If you read through the workflow details in the "cause" section above, you may already have an idea of why your asset jobs are not running. The common bottleneck for asset jobs are the engines processing the collect task for each scalability server.
On the front-end, after an asset job is linked to an agent or computer group, the collect task for each scalability server needs to be run, in order to synchronize the asset job and its scheduling information with each scalability server.
In the middle, what's triggering the asset job to run? The AM agent plugin is 1-CAF scheduler policy based and 2-Event trigger based. Asset jobs are not proactively triggered like software jobs. Check the CAF scheduler policy, "Run the UAM Agent".
On the back-end, after the asset job has run on the agent, its status is uploaded to the sector on the scalability server, among all the other files waiting to be collected and processed by the engine-- computer and user registrations, hardware and software inventory files, execution data and status files, along with any custom inventory files.
Hence a very common factor in the perception that asset jobs are not working, is environment, scaling and architecture.
Here is a list of items you should check and consider--
1. Are all the scalability server collect tasks assigned/linked to an engine? You can check this by using the DSM Explorer --> Control Panel --> Engines --> All Engines --> Pick an engine (i.e. SystemEngine) --> Link Existing Task. Ensure the collect task for each scalability server is mapped to an engine, otherwise its files will not be collected.
2. Are all the engine plugins running? Check the "caf status" output to be sure.
3. Are any engines stuck processing a large backlog of files from any scalability server?
4. Architecture and scaling plays a BIG ROLE in how efficiently asset jobs run:
a) The overall number of agents in the environment. A good rule of thumb is about 10,000 agents per ITCM domain manager.
b) The overall number of scalability servers in the environment. A good rule of thumb is about 1,000 agents per ITCM scalability server. Could they handle more? Absolutely, but at what trade-off? The location of each scalability server is also quite important. What's the point of a scalability server, if it's not locally/regionally serving its registered agents?
c) The overall number of engines on the domain manager. A good rule of thumb is no more than 8 or 10 additional engine instances, but varying depending on the architecture and available resources. Remember, the engines are the portals into SQL for processing information. The database schema is remaining constant, so at some point, adding more engines and too much parallelism may only serve to increase delays and wait times for SQL commands to process. There are no hard and fast numbers here-- it is up the the ITCM and SQL administrators to see what configuration is working most efficiently compared to the volume of incoming data from the agents.
d) The distribution of agents across the scalability servers. You can download the WinOffline utility from the ITCM community site, as the scalability servers portal, under "Database Tools" will reveal the count of registered agents per scalability server (versus manually filtering for each scalability server in DSM Explorer):
e) How often are the agents configured to send a hardware inventory delta? Reference sister scaling document, kb000046211, "Scaling Client Automation: How to improve collect task and replication task performance by limiting the amount of hardware and software scans sent by agents":
5. Engine advanced settings-- right click on an engine --> properties --> advanced tab.
a) Number of files the engine will collect during one collect cycle. The default value is 10,000. A good rule of thumb is to change this to 500-1,000 files. Why? It allows the engine to cycle through a collect task (or list of collect tasks) faster, so engines are "validated" more often. When engines are validated more often, computer registrations are updated more frequently, and most importantly for asset jobs, the scalability servers are validated more frequently, so asset jobs are not waiting for a lengthy list of files to be collected, before they are synchronized.
b) The interval in seconds between engine jobs. The default value is 60 seconds. A good rule of thumb is to change this to 20-30 seconds. Why? If you have a lengthy list of collect tasks assigned/linked to an engine, you'll want the collect task resting less between processing tasks, so it can cycle more tasks in a shorter period of time.
6. Asset jobs are not automatically triggered by the scalability servers. Their execution are agent-driven by the AM agent plugin on the agent side. The AM agent plugin is CAF scheduler policy based and event trigger based. Asset jobs are not proactively triggered like software jobs.
a) CAF scheduler policy. DSM Explorer --> Control Panel --> Configuration --> Configuration Policy --> DSM --> Common Components --> CAF --> Scheduler --> Run the UAM Agent. Review the settings within. By default, the AM agent plugin is only triggered once per day by the CAF Scheduler.
b) Event triggers. These are external triggers that automatically cause the AM agent plugin to run. These include:
- Computer reboot.
- User login.
- Network address change.
- Local or remote "caf register".
- Local or remote "caf start amagent".
- Remote DSM Explorer Asset Job Check functionality.