We can see all jobs are having issues on the same System Agent.
They started well but not going to " Running" status.
Status of the job shows as "ST" but as per Application teams jobs are running in background.
System Agent release 11.3 SP2 on Linux but it might happen with any other release
Autosys is configured in a full High Availability System.
The primary scheduler failed over and the shadow machine took over.
During this process, the agentparm.txt file of the System Agent is automatically updated and the communication address of the primary scheduler is replaced by the same for the shadow scheduler
When the primary scheduler is back and up again, same process is done again and the communication address of the shadow scheduler is replaced by the primary scheduler
On this System Agent, someone manually updated the agentparm.txt file and the first communication address still belonged to the shadow scheduler and there was another one for the primary scheduler.
When the System Agent tried to send the job status to the scheduler, it was trying to communication with the shadow scheduler which was the first in the list
The shadow scheduler rejected this communication attempt with a NAK signal, hence the problem with the job stuck in Starting status
Cleaned up the communication addresses in the agentparm.txt file of the System Agent to only keep the address of the primary scheduler which is the active scheduler
Variables to cleanup are:
To only keep details of the active scheduler