Diagnosing Rare Delays of Tiny Jobs

One of our users contacted us with a puzzling issue: several very small jobs, each running once a minute and typically completing in 1–2 seconds, would occasionally take 30 seconds or more to return a result.

‍

The delay happened so rarely—and the jobs ended normally and delivered correct results—that operations couldn’t catch them in time to investigate. Something was clearly causing the job to wait, but what?

‍

Finding the hidden wait

In cases like this, the best way to diagnose the issue is to retrieve the job’s call stack while the delay is happening. Our performance analysis tool GiAPA includes a “Watch job” function for precisely this purpose, but you can also set up a lightweight workaround using just a few lines of CL code.

‍

Here’s how it works:

‍

Step-by-step workaround

1. Start with RTVJOBA and SBMJOB
   At the beginning of the job experiencing delays, insert a command to retrieve the job’s ID:

‍

RTVJOBA JOB(&NAME) USER(&USER) NBR(&NBR)

1. Then immediately submit a separate job that will attempt to examine the call stack after a short wait:

‍

SBMJOB CMD(CALL PGM(WATCHCL)) JOBQ(QINTER) JOB(WATCH_DELAY)


1. In the WATCHCL program:
   Add these three lines:

DLYJOB DLY(10)
DSPJOB JOB(&NBR/&USER/&NAME) OPTION(*PGMSTK)
MONMSG CPF0000


1. DLYJOB DLY(10) ensures the analysis job waits until after the tiny job should have finished.
2. DSPJOB ... OPTION(*PGMSTK) prints the call stack if the job is still running.
3. MONMSG CPF0000 prevents errors if the job already ended (which it normally will, unless it’s delayed).

This setup lets you quietly “watch” the job from the side, without affecting its behavior. And if a delay does occur, you’ll get a snapshot of what the job was doing at that moment—including which program or statement was active.

‍

When it works best

This method is especially useful for rare, unpredictable delays where the job exits cleanly before anyone can inspect it. With this in place, you’ll have a clear view of any resource the job was waiting on—whether it’s file locks, communication with another process, or system contention.

‍