Why a One-Second Job Sometimes Takes 30 Seconds

One example involved a key application where a transaction that normally completed in about one second could occasionally take as long as 30 seconds. The issue was serious enough to frustrate users, but not frequent enough to be easy to catch in real time. An experienced IBM performance specialist had already looked at the case, but the normal logs did not reveal the root cause.

When more detailed performance data became available, the explanation became clearer. The delay did not appear to come from CPU pressure. CPU utilization was well below 100%. Instead, the relevant clue was memory-related activity. Storage management page-out activity showed a spike at the exact time of the slowdown.

The slowdown was real, but the evidence was fragile

This is what makes intermittent problems so difficult in practice. A job that is usually fast can become slow only briefly and then return to normal. By the time someone starts looking, the visible symptom may already be gone. Standard logs may confirm that the delay happened, but still say very little about why it happened.

That is why these cases often feel more mysterious than they really are. The problem is not always that the explanation is deeply hidden. The problem is that the evidence disappears before the investigation begins.

The real cause was somewhere else

The page-out spike pointed in a different direction from what many people first expected. It suggested that the delayed transaction was not itself creating the main problem. Instead, another job on the system was putting pressure on memory in a way that affected the transaction indirectly.

That turned out to be the case. The real cause was an SQL job creating a large temporary table. Because of its priority, it effectively took memory away from other jobs. The one-second transaction was delayed not because its own logic had suddenly changed, but because the surrounding run environment had shifted at exactly the wrong moment.

Why this kind of case matters

This is a good example of why performance analysis cannot focus only on the job that appears slow. A delayed transaction is not always the job creating the problem. Sometimes the actual cause sits elsewhere in the workload, and only becomes visible when you can correlate the slowdown with resource behaviour across the system.

That is also why broad resource collection matters so much. If you only look inside the delayed job, you may miss the fact that another job changed the conditions around it. In this case, the most useful clue was not hidden in the transaction logic itself. It was visible in the system behaviour that surrounded the event.

A problem can be easy to explain once the right moment is captured

One of the most important lessons in this case is that the explanation did not depend on heroic investigation. It depended on having the right evidence at the right moment. Once the relevant data had been collected, the case became much easier to understand.

That is often the pattern with rare delays. They feel elusive because the opportunity to observe them is small. But when that opportunity is captured properly, they may turn out to be far less mysterious than their reputation suggests.

The broader lesson

A one-second job that sometimes takes 30 seconds does not always point to bad code in that job. It may point to something else happening elsewhere in the system at the same time. That is why intermittent performance analysis needs visibility not only into individual jobs, but into the wider resource picture around them.

Without that visibility, the investigation begins with guesswork. With it, even an unusual and frustrating slowdown can become a concrete, explainable event.

This article is part of GiAPA’s guide: Why IBM i Application Performance Analysis Feels Hard.

If you want the broader context — why IBM i performance problems often seem difficult, what data makes them easier to explain, and how real cases point to practical improvements — the full guide brings the pieces together.