Continuous Collection Only Works If Overhead Stays Low

Continuous collection sounds attractive in theory. If performance data is always available, then investigations can begin with evidence instead of guesswork. Rare delays become easier to explain, CPU-heavy jobs can be examined while the relevant activity is still visible, and teams no longer have to hope that the problem happens again under controlled conditions. That is the promise. But there is also an obvious objection: continuous collection is only useful if the collection itself does not become part of the problem.

The idea fails if overhead is too high

Many IBM i teams are understandably cautious about broad performance collection. If you collect too much data, too often, and store it inefficiently, the monitoring system starts competing with the application workload it is meant to observe. At that point, the cure begins to look suspiciously like a new source of cost.

That is why the question is not simply whether data should be collected continuously. The real question is whether it can be collected continuously without creating unacceptable overhead. If the answer is no, then teams will only enable collection occasionally, or only after users complain. And once that happens, many of the most valuable moments are already lost.

Availability is not enough. Efficiency matters too

This is where many performance approaches become weaker than they first appear. They may be capable of collecting useful information, but only at a cost that makes continuous use unattractive. As a result, the data is available in principle, but not in practice when the difficult case actually occurs.

That is not good enough. For continuous collection to be genuinely useful in production, two things have to be true at the same time: the data must be relevant enough to explain real application problems, and the overhead must be low enough that collection can remain active without hesitation.

Those two requirements belong together. If the data is lightweight but unhelpful, you still cannot solve much. If the data is useful but expensive to collect, it will not stay on long enough to catch the important cases.

Why low overhead changes behaviour

When collection overhead is low, something important changes in the way teams work. They stop treating performance data as an emergency measure. Instead, they can treat it as part of the normal operating environment.

That matters more than it may sound. Because once relevant data is already being gathered, performance analysis becomes far less reactive. A slowdown no longer begins with a debate about what should be turned on, what should be captured, or whether the system can afford extra collection right now. The evidence is simply there.

That shortens the path from symptom to explanation. It also makes rare and intermittent problems much less expensive to investigate, because the opportunity to observe them is not lost while people are still deciding how to start.

The collection method itself becomes a design problem

This is why data collection cannot be treated as a secondary technical detail. The way data is gathered is part of the performance strategy.

If the collection process depends heavily on I/O, stores everything inefficiently, or creates too much extra work on the system, then continuous use becomes hard to justify. Even if the collected data is valuable, the operational cost may still be too high.

A practical solution therefore needs more than good analysis logic. It also needs a collection method that is designed to stay lightweight under real production conditions. Otherwise the system will always be most useful in theory, and least useful when it matters most.

Why this matters in GiAPA

This is one of the key ideas behind GiAPA. Continuous collection only becomes strategically useful when the overhead stays small enough that the data can remain available all the time, not just during special investigations.

That is what changes performance work from something reactive and episodic into something more systematic. The goal is not simply to collect more data. The goal is to make sure the right data is available when the slowdown actually happens.

Why this is strategically important

In practice, that changes the whole value of performance analysis. When relevant evidence is already present, the work becomes faster, more precise, and less dependent on luck. Teams can move more quickly from symptom to cause, and from cause to improvement.

That is why overhead is not a side issue. It determines whether continuous collection becomes a realistic operating model or just a technical ambition that nobody dares leave running.

This article is part of GiAPA’s guide: Why IBM i Application Performance Analysis Feels Hard.

If you want the broader context — why IBM i performance problems often seem difficult, what data makes them easier to explain, and how real cases point to practical improvements — the full guide brings the pieces together.