15 Key Terms in Performance Tuning Everyone Should Know
Introduction
Performance Tuning includes many ideas that can feel disconnected when you first encounter them. The easiest way to build a useful foundation is to learn a small set of terms that represent different parts of the field. The 15 terms below move from basic concepts toward design, runtime, testing, deployment, or operational ideas where those areas apply. Each term is explained in simple language and paired with a practical example. The goal is not to turn this article into a textbook or implementation guide. Instead, it gives you a mental map so that documentation, technical discussions, courses, and deeper articles become easier to follow.
1. Baseline
A baseline is a trusted measurement of current performance used for comparison. Before tuning, engineers record response time, throughput, errors, and resource use under a controlled workload. After a change, the same test is repeated so improvement or regression can be measured instead of guessed.
For example, a service starts with p95 latency of 900 ms; after tuning, the same test shows 620 ms.
2. Bottleneck
A bottleneck is the part of a system that currently limits performance. Tuning is most effective when it targets the limiting resource rather than changing everything at once. Bottlenecks can move after one problem is fixed, so tuning is usually iterative.
For example, optimizing application code may expose the database as the next bottleneck.
3. Profiling
Profiling measures where an application spends CPU time, memory, or other resources. A profiler can identify hot methods, excessive allocations, blocking calls, or expensive code paths. It gives code-level evidence that complements system-level metrics.
For example, a CPU profile may show that JSON serialization consumes 35% of request processing time.
4. Caching
Caching stores frequently used data closer to where it is needed. A cache can avoid repeated database queries, remote calls, or expensive calculations. Good caching improves latency and reduces load, but stale data and invalidation must be handled carefully.
For example, a product service can cache catalog details that change only a few times per day.
5. Connection Pool
A connection pool keeps reusable connections ready for databases or remote services. Opening a connection for every request is expensive. Pools reduce setup overhead, but a pool that is too small creates waiting and one that is too large can overload the downstream system.
For example, a web service may reuse 40 database connections across hundreds of incoming requests.
6. Thread Pool
A thread pool is a controlled set of worker threads used to execute tasks. Pools avoid creating unlimited threads and help control concurrency. Too few workers can leave work waiting, while too many can increase context switching, memory use, and contention.
For example, an API server may process requests using a fixed worker pool rather than creating a new thread for every request.
7. Index
A database index is a data structure that helps the database find rows faster. Without a useful index, a query may scan a large table. An index can reduce read time dramatically, although indexes consume storage and add work to inserts and updates.
Get Bishal Goutam’s stories in your inbox
Join Medium for free to get updates from this writer.
For example, adding an index on customer_id can speed up queries that repeatedly search orders by customer.
8. Query Plan
A query plan is the database’s chosen strategy for executing a query. The plan shows operations such as scans, index lookups, joins, sorts, and estimated costs. Reading it helps engineers understand why a query is slow and whether indexes or query changes may help.
For example, a plan may reveal a full table scan where an index lookup was expected.
9. Batching
Batching processes several items together instead of handling each item separately. Batching can reduce network calls, transaction overhead, and repeated setup work. Very large batches, however, can increase latency and memory use, so the batch size should match the workload.
For example, a logging client can send 100 events in one request instead of 100 separate requests.
10. Asynchronous Processing
Asynchronous processing moves work off the immediate request path when the user does not need the result right away. A request can place work on a queue and return quickly while a background consumer completes the task. This can improve user-facing latency and absorb bursts, but it introduces queues and eventual completion.
For example, an order API can confirm receipt first and generate the invoice in the background.
11. Garbage Collection
Garbage collection automatically reclaims memory that an application no longer uses. Managed runtimes periodically identify unused objects and free memory. Excessive allocation or poorly tuned heap settings can cause frequent or long pauses that affect latency.
For example, a Java service may show response-time spikes during long full garbage-collection pauses.
12. Compression
Compression reduces the amount of data sent or stored by representing it more compactly. Smaller payloads can reduce network transfer time, especially for text-based responses. Compression also uses CPU, so it should be applied where bandwidth savings are worth the processing cost.
For example, an API can gzip a large JSON response before sending it to a browser.
13. Contention
Contention happens when multiple tasks compete for the same limited resource. Locks, shared data, database rows, CPU cores, and connection pools can all create contention. As concurrency rises, waiting can become a major part of response time.
For example, many threads updating the same synchronized object can spend more time waiting than doing useful work.
14. Capacity Headroom
Capacity headroom is spare system capacity kept above normal demand. Running permanently near maximum utilization leaves little room for traffic spikes, failures, or scaling delays. Headroom gives the system space to absorb temporary increases without immediate saturation.
For example, a service designed for 10,000 requests per second may normally operate around 6,000 to leave room for bursts.
15. Measure-Change-Verify Loop
The measure-change-verify loop is the basic discipline of performance tuning. Engineers measure a problem, form a hypothesis, make one controlled change, and repeat the same test to verify the effect. This avoids tuning based on assumptions and makes improvements reproducible.
For example, after finding a slow query, a team adds an index and reruns the same workload to confirm the improvement.
