The Fastest Way to Read a Raw Image Stream
I compared several ways to load a raw (uncompressed, unprocessed) image stream from a file in C++, looking at what matters when dealing with large amounts of data in this format.
In video processing or medical imaging, we often work with raw data streams. And the libraries we do the computation with often require their data in a different shape, to squeeze every bit of performance out of the underlying hardware. Bridging this gap is necessary and important, yet it’s not talked about often enough.
I wanted to know three things:
- Which approach is fastest, from opening the file until the data is ready for computation?
- What happens inside the libraries when we read a file, and why does it matter?
- Does introducing abstractions for testability affect the performance measurably? (Part 2)
The file format
The file format is boring, because real ones are. A file header describes the stream: how many views it contains, their dimensions, and the size of each payload. Each view then follows as a view header and its pixel payload. Here, a view is one image: a projection in a CT scan, but it could just as well be a frame of a video.
The records follow each other directly in the stream, just as they arrive from the hardware or any other source. All padding is explicit: the header structs contain reserved bytes, so the compiler never has to insert hidden ones.
For the measurements, I used the following data structures:
There is a lot of empty space in both headers, and that is on purpose. Reserved bytes let us add new fields later without changing the size of the headers, so files and readers stay compatible. The power-of-two sizes help too: the 64-byte file header is exactly one cache line, and two 32-byte view headers fit into one. Most importantly, there is no implicit padding, neither inside a header nor between headers in an array, and every field stays naturally aligned. What that means for cache efficiency and SIMD operations is a topic for another article.
What we actually want in memory
The file interleaves headers and payloads, because that’s the natural order for whoever writes it: one view at a time, as it is acquired. The code that consumes it wants something else. The computational library in this example expects all pixels in one contiguous array. The view headers also contain information needed for processing and validation, so we collect them in a separate array.
This is a structure of arrays: data grouped by how it will be accessed. Why that pays off is beyond the scope of this post. Mike Acton’s talk Data-Oriented Design and C++ (CppCon 2014) makes the case.
So the target data structure looks like this:
struct Dataset {
FileHeader fileHeader;
std::vector<ViewHeader> headers; // all view headers
std::vector<Pixel> pixels; // all views, back to back
};
Not every consumer wants this. A video player shows one frame at a time, so it never needs the whole clip in memory. The CT reconstruction library in this example needs all views at once, in one block. Same kind of file, different consumer, different best loader.
First comparison
When I started writing the loader, I thought of three intuitive approaches to read the file and produce the Dataset expected by the computational library.
V1. Bulk read
Read all the records from disk into memory with a single read call, then copy the data into the two (pre-allocated) vectors. Although it uses twice the memory of the data structure, it makes only one read call, and copying in memory might be faster than many smaller reads from the file.
V2. Read directly into the Dataset
Read the file sequentially: a view header, then its payload, then the next header and payload. Each read goes directly into its final location, alternating between the header and pixel vectors.
V3. Two pass read, with jumps
V2 reads the file sequentially, but alternates between the two vectors. What if we filled one vector at a time instead? Read the file in two passes: first the view headers, skipping each payload, then the payloads, skipping each header. Each read goes directly into its final location, filling the header vector first and the pixel vector second.
How I measured
To keep the comparison fair and the noise as low as possible:
- I timed from opening the file until the `Dataset` was complete; opening is inside the timer.
- Every measurement ran in a fresh process, because the real system loads each file once.
- The file was in the operating system’s page cache (warm cache); cold reads from disk are a topic for part 2.
- Every load was checked against the generated data before its time counted.
- I report the median of 10 processes per bar; the whisker shows the middle 50%.
- Machine: a laptop with an AMD Ryzen 7 7735HS (8 cores, 16 threads), 32 GB DDR5–4800 and a 512 GB NVMe SSD (Micron MTFDKCD512QGN). Windows 11 with MSVC 14.51 (Visual Studio 18, Release build); Ubuntu 26.04 under WSL2 (kernel 6.18, 13 GB of the RAM assigned) with GCC 15.2 (`-O2`). The power settings were the same for every measurement.
Of course, many other factors could have influenced the results, like process priority or whatever else was running in the background. But the trends were clear, and the measurements were consistent with each other.
Results
First, I implemented all three approaches using std::ifstream, the standard C++ file input stream. You can find the source code here.
I generated a test file containing 2,000 views, each with 896 × 64 unsigned 16-bit pixels. The total size was about 229 MB.
Real-world datasets can be much larger. In my tests, the same overall pattern held across the platforms I tested, including with files several gigabytes in size. To try different dimensions or file sizes on your own system, follow the data-generation and benchmarking instructions in the repository.
V2 performed the best, and V3 came close. V1 was the slowest on both platforms: reading everything at once and splitting it afterwards took up to twice as long as reading each piece straight into place.
Get Pinter Balazs’s stories in your inbox
Join Medium for free to get updates from this writer.
Interestingly, Linux under WSL outperformed the native Windows build on the same machine for V2 and V3: it needed about 30% less time for V2 and 34% less for V3. Only V1 was faster on Windows. Later we will see why!
With
std::ifstream, reading directly into the final layout was faster than loading everything into a temporary buffer first.
Let’s try C and go native
As C++ is (almost) a superset of C, we could also use the FILE* operations of the C standard library: fopen, fread, fseek. And as the target system runs on Linux, we could even go one level lower and use the POSIX system calls: open, read, lseek. This is as close to the operating system as it gets.
The algorithm stays exactly the same, only the calls change. This is the core of V2 with all three APIs:
// std::ifstream
mIn.read(reinterpret_cast<char *>(&dataset.headers[i]), sizeof(ViewHeader));
mIn.read(reinterpret_cast<char *>(dataset.view(i)), mFileHeader.payload_size);
// FILE*
std::fread(&dataset.headers[i], sizeof(ViewHeader), 1, mFile);
std::fread(dataset.view(i), mFileHeader.payload_size, 1, mFile);
// POSIX
read(mFd, &dataset.headers[i], sizeof(ViewHeader));
read(mFd, dataset.view(i), mFileHeader.payload_size);You can find the full readers in the repository: stdioReader.hand posixReader.h. POSIX is not available with MSVC, so it only runs on Linux.
On Linux, nothing changes, all three APIs are within a few percent of each other, well inside the run-to-run noise. Going native didn’t buy anything here.
On Windows, the picture flips completely. FILE* is around 2.5× faster than std::ifstream, for every approach. It is even twice as fast as anything measured under WSL. So “Linux beat Windows” from the previous section is only true for std::ifstream. Here is the promised why.
What ifstream::read does on Windows
In the MSVC version I used, a large ifstream::read is split into small chunks (4,095 bytes each), turning one visible read into many fread calls. That chunk size is hard-coded in the library. It is a workaround for text-mode newline translation, but applies in binary mode too.
The small reads add call overhead and an extra copy through the stream buffer. GCC’s libstdc++ handles large reads more like fread, which helps explain why the two APIs performed similarly on Linux but differently with MSVC.
Letting V1 go
V1 remained the slowest approach with every API tested. Its single bulk read did not compensate for allocating and initializing a temporary buffer, then copying the data into the destination vectors. From here on, I’ll focus on V2 and V3.
The POSIX V1 implementation also fails on files larger than 2 GiB, because Linux limits a single read() to just under that amount. Real production code should check the returned byte count, handle errors, and repeat incomplete reads. My example omits these checks, keeping the focus on the read patterns.
So far,
FILE*is the clear winner with MSVC. On Linux with GCC, the choice of API makes little difference for V2 and V3 with this data.
But what happens when we change the buffering?
Buffering
FILE* and std::ifstream keep a small buffer inside the process. Small sequential reads are served from it; when the available bytes run out, the library asks the operating system for more. Large reads can go directly into the destination, although this depends on the implementation.
POSIX read() has no user-space buffer: every call is a system call that reads directly into the destination. (The operating system’s page cache still applies.) This was the reasoning behind a similar loader that inspired this article: POSIX has no buffer and no extra copy, so it should be the fastest.
So the effectiveness of buffering depends on both read size and access pattern. It can serve many small reads with fewer system calls, while large reads may already bypass the buffer. Seeking can change how much buffered data gets reused.
To test the difference, I disabled buffering in the other two APIs as well (02_buffered/):
std::setvbuf(mFile, nullptr, _IONBF, 0); // FILE*: before the first read
mIn.rdbuf()->pubsetbuf(nullptr, 0); // std::ifstream: before (GCC) or after (MSVC) open On my system, the default buffer sizes were 4 KiB for FILE* with glibc, 8 KiB for std::ifstream with libstdc++, and 4 KiB for both with MSVC on Windows. These can be configured using setvbuf for FILE* and pubsetbuf for std::ifstream.
With the original 112 KiB views, it made no difference on Linux: every payload read is far bigger than the buffer. So I generated a file with tiny views instead: 200,000 views of 256 bytes.
Now V2 with a buffer is about 6× faster on Linux and 13–14× faster on Windows. Without a buffer, one load takes about 400,000 read() system calls on Linux; with it, std::ifstream needs around 7,000 and FILE* about 14,000.
The exception is V3 with std::ifstream on Linux, where the buffer makes it slower. libstdc++ throws its buffer away on every seekg, so each 32-byte header read refills a full 8 KiB buffer, only to jump over most of it.
Where is the line?
I ran the same comparison on Linux with view sizes from 256 B to 1 MiB:
For V2, the buffer wins clearly below 1 KiB. Around the buffer size (4 KiB), the difference shrinks. From 16 KiB on, the lines stay close to 1. Unbuffered only wins with seeks: by up to about 1.4× for std::ifstream V3, otherwise only by a little.
So the buffer matters when the average read is smaller than the buffer. For the original 112 KiB views, buffered or unbuffered made no measurable difference.
Conclusions:
V2 was the best overall choice: read each piece straight into its final place. For the original 112 KiB views on Linux, POSIX, FILE*, and std::ifstream performed similarly, and buffering made no measurable difference. On Windows with MSVC, FILE* was about 2.5× faster than std::ifstream.
V1 was the slowest and needed roughly twice as much memory for the data: the temporary buffer alongside the final Dataset.
Buffering made the biggest difference with small reads. With 256-byte views, disabling it made V2 about 6× slower on Linux and 13–14× slower on Windows. With V3, seeking could make buffering work against us. So “POSIX, because it’s unbuffered” was a reasonable choice for our original data, but the measurements didn’t support that reasoning.
Know what you’re optimizing for: throughput, latency, or memory. Find the bottleneck and look at the whole system, including the libraries beneath the code.
But the biggest takeaway is to test the assumptions. Measuring showed which approach worked best; looking at the implementations helped explain why. “We’ve always done it this way” is not a reason.
Coming in part 2
- Does an interface with virtual calls make the code easier to test at a measurable performance cost?
- Does the file system matter? Comparing FAT32, exFAT, NTFS, and ext4 on an NVMe drive and a USB stick.
- What changes with a cold cache, when the file isn’t already in memory?
Thanks for reading! If you’re interested in part 2 or similar topics, hit the follow button ;)
