How Spring Boot Implements File Streaming for Large Files

Image Source

Uploading and downloading large files efficiently is a common challenge in web applications. Spring Boot makes this easier with built-in streaming mechanisms like InputStreamResource and OutputStream, which keep memory usage low while handling large files. In this article, we’ll take a look at how these components work under the hood and why they’re important for managing file streaming efficiently.

Streaming Large File Uploads in Spring Boot

Handling large file uploads in a web application requires an efficient way to process incoming data without overwhelming the server’s memory. By default, Spring Boot supports file uploads through its built-in multipart handling, but when dealing with large files, loading everything into memory at once can lead to performance problems. Instead of storing the entire file in RAM, Spring Boot allows streaming the data in chunks, making the upload process smoother and more reliable.

How Spring Boot Handles File Uploads

Spring Boot processes file uploads using the MultipartFile interface, which is part of Spring MVC. When a file is uploaded, the multipart resolver processes the request and stores the uploaded file in a temporary location before making it available for further processing. This works well for smaller files, but for larger ones, reading the entire content into memory can quickly become a problem. Instead of handling files as whole objects, reading them as a stream allows processing data in chunks, reducing the risk of memory overflows. This method is especially useful for applications dealing with video uploads, large datasets, or logs that need to be stored on disk or in cloud storage.

Streaming File Uploads with InputStream

Using an InputStream for file uploads allows reading and writing data in smaller portions rather than keeping the entire file in memory. Instead of loading the full file at once, the application processes it in chunks, reducing the risk of memory overload and improving performance.

Uploading a Large File Using InputStream:

@PostMapping("/upload")
public ResponseEntity<String> uploadFile(@RequestParam("file") MultipartFile file) {
    Path filePath = Paths.get("uploads/" + file.getOriginalFilename());

    try (InputStream inputStream = file.getInputStream();
         OutputStream outputStream = Files.newOutputStream(filePath, StandardOpenOption.CREATE)) {
        byte[] buffer = new byte[8192];
        int bytesRead;
        while ((bytesRead = inputStream.read(buffer)) != -1) {
            outputStream.write(buffer, 0, bytesRead);
        }
    } catch (IOException e) {
        return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).body("Error saving file");
    }

    return ResponseEntity.ok("File uploaded successfully");
}

Reading the file in chunks prevents excessive memory usage. The InputStream reads 8 KB blocks at a time and writes them to an OutputStream, allowing the file to be stored as it arrives instead of holding everything in memory. The try-with-resources block automatically closes the streams, preventing resource leaks. This method is useful for local file storage, but cloud-based applications or distributed systems may require different handling.

Streaming File Uploads in a Reactive Application

Spring WebFlux, a reactive programming framework, provides a non-blocking way to handle file uploads. Instead of keeping the entire file in memory, WebFlux processes it as a continuous stream using Flux<DataBuffer>. This is particularly useful in cloud environments or when handling multiple large file uploads simultaneously.

Handling Large File Uploads in WebFlux

@PostMapping(value = "/upload-stream", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public Mono<Void> uploadLargeFile(@RequestPart("file") Flux<DataBuffer> fileBuffer) {
    Path path = Paths.get("uploads/large-file.dat");

    return DataBufferUtils.write(fileBuffer, path, StandardOpenOption.CREATE).then();
}

This method processes the file as a stream of DataBuffer objects rather than reading everything into memory at once. As each chunk arrives, DataBufferUtils.write() writes it to disk without blocking the main thread. This approach is useful for handling multiple large uploads concurrently, as it keeps memory usage low and prevents slowdowns.

Multipart Configuration for Large File Uploads

Spring Boot allows configuring file upload limits in application.properties to manage large files efficiently:

spring.servlet.multipart.max-file-size=1GB
spring.servlet.multipart.max-request-size=2GB

• max-file-size: Defines the maximum size for a single file.
• max-request-size: Sets the total size limit for a multipart request.

Adjusting these properties allows applications to handle large files while keeping memory usage under control.

Choosing the Right File Upload Method

The best way to handle file uploads depends on the specific requirements of an application:

• For simple uploads: MultipartFile works fine but may not be ideal for large files.
• For efficient memory usage: Using InputStream helps process files in chunks.
• For reactive applications: Flux<DataBuffer> in WebFlux is a better choice, especially for handling concurrent uploads.

Each method has its own advantages, and choosing the right one depends on the expected file sizes, server resources, and application architecture.

Streaming Large File Downloads in Spring Boot

Handling large file downloads efficiently is just as important as managing uploads, especially when dealing with large datasets, media files, or logs. Without streaming, loading a large file into memory before sending it to the client can lead to performance issues, slow response times, and even out-of-memory errors. Spring Boot provides different ways to send large files without consuming too much memory by processing data in chunks and streaming it directly to the response.

Streaming Large Files with InputStreamResource

InputStreamResource is one of the most effective ways to handle file downloads in Spring Boot. Instead of loading the entire file into memory, it reads the file in small sections and sends it to the client as the data becomes available.

Get Alexander Obregon’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Streaming a Large File Using InputStreamResource

@GetMapping("/download")
public ResponseEntity<InputStreamResource> downloadFile() throws IOException {
    Path filePath = Paths.get("uploads/large-file.dat");
    InputStreamResource resource = new InputStreamResource(Files.newInputStream(filePath));

    return ResponseEntity.ok()
            .header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=large-file.dat")
            .contentType(MediaType.APPLICATION_OCTET_STREAM)
            .body(resource);
}

The file is read using Files.newInputStream(filePath), which opens a stream to the file without loading everything into memory. As the client reads data from the response, the InputStreamResource provides chunks of the file, keeping memory usage low. This method works well for most cases where files need to be served efficiently.

Streaming Large Files Using OutputStream

For even better control over how data is sent, an OutputStream can be used to directly write file contents to the HTTP response. This allows finer control over buffering and chunk sizes, making it a good option for large files where memory efficiency and transfer speed matter.

Streaming Large Files with OutputStream

@GetMapping("/download-stream")
public void downloadFileStream(HttpServletResponse response) throws IOException {
    Path filePath = Paths.get("uploads/large-file.dat");
    response.setContentType("application/octet-stream");
    response.setHeader("Content-Disposition", "attachment; filename=large-file.dat");

    try (OutputStream out = response.getOutputStream();
         InputStream in = Files.newInputStream(filePath)) {
        byte[] buffer = new byte[8192];
        int bytesRead;
        while ((bytesRead = in.read(buffer)) != -1) {
            out.write(buffer, 0, bytesRead);
        }
    }
}

This method opens a file input stream and writes its contents directly to the response output stream. The data is sent in chunks of 8 KB, reducing memory load while maintaining a steady data transfer rate. The try-with-resources block makes sure that both streams are closed automatically when the download is complete.

Unlike InputStreamResource, this approach gives more control over how data is handled. It allows adjusting buffer sizes for different file types, optimizing performance for specific workloads.

Streaming Large Files with WebFlux

For reactive applications, Spring WebFlux provides a fully non-blocking way to serve large files. This method is useful for applications that need to handle multiple large downloads at the same time without blocking server threads.

Streaming Files Using WebFlux and DataBuffer

@GetMapping(value = "/download-reactive", produces = MediaType.APPLICATION_OCTET_STREAM_VALUE)
public Mono<Void> downloadFileReactive(ServerHttpResponse response) {
    Path filePath = Paths.get("uploads/large-file.dat");
    Flux<DataBuffer> fileStream = DataBufferUtils.read(filePath, new DefaultDataBufferFactory(), 8192);
    
    return response.writeWith(fileStream);
}

Instead of blocking a thread while reading the file, this method streams the data using Flux<DataBuffer>. The file is read in 8 KB chunks, and each chunk is sent to the client as soon as it’s available. This reduces memory usage while keeping the server responsive, even when handling multiple downloads at once.

WebFlux works best in high-concurrency environments, such as media streaming applications, cloud-based file storage, and APIs that need to serve large files without slowing down other requests.

Conclusion

Spring Boot provides multiple ways to handle large file uploads and downloads without overwhelming memory. Uploading files with InputStream processes data in chunks, reducing the load on system resources, while WebFlux offers a reactive solution for handling multiple large files at once. For downloads, InputStreamResource and OutputStream allow efficient streaming, while WebFlux keeps downloads non-blocking for high-concurrency environments. Choosing the right method depends on the application's needs, server capacity, and expected file sizes.

1. Spring Boot File Upload Guide
2. Spring WebFlux Documentation
3. Spring Boot Multipart Configuration
4. Java NIO Files API

Thank you for reading! If you find this article helpful, please consider highlighting, clapping, responding or connecting with me on Twitter/X as it’s very appreciated and helps keeps content like this free!

Spring Boot icon by Icons8