The Rise of Async Generator Functions: How AI Made ‘On-Demand Processing’ Essential
When ChatGPT first showed responses appearing word by word instead of all at once, it didn’t just introduce a new interface pattern — it reset user expectations about how software should behave. Users who once accepted loading spinners and progress bars now expect to see results as they’re generated, not after they’re complete.
This shift has made async generator functions increasingly relevant in JavaScript development. What was once a specialized feature for handling large datasets has become useful for building applications that feel responsive and modern. Async generators provide a natural way to implement the “show results as they come” pattern that AI applications have made standard.
Understanding the Expectation Shift
Traditional web applications follow a request-response cycle: users submit input, wait for processing, then receive complete results. This approach worked when computational tasks were simple and fast. But AI workloads are different — they involve processing large context windows, generating responses incrementally, and handling user interruptions mid-stream.
Async generators align well with these requirements because they produce values on-demand rather than all at once. Instead of building all results in memory before returning them, async generators can yield values as they become available, allowing applications to respond immediately to user input.
Example 1: Server-Side AI Response Streaming
The most practical application of async generators today is streaming AI responses from backend services. Here’s a realistic implementation using LangChain:
// Async Generator for LangChain Streaming
import { ChatOpenAI } from "@langchain/openai";
async function* streamAIResponse(userMessage) {
// Initialize streaming LLM
const llm = new ChatOpenAI({
modelName: "gpt-4",
streaming: true
});
// Step 1: Start processing
yield { status: "🔍 Searching context..." };
// Simulate context retrieval
const context = await getRelevantContext(userMessage);
yield {
status: "📚 Found context",
preview: context.slice(0, 100) + "..."
};
// Step 2: Begin streaming response
yield { status: "🤖 Generating response..." };
const prompt = `Context: ${context}\n\nUser: ${userMessage}`;
let fullResponse = "";
// Step 3: Stream tokens as they arrive
for await (const chunk of await llm.stream(prompt)) {
if (chunk.content) {
fullResponse += chunk.content;
yield {
type: "token",
content: chunk.content,
fullResponse
};
}
}
// Step 4: Complete
yield {
type: "complete",
response: fullResponse,
wordCount: fullResponse.split(' ').length
};
}
// Express route - clean and simple
app.get('/chat/stream', async (req, res) => {
res.writeHead(200, {
'Content-Type': 'text/event-stream',
'Access-Control-Allow-Origin': '*'
});
try {
for await (const update of streamAIResponse(req.query.message)) {
res.write(`data: ${JSON.stringify(update)}\n\n`);
}
} finally {
res.end();
}
});
// Helper function
async function getRelevantContext(query) {
// Simulate vector search
await new Promise(resolve => setTimeout(resolve, 200));
return "Async generators enable memory-efficient streaming...";
}This example demonstrates how async generators naturally handle multi-step AI workflows. Users see progress at each stage — context retrieval, prompt building, and response generation — rather than waiting for a black-box result.
Example 2: Memory-Efficient Data Processing
Beyond AI applications, async generators excel at processing large datasets without consuming excessive memory. Here’s a practical example for processing CSV files:
import Papa from 'papaparse';
async function* processLargeCSV(fileStream, filterFn, transformFn) {
let rowCount = 0;
let processedCount = 0;
// Parse CSV in streaming mode
const parser = Papa.parse(Papa.NODE_STREAM_INPUT, {
header: true,
skipEmptyLines: true,
step: null // We'll handle rows manually
});
for await (const chunk of fileStream) {
const rows = Papa.parse(chunk, { header: true }).data;
for (const row of rows) {
rowCount++;
// Apply filter if provided
if (filterFn && !filterFn(row)) {
continue;
}
// Transform the row
const transformedRow = transformFn ? transformFn(row) : row;
processedCount++;
yield {
data: transformedRow,
progress: {
rowsRead: rowCount,
rowsProcessed: processedCount
}
};
// Yield control periodically to prevent blocking
if (rowCount % 1000 === 0) {
await new Promise(resolve => setTimeout(resolve, 0));
}
}
}
yield {
complete: true,
summary: {
totalRows: rowCount,
processedRows: processedCount
}
};
}
// Usage example
async function analyzeCustomerData(fileStream) {
const results = [];
const filter = (row) => row.purchaseAmount > 100;
const transform = (row) => ({
customerId: row.id,
value: parseFloat(row.purchaseAmount),
category: row.category,
date: new Date(row.purchaseDate)
});
for await (const update of processLargeCSV(fileStream, filter, transform)) {
if (update.complete) {
console.log('Processing complete:', update.summary);
break;
}
results.push(update.data);
// Update UI with progress
if (update.progress.rowsRead % 10000 === 0) {
updateProgressBar(update.progress);
}
// Early exit if we have enough data
if (results.length >= 50000) {
console.log('Collected sufficient data, stopping early');
break;
}
}
return results;
}This pattern is particularly valuable for data analysis applications where users don’t always need complete results. Processing large files with this approach reduces peak memory usage significantly, and users can stop processing when they have enough data.
Real-World Adoption
Async generators are gaining traction in production applications, though adoption is more gradual than revolutionary. Next.js 13+ supports streaming responses that can be implemented using ReadableStream or community wrappers that adapt async generators, allowing server components to yield partial content as data becomes available.
The Quantlane trading platform uses async generators in their ETL pipelines to process financial data streams efficiently. Their approach demonstrates how async generators can handle continuous data flows without the memory accumulation problems of traditional batch processing.
Get Yarinsasson’s stories in your inbox
Join Medium for free to get updates from this writer.
Several npm packages now provide React integration for async generators, including @repeaterjs/react-hooks, which offers hooks for consuming async iterators in React components.
Performance Characteristics
The primary benefits of async generators in data processing applications are:
Reduced Memory Usage: Unlike loading entire datasets into memory, async generators process items individually, reducing peak memory consumption regardless of input size.
Natural Backpressure: If consuming code can’t keep up with data production, async generators automatically pause, preventing system overload without complex flow control logic.
Early Termination: Processing can stop as soon as enough results are found, saving computational resources in scenarios where complete results aren’t necessary.
Responsive UIs: By yielding control periodically, async generators prevent long-running operations from blocking the main thread.
Implementation Considerations
When implementing async generators for data processing, consider these patterns:
Progressive Enhancement: Start with simple functionality and add streaming later. Many applications can benefit from async generators without requiring complete architectural changes.
Error Handling: Async generators propagate errors naturally through the async call stack, but consider how partial results should be handled when errors occur mid-stream.
Cancellation Support: Implement cancellation mechanisms for long-running operations, especially in user-facing applications where requirements might change mid-process.
Memory Management: While async generators help with memory efficiency, ensure that consuming code doesn’t accumulate all yielded values if only recent data is needed.
Looking Forward
Async generators represent a shift toward more responsive, user-centric application design. Rather than optimizing for raw throughput, they prioritize user experience and resource efficiency. This aligns well with modern web development trends toward real-time interfaces and progressive disclosure of information.
As AI applications become more common, the patterns enabled by async generators — streaming results, incremental processing, and graceful cancellation — will likely become standard expectations rather than nice-to-have features.
For developers working on data-heavy applications or AI integrations, async generators offer a path toward more responsive user experiences without requiring complex state management or memory optimization techniques. They provide a straightforward way to handle the “show me results as they come” pattern that users increasingly expect.
The key insight isn’t that async generators solve new problems, but that they make existing solutions more elegant and user-friendly. In an era where user attention is limited and computational tasks are getting more complex, this matters more than raw performance optimizations.
Getting Started
If you’re considering async generators for your applications, start with scenarios where you currently use loading states or progress indicators. Ask whether users would benefit from seeing partial results immediately rather than waiting for complete processing.
The most straightforward applications are data streaming, search result generation, and any workflow where early termination provides value. Focus on user experience improvements rather than architectural complexity, and implement async generators as a progressive enhancement to existing functionality.
The future belongs to applications that respond immediately and refine results over time, rather than those that process completely before responding. Async generators provide a practical path toward that future.
