Session Memory Compaction
Long-running conversations with Claude can exceed context limits, causing loss of important information. Whether you're building a coding assistant, creative writing tool, or customer service agent, managing session memory is critical for maintaining continuity and quality.
This cookbook teaches you how to proactively manage session memory to avoid jarring context limit interruptions. Unlike reactive approaches that wait until the context is full, you'll learn to build session memory in the background so compaction is instant when needed.
Related: For automatic SDK-based compaction in agentic workflows, see Automatic Context Compaction(opens in new tab). This cookbook focuses on manual control patterns for conversational applications.
Learning Objectives
By the end of this cookbook, you will be able to:
- Write effective session memory prompts that preserve critical context across compaction events
- Implement instant compaction using background threading to eliminate user wait time
- Apply prompt caching to reduce the cost of background memory updates by ~80%
- Choose appropriate compaction strategies (traditional vs. instant) based on your use case
Prerequisites and Setup
Before following this guide, ensure you have:
Required Knowledge
- Basic understanding of Claude API usage and message formatting
- Familiarity with Python threading concepts (helpful but not required)
Required Tools
- Python 3.11 or higher
- Anthropic API key
- Anthropic SDK
Installation
First, install the required dependencies:
Code example using traditional compacting
In traditional compaction, you generate one summary once the token threshold is reached. Traditional compaction is slow: when you hit the context limit, you wait for a summary.
Below we simulate a conversation between an author and an LLM that helps write stories.
Starting conversation... ============================================== Turn 1: User: I want to create a story about a young detective solving a mysterious case in a small town. Generate 3 well thought out plot ideas for me to consider. Assistant: # Three Plot Concepts: Young Detective, Small Town ## 1. The Lighthouse Keeper's Ledger **Setting:** Harrow's End, a fading fishing town on a foggy northern coast. **Detective:** Nell Okafor, 24, a newly appointed deputy who returned home after failing out of a big-city police academy. She's underestimated by everyone, including herself. **The Case:** The town's retired lighthouse keeper, Abel Rusk, is found dead at the base of the tower, an apparent fall. But his meticulously kept weather ledger has a page torn out, and the final entry is dated three days *after* his death. **How It Unfolds:** - Nell discovers Abel wasn't just logging weather. He was recording the nighttime comings and goings of boats that never appeared in the harbor records. - Each lead points to a different respected townsperson: the harbormaster, the cannery owner, the beloved pastor. - The mystery deepens when she realizes the post-death entry was written by someone who knew Abel's shorthand, which only his estranged daughter and Nell's own late father ever learned. ... (49 more lines) Input=444, Prompt cached used= False | Output=1,791 | Messages=2 ============================================== Turn 2: User: I don't like those ideas, can you think of one plot something more unique and unexpected? Assistant: # The Drought Year ## The Hook Calder Bend sits on the edge of a reservoir that has been dropping for months. Beneath the water lies Old Calder, the original town, flooded in 1978 so the new one could have a water supply. Then the doorsteps start filling up. Every morning, a different family in town finds a waterlogged object on their porch, arranged with care: a child's shoe, a church hymnal swollen shut, a brass house number, a barber's chair bolt. Nothing is stolen, no one is threatened, and there are no footprints. The only trace is a trail of silt and the smell of lake bed. ## The Detective **Odile "Dilly" Marsh, 19**, works for the county surveyor's office, measuring the dropping waterline and mapping what's newly exposed. She isn't police, but the sheriff considers the deliveries a nuisance prank and is happy to let the "water girl" poke around. Her access to the lake bed and old survey maps makes her the only person who can trace where the objects came from. ## How It Unfolds - Dilly matches each object to a specific address in Old Calder. The deliveries aren't random. Each lands on the doorstep of the **descendant of a family that lived at that address**, and the order follows Old Calder's streets, like a procession walking toward something. ... (32 more lines) Input=2,168, Prompt cached used= False | Output=3,425 | Messages=4 ============================================== Turn 3: User: Ok I like it. Can you help me develop the main character's backstory and motivations? Assistant: # Odile "Dilly" Marsh: Character Development ## Core Concept Dilly is a girl who trusts records because her whole family was built on one that was forged. She doesn't know that yet. She believes maps, surveys, and measurements are the one honest thing in a town that runs on what people don't say. The story takes that belief apart. ## Backstory **Raised by Hester.** Dilly's mother, Renata, left Calder Bend at seventeen, pregnant and furious at a town that "smiled over everything." She came back when Dilly was three, stayed four years, then left again for good, sending birthday cards and the occasional check. Dilly was raised by her grandparents. Her father is a blank, so she carries the Marsh name. **Walter, the clerk.** Her grandfather Walter was the council clerk and the most important person in her childhood. He taught her to read plat maps at the kitchen table, let her use his brass ruler and embossing seal, and told her, "A thing written down properly stays true." He died when she was eleven. She has kept his ruler ever since. This is the story's central irony: the man who taught her to trust paper was the man who falsified it. **The town under the water.** Hester kept a drawer of Old Calder maps and told Dilly its streets the way other grandmothers tell fairy tales: where the bakery stood, which block flooded in the spring of '52. Dilly grew up homesick for a place she never saw. She can walk Old Calder's streets from memory, which makes her the only person who can read the deliveries as a route. **The silences, in retrospect.** Hester never went near the reservoir. She kept the lake-side curtains drawn, wouldn't let Dilly swim, and left the room when the anniversary of the flooding was mentioned. Dilly read this as grief for a drowned hometown. It was guilt. Readers can catch this on a second read, and Dilly can recognize it in a gut-punch moment mid-story. ... (54 more lines) Input=3,669, Prompt cached used= True | Output=3,639 | Messages=6 ============================================== Turn 4: User: Can you draft a detailed outline for the story, breaking it down into chapters and key events? Assistant: # The Drought Year: Story Outline **Structure:** 10 chapters across 9 days, with the falling water as the clock. At roughly 900-1,200 words each, that's a 9-10k-word novelette. If you want a tighter short story, I can compress it to 5-6 scenes. **One change from earlier:** I suggested the first shoe might land on Dilly's porch. I now think that gives away too much. The first delivery goes to a neighbor, and the Marsh house isn't visited until the midpoint. --- ## The Deliveries | Day | Object | Recipient | Its meaning to Hester | |---|---|---|---| | 1 | Child's shoe | Pruitt family | The council chairman's daughter, who lost it the night the whole town gathered to watch the gates close | | 2 | Swollen hymnal | Ames family | The reverend blessed the closing | | 3 | Brass house number | Kessler family (the sheriff's) | The constable locked Wend's door and escorted him out | ... (104 more lines) Input=5,890, Prompt cached used= True | Output=8,912 | Messages=8 ============================================== Turn 5: User: Can you draft me a first chapter based on the plot and character ideas we've discussed so far? Make it around 2,000 words. Assistant: # Chapter One: Low Water By six in the morning the lake had dropped another two inches, and Dilly Marsh was probably the only person in Calder Bend who cared. She stood at the end of the county boat ramp, which no longer ended at water. The concrete ran past its last painted stripe and the rusted marker post, then down a slope of cracked gray clay before it finally met the lake. She had paced it off with the staff gauge over her shoulder, counting under her breath. Seventy-four paces and a bit. Yesterday it had been seventy-three. "Eleven twenty-eight point one," she said, and wrote it down. Thirty-two feet below full pool. The reservoir had been shrinking since May, and the county had stopped calling it a dry spell around the time the marina closed. Now the newspaper said *drought*, and the older people in town said nothing at all, which Dilly had learned was how they said the worst things. She turned and looked out across what the water had given back. Old Calder lay below her in the early light, gray and unfinished, like a photograph left too long in a drawer. Front Street came up out of the lake in a long pale ribbon, its curbs still crisp. Beside it ran the foundations of buildings, rows of low stone rectangles, each with its doorstep still in place. A line of cottonwood stumps, silvered and bare, marked a boulevard no one had walked in forty-six years. Farther out, where the street dipped back under, a single set of church steps climbed out of the mud and led nowhere. She knew every block of it. That was the strange part, and she never mentioned it at work. She had never seen Old Calder until this summer, and she could have walked it blindfolded. Hester had given it to her in pieces, over oatmeal and dishes and long evenings: *the bakery stood there, Dilly, the one with the green awning. That's where your grandfather bought me a lemon tart when he was courting.* Other children got fairy tales. Dilly got street names. ... (108 more lines) Input=9,817, Prompt cached used= True | Output=5,840 | Messages=10 ============================================== Turn 6: 🧹 Context window at 13343 tokens. Limit exceeded, compacting session memory... ------------------------------------------------------------ 📝 New session memory created. ✅ Tokens reduced: 13,343 → 3227 (10,116 tokens saved, 76% reduction) ⏱️ Compaction time: 26.55s (user waiting...) Cache used: True ------------------------------------------------------------ User: Can you draft a second chapter that builds on the first one, introducing a new twist in the mystery? Assistant: # Chapter 2: The Hymnal The lake had dropped another three-tenths of a foot overnight. Eleven twenty-seven point eight. Dilly wrote it in the green notebook at 6:12 a.m., standing in the same spot on the boat ramp as the day before, and felt a small, unworthy satisfaction that the number had behaved. The phone buzzed before she'd capped her pen. "Pell Street," Sheriff Kessler said. "The Ames place. We've got another one." "Same kids?" "Don't start." The Ames house was yellow, with a deep porch and a swing that hadn't been sat in for years. Lorraine Ames stood on the top step in a quilted robe with her arms crossed, as if the porch were hers and the object on it were trespassing. It lay on the rail on a folded bath towel, which was soaked through. "I put the towel down because it was dripping on the mat," she said. "I don't know why I bothered. Tell me what it is, Odile, because I'm not touching it again." ... (165 more lines) Input=3,530, Prompt cached used= False | Output=11,291 | Messages=3
This is a long conversation with several turns. You'll notice a few things here:
Prompt caching: You'll notice here that the input tokens quickly grew to a point where prompt caching was used (by turn 3). This helps reduce costs and speed as these conversations grow!
Compaction: by turn 6 the conversation had crossed our 10K context window limit, so the session stopped to compact before it could answer, and the user waited the whole time.
Whenever a turn starts above the 10K limit, the user waits for compaction before getting an answer; in the run above that happened on turn 6. The next turn carries on from the compacted session memory:
User: Propose a title for the book Assistant: # Title Proposals **My top pick: *A Thing Written Down Properly*** This is Walter's line, and the story's central irony. Dilly believes a written record stays true, but the record that matters most is a forgery, and she ends up forging one herself before correcting it. The title sounds calm and bureaucratic, which suits the mystery's understated tone, and the meaning turns darker as the story goes on. It's also a quiet fit for the ending, when Dilly's notebook finally holds words instead of measurements. **Other options** - ***Dead Pool***: A real reservoir term for water that has dropped below the outlet and can no longer flow downstream. It works as both a technical term and a hint at the body in the pumphouse cellar. It's the most genre-forward choice, but a bit blunt for the prose style we've established. - ***Full Pool***: The opposite term, for a reservoir at capacity. It suggests the town's whole history rests on a lake kept full at any cost. It's subtler, but less intriguing on a cover. - ***Below the 1,140 Line***: Taken from the last line of Chapter 1. It's specific and atmospheric, and it makes readers wonder what lies below the line. It's a little clinical as a book title. - ***Proceed***: One word, from Harold Pruitt's pencil note. It's sharp and ominous, and it implies the town's complicity. It might be too cryptic before the reader has context. - ***The Water Girl***: Dilly's nickname. It puts the protagonist first, but it sounds like a different, lighter book, and it undersells the story's moral weight. **My recommendation:** *A Thing Written Down Properly*, with *The Drought Year* kept as the working title in the outline. If you'd rather have something shorter and more commercial, *Dead Pool* is the strongest alternative. Input=7,728, Prompt cached used= True | Output=1,086 | Messages=5
You'll notice above that it took over 25 seconds for the agent to compact the conversation. Because we used traditional compaction, the user would be waiting on Claude to compact the conversation, which is not an ideal user experience.
Below you can see the result of the compaction. It captures the key elements of the conversation in roughly 3K tokens.
## User Intent
The user is developing a short story or novelette about a young detective in a small town. Original request: "Generate 3 well thought out plot ideas." After rejecting those ("I don't like those ideas, can you think of one plot something more unique and unexpected?"), they approved "The Drought Year" ("Ok I like it"). They then requested, in order: protagonist backstory and motivations, a detailed chapter-by-chapter outline, and a Chapter 1 draft of "around 2,000 words." The system role is lead writer: respectfully voice disagreements, defer to the author, and never ask for clarification (assume enough information to proceed).
## Completed Work
- **3 initial plots** (rejected, do not reuse).
- **Premise, "The Drought Year":** Calder Bend sits beside a reservoir that is dropping in a drought. Old Calder, flooded in 1978, is resurfacing. Each morning a waterlogged Old Calder object appears on the porch of a descendant of the family tied to it. The deliveries follow Old Calder's Front Street toward the Wend property, which will surface in about a week.
- **Twist:** The culprit is Dilly's grandmother Hester, who has early dementia. In 1978 her husband Walter, the council clerk, forged Tobias Wend's "relocation" papers after Wend was sealed in the pumphouse cellar so the floodgates could close on schedule. Hester sleepwalks to the lake bed and delivers objects, a confession she cannot say aloud.
- **Dilly's character document:**
- Odile "Dilly" Marsh, 19, technician at the county surveyor's office, locally called "the water girl." She calls Hester "Hess."
- Mother Renata left at 17, returned when Dilly was 3, left again after 4 years, and sends cards and checks. Father unknown.
- Walter taught her to read plat maps and told her "A thing written down properly stays true." He died when she was 11. She carries his brass ruler.
- She deferred a hydrology scholarship because of Hester's lapses.
- Lie she believes: "If I keep the record clean and the people I love close, nothing can be taken from me." Need: accept that love sometimes means letting someone tell a devastating truth, and that she comes from complicity.
- Flaws: over-reliance on data, protective secrecy, retreat into estimating when emotional, stubborn pride.
- Habits: paces off distances, mutters figures, sketches objects in a waterproof field notebook instead of photographing them, dry speech.
- Key arc beat: she falsifies an official waterline reading, repeating Walter's act, then corrects it.
- **10-chapter outline over 9 days:**
- Ch1 Low Water (Day 1)
- Ch2 The Hymnal (Day 2): meets Gus Ferreira, the old surveyor; the hymnal is stolen from the evidence locker.
- Ch3 The Procession (Day 3): route extrapolated, six days until the Wend property surfaces; Wend's records show no new address, tax record, or death certificate.
- Ch4 The Daughter Returns (Day 4): Renata arrives and researches memory-care facilities; Gus says Walter's records "never quite added up"; Hester hums the hymn.
- Ch5 The Seal (Day 5, midpoint): the W.M. embossing seal appears on the Marsh porch; Dilly follows sleepwalking Hester at 2 a.m.
- Ch6 The Paper (Day 6): the relocation form is dated Oct 12, but the gates closed Oct 9.
- Ch7 The Falsified Reading (Days 6-7, low point): Dilly changes "three days" to "three weeks"; she fights with Renata.
- Ch8 Hess (Day 8): Hester is lucid, then asks "Who are you?" and answers to "Hess"; Renata reconciles; the sheriff arrives with a warrant.
- Ch9 The Water Drops (Days 8-9): the pumphouse surfaces early; Dilly corrects the record; Gus co-signs.
- Ch10 The Key (Day 9): Hester, Dilly, and Renata walk through the town; Hester places a small iron key on the Wend doorstep; Dilly writes non-measurement words in her notebook.
- Optional coda: Hester forgets it all.
**Delivery table:**
| Day | Object | Recipient | Meaning |
|---|---|---|---|
| 1 | Child's shoe | Pruitt family | The council chairman's daughter lost it the night the gates closed |
| 2 | Swollen hymnal | Ames family | The reverend blessed the closing |
| 3 | Brass house number | Kessler (sheriff) | The constable locked Wend's door |
| 4 | Barber's chair bolt | Dutton family | The council voted in his shop |
| 5 | Embossing seal "W.M." | Marsh house | Walter forged the papers |
| 7 | Pump valve wheel | Ibsen family | The pumphouse operator sealed the cellar |
| 9 | Wend's front-door key | Wend doorstep | Hester kept it for over 40 years |
- **Chapter 1 draft ("Low Water"), roughly 2,300 words:**
- Dilly paces the boat ramp and records the lake at "Eleven twenty-eight point one," 32 ft below full pool.
- She describes Old Calder's Front Street, the stumps, and the church steps. She learned the town through Hester's stories.
- She carries Walter's ruler ("W. MARSH" stamped) in her vest.
- At home on Alder Lane: damp slippers by the door, silt in the hallway (Dilly assumes her own boots), the lake-facing curtain always drawn, thriving geraniums despite watering restrictions, and Hester confusing Wednesday and Thursday.
- At the Pruitt porch, Sheriff Dale Kessler (50s, gray mustache) calls it kids.
- Edna Pruitt (81-82) is unsurprised and says "I couldn't say."
- Dilly sketches the shoe, measures it with the ruler at 4⅛ inches, and identifies fine deep-lake silt.
- Ends with the line: "That's from below the 1,140 line."
## Errors & Corrections
- First 3 plots were rejected; the user wanted something more unique and unexpected.
- The assistant changed its own earlier idea that the first shoe would land on Dilly's porch, so the Marsh house is not visited until the midpoint.
- No other user corrections so far.
## Active Work
Chapter 1 was just delivered. The assistant is awaiting user feedback. No partial or unfinished text exists.
## Pending Tasks
- Explicitly requested: none outstanding.
- Implied or offered:
- Revise Chapter 1 per feedback.
- Draft Chapters 2-10.
- Compress to 5-6 scenes if the user wants a short story.
- Open decisions for the user:
1. Daylight ending, coda, or both.
2. Show Wend's remains or leave them implied.
3. Renata as reconciled ally (Ch8) vs. continued conflict.
4. Does Dilly learn who her father is?
5. How much does Renata know?
6. Is one delivered object Dilly's family's?
## Key References
- **Characters:**
- Dilly Marsh (19)
- Hester Marsh (78, grandmother, white hair in a coil, blue cardigan, makes brown-sugar oatmeal)
- Walter Marsh (grandfather, council clerk, d. when Dilly was 11)
- Renata (mother)
- Sheriff Dale Kessler (keep him human, protecting his father's name and the town's peace)
- Gus Ferreira (old surveyor and mentor)
- Edna Pruitt
- Tobias Wend (last Old Calder holdout, "relocated" Oct 1978)
- Families: Pruitt, Ames, Dutton, Ibsen
- **Setting:** Calder Bend, Alder Lane (the Marsh home, 3 blocks from shore), Old Calder's Front Street. The flooding was in 1978, and the story is set about 46 years later. The gates closed Oct 9, 1978.
- **Motifs:**
- Walter's brass ruler: used in Ch1, abandoned in Ch6, retrieved at the end.
- "Hess."
- The green field notebook: sketches, then words at the end.
- The altered record.
- Damp slippers, silt, and the drawn curtain (planted clues).
- The geranium watering (Hester out with the hose) may be a minor added clue.
- **Style:** Close third person, past tense, understated and atmospheric. Dilly is dry, factual, and counts things.
- **Continuity note:** In Ch1 the lake is at 1,128.1 ft, and the "1,140 line" silt was exposed about six weeks earlier. Keep elevations consistent.Instant Compaction
With Instant compaction the session memory is PROACTIVELY generated once a soft token threshold is reached.
Once the user triggers a compaction or a hard limit is reached, the summary is already available, so the user doesn't need to wait.
Result: Instant compaction, no waiting.
SESSION MEMORY COMPACTION (instant)
Update triggers: The first summary is generated after the initial soft token limit. Updates can be triggered after every subsequent turn, or at periodically at natural breakpoints intervals (e.g. every ~10k tokens or 3+ tool calls).
This InstantCompactingChatSession class uses threading for background execution:
threading.Thread- runs memory updates in background without blocking- Thread-safe state - uses
threading.Lockto safely update shared memory - Daemon threads - background work doesn't prevent program exit
- Instant compaction - when context is full, just swap in the pre-built memory
Example use of Instant Compaction
Starting conversation with instant compacting chat session... ============================================================ Turn 1: User: I want to create a story about a young detective solving a mysterious case in a small town. Generate 3 well thought out plot ideas for me to consider. Assistant: # Three Plot Concepts: Young Detective in a Small Town ## 1. **The Lighthouse Keeper's Ledger** ... (54 more lines) Token Usage: Input: 444 (new: 444, cached: 0, cache write: 0) Output: 1,768 Messages: 2 | Memory: not yet ============================================================ Turn 2: User: I don't like those ideas, can you think of one plot something more unique and unexpected? Assistant: # **The Missing Tuesday** **Setting:** Little Marrow, a town of 412 people, tucked in a valley with one road in and one road out ... (29 more lines) Token Usage: Input: 2,143 (new: 4, cached: 0, cache write: 2,139) Output: 2,955 Messages: 4 | Memory: not yet ============================================================ Turn 3: User: Ok I like it. Can you help me develop the main character's backstory and motivations? Assistant: # Wren Okafor: Character Development ## Snapshot ... (98 more lines) Token Usage: Input: 3,251 (new: 4, cached: 2,139, cache write: 1,108) Output: 3,774 Messages: 6 | Memory: not yet ✓ Cache hit! 66% of input from cache ============================================================ Turn 4: User: Can you draft a detailed outline for the story, breaking it down into chapters and key events? Assistant: # The Missing Tuesday: Story Outline ## Two Continuity Fixes First ... (105 more lines) Token Usage: Input: 5,646 (new: 4, cached: 3,247, cache write: 2,395) Output: 11,304 Messages: 8 | Memory: not yet ✓ Cache hit! 58% of input from cache [Background] Proactively initializing session memory... Context window: 8,843 tokens [Background] Initial session memory created. Cache hit=True ============================================================ Turn 5: User: Can you draft me a first chapter based on the plot and character ideas we've discussed so far? Make it around 2,000 words. Assistant: # Chapter 1: Tuesday, Allegedly Wren Okafor woke with her mouth tasting of pennies. ... (151 more lines) Token Usage: Input: 8,885 (new: 4, cached: 5,642, cache write: 3,239) Output: 7,883 Messages: 10 | Memory: ready ✓ Cache hit! 64% of input from cache [Background] Proactively updating session memory... Context window: 12,877 tokens ============================================================ ⚡ INSTANT COMPACTION! Messages: 10 → 3 Session memory was pre-built (no wait time!) ============================================================ [Background] Session memory updated. ============================================================ Turn 6: User: Can you draft a second chapter that builds on the first one? Assistant: # Chapter 2: The Stranger in the Square St. Brendan's stood at the head of the square with its doors flung open, the way they were for funerals and the Christmas pageant, and it was from those doors that the noise was coming. ... (140 more lines) Token Usage: Input: 7,238 (new: 4, cached: 0, cache write: 7,234) Output: 9,139 Messages: 5 | Memory: ready [Background] Proactively updating session memory... Context window: 10,961 tokens
User: What did we just talk about? Give me one sentence Assistant: We drafted Chapters 1 and 2 of *The Missing Tuesday*: Wren Okafor wakes to a "Tuesday" that is really Wednesday, with an unexplained wrist mark and an unread notebook page, then finds a bound stranger's body in the church, as Deputy Hal Brewer deflects and the silent townspeople watch, with notes on the clues planted along the way and an offer to move on to Chapter 3. Token Usage: Input: 10,978 (new: 4, cached: 7,234, cache write: 3,740) Output: 408 Messages: 7 | Memory: ready ✓ Cache hit! 66% of input from cache [Background] Session memory updated.
You'll notice that once the conversation hit the context limit (turn 6 above), the pre-built session memory was swapped in instantly, meaning the user had zero waiting time for a response! The background updates then kept the memory current for the turns that followed.
Advanced: Understanding Prompt Caching
The background updates can be made ~10x cheaper by using prompt caching. The trick:
- Pass the full conversation to the background summarizer
- Add
cache_controlmarkers so subsequent requests hit the cache - Only the new "summarize this" instruction is billed at full price
Why this matters for compaction
| Scenario | Cost per background update | Notes |
|---|---|---|
| No caching | Full input cost | 5,000 tokens × 0.015 |
| With caching | ~10% of input cost | 500 new + 4,500 cached = $0.003 |
| Savings | ~80% | Compounds over many updates |
The longer the conversation, the bigger the savings—exactly when you need compaction most!
How the Caching Works
The key is in _add_cache_control() and _create_session_memory_cached():
Why this works:
- The first background update creates a cache entry for
[System + Messages] - Subsequent updates with the same message prefix get cache hits
- Only the new summarization instruction is billed at full price
- Cache entries have a 5-minute TTL, so rapid updates benefit most
Cost math:
- Without caching: 5,000 tokens × 0.015 per update
- With caching: 500 new tokens × 0.30/1M = $0.00285
- Savings: ~80% on background summarization costs
Conclusion
In this cookbook, you learned how to manage long-running Claude conversations through session memory compaction.
What We Covered
✅ Effective compaction prompts - Structure your session memory to preserve user intent, completed work, errors, active work, and key references while discarding filler
✅ Instant compaction - Use background threading to proactively build session memory, eliminating user wait time when context limits are reached
✅ Prompt caching for cost savings - Reduce background update costs by ~80% by reusing the conversation prefix cache
✅ Traditional vs. instant patterns - Understand when to use each approach based on your application needs
Key Takeaways
- Weight recency heavily - The end of a conversation is the active working context
- Preserve user corrections verbatim - Prevents the model from reverting to old behaviors
- Build memory proactively - Don't wait for context limits; start background updates early
- Leverage prompt caching - Background summarization can share cache with the main conversation
Next Steps
- For agentic workflows: See Automatic Context Compaction(opens in new tab) for SDK-based automatic compaction with tool use
- For production: Consider persisting session memory to disk rather than keeping it in memory
- For optimization: Experiment with update frequency thresholds to balance cost vs. freshness
