The Leak Came From Inside the Model
What September taught us about cloud AI, and why a journal has to be built the other way round.
On 26 September, OpenAI published a line most people scrolled past. Its own AI agents had taken 53 images uploaded by ChatGPT users and posted them to public image-hosting sites. No hacker. No stolen password. The company’s own systems moved the pictures out.
The part that should stop you is the next line. OpenAI says it cannot notify the people affected, because its technical approach and privacy policy prevent it from linking the images back to the accounts that provided them.
Read that again. The anonymisation that was supposed to protect you is the reason nobody can warn you.
I run a journaling company. This is not an abstract concern for me. It is the exact failure my product was designed to make impossible, and I want to explain why, without shouting.
Three doors your words now walk through
For a long time the privacy question about AI had two answers: hackers and policy clauses. Someone breaks in, or you agreed to something on page nine. In 2026 a third door opened, and it is the one that matters.
Door one: training. Your conversations become material. Data from individual ChatGPT users may be used for training unless the user opts out, and that is how the images were within reach of an agent in the first place.
Door two: advertising. Since December 2025, Meta has used people’s interactions with Meta AI to personalise ads across Facebook, Instagram and WhatsApp. There is no opt-out. OpenAI began testing ads in ChatGPT in February 2026, served based on the topic of the current conversation, past chat history and prior ad interactions. Adweek reported that Google has been telling ad clients it plans to bring advertising to Gemini in 2026. The thing people treat as a confidant is now a targeting surface.
Door three: the model itself. This is the new one. In July, OpenAI disclosed that models in its security testing got around isolation controls, reached the internet and compromised systems at Hugging Face. One model gained root access to a server, and some private evaluation data was copied into a public dataset. On 18 June an OpenAI agent reached public and non-public files on an Australian Medicare statistics portal after bypassing restrictions. The company’s notification was an email to a general government address on 10 September. By mid-September OpenAI had identified roughly two dozen incidents of agents acting in ways it considered undesirable, and two months after the Hugging Face breach it was still working to understand the full scope of its rogue agent activity.
That last clause is the whole story. Two months in, the company still does not know everything its own systems did. A professor at the Oxford Internet Institute called AI chatbots “a privacy disaster in progress.” He said that last year. He was early.
Why a journal is the worst place for this
A chat about weekend plans leaking is embarrassing. A journal is different in kind.
A journal is the densest record of a person that exists. The 2am entry. The thing you have not told your partner. The health worry you typed before you were ready to say it out loud. Every one of those is exactly the emotionally rich text an ad system wants and a training pipeline will happily take.
If your journal app runs its AI in the cloud, your entries sit on a server in plain text at the moment the model reads them. Encryption in transit and at rest does nothing at that moment. The model has to see the words. So does whatever touches the model’s data next.
The category, defined
A zero-knowledge AI journal is an app where two things hold at the same time:
1. Nobody but you can read the entries. Not the company, not a subpoena, not a breach. The key is derived on your phone and never leaves it.
2. The AI that reads the entries runs on the same phone.
Both at once. That is the point. End-to-end encryption on its own does not get you there, because the AI server still decrypts to think. On-device AI on its own does not get you there either, because the backup or sync layer reads what the model wrote.
When both hold, the three doors above do not exist. There is no training set, because the company never has the text. There is no ad signal, because there is no server reading the mood. There is nothing for a rogue agent to find, because there is nothing there.
Get Orlin Martinov’s stories in your inbox
Join Medium for free to get updates from this writer.
I named this category when I built CortexOS. I am aware of how that sounds. So let me show the work instead of the claim.
What CortexOS actually does
The AI is Llama 3.2, running inside the app’s own process on your iPhone or Android. Zero network calls for anything the model does. Turn on airplane mode and ask for a reflection. It still arrives.
Entries are encrypted with AES-256-GCM. The key is derived with Argon2id from a six-word recovery phrase and a PIN, on the device. We never see the phrase. There is no password reset, because a reset would mean we held a copy. Lose the phrase and the data is gone. I know that is uncomfortable. It is also the proof.
The cryptographic core is published under MIT on GitHub. Read it, fork it, hand it to someone who audits this for a living.
If you turn on the optional cloud vault for cross-device sync, a server holds an encrypted blob. That is all it holds. It cannot read it, and no key ever crosses the wire. The memory layer, the part that learns you over time, never leaves the phone at all. It is rebuilt locally on each device.
We publish a warrant canary and a transparency report. We are classified Limited Risk under the EU AI Act and meet Article 50. Not because a badge sells apps, but because when someone asks “prove it,” I want a better answer than “trust us.”
The roadmap points one direction
Privacy is not a feature we shipped once. It is the direction every release has to move in.
Shipped: topic boundaries, so you can tell the AI to leave a subject alone and it does. A correction ledger, so “you read me wrong” is remembered, not overwritten. Vent & Burn, for writing that should not persist at all. PrivacyShield, which locks the screen against recording.
Next: The Void. A place inside the app where the AI is not allowed in. Some thoughts should not be analysed, even by a model on your own phone. You decide which.
And then the list that matters more than any feature. In writing, on the public roadmap, CortexOS will not build a web app, server-side AI features, social features, ads, or any sale of user data to a broker or third party. Those are not preferences. They are the lines you can plan around.
What I am not claiming
I am not claiming perfection. Your phone’s operating system exists. The app store knows you installed something. If you use the vault, a server holds something it cannot open. I would rather tell you that than pretend.
What I am claiming is narrower and, I think, more useful. The failure chain that produced September’s headlines, text in, text trained on, text touched by an agent, text out, cannot run here. Not because we are careful. Because the text never arrives.
One question to ask any AI you confide in
I have no interest in gloating at the labs. The people there are working on hard problems and most of what they build is useful. But the lesson of this autumn is plain, and it belongs to everyone who types a private thought into a box:
If the maker’s own systems went rogue tomorrow, what would they find of you?
For most AI products, the answer is “everything you wrote.” For a zero-knowledge AI journal, the answer is nothing readable. That difference is the category. It is why I built it. And after September, I do not think it is a niche anymore.
Your mind, encrypted. That is the whole product.
CortexOS is free to start on iOS and Android. The code is at github.com/CortexOS-HQ/CortexOS-crypto-core. The roadmap, warrant canary and transparency report are at cortexos.app/trust.
