This is a submission for the Weekend Challenge: Generosity Edition
Don't Just Ask AI. Give the Answer Back.
AI is a real force multiplier for software development. It's also the ideal companion for solving technical problems fast. But all that knowledge — we keep it to ourselves. Or rather, we lose it.
The story always stops there.
Question → answer → problem solved — and the conversation sinks into the chat history, gone.
Then someone else hits the exact same wall. Same cycle: question → answer → problem solved — and the conversation sinks into the chat history, gone.
That's the problem. Not that AI can't solve the same issue twice — it's that a working solution already exists somewhere: someone already investigated, tested, found the fix, and had a conversation detailed enough to explain it properly.
Why should that knowledge evaporate the moment the session ends? Why keep asking the same question over and over — burning electricity, water, and time that's already been spent — instead of recycling that raw material?
That's the idea behind Shared Knowledge MCP.
What I Built
Shared Knowledge is an MCP server that turns a solution from an AI conversation into a proposed Markdown article, then into a GitHub Pull Request submitted for human review. Once merged, the contribution is published to a documentation site and gets an audio version generated with ElevenLabs.
The project turns a solved problem into a reusable piece of community knowledge — but only when the user makes the explicit decision to share it.
The conversation itself stays strictly private. The MCP server extracts only the relevant solution, structures it as a standalone English Markdown article, validates it, and opens a Pull Request on GitHub.
Nothing gets published automatically. A human reviews the contribution and decides whether it belongs in the shared knowledge base. Only once the PR is merged does the article land on the public documentation site, which in turn kicks off its audio version.
The pipeline is deliberately minimal:
AI conversation → explicit sharing → MCP → Markdown → Pull Request → human review → merge → docs + audio
The one boundary that matters is human review. The MCP can structure knowledge and prepare a contribution — it can't decide, on anyone's behalf, what deserves to become public knowledge.
Demo
The public documentation site is live:
It currently hosts four published articles, each with a generated audio version.
To prove the system actually works across different clients — no shortcuts, nothing hard-coded — I tested publish_knowledge through two very different paths.
Direct script execution
The first contribution published through Shared Knowledge went out as a real Pull Request: Optional dependency crashes the import chain when the import itself isn't optional.
That PR walks through the whole flow end to end:
- a real technical problem solved with an AI assistant (an optional dependency in
pyproject.tomlthat crashed the import chain); - an explicit turn into a structured Markdown article (sections, metadata, tags);
- automatic validation by the MCP server, which opens the PR;
- an automated review from GitHub Copilot, flagging YAML formatting and heading issues;
- fixes applied, then a merge into
main; - and, after the merge, an ElevenLabs audio version generated and published to the docs site.
That first PR is also a concrete act of generosity: someone took the time to turn their fix into a resource every future developer hitting the same wall can reuse.
Conversational Agent mode
Next, I used a real AI client — GitHub Copilot Chat in Agent mode, running inside a Codespace.
Here, the assistant spontaneously started by querying the existing base through search_knowledge to check for duplicates before deciding to publish: PR #2.
That behavior wasn't hard-coded anywhere in the MCP server: the server just provides the tools, and the calling assistant decides how to use them. Shared Knowledge isn't a closed AI app tied to one model — it's an open MCP interface for sharing knowledge.
Code
The full project is open source on GitHub:
pcescato
/
shared-knowledge
community-driven knowledge base built around AI-assisted problem solving
Shared Knowledge MCP
Turn solved problems into shared knowledge. Don't just get the answer. Give the answer back.
Shared Knowledge MCP is a community-driven knowledge base built around AI-assisted problem solving. It is an MCP (Model Context Protocol) server that lets any MCP-compatible AI assistant (Claude, ChatGPT/Codex, Cursor, …) search a shared knowledge base before solving a problem from scratch, and publish a freshly solved problem as a community knowledge article — submitted as a GitHub Pull Request for human review, then published as a static documentation website.
The core principle:
The conversation remains private. The knowledge extracted from it can be shared. Sharing is always explicit and voluntary — nothing is ever published without the user asking for it.
Private conversation
↓
AI-assisted solution
↓
User chooses to share
↓
Caller structures the article (guidelines prompt)
↓
GitHub Pull Request
↓
Human review
↓
Shared knowledge base
↓
Available…The repo includes the MCP server, the knowledge articles, the Astro/Starlight static site, the GitHub Actions workflows, the test suite, and the documentation.
How I Built It
Weekend-MVP pragmatism
Building a solo PoC in a weekend forces hard limits.
I could have designed a PostgreSQL database, added vector search with embeddings, built a full authentication system, and shipped an admin dashboard.
I didn't.
For this MVP:
- GitHub is the only source of truth. The knowledge base is plain Markdown. Git already handles versioning, history, branches, diffs, and code review — no reason to rebuild any of that.
- Astro + Starlight handles the rendering. No need to run an application server around the clock just to serve static content that a human has already approved.
- Search stays intentionally simple. Field-weighted keyword search is more than enough at this scale. At four articles, adding a vector database wouldn't have made the system smarter — it would have just added weight to the prototype for no real benefit.
Three narrowly-scoped MCP tools
The server exposes three deliberately simple tools:
-
search_knowledge— searches the base with field-weighted ranking. -
get_knowledge— fetches a specific article safely (path-traversal protected). -
publish_knowledge— validates the structure and opens a GitHub Pull Request.
Using it today
Any stdio-compatible MCP client can connect to the server — Claude Desktop, VS Code with GitHub Copilot Chat in Agent mode, and others. A minimal .vscode/mcp.json is enough:
{
"servers": {
"shared-knowledge": {
"type": "stdio",
"command": "${workspaceFolder}/.venv/bin/python3",
"args": ["${workspaceFolder}/server.py"],
"env": {
"GITHUB_TOKEN": "${env:GITHUB_TOKEN}",
"GITHUB_REPO": "pcescato/shared-knowledge",
"KNOWLEDGE_DIR": "${workspaceFolder}/knowledge"
}
}
}
}
Clone the repo, pip install -e ., set your own GITHUB_TOKEN, point it at your own knowledge repo — and any assistant that speaks MCP can start reading from and contributing to the base, opening Pull Requests under your own GitHub identity.
The architectural pivot: interoperability over prize categories
In my first implementation, the MCP server called Gemini directly, internally, to turn a conversation summary into a structured article. It worked — but it went against the whole point of MCP: the server was turning into a monolithic app tied to one specific AI vendor.
So I reworked the architecture mid-build.
The server now exposes an MCP prompt (knowledge_article_guidelines) that lays out the structuring rules — English output, mandatory ## Problem / ## Solution sections, a controlled category and tag vocabulary. It's the calling assistant that uses its own model to structure the content, while publish_knowledge only validates and publishes.
It doesn't matter whether the client runs on Claude, GPT, Gemini, or a local model.
That choice had a real cost for the challenge: by taking Gemini out of the critical path, I knowingly gave up eligibility for the Best Use of Google AI prize. But I'd rather submit a project whose architecture stays true to the idea of interoperability than force a dependency in just to check a prize-category box.
Multi-agent collaboration: practicing what it preaches
There's a nice loop in this project: it was built using exactly the paradigm it argues for.
Instead of relying on a single assistant, the repo was developed by having several AI environments collaborate, each in a specific role, under human supervision:
- FreeBuff + GLM 5.3 — drove development through seven sequential, sealed-off batches, each with explicit guardrails ("Do NOT implement...", "Do ONLY...") to keep scope from drifting and stop one component's changes from breaking another.
- GitHub Copilot CLI + Haiku 4.5 — acted as an independent code reviewer, inspecting PRs and running critical checks.
- OpenCode + Big Pickle — handled local code iteration, refactoring, fast edge-case fixes, and bug squashing.
Real bugs from the real world
A weekend PoC is a fast reminder that architecture diagrams aren't reality:
The AI-generated bug that wasn't there
An automated review from Copilot CLI reported a very convincing GitHub authentication bug — code excerpt and line number included. I checked before touching anything: the line, and the code it described, simply didn't exist in the repo. A useful reminder that AI review is genuinely valuable, but it still needs a human to check its homework.
The ElevenLabs Voice Library trap
Audio generation kept failing with a deeply misleading invalid_uid error. The voice picked from the web interface belonged to the shared Voice Library, which isn't reachable through the API on a free account. Once I added the voice to "My Voices," a second issue surfaced: an empty ELEVENLABS_MODEL_ID variable in GitHub Actions was silently overriding the code's default value — because os.environ.get() only falls back to its default when the key is missing, not when it's set to an empty string.
The manually-broken state problem
During a local test, I deleted an MP3 file without updating .audio_manifest.json to match. On the next CI run, the workflow read the manifest and concluded the audio was already up to date. The code wasn't broken — I had just thrown the system's state out of sync by editing files by hand.
What's Left to Do
Improving audio generation
Audio generation through ElevenLabs happens strictly after human review and merge. No audio is ever produced for an unreviewed Pull Request.
That said, listening back to the generated MP3s surfaced an interesting engineering reality: a format that's perfectly suited for human reading and Git diffs isn't necessarily suited for a text-to-speech engine. Markdown is built for Git, not for speech.
When a TTS engine reads raw Markdown, section headings (## Problem, ## Context) run straight into the following text with no real prosodic break. The result lacks breathing room and narrative structure.
The audio pipeline will need an intermediate scripting step:
Validated Markdown article
↓
Structure parser
↓
Narration script (pauses & transition cues)
↓
ElevenLabs API
↓
Final MP3 file
Instead of sending raw Markdown syntax straight to ElevenLabs, the pipeline will turn the editorial structure into an actual narration script. The Markdown article stays the single source of truth — the audio becomes a purpose-built derived format.
The obvious temptation would be to turn this into a full community platform — voting, comments, vector search, dashboards.
I don't think that's the right priority. The point, first, is to validate the simplicity of the loop itself: someone solves a problem → chooses to give the answer back → someone else gets to reuse it.
The natural next step toward a genuinely usable product is hosting the MCP server remotely over Streamable HTTP — no local install, just a URL added to whatever assistant you already use.
But the loop doesn't need voting, dashboards, or a bigger model to work. It needs someone willing to give the answer back — and someone else willing to trust it enough to read it.
Prize Categories
Best Use of ElevenLabs
Shared Knowledge uses ElevenLabs to make validated technical solutions accessible as audio. Audio generation is decoupled from the MCP server and runs exclusively inside GitHub Actions after a Pull Request is merged — guaranteeing that only reviewed, approved content ever gets synthesized into speech.
Not submitted for Best Use of Google AI: the architectural pivot described above deliberately took Gemini out of the critical path in favor of model-agnostic structuring. That's a trade made on purpose, not an oversight.
Top comments (66)
One angle I haven’t seen mentioned yet is what happens when the knowledge base itself becomes untrusted model input.
Since these articles aren’t only documentation for humans but are retrieved through MCP and consumed by agents, a merged contribution effectively enters the agent’s context boundary. That makes this a knowledge-poisoning / indirect prompt injection problem as well as a content review problem.
Human PR review is a strong publication gate, but “this technical solution is correct” and “this Markdown is safe for an AI agent to consume as data rather than instructions” are slightly different checks.
I’d be interested in how you see that boundary evolving as contributions become less trusted: whether retrieved articles should be structurally treated as quoted/untrusted data, whether certain content patterns should be rejected at publication time, or whether the MCP should enforce that distinction independently of whatever client/model is calling it.
The model agnostic design actually makes that question more interesting, because you can’t assume every downstream client will interpret retrieved content with the same trust semantics.
Maybe I just missed this being mentioned but there are a lot of comments in this post. Good work
I think there may be a slight category error here.
The knowledge base isn't intended to be an untrusted collection of arbitrary model-generated content. A contribution goes through the MCP, becomes a PR, and is explicitly reviewed by a human before it becomes part of the knowledge base.
So if a reviewer validates the solution, the resulting Markdown is simply validated knowledge. The fact that it can subsequently be consumed either by a human or by an AI doesn't fundamentally change the trust model of the knowledge itself.
Of course an agent can be vulnerable to prompt injection from external content in general, but that isn't specific to this knowledge base. The same question applies to any documentation, API response, web page, repository, etc. that an agent is allowed to consume.
I would therefore distinguish “untrusted contribution” from “published knowledge”. The former is exactly why the PR/review boundary exists. Once merged, the content has passed that boundary.
If we started treating every published knowledge artifact as inherently untrusted model input, we'd effectively be saying that human validation doesn't establish anything about the trustworthiness of the knowledge — which isn't really the model I'm building.
That said, the distinction between knowledge and instructions to the agent is an interesting one. But I'd see that as a property of how a particular MCP client consumes the returned data, rather than something that turns a validated knowledge base into a poisoning vector by itself.
I think I may have used “untrusted” too broadly in my first sentence, because I’m not arguing that a merged article should remain epistemically untrusted after human review.
What I’m distinguishing is trust in the knowledge from authority over the consuming agent.
A reviewer can establish “this is a technically valid solution” without necessarily establishing “this content is authorized to influence downstream agent control flow.” To me those are separate properties.
So I’m not sure the distinction is really untrusted contribution vs. published knowledge. I think it’s more like publication trust vs. execution authority.
And I agree this isn’t unique to this knowledge base. Documentation, API responses, repositories, and web pages all create the same boundary when agents consume them. That’s actually why I’d expect the MCP client to preserve a strong data/instruction distinction regardless of how trusted the upstream source is.
In that sense I think we may be closer than my use of “untrusted” initially made it sound.
Yes, I think I see the distinction you're making, but I'd probably stop the boundary there.
My responsibility is to make the application robust and ensure that what gets published has been properly validated. Whether a human reads that knowledge, an agent consumes it, or someone tries to use it in a way I never intended isn't something the knowledge base itself can really control.
So for me the important boundary remains the publication gate: is this knowledge valid and safe to publish? Once published, it's a knowledge resource. What a downstream system does with it is a separate problem.
That makes sense. I think we’re aligned on the distinction then we’re mostly drawing the system boundary at different places. Your publication gate establishes the properties of the knowledge resource, while the consuming client owns the instruction/authority boundary downstream. Thanks for engaging .
Exactly. I think that's where we landed. Thanks for the thoughtful discussion! 👍
This raises some important points. In practice, I've found that the key is balancing theoretical best practices with pragmatic trade-offs — what works in a blog post doesn't always survive contact with a legacy codebase.
Absolutely. And I think that's actually one of the reasons I'm interested in synthesizing knowledge rather than simply collecting “best practices”.
A solution can be perfectly valid in isolation and still be the wrong recommendation once you add a legacy codebase, existing constraints, or a particular environment.
That's another reason why I don't want Shared Knowledge to behave like a static cookbook. The useful part is not just “this works”, but understanding the scope and conditions under which it works — and being explicit when those conditions matter.
In the end, “pragmatic trade-offs” are often part of the knowledge, not a deviation from it.
Treating knowledge as a living synthesis rather than a static dump of best practices is exactly what prevents technical debt from compounding. When we ignore legacy constraints, even the most elegant isolated solution becomes a liability. I'm curious how you plan to structure the MCP to continuously ingest and adapt to those environmental shifts without turning into just another outdated wiki.
The architectural pivot is the part that stuck with me most, deliberately taking Gemini out of the critical path and giving up the prize category eligibility rather than forcing a dependency that would have contradicted the whole point of the project. That's a genuinely principled call and it shows in the architecture.
The "conversation stays private, knowledge extracted from it can be shared" boundary is also really cleanly drawn. A lot of knowledge-sharing tools blur that line or make sharing the default, and the explicit human review gate before anything goes public is the kind of design decision that feels obvious in hindsight but actually requires discipline to hold to when you're building fast.
The ElevenLabs Voice Library bug is a useful one too, the empty environment variable silently overriding the default because os.environ.get() only falls back when the key is missing rather than when it's empty is exactly the kind of thing that takes way too long to find the first time you hit it.
Thank you — especially for taking the time to read the article at that level of depth.
What you picked up on is actually very close to what I was trying to do with the article itself. I deliberately didn't want to turn it into a polished “here's what I built and why it works” story. I wanted to document the reasoning, the architectural decisions, the things that didn't work, and the solutions that emerged from those problems.
In a way, I tried to apply the same approach to the article as to the project: don't hide the messy parts behind the final result, because that's often where the useful knowledge is.
So I'm particularly glad you noticed the Gemini decision, the privacy/knowledge boundary, and even the rather mundane ElevenLabs environment-variable bug. Those details are part of the story precisely because they shaped the system that eventually emerged.
And yes, that
os.environ.get()behavior is one of those wonderfully small bugs that can consume a disproportionate amount of time the first time you encounter it. 😄Well, technically Gemini users can still use it? While not explicitly used, it does offer a service worthwhile having. I know that finding solutions is often hit-or-miss, especially now, so having a repo to query, with a MCP in order to prioritize it, should help alot of people with common issues.
Especially if it comes to project scoped work, eg. how to set up a CLAUDE.md file for a Blazor project. It becomes a template grab system that works similar to how we used to use stack overflow.
Overall really neat project that tackles a real world frustration, getting working solutions, that would make alot of people's lives easier, especially given that it's user-fed, it can really push the boundaries.
Yes — and that's an important distinction. Removing Gemini from the critical path doesn't mean Gemini users can't benefit from Shared Knowledge. Quite the opposite: the MCP is model-agnostic precisely so that the knowledge layer can sit underneath whatever AI assistant or model you're using.
And I really like the Stack Overflow comparison. In a way, that's exactly the kind of friction I'm trying to address: instead of asking an AI to rediscover the same solution from scratch every time, it can first query knowledge that came from someone who already went through the problem.
The project-scoped example is a particularly good one. Something like “how should I structure a CLAUDE.md for a Blazor project?” could eventually become reusable knowledge rather than another one-off answer generated from scratch.
The important difference from Stack Overflow, though, is that I'm trying not to turn it into a collection of individual answers. The goal is to progressively synthesize those experiences into generalized knowledge: not “here's how Pascal solved it”, but “here's what actually works for this class of problem”.
And that's also where the human review becomes important. If this really becomes user-fed at scale, the interesting challenge isn't just collecting more knowledge — it's making sure the knowledge base gets better as it grows.
Thanks for the thoughtful comment! That's very close to the direction I'm hoping to take this. 😄
Hm... Would also be a good system to have a 'librarian' model for. Something tiny, literally 100m params, that serves as a lookup index. Model A that you're using, queries librarian model, who essentially uses the context from Model A, to index and find relevant data from the knowledge base, instead of Model A having to do it, that way it saves a ton of tokens on an expensive model, instead uses one that runs practically for free. I had thought for a while now that we've been seriously under utilizing tiny models, eg. they could be used to strip credentials from prompts as a pre-filter, then index the real credentials vs their placeholder, so when the response comes back, it can re-insert it. This would be another excellent use-case. I really need to build 1 at some point to test the theory 😂
I think the “librarian” could become really interesting at scale, but I probably wouldn't introduce it too early.
For a corpus of a few thousand answers — even a few tens of thousands — a good full-text search should be more than sufficient, especially for a technical corpus in English. In that situation, adding a model just to decide what to retrieve could actually add complexity without giving us much in return.
Where I could see your idea becoming really useful is when the corpus gets large enough, or when queries become semantically ambiguous enough, that retrieval itself becomes the bottleneck. Then a tiny specialised model acting as a router/reranker could be a very interesting optimisation.
So I'd rather let the simple solution hit its limits first, measure where it actually breaks, and only then introduce a librarian if the data justifies it.
Which is perhaps another good example of the principle behind the project: don't add an AI component just because you can. 😄
Very true, definitely not something meant as a small scale solution where postgres + pgvector would do a better job, but lets say it gets to stack overflow's scale... That's when the answer hides in plain sight and while a vector lookup can help, it'll be too specific to catch every case and if a LLM needs to reason through a knowledgebase to get to the answer, it loses the efficiency gain, albeit a worthwhile sacrifice for accuracy, it's still a solution derived from a different problem, which means 99% right, 99% of the time.
But overall, on it's own, I dont think it'd be worth it. Rather, I'm more interested in the concept of a drafter for general purpose reasoning. How many tokens do we burn on grep searches? Or indexing a codebase, or diffing vs git? All things that are braindead simple for a LLM to do, yet people waste 3t models on it daily, while their cpu's NPU and gpu sit idle? The use-cases are endless, eg. credential stripping JIT before sending the content to a cloud LLM? Acting like a safe guard between the model and the tool?
This is the part I'd bet on soonest. I build an MCP that a coding agent calls into, and the thing that keeps proving true is that the mediating server, not the model, is where any rule you actually need to hold has to live. The model will call your tools in whatever order it feels like, skip the check step, or confidently act on a premise that was never there, so treating the MCP layer as the place that enforces and validates rather than trusting the caller has saved me more grief than any prompt tweak. The tiny-model-as-pre-filter idea fits that nicely: cheap, deterministic, and it never has to be talked into doing its job.
Yes — I think we're converging on the same architectural principle from two slightly different angles.
The MCP layer is where I want the actual invariants to live, precisely because the caller cannot be trusted to preserve them. A model can be very good at reasoning and still skip a step, misunderstand the state, or confidently act on a premise that simply isn't true. Once the call reaches the mediating layer, though, you can enforce the rules rather than asking the model to remember them.
That's also why I find the tiny-model idea interesting, with one small caveat: I wouldn't call the model itself deterministic. I'd rather make the decision boundary deterministic — the model can classify or filter, but the MCP remains the authority that decides whether an action is actually allowed.
That gives you a nice separation: model for interpretation, MCP for enforcement. And if a tiny model can make that boundary cheaper without becoming part of the trust model, that's potentially a very useful place for it.
Exactly, and that's a sharper way to put it than I did. "Deterministic" was the wrong word for the model; the boundary is what should be deterministic, not the classifier. In my case the rule lives in the tool layer: the small model may only compress prose, it's structurally barred from source code or structured data, and the exact original is always one call away. So the model interprets, but what it's allowed to stand in for is decided outside it, which keeps it out of the trust model like you said. Here's that boundary in code if useful: github.com/olosegres/aia/commit/c9... . Curious whether you'd draw the prose vs structured line differently.
Exactly, it acts as a coffee filter essentially. It catches the secrets and filters them, it directs calls where it's more dense and it prevents context growth into regions that simply make no sense. That being said, the percolator is still the thing turning water and powder into coffee. The MCP is still the authoritative execution layer that has rule gating. All the tiny model does is penny pinching and redacting.
Essentially on the same premise as a PA, it directs, it filters, but it doesnt enforce the rules, it complies to them the same way the main model does, because the rules execute at a different boundary, the tool execution layer (MCP), which fits nicely with what I've been building with velocity MCP, because of the audit log feature, it can show exactly what rules fired and why it was allowed to trigger.
The "AI-generated bug that wasn't there" section hit home — I just published a piece where my own skill read my Zapier test posts as a promise I'd made to my audience and confidently offered to "finish the thread." Same shape as your Copilot phantom bug: the model wasn't wrong in a way that looks wrong, it was wrong in a way that looks completely reasonable, which is the only kind that actually costs you.
And the human-review gate as the one non-negotiable boundary is exactly where I keep landing too. Not because the model is dumb — because "structure a contribution" and "decide it deserves to be public" are different jobs, and only one of them is safe to hand over. Nice to see someone draw that line so cleanly, and hold it even when it cost you a prize category.
Yes — “wrong in a way that looks completely reasonable” is exactly the dangerous part.
A completely absurd answer is easy to catch. The difficult cases are the ones where the model constructs a perfectly coherent explanation from something that was never actually there. That's what made the phantom bug in my Copilot experiment interesting: the failure wasn't a lack of reasoning, it was reasoning applied to a false premise.
And I really like your distinction between “structure a contribution” and “decide it deserves to be public”. That's very close to how I see the human-review gate.
The model can help turn messy experience into a structured, potentially useful contribution. But whether that contribution is accurate, sufficiently general, non-sensitive, and actually worth adding to the shared knowledge base is a different kind of decision.
That's also why I was willing to lose the prize category. If removing that boundary made the project more eligible but less trustworthy, it would have defeated the point of the experiment.
And your Zapier example is a great illustration of the same problem from another angle: the model didn't invent something random — it inferred a plausible intention from context that didn't actually imply it. Those are exactly the errors that deserve a human in the loop.
"Reasoning applied to a false premise" is the cleanest way anyone's put it, and it reframes the whole thing for me. We keep building guardrails against models that reason badly, when the expensive failures come from models that reason well from a wrong starting point. You can't catch that by checking the logic — the logic is fine. You have to check the premise, and the premise is the one thing the model can't verify on its own.
Which loops right back to your review gate: the human isn't there to grade the reasoning, they're there to confirm the premise was real. Good conversation to have stumbled into — I'll be keeping an eye on where Shared Knowledge goes.
Exactly. And I think that's the part I hadn't fully articulated when I started the project.
We tend to think of AI guardrails in terms of how the model reasons: constrain the output, validate the logic, add more checks around the reasoning process. But if the model is reasoning correctly from a premise that never existed, all of those checks can happily pass.
That's why I increasingly see the human review as an epistemic boundary rather than just a quality-control step. The question isn't only “is this reasoning sound?” but “is this actually grounded in something that happened, and does the resulting knowledge deserve to become shared knowledge?”
And there is an interesting consequence for Shared Knowledge: the MCP can help structure and generalize an experience, but it can't be the final authority on whether that experience is real or whether the generalization is valid.
So yes — “reasoning applied to a false premise” may end up being one of the concepts I carry forward from this experiment. 😄
And thank you for following the project. I'm genuinely curious to see where it leads too.
"Epistemic boundary rather than quality-control step" — that's the upgrade. Quality control asks if the work is good; an epistemic boundary asks if it's true, and those come apart exactly where a fluent model is most convincing. I think that's the sentence I'll be stealing back from you. 😄
This was one of the better threads I've fallen into here. I'll be watching Shared Knowledge — and if that premise-checking layer ever turns into something concrete, I'd love to read the writeup.
Haha, go ahead — consider “epistemic boundary” officially open source. 😄
And now that I think about it, there’s a pretty interesting parallel with the problems you’re dealing with in Publora.
Your “right shape, wrong target” failure is almost the operational version of the same problem: the system can validate that something is technically correct without being able to establish that it is actually what was intended.
For Shared Knowledge, the equivalent is: the model can produce a perfectly coherent explanation from a premise that was never true.
In both cases, the dangerous failure isn't a broken call or obviously bad reasoning. It's a successful operation built on a false assumption.
That makes me think the “epistemic boundary” may actually be more useful as a design principle than I initially expected. The interesting challenge now is turning it into something concrete without pretending that truth itself can be automated.
And yes — if I manage to turn that premise-checking layer into more than an architectural box on a diagram, there will definitely be a writeup. 😉
Thanks again for the conversation. This has been one of the more interesting threads I've had around the project so far.
That's the real find in this whole thread — same failure wearing two costumes. Mine ships a post to the wrong account, yours ships an explanation from a premise that never happened, and both pass every check that only knows how to ask "is this well-formed?" You just gave the shared root a name.
"Without pretending truth itself can be automated" is the honest constraint, and it's why the human stays in it. Thanks right back — this was the good kind of conversation, the one that leaves you with a sharper question than you came in with. I'll be reading whenever that writeup lands. 🙂
Exactly. I think “same failure wearing two costumes” is probably the best summary of the whole thread. 😄
And I really like that we ended up not with an answer, but with a sharper question. That's usually a much better outcome for an architectural discussion.
I'll definitely keep you posted when the premise-checking side becomes concrete. And I'll be watching what you discover on your side too — I suspect these two problems have more in common than we initially thought.
Thanks again for the conversation. This was genuinely fun. 🙂
The duplicate check has a race that only shows up once the base is worth searching. search_knowledge reads merged articles, so two people who hit the same wall in the same week both search, both find nothing because the first article is still an open PR, and both open one.
Your Copilot run is the good case and it shows the limit: the assistant did check before publishing, and it could still only see main.
Having publish_knowledge search open PRs as well, or deriving the branch name from a slug of the title so a second attempt collides on the branch instead of on the reviewer, keeps that off the human's desk.
That's a fair observation, although I wouldn't actually consider this a race condition that the architecture needs to eliminate.
Two people can independently encounter the same problem within a short enough window that neither can see the other's contribution yet. That's not necessarily a failure of the system — it's simply a property of collaborative knowledge creation.
The important distinction for me is between preventing duplicate published knowledge and preventing duplicate proposals. The former is something the system should handle automatically; the latter is perfectly legitimate.
That's also why the human review gate is there. If two PRs describe essentially the same solution, the reviewer can reject the second one. But they can also merge the useful parts of both, or even decide that the second solution is better and replace the first one.
So in this particular case, I actually want the architecture to tolerate the race rather than trying to eliminate it. The PR is a proposal, not yet knowledge.
And I think your observation about the Copilot run is still valuable: it makes that boundary very explicit — the assistant can verify against published knowledge, but it cannot know what another contributor is currently proposing. That's a feature of the workflow rather than something I necessarily want to hide from it.
The next thing I'd want as a reader is a small "tested under" block: dependency versions, a minimal input, expected output, and the date someone actually reran it. That can stay in Markdown alongside the solution, without retaining the private conversation.
Your optional-dependency example is a good candidate: can a second person reproduce the import failure and then verify the fix in a clean environment? A merged article and an independently reproduced fix are different kinds of evidence; both are useful, but I'd label them separately.
Would you consider letting the next reader contribute a reproduction result to the existing article rather than another solution? That could make the smallest useful contribution "I checked this on version X; here's what changed," not writing a whole new post.
I like the idea of recording independent reproduction, but I think there is an important constraint here: we can't simply accumulate individual experiences alongside the original solution.
The goal is to synthesize knowledge, not to build a collection of personalized installation stories. If several people independently solve the same problem, the useful result should become something like “Installing n8n on Fedora 18+” or “Installing n8n on Debian / Ubuntu”, rather than a collection of “how Pascal did it”, “how X did it”, etc.
So I agree that an independent reproduction is valuable evidence, but I think that evidence should feed back into the synthesis rather than become another public artifact on its own.
That also means the MCP would need to distinguish between an experience, the synthesized knowledge extracted from multiple experiences, and new evidence that can confirm or challenge that synthesis.
That's actually a rather interesting consequence of the model: the contribution doesn't necessarily produce another article. It can improve an existing piece of shared knowledge.
Yes, that distinction improves what I was suggesting: reproduction should support a shared procedure, not generate another installation article.
I'd keep a compact evidence record attached to the revision it tested: environment, result, and date. The synthesis can then state its supported environments and any unresolved contradictions. A failure on a different OS shouldn't automatically invalidate the procedure for environments where it still works.
The interesting editorial decision is whether new evidence changes the procedure, narrows its scope, or leaves an open question. Would you have a maintainer review those changes, or let the MCP propose a revision with the conflicting evidence attached?
Great project, Pascal, as usual! And I really appreciate the practical, common-sense approach to AI, especially in the middle of all the current AI paranoia. 😄
Thank you, Sylwia! 😄
And coming from you, “as usual” probably means more than you realize. We’ve been around quite a few AI discussions together by now, so I’m glad that this practical, common-sense approach still comes through.
I think there’s a middle ground that often gets lost: neither “AI will solve everything” nor “AI is the source of everything that’s wrong with software and society.” Just build things, see what actually works, understand where it fails, and make sensible decisions from there.
And yes, there is quite enough AI paranoia around at the moment to keep us busy. 😄
Respect for dropping Gemini eligibility to keep the architecture model-agnostic. That's the same kind of scope call I had to make this weekend (cut ElevenLabs entirely rather than force it in for a prize category). The os.environ.get() empty-string-vs-missing-key bug is a good catch too, that exact gotcha gets everyone eventually.
Thank you — and having read your Ajo Chain submission, I think I understand the parallel much better now.
We ended up making almost the same kind of scope decision, for different technologies and for slightly different reasons, but with the same underlying principle: don't let the prize category dictate the architecture.
In my case, keeping Gemini in the critical path would have contradicted the model-agnostic design I was aiming for. In yours, keeping ElevenLabs would have meant spending more time forcing an integration that wasn't essential to the actual product. In both cases, the category became less important than the integrity of the project.
And I particularly appreciate the comparison because that's exactly what I tried to preserve in my article: not a polished account where every technology ends up working perfectly, but the actual reasoning, failures and scope decisions that shaped the final architecture.
Also, apparently the empty-string
os.environ.get()trap is now officially a shared experience. 😄This is a really great idea. I love the thought of taking those useful little things we figure out while working with AI and turning them into something other people can actually find and use later, instead of letting them disappear into chat history.
I’ve run into a few things that I think could be useful to share, so I might have to contribute something. The human review part is especially nice too. AI can help put the knowledge together, but that doesn’t mean everything it says should automatically become trusted documentation.
Really cool project. I’m interested to see what people add to it.
Thank you! And if you already have a few things in mind that you've learned the hard way, that's exactly the kind of material I'm hoping Shared Knowledge can capture. 😄
The contributor shouldn't have to turn that experience into polished documentation first — that's part of what the MCP and the synthesis workflow are supposed to help with.
And I completely agree about the review boundary. The AI can help extract, structure and generalize something useful, but “the model produced a plausible explanation” is very different from “this is now trusted shared knowledge.”
I'm really curious to see what people end up contributing. The interesting part of the project will probably be what emerges from those small, otherwise-forgotten discoveries once they start accumulating.
I know I definitely have a few “well, that took way longer to figure out than it should have” discoveries sitting around.
I’ve been experimenting with running and converting smaller models lately, and some of the compatibility issues and workarounds might be useful to other people. I’ll have to look through what I’ve got and see what would make a good first contribution.
And yeah, I think that distinction between a plausible explanation and trusted knowledge is really important. It’s easy for something to sound right, especially when an AI explains it confidently, so it's definitely a good idea to have that review step.
I’m really curious to see what people contribute too. I think the little things people figure out along the way could end up being some of the most useful stuff in there.
Absolutely — the smaller-model stuff sounds like exactly the kind of thing that could be valuable there.
Compatibility issues are often a perfect example of knowledge that is painful to discover once, but incredibly useful when someone else can find the answer instead of repeating the same experiment. And the fact that you had to work around them is probably more interesting than a generic “this model works with X” statement — the constraints and the actual workaround are what make the knowledge reusable.
So if you dig through your notes and find a couple of those “why does this work that way?” moments, I'd definitely be interested in seeing them. 😄
And yes, I think that's the core distinction: the AI can help turn those experiences into something structured and reusable, but someone still needs to decide whether the resulting explanation actually deserves to become shared knowledge.
Those little discoveries may well end up being the most valuable part of the corpus precisely because nobody thinks they're worth writing down at the time.
Yeah, exactly! I think the constraints and workarounds are usually the most useful part too. A generic “this model works” doesn’t tell you much when you’ve spent three hours trying to figure out why it doesn’t work on your setup.
I’ve got some notes from a recent experiment that could be a good fit. I might dig through them and see what I can pull together. I like that the MCP can help with the structuring part, because that’s usually the bit that makes me put off writing things down.
Really looking forward to seeing how this grows!
Exactly — and I think that's one of the less obvious problems Shared Knowledge is trying to solve.
The frustrating part is that the useful information is often already there in someone's notes, terminal history, or “three hours of figuring out why this doesn't work” — but turning that into a proper piece of documentation feels like a second, completely different task.
If the MCP can take the messy experience and help structure and generalize it, without requiring the contributor to become a technical writer, then the barrier to sharing becomes much lower.
So definitely dig through those notes when you get a chance. I'd much rather see a rough contribution that can be refined through the workflow than have another useful workaround disappear into someone's project folder. 😄
And I'm curious to see what comes out of your experiment too!
Some comments may only be visible to logged-in visitors. Sign in to view all comments.