Knowledge Base showing no Documents or Code Examples

[Magowntown]

Hi all,

Lately I have been trying out Archon more and it's knowledge base crawler/scraper.

However any time I use this, including the default Pydantic one that came with it, or even a PDF, no Documents or Code Examples ever show up.

I am also getting the warning error when on the Projects page and trying to choose the "Docs" tab:

Project Documents Under Migration
Editing and uploading project documents is currently disabled while we migrate to a new storage system. Please backup your existing project documents elsewhere as they will be lost when the migration is complete.

Note: This only affects project-specific documents. Your knowledge base documents are safe and unaffected.

Do i need to re-download the repo or something?

[j105rob]

I am having the same issue after installing the stable branch as a first time user 4 days ago. Is there any update on this issue? Should I try main?

[meijlp]

Hi @Magowntown, @j105rob I had the same issue and noticed in my archon server docker logs that there was a missing column in the database schema that I had deployed in Supabase. I was able to fix it by running the RESET_DB.sql and did a fresh complete_setup.sql to make it work. You also need to set up the openai key.

[Aston77]

I am running into a similar issue. Seems like the system desperately needs some sort of configuration health check to test itself against known good outcomes to determine if there are configuration issues, otherwise it just repeatedly fails silently. I've had incorrect embedding keys, missing columns after upgrades and it is something you purposefully have to make an effort to double check.

[Aston77]

I am not technically proficient enough to submit this for a pull request but I was able to get my issue resolved this morning and had Claude outline what the problem was and how it was addressed. Hopefully a better programmer can review this for broader use.

Knowledge Base Search Fix - Multi-Dimensional Embeddings

Problem: Knowledge Base Search Returning No Results

Despite successfully crawling and embedding data with Google Gemini embeddings, RAG searches were returning empty results. The UI showed data existed (10,947 crawled pages), but queries returned 0 matches.

Root Cause

The issue had three components:

1. Database Schema: Multi-dimensional embedding columns existed (embedding_384, embedding_768, embedding_1536, etc.) but the multi-dimensional RPC functions were missing
2. Missing Functions: The database lacked match_archon_crawled_pages_multi() and hybrid_search_archon_crawled_pages_multi() functions
3. Python Code: Search services were calling legacy single-dimension RPC functions that only queried the embedding_1536 column with hardcoded dimension parameter

Error Observed

ERROR: Could not find the function public.hybrid_search_archon_crawled_pages_multi(...) in the schema cache

Solution (Without Data Loss)

1. Database Migration

Run complete_setup.sql on your existing database to add missing multi-dimensional functions:

export PGPASSWORD=postgres
psql -h 127.0.0.1 -p 54322 -U postgres -d postgres -f migration/complete_setup.sql

Note: This is idempotent and preserves all existing data. Ignore "already exists" errors - they're expected.

2. Python Code Updates

File: python/src/server/services/search/base_search_strategy.py

Before (legacy single-dimension):

# Build RPC parameters
rpc_params = {"query_embedding": query_embedding, "match_count": match_count}

# Add filter parameters
if filter_metadata:
    if "source" in filter_metadata:
        rpc_params["source_filter"] = filter_metadata["source"]
        rpc_params["filter"] = {}
    else:
        rpc_params["filter"] = filter_metadata
else:
    rpc_params["filter"] = {}

# Execute search
response = self.supabase_client.rpc(table_rpc, rpc_params).execute()

After (multi-dimensional support):

# Detect embedding dimension
embedding_dimension = len(query_embedding)

# Map base RPC function names to multi-dimensional versions
multi_rpc_map = {
    "match_archon_crawled_pages": "match_archon_crawled_pages_multi",
    "match_archon_code_examples": "match_archon_code_examples_multi",
}

# Use multi-dimensional RPC function if available
multi_table_rpc = multi_rpc_map.get(table_rpc, table_rpc)

# Build RPC parameters with embedding dimension
rpc_params = {
    "query_embedding": query_embedding,
    "embedding_dimension": embedding_dimension,
    "match_count": match_count,
}

# Add filter parameters
if filter_metadata:
    if "source" in filter_metadata:
        rpc_params["source_filter"] = filter_metadata["source"]
        rpc_params["filter"] = {}
    else:
        rpc_params["filter"] = filter_metadata
else:
    rpc_params["filter"] = {}

# Execute search with multi-dimensional RPC
span.set_attribute("embedding_dimension", embedding_dimension)
span.set_attribute("rpc_function", multi_table_rpc)

response = self.supabase_client.rpc(multi_table_rpc, rpc_params).execute()

File: python/src/server/services/search/hybrid_search_strategy.py

Update both search_documents_hybrid() and search_code_examples_hybrid() methods:

For Documents:

# Detect embedding dimension
embedding_dimension = len(query_embedding)

# Call the multi-dimensional hybrid search PostgreSQL function
response = self.supabase_client.rpc(
    "hybrid_search_archon_crawled_pages_multi",  # Changed from legacy function
    {
        "query_embedding": query_embedding,
        "embedding_dimension": embedding_dimension,  # Added
        "query_text": query,
        "match_count": match_count,
        "filter": filter_json,
        "source_filter": source_filter,
    },
).execute()

span.set_attribute("embedding_dimension", embedding_dimension)  # Added

For Code Examples:

# Detect embedding dimension
embedding_dimension = len(query_embedding)

# Call the multi-dimensional hybrid search PostgreSQL function
response = self.supabase_client.rpc(
    "hybrid_search_archon_code_examples_multi",  # Changed from legacy function
    {
        "query_embedding": query_embedding,
        "embedding_dimension": embedding_dimension,  # Added
        "query_text": query,
        "match_count": match_count,
        "filter": filter_json,
        "source_filter": final_source_filter,
    },
).execute()

span.set_attribute("embedding_dimension", embedding_dimension)  # Added

3. Restart Services

docker compose restart archon-server

Result

✅ RAG searches now work with any embedding dimension (384, 768, 1024, 1536, 3072)
✅ Automatically detects query embedding dimension and queries correct database column
✅ Works with OpenAI, Google Gemini, Ollama, and other embedding providers
✅ Zero data loss - all existing crawled pages preserved

Verification

Test your RAG search:

curl -X POST http://localhost:8181/api/rag/query \
  -H "Content-Type: application/json" \
  -d '{"query": "How do I use embeddings?", "match_count": 5}'

You should receive results with "total_found" > 0.

Alternative: Nuclear Option (NOT Recommended)

Some users have resolved this by running RESET_DB.sql followed by complete_setup.sql, but this deletes all your data (crawled pages, projects, tasks). The solution above preserves everything.

Comparison

Approach	Data Preserved	Downtime	Re-crawl Required
Our Fix	✅ Yes	Minimal (~30s)	❌ No
RESET_DB.sql	❌ No	Significant	✅ Yes

Why This Happened

The Archon database schema supports multi-dimensional embeddings (added in commits 85bd6bc and ee3af43), but the Python service layer code wasn't updated to use the new multi-dimensional RPC functions. This fix bridges that gap.

Architecture Context

Archon's multi-dimensional embedding support works as follows:

1. Database Storage: Data is stored in dimension-specific columns (embedding_384, embedding_768, embedding_1536, etc.)
2. RPC Functions: Multi-dimensional functions (*_multi) dynamically select the correct column based on the embedding_dimension parameter
3. Legacy Functions: Single-dimension functions still exist for backward compatibility but default to 1536D only

The Python code was still calling the legacy functions, which caused dimension mismatches when using non-1536D embeddings.

Supported Embedding Models

This fix enables proper support for:

• OpenAI: text-embedding-3-small (1536D), text-embedding-3-large (3072D)
• Google Gemini: gemini-embedding-001 (768D default, or 1536D/3072D with output_dimensionality)
• Ollama: nomic-embed-text (768D), mxbai-embed-large (1024D)
• Custom models: Any dimension in [384, 768, 1024, 1536, 3072]

Configuration

Set your embedding dimension in .env:

# Match your model's output dimension
EMBEDDING_DIMENSIONS=1536  # For Gemini with output_dimensionality=1536
# OR
EMBEDDING_DIMENSIONS=768   # For Gemini default or nomic-embed-text

Files Modified

• python/src/server/services/search/base_search_strategy.py
• python/src/server/services/search/hybrid_search_strategy.py

Related Issues

• GitHub Discussion: "Knowledge base search returning no results"
• Related to commits: 85bd6bc (Fix multi-dimensional vector hybrid search functions), ee3af43 (Ollama Integration)

Credits

Solution developed through analysis of Archon's multi-dimensional embedding architecture and database schema inspection.

[bilzee]

Had the same problem and the above solutions didn't work for me. What worked was to simply clone the "main" branch rather than the "stable" branch which is the default in the readme file (or you could just switch to the main branch) and it worked.

[Aston77]

FWIW I am and have been on the main branch, but my issue very well may have been specific to my implementation.