Most of the data a company cares about does not live in a database an LLM can talk to. It lives in a data lake: thousands of CSV and Parquet files, organised in folders by domain, year or source system. Getting an AI agent to use that data usually means writing a custom pipeline, exporting samples by hand, or pasting rows into a chat window.

The Model Context Protocol (MCP) gives us a cleaner option. If the data lake is exposed as an MCP server, any MCP-capable client (Claude Desktop, Claude Code, your own agent) can discover and use it as a set of typed tools.

That is what Datalake-MCP does. It is a .NET 9 MCP server that connects to an Azure Data Lake Storage Gen2 account and lets an agent:

- Navigate filesystems and directories,

- Read CSV, Parquet, XML, text and markdown files, with filters, column projection and pagination,

- Save text, markdown, CSV, Parquet and PDF files, with a searchable index of everything that has been written.

This post covers how it works internally and what it makes possible.

Architecture at a glance

User <-> Claude (Desktop / Code) ← MCP over stdio → Datalake.Mcp.Server (.NET 9)

The server runs as a .NET Generic Host and talks to the client over stdio. The client launches it as a child process, so there is no HTTP endpoint to secure and nothing to deploy. Connection settings (`AccountName`, `AccountKey`, `DefaultFileSystem`) come from .NET user-secrets in development, or from environment variables when the server is registered in the client.

The code is split into three layers:

- Tools (`DataLakeTools`): the MCP surface. Each method is a tool with a description and typed parameters that the model sees.

- Services: format-specific readers, the writer, the index and the row filter. They contain no Azure-specific code.

- DataLakeService: a thin wrapper over `Azure.Storage.Files.DataLake`, the only class that talks to the storage account.

The tool surface

The tools are designed to be called in sequence. A typical exploration goes `list_filesystems`, then `list_directory`, then `get_parquet_schema`, then `read_parquet`. The agent sees column names and types before it asks for rows, so it can write sensible filters instead of guessing.

Reading without loading: streaming by design

A data lake file can be gigabytes in size, and an LLM can only consume a tiny fraction of it. The server is built around that mismatch.

Lazy remote reads. Files are opened with `DataLakeFileClient.OpenReadAsync`, which returns a stream that downloads only the bytes actually requested. Nothing is fetched up front.

Format-aware streaming

- CSV, text and markdown are read line by line and the read stops at the requested limit.

- XML is read element by element (tabular XML: a root with repeated record elements). DTD processing and external entity resolution are disabled, which prevents XXE attacks.

- Parquet is read one row group at a time, with column projection: if you ask for three columns, the others are never decoded. This is where Parquet’s columnar layout pays off.

Pagination that does not re-read. Every read tool returns `truncated` and `nextOffset`. When a result is cut off, either by `maxRows` or by the tool-call size limit (about 1 MB), the agent calls the tool again with `offset = nextOffset` and gets the next page. The model walks a large file in chunks without any hidden state in the server.

Querying: server-side filters

The most useful feature for analytical work is the `filter` parameter on `read_csv`, `read_parquet` and `read_xml`. It is a list of conditions combined with AND:

```json

[

{“column”: “stato”, “operator”: “eq”, “value”: “APERTO”},

{“column”: “importo”, “operator”: “gt”, “value”: “1000”}

]

```

Supported operators are `eq`, `ne`, `gt`, `gte`, `lt`, `lte`, `contains`, `startswith`, `endswith`, `isnull` and `isnotnull`. Comparison semantics:

- numeric when both the cell and the value parse as numbers (invariant format, `12.5`),

- timestamp-aware for Parquet date columns when the value is a date,

- case-insensitive text otherwise.

Two design decisions matter here:

1. The filter runs inside the streaming loop, before `offset` and `maxRows`. The pagination counters therefore count only matching rows. “Give me the next 100 open orders over 1000” works as expected, and non-matching rows never reach the model’s context window.

2. Filters can reference columns that are not projected. You can filter on `stato` and return only `id` and `importo`.

A wrong column name or operator returns an error that lists the allowed values, so the agent can correct itself in the next call without human help.

A request like *”show me the open orders above 1000"* becomes: look at the schema, build a filter, read with a projection, and summarise the result. The model never sees the other 99% of the file.

Writing back: safe by default

Reading is only half of the story. Agents are far more useful when they can leave something behind: a report, a cleaned dataset, a summary of an analysis. Write access to shared storage is also where things go wrong, so the write path was designed conservatively.

The `save_*` tools take a `path` (folders and subfolders are created if missing), a `fileSystem`, a `description` and an `overwrite` flag.

No silent overwrite. Uploads are conditional (`If-None-Match: *`). If the file already exists and `overwrite` is not `true`, the call fails and the existing file is untouched. An agent cannot destroy data by accident, only by being explicitly told to.

Validation before upload.

- The path extension must match the format.

- CSV must have the same number of fields on every row.

- Parquet is built from a list of rows (column to value objects), with types inferred (bool, integer, decimal, string).

- PDF is received as base64, and the `%PDF-` header is checked.

The index: files you can find again

Each successful save is recorded in an index file, a CSV at the filesystem root (configured with `DataLake:SavedFilesIndexPath`, default `saved-files-index.csv`) with the columns:

```

path, name, format, sizeBytes, savedAt, description

```

Saving the same path again updates its row. Because several agents or sessions may write at the same time, the index is updated with optimistic concurrency: read the index and its ETag, apply the change, write back with `If-Match: <ETag>`. On a conflict the cycle restarts, up to 5 attempts. No locks, no lease management, and no lost updates.

The index is a plain CSV, so it can be read with `read_csv`, opened in Excel or loaded by any other tool.

On top of the index sits `search_file(nomeFile, tipoFile)`. It matches the file name with *contains* logic (case-insensitive) and can filter by type (csv, parquet, pdf, md, txt). When exactly one file matches, the tool also returns its content: the first `maxRows` rows for tabular files, text for txt and md, metadata only for PDF. The agent can ask for “last quarter’s margin report” and get the document in a single call.

What this enables

Once the connection is in place, several patterns become possible.

1. Natural-language analytics over raw files. Ask questions about costs, orders or KPIs without building a dashboard first. The agent inspects the schema, filters, aggregates what comes back and explains it. It works well for ad-hoc questions that do not justify a BI project.

2. Data exploration and profiling. Point the agent at an unfamiliar folder. It can list the files, read the schemas, sample rows, spot null-heavy columns and describe what the dataset seems to contain, which is a good first step when you inherit a lake nobody documented.

3. Persistent agent output. Reports, generated markdown documentation, cleaned CSVs and Parquet extracts can be saved next to the data, indexed and found again later. The lake becomes the shared memory of your agents, backed by your existing storage, access control and retention rules.

4. Multi-agent workflows with a shared medium. One agent extracts and filters data and saves a Parquet file. Another finds it with `search_file` and builds a summary. A third saves the PDF. The agents never talk directly. They coordinate through files and the index, and every step leaves an auditable artifact.

5. Reference data lookups. Price lists, catalogues and code tables stored as CSV with a markdown index file can be searched by keyword and compared across years or regions, with the agent doing the matching that used to be done by hand.

6. Domain skills on top of the tools. The repository includes a `SKILL.md` that teaches Claude how a specific lake is organised: folder conventions, table meanings, business terms (EVM, WBS, SAL and so on). The MCP provides the capabilities and the skill provides the domain knowledge. Together they let a general-purpose model act like a specialist for your data.

Getting started

1. Build the server: `dotnet build src/Datalake.Mcp.Server -c Release`

2. Add it to `%APPDATA%\Claude\claude_desktop_config.json` under `mcpServers`, with the DLL path in `args` and `DataLake__AccountName`, `DataLake__AccountKey` and `DataLake__DefaultFileSystem` in `env`.

3. Restart Claude Desktop. The `datalake` server and its tools appear in the tool list.

Then ask something like: *”List the filesystems, find the Parquet files under `sales/2026`, and show me the open orders above 1000 with only id, customer and amount.”*

Security notes

- Never commit credentials. Use user-secrets or environment variables; `appsettings.json` ships with empty placeholders.

- The server authenticates with the account key, which grants full access to the account. For production, consider a dedicated storage account or a narrower credential, and be deliberate about which lake you connect to an agent.

- Writes are additive by default. `overwrite=true` is the only way to replace a file.

- Reads are bounded (`maxRows`, `maxLines`, the response size cap), so a careless prompt cannot stream a whole file into the model’s context.

Conclusion

An agent is only as useful as the data it can reach. Exposing Azure Data Lake Gen2 over MCP removes the glue code between the two: the agent can browse, filter, read and write through a small set of typed tools, while streaming reads, conditional writes and ETag-based indexing keep the server efficient and safe.

see open source project on Github

https://github.com/alex1976/Datalake-MCP

Datalake-MCP is listed on mcpservers.org

https://mcpservers.org/servers/alex1976/datalake-mcp

If you have a datalake on Azure and an agent, try connecting them.

Tags: #MCP, #Azure Data Lake, #AI Agents, #Claude, #DotNET, #Data Engineering

Connect your AI agents to Datalake to read and store data was originally published in Dev Genius on Medium, where people are continuing the conversation by highlighting and responding to this story.

In Memory-MCP, a persistent-memory MCP server for AI agents, this responsibility lives in a single class, PdfTextExtractor, built on PdfPig (v0.1.15). It's worth walking through the implementation because the failure mode in this domain is silent: the document "looks" extracted fine, but the text is in the wrong order.

Why PdfPig, and why that matters architecturally

PdfPig is a fully managed .NET library — no native dependencies, no wrapper around Ghostscript or a C library. That’s not a cosmetic detail. Elsewhere in the same system, IEmbeddingProvider and IFactExtractor require an external API key and may simply not be configured. IPdfTextExtractor has no such dependency, so it's always available — a user can upload and store a PDF's text even in a deployment with no LLM configured at all.

The problem Page.Text doesn't solve

PdfPig’s most obvious API is Page.Text, which concatenates glyphs in content-stream order — the order operators appear in the PDF's internal drawing instructions. For a single-column document, that's fine. For anything more realistic — two columns, a table, a header repeated on every page — content-stream order does not match human reading order. You get output like:

Alpha one Beta one Alpha two Beta two Alpha three Beta three...

when the actual intent was two columns read top-to-bottom in sequence: “Alpha one, Alpha two, Alpha three… then Beta one, Beta two…”.

This isn’t cosmetic. The extracted text is exactly what gets embedded and handed to the downstream fact extractor: wrong reading order means wrong or unrecoverable extracted facts. In the codebase this is treated explicitly as a correctness concern, not a nice-to-have.

The three-stage layout analysis pipeline

PdfTextExtractor.ExtractPageText doesn't reach for Page.Text first. It runs PdfPig's document layout analysis pipeline in three sequential stages:

var words = NearestNeighbourWordExtractor.Instance.GetWords(page.Letters);
var blocks = DocstrumBoundingBoxes.Instance.GetBlocks(words);
var ordered = UnsupervisedReadingOrderDetector.Instance.Get(blocks);
var pageText = string.Join("\n\n", ordered.Select(block => block.Text));

1. NearestNeighbourWordExtractor — PdfPig exposes individual glyphs (Letter) with their positions. This stage groups them into words using spatial proximity as the heuristic.
2. DocstrumBoundingBoxes — implements the Docstrum (Document Structure Analysis) algorithm, which groups words into blocks/paragraphs based on bounding-box geometry: line alignment, typical spacing between words versus between columns.
3. UnsupervisedReadingOrderDetector — reorders the resulting blocks according to an estimated human reading order, not content-stream order. This is the stage that actually fixes the two-column case.

The ordered blocks are joined with "\n\n" — a blank line. That's the only "structure" the system preserves: each block is a paragraph, and the boundary between blocks survives into the stored text as an empty line. This is not Markdown in any literal sense — no #, no reconstructed lists, no tables — but it's the minimum useful granularity: it gives the downstream fact extractor a coherent unit to reason over instead of an undifferentiated wall of text.

The parsing options, and why each one exists

private static readonly ParsingOptions Options = new()
{
    UseLenientParsing = true,
    SkipMissingFonts = true,
    UseActualText = true,
};

• UseLenientParsing and SkipMissingFonts: real-world PDFs are frequently slightly malformed (incomplete font embedding, dangling object references). Without these, PdfPig would throw on documents that any ordinary viewer opens without complaint.
• UseActualText: honors the PDF's /ActualText attribute, a spec-level mechanism that lets an author specify the "real" text behind a glyph sequence that would otherwise be unreadable — typographic ligatures, or text drawn out of logical order, which is common in tables produced by layout tools.

A two-level fallback, without swallowing errors entirely

try
{
    // NearestNeighbourWordExtractor -> DocstrumBoundingBoxes -> UnsupervisedReadingOrderDetector
    return string.IsNullOrWhiteSpace(pageText) ? page.Text : pageText;
}
catch (Exception ex) when (ex is not OperationCanceledException)
{
    return page.Text;
}

A page with degenerate geometry (corrupted fonts, malformed coordinates) can make the segmentation algorithm throw, or produce zero blocks. In that case, falling back to page.Text for that one page beats both worse alternatives: failing the entire upload, or silently storing an empty document.

At the document level, any other exception (corrupted bytes, an unopenable PDF) is wrapped into a DocumentExtractionException carrying the original message — handled as an ordinary MCP tool error, not an unhandled exception that crashes the process.

What this pipeline deliberately does not do

Being explicit about the boundaries matters as much as explaining what works:

• No OCR. A scanned (image-only) PDF with no text layer extracts as empty text, with no error. The UI surfaces this as “no extractable text found — memory extraction skipped.” OCR is explicitly out of scope at this stage.
• No embedded image extraction.
• No structural table reconstruction. Tables benefit only indirectly: UseActualText recovers correct text from cells drawn out of logical order, and the reading-order detector mitigates — but doesn't fully solve — column/cell interleaving.
• No heading detection based on font size or boldness — there’s no generated #/## markup.

Where this text goes: two decoupled steps, not one automatic pipeline

One architectural detail that’s often assumed implicitly but is explicit here: uploading a PDF and generating memories are two separate MCP tools, called in sequence by the client — not a single operation.

1. create_document receives the PDF as base64 bytes, calls PdfTextExtractor.ExtractTextAsync, and stores the result as the Document's RawContent. That's it — no fact extraction, no embedding, just source-of-truth storage.
2. add_memory, called separately, takes that text (the full content, not chunked into fixed windows), computes an embedding, searches for similar candidate memories already in the space, and hands everything to an LLM-backed IFactExtractor, which pulls out atomic facts and links them semantically to existing memories in a graph. If no LLM is configured, the entire text becomes a single memory — a "pre-graph" fallback behavior.

There’s no fixed-token chunker anywhere in the PdfPig path: the real chunking is semantic, and it happens inside the fact extractor’s LLM, not in the PDF parsing layer. PdfPig’s job ends earlier — handing the LLM text in the right order.

The general lesson

This kind of pipeline illustrates something often underweighted in RAG systems: upstream extraction quality is a silent multiplier on everything downstream. An embedding computed over text with interleaved columns produces vectors that don’t represent the actual document — and no retriever, however sophisticated, can fix that after the fact. Investing in layout analysis before embedding isn’t over-engineering. It’s where the game is actually won or lost.

If you want to learn more, see (and star) my project at

https://github.com/alex1976/Memory-MCP

The problem

Anyone who works with data spends time writing the same queries over and over: “show me the tables,” “what columns does this view have?,” “pull me the 2025 orders as CSV.” These are mechanical tasks, but they require knowing the schema by heart and writing correct SQL for the right dialect (T-SQL or PostgreSQL).

Database-MCP is a server that implements the Model Context Protocol (MCP) and lets Claude (or others agents) explore and extract data from a SQL Server (Azure or on-prem) or PostgreSQL database, using natural language. The idea isn’t to hand an LLM the keys to the database: it’s to give it only the ability to read.

What it does, from the user’s perspective

The server exposes eight MCP tools that Claude can call:

Tool and what it does:

list_schemas:Lists user schemas (system schemas excluded)

list_tables:Lists tables, with an approximate row count

list_views:Lists views

describe_table:Columns, primary key, foreign keys, indexes for a table

describe_view:Columns and SQL definition of a view

query_data:Queries a table/view with structured filters (no raw SQL)

execute_sql:Runs a raw SELECT (joins, aggregates, CTEs)

export_data:Runs a SELECT and streams the full result set to a CSV/TXT file

The typical flow is: orient (list_schemas/list_tables), inspect (describe_table so column names are never guessed), query (query_data for simple cases, execute_sql for joins and aggregations), export (export_data when the full result set needs to land on disk instead of being pasted into the chat).

This sequence is also encoded in a Claude skill (skill/SKILL.md) that instructs the model never to guess table or column names and to always prefer query_data when it's expressive enough.

Architecture

The server is a single .NET console application that speaks MCP over stdio: Claude launches the process, and JSON-RPC messages flow over stdin/stdout, while all logging goes to stderr so it never corrupts the protocol stream.

Claude  <--stdio(JSON-RPC)-->  DatabaseMcp.dll
                                   │
                        ModelContextProtocol SDK
                         (tool discovery/dispatch)
                                   │
                    ┌──────────────┴──────────────┐
                 Tools/                          Data/
          SchemaTools, DataTools          ISchemaService, QueryService
                                                   │
                                            Security/
                                    SqlGuard, SqlIdentifier
                                                   │
                                   ┌───────────────┴───────────────┐
                            SQL Server                        PostgreSQL

Key design points:

• One database per instance. The DB_PROVIDER variable selects the engine (sqlserver, the default, or postgresql). If both database types need to be reachable, two instances of the server are launched, each under its own tool namespace.
• No ORM. All data access is hand-written, parameterized ADO.NET against the provider-agnostic DbConnection/DbCommand/DbDataReader interfaces. This gives full control over streaming and predictable performance — an explicit project requirement.
• Two implementations of ISchemaService. SqlServerSchemaService uses the sys.* system catalog views; PostgreSqlSchemaService uses information_schema wherever the two engines agree (tables, views, columns, keys), falling back to pg_catalog/pg_stat_user_tables only where needed (indexes, row counts). All queries are fixed and parameterized: no user input is ever concatenated into the SQL text.
• Streaming, not buffering. QueryService + DelimitedWriter execute the SELECT and stream the DbDataReader directly to CSV/TXT — either inline (row-capped) or to a file (uncapped). Extracting millions of rows never holds the entire result set in memory.

The safety model: read-only, for real

This is the heart of the project. An LLM that generates SQL is powerful but unpredictable, so the server applies defense in depth across several layers:

1. SqlGuard — only SELECT/WITH. Any raw SQL passed to execute_sql/export_data is statically validated: comments are stripped, it's checked to be a single statement (no multiple ;-separated statements), it must start with SELECT or WITH, and it's checked against a denylist of dangerous keywords covering both engines — INSERT/UPDATE/DELETE/MERGE/DROP/ALTER, but also engine-specific surface like xp_cmdshell, OPENROWSET (SQL Server) or COPY, dblink, pg_read_file (PostgreSQL).

2. query_data never sees raw SQL. It builds a parameterized SELECT from structured arguments (table, columns, filters, ordering, row limit). Identifiers go through SqlIdentifier, which validates and quotes them correctly per engine (square brackets for SQL Server, double quotes for PostgreSQL); filter values are always bound as parameters, never concatenated text — zero injection surface.

3. Caps on inline results. execute_sql/query_data return at most DB_HARD_MAX_ROWS rows (default 10,000) to protect the model's context window, reporting truncated: true if more rows exist. export_data, by contrast, has no cap: it writes everything to a file.

4. The real backstop is still the database. The README is explicit about this: application-level protection doesn’t replace a login with SELECT-only permissions on the target database. It's defense in depth, not the only line of defense.

One interesting technical detail: string parameters sent to PostgreSQL are marked as NpgsqlDbType.Unknown, because — unlike SQL Server — Postgres doesn't implicitly cast a text parameter to compare it against a numeric or date column. Without this workaround, a filter on a date column passed as a string would silently fail.

Technology stack

• .NET 10 / C#, console application
• ModelContextProtocol, the official C# SDK for MCP — stdio transport, tool discovery via [McpServerToolType]/[McpServerTool] attributes
• Microsoft.Data.SqlClient for SQL Server, Npgsql for PostgreSQL
• Microsoft.Extensions.Hosting for DI, configuration, and logging

Configuration is entirely environment-variable driven: DB_PROVIDER selects the engine, then SQLSERVER_*/POSTGRES_* (or the generic DB_* equivalents) set host, credentials, timeouts, and limits. For SQL Server, if SQLSERVER_USER/SQLSERVER_PASSWORD are omitted, it falls back to Windows Integrated Security — convenient for on-prem AD environments.

Why this design matters

The general lesson here, one that extends beyond this specific project, is that giving an LLM access to an external system doesn’t mean giving it a raw channel into that system. Every layer here — the SQL guard, the parameterized query construction, the cap on inline rows, the file-based streaming for bulk exports — exists to make it impossible (or at least very hard) for an unexpected model generation to produce a real side effect on the database. The model can explore, query, export. It can’t write.

The project is tested with xUnit on the pure-logic parts (guard, quoting, query builder, CSV writer) and with optional integration tests against a real database (which create and drop a temporary table, never touching production data).

See more (and star) at:

https://github.com/alex1976/Database-mcp

Database-MCP: giving AI agents access to your SQL Server or PostgreSQL database was originally published in Dev Genius on Medium, where people are continuing the conversation by highlighting and responding to this story.

Systems like ChatGPT, modern search engines, and recommendation systems no longer rely on simple keyword matching. Instead, they understand semantic context.

The Context: Project Memory-MCP

The concepts explored in this article are a core part of Memory-MCP, an ongoing project I am actively developing. Memory-MCP is designed to be a graph-based memory system engineered to support various AI agents in retrieving highly meaningful, context-aware information. By combining structured knowledge graphs with vector spaces, it bridges the gap between raw data and cognitive recall.

But how do computers actually grasp that “The cat jumps onto the bed” and “A feline leaps onto the mattress” express the exact same concept, even though they share almost no words? How does a system like Memory-MCP rank what is truly “meaningful”?

The answer lies in the combination of three key elements: Embeddings, Cosine Similarity, and ANN (Approximate Nearest Neighbors) search algorithms. In this article, we will break down how these technologies work step by step.

1. The Starting Point: What Are Embeddings?

An embedding is the conversion of a real-world object (such as text, images, or audio) into a fixed-length list of numbers, known as a vector.

When we pass a sentence into an AI model (like those provided by OpenAI or Hugging Face), the text is mapped into an abstract, multidimensional space (often exceeding 1,500 dimensions).

• Semantic Placement: The model assigns numerical coordinates so that words, phrases, or documents with a similar meaning are placed geometrically close to each other.
• The Geometry of Meaning: The words “king” and “queen” will have very similar vectors, just like “dog” and “puppy”.

Embeddings essentially turn the nuance and ambiguity of human language into pure linear geometry.

2. Measuring Meaning: Cosine Similarity

Once texts are transformed into numerical vectors, we need a metric to measure how close or far apart they are in that geometric space. The standard technique for this is cosine similarity.

Unlike Euclidean distance (which draws a straight line “as the crow flies” between the tips of two vectors), cosine similarity measures the size of the angle formed by two vectors starting from the same origin.

The mathematical formula normalizes the vectors and calculates their dot product:

\(\text{Cosine\ Similarity}=\frac{A\cdot B}{\|{}A\|{}\|{}B\|{}}\)

The result is a pure value, always ranging between -1 and 1:

• 1 (0° Angle): The vectors point in the exact same direction. This means maximum semantic similarity.
• 0 (90° Angle): The vectors are orthogonal. The concepts are independent or unrelated.
• -1 (180° Angle): The vectors point in opposite directions (opposite concepts).

Why Use Cosine Instead of Euclidean Distance?

In machine learning applied to text, the length of a vector (magnitude) is heavily influenced by word count. A 10-page article and a two-line tweet about the exact same topic will have vectors of vastly different lengths (making them far apart by Euclidean standards).

Cosine similarity solves this: because it evaluates only the direction and ignores the length, it recognizes that both the article and the tweet are talking about the same thing, returning a score close to 1.

3. The Search Infrastructure: The Workflow (RAG)

In modern enterprise architectures, such as RAG (Retrieval-Augmented Generation) systems, the search process follows four main phases:

1. Indexing: All company documents are converted into vectors using an embedding model and saved into a vector database (e.g., Qdrant, Pinecone, Milvus).
2. Query Vectorization: When a user types a question (e.g., “How do I take care of my puppy?”), the query is transformed into a vector using the exact same model.
3. Distance Calculation: The database compares the query vector against the stored document vectors.
4. Ranking: The system returns the text chunks with the highest cosine similarity scores.

4. The Scale Challenge: Exact k-NN vs. ANN

When a database contains only a few texts, calculating the cosine between the query and all documents takes fractions of a millisecond. But what happens when the vectors scale to millions or billions?

This is where the engineering trade-off between mathematical precision and computational speed comes into play.

Exact k-NN (Exact Nearest Neighbors)

Exact k-NN performs a sequential linear search (brute-force). It compares the query vector against every single item in the database.

• Accuracy: 100%. It mathematically finds the absolute closest vectors.
• Complexity: O(N). If the data doubles, the computation time doubles. This becomes unsustainable at large scale.

ANN (Approximate Nearest Neighbors)

ANN gives up 100% mathematical certainty in exchange for blazing-fast execution speeds. It promises to find results that are extremely close to the true best neighbors in near-instantaneous time (\(O(\log N)\) or O(1)).

• Accuracy: 95% — 99% (measured in Recall).
• How does it approximate? Instead of checking every vector, it organizes the space into pre-computed advanced data structures (indexes), instantly isolating the geometric region where the answer is most likely to be found.

The 3 Most Common ANN Algorithms

To optimize search, vector databases organize data using three main approaches:

1. Graphs (HNSW — Hierarchical Navigable Small World): The current state-of-the-art. It creates a multi-layered network where upper layers allow for “big jumps” across the vector space, while lower layers handle local, precise navigation.
2. Quantization (IVF-PQ): Divides the space into clusters. The system only searches within the cluster closest to the query. It also compresses vectors to save RAM.
3. Trees (KD-Trees): Recursively splits the space using hyperplanes. Highly effective, but tends to lose efficiency when the space has too many dimensions (the curse of dimensionality).

FeatureExact k-NNANN (e.g., HNSW)Accuracy (Recall)Maximum (100%)Approximate (95–99%)Search TimeSlow (scales linearly)Ultra-fast (milliseconds on billions of rows)Memory UsageLow / ModerateHigh (graph indexes require significant RAM)ScalabilityPoorExcellent

5. Conclusion

Building an efficient semantic search system isn’t just about choosing the smartest embedding model; it’s about balancing business needs. If you have a small, niche dataset, Exact k-NN will give you total precision at zero index cost. However, if you are designing an enterprise system meant to scale, adopting ANN indexes like HNSW within a dedicated vector database becomes an absolute necessity.

if you want to learn more, see (and star) my project at

https://github.com/alex1976/Memory-MCP

Memory-MCP: Guide to Semantic Search From Embeddings to HNSW and Cosine Similarity was originally published in CodeX on Medium, where people are continuing the conversation by highlighting and responding to this story.

The problem nobody likes to admit

You ask one agent to research a vendor. It reads the docs, extracts the pricing model, writes a summary — and that knowledge dies with the session. Two days later, a different agent, working on the same project for the same team, has to rediscover it from scratch, because there was never a shared place to put it.

This isn’t a “context window” problem. It’s a coordination problem. Every agent, every tool, every session tends to keep what it learns to itself. Multiply that across a team running several agents on the same project — a research agent, a coding agent, a support agent — and you get the same facts rediscovered independently, sometimes inconsistently, because nothing connects what one agent knows to what another needs.

What’s missing isn’t a bigger prompt. It’s a shared layer of memory that any authorized agent — or person — can read from and write to, so knowledge accumulates once and stays available to everyone who’s supposed to see it.

That’s what Memory-MCP is: a remote MCP (Model Context Protocol) server for storing, searching, and forgetting memories, organized into multi-tenant spaces and protected by API keys — designed so multiple agents and teammates can genuinely share what they know, without stepping on each other or leaking into contexts they shouldn’t.

What it actually is

Memory-MCP exposes a small set of MCP tools — search_memory, add_memory, listMemories, listDocuments, getDocument, listSpaces, whoAmI — plus resources, a context prompt, and interactive widgets. Any MCP-capable client can connect over HTTP with an API key: Claude Desktop, an agent built on the Claude Agent SDK, a CI pipeline, MCP Inspector — anything speaking the protocol.

The unit that makes sharing possible is the space: a logical container for a project, a client, or a domain. What makes it a genuine collaboration mechanism rather than just a namespace is how access is granted. An API key isn’t tied to one agent — it’s tied to one identity (a person, a service, an automation), with Read or ReadWrite permission on each space it's allowed to touch, and one space marked as its default. Give three different agents three different keys, all granted ReadWrite on the same acme-project space, and every fact one of them saves is immediately visible to the other two on their very next search_memory call — no export, no sync job, no copy-pasting a summary between chat windows.

Memory as a shared surface, not a private notebook

A few design choices only make sense once you stop thinking of memory as “one agent’s private cache” and start thinking of it as a surface multiple agents read and write concurrently.

Atomic facts instead of whoever-wrote-it-last text blobs. When content is saved through add_memory, an IFactExtractor doesn't just store the raw paragraph — it decomposes it into atomic, self-contained facts ("Alex left Stripe in March"), each becoming its own memory. That matters a lot more once several contributors are writing into the same space over time: atomic facts are unambiguous to combine, re-rank, and search, in a way that competing free-text paragraphs from different agents are not.

Relations that reconcile contributions instead of just concatenating them. If a new fact updates, extends, or derives from an existing one — regardless of which agent originally wrote that existing one — a typed graph edge (Updates / Extends / Derives) is created at save time, and an Updates relation automatically deactivates the memory it supersedes. This is the piece that makes multi-agent writing safe: agent B correcting something agent A wrote yesterday doesn't produce two contradictory memories sitting side by side — it produces a coherent, superseding chain that search_memory's relatedMemories surfaces automatically.

Spaces as a hard boundary, not a soft convention. Because access is enforced per space at the API key level — not by an agent politely deciding what’s relevant — a support agent scoped to customer-x structurally cannot see or pollute customer-y's memory, even if both are backed by the same Memory-MCP instance. Sharing within a space is deliberate and total; leakage across spaces isn't possible by construction.

category and keyword as shared vocabulary. When several contributors write into the same space, category becomes a lightweight shared taxonomy ("pricing", "incidents", "decisions") that any agent can filter on, and keyword/fuzzy search (more on that below) means a teammate searching by hand doesn't need to remember the exact phrasing another agent used when it saved the fact.

A context prompt that hands off state cleanly. The context MCP prompt returns a ready-to-attach message — the active space's recent profile, plus other recently active spaces — which is as useful for onboarding a new agent into an ongoing shared space as it is for resuming a session: whoever picks up the work next starts from what the group already knows, not from an empty conversation.

Under the hood: four layers, one direction

Memory-MCP is a .NET 10 / ASP.NET Core project, built as Clean Architecture with dependencies flowing one way — Api → Infrastructure/Application → Domain:

Memory-MCP/
├── Domain/          # Pure entities: Space, ApiKey, Document, Memory — zero framework dependencies
├── Application/     # Use cases: IMemoryService, IEmbeddingProvider, repository interfaces
├── Infrastructure/  # EF Core, Postgres, the embedding provider, the fact extractor
└── Api/             # ASP.NET Core host: MCP tools/resources/prompts as thin adapters

Every tool in Api/Tools does exactly three things: resolve the access context, call an application service, format the output. That separation is what lets the same IMemoryService back search_memory, the MCP resources, the context prompt, and every interactive widget identically — whichever surface an agent or a human happens to be using, the sharing and access-control rules behind it are enforced in exactly one place.

Persistence is plain PostgreSQL via EF Core. Embeddings are stored as native Postgres real[] arrays, and IEmbeddingProvider is pluggable across OpenAI, Azure OpenAI, or Gemini.

Graph memory: reconciling knowledge from many contributors

When add_memory saves content, an LLM-backed IFactExtractor splits it into atomic facts and classifies each fact's relationship to a handful of similar existing memories — regardless of who or what originally wrote them. Each fact becomes its own row; each relation becomes a typed, directed MemoryEdge.

Traversing that graph — “what’s connected to this memory, up to two hops away” — doesn’t need a dedicated graph database: it’s a WITH RECURSIVE CTE over a plain memory_edges table, bounded by a hop count and a visited-node array to guarantee termination. search_memory attaches each top match's related memories to the result, so whoever — or whichever agent — searches next gets the reconciled picture, not just the most recent write.

If no extraction model is configured, none of this breaks: add_memory falls back to saving the whole content as a single flat memory. Graph memory is additive on top of shared storage, not a hard dependency of it.

A small, honest case study in tuning: fuzzy keyword search

Not every feature came from a grand design session. search_memory's keyword parameter started as a plain case-insensitive ILIKE '%keyword%' match — fine until a teammate searches for something a different agent phrased slightly differently, or just mistypes it.

The fix is PostgreSQL’s pg_trgm extension, exposed through EF Core as TrigramsAreWordSimilar / TrigramsWordSimilarity, backed by a GIN trigram index. The interesting part was resisting the instinct to lower the similarity threshold below Postgres's default of 0.6 to catch more typos. Testing that instinct against real data:

word_similarity('migraiton', 'migration')  = 0.5    -- a real typo, useful to catch
word_similarity('recieve',   'receive')    = 0.375  -- also a real typo
word_similarity('plan',      'plant')      = 0.8    -- NOT a typo, false positive
word_similarity('sky',       'skip')       = 0.5    -- NOT a typo, false positive

Short keywords are inherently noisy under trigram similarity: a 4-letter word only has two or three trigrams, so any partial overlap scores deceptively high. Lowering the threshold to catch recieve/receive would also start matching plan/plant as if they were typos of each other — actively worse for a shared space where multiple people search by hand with their own habits and typos. The default threshold stayed, documented explicitly in the codebase, because in a shared search surface, precision matters as much as recall.

Interactive widgets: a shared memory people can see, too

For MCP clients that support MCP Apps, Memory-MCP ships four widgets that render inside the conversation: a searchable space picker, a guided save form, a file upload flow (text, Markdown, CSV, and PDF via PdfPig), and a force-directed graph of a space's memories and relations. These matter specifically because the memory isn't private to one agent's session — a teammate can open the graph widget and see what the team's agents have collectively learned, correct a fact through the same guided-save form an agent would use, or switch which shared space they're currently working in. The widgets are thin: each is a small postMessage-based bridge to the same tools and resources any agent calls, so a human and an agent editing the same space go through identical rules.

Building real infrastructure under real constraints

Worth saying plainly: this was built where Docker Desktop is blocked by company policy and the available Postgres has neither pgvector nor admin rights to install it. The original spec called for pgvector with an HNSW index and a dedicated graph database. Neither was available, so both became deliberate trade-offs instead of blockers: cosine similarity computed in-app over a plain real[] column, a recursive CTE instead of a graph engine, pg_trgm — a contrib extension needing no elevated privileges — instead of anything requiring installation. Every one of these is documented in the codebase as a conscious choice, with a note on what migrating to the "ideal" version would take if the constraint ever lifts.

Try it

Memory-MCP runs anywhere .NET 10 and Postgres run — locally via dotnet run, in Docker via the provided docker-compose.yml, or deployed to Fly.io (coming soon). Give each agent (or each teammate) its own API key scoped to the spaces it should share, point Claude Desktop or your agent framework at the /mcp endpoint, and the same shared memory shows up identically across every one of them.

Full setup instructions and the reasoning behind every trade-off mentioned here live in the repo: https://github.com/alex1976/Memory-MCP.

What’s next

The roadmap is explicit about what’s deferred: a real graph database (Neo4j) if a space’s graph outgrows recursive CTEs, a native vector index (Qdrant or pgvector, if it becomes available) as more agents share more history in the same space, and richer document ingestion — Word, image OCR, audio transcription — for the upload widget.

None of it changes the core bet: the more agents and people work on the same problem, the more valuable it is that what one of them learns doesn’t stay locked in a single conversation.

Memory-MCP: A shared memory layer for AI Agents that work as a team was originally published in Dev Genius on Medium, where people are continuing the conversation by highlighting and responding to this story.

link: https://platform.openai.com/chat/edit?optimize=true

Prompt optmizer uses the guidelines proposed by OpenAI itself for the gpt 5 model.

link: https://cookbook.openai.com/examples/gpt-5/gpt-5_prompting_guide

💡 How does it work?
1. Choose the model for which you want to optimize the prompt

2. Paste your original prompt

3. Click “Optimize”

4. The tool rewrites it, applying the official prompting guidelines for the various models (those published by OpenAI itself)

5. Use the new optimized prompt

🎯 Result:
- Highlights the steps to follow
- Eliminates ambiguity
- Clearly defines inputs and outputs
- Ability to save “optimized” versions and reuse them

Conclusion:

This tool is really very useful to make prompts complete and effective

Definition:
Data products are foundational components in a modern data ecosystem, particularly within a data mesh architecture. They are designed to serve analytical data effectively, applying the principles and practices of software product development to the data domain. By doing so, data products ensure data is accessible, understandable, trustworthy, and valuable to its users.

Key Characteristics of Data Products

The following attributes define an effective data product:

1. Discoverable
Users should easily locate and understand the data products available to them. This involves robust metadata, intuitive search, and clear documentation.

2. Addressable
Each data product must have a unique identifier (e.g., a URI or URL), ensuring consistent and reliable access programmatically or manually.

3. Understandable (Self-Describable)
Comprehensive documentation is critical. This includes the data product’s purpose, structure, field-level descriptions, usage instructions, and sample data for easy evaluation.

4. Trustworthy
Trust is built through transparency about the product’s quality, performance, and reliability. This includes publishing Service Level Objectives (SLOs) and adhering to measurable Service Level Indicators (SLIs).

5. Natively Accessible
Data products should be tailored for various consumer needs:

• Managers might need prebuilt reports.
• Data scientists could require SQL-based access.
• Engineers might prefer API-based programmatic access.

6. Interoperable (Composable)
To enable seamless integration with other data products, interoperability is key. This includes supporting standard business keys and common access patterns for joining, filtering, and aggregating data.

7. Valuable on Its Own
A data product must deliver independent value, representing a cohesive concept within its domain without requiring additional context or joins to be useful.

8. Secure
Strong access controls, encryption, and compliance with domain-specific regulations ensure the data product is protected against unauthorized use.

Note: to learn more about the topic see Zhamak Dehghani’s book “Data Mesh: Delivering Data-Driven Value at Scale”

Functional Perspective

A data product is not merely a dataset; it is a packaged, self-contained entity that:

• Provides structured, semi-structured, or unstructured analytical data.
• Caters to specific user groups and their data consumption patterns.
• Embeds necessary automation and infrastructure to efficiently achieve its purpose.

Business Value of Data Products

By aligning with the principles of modern software development, data products:

• Decompose large, monolithic data systems into manageable, reusable components.
• Encourage scalability, discoverability, and maintainability.
• Support consistent delivery of meaningful business insights and value.

Data products are the smallest valuable unit of analytical data within a data mesh, facilitating data-driven decision-making while prioritizing usability and reliability for diverse user personas.

What Data Products Are Not

A robust definition of data products requires not only specifying what they are but also clarifying what they aren’t. By doing so, organizations can avoid common pitfalls, ensure focus, and streamline the creation and management of effective data products.

Why Clarity Matters

• A narrower, well-defined scope prevents confusion across teams.
• It enables the development of reusable blueprints and “paved paths” that simplify building and managing data products.
• It maintains focus on solving problems specific to analytical data rather than trying to address all data-related challenges simultaneously.

Common Misrepresentations and Missing Characteristics

Key Differentiators

• Data Products vs. Data-Driven Applications:
• Data Products are designed for programmatic access and composability, making them building blocks in a data mesh.
• Data-Driven Applications are intended for human interaction and lack inherent composability.
• Focus on Analytical Data:
  Data products apply software engineering principles to analytical data to address ownership and quality issues, but they are not the catch-all solution for broader enterprise data challenges.

Why Misunderstandings Happen

• Conflation of Concepts: Teams sometimes mix up data products with other data artifacts or tools due to the overlap in terminology or functionality.
• Diluted Language and Focus: Attempting to solve all ownership and quality issues under the banner of data mesh risks diluting the transformative impact of focused data product design.

Conclusion

By clearly distinguishing what data products are and aren’t, teams can maintain focus, improve clarity, and build scalable, reusable solutions that align with the principles of a data mesh. This ensures data products deliver on their promise of creating composable, high-quality analytical data units that drive business value.

Credits: https://martinfowler.com/articles/designing-data-products.html

In this short DAX pill we see how to find the first and the last value of a given column within a selected time range.

the context is that of the analysis of the trend of a cryptocurrency (e.g. BITCOIN) in a predefined time frame (e.g. quarter)

Let’s create the following calculated measurs:

1. “FirstAdjClose”
2. “LastAdjClose”

where:

CryptoHistorical is the table where we have the historical data of a cryptocurrency

FIRSTDATE is the function that returns the first date in the current context for the specified date column.

LASTDATE is the function that returns the last date in the current context for the specified date column.

see the link:

https://learn.microsoft.com/it-it/dax/firstdate-function-dax

https://learn.microsoft.com/it-it/dax/lastdate-function-dax

the DAX sintax is the following:

FirstAdjClose =

VAR FirstDateValue = FIRSTDATE(CryptoHistorical[historical.date])

VAR result = CALCULATE(MAX(CryptoHistorical[historical.adjClose]), FILTER(CryptoHistorical, CryptoHistorical[historical.date]=FirstDateValue))

RETURN result

**********************************************************************

LastAdjClose =

VAR LastDateValue = LASTDATE(CryptoHistorical[historical.date])

VAR result = CALCULATE(MAX(CryptoHistorical[historical.adjClose]), FILTER(CryptoHistorical, CryptoHistorical[historical.date]=LastDateValue))

RETURN result

**********************************************************************

Note that the date (first or last) is stored in a variable and then passed as a filter to the CALCULATE function

Reference to my previous post about Api economy:

https://blog.devgenius.io/the-apis-economy-c3ecd72061cd

I will talk about APIs available online and usable for real purposes and applications, I will talk about tools for API development, I will also talk about products (e.g. API marketplace) dedicated to the world of APIs.

In this first pill we will see a very useful list of APIs and resources (work in progress) relating to the world of weather (current, forecast), air-quality and other data.

AccuWeather:

Weather and forecast data.

Access takes place via an ApiKey in HTTP mode and there are various free and paid plans for using the APIs

API Reference

Weatherstack:

Real-Time & Historical World Weather Data API.

Access takes place via an ApiKey in HTTPS mode and there are various free and paid plans for using the APIs

Weatherstack - Real-Time World Weather REST API

OpenWeatherMap:

Weather forecasts, nowcasts and history data.

Access takes place via an ApiKey in HTTPS mode and there are various free and paid plans for using the APIs

Weather API

WeatherApi:

JSON and XML Weather API and Geolocation API

Access takes place via an ApiKey in HTTP or HTTPS mode and there are various free and paid plans for using the APIs. Support for CORS is present.

Weather and Geolocation API - Weather and Geolocation API JSON and XML - WeatherAPI.com

US Weather API:

Data from US National Weather Service.

All of the information presented via the API is intended to be open data, free to use for any purpose. Access takes place in HTTPS mode.

API Web Service

WeatherBit:

Current, forecasts and alert weather data. Air quality data.

Access takes place via an ApiKey in HTTP or HTTPS mode and there are various free and paid plans for using the APIs.

API Documentation

Tomorrow.io:

Enterprise-grade Weather API (current, real-time, forecast)

Access takes place via an ApiKey in HTTPS mode and there are various free and paid plans for using the APIs.

Tomorrow.io APIs

Stormglass.io:

Weather forecasts & historical data from the world’s most trusted meteorological institutions. Include marine weather (wave and wind).

Access takes place via an ApiKey in HTTPS mode and there are various free and paid plans for using the APIs.

stormglass.io Weather API

M3O Weather:

Real time weather forecast.

Access takes place via an ApiKey in HTTPS mode and there are various free and paid plans for using the APIs.

Weather API - Real time weather forecast

OpenSenseMap:

OpenSenseMap is an open sensor data platform that anyone can participate in. Data comes from Personal Weather Stations called “SenseBoxes”. Air-quality, temperature and humidity data are available.

Access requires token authentication only for some APIs. Access in HTTP mode.

openSenseMap.org

INGV Earthquake events:

Italian open data on earthquakes.

All of the information presented via the API is intended to be open data, access is free and does not require ApiKey. Access in HTTP mode.

INGV Earthquake event

API economy : pills n.1 — A very useful collection of weather API was originally published in Dev Genius on Medium, where people are continuing the conversation by highlighting and responding to this story.

What is an API:

An API (Application Programming Interface) is a tool that allows companies to make the data and functionality of their applications available to external developers, business partners and internal departments of the company itself. This allows different services and products to communicate with each other and leverage each other’s information and functionality via a well-documented interface. Developers don’t need to know the technical details of how the API was created; they simply use the interface to interact with other products and services. The use of APIs has grown significantly over the last decade, so much so that many of the most popular web applications today could not exist without the use of APIs.

How the API works:

An API (Application Programming Interface) is a set of clearly defined rules that dictate how computers or applications should communicate with each other. APIs serve as an intermediary layer situated between an application and a web server, facilitating the processing of data exchange between different systems.

Here’s how an API works:

1. A client application initiates an API call, often referred to as a request, to obtain specific information. This request is transmitted from the client application to the web server using the API’s Uniform Resource Identifier (URI). It typically includes a request verb, headers, and sometimes a request body.

2. Once a valid request is received, the API acts as an intermediary and makes a call to the external program or web server that holds the requested data or functionality.

3. The server processes the request and sends a response back to the API, containing the requested information.

4. The API then passes this data back to the initial requesting client application.

This request and response process takes place entirely through the API. While user interfaces are designed for human interaction, APIs are specifically designed for computer or application-to-application communication.

APIs are designed with security in mind. Their intermediate position helps simplify the abstraction of functionality between two systems, with the API endpoint acting as a separation layer between the requesting application and the underlying infrastructure providing the service. Here are some key aspects of API security:

1. Authorization Credentials: API calls typically include authorization credentials, such as API keys or tokens. These credentials help verify the identity of the requesting application and reduce the risk of unauthorized access to the server.

2. API Gateway: An API gateway can be used to manage and secure API access. It serves as a central entry point for API requests and can enforce security policies, throttle requests, and provide logging and monitoring to minimize security threats.

3. HTTP Headers, Cookies, and Query Parameters: During the data exchange between the client and server via the API, HTTP headers, cookies, and query string parameters can be used to add additional layers of security. For example, secure cookies can help maintain session state, and headers can be used to specify content types and enforce security policies.

4. Rate Limiting: API security often includes rate limiting to prevent abuse. By restricting the number of requests a client can make within a specific time frame, it helps protect the server from potential denial-of-service attacks.

5. Encryption: To secure data in transit, APIs commonly use encryption protocols such as HTTPS (HTTP Secure) to encrypt the communication between the client and server, ensuring that data remains confidential and cannot be easily intercepted.

6. Authentication: APIs can implement various authentication methods, such as OAuth, API keys, or token-based authentication, to verify the identity of clients and grant access only to authorized users or applications.

7. Access Control: Access control mechanisms within APIs ensure that users or applications can only access the data and functionality they are authorized to use. This helps prevent unauthorized access to sensitive information.

In summary, APIs incorporate multiple layers of security measures, including authentication, authorization, encryption, and access control, to protect both the API provider’s data and the users of the API. These security features are essential for ensuring the integrity and confidentiality of data exchanged through APIs.

For instance, let’s take the case of an API provided by a payment processing service. In an e-commerce store application, customers input their card details into the frontend. Instead of needing access to the user’s bank account directly, the payment processor’s API generates a unique token for that specific transaction and includes it in the API request sent to the server. This approach enhances security by reducing the risk of potential hacking threats because the actual card details are not exposed during the transaction.

Why we need APIs:

Using APIs offers numerous advantages for both managing existing tools and designing new ones. Some of the key benefits of APIs include:

1. Improving Collaboration: Many companies use a wide range of cloud applications, often not integrated with each other. APIs enable integration, allowing these platforms to communicate seamlessly. This integration automates workflows and improves workplace collaboration. Without APIs, companies would be forced to use siled data, compromising productivity and performance.

2. Simple Innovation: APIs offer flexibility, allowing companies to establish connections with new business partners, offer new services to their existing market and enter new markets. For example, Stripe started as an API with a few lines of code and has grown into a billion-dollar company through partnerships with large companies and diversification of services.

3. Data Monetization: Many companies start by offering APIs for free to build a developer base and establish relationships with potential partners. However, if the API offers access to valuable data, you can monetize it by selling access. AccuWeather, for example, has attracted thousands of developers and created a growing community by selling API keys.

4. Added Security: APIs add an extra layer of security between the data and the server. Developers can further enhance API security using tokens, digital signatures, and Transport Layer Security (TLS) encryption. Using API gateways to manage and authenticate traffic helps further protect resources. Additionally, effective API management is critical to ensuring secure operations.

In summary, APIs are essential tools that facilitate collaboration, foster innovation, enable data monetization, and strengthen the security of business applications and services. They are a key element in digital transformation and the creation of new business opportunities.

Types of API protocols:

As the use of Web APIs has increased, various protocols have been developed to provide users with a set of well-defined rules that specify the commands and data types accepted. These API protocols simplify the standardized exchange of information. Here are some of them:

1. SOAP (Simple Object Access Protocol): This is an XML-based API protocol that allows users to send and receive data via protocols such as SMTP and HTTP. SOAP APIs make it easy to exchange information between applications or software components running in different environments or written in different programming languages. It is known for its robustness and support for complex operations.

2. XML-RPC: This protocol uses the XML format for data transfer, but unlike SOAP, it uses a simpler XML format. XML-RPC is older than SOAP but is known for its lightweight and minimal bandwidth usage.

3. JSON-RPC: Similar to XML-RPC, but uses JSON (JavaScript Object Notation) format for data transfer. Both protocols are known for their simplicity and support remote procedure calls (RPCs) with parameters and results.

4. REST (Representational State Transfer): This is a principle-based Web API architecture rather than a specific protocol. REST APIs (known as RESTful APIs) must respect certain architectural constraints, such as the use of URIs (Uniform Resource Identifier) ​​to identify resources and the use of HTTP methods (GET, POST, PUT, DELETE) to interact with them . There is no official standard for RESTful APIs, but they follow a set of key API design principles.

It’s important to note that while it’s possible to build RESTful APIs using protocols like SOAP, the two approaches are often thought of as competing specifications and follow different design philosophies. The choice of protocol often depends on the specific needs of the project and the preferences of the development team.

Common API examples:

APIs have become a valuable asset in the modern business world as they allow companies to offer access to their resources while maintaining security and control. Below are some popular examples of APIs you may encounter:

1. Universal Logins: Many platforms allow users to log in to websites using the login details of their accounts on services such as Facebook, Twitter or Google. This is made possible through APIs, which allow websites to use the authentication credentials of these services to simplify the login process. This saves users the hassle of having to create new accounts for every web service they use.

2. Third Party Payment Processing: Services like “Pay with PayPal” on e-commerce websites use APIs to allow users to make payments securely without having to share sensitive financial information with the website. Payment APIs enable secure and convenient transactions.

3. Travel Booking Comparisons: Travel booking websites aggregate information on flights, hotels and other travel options from multiple sources. This is possible thanks to APIs that provide application users access to a wide range of data updated in real time. Users can easily compare travel options without having to manually consult different sources.

4. Google Maps: Google Maps offers a set of APIs that allow developers to integrate interactive maps and geolocation services into their applications. These APIs allow users to view maps, get directions, find places of interest, and much more. Google Maps APIs are widely used in web and mobile applications.

5. Twitter: Twitter offers an API that allows developers to access tweet data and interact with the platform. This allows developers to create applications and services that use public tweets and replies. For example, social media management applications use the Twitter API to publish tweets and analyze social media activity.

In summary, APIs are critical to building modern services and applications that rely on data and functionality provided by third parties. They enable integration, security and simplified access to a wide range of digital assets, improving the user experience and expanding business opportunities.

The APIs economy was originally published in Dev Genius on Medium, where people are continuing the conversation by highlighting and responding to this story.