This project implements a minimal metadata service for blobs using:
- FoundationDB (FDB) → strongly consistent transactional catalog.
- MinIO → blob storage backend.
- Java (Javalin + FDB Java bindings + MinIO client) → lightweight HTTP API to register and query metadata.
The goal is to demonstrate how FDB can serve as a metadata store in a realistic design, ready to evolve toward advanced features (idempotency, CAS, secondary indexes, SNAPs).
-
Key schema in FDB (Tuple Layer):
("b", tenant, objId, "meta") -> JSON metadata ("b", tenant, objId, "state") -> byte (0=pending, 1=committed)
-
Minimal API (Javalin):
PUT /o/{tenant}/{id}
→ register/update metadata (validates blob in MinIO).GET /o/{tenant}/{id}
→ return metadata JSON.HEAD /o/{tenant}/{id}
→ existence check.GET /o/{tenant}?prefix=abc
→ list object IDs by prefix.DELETE /o/{tenant}/{id}
→ delete metadata (optionally delete blob).
-
Dockerized stack:
- FoundationDB 7.4
- MinIO server
- Java app (Gradle + ShadowJar)
-
Designed to grow:
- Multi-tenant isolation.
- Extendable to CAS, secondary indexes, SNAP composition.
# Build images
docker compose build
# Start stack
docker compose up -d
# Initialize FDB cluster (first run only)
docker compose exec fdb fdbcli --cluster_file /var/fdb/fdb.cluster --exec "configure new single ssd"
# Create a bucket in MinIO
mc alias set local http://localhost:9000 minio minio12345
mc mb local/metadata
Register metadata example:
curl -X PUT http://localhost:8080/o/tenant1/foto-001 \
-H "Content-Type: application/json" \
-d '{
"bucket": "metadata",
"object": "networking.jpg",
"contentType": "image/jpeg",
"size": 12345,
"sha256": "abc123...",
"tags": {"owner":"fernando","type":"photo"}
}'
- Advanced idempotency with
Idempotency-Key
. - Secondary indexes (tags, owner).
- Deduplication via SHA-256 (CAS).
- SNAP integration (queues, quotas, profiles).
- Observability (structured logs, p95/p99 metrics).
- FoundationDB Record Layer (SIGMOD’19)
- [Apple CloudKit (VLDB’20)]
- DeepSeek 3FS (arXiv 2025)