A text embeddings inference server built with Zig, leveraging the power of Zig, ZML, and Zap for blazing fast HTTP serving capabilities.
The project uses bazel as its build system, primarily because ZML is built with Bazel. This ensures compatibility and reproducible builds.
MacOS installation :
brew install bazeliskLinux installation (amd64) :
curl -L -o /usr/local/bin/bazel 'https://github.com/bazelbuild/bazelisk/releases/download/v1.25.0/bazelisk-linux-amd64'
chmod +x /usr/local/bin/bazelbazel run -c opt //:ModernBERT-largebazel run -c opt --@zml//runtimes:cuda=true //:ModernBERT-large| Model ID | Description | Status |
|---|---|---|
ModernBERT-large |
Answer.AI's ModernBERT large model | ✅ Available |
nomic-ai/modernbert-embed-base |
A ModernBERT-based embedding model from Nomic AI | |
Alibaba-NLP/gte-modernbert-base |
An improved GTE (General Text Embeddings) variant based on ModernBERT |