Antonello Fratepietro·Nov 6Rust Inference Engine: How to Run AI Models in RustA Rust inference engine runs a trained model from Rust code: an LLM, an embedding model, a classifier or a recommender. You give it weights…
Antonello Fratepietro·Oct 22Admission Control for Distributed LLM InferenceMost of what I read about distributed LLM inference is about routing. Which GPU should take this request? Ideally the one that already has…
Antonello Fratepietro·Oct 19How to Build a Decision Model Like JevMost software decisions do not want text generation at all. Routing a support ticket, gating a risky tool call for human review, scoring an…
Antonello Fratepietro·Oct 7Google Cloud’s Agentic AI Blueprint: From Design Patterns to Private Inference NetworkingMost agent demos stop at the interesting part: the model reasons, calls a tool, answers. The questions that decide whether it survives…
Antonello Fratepietro·Sep 24A 322M AI Model Traded NVIDIA on Paper and Beat Buy and Hold (+7.5% vs +2.0%)I let a small local AI model paper-trade crypto and S&P 500 stocks. On NVIDIA: +7.5% vs +2.0% for buy and hold. Every loss included too.A response icon1A response icon1
Antonello Fratepietro·Sep 23Laya vs Dijkstra: can a typed-decision model find its way out of a maze?laya-mlx is an MLX port of Laya, a model that doesn’t generate text. You give it a state and a typed question: a choice, a score, or a…
Antonello Fratepietro·Sep 22LLM Inference: a learning roadmapMost conversations I have about language models are not about training. They are about what happens after you hit send: where the time…
Antonello Fratepietro·Sep 2gitgui: a Rust real git GUI inside cmux, next to Pihttps://github.com/antonellof/gitguiA response icon1A response icon1
Antonello Fratepietro·Aug 23Ferrox v0.9.1: a Rust GGUF engine, measured against llama.cppFerrox is a pure-Rust inference engine for GGUF models. Dense and mixture-of-experts, on CPU, Apple Metal, or CUDA. No bindings to…A response icon1A response icon1
Antonello Fratepietro·Aug 4Ferrox: Building a Rust Inference Engine That Matches llama.cppI’ve spent the last few days building Ferrox, a pure-Rust inference engine for running open LLMs locally — dense models and…A response icon2A response icon2
