Member-only story
AI on Desktop
How Microsoft Foundry Local and Olive Simplify Model Optimization and running it locally
GenAI | Foundry Local | Cloud | Olive | ONNX | Inference | Open WebUI
Foundry Local is available in preview. Public preview releases provide early access to features that are in active deployment.
AI development has never moved faster, and for many engineers and applied ML teams, the real challenge is no longer training cutting-edge models it’s deploying them efficiently and making them easily accessible to the developers who will use them to build applications. Running advanced AI models locally, without relying on the cloud, complex server setups, or expensive infrastructure, has become essential for developers, researchers, and enterprises alike.
While tools like Ollama and LM Studio have made local LLMs incredibly accessible, Foundry Local targets a broader and more ambitious goal. It integrates deeply with Microsoft’s optimization ecosystem including ONNX Runtime and Olive to deliver hardware-aware acceleration, reproducible optimization, and seamless local execution of models across GPUs, NPUs, and CPUs from virtually every major hardware vendor. In this post we will try to cover all these topics and see how this ecosystem work.
