<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Function on Medium]]></title>
        <description><![CDATA[Stories by Function on Medium]]></description>
        <link>https://medium.com/@fxnai?source=rss-09537b3a5896------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*qT-Zlx9bPmqwOMLOgZjc5w.png</url>
            <title>Stories by Function on Medium</title>
            <link>https://medium.com/@fxnai?source=rss-09537b3a5896------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 05:32:38 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@fxnai/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Keep Your Next Viral AI App Free For Longer With Local Embeddings]]></title>
            <link>https://medium.com/fxnai/keep-your-next-viral-ai-app-free-for-longer-with-local-embeddings-33d209ceb8e1?source=rss-09537b3a5896------2</link>
            <guid isPermaLink="false">https://medium.com/p/33d209ceb8e1</guid>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[nextjs]]></category>
            <category><![CDATA[function]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Function]]></dc:creator>
            <pubDate>Thu, 19 Sep 2024 19:27:06 GMT</pubDate>
            <atom:updated>2024-09-19T19:27:06.017Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/736/1*GIQeXQolGXF4-hcFkz2oqw.gif" /></figure><h4>Patch an OpenAI client with Function LLM in one line of code.</h4><p>Whether you’re a solopreneur building an intelligent search app; or you’re a series A startup finding product-market fit in enterprise knowledge management; you can be saving 60% or more on your monthly OpenAI bills by generating embeddings on your users’ devices. We’ll show you how you can extract huge margins (and hopefully reach your next funding round) in about 5 minutes with one extra line of code:</p><h3>Your Current Stack Is Leaving Money On The Table</h3><p>Text embeddings form the foundation of many AI-powered apps today. They work by turning any arbitrary block of text into a vector of numbers that retains the semantic meaning of the text:</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*bPGIAjGkRSp3ys4z.png" /><figcaption><a href="https://dagshub.com/blog/how-to-train-a-custom-llm-embedding-model/">Source</a>.</figcaption></figure><p>Right now, a lot of developers rely on OpenAI to generate these embeddings, and are charged on a per-token basis. Each token costs a fraction of a cent, but when you multiply that by thousands (or millions) of users, each throwing tons of text at you every month (millions of tokens), you end up with monthly bills in the tens of thousands of dollars or more.</p><h3>Enter: On-Device Embeddings with Function LLM</h3><p>It is now feasible to run state-of-the-art text and vision embedding models directly on your users’ devices. To do so, we have created a lightweight library, <a href="https://github.com/fxnai/fxn-llm-js">Function LLM</a>, which patches your OpenAI client to generate embeddings fully locally, both in the browser and on Node.js:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/107997232c9389a7b084ac70adaa10a6/href">https://medium.com/media/107997232c9389a7b084ac70adaa10a6/href</a></iframe><p>To launch, we’ve partnered up with <a href="https://www.nomic.ai/">Nomic</a>, pioneers of the on-device AI movement, and the makers of the popular <a href="https://www.nomic.ai/gpt4all">GPT4All</a> app. Function LLM supports their state-of-the-art <a href="https://www.fxn.ai/@nomic/nomic-embed-text-v1.5-quant">Nomic Embed Text v1.5</a> embedding model, which boasts <a href="https://www.nomic.ai/blog/posts/nomic-embed-matryoshka">better performance</a> than OpenAI’s text-embedding-3-small model at similar dimensionality. Try it out:</p><p><a href="https://fxn-llm-js.vercel.app/">Function LLM Demo</a></p><h3>How Function LLM Works</h3><p><a href="https://github.com/fxnai/fxn-llm-js">Function LLM</a> is a nanolibrary (currently <a href="https://github.com/fxnai/fxn-llm-js/blob/8f5e29577cd1e6f3b74ad94576f8f836c95af512/src/index.ts#L123">123 lines of code</a>) which patches the OpenAI client to run Nomic’s embedding model on-device. It does so using our platform, <a href="https://fxn.ai/">Function</a>, which enables developers to run arbitrary AI functions on-device:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a90f1d6357b48dd963fdb0631c3451f7/href">https://medium.com/media/a90f1d6357b48dd963fdb0631c3451f7/href</a></iframe><p>Function works by compiling Python functions for most platforms (Android, browser, iOS, macOS, Linux, and Windows) and airdropping the compiled binaries to users’ devices to run locally.</p><h3>But Why Not Ollama?</h3><p>In the past few months, there’s been a crop of amazing startups building infrastructure to run large language models (LLM’s) locally. You’ve probably heard of a few of them: Ollama, LMStudio, LocalAI, and others. Looking over their tech, we can’t help but feel like there’s a sizable, underserved gap in what they offer developers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8ak2C2Jieo2zz8WYQKZTHw.png" /><figcaption>The cost of using OpenAI’s embeddings vs. generating embeddings locally with Function.</figcaption></figure><p>The goal of on-device AI isn’t to go from paying OpenAI to paying AWS for your containerized “local” AI service running in the cloud. It’s quite the opposite: the goal is to push as much of your product’s compute to your users’ devices, thereby sidestepping the need for hosted inference entirely. The benefits in cost and privacy are almost too good to be true, and Apple is perhaps the only major company leading the way on this.</p><h3>Make It Your Own</h3><p>One-click deploy the demo to your GitHub account with Vercel, and modify it to fit your needs. Also, make sure to <a href="https://github.com/fxnai/fxn-llm-js">star Function LLM on GitHub</a>!</p><p><a href="https://github.com/fxnai/fxn-llm-js/tree/main/example">fxn-llm-js/example at main · fxnai/fxn-llm-js</a></p><blockquote>We’re offering $5 in credits to the first 250 users who sign up to <a href="https://fxn.ai/">Function</a>.</blockquote><blockquote>If you already have an application in production, we’d love to work with you on migrating your app to use on-device embeddings. <a href="/service/mailto:hi@fxn.ai">Reach out to us</a>!</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=33d209ceb8e1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/fxnai/keep-your-next-viral-ai-app-free-for-longer-with-local-embeddings-33d209ceb8e1">Keep Your Next Viral AI App Free For Longer With Local Embeddings</a> was originally published in <a href="https://medium.com/fxnai">Function</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>