<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Omar Alva on Medium]]></title>
        <description><![CDATA[Stories by Omar Alva on Medium]]></description>
        <link>https://medium.com/@omargohan?source=rss-e8ef083caee4------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*wWqAohty3rjGXnF_</url>
            <title>Stories by Omar Alva on Medium</title>
            <link>https://medium.com/@omargohan?source=rss-e8ef083caee4------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 11:31:07 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@omargohan/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[A new challenger: Claude 3.5 Sonnet]]></title>
            <link>https://medium.com/@omargohan/a-new-challenger-claude-3-5-sonnet-4ccd1bdc0a01?source=rss-e8ef083caee4------2</link>
            <guid isPermaLink="false">https://medium.com/p/4ccd1bdc0a01</guid>
            <category><![CDATA[anthropic-ai]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[sota]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Omar Alva]]></dc:creator>
            <pubDate>Mon, 24 Jun 2024 21:12:51 GMT</pubDate>
            <atom:updated>2024-06-24T21:12:51.064Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0qWH-9KhqU4noGLNUJC2zw.png" /><figcaption>Image generated using AI by author.</figcaption></figure><h3><strong>Introduction</strong></h3><p>Anthropic, a leading AI research company, has recently unveiled its latest large language model (LLM), Claude 3.5 Sonnet. This release marks a significant milestone in the evolution of AI, showcasing Anthropic’s commitment to advancing AI technology while ensuring alignment with human values. Founded in 2021 by former OpenAI employees Dario and Daniela Amodei, Anthropic has quickly risen to prominence in the AI landscape. The company’s mission is to develop advanced AI systems that are safe, transparent, and ethical, capable of addressing some of the world’s most pressing challenges.</p><h3><strong>Changes in Claude 3.5 Sonnet</strong></h3><p>Claude 3.5 Sonnet introduces several groundbreaking features and improvements over its predecessors. Here are some of the key changes:</p><ul><li><strong>Speed and Efficiency</strong>: Claude 3.5 Sonnet operates at twice the speed of its predecessor, Claude 3 Opus, making it ideal for complex tasks such as context-sensitive customer support and multi-step workflows.</li><li><strong>Enhanced Visual Reasoning</strong>: The model surpasses previous versions in visual comprehension, excelling at interpreting charts, graphs, and transcribing text from imperfect images. This capability is particularly beneficial for industries like retail, logistics, and financial services.</li><li><strong>Artifacts Feature</strong>: One of the most notable additions is the Artifacts feature, which allows users to generate and interact with content such as code snippets, text documents, and website designs in a dedicated workspace alongside their conversation with Claude. This transforms Claude from a conversational AI into a collaborative work environment.</li><li><strong>Improved Language Understanding</strong>: Claude 3.5 Sonnet shows marked improvement in grasping nuance, humor, and complex instructions, producing high-quality content with a natural, relatable tone.</li><li><strong>Coding Proficiency</strong>: The model demonstrates significant advancements in coding capabilities, solving 64% of coding problems in internal evaluations, compared to 38% by Claude 3 Opus.</li></ul><h3><strong>Claude vs. OpenAI</strong></h3><p>The competition between Anthropic’s Claude 3.5 Sonnet and OpenAI’s GPT-4o highlights the rapid advancements in AI technology. Here are some key comparisons:</p><ul><li><strong>Context Window</strong>: Claude 3.5 Sonnet features a 200,000 token context window, significantly larger than GPT-4o’s 8,192 tokens, allowing it to handle much larger chunks of information at once.</li><li><strong>Cost Efficiency</strong>: Claude 3.5 Sonnet is more cost-effective, with input tokens priced at $3 per million and output tokens at $15 per million, compared to GPT-4o’s $30 and $60 per million tokens, respectively.</li><li><strong>Benchmark Performance</strong>: Claude 3.5 Sonnet outperforms GPT-4o in several benchmarks, including graduate-level reasoning, coding proficiency, and visual comprehension. However, GPT-4o still holds an edge in certain areas like math problem-solving.</li><li><strong>User Experience</strong>: Both models are available for free, with higher rate limits for paid subscribers. Claude 3.5 Sonnet’s Artifacts feature offers a unique collaborative workspace, setting it apart from GPT-4o.</li></ul><h3>Demo</h3><ol><li>Coding</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/626/1*IhmzPqjjGfFwjHmjgD9iqg.png" /><figcaption>Function in python. Image by author</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/626/1*9XWu6IuUKsMrG2GWiAWg0Q.png" /><figcaption>Code output. Image by author.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/313/1*ZF2CPw5DUPPmrIcmSsjwfw.png" /><figcaption>Testing the function. Image by author.</figcaption></figure><p>2. Reasoning</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/1*l1OBc8hWpXpglt9WjORb8g.png" /><figcaption>First riddle. Image by author.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/1*fity36Pxp5E0nayHDd_Wng.png" /><figcaption>Second riddle. Image by author.</figcaption></figure><p>3. Math</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/1*1I5cIm1Svrko08RREOsxMw.png" /><figcaption>Math problem A. Image by author.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/1*ZMn9mZwvrE6PAQqCRfQ0NQ.png" /><figcaption>Math problem B. Image by author.</figcaption></figure><h3><strong>Conclusion</strong></h3><p>The ongoing competition between AI providers like Anthropic and OpenAI is a testament to the dynamic and rapidly evolving nature of the AI industry. This healthy rivalry drives innovation, pushing each company to develop more advanced, efficient, and user-friendly models. For end users, this competition translates into better, more affordable AI solutions that can tackle a wide range of tasks with increasing sophistication.</p><p>As AI technology continues to advance, the benefits of this competition will become even more apparent, leading to new applications and capabilities that were once thought impossible. Ultimately, the end users stand to gain the most from this relentless pursuit of excellence, as AI becomes an ever more integral part of our daily lives and work environments.</p><p>By embracing the advancements brought by models like Claude 3.5 Sonnet and GPT-4o, we can look forward to a future where AI not only enhances productivity and efficiency but also aligns closely with human values and needs.</p><h3>References</h3><ul><li><a href="https://digitaleprofis.de/kuenstliche-intelligenz/ki-chatbots/claude-3-5-das-erste-ki-modell-das-gpt4o-schlaegt/">https://digitaleprofis.de/kuenstliche-intelligenz/ki-chatbots/claude-3-5-das-erste-ki-modell-das-gpt4o-schlaegt/</a></li><li><a href="https://www.oecd-ilibrary.org/sites/3acbe1cd-en/index.html?itemId=%2Fcontent%2Fcomponent%2F3acbe1cd-en">https://www.oecd-ilibrary.org/sites/3acbe1cd-en/index.html?itemId=%2Fcontent%2Fcomponent%2F3acbe1cd-en</a></li><li><a href="https://www.pubaffairsbruxelles.eu/opinion-analysis/big-tech-rivalry-could-be-the-key-to-competition-in-ai/">https://www.pubaffairsbruxelles.eu/opinion-analysis/big-tech-rivalry-could-be-the-key-to-competition-in-ai/</a></li><li><a href="https://context.ai/compare/gpt-4/claude-3-sonnet">https://context.ai/compare/gpt-4/claude-3-sonnet</a></li><li><a href="https://happyfutureai.com/the-rise-of-anthropic-and-the-birth-of-claude-a-saga-of-artificial-general-intelligence/">https://happyfutureai.com/the-rise-of-anthropic-and-the-birth-of-claude-a-saga-of-artificial-general-intelligence/</a></li><li><a href="https://apix-drive.com/en/blog/useful/anthropic-pbc-history-development-products">https://apix-drive.com/en/blog/useful/anthropic-pbc-history-development-products</a></li><li><a href="https://www.tomsguide.com/ai/anthropic-just-dropped-claude-35-sonnet-with-better-vision-and-a-sense-of-humor">https://www.tomsguide.com/ai/anthropic-just-dropped-claude-35-sonnet-with-better-vision-and-a-sense-of-humor</a></li><li><a href="https://nomusica.com/claude-3-5-sonnet-new-features-improvements-availability-future-plans/">https://nomusica.com/claude-3-5-sonnet-new-features-improvements-availability-future-plans/</a></li><li><a href="https://observer.com/2024/06/anthropic-release-claude-ai-model-gpt-comparison/">https://observer.com/2024/06/anthropic-release-claude-ai-model-gpt-comparison/</a></li><li><a href="https://www.pcmag.com/news/anthropic-our-claude-35-model-beats-openais-gpt-4o">https://www.pcmag.com/news/anthropic-our-claude-35-model-beats-openais-gpt-4o</a></li><li><a href="https://www.anthropic.com/news/claude-3-5-sonnet">https://www.anthropic.com/news/claude-3-5-sonnet</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4ccd1bdc0a01" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Open WebUI: The LLM Web UI]]></title>
            <link>https://medium.com/@omargohan/open-webui-the-llm-web-ui-66f47d530107?source=rss-e8ef083caee4------2</link>
            <guid isPermaLink="false">https://medium.com/p/66f47d530107</guid>
            <category><![CDATA[ollama]]></category>
            <category><![CDATA[web-ui]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[gen-ai-tools]]></category>
            <category><![CDATA[chatbots]]></category>
            <dc:creator><![CDATA[Omar Alva]]></dc:creator>
            <pubDate>Tue, 21 May 2024 15:45:27 GMT</pubDate>
            <atom:updated>2024-05-21T15:45:27.663Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8YQ5HbIbKrRzGx3PzXcuBw.png" /><figcaption>Futuristic Web UI — AI generated by author</figcaption></figure><h3>Introduction</h3><p>Open WebUI, formerly known as Ollama WebUI, is an extensible, feature-rich, and user-friendly self-hosted web interface designed to operate entirely offline. It supports various Large Language Model (LLM) runners, making it a versatile tool for deploying and interacting with language models.</p><p>Open WebUI provides a ChatGPT-style interface, allowing users to chat with remote servers running language models. This web UI is particularly useful for those who want to run language models locally or in a self-hosted environment, ensuring data privacy and control.</p><p><a href="https://github.com/open-webui/open-webui">GitHub - open-webui/open-webui: User-friendly AI Interface (Supports Ollama, OpenAI API, ...)</a></p><h3>Concepts</h3><h3>Extensibility and Features</h3><p>Open WebUI is built with extensibility in mind. It supports multiple LLM runners, which means it can be configured to work with different language models and frameworks. This flexibility allows users to choose the best model for their specific needs. The web UI is designed to be user-friendly, with a clean interface that makes it easy to interact with the models.</p><h3>Self-Hosted and Offline Operation</h3><p>One of the key features of Open WebUI is its ability to operate entirely offline. This is particularly important for users who are concerned about data privacy and security. By running the web UI locally, users can ensure that their data is not sent to external servers. This self-hosted approach also provides greater control over the deployment and management of language models.</p><h3>Community-Driven Development</h3><p>Open WebUI is a community-driven project, which means it benefits from contributions and feedback from a diverse group of users and developers. This collaborative approach helps ensure that the web UI continues to evolve and improve over time, incorporating new features and addressing any issues that arise.</p><h3>Usage</h3><h3>Installation and Setup</h3><p>To get started with Open WebUI, users need to install the necessary software and configure their environment. The installation process typically involves setting up Docker, as Open WebUI runs inside a Docker container. This ensures that the web UI is isolated from the host system and can be easily managed.</p><p><strong>Install Docker</strong>: Ensure that Docker is installed on your system. Docker provides a convenient way to package and run applications in isolated containers.</p><p><a href="https://www.docker.com/">Docker: Accelerated Container Application Development</a></p><p><strong>Launch Open WebUI</strong>: Use Docker commands to pull the Open WebUI image and start the container (instructions below). This will set up the web UI and make it accessible via a web browser.</p><pre># If Ollama is on your computer, use this command:<br><br>docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main<br><br># If Ollama is on a Different Server, use this command:<br># To connect to Ollama on another server, change the OLLAMA_BASE_URL to the server&#39;s URL:<br><br>docker run -d -p 3000:8080 -e OLLAMA_BASE_URL=https://example.com -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main<br><br># To run Open WebUI with Nvidia GPU support, use this command:<br><br>docker run -d -p 3000:8080 --gpus all --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:cuda</pre><p><strong>Create an Admin Account</strong>: The first user to sign up on Open WebUI will be granted administrator privileges. This account will have comprehensive control over the web UI, including the ability to manage other users and configure settings.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/570/1*549PgsZExPL8t9bLBbB6Lw.png" /><figcaption>Open WebUI Sign Up — Image by author</figcaption></figure><h3>Connecting to Language Models</h3><p>Once Open WebUI is up and running, users can connect it to various language models. This involves configuring the web UI to communicate with the servers running the models.</p><p><strong>Configure Ollama</strong>: Set up your Ollama instances that Open WebUI will connect to. This may involve specifying the server addresses.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/722/1*-v83LNkhA0nYg05jM9aANQ.png" /><figcaption>Open WebUI Settings — Image by author</figcaption></figure><p><strong>Configure OpenAI (optional)</strong>: Set the OpenAI API key. This allows Open WebUI to connect to OpenAI directly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/722/1*_z-4wJEY0lLXDxg07nyUhA.png" /><figcaption>Open WebUI Settings — Image by author</figcaption></figure><h3>Demo</h3><p>To demonstrate the capabilities of Open WebUI, let’s walk through a simple example of setting up and using the web UI to interact with a language model.</p><p><strong>Access the Web UI</strong>: Open a web browser and navigate to the address where Open WebUI is running. You will be prompted to create an admin account if this is the first time accessing the web UI.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/722/1*veiWjKhOAxALaL9gQ3HY2A.png" /><figcaption>Open WebUI Sign In — Image by author</figcaption></figure><p><strong>Start a Chat Session</strong>: Once logged in, you can start a chat session with the language model. The interface is designed to be intuitive, with a text input field for entering your queries and a chat window for displaying the model’s responses.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m3t8X6xjaiXtuvFv4ZAlWA.png" /><figcaption>Open WebUI Welcome page — Image by author</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/583/1*FxzrioWs5vUjOAI6MphZJw.png" /><figcaption>Open WebUI Select a model — Image by author</figcaption></figure><p><strong>Manage Settings</strong>: As an admin, you have access to various settings and configurations. You can add or remove model instances, adjust load balancing settings, and manage user accounts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/706/1*7Isw6YozpOARziVW-0ADcA.png" /><figcaption>Open WebUI Admin settings — Image by author</figcaption></figure><h3>Conclusion</h3><p>Open WebUI, the Ollama web UI, is a powerful and flexible tool for interacting with language models in a self-hosted environment. Its extensibility, user-friendly interface, and offline operation make it an ideal choice for users who value data privacy and control. By leveraging the capabilities of Open WebUI, users can deploy and manage language models with ease, ensuring optimal performance and a seamless user experience.</p><p>Whether you are a researcher, developer, or enthusiast, Open WebUI provides the tools you need to harness the power of language models in a secure and efficient manner. With its community-driven development and robust feature set, Open WebUI is poised to become a leading solution for self-hosted language model interfaces.</p><h3>References</h3><ul><li><a href="https://docs.openwebui.com/">https://docs.openwebui.com</a></li><li><a href="https://github.com/open-webui/open-webui">https://github.com/open-webui/open-webui</a></li><li><a href="https://dev.to/timesurgelabs/how-to-run-llama-3-locally-with-ollama-and-open-webui-297d">https://dev.to/timesurgelabs/how-to-run-llama-3-locally-with-ollama-and-open-webui-297d</a></li><li><a href="http://anakin.ai/de/blog/open-webui-ollama/">http://anakin.ai/de/blog/open-webui-ollama/</a></li><li><a href="https://docs.openwebui.com/getting-started/">https://docs.openwebui.com/getting-started/</a></li><li><a href="https://community.hetzner.com/tutorials/ai-chatbot-with-ollama-and-open-webui/de/">https://community.hetzner.com/tutorials/ai-chatbot-with-ollama-and-open-webui/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=66f47d530107" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Ollama Adds Support for Embeddings]]></title>
            <link>https://medium.com/@omargohan/ollama-adds-support-for-embeddings-d2646b9fc326?source=rss-e8ef083caee4------2</link>
            <guid isPermaLink="false">https://medium.com/p/d2646b9fc326</guid>
            <category><![CDATA[vector-database]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[ollama]]></category>
            <category><![CDATA[embedding]]></category>
            <dc:creator><![CDATA[Omar Alva]]></dc:creator>
            <pubDate>Wed, 10 Apr 2024 13:02:12 GMT</pubDate>
            <atom:updated>2024-04-10T13:02:12.925Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HLbScqT29hlYoPlOkLjPDg.png" /><figcaption>Robot reading a book (AI-generated by author)</figcaption></figure><h3>Introduction</h3><p>Ollama, a leading platform in the development of advanced machine learning models, has recently announced its support for embedding models in version <a href="https://github.com/ollama/ollama"><strong>0.1.31</strong></a>. This significant update enables the creation of retrieval augmented generation (RAG) applications that leverage text prompts alongside existing documents or other data sources. But what exactly does this entail?</p><p>Embedding models are specialized models designed to generate vector embeddings — long arrays of numbers that encapsulate the semantic meaning of a given text sequence. These vector embeddings can be stored in a database, facilitating the search for data with similar meanings.</p><p><a href="https://ollama.ai/">Ollama</a></p><p>The importance of embeddings cannot be overstated; they are crucial for developing applications that require a deep understanding of text semantics, such as recommendation systems, search engines, and more. By supporting embeddings, Ollama is enhancing the capabilities of developers and companies to build more intelligent and responsive applications.</p><h3>Use Cases</h3><p>The introduction of embedding models by Ollama opens up plenty of use cases across various industries. For instance, in the e-commerce sector, embeddings can improve product recommendation systems by understanding customer’s preferences. In the realm of customer service, chatbots and virtual assistants can leverage embeddings to provide more accurate and contextually relevant responses.</p><p>Furthermore, in the field of information retrieval, embeddings can significantly enhance the precision of search results, making it easier for users to find the information they need. The versatility of embeddings means that any application requiring a complex understanding of text can benefit from this technology.</p><h3>Models</h3><p>Ollama supports a variety of models for generating embeddings, each with its unique characteristics and advantages, some of those models are:</p><h4>mxbai-embed-large</h4><p>The <em>mxbai-embed-large</em> model is known for its ability to generalize well across several domains, tasks, and text lengths. It has been trained without overlapping the MTEB data, indicating robust performance in diverse scenarios. This model is suitable for applications that require understanding and processing large and varied datasets.</p><h4>nomic-embed-text</h4><p>The <em>nomic-embed-text</em> model is a large context length text encoder that outperforms some of OpenAI’s text-embedding models. It is particularly adept at handling both short and long context tasks, making it a versatile choice for embedding generation. This model is open source, open data, and fully reproducible, adding an extra layer of transparency and reliability for developers.</p><h4>all-minilm</h4><p>The <em>all-minilm</em> model aims to train sentence embedding models on very large sentence-level datasets using a self-supervised contrastive learning objective. It is designed to be efficient and effective for sentence-level embeddings, making it an excellent choice for tasks like clustering or semantic search on a more granular level.</p><h3>Demo</h3><p>To illustrate the practical application of Ollama’s new embedding capability, consider the following simple Python code example:</p><pre># Import the necessary modules<br>import ollama<br>import chromadb<br>import argparse<br><br># Create an argument parser<br>parser = argparse.ArgumentParser()<br><br># Add arguments<br>parser.add_argument(&quot;-p&quot;, &quot;--prompt&quot;, type=str, required=True, help=&quot;Prompt to generate response for&quot;)<br>parser.add_argument(&quot;-m&quot;, &quot;--model&quot;, type=str, default=&quot;mistral&quot;, help=&quot;LLM model to use for generation&quot;)<br>parser.add_argument(&quot;-e&quot;, &quot;--embedding_model&quot;, type=str, default=&quot;mxbai-embed-large&quot;, help=&quot;Embedding model to use&quot;)<br>args = parser.parse_args()<br><br># Create a ChromaDB client<br>client = chromadb.Client()<br><br># Create a collection named &quot;test&quot;<br>collection = client.create_collection(name=&quot;test&quot;)<br><br># Define the documents (this part is hardcoded for demostration purposes)<br>documents = [&quot;Artificial intelligence in healthcare offers significant benefits such as improved diagnostic accuracy and speed, enabling earlier and more precise detection of diseases through the analysis of medical images and patient data.&quot;,&quot;AI-driven technologies enhance personalized patient care by analyzing genetic, clinical, and lifestyle data to create tailored treatment plans, and they also streamline administrative tasks, reducing paperwork and improving operational efficiency within healthcare organizations.&quot;,&quot;In drug discovery and development, AI accelerates the process by analyzing vast datasets to identify potential drug candidates, predict drug efficacy, and interactions, thereby saving resources and speeding up development.&quot;,&quot;Additionally, AI applications in healthcare can improve access to care, particularly in remote and underserved areas, through telemedicine services and remote patient monitoring, ensuring healthcare services are more accessible regardless of location.&quot;]<br><br># Define the embedding model<br>embedding_model = args.embedding_model<br><br># Add the documents and embeddings to the collection<br>for i, d in enumerate(documents):<br> response = ollama.embeddings(model=embedding_model, prompt=d)<br> embedding = response[&quot;embedding&quot;]<br> collection.add(<br>  ids=[str(i)],<br>  embeddings=[embedding],<br>  documents=[d]<br> )<br><br># Define the prompt<br>prompt = args.prompt<br><br># Create embeddings for the prompt<br>response = ollama.embeddings(<br> prompt=prompt,<br> model=embedding_model<br>)<br><br># Query the collection for the most similar document<br>results = collection.query(<br> query_embeddings=[response[&quot;embedding&quot;]],<br> n_results=1<br>)<br><br># Get the most similar document<br>data = results[&#39;documents&#39;][0][0]<br><br># Generate a response using the model<br>output = ollama.generate(<br> model=args.model,<br> prompt=f&quot;Using this data: {data}. Respond to this prompt: {prompt}&quot;<br>)<br><br># Print the response<br>print(output[&#39;response&#39;])</pre><p>Result:</p><pre>$ python embeddings.py -p &quot;Please provide me a sentence with two benefits of AI in the healthcare sector?&quot; -e &quot;nomic-embed-text&quot;<br>Artificial intelligence in healthcare enhances diagnostic accuracy and speed by analyzing medical images and patient data to enable earlier and more precise disease detection.</pre><p>This code snippet demonstrates how to generate vector embeddings for a document using Ollama and then store these embeddings in a ChromaDB collection.</p><p><a href="https://www.trychroma.com/">Chroma</a></p><p>The process involves initializing Ollama, creating a collection in ChromaDB, generating embeddings for a document, and finally storing these embeddings. This example underscores the ease with which developers can integrate Ollama’s embedding capabilities into their applications.</p><h3>Conclusion</h3><p>The support for embedding models by Ollama marks a significant milestone in the evolution of open-source models and tools. As these technologies continue to advance, they become increasingly powerful and accessible, enabling developers and companies to build more sophisticated and intelligent applications.</p><p>The introduction of embeddings by Ollama is a testament to the ongoing innovation in the field of machine learning and artificial intelligence, promising a future where applications can understand and respond to human language with unprecedented accuracy and precision.</p><h3>References</h3><ul><li><a href="https://ollama.ai/">https://ollama.ai</a></li><li><a href="https://github.com/ollama/ollama/">https://github.com/ollama/ollama/</a></li><li><a href="https://www.trychroma.com/">https://www.trychroma.com/</a></li><li><a href="https://www.sbert.net/">https://www.sbert.net/</a></li><li><a href="https://www.mixedbread.ai/blog/mxbai-embed-large-v1">https://www.mixedbread.ai/blog/mxbai-embed-large-v1</a></li><li><a href="https://blog.nomic.ai/posts/nomic-embed-text-v1">https://blog.nomic.ai/posts/nomic-embed-text-v1</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d2646b9fc326" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using Ollama in your IDE with Continue]]></title>
            <link>https://medium.com/@omargohan/using-ollama-in-your-ide-with-continue-e8cefeeee033?source=rss-e8ef083caee4------2</link>
            <guid isPermaLink="false">https://medium.com/p/e8cefeeee033</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[continue]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[ide]]></category>
            <category><![CDATA[ollama]]></category>
            <dc:creator><![CDATA[Omar Alva]]></dc:creator>
            <pubDate>Mon, 26 Feb 2024 14:06:57 GMT</pubDate>
            <atom:updated>2024-02-26T14:06:57.971Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FrcREygdJwvRpvkEFHoKWg.png" /><figcaption>Ollama as coding assistant (AI generated by author)</figcaption></figure><h3>Introduction</h3><p>Integrated Development Environments (IDEs) are software suites that consolidate the tools required for software development, such as text editors, compilers, debuggers, and test platforms. They enhance productivity by streamlining the coding process, particularly for complex projects. The integration of Large Language Models (LLMs) into IDEs, like the Continue extension for VSCode, brings the power of AI to the fingertips of developers, offering benefits such as code generation, error checking, and context-aware suggestions.</p><p>Ollama is a tool that enables the local operation of open-source LLMs on your machine, such as Mistral, Llama 2 and more. It bundles model weights, configurations, and datasets into a unified package, optimizing setup and configuration details, including GPU usage. Ollama is designed for privacy, ease of use, and seamless integration, supporting a variety of use cases from AI-powered web applications to local note-taking tools.</p><p><a href="https://ollama.com/">Ollama · Run AI models locally and in the cloud</a></p><h3>Install VSCode or vscodium</h3><p>Visual Studio Code (VSCode) is a popular, open-source IDE developed by Microsoft, known for its powerful features like IntelliSense, debugging, and extension support. VSCodium is a community-driven, freely-licensed binary distribution of VSCode. The main difference between VSCode and Vscodium lies in telemetry; VSCodium does not track users’ activity, as it removes all telemetry and tracking from the Visual Studio Code source code.</p><p><a href="https://vscodium.com/">VSCodium</a></p><p>The extension to be installed is compatible with both VSCode and VSCodium, ensuring that developers can benefit from its features regardless of their preference for or against telemetry. Ultimately, the choice between using VSCode or VSCodium is up to you, allowing for flexibility based on personal or organizational policies regarding privacy and data collection.</p><p><a href="https://code.visualstudio.com/">Visual Studio Code - Build what comes next</a></p><h3>Install the Extension Continue</h3><p>Continue is an open-source autopilot for software development that integrates the capabilities of LLMs into VSCode and JetBrains IDEs. It allows developers to ask questions about highlighted code, get inline completions, and generate files from scratch, among other features.</p><p><a href="https://continue.dev/">Quality control for your software factory. | Continue</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/496/1*iKWYJoyIPPoUK2Thjs9PeA.png" /><figcaption>Search for the “Continue” extension (by author)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TzkuQeLdLhviuohemt92Ww.png" /><figcaption>You can disable telemetry (by author)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/56/1*oDJjkd6UFwV3UjKGxW8JkA.png" /><figcaption>Continue should be available on the sidebar (by author)</figcaption></figure><h4>Configure Continue</h4><p>To connect Continue to a local instance of Ollama, you need to:</p><ol><li>Download Ollama and run it locally. If you have questions about how to install and use Ollama, you can visit the comprehensive guide at <a href="https://medium.com/@omargohan/running-llms-locally-with-ollama-b87b087e70e6">Running LLMs Locally with Ollama</a> for more information.</li><li>Click on the Continue extension (sidebar menu)</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/499/1*TuH45vOZDIQT2vTh8iRY5g.png" /><figcaption>Continue (by author)</figcaption></figure><p>3. Open Continue Setting (bottom-right icon)<br>4. Add the Ollama configuration and save the changes. To ad mistral as an option, use the following example:</p><pre>  &quot;models&quot;: [<br>    {<br>      &quot;title&quot;: &quot;GPT-4 (Free Trial)&quot;,<br>      &quot;provider&quot;: &quot;free-trial&quot;,<br>      &quot;model&quot;: &quot;gpt-4&quot;<br>    },<br>    {<br>      &quot;title&quot;: &quot;GPT-4 Vision (Free Trial)&quot;,<br>      &quot;provider&quot;: &quot;free-trial&quot;,<br>      &quot;model&quot;: &quot;gpt-4-vision-preview&quot;<br>    },<br>    {<br>      &quot;title&quot;: &quot;Gemini Pro (Free Trial)&quot;,<br>      &quot;provider&quot;: &quot;free-trial&quot;,<br>      &quot;model&quot;: &quot;gemini-pro&quot;<br>    },<br>    {<br>      &quot;title&quot;: &quot;Codellama 70b (Free Trial)&quot;,<br>      &quot;provider&quot;: &quot;free-trial&quot;,<br>      &quot;model&quot;: &quot;codellama-70b&quot;<br>    },<br>    {<br>      &quot;title&quot;: &quot;Mistral&quot;,<br>      &quot;provider&quot;: &quot;ollama&quot;,<br>      &quot;model&quot;: &quot;mistral:latest&quot;,<br>      &quot;apiBase&quot;: &quot;http://localhost:11434/&quot;<br>    }<br>  ],</pre><p>5. Back to Continue and click on the bottom-left to select the “mistral” model from the list.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/250/1*gS_eSMGroDuglNZPFX21Hg.png" /><figcaption>Mistral is now available (by author)</figcaption></figure><h3>Usage</h3><p>To test Continue and Ollama, open the sample <strong>continue_tutorial.py</strong> file that comes with the extension. You can interact with the extension by highlighting code sections and asking questions or instructing Continue to refactor code. The extension provides a seamless experience by integrating AI-powered assistance directly into the coding workflow.</p><p>a) Validate that Continue can talk to Ollama.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/454/1*GGjYPg39frNnRUG1t_KJsA.png" /><figcaption>Click on the Continue extension (sidebar) and type “Hi” (by author)</figcaption></figure><p>b) Explore the tutorial, for example ask Continue to explain a function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/486/1*IEmkay_dEvkUxbVqHqTjiw.png" /><figcaption>Highlight the function and use Ctrl + M (by author)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/453/1*9GFnxqCeWKFBEeJHOH7-pg.png" /><figcaption>Ask Mistral to explain the function (by author)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/453/1*fQ3RZMreh9O-j3HUUJA-nQ.png" /><figcaption>Success! Continue and Ollama for the win! (by author)</figcaption></figure><h3>Conclusion</h3><p>Incorporating AI into an IDE like VSCode or vscodium with the Continue extension offers significant advantages, such as increased efficiency, error reduction, and the ability to quickly generate and refactor code. However, it’s important to consider the potential drawbacks, such as over-reliance on AI suggestions and the need for developers to remain vigilant about the quality and accuracy of the generated code. When used correctly, AI in programming can be a powerful tool that enhances the development process.</p><h3>References</h3><ol><li><a href="https://continue.dev/docs/walkthroughs/codellama">https://continue.dev/docs/walkthroughs/codellama</a></li><li><a href="https://marketplace.visualstudio.com/items?itemName=Continue.continue">https://marketplace.visualstudio.com/items?itemName=Continue.continue</a></li><li><a href="https://yellow.ai/blog/large-language-models/">https://yellow.ai/blog/large-language-models/</a></li><li><a href="https://www.techtarget.com/searchsoftwarequality/definition/integrated-development-environment">https://www.techtarget.com/searchsoftwarequality/definition/integrated-development-environment</a></li><li><a href="https://github.com/ollama/ollama/">https://github.com/ollama/ollama/</a></li><li><a href="https://www.carteakey.dev/running-your-own-copilot-in-vscode-on-a-mac/">https://www.carteakey.dev/running-your-own-copilot-in-vscode-on-a-mac/</a></li><li><a href="https://radixweb.com/blog/what-is-ide">https://radixweb.com/blog/what-is-ide</a></li><li><a href="https://www.codegrip.tech/productivity/top-10-ide-every-developer-should-know/">https://www.codegrip.tech/productivity/top-10-ide-every-developer-should-know/</a></li><li><a href="https://blog.logrocket.com/extending-vs-code-continue-leverage-chatgpt/">https://blog.logrocket.com/extending-vs-code-continue-leverage-chatgpt/</a></li><li><a href="https://www.maginative.com/article/how-to-get-started-with-code-llama/">https://www.maginative.com/article/how-to-get-started-with-code-llama/</a></li><li><a href="https://www.g2.com/articles/ide">https://www.g2.com/articles/ide</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e8cefeeee033" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Video Summarization with Retrieval Augmented Generation and Ollama]]></title>
            <link>https://medium.com/@omargohan/video-summarization-with-retrieval-augmented-generation-and-ollama-5879051d876f?source=rss-e8ef083caee4------2</link>
            <guid isPermaLink="false">https://medium.com/p/5879051d876f</guid>
            <category><![CDATA[ollama]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[rags]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[text-summarization]]></category>
            <dc:creator><![CDATA[Omar Alva]]></dc:creator>
            <pubDate>Mon, 19 Feb 2024 13:57:49 GMT</pubDate>
            <atom:updated>2024-02-19T13:57:49.886Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XQyuLt83O3Q9GSKREORDXA.png" /><figcaption>A robot searching for documents (AI generated by author)</figcaption></figure><blockquote><strong>Disclaimer</strong></blockquote><blockquote>Please note that the information provided in this article is intended for educational and research purposes only. Users should ensure that they comply with all applicable laws and regulations when using the tools and techniques discussed. The code examples and methodologies are shared to foster learning and the advancement of knowledge within the community. Users should exercise caution and discretion when applying these in practice.</blockquote><h3>Introduction</h3><p>Retrieval Augmented Generation (RAG) is a technique that enhances the accuracy and reliability of generative AI models by incorporating relevant information from external knowledge sources. It combines large language models (<strong>LLMs</strong>) with a retrieval component to provide contextually accurate responses.</p><h3>Components</h3><ul><li><strong>Retrieval Augmented Generation (RAG)</strong>: RAG is a framework that enhances language models by integrating the ability to retrieve information from external databases or knowledge sources. This approach improves the output quality by providing contextually relevant information.</li><li><strong>Vector Embeddings</strong>: Vector embeddings are high-dimensional numerical representations of text that capture semantic meanings. These embeddings enable efficient retrieval of information by measuring the similarity between vectors, facilitating the identification of relevant content from large datasets.</li><li><strong>ColBERT</strong>: ColBERT is a model within the RAG framework that utilizes vector embeddings for efficient information retrieval. It leverages these embeddings to quickly find and retrieve the most relevant information from a database, enhancing the model’s ability to generate informed responses.</li></ul><p><a href="https://github.com/stanford-futuredata/ColBERT">GitHub - stanford-futuredata/ColBERT: ColBERT: state-of-the-art neural search (SIGIR&#39;20, TACL&#39;21, NeurIPS&#39;21, NAACL&#39;22, CIKM&#39;22, ACL&#39;23, EMNLP&#39;23)</a></p><ul><li><strong>Large Language Model (LLM)</strong>: LLMs are AI algorithms that use deep learning techniques to understand and generate text. They are trained on massive datasets, enabling them to perform a variety of NLP tasks such as translation, content summarization, and sentiment analysis. LLMs are characterized by their ability to generate text on any topic they have been trained on, making them versatile tools for a wide range of applications.</li><li><strong>Ollama</strong>: Ollama is a user-friendly interface for running large language models (LLMs) locally, specifically on MacOS, Linux, and Windows. It is a valuable tool for researchers, developers, and anyone who wants to experiment with language models. Ollama supports a wide range of models, including Lama 2, Lama 2 uncensored, and the newly released Mistal 7B, among others.</li></ul><p><a href="https://github.com/ollama/ollama">GitHub - ollama/ollama: Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.</a></p><h3>Requirements</h3><p>To successfully run the Python code provided for summarizing a video using Retrieval Augmented Generation (RAG) and Ollama, there are specific requirements that must be met:</p><ul><li><strong>Python</strong>: Make sure to be running Python 3.9 or higher installed on your system. This is necessary to ensure compatibility with the libraries and syntax used in the code.</li><li><strong>Python modules</strong>: Make sure the following modules are installed ragatouille, langchain, langchain-community, youtube-transcript-api, pytube. Poetry or conda is recommended here.</li><li><strong>Ollama</strong>: Ollama must be preinstalled for the code to work. Ollama is a tool that allows to run open-source Large Language Models (LLMs) locally, which is essential for the summarization process described.</li></ul><p>If you don’t know how to install Ollama, then please refer to the guide <a href="https://medium.com/@omargohan/running-llms-locally-with-ollama-b87b087e70e6">Running LLMs Locally with Ollama</a> for detailed instructions on how to install and use Ollama.</p><p><a href="https://medium.com/@omargohan/running-llms-locally-with-ollama-b87b087e70e6">Running LLMs locally with Ollama</a></p><h3>Example code</h3><p>The following Python code demonstrates how to use RAG and Ollama to summarize the YouTube video about <a href="https://www.youtube.com/watch?v=ORMx45xqWkA">PyTorch</a>.</p><pre># Import necessary libraries and modules<br>import uuid<br>from langchain.chains import create_retrieval_chain<br>from langchain.chains.combine_documents import create_stuff_documents_chain<br>from langchain_community.chat_models import ChatOllama<br>from langchain_community.document_loaders import YoutubeLoader<br>from langchain_core.prompts import ChatPromptTemplate<br>from ragatouille import RAGPretrainedModel<br><br># Function to get transcript from a YouTube video link<br>def get_transcript(link):<br>    return YoutubeLoader.from_youtube_url(https://medium.com/%3Cbr%3E%20%20%20%20%20%20%20%20%20%20%20%20link,%20add_video_info=True).load()[0].page_content<br><br># Load RAG pretrained model<br>RAG = RAGPretrainedModel.from_pretrained(&quot;colbert-ir/colbertv2.0&quot;)<br><br># Create a chat prompt template<br>rag_prompt = ChatPromptTemplate.from_template(<br>&quot;&quot;&quot;Answer the following question based only on the provided context:<br><br>&lt;context&gt;<br>{context}<br>&lt;/context&gt;<br><br>Question: {input}&quot;&quot;&quot;<br>)<br><br># Initialize ChatOllama model<br>llm = ChatOllama(model=&quot;mistral&quot;, temperature=0.4)<br><br># YouTube video link and questions<br>link = &quot;https://www.youtube.com/watch?v=ORMx45xqWkA&quot;<br>questions = [<br>  &quot;what is the video about?&quot;, <br>  &quot;please provide a summary of the video&quot;<br>]<br><br># Generate a unique index name<br>index_name = str(uuid.uuid4())<br><br># Get transcript from the YouTube video<br>transcript = get_transcript(link)<br><br># Index the transcript using RAG model<br>RAG.index(<br>    collection=[transcript],<br>    index_name=index_name,<br>    max_document_length=180,<br>    split_documents=True,<br>)<br><br># Create retriever and document chains<br>retriever = RAG.as_langchain_retriever(k=4)<br>document_chain = create_stuff_documents_chain(llm, rag_prompt)<br>retrieval_chain = create_retrieval_chain(retriever, document_chain)<br><br># Retrieve and print answer for all questions<br>for question in questions:<br>    response = retrieval_chain.invoke({&quot;input&quot;: question})<br>    print(f&quot;Question: {question}\n&quot;)<br>    print(f&quot;Answer: {response[&#39;answer&#39;]}\n&quot;)</pre><p>Processing and embedding logs:</p><pre>[Feb 18, 22:10:07] #&gt; Creating directory .ragatouille/colbert/indexes/ef67f308-224b-42d7-8233-cf2685895117<br><br><br>[Feb 18, 22:10:08] [0]           #&gt; Encoding 4 passages..<br>[Feb 18, 22:10:09] [0]           avg_doclen_est = 176.25         len(local_sample) = 4<br>[Feb 18, 22:10:09] [0]           Creating 256 partitions.<br>[Feb 18, 22:10:09] [0]           *Estimated* 705 embeddings.<br>[Feb 18, 22:10:09] [0]           #&gt; Saving the indexing plan to .ragatouille/colbert/indexes/ef67f308-224b-42d7-8233-cf2685895117/plan.json ..<br>WARNING clustering 670 points to 256 centroids: please provide at least 9984 training points<br>Clustering 670 points in 128D to 256 clusters, redo 1 times, 20 iterations<br>  Preprocessing in 0.00 s<br>  Iteration 19 (0.02 s, search 0.02 s): objective=86.4826 imbalance=1.520 nsplit=0<br>[Feb 18, 22:10:09] Loading decompress_residuals_cpp extension (set COLBERT_LOAD_TORCH_EXTENSION_VERBOSE=True for more info)...<br>[Feb 18, 22:10:09] Loading packbits_cpp extension (set COLBERT_LOAD_TORCH_EXTENSION_VERBOSE=True for more info)...<br>[0.04, 0.034, 0.039, 0.042, 0.037, 0.052, 0.041, 0.027, 0.034, 0.033, 0.042, 0.033, 0.029, 0.034, 0.029, 0.036, 0.025, 0.037, 0.032, 0.04, 0.033, 0.031, 0.043, 0.027, 0.042, 0.036, 0.038, 0.028, 0.034, 0.035, 0.031, 0.035, 0.031, 0.036, 0.033, 0.038, 0.037, 0.031, 0.049, 0.039, 0.032, 0.03, 0.038, 0.043, 0.047, 0.03, 0.038, 0.046, 0.031, 0.026, 0.039, 0.029, 0.033, 0.034, 0.033, 0.059, 0.038, 0.055, 0.033, 0.043, 0.031, 0.043, 0.039, 0.037, 0.037, 0.036, 0.035, 0.029, 0.037, 0.037, 0.046, 0.037, 0.038, 0.054, 0.048, 0.027, 0.04, 0.036, 0.037, 0.044, 0.036, 0.031, 0.031, 0.035, 0.041, 0.043, 0.037, 0.034, 0.034, 0.03, 0.029, 0.038, 0.031, 0.037, 0.039, 0.043, 0.039, 0.036, 0.051, 0.043, 0.039, 0.045, 0.03, 0.038, 0.028, 0.038, 0.04, 0.034, 0.027, 0.042, 0.038, 0.045, 0.037, 0.03, 0.041, 0.04, 0.034, 0.031, 0.037, 0.047, 0.037, 0.038, 0.044, 0.034, 0.034, 0.051, 0.045, 0.04]<br>0it [00:00, ?it/s][Feb 18, 22:10:09] [0]                 #&gt; Encoding 4 passages..<br>1it [00:00, 31.03it/s]<br>100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00&lt;00:00, 3404.47it/s]<br>[Feb 18, 22:10:09] #&gt; Optimizing IVF to store map from centroids to list of pids..<br>[Feb 18, 22:10:09] #&gt; Building the emb2pid mapping..<br>[Feb 18, 22:10:09] len(emb2pid) = 705<br>100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 256/256 [00:00&lt;00:00, 87409.79it/s]<br>[Feb 18, 22:10:09] #&gt; Saved optimized IVF to .ragatouille/colbert/indexes/ef67f308-224b-42d7-8233-cf2685895117/ivf.pid.pt<br>Done indexing!<br>Loading searcher for index ef67f308-224b-42d7-8233-cf2685895117 for the first time... This may take a few seconds<br>[Feb 18, 22:10:11] #&gt; Loading codec...<br>[Feb 18, 22:10:11] #&gt; Loading IVF...<br>[Feb 18, 22:10:11] #&gt; Loading doclens...<br>100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00&lt;00:00, 18808.54it/s]<br>[Feb 18, 22:10:11] #&gt; Loading codes and residuals...<br>100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00&lt;00:00, 2700.78it/s]<br>Searcher loaded!<br><br>#&gt; QueryTokenizer.tensorize(batch_text[0], batch_background[0], bsize) ==<br>#&gt; Input: . what is the video about?,            True,           None<br>#&gt; Output IDs: torch.Size([32]), tensor([ 101,    1, 2054, 2003, 1996, 2678, 2055, 1029,  102,  103,  103,  103,<br>         103,  103,  103,  103,  103,  103,  103,  103,  103,  103,  103,  103,<br>         103,  103,  103,  103,  103,  103,  103,  103], device=&#39;cuda:0&#39;)<br>#&gt; Output Mask: torch.Size([32]), tensor([1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,<br>        0, 0, 0, 0, 0, 0, 0, 0], device=&#39;cuda:0&#39;)</pre><p>The final answer:</p><pre>Question: what is the video about?<br><br>Answer:  The video is about using PyTorch, an open-source deep learning <br>framework, to build a neural network from scratch. The speaker explains <br>that PyTorch is a library for programming with tensors, which are <br>multi-dimensional arrays used in deep neural networks. They also <br>mention that PyTorch supports dynamic computational graphs and high <br>performance parallel computing on GPUs. The speaker then goes on to <br>demonstrate how to define and instantiate a simple neural network <br>using PyTorch, specifically for image classification tasks. They explain <br>the role of various components such as linear layers, non-linear <br>activation functions, and fully connected layers. The video also <br>mentions some applications of PyTorch in computer vision, speech <br>recognition, and more.<br><br>Question: please provide a summary of the video<br><br>Answer:  The video describes how to build a neural network using <br>PyTorch, an open-source deep learning framework. PyTorch was derived <br>from the Lua-based torch library and is focused on usability, allowing <br>users to train machine learning models with just a few lines of <br>Python code. It supports dynamic computational graphs that can be <br>optimized at runtime. The video demonstrates building a neural <br>network for image classification by defining a new class that <br>inherits from PyTorch&#39;s neural network module and creating layers <br>using the Sequential container. Linear layers are used to transform <br>multi-dimensional input into one dimension, followed by non-linear <br>activation functions and another fully connected layer to output labels. <br>PyTorch has been used for various applications such as computer vision <br>AI, image generators, and speech recognition models. To get started, <br>users need to install PyTorch and optionally Cuda for GPU acceleration. <br>The video provides a step-by-step guide on creating tensors using <br>Python and performing computations like linear algebra.</pre><h3>Conclusion</h3><p>The Retrieval Augmented Generation (RAG) technology is a testament to the ongoing innovation in the field of artificial intelligence, particularly in the realm of natural language processing.</p><p>The Python code provided exemplifies the simplicity with which RAG, coupled with Ollama, can be used to summarize the content of a video. This ease of use opens up possibilities for researchers and developers to explore and understand the inner workings of RAG without the need for complex setups.</p><p>However, it is crucial to recognize that the code presented is not intended for production use. Its primary purpose is educational, to illustrate the process of how RAG functions and how it can be integrated with tools like Ollama for local execution. The code serves as a starting point for those interested in delving into the mechanics of RAG and should be seen as a conceptual demonstration rather than a production-ready solution.As RAG continues to evolve, it is becoming increasingly adept at handling complex tasks, including summarizing large documents, although it still faces challenges in this area.</p><p>The technology is not perfect, but its trajectory shows a clear path of improvement, with enhancements being made regularly. By understanding the foundational aspects of RAG through such examples, users can better appreciate its potential and contribute to its development, pushing the boundaries of what’s possible with AI-driven text generation.</p><h3>References</h3><ul><li><a href="https://python.langchain.com/docs/integrations/retrievers/ragatouille">https://python.langchain.com/docs/integrations/retrievers/ragatouille</a></li><li><a href="https://blogs.nvidia.com/blog/what-is-retrieval-augmented-generation/">https://blogs.nvidia.com/blog/what-is-retrieval-augmented-generation/</a></li><li><a href="https://en.wikipedia.org/wiki/Large_language_model">https://en.wikipedia.org/wiki/Large_language_model</a></li><li><a href="https://aws.amazon.com/what-is/retrieval-augmented-generation/">https://aws.amazon.com/what-is/retrieval-augmented-generation/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5879051d876f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Running LLMs locally with Ollama]]></title>
            <link>https://medium.com/@omargohan/running-llms-locally-with-ollama-b87b087e70e6?source=rss-e8ef083caee4------2</link>
            <guid isPermaLink="false">https://medium.com/p/b87b087e70e6</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[tools]]></category>
            <dc:creator><![CDATA[Omar Alva]]></dc:creator>
            <pubDate>Sat, 10 Feb 2024 15:09:06 GMT</pubDate>
            <atom:updated>2024-02-10T15:09:06.040Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7k7pJ4gDYyH5oErC00bjhg.png" /><figcaption>Cyber-llamas (AI generated by author)</figcaption></figure><h3>Introduction</h3><h4>AI</h4><p>Artificial Intelligence (AI) is a branch of computer science dedicated to creating systems capable of performing tasks that typically require human intelligence. These tasks include learning, reasoning, problem-solving, perception, and understanding language. The evolution of AI has been a journey of ambition, creativity, and innovation, deeply rooted in human history and philosophy. The concept of AI dates back to ancient civilizations, where philosophers like Aristotle and Plato explored the nature of human cognition and reasoning, laying the groundwork for symbolic AI, which represents human thought and reasoning using symbols.</p><h4>LLMs</h4><p>Large Language Models (LLMs) are a subset of AI focusing on understanding and generating human language. They are built on deep learning algorithms, particularly transformer models, and are trained on vast datasets, including texts from the web, books, and Wikipedia. This training allows LLMs to learn statistical relationships between words and phrases, enabling them to perform a wide range of language-related tasks such as translation, summarization, question-answering, and content creation.</p><h3>Overview of Ollama</h3><p>Ollama is a user-friendly interface for running large language models (LLMs) locally, specifically on MacOS and Linux, with Windows support on the horizon. It is a valuable tool for researchers, developers, and anyone who wants to experiment with language models. Ollama supports a wide range of models, including Lama 2, Lama 2 uncensored, and the newly released Mistal 7B, among others.</p><p><a href="https://github.com/ollama/ollama/">GitHub - ollama/ollama: Get up and running with Llama 2, Mistral, and other large language models locally.</a></p><h3>Features</h3><ul><li><strong>Ease of Use</strong>: Ollama is easy to install and use, even for users with no prior experience with language models. It provides a simple API for creating, running, and managing models.</li><li><strong>Versatility and Model Installation</strong>: Ollama supports a wide range of models, making it versatile for various applications. It also provides a straightforward installation process, making it appealing to individuals and small teams.</li><li><strong>GPU Acceleration</strong>: Ollama leverages GPU acceleration, which can speed up model inference by up to 2x compared to CPU-only setups. This feature is particularly beneficial for tasks that require heavy computation. This feature is included out of the box and it requires zero intervention.</li><li><strong>Integration Capabilities</strong>: Ollama is compatible with several platforms like Langchain, llama-index, and more.</li><li><strong>Privacy and Cost</strong>: Running LLMs locally with Ollama ensures data privacy as your data is not sent to a third party. It also eliminates inference fees, which is important for token-intensive applications.</li></ul><h3>Installing ollama</h3><h4>Mac</h4><p>To install Ollama on a Mac, you need to have macOS 11 Big Sur or later. The installation process can be done in a few steps:</p><ul><li><strong>Download Ollama</strong>: You can download Ollama for macOS from the official <a href="https://ollama.com/download">website</a>.</li><li><strong>Install with Homebrew</strong>: If you prefer using Homebrew, you can install Ollama with the following command:</li></ul><pre>brew install ollama</pre><h4>Linux</h4><p>For Linux users, the installation process is straightforward:</p><ul><li><strong>Manual Installation</strong>: You can manually download the Ollama binary from the GitHub repository and follow the instructions provided there, which may include adding Ollama as a startup service and installing CUDA drivers if you’re using Nvidia GPUs.</li><li><strong>Post-Installation Configuration</strong>: After installation, you may need to modify Ollama to start automatically. This involves creating a configuration file and setting the appropriate environment variables.</li><li><strong>Install with a Single Command</strong>: Alternatively, you can install Ollama using a single command by running in your terminal:</li></ul><pre>curl https://ollama.ai/install.sh | sh</pre><p>Before installing, it’s important to check if there’s a specific package for the Linux distribution you’re running, such as Ubuntu, Fedora, or Arch. This can ensure compatibility and smooth operation of Ollama on your system and automatic updates.</p><h3>Basic Operations with Ollama</h3><h4>Find a model</h4><p>To find a model in Ollama, you can visit the <a href="https://ollama.ai/library">Ollama library page</a>. This page lists all the available models that you can pull and run locally using Ollama. The models are listed by their capabilities, and each model’s page provides detailed information about the model, including its size, capabilities, and in most cases, a description how to use it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/818/1*NOxdIekKu9HPWdzvqgpD7Q.png" /><figcaption>Mistral library’s page (<a href="https://ollama.com/library/mistral">https://ollama.com/library/mistral</a>)</figcaption></figure><h4>Pull a Model</h4><p>To pull a model using Ollama, you can use the <em>pull</em> command followed by the model name. For instance, to pull the latest version of the Mistral model, you would use the following command:</p><pre>ollama pull mistral:latest</pre><p>This command will download the specified model (in this case, the latest version of Mistral) to your local machine.</p><h4>List Models</h4><p>To view the models you have pulled to your local machine, you can use the <em>list</em> command:</p><pre>ollama list</pre><p>This command will display a list of all models that you have downloaded locally. It’s important to note that the <em>ollama list</em> command only shows models that you have already pulled to your machine, not all available models on the Ollama platform.</p><h4>Run a Model</h4><p>Once you’ve pulled a model, you can run it using the <em>run</em> command. For example, to run the Mistral model you just pulled, you would use:</p><pre>ollama run mistral:latest</pre><p>This command will start the model, and you can then interact with it through the Ollama CLI.</p><p><strong>Example</strong>:</p><pre>ollama run mistral:latest &quot;What is the distance from Los Angeles <br>to San Francisco by car?&quot;<br>The driving distance between Los Angeles and San Francisco is <br>approximately 383 miles (616 kilometers) if you take the most<br>direct route via Interstate 5 North. The actual travel time<br>depends on traffic conditions, but it usually takes around <br>6 hours without stops. Please keep in mind that these estimates<br>are approximate, and for accurate information, consider checking <br>a reliable GPS or mapping tool.</pre><h4>Remove a Model</h4><p>If you want to remove a model from your local machine, you can use the <em>rm</em> command followed by the model name. For instance, to delete the Mistral model, you would use:</p><pre>ollama rm mistral:latest</pre><p>This command will remove the specified model (in this case, the latest version of Mistral) from your local machine.</p><h3>Conclusion</h3><p>In conclusion, Ollama is an open-source platform that significantly simplifies the process of running Large Language Models (LLMs) locally, particularly on Linux and macOS systems. It stands out as a tool that democratizes access to advanced AI technologies, enabling researchers, developers, and enthusiasts to leverage the power of LLMs without relying on cloud services.</p><p>Ollama’s contribution to the AI field is significant, as it empowers users to maintain control over their AI models and data. It is one of the best Open Source AI tools for running LLMs locally, offering a blend of privacy, cost savings, and user-friendliness. As AI continues to evolve, tools like Ollama will play a crucial role in shaping how individuals and organizations interact with and benefit from AI technologies.</p><h3>References</h3><ul><li><a href="https://ollama.ai/">https://ollama.ai</a></li><li><a href="https://github.com/ollama/ollama/">https://github.com/ollama/ollama/</a></li><li><a href="https://cheatsheet.md/llm-leaderboard/ollama">https://cheatsheet.md/llm-leaderboard/ollama</a></li><li><a href="https://python.langchain.com/docs/integrations/llms/ollama">https://python.langchain.com/docs/integrations/llms/ollama</a></li><li><a href="https://python.langchain.com/docs/integrations/chat/ollama">https://python.langchain.com/docs/integrations/chat/ollama</a></li><li><a href="https://openpedia.io/ollama-ai/">https://openpedia.io/ollama-ai/</a></li><li><a href="https://youtube.com/watch?v=XGlK4bBAzcg">https://youtube.com/watch?v=XGlK4bBAzcg</a></li><li><a href="https://replicate.com/guides/language-models/">https://replicate.com/guides/language-models/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b87b087e70e6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Comparing Kestra, Rundeck and Jenkins]]></title>
            <link>https://medium.com/@omargohan/comparing-kestra-rundeck-and-jenkins-5f4982816fba?source=rss-e8ef083caee4------2</link>
            <guid isPermaLink="false">https://medium.com/p/5f4982816fba</guid>
            <category><![CDATA[orchestration]]></category>
            <category><![CDATA[event-driven]]></category>
            <category><![CDATA[automation]]></category>
            <category><![CDATA[tools]]></category>
            <category><![CDATA[linux]]></category>
            <dc:creator><![CDATA[Omar Alva]]></dc:creator>
            <pubDate>Sat, 03 Feb 2024 00:23:36 GMT</pubDate>
            <atom:updated>2024-02-03T00:23:36.286Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*myM1_A60ihvMrKpDVUf-8A.png" /><figcaption>Automation (by author)</figcaption></figure><h3><strong>Introduction</strong></h3><p>In this article, we will compare three orchestration tools: Jenkins, Rundeck, and Kestra. These tools are used to automate tasks and processes in software development and IT operations. We will look at their use cases, features, and where each tool excels or falls short. This comparison aims to provide general information to newcomers and help them understand each tool’s capabilities.</p><h3><strong>Jenkins</strong></h3><p>Jenkins is an open-source automation server that provides hundreds of plugins to support building, deploying, and automating any project. It can be used as a simple CI server or turned into a continuous delivery hub for any project. Jenkins is Java-based and offers a vast array of plugins, making it highly extensible for integrating with CI/CD tools, source control, build tools, and automation testing frameworks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g52LZZg1zJD7uMt_GjxgSA.png" /><figcaption>Source: <a href="https://riptutorial.com/jenkins">https://riptutorial.com/jenkins</a></figcaption></figure><p>Jenkins was created to automate tasks related to building, testing, and deploying software. It is widely used for continuous integration and continuous delivery (CI/CD) pipelines, which automate the process of delivering software to production. Jenkins supports a variety of plugins that extend its functionality and integrate with other software development tools. These plugins can be used to add new user interface elements, help with administration, and enhance Jenkins for build and source code management.</p><p>However, Jenkins also has its limitations. It can be complicated to set up and manage, especially for complex pipelines. It can also be time-consuming to run. Furthermore, while the extensive plugin ecosystem is a strength, it can also be a weakness as there are thousands of different plugins, and not all of them are actively supported.</p><p><a href="https://github.com/jenkinsci/jenkins">GitHub - jenkinsci/jenkins: Jenkins automation server</a></p><h3>Rundeck</h3><p>Rundeck is an open-source automation tool that provides self-service access to operational processes and tools. It is part of the PagerDuty platform and is used by both small-scale and large-scale organizations to deploy and manage automation.</p><p>Rundeck is often used for server automation tasks, such as applying the latest patches. It is flexible, which can be both an advantage and a disadvantage. It has over 120+ plugins available, which is less than Jenkins, but still offers a wide range of functionality. However, its API is not RESTful, which can be a limitation for some use cases. Another limitation is that you cannot “version” a job, so an API call is just the latest available job, and you cannot run a previous job.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TWUQSMSv0oKy8C9vbjjbhg.png" /><figcaption>Source: <a href="https://low-orbit.net/rundeck">https://low-orbit.net/rundeck</a></figcaption></figure><p>Rundeck’s user interface can be configured intuitively, and it provides fine-grained role-based access control policy to safely grant or deny access to job nodes or activity logs. However, the remote command function provided by Rundeck, while convenient, can cause unpredictable situations if arbitrary commands that are not templated are used.</p><p><a href="https://github.com/rundeck/rundeck">GitHub - rundeck/rundeck: Enable Self-Service Operations: Give specific users access to your existing tools, services, and scripts</a></p><h3>Kestra</h3><p>Kestra is an open-source orchestrator designed to bring Infrastructure as Code (IaC) best practices to business processes, data pipelines, and simple automation. Built with an API-first philosophy, Kestra enables users to define and manage data pipelines through a simple YAML configuration file.</p><p>Kestra excels with its flexible and extensible plugin ecosystem. Its REST API allows seamless interaction with third-party systems. It also offers a comprehensive developer experience with built-in documentation and a code as well as a live-updating topology view. Kestra’s user-friendly interface aims to invite not just engineers but also business stakeholders.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*zD4fkDolOuVolfIw" /><figcaption>Source: <a href="https://kestra.io/blogs/2023-06-26-end-to-end-data-orchestration">https://kestra.io/blogs/2023-06-26-end-to-end-data-orchestration</a></figcaption></figure><p>However, adding scripts into a YAML configuration file makes the scripts less readable, especially if they get long. Inline scripts also lack the syntax highlighting and testability that you would get when leveraging more complex use cases.</p><p><a href="https://github.com/kestra-io/kestra">GitHub - kestra-io/kestra: Event Driven Orchestration &amp; Scheduling Platform for Mission Critical Applications</a></p><h3>Comparative</h3><p>When comparing Jenkins, Rundeck, and Kestra, it’s important to consider the specific needs of your project or organization. Jenkins is a powerhouse for CI/CD with a strong focus on integration and testing automation. It excels in environments where continuous integration and delivery are critical. However, its complexity can be a barrier for those without technical expertise.</p><p>Rundeck, on the other hand, is tailored more towards operational tasks and runbook automation. It is particularly useful for incident management and reducing repetitive operational work. While it can handle job scheduling and automation, it may not be as robust for CI/CD purposes as Jenkins.</p><p>Kestra is designed with data workflows in mind, making it an excellent choice for organizations that need to orchestrate complex data processes. It offers real-time monitoring and is built to scale, but it might not be the first choice for general IT automation outside of data-centric tasks.</p><h4>Integration and collaboration</h4><p>Jenkins is renowned for its robust continuous integration and delivery capabilities, automating the build, test, and deployment phases of the software development lifecycle. Rundeck specializes in operational tasks and runbook automation, focusing on incident management and routine operations.</p><p>Kestra, a newer player in the field, is designed for data orchestration, enabling complex workflows across various services and data sources. It aims to be a versatile platform that can handle not just data orchestration but also microservice and infrastructure orchestration. Kestra’s approach brings the infrastructure-as-code model to data orchestration, which can be particularly beneficial for companies with complex data ecosystems spread across multiple storage locations.</p><h4>Scalability and agent management</h4><p>Jenkins and Rundeck differ in their scalability approach, with Jenkins using agents and Rundeck operating agentlessly. Kestra, built upon Apache Kafka and ElasticSearch, offers scalability through its Kafka architecture, where every worker in the Kestra cluster is implemented as a Kafka consumer. This allows Kestra to handle large datasets and complex workflows efficiently, making it a strong contender for data-heavy environments.</p><h4>Plugin ecosystem and community support</h4><p>Jenkins has a vast plugin ecosystem, while Rundeck offers a more modest selection. Kestra, though newer, has a growing library of integrations with official plugins for major cloud providers, data warehouses, and data tools like dbt and Airbyte. Kestra’s user interface also facilitates the creation of scheduled and event-driven workflows, which can be appealing to business users who need to create SQL queries and build internal tools.</p><h4>Data orchestration and workflow management</h4><p>Kestra stands out for its data orchestration capabilities, with a user-friendly interface that simplifies the creation and management of complex data flows. It is API-driven, allowing for automation and integration with existing tools and platforms. Kestra’s design is data-first, which can democratize the use of the tool within an organization, bridging gaps in data processing and orchestration.</p><h3>Conclusion</h3><p>In summary, Jenkins is best suited for organizations with a strong emphasis on continuous integration and delivery. Rundeck is ideal for those looking to automate operational tasks and manage incidents effectively. Kestra is the go-to for data workflow orchestration at scale. Each tool has its strengths and weaknesses, and the best choice will depend on your specific use cases and requirements.</p><h3>Github repositories</h3><ul><li><a href="https://github.com/kestra-io/kestra">Kestra</a> 5.3k ⭐</li><li><a href="https://github.com/rundeck/rundeck">Rundeck</a> 5.2k ⭐</li><li><a href="https://github.com/jenkinsci/jenkins">Jenkins</a> 22k ⭐</li></ul><h3>References</h3><ul><li><a href="https://www.spiceworks.com/tech/devops/articles/what-is-jenkins/">https://www.spiceworks.com/tech/devops/articles/what-is-jenkins</a></li><li><a href="https://www.simplilearn.com/tutorials/jenkins-tutorial/what-is-jenkins">https://www.simplilearn.com/tutorials/jenkins-tutorial/what-is-jenkins</a></li><li><a href="https://www.rundeck.com/">https://www.rundeck.com</a></li><li><a href="https://kestra.io/">https://kestra.io</a></li><li><a href="https://www.jenkins.io/">https://www.jenkins.io</a></li><li><a href="https://hackr.io/blog/what-is-jenkins">https://hackr.io/blog/what-is-jenkins</a></li><li><a href="https://www.techtarget.com/searchsoftwarequality/definition/Jenkins">https://www.techtarget.com/searchsoftwarequality/definition/Jenkins</a></li><li><a href="https://www.restack.io/docs/kestra-introduction">https://www.restack.io/docs/kestra-introduction</a></li><li><a href="https://codefresh.io/learn/jenkins/">https://codefresh.io/learn/jenkins/</a></li><li><a href="https://www.g2.com/products/jenkins/features">https://www.g2.com/products/jenkins/features</a></li><li><a href="https://www.devopsschool.com/blog/rundeck-community-vs-rundeck-enterprise-vs-rundeck-cloud/">https://www.devopsschool.com/blog/rundeck-community-vs-rundeck-enterprise-vs-rundeck-cloud</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5f4982816fba" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>