<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Vince Lam on Medium]]></title>
        <description><![CDATA[Stories by Vince Lam on Medium]]></description>
        <link>https://medium.com/@vince-lam?source=rss-ce24b5d6cbb------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*FdY_N_dCL4n7tdWO_FJrng.png</url>
            <title>Stories by Vince Lam on Medium</title>
            <link>https://medium.com/@vince-lam?source=rss-ce24b5d6cbb------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 14:25:12 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@vince-lam/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Why I Switched My Blog Framework from Jekyll to Astro]]></title>
            <link>https://medium.com/@vince-lam/why-i-switched-my-blog-framework-from-jekyll-to-astro-101c39d28745?source=rss-ce24b5d6cbb------2</link>
            <guid isPermaLink="false">https://medium.com/p/101c39d28745</guid>
            <category><![CDATA[blog]]></category>
            <category><![CDATA[front-end-development]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Vince Lam]]></dc:creator>
            <pubDate>Wed, 16 Oct 2024 00:00:49 GMT</pubDate>
            <atom:updated>2024-10-19T12:17:21.122Z</atom:updated>
            <content:encoded><![CDATA[<h4>I recently gave my website a new look by switching from Jekyll to Astro as my static site generator.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ydupt3-REOWyc3Jx9X748A.png" /><figcaption>Jekyll vs Astro (<a href="https://www.kooslooijesteijn.net/blog/jekyll-or-astro-which-is-better-static-site-generator">source</a>)</figcaption></figure><p><a href="https://github.com/jekyll/jekyll">Jekyll</a>, a site generator in Ruby and the engine behind <a href="https://pages.github.com/">GitHub pages</a>, has served me well these past 14 months and has been perfectly sufficient to my needs. However, I got that developers itch and fancied a change.</p><p>Several factors prompted my switch:</p><ul><li>Improved reader experience</li><li>Curiosity about new tech</li><li>Desire to learn a new frontend framework</li><li>Website modernisation</li><li>Enhanced SEO capabilities</li></ul><p>I still highly recommend using Jekyll + GitHub pages for those looking to start a personal site. This <a href="https://youtu.be/g6AJ9qPPoyc">video</a> is a great tutorial to get started that I wish I had, would’ve saved a few headaches.</p><h3>Exploring Static Site Generator Options</h3><p>There are many static site generators for developers to choose from. I researched the most popular frameworks and found these contenders:</p><ul><li><a href="https://github.com/vercel/next.js">NextJS</a> (126k ⭐) — a React-based framework for building server-side rendered and static websites with excellent performance and developer experience.</li><li><a href="https://github.com/gohugoio/hugo">Hugo</a> (75k ⭐) — a fast static site generator with a focus on speed and simplicity, using Go templates.</li><li><a href="https://github.com/gatsbyjs/gatsby">Gatsby</a> (55k ⭐) — a React-based static site generator optimised for fast web apps, offering seamless integration with GraphQL and extensive plugin ecosystem.</li><li><a href="https://github.com/withastro/astro">Astro</a> (46k ⭐) — a modern static site builder that delivers lightweight, minimal JavaScript pages with a focus on content-first and component-based development.</li><li><a href="https://github.com/11ty/eleventy">Eleventy</a> (11k ⭐) — a flexible static site generator that supports multiple template languages and emphasises simplicity and speed.</li></ul><p>Each have their own strengths, but Astro resonated with me the most as it struck the right balance between being feature-rich, flexible and simple.</p><h3>Why Astro Emerged as the Top Choice</h3><p>After careful consideration, Astro stood out as the clear winner, despite being a relatively new player in the field. Launched in August 2022, Astro has quickly gained traction in the developer community, and for good reason. Its innovative approach to building websites addresses many of the pain points developers face with traditional frameworks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wC4nDyvf5jbb6FJT6EEafg.png" /><figcaption>Popular JavaScript frameworks</figcaption></figure><p>What sets Astro apart is not just its feature set, but its philosophy of providing powerful capabilities while maintaining simplicity and performance. This balance of power and efficiency made it an attractive choice for my blog redesign.</p><p>Here’s why Astro became the top choice:</p><ul><li><strong>Performance-First Architecture:</strong> (i) Zero JavaScript by default (ii) Static HTML output for fast load times (iii) Partial hydration for interactive components</li><li><strong>Flexible Framework Support:</strong> (i) Seamless integration with React, Vue, Svelte, and more (ii) Use multiple frameworks in a single project (iii) Framework-agnostic approach</li><li><strong>Developer Experience:</strong> (i) Intuitive component-based structure (ii) Built-in Markdown support (iii) Hot module replacement for rapid development</li><li><strong>SEO Optimisation:</strong> (i) Server-side rendering for improved search engine visibility (ii) Automatic sitemap generation (iii) Easy metadata management</li><li><strong>Growing Ecosystem:</strong> (i) Active and supportive community (ii) Expanding library of integrations and themes (iii) Regular updates and improvements (Astro 5.0 is now in beta)</li><li><strong>Content-Focused: </strong>(i) Markdown-centric workflow (ii) Content Collections for organised content management</li><li><strong>Build Performance: </strong>(i) Fast build times, even for large sites (ii) Efficient caching mechanisms (iii) Incremental builds for quick updates</li><li><strong>Scalability:</strong> (i) Suitable for small personal blogs to large corporate sites (ii) Easy deployment to various hosting platforms (iii) Built-in image optimisation</li></ul><h3>Site Improvements Post-Switch</h3><p>With Astro’s flexible framework and powerful features, I was able to implement a range of enhancements that significantly improved both the aesthetics and functionality of the site. These improvements not only modernised the look and feel of the blog but also made it more accessible and user-friendly for my readers.</p><p>The move to Astro enabled several enhancements (although many of these features are also possible in Jekyll with workarounds):</p><ul><li>Light and dark mode toggle</li><li>Site-wide search functionality</li><li>Reading progress indicator</li><li>Modern page transitions</li><li>“Scroll to top” button</li><li>Improved mobile responsiveness</li><li>Better code syntax highlighting</li></ul><p>You can see my old site <a href="https://vince-lam.github.io/">here</a> and compare the changes yourself.</p><h3>Future Plans</h3><p>One of the most exciting aspects of switching to Astro is the potential it unlocks for future development. As I’ve become more familiar with the framework, I’ve started to think of new features and blog content that were previously not possible.</p><p>The switch to Astro opens up new possibilities:</p><ul><li>Implementation of dynamic content areas</li><li>Creation of interactive data visualisations</li><li>Improved accessibility features</li></ul><p><em>Originally published at </em><a href="https://vinlam.com/posts/why-switch-from-jekyll-to-astro/"><em>https://vinlam.com</em></a><em> on October 16, 2024.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=101c39d28745" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[5 Networking Tips I’ve Learnt to Build My Professional Network in a New Country with Zero…]]></title>
            <link>https://long.sweet.pub/5-networking-tips-ive-learnt-to-build-my-professional-network-in-a-new-country-with-zero-9c73e4b74e5d?source=rss-ce24b5d6cbb------2</link>
            <guid isPermaLink="false">https://medium.com/p/9c73e4b74e5d</guid>
            <category><![CDATA[networking]]></category>
            <category><![CDATA[business]]></category>
            <category><![CDATA[careers]]></category>
            <category><![CDATA[relationships]]></category>
            <category><![CDATA[career-advice]]></category>
            <dc:creator><![CDATA[Vince Lam]]></dc:creator>
            <pubDate>Thu, 13 Jun 2024 11:10:33 GMT</pubDate>
            <atom:updated>2024-06-13T20:15:03.086Z</atom:updated>
            <content:encoded><![CDATA[<h3>5 Networking Tips I’ve Learnt to Build My Professional Network in a New Country with Zero Connections</h3><p>When I relocated to Singapore last year I had zero connections and had to build my professional network from scratch.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vluc2b9h6kwENHxxj7GeuA.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@productschool?utm_content=creditCopyText&amp;utm_medium=referral&amp;utm_source=unsplash">Product School</a> on <a href="https://unsplash.com/photos/crowd-of-people-in-building-lobby-nOvIa_x_tfo?utm_content=creditCopyText&amp;utm_medium=referral&amp;utm_source=unsplash">Unsplash</a></figcaption></figure><p>Networkings offers many benefits. It can change your career trajectory and open doors that might otherwise be closed. It’s also a great way to exchange ideas, stay updated on industry trends, and build connections with peers and mentors.</p><p>Here are five tips I’ve learnt from attending 50+ events to make networking more enjoyable:</p><h4>1. Focus on Building Genuine Connections</h4><p>Instead of targeting people based on their job titles, industry, or seniority, aim to make one genuine connection (and hopefully friend). This approach reduces pressure and makes networking feel less like work. Broadening your network naturally follows when you prioritise building authentic relationships.</p><h4>2. Prepare Conversation Starters</h4><p>Having pre-prepared questions can ease initial conversations. Some of my favourites include:</p><blockquote>- What brought you to this event? What interests you about it?</blockquote><blockquote>- How do you know the speaker or organisers?</blockquote><blockquote>- What are you most excited about currently, both at work and outside of work?</blockquote><h4>3. Remember: Everyone’s in the Same Situation</h4><p>If you find yourself standing alone and feeling awkward, remember that others are likely feeling the same way. Most people appreciate a friendly introduction. A simple smile and a “Hi, I’m [Vince]” can kickstart a conversation, which will likely flow naturally from there.</p><p>Don’t overthink it. Remember that everyone is too busy worrying about themselves to focus on you.</p><h4>4. Apply Classic Networking Advice</h4><p>Principles from <em>“How to Win Friends and Influence People”</em> still apply:</p><ul><li>Listen actively</li><li>Show genuine interest in others</li><li>Smile often</li><li>Remember people’s names</li></ul><h4>5. View Networking as a Skill</h4><p>I approach each event as an opportunity to practice, reminding myself that networking is a skill that can be improved with reps.</p><h4>Bonus Tip: Take Notes</h4><p>After each event, I keep notes on the people I met and conversations we had. This helps me strengthen existing relationships and make future interactions more meaningful.</p><figure><a href="https://medium.com/p/23995df2a3e3"><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*wzZp-tXvY6bR07t0PsqJHA.png" /></a></figure><p><strong><em>Liked the story? </em></strong><a href="https://medium.com/p/23995df2a3e3"><strong><em>Join Sweet Publications!</em></strong></a></p><p>Sweet.pub is a family of three — 💚 <a href="https://medium.com/p/6c1553fcd541"><strong>Short</strong></a>, 💙 <a href="https://medium.com/p/23995df2a3e3"><strong>Long</strong></a>, and 🧡 <a href="https://medium.com/p/11993686e2a4"><strong>Deep</strong></a>. <br>Discover the stories that will make your 🤍 beat!</p><p><em>This article was published on June 13th, 2024 in </em><a href="https://medium.com/p/23995df2a3e3"><em>Long. Sweet. Valuable.</em></a><em> publication.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*xNjg-YJL_tqFx9Kfe7sAlQ.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9c73e4b74e5d" width="1" height="1" alt=""><hr><p><a href="https://long.sweet.pub/5-networking-tips-ive-learnt-to-build-my-professional-network-in-a-new-country-with-zero-9c73e4b74e5d">5 Networking Tips I’ve Learnt to Build My Professional Network in a New Country with Zero…</a> was originally published in <a href="https://long.sweet.pub/">Long. Sweet. Valuable.</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Traditional Natural Language Processing (NLP) Tasks: Understanding What Large Language Models…]]></title>
            <link>https://medium.com/thedeephub/traditional-natural-language-processing-nlp-tasks-understanding-what-large-language-models-e8899161f202?source=rss-ce24b5d6cbb------2</link>
            <guid isPermaLink="false">https://medium.com/p/e8899161f202</guid>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[nlp]]></category>
            <dc:creator><![CDATA[Vince Lam]]></dc:creator>
            <pubDate>Wed, 12 Jun 2024 08:10:59 GMT</pubDate>
            <atom:updated>2024-07-10T06:26:27.914Z</atom:updated>
            <content:encoded><![CDATA[<h3>Traditional Natural Language Processing (NLP) Tasks: Understanding What Large Language Models (LLMs) Can Do For You</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pHpf_aITeICAx1pq5vFJyg.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@joshua_hoehne?utm_content=creditCopyText&amp;utm_medium=referral&amp;utm_source=unsplash">Joshua Hoehne</a> on <a href="https://unsplash.com/photos/text-YPgTovTiUv4?utm_content=creditCopyText&amp;utm_medium=referral&amp;utm_source=unsplash">Unsplash</a></figcaption></figure><p>Since Transformers were introduced in 2017, Natural Language Processing (NLP) capabilities have skyrocketed. Knowing traditional NLP tasks is useful to understand the capabilities of (Large) Language Models. Although LLMs may outperform in many areas, smaller fine-tuned models will always have a place due to their faster latency and lower costs.</p><p>Here is an overview of NLP task categories:</p><h3>Natural Language Understanding (NLU)</h3><ul><li><strong>Text Classification:</strong> Sorting text into predefined groups — can be <strong>binary</strong>, <strong>multiclass</strong>, or <strong>multilabel</strong> classification. Subtasks include <strong>sentiment analysis</strong>, spam detection, and emotion detection.</li><li><strong>Topic Modelling: </strong>Discovering main themes in a collection of documents.</li><li><strong>Named Entity Recognition (NER):</strong> Identifying and categorising entities like names, places, and dates.</li><li><strong>Part-of-Speech Tagging (POS): </strong>Assigning parts of speech to each word in a sentence.</li><li><strong>Information Retrieval (IR):</strong> Searching large datasets to find relevant information.</li><li><strong>Semantic Textual Similarity (STS):</strong> Comparing similarity between words, text spans, and documents.</li></ul><h3>Natural Language Generation (NLG):</h3><p>These are types of <strong>sequence-to-sequence (Seq2seq)</strong> task, where inputs and outputs are of varying lengths.</p><ul><li><strong>Machine Translation:</strong> Translating text from one language to another.</li><li><strong>Text Style Transfer (TST):</strong> Translating one style of text to another, for example modern English to Shakespearean English.</li><li><strong>Text-to-Text Generation:</strong> Generating relevant and coherent text, includes <strong>Next Sentence Prediction (NSP)</strong>.</li><li><strong>Question Answering:</strong> Generating answers to user queries, given a context. This involves using multiple NLP techniques, such as <strong>Natural Language Understanding (NLU)</strong> and <strong>Intent Classification</strong>.</li><li><strong>Abstractive Summarisation:</strong> Paraphrasing content to create concise summaries. This approach is more common and performs better than extractive summarisation.</li></ul><p>Understanding NLP tasks and their applications, allows us leverage them to solve real-world problems and create innovative solutions.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e8899161f202" width="1" height="1" alt=""><hr><p><a href="https://medium.com/thedeephub/traditional-natural-language-processing-nlp-tasks-understanding-what-large-language-models-e8899161f202">Traditional Natural Language Processing (NLP) Tasks: Understanding What Large Language Models…</a> was originally published in <a href="https://medium.com/thedeephub">The Deep Hub</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How AI Assistants Learn: The Training Process of Large Language Models]]></title>
            <link>https://medium.com/thedeephub/how-ai-assistants-learn-the-training-process-of-large-language-models-8c512e6d165f?source=rss-ce24b5d6cbb------2</link>
            <guid isPermaLink="false">https://medium.com/p/8c512e6d165f</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <dc:creator><![CDATA[Vince Lam]]></dc:creator>
            <pubDate>Wed, 12 Jun 2024 08:08:31 GMT</pubDate>
            <atom:updated>2024-06-14T13:39:42.060Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wPHPf7BgFoWvUPYwDB6Pwg.jpeg" /><figcaption>Image generated by Stable Diffusion XL</figcaption></figure><p>Understanding the training process of AI assistants reveals how they evolve from simple language models to sophisticated, useful tools. Here’s a breakdown of the stages involved.</p><h4>Step 1: Pre-training on Massive Text Data</h4><p>The initial stage is about compressing knowledge to obtain a base model. Terabytes of unlabelled internet data and months of GPU compute are required to create a base model that fulfils a simple objective: to predict a missing or next word in a sentence.</p><p>This simple yet powerful objective forces the neural network to internalise a wealth of knowledge about the world. The model undergoes self-supervised learning to identify patterns in how words relate to each other, essentially compressing this vast knowledge into gigabytes of parameters.</p><p>However, these base models have limitations. Although they can predict the next word, the output is not usable. It is simply a probability distribution learned from internet data.</p><h4>Step 2: Instruction Fine-tuning for an Assistant Model</h4><p>To transform the base model into a useful assistant, it undergoes further training on high-quality, human-generated question-answering datasets instead of broad internet documents. This fine-tuning stage aligns the model’s capabilities with the desired task of answering questions, enabling it to provide more relevant and coherent responses.</p><p>While the model can learn new information during fine-tuning, this learning is less comprehensive compared to full training. The model’s knowledge is primarily based on data from the initial pre-training stage, known as the knowledge cut-off date.</p><h4>Step 3: Enhancing Performance with Feedback (Optional)</h4><p>Usually there is a final stage involving reinforcement learning. The model generates multiple responses, which are judged by humans or more advanced models. This feedback is then used to fine-tune the model further, improving its performance and accuracy.</p><p>Through these stages, AI assistants become more than just language predictors. They evolve into powerful tools capable of understanding and responding to human queries effectively.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8c512e6d165f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/thedeephub/how-ai-assistants-learn-the-training-process-of-large-language-models-8c512e6d165f">How AI Assistants Learn: The Training Process of Large Language Models</a> was originally published in <a href="https://medium.com/thedeephub">The Deep Hub</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[8 Simple Tips To Improve Your LLM Prompts for Better Responses]]></title>
            <link>https://medium.com/thedeephub/prompting-tips-for-better-llm-outputs-bc17f9f3138a?source=rss-ce24b5d6cbb------2</link>
            <guid isPermaLink="false">https://medium.com/p/bc17f9f3138a</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[prompt-engineering]]></category>
            <dc:creator><![CDATA[Vince Lam]]></dc:creator>
            <pubDate>Wed, 12 Jun 2024 08:03:49 GMT</pubDate>
            <atom:updated>2024-06-14T13:39:23.731Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wI7rxWzr0cs3QO7QKgm-HQ.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@sebastien_bonneval?utm_content=creditCopyText&amp;utm_medium=referral&amp;utm_source=unsplash">Sebastien Bonneval</a> on <a href="https://unsplash.com/photos/man-in-gray-shirt-facing-sticky-notes-UIpFY1Umamw?utm_content=creditCopyText&amp;utm_medium=referral&amp;utm_source=unsplash">Unsplash</a></figcaption></figure><p>Large Language Models (LLMs) are complex probabilistic machines that predict the next word. Prompt engineering is about conditioning them for desired outputs. Each instruction steers the model’s generation. Here are tips to prompting LLMs to generate useful outputs to your problems.</p><ol><li><strong>Start with roles.</strong> Who is your AI pretending to be? A writing coach? A data analyst? By assigning a persona and responsibilities, you steer content, tone, and style.</li><li><strong>Set clear goals.</strong> What do you want to achieve? Be specific on the LLM’s task. The more descriptive and detailed the prompt, the better the results. Avoid imprecise descriptions.</li><li><strong>Use positive language.</strong> LLMs prefer “do this” over “don’t do that.” It’s more specific, easier to grasp. So, frame your requests positively.</li><li><strong>Break tasks down.</strong> Treat your AI like a human intern. Complex task? Break it into simpler prompts. They’re easier to understand and complete.</li><li><strong>Structure with delimiters.</strong> Use tags like &lt;example&gt; {your example} &lt;/example&gt; or ###Instruction### to separate parts. This helps the LLM parse your input effectively.</li><li><strong>Specify output format.</strong> Want Markdown? Lists? JSON? Ask for it. The more detail you provide about context, length, format, and style, the better. This prevents misinterpretation and ensures usable output.</li><li><strong>Try few-shot prompting.</strong> This technique enables in-context learning where specific format examples are provided in the prompt to steer the LLM to better performance.</li><li><strong>Use Chain-of-Thought (CoT).</strong> Give your AI “space to think” with phrases like “think step by step” or “provide your thought process to explain how you reasoned this response”. This intermediate reasoning yields better results for complex tasks.</li></ol><p>For the power users out there:</p><ul><li><strong>Use the System Prompt Directly. </strong>Use the system prompt to change settings directly instead of the user prompt for more control over the model’s behaviour.</li><li><strong>Adjust Temperature and Top P for more (or less) creativity. </strong>Lower values yield more deterministic outputs, while higher values increase variability.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bc17f9f3138a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/thedeephub/prompting-tips-for-better-llm-outputs-bc17f9f3138a">8 Simple Tips To Improve Your LLM Prompts for Better Responses</a> was originally published in <a href="https://medium.com/thedeephub">The Deep Hub</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Writing Online Landed Me Freelance Work Despite a Tough Job Market]]></title>
            <link>https://short.sweet.pub/how-writing-online-landed-me-freelance-work-despite-a-tough-job-market-9bfcd3d0d00a?source=rss-ce24b5d6cbb------2</link>
            <guid isPermaLink="false">https://medium.com/p/9bfcd3d0d00a</guid>
            <category><![CDATA[career-advice]]></category>
            <category><![CDATA[writing]]></category>
            <category><![CDATA[freelance]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[careers]]></category>
            <dc:creator><![CDATA[Vince Lam]]></dc:creator>
            <pubDate>Thu, 06 Jun 2024 23:28:12 GMT</pubDate>
            <atom:updated>2024-06-06T23:28:45.956Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BDc4-_JrDmOskrpdXqG3fQ.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@peyankov?utm_content=creditCopyText&amp;utm_medium=referral&amp;utm_source=unsplash">Y. Peyankov</a> on <a href="https://unsplash.com/photos/red-flower-on-green-grass-field-during-daytime-CMlkM-Sd6Lc?utm_content=creditCopyText&amp;utm_medium=referral&amp;utm_source=unsplash">Unsplash</a></figcaption></figure><p>In today’s tough job market, especially in the competitive data science field, writing online and building a personal profile helped me stand out and find freelance work.</p><p>Since moving to Singapore, I’ve shared my AI journey and learnings on LinkedIn. While researching local large language model (LLM) inferencing, I noticed a gap — there was a lack of cohesive resources. So, I compiled and shared this information, which resonated on Medium, LinkedIn, and Reddit. This piece brought freelance opportunities to me.</p><p>Becoming a thought leader in your industry has significant benefits. Instead of you finding opportunities, opportunities can come to you.</p><h3><strong>Reason #1: No one’s going to find you if you don’t make yourself findable</strong></h3><p>Visibility is key. If you don’t put yourself out there, potential clients won’t know you exist. Writing online, whether through blog posts, articles, or social media, increases your online presence. From 0 to 1. You open yourself to infinite more opportunities.</p><p>The more you put out, the more likely someone will discover your content. Each piece is an opportunity flywheel and advertisement that works for you 24/7, provided it’s valuable and high quality.</p><h3><strong>Reason #2: Writing Clarifies Your Thinking</strong></h3><p>Writing about my field forces me to research thoroughly and stay updated with the latest trends. This process deepens my understanding and keeps my knowledge current. Writing helps me articulate what I know and identify gaps. This reflection makes me a more competent and confident professional.</p><h3><strong>Reason #3: Build Trust and Credibility</strong></h3><p>Consistent content showcases my expertise. Regular posts about my field demonstrate my knowledge and keep me top of mind for potential clients. Consistency builds credibility and shows I am serious about my craft. Sharing my expertise positions me as a go-to resource. Being transparent about my process builds trust and credibility.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9bfcd3d0d00a" width="1" height="1" alt=""><hr><p><a href="https://short.sweet.pub/how-writing-online-landed-me-freelance-work-despite-a-tough-job-market-9bfcd3d0d00a">How Writing Online Landed Me Freelance Work Despite a Tough Job Market</a> was originally published in <a href="https://short.sweet.pub/">Short. Sweet. Valuable.</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why I Moved to Singapore Jobless, Leaving Behind My Free Data Science Degree]]></title>
            <link>https://long.sweet.pub/why-i-moved-to-singapore-jobless-leaving-behind-my-free-data-science-degree-6548ee2a1469?source=rss-ce24b5d6cbb------2</link>
            <guid isPermaLink="false">https://medium.com/p/6548ee2a1469</guid>
            <category><![CDATA[culture]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[travel]]></category>
            <category><![CDATA[careers]]></category>
            <category><![CDATA[career-change]]></category>
            <dc:creator><![CDATA[Vince Lam]]></dc:creator>
            <pubDate>Tue, 28 May 2024 22:15:13 GMT</pubDate>
            <atom:updated>2024-05-28T22:15:13.839Z</atom:updated>
            <content:encoded><![CDATA[<h4>Last year, my partner and I made the life-changing decision to uproot our comfortable lives in London and move to Singapore. Here’s why.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*yAwziv4KWo8VH2_b" /></figure><p>Last year, my partner and I decided to move across the world from London to Singapore. Her job offered her the opportunity to transfer offices. We’d never been there before and had no connections.</p><p>It was a daunting choice, filled with uncertainty. It meant leaving behind our family and friends, quitting my data scientist role for the UK Government, and dropping out of a sponsored bachelor’s degree. I had already completed two out of three and a half years.</p><p>Several reasons drove my decision to go.</p><h3>Fulfilling a Lifelong Dream</h3><p>We had always wanted to live abroad, meet people from all walks of life, experience new cultures, and broaden our horizons.</p><p>Singapore was one of the countries on our list. It’s a melting pot of cultures, different from what we’re used to, hot and sunny, English-speaking, safe, convenient, with good job opportunities, and great for traveling.</p><p>While there was an initial culture shock, we have since settled in and are enjoying the life we’ve created here.</p><h3>Embracing Discomfort and Growth</h3><p>I was content and comfortable in my previous job and degree. But I knew moving countries would force me to grow and develop.</p><p>I knew it would be difficult to find work as a foreigner in Singapore, especially without connections. It would force me to get out of my comfort zone, become comfortable with networking and cold messaging, enhance my data science and AI engineering skills, find ways to stand out in a saturated market and deal with the stress of job searching.</p><p>During this career break, I’ve undergone significant personal growth. I’m more mindful of how I spend my time and have built daily habits for journaling, coding, and reading. I’ve built generative AI applications, established an online presence, had freelancing opportunities, and am exploring entrepreneurship.</p><h3>Deciding with The Regret Minimization Framework</h3><p>To make this pivotal decision, I turned to Jeff Bezos’ Regret Minimization Framework, a tool I had previously used when transitioning from mechanical engineering to data science. The framework prompts you to envision your future self and consider which option would minimize regret.</p><p>I asked myself, “What would I regret more? Staying in London and wondering ‘what if?’ Or moving to Singapore with uncertainty, but trying to make life work out here, even if it doesn’t? At least we can say we tried and have amazing memories to look back on.”</p><p>I chose the second option, and I’m glad I did. The experiences and personal growth I’ve gained have been invaluable, and the memories we’ve created are priceless.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6548ee2a1469" width="1" height="1" alt=""><hr><p><a href="https://long.sweet.pub/why-i-moved-to-singapore-jobless-leaving-behind-my-free-data-science-degree-6548ee2a1469">Why I Moved to Singapore Jobless, Leaving Behind My Free Data Science Degree</a> was originally published in <a href="https://long.sweet.pub/">Long. Sweet. Valuable.</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How I Stay Up-To-Date On The Latest AI Advancements Without Feeling Overwhelmed]]></title>
            <link>https://generativeai.pub/how-i-stay-up-to-date-on-ai-without-feeling-overwhelmed-ff07ea9e41dc?source=rss-ce24b5d6cbb------2</link>
            <guid isPermaLink="false">https://medium.com/p/ff07ea9e41dc</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[learning]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <dc:creator><![CDATA[Vince Lam]]></dc:creator>
            <pubDate>Fri, 24 May 2024 15:39:14 GMT</pubDate>
            <atom:updated>2024-05-24T15:39:14.660Z</atom:updated>
            <content:encoded><![CDATA[<h4>Keeping up with the latest breakthroughs and industry trends can feel like an endless pursuit, leading to information overload.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*VCliZlINfS8KNzPa" /><figcaption>Source: Image by <a href="https://unsplash.com/@darktez">Kamil Sudoł</a> on <a href="https://unsplash.com/photos/a-very-large-cloud-in-the-sky-over-a-field-5CwFM0_U4Hc">Unsplash</a></figcaption></figure><p>In the rapidly evolving field of AI, staying adaptable and committed to lifelong learning is crucial for long-term career success. Embracing a mindset of continuous growth and adaptation is key to staying ahead.</p><p>With so much information available, it’s easy to feel overwhelmed. Here’s how I stay updated without getting bogged down.</p><h3>Start with a Broad Overview and Curate Sources</h3><p>When diving into AI, I begin with a broad overview. This approach allows me to identify key areas of interest, provide context, and make connections across different domains. It allows me to make informed decisions on where to focus my efforts.</p><p>To avoid information overload, I curated a mix of resources to allow me to navigate the AI landscape efficiently, separating the signal from the noise.</p><h3>Newsletters: Curated Overview of the AI Landscape</h3><p>Newsletters are an efficient way to stay informed. They provide curated summaries of the most exciting and impactful AI developments, saving you countless hours of sifting through papers, social media, and forums.</p><h4>Succinct Overviews: Alpha Signal</h4><p><a href="https://alphasignal.ai/">Alpha Signal</a> offers concise 5-minute daily summaries of the latest state-of-the-art projects, notebooks, and GitHub repositories. This allows you to learn and build with the latest LLM techniques quickly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UsYjWsNADA2wQflemdfKhA.png" /><figcaption>Alpha Signal Landing Page</figcaption></figure><p>I appreciate its concise and matter-of-fact approach, providing a clear overview of the most exciting ideas and trends capturing the community’s interest.</p><p>You can read their latest newsletter <a href="https://flask-bootstrap-mbs11.replit.app/">here</a>.</p><h4>Comprehensive Coverage: AI News</h4><p>AI News by <a href="https://twitter.com/swyx">Sywx</a> and <a href="https://www.latent.space/">Latent Space</a> is an excellent choice for those who regularly use and build with AI.</p><p>It provides comprehensive daily updates using web scraping and AI to summarize content from 800 sources, quantifying the time saved.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MF72fJ3wFF2GRDt6WAZ5sg.png" /><figcaption>AI News Time Saving</figcaption></figure><p>Although the AI-powered summaries may occasionally contain inaccuracies, the direct links allow you to dive deeper into conversations that catch your interest.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*v3uOqr93E-0BDkUicsgohA.png" /><figcaption>Table of Contents from a typical AI News Newsletter</figcaption></figure><p>You can read through the archives <a href="https://buttondown.email/ainews/archive/">here</a> to try before subscribing.</p><h4>Other Newsletters Worth Trying</h4><p>I’m selective with my newsletters to avoid inbox and mental clutter. Here are other newsletters that may be a better fit for you:</p><ul><li><a href="https://tldr.tech/ai">TLDR AI</a>: For a more informal daily digest</li><li><a href="https://www.deeplearning.ai/the-batch/">The Batch</a> by Andrew Ng and DeepLearning.ai</li><li><a href="https://www.linkedin.com/newsletters/the-atlas-7075090904906149891/">The Atlas</a> on LinkedIn</li><li><a href="https://nlp.elvissaravia.com/">🥇Top ML Papers of The Week</a> by Elvis and DAIR</li></ul><p>For newsletters from individual AI practitioners, I’ll cover those in a future post. Typically, these newsletters and posts are longer and dive deeper into topics.</p><h3>X (Twitter): The Pulse of Cutting-Edge AI</h3><p>To hear news fresh off the press, I turn to X. This platform remains the go-to place for AI researchers and builders to share their unfiltered and honest thoughts and opinions on papers and the industry direction.</p><p>Although AI News covers X, I still schedule time to check these accounts and other platforms. This way, I can read the source the tweet is referring to. And often, the comment section is where the true insight lies.</p><p>I follow several AI founders, researchers, and open-source builders: <a href="https://twitter.com/ylecun">Yann LeCun</a>, <a href="https://twitter.com/karpathy">Andrej Karpathy</a>, E<a href="https://twitter.com/omarsar0">lvis</a>, <a href="https://twitter.com/drjimfan">Jim Fan</a>, <a href="https://twitter.com/_akhaliq">AK</a>, <a href="https://twitter.com/GregKamradt">Greg Kamradt</a>, <a href="https://twitter.com/JeffDean">Jeff Dean</a>, <a href="https://twitter.com/gdb">Greg Brockman</a>, <a href="https://twitter.com/demishassabis">Demis Hassabis</a>, <a href="https://twitter.com/ClementDelangue">Clement Delangue</a>, <a href="https://twitter.com/jerryjliu0">Jerry Liu</a>, <a href="https://twitter.com/hwchase17">Harrison Chase</a>, <a href="https://twitter.com/chipro">Chip Huyen</a>, <a href="https://twitter.com/eugeneyan">Eugene Yan</a>, <a href="https://twitter.com/swyx">Swyx</a>, <a href="https://twitter.com/simonw">Simon Willison</a>, <a href="https://twitter.com/_philschmid">Philipp Schmid</a>, and <a href="https://twitter.com/rasbt">Sebastian Raschka</a>.</p><h3>Meetups and Events: High Interactivity</h3><p>Attending meetups and events, both in-person and virtual, is a great and engaging way to learn from experienced professionals and gain insights into real-world applications of AI.</p><p>These gatherings provide opportunities for Q&amp;A sessions, one-on-one conversations, and idea exchanges with other AI practitioners.</p><p>I recommend using <a href="https://www.meetup.com/">meetup.com</a> and <a href="https://lu.ma/discover">lu.ma</a> to find meetups and events in your area.</p><h3>Online Communities: Crowd-Sourced AI Insights</h3><p>Online communities provide a dynamic platform for engaging with peers and experts in the field.</p><p>I engage with various online communities to expose myself to more diverse ideas. There are plenty of crowd-sourced knowledge and discussions on these platforms:</p><ul><li><strong>LinkedIn:</strong> Many people on X will cross-post to LinkedIn, so you can follow them there if that’s your preferred platform</li><li><strong>Medium: </strong>There are many great Medium publications, including this one</li><li><strong>Reddit:</strong> <a href="https://www.reddit.com/r/localLLaMA/">r/localLlama</a></li><li><strong>Discord</strong></li><li><strong>HackerNews</strong></li><li><strong>Slack</strong></li><li><strong>WhatsApp / Telegram groups</strong></li></ul><p>Participating in discussions and sharing insights with peers can foster learning, networking, and collaboration.</p><h3>Conclusion</h3><p>Embracing a mindset of continuous learning and adaptation is crucial for long-term success in the rapidly evolving field of AI.</p><p>By curating resources, setting aside dedicated time, and engaging with the AI community, you can stay up-to-date on the latest advancements without feeling overwhelmed.</p><p>Thanks for reading. If you end up trying these resources, let me know what you think! Or share your favorite AI resources.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*lh7EVYCJ8RLEAlwb.png" /></figure><p>This story is published on <a href="https://generativeai.pub/">Generative AI</a>. Connect with us on <a href="https://www.linkedin.com/company/generative-ai-publication">LinkedIn</a> and follow <a href="https://www.zeniteq.com/">Zeniteq</a> to stay in the loop with the latest AI stories.</p><p>Subscribe to our <a href="https://www.generativeaipub.com/">newsletter</a> to stay updated with the latest news and updates on generative AI. Let’s shape the future of AI together!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*gBPWHAylx1SlQJ_V.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ff07ea9e41dc" width="1" height="1" alt=""><hr><p><a href="https://generativeai.pub/how-i-stay-up-to-date-on-ai-without-feeling-overwhelmed-ff07ea9e41dc">How I Stay Up-To-Date On The Latest AI Advancements Without Feeling Overwhelmed</a> was originally published in <a href="https://generativeai.pub/">Generative AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Raycast Transformed My MacBook Workflow]]></title>
            <link>https://medium.com/thedeephub/how-raycast-transformed-my-macbook-workflow-005d736f8f8b?source=rss-ce24b5d6cbb------2</link>
            <guid isPermaLink="false">https://medium.com/p/005d736f8f8b</guid>
            <dc:creator><![CDATA[Vince Lam]]></dc:creator>
            <pubDate>Tue, 23 Apr 2024 01:29:40 GMT</pubDate>
            <atom:updated>2024-04-24T22:20:26.332Z</atom:updated>
            <content:encoded><![CDATA[<p>As knowledge workers, the tools we use and how we integrate them into our workflow can greatly impact our productivity, minimise context switching and distractions, and improve the overall work experience.</p><p>When I switched from a Windows machine to a MacBook M2 Pro in February 2023, I discovered Raycast, a free macOS launcher that has become an indispensable part of my daily routine. Raycast has transformed the way I interact with my MacBook, making it more enjoyable and efficient, and has quickly become my favourite Mac app.</p><p>In this blog post, I’ll share my thoughts on why Raycast is a game-changer for productivity and I’ll highlight my most frequent use cases and demonstrate how you can significantly improve your workflow.</p><h3>What is Raycast?</h3><p>Raycast is a free, fast, and powerful macOS launcher, think Spotlight (default launcher) or Alfred but better. With just two keystrokes, Option + Space, you can access Raycast and its productivity-boosting commands and extensions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C5m2zg-JsiotY69t-Gihsw.gif" /><figcaption>Example Raycast usage <a href="https://manual.raycast.com/the-basics">(Source)</a></figcaption></figure><h3>Why Raycast Stands Out?</h3><p>Raycast is actively developed by a dedicated startup that engages with its community. The Raycast YouTube channel has great tips and inspiration for creative ways to use Raycast.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fw84WYPPY3ck%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dw84WYPPY3ck&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fw84WYPPY3ck%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/18516f402055828c5346b9c2c911d7a1/href">https://medium.com/media/18516f402055828c5346b9c2c911d7a1/href</a></iframe><p>The best reason to use Raycast though is its thriving ecosystem of developers and the extensions that they build. This translates to a constantly evolving tool that adapts to your needs. I’ll share the extensions I most commonly use below.</p><h3>Native Raycast Commands That Streamline My Workflow</h3><p>Many inbuilt Raycast commands replace third-party apps I previously installed.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6017c51b89f0dfc53bdfcff5437d4ba5/href">https://medium.com/media/6017c51b89f0dfc53bdfcff5437d4ba5/href</a></iframe><h3>Raycast Extensions: Customising Your Workflow to Fit Your Needs</h3><p>Those features alone are are enough to make me use Raycast. However, what truly sets Raycast apart is their Extension store, where anyone can develop an extension that can be approved.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Nrud0t7katIzdLlH.jpeg" /><figcaption>Raycast Extension Store</figcaption></figure><p>Below are some of the extensions I use and how. Most extensions are activated simply by opening Raycast and typing the first few characters or initials. I’ve starred ⭐ high-usage shortcuts. Raycast learns your most frequently used commands and ranks them accordingly, ensuring that your favourite actions are always at your fingertips.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f74e094e94597952a825d68868d8a41d/href">https://medium.com/media/f74e094e94597952a825d68868d8a41d/href</a></iframe><h3>Conclusion</h3><p>Raycast has become an essential tool in my toolkit and I can’t imagine using my MacBook without it. By leveraging its native commands and powerful extensions, I save countless hours every month. If you’re looking to boost your productivity and enjoy a more streamlined workflow, give Raycast a try and download it <a href="https://www.raycast.com/">here</a> or via Homebrew brew install --cask raycast.</p><p>Trust me, you won’t look back. Hope you’ve found this useful!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=005d736f8f8b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/thedeephub/how-raycast-transformed-my-macbook-workflow-005d736f8f8b">How Raycast Transformed My MacBook Workflow</a> was originally published in <a href="https://medium.com/thedeephub">The Deep Hub</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[50+ Open-Source Options for Running LLMs Locally]]></title>
            <link>https://medium.com/thedeephub/50-open-source-options-for-running-llms-locally-db1ec6f5a54f?source=rss-ce24b5d6cbb------2</link>
            <guid isPermaLink="false">https://medium.com/p/db1ec6f5a54f</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <dc:creator><![CDATA[Vince Lam]]></dc:creator>
            <pubDate>Tue, 12 Mar 2024 10:48:57 GMT</pubDate>
            <atom:updated>2024-05-16T10:07:58.119Z</atom:updated>
            <content:encoded><![CDATA[<p>In my <a href="https://medium.com/thedeephub/why-i-use-locally-hosted-llms-9146e1fd55fa">previous post</a>, I discussed the benefits of using locally hosted open weights LLMs, like data privacy and cost savings. By using mostly free models and occasionally switching to GPT-4, my monthly expenses dropped from $20 to $0.50. Setting up a port-forward to your local LLM server is a free solution for mobile access.</p><p>There are many open-source tools for hosting open weights LLMs locally for inference, from the command line (CLI) tools to full GUI desktop applications. Here, I’ll outline some popular options and provide my own recommendations. I have split this post into the following sections:</p><ol><li>All-in-one desktop solutions for accessibility</li><li>LLM inference via the CLI and backend API servers</li><li>Front-end UIs for connecting to LLM backends</li></ol><p>Each section includes a table of relevant open-source LLM GitHub repos to gauge popularity and activity. Click <a href="https://docs.google.com/spreadsheets/d/1Xv38p90V3GiJXjq0a3qc24056Vicn1I5MG6QiFE6nVE/edit?usp=sharing">here</a> for the <strong>full table</strong> and <a href="https://github.com/vince-lam/awesome-local-llms">here</a> for the associated <strong>GitHub repo</strong>.</p><p>These projects can overlap in scope and may split into different components of inference backend server and UI. This field evolves quickly, so details may soon be outdated.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SmFROaOLlZDXxARO99QGyA.png" /><figcaption>Google Sheets of open-source local LLM repositories, available <a href="https://docs.google.com/spreadsheets/d/1Xv38p90V3GiJXjq0a3qc24056Vicn1I5MG6QiFE6nVE/edit#gid=0">here</a></figcaption></figure><h3>#1. Desktop Solutions</h3><p>All-in-one desktop solutions offer ease of use and minimal setup for executing LLM inferences, highlighting the accessibility of AI technologies. Simply download and launch a .exe or .dmg file to get started. Ideal for less technical users seeking a ready-to-use ChatGPT alternative, these tools provide a solid foundation for anyone looking to explore AI before advancing to more sophisticated, technical alternatives.</p><h4>Popular Choice: GPT4All</h4><p><strong>GPT4All</strong> is an all-in-one application mirroring ChatGPT’s interface and quickly runs local LLMs for common tasks and RAG. The provided models work out of the box and the experience is focused on the end user.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*_tX6k4bDmILRt5Dc.gif" /><figcaption>GPT4All UI realtime demo on M1 MacOS Device <a href="https://github.com/nomic-ai/gpt4all">(Source)</a></figcaption></figure><h4>Open-Source Alternatives to LM Studio: Jan</h4><p><strong>LM Studio</strong> is often praised by YouTubers and bloggers for its straightforward setup and user-friendly interface. It offers features like model card viewing, model downloading, and system compatibility checks, making it accessible for beginners in model selection.</p><p>Despite its advantages, I hesitate to recommend LM Studio due to its proprietary nature, which may limit its use in business settings because of licensing constraints. Additionally, the inevitable monetization of the product is a concern. I favour open-source solutions when available.</p><p><a href="https://github.com/janhq/jan"><strong>Jan</strong></a> is an open-source alternative to LM Studio with a clean and elegant UI. The developers actively engage with their community (<a href="https://twitter.com/janframework">X</a> and <a href="https://discord.com/invite/FTk2MvZwJH">Discord</a>), maintain good <a href="https://jan.ai/guides">documentation</a>, and are transparent with their work — for example, their <a href="https://github.com/orgs/janhq/projects/5/views/16">roadmap</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/972/0*m12it2ztRjQDuOVd.gif" /><figcaption>Jan UI realtime demo: Jan v0.4.3-nightly on a Mac M1, 16GB Sonoma 14 <a href="https://github.com/janhq/jan">(Source)</a></figcaption></figure><p>With a recent <a href="https://github.com/janhq/jan/releases/tag/v0.4.8">update</a>, you can easily download models from the Jan UI. You can also use any model available from HuggingFace or <a href="https://jan.ai/guides/using-models/import-manually/">upload your custom models</a>.</p><p>Jan also has a minimal LLM inference server (3 MB) built on top of <strong>llama.cpp</strong> called <a href="https://github.com/janhq/nitro/"><strong>nitro</strong></a>, and it powers their desktop application.</p><h4>Feature-Rich: h2oGPT</h4><p><strong>H2O.ai</strong> is an AI company that has greatly contributed to the open-source community with its AutoML products and now their Generative AI products. <a href="https://github.com/h2oai/h2ogpt"><strong>h2oGPT</strong></a> offers extensive features and customisation, ideal for NVIDIA GPU owners:</p><ul><li><a href="https://github.com/h2oai/h2ogpt/blob/main/docs/README_LangChain.md#supported-datatypes">Many file formats supported for offline RAG</a></li><li>Evaluation of model performance using reward models</li><li>Agents for Search, Document Q/A, Python code, CSVs</li><li>Robust testing with 1000s of unit and integration tests</li></ul><p>You can explore a demo at <a href="https://gpt.h2o.ai/"><strong>gpt.h2o.ai</strong></a><strong> </strong>to experience the interface before installing it on your system. If the UI meets your needs and you’re interested in more features, a basic version of the app is available for download, offering limited document querying capabilities. For installation, refer to the <a href="https://github.com/h2oai/h2ogpt/blob/main/README.md#get-started">setup instructions</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*1FtNGnjzVoHnMo_Y.png" /><figcaption>h2oGPT available at <a href="https://gpt.h2o.ai/">https://gpt.h2o.ai/</a></figcaption></figure><h4>Other Desktop Solutions</h4><p>Table of Top 5 most popular FOSS (Free Open-Source Software) LLM desktop solutions:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JJL4R1h7ODy00Px_ntfpfQ.png" /><figcaption>Top 5 open-source LLM desktop apps, full table available <a href="https://docs.google.com/spreadsheets/d/1Xv38p90V3GiJXjq0a3qc24056Vicn1I5MG6QiFE6nVE/edit#gid=0">here</a></figcaption></figure><h3>#2. LLM inference via the CLI and backend API servers</h3><p>CLI tools enable local inference servers with remote APIs, integrating with front-end UIs (listed in Section 3) for a custom experience. They often offer OpenAI API-compatible endpoints for easy model swapping with minimal code changes.</p><p>Although chatbots are the most common use case, you can also use these tools to power Agents, using frameworks such as <a href="https://github.com/joaomdmoura/crewAI">CrewAI</a> and Microsoft’s <a href="https://microsoft.github.io/autogen/">AutoGen</a>.</p><h4>High Optimisation: llama.cpp</h4><p><a href="https://github.com/ggerganov/llama.cpp"><strong>llama.cpp</strong></a><strong> </strong>offers minimal setup for LLM inference across devices. The project is a C++ port of Llama2 and supports <a href="https://github.com/ggerganov/ggml/blob/master/docs/gguf.md">GGUF format</a> models, including multimodal ones, such as <a href="https://github.com/haotian-liu/LLaVA">LLava</a>. Its efficiency suits consumer hardware and edge devices.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*pq52Sb2opX70EIsf.png" /><figcaption>llama.cpp <a href="https://github.com/ggerganov/llama.cpp">(Source)</a></figcaption></figure><p>There are many bindings based on llama.cpp, like <a href="https://github.com/abetlen/llama-cpp-python">llama-cpp-python</a> (more are listed in the README’s description). Hence, many tools and UIs are built on llama.cpp and provide a more user-friendly interface.</p><p>To get started, follow the instructions <a href="https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage">here</a>. You will need to <a href="https://huggingface.co/docs/hub/en/models-downloading">download models</a> in GGUF format from HuggingFace.</p><p>llama.cpp has its own HTTP server implementation, just type ./server to start.</p><pre># Unix-based example<br>./server -m models/7B/ggml-model.gguf -c 2048</pre><p>This means you can easily connect it with other web chat UIs listed in section 2.</p><p>This option is best for those who are comfortable with the command line interface (CLI) and prefer writing custom scripts and seeing outputs in the terminal.</p><h4>Intuitive CLI Option: Ollama</h4><p><a href="https://github.com/ollama/ollama"><strong>Ollama</strong></a> is another LLM inference command-line tool — built on llama.cpp and abstracts scripts into simple commands. Inspired by Docker, it offers simple and intuitive model management, making it easy to swap models. You can see the list of available models at <a href="https://ollama.ai/library">https://ollama.ai/library</a>. You can also run any GGUF model from HuggingFace following these <a href="https://github.com/ollama/ollama?tab=readme-ov-file#import-from-gguf">instructions</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7r7nahWLdx-UoauM.png" /><figcaption>Ollama (<a href="https://ollama.com/">Source</a>)</figcaption></figure><p>Once you have followed the instructions to download the Ollama application. You can run simple inferences in the terminal by running:</p><pre>ollama run llama2</pre><p>A useful general heuristic for selecting model sizes from Ollama’s <a href="https://github.com/ollama/ollama/blob/main/README.md">README</a>:</p><blockquote>You should have at least 8 GB of RAM available to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models.</blockquote><blockquote>By default, Ollama uses 4-bit quantization. To try other quantization levels, try the other tags. The number after the q represents the number of bits used for quantization (i.e. q4 means 4-bit quantization). The higher the number, the more accurate the model is, but the slower it runs, and the more memory it requires.</blockquote><p>With ollama serve, Ollama sets itself up as a local server on port 11434 that can connect with other services. The <a href="https://github.com/ollama/ollama/blob/main/docs/faq.md">FAQ</a> provides more information.</p><p>Ollama stands out for its strong community support and active development. Its frequent updates are driven by user feedback on Discord. Ollama has many integrations, and people have developed mobile device compatibility.</p><h4>Other LLM Backend Options</h4><p>Table of Top 5 LLM inference repos:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IKmZDHMTyKFHfNEYHInmFA.png" /><figcaption>Top 5 open-source LLM backends, full table available <a href="http://here/">here</a></figcaption></figure><h3>#3. Front-end UIs for connecting to LLM backends</h3><p>The tools discussed in Section 2 can handle basic queries using the pre-trained data of LLMs. Yet, their capabilities significantly expand with integrating external information via web search and Retrieval Augmented Generation (RAG). Utilizing user interfaces that leverage existing LLM frameworks, like LangChain and LlamaIndex, simplifies embedding data chunks into vector databases.</p><p>The UIs mentioned in this section seamlessly interface with the backend servers set up with Section 1’s tools. They are compatible with various APIs, including OpenAI’s, allowing easy integration with proprietary and open weights models.</p><h4>Most similar to ChatGPT visually and functionally: Open WebUI</h4><p><a href="https://openwebui.com/"><strong>Open WebUI</strong></a> is a web UI that provides local RAG integration, web browsing, voice input support, multimodal capabilities (if the model supports it), supports OpenAI API as a backend, and much <a href="https://docs.openwebui.com/">more</a>. Previously called <a href="https://github.com/ollama-webui/ollama-webui">ollama-webui</a>, this project is developed by the Ollama team.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*2B3odohYq-2Za56z.gif" /><figcaption>OpenWeb UI demo <a href="https://github.com/open-webui/open-webui">(Source)</a></figcaption></figure><p>To connect Open WebUI with Ollama all you need is Docker already installed, and then simply run the following:</p><pre>docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main</pre><p>Then, you can simply access Open WebUI at <a href="http://localhost:3000/">http://localhost:3000</a>.</p><h4>Feature Rich UI: Lobe Chat</h4><p><a href="https://chat-preview.lobehub.com/welcome"><strong>Lobe Chat</strong></a> has more features, including its Plugin System for Function Calling and Agent market. Plugins include search engines, web extraction, and many custom ones from the community. Their prompt agent market is similar to the ChatGPT market, which allows users to share and optimize prompt agents for their own use.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*W4ngP9X_oh4xc_LO.png" /><figcaption>Lobe Chat UI <a href="https://github.com/lobehub/lobe-chat/">(Source)</a></figcaption></figure><p>To start a Lobe Chat conversation with a local LLM with Ollama, simply use Docker:</p><pre>docker run -d -p 3210:3210 -e OLLAMA_PROXY_URL=http://host.docker.internal:11434/v1 lobehub/lobe-chat</pre><p>Then, you can simply access Lobe Chat at <a href="http://localhost:3000/">http://localhost:3210</a>. Read <a href="https://chat-docs.lobehub.com/en/usage/features/local-llm">here</a> for more information.</p><h4>UI that supports many backends: text-generation-webui</h4><p><a href="https://github.com/oobabooga/text-generation-webui"><strong>text-generation-webui</strong></a> by Oogabooga is a fully featured Gradio web UI for LLMs and supports many backend loaders like transformers, GPTQ, autoawq (AWQ), exllama (EXL2), llama.cpp (GGUF), and Llama models — which are refactors of the transformers code libraries with extra tweaks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*BIBCp2yOyPNa_G12.png" /><figcaption>text-generation-webui UI <a href="https://github.com/oobabooga/text-generation-webui">(Source)</a></figcaption></figure><p>text-generation-webui is highly configurable and even offers finetuning with QLoRA due to its transformers backend. This allows you to enhance a model’s capabilities and customize based on your data. There is thorough documentation on their <a href="https://github.com/oobabooga/text-generation-webui/wiki">wiki</a>. I find this option useful for quickly testing models due to its great out-of-the-box support.</p><h3>Other UI Options</h3><p>Table of Top 5 open-source LLM UIs:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VQBjL-1wBcw7I5-KMW9HLg.png" /><figcaption>Top 5 LLM UIs, full table available <a href="https://docs.google.com/spreadsheets/d/1Xv38p90V3GiJXjq0a3qc24056Vicn1I5MG6QiFE6nVE/edit#gid=0">here</a></figcaption></figure><h3>Conclusion: What I use and recommend</h3><p>I’ve been using Ollama for its versatility, easy model management, and robust support, especially its seamless <a href="https://ollama.com/blog/openai-compatibility">integration with OpenAI models</a>. For coding, Ollama’s API connects with the <a href="https://continue.dev/">continue.dev</a> VS Code plugin, replacing GitHub Copilot.</p><p>For UIs, I prefer Open WebUI for its professional, ChatGPT-like interface or Lobe Chat for additional plugins. For those seeking a user-friendly desktop app akin to ChatGPT, Jan is my top recommendation.</p><p>I use both Ollama and Jan for local LLM inference, depending on how I wish to interact with an LLM. You can <a href="https://jan.ai/docs/local-models/ollama#integrate-ollama-with-jan">integrate</a> Ollama and Jan to save system storage and avoid model duplication.</p><h3>References</h3><ol><li><a href="https://github.com/janhq/awesome-local-ai">https://github.com/janhq/awesome-local-ai</a></li><li><a href="https://www.reddit.com/r/LocalLLaMA/">https://www.reddit.com/r/LocalLLaMA/</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=db1ec6f5a54f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/thedeephub/50-open-source-options-for-running-llms-locally-db1ec6f5a54f">50+ Open-Source Options for Running LLMs Locally</a> was originally published in <a href="https://medium.com/thedeephub">The Deep Hub</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>