<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Raj Hammeer S. Hada on Medium]]></title>
        <description><![CDATA[Stories by Raj Hammeer S. Hada on Medium]]></description>
        <link>https://medium.com/@hammeerraj?source=rss-a944615efe93------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*_mzHm1islDIoq_6Mnl5qFA.jpeg</url>
            <title>Stories by Raj Hammeer S. Hada on Medium</title>
            <link>https://medium.com/@hammeerraj?source=rss-a944615efe93------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 02:37:38 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@hammeerraj/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Run Florence — 2 model on Colab (free)]]></title>
            <link>https://medium.com/@hammeerraj/run-florence-2-model-on-colab-free-800b4e4e8b17?source=rss-a944615efe93------2</link>
            <guid isPermaLink="false">https://medium.com/p/800b4e4e8b17</guid>
            <category><![CDATA[florence-2]]></category>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[vision]]></category>
            <category><![CDATA[microsoft]]></category>
            <dc:creator><![CDATA[Raj Hammeer S. Hada]]></dc:creator>
            <pubDate>Wed, 26 Jun 2024 05:33:33 GMT</pubDate>
            <atom:updated>2024-06-26T05:33:33.392Z</atom:updated>
            <content:encoded><![CDATA[<h3>Run Florence — 2 model on Colab (free)</h3><h4>Microsoft’s 400M model beats OpenAI’s 80B model in Benchmarks</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lTpRa7xBy6vhithm" /><figcaption>Photo by <a href="https://unsplash.com/@pawelskor?utm_source=medium&amp;utm_medium=referral">Paul Skorupskas</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4>GPU inferencing —</h4><ol><li><a href="https://colab.research.google.com/drive/1FF1l27-Vma665G2YP6DgqR4-zvgEsdbH?usp=sharing">Colab Link</a></li><li><a href="https://huggingface.co/spaces/SixOpen/Florence-2-large-ft">Demo Link</a></li></ol><h4>CPU inferencing —</h4><ol><li>Open <a href="https://colab.research.google.com/drive/1dUsztvbuxMY4Y3U1p9hSyo1mDjtQz4gt?usp=sharing">Colab Link</a>, Run all cells, It will throw a flash_attn error on CPU.</li><li>Comment out these lines in modeling_florence2.py file to bypass flash_attn errors and run the last cell again.</li></ol><pre>#somewhere in line 63,64 in modeling_florence2.py<br> if is_flash_attn_2_available():<br>    from flash_attn.bert_padding import index_first_axis, pad_input, unpad_input  # noqa</pre><pre>#somewhere in line 684,685,686 in modeling_florence2.py<br>if is_flash_attn_2_available():<br>    from flash_attn import flash_attn_func, flash_attn_varlen_func<br>    from flash_attn.bert_padding import index_first_axis, pad_input, unpad_input  # noqa</pre><h4>How to Finetune the model?</h4><p><a href="https://huggingface.co/blog/finetune-florence2">https://huggingface.co/blog/finetune-florence2</a></p><h4>Packaging —</h4><ol><li>Llama.cpp doesn’t support this model yet — <a href="https://github.com/ggerganov/llama.cpp/issues/8012">https://github.com/ggerganov/llama.cpp/issues/8012</a></li><li>The only way I was able to package this for Windows was using <strong>PyInstaller</strong>, hoping for a release from <a href="https://github.com/ggerganov/llama.cpp">llama.cpp</a> team to run it on edge devices.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=800b4e4e8b17" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build your own Perplexity.ai (Discussion)]]></title>
            <link>https://medium.com/@hammeerraj/perplexity-unperplexed-discussion-3ac57ea87f58?source=rss-a944615efe93------2</link>
            <guid isPermaLink="false">https://medium.com/p/3ac57ea87f58</guid>
            <category><![CDATA[perplexity-ai]]></category>
            <category><![CDATA[search]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Raj Hammeer S. Hada]]></dc:creator>
            <pubDate>Sun, 12 May 2024 12:36:58 GMT</pubDate>
            <atom:updated>2024-06-26T13:48:53.508Z</atom:updated>
            <content:encoded><![CDATA[<h4>Steps to produce the same (almost) results from <a href="https://www.perplexity.ai/">perplexity</a>, <a href="https://you.com/">you</a> or <a href="https://www.morphic.sh/">morphic</a>, for your own search engine</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*qYVLWbicGgnMpLD8" /><figcaption>Photo by <a href="https://unsplash.com/@oskaryil?utm_source=medium&amp;utm_medium=referral">Oskar Yildiz</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4>Prerequisites —</h4><ol><li><a href="https://serper.dev/"><strong>SerperDev</strong></a><strong> API key</strong> for web search and images or use <a href="https://pypi.org/project/duckduckgo-search/"><strong>DDG search</strong></a><strong> (free alternative).</strong></li><li><a href="https://unstructured.io/"><strong>Unstructured.io</strong></a> for web scraping (<strong>optional</strong> — only needed if you want detailed and more refined answers, a must of production use-case).</li><li>Choose any one API provider from these <a href="https://www.together.ai/"><strong>Together.AI</strong></a><strong>, </strong><a href="https://fireworks.ai/"><strong>Fireworks.AI</strong></a><strong>, </strong><a href="https://groq.com/"><strong>Groq</strong></a><strong>, </strong><a href="https://www.anthropic.com/"><strong>Anthorpic</strong></a><strong>, </strong><a href="https://openai.com/"><strong>OpenAI</strong></a> for LLM inferencing (recommended model — anything above 56B parameters is good to go).</li></ol><h4>Background —</h4><p>Now let’s talk about some of my findings from sniffing <a href="https://you.com/">You</a> API calls —</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t5tykOoY6ErPn2vflXVnLQ.png" /><figcaption>network calls from you to its backend</figcaption></figure><p><strong><em>Example Query</em> — <em>Is it worth buying apple vision pro in 2024?</em></strong></p><p><strong><em>Results from SerperDev</em></strong> (<strong><em>type →Search</em></strong>)<strong><em>—</em></strong></p><pre>{<br>  &quot;link&quot;: &quot;https://www.cnn.com/cnn-underscored/electronics/apple-vision-pro-worth-it&quot;,<br>  &quot;link&quot;: &quot;https://www.cnet.com/tech/computing/apple-vision-pro-one-month-later-its-in-my-life-sometimes/&quot;,<br>  &quot;link&quot;: &quot;https://fortune.com/2024/02/24/apple-vision-pro-review-by-working-parent/&quot;,<br>  &quot;link&quot;: &quot;https://www.wired.com/review/apple-vision-pro/&quot;,<br>  &quot;link&quot;: &quot;https://www.reddit.com/r/VisionPro/comments/1ahybno/a_review_of_the_reviews_of_apple_vision_pro_and/&quot;,<br>  &quot;link&quot;: &quot;https://www.cosmopolitan.com/lifestyle/a46425557/apple-vision-pro-review/&quot;,<br>  &quot;link&quot;: &quot;https://www.reddit.com/r/AppleVisionPro/comments/1al9kpl/thoughts_on_buying_an_apple_vision_pro_and/&quot;,<br>  &quot;link&quot;: &quot;https://www.tomsguide.com/computing/vr-ar/ive-been-testing-the-apple-vision-pro-for-3-weeks-heres-the-pros-and-cons&quot;,<br>  &quot;link&quot;: &quot;https://www.theverge.com/24054862/apple-vision-pro-review-vr-ar-headset-features-price&quot;,<br>  &quot;link&quot;: &quot;https://ktla.com/news/hands-on-apple-vision-pro-amazing-but-expensive/&quot;<br>}</pre><p><strong><em>Results from SerperDev</em></strong> (<strong><em>type →News</em></strong>)<strong><em> —</em></strong></p><pre>{<br>  &quot;link&quot;: &quot;https://www.cnn.com/cnn-underscored/electronics/apple-vision-pro-worth-it&quot;,<br>  &quot;link&quot;: &quot;https://www.zdnet.com/article/the-3-apple-products-you-shouldnt-buy-this-month-including-this-ipad/&quot;,<br>  &quot;link&quot;: &quot;https://www.cnet.com/tech/computing/apple-vision-pro-one-month-later-its-in-my-life-sometimes/&quot;,<br>  &quot;link&quot;: &quot;https://www.cosmopolitan.com/lifestyle/a46425557/apple-vision-pro-review/&quot;,<br>  &quot;link&quot;: &quot;https://www.goodhousekeeping.com/electronics/a46462295/apple-vision-pro-review/&quot;,<br>  &quot;link&quot;: &quot;https://www.cosmopolitan.com/lifestyle/a46425557/apple-vision-pro-review/&quot;,<br>  &quot;link&quot;: &quot;https://www.washingtonpost.com/technology/2024/02/12/apple-vision-pro-review/&quot;,<br>  &quot;link&quot;: &quot;https://www.sportskeeda.com/gaming-tech/is-apple-vision-pro-worth&quot;,<br>  &quot;link&quot;: &quot;https://www.theverge.com/24054862/apple-vision-pro-review-vr-ar-headset-features-price&quot;,<br>  &quot;link&quot;: &quot;https://ktla.com/news/hands-on-apple-vision-pro-amazing-but-expensive/&quot;<br>}</pre><p><strong><em>Top Results from You—</em></strong></p><pre>{<br>  &quot;link&quot;: &quot;https://www.cosmopolitan.com/lifestyle/a46425557/apple-vision-pro-review/&quot;,<br>  &quot;link&quot;: &quot;https://www.apple.com/newsroom/2024/01/apple-vision-pro-available-in-the-us-on-february-2/&quot;,<br>  &quot;link&quot;: &quot;https://www.theverge.com/24054862/apple-vision-pro-review-vr-ar-headset-features-price&quot;,<br>  &quot;link&quot;: &quot;https://www.cnbc.com/2024/01/30/apple-vision-pro-review-the-future-of-computing-and-entertainment.html&quot;,<br>  &quot;link&quot;: &quot;https://www.esquire.com/lifestyle/cars/a44104790/apple-headset-vision-pro-review/&quot;,<br>  &quot;link&quot;: &quot;https://www.wired.com/review/apple-vision-pro/&quot;,<br>  &quot;link&quot;: &quot;https://forums.macrumors.com/threads/worth-buying-an-apple-vision-pro-to-sell-in-future-for-profit.2415211/&quot;,<br>  &quot;link&quot;: &quot;https://www.forbes.com/sites/timbajarin/2023/10/31/why-apple-will-sellout-of-vision-pros-in-2024/&quot;,<br>  &quot;link&quot;: &quot;https://www.sportskeeda.com/gaming-tech/is-apple-vision-pro-worth&quot;,<br>  &quot;link&quot;: &quot;https://www.zdnet.com/article/apple-vision-pro-hands-on-buying-advice/&quot;<br>}</pre><p>A lot of web links seem to be similar on You.com and SerperDev (the discrepancies could be because of geo location, not really sure? 😥), I have a hunch they are using SerperDev for their web search with an addition to an in-house re-ranking algorithm (we will talk about this later in this blog).</p><p>Finding that You.com uses SerperDev is not a new thing though, SerperDev is cheap, does an okay job and is pretty reliable so most of the AI based search engines seem to use it. The juice lies in the <strong>llm_intent </strong>(refer image above)<strong> </strong>call that gives them the intent of user’s query, <strong>for example</strong> — the query, <strong>is it worth buying Apple vision pro in 2024?</strong> came under <strong>Finance and Crypto</strong> (top probability) and <strong>General Information Search</strong> (2nd top probability).</p><p>This segregation of queries according to the intent of the question from the user seems to be a nice move from You.com. And using this information of intent, we could —</p><ol><li>Get search results from SerperDev b/w search, news and more, based on the intent of the query.</li><li>Build a re-ranking algorithm which would be more effective now because the reddit URLs can be removed from SerperDev results if user queries don’t fall under the type Reddit. For queries that fall under Causal Chat, a SerperDev search call is not even needed so LLM can answer directly. For queries that fall under Twitter or any other social media, for the matter, links from Wikipedia won’t make much sense and hence can be re-ranked to hold less weightage in the final answering from LLM.</li></ol><p>Now with all this information in hand, let’s write an end-to-end logic for our search engine to wield results just like perplexity.ai or you.com.</p><h4>Final End-to-End Flow —</h4><ol><li>Receive user’s query</li><li>Judge the intent of the user’s query (<em>Like how You.com is doing it, Intent can be classified from a list of pre-defined query types, for example — Finance &amp; Crypto, Twitter &amp; social media, Live Sports News, General News, Code, Wikipedia &amp; Facts, etc., you get the point right?</em>)</li><li>Get SerperDev results based on the intent of the query (<em>is it a basic search, a news search, a shopping related query, how many URLs you might need, Do you even need images in this query type? All these questions are answered here.</em>)</li><li>Now re-rank the results running semantic search on snippets of the URLs against user’s query. (<em>re-ranking is also based on the intent, if it’s a fact-based question consider Wikipedia and Britannica to have more weightage, queries related to videos will have more URLs from YouTube and Vimeo, queries that are more shopping related will have Amazon, Walmart, Ebay as top URLs</em>).</li><li>Scrape top <strong>n</strong> pages from re-ranked URLs (<strong><em>optional </em></strong><em>— uses </em><strong><em>unstructured.io</em></strong><em> for scraping the web URLs to get better answers</em>).</li></ol><p><strong>Note </strong>— <strong>n </strong>depends on how refined and quality answer you are expecting, intent type (more for news and less for general search). And it totally depends how qualitative you want the answer to be.</p><p>6. Finally provide all the scraped URLs content as context and ask LLM to answer user’s query with the intent attached. The answer should be streamed and must be in markdown format for parsing it beautifully in the frontend.</p><p>That’s it! This is all you need to build an AI based search engine that answers your questions and doesn’t throw a bunch of links on your face like Google or Bing. I am thinking of forking the <a href="https://github.com/miurla/morphic">morphic project</a> and implement the stack as discussed above. Please feel free to share your ideas in the comment section.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3ac57ea87f58" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building Local RAG (Windows) P1]]></title>
            <link>https://medium.com/@hammeerraj/building-local-rag-windows-p1-f01106c50afb?source=rss-a944615efe93------2</link>
            <guid isPermaLink="false">https://medium.com/p/f01106c50afb</guid>
            <category><![CDATA[rags]]></category>
            <category><![CDATA[local]]></category>
            <category><![CDATA[llama-cpp]]></category>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <dc:creator><![CDATA[Raj Hammeer S. Hada]]></dc:creator>
            <pubDate>Wed, 24 Apr 2024 06:28:21 GMT</pubDate>
            <atom:updated>2024-04-24T06:34:29.739Z</atom:updated>
            <content:encoded><![CDATA[<h3>Local RAG approach using FAISS (Semantic Search) + Llama.cpp (Embeddings Generation)</h3><h4>1. Understanding Basic RAG —</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/956/1*xTgBi0nWdZvFmcBAcV_NXg.png" /><figcaption>shoutout to draw.io for making such a kickass UML application</figcaption></figure><p><strong>1.1 Adding to Vector DB (Upserting) —</strong></p><p>Text input strings are chunked in smaller strings, followed by converting the text inputs to embeddings and finally adding them to the vector DB (or keeping them in memory → preferred when you don’t need a lot of data to be stored in the vector DB).</p><p><strong>1.2 Querying the DB (Search) —</strong></p><p>Searching operation involves converting the query string into embeddings and doing a simple semantic search operation to get top similar results.</p><h4>2. Generating Embeddings —</h4><p><strong>2.1 Downloading Llama.cpp —</strong></p><p>Download this <a href="https://github.com/ggerganov/llama.cpp/releases/download/b2715/llama-b2715-bin-win-openblas-x64.zip">pre-built llama.cpp</a> and unzip this on your machine.</p><p><strong>2.2 Text input to Embeddings —</strong></p><p>Download this <a href="https://huggingface.co/CompendiumLabs/bge-large-zh-v1.5-gguf/blob/main/bge-large-zh-v1.5-q4_k_m.gguf">model</a>, and run the command in the root directory of your unzipped llama.cpp binaries—</p><pre>embedding.exe -m models\bge-large-zh-v1.5-q4_k_m.gguf -p &quot;An apple a day keeps the doctor away&quot;</pre><p>The output would be an array of float values, yep those are the embeddings. Save them somewhere we will need these in future.</p><h4>3. FAISS</h4><p><strong>3.1 FAISS Compilation —</strong></p><p>Follow this <a href="https://github.com/facebookresearch/faiss/blob/main/INSTALL.md">link</a> to build FAISS with C/C++ bindings on Windows.</p><p><strong>— Note —</strong></p><ol><li>openblas &amp; lapack lib can be downloaded and attached while building FAISS using cmake</li></ol><pre>cmake -B build -DFAISS_ENABLE_GPU=OFF -DFAISS_ENABLE_PYTHON=OFF -DFAISS_ENABLE_C_API=ON -DBLAS_LIBRARIES=&quot;OpenBLAS-0.3.27-x86/lib/libopenblas.lib&quot; -DLAPACK_LIBRARIES=&quot;liblapack.lib&quot;</pre><p>After the compilation is done, Try running the example_c.c code in build/c_api and it should work now.</p><p>Now that you have all the legos ready, all you need is to put everything together and you will have a local RAG solution.</p><h4>Let’s see the integration in PART 2.</h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f01106c50afb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Try Llama3 on Colab (Free)]]></title>
            <link>https://medium.com/@hammeerraj/try-llama3-on-colab-free-f8c30eba19d7?source=rss-a944615efe93------2</link>
            <guid isPermaLink="false">https://medium.com/p/f8c30eba19d7</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[meta-llama-3]]></category>
            <category><![CDATA[llama-3]]></category>
            <category><![CDATA[llama-2]]></category>
            <dc:creator><![CDATA[Raj Hammeer S. Hada]]></dc:creator>
            <pubDate>Sat, 20 Apr 2024 21:46:20 GMT</pubDate>
            <atom:updated>2024-04-20T21:46:20.941Z</atom:updated>
            <content:encoded><![CDATA[<h4>Local inferencing on LLama3</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*u6_2cf7nQLZvtZaa" /><figcaption>Photo by <a href="https://unsplash.com/@josiahfarrow?utm_source=medium&amp;utm_medium=referral">Josiah Farrow</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4>About —</h4><p>LLama3 was recently released in 2 model variants — <strong>8B and 70B</strong> parameter models, <strong>pre-trained and instruction fine-tuned</strong> versions, with knowledge cut-off in <strong>March 2023 for the smaller model</strong> and <strong>December 2023 for the 70B</strong> model. Both the models have context length of <strong>8K tokens</strong>.</p><h4>Benchmarking —</h4><p>Benchmark comparison against the old Llama2 release from Meta</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/568/1*q9utvVGZInb4Yc1bIpxiLA.png" /><figcaption><a href="https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md">llama3/MODEL_CARD.md at main · meta-llama/llama3 (github.com)</a></figcaption></figure><p>Instruction Finetuned<strong> LLama3 8B model performs better on all benchmarks against Gemini models and Mistral models.</strong> Whereas <strong>Llama3 70B outperforms Gemini, Claude3 Sonnet &amp; Mistral Medium in almost all of the benchmarks.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/611/1*ChhlM4X_vagNS-bkMEVsjw.png" /></figure><h4>Demo on free Colab notebook (T4 GPU)—</h4><p><strong>Note </strong>— T4 doesn’t support bf16, bf16 is only supported on Ampere and above. So use <strong>float16 </strong>instead. However in production use-cases it is recommended to use bf16 for better output quality.</p><p><strong>You will need</strong> —</p><ol><li><a href="https://huggingface.co/">Huggingface Account</a></li><li><a href="https://huggingface.co/settings/tokens">Read access token</a></li><li><a href="https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct">LLama3 access on Huggingface</a></li></ol><p>All set? Follow the demo now —</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/38f651a7a34e0066b3951fb5153556b2/href">https://medium.com/media/38f651a7a34e0066b3951fb5153556b2/href</a></iframe><h4><strong>Next Tasks —</strong></h4><ol><li>Fast inference using <a href="https://github.com/vllm-project/vllm">vLLM</a> (vLLM already supports Llama3 models for inferencing).</li><li>Check <a href="https://www.promptingguide.ai/models/llama-3">prompting guide</a> to get more predicted responses from the model.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f8c30eba19d7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Running LLMs on CPU]]></title>
            <link>https://medium.com/@hammeerraj/running-llms-on-cpu-52e7bb13ea28?source=rss-a944615efe93------2</link>
            <guid isPermaLink="false">https://medium.com/p/52e7bb13ea28</guid>
            <category><![CDATA[llama-cpp]]></category>
            <category><![CDATA[ggml]]></category>
            <category><![CDATA[koboldcpp]]></category>
            <category><![CDATA[gguf]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Raj Hammeer S. Hada]]></dc:creator>
            <pubDate>Sat, 13 Apr 2024 16:44:07 GMT</pubDate>
            <atom:updated>2024-04-13T16:44:07.952Z</atom:updated>
            <content:encoded><![CDATA[<h4>Easy Guide to using <a href="https://github.com/ggerganov/llama.cpp">Llama.cpp</a> and <a href="https://github.com/LostRuins/koboldcpp">Koboldcpp</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*yLQ56CLT7prx7irA" /><figcaption>Photo by <a href="https://unsplash.com/@bogzilla?utm_source=medium&amp;utm_medium=referral">Alexandru-Bogdan Ghita</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3><strong><em>Recipe →</em></strong></h3><h4>Nutritional information (Definitions &amp; Terminology) →</h4><p><strong><em>What is GGUF?</em></strong></p><p><strong><em>GGUF </em></strong><em>is a file format for GPT like language models for storing, sharing and running inferences (also on CPU if needed) using a single model file. </em><strong><em>GGML </em></strong><em>is kind of the same thing but deprecated and comes with a bunch of limitations.</em></p><h4>Ingredients (Prerequisites) →</h4><p>This blog is more focused on how to leverage this marvel instead of just the definitions of the tech involved. So let’s start cooking, shall we?</p><p><strong>You will need:</strong></p><ol><li>A Model (GGUF File) — (We will learn to mine the GGUF files online and mint our own as well)</li><li>A Computer with or without GPU (CPU will do the job too) — Check?</li><li>Internet Connectivity (Which you might already have if you’re reading this blog) — Check 👍</li></ol><p><strong>Gather the ingredients (THE MINING PART):</strong></p><ol><li>Pick this <a href="https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF/blob/main/mistral-7b-instruct-v0.1.Q4_K_M.gguf">GGUF file</a> from <a href="https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF">TheBloke/Mistral-7B-Instruct-v0.1-GGUF · Hugging Face</a></li></ol><p>2(<strong><em>type — 1</em></strong>). Build Llama.cpp (if LLama.cpp is your calling, Koboldcppp is in the next step) → <a href="https://github.com/ggerganov/llama.cpp">Follow the build guide provided on the GitHub page of llama.cpp</a></p><p><strong><em>OR</em></strong></p><p>2(<strong><em>type — 2</em></strong>). Download Koboldcpp.exe</p><p><strong><em>Note:</em></strong> Download <a href="https://github.com/LostRuins/koboldcpp/releases/tag/v1.62.2"><strong>koboldcpp_nocuda</strong></a><strong>.exe </strong>if you want to run LLM models on CPU</p><h4>Cooking and assembling the dish (Inferencing) →</h4><ol><li>With Llama.cpp, open terminal and run the command —</li></ol><pre>./main -m models/mistral-7b-instruct-v0.1.Q4_K_M.gguf -p &quot;Building a website can be done in 10 simple steps:\nStep 1:&quot; -n 400 -e</pre><ol><li>With Koboldcpp, open terminal and run the command —</li></ol><pre>koboldcpp.exe models/mistral-7b-instruct-v0.1.Q4_K_M.gguf</pre><p><strong>Voila! 🙌 You cooked it right<em>. </em></strong>You might be getting an output from the <strong>LLM running on your CPU</strong> right now, as we speak. If not, drop your error in the comments and let’s debug it together.</p><p><strong>Gather the Ingredients continued (THE MINTING PART):</strong></p><p>If you’re looking to create GGUF files of your own pytorch models or models that don’t have a GGUF file yet then before you start creating your own GGUF, give a search on <a href="https://huggingface.co/">huggingface</a> to check if the GGUF file already exists.</p><p>Here is the recipe to quantize pytorch models into GGUF format → (list of all the supported models on LLama.cpp is attached below)</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e350e74d3b6ad119b24f43da48c8bbea/href">https://medium.com/media/e350e74d3b6ad119b24f43da48c8bbea/href</a></iframe><p><strong>Note :</strong> This is the list of models that is supported for inferencing, the fine-tuned models of these supported model architectures will work just fine.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/669/1*7YPmByrNnkmGVSXVjJnA7g.png" /><figcaption>Source — <a href="https://github.com/ggerganov/llama.cpp">GitHub — ggerganov/llama.cpp: LLM inference in C/C++</a></figcaption></figure><p>Inferencing on your own GGUF converted model is the same —</p><pre>./main -m your_own_gguf_model_here.gguf -p &quot;Building a website can be done in 10 simple steps:\nStep 1:&quot; -n 400 -e</pre><h4><strong>Next Steps:</strong></h4><ol><li>Understanding the parameters and using them correctly for your own use-case.</li><li>How different quantization have different output quality and different resource consumption.</li><li>Benchmarking different GGUF models based on the resources they consume, time taken for first token, token throughput.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=52e7bb13ea28" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Valorant Aimbot using OpenCV]]></title>
            <link>https://medium.com/@hammeerraj/valorant-aimbot-using-opencv-ac24b12dc975?source=rss-a944615efe93------2</link>
            <guid isPermaLink="false">https://medium.com/p/ac24b12dc975</guid>
            <category><![CDATA[windows]]></category>
            <category><![CDATA[aimbot]]></category>
            <category><![CDATA[opencv]]></category>
            <category><![CDATA[gaming]]></category>
            <category><![CDATA[valorant]]></category>
            <dc:creator><![CDATA[Raj Hammeer S. Hada]]></dc:creator>
            <pubDate>Sat, 22 Apr 2023 22:20:35 GMT</pubDate>
            <atom:updated>2023-04-22T22:20:35.756Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xnpbqa9GMrT2ZdmS3gLXDQ.jpeg" /></figure><p>I <strong>suck </strong>at Valorant, well that’s not the only reason why my mind vented into creating an aimbot for jumping some ranks. I also really wanted to know if it could be done? (I took it on my ego NGL)</p><p>The fact that hitting head every time being a Silver is very realistic and not that bad after all. (Catching my sarcasm low elo players? 😅)</p><h4>The Start — 💚</h4><p>Here is the map I wanted to follow after doing some experiments and scraping the internet. I am embarrassed to say that my internet history looks like a typical cheater lol, with search keywords like ‘<strong>Valorant Aimbot</strong>’ ‘<strong>Valo Aimbot Github</strong>’. (The guilt wasn’t stopping me to look at some code examples online). Most folks on the internet agreed to something like this —</p><ol><li>Get screen capture and feed it to the detection engine.</li><li>Detect purple outline of enemy body on the screen.</li><li>Trigger a mouse click on the detected blob.</li></ol><p>Sounds easy right? I mean excluding the sleepless nights with false body detections and wanting to give up because <strong><em>mouse_move()</em></strong> and <strong><em>mouse_left_click()</em></strong> just won’t work (Valorant seems to have blocked <strong>SendInput()</strong> coming from the system calls on Windows, more on this later…)</p><h4>Screen Capture —</h4><p>This might be the worst screen capture code I could have written but but ✋. It gets the work done. It basically returns me a Mat object of the screen for my further processing, pretty cool right? (Pair this with <strong>ffmpeg </strong>and you have your own OBS ready 💁)</p><pre>#include &lt;iostream&gt;<br>#include &lt;Windows.h&gt;<br>#include &lt;opencv2/opencv.hpp&gt;<br><br>using namespace std;<br>using namespace cv;<br><br>Mat captureScreenMat(HWND hwnd)<br>{<br>       HDC hwindowDC,hwindowCompatibleDC;<br><br>       int height,width,srcheight,srcwidth;<br>       HBITMAP hbwindow;<br>       Mat src;<br>       BITMAPINFOHEADER  bi;<br><br>       hwindowDC=GetDC(hwnd);<br>       hwindowCompatibleDC=CreateCompatibleDC(hwindowDC);<br>       SetStretchBltMode(hwindowCompatibleDC,COLORONCOLOR);<br><br>       RECT windowsize;    // get the height and width of the screen<br>       GetClientRect(hwnd, &amp;windowsize);<br><br>       srcheight = windowsize.bottom;<br>       srcwidth = windowsize.right;<br>       height = windowsize.bottom/1;  //change this to whatever size you want to resize to<br>       width = windowsize.right/1;<br><br>       src.create(height,width,CV_8UC4);<br><br>       // create a bitmap<br>       hbwindow = CreateCompatibleBitmap( hwindowDC, width, height);<br>       bi.biSize = sizeof(BITMAPINFOHEADER);<br>       bi.biWidth = width;<br>       bi.biHeight = -height;  //this is the line that makes it draw upside down or not<br>       bi.biPlanes = 1;<br>       bi.biBitCount = 32;<br>       bi.biCompression = BI_RGB;<br>       bi.biSizeImage = 0;<br>       bi.biXPelsPerMeter = 0;<br>       bi.biYPelsPerMeter = 0;<br>       bi.biClrUsed = 0;<br>       bi.biClrImportant = 0;<br><br>       // use the previously created device context with the bitmap<br>       SelectObject(hwindowCompatibleDC, hbwindow);<br>       // copy from the window device context to the bitmap device context<br>       StretchBlt( hwindowCompatibleDC, 0,0, width, height, hwindowDC, 0, 0,srcwidth,srcheight, SRCCOPY); //change SRCCOPY to NOTSRCCOPY for wacky colors !<br>       GetDIBits(hwindowCompatibleDC,hbwindow,0,height,src.data,(BITMAPINFO *)&amp;bi,DIB_RGB_COLORS);  //copy from hwindowCompatibleDC to hbwindow<br><br>       // avoid memory leak<br>       DeleteObject (hbwindow);<br>       DeleteDC(hwindowCompatibleDC);<br>       ReleaseDC(hwnd, hwindowDC);<br><br>       return src;<br> }<br><br>int main()<br>{<br>    namedWindow(&quot;Screen Capture&quot;, CV_WINDOW_NORMAL);<br>    HWND hwnd = GetDesktopWindow();<br>    int key = 0;<br>    while(key != 27){<br>        Mat src = captureScreenMat(hwnd);<br>        key = waitKey(30);<br>    }<br><br>    return 0;<br>}</pre><h4>Body Detection —</h4><p>After a bunch of hit and trial I was able to detect enemy bodies almost immediately as they show up on screen (At close to mid-range distances) and below are the settings/filters I have used from OpenCV.</p><p><strong>Note</strong> — for people who haven’t played hefty Valorant, the outline of the enemy bodies is marked purple.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/205/1*Eqdml9-cgCqnhP-CK9mFtg.png" /><figcaption>You see the purple outline?</figcaption></figure><p>Hence you would see the <strong>inRange()</strong> method being used to mask purple color in the source image.</p><p>Later I am detecting all the blobs larger than a particular area using <strong>findContours()</strong> and then only selecting the blob that is closest to the center (considering that a bit of manual aiming would be done by the user themselves).</p><pre>void bodyDetector(Mat frame){<br>    Mat image;<br>    cvtColor(frame, image, COLOR_BGR2HSV);<br>    inRange(image, Scalar(130, 50, 50), Scalar(150, 255, 255), image);<br><br>    int morph_size = 1;<br>    Mat element = getStructuringElement(<br>    MORPH_DILATE, Size(2 * morph_size + 1,<br>                     2 * morph_size + 1),<br>    Point(morph_size, morph_size));<br><br>    //dilate to make everything bigger a bit<br>    dilate(image, image, element, Point(-1, -1), 5, 1);<br><br>    threshold(image, image, 60, 255, THRESH_BINARY);<br><br>    vector&lt;vector&lt;Point&gt; &gt; contours;<br>    findContours( image, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_NONE );<br><br>    vector&lt;vector&lt;Point&gt; &gt;hull( contours.size() );<br><br>    for( size_t i = 0; i &lt; contours.size(); i++ )<br>    {<br>        convexHull( contours[i], hull[i] );<br>    }<br><br>    vector&lt;Moments&gt; mu(hull.size()); //get moments<br>    for (int i = 0; i &lt; hull.size(); i++)<br>    {<br>        mu[i] = moments(hull[i], false);<br>    }<br><br>    vector&lt;Point2f&gt; mc(hull.size()); //get centers<br>    for (int i = 0; i &lt; hull.size(); i++)<br>    {<br>        mc[i] = Point2f(mu[i].m10 / mu[i].m00, mu[i].m01 / mu[i].m00);<br>    }<br><br>    vector&lt;int&gt; largeBlobIndexes;<br>    for (int j = 0; j &lt; hull.size(); j++) {<br>        double newArea = contourArea(hull.at(j));<br>        if (newArea &gt; MAX_AREA_CAP) {<br>            largeBlobIndexes.push_back(j);<br>        }<br>    }<br><br>    int closestToCenterBlobDistance = 1920;<br>    int closestToCenterBlobDistanceIndx = 0;<br>    for(int i=0;i&lt;largeBlobIndexes.size();i++){<br>        Point2f frameCenter = Point2f(frame.cols/2, frame.rows/2);<br>        if(sqrt(pow(frameCenter.x - mc[i].x, 2) + pow(frameCenter.y - mc[i].y, 2) * 1.0) &lt; closestToCenterBlobDistance){<br>            closestToCenterBlobDistance = sqrt(pow(frameCenter.x - mc[i].x, 2) + pow(frameCenter.y - mc[i].y, 2) * 1.0);<br>            closestToCenterBlobDistanceIndx = i;<br>        }<br>    }<br><br>    drawContours( frame, hull, closestToCenterBlobDistanceIndx, Scalar(255, 0, 0), CV_FILLED, 8 );<br><br>    //trigger click at this x and y on SHIFT_PRESS<br>    if(GetKeyState(VK_SHIFT) &amp; 0x8000)<br>    {<br>        click_x = mc[closestToCenterBlobDistanceIndx].x;<br>        click_y = mc[closestToCenterBlobDistanceIndx].y;<br>        EnumDisplayMonitors(NULL,NULL, Monitorenumproc,0);<br>    }<br><br>    imshow(&quot;Head Hitter&quot;, frame);<br>}</pre><p>I am being 100% honest when I say this piece of code doesn’t hit the body every time, let alone the required head hit to win a kill.</p><p>On finding the <strong>cvMoments</strong>(weighted center of the blob) of the closet blob to the center, we trigger a click event using the normal <strong>SendInput()</strong> WIN system call. The click events works just fine on the test video that I feed to the algorithm.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/367/1*YWASVFPAshAgU-3Q--OYaw.png" /><figcaption>Body detection doing its job 👍</figcaption></figure><h4>But But ✋What’s the catch here? (Aimbot works? 😶) —</h4><p>As soon as I launch Valorant, the click events from <strong><em>SendInput()</em></strong> on Windows doesn&#39;t work. Looks like Valorant has worked a bit on their anti-cheat innit? I am suspecting Valorant blocks the <strong>SendInput()</strong> calls. I have tried a bunch of other methods including (<strong><em>mouse_event()</em></strong> calls on Windows, but that’s a failure too).</p><p>Well, I am really sorry for a disappointing end. I am still looking for a workaround, let’s see if I am able to figure it out. I will keep pushing updates here. Care to help me on this? Comment down and I will give you access to the source code.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ac24b12dc975" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[OTP View in Android (Easy)]]></title>
            <link>https://medium.com/@hammeerraj/otp-view-in-android-easy-c5a449d3da42?source=rss-a944615efe93------2</link>
            <guid isPermaLink="false">https://medium.com/p/c5a449d3da42</guid>
            <category><![CDATA[android-app-development]]></category>
            <category><![CDATA[otp-verification]]></category>
            <dc:creator><![CDATA[Raj Hammeer S. Hada]]></dc:creator>
            <pubDate>Mon, 22 Nov 2021 16:23:47 GMT</pubDate>
            <atom:updated>2024-04-27T12:09:44.264Z</atom:updated>
            <content:encoded><![CDATA[<h4>OTP View implementation from — https://github.com/criminact/OTPView</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2ZGwrenHcYqr9qq6qrniUw.png" /><figcaption>Credits — <a href="https://dribbble.com/arunpsingh">https://dribbble.com/arunpsingh</a></figcaption></figure><p>Since Firebase Mobile Auth seems razzmatazz to the Android newbies. They fell short of a way to fill OTPs in candy UI and Since native EditText can’t be styled into divided Boxes. <strong>OTPView </strong>Lib is at the rescue —</p><h4>Step 1.1— Add Dependencies to build.gradle(Module: Example.app)</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cd3b99f9f2e91f72114c69aa17f387eb/href">https://medium.com/media/cd3b99f9f2e91f72114c69aa17f387eb/href</a></iframe><h4>Step 1.2— Add JitPack Repo to build.gradle(Project: Example)</h4><blockquote>For versions &lt; Artic Fox</blockquote><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b929ce888abf034731612ced2c7e97f0/href">https://medium.com/media/b929ce888abf034731612ced2c7e97f0/href</a></iframe><blockquote>For versions ≥ Artic Fox</blockquote><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cc0c42530921bc1bae88d9c4ff8f8635/href">https://medium.com/media/cc0c42530921bc1bae88d9c4ff8f8635/href</a></iframe><h4>Step 2 — Add OTPView in the xml</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2b5709266ce2aa7e9acf54f6966ac66b/href">https://medium.com/media/2b5709266ce2aa7e9acf54f6966ac66b/href</a></iframe><h4>Step 3— Handle OTPView Completion in the Fragment or Activity</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2f3c4d3f6e61ab734a47992b3514e4b9/href">https://medium.com/media/2f3c4d3f6e61ab734a47992b3514e4b9/href</a></iframe><p>And Voila! The implementation is done and ready to test &amp; deploy.</p><p>Thanks for following up until here. Feel free to request for features and contribute to this open source lib 🙌.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c5a449d3da42" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[RecyclerView isn’t so tough]]></title>
            <link>https://medium.com/@hammeerraj/recyclerview-aint-so-tough-559ac87975a8?source=rss-a944615efe93------2</link>
            <guid isPermaLink="false">https://medium.com/p/559ac87975a8</guid>
            <category><![CDATA[android-apps]]></category>
            <category><![CDATA[android-app-development]]></category>
            <category><![CDATA[recyclerview-in-android]]></category>
            <category><![CDATA[recyclerview]]></category>
            <category><![CDATA[android]]></category>
            <dc:creator><![CDATA[Raj Hammeer S. Hada]]></dc:creator>
            <pubDate>Sun, 14 Nov 2021 08:21:22 GMT</pubDate>
            <atom:updated>2024-05-06T10:21:25.130Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>The Gods’ very own way to populate RecyclerViews in Android</strong></p><p>Note — Below tutorial is in <strong>Kotlin</strong>.</p><p>The RecyclerView has multiple custom views either aligned horizontally, vertically or in Grid. The only requirement for these ‘custom views’ is they are ought to be somewhat similar since they all have a common xml. Here is the Visual Explanation —</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gLZt_aejKpNx360qFaX7ww.png" /></figure><p>Image the view hierarchy(xml) in this case to be somewhat like this —</p><pre>&lt;RecyclerView&gt;<br>   ...   <br>   //list of custom views<br>   &lt;CustomView&gt;</pre><pre>   &lt;/CustomView&gt;</pre><pre>   &lt;CustomView&gt;</pre><pre>   &lt;/CustomView&gt;<br>   ...<br>&lt;/RecyclerView&gt;</pre><h4><strong>Done with the explanation part. Let’s code, yeah?</strong></h4><p>oh wait wait, now before we start. Prepare the Data, Which is needed to be shown to the end user.</p><ol><li><strong>Preparing Data</strong></li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/738/1*17leGgiMcEOycg0x5t8Lxw.png" /><figcaption>Custom View</figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/33bd55677dd23bbe34e3aa34a0194303/href">https://medium.com/media/33bd55677dd23bbe34e3aa34a0194303/href</a></iframe><p>2.<strong> Creating Custom View XML</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/978/1*nBO5tGa0CWRSs2VYRwfUZQ.png" /><figcaption>Custom View XML output</figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5279d3c7600e49e01595f03c7f0f7402/href">https://medium.com/media/5279d3c7600e49e01595f03c7f0f7402/href</a></iframe><p>3.<strong> Making a View that Carries multiple Custom Views together — RecyclerView</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/477f95f8070ca08efc529effe0ab9f01/href">https://medium.com/media/477f95f8070ca08efc529effe0ab9f01/href</a></iframe><p>4. <strong>Creating Adapter(DataBinding Required)</strong></p><p>4.1 — <strong><em>Enable DataBinding</em></strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a012a394ebbe3d234ea0cef31c9e2ff2/href">https://medium.com/media/a012a394ebbe3d234ea0cef31c9e2ff2/href</a></iframe><p>4.2 — <strong><em>Proceeding with the Adapter Creation</em></strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7461267d548ee38a926bf01710cdb357/href">https://medium.com/media/7461267d548ee38a926bf01710cdb357/href</a></iframe><p>5.<strong> Fill the Data and VOILA!!</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/43c2329f2cbe682cc83cc592ae2be686/href">https://medium.com/media/43c2329f2cbe682cc83cc592ae2be686/href</a></iframe><h4>BONUS</h4><p><strong><em>In most cases, The data is to be updated without defining the Adapter again, To do that, Follow this —</em></strong></p><p>Changes needed in Adapter(Notice the new Function <strong>updateData()</strong>) —</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/70996622bd125e2de488f0d254241a42/href">https://medium.com/media/70996622bd125e2de488f0d254241a42/href</a></iframe><p>Call the AppDataAdapter’s updateData function anytime you want to update the data in the RecyclerView.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/765ec05250180f8102d100d8afa8259c/href">https://medium.com/media/765ec05250180f8102d100d8afa8259c/href</a></iframe><p>I hope the tutorial was easy to understand and implement. Thank you.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=559ac87975a8" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>