PinnedInTowards AIbyArion Das·Apr 19, 2024The Infamous Attention Mechanism in the Transformer architectureTHE WHY & WHEN ?A response icon1A response icon1
PinnedInTowards AIbyArion Das·Nov 13, 2024Advanced Attention Mechanisms — IIFlash Attention. You can refer to it’s predecessors here: KV cache, sliding window attention, MHA, MQA, uptraining, & GQA. These methods…
PinnedInTowards AIbyArion Das·Nov 3, 2024Advanced Attention Mechanisms-II would recommend you go through this blog first to develop the intuition behind the infamous attention mechanism.A response icon1A response icon1
PinnedArion Das·Aug 26, 2024Representing Words, Phrases & their Compositionality — Skip Gram ModelRepresenting words in a vector space helps achieve better performances on NLP tasks as it helps learning the algorithms better. I have…
PinnedInGenerative AIbyArion Das·Feb 10, 2024RAGs from scratch — Why & What?!!Ok. It’s true; LLMs are answering most of the questions out there. Gone are those days when one had to memorize repetitive stuff. LLMs are…
Arion Das·Feb 18, 2025Fine-tuning DeepSeek-R1 on Bhagvad Gita text chunks using Glows-AI GPUsIn this article, I’ll discuss how to use the GPU service provided by Glows.ai. It is not advisable to refer to this fine-tuning pipeline…
InGenerative AIbyArion Das·Aug 31, 2024Weighing Down — Subsampling & Negative SamplingBased on the math in the skip gram model, we can identify two major drawbacks:
Arion Das·Aug 27, 2024Hierarchical SoftmaxSoftmax is the output layer function which activates the nodes in the last step of the neural network computation. It is a very popular…
InGenerative AIbyArion Das·Aug 19, 2024Stemming & LemmatizationSentence segmentation and the removal of punctuation may help in sentence-level analysis while working with textual data. But what if we…
Arion Das·Aug 13, 2024Sentence SegmentationThis is simply, as the name suggests — breaking up the text into sentences at the appearance of full stops, questions marks, exclamation…