<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Romee Panchal on Medium]]></title>
        <description><![CDATA[Stories by Romee Panchal on Medium]]></description>
        <link>https://medium.com/@romeepanchal?source=rss-9f51633454f8------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*NqNW_va1ZdNeUixUtLNzRA.jpeg</url>
            <title>Stories by Romee Panchal on Medium</title>
            <link>https://medium.com/@romeepanchal?source=rss-9f51633454f8------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 10:30:31 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@romeepanchal/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[From Word Vectors to Reasoning Models: The Engineering Evolution of NLP]]></title>
            <link>https://medium.com/@romeepanchal/from-word-vectors-to-reasoning-models-the-engineering-evolution-of-nlp-e9d7148aab17?source=rss-9f51633454f8------2</link>
            <guid isPermaLink="false">https://medium.com/p/e9d7148aab17</guid>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Romee Panchal]]></dc:creator>
            <pubDate>Tue, 15 Jul 2025 09:27:00 GMT</pubDate>
            <atom:updated>2025-07-15T09:27:00.069Z</atom:updated>
            <content:encoded><![CDATA[<h3>Understanding the Technical Trajectory</h3><p>Modern NLP development follows a clear pattern: each breakthrough emerges from solving concrete computational bottlenecks in existing systems. For ML practitioners, understanding this progression means recognising which techniques solve which problems, when to apply them, and how to implement them effectively.</p><p>This analysis traces the technical lineage from static embeddings to current reasoning models, focusing on the engineering decisions that shaped each advance and their practical implications for deployment.</p><h3>Foundational Context: The Attention Revolution (2017)</h3><p>Before diving into the relay race, practitioners must understand the baseline: <strong>Vaswani et al.’s “Attention Is All You Need” (2017)</strong> eliminated recurrence, replacing LSTMs with parallelizable self-attention mechanisms. This architectural shift enabled the scaling that made everything else possible.</p><p>Self-attention’s O(n²) complexity was acceptable for sequences up to 512 tokens, but the parallelisation gains on modern hardware outweighed the quadratic cost for most practical applications.</p><p>The original Transformers architecture’s success stemmed from three specific design choices:</p><ol><li>Multi-head attention for capturing diverse relationships</li><li>Positional encodings for sequence order</li><li>Layer normalisation for training stability.</li></ol><h3>Global and Contextual Representation: Solving the Static Embedding Problem</h3><h4>The Static Limitation</h4><p><strong>GloVe (2014)</strong> demonstrated that global co-occurrence statistics outperformed local window methods (word2vec) for analogy tasks. However, both approaches suffered from the same fundamental flaw: one vector per word type, regardless of context.</p><h4>Dynamic Contextualization</h4><p><strong>ELMo (2018)</strong> introduced the first practical solution: bidirectional LSTM language models producing context-dependent representations. The key engineering insight was layer-weighted combinations — different layers captured different linguistic phenomena (syntax vs. semantics).</p><p>ELMo’s architecture stacked bidirectional LSTMs and learned task-specific weightings across layers. This approach improved downstream tasks by 10–15% but required separate training for each target domain.</p><h3>The Transformer Advantage</h3><p><strong>BERT (2018)</strong> applied the Transformer architecture to bidirectional language modelling through masked language modelling (MLM). The critical innovation was joint training on bidirectional context with a single model architecture.</p><p>MLM’s random masking strategy (15% of tokens) provided a sufficient training signal while avoiding information leakage. The [CLS] token aggregation mechanism enabled straightforward fine-tuning for classification tasks.</p><p><strong>RoBERTa (2019)</strong> validated a crucial hypothesis: architecture improvements often matter less than training methodology. By removing Next Sentence Prediction, increasing batch sizes, and extending training duration, RoBERTa achieved superior performance with identical architecture.</p><p>Optimisation details frequently outweigh architectural novelty. Modern practitioners should prioritise training efficiency before architectural complexity.</p><h3>Scaling Context: Solving the Quadratic Bottleneck</h3><h4>The Length Problem</h4><p>Transformer self-attention’s O(n²) complexity became prohibitive for sequences exceeding 1,000 tokens. This limitation blocked applications requiring long-form understanding: document analysis, code generation, and multi-turn conversations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*m_0Iq31Yfwn2OkRf.png" /><figcaption>Transformer (Vaswani et al., 2017). Image source: <a href="https://arxiv.org/abs/2009.06732">https://arxiv.org/abs/2009.06732</a></figcaption></figure><h4>Recurrence Revival</h4><p><strong>Transformer-XL (2019)</strong> reintroduced recurrence through segment-level memory while maintaining attention benefits. The relative positional encoding scheme enabled consistent performance across arbitrary sequence lengths.</p><p>Cached hidden states from previous segments provided extended context without recomputation. The relative position bias eliminated the absolute position limitations of standard Transformers.</p><h4>Attention Efficiency</h4><p>Multiple approaches emerged to reduce attention complexity:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_khiCZuNg7PZiS9V.png" /><figcaption>Taxonomy of Efficient Transformer Architectures. Image source: <a href="https://arxiv.org/pdf/2009.06732">https://arxiv.org/pdf/2009.06732</a></figcaption></figure><p><strong>Linformer (2020)</strong> projected keys and values to fixed-dimension spaces, reducing complexity to O(nk) where k &lt;&lt; n. This approach maintained performance while enabling longer sequences.</p><p><strong>Longformer (2020)</strong> combined local sliding-window attention with sparse global attention patterns. Documents up to 8,192 tokens became tractable on standard hardware.</p><p>These approaches enabled document-level processing but required careful tuning of sparsity patterns for optimal performance.</p><h3>Task Unification and Scale: The Generative Paradigm</h3><h4>Universal Task Formulation</h4><p><strong>T5 (2020)</strong> demonstrated that diverse NLP tasks could be unified under a single text-to-text framework. Every problem became a sequence generation with appropriate prefixes and formatting.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*izgWF8hBNv12kYtM.png" /><figcaption>Text-To-Text Transfer Transformer (T5). Image source: <a href="https://jmlr.org/papers/volume21/20-074/20-074.pdf">https://jmlr.org/papers/volume21/20-074/20-074.pdf</a></figcaption></figure><p>Task unification eliminated the need for task-specific architectures. A single model could handle translation, summarisation, and question answering through prompt engineering.</p><p>T5’s success required careful prompt design and massive clean training data (750GB Colossal Clean Crawled Corpus). The seq2seq architecture enabled flexible output generation.</p><h4>Scaling Laws and Emergence</h4><p><strong>GPT-3 (2020)</strong> validated the scaling hypothesis: larger models exhibit emergent capabilities without explicit training. The 175B parameter autoregressive model demonstrated in-context learning — task performance from examples alone.</p><p><strong>Kaplan et al. (2020)</strong> quantified these phenomena through power-law scaling relationships. Model performance scales predictably with compute, data, and parameters, providing ROI frameworks for resource allocation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*FXnPB25AVz8VDpLN.png" /><figcaption>Image from <a href="https://arxiv.org/pdf/2001.08361.pdf">https://arxiv.org/pdf/2001.08361.pdf</a></figcaption></figure><p>In-context learning emerged from scale, not architectural innovation. This finding shifted focus from task-specific fine-tuning to prompt engineering and few-shot learning.</p><h3>Human Alignment: Making Models Useful</h3><h4>The Usability Problem</h4><p>Raw language models optimise for text completion, not human intent. GPT-3’s impressive capabilities required careful prompt engineering and few-shot examples to achieve useful behaviour. This complexity limited practical deployment and user adoption.</p><h4>The Instruction Tuning Breakthrough</h4><p><strong>FLAN (2022)</strong> demonstrated that instruction tuning alone could dramatically improve zero-shot performance. By fine-tuning language models on diverse tasks formatted as natural language instructions, FLAN substantially enhanced performance and surpassed zero-shot GPT-3 on 20 of 25 evaluated tasks.</p><p>The key insight was task diversity — training on 62 text-to-text tasks clustered across reasoning, natural language inference, reading comprehension, and closed-book QA. This multi-task instruction tuning enabled zero-shot generalisation to unseen tasks.</p><p>Instruction tuning substantially improved zero-shot performance on unseen tasks, eliminating the need for complex few-shot prompting in many applications. Models became immediately useful for diverse tasks without task-specific examples.</p><h4>The RLHF Solution</h4><p><strong>InstructGPT (2022)</strong> was built on instruction tuning foundations through a three-stage process:</p><ol><li><strong>Supervised Fine-tuning (SFT)</strong>: Starting with labeller-written prompts and API-submitted prompts, collected demonstrations of desired model behaviour for supervised learning fine-tuning.</li><li><strong>Reward Modelling</strong>: Learn human preferences from comparative rankings.</li><li><strong>Policy Optimisation</strong>: Use PPO to maximise reward model scores.</li></ol><p>The first step was exactly instruction tuning — supervised fine-tuning on a dataset of natural language instructions. RLHF enhanced this foundation rather than replacing it.</p><p>SFT and RLHF unlock capabilities that pretrained models already have but are hard for users to access via prompting alone. The reward model architecture typically mirrors the base model but outputs scalar values.</p><p>This combined approach enabled ChatGPT’s deployment and established the standard alignment pipeline for production systems.</p><h3>Current Frontiers: 2024–2025 Developments</h3><h4>Reasoning Models</h4><p>OpenAI introduced the o1 model in September 2024, followed by o3 in April 2025, marking a shift toward explicit reasoning capabilities. These models are trained to generate step-by-step solutions before providing final answers, similar to human problem-solving processes.</p><p>OpenAI o3 is designed for complex, multi-step problem-solving across coding, math, science, and visual tasks, representing a departure from pure pattern matching toward structured reasoning.</p><h4>Mixture of Experts Evolution</h4><p>Mixture of Experts (MoE) models provide highly efficient deep learning by activating only relevant experts for each task, allowing complex operations with reduced computational resources. Modern implementations like DeepSeek-R1 with 671B parameters and 37B activated parameters per token demonstrate practical scaling strategies.</p><p>MoE architectures enable massive parameter counts while maintaining inference efficiency. Only a subset of parameters activate for each input, reducing computational requirements.</p><h4>Multimodal Integration</h4><p>Current systems extend beyond text to incorporate visual, audio, and code modalities. Models like GPT-4V and Claude 3.5 Sonnet handle diverse input types within unified architectures.</p><p>Cross-modal alignment requires careful training strategies to maintain performance across modalities without interference.</p><h4>Extended Context Windows</h4><p>Recent developments include LLaMA 4 in April 2025 and other models supporting context windows exceeding 100,000 tokens, enabling document-level analysis and extended conversations.</p><p>Modern approaches combine efficient attention mechanisms with hierarchical processing to manage computational costs while maintaining coherent long-range dependencies.</p><h3>Practical Implementation Guidance</h3><h4>Choosing the Right Approach</h4><p><strong>For Representation Tasks</strong>: Use pre-trained BERT variants (RoBERTa, DeBERTa) for classification, named entity recognition, and information extraction. Fine-tune on domain-specific data for optimal performance.</p><p><strong>For Generation Tasks</strong>: Implement T5-based approaches for structured generation (summarisation, translation) or GPT-style models for open-ended generation. Consider instruction-tuned variants for better human alignment.</p><p><strong>For Long Documents</strong>: Apply Longformer or recent models with extended context windows. For extremely long sequences, consider retrieval-augmented generation (RAG) approaches.</p><p><strong>For Instruction Following</strong>: Start with instruction tuning using diverse task formats before considering RLHF. FLAN-style multi-task instruction tuning dramatically improves zero-shot performance and reduces prompting complexity. Add RLHF only when alignment with human preferences becomes critical.</p><h4>Current Best Practices</h4><p><strong>Data Preparation</strong>: Clean, deduplicate, and carefully curate training data. Data quality impacts model performance more than architecture choices.</p><p><strong>Training Efficiency</strong>: Use gradient checkpointing, mixed precision, and efficient optimisers (AdamW with warmup schedules). Monitor training dynamics through loss curves and validation metrics.</p><p><strong>Evaluation Strategy</strong>: Implement a comprehensive evaluation across multiple dimensions: accuracy, robustness, efficiency, and alignment with human preferences.</p><p><strong>Deployment Considerations</strong>: Plan for hardware requirements, latency constraints, and scaling patterns. Consider serving optimisations like quantisation and caching.</p><h4>Emerging Bottlenecks</h4><p><strong>Reasoning Consistency</strong>: Current models excel at pattern matching but struggle with multi-step logical reasoning. The o-series models represent early attempts at systematic reasoning.</p><p><strong>Factual Grounding</strong>: Models frequently generate plausible but incorrect information. Retrieval-augmented generation and explicit knowledge bases address this limitation.</p><p><strong>Efficiency at Scale</strong>: Training costs grow exponentially with model size. MoE architectures and parameter-efficient fine-tuning methods provide scaling alternatives.</p><h4>Implementation Implications</h4><p><strong>Architecture Selection</strong>: Choose architectures based on specific bottlenecks rather than general performance metrics. Understand the trade-offs between efficiency, accuracy, and deployment complexity.</p><p><strong>Training Strategy</strong>: Invest in data quality and training methodology before scaling model size. Proper optimisation often outperforms architectural complexity.</p><p><strong>Evaluation Framework</strong>: Develop comprehensive evaluation protocols that assess reasoning, factual accuracy, and alignment, not just traditional NLP metrics.</p><h3>Conclusion: Engineering Lessons for Practitioners</h3><h4>Core Principles</h4><ol><li><strong>Bottleneck-Driven Innovation</strong>: Each major advance solved specific computational or representational limitations. Identify your system’s bottlenecks before selecting techniques.</li><li><strong>Implementation Over Innovation</strong>: Training methodology, data quality, and optimisation details frequently matter more than architectural novelty. Master the fundamentals before pursuing cutting-edge approaches.</li><li><strong>Systematic Evaluation</strong>: Comprehensive evaluation across multiple dimensions prevents overfitting to specific benchmarks. Consider efficiency, robustness, and alignment alongside accuracy.</li></ol><h4>Insights</h4><p>The NLP field’s rapid evolution continues, but the underlying pattern remains consistent: identify bottlenecks, develop targeted solutions, and validate through systematic evaluation. Understanding this progression provides the foundation for navigating future developments and making informed technical decisions.</p><p>Success in modern NLP requires balancing cutting-edge techniques with engineering practicality. Master the fundamentals, understand the trade-offs, and choose approaches that solve your specific problems — not just the latest research trends.</p><h3>References</h3><ol><li>Hochreiter, S., &amp; Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation, 9(8), 1735–1780.</li><li>Pennington, J., Socher, R., &amp; Manning, C. D. (2014). GloVe: Global Vectors for Word Representation. EMNLP.</li><li>Peters, M. E., et al. (2018). Deep Contextualized Word Representations. NAACL.</li><li>Devlin, J., et al. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. NAACL.</li><li>Liu, Y., et al. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv:1907.11692.</li><li>Dai, Z., et al. (2019). Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context. ACL.</li><li>Wang, S., et al. (2020). Linformer: Self-Attention with Linear Complexity. arXiv:2006.04768.</li><li>Beltagy, I., Peters, M., &amp; Cohan, A. (2020). Longformer: The Long-Document Transformer. arXiv:2004.05150.</li><li>Raffel, C., et al. (2020). Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. JMLR, 21(140).</li><li>Brown, T. B., et al. (2020). Language Models are Few-Shot Learners. NeurIPS.</li><li>Kaplan, J., et al. (2020). Scaling Laws for Neural Language Models. arXiv:2001.08361.</li><li>Ouyang, L., et al. (2022). Training Language Models to Follow Instructions with Human Feedback. arXiv:2203.02155.</li><li>Radford, A., et al. (2021). Learning Transferable Visual Models from Natural Language Supervision. ICML.</li><li>Gu, A., et al. (2021). Efficiently Modeling Long Sequences with Structured State Spaces. arXiv:2111.00396.</li><li>Fedus, W., et al. (2021). Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity. JMLR, 23(120).</li><li>Borgeaud, S., et al. (2022). Improving Language Models by Retrieving from Trillions of Tokens. Nature, 601, 590–595.</li><li>Tay, Y., et al. (2022). Efficient Transformers: A Survey. ACM Computing Surveys, 55(6), 1–28.</li><li>Heidloff, N. (2025). Key Concepts of DeepSeek-R1. Heidloff</li><li>Modular AI. (2025). Exploring DeepSeek-R1’s Mixture-of-Experts Architecture. Modular</li><li>Meta AI. (2025). LLaMA 4 release blog &amp; documentation. AiWire</li><li>OpenAI. (2025). Introducing GPT-4.1 in the API. OpenAI</li><li>Reuters. (2025, Apr 14). OpenAI launches GPT-4.1 models with improved coding &amp; long context. Reuters</li><li>Anthropic. (2024). Introducing Claude 3.5 Sonnet. Anthropic</li><li><a href="https://huggingface.co/blog/long-range-transformers">Huggingface. Long-range Transformers</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e9d7148aab17" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From LSTMs to RLHF — How One Idea Ignites the Next]]></title>
            <link>https://medium.com/@romeepanchal/from-lstms-to-rlhf-how-one-idea-ignites-the-next-8cbbe3fd87b2?source=rss-9f51633454f8------2</link>
            <guid isPermaLink="false">https://medium.com/p/8cbbe3fd87b2</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Romee Panchal]]></dc:creator>
            <pubDate>Sat, 12 Jul 2025 07:32:56 GMT</pubDate>
            <atom:updated>2025-07-12T07:36:08.818Z</atom:updated>
            <content:encoded><![CDATA[<h3>From LSTMs to RLHF — How One Idea Ignites the Next</h3><h3>Innovation as a Relay Race</h3><p>Every landmark in modern NLP began as a modest answer to a concrete limitation. One group publishes a clever fix, another notices the remaining crack, and a third discovers a shortcut the first two never imagined. LSTMs rescued RNNs from vanishing gradients; contextual embeddings made static vectors obsolete; transformers smashed the parallel-computing ceiling; instruction-tuned models closed the human-alignment gap.</p><p>Tracking that relay race is more instructive than memorising any single result: it shows <em>why</em> each breakthrough mattered and <em>how</em> the field’s centre of gravity kept shifting.</p><p>Below is a narrative — rather than a checklist — of the papers that turned yesterday’s impossibilities into today’s defaults.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Eqab6Vxxk7FIL6ea" /><figcaption>Photo by <a href="https://unsplash.com/@mangofantasy?utm_source=medium&amp;utm_medium=referral">Tim Johnson</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Global and Contextual Word Representation</h3><p><strong>GloVe (2014)</strong> argued that local window-based prediction (word2vec) ignores global statistics, so it factorises a log-co-occurrence matrix instead. The jump in analogy accuracy hinted that richer context <em>matters</em>.</p><p>Four years later, <strong>ELMo (2018)</strong> showed an even bigger flaw: <em>static</em> embeddings cannot capture homonyms or polysemy. By feeding sentences through a bi-directional LSTM language model and taking layer-weighted mixtures, ELMo made word meaning conditional on the whole sequence — fuel for the first double-digit gains across QA and coreference tasks.</p><p>That insight reached escape velocity with <strong>BERT (2018)</strong>. Replace RNNs with Transformers, mask 15 % of tokens, predict them using bidirectional context, and you have a universal encoder that can be fine-tuned for almost anything. <strong>RoBERTa (2019)</strong> then asked a heretical question: <em>What if BERT’s magic is just large batches, more data, and longer training?</em> By stripping away Next-Sentence Prediction and scaling aggressively, it proved that optimisation tricks sometimes trump architecture tweaks.</p><h3>Remembering More Than a Page</h3><p>As soon as Transformers won short-sequence tasks, researchers hit the quadratic wall: self-attention’s cost explodes with length. <strong>Transformer-XL (2019)</strong> revived recurrence — storing hidden states from the previous segment and using a relative positional scheme so that textual horizons could extend from 512 tokens to thousands. The leap mattered for story generation and long-form language modelling.</p><p>While Transformer-XL kept the full attention pattern, others chased sparsity. <strong>Linformer (2020)</strong> projected keys and values to a low-rank space, shrinking complexity from <em>O(n²)</em> to <em>O(nk)</em>; <strong>Longformer (2020)</strong> combined sliding-window locality with a few global tokens to process 8 k-token scientific papers on a single GPU. By 2022, Tay et al.’s survey catalogued more than 50 such variants, signalling consensus that hardware — not theory — is now the bottleneck.</p><h3>Unifying Tasks and Upsizing Everything</h3><p>If context length was one ceiling, task-specific heads were another. <strong>T5 (2020)</strong> demolished both by casting <em>all</em> problems — translation, summarisation, classification — as “input text -&gt; output text.” With a cleaned 750 GB corpus and a seq-to-seq Transformer, it showed that careful task formatting can unlock as much transfer as bigger models.</p><p>Speaking of bigger: <strong>GPT-3 (2020)</strong> asked what happens when you simply scale an autoregressive Transformer to 175 B parameters and skip fine-tuning. The answer was <em>in-context learning</em> — the ability to perform new tasks from a few demonstration prompts. <strong>Kaplan et al. (2020)</strong> then put numbers on the intuition, deriving power-law scaling rules that still guide GPU-budget spreadsheets at every lab.</p><h3>Aligning Models with Humans</h3><p>Raw capability is useless — or dangerous — if a model ignores human intent. <strong>Ouyang et al. (2022)</strong> introduced the now-canonical RLHF pipeline: supervised instruction tuning, a reward model trained on preference comparisons, and PPO reinforcement learning to steer generation. That recipe underpins ChatGPT and every serious deployment concerned with safety, helpfulness, or brand tone.</p><h3>Emerging Frontiers</h3><p>With text well-tamed, attention is turning to the edges: multimodality (<strong>CLIP</strong>, 2021), 100 k-token memory via state-space models (<strong>S4</strong>, 2021), retrieval-augmented generation for factual reliability (<strong>RETRO</strong>, 2022), and sparse mixtures-of-experts to grow capacity without quadratic FLOPs (<strong>Switch Transformer</strong>, 2021). Each frontier picks up where the last breakthrough shows strain — proof that the relay race continues.</p><p>Then there is a whole new race of reasoning models.</p><ul><li><strong>Reasoning-first models — OpenAI o1 / o3. </strong>Instead of squeezing ever more tokens into a single forward pass, these variants <em>slow the clock</em> and explicitly allocate extra computation to step-by-step reasoning, tool use and self-reflection. The result is higher accuracy on multi-hop problems and code synthesis without a parameter-count arms race. <a href="https://www.datacamp.com/tutorial/o3-api?utm_source=chatgpt.com">DataCamp</a><a href="https://www.techtarget.com/whatis/feature/OpenAI-o3-explained-Everything-you-need-to-know?utm_source=chatgpt.com">TechTarget</a></li><li><strong>Next-gen Mixture-of-Experts — DeepSeek-R1 (671 B total / 37 B active). </strong>DeepSeek shows MoE is no longer an exotic Google-only trick: by activating a tiny slice of the network per token, it achieves GPT-4-class math-and-coding scores while running on commodity H800s — an order-of-magnitude cut in training FLOPs. <a href="https://heidloff.net/article/deepseek-r1/?utm_source=chatgpt.com">Heidloff</a><a href="https://www.modular.com/ai-resources/exploring-deepseek-r1-s-mixture-of-experts-model-architecture?utm_source=chatgpt.com">Modular</a><a href="https://www.wsj.com/tech/ai/deepseek-ai-how-it-works-725cb464?utm_source=chatgpt.com">The Wall Street Journal</a></li><li><strong>Open-source scale — LLaMA 4 (April 2025). </strong>Meta’s fourth-generation family adds two open-weights variants (Maverick &amp; Scout) that match GPT-4-level benchmarks while remaining fine-tune-friendly. For academics and start-ups, it resets the “free-to-tinker” baseline. <a href="https://www.aiwire.net/2025/04/08/meta-unleashes-new-llama-4-ai-models/?utm_source=chatgpt.com">AiWire</a><a href="https://en.wikipedia.org/wiki/Llama_%28language_model%29?utm_source=chatgpt.com">Wikipedia</a></li><li><strong>Ultra-long context — GPT-4.1 (up to 1 M tokens). </strong>The new flagship pushes context length from 128 k to a <em>million</em> tokens, enough to hold eight copies of the entire React codebase or a week of Slack history. It also closes the coding-accuracy gap with domain-specific copilots. <a href="https://openai.com/index/gpt-4-1/?utm_source=chatgpt.com">OpenAI</a><a href="https://www.techtarget.com/whatis/feature/GPT-41-explained-Everything-you-need-to-know?utm_source=chatgpt.com">TechTarget</a><a href="https://www.reuters.com/technology/artificial-intelligence/openai-launches-new-gpt-41-models-with-improved-coding-long-context-2025-04-14/?utm_source=chatgpt.com">Reuters</a></li><li><strong>Vision-enhanced alignment — Claude 3.5 Sonnet. </strong>Anthropic’s mid-sized model leap-frogs its own Claude 3 Opus on vision and reasoning tasks, accurately reading charts, receipts and low-quality images — critical for retail and logistics workflows. <a href="https://www.anthropic.com/news/claude-3-5-sonnet?utm_source=chatgpt.com">Anthropic</a><a href="https://www.datacamp.com/blog/claude-sonnet-anthropic?utm_source=chatgpt.com">DataCamp</a></li></ul><p>Each effort tackles the pressure points revealed by the previous wave. These additions reinforce the same relay pattern we’ve seen:</p><ol><li><strong>Identify a practical ceiling</strong> (reasoning depth, training cost, openness, context window, multimodal grounding).</li><li><strong>Engineer a targeted workaround</strong> (scheduled reflection, sparse experts, permissive licensing, memory-efficient attention, vision encoders).</li><li><strong>Watch the workaround create its follow-up questions</strong> (how to debug hidden experts? how to search a million-token prompt?).</li></ol><h3>Conclusion — What This Journey Teaches</h3><ol><li><strong>Progress is iterative but not incremental.</strong> A single clever hack (masking, recurrence, RLHF) can open a new design space.</li><li><strong>Bottlenecks create research themes.</strong> Once representation quality plateaued, context length and compute cost became the urgent problems; now, alignment and grounding dominate.</li><li><strong>Adopt selectively.</strong> Map your pain point — representation, context, scale, or alignment — to the paper that solved it. Implement that solution before chasing the next hype wave.</li></ol><h3>References</h3><ol><li>Hochreiter, S., &amp; Schmidhuber, J. (1997). <em>Long Short-Term Memory</em>. <strong>Neural Computation, 9</strong>(8), 1735–1780.</li><li>Pennington, J., Socher, R., &amp; Manning, C. D. (2014). <em>GloVe: Global Vectors for Word Representation</em>. EMNLP.</li><li>Peters, M. E., et al. (2018). <em>Deep Contextualized Word Representations</em>. NAACL.</li><li>Devlin, J., et al. (2018). <em>BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding</em>. NAACL.</li><li>Liu, Y., et al. (2019). <em>RoBERTa: A Robustly Optimized BERT Pretraining Approach</em>. arXiv:1907.11692.</li><li>Dai, Z., et al. (2019). <em>Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context</em>. ACL.</li><li>Wang, S., et al. (2020). <em>Linformer: Self-Attention with Linear Complexity</em>. arXiv:2006.04768.</li><li>Beltagy, I., Peters, M., &amp; Cohan, A. (2020). <em>Longformer: The Long-Document Transformer</em>. arXiv:2004.05150.</li><li>Raffel, C., et al. (2020). <em>Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer</em>. <strong>JMLR, 21</strong>(140).</li><li>Brown, T. B., et al. (2020). <em>Language Models are Few-Shot Learners</em>. NeurIPS.</li><li>Kaplan, J., et al. (2020). <em>Scaling Laws for Neural Language Models</em>. arXiv:2001.08361.</li><li>Ouyang, L., et al. (2022). <em>Training Language Models to Follow Instructions with Human Feedback</em>. arXiv:2203.02155.</li><li>Radford, A., et al. (2021). <em>Learning Transferable Visual Models from Natural Language Supervision</em>. ICML.</li><li>Gu, A., et al. (2021). <em>Efficiently Modeling Long Sequences with Structured State Spaces</em>. arXiv:2111.00396.</li><li>Fedus, W., et al. (2021). <em>Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity</em>. <strong>JMLR, 23</strong>(120).</li><li>Borgeaud, S., et al. (2022). <em>Improving Language Models by Retrieving from Trillions of Tokens</em>. <strong>Nature, 601</strong>, 590–595.</li><li>Tay, Y., et al. (2022). <em>Efficient Transformers: A Survey</em>. <strong>ACM Computing Surveys, 55</strong>(6), 1–28.</li><li>Heidloff, N. (2025). <em>Key Concepts of DeepSeek-R1</em>. <a href="https://heidloff.net/article/deepseek-r1/?utm_source=chatgpt.com">Heidloff</a></li><li>Modular AI. (2025). <em>Exploring DeepSeek-R1’s Mixture-of-Experts Architecture</em>. <a href="https://www.modular.com/ai-resources/exploring-deepseek-r1-s-mixture-of-experts-model-architecture?utm_source=chatgpt.com">Modular</a></li><li>Meta AI. (2025). <em>LLaMA 4 release blog &amp; documentation</em>. <a href="https://www.aiwire.net/2025/04/08/meta-unleashes-new-llama-4-ai-models/?utm_source=chatgpt.com">AiWire</a></li><li>OpenAI. (2025). <em>Introducing GPT-4.1 in the API</em>. <a href="https://openai.com/index/gpt-4-1/?utm_source=chatgpt.com">OpenAI</a></li><li>Reuters. (2025, Apr 14). <em>OpenAI launches GPT-4.1 models with improved coding &amp; long context</em>. <a href="https://www.reuters.com/technology/artificial-intelligence/openai-launches-new-gpt-41-models-with-improved-coding-long-context-2025-04-14/?utm_source=chatgpt.com">Reuters</a></li><li>Anthropic. (2024). <em>Introducing Claude 3.5 Sonnet</em>. <a href="https://www.anthropic.com/news/claude-3-5-sonnet?utm_source=chatgpt.com">Anthropic</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8cbbe3fd87b2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Finding Light in Unpredictability]]></title>
            <link>https://medium.com/@romeepanchal/finding-light-in-unpredictability-18a9c7599445?source=rss-9f51633454f8------2</link>
            <guid isPermaLink="false">https://medium.com/p/18a9c7599445</guid>
            <category><![CDATA[life-lessons]]></category>
            <category><![CDATA[life]]></category>
            <dc:creator><![CDATA[Romee Panchal]]></dc:creator>
            <pubDate>Mon, 07 Oct 2024 04:08:45 GMT</pubDate>
            <atom:updated>2024-10-07T04:08:45.106Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jxLoAYQtpTHgayL8RPYaWw.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@peterconlan">Peter Conlan</a> on <a href="https://unsplash.com/">Unsplash</a></figcaption></figure><p>One of the most important facts to understand in life is that you don’t always get what you want. You can hope to achieve things in life. You can even work harder than others. However, there are factors beyond our control that will affect the overall outcome. Those factors play a significant role in our life.</p><p>But why am I writing about this? I think, when life goes smoothly, we rarely question things. We feel content, and everything seems to be going according to plan. Why worry at such a time, right? But as soon as things do not go according to our plan, then we realise the more realistic world around us. We suddenly see things from a philosophical angle.</p><p>Recently, I have been thinking that everything in the world is going well. Then BAM. You get a reality check. You get a reminder. Nothing is permanent. Even if you work your ass off towards something, if you are unlucky, you may not get the desired results.</p><p>This realisation seems negative at first, doesn’t it? Seems like things can go south at the most crucial time of your life and what you worked for so long will completely go to waste. But that is not the case. I once read that with determination (and a positive outlook), you can find light even in the darkest hour. If nothing is permanent in life then the same applies to our bad fate. It will also change one day and come to work with us. That day will be awesome. Eventually, circumstances will align in our favour, and our efforts will yield results.</p><p>So, what’s my takeaway from this realisation?</p><p>First of all, that writing helps a lot. Before writing this article, I was in a questioning mode. But as I was writing this article, things started to become a little bit clearer, which is a big mental shift by the way.</p><p>Second, nothing is permanent in life. It applies both ways. When things are good and when things are bad. This understanding offers hope in bad times and fosters gratefulness in the good times.</p><p>Third, luck plays a big role in life. However, this does not mean we should stop making efforts. In fact, use it to your advantage. You never know when good luck will knock on your door. Keep putting in effort and be ready for that moment.</p><p>In conclusion, life’s unpredictability can be both challenging and exciting. By acknowledging this reality and maintaining a balanced perspective, we can navigate life’s ups and downs with resilience and optimism.</p><p><em>This story was originally posted on my blog </em><a href="https://romeepanchal.com/">romeepanchal.com</a>. <em>Follow me to get the stories related to AI and Machine Learning.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=18a9c7599445" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why You Should Learn Mathematics Even if You Have Experience in Building ML Models]]></title>
            <link>https://medium.com/@romeepanchal/why-you-should-learn-mathematics-even-if-you-have-experience-in-building-ml-models-2ec596a1886b?source=rss-9f51633454f8------2</link>
            <guid isPermaLink="false">https://medium.com/p/2ec596a1886b</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[mathematics]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Romee Panchal]]></dc:creator>
            <pubDate>Fri, 13 Sep 2024 08:08:09 GMT</pubDate>
            <atom:updated>2024-09-13T08:08:09.189Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>How Mathematical Mastery Enhances Your Machine Learning Success</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Ce6JkM1FPRcBO6dG" /><figcaption>Photo by <a href="https://unsplash.com/@jplenio?utm_source=medium&amp;utm_medium=referral">Johannes Plenio</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>In recent years, machine learning (ML) has rapidly advanced and become a key component of modern technology. Whether applied to recommendation systems, autonomous driving, healthcare, or finance, machine learning’s influence is undeniable. Many practitioners in the field come from diverse backgrounds, sometimes with limited formal mathematical training but strong programming and data manipulation skills. While this might be sufficient to implement machine learning models using popular libraries like TensorFlow, PyTorch, or scikit-learn, there is a compelling case for deepening one’s understanding of mathematics even when you already have hands-on experience in building models. Mathematics provides the theoretical foundation for machine learning and enables practitioners to elevate their work, enhance model performance, and solve complex problems with greater insight and creativity.</p><p>This essay will explore several key reasons why learning mathematics is essential for machine learning professionals. It will cover the role of math in understanding model behaviour, optimizing performance, creating new models, avoiding common pitfalls, and effectively communicating ideas.</p><h3>Understanding Model Behaviour and Inner Workings</h3><p>Machine learning models, particularly complex ones like deep neural networks, are often treated as black boxes by those who lack mathematical training. With strong libraries available to abstract much of the underlying complexity, it’s possible to build and deploy models without fully understanding how they work. However, this is a risky approach. Without a solid grasp of the mathematical principles that govern these algorithms, you might miss important insights into their behaviour, limitations, and potential failure modes.</p><p>For example, linear regression is a simple algorithm used frequently in ML. On the surface, it seems straightforward: find the best fit line for a given set of data points. But what does “best fit” mean? How does the algorithm determine this? The answer lies in concepts from calculus (gradient descent) and linear algebra (matrix operations). Understanding these mathematical concepts allows you to grasp how the model makes predictions, why it behaves the way it does, and how to interpret the coefficients and performance metrics meaningfully.</p><p>By studying the underlying mathematics, you can understand not just <em>how</em> models like neural networks or support vector machines (SVMs) work, but <em>why</em> they behave the way they do under different conditions. This depth of understanding empowers you to make better decisions when tuning models, selecting algorithms, or designing architectures.</p><h3>Optimising Model Performance</h3><p>When building machine learning models, practitioners often focus on improving metrics like accuracy, precision, or recall. Achieving high performance typically involves hyperparameter tuning, feature engineering, and experimenting with different algorithms. While practical experience helps with these tasks, a firm mathematical foundation can give you a more sophisticated toolkit for optimisation.</p><p>For instance, consider the concept of regularisation, a critical technique used to prevent overfitting. Regularisation adds a penalty term to the loss function, which discourages complex models from fitting the noise in the data. Understanding L1 and L2 regularisation in terms of norms and vector spaces can help you determine when to use them, how to adjust the penalty terms, and why they are effective in certain situations. Without this mathematical insight, regularisation might just seem like an arbitrary trick.</p><p>Similarly, optimisation algorithms like stochastic gradient descent (SGD), Adam, and RMSprop are at the heart of most modern ML models, but they operate on mathematical principles from calculus and statistics. Knowing how gradients work, why learning rates matter, and how loss surfaces behave gives you greater control over the training process, helping you avoid issues like vanishing or exploding gradients, slow convergence, or getting stuck in local minima.</p><h3>Creating and Innovating New Models</h3><p>While existing machine learning models and algorithms can handle many real-world problems, innovation in the field requires a deep understanding of mathematics. Practitioners who want to go beyond using pre-built models and instead contribute to advancing the state of the art will inevitably need to master the mathematical theories that underlie machine learning.</p><p>For example, generative models like Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs) rely on advanced topics from probability theory, linear algebra, and information theory. To truly innovate with these models, you need to understand concepts like Kullback-Leibler divergence, latent space, and Jensen-Shannon divergence. Armed with this knowledge, you can push the boundaries of current models or even create new ones better suited to specific tasks.</p><p>Moreover, learning mathematics equips you with the tools to break down complex problems and formulate new algorithms from scratch. As the field of machine learning continues to evolve, professionals with strong mathematical expertise are more likely to lead these innovations by designing algorithms that address limitations in existing methods or tackle new challenges altogether.</p><h3>Avoiding Common Pitfalls</h3><p>Machine learning models are often sensitive to the quality of the input data, and incorrect assumptions or overlooked details can lead to flawed results. A solid understanding of statistics, probability, and linear algebra helps practitioners recognize these pitfalls before they become major issues.</p><p>One common pitfall is overfitting, where a model performs well on training data but poorly on unseen data. Knowing the mathematical foundations behind model complexity and regularization allows you to effectively combat overfitting. Similarly, understanding the bias-variance tradeoff, another core statistical concept, can help you balance model flexibility with generalization.</p><p>Another example is in handling data. Matrix operations are central to many machine learning algorithms, and errors in understanding matrix dimensions, properties, or decompositions can lead to incorrect implementations. Additionally, probability theory is essential for tasks like estimating uncertainty in predictions, interpreting results, and designing models for tasks involving randomness, like reinforcement learning or Bayesian inference.</p><p>Without this mathematical foundation, you might miss subtle issues in data processing, model evaluation, or even in the assumptions of the algorithms themselves, leading to inaccurate or misleading results.</p><h3>Effective Communication and Collaboration</h3><p>Finally, mathematics is the universal language of machine learning and data science. Whether you are working in a team of engineers, collaborating with researchers, or presenting your work to stakeholders, the ability to communicate the principles behind your models is invaluable. Even though many people can understand the output of a model (such as a prediction or classification), fewer can appreciate the nuances of how that output was achieved.</p><p>By developing a solid understanding of the mathematical principles that govern machine learning models, you can effectively explain your choices, justify trade-offs, and collaborate with others who are more mathematically inclined. This skill is particularly important in interdisciplinary teams where data scientists, engineers, statisticians, and domain experts work together. Clear communication based on a shared understanding of mathematical principles ensures that the entire team can align on model goals, limitations, and capabilities.</p><h3>Conclusion</h3><p>While it is possible to build machine learning models with a basic understanding of mathematics, a deeper knowledge of mathematical principles offers numerous advantages. It allows you to understand how models work, optimise their performance, create new algorithms, avoid common pitfalls, and communicate effectively with others in the field. As machine learning continues to evolve, those with strong mathematical foundations will be better positioned to innovate, solve complex problems, and contribute meaningfully to the advancement of the field. Therefore, investing in learning mathematics is a crucial step for anyone serious about mastering machine learning.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2ec596a1886b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Starting Over: How will I Prepare for a Machine Learning Career Again]]></title>
            <link>https://medium.com/@romeepanchal/starting-over-how-will-i-prepare-for-a-machine-learning-career-again-9d3fd934383a?source=rss-9f51633454f8------2</link>
            <guid isPermaLink="false">https://medium.com/p/9d3fd934383a</guid>
            <category><![CDATA[interview-preparation]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Romee Panchal]]></dc:creator>
            <pubDate>Wed, 28 Aug 2024 13:51:29 GMT</pubDate>
            <atom:updated>2024-08-28T13:51:29.699Z</atom:updated>
            <content:encoded><![CDATA[<p>I love machine learning. It is one of the fastest-growing and most exciting fields today. The possibilities of things that we can achieve using machine learning are phenomenal. That is the reason I switched to machine learning as a software engineer.</p><p>When I first started preparing for the machine learning interviews back in 2017, I had very little information on what to do. There were limited resources available online. And I had no formal education in machine learning. I got my first data science job through a trial and error method. I put many personal projects in my resume. I applied to 100s of roles. Failed almost all those interviews. Made a list of all the questions interviewers used to ask. Prepared the answers to those questions. And, finally, got an offer.</p><p>This kind of trial-and-error method worked for me back then. However, this is not an efficient approach to finding a machine learning job. There should be a framework that can help a person like me to prepare for the machine learning interviews. After working in this field for almost 7 years and sitting on the other side of the table, I have learned a few things about ML interviews.</p><p>In this article, I will share my learnings about the machine learning preparation. My goal is to provide a step-by-step framework that anyone can follow and get a machine learning job.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PZcWx0tStr7yWuTh6vQx4A.png" /><figcaption>Make Your Way Forward</figcaption></figure><h3>Assessing Your Skills</h3><p>To begin your interview preparation journey, it is essential to identify your strengths. It’s crucial to assess your current skills and strengths. Here are some key areas:</p><ul><li><strong>Programming</strong>: Machine learning engineers need to write functional code to explore data, train models, and present results. Python is a popular choice.</li><li><strong>Data Analysis</strong>: A machine learning engineer must be able to clean, manipulate, and analyse data. Statistical concepts and visualisation methods are essential tools for data exploration.</li><li><strong>Machine Learning</strong>: Understanding machine learning and building predictive models is crucial. While it may be tempting to start with models directly, a solid foundation in the mathematical underpinnings of machine learning algorithms allows for a deeper understanding and better long-term results.</li><li><strong>Communication</strong>: ML engineers need to effectively communicate their findings to both technical and non-technical audiences.</li></ul><p>Once you’ve evaluated your skills, identify areas for improvement and focus on building a strong foundation before applying for jobs. By taking this proactive approach, you’ll be well-positioned to impress potential employers.</p><h3>Building a Solid Foundation</h3><p>After evaluating your skills, you will know skills are required for a machine learning job. Depending on the skill gap, there are several ways you can fill these gaps.</p><ol><li>Take courses.</li><li>Read books.</li><li>Participate in hackathons.</li><li>Work on personal projects.</li><li>Self-learning.</li><li>Watch YT videos.</li></ol><p>Everyone has their own strengths and weaknesses. You are the best person to tell what works for you. Some prepare by reading books. Others prefer watching YouTube videos. The main point I want to convey here is you need to fill the gaps in your current skills. Do whatever works for you.</p><p>Personally, when I decided to transition to data science, I actively participated in machine learning hackathons. From personal experience, I can tell you that such competitions are a fantastic way to learn new things from your peers, build a network in the field, and grab the attention of potential hiring managers.</p><h3>Developing a Portfolio</h3><p>One of the best ways to test your skills is to apply it. You work hard to acquire the necessary skills of an ML engineer. Now, it is time to put those skills to work to create something concrete. Find some real-world data to work on. And build something. While building you also improve your programming and problem-solving skills. This way you will immediately know what works and what does not. Focus on what works.</p><p>Once you have a few machine learning projects, create an online portfolio. You can build such a portfolio using GitHub Pages. It is free to use and easy to set up. This also showcases your skill to work with version control tools such as GitHub. Include the links to your projects, the data sources used, and the outcome of the projects.</p><p>This will show your future employer your ability to frame problems, code, analyse, and build models. Your ability to craft good stories about your projects will also showcase your communication skills.</p><h3>Creating a Compelling Resume</h3><p>Your resume is the first impression you make on a potential employer, so it’s crucial to create a compelling resume that showcases your qualifications.</p><ul><li><strong>Gather Your Content</strong>: Reflect on your education, skills, projects (including personal projects!), hackathon experiences, and any other relevant achievements.</li><li><strong>Targeted Tailoring</strong>: Use online resources to find a resume template that complements your experience level and the type of roles you’re targeting.</li><li><strong>Keyword Optimisation</strong>: Carefully review job descriptions and incorporate relevant keywords throughout your resume, especially in your skills section. This can help your resume pass applicant tracking systems (ATS) used by many companies.</li><li><strong>One Strong Base</strong>: Instead of creating several entirely different resumes, develop a strong base resume that highlights your core skills and experiences.</li><li><strong>Targeted Tweaks</strong>: When applying for specific jobs, customise your base resume by emphasizing the skills and experiences most relevant to the particular role. This targeted approach demonstrates that you understand the position’s requirements and strengthens your candidacy.</li></ul><h3>Searching and Applying for Jobs</h3><p>Once you feel comfortable with the fundamentals, start searching for relevant jobs in the market. Begin by researching companies and organisations that offer machine learning positions. Shortlist the companies that align well with your interests in the field of artificial intelligence. You can visit their career pages and apply directly.</p><p>Another way to search and apply for jobs is through career portals. Platforms such as LinkedIn, Glassdoor, and Indeed usually have jobs posted directly by company recruiters. It’s quite easy to search for jobs on these portals by applying filters and using specific keywords.</p><p>A third way to apply for a job is through referrals. You can ask your connections or online community members to refer you for roles that interest you.</p><h3>Preparing for Interviews</h3><p>Each company has its own selection criteria. Regardless of the number of rounds or exact interview format, all hiring managers look for the following qualities in a candidate:</p><ol><li>Technical Skills — Familiarity with particular technologies to do the job effectively.</li><li>Problem-Solving Skills — Ability to solve complex problems with minimal guidance.</li><li>Communication Skills — Ability to communicate your ideas effectively with other team members and stakeholders.</li></ol><p>Let’s look into each of these areas:</p><h4>Technical Skills</h4><p>For technical skills, there are two main aspects: 1) the tech stack and 2) conceptual understanding.</p><p>Hiring managers are interested in knowing your comfort level with the tech stack they work with. They may judge you based on your proficiency with particular software packages. For example, if you’re applying for an AI engineer role, you need to know deep learning frameworks like PyTorch or TensorFlow. For machine learning, you need to know Scikit-learn, NumPy, Pandas, etc. You may be asked questions related to the usage of these technologies. So, check the tech stack used for the role you’re applying for and prepare accordingly.</p><p>The second aspect you may be judged on is conceptual understanding. Interviewers may ask questions to gauge your comfort level with particular topics. For example, for a data science role, they might ask questions related to machine learning, probability, statistics, or mathematics.</p><p>The ability to code is also very important for data science and machine learning roles. Hence, it’s crucial to know the fundamental topics of your programming language. For example, in Python, you need to know about lists, tuples, dictionaries, OOP concepts, iterators, and generators.</p><h4>Problem-Solving Skills</h4><p>Hiring managers want to recruit people who can solve complex problems. To check problem-solving skills, different companies interview candidates in various ways. Typically, they conduct one or more rounds of coding and algorithmic problem-solving interviews. Based on the seniority levels, they may also conduct case study and/or system design interviews.</p><p>Prepare for coding problem-solving at least a month before. You can practice on platforms like LeetCode or HackerRank.</p><p>Similarly, prepare for case studies or design interviews. You usually learn these skills while working. There’s no shortcut to improving these skills. However, you can read examples of case studies and system designs for many different problems.</p><h4>Communication Skills</h4><p>Finally, communication skills are equally important. Even if a candidate has all the technical knowledge required for a role, they might be rejected if they fail to communicate clearly with the interviewers. Within companies, projects are team efforts. You need to communicate regularly within your team as well as with other teams. In a data science role, you often need to present your model’s output to stakeholders. Hence, communication skills are crucial.</p><p>Interviewers don’t typically have a round focused solely on communication skills. Instead, they evaluate your communication skills throughout the entire interview process.</p><p>For experienced candidates, hiring managers like to know about your past projects. They want to know what you did in your previous company, what situations you faced, and how you tackled them. They also like to know the value or impact you created. So, practice how you will present your previous projects.</p><h3>Additional Tips</h3><ol><li>Mock interviews: Practice with peers or mentors</li><li>Time management: Solve problems under timed conditions</li><li>Whiteboard practice: Explain solutions visually</li><li>Verbalise thought process: Practice thinking aloud</li><li>Prepare questions: Show genuine interest in the role</li><li>Review your own projects: Be ready to discuss in depth</li><li>Develop a problem-solving framework: Approach questions systematically</li><li>Practice explaining complex concepts simply</li></ol><h3>Negotiating Job Offers</h3><p>You’ve successfully cleared all the technical interview rounds. However, your journey isn’t quite over. Negotiating your job offer is a crucial step that can significantly impact your career trajectory and job satisfaction.</p><p>Before entering negotiations, arm yourself with knowledge. Research industry standards for salaries in your area using resources like Glassdoor, PayScale, or the annual Stack Overflow Developer Survey. This information will provide a solid foundation for your negotiation strategy.</p><p>Remember, negotiation isn’t just about salary. Consider the entire compensation package, including benefits, bonuses, stock options, remote work flexibility, and professional development opportunities. A lower base salary might be offset by excellent benefits or growth potential.</p><p>When you begin negotiations, approach the conversation as a collaborative discussion rather than an adversarial process. Articulate your value proposition clearly, highlighting your unique skills, experience, and potential contributions to the company. Be confident, but maintain a positive and professional tone throughout.</p><p>From personal experience, I can tell you that negotiation is a normal and expected part of the hiring process. In my transition from software engineering to data science, I initially received an offer that was below my expectations. By discussing my skills and the value I could bring to the role, we were able to reach a mutually beneficial agreement that included not just a higher salary, but also additional training opportunities.</p><p>Remember, if you don’t ask, you don’t get it.</p><h3>Conclusion</h3><p>Starting a career in machine learning is an exciting journey filled with opportunities for growth, innovation, and impact. As you start your ML career, keep in mind that this field is characterised by constant evolution. The learning doesn’t stop once you land a job. Stay curious, keep updating your skills, and remain open to new challenges and opportunities.</p><p>Your journey in ML is uniquely yours. There will be challenges along the way, but with perseverance, continuous learning, and a passion for solving complex problems, you can build a rewarding career in this dynamic field.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9d3fd934383a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Beyond Memorisation: How Summaries Build Deeper Understanding]]></title>
            <link>https://medium.com/@romeepanchal/beyond-memorisation-how-summaries-build-deeper-understanding-d567e4723ea8?source=rss-9f51633454f8------2</link>
            <guid isPermaLink="false">https://medium.com/p/d567e4723ea8</guid>
            <category><![CDATA[learning]]></category>
            <category><![CDATA[exam-preparation]]></category>
            <category><![CDATA[education]]></category>
            <category><![CDATA[memory-improvement]]></category>
            <category><![CDATA[understanding]]></category>
            <dc:creator><![CDATA[Romee Panchal]]></dc:creator>
            <pubDate>Sun, 21 Jul 2024 12:57:49 GMT</pubDate>
            <atom:updated>2024-08-06T13:54:12.396Z</atom:updated>
            <content:encoded><![CDATA[<p>Today, the internet is flooded with information. It isn&#39;t easy to keep up with everything happening in the field. However, to be successful you have to keep yourself informed. The problem is you can only read some things due to time constraints. So, it becomes crucial that whatever little time you spend in consuming information you make the best out of it. But how can you do that? How can you understand and retain most of the information that you read? There’s a simple yet powerful technique that can help you gauge and improve your understanding: summarising content in your own words.</p><p>Summarising is the process of distilling information down to its core ideas and concepts. By writing a summary of what you’ve read without looking back at the original material, you force yourself to engage with the content actively. This process not only reveals how much you’ve understood but also reinforces learning and retention.</p><p>In this article, we’ll explore the benefits of summarising, break down the process, and provide tips for effectively using this technique to enhance your comprehension and mastery of any subject.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wx6O2GdflwTZ8iiH" /><figcaption>Photo by <a href="https://unsplash.com/@sauravmahto?utm_source=medium&amp;utm_medium=referral">Saurav Mahto</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Why Summarise the Content</h3><p>We consume educational content through various mediums. Some prefer watching videos. Others prefer to read. Irrespective of the medium, learning aims to deeply understand the concept.</p><p>But how do you know if you have thoroughly understood the concept? When you go through the content, you feel fairly confident. However, as time passes you start to forget a lot of details without even realising this fact even if you remember all the details. What if someone asks you a question related to the topic? You will not be able to answer the question unless you have a deep understanding of the topic. At times like this, you realise that there might be a gap in your understanding.</p><p>You can avoid this pain if you assess your understanding of the topic right after finishing it. There are several techniques to do this. My favourite is summarisation. It is a quick and effective way to assess your understanding of the topic. After going through a topic, take some time and write a summary of it. Writing a summary is not an easy process unless you understand the topic. It requires you to identify the core ideas and concepts. If you struggle to write a summary this would indicate that there is a knowledge gap. The summarisation process will reveal exactly where are you struggling and what you need to work on.</p><p>Summarisation is not only helpful in revealing a knowledge gap. It also improves memory and critical thinking. When you summarise something, it forces your brain to recall the right pieces of information and connect those pieces into a much shorter summary than the original text. It improves your overall comprehension and helps you save a ton of time later.</p><p>Having understood the importance of summarisation, let’s dive into the practical aspects of how to effectively summarise content.</p><h3>The Summarisation Process</h3><p>The summarisation process is very straightforward. Here’s how I do it:</p><p>I first identify the core idea of the topic. Then, I identify the supporting ideas and key characteristics of the core idea. Once I’ve identified these details, I write a summary in simple terms, omitting specific details. If I encounter multiple smaller core ideas, I try to connect them and explain how they fit together to form the bigger picture. This reveals a lot about my understanding of the text.</p><p>While writing the summary I keep the following things in my mind.</p><ul><li>Did I understand the concept well?</li><li>Am I able to recognise the characteristics of the idea?</li><li>Can I make the connection to the other related ideas?</li><li>Can I write an even shorter summary of this?</li></ul><p>There are many different techniques to summarise content.</p><ol><li>Mind Mapping: Create a visual diagram with the main topic in the centre and related ideas branching out. Start with the central concept, then add major themes as primary branches. Add details and sub-ideas as secondary branches. A mind map helps visualise connections between ideas and organise information hierarchically.</li><li>The Feynman Technique: No summarisation method is complete without mentioning the famous Feynman technique. Named after physicist Richard Feynman, it involves explaining a concept in simple terms. After reading a topic, try to explain it to a child or someone unfamiliar with the topic. Identify gaps in your explanation and revisit the material. Simplify and use analogies to make the concept clearer. It helps identify gaps in understanding and reinforces learning through teaching.</li><li>Concept Mapping: Similar to mind mapping, but focuses more on showing relationships between concepts. Write key concepts in boxes or circles, then connect them with labelled lines showing their relationships. Helps understand complex systems and how different ideas relate to each other.</li><li>Cornell Method: Divide your page into three sections: notes, cues, and summary. Take notes in the largest section during reading or lecture. Write cues/questions in the left column to test your understanding later. Summarise the main ideas in the bottom section. Encourages active engagement with the material and provides a built-in review system.</li><li>Outlining: Organise information in a hierarchical structure with main points and supporting details. Identify main topics as Roman numerals, subtopics as capital letters, and details as numbers or lowercase letters. It helps structure information logically and see relationships between ideas.</li></ol><h3>How do I use summarisation?</h3><p>When I first learned linear algebra, I encountered the concepts of vectors and matrices. These are abstract concepts. I learned the formal definitions of vectors and their properties. Then, I learned how they relate to matrices. To solidify my understanding of these abstract concepts, I found it helpful to write summaries of what I learned.</p><p>For example, I wrote the following summary: Vectors are simply arrays of numbers. There are defined rules for operating on these arrays, such as adding two vectors or multiplying a vector by a number.</p><p>How do you add two vectors? The first thing to consider is that the vectors you want to add must have the same size. If the first array has 10 numbers, then the second array must also have 10 numbers. To add these arrays, suppose you have two rows. Write one array in the first row and the second array in the second row. Then, look at the corresponding numbers within those arrays and add them together like regular numbers. The result is another array with the same number of elements as the original two arrays.</p><p>Similarly, to interpret the equation Ax = b, where A is a matrix, x is a vector of unknowns, and b is a resultant vector, I thought of A as a map. Upon multiplication, a matrix transforms vector x to vector b.</p><p>These kinds of summaries helped me develop the intuition needed to understand the subject. While summarisation is a powerful tool, it’s not without its difficulties. Let’s examine some common challenges and how to overcome them.</p><h3>Challenges in Summarisation</h3><p>While summarisation is a powerful tool for enhancing understanding. However, it comes with its own set of challenges. Recognising these difficulties and learning how to overcome them is crucial for effective summarisation. Let’s explore some common challenges and strategies to address them.</p><p>It is important to distinguish the main idea from the supporting ideas. Many readers struggle to identify the core concept in a sea of information. Practice the “so what?” test. For each point, ask yourself, “So what? Why is this important?” If it’s crucial to the overall message, it’s likely a main idea. For example, if you are reading about the vectors in linear algebra. You can find the text talks about a bunch of stuff. Ask yourself, so what? The answer should you the importance of vectors.</p><p>Another important thing to keep in mind is the personal biases. Personal biases can influence how you interpret and summarise information. Try to go through the material with an open mind. Ask as many questions as possible to understand the underlying concept. This will help you overcome your own biases and understand the content better. Then write a summary based on your understanding. Focus on summarising what the text says, not your opinion of it.</p><p>Summaries can become too long or too brief, missing the balance between conciseness and comprehensiveness. Follow the 80/20 rule. Aim to capture 80% of the main content in 20% of the original length.</p><h3>Conclusion</h3><p>Throughout this essay, we’ve explored the powerful impact of summarisation on learning and understanding. The key benefits we’ve discussed include:</p><ol><li>Enhanced comprehension of complex topics</li><li>Improved retention of information</li><li>Development of critical thinking skills</li><li>Identification of knowledge gaps</li><li>More efficient learning and studying</li></ol><p>Summarisation serves as a versatile tool, applicable across various fields and learning contexts. It not only helps in academic settings but also in professional development and personal growth.</p><p>Now, it’s time to put this knowledge into action. Challenge yourself to incorporate summarisation into your daily learning routine. Start small — perhaps summarise this article you are reading. As you practice, you’ll find your ability to distil and retain information improving significantly.</p><p>It’s a skill that, once mastered, will serve you well in all areas of life where learning and information processing are key. So, pick up that pen, open that notebook, and start summarising.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d567e4723ea8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Step-by-Step Guide to Developing ML Models with SentenceTransformers]]></title>
            <link>https://medium.com/@romeepanchal/a-step-by-step-guide-to-developing-ml-models-with-sentencetransformers-9b11a4836e39?source=rss-9f51633454f8------2</link>
            <guid isPermaLink="false">https://medium.com/p/9b11a4836e39</guid>
            <category><![CDATA[sentence-embedding]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[sentiment-analysis]]></category>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Romee Panchal]]></dc:creator>
            <pubDate>Sun, 16 Jun 2024 06:05:10 GMT</pubDate>
            <atom:updated>2024-06-16T06:05:10.200Z</atom:updated>
            <content:encoded><![CDATA[<h4>A Deep Dive into Transformer-based Text Representations for Sentiment Analysis</h4><p>Machine Learning is fantastic. We can achieve marvellous results on pattern recognition tasks using machine learning. The trouble is a machine learning model only understands the numbers. And when we have text data, they are not directly applicable. We need some way to translate the text into numbers.</p><p>Several techniques, such as bag-of-words, tf-idf, word2vec, etc., transform text into numbers. The traditional bag-of-words approach often fails to capture the details and context in the text. This leads to inaccurate predictions. However, recent advancements in NLP have introduced powerful techniques like transformers. They have revolutionised how to represent the text data numerically and preserve the meaning of the data.</p><p>In this guide, we will explore SentenceTransformers and their applications in sentiment identification. We’ll start by understanding the SentenceTransformers package. It can capture context and handle complex language. From there, we’ll cover the practical steps. We will cover data prep, training, and evaluation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Ac6mfA3M9SkZcnPJ" /><figcaption>Photo by <a href="https://unsplash.com/@ikukevk?utm_source=medium&amp;utm_medium=referral">Kevin Ku</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Understanding Sentence Transformers</h3><p>SentenceTransformers is a Python framework based on <a href="https://pytorch.org/">PyTorch</a> and <a href="https://huggingface.co/transformers/">Transformers</a>. It offers a large collection of <a href="https://sbert.net/docs/pretrained_models.html">pre-trained models</a> tuned for various tasks. It can generate dense vector representations, or embeddings, for entire sentences or passages. These embeddings capture the semantic and contextual information within the text. These embeddings enable more accurate sentiment classification compared to traditional word-level embeddings. SentenceTransformers can harness transfer learning and pre-trained models. They can be fine-tuned for sentiment tasks. This results in better performance and adaptability.</p><p>You can use this framework to calculate sentence and text embeddings. It works for over 100 languages. You can then compare them using cosine similarity. This lets you find sentences with a similar meaning.</p><p>You can install the SentenceTransformers package using the following command.</p><blockquote>pip install -U sentence-transformers</blockquote><h3>Build a Sentiment Model</h3><h3>Import necessary libraries</h3><p>First, let’s import the required libraries.</p><pre>import pandas as pd<br>import numpy as np<br>from sklearn import metrics<br>from sklearn import model_selection<br>from sklearn import linear_model<br>from sentence_transformers import SentenceTransformer</pre><p>We need pandas and numpy to load and manipulate data. For the model development, we will use scikit-learn library. Here, we have imported the metrics module to evaluate the mode, model_selection to split the data into train and test sets, and the linear_model to train a model.</p><p>We imported the SentenceTransformer class from the sentence_transformers package. This will help us to create sentence embeddings.</p><h3>Load data</h3><p>We will use Amazon’s customer review data to build a sentiment classification model.</p><p>This data has 20k rows with 76.2% positive and 23.8% negative reviews.</p><pre># load the data<br>df = pd.read_csv(<br>    &quot;&lt;https://raw.githubusercontent.com/pycaret/pycaret/master/datasets/amazon.csv&gt;&quot;<br>)<br><br># let&#39;s rename columns for clarity<br>df = df.rename(columns={&quot;reviewText&quot;: &quot;reviews&quot;, &quot;Positive&quot;: &quot;label&quot;})<br><br># add a new column in the data to represent sentiment<br>df[&quot;sentiment&quot;] = np.where(df[&quot;label&quot;] == 1, &quot;positive&quot;, &quot;negative&quot;)<br><br># show sample<br>df.sample(5)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-GZa74O1xr1IJfOZw6RNSQ.png" /></figure><p>So, we have 20000 reviews where 15.2k(76%) are positive and 4.7k(24%) are negative.</p><h3>Create samples</h3><p>We will split the data into train (80%) and test (20%) sets. We will train the model on the train set and then will evaluate the model performance on the holdout set (test).</p><pre># create training sample<br>X = df[&quot;reviews&quot;].values<br>y = df[&quot;label&quot;].values<br><br># split the dataset into train and test samples<br>X_train, X_test, y_train, y_test = model_selection.train_test_split(<br>    X, y, test_size=0.2, stratify=y, random_state=42<br>)</pre><p>While splitting the dataset we used the stratify parameter so that we have an equal proportion of positive and negative samples in both train and set sets.</p><p>After this step, we have the following shapes for the train and test sets respectively.</p><pre># train shape<br>(16000,)<br><br># test shape<br>(4000,)</pre><h3>Create features</h3><p>Now, it is time to use the magic of sentence-transformer library to get the numerical representation of the customer reviews.</p><pre># define a function to get document vectors<br>def get_document_vector(model: SentenceTransformer, data: list) -&gt; np.array:<br>    document_vector = model.encode(<br>        sentences=data, convert_to_numpy=True, show_progress_bar=True<br>    )<br>  <br>  return document_vector<br><br># load sentence transformer model<br>embedding_model = &quot;all-MiniLM-L6-v2&quot;<br>text_model = SentenceTransformer(embedding_model)<br># get the embeddings for the train data<br>X_train = get_document_vector(text_model, X_train)<br># get the embeddings for the test data<br>X_test = get_document_vector(text_model, X_test)</pre><p>A lot is going on in the above code. Let’s understand it piece by piece.</p><p>First, we instantiate an object of SentenceTransformer class. For this, we used a pre-trained language model called all-MiniLM-L6-v2. This model is trained on 2 billion word pairs and is well-equipped to generate numerical features from the text data. This model generates a feature vector of 384 dimensions for each row in the input data.</p><p>Then we applied this model to our training and test data to get the numpy arrays of numbers.</p><p>After this step, we have the following shapes for the train and test sets respectively.</p><pre># train shape<br>(16000, 384)<br><br># test shape<br>(4000, 384)</pre><p>Notice that we got 384 for each training example.</p><h3>Train model</h3><p>Now, we have all we need to train a classification model. We will train a simple Logistic Regression model for this demonstration.</p><pre># train model<br>clf_model = linear_model.LogisticRegression()<br>clf_model.fit(X_train, y_train)<br><br># model inference<br>y_train_pred = clf_model.predict(X_train)<br>y_test_pred = clf_model.predict(X_test)</pre><p>This simple model achieves an accuracy of 90% on the train and test sets. Which is fantastic. We can further improve this accuracy by training a more complex model such as XGBoost or neural networks.</p><h3>Conclusion</h3><p>In the guide, we saw how sentence-transformer makes our life a bit easier. While traditional methods of transforming text to numbers do require a lot of data preprocessing. The sentence-transformer package works out of the box.</p><p><em>Note: This article was originally published on </em><a href="https://romeepanchal.substack.com/p/a-step-by-step-guide-to-developing"><em>Substack</em></a><em>. Follow me on substack to get AI essays directly in your inbox.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9b11a4836e39" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How do you find insights in data?]]></title>
            <link>https://medium.com/@romeepanchal/how-do-you-find-insights-in-data-cb31799312de?source=rss-9f51633454f8------2</link>
            <guid isPermaLink="false">https://medium.com/p/cb31799312de</guid>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[insights]]></category>
            <dc:creator><![CDATA[Romee Panchal]]></dc:creator>
            <pubDate>Wed, 08 May 2024 13:18:20 GMT</pubDate>
            <atom:updated>2024-05-08T13:18:20.637Z</atom:updated>
            <content:encoded><![CDATA[<h4>A Strategic Guide to Uncovering Hidden Insights</h4><h3>Introduction</h3><p>In today’s data-driven world, businesses hold a treasure trove of information. But simply having data isn’t enough. The true power lies in extracting valuable insights to inform better decision-making and fuel growth. This article will guide you through a strategic approach to uncover hidden gems within your data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*F8wK-6sIJXkxzVHG" /><figcaption>Photo by <a href="https://unsplash.com/@pietrozj">Pietro Jeng</a> on <a href="https://unsplash.com/">Unsplash</a></figcaption></figure><h3>What are Insights?</h3><p>Insights are not just random observations; they are previously unknown, <strong>actionable</strong> business trends or knowledge. They provide a deeper understanding of your customers, market dynamics, or internal operations, empowering you to take strategic actions.</p><h3>Asking Questions</h3><p>Before diving headfirst into data exploration, having a clear direction is crucial. Start by formulating insightful questions that align with your business goals. Here’s why this is important:</p><ul><li><strong>Focused Exploration:</strong> Asking specific questions prevents you from getting lost in a maze of data. It ensures your analysis is targeted and delivers valuable results.</li><li><strong>Uncovering the Unknown:</strong> By asking the right questions, you may unearth patterns or relationships that were previously hidden. This can lead to surprising discoveries that can significantly impact your business.</li></ul><h3>Examples of Insightful Questions:</h3><p>Imagine you’re analyzing sales data. Here are some questions to get you started:</p><ul><li>What are the sales trends over the past year? Are there any seasonal fluctuations?</li><li>Who are our most valuable customers? What are their key characteristics?</li><li>Do specific regions or outlets consistently outperform others?</li><li>Is there a correlation between marketing campaigns and sales spikes?</li></ul><h3>Beyond the Obvious</h3><p>While some initial questions might reveal readily apparent trends, the true gems lie deeper. Don’t be discouraged if your initial findings seem familiar. Here’s how to push the exploration further:</p><ul><li><strong>Business Context:</strong> Gain a deeper understanding of your industry and how your company operates. Talk to colleagues from different departments to gain a well-rounded perspective.</li><li><strong>Hypothesis-Driven Analysis:</strong> Based on your understanding, formulate hypotheses about the data. For instance, you might hypothesize that sales data follows a normal distribution.</li><li><strong>Data Validation:</strong> Use statistical methods or visualizations to test your hypotheses. Deviations from your expectations can be significant insights.</li></ul><h3>Finally Something Interesting</h3><p>If the data doesn’t conform to your expectations, congratulations! You’ve potentially stumbled upon a valuable insight. Investigate further to understand the reasons behind the anomaly. This deeper exploration can lead to game-changing discoveries.</p><h3>Conclusion</h3><p>By combining a question-driven approach with a solid understanding of your business, you can transform raw data into actionable insights. Remember, data is a powerful tool, but it’s the questions you ask that unlock its true potential. So, start asking, explore, and unearth the hidden gems waiting to be discovered within your data!</p><p><em>Note: This article is originally posted on </em><a href="https://romeepanchal.substack.com/p/how-to-find-insights-in-data"><em>Substack</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cb31799312de" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Bias Variance Trade-off]]></title>
            <link>https://medium.com/@romeepanchal/bias-variance-trade-off-528304b0c019?source=rss-9f51633454f8------2</link>
            <guid isPermaLink="false">https://medium.com/p/528304b0c019</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[bias-variance-tradeoff]]></category>
            <dc:creator><![CDATA[Romee Panchal]]></dc:creator>
            <pubDate>Sat, 22 Jul 2023 13:44:34 GMT</pubDate>
            <atom:updated>2023-07-22T13:44:34.264Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Bias-variance Tradeoff" src="https://cdn-images-1.medium.com/max/1024/0*VkO7bcA7IMQBI8Xz.jpeg" /><figcaption>Photo by Suket Dedhia from Pexels</figcaption></figure><h3>Introduction</h3><p>The bias-variance trade-off is a fundamental concept in machine learning that refers to the trade-off between the ability of a model to fit the training data well (low bias) and its ability to generalize to new, unseen data (low variance). In other words, the bias-variance trade-off is the balance between the complexity and flexibility of a model, and its ability to make accurate predictions on both the training and testing datasets.</p><p>Understanding the bias-variance trade-off is critical in machine learning because it helps us to build models that can make accurate predictions on new data. A model with high bias or high variance will not perform well on new data and may lead to incorrect predictions and poor decision-making. Therefore, finding the optimal balance between bias and variance is essential in building robust and accurate machine learning models.</p><h3>Bias and Variance</h3><p>In the context of machine learning, bias and variance are two types of errors that can occur in a model.</p><p>Bias refers to the difference between the expected or true output and the predicted output of the model. A model with high bias tends to underfit the data, meaning it oversimplifies the relationship between the input features and the target variable. This can result in poor performance on both the training and testing datasets, as the model may not capture all of the relevant patterns in the data.</p><p>On the other hand, variance refers to the variability of the model’s predictions for different training datasets. A model with high variance tends to overfit the data, meaning it captures the noise and idiosyncrasies of the training dataset too well. This can result in poor performance on the testing dataset, as the model may not generalize well to new data.</p><p>Let’s consider the example of a regression problem, where we want to predict the price of a house based on its square footage. Here are some examples of models with high and low bias/variance:</p><p>1. High bias, low variance: A linear regression model with only one feature, square footage, would have high bias and low variance. It is too simple to capture the underlying relationship between the square footage and the price of a house, resulting in underfitting. This model would have a high mean squared error on both the training and testing datasets.</p><p>2. Low bias, high variance: A complex model, such as a neural network with multiple hidden layers, would have low bias and high variance. It has the flexibility to capture the complex, nonlinear relationship between the square footage and the price of a house, but it may overfit the data by capturing the noise and idiosyncrasies of the training dataset too well. This model would have a low mean squared error on the training dataset, but a high mean squared error on the testing dataset.</p><p>3. Optimal bias-variance trade-off: A model with an optimal balance between bias and variance would have a moderate number of features or hidden layers. For example, a polynomial regression model with second-degree features could capture the underlying nonlinear relationship between the square footage and the price of a house without overfitting. This model would have a low mean squared error on both the training and testing datasets.</p><h3>The Trade-off</h3><p>Bias and variance are related in a way that increasing one may decrease the other, and vice versa. This relationship is called the bias-variance trade-off.</p><p>In machine learning, the bias-variance trade-off refers to the trade-off between the complexity of a model and its ability to generalize well to new data. A model with high bias has low complexity and may be too simple to capture the underlying patterns in the data. This model is likely to underfit the data and has poor performance on both the training and testing datasets. On the other hand, a model with high variance has high complexity and may be too flexible, resulting in overfitting and poor performance on the testing dataset.</p><p>In practice, we can control the bias-variance trade-off by adjusting the hyperparameters of the model, such as the number of features or the regularization strength. For example, we can increase the complexity of a model by adding more features or using a more powerful algorithm, which reduces bias but increases variance. Conversely, we can decrease the complexity of a model by using regularization techniques, which increases bias but reduces variance.</p><h3>Underfitting and Overfitting</h3><p>Underfitting and overfitting are two common problems that can arise in machine learning when trying to build models that can generalize well to new data.</p><p>Underfitting occurs when the model is too simple and is not able to capture the underlying patterns in the data. As a result, the model may perform poorly on both the training and testing data, indicating that it has not learned the relevant features of the data. This can happen when the model has high bias and low variance, and it is not able to capture the complexity of the data. In other words, underfitting occurs when the model is too simple and does not capture the complexity of the data, leading to high bias and low variance. A linear regression model that tries to fit a non-linear relationship between the features and the target variable. The model is too simple and cannot capture the non-linear patterns in the data, leading to high bias and low variance.</p><p>Overfitting occurs when the model is too complex and is too tightly fit to the training data. As a result, the model may perform very well on the training data but poorly on the testing data, indicating that it has learned noise or idiosyncrasies in the training data. This can happen when the model has low bias and high variance, and it is too flexible and overfits the training data. A decision tree model with very deep and complex trees that capture every detail of the training data. The model is too flexible and captures the noise and idiosyncrasies of the training data, leading to low bias and high variance.</p><h3>Finding the Optimal Model Complexity</h3><p>Finding the optimal model complexity involves balancing the bias-variance trade-off to minimize the total error of the model. This requires choosing the appropriate model complexity that can capture the relevant features of the data without overfitting or underfitting. Several techniques can be used to find the optimal model complexity, including cross-validation, regularization, and model selection.</p><p>Cross-validation is a technique used to evaluate the performance of a model by splitting the data into training and testing sets, and iteratively training the model on different subsets of the data. This allows us to estimate the generalization error of the model and choose the appropriate complexity that minimizes the error. For example, k-fold cross-validation involves splitting the data into k subsets and iteratively training the model on k-1 subsets and evaluating the performance on the remaining subset. This is repeated k times with different subsets used for testing each time.</p><p>Regularization is a technique used to prevent overfitting by adding a penalty term to the objective function of the model. This penalty term discourages the model from fitting the noise or idiosyncrasies in the data and encourages it to focus on the relevant features. Examples of regularization techniques include L1 and L2 regularization, which add a penalty term based on the magnitude of the weights of the model, and dropout, which randomly drops out some of the neurons in a neural network during training to prevent overfitting.</p><p>Model selection is a technique used to choose the appropriate algorithm and hyperparameters for the model. This involves trying different algorithms with different hyperparameters and evaluating their performance using cross-validation or other techniques. Examples of model selection techniques include grid search, which exhaustively searches the hyperparameter space for the best combination of hyperparameters, and random search, which randomly samples the hyperparameter space to find a good combination of hyperparameters.</p><p>For example, suppose we are building a classification model to predict whether a customer will churn or not based on their demographic and behavioural data. We start by trying different algorithms, such as logistic regression, decision trees, and neural networks, with different hyperparameters. We use k-fold cross-validation to evaluate the performance of each algorithm and choose the best one based on its generalization error. Once we have chosen the best algorithm, we use regularization techniques such as L1 or L2 regularization to prevent overfitting and fine-tune the hyperparameters using grid search or random search. This allows us to find the optimal model complexity that balances bias and variance and achieves the best performance on the testing data.</p><h3>Real-World Applications</h3><p>The bias-variance trade-off is a fundamental concept in machine learning that applies to a wide range of real-world scenarios. Here are some examples of how the trade-off applies in real-world scenarios:</p><p>1. Medical diagnosis: In medical diagnosis, the bias-variance trade-off can impact the accuracy of the diagnosis. A high-bias model may oversimplify the diagnosis, leading to a high error rate due to underfitting. On the other hand, a high-variance model may overfit the diagnosis, leading to low accuracy due to noise in the data. A model with an appropriate bias-variance balance can achieve high accuracy and reliable diagnosis.</p><p>2. Financial forecasting: In financial forecasting, the bias-variance trade-off can affect the interpretability of the model. A high-bias model may produce simple and interpretable forecasts but may fail to capture complex patterns and underlying relationships in the data. A high-variance model may produce accurate forecasts but may be difficult to interpret and explain due to its complexity.</p><p>3. Image recognition: In image recognition, the bias-variance trade-off can impact the robustness of the model. A high-bias model may miss subtle patterns in the image, leading to low accuracy and poor recognition performance. A high-variance model may fit the noise in the data, leading to overfitting and poor generalization performance. A model with an appropriate bias-variance balance can achieve high accuracy and robust recognition performance.</p><h3>Conclusion</h3><p>Understanding the bias-variance trade-off is essential in machine learning because it allows us to develop models that generalize well to unseen data. By finding the optimal balance between bias and variance, we can create models that are both accurate and interpretable, making them useful in real-world applications.</p><p>In conclusion, the bias-variance trade-off is a critical concept for machine learning practitioners to understand. By balancing the trade-off between bias and variance, we can develop models that are both accurate and interpretable, resulting in better performance and better results in real-world applications.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=528304b0c019" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Version control machine learning experiments]]></title>
            <link>https://medium.com/analytics-vidhya/version-control-machine-learning-experiments-cf52d092e468?source=rss-9f51633454f8------2</link>
            <guid isPermaLink="false">https://medium.com/p/cf52d092e468</guid>
            <category><![CDATA[dvc]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[version-control]]></category>
            <category><![CDATA[git]]></category>
            <dc:creator><![CDATA[Romee Panchal]]></dc:creator>
            <pubDate>Mon, 22 Jun 2020 04:48:53 GMT</pubDate>
            <atom:updated>2020-07-15T16:13:37.316Z</atom:updated>
            <content:encoded><![CDATA[<h3>Version control your machine learning experiments</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*c1mGZbLB_97jr1lx" /><figcaption>Photo by <a href="https://unsplash.com/@hania09?utm_source=medium&amp;utm_medium=referral">Hani Bdran</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Nowadays, learning from data to gain business insights is common for almost every industry. These insights include— predictability, customer churn behavior, forecasting, etc.. Machine learning is the key player in generating these insights.</p><p>Building a good ML model requires a lot of experiments that involves multiple iterations of different algorithms over data, creating new variables, adding more data etc.. As the number of iterations grows, it becomes harder to keep track of these experiments.</p><p>In this article I will talk about a system to effectively version control machine learning project. I will also share some tools that will help you in easily implementing this system.</p><h3>In what scenarios you get a new version?</h3><h4><strong>1. Data changes</strong></h4><p>Whenever there is a change in the modeling data, you create a new version of the model. ML models are trained on the modeling data. As the modeling data changes, model parameters will also change. You change the data when you do the following:</p><ul><li>When you capture more data.</li><li>When you normalize/standardized data.</li><li>When you create new variables — combination of variables, dummy variables, trends, etc..</li><li>When you drop existing variables.</li><li>When you fill missing data or drop missing data.</li><li>Or any other way, in which some change happen in the data.</li></ul><h4>2. Model changes</h4><p>Different modeling algorithms have different performance on the same training data. Each new algorithm you apply, you do a new experiment. This new experiment is a new version of the model.</p><p>Another scenario in which you create a new version, where you keep the data and the algorithm same, but, the model hyper-parameters changes.</p><h3>Why do you need to keep track of model versions?</h3><p>In the modeling phase, a unique combination of training data and algorithm, will result in a single experiment. When you run different experiments, each experiment has different performance on the validation data. With each experiment, model performance may improve or degrade. Consider following table:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/757/1*g7dlCM5huHrN40HTeaoX6g.png" /><figcaption>Model experiments results</figcaption></figure><p>Suppose, you have done 50+ experiments, and realize that one of the earlier experiments had better generalization than the current one. In the above table, we see that the model 4 has the better performance of both train and validation data. Because you did a lot of experiments, you don’t remember exactly what data and what algorithm you used in that model. In such cases, it becomes very difficult to reproduce the exact same results. That is why you need a versioning system with which you can easily keep track of all of the experiments without losing your older work.</p><h3>Model Versioning</h3><p>Model versioning is a systematic approach to keep track of all your modeling experiments, so that, when you need it, you can retrieve it.</p><h4>Version Naming</h4><p>As you saw in earlier section that a model changes either when data changes or algorithm changes or both. Given all this kind of changes, it is important to have a meaningful and consistent version naming system, so that, you know at a high level what is going on with a version.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qtFv9SUOARk0_rzYi1T1jg.png" /></figure><p>I use the four number system to specify a model number.</p><ul><li>First number tells me the project version. If I am working on it for the first time, I will assign x=1.</li><li>Second number tells me the data version that I used for the modeling. Initially, I set y=1. During the modeling process, we may need to modify data multiple times. Each time I modify data, I increment the number y.</li><li>Third number I use for the modeling algorithm. For each data version, I start it from 1. For example, for data version y, for the first algorithm I apply, I set z=1, for the second algorithm, z=2, etc.</li><li>Forth number I use for the minor changes to model, like when algorithm hyper-parameters changes, it produces different results so different number for w in that case.</li></ul><p>For example, if I am working on an ML project for the first time. I created one version of the modeling data and I applied logistic regression algorithm to that data. In this case, my complete version number will become 1.1.1.0. Now, if I change the algorithm to random forest for the same data, I will call my version 1.1.2.0. In case, I find better parameters setting for the random forest, I will call that version 1.1.2.1. This way, I can easily track my models.</p><h4>Versioning Tools</h4><p>The aforementioned version system can be implemented using regular file system. For if your data is in CSV, you name your file as &lt;filename&gt;_&lt;version&gt;.csv. Similarly, code files can be managed in the same way.</p><p>But above approach starts to fail when you are dealing with a lot of files. An efficient approach to deal with versioning is to use dedicated software. There the two version control software that we can use for this purpose —</p><ul><li><a href="https://git-scm.com/">git</a> — for the code version control.</li><li><a href="https://dvc.org/">dvc</a> — for the data version control.</li></ul><p>Both git and dvc are open-source software. For the complete usage information, checkout their documentation. For examples on how to use these tool, checkout following articles —</p><ul><li><a href="https://towardsdatascience.com/start-version-controlling-your-machine-learning-datasets-2b872e109856">Start Version Controlling your Machine Learning Datasets</a></li><li><a href="https://towardsdatascience.com/version-control-ml-model-4adb2db5f87c">Version Control ML Model</a></li></ul><h3>Conclusion</h3><p>You can use model versioning using the above system or you can define your own system. The important thing is follow a systematic approach while doing experiment so that it becomes easy to reproduce results when you need them.</p><p>Follow me on <a href="https://www.quora.com/profile/Romee-Panchal">Quora</a>. I write answers related to data science and machine learning.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cf52d092e468" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/version-control-machine-learning-experiments-cf52d092e468">Version control machine learning experiments</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>