Machine Learning Models For Predictive Analytics

RAG stands for Retrieval-Augmented Generation. It’s a technique that combines the power of LLMs with real-time access to external information sources. Instead of relying solely on what an AI model learned during training (which can quickly become outdated), RAG enables the model to retrieve relevant data from external databases, documents, or APIs—and then use that information to generate more accurate, context-aware responses. How does RAG work? 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗲: The system searches for the most relevant documents or data based on your query, using advanced search methods like semantic or vector search. 𝗔𝘂𝗴𝗺𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻: Instead of just using the original question, RAG 𝗮𝘂𝗴𝗺𝗲𝗻𝘁𝘀 (enriches) the prompt by adding the retrieved information directly into the input for the AI model. This means the model doesn’t just rely on what it “remembers” from training—it now sees your question 𝘱𝘭𝘶𝘴 the latest, domain-specific context 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗲: The LLM takes the retrieved information and crafts a well-informed, natural language response. 𝗪𝗵𝘆 𝗱𝗼𝗲𝘀 𝗥𝗔𝗚 𝗺𝗮𝘁𝘁𝗲𝗿? Improves accuracy: By referencing up-to-date or proprietary data, RAG reduces outdated or incorrect answers. Context-aware: Responses are tailored using the latest information, not just what the model “remembers.” Reduces hallucinations: RAG helps prevent AI from making up facts by grounding answers in real sources. Example: Imagine asking an AI assistant, “What are the latest trends in renewable energy?” A traditional LLM might give you a general answer based on old data. With RAG, the model first searches for the most recent articles and reports, then synthesizes a response grounded in that up-to-date information. Illustration by Deepak Bhardwaj

Small variations in prompts can lead to very different LLM responses. Research that measures LLM prompt sensitivity uncovers what matters, and the strategies to get the best outcomes. A new framework for prompt sensitivity, ProSA, shows that response robustness increases with factors including higher model confidence, few-shot examples, and larger model size. Some strategies you should consider given these findings: 💡 Understand Prompt Sensitivity and Test Variability: LLMs can produce different responses with minor rephrasings of the same prompt. Testing multiple prompt versions is essential, as even small wording adjustments can significantly impact the outcome. Organizations may benefit from creating a library of proven prompts, noting which styles perform best for different types of queries. 🧩 Integrate Few-Shot Examples for Consistency: Including few-shot examples (demonstrative samples within prompts) enhances the stability of responses, especially in larger models. For complex or high-priority tasks, adding a few-shot structure can reduce prompt sensitivity. Standardizing few-shot examples in key prompts across the organization helps ensure consistent output. 🧠 Match Prompt Style to Task Complexity: Different tasks benefit from different prompt strategies. Knowledge-based tasks like basic Q&A are generally less sensitive to prompt variations than complex, reasoning-heavy tasks, such as coding or creative requests. For these complex tasks, using structured, example-rich prompts can improve response reliability. 📈 Use Decoding Confidence as a Quality Check: High decoding confidence—the model’s level of certainty in its responses—indicates robustness against prompt variations. Organizations can track confidence scores to flag low-confidence responses and identify prompts that might need adjustment, enhancing the overall quality of outputs. 📜 Standardize Prompt Templates for Reliability: Simple, standardized templates reduce prompt sensitivity across users and tasks. For frequent or critical applications, well-designed, straightforward prompt templates minimize variability in responses. Organizations should consider a “best-practices” prompt set that can be shared across teams to ensure reliable outcomes. 🔄 Regularly Review and Optimize Prompts: As LLMs evolve, so may prompt performance. Routine prompt evaluations help organizations adapt to model changes and maintain high-quality, reliable responses over time. Regularly revisiting and refining key prompts ensures they stay aligned with the latest LLM behavior. Link to paper in comments.

Machine learning beats traditional forecasting methods in multi series forecasting. In one of the latest M forecasting competitions, the aim was to advance what we know about time series forecasting methods and strategies. Competitors had to forecast 40k+ time series representing sales for the largest retail company in the world by revenue: Walmart. These are the main findings: ▶️ Performance of ML Methods: Machine learning (ML) models demonstrate superior accuracy compared to simple statistical methods. Hybrid approaches that combine ML techniques with statistical functionalities often yield effective results. Advanced ML methods, such as LightGBM and deep learning techniques, have shown significant forecasting potential. ▶️ Value of Combining Forecasts: Combining forecasts from various methods enhances accuracy. Even simple, equal-weighted combinations of models can outperform more complex approaches, reaffirming the effectiveness of ensemble strategies. ▶️ Cross-Learning Benefits: Utilizing cross-learning from correlated, hierarchical data improves forecasting accuracy. In short, one model to forecast thousands of time series. This approach allows for more efficient training and reduces computational costs, making it a valuable strategy. ▶️ Differences in Performance: Winning methods often outperform traditional benchmarks significantly. However, many teams may not surpass the performance of simpler methods, indicating that straightforward approaches can still be effective. Impact of External Adjustments: Incorporating external adjustments (ie, data based insight) can enhance forecast accuracy. ▶️ Importance of Cross-Validation Strategies: Effective cross-validation (CV) strategies are crucial for accurately assessing forecasting methods. Many teams fail to select the best forecasts due to inadequate CV methods. Utilizing extensive validation techniques can ensure robustness. ▶️ Role of Exogenous Variables: Including exogenous/explanatory variables significantly improves forecasting accuracy. Additional data such as promotions and price changes can lead to substantial improvements over models that rely solely on historical data. Overall, these findings emphasize the effectiveness of ML methods, the value of combining forecasts, and the importance of incorporating external factors and robust validation strategies in forecasting. If you haven’t already, try using machine learning models to forecast your future challenge 🙂 Read the article 👉 https://buff.ly/3O95gQp

I just came across a groundbreaking paper titled "Benchmarking LLMs in Recommendation Tasks: A Comparative Evaluation with Conventional Recommenders" that provides comprehensive insights into how large language models (LLMs) perform in recommendation tasks. The researchers from The Hong Kong Polytechnic University, Huawei Noah's Ark Lab, Nanyang Technology University, and National University of Singapore have developed RecBench - a systematic evaluation platform that thoroughly assesses the capabilities of LLMs in recommendation scenarios. >> Key Technical Insights: This benchmark evaluates various item representation forms: - Unique identifiers (traditional approach) - Text representations (using item descriptions) - Semantic embeddings (leveraging pre-trained LLM knowledge) - Semantic identifiers (using discrete encoding techniques like RQ-VAE) The study covers two critical recommendation tasks: - Click-through rate (CTR) prediction (pair-wise recommendation) - Sequential recommendation (list-wise recommendation) Their extensive experiments evaluated 17 different LLMs across five diverse datasets from fashion, news, video, books, and music domains. The results are eye-opening: - LLM-based recommenders outperform conventional recommenders by up to 5% AUC improvement in CTR prediction and a staggering 170% NDCG@10 improvement in sequential recommendation - However, these performance gains come with significant computational costs, making real-time deployment challenging - Conventional deep learning recommenders enhanced with LLM support can achieve 95% of standalone LLM performance while being thousands of times faster Under the hood, the researchers implemented a conditional beam search technique for semantic identifier-based models to ensure valid item recommendations. They also employed low-rank adaptation (LoRA) for parameter-efficient fine-tuning of the large models. Most interestingly, they found that while most LLMs have limited zero-shot recommendation abilities, models like Mistral, GLM, and Qwen-2 performed significantly better, likely due to exposure to more implicit recommendation signals during pre-training. This research opens exciting avenues for recommendation system development while highlighting the need for inference acceleration techniques to make LLM-based recommenders practical for industrial applications.

Thank you to the University of Toronto Machine Intelligence Student Team for inviting me to present a keynote on augmenting human-labeled datasets using Large Language Models (LLMs). Human-labeled data is crucial for testing, tuning, customizing, and validating LLMs in organizations. This is because human labeled data provides the ground truth for developing trustworthy #GenerativeAI applications and #AgenticAI systems. Yet acquiring sufficient human labeled data is often a bottleneck in many organizations. Subject matter experts and domain specialists typically have limited time for labeling tasks due to competing professional demands, making large-scale manual labeling difficult to sustain. My talk focused on how LLMs can be used not to substitute human labels, but to systematically augment them—extending the utility of existing human labeled data and improving model robustness without proportionally increasing manual labeling effort. I described practical methods for implementing two augmentation techniques with strong empirical grounding: • Negative Reinforcement with Counterfactual Examples – This technique involves analyzing labeled examples to generate counterfactual examples—outputs that are intentionally incorrect or undesirable—and using them to teach the model about what not to generate. By guiding the model using these negative samples, the model learns sharper decision boundaries, increasing robustness against hallucinations and confabulations. • Contrastive Learning with Controlled Perturbations – This technique creates diverse, label-preserving variants of human-labeled examples by introducing controlled modifications to the prompts and/or completions. These perturbations maintain core semantic meaning while varying surface-level features such as syntax, phrasing, or structure, encouraging the model to generalize beyond shallow lexical or syntactic cues. These techniques have been shown to drive measurable improvements in model behavior: • Lower Perplexity → More predictable completions and improved alignment with ground-truth targets. • Reduced Token Entropy → More focused and efficient completions, reducing inference complexity. • Higher Self-Consistency → More stable completions across repeated generations of the same prompt—a key requirement for dependable downstream use. These are not theoretical constructs—they are practical techniques for overcoming constraints in human-labeled data availability and scaling of #LLM applications with greater efficiency and rigor. Appreciate the University of Toronto Machine Intelligence Student Team (UTMIST) for a well-curated conference, and the UofT AI group for their initiatives in the space. Grateful to my research partner, Olga, for her contributions in collaboratively developing content for this presentation. Kudos to my PwC Canada teammates including Michelle B, Annie, Chris M, Michelle G, Chris D, Brenda, Bahar, Danielle, and Abhinav for their partnership on our PwC #AI portfolio.

Mixture-of-Experts (MoE) LLMs are more prone to training instability than standard LLMs. Here’s why this is the case and how we can fix it… Where do instabilities come from? There are two main issues that occur when training an MoE: 1. Routing collapse: the model converges to using the same expert(s) over and over. 2. Numerical instability: the MoE experiences round-off errors, especially in the router. These issues lead to training instability, meaning that the model’s loss may simply diverge (i.e., go up instead of down) during the training process. Avoiding routing collapse: We need to add auxiliary losses to our training objective that encourage the model to use experts uniformly. The most common auxiliary loss for MoEs is the load balancing auxiliary loss [1], which is minimized when the MoE i) assigns probability uniformly to experts and ii) routes an equal number of tokens to each expert within a batch. Avoiding numerical instability: The biggest source of numerical instability occurs in the MoE’s router because the router includes an (exponential) softmax function. To avoid numerical instabilities in this layer, we can add an auxiliary loss that encourages the values going into the softmax function to not be too large–this is called the router z-loss [2]. Although many LLMs are trained in lower (bfloat16) precision, we should avoid using low precision within the router. Mixed / low precision training greatly improves training efficiency, but it can also make round-off errors more frequent within the router! Weight initialization: Traditionally, we made the training of large, deep neural networks more stable by discovering better weight initialization (e.g., He or Glorot init) and normalization (e.g., batch normalization) techniques. Similarly, we can improve MoE training stability by using a weight initialization strategy that’s more tailored to MoEs. As proposed in [1], we can sample from a truncated normal distribution with a mean of zero (µ = 0) and standard deviation given by σ = SQRT(s/n), where s (0.1 by default) is a scale hyperparameter and n is the size of the input to the layer being initialized. Putting everything together: I’ve tried out each of these techniques within nanoMoE, a simple and functional MoE pretraining implementation that I recently released. We can see that each of these tricks improves the MoE’s training stability. When we use them all together, nanoMoE is able to fully complete pretraining without having any instabilities!

𝗔𝗜 𝗶𝗻 𝗣𝗿𝗲𝗱𝗶𝗰𝘁𝗶𝘃𝗲 𝗠𝗮𝗶𝗻𝘁𝗲𝗻𝗮𝗻𝗰𝗲: 𝗔 𝗟𝗲𝗮𝗽 𝗕𝗲𝘆𝗼𝗻𝗱 𝗜𝗻𝗱𝘂𝘀𝘁𝗿𝘆 𝟰.𝟬 Predictive maintenance (PdM) is one of the most popular use cases in Industry 4.0. Nearly every organization exploring IoT and analytics has a PdM initiative because the promise is clear: anticipate failures, minimize downtime, and cut maintenance costs. But here’s the reality: traditional PdM often struggles with siloed data, model degradation, and weak integration with day-to-day operations. That’s where AI agent-driven predictive maintenance makes the difference. Unlike conventional PdM, which relies on isolated models and static rules, AI-powered PdM leverages: 𝗔𝘂𝘁𝗼𝗻𝗼𝗺𝗼𝘂𝘀 𝗔𝗜 𝗮𝗴𝗲𝗻𝘁𝘀 – not just detecting anomalies but reasoning, planning, and executing maintenance actions. 𝗟𝗮𝗿𝗴𝗲 𝗹𝗮𝗻𝗴𝘂𝗮𝗴𝗲 & 𝗺𝘂𝗹𝘁𝗶𝗺𝗼𝗱𝗮𝗹 𝗺𝗼𝗱𝗲𝗹𝘀 – combining sensor data with logs, manuals, and operator notes. 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹-𝗔𝘂𝗴𝗺𝗲𝗻𝘁𝗲𝗱 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻 (𝗥𝗔𝗚) – grounding predictions in real-time technical knowledge. 𝗔𝗜 + 𝗗𝗶𝗴𝗶𝘁𝗮𝗹 𝗧𝘄𝗶𝗻𝘀 – simulating “what-if” scenarios before interventions, avoiding costly missteps. 𝗛𝗼𝘄 𝗶𝘁 𝘄𝗼𝗿𝗸𝘀 𝗶𝗻 𝗽𝗿𝗮𝗰𝘁𝗶𝗰𝗲: 𝗘𝗱𝗴𝗲 & 𝗙𝗼𝗴 𝗹𝗮𝘆𝗲𝗿𝘀 handle real-time monitoring and anomaly detection, filtering data and running lightweight AI models for quick decisions. 𝗥𝗔𝗚𝗙𝗹𝗼𝘄 enriches anomaly data with relevant manuals and past cases, reducing false alarms and improving diagnostic accuracy. 𝗖𝗼𝗿𝗲 𝗔𝗜 agents work together: a diagnostic agent identifies root causes and risk levels, a scheduling agent optimizes work orders, a digital twin agent tests scenarios before execution, and a summing-up agent closes the loop by learning from outcomes. 𝗖𝗹𝗼𝘂𝗱 𝗶𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲 manages complex diagnostics, lifespan forecasting, and global optimization of resources. The result is a continuous learning cycle, where every maintenance action refines the system’s knowledge base, making it smarter and more reliable over time. 𝗧𝗵𝗲 𝗗𝗶𝗳𝗳𝗲𝗿𝗲𝗻𝗰𝗲: Industry 4.0 PdM → data-driven, efficiency-focused. AI agent-driven PdM → adaptive, knowledge-rich, human-centric, aligned with priorities like resilience and sustainability. The outcome is a closed-loop system where sensing, reasoning, and action converge, turning maintenance from a cost center into a strategic enabler of resilience and value. Ref: Artificial Intelligence Agent-Enabled Predictive Maintenance: Wenyu Jiang et.all.

🌍 Transforming Process Safety with AI-Driven Insights | Successful POC with Energy Operator in #Oman Pain: The traditional approach to LOPA and SIL processes is manual, time-consuming, and prone to human error. Identifying Risk Reduction Factors (RRF), Independent Protection Layers (IPLs), and ICLs (Initiating Cause Likelihood) takes significant effort, and the decisions made based on these analyses are critical but not always optimized. Value: With AI inference models, specifically using Artificial Neural Networks (ANN) and Binary Classification, we can convert the LOPA and SIL processes into actionable data-driven insights. By automating risk assessments, we gain a deeper understanding of how independent and dependent variables interact dynamically allowing us to predict system behavior with higher precision. This leads to safer operations, better risk reduction decisions, and greater efficiency. Vision: The future of process safety lies in leveraging hybrid models and AI to not only automate safety assessments but to democratize safety across organizations. By embedding AI in these critical processes, we enable faster, more informed decision-making ensuring every near-miss, maintenance log, and process update is accounted for in real-time. The goal? To create a safer, more resilient operational environment with enhanced autonomy and predictability. Learned Lessons: From deploying AI in gas and liquid separation use cases, we've learned the importance of integrating first principles with AI. This hybrid approach allows us to build systems that not only rely on historical data but also incorporate deep engineering insights for a more holistic safety analysis. Deployment: The proposed solution used to encompass #HYSYS, #AIMB and #SelexMB The future of process safety is here, and the synergy between AI and First Principles is leading the positive change. #ProcessSafety #AIinSafety #LOPA #SIL #RiskManagement #ArtificialIntelligence #HybridModeling #OperationalExcellence #SafetyIntegrityLevel #DigitalTransformation #EnergySafety #HYSYS #AIMB #SelexMB #Extension

LLM literacy is now part of modern UX practice. It is not about turning researchers into engineers. It is about getting cleaner insights, predictable workflows, and safer use of AI in everyday work. A large language model is a Transformer based language system with billions of parameters. Most production models are decoder only, which means they read tokens and generate tokens as text in and text out. The model lifecycle follows three stages. Pretraining learns broad language regularities. Finetuning adapts the model to specific tasks. Preference tuning shapes behavior toward what reviewers and policies consider desirable. Prompting is a control surface. Context length sets how much material the model can consider at once. Temperature and sampling set how deterministic or exploratory generation will be. Fixed seeds and low temperature produce stable, reproducible drafts. Higher temperature encourages variation for exploration and ideation. Reasoning aids can raise reliability when tasks are complex. Chain of Thought asks for intermediate steps. Tree of Thoughts explores alternatives. Self consistency aggregates multiple reasoning paths to select a stronger answer. Adaptation options map to real constraints. Supervised finetuning aligns behavior with high quality input and output pairs. Instruction tuning is the same process with instruction style data. Parameter efficient finetuning adds small trainable components such as LoRA, prefix tuning, or adapter layers so you do not update all weights. Quantization and QLoRA reduce memory and allow training on modest hardware. Preference tuning provides practical levers for quality and safety. A reward model can score several candidates so Best of N keeps the highest scoring answer. Reinforcement learning from human feedback with PPO updates the generator while staying close to the base model. Direct Preference Optimization is a supervised alternative that simplifies the pipeline. Efficiency techniques protect budgets and service levels. Mixture of Experts activates only a subset of experts per input at inference which is fast to run although the routing is hard to train well. Distillation trains a smaller model to match the probability outputs of a larger one so most quality is retained. Quantization stores weights in fewer bits to cut memory and latency. Understanding these mechanics pays off. You get reproducible outputs with fixed parameters, bias-aware judging by checking position and verbosity, grounded claims through retrieval when accuracy matters, and cost control by matching model size, context window, and adaptation to the job. For UX, this literacy delivers defensible insights, reliable operations, stronger privacy governance, and smarter trade offs across quality, speed, and cost.