Understanding AI Systems

Multi-Head Attention (MHA) is the engine of LLMs. But over the years, we have added several tweaks to make it more efficient for long-context settings, especially when using KV caching during inference. I implemented the most common variants from scratch: 1) Grouped-Query Attention (GQA): Instead of having a unique key and value for each query head, multiple queries share the same key and value. As long as the sharing ratio is not too extreme, this has minimal impact on model quality. It is the most widely used variant today and found in almost every modern LLM including Llama 2-4, GPT-OSS, Gemma 3, Qwen 3, GLM 4.6, and many others. 2) Multi-Head Latent Attention (MLA): This variant introduces a compressed latent representation for the keys and values that are stored in the KV cache. During inference, these latent keys and values are up-projected back to the full dimension. The extra projection adds a small computational cost, but the memory savings make it worth it. This approach is currently used by DeepSeek V3 and Kimi K2. 3) Sliding-Window Attention (SWA): SWA restricts each token’s attention span to a fixed local window, which reduces memory needs by shrinking the KV cache in long-context regimes. It is usually applied selectively, for example every other layer, or in Gemma 3's case, five SWA layers for each full-attention layer. While less common today, it remains an important optimization, notably in Gemma 3. All three variants, GQA, MLA, and SWA, can also be combined freely within the same model. Here's a link to check them out: 1️⃣ GQA: https://lnkd.in/grDPXUUi 2️⃣ MLA: https://lnkd.in/gm4FzE32 3️⃣ SWA: https://lnkd.in/g7x-fdgn

Microsoft just released a 35-page report on medical AI - and it’s a reality check for healthcare. The paper, “The Illusion of Readiness”, tested six of the most popular models (OpenAI, Gemini, etc)… across six multimodal medical benchmarks. And the verdict? The models scored high on medical exams. But they’re not even close to being real-world ready. Here’s what the stress tests revealed: ▶ 1. Shortcut learning Models often answered correctly even when key information, like medical images, was removed. They weren’t reasoning - they were exploiting statistical shortcuts. That means benchmark wins may hide shallow understanding. ▶ 2. Fragile under small changes Making small tweaks caused big swings in predictions. This fragility shows how unreliable model reasoning becomes under stress. In visual substitution tests, accuracy dropped from 83% to 52% when images were swapped - exposing shallow visual–answer pairings. ▶ 3. Fabricated reasoning Models produced confident, step-by-step medical explanations - but many were medically unsound… or entirely fabricated. Convincing to the eye, dangerous in practice. And more importantly, healthcare isn’t a multiple-choice exam. It’s uncertainty, incomplete data, and high stakes. So Microsoft’s team calls for new standards: - Stress tests that expose fragility - Clinician-guided guidelines that profile benchmarks - Evaluation of robustness and trustworthiness - not just leaderboard scores The takeaway is simple: Medical AI may ace tests today. But until it proves reliable under stress, it’s not ready for the clinic. When do you think popular LLMs will be clinic-ready? #entrepreneurship #healthtech #AI

Ever wondered where the future of AI is being built? I just visited the data centre in Finland that's making it happen. Nebius’ data centre is the powerhouse where AI models are trained. Thousands of GPUs working in unison. It’s expanding to host up to 60,000 GPUs dedicated to intensive AI workloads. They’re building a full-stack AI cloud platform. Here’s what I learned: 1. There is a scarcity of GPUs in the US • Clusters are being sold in massive packages • People who need smaller requirements can’t find them 2. Nebius are building a self-serve platform • Cover infrastructure requirements from a single GPU to big GPU clusters • They’re not a GPU reseller—they’re designing the servers and the racks from the ground up 3. Applications • Helped Mistral train their multimodal models • Provide full-stack infrastructure for AI model development Something else that was unique about the visit. Nebius cools the servers in Finland using the outside air. The heat that’s generated from the servers is then shipped back into the grid. This means Nebius not only heats the onsite building, But it also heats homes nearby, benefitting the local community. They’re able to recover 70% of the heat generated. And it’s the first in the world to have this heat reuse application connected to the local municipal grid. They’re now investing over $1B in AI data centres in Europe. I feel the future of AI depends on infrastructure like this that balances performance with sustainability. Follow me Alex Banks for daily AI highlights & insights.

Many engineers can build an AI agent. But designing an AI agent that is scalable, reliable, and truly autonomous? That’s a whole different challenge.  AI agents are more than just fancy chatbots—they are the backbone of automated workflows, intelligent decision-making, and next-gen AI systems. However, many projects fail because they overlook critical components of agent design.  So, what separates an experimental AI from a production-ready one?  This Cheat Sheet for Designing AI Agents breaks it down into 10 key pillars:  🔹 AI Failure Recovery & Debugging – Your AI will fail. The question is, can it recover? Implement self-healing mechanisms and stress testing to ensure resilience.  🔹 Scalability & Deployment – What works in a sandbox often breaks at scale. Using containerized workloads and serverless architectures ensures high availability.  🔹 Authentication & Access Control – AI agents need proper security layers. OAuth, MFA, and role-based access aren’t just best practices—they’re essential.  🔹 Data Ingestion & Processing – Real-time AI requires efficient ETL pipelines and vector storage for retrieval—structured and unstructured data must work together.  🔹 Knowledge & Context Management – AI must remember and reason across interactions. RAG (Retrieval-Augmented Generation) and structured knowledge graphs help with long-term memory.  🔹 Model Selection & Reasoning – Picking the right model isn't just about LLM size. Hybrid AI approaches (symbolic + LLM) can dramatically improve reasoning.  🔹 Action Execution & Automation – AI isn't useful if it just predicts—it must act. Multi-agent orchestration and real-world automation (Zapier, LangChain) are key.  🔹 Monitoring & Performance Optimization – AI drift and hallucinations are inevitable. Continuous tracking and retraining keeps your AI reliable.  🔹 Personalization & Adaptive Learning – AI must learn dynamically from user behavior. Reinforcement learning from human feedback (RHLF) improves responses over time.  🔹 Compliance & Ethical AI – AI must be explainable, auditable, and regulation-compliant (GDPR, HIPAA, CCPA). Otherwise, your AI can’t be trusted.  An AI agent isn’t just a model—it’s an ecosystem. Designing it well means balancing performance, reliability, security, and compliance.  The gap between an experimental AI and a production-ready AI is strategy and execution.  Which of these areas do you think is the hardest to get right?

Last week, Chinese AI company DeepSeek shocked the AI industry with the release of R1, their open-sourced reasoning model. Yesterday, the stock market noticed too. To help us understand the significance of this technological and geopolitical moment, I’ve co-authored a piece in The Washington Post about DeepSeek and open-source models. DeepSeek-R1, which matches models like OpenAI’s o1 in logic tasks including math and coding, costs only 2% of what OpenAI charges to run, and was built with far fewer resources. And most importantly, it’s an open-source model, meaning that DeepSeek has published the model’s weights, allowing anyone to use them to create and train their own AI models.   Up until now, closed-source models like those coming out of American tech companies have been winning the AI race. But my co-author Dhaval Adjodah and I argue in our piece that DeepSeek-R1 should make us question our assumption that closed-source models will necessarily remain dominant. Open-source models may become a key component of the AI ecosystem, and the United States should not cede leadership in this space. As we conclude in our article: “America’s competitive edge has long relied on open science and collaboration across industry, academia and government. We should embrace the possibility that open science might once again fuel American dynamism in the age of AI.” It was a pleasure to collaborate on this article with Dhaval, whose company MakerMaker.AI is on the cutting-edge of AI technology, building AI agents that build AI agents. What do you think about the future of open vs. closed-source AI? Read the full op-ed here: https://lnkd.in/eXK5YdWk. 

Most people think having a human approve an AI decision means the decision is safe. It does not. 👀 There is a term for what actually happens when humans rubber stamp AI outputs under time pressure. Automation bias. It is one of the most documented and underreported risks in enterprise AI right now. After 13 years and 200+ deployments, here is what I have learned about building genuine oversight into AI systems. The human reviewing an output needs three things to actually be in the loop. They need to understand what they are reviewing. They need the context to catch what the model gets wrong. And they need to be genuinely empowered to say no without institutional pressure to simply keep moving. Most organisations have none of those three in place. They have a signature process. That is not the same thing. Before any high-stakes AI output reaches a decision point in your organisation, ask these questions. ➡️ Does the person approving this understand the underlying data well enough to catch an error? ➡️ Is there time built in for genuine review or just enough time to click approve? ➡️ What happens if someone says no? Is that genuinely supported? If the answer to any of those is no… you do not have human oversight. You have automation bias with a human signature attached. What does genuine human oversight look like in your organisation right now? #ai #leadership #futureofwork #artificialintelligence #aistrategy #teamhuman #intellectualatrophy #criticalthinking

I taught myself machine learning > 10 years ago. If I had to start again today, I wouldn’t touch models, LLMs, or agents first, as many AI experts suggest. I'd start with the math and the code. Ugly truth: 90% of people skip the foundations, then wonder why everything feels like magic or falls apart in production. If you want to be different, actually understand ML, not just copy-paste, this is the roadmap I'd follow: Start with fundamentals: Because no matter how fast LLMs or GenAI evolve, your math, code, and logic will keep you relevant. Here's what you should focus on: 📐 1. Linear Algebra Learn these core ideas: Vectors, matrices, tensors Matrix multiplication (dot products, broadcasting) Transpose, inverse, rank, determinants Eigenvalues & eigenvectors (especially for PCA & embeddings) Projections and orthogonality ✅ Use NumPy to implement everything yourself → Practice matrix ops, dot products, and visualizing transformations with Matplotlib 🔁 2. Calculus Focus on: Derivatives & partial derivatives Chain rule (for backpropagation in neural nets) Gradient descent Convex functions, minima/maxima ✅ Use SymPy or JAX to visualize and compute derivatives → Plot functions and their gradients to develop deep intuition 🎲 3. Probability You need a solid grip on: Random variables (discrete & continuous) Conditional probability & Bayes' rule Joint & marginal probability The Chain rule Expectation, variance, entropy Common distributions: Bernoulli, Binomial, Gaussian, Poisson Central limit theorem The law of large numbers ✅ Simulate simple probability experiments in Python with NumPy → E.g. simulate sampling from distributions 📊 4. Statistics These are must-know topics: Descriptive stats: mean, median, mode, standard deviation Hypothesis testing: p-values, confidence intervals, t-tests Correlation vs. causation Sampling, bias, and variance Overfitting/underfitting A/B testing basics ✅ Use Pandas & SciPy to explore real datasets → Calculate descriptive stats, create histograms/box plots, run t-tests 🔧 Essential Python libraries to learn early NumPy – for vectorized math and fast array ops Pandas – for loading, cleaning, and analyzing tabular data Matplotlib / Seaborn – for plotting and visualizing distributions, relationships, and trends SymPy – for symbolic math and calculus SciPy – for stats, optimization, and numerical methods Use Jupyter Notebooks(to combine math, code, & visuals in one place) 📚 Best resources to nail the fundamentals: ✅ Machine Learning Foundations Math series (ML Foundations: Linear Algebra, Calculus, Probability, and Statistics)-series of 4 courses that I've created together with LinkedIn learning ✅ Hands-On ML with TensorFlow & Keras book by Aurélien Géron ✅ The Hundred-page Machine Learning Book by Andriy Burkov If you want to become an actual ML engineer, not just someone who watches and copies demos, start here. ♻️ Repost to help others💚

Everyone's talking about LLMs. I went a different direction 🧠 While everyone's building RAG systems with document chunking and vector search, I got curious about something else after Prof Alsayed Algergawy and his assistant Vishvapalsinhji Parmar's Knowledge Graphs seminar. What if the problem isn't just retrieval - but how we structure knowledge itself? 🤔 Traditional RAG's limitation: Chop documents into chunks, embed them, hope semantic search finds the right pieces. But what happens when you need to connect information across chunks? Or when relationships matter more than text similarity? 📄➡️❓ My approach: Instead of chunking, I built a structured knowledge graph from Yelp data (220K+ entities, 555K+ relationships) and trained Graph Neural Networks to reason through connections. 🕸️ The attached visualization shows exactly why this works - see how information naturally exists as interconnected webs, not isolated chunks. 👇🏻 The difference in action: ⚡ Traditional RAG: "Find similar text about Italian restaurants" 🔍 My system: "Traverse user→review→business→category→location→hours and explain why" 🗺️ Result: 94% AUC-ROC performance with explainable reasoning paths. Ask "Find family-friendly Italian restaurants in Philadelphia open Sunday" and get answers that show exactly how the AI connected reviews mentioning kids, atmosphere ratings, location data, and business hours. 🎯 Why this matters: While others optimize chunking strategies, maybe we should question whether chunking is the right approach at all. Sometimes the breakthrough isn't better embeddings - it's fundamentally rethinking how we represent knowledge. 💡 Check my script here 🔗: https://lnkd.in/dwNcS5uM The journey from that seminar to building this alternative has been incredibly rewarding. Excited to continue exploring how structured knowledge can transform AI systems beyond what traditional approaches achieve. ✨ #AI #MachineLearning #RAG #KnowledgeGraphs #GraphNeuralNetworks #NLP #DataScience 

Use this Super Simple Post to Understand the Evolution of AI Agents in 6 Key Phases.     Often, I see confusion surrounding the development pathway from basic LLMs to fully-fledged AI Agents.   To clear the fog, I've put together a straightforward, step-by-step visualization that encapsulates the entire evolutionary journey.   Remember, this isn't merely a technical diagram, but harmoniously intertwined view of how AI systems have evolved to become increasingly capable and autonomous.   👉 Phase 1: The Foundation - Basic LLM - Simple workflow: Input (Text) → LLM → Output (Text) - Transformer-based architecture trained on vast datasets - Limited to text processing within context window - No external tools or memory capabilities   👉 Phase 2: Document Processing Capabilities - Enhanced workflow: Input (Text/Documents) → LLM → Output (Text/Documents) - Expanded context window for processing larger documents - Improved tokenization for handling structured content - Limited by static knowledge from training data   👉 Phase 3: Introduce RAGs and Tool Integration to: - Enable access to up-to-date information - Supplement LLM knowledge with external data - Improve factual accuracy and reduce hallucinations - Support specialized operations through API calls   👉 Phase 4: Integrating Memory Systems to: - Maintain context across interactions - Enable personalization based on past exchanges - Store and retrieve relevant information - Support long-running tasks and conversations   👉 Phase 5: Implement Multi-Modal Processing by: - Handling diverse input types (text, images, tables) - Generating varied output formats - Creating more comprehensive understanding - Enabling richer information exchange   👉 Phase 6: Future of AI Agent Architecture through: - Chain-of-thought processing for complex problems - Step-by-step evaluation of solutions - Dynamic tool selection based on tasks - Goal-oriented execution with self-correction   If you're looking to implement AI agents in your systems, understanding this evolutionary path is crucial.   Here are some additional tips for building AI Agents:   Start small. Don't try to build a fully autonomous agent with all capabilities at once.   Start with enhancing a basic LLM with one capability (like RAG) and then gradually add more components as you validate each integration.   Integrate thoughtfully. The more capabilities you add to your agent, the more complex the system becomes.   Monitor extensively. Track not just technical metrics but also output quality, hallucination rates, tool usage patterns, and user satisfaction to continuously refine ai agents.   Here are key capabilities to build into your architecture:   🧠 Strong Foundation LLM 🔄 Effective RAG Implementation 🛠️ Versatile Tool Use Integration 💾 Contextual Memory Systems 🖼️ Multi-Modal Processing 🔍 Self-Monitoring Capabilities 🔒 Safety Systems   Over to you: What fascinate you most about the future architecture of AI agents?

AI agents are not yet safe for unsupervised use in enterprise environments The German Federal Office for Information Security (BSI) and France’s ANSSI have just released updated guidance on the secure integration of Large Language Models (LLMs). Their key message? Fully autonomous AI systems without human oversight are a security risk and should be avoided. As LLMs evolve into agentic systems capable of autonomous decision-making, the risks grow exponentially. From Prompt Injection attacks to unauthorized data access, the threats are real and increasingly sophisticated. The updated framework introduces Zero Trust principles tailored for LLMs: 1) No implicit trust: every interaction must be verified. 2) Strict authentication & least privilege access – even internal components must earn their permissions. 3) Continuous monitoring – not just outputs, but inputs must be validated and sanitized. 4) Sandboxing & session isolation – to prevent cross-session data leaks and persistent attacks. 5) Human-in-the-loop, i.e., critical decisions must remain under human control. Whether you're deploying chatbots, AI agents, or multimodal LLMs, this guidance is a must-read. It’s not just about compliance but about building trustworthy AI that respects privacy, integrity, and security. Bottom line: AI agents are not yet safe for unsupervised use in enterprise environments. If you're working with LLMs, it's time to rethink your architecture.