This year’s ACL featured more than 2,200 main conference papers, 2,100 findings, and a growing industry track, reflecting a convergence of academic rigor and production-driven innovation. Despite this scale, the acceptance rate remained around 18%, reinforcing the continued selectivity and quality bar of the venue.

Amid this, I had the opportunity to present our work on CascadeDebate, which tackles a very practical inefficiency in production AI systems: the tendency to escalate uncertain cases too quickly to larger models or human experts. Instead, we introduce a lightweight debate mechanism where smaller models resolve uncertainty collaboratively before escalation. The result is simple but powerful, smaller models outperform significantly larger standalone systems, while reducing expert intervention costs.

A Field Shifting Its Priorities

One of the more interesting signals from ACL 2026 was not just what people are building, but what the community is prioritizing. The largest research category this year was Resources and Evaluation (~10.2%), which is telling. After years of rapid model scaling, the field is now investing heavily in how we measure progress. Static benchmarks are increasingly seen as insufficient, and there is a growing push toward dynamic, robust, and context-aware evaluation frameworks.

This shift is also reflected in emerging themes:

• Agentic systems and multi-agent reasoning
• Multimodal grounding beyond text-only intelligence
• Safety, alignment, and responsible deployment
• Bias and robustness in evaluation pipelines

Geographically, contributions were dominated by China and the United States, highlighting both the scale and competitiveness of research in these regions.

Papers That Stood Out

Several works this year captured deeper structural challenges in current LLM paradigms.

A particularly interesting direction was around uncertainty and self-awareness in models. For example, “Beyond ‘I Don’t Know’” explores whether models can distinguish between data uncertainty and model uncertainty — a crucial step toward reliable deployment. This aligns closely with real-world needs where abstention alone is not enough; systems must reason about why they are uncertain. In parallel, work like “Early-Token Confidence Predicts Reasoning Quality” suggests that signals available very early in generation can predict downstream reasoning success. This opens the door to adaptive inference strategies, something highly relevant for latency-sensitive systems.

Another cluster of papers focused on LLMs as evaluators. Findings such as “Prior Beliefs Prejudice LLM-as-Judge” and “Stability vs. Manipulability” highlight a fundamental issue: evaluation models themselves are not neutral. They inherit biases, can be manipulated, and may not remain stable under interaction. This raises important questions about the reliability of automated evaluation pipelines that many systems now depend on.

Multi-Agent Systems Are Maturing. Multi-agent reasoning has moved from a conceptual curiosity to a more structured research direction. Papers like “A Diagnostic Study of Multi-Agent LLMs for Real-World Debates” and “Semantic Equivalence Self-Play with Formal Verification” show that the community is beginning to systematically analyze how agents collaborate, debate, and refine outputs.

What stood out to me is that we are starting to see mechanistic insights, not just performance gains. For instance:

• Debate frameworks are being analyzed rather than just benchmarked
• Confidence signals are being used as control variables
• Formal verification is entering the loop for reasoning tasks

This is a meaningful shift toward understanding rather than just scaling.

Creativity, Bias, and Latent Spaces

Another interesting direction explored more cognitive aspects of AI.

The work on divergent thinking in latent space reframes generation as intention optimization rather than discrete search. This aligns with a broader trend of moving away from token-level heuristics toward more structured representations of intent and reasoning.

At the same time, papers like “Objectifying the Subjective” and work on cognitive biases in topic interpretation highlight that even as models improve, they continue to inherit and amplify subtle biases. The challenge is no longer just performance, it is interpretability and trust.

Evaluation Is Being Rewritten

One of the most important signals from ACL 2026 is that evaluation itself is undergoing a redesign.

Work like BeyondBench proposes moving away from fixed benchmarks toward more flexible, scenario-driven evaluation. This reflects a growing realization: models are no longer static systems solving static tasks. Evaluation must match the dynamic, interactive nature of modern AI systems. Similarly, context-sensitive studies in acceptability judgment show that even fundamental NLP tasks are far more dependent on context than previously assumed.

Personal Takeaways

ACL 2026 reinforced a few key ideas for me:

• Scaling alone is no longer sufficient; efficiency and control are becoming first-class concerns.
• Evaluation is now a central research problem, not a supporting tool.
• Multi-agent systems are transitioning from heuristics to principled frameworks.
• Bias, robustness, and reliability are moving from “nice-to-have” to essential requirements.

Most importantly, the field feels like it is entering a phase of consolidation and reflection, where we are not just building more powerful models, but trying to understand, evaluate, and control them in meaningful ways. If the past few years were about demonstrating what large models can do, ACL 2026 suggests the next phase is about making them usable, reliable, and aligned with real-world constraints. The problems are becoming less about raw capability and more about how these systems behave under uncertainty, interaction, and scale.

And that shift, in many ways, is where the most interesting research now lies.

Appendix: Extended List of selected articles

The following papers represent a selection of notable works presented at the conference, covering diverse topics.

The Imperfective Paradox in Large Language Models

• Link: https://aclanthology.org/2026.acl-long.689/
• This paper explores the imperfections and paradoxes observed in large language models when dealing with imperfective aspects of language. It highlights the challenges LLMs face in understanding and generating nuanced temporal and aspectual information, which is crucial for accurate natural language understanding.

Influence-based Online Experience Selection for Effective RLHF

• Link: https://aclanthology.org/2026.acl-long.2206/
• This work focuses on improving Reinforcement Learning from Human Feedback (RLHF) by introducing an influence-based online experience selection mechanism. The method aims to more effectively leverage human preferences to guide model training, leading to more aligned and performant LLMs.

Beyond Discrete Search: Divergent Thinking as Intention Optimization in Latent Space

• Link: https://aclanthology.org/2026.acl-srw.88/
• This paper explores divergent thinking in AI, moving beyond traditional discrete search methods. It proposes an intention optimization framework in latent space, enabling AI systems to generate more creative and varied solutions, mimicking human-like innovative thought processes.

Prior Beliefs Prejudice LLM-as-Judge: Evidence from Persuasion Evaluation ConvinceQA + RevealQA Datasets

• Link: https://aclanthology.org/2026.findings-acl.2087/
• This paper investigates how prior beliefs can prejudice LLMs when used as judges in persuasion evaluation tasks. Using the ConvinceQA and RevealQA datasets, the study provides evidence of biases, highlighting the need for careful consideration when deploying LLMs in evaluative roles.

Assessing the Effect of Context in Multi-domain Acceptability Judgment

• Link: https://aclanthology.org/2026.findings-acl.2096/
• This paper examines the impact of context on multi-domain acceptability judgments in NLP. It provides insights into how different contextual factors influence human and model judgments of linguistic acceptability across various domains, contributing to more robust evaluation methodologies.

Role-Sensitive Neurons: A Neuron-Level Gain Control Mechanism for Confidence Steering

• Link: https://aclanthology.org/2026.findings-acl.294/
• This work introduces a novel concept of role-sensitive neurons and a gain control mechanism at the neuron level to steer the confidence of neural networks. This has implications for improving the reliability and interpretability of AI models.

Improving LLM Code Reasoning via Semantic Equivalence Self-play with Formal Verification

• Link: https://aclanthology.org/2026.findings-acl.1615/
• This research focuses on improving the code reasoning abilities of LLMs through a semantic equivalence self-play mechanism combined with formal verification. This innovative approach helps LLMs generate more accurate and robust code, addressing a significant challenge in AI-assisted programming.

Objectifying the Subjective: Cognitive Biases in Topic Interpretations

• Link: https://aclanthology.org/2025.tacl-1.70/
• This research investigates cognitive biases in how AI models interpret topics, particularly in subjective contexts. It sheds light on the inherent biases that can influence topic modeling and interpretation, advocating for more robust and fair AI systems.

Beyond "I Don't Know": Evaluating LLM Self-Awareness in Discriminating Data and Model Uncertainty

• Link: https://aclanthology.org/2026.acl-long.547/
• This paper delves into the self-awareness of LLMs, moving beyond simple "I don't know" responses to evaluate their ability to discriminate between data uncertainty and model uncertainty. This research is vital for building more reliable and trustworthy AI systems.

STABILITY VS. MANIPULABILITY Evaluating Robustness Under Post-Decision Interaction In LLM Judges

• Link: https://aclanthology.org/2026.gem-main.57/
• This paper investigates the robustness of LLM judges under post-decision interaction, analyzing their stability versus manipulability. The findings are crucial for understanding the reliability of LLMs in critical evaluation tasks and for designing more resilient AI systems.

Early-Token Confidence Predicts Reasoning Quality in Multi-Agent LLM Debate

• Link: https://aclanthology.org/2026.gem-main.60/
• This research demonstrates that the confidence expressed by LLMs at early tokens can predict the overall reasoning quality in multi-agent LLM debates. This insight can be used to develop more efficient and effective multi-agent AI systems by identifying promising reasoning paths early on.

A Diagnostic Study of Multi-Agent LLMs for Real-World Debates

• Link: https://aclanthology.org/2026.acl-long.1527/
• This study provides a diagnostic analysis of multi-agent LLMs in the context of real-world debates. It identifies key challenges and opportunities for improving the performance and reliability of AI systems engaged in complex argumentative interactions.

BeyondBench: Benchmark-Free Evaluation of Reasoning in Language Models

• Link: https://aclanthology.org/2025.acl-long.100/
• BeyondBench introduces a benchmark-free evaluation methodology for assessing reasoning capabilities in language models. This approach offers a more flexible and comprehensive way to evaluate LLMs, moving beyond static benchmarks to understand their true reasoning potential.

LLMs, though powerful, often generate unhelpful or unpredictable outputs if left solely to their pretraining data. Post-training alignment algorithms steer models to match human preferences — whether that’s politeness, factuality, or safe reasoning — by rewarding desired behavior and discouraging mistakes.

Core Post-training Techniques

Supervised Fine-Tuning (SFT)

The most direct approach, SFT, teaches models with curated human-written examples, laying a foundation for desired behaviors before deeper optimization.

Reinforcement Learning from Human Feedback (RLHF)

PPO and its variants, like GRPO and DAPO, use reward models trained on user preferences to further refine outputs, constantly balancing exploration and accuracy. PPO’s clipping mechanism protects against destabilization while GRPO and DAPO offer efficient alternatives for complex tasks.

Preference Optimization: PPO, DPO and Beyond

DPO reframes alignment as a simple loss function that increases the likelihood of preferred answers — no critic required — streamlining the process for scalable deployment. Models like PPO and GSPO use advantage estimates to foster robust policy learning, making alignment less resource-intensive and more dynamic.

Cutting-edge alignment in 2025 blends supervised data, RL-based tuning, and self-play. New methods like SPIN, RSPO, and AMPO extend flexibility, allowing LLMs to adapt to more subjective and varied human needs, while keeping outputs safe and accurate. Attached is a list of papers to build the understanding:

1. PPO (Proximal Policy Optimization) -> Proximal Policy Optimization Algorithms (1707.06347) Clips the probability ratio to prevent the new policy from diverging too far from the old one. It helps keep everything stable
2. DPO (Direct Preference Optimization) -> Direct Preference Optimization: Your Language Model is Secretly a Reward Model (2305.18290) It’s a non RL method, where an LM is an implicit reward model. It uses a simple loss to boost the preferred answer’s probability over the less preferred one
3. GRPO (Group Relative Policy Optimization) -> DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models (2402.03300) An RL method that compares a group of model outputs for the same input and updates the policy based on relative rankings. It doesn’t need a separate critic model
4. DAPO (Decoupled Clip and Dynamic sAmpling Policy Optimization) -> DAPO: An Open-Source LLM Reinforcement Learning System at Scale (2503.14476) Decouples the clipping bounds for flexibility, introducing 4 key techniques: clip-higher (to maintain exploration), dynamic sampling (to ensure gradient updates), token-level loss (to balance learning across long outputs), and overlong reward shaping (to handle long, truncated answers)
5. Supervised Fine-Tuning (SFT) -> Training language models to follow instructions with human feedback (2203.02155) Often the first post-pretraining step. A model is fine-tuned on a dataset of high-quality human-written input-output pairs to directly teach desired behaviors
6. Reinforcement Learning from Human Feedback (RLHF) -> https://huggingface.co/papers/2203.02155 The classic approach that combines supervised fine-tuning with RL on a reward model trained from human preference data.
7. Monte Carlo Tree Search (MCTS) -> https://huggingface.co/papers/2305.10601 This planning algorithm builds a search tree by simulating many reasoning paths from the current state, balancing exploration and exploitation.
8. AMPO (Active Multi-Preference Optimization) -> https://huggingface.co/papers/2502.18293
   Combines on-policy generation, contrastive learning, and smart selection of training examples. From many possible responses, it picks a small, diverse set with both high- and low-quality answers and unique styles
9. SPIN (Self-Play Fine-Tuning) -> https://huggingface.co/papers/2401.01335 Uses self-play, where the model learns by comparing its own generated responses to earlier outputs and human examples
10. SPPO (Self-Play Preference Optimization) -> https://huggingface.co/papers/2405.00675 Aligns LMs by framing training as a two-player game where the model learns to improve against itself through preference comparisons, aiming to reach a Nash equilibrium
11. RSPO (Regularized Self-Play Policy Optimization) -> https://huggingface.co/papers/2503.00030 Lets models learn through self-play, with an extra regularization term added to keep training stable. It achieves best results with a linear combination of forward and reverse KL divergence regularization

I’ll try to collect my thoughts and share, for reference, what I noted during ACL 2025. To begin with — it was the largest NLP conference in history, and I’m genuinely happy to have been a part of it. [Feel free to comment and extend the article with your thoughts, if you are attending too.]

Backstory: My paper got accepted this year to the ACL Industry Track, so I got the chance to attend the conference — albeit virtually, not in person. To be honest, it felt surreal seeing so many of my NLP heroes (almost) all in one place — Prof. Christopher Manning, MSR’s Sunayna Sitaraman, Cohere’s rockstar Sarah Hooker, and the legend Daniel Jurafsky (joining virtually too, so I didn’t feel too left out!). I also saw a few familiar names pop up — friends like Letitia (aka Miss Coffee Bean from YouTube), and delegations from pretty much all the major labs: Google, Meta, LinkedIn, Apple, JPMC, Microsoft, Huawei, MILA, IIT Madras, and several other research hubs.

As has already been widely shared online, the paper stats were no surprise — the leaderboard was dominated by China, US, South Korea, India, and Germany.

I’m still not sure how they managed to squeeze so much into such a tight schedule. That was honestly one of the main concerns — every presentation was limited to just 10 minutes (7 minutes for the author talk + 3 minutes for questions), and more often than not, the Q&A part was skipped due to time constraints.

With so many parallel sessions running at the same time, it got a bit overwhelming to decide where to go — one room vs another. So, I took a more focused approach: I tried to attend talks that covered things that were either new, genuinely exciting, or that I could relate to (especially if I had worked on something similar in the past). The keywords that drew me in were things like: self-evaluative models, post-training techniques, low-precision training, multimodal alignment, reasoning and planning methods, LLM alignment techniques, jailbreaking, attention mechanisms, sparsity, multilingualism, PEFT techniques, quantization, reward optimization, and agent-based systems.

I must say — my taste isn’t bad at all (as I later confirmed during the Award Ceremony 😄). Turns out, out of the four Best Paper Awards distributed, two were from the sessions I had attended! These were:

1. A Theory of Response Sampling in LLMs: Part Descriptive and Part Prescriptive Read here
2. Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention

Another paper that, in my opinion, deserved a top spot — and was highlighted by the keynote speaker — was Byte Latent Transformer: Patches Scale Better Than Tokens, which explored the idea of training on patches of text instead of tokens. The results were compelling and might even hint at the direction models like LLaMA 4 could take. I’d also place Language Models Resist Alignment: Evidence From Data Compression in that same league — sharp, thought-provoking, and well-executed. For anyone curious, the full list of Best Paper and Area Chair Recommendations can be found here.

The major star-performing LLMs that were on everyone’s lips this year were LLaMa, Gemini, GPT-4-o3, DeepSeek, and the distilled versions of Qwen and Grok — forming the usual suspects of the top list. At first glance, it honestly seemed like most people were just using these models — no fine-tuning, no architectural changes — just building applications on top of them. But as I dug deeper into the sessions, I found quite a few works that were doing exactly what I initially thought they weren’t! That was a pleasant surprise. That said, the majority were still focused on using LLMs as agents to power specific applications.

A lot of exciting work was also happening in the multimodal space — moving beyond just-text. This trend was echoed in the closing ceremony as part of the “future predictions” segment. Although to be fair, this prediction had already been quietly shaping the landscape throughout the conference.

Being someone who’s quite alignment-focused, I really enjoyed how Meta is thinking about combining Byte Latent Transformer learning with online policy learning algorithms. This was brought up during the keynote address, and I found it particularly exciting. The keynote speaker also shared a curated list of papers he’s personally keeping an eye on — a goldmine for anyone interested in where things are headed.

As I continue collecting more papers for my reading list, I thought I’d start sharing a few that really stood out to me. These are mostly centered around foundational innovations or alignment theory for LLMs — areas close to my heart.

1. Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyond
2. SimBench: Benchmarking the Ability of Large Language Models to Simulate Human Behaviors
3. Think Again! The Effect of Test-Time Compute on Preferences, Opinions, and Beliefs of Large Language Models
4. CRPO: Confidence-Reward Driven Preference Optimization for Machine Translation
5. Large Vocabulary Size Improves Large Language Models
6. EnerGIZAr: Leveraging GIZA++ for Effective Tokenizer Initialization
7. MoRE: A Mixture of Low-Rank Experts for Adaptive Multi-Task Learning
8. Drop Dropout on Single Epoch Language Model Pretraining
9. RaaS: Reasoning-Aware Attention Sparsity for Efficient LLM Inference with Long Decoding Chains
10. Revisiting Weak-to-Strong Generalization in Theory and Practice: Reverse KL vs. Forward KL
11. Position-Aware Depth Decay Decoding (D3): Boosting Large Language Model Inference Efficiency
12. DALR: Dual-level Alignment Learning for Multimodal Sentence Representation Learning
13. Natural Language Reasoning in Large Language Models: Analysis and Evaluation
14. Self-Improvement Towards Pareto Optimality: Mitigating Preference Conflicts in Multi-Objective Alignment
15. SGDPO: Self-Guided Direct Preference Optimization for Language Model Alignment
16. Token-level Preference Self-Alignment Optimization for Multi-style Outline Controllable Generation
17. VITAL: A New Dataset for Benchmarking Pluralistic Alignment inHealthcare
18. EfficientQAT: Efficient Quantization-Aware Training for Large Language Models
19. Beyond Browsing: API-Based Web Agents
20. Navigating Rifts in Human-LLM Grounding: Study and Benchmark
21. TokAlign: Efficient Vocabulary Adaptation via Token Alignment
22. Byte Latent Transformer: Patches Scale Better Than Tokens
23. SPECTRA: Faster Large Language Model Inference with Optimized Internal and External Speculation
24. Training Dynamics Underlying Language Model Scaling Laws: Loss Deceleration and Zero-Sum Learning
25. SimUSER: Simulating User Behavior with Large Language Models for Recommender System Evaluation
26. Intuitive Fine-Tuning: Towards Simplifying Alignment into a Single Process
27. Nudging: Inference-time Alignment of LLMs via Guided Decoding
28. Language Models Resist Alignment: Evidence From Data Compression
29. MPO: Multilingual Safety Alignment via Reward Gap Optimization
30. Unintended Harms of Value-Aligned LLMs: Psychological and Empirical Insights
31. Language Models Resist Alignment: Evidence From Data Compression

Inference scaling or Test-time scaling is the latest buzzword capturing the interest of everyone working in the high-boiling field of AI research. With no fundamentally new methods, all the previous prompt-based or incrementally advancing research is now neatly repackaged under this new label. But the ideas being discussed remain relevant, with meaningful contributions continuing to emerge.

Today, we’re diving into some key questions in this domain using a recent survey article by Zang et al., titled “What, How, Where, and How Well? A Survey on Test-Time Scaling in Large Language Models.” We’ll explore what techniques researchers are using to scale test-time resources for better answers, how these methods are applied to different datasets/problems, and which metrics are used for evaluation. For a detailed review of papers in each category, I highly recommend checking out this and other relevant works in the field.

The type of scaling technique can be easily classified into four major brackets: Sequential, Parallel, Hybrid, and Internal scaling. In the Sequential pathway, we focus on increasing the token budget, forcing the model to generate more output when it’s close to the EOS token prediction. In the Parallel pathway, we generate multiple completions for a problem and then select the most promising one based on some metric.

In the Hybrid approach, we perform Divergent-Convergent thinking — first generating parallel solutions and then refining the final response through verification or heuristics. Internal reasoning, however, is a completely different beast — it requires tuning models to reason at the learning stage and mimic the same behavior during generation, eliminating reliance on any prompting strategy or specific token choices.

Whether fine-tuning is used or not, even a small amount of training can provide a significant performance boost. Fine-tuning techniques can be conventional, such as Supervised Fine-tuning (SFT) with larger, more reasoning-oriented datasets distilled from stronger models, or Reinforcement Learning (RL) methods that may (or may not) involve reward models to quantify the importance of generated outputs.

For inference-only techniques, the simplest approach is using different prompting strategies like Chain-of-Thought (CoT), Self-consistency, Step-back reasoning, or task-specific prompts. Another approach is leveraging multiple models and selecting the best response. Further tuning can be achieved using Process Reward Models (PRMs) to verify the final output or each intermediate step. If keeping it even simpler, search-based approaches or ensemble methods can also be used to improve reliability.

In theory, TTS can be applied everywhere, but past studies have primarily focused on reasoning-heavy problems. Some domains naturally lend themselves to step-by-step verification, such as formal verification of generated code or mathematical theorem proving. However, agents, knowledge-based Q&A, and multimodal applications are also well-explored areas.

The relevance of scaling can be assessed using all the usual metrics — performance, efficiency, scalability, and controllability of generation. Beyond just accuracy for task completion, the focus extends to factors like number of tokens, number of LM calls, FLOPs used on GPUs, cost, cache size, and robustness to different inputs.

Overall, Test-Time Scaling is a meaningful direction for improving responses based on the training an LLM has already received, maximizing the knowledge stored within its parameters. While there’s no magical new technique, refining inference strategies is an exciting way to push the limits of what large models can achieve without requiring additional large-scale training.

For more details, check out the original paper here: https://arxiv.org/pdf/2503.24235

Gone are the days when a basic understanding of machine learning sufficed for tech interviews. Today, employers seek candidates who can navigate the intricacies. If you are looking for basic Machine Learning topics to prepare, I would encourage you to check my Data Science Interview Preparation blog. In this article, I go beyond the basics, providing you with an extensive interview questions bank that will not only test your knowledge but also challenge you to apply these concepts in real-world scenarios. Question bank is prepared by studying the Google DeepMind, Google Brain, Microsoft, Samsung Research, OpenAI and Meta’s interview details shared online along with theoretical and practical observations, collected by self-reading papers in particular domain of choice such as Deep Learning, Transformer, LLMs and others.

The interview room can be an intimidating space, especially when the discussion revolves around complex concepts like Deep Learning, Natural Language Processing, and Diffusion Models. To equip you with the confidence and knowledge needed to excel, I present a treasure trove of interview questions that cover the breadth and depth of these fascinating domains. Let’s unravel the secrets to cracking the code of your next technical interview. In case, there are any other related questions/topics in your mind, please feel free to add them to comments of this blog. If you are looking to practise DL/MLE/Software interview live with me, please reach out me at Topmate.

Disclaimer: This blog do not in any way represent the views of my present or any previous employer(s).

The topics covered in this post include:

• Probability and Information Theory
• Optimization
• Gradient Descent, LR, Activation functions and Optimizers
• Transformer
• Diffusion Models

Probability and Information

• What is Random Variable
• What are some of famous Probability distributions: Bernoulli, Multinoulli, Gaussian, Laplace, Exponential, Mixture-of-Distributions(GMM)
• Expectation, variance, covariance and correlation
• How would you find the 95% confidence intervals for a given mean?: find 2sigma
• What is Information and Entropy?: Learning that a less likely event has happened is more informative then learning that a likely event has happened. Therefore we measure, self-information of an event given the prob distribution as I(X=x) = -log(P(X=x)) nats while Expected information in an event drawn from prob distribution is Entropy, H(x) = E[x~P](I(x)) = -∑(p_i*log(p_i))
• Distributions that are nearly deterministic have low entropy otherwise high entropy
• What is KL Divergence: If we have two separate prob distributions P(x) and Q(x) over the same random variable, we can measure how different these two distribution brae using the KL Divergence; Mathematically KL(P||Q) = E[x~P](log(P(x)/Q(x)))

Optimization

• What is difference between MLE and MAP estimation
• Difference in optimization and regularization
• Optimization: refers to task of either minimizing or maximizing some function f(x) by altering x
• Regularization: Any changes made to the ML algorithm such that its performance on training data do not changes, but it improves for unseen test data. For instance, Regularization methods aim to mitigate this issue of overfitting by introducing additional constraints or penalties on the learning algorithm.
• What is the difference in convex and concave optimisation?
• What are the benefits of convex function for optimization?
• Convex optimisation algos are applicable only to convex functions(function for which Hessian is Positive semi definite everywhere(Eigen values are positive or 0)). Such functions are well-behaved because they lack saddle points and all their local minima are necessarily global minima. Most functions in DL are difficult to express as Convex optimisation
• What are some of the commonly used optimization technqiues in neural networks: Gradient based optimization like GD, SGD, RMSProp, Adam, Ada, Momentum, Nestrov. Nature inspired optimization like Genetic Algo, PSO, Differential Evolution, Bee Colony algos.
• How do we update the parameters using Gradient descent and why: During back propagation, we update parameters by subtracting negative of gradient wrt input. Negative of gradient points in a direction which minimizes the value of cost function
• What is the role of learning rate in gradient descent?: The LR must be small enough to avoid overshooting minimum and gaining uphill in directions with strong positive curvature
• What are Jacobian and Hessian Matrices of a function?
• What is the role of second derivative in optimisation?: This is important because it tells whether a gradient step will cause as much of an improvement as we would expect based on gradient alone. It measures curvature I.e. concave or convex
• What is Saddle point?
• What is Taylor series expansion?
• What is Newtons method?: Its a second order optimisation algorithm
• What is conditioning number?: Conditioning refers to how rapidly a function changes wrt small changes in its inputs. f(x) = A(-1)x and A has an Eigen value decomposition then its condition number = max[i,j] abs(𝜆i/𝜆j), i.e. the ration of largest and smallest eigenvalue. In multiple dimensions, there can be variety of second derivatives at a single point, because of second order derivative in each direction. The conditioning number of hessian measures how much the second derivative vary. Poor conditioning number implies poor performance of gradient descent as well.
• What are draw backs of using gradient descent?: GD fails to exploit the curvature information contained in the Hessian Matrix. Cost function of NN is neither convex or concave like sin(x) for x belongs to R with multiple maxima and minima

Gradient Descent, LR, Activation functions and Optimizers

• Difference in GD, SGD, mini-batch GD
• GD: Optimisation algo, used to minimize some function by iteratively moving in a direction of steepest descent as defined by negative of gradient. Since in SGD, only one sample from dataset is chosen at random for each iteration, path taken by also to reach minima is noisy than vanilla GD. Goal is to reach minima fastly by taking greedy decisions
• Difference in GD, Momentum and Nestrov GD — Momentum: Takes into account past gradients to smooth out the update by taking a exponential weighted average. Nestrov: It works on the principle of “Look before you leap”
• How to tune LR in NN training? Are there any particular optimizers doings so?: Annealing LR: Step decay, Exponential decay, Adaptive LR
• Optimizer which adapt LR — Adagrad, RMSprop, Adam, AdamW=> Adagrad: Parameters that have higher gradient or frequent updates, should have slower LR, so that we do not overshoot the minimum value, while params with lower gradient/infrequent updates should have higher LR. It decays LR very aggressively, so params will start reaching small updates because of decayed LR. It gets stuck when its close to convergence to minima
• RMSprop: It overcomes adagrad problem by being less aggressive to the decay of LR. It works by keeping an exponentially weighted average of the square of past gradients
• Adam: adaptive momentum estimation. Combine idea of Momentum and RMSprop. Two params are beta_1, beta_2. Convergence through momentum, model do not get stuck in saddle point
• AdamW: Adam with Weight Decay Regularisation.
• Implement Softmax — Possible issue: Numerical stability for Overflow and Underflow
• What is difference between Cross entropy and Binary Cross entropy
• What is vanishing and exploding gradient?
• What is ReLU activation? Are there any pros and cons of using it?
• Pros: Avoid & rectifies vanishing gradient. Less computationally expensive than tank or sigmoid. Cons: Can only be used within hidden layers. Some gradients can be fragile during training and can die. It results in Dead neuron. The neurons which go in that state, will stop responding to the variation in error. “Dying ReLU” problem: If a neuron’s output is consistently negative, the gradient is consistently zero, and the neuron effectively becomes inactive. Remedy is to use Leaky Relu or p-ReLU, Its an attempt to fix “dying ReLU” by having small negative slope when x is negative
• What is squashing in Sigmoid activation
• Problem with sigmoid — Saturate quickly and kill gradients
• What are some of the regularization techniques we can use while training a NN? — Data Augmentation, Using Dropout, Early Stopping, L1/L2 regularization, Batch Normalisation, Skip-connection: allows gradients to bypass certain layers, helping to address vanishing gradients
• Can I use L2 regularization to deal with vanishing gradient problem while training NN? — This will worsen the problem, by shrinking weights towards 0. Its main purpose is to control the complexity of the model and encourage smaller weights, rather than directly addressing vanishing gradients.
• How to create ensemble of NN, in a practically feasible manner? — Using dropout creates an ensemble of NN
• What if all weights are initialized with same value in a NN? — Each hidden unit gets same value and same activation output then. Same derivative of cost function during back prop. No learning/Under-fitting
• How does weight initialization impact the training process of a neural network? What are some of the techniques? — It can affect how quickly the network converges to a solution, the quality of the solution reached, and whether the network gets stuck in local minima. Xavier/Glorot Initialization: This method initializes weights with values drawn from a Gaussian distribution with zero mean and a variance, He Initialization, Uniform Initialization
• What if I initialize all weights to 0? — Initializing all weights to zero can cause issues since all neurons will learn the same features and gradients will be equal. This results in symmetric weights and no learning capacity.
• Describe the purpose and benefits of batch normalization in neural networks. — It normalize the activations of hidden layer, so that the weights of next layer can be updated faster. It tackles the problem of internal covariate shift
• What is internal covariate shift — It refers to the change in the distribution of activations within a neural network layer during training due to the changing input distribution.
• Can we do not use Batch Normalization in RNN or Transformer architectures — Batch Normalization was originally designed for feedforward neural networks and convolutional neural networks (CNNs). It aims to normalize the activations of each layer by adjusting them to have zero mean and unit variance. RNNs process sequential data over time steps. Traditional BN treats each time step as a separate batch, which can break the temporal dependencies in the data. Techniques like “Layer Normalization” or “Sequence Batch Normalization” have been proposed to address this issue.
• What will be the last layer activation function and loss-function for multi-label and multi-class classification — Multi-class: Softmax with cross entropy, Multi-label: Sigmoid with Binary cross entropy
• What is Catastrophic forgetting — The process of learning new information interferes with or overrides the previously learned information. Common in Sequence learning networks. As the network adapts to the new tasks, the weights and representations that were useful for the previous tasks might be altered → degradation in performance

Transformer

• What challenges do RNNs face when processing long sequences, and how does the Long Short-Term Memory (LSTM) architecture aim to address these issues?
• Transformers use attention. Attention helps to draw connections between any parts of the sequence, so long-range dependencies are not a problem anymore.
• What is attention?
• What is the purpose of attention mechanisms in neural networks?
• Why do we need Positional embeddings in transformer? — Since transformer model contains no recurrence and no convolution, in order for the model to make use of the order of the sequence, we must inject some information about the relative or absolute position of the tokens in the sequence. To this end, we add “positional encodings” to the input embeddings at the bottoms of the encoder and decoder stacks. The positional encodings have the same dimension as the learned word embeddings, so that the two can be summed.
• What is input to a transformer model for MT?
• How can I tokenise the text before training the Transformer model — Areas of concern: 1) Unit Selection, 2) Vocab Size, 3) handling languages with complex morphology, 4) out-of-vocabulary words, 5)Word level or White space tokenisation, 6) Character level: Text is tokenized at the character level, where each character becomes a token. Used in ELMO using 1-D convolution, 7) Sub-word level: To handle rare or OOV words/character sequences, 8) Byte-pair encoding: BPE works by iteratively merging the most frequent pairs of characters or subword units in the text. It starts by treating each character as a token and then merges pairs of tokens based on their frequency until a specified vocabulary size is reached. Its a greedy algo. 9) Word piece: WordPiece also breaks down words into subword units, but it performs the splitting based on maximizing a predefined criterion (such as likelihood or entropy) while maintaining a specified vocabulary size. Its an optimal algo.
• What different kind of attention mechanisms are utilized in Transformer?
• Self-attention in Encoder: All of the keys, values and queries come from the same place, in this case, the output of the previous layer in the encoder. Each position in the encoder can attend to all positions in the previous layer of the encoder
• Cross-attention in Decoder: The queries come from the previous decoder layer, and the memory keys and values come from the output of the encoder. This allows every position in the decoder to attend over all positions in the input sequence
• Why do we use Multi-headed self attention?
• What is masked multi-headed attention?
• What are QKV in attention(self and cross)?
• How to create mask for masked multi-head attention? — Create upper triangular matrix with negative infinity values and apply exp operation i.e. all values in the input of the softmax which correspond to illegal connections
• What are different decoding strategies for language modelling task — 1) Greedy decoding, 2) Beam Search: This strategy maintains a fixed number (beam width) of partial sequences in parallel. At each step, the model generates multiple candidate tokens and keeps the top candidates based on their probabilities, 3) Random Sampling, 4) Tok-K: This strategy limits the sampling to the top-k most likely tokens at each step. This allows for controlled randomness while ensuring that the generated sequences are coherent and contextually relevant. 5) Top-p/Nucleus sampling: It keeps the top-p most likely tokens at each step, where p is a probability threshold. This technique helps in maintaining diversity while controlling the number of options considered, 6) Temperature Scaling: Using Softmax temperature. Higher temperature values introduce more randomness, while lower values make the selection more deterministic.
• Why do we use Layer Norm in Transformer and not Batch Norm?
• In batch normalization, we use the batch statistics: the mean and standard deviation corresponding to the current mini-batch. However, when the batch size is small, the sample mean and sample standard deviation are not representative enough of the actual distribution and the network cannot learn anything meaningful.
• As batch normalization depends on batch statistics for normalization, it is less suited for sequence models. This is because, in sequence models, we may have sequences of potentially different lengths and smaller batch sizes corresponding to longer sequences.
• How BERT and GPT are different?
• What tasks BERT was pre-trained upon?
• Why do we consider bi-directional context in BERT and not in GPT?
• What is KV cache? — The KV cache in Transformers is a memory-efficient technique that stores key (K) and value (V) vectors from the encoder’s output to optimize the attention mechanism during decoding in autoregressive language models, leading to faster and more efficient generation of tokens.
• What is the time complexity of attention mechanism?
• Can we reduce this time complexity?
• Why there is Shift right token operation when giving inputs to decoder — Input to the decoder is the target sequence shifted one position to the right by the token that signals the beginning of the sentence. The logic of this is that the output at each position should receive the previous tokens (and not the token at the same position, of course), which is achieved with this shift together with the self-attention mask.

Diffusion Models

• Most of the questions along with results are mentioned with response here in this article. Noting down the questions ahead.
• Are there any assumptions for noise functions added at each step of Diffusion?
• Why only the noise added at each step is gaussian. Can I use any other probability distribution of noise?
• Why diffusion models perform better than GANs, Is there any intuition?
• What is the role of skip-connections in U-net architecture and how are they different from skip-connections in ResNet?
• Why the variance schedule have small value(i.e. β_t << 1) or why the step size is small?
• Why we need to do so many reverse steps to obtain clear image. What benefit it provides?
• Let’s compare different Generative architecture such as VAE, GANs and Diffusion. When should one prefer to use one architecture over the other?
• How is stable-diffusion different from DALLE-2 or IMAGEN model architectures. What added advantage do it provides?

Keep Learning, Keep Hustling

In the vast expanse of air travel, unexpected delays can become emotionally charged chapters in one’s journey. My recent experience with the British Airways flight “BA 256" from Delhi to London unfolded as an unforeseen saga, shedding light on the profound impact these delays can have on travelers.

The journey was meant to be a swift transition from the vibrant celebrations of Diwali in India to the routine of work life in Toronto. However, the wheels of time seemed to slow down as the scheduled departure approached. The initial excitement of boarding turned into an extended waiting game at Delhi Airport, with the flight BA 256 grappling with unforeseen delays.

As the minutes turned into hours, the emotional landscape of the passengers underwent a transformation. Anticipation evolved into frustration, and the shared camaraderie among fellow travellers morphed into collective concern. The ripple effect of delayed flights is not confined to the ticking clock but extends to the emotional well-being of those waiting anxiously.

Beyond the logistical intricacies, flight delays weave a tapestry of emotions, ranging from impatience to resilience. Each passenger carries a unique narrative, and the shared experience becomes a bond, even if fleeting, in the transient space of the Airport lounge and the Hotel rooms arranged for passengers by the respective airways.

The uncertainty of travel plans and the need to recalibrate expectations give rise to a shared vulnerability. Stranded in the departure lounge, we become storytellers of our journeys, sharing snippets of life with fellow passengers who, moments ago, were strangers.

I am sharing the stories of few fellow passengers with their prior permission to add into an online article. In sharing these personal narratives, we delve into the emotional intricacies of travellers who, like me, found their voyages disrupted by the twists of fate.

• Navigating Work-Life Balance: For me, the delayed flight turned into a tale of unexpected twists, introducing uncertainties about reaching my workplace in Toronto on time. The joy of returning home was eclipsed by the looming question of how the delay would impact my professional commitments, underscoring the delicate balance between personal and work life.

• A Lesson in Embracing Diversity: Among the passengers were 17 individuals from Europe on a school trip to India, eager to embrace the country’s diversity during the festive season. Their shared sentiment turned the delay into an opportunity to explore the vibrant streets of Delhi and savor the culinary delights near Connaught Place.
• The Long-Awaited Reunion: A couple from the Royal city of Patiala was on their way to Ottawa to reunite with their son after 12 long years. The delay, though tedious, added a unique layer to their journey. Their son’s excitement to welcome them had turned the delay into an unexpected but memorable part of their travel saga.
• Work Commitments in Disarray: The flight carried professionals from various fields, including software developers, machine testing engineers, research scientists, data analysts, surgeons, and lawyers. Their work commitments in different corners of the world were hampered, adding an unexpected layer to their travel tales
• Bittersweet Departures: A group of elderly couples, leaving the diverse landscapes of India, experienced a bitter taste as their departure plans were thwarted by the delay. Their varied emotions echoed the sentiment of bidding farewell to a land of rivers, mountains, deserts, plateaus, and plains.
• I may inadvertently be doing an injustice to the numerous untold narratives on the board. Each of these untold stories carries its own urgency, particularly those entangled with pressing work commitments that demand attention and resolution. It’s a reminder that behind every delayed flight, there’s a multitude of personal and professional journeys, each deserving recognition and acknowledgment.

In these shared narratives, the delay becomes more than a logistical hiccup — it transforms into a shared human experience. Each story, unique in its essence, contributes to the collective tapestry of our shared journeys, emphasizing the emotional intricacies woven into the fabric of travel.

As I navigate this unforeseen delay, I reflect on the broader implications of flight disruptions. Beyond the inconvenience, they underscore the fragility of our meticulously planned lives. The emotional toll of uncertainty becomes palpable, with each announcement echoing a ripple of reactions, from sighs of resignation to the nervous tapping of fingers on smartphones. If these passengers weren’t stuck waiting for over a day, they could have otherwise been actively contributing to the economy, either through their dedicated work or through regular purchases of goods and services, especially those who were just traveling through India. Delay hindered their economic contributions.

In conclusion, the delayed flight BA 256 serves as a poignant reminder that the emotional impact of travel extends far beyond the logistics of reaching a destination. It unearths the human stories hidden behind departure gates and arrival boards, emphasizing the need for empathy and understanding in an environment where shared journeys converge and diverge.