<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Amrin Kareem on Medium]]></title>
        <description><![CDATA[Stories by Amrin Kareem on Medium]]></description>
        <link>https://medium.com/@amrinkareem?source=rss-f16c954592b0------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Vrql1wJVF6e3I1Fb6ZcGyQ.jpeg</url>
            <title>Stories by Amrin Kareem on Medium</title>
            <link>https://medium.com/@amrinkareem?source=rss-f16c954592b0------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 00:51:41 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@amrinkareem/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[MMDuet2: Enhancing Proactive Interaction of Video MLLMs with Multi-Turn Reinforcement Learning]]></title>
            <link>https://medium.com/the-curious-algorithm/mmduet2-enhancing-proactive-interaction-of-video-mllms-with-multi-turn-reinforcement-learning-0da2e0d07af5?source=rss-f16c954592b0------2</link>
            <guid isPermaLink="false">https://medium.com/p/0da2e0d07af5</guid>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[video-understanding]]></category>
            <category><![CDATA[phd]]></category>
            <category><![CDATA[research]]></category>
            <dc:creator><![CDATA[Amrin Kareem]]></dc:creator>
            <pubDate>Fri, 14 Aug 2026 12:24:47 GMT</pubDate>
            <atom:updated>2026-08-14T12:24:47.073Z</atom:updated>
            <content:encoded><![CDATA[<h4>Paper Review 2 on streaming video understanding</h4><p>The paper of the week is MMDuet2 published in ICLR 2026.</p><h4>What is the task at hand?</h4><p>Proactive interaction in streaming videos. (Deciding when to respond, and silent otherwise)</p><h4>Why did prevailing methods fail?</h4><p>Previous attempts at proactive interaction such as <em>VideoLLM-Online</em>, <em>Dispider</em>, and <em>VideoLLM Knows When to Speak</em>, a video MLLM decides whether it should respond after a certain video frame by one of 3 ways: it can have an additional module such as a classifier that predicts a score, or it can look at the probability of a special token like &lt;silent&gt; or &lt;response&gt;, or the visual token drop ratio. These scores then get compared to a pre-defined threshold — based on which the model takes the yes/no decision. However, the authors find two issues with these approaches:</p><p>(1) If the threshold is not set to the right value, the model may not respond at all, or give many repetitive responses.</p><p>(2) Methods that use supervised fine-tuning to train proactive interaction models require training data with exact timestamps for each model response — this is difficult to collect. Even when such interleaved video-text data is created, slight differences between the model response timing and the exact point of an event occurence in a frame can affect the model’s timeliness.</p><h4>Unique Selling Point</h4><p>To sum up the difficulties in prior methods:</p><ul><li>manually tuning response decision thresholds</li><li>annotating precise reply times</li></ul><p>The paper introduces a<em> multi-turn RL-based training method</em>.</p><p>This method urges the model to generate correct responses as early as possible and penalizes it for producing incorrect or excessively delayed responses. This way they enhance the model’s response timing without annotating the timestamp of each model reply in the training data.</p><h4>Examples of model usage scenarios</h4><p>Live-stream analysis, intelligent surveillance, egocentric assistance agents, and socially interactive AI agents.</p><h4>So, how does the model work?</h4><p>Given a video (or webcam input), the model starts speaking about the video content every few seconds — if asked a question specifically at any point of the stream, e.g, <em>“what are the features of this &lt;object on screen&gt;?”</em>, the model can choose to generate some text content as a reply in real-time, or generate “NO REPLY” to indicate that it does not want to reply in this round. The response is neither too sparse nor too dense and repetitive.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3UGgGJiUx2KB3fqsc4BGRg.png" /><figcaption>Demo video (source: GitHub)</figcaption></figure><h4>How did they train it?</h4><p><strong>Training:</strong> SFT + RL (GRPO)</p><p>RL reward design:</p><p>(1) <strong>Modified PAUC (Proactive Area Under Curve) reward</strong></p><p>(2) <strong>Replication reward:</strong> To prevent the model from generating replicated replies</p><p>(3) <strong>In-span reward:</strong> To prevent the model from generating replies during video spans unrelated to the question</p><p>(4) <strong>Prefix reward: </strong>To prevent the model from repeating the previous replies before adding new content when generating new replies, making the replies more verbose than necessary</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/782/1*DNlmcK-h4OBa_bWlz6ziWQ.png" /><figcaption>total reward is the weighted sum of each reward</figcaption></figure><p>The chat template used in MMDuet2 formats the entire interaction process.</p><p>The system prompt is customized for proactive interaction. The user asks a question and provides video input as frames. The model can answer if the frames offer sufficient evidence to respond, or just say <em>“NO REPLY”</em>. The video frames keep coming in the user’s turns and the model once again can choose to reply or refrain in the assistant’s turns. This keeps happening until all the frames in the video have been input to the model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Fao3AmVGI5xMRfQj4DWHRg.png" /><figcaption>Chat template of MMDuet2</figcaption></figure><h4>Dataset</h4><p>Dataset of web videos and ego-centric videos from: <a href="https://huggingface.co/datasets/chenjoya/Live-WhisperX-526K/">Live-WhisperX</a>, <a href="https://github.com/OpenGVLab/EgoExoLearn">EgoExoLearn</a>, <a href="https://ego-exo4d-data.org/">Ego-Exo4D</a>, <a href="https://huggingface.co/datasets/lmms-lab/LLaVA-Video-178K">LLaVA-Video</a>, and <a href="https://huggingface.co/datasets/omni-research/Tarsier2-Recap-585K">Tarsier2</a></p><p>SFT data:</p><ul><li>52k high-quality and diverse proactive dialogues from Live-WhisperX, Ego-Exo4D, EgoExoLearn</li><li>25k offline video QA data from LLaVA-Video and 25k video captioning data from tarsier2 formatted using Qwen2.5-VL’s default chat template and system prompt.</li></ul><p>Hold out data for RL:</p><ul><li>1500 web and 400 ego-centric</li></ul><h4>What Model and Hardware did they Use?</h4><p>Qwen2.5-VL 3B</p><p>SFT process is conducted on 16 H800 GPUs and takes about 8 hours.</p><h4>Evaluation Benchmarks</h4><p>ProactiveVideoQA</p><p>StreamingBench</p><h4>Summing up: What did this paper introduce?</h4><p>The contributions of this work include:</p><p>(1) An RL-based training method that can significantly improve proactive interaction experience without requiring precise reply timestamp in training data,</p><p>(2) A pipeline for constructing proactive dialogue from videos, and a dataset consisting of 52k high-quality and diverse proactive dialogues, and</p><p>(3) MMDuet2, a video MLLM that has state-of-the-art performance on proactive video QA benchmarks and provides a better interaction experience.</p><p>Here is the paper, go check it out!</p><p><a href="https://arxiv.org/abs/2512.06810">MMDuet2: Enhancing Proactive Interaction of Video MLLMs with Multi-Turn Reinforcement Learning</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0da2e0d07af5" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-curious-algorithm/mmduet2-enhancing-proactive-interaction-of-video-mllms-with-multi-turn-reinforcement-learning-0da2e0d07af5">MMDuet2: Enhancing Proactive Interaction of Video MLLMs with Multi-Turn Reinforcement Learning</a> was originally published in <a href="https://medium.com/the-curious-algorithm">The Curious Algorithm</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Proactive vs. Reactive Streaming Video Understanding]]></title>
            <link>https://medium.com/the-curious-algorithm/proactive-vs-reactive-streaming-video-understanding-a34de19fb842?source=rss-f16c954592b0------2</link>
            <guid isPermaLink="false">https://medium.com/p/a34de19fb842</guid>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[research]]></category>
            <category><![CDATA[video-understanding]]></category>
            <category><![CDATA[phd]]></category>
            <dc:creator><![CDATA[Amrin Kareem]]></dc:creator>
            <pubDate>Fri, 14 Aug 2026 11:49:46 GMT</pubDate>
            <atom:updated>2026-08-14T11:49:46.547Z</atom:updated>
            <content:encoded><![CDATA[<h4>Understand the tasks, techniques and works of each paradigm</h4><p>In Part 1, we looked at streaming video understanding and tried to understand the paradigms that govern how it is implemented in the literature. The most important aspect of streaming video understanding is the processing of video frames in a strictly causal manner. The proactive paradigm features autonomous interaction whereas reactive paradigm is specifically triggered by user’s queries.</p><p>Today, we’ll look at the types of tasks in each type of response paradigm.</p><h3>Types of Proactive vs Reactive Streaming Video Understanding Tasks</h3><p>Proactive tasks monitor the video stream and decide <em>when </em>and <em>what </em>to output. Some of these tasks include:</p><ul><li><strong>Streaming real-time narration:</strong> Model performs <em>live commentary</em> and generates continuous descriptions of events in a video stream.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/708/1*f_EnsqSO8yYMQU028Btv4w.png" /><figcaption>From the paper: Towards Online Interactors: A Comprehensive Survey on Streaming Video Understanding</figcaption></figure><ul><li><strong>Continuous state monitoring:</strong> Model keeps updating response based on an initial query — such as “count the number of clothes I folded”.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/708/1*VPAQdyQXDMrTKOcSPa6qyg.png" /></figure><ul><li><strong>Event conditioned responding:</strong> Model responds when some criteria/occurence of event/trigger is met, such as “Alert me if a fire occurs.”</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/709/1*wMuH_XDgE6-3fXR0Iv6HfA.png" /></figure><ul><li><strong>Proactive multi-turn interaction:</strong> The model must also engage in turn-based conversation juggling interactions by proactively initiating turns.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/711/1*qU4l_l9VvCQilCehamWMkA.png" /></figure><p>Reactive streaming tasks are only triggered by a user query. The job of the model is to analyze a potentially long history of video content and provide an accurate answer based on the analysis. Some of these tasks are:</p><ul><li><strong>Real-time visual perception: </strong>The model must be able to answer questions related to what is happening at the current timestamp.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/704/1*Rgisa3eFh4_mbNrfOLRc6w.png" /></figure><ul><li><strong>History backward tracing:</strong> The model should be able to <em>recall </em>specific details or instances <em>from the past</em>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/709/1*QgKS3H2KshCQINWVeU_tpg.png" /></figure><ul><li><strong>Future action prediction: </strong>The model must <em>anticipate the next steps</em> or future based on the present content and reasoning.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/710/1*SbCKmupms20VgB08_rLQxw.png" /></figure><ul><li><strong>Contextual multi-turn QA: </strong>Model must mainatin coherent conversations over multiple turns.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/706/1*BSlV_sq_DDMJKsZ2Ipjs4g.png" /></figure><h3>Proactive Streaming Models</h3><p>There are many approaches to the autonomous response triggering capability of proactive streaming models, the key difference between them being <em>how they decide when to trigger a response</em>. Here are 4 technical classes of approaches the paper talks about:</p><h4>Token-Driven Triggering via EOS/Action Token</h4><p>As the name suggests, this class of models learn an additional token which are generated at inference time to trigger a response. At each time step, the model generates output tokens — for example, let’s look at the binary case: an EOS (End of Sequence) token means no response will be generated, and any other token initiates a response.</p><p>Thus, the proactive response generation is treated as a token prediction problem. In other works, there are more action tokens such as &lt;silent&gt;, &lt;response&gt;, &lt;standby&gt;, &lt;next&gt;, &lt;feedback&gt;, &lt;ask-high-res&gt;, each differing in the semantic role of the actions intended. For example, &lt;silent&gt; is output when the model chooses not to output a response, and &lt;response&gt; when the model has accumulated enough visual evidence to generate a response; &lt;next&gt; seeks more observations and similarly, &lt;ask-high-res&gt; requests a high resolution input when the input is insufficient.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/634/1*2amFo-0RCf9fbxI0TfD0Hg.png" /></figure><p><strong>Challenges:</strong></p><p>Token-driven models are sensitive to supervision — the sparsity of response events and the cost of high-accuracy frame-by-frame annotations have necessitated mitigations at the data source, training distribution, and supervisory signal.</p><h4>Dedicated Classification Heads and Detectors</h4><p>Here, the video frames or their textual timestamped summaries are given to a lightweight external module (classification heads, structured detectors or agents) that output the decision about <em>when to respond. </em>Generally, works differ on the following:</p><ul><li>Is the trigger signal a probability (graded signal) or binary yes/no decision?</li><li>Where is the module placed — a head on top of the existing Video LLM or is it a standalone external model?</li><li>How is this module trained?</li></ul><p><strong>Challenges:</strong></p><p>Separating language generation from perception and decision-making might cause problems in coordination since these modules are trained separately.</p><p>Lack of annotated, timestamped data for “speak” and “don’t speak” to supervise these modules might be difficult to obtain.</p><h4>Uncertainty and Perplexity Validation</h4><p>The model computes perplexity or a measure of uncertainty of the last generated response conditioned on new frames. If the PPL score is low, it means that the response is consistent with the new frames and thus there is no update required to it. However, if the PPL score is high, there is higher uncertainty — there is new information in the new frames that warrants an update to the response. Thus, the full decoding of a new response is triggered only when the PPL score exceeds a threshold.</p><p><strong>Challenges:</strong></p><p>Careful selection of the threshold would be required since the accuracy of this method depends on how aligned the PPL or other metric is with a key semantic change in the video.</p><p>This method is especially suited for tasks such as stability monitoring for videos where the content varies slowly.</p><h4>Visual Change and Event-Based Trigger</h4><p>In this approach, the works point to the belief that any significant visual changes such as the appearance or disappearance of an object, transitions, etc signal a new event. Thus, sharp visual changes can be treated as response triggers.</p><p>How to quantify this visual change? It could be:</p><ul><li>Pixel-level frame differences</li><li>Motion-level optical flow changes</li><li>Semantic-level event detection</li></ul><p><strong>Challenges:</strong></p><p>Lightweight, training-free heuristics decide when and whether to respond — so the heavy LLM does not need to be invoked on every frame.</p><p>But the visual change cannot be reliably associated with high-level semantic changes, so defining a threshold that does not trigger false alarms and also doesn’t miss relevant events becomes tricky.</p><h3>Reactive Streaming Models</h3><p>As discussed in part 1, reactive streaming models accept continuously arriving frames and generates an accurate answer when the user asks a question. The paradigm faces an important constraint: video streams are of an unbounded nature whereas compute and memory resources and strictly bounded.</p><p>The works in the literature work around these constraints by implementing improvements in the stages of information flow within the streaming system:</p><h4>Information Input Interception</h4><p>This is the first stage of reducing the compute burden on the system. The data stream has potentially many reducndant visual tokens. By identifying and intercepting these tokens, the data stream can be sparsified and a high-throughput low-latency system can be maintained — this can be done in 2 ways:</p><ul><li>Modify attention mechanisms (self-attention cost scales quadratically with sequence length). One idea was to introduce a cross-attention mechanism where only text tokens are used as queries to retrieve visual tokens so that the number of pairwise interactions is reduced.</li><li>Prune tokens to eliminate spatial and temporal redundancy — one strategy is to reuse static frames and only recompute attention for dynamic regions during the visual encoding stage. Further, the tokens can be selected based on their relevance to the current frame and previous context.</li></ul><p>This method reduces latency very effectively, but aggressive pruning can often lose important visual cues that might become relevant in a later query or conversation with the model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/586/1*RtrEmVlhnQvKLqNgQwcP6g.png" /></figure><h4>Working Memory Maintenance</h4><p>The input stage and visual decoding is completed; now the visual data enters the transfomer, where it is stored in the KV cache. For endless video stream, storing the KV states will eventually exhause the memory. There needs to be a strategy to effectively keep, merge, or remove cache tokens.</p><p>One way to do this is by inferring token importance from attention scores, spatial redundancy etc and evict the less significant ones. For example, measure the similarity of a token to its predecessor — if it exceeds a threshold, it can be treated as a redundant token and removed from the cache. (Explored in InfiniPot-V).</p><p>Another consideration is that there is a significant difference between the offline training and online inference in terms of how the working memory is used. StreamingVLM and TaYs explore this area further.</p><p>Overall, KV cache eviction prevvents OOM errors, but discarding unimportant tokens disregards what the user might want from the model at a later stage. Context-aware eviction is a better choice for safe compression of information.</p><h4>Long-Term Memory Compression</h4><p>Short term working memory is organized into the KV cache, what about longer sequences? This historical data needs to be compressed and stored in a long-term memory — often into high-level semantic structures such as knowledge graphs or event hierarchies. How this works is as follows: a short term buffer for immediate processing and a long term store for global understanding. Several papers have explored this area — StreamChat works by splitting the storage into short-term visual features and compressed long-term semantic summaries while also maintaining a dialogue memory.</p><p>Another strategy is to bstract context/historical data into text summaries or implicit model parameters. An example is ProVideLLM which compresses procedural observations of length running into minutes, into brief text summaries interleaved with short-term high-resolution visual tokens. The tradeoff is in visual grounding. A model that has a scene abstracted into text or parametric updates cannot answer questions relating to fine-grained spatial details.</p><h4>On Demand History Recall</h4><p>Finally, the user submits a query. Now the model must make use of Retrieval Augmented Generation to retrieve context from the compressed historical data to guide the LLM to generate an answer. The first step here is to make sure that the continuous video stream is made organized and indexable, by first segmenting the incoming frames into segments, computing their Key-Value pairs and storing them in memory— thus the video stream become a knowledge base from which information can be retrieved on demand using some similarity measure.</p><p>The challenge here is to balance the continuous spatiotemporal causality and semantic similarity between chunks of frames. For example, Event A and Event B might appear similar, but the key question could be whether Event A caused Event B.</p><p>Thus, we have come to the end of the discussion about the key consideration in designing proactive and reactive streaming video models. In the next part we will look at benchmarks and datasets, challenges in this area of work, and questions that remain yet to be answered.</p><p>Adios!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a34de19fb842" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-curious-algorithm/proactive-vs-reactive-streaming-video-understanding-a34de19fb842">Proactive vs. Reactive Streaming Video Understanding</a> was originally published in <a href="https://medium.com/the-curious-algorithm">The Curious Algorithm</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Thinking in Streaming Video: ECCV 2026]]></title>
            <link>https://medium.com/the-curious-algorithm/thinking-in-streaming-video-eccv-2026-118874d9b413?source=rss-f16c954592b0------2</link>
            <guid isPermaLink="false">https://medium.com/p/118874d9b413</guid>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[video-understanding]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[research-paper]]></category>
            <dc:creator><![CDATA[Amrin Kareem]]></dc:creator>
            <pubDate>Fri, 07 Aug 2026 14:28:42 GMT</pubDate>
            <atom:updated>2026-08-07T14:28:42.300Z</atom:updated>
            <content:encoded><![CDATA[<h4>Paper Review 1 in streaming video understanding</h4><p>The paper of the week in Thinking in Streaming Video published in ECCV 2026.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/838/1*jqSY97yYglcc1EpZuvzMhw.png" /></figure><h4>What is the task at hand?</h4><p>Real-time understanding of continuous video streams.</p><h4>Why did prevailing methods fail?</h4><p>Existing video reasoning models first consume a long video context and only then performs multi-step reasoning to generate an answer. This creates a distance between the thinking steps of the model and the video frames that triggered the thought and eventually led to a response. This latency is unacceptable for anything real-time.</p><p>3 challenges:</p><ul><li>strict causality — the model must reason on only what it has seen up until the current moment</li><li>stable inference cost (computation and memory use) even for long unbounded streams</li><li>timely intervention/interaction — the model must be able to decide when to interact with the user and respond accordingly</li></ul><h4>Examples of model usage scenarios</h4><ul><li>Real-time assistants observing user activities,</li><li>Monitoring systems detecting emerging events,</li><li>Embodied agents interacting with dynamic physical environments.</li></ul><h4>So, how does the model work?</h4><p>It works on a watch-think-speak paradigm. At every step the model watches a new video chunk and thinks/reasons about it. This reasoning is accumulated over every video chunks from start to end. If the accumulated at any point of the video reasoning history contains suitable evidence to trigger a response, such as a fire alert, or an anomalous incident, or a scene change, or an update such as the number of lunges a person has been doing, then it should respond. Otherwise the model stays silent.</p><p>To do this, it uses 3 special tokens: &lt;think&gt;, &lt;silent&gt; and &lt;response&gt;.</p><p>Given a video stream V (sequence of video chunks) and an instruction I (<em>e.g., alert me if you see a fire break out</em>) , the model maintains a history of context H, and produces a structured output consisting of:</p><ul><li>a reasoning segment enclosed in a <strong>&lt;think&gt;</strong> block and either of &lt;silent&gt; or &lt;response&gt;.</li><li>Emitting ⟨silent⟩ indicates that the model continues observing the stream without responding, while emitting</li><li>⟨response⟩ signals that the model has accumulated sufficient evidence to produce an answer.</li></ul><h4>How did they train it?</h4><p>SFT + RL</p><p>SFT to observe the current video chunk along with the accumulated video history/context and generate an output reasoning (&lt;think&gt;) and an action (&lt;silent&gt; or &lt;response&gt;).</p><p>RL (GRPO) is used to create many potential outputs (reasoning-action pairs) for each streaming instance and optimize using the standard clipped objective. They have an accuracy reward, format reward, and time reward.</p><h4>Dataset</h4><p>They also constructed a large-scale dataset featuring time-grounded CoT reasoning traces and verifiable question-answering pairs for cold-start supervision and a carefully formatted subset for RLVR training. Here’s an example:</p><pre>{&quot;video_path&quot;: &quot;./datasets/tarsier2_unzip/Kinetics-700/videos/lAyUBTnETTU_000053_000063.mp4&quot;, &quot;interaction_mode&quot;: &quot;Continuous Output&quot;, &quot;content_dimension&quot;: &quot;Spatial &amp; Geometric Relationships&quot;, &quot;temporal_scope&quot;: &quot;Current&quot;, &quot;response_format&quot;: &quot;Open-ended&quot;, <br>&quot;conversations&quot;: [{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Describe what happens as the man climbs the frame and how the surroundings change.&quot;, &quot;timestamp&quot;: 1.0}, <br>{&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;The man in the maroon&quot;, &quot;timestamp&quot;: 2.0, }, <br>{&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;shirt starts climbing the red&quot;, &quot;timestamp&quot;: 3.0, }, <br>{&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;frame, gripping the pole tightly.&quot;, &quot;timestamp&quot;: 4.0, }, <br>{&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;The Marine beside him&quot;, &quot;timestamp&quot;: 5.0, }, <br>{&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;gestures while holding the&quot;, &quot;timestamp&quot;: 6.0, }, <br>{&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;frame, watching closely.&quot;, &quot;timestamp&quot;: 7.0, }], <br>&quot;thoughts&quot;: [{&quot;think&quot;: &quot;User asks about the man climbing and surroundings. I need to&quot;, &quot;timestamp&quot;: 1.0}, <br>{&quot;think&quot;: &quot;describe his actions and environment based on current visual input.&quot;, &quot;timestamp&quot;: 2.0,}, <br>{&quot;think&quot;: &quot;The man in maroon climbs the red frame, gripping the pole. Sky&quot;, &quot;timestamp&quot;: 3.0, }, <br>{&quot;think&quot;: &quot;is partly cloudy. Low camera angle adds drama to his movement.&quot;, &quot;timestamp&quot;: 4.0, }, <br>{&quot;think&quot;: &quot;Marine in camouflage stabilizes the frame, gestures guidance. Red tent with&quot;, &quot;timestamp&quot;: 5.0, }, <br>{&quot;think&quot;: &quot;&#39;MARINES&#39; visible. Urban plaza setting with building and shrubbery in background.&quot;, &quot;timestamp&quot;: 6.0, }, <br>{&quot;think&quot;: &quot;Man reaches top of frame, pauses, then starts descending using horizontal&quot;, &quot;timestamp&quot;: 7.0}, <br>{&quot;think&quot;: &quot;bars. Camera reveals more personnel near tent and equipment box.&quot;, &quot;timestamp&quot;: 8.0, }, <br>{&quot;think&quot;: &quot;Activity continues under military supervision. Structured setup suggests fitness demo or recruitment. Atmosphere remains focused and supportive.&quot;, &quot;timestamp&quot;: 9.0, }]}</pre><h4>What Model and Hardware did they Use?</h4><p>Qwen-2.5-VL-3B</p><p>Trained 8 NVIDIA H20 GPUs.</p><h4>Summing up: What did this paper introduce?</h4><p>The authors introduce ThinkStream — a framework that enables reasoning over long video streams without using excessive compute.</p><p>To prevent the KV cache from grow rapidly due to visual tokens, they also propose Reasoning-Compressed Streaming Memory (RCSM). It is a technique that evicts outdated visual tokens but preserve the information as a reasoning trace which is much more compact.</p><p>To train such a model, they also introduced a Streaming Reinforcement Learning with Verifiable Rewards (RLVR) framework tailored to the Watch–Think–Speak paradigm that they introduced.</p><p>They also created a custom backend for efficient streaming inference utilizing Eager prefill and replayable CUDA Graphs.</p><p>Paper credits:</p><pre>@misc{liu2026thinkingstreamingvideo,<br>      title={Thinking in Streaming Video}, <br>      author={Zikang Liu and Longteng Guo and Handong Li and Ru Zhen and Xingjian He and Ruyi Ji and Xiaoming Ren and Yanhao Zhang and Haonan Lu and Jing Liu},<br>      year={2026},<br>      eprint={2603.12938},<br>      archivePrefix={arXiv},<br>      primaryClass={cs.CV},<br>      url={https://arxiv.org/abs/2603.12938}, <br>}</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=118874d9b413" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-curious-algorithm/thinking-in-streaming-video-eccv-2026-118874d9b413">Thinking in Streaming Video: ECCV 2026</a> was originally published in <a href="https://medium.com/the-curious-algorithm">The Curious Algorithm</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is Streaming Video Understanding?]]></title>
            <link>https://medium.com/the-curious-algorithm/what-is-streaming-video-understanding-92e577b594cf?source=rss-f16c954592b0------2</link>
            <guid isPermaLink="false">https://medium.com/p/92e577b594cf</guid>
            <category><![CDATA[video-understanding]]></category>
            <category><![CDATA[vlm]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[multi-modal-llm]]></category>
            <category><![CDATA[streaming-video]]></category>
            <dc:creator><![CDATA[Amrin Kareem]]></dc:creator>
            <pubDate>Fri, 07 Aug 2026 09:28:03 GMT</pubDate>
            <atom:updated>2026-08-07T09:31:03.812Z</atom:updated>
            <content:encoded><![CDATA[<h4>Part one of a break-down of recent advances towards online interactors, based on the paper ‘<em>A Comprehensive Survey on Streaming Video Understanding’</em>.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*h8p_4lKxJmrAm2Kp" /><figcaption>Photo by <a href="https://unsplash.com/@jakobowens1?utm_source=medium&amp;utm_medium=referral">Jakob Owens</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Video is one of the most popular modes of dispersing information, largely owing to video sharing platforms and the appeal of the dynamic and engaging format of the medium. Other reasons would include ubiquitous sensing devices, surveillance, and autonomous systems which operate mostly on video data. This abundance of video data is partly responsible for the recent advancements in video understanding — notably, Multimodal Large Language Models (MLLMs) — both closed-source such as GPT-5 and open-weight such as Qwen3-VL. These models have been largely successful in fine-grained spatio-temporal reasoning and long-context understanding.</p><p>However, mainstream Video Large Language Models (Video-LLMs) work in an offline manner — they respond to queries based on complete, pre-recorded videos which have been fully buffered already. In reality, many videos are recorded live and might need low latency responses that do not depend on having all frames ready beforehand. Moreover, offline video understanding also slows with increasing video duration. For applications within the surveillance, autonomous driving, and embodied intelligence, this calls for a paradigm shift to streaming video understanding — where a model processes video frames as they arrive continuously and responds under real-time constraints. This change, however, comes with its own challenges.</p><p>(1) In offline video understanding, a single static answer is just fine. Streaming video understanding requires responses that can evolve with the accumulating visual evidence. The prediction depends on both ‘<em>what the model sees’ </em>and <em>‘when it sees’ it</em>. For instance, a query,<em> </em><strong><em>“Did the chef use the sour cream?”</em> </strong>will have <em>‘not yet’ </em>as an answer until it changes to <em>‘yes, he just poured the sour cream into the bowl’</em> based on the accruing video content.</p><p>(2) For unbounded streaming videos, another challenge is memory and compute. Summarization of a long video history and reasoning over long video streams are expensive in memory, as is caching all visual tokens in the key-value (KV) cache. With increasing stream length, the cost of computing the attention also grows quadratically.</p><p>(3) Finally, the real-time interaction proves to be another significant challenge. The video understanding system might often need to support interactive behaviour, such as handling interruptions and engaging in conversations, while maintaining an inference throughput at or above the input frame rate.</p><p>What distinguishes streaming from offline video understanding are these challenges applied in unison —</p><ul><li>the model must incrementally update its representation of the scene as it evolves, and</li><li>produce temporally consistent interpretations or responses</li><li>all while working within strict latency and memory budgets.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/728/1*Pvw6K1glW0cEHsIcaWofwg.png" /><figcaption>Source: <a href="https://sotayang.github.io/Streaming_Video_Understanding_Survey.pdf">Paper</a></figcaption></figure><h4>Mathematical Intuition</h4><p>In streaming video understanding, a model processes video frames in a <strong>strictly causal manner:</strong> at any time step, it only sees past and present video frames, and cannot see future frames. This is in sharp contrast to conventional offline video understanding, where the entire video is available prior to inference.</p><p>Formally, a streaming video <em>V </em>is represented as a (potentially unbounded) sequence of frames <em>f</em>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/772/1*e1r0bXr5fmJXSq9_j1DR_Q.png" /></figure><p>The two complementary interaction paradigms examined in the paper are Proactive Streaming Models and Reactive Streaming Models.</p><h4>Proactive Streaming Models</h4><p>Proactive streaming models do not wait for an input query to provide a response: they actively decide when and whether to initiate interactions. For instance, the issuing early hazard warnings like an impending collision, real-time narration of events like football commentary, or requesting clarification/delegation when a question is ambiguous. The emphasis is on decision-making under uncertainty. Models approach this task by designing reliable and generalizable triggering mechanisms that identify the right moment for interaction — not reacting late enough to miss an event, and also not reacting prematurely so as to be inaccurate.</p><p>The proactive paradigm follows a formulation as follows: the model processes continuous visual streams <em>f </em>from <em>1…t </em>while also learning to decide when to generate responses. At each time step t, a response triggering variable a is introduced:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*6QeeQw5g0bvOrVLerew1OA.png" /><figcaption>Response triggering variable a can take the values 0 or 1 at any time step t</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/774/1*U6pLCmzxvBencAnLAwVoTw.png" /></figure><h4>Reactive Streaming Models</h4><p>In reactive streaming models response generation is explicitly triggered by external user queries. They follow a query-driven paradigm: they continuously encode and store the incoming video stream into a persistent memory — and remain idle until the user asks an explicit query (e.g., “What happened just now?” or “Do you spot an anomaly?”). Only upon receiving the query does the model retrieve and aggregate relevant information from the temporal context (could be long or short depending on the video length) to produce responses.</p><p>The emphasis here is on scalability and efficiency: maintaining accurate representations of long context histories while staying within memory and latency constraints.</p><p>The reactive paradigm can be seen as a special case of the proactive streaming case; At time t, given a query Qt, the model produces a response conditioned on both the query and the observed video history:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/804/1*NdAYaHQXDkDaNzI2BZ1nCA.png" /></figure><p>In this case, the response triggering mechanism is not learned by the model. Rather, it is determined by the user’s input to the model. There is no longer autonomous decision-making, only query-driven reasoning.</p><p>So far, we explored:</p><ul><li>Streaming video understanding</li><li>Offline Video Understanding</li><li>Proactive Streaming Models</li><li>Reactive Streaming Models</li></ul><p>That’s all for now. In the next part of this series, we examine the different video understanding tasks under each paradigm and some notable models in detail.</p><p>This article is summarised from the paper:</p><pre>@article{202606.1674,<br> doi = {10.20944/preprints202606.1674.v1},<br> url = {https://doi.org/10.20944/preprints202606.1674.v1},<br> year = 2026,<br> month = {June},<br> publisher = {Preprints},<br> author = {Zhenyu Yang and Kairui Zhang and Qi Liu and Tiancheng Liu and Long Ying and Dizhan Xue and Qibin Hou and Shengsheng Qian and Changsheng Xu},<br> title = {Towards Online Interactors: A Comprehensive Survey on Streaming Video Understanding},<br> journal = {Preprints}<br>}</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=92e577b594cf" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-curious-algorithm/what-is-streaming-video-understanding-92e577b594cf">What is Streaming Video Understanding?</a> was originally published in <a href="https://medium.com/the-curious-algorithm">The Curious Algorithm</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Zero-Shot Video Object Grounding with Qwen-3.5-VL and vLLM: A Step-by-Step Implementation Guide]]></title>
            <link>https://medium.com/the-curious-algorithm/zero-shot-video-object-grounding-with-qwen-3-5-vl-and-vllm-a-step-by-step-implementation-guide-b16334a3a0c3?source=rss-f16c954592b0------2</link>
            <guid isPermaLink="false">https://medium.com/p/b16334a3a0c3</guid>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[tutorial]]></category>
            <category><![CDATA[video-analysis]]></category>
            <category><![CDATA[visual-grounding]]></category>
            <category><![CDATA[vllm]]></category>
            <dc:creator><![CDATA[Amrin Kareem]]></dc:creator>
            <pubDate>Wed, 05 Aug 2026 16:11:21 GMT</pubDate>
            <atom:updated>2026-08-05T16:32:57.452Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8OjAb4kKkpaxvqH7" /><figcaption>Photo by <a href="https://unsplash.com/@arianaioana?utm_source=medium&amp;utm_medium=referral">A. I.</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Visual grounding — the task of locating specific objects in an image or video frame using natural language queries — is transforming how we index, search, and analyze video content. Instead of training custom object detectors for every new class, modern Vision-Language Models (VLMs) like Qwen-3.5-VL allow us to detect any object on the fly using zero-shot prompting.</p><p>In this post, we’ll build a high-performance video object-grounding pipeline. We will explore how to cache video frames efficiently, run distributed VLM inference using <strong>vLLM</strong>, and draw annotations using <strong>Roboflow’s Supervision</strong> library.</p><h4>The Video Grounding Pipeline Architecture</h4><p>Processing video is computationally heavy. A naive pipeline that re-opens video files for every frame and runs slow single-frame model inferences will quickly hit a performance bottleneck.</p><p>Our optimized pipeline consists of three major steps:<br>1. <strong>Smart Frame Caching</strong>: Efficiently seeks and reads video frames at specific timestamps using OpenCV.<br>2. <strong>Fast VLM Inference:</strong> Utilizes vLLM to run parallel, batch-based model generation.<br>3.<strong> Professional Annotation:</strong> Renders high-quality bounding boxes, text labels, and translucent masks.</p><h4>1. Efficient Frame Extraction</h4><p>When running analysis over a long video, frequently seeking and loading frames can be a major bottleneck. Re-opening a video file for every single timestamp query is highly inefficient.</p><p>To solve this, we implement a class that holds open <em>cv2.VideoCapture</em> handles and caches frames using PyPillow and OpenCV.</p><pre>import os<br>import cv2<br>from PIL import Image<br><br>class FrameCache:<br><br>  def __init__(self):<br>    self._caps = {}<br>    self._frames = {}<br><br>  def get(self, video_path: str, timestamp_sec: float):<br> <br>    key = (video_path, round(timestamp_sec, 3))<br>    if key in self._frames:<br>     return self._frames[key]<br>    cap = self._caps.get(video_path)<br>    if cap is None:<br>      if not os.path.exists(video_path):<br>        self._frames[key] = None<br>        return None<br>      cap = cv2.VideoCapture(video_path)<br>      self._caps[video_path] = cap<br>    if not cap.isOpened():<br>      self._frames[key] = None<br>      return None<br>   <br>   fps = cap.get(cv2.CAP_PROP_FPS) or 25.0<br>   frame_count = cap.get(cv2.CAP_PROP_FRAME_COUNT) or 0<br>   frame_idx = int(round(timestamp_sec * fps))<br>   if frame_count:<br>     frame_idx = min(frame_idx, max(int(frame_count) - 1, 0))<br>   cap.set(cv2.CAP_PROP_POS_FRAMES, frame_idx)<br>   ret, frame = cap.read()<br>   if not ret:<br>     self._frames[key] = None<br>     return None<br>   <br>   img = Image.fromarray(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))<br>   self._frames[key] = img<br>   return img<br><br>def close(self):<br> for cap in self._caps.values():<br>   cap.release()<br> self._caps.clear()<br> self._frames.clear()</pre><h4>2. Fast Grounding with Qwen-3.5 and vLLM</h4><p>For inference, we leverage the <strong>Qwen-3.5–35B-A3B</strong> multimodal model. Because of its size, running it in standard Transformers code can be slow. By using <strong>vLLM</strong>, we can execute inference with optimized memory management, dynamic batching, and tensor parallelization (e.g., using <em>tensor_parallel_size=4 </em>to split the model across 4 GPUs).</p><p>The grounding prompt instructs Qwen to locate objects from a text query and output a structured JSON array containing <em>bbox_2d</em> and <em>label</em> fields:</p><pre>from vllm import LLM, SamplingParams<br>from transformers import AutoProcessor<br>import torch<br><br>class QwenGrounder:<br> def __init__(self, model_id=&quot;Qwen/Qwen3.5–35B-A3B&quot;, device=None):<br>   self.llm = LLM(<br>   model=model_id,<br>   trust_remote_code=True,<br>   max_model_len=4096,<br>   limit_mm_per_prompt={&quot;image&quot;: 1}, <br>   tensor_parallel_size=4 # Run across 4 GPUs<br>   )<br>   self.processor = AutoProcessor.from_pretrained(model_id)<br>   img_proc = getattr(self.processor, &quot;image_processor&quot;, None)</pre><p>Qwen-VL models output normalized bounding boxes in a [0, 1000] coordinate format (i.e., [xmin, ymin, xmax, ymax]). However, Qwen internally resizes input images to fit within a specific grid size.</p><p>To correctly map the coordinate space back to the original image dimensions without coordinate drifting, we divide Qwen’s output by 1000 to get relative coordinates in the range [0, 1] and then scale them back up to the original image width and height.</p><p>Here is the <em>ground</em> method of our <em>QwenGrounder</em> class:</p><pre>@torch.no_grad()<br>def ground(self, image_pil: Image.Image, text: str) -&gt; dict:<br>   if image_pil is None:<br>     return {}<br> <br>   prompt = (<br>   f&quot;Locate all visible objects in this frame as mentioned in the caption: {text}. &quot;<br>   &quot;Output a JSON array where each element has \&quot;bbox_2d\&quot;: &quot;<br>   &quot;[x1, y1, x2, y2] and \&quot;label\&quot;: the matching object name from &quot;<br>   &quot;the list. Only include objects you can actually see. Output &quot;<br>   &quot;JSON only, no other text.&quot;<br>   )<br>   messages = <br>  [<br>   {<br>   &quot;role&quot;: &quot;user&quot;,<br>   &quot;content&quot;: <br>    [<br>     {&quot;type&quot;: &quot;image&quot;, &quot;image&quot;: image_pil},<br>     {&quot;type&quot;: &quot;text&quot;, &quot;text&quot;: prompt}, <br>    ],<br>      }<br>       ]<br>   <br>   formatted_prompt = self.processor.apply_chat_template(<br>   messages, tokenize=False, add_generation_prompt=True<br>   )<br>   image_inputs, video_inputs = process_vision_info(messages)<br>   <br>   mm_data = {}<br>   if image_inputs is not None:<br>     mm_data[&quot;image&quot;] = image_inputs<br> <br>   llm_inputs = {<br>   &quot;prompt&quot;: formatted_prompt,<br>   &quot;multi_modal_data&quot;: mm_data<br>   }<br> <br>   sampling_params = SamplingParams(temperature=0.8, max_tokens=1024)<br>   outputs = self.llm.generate([llm_inputs], sampling_params=sampling_params)<br>   generated_text = outputs[0].outputs[0].text<br> <br>   # Clean reasoning tokens and markdown code blocks<br>   if &quot;&lt;/think&gt;&quot; in generated_text:<br>     generated_text = generated_text.split(&quot;&lt;/think&gt;&quot;, 1)[1]<br>   cleaned = re.sub(r&quot;^```(json)?|```$&quot;, &quot;&quot;, generated_text.strip(), flags=re.MULTILINE).strip()<br>   detections = json.loads(cleaned)<br>   boxes = {}<br>   for det in detections:<br>     x1, y1, x2, y2 = det[&quot;bbox_2d&quot;]<br>     label = str(det.get(&quot;label&quot;, &quot;&quot;)).strip().lower()<br>     # Map Qwen&#39;s [0, 1000] scale to normalized float values [0.0, 1.0]<br>     boxes[label] = [x1/1000, y1/1000, x2/1000, y2/1000]<br>     <br>   return boxes</pre><h4>3. Visualizing Detections with Supervision</h4><p>Once we have normalized [0, 1] bounding boxes, we convert them back to actual pixel dimensions. Instead of drawing basic boxes using OpenCV’s cv2.rectangle, we use the supervision library to build a modern, high-quality visualization.</p><pre>import numpy as np<br>import supervision as sv<br><br>def draw_boxes_on_image(image_pil: Image.Image, boxes: dict) -&gt; np.ndarray:<br> &quot;&quot;&quot;Draws bounding boxes and labels on the image and returns an OpenCV BGR frame.&quot;&quot;&quot;<br> frame = cv2.cvtColor(np.array(image_pil), cv2.COLOR_RGB2BGR)<br> h, w, _ = frame.shape<br> <br> if not boxes:<br>   return frame<br> <br> xyxy_list = []<br> labels = []<br> <br> # Scale coordinates to fit image dimensions<br> for label, box in boxes.items():<br>   x1, y1, x2, y2 = box<br>   xmin_pix = max(0, min(int(x1 * w), w - 1))<br>   ymin_pix = max(0, min(int(y1 * h), h - 1))<br>   xmax_pix = max(0, min(int(x2 * w), w - 1))<br>   ymax_pix = max(0, min(int(y2 * h), h - 1))<br> <br>   xyxy_list.append([xmin_pix, ymin_pix, xmax_pix, ymax_pix])<br>   labels.append(label)<br> <br> xyxy = np.array(xyxy_list, dtype=np.float32)<br> <br> # Generate binary masks for the box regions<br> masks = np.zeros((len(xyxy_list), h, w), dtype=bool)<br> for i, (xmin, ymin, xmax, ymax) in enumerate(xyxy_list):<br>   masks[i, ymin:ymax, xmin:xmax] = True<br> <br> # Instantiate supervision detections<br> dets = sv.Detections(xyxy=xyxy, mask=masks)<br> <br> # Draw annotations<br> scene = frame.copy()<br> scene = sv.MaskAnnotator(color=sv.Color.YELLOW, opacity=0.18).annotate(scene=scene, detections=dets)<br> scene = sv.BoxAnnotator(color=sv.Color.YELLOW, thickness=2).annotate(scene=scene, detections=dets)<br> scene = sv.LabelAnnotator(<br> color=sv.Color.YELLOW, <br> text_color=sv.Color.BLACK, <br> text_scale=0.5, <br> text_padding=6<br> ).annotate(scene=scene, detections=dets, labels=labels)<br> <br> return scene</pre><p>By combining Qwen-3.5-VL’s robust reasoning capabilities with <strong>vLLM</strong>’s fast distributed inference, we can perform zero-shot visual grounding on video frames in real-time.</p><p>This approach yields a production-ready pipeline for next-generation video search engines and automated labeling pipelines.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b16334a3a0c3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-curious-algorithm/zero-shot-video-object-grounding-with-qwen-3-5-vl-and-vllm-a-step-by-step-implementation-guide-b16334a3a0c3">Zero-Shot Video Object Grounding with Qwen-3.5-VL and vLLM: A Step-by-Step Implementation Guide</a> was originally published in <a href="https://medium.com/the-curious-algorithm">The Curious Algorithm</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[George, My Childhood Obsession]]></title>
            <link>https://medium.com/the-window-seat/george-my-childhood-obsession-c67f0ce282ff?source=rss-f16c954592b0------2</link>
            <guid isPermaLink="false">https://medium.com/p/c67f0ce282ff</guid>
            <category><![CDATA[enid-blyton]]></category>
            <category><![CDATA[books]]></category>
            <category><![CDATA[fiction]]></category>
            <category><![CDATA[childrens-books]]></category>
            <category><![CDATA[writing]]></category>
            <dc:creator><![CDATA[Amrin Kareem]]></dc:creator>
            <pubDate>Thu, 29 Jan 2026 08:47:48 GMT</pubDate>
            <atom:updated>2026-01-29T08:47:48.863Z</atom:updated>
            <content:encoded><![CDATA[<h4>Bold, brave, and brilliant — What made George my favourite Enid Blyton character?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/468/1*tFRCH-zbv_ZE71QPeoeTkQ.jpeg" /><figcaption>Typical unapologetic George.</figcaption></figure><p>An athletic frame dressed in a long-sleeved T-shirt and a pair of baggy jeans, cropped curly hair tousled by the wind, a sun-kissed face, scented by sea spray, a fiery temper, fearless language and a love for adventure — George was my first obsession at the age of nine.</p><p>When I moved on from short stories and started reading books and other longer literature, Dad introduced me to Enid Blyton and her daring characters who magically sprang out of the tip of her pen onto pages and pages of thrilling feats of bravery.</p><p>If you’re new to the Blyton universe: Enid Blyton was a highly popular English author who authored several children’s book series, including The Famous Five, The Naughtiest Girl, Malory Towers, The Secret Seven and others I never got to lay my hands on. The Famous Five was made into a British TV series in the 1970s and became hugely successful, and was reproduced again in the 1990s.</p><p>The point is, I got to know George through the books Dad bought me. She was everything I was like — and wanted to be. With my curly hair cut short, baggy jeans and an oversized T-shirt, I was the George in my circles. I loved the sudden confusion in the eyes of an elderly person when they saw me, and finally ended up calling me a boy. It was <em>victory</em>, the very label.</p><p>Once my pace picked up and I began consuming books voraciously, my Dad realized that this wasn’t all well for his finances. And he ever so gently prodded me in the direction of the library. This was back when I was eight years old, when kids weren’t allowed to borrow books home from the school library.</p><p>I had a way out, though. I put my elder sister in charge, and she became my agent at the library. Once I finished the series with her warm cooperation, I was old enough to borrow books, but my love for Enid Blyton’s books had survived and thrived. I had been exploring The Naughtiest Girl and some of the Secret Seven series in parallel, and now the school library was my haven.</p><p>But dear God, The Famous Five was something else entirely. It’s probably my bias for first love. I read and reread and repeated. I remember the smell of the paperback, with its off-white colored paper, its amazing smell, and the way it felt between my fingers — a little coarse, but just right for the touch. I spent hours looking at the illustration on the cover page and mimicking Enid Blyton’s signature at the top.</p><p>The rustle of pages and that familiar smell of paper is as intoxicating as reading the stories themselves. The Blyton universe is a wonderful place to get lost in, and a great way to discover yourself. Just like I discovered George. Although Julian was the oldest of the lot, George was at the centre of every story, Timmy at her side. It was unbearable when she was sidelined in any plot — I’d feel bad about it all day.</p><p>I find the name very personal, so much so that even today, if someone says <em>Georgina Kirrin </em>of Famous Five, I’m ready to go all out on a spat and fight it out with them. The name’s George. <em>You might want to keep it that way.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c67f0ce282ff" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-window-seat/george-my-childhood-obsession-c67f0ce282ff">George, My Childhood Obsession</a> was originally published in <a href="https://medium.com/the-window-seat">The Window Seat</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Sundown in Kovai]]></title>
            <link>https://medium.com/the-window-seat/sundown-in-kovai-c8b9c06542e3?source=rss-f16c954592b0------2</link>
            <guid isPermaLink="false">https://medium.com/p/c8b9c06542e3</guid>
            <category><![CDATA[culture]]></category>
            <category><![CDATA[self-discovery]]></category>
            <category><![CDATA[travel]]></category>
            <category><![CDATA[india]]></category>
            <category><![CDATA[landscape]]></category>
            <dc:creator><![CDATA[Amrin Kareem]]></dc:creator>
            <pubDate>Tue, 27 Jan 2026 14:25:40 GMT</pubDate>
            <atom:updated>2026-01-27T14:25:40.135Z</atom:updated>
            <content:encoded><![CDATA[<h3>Sundown in Kovai — Part 1</h3><h4>A tiny excerpt from my life in the Manchester of South India.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ucj0eQs6MEvDdEFPgdGovA.jpeg" /><figcaption>Unsplash</figcaption></figure><p>The evening breeze is cool and gentle as it strokes my hair. The bushes here grow tall, taller than me, and stretch as far as the eye can see, and seem to be swallowed by the sky as they approach the intersection of the narrow inroads. As I approach them, I realise the bushes are actually crops; the road runs through what must have been a very large field. It feels comfortingly strange to be in the middle of nowhere with nowhere to be anytime soon. The structure on my left is a government school, my friend tells me. I’m still learning how to read Tamil. Apart from the colourful childish drawings on its high compound wall and the Tamil writing, there is nothing that suggests to me the functioning of a school here.</p><p>The road is mostly empty, as it is almost all the time. The breeze is cooler now. <em>There’s no better weather in all of South India after five.</em> I remember a colleague’s comment that rings very much true. A factory makes some chugging noise at the far end of the road. The most striking part of the walk is that evenings here are not filled with the typical cawing sounds of crows flying home before sundown that I had grown up listening to. The silence is sometimes a bit too much. Overgrown crops stand prominently to the right side of the road, and from within the field, brimming with long stalks, I hear peacocks. Some days, I see them, all the bright hues in one magnificent flightless creature. A simple reminder that even when you’re great at many things, you can’t have <em>everything</em>.</p><p>As a faraway Masjid faintly announces the call to prayer, I turn around and slowly trace my steps back to the farthest building two blocks away, where I live.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c8b9c06542e3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-window-seat/sundown-in-kovai-c8b9c06542e3">Sundown in Kovai</a> was originally published in <a href="https://medium.com/the-window-seat">The Window Seat</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI, Analytics, and a New Age of Data]]></title>
            <link>https://medium.com/the-curious-algorithm/ai-analytics-and-a-new-age-of-data-81d2faa270fb?source=rss-f16c954592b0------2</link>
            <guid isPermaLink="false">https://medium.com/p/81d2faa270fb</guid>
            <category><![CDATA[big-data]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[ai-tools]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <dc:creator><![CDATA[Amrin Kareem]]></dc:creator>
            <pubDate>Sun, 04 Jan 2026 12:53:55 GMT</pubDate>
            <atom:updated>2026-01-04T12:53:55.313Z</atom:updated>
            <content:encoded><![CDATA[<h4>How does AI revolutionise data analytics?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2DvS0Qn0gxCHlzTmNagJhA.png" /><figcaption>Image from alphacoders</figcaption></figure><p>Every organization collects data — it is perhaps the most important asset of any company that wants to make data-driven decisions and draw insights from huge volumes of data collected over years or decades. This article is about data analytics and how AI is changing this field. Let’s get definitions out of the way first.</p><p>Data analytics — “the process of collecting, cleansing, transforming and modeling data using methods from statistics, machine learning, and data science to discover useful and actionable insights that support business decision-making.”</p><blockquote>Simply put, we use data analytics to understand the message hidden in our data and use that to improve our business.</blockquote><h3>4 Types of Data Analytics</h3><p><strong>Descriptive Analytics: </strong>Asks the question <em>“What happened?” </em>It tries to understand and make sense of what happened in the past. These are simple factual things, such as, <em>“What is the number of monthly users product XYZ has?”</em></p><p><strong>Diagnostic Analytics:</strong> Asks the question <em>“Why did it happen?” </em>In this kind of analytics, we’re trying to get to the cause of an event by examining the relationships between different elements. For instance, <em>“Did the holiday season campaign affect the web traffic/number of new signups?”</em></p><p><strong>Predictive Analytics:</strong> Asks the question <em>“What could happen next?” </em>It is a method that makes forecasts about the future, and try to say what’s most likely to happen. Businesses often derive a lot of value from predictive analytics, by finding answers to questions like, <em>“In the next quarter, will sales of product XYZ go up?”</em></p><p><strong>Prescriptive Analytics: </strong>Advises the next course of action. Asks the question, <em>“What should we do next?” </em>It offers recommendations for what a business should do to reach its goals in future.</p><h3>When and How did we start applying AI in Data Analytics?</h3><p>Data analytics has been around for centuries, with some early applications in the preparation of census reports. As databases became popular towards the 1970s, RDBMs and SQL were widely used to store and retrieve information accurately.</p><p>When the internet went mainstream in the 1990s, the number of sources and datatypes grew rapidly, and NoSQL was developed as a means to store unstructured data in non-relational databases. In the late 1980s, as the cost of data storage in hard disk drives fell, the amount of data being collected continued to grow even more significantly. Around this time, data warehouses were developed to transform data coming from operational systems to be used in systems that support decision-making.</p><p>Around 1989, Howard Dresner adapted the term <em>Business Intelligence </em>to describe decision-making based on data gathered by an organization. In the 1990s, bolstered by the developments in data warehouse technologies and databases, data mining emerged as a field of growing interest. It allowed patterns to be learnt quickly from large amounts of data, and witmnessed wide application in analyzing customer purchase data.</p><p>With the post-2010 resurgence in deep learning, it became possible to perform more nuanced analysis of text and visual data in addition to tabular data analysis and forecasting made possible by traditional machine learning (ML) techniques.</p><p>Today, data analytics is a quintessential part of any business governing its decision making and operations. Like we said, it involves uncovering trends and patterns in data through close and thorough examination. While <strong>traditional analytics relies on basic statistical analysis techniques like regression analysis, hypothesis testing, and descriptive statistics</strong> (mean, median, mode, range, standard deviation, etc), modern AI based analytics approaches automate and simplify every steps of the data analysis process.</p><h3>The Data Analytics Process</h3><h4>Analysis vs. Analytics</h4><p>Data analysis is just one step in the data analytics process. While data analytics encompasses the whole process from data collection, cleaning, transformation, interpretation, analysis and reporting, data analysis is the one step where we draw insights from the analysis-ready, organized data.</p><h4>1. Define the Problem</h4><p>In this step, a data analyst clearly defines what questions they’re trying to find answers for. Once the problem is known, they can proceed to identify what data is required, and what specific results or output they are looking for. They will also need to define clear KPIs that will measure the success of this project.</p><h4>2. Collect the Data</h4><p>The next step is to gather data from various sources. Some companies have subscriptions to reliable external sources of data, such as for market intelligence purposes, and it might be useful to have a data engineer working with the analyst to understand how to organize these various formats for the analysis.</p><h4>3. Perform Exploratory Data Analysis and Cleaning</h4><p>This step aggregates the data and visualizes it to form a solid understanding about its distribution, properties, and features. A domain expert might be able to shed more light on the variables and their behaviours. At this stage, the data might need cleaning to fix errors, omissions, or irregularities such as defaults.</p><h4>4. Perform Data Analysis</h4><p>The actual data analysis process is what uncovers insights hidden in the cleaned data. It could use simple statistical models or complex ML algorithms. Most of the time, the data has to be split into a separate train set on which an ML model is trained and a test set on which it might be tested. These methods expose the underlying causal relationships in the data, providing context and allowing business owners to make more informed decisions.</p><h4>5. Interpretation and Reporting</h4><p>The findings of the analysis now have to be translated into bsuiness context and presented in a suitable format — a dashboard, report, or infographic that summarizes all the key insights and their implications.</p><h3>What are some really great AI tools for Data Analytics?</h3><p><strong>AI tools automate the entire flow</strong> from data ingestion from multiple sources, cleaning (identifying headers and renaming them, removing or imputing null values, identifying ML features from categorical and numerical variables), applying transformations, and organizing the data for analysis. Generative AI has further revolutionized the analytics capability by allowing users to interact in natural language and experiment with plausible scenarios, all within a single conversation. In many cases, these tools run the analysis automatically from just a natural language prompt guiding the whole process. My recommendations:</p><h4><a href="https://julius.ai/">Julius AI</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*j7jM81PJVWjl3JZc" /><figcaption>Source: Julius.ai</figcaption></figure><p>This is a fantastic tool for data analysts, students, marketers or researchers— you could use it to simply upload files or connect live data sources, allow collaboration with teams, and get amazing visualizations such as bar graphs and pie charts or correlation matrices, or perform analysis such as significance testing, time series analysis, etc, all through natural language prompts. There are examples online of people using it in many ways, for instance, to find the break even point of a coffee business simply by sharing the menu and prices, then using that insight to build multiple what-if scenarios and eventually make a decision for or against starting the business. Overall, Julius is a great conversational way to explore data and even learn from the code running in the background which it openly shares.</p><h4>ChatGPT</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*2g9FuKs_wqzXmkrd" /><figcaption>Source: openai</figcaption></figure><p>ChatGPT is also a valuable tool for data analysis — with the same conversational abilities that power tools like Julius AI. The difference is that tools like Julius have an entire system built around it to support data analysis, whereas ChatGPT doesn’t. However, some of its standout features are: it automatically creates an interactive table view- users can view and scroll through all of their data as rows and columns. ChatGPT also determines the best chart type fitting the dataset, but it can also be specified through the user’s prompt, graphas are customizable, and summaries of findings are generated automatically. With its line of reasoning models, users can perform regressions on test data, visualize complex business metrics, or run multiple scenario simulations.</p><h4><a href="https://www.quadratichq.com//">Quadratic AI</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Kpw2RRwxa9uLU99bScnkqg.png" /><figcaption>Source: quadratic.ai</figcaption></figure><p>This is another useful tool that works in a spreadsheet format — it allows to upload CSVs, Excel, PDFs, or connect databases directly — allows to ask natural language questions and build repeatable analyses, visualizations, all within the same spreadsheet.</p><h4><a href="https://www.thebricks.com/">Bricks</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/796/1*ZOotZ2lOa8MGspQFbe2aCg.png" /><figcaption>Source: thebricks.com</figcaption></figure><p>It is a simple platform to use natural language to quickly build editable dashboards, slides or reports from a data source such as a CSV or Excel file, or a connected database. With features such as interactive filters, quick refreshes for updated data, and reusable templates, Bricks gains acceptability as something more automated than Excel, but less complicated than Tableau or PowerBI.</p><h4>AI Features in Power BI</h4><p>Traditionally, Power BI has been one of the most popular options for business intelligence in the industry. It has also been quick to integrate AI into its assortment of features. Power BI comes with Microsoft Copilot’s natural language conversation abilities, allowing users to ask questions about the data and get them answered immediately, for e.g., <em>“What are our best-selling products?”</em></p><p>Other features include sentiment analysis — if the dataset uploaded into Power BI has textual data such as reviews, then it can automatically classify the sentiment of this text as positive, negative, or neutral — this is part of the Text Analytics feature. Root cause analysis is a feature to explore the data and drill down to the root cause of an event based on certain criteria. In addition, the anomaly detection feature identifies outliers or irregularities in datasets. Predictive analytics built into Power BI such as its forecasting ability can help forecast trends and give insights into potential future scenarios.</p><p>It also allows image and video analysis — summarizing or captioning an image or video accurately, generating metadata, etc. It also has AI-assisted SQL generation built in, which enhances user productivity. Overall, Power BI is also an exciting AI-augmented data analysis tool used in the industry to prepare interactive, information-packed dashboards.</p><h4>Conclusion</h4><p>This is not an exhaustive list of AI tools used in data analysis processes. In conclusion, it seems that these tools can do most of the heavylifting when it comes to cleaning and organizing data for analysis, at least for most simple analyses, provided the instructions to do this are given with clarity and with due planning.</p><p>As AI empowers us with the speed and versatility to deal with many kinds of file formats in different ways all at once, it becomes all the more important to design a process which gives importance to the core problem and solving it in the simplest way.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=81d2faa270fb" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-curious-algorithm/ai-analytics-and-a-new-age-of-data-81d2faa270fb">AI, Analytics, and a New Age of Data</a> was originally published in <a href="https://medium.com/the-curious-algorithm">The Curious Algorithm</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fantastic AI Tools and Where to Find Them]]></title>
            <link>https://medium.com/the-curious-algorithm/fantastic-ai-tools-and-where-to-find-them-9a15b4990a5e?source=rss-f16c954592b0------2</link>
            <guid isPermaLink="false">https://medium.com/p/9a15b4990a5e</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[women-in-tech]]></category>
            <dc:creator><![CDATA[Amrin Kareem]]></dc:creator>
            <pubDate>Sat, 20 Sep 2025 13:26:59 GMT</pubDate>
            <atom:updated>2025-09-20T13:26:59.898Z</atom:updated>
            <content:encoded><![CDATA[<h4>A generative AI tool quest</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*unQFL5wq3LYpW3oRUXWO4A.png" /><figcaption>AI-generated image</figcaption></figure><p>So you’ve heard of generative AI. At this point, it’s quite impossible to ignore it, actually. You’ve seen the generated images, videos, and you’ve used a little bit of ChatGPT, but that’s about it. Now you’d like to know more about how to play around with the technology. If this is you, you’re at the right place.</p><p>In this article, we look at the capabilities of generative AI and the tools that enable it. Before we go into that, I’d like to distinguish generative AI and discriminative AI.</p><h4>Generative AI vs Discriminative AI</h4><p>The textbook definition of discriminative AI is an approach that learns the conditional probability distribution of the training data, such that, given new or unseen data points, it can classify them into distinct classes based on the distribution. This means that a discriminative AI model can draw a decision boundary between different classes of data. For instance, distinguishing between spam and non-spam emails is an example of a discriminative model at play. Examples include k-NN, Logistic Regression and decision trees.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/868/1*-D5c8_iDXszahnR7FoHNcw.png" /><figcaption>Source: <a href="https://www.datacamp.com/blog/generative-vs-discriminative-models">This excellent article from datacamp</a></figcaption></figure><p>However, discriminative models cannot generate new data similar to the training data or understand the context surrounding the data.</p><p>On the other hand, generative AI models capture the underlying joint probability distribution of the data and its associated labels. Using this distribution, it can generate new data, much like the data it was trained on. Some examples of generative models include Naive Bayes models, Hidden Markov models, and GANs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/861/1*1JZN5_YJiyR7VY5VqH3h4g.png" /><figcaption>Source: <a href="https://www.datacamp.com/blog/generative-vs-discriminative-models">datacamp</a></figcaption></figure><p>Today, generative models are more advanced than HMMs or Gaussian Mixture Models, which were some early examples. Generative AI models take a prompt and produce outputs that adhere to the context and intent of the input. The modalities of these model inputs and outputs include, but are not limited to, text, image, audio, video, point clouds, files, and programming languages.</p><p>Let’s start with a high-level overview of generative AI capabilities:</p><ul><li>Text Generation</li><li>Image Generation</li><li>Video Generation</li><li>Code Generation</li><li>Data generation</li><li>Virtual World Generation</li></ul><h4>Text Generation</h4><p>Text generation models generate new text, given a piece of text as input. The tasks they can do include question answering, paraphrasing, summarisation, translation, and text completion. Where do these capabilities come from? The short answer is LLMs. LLMs were trained on vast amounts of textual data — covering entire books, research papers, journals, and online information sources, followed by multiple stages of post-training. With its concrete understanding of the human language structure and usage patterns, an LLM can generate coherent, grammatically immaculate and contextually relevant text.</p><p>Here are some examples of text generation models in the industry and how they make our lives easier:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*FkIVwEdaod0jpdfw" /><figcaption>Chatbots and virtual assistants — Atlassian Intelligence</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*rKQIQWo-bg8GhG6_" /><figcaption>A screenshot of Rytr, an AI text/blog generator. Source: Zapier</figcaption></figure><h4>Text Generation Tools</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GPEudkbQ5NRB0Hy3_qiTwA.png" /><figcaption>AI text generation tools</figcaption></figure><h4>Image Generation</h4><p>Image generation models generate high-quality images from text prompts. Sometimes they also take an image as an input to generate an output image. These models are trained on huge datasets comprising billions of images. There are 2 types of image generation models — diffusion models and generative adversarial networks.</p><p>Diffusion models are trained as follows: the first step is a forward diffusion process. In this process, noise is added to an image in a number of steps. This goes on until the data is completely transformed into noise. The next step is the reverse process — here, the model learns to remove the noise added in each step of the noising process. At inference time, a noise vector is refined until an image or video is produced as output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*UPYoeevSQX1WLf5LliWzBA.png" /><figcaption>Source: <a href="https://guides.csbsju.edu/AI-Images">How AI creates images — Artificial Intelligence and Images — Research Guides at College of St. Benedict/St. John’s University</a></figcaption></figure><p>Generative adversarial networks have been around since 2014. These models consist of 2 sub-models- the generator and the discriminator, which are trained in an adversarial setup. The generator receives random noise as input and produces an output image similar to the training dataset. The discriminator now tries to classify this image as a fake/generated image or a real input from the training dataset. The training proceeds with feedback to the generator until the discriminator fails to correctly distinguish real images from generated ones.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*4Hvhh238ZxwLh9mcU3zZeA.png" /><figcaption>Source: <a href="https://blogs.mathworks.com/deep-learning/2021/12/02/synthetic-image-generation-using-gans/">Mathworks</a></figcaption></figure><p>In addition to image generation, these models can perform AI-based image manipulation or editing. For example:</p><p>Image-to-image translation transforms an image from one domain, such as sketches, to another domain, like cartoons.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*q3A1y1w1ECXQHp1mBRMlRg.png" /><figcaption>Source: Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks (ICCV 2017)</figcaption></figure><p>Inpainting reconstructs the missing or damaged parts of an image, finding application in art restoration and forensics.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*1_X7aeuN1Fu5k6Ay3pGhrg.png" /></figure><p>Outpainting extends the image outwards by generating new context and landscape around the actual subject. This can make images larger and improve resolution.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/1*bqv3d0Kb6atrTJEEsJvqdw.png" /></figure><h4>Image Generation Tools</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e0dtDcGSXActn6z5VkDOJg.png" /></figure><h4>Audio Generation</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*FrjajH3emfOlLRNXtPlV7Q.png" /><figcaption>Source: Meta</figcaption></figure><p>Generative audio tools mainly belong to one of three categories: speech generation tools, music generation tools, and audio editing tools. Speech generation tools are common, such as TTS (text-to-speech) tools that convert written text into audio. Music generation tools can create new compositions with preset style, language, and choice of instruments. Suno.ai is an example of an AI-generated music platform. The voices generated by the underlying models of these applications often sound completely natural, although they’re synthetic. The third kind of tools can convert voices, clean up audio, reduce noise and improve quality.</p><p>The following are some examples from Bytedance’s Make-an-Audio repo:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*j4vObH58Vqzd5co1ZUWTZg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*ykdonXKgmZp-IRCfBqnFww.png" /></figure><p>These models are built on top of deep learning algorithms trained on enormous datasets of human speech. They have a good understanding of vocal characteristics and can generate similar-sounding audio in terms of pronunciation, emotion, speed, etc.</p><h4>Audio Generation Tools</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PB43nS2rdiad6NH8WoA6iw.png" /></figure><h4>Video Generation</h4><p>Generative models can generate videos of varied complexity and style, from simple animations to complex, realistic and dynamic scenes. Sora AI, for example, uses a diffusion transformer to generate videos in response to textual prompts. Imagen by Google, on the other hand, uses about 7 models in a cascading fashion to generate video output. These models can integrate seamless transitions and smooth motion, with the potential to guide the video generation process.</p><h4>Video Generation Tools</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OJWw2sFxP1Fd0lz64wczcA.png" /></figure><h4>Code Generation</h4><p>Like text, generative models that are trained on code repositories learn the syntactical details and semantics of programming languages, generating new code snippets, functions or performing refactoring changes, fixing bugs, writing comments, and more. GitHub Copilot, OpenAI Codex, Claude Code, Cursor are examples of AI programming assistants with varying degrees of autonomy and specialisation. While specialised tools focus on specific tasks such as writing tests and PR reviews, generic tools support many coding activities such as code completion, error explanation, bug fixes, etc. Autonomous coding tools such as lovable or bolt.new often use a single text prompt to arrive at a complete output, generating both code and a preview of the solution, whereas assistants provide guidance at steps along the coding journey of the user.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*y54IQDs5-K0fLKRcrKgm4g.png" /><figcaption>Source: GenerativeProgrammer</figcaption></figure><h4>Virtual World Generation</h4><p>Another capability of generative AI models is virtual world generation. Models can create avatars and virtual environments with complex detailing in texture, sounds, and unique elements. While avatars can copy expressions, behaviour, and language, virtual environments are close in appearance and perception to the real world. An example is of metaverse platforms with personalised experiences for individual users. Metaverse is a virtual 3D environment where people can interact with the support of virtual and augmented reality. Some of these aspects can be found in games such as Second Life, Roblox, etc, and Mesh for Microsoft Teams. Other applications are found in education, gaming, entertainment, and virtual influencers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AsB5gibrp8TF3bpnjORu5g.png" /></figure><p>So far, we have seen some interesting and creative generation capabilities of AI. This is not an exhaustive list, and new innovative applications and use cases are discovered every day. Also, <a href="https://www.coursera.org/learn/generative-ai-introduction-and-applications?specialization=ibm-generative-ai-engineering">this course by IBM</a> helped with writing this article. Thank you for reading this post, and if this helped you, please feel free to leave a clap or comment.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9a15b4990a5e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-curious-algorithm/fantastic-ai-tools-and-where-to-find-them-9a15b4990a5e">Fantastic AI Tools and Where to Find Them</a> was originally published in <a href="https://medium.com/the-curious-algorithm">The Curious Algorithm</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Future of Jobs in a World of AI]]></title>
            <link>https://medium.com/the-curious-algorithm/the-future-of-jobs-in-a-world-of-ai-6ab07a8016d4?source=rss-f16c954592b0------2</link>
            <guid isPermaLink="false">https://medium.com/p/6ab07a8016d4</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[women-in-tech]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[future-of-work]]></category>
            <category><![CDATA[job-search]]></category>
            <dc:creator><![CDATA[Amrin Kareem]]></dc:creator>
            <pubDate>Sat, 23 Aug 2025 10:38:21 GMT</pubDate>
            <atom:updated>2025-08-23T10:38:21.416Z</atom:updated>
            <content:encoded><![CDATA[<h4>Are our jobs AI-proof?</h4><p>At work, things change. This is how it has always been, and these changes are largely due to technological advancements. Examples of this phenomenon have appeared over the years — desktop computers, digital commerce, mobile phones, and social media. Fast forward to today, and more than 10% of professionals hired today have job titles that didn’t exist in 2000.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DX8mWQvnJs4mL_YZ6zRVpA.jpeg" /></figure><p>At this point, we are in flux again. LinkedIn’s <a href="https://economicgraph.linkedin.com/research/work-change-report">Work Change Report</a> shows that the jobs available and the skills associated with them are changing rapidly. The workplace is shifting — driven by AI and generative AI in particular, these technological developments are set to transform the labour market.</p><p>It is predicted that by 2030, 70 per cent of skills used in most jobs will have evolved, and AI will be a catalyst driving this change. <em>“Artificial Intelligence Engineer”</em> is one of the fastest-growing job roles in 15 countries.</p><p>The major takeaway here is that professionals have to work to keep up with the breakneck pace at which these changes are happening. LinkedIn further reported that there has been a 140 per cent increase in the pace at which new skills are being added to LinkedIn profiles after 2022.</p><h4>Attitudes and Perceptions of Business Leaders: Are They Ready to Embrace the Change?</h4><p>McKinsey, in their 2025 report <a href="https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/superagency-in-the-workplace-empowering-people-to-unlock-ais-full-potential-at-work">Superagency in the Workplace: Empowering People to Unlock AI’s Full Potential</a>, compares AI to the steam engine, citing their transformative potential to be equally powerful. Since the release of ChatGPT in 2022, similar models from Meta, Google, Anthropic, Cohere, and others have been launched and popularised, ushering in a new era of LLMs.</p><p>Most senior executives appreciate the speed, but at the same time, they are wary of safety challenges such as data security and accuracy. Studies put the onus on the leaders of organisations to establish robust governance structures and ensure continuous training and adherence to regulatory requirements.</p><p>In the UAE (where I am based), the 2025 State of AI report by Deloitte AI Institute and MBZUAI sheds new light on the challenges faced by organisations when investing in AI and related technologies, and ensuring they understand its true potential. The report gathered insights from more than 150 business leaders in the UAE, KSA, and Qatar. Their findings are interesting:</p><ul><li>About 69 per cent of leaders and organisations across the Middle East are increasing AI investments despite challenges</li><li>The major challenges identified include selecting the right tools and technology aligned with the organisational objective</li><li>Another main challenge is identifying use cases within the organisation</li></ul><p>In another finding by the PwC Middle East Workforce Survey 2024, employees in the Middle East are far ahead of their global counterparts in terms of AI/ML tool use, understanding of their organisation’s goals, and awareness of new skills, tools, and technologies to perform better in their jobs. A significant percentage of this population prioritises flexibility and adaptability to a volatile job market.</p><p>In short, the UAE market is strongly positioned for an AI wave of transformation.</p><h4>AI in the Workplace</h4><p>By and large, the use of AI has been, like all technology, to make lives easier. To augment work and not to substitute or replace humans. Roughly speaking, if you’re a business leader, you’re probably using AI applications to support decision-making. Maybe you want guidelines on the best business processes or deep market research-based insights; there are AI applications that support this.</p><p>If you’re a non-technical employee or manager, you’d probably want to automate a routine task such as document processing or data entry. The goal here is to reduce human error, increase efficiency, and enhance productivity and creativity, as your time is now freed up to focus on more human tasks and less robotic ones.</p><p>If you’re a technical person, you’re looking for more speed and efficiency, as well as higher productivity. You would want to utilise AI tools to support the development process, and we routinely see this, especially with generative AI in application modernisation and platform engineering, providing code refactoring, research support, and more.</p><p>Apart from the use cases mentioned above, specific departments such as legal, finance, HR, or marketing can identify gaps and implement enterprise AI solutions to ease their work and streamline operations, processes, and efficiency.</p><h4>On AI and Job Displacement</h4><p>This has been talked about a lot as long as technology has been in existence — the disappearance of jobs and the replacement of humans in mainstream work. Last month, Microsoft researchers released a list of 40 roles that AI is expected to impact, and I was quite surprised to find that data scientists made the list. There are two aspects to the job replacement/displacement debate that I see:</p><p>For instance, in my first job, we had to create folders for each test case that we had prepared — this could go up to 100 folders at a time. A simple Python script could do what would otherwise take up half your day and be perceived as an extremely distasteful task. McKinsey has some estimates — about 30% of work hours could be automated, which means part of our everyday tasks could go away, leaving space for high-quality, creative work. But over time, job roles will evolve like they always have. It is predicted that eventually, there will be over 12 million job transitions.</p><p>Overall, AI deployment has had an impact on revenue growth and employee retention. However, the job market is shifting. The long-term impact is that if your job doesn’t involve any human value creation or addition, there is a significant need for upskilling/reskilling so that the workforce uses AI tools to their benefit — they are present when the changes happen, and they know how to wield it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*efcNqxU5EN35YPKp" /></figure><p>LinkedIn Work Change Report emphasises the term adaptive talent — also known as agile talent, the demand for which is rising. The ideal candidate can collaborate with others to solve complex challenges and can leverage AI to automate and improve existing processes. AI literacy is also a deciding factor as the emerging skills-based job market seeks AI-literate non-technical professionals with people skills.</p><p>In closing, the outlook isn’t bad. What we truly need is the mindset shift that does three things: Adapt — embraces changes as they appear, Innovate — build solutions to challenges, and Upskill — active learning is the way forward.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6ab07a8016d4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-curious-algorithm/the-future-of-jobs-in-a-world-of-ai-6ab07a8016d4">The Future of Jobs in a World of AI</a> was originally published in <a href="https://medium.com/the-curious-algorithm">The Curious Algorithm</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>