<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Tech @Tally on Medium]]></title>
        <description><![CDATA[Stories by Tech @Tally on Medium]]></description>
        <link>https://medium.com/@techattally?source=rss-996d4db5c7dc------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*25hruZJY2JCwby6_0at2YA.jpeg</url>
            <title>Stories by Tech @Tally on Medium</title>
            <link>https://medium.com/@techattally?source=rss-996d4db5c7dc------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 20:47:55 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@techattally/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[When Vibe Coding Meets the Enterprise: A Live Comparison]]></title>
            <link>https://medium.com/@techattally/vibe-coding-vs-spec-driven-development-enterprise-test-fccaed4b6351?source=rss-996d4db5c7dc------2</link>
            <guid isPermaLink="false">https://medium.com/p/fccaed4b6351</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[enterprise-technology]]></category>
            <category><![CDATA[vibe-coding]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Tech @Tally]]></dc:creator>
            <pubDate>Thu, 06 Aug 2026 09:01:08 GMT</pubDate>
            <atom:updated>2026-08-12T12:52:17.558Z</atom:updated>
            <content:encoded><![CDATA[<p>My manager sent a link on a Tuesday afternoon. No preamble, just a URL and a message: <em>“Can you take a look? We need this ready for the event — it’s in five days.”</em></p><p>The link opened a responsive web application. Registration forms, a dashboard, data persisting in real time. It worked. Built by a business head using vibe coding — iterating with an AI model, prompt by prompt, until something functional emerged — and hosted on a free platform backed by Supabase. For a solo, non-engineering build completed under deadline pressure, it was genuinely impressive.</p><p>But the event was an enterprise function. The app needed to run on our cloud, pass a security scan, and survive a Vulnerability Assessment and Penetration Test (VAPT). That changed the picture entirely.</p><p>What followed was one of the more instructive weeks I’ve had as an engineer: running two approaches to the same problem in parallel, under real conditions, with real security requirements. This is what I found.</p><h3>The two approaches, briefly</h3><p><strong>Vibe coding </strong>is intuition-led, prompt-driven development. You describe what you want, iterate on the output, patch what breaks, and keep going until it works. The primary artefact is the working code and the prompt history that produced it. Context lives in the session. Requirements are implicit — embedded in what the model understood you to mean.</p><p><strong>Spec-driven development </strong>inverts that. Requirements are made explicit first, captured in a structured specification document (I’ll call mine SPEC.md ). An AI coding tool — in this case, Claude.ai — works from that document rather than from freeform prompts. The spec is persistent context. It travels across sessions, enforces architectural decisions, and gives reviewers something to audit.</p><p>Neither is inherently superior. They optimise for different things. What this article does is show what that difference looks like when the stakes are real.</p><h3>1. Reverse-engineering the original app</h3><p>Before I could build the spec-driven version, I needed to understand what the original app actually did — not what it was supposed to do, but what it had been prompted into doing.</p><p>I spent a day working through it: tracing data flows, reading the Supabase schema, mapping the UI states. A few things became clear quickly.</p><p>The core functionality was sound. Users could register, submit data, and see results. The Supabase integration was functional. The UI was responsive and reasonably polished for a vibe-coded build.</p><p>What was missing was everything below the surface. Authentication logic had gaps. API calls were made without error handling in several places. Environment variables were managed in ways that wouldn’t survive a security audit. There was no documentation — no README, no data model, nothing that would tell a second developer what the system was doing or why.</p><p>None of this was the builder’s fault. Vibe coding optimises for getting something working. Security hygiene, error handling, and documentation aren’t things a prompt naturally surfaces unless you ask for them explicitly — and even then, consistently enforcing them across a multi-session build is hard.</p><p>The reverse-engineering process became the foundation of the spec. Every gap I found was a requirement I could write down.</p><h3>2. Building the spec-driven app</h3><p>I wrote the SPEC.md in about 45 minutes. It covered:</p><ul><li><strong>Functional requirements</strong> — The features the original app had, translated into explicit acceptance criteria</li><li><strong>Data model</strong> — The Supabase schema, documented and validated against the actual tables</li><li><strong>Authentication and authorisation</strong> — Rules for who could access what, and how sessions should be managed</li><li><strong>Input validation</strong> — Field-level rules for every form input</li><li><strong>Error handling</strong> — Expected behaviour for network failures, validation errors, and edge cases</li><li><strong>Security requirements</strong> — Explicit constraints on environment variable handling, API exposure, and data access patterns</li><li><strong>Non-functional requirements</strong> — Performance expectations, deployment target, logging</li></ul><p>The document wasn’t long. But it was the first time the app’s requirements existed anywhere other than inside a series of prompts.</p><p>I fed the spec to Claude.ai and worked through the build section by section. The key difference from vibe coding was visible immediately: when the model made a decision I hadn’t anticipated — a different state management pattern, an unexpected API structure — I could evaluate it against the spec and either accept or correct it. There was a reference point. Vibe coding doesn’t give you that. You evaluate the output against your mental model, which degrades across sessions, and against whether it runs, which is a much lower bar.</p><p>The re-engineered app took longer to start. The spec writing, the initial setup, the structured review process — that overhead is real. But it compressed dramatically in the later stages, where the original app would have started accumulating friction.</p><h3>The security test: same bar, very different results</h3><p>Both apps went through identical enterprise onboarding: deployment to our cloud environment, a security scan, and a full VAPT.</p><p>The results weren’t close.</p><p><strong>| Original app (vibe coding): </strong>Returned more than 10 high-priority findings on the security scan. The VAPT surfaced several priority-level vulnerabilities — issues that, in a production system, could expose user data or allow unauthorised access. The remediation work was significant. The app needed to be patched before it could be approved for deployment, which introduced its own risk: patching a vibe-coded app without a spec means reasoning about what each change might break, without documentation to guide you.</p><p><strong>| Re-engineered app (spec-driven): </strong>Returned 2 low-priority security issues on the scan. Both were straightforward — caught early, fixed cleanly, and verified before the VAPT ran. The VAPT findings were addressed in the same cycle. The app cleared enterprise onboarding.</p><p>To be clear: <strong>neither result was about the quality of the builder</strong>. The business head who built the original app shipped something functional in a few days with no development background. That’s not a small thing. The security gaps were a predictable consequence of the approach and the context — not the person.</p><p>What the spec-driven process did was make security a first-class requirement from the start, rather than something to retrofit after the fact. That’s the structural difference.</p><h3>Side-by-side comparison</h3><p>The table below maps both apps across all 12 dimensions. The security and VAPT rows are the ones that matter most for enterprise contexts. The others explain why those rows turn out the way they do.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/736/1*Wlkw0vrgzlKzniKC4XXjlQ.png" /></figure><h3>When to use which</h3><p>This is not a verdict against vibe coding. It’s a map.</p><p><strong>Vibe coding is well-suited when:</strong></p><ul><li>You need a working prototype in hours or days, not a production system</li><li>The builder is non-technical, and the goal is to prove an idea</li><li>The app won’t handle sensitive data or face external users</li><li>You’re exploring a problem space before committing to an approach</li><li>Failure is low-stakes and reversible</li></ul><p><strong>Spec-driven development is the right choice when:</strong></p><ul><li>The app will be deployed to an enterprise environment</li><li>It will handle user data, authentication, or any security surface</li><li>Multiple people need to work on it, maintain it, or hand it over</li><li>You need to demonstrate compliance or pass a security review</li><li>The build will span multiple sessions or weeks</li></ul><p>The honest version of this is: the original app was the right tool for getting to “does this idea work?” in five days. It was the wrong tool for getting to “can we deploy this in production?” Spec-driven development answered the second question, but it needed the original app’s existence to know what to spec.</p><p>A hybrid approach — vibe code a rough proof of concept, then formalise a spec before any production work begins — is probably the most pragmatic path for teams that want speed and safety. The cost of writing a spec from a working prototype is low. The cost of retrofitting security into a vibe-coded app under deadline pressure is not.</p><p><em>Originally published at </em><a href="https://tallysolutions.com/technology/when-vibe-coding-meets-the-enterprise-a-live-comparison/"><em>https://tallysolutions.com</em></a><em> on August 6, 2026.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fccaed4b6351" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mastering AWS Cloud Costs: Automation, Optimization, and Analytics for FinOps Success]]></title>
            <link>https://medium.com/@techattally/7-pillars-of-finops-maturity-for-cloud-cost-savings-in-2026-7c52a442073e?source=rss-996d4db5c7dc------2</link>
            <guid isPermaLink="false">https://medium.com/p/7c52a442073e</guid>
            <category><![CDATA[dashboard]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[finops]]></category>
            <category><![CDATA[automation]]></category>
            <category><![CDATA[cloud]]></category>
            <dc:creator><![CDATA[Tech @Tally]]></dc:creator>
            <pubDate>Thu, 02 Jul 2026 09:17:29 GMT</pubDate>
            <atom:updated>2026-07-29T09:24:58.471Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>30 second summary |</strong> FinOps is a cloud engineering discipline that embeds cost accountability into infrastructure. Built on seven pillars — intelligence, optimisation, automation, savings strategy, and multi-cloud FOCUS — mature practices cut cloud waste 20–30% in year one, not by restricting engineers but by giving them shared visibility and ownership.</p><p>Your AWS bill grew 40% last quarter. Your engineers are shipping faster than ever. But nobody can explain exactly why costs spiked — or who owns them.</p><p>This is the FinOps gap: the space between cloud agility and financial accountability. At scale, especially across multiple payer accounts, cloud cost management is no longer a matter of reviewing monthly invoices. It requires a fundamentally different operating model.</p><p>True FinOps maturity combines cost intelligence, automation, and optimisation intelligence into a single, continuously running discipline. This blog walks through the seven pillars that make that possible.</p><h3>1. The foundation: Cost intelligence dashboard</h3><p>At the heart of effective FinOps is a centralised cost intelligence dashboard — a single source of truth that spans every payer account, every service, and every team. Without this layer, everything else is reactive.</p><p>Think of it as your financial control plane for the cloud.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wXd_s282m_VEVyOs.jpg" /></figure><p><em>Cost Intelligence Dashboard: multi-payer overview</em></p><h3>What it should solve</h3><p><strong>✔ </strong>Unified visibility across all payer accounts and linked accounts</p><p><strong>✔ </strong>Granular drill-down from service → account → tag → resource</p><p><strong>✔ </strong>Budget vs. actual tracking with threshold alerts</p><p><strong>✔ </strong>Savings plan utilisation and coverage monitoring</p><p><strong>✔ </strong>Automated anomaly detection before month-end surprises</p><p><strong>✔ </strong>Business-level cost allocation and chargeback models</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1009/0*WoMJNFSca1N5hHjM.jpg" /></figure><p><em>Granular drill-down view-per-service cost breakdown</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/955/0*4XfHqzqDgKgxlK4E.jpg" /></figure><p><em>Budget vs. Actual tracking with anomaly flags</em></p><h3>Architecture overview</h3><p>A production-grade cost intelligence platform typically combines:</p><ul><li>AWS Cost &amp; Usage Reports (CUR) exported from every payer account</li><li>Centralised S3 bucket for consolidation</li><li>Amazon Athena for ad-hoc SQL queries on raw CUR data</li><li>QuickSight (or a BI tool of your choice) for dashboards and visualisations</li><li>Cross-account IAM roles to enable secure data ingestion</li></ul><p>This architecture enables plan-level cost visibility, per-user cost analytics, service-level costing, and chargeback/showback reporting. A mature dashboard should be able to answer four questions at any time:</p><p><strong>The four questions a dashboard must answer </strong><em>“Why did cost increase? | Who owns it? | Is it optimised? | Can it be automated?”</em></p><h3>2. Unified cost optimisation intelligence</h3><p>Visibility is step one. But visibility without action is just an expensive reporting exercise.</p><p>Step two is Optimisation Intelligence — taking the scattered recommendations that live across AWS tools and consolidating them into a single, actionable cockpit. Without this, teams end up with siloed insights: Compute Optimizer says one thing, Cost Explorer says another, and nobody has the full picture.</p><p>A mature FinOps organisation centralises signals from:</p><ul><li>AWS cost optimisation hub</li><li>CUDOS dashboards (supporting multi-payer environments)</li><li>Compute Optimiser for rightsizing signals</li><li>Savings plans utilisation and gap analysis</li><li>Graviton migration candidates identified automatically</li></ul><p><strong>The goal: </strong>A single optimisation cockpit where every recommendation has an owner, an estimated savings, and a status.</p><h3>3. High-impact optimisation levers</h3><p>Across hundreds of AWS accounts, four categories consistently unlock the largest savings. None of these requires architectural changes — they are operational wins available to any team willing to prioritise them.</p><h3>Compute optimisation</h3><ul><li>EC2 rightsizing via Compute Optimiser — typically yields 15–30% savings on over-provisioned instances</li><li>Intel → AMD instance transitions (e.g., m5 → m6a) for equivalent workloads at lower cost</li><li>x86 → Graviton modernisation — up to 20% better price-performance</li><li>Identification and termination of idle instances running below 5% CPU</li><li>Compute Savings Plans and Database Savings Plans for steady-state workloads</li></ul><h3>Storage optimisation</h3><ul><li>GP2 → GP3 EBS migration — GP3 costs ~20% less and offers better baseline performance</li><li>Automated cleanup of unattached EBS volumes (a silent cost driver in most accounts)</li><li>Aged snapshot policies to remove snapshots older than retention thresholds</li><li>S3 lifecycle policies to transition infrequently accessed data to cheaper storage tiers</li></ul><h3>Network optimisation</h3><ul><li>Audit idle VPC Endpoints and remove unused ones</li><li>Release unassociated Elastic IPs — AWS charges for reserved IPs not in use</li><li>Review inactive VPN connections and NAT gateway utilisation</li></ul><h3>Logging &amp; observability</h3><ul><li>Migrate CloudWatch Logs to Infrequent Access tier where daily query volume is low</li><li>Right-size log retention policies — 90-day default is rarely needed for all log groups</li><li>Remove orphaned log groups from deleted services</li></ul><h3>4. Automation: The force multiplier</h3><p>Optimisation without automation is temporary. You identify the waste, you clean it up, and three months later it’s back — because the underlying behaviour hasn’t changed.</p><p>The real transformation in FinOps happens when tagging standards, lifecycle scripts, and governance policies combine into a self-healing system. Instead of monthly cleanup sprints, resources are automatically right-sized, stopped, or terminated based on declared policy.</p><h3>Smart tagging strategy</h3><p>Tags are the foundation of automation. Without reliable tags, scripts cannot safely identify what to stop, resize, or delete. A mature tagging strategy captures:</p><p><strong>✔ </strong>Ownership — Owner, Business Unit, Project, Cost Centre</p><p><strong>✔ </strong>Environment — Prod, Staging, Dev, POC</p><p><strong>✔ </strong>Lifecycle — Persistent, Scheduled-Termination, Ephemeral</p><p><strong>✔ </strong>Criticality — For blast-radius assessment before automation acts</p><h3>Automation use cases in practice</h3><p>Once tagging is standardised, the following automations become safe to run at scale:</p><p><strong>✔ </strong>Stop all non-prod EC2 and RDS instances outside business hours — saving up to 65% on non-production compute costs</p><p><strong>✔ </strong>Auto-terminate POC and sandbox resources after a defined TTL (e.g., 14 days)</p><p><strong>✔ </strong>Delete unattached EBS volumes after X days of being in ‘available’ state</p><p><strong>✔ </strong>Resize Auto Scaling Group minimum capacity overnight during low-traffic windows</p><p><strong>✔ </strong>Clean up aged AMIs and their backing snapshots on a rolling 90-day schedule</p><p><strong>✔ </strong>Alert on unassociated Elastic IPs before the billing cycle closes</p><h3>5. Savings strategy: Reserved commitments done right</h3><p>One of the most common FinOps mistakes is treating Savings Plans and Reserved Instances as a procurement exercise — something the finance team “buys” once a year. That approach leaves money on the table.</p><p>Commitment strategies must be analytics-backed, modelled against actual usage patterns, and purchased in tranches to maintain flexibility.</p><h3>Compute Savings Plans</h3><ul><li>Track coverage across EC2, Fargate, and Lambda continuously — not just at renewal</li><li>Monitor utilisation: an unused Savings Plan is a liability, not a saving</li><li>Model break-even before committing — especially for variable or seasonal workloads</li></ul><h3>Database Savings Plans</h3><ul><li>Analyse RDS and Aurora usage for consistency before committing</li><li>Use usage consistency modelling to identify workloads suitable for 1-year vs. 3-year terms</li></ul><p><strong>The FinOps commitment rule </strong><em>You don’t “buy” Savings Plans. You model them, simulate break-even scenarios, validate utilisation trends — and then purchase strategically. Every commitment should be traceable to a utilisation forecast.</em></p><h3>6. Multi-cloud FinOps: The FOCUS dashboard</h3><p>Most enterprises today operate across AWS, Azure, and GCP — often as a result of acquisitions, team preferences, or strategic redundancy. This creates a fragmentation problem: each cloud has its own billing model, its own cost categories, and its own tagging conventions.</p><p>The result is that FinOps teams end up building three separate dashboards, three separate governance frameworks, and three separate anomaly detection pipelines. Cloud silos become cost silos.</p><p>The solution is FOCUS — the FinOps Open Cost and Usage Specification — an open standard that normalises cost data across cloud providers into a single schema.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*c9UY4Ic9qL1NZhTH.jpg" /></figure><p><em>Multi-cloud tagging strategy aligned to FOCUS schema</em></p><h3>What FOCUS enables</h3><p><strong>✔ </strong>Standardised cost datasets with consistent field names and semantics across providers</p><p><strong>✔ </strong>Cross-cloud cost normalisation — compare apples to apples across AWS, Azure, and GCP</p><p><strong>✔ </strong>Unified tagging strategy that maps to a common taxonomy</p><p><strong>✔ </strong>Single-pane analytics with one dashboard, one query layer, one governance model</p><p><strong>✔ </strong>Cross-cloud benchmarking to identify which provider delivers better unit economics</p><h3>7. Operationalising FinOps: From dashboard to action</h3><p>Tools and dashboards are necessary but not sufficient. The organisations that achieve lasting cost efficiency treat FinOps as an operating model — not a project. That means embedding cost accountability into the engineering culture, not delegating it to a central team.</p><p>A complete FinOps operating model has five layers working in concert:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4VrjWKiw1zBrMnB4-TsdwA.png" /></figure><p>Each layer feeds the next. Intelligence reveals the waste. Optimisation prioritises it. Automation eliminates it. Governance prevents it from returning. Strategy positions the organi s ation for the next wave of growth.</p><h3>Final thoughts</h3><p>Cloud cost mastery is not a destination — it is an operating discipline that compounds over time. The organisations that get it right share one trait: they treat cost as a first-class engineering concern, not an afterthought reviewed at month-end.</p><p>Ensuring the reliability and performance of your cloud applications without overspending is critical, and FinOps plays a key role in achieving this balance.</p><p>When visibility becomes granular,</p><p>recommendations become centralised,</p><p>automation becomes the default,</p><p>and optimisation becomes cultural — FinOps stops being a function and becomes the way your engineering organisation thinks about infrastructure.</p><p><strong>Key Takeaway </strong><em>FinOps is not a finance function. It is a cloud engineering discipline with financial intelligence embedded — and the teams that treat it that way consistently outperform those that don’t.</em></p><p><em>Originally published at </em><a href="https://tallysolutions.com/technology/mastering-aws-cloud-costs-automation-optimization-and-analytics-for-finops/"><em>https://tallysolutions.com</em></a><em> on July 2, 2026.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7c52a442073e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From LLM to SLM: The Art of Transferring Intelligence On-Device]]></title>
            <link>https://medium.com/@techattally/from-llm-to-slm-the-art-of-transferring-intelligence-on-device-301107512278?source=rss-996d4db5c7dc------2</link>
            <guid isPermaLink="false">https://medium.com/p/301107512278</guid>
            <category><![CDATA[knowledge-distillation]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Tech @Tally]]></dc:creator>
            <pubDate>Sun, 14 Jun 2026 06:05:10 GMT</pubDate>
            <atom:updated>2026-06-22T06:12:20.117Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>30 second summary |</strong> Knowledge Distillation is a technique that enables a smaller, more efficient AI model to learn from a larger, highly capable model by mimicking its predictions and reasoning patterns. By transferring knowledge through soft labels and learned relationships between classes, the student model can achieve performance close to the teacher while requiring far less memory and computational power. This makes it possible to deploy AI locally on devices, improving privacy, reducing cloud dependency, and supporting offline functionality. With applications ranging from mobile AI and healthcare to NLP and edge computing, Knowledge Distillation plays a key role in making advanced AI more accessible, efficient, and trustworthy.</p><p>The AI industry is growing rapidly, with every product and service incorporating AI features to deliver better, more user-friendly interactions that enhance productivity across various fields. The driving force behind many of these successful AI capabilities is Large Language Models (LLMs). Leveraging the power of LLMs has become a popular approach to boosting productivity, as exemplified by ChatGPT from OpenAI, widely used for its advanced language understanding.</p><p>However, despite its benefits, users often face concerns about privacy and data security. Even when AI features guarantee that user data is not used for model training, the lingering question remains: “Is my data safe?” Addressing this trust issue requires innovative solutions that ensure no user data ever leaves the local device.</p><p>Model efficiency is crucial in AI deployments, especially when balancing performance with the constraints of device resources and privacy needs. Knowledge Distillation offers a compelling approach to achieve this balance by transferring the knowledge of a large, complex teacher model into a smaller, more efficient student model. This teacher-student paradigm enables the creation of Small Language Models (SLMs) capable of running locally, without internet connectivity, providing users with privacy assurance and maintaining strong language understanding.</p><p>In this article, we explore how Knowledge Distillation can be leveraged to train SLMs with comparable comprehension to LLMs while addressing privacy and efficiency challenges in real-world AI applications.</p><h3>Introducing Knowledge Distillation!</h3><p>Knowledge Distillation (KD) is a model compression technique where a large, high-capacity teacher model imparts its knowledge to a smaller student model. The core idea is to train the student to mimic the behaviour of the teacher, i.e. to train the student to replicate the teacher’s predictive behaviour with significantly reduced computational and memory requirements, enabling deployment in resource-constrained environments without substantial accuracy loss, as large models have higher number of parameter indicating large layers and neurons for which to generate any response, each performs certain operations which requires more computation resources and more Memory to load for large models.</p><p>Before we dive into the Knowledge Distillation workflow, let’s get familiar with a few terminologies.</p><h3>Teacher and student models</h3><ul><li><strong>Teacher model:</strong> A well-trained, often large model, with high accuracy but has high resource requirements and complex architecture. This model generally requires a high-end GPU (usually runs on bare metal servers hosted locally or on cloud) for loading and processing.</li><li><strong>Student model:</strong> A smaller, lighter model designed to simulate the teacher’s performance with fewer resources and high speed. The purpose of student models is to be able to serve on low spec systems like smartphones, or IoT devices for providing enhanced assistance to the device’s features.</li></ul><h3>Hard labels vs soft labels</h3><ul><li><strong>Hard labels: </strong>These consist of one-hot encoded ground-truth classes used in conventional supervised learning</li><li><strong>Soft labels:</strong> These are the core of Knowledge Distillation and are the logits or softened probability distributions about inter-class similarities produced by the teacher model, providing richer information about the teacher’s confidence. Basically, soft labels are the genes inherited by learning from the teacher model with some variations, providing more generalization to the student model.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/848/0*VXZjn4yBFG2F1yC2.png" /></figure><p>Ref: Generated from Gemini 2.5 flash</p><h3>How is knowledge transferred?</h3><h3>Teacher model training</h3><p>Teacher model training is the foundational step in the Knowledge Distillation (KD) pipeline. The aim is to build a highly capable teacher model, which later will become a role model or idol to serve as a guide for a smaller student network. Hence, Teacher model works as a performance benchmark. It must capture informative and comprehensive patterns from training dataset and produce fine-tuned probability distribution that signifies inter class relationships. The more generalised and richer these insights are, the more effectively it can teach the student model. This stage determines the quality of information available for distillation and making its design, dataset, and optimization strategies critical.</p><h3>Importance of teacher selection</h3><p>Before directly jumping to teacher selection, let us investigate its significance. Selection and tuning of the teacher model are critical in Knowledge Distillation workflows because the quality, diversity, and calibration of the teacher’s knowledge directly determine the upper bound of student performance. The teacher’s output distributions (soft logits) are the primary source for the student, so any weakness — be it in overfitting, bias, poor calibration, or scope, will propagate or even amplify in the distilled student as generalization on top of these small mis-judgements extrapolates to even larger miscalculations.</p><p>Now we’ll have a look at factors defining the severity of selection process -</p><ul><li><strong>Generalization power</strong>: A student model learns not only what the teacher does well but also its weaknesses. If the teacher is overfitted or unstable, the student will likely repeat those problems, limiting how well it performs outside the training data.</li><li><strong>Calibration</strong>: A well calibrated teacher provides “soft” predictions that accurately reflect how confident it is in each class, including any uncertainty between them. If the teacher’s outputs are poorly calibrated, the student may learn misleading confidence patterns, which can reduce the final model’s reliability and robustness.</li><li><strong>Knowledge richness</strong>: Teachers that capture deeper patterns and relationships between classes, using methods like self-supervised pretraining or model ensembling or recently proficient training techniques like supervised reinforcement learning (a step wise framework with expert trajectories to teach small language models to reason hard problems), offers more detailed and meaningful “knowledge”. This rich information helps the student learn more effectively during distillation.</li><li><strong>Task alignment</strong>: Misalignment in task or domain between teacher and student can introduce knowledge transfer gaps, reducing practical benefit.</li></ul><h3>Considerations for teacher selection</h3><ul><li><strong>Model capacity vs overfitting</strong>: Select a model with sufficient capacity suitable and sufficient for the purpose of training to capture complexity but avoid excessive overparameterization, which may lead to noisy, non-generalizable outputs, as true strength lies not in force, but in focus. If we consider a large model with super high parameters for a simple classification task, all it will do is waste resources, when a simple and smaller model can do it proficiently, saving a lot of memory and computational power. Even sometimes, pruning or regularization is also needed post-pretraining.​</li><li><strong>Pretraining strategies</strong>: Utilizing foundation models pretrained on large, diverse corpus of data often builds teachers with better out-of-distribution generalization and richer soft labels. Fine-tuning on task-specific data is essential for relevance.​</li><li><strong>Optimization techniques</strong>: Apply loss landscape smoothing (e.g., SAM, an optimization algorithm that acts as a powerful regularizer by guiding models toward flat minima of the loss landscape, which generally leads to better generalization performance on unseen data), or Information bottleneck constraints make the model compress information intelligently, which leads to cleaner features and more general outcomes. This improves how well the model’s knowledge can be transferred or distilled into student models.</li><li><strong>Calibration methods</strong>: After or during training, fine tuning the teacher’s output probabilities using methods like temperature or Platt scaling, ensures that the teacher’s confidence is more accurate and well-calibrated, improving student learning from soft labels.​ How about having dynamic temperature during distillation? Does it ensure better calibration? Answer is not always, in worst case scenario it extrapolates the error patterns, resulting in bad outcomes. However, to compensate this generalization, taking loss of teacher into account can reduce the chance of these error patterns.</li><li><strong>Diversity via ensembles or multi-teacher methods</strong>: Use ensemble teachers to blend complementary error patterns or domain knowledge, providing students with a more robust supervisory signal (soft logits). Some methods build explicit multi-teacher assistant networks to smooth knowledge transfer and improve generalization.​</li></ul><h3>Training process (teacher model)</h3><p><strong>1. Prepare data </strong>: Start with a clean and representative dataset. Use data augmentation if the task benefits: this helps the teacher model learn all relevant patterns and reduces overfitting. ​ Data Augmentation techniques can be paraphrasing, noise injection, Synonym Replacement, Random Deletion, Translation, etc.</p><p><strong>2. Train with supervised learning: </strong>Train the teacher model on the full dataset using standard techniques, such as cross-entropy loss for classification tasks. Monitor validation accuracy and loss to track overfitting or underfitting. Adjust hyperparameters like batch size, learning rate, and optimizer for best results, for which technique like hyperparameter tuning can be used, to get an optimized set of parameters.</p><p><strong>3. Calibrate model outputs: </strong>Make sure the teacher model’s probability predictions are well-calibrated i.e. when the model is less certain, it shows that uncertainty in its output. “Temperature scaling” can be used to smooth predictions, which later helps the student learn better from soft labels. ​</p><p><strong>4. Evaluate and fine-tune: </strong>Check final model accuracy, calibration, and robustness before starting distillation. If needed, fine-tune the teacher on hard examples or relevant sub-tasks to ensure it excels at the purpose of student model. ​</p><h3>Distillation process</h3><p>Once the teacher model is up and running with strong performance and accuracy, just like a teacher who has studied and passed the exam, it is ready to transfer its knowledge to the student model. This is carried out in the following manner:</p><ol><li><strong>Use the trained teacher model</strong></li></ol><ul><li>Start with a teacher model that is well-trained and produces accurate predictions (probability outputs, not just class labels).​</li></ul><p><strong>2. Generate soft labels (knowledge extraction)</strong></p><ul><li>For each example in the training data, pass it through the teacher model and record the output probabilities (soft labels).</li><li>These soft labels are adjusted using a technique called “temperature scaling.” With a higher temperature, the outputs become softer (showing more uncertainty and similarities between classes, providing high level of generalization), giving richer information to the student model.​</li></ul><p><strong>3. Prepare the student model</strong></p><ul><li>Choose a smaller student model architecture designed for fast and efficient inference on target devices (like mobile or edge hardware). These are generally chosen from the same model family having similar architecture ensuring similar pattern recognition abilities.</li></ul><p><strong>4. Train the student model</strong></p><ul><li>Train the student model using two types of information: <br><strong>a) Hard labels:</strong> The true class labels from the dataset (enhancing accuracy). <br><strong>b) Soft labels:</strong> The probability distributions generated by the teacher model.</li><li>The student’s loss function balances matching the soft labels and correctly predicting hard labels, so the student learns both the correct answer and the reasoning behind it.​</li></ul><p><strong>5. Evaluate and tune</strong></p><ul><li>At the end of training, test the student model’s accuracy, speed, and resource usage. If the results are not satisfactory, adjust hyperparameters, re-run distillation.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/848/0*w7ucsh3Fc_aMbMwq.png" /></figure><p>Ref: Generated from Gemini 2.5 flash</p><p>In this way, K nowledge Distillation transfers not only the teacher model’s final predictions but also the detailed relationships between classes (soft logits). This helps the smaller student model learn more effectively, achieving performance close to the teacher’s accuracy while using much less computing power and memory.</p><p>Since we’ve looked at how actually distillation is done, let’s have a look at different variations of Knowledge Distillation.</p><p><em>“Experience truly trustworthy AI. With compact models that deliver high performance and unparalleled data security right on your device, boosting strategic success while maintaining complete control. “</em></p><h3>Variations of Knowledge Distillation</h3><h3>1. Knowledge types: How information is gathered</h3><p>These types are based on where the information is derived from the deep teacher model:</p><ul><li><strong>Response-based distillation: <br></strong>Information is obtained from the output layer of the teacher model. The student model is expected to mimic the teacher’s logits (class probabilities). This is the most popular genre of KD. The distillation loss often uses a divergence metric, such as Kullback-Leibler Divergence, to compare the teacher and student logits.</li><li><strong>Feature-based distillation:<br></strong>Knowledge is derived from the feature maps of the intermediate layers of the teacher network. Each layer of student learns from increasingly abstract feature representations of intermediate outputs. For example, this knowledge can be extracted from the penultimate layer of the teacher model for tasks like image classification.</li><li><strong>Relation-based distillation:<br></strong>This type explores relationships between different layers or data samples, going beyond the specific layer outputs used in response-based and feature-based methods. An example is the Flow of Solution Process (FSP), which is defined by the Gram matrix and summarizes the relations between pairs of feature maps.</li></ul><h3>2. Distillation schemes: Training strategy</h3><p>Distillation schemes classify the training setup based on whether the teacher network is updated simultaneously with the student model:</p><ul><li><strong>Online distillation:<br></strong>The teacher network is pre-trained and then frozen (not updated). The student network is trained using the knowledge from the static teacher model. Most conventional KD methods, including the original base paper by Hinton et al., operate in this manner.</li><li><strong>Online distillation:<br></strong>The teacher and student networks are trained simultaneously. This strategy is necessary when a large pre-trained teacher model is unavailable. It can involve mutual teaching, where sub-networks learn by mutually guiding each other via KD.</li><li><strong>Self-distillation: <br></strong>The same network acts as both the teacher and the student. This method addresses problems related to selecting an optimal teacher and preventing accuracy degradation in student models. Deeper classifiers (teachers) within the network are used to guide the training of shallower classifiers (students) using divergence and L2 losses. During the inference period, the additional shallow classifiers are dropped.</li></ul><h3>3. Distillation algorithms: Advanced frameworks</h3><p>These specialized algorithms are proposed to make the knowledge transfer process more efficient in complex and diverse settings:</p><ul><li><strong>Adversarial distillation:<br></strong>Applies principles of Adversarial Learning (like GANs). A discriminator measures the similarity between the teacher’s and student’s outputs, and the student acts as a generator attempting to “fool” the discriminator by producing outputs that imitate the teacher.</li><li><strong>Multi-teacher distillation:<br></strong>Involves distilling knowledge from an ensemble of several heavy teacher architectures to obtain diverse types of knowledge. The soft label outputs of the ensemble are typically averaged to provide supervision to the student.</li><li><strong>Cross-modal distillation:<br></strong>The process of transferring knowledge between different data modalities (e.g., image to audio). This is used when data or labels for certain modalities are missing, corrupt, or unusable.</li><li><strong>Attention-based distillation:<br></strong>Leverages attention mechanisms to focus the network on specific input details. Knowledge is transferred by matching the attention maps of intermediate layers between the teacher and student models.</li><li><strong>Quantized distillation:<br></strong>Aims to train a low-precision student model (e.g., 2-bit or 8-bit integer) from a high-precision teacher network (e.g., 32-bit floating point). This ensures the model is compatible with edge devices that require fixed-point inference for computation and power efficiency.</li></ul><h3>Practical cases of Knowledge Distillation</h3><p>Having a look into multiple variations of Knowledge Distillation, let have a look at how Knowledge Distillation is widely used in real-world AI systems where balancing model performance with resource constraints is crucial.</p><ul><li><strong>Mobile and edge AI:<br></strong>Distilled models run efficiently on smartphones, wearables, and IoT devices, enabling fast, low-power AI for tasks like real time call translation, image processing and predictive typing — all without reliance on cloud connectivity.</li><li><strong>Autonomous vehicles and robotics:<br></strong>Real-time perception and decision-making in autonomous cars and robots require models that are both accurate and fast. Distillation compresses large perception networks so they can operate within strict latency and hardware limits.</li><li><strong>Healthcare and medical devices:<br></strong>AI-powered medical diagnostics often need to run on privacy-sensitive devices. Distillation enables lightweight models that keep data local without sacrificing diagnostic performance.</li><li><strong>Natural Language Processing (NLP):<br></strong>Large pre-trained language models are resource intensive. Distillation lets developers create smaller, efficient language models for chatbots, translation, and sentiment analysis while maintaining quality.</li></ul><p>As such Knowledge Distillation offers an effective solution for deploying high-performance AI models on devices with limited resources or strict privacy requirements. By transferring knowledge from a large, well-trained teacher to a compact, efficient student model, organizations can achieve near-state-of-the-art accuracy while drastically reducing computational and memory demands.</p><p>Distillation can be performed through multiple variations, giving developers the flexibility to adapt techniques to their use cases and data availability. As AI continues to expand into everyday applications, knowledge distillation enables trustworthy, scalable intelligence for both commercial and edge deployments. Moving forward, ongoing research and practical innovations will further broaden its impact, making intelligent, efficient AI more accessible than ever. ​</p><p><em>Originally published at </em><a href="https://tallysolutions.com/technology/the-art-of-transferring-intelligence-on-device/"><em>https://tallysolutions.com</em></a><em> on June 14, 2026.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=301107512278" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Hidden Cost of Structured Output: How It Can Suppress LLM Reasoning]]></title>
            <link>https://medium.com/@techattally/the-hidden-cost-of-structured-output-how-it-can-suppress-llm-reasoning-334abc534cb9?source=rss-996d4db5c7dc------2</link>
            <guid isPermaLink="false">https://medium.com/p/334abc534cb9</guid>
            <category><![CDATA[architecture]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[prompt-chaining]]></category>
            <category><![CDATA[workflow]]></category>
            <dc:creator><![CDATA[Tech @Tally]]></dc:creator>
            <pubDate>Tue, 12 May 2026 09:49:45 GMT</pubDate>
            <atom:updated>2026-05-25T12:09:30.771Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>30 second summary |</strong> Structured Output makes LLM systems reliable and production-safe by forcing responses into predefined schemas. In an automated code review system, the model often skipped tool calls needed to fetch missing dependency code and instead produced confident structured answers based on assumptions. A path traversal vulnerability went undetected because the model treated storage.save() as a safe abstraction without inspecting its implementation. The solution was Prompt Chaining, separating reasoning from formatting. First, the model freely investigates dependencies and retrieves context; then, a second prompt converts that reasoning into structured output.</p><p>LLMs generate unconstrained text by default. To manage this, we often provide the target structure upfront and let the models fill it in — this is called Structured Output; It is vital for deterministic LLM workflows, thereby enabling the possibility of reliable production integration. By constraining the model to return responses in a predefined schema, we can reliably parse, validate, and route its outputs within automated pipelines.</p><p>This transforms LLM responses from free-form text into structured data that downstream systems can safely consume. For a wide range of tasks, the underlying assumption is that with sufficient context and a strict schema, the model can reason about the problem and produce a correct Structured Output in a single pass.</p><p>In theory, this approach looked perfect; in practice, it failed unexpectedly.</p><h3>When the architecture looked right but the behavior wasn’t</h3><p>The goal was to build an automated code review system that looks at dependent code for evaluation. This means the system must fetch definitions of classes, functions, and variables being used in the code diff to evaluate.</p><p>To support this, the model was given access to tools so whenever it needed more information, it could fetch code, inspect dependencies, or pull surrounding context before reviewing. Combined with a strict schema, this felt ideal. The model could reason as needed and still return production-safe outputs.</p><p>However, something odd kept happening during testing. In situations where the surrounding context was clearly missing, the model was expected to invoke the retrieval tool to fetch the required definitions. Instead, it often skipped the tool call entirely and returned a structured response as if it already had enough information. In other words, rather than pausing to reason about the missing context, the model proceeded to produce an answer directly.</p><p>At first, the issue seemed to be related to implementation. Perhaps the tool definitions were unclear, or the system prompt did not explicitly instruct the model to avoid hallucinating when context was incomplete. The prompts were revised, tool descriptions were clarified, and stronger instructions were added encouraging the model to call the tool whenever it encountered unknown functions or classes.</p><p>Despite these changes, the behavior remained largely unchanged. The model consistently produced well-formatted structured responses, but it did not reliably engage in the intermediate reasoning step that should have triggered a tool call. Instead of explicitly recognizing that context was missing and retrieving it, the model often attempted to complete the task using its own assumptions. The model wasn’t failing to achieve task; It was failing to reason before formatting.</p><h3>When structured schema skips reasoning</h3><p>Consider the following code snippet:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LdojJ404hCvewjh6IKiPwQ.png" /></figure><p>At first glance, nothing immediately stands out. The function extracts a filename from the request and asks the storage layer to save the file. When the model analyzed this snippet in isolation, the reviews usually looked reasonable. It might occasionally suggest validating user input but rarely flag anything serious.</p><p>The missing piece is how storage was expected to behave. That implementation lived in another file:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gYzQ9P_gDTDBwj8fXbYM3A.png" /></figure><p>The moment you see that implementation; the problem becomes obvious — upload_file() function passes user input directly as a file path. The vulnerability isn’t in storage.py; it’s in the function trusting User input without sanitization. But without seeing how save() function worked, the risk didn’t look real.</p><p>This is where the Structured workflow kept failing. The model often treated storage.save() function as a safe abstraction and moved on, because the schema pushed it toward producing a finalized answer quickly. Even when tools were available, it didn’t always fetch the implementation before committing to the Structured Output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9eVN4Q4f58kDT1xkeWkeDQ.png" /></figure><h3>The fix: Prompt Chaining</h3><p>It wasn’t about adding more instructions but breaking the tasks into sub-tasks. The process was split into two smaller prompts — first, let the model leverage the tools provided and reason about the code snippet, followed by extracting the desired structure from the previous inferences’ output.</p><p>The core idea here is to break down the problem into smaller problems, where each sub-problem can be solved by individual unique prompts tailored only for that sub-problem and the output generated by solving one problem is fed as input into the subsequent prompts following a chain pattern, also known as Prompt Chaining.</p><p>To address the issue discussed, the review pipeline was separated into two prompts. The first prompt is responsible for analyzing the code and identifying potential trust boundaries, external dependencies, and assumptions within the provided snippet. Rather than immediately producing a final structured answer, the model is encouraged to retrieve additional context when required, such as implementations of dependent functions. The second prompt then takes the reasoning and retrieved context from the first step and focuses on extracting the final Structured Output, including the identified vulnerability, its root cause, and the relevant code locations. By separating reasoning from Structured response, the system ensures that the model first gathers sufficient context before generating the final analysis.</p><h4>“The bug never moved. The model just wasn’t looking for it.”</h4><p>The vulnerability was present in the code from the start, and the system provided all the necessary tools to retrieve the missing context. However, the Structured Output constraint implicitly encouraged the model to finalize an answer early rather than investigate dependencies.</p><p>When the model retrieved the Storage.save() function implementation, the trust boundary violation became obvious. But without that retrieval step, the model assumed the abstraction was safe and proceeded with the review.</p><p>This revealed an important failure mode: the model wasn’t struggling with bug detection , but with deciding when to gather more information before answering. With prompt chaining, the first step focused only on understanding trust boundaries and hidden assumptions. That reasoning step reliably pulled the storage implementation, and once that context was visible, the vulnerability in the original function became clear. The same can be seen in the screenshot below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1023/0*dmR5KOrILcku-6zg.jpg" /></figure><p><em>Detection of a path traversal vulnerability caused by unsanitized user-controlled filename input passed to the storage layer.</em></p><h3>The takeaway</h3><p>Structured Output works extremely well for deterministic pipelines, but it also introduces a subtle bias. When the schema is enforced too early, the model optimizes for producing a valid response rather than reasoning towards the right one.</p><p>This isn’t a bug. It is the consequence of how constraints shape behavior. When the output format is fixed upfront, the model is implicitly encouraged to fill that structure immediately. Intermediate steps like checking assumptions or invoking tool call gets deprioritized. It is not because the model can’t do them, but the schema creates a bias towards finalization. The result is a system that looks correct on the surface. The responses are well-formatted, parseable, and pipeline-safe. But the reasoning that should precede the answers get quietly suppressed.</p><p>Prompt Chaining breaks this pattern by separating what the model needs to figure out from how the output should be structured. The first step reasons freely, invoking tool calls, retrieving context, identifying unknowns. The second step formats and consequently, by the time the schema is enforced, the model has already done the work.</p><p>Sometimes improving an LLM system isn’t about better prompts or bigger models, it’s about breaking down the tasks into simpler sub-tasks, through prompt chaining, ensuring both reliability and quality across repeated inferences.</p><p><em>Originally published at </em><a href="https://tallysolutions.com/technology/the-hidden-cost-of-structured-output-how-it-can-suppress-llm-reasoning/"><em>https://tallysolutions.com</em></a><em> on May 12, 2026.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=334abc534cb9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Enterprise AI Isn’t Just About Training Models | Tally Solutions]]></title>
            <link>https://medium.com/@techattally/enterprise-ai-isnt-just-about-training-models-tally-solutions-b89959e0ce00?source=rss-996d4db5c7dc------2</link>
            <guid isPermaLink="false">https://medium.com/p/b89959e0ce00</guid>
            <category><![CDATA[scalability]]></category>
            <category><![CDATA[workflow]]></category>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[research]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Tech @Tally]]></dc:creator>
            <pubDate>Tue, 28 Apr 2026 12:17:24 GMT</pubDate>
            <atom:updated>2026-05-11T09:11:48.510Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>30 second summary |</strong> Enterprise AI isn’t about training new models from scratch. Instead, businesses get better results by using powerful pre-trained models and connecting them to their own data. The key is combining retrieval (fetching the right company information in real time) with orchestration (adding workflows, rules, and checks). This approach makes AI faster to deploy, more accurate, cost-effective, and easier to maintain. In practice, successful enterprise AI is less about building intelligence and more about making existing intelligence reliable, safe, and useful in real-world workflows.</p><p>When people think of “AI in enterprises,” they often picture large teams of engineers training massive models from scratch, racks of GPUs crunching data, custom algorithms tuned for every problem, and researchers pushing the limits of intelligence. Training certainly has its place- it powers foundational models, enables breakthroughs in areas like speech recognition or medical imaging, and can be useful when a company’s data is so unique that off-the-shelf models won’t suffice.</p><p>But for many enterprises, this path is impractical. Training large models requires a large amount of data, significant infrastructure and a lot of time. Even then, the results aren’t guaranteed to automatically align with day-to-day business needs.</p><p>That’s why the real opportunity for enterprises lies in using powerful pre-trained models like GPT-4, Gemini, or Llama and focusing on how to connect them to company data. The challenge isn’t inventing new intelligence, but making sure existing intelligence delivers answers that are accurate, safe, and trustworthy.</p><p>In this article, we’ll bust the myth that enterprise AI is all about model training and show why information retrieval and orchestration are often the real heroes of practical AI adoption.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/0*MRa7GGw-xHv6AVCN.png" /></figure><h3>What is Enterprise AI?</h3><p>In a business context, enterprise AI means applying existing AI to solve real problems at scale. Instead of training new models, companies embed pre-trained ones into their workflows — powering things like HR chatbots that answer leave policy questions, finance assistants that help check expenses, or customer support systems that handle routine queries.</p><p>The emphasis is on reliability, compliance, and usability. Unlike research AI, which focuses on breakthroughs, enterprise AI succeeds when employees and customers can trust it in day-to-day use.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/0*Isk9QcTu7o2fby_o.png" /></figure><h3>The myth of model training</h3><p>The idea that AI always means training from scratch comes from the research world, where the goal is to build new systems. But enterprises rarely need that.</p><p>Modern models are already trained on vast amounts of information, giving them strong language and reasoning abilities out of the box. Rebuilding that inside a company would cost crores and require specialized teams.</p><p>When customization is needed, businesses adapt existing models. This might mean refining the way prompts are written, making small adjustments, or giving the model access to the right documents at the right time. It’s far more practical to extend what works than to reinvent the wheel.</p><h3>What enterprises could do</h3><p>Instead of training new models, companies wrap existing ones in systems that make them useful for day-to-day work. This usually involves two pillars:</p><h4>1. Retrieval-Augmented Generation (RAG)</h4><p>Rather than expecting the model to “remember everything,” RAG lets it look up information when needed. For example, if an employee asks a policy question, the system searches company documents for the most relevant section and passes it to the model. The model then uses that snippet to give a grounded answer. This way, responses are always tied to actual company data instead of just relying on the model’s memory.</p><h4>2. Orchestration (Workflows around the model)</h4><p>Orchestration is the glue that ties everything together. It’s not just the model producing data, it’s the workflow around it: interpreting the user’s question, retrieving the right data, generating a response, cleaning it up, and applying company rules before showing it. For instance, if a chatbot is connected to financial data, orchestration might add a rule that prevents it from showing numbers without double-checking them. This ensures answers are both useful and safe.</p><p>Together, retrieval and orchestration turn a general-purpose model into a trusted enterprise assistant.</p><h3>Why this approach matters</h3><p>This approach matters because it directly improves the things enterprises care about most:<br> 1. <strong>Faster deployment</strong>: Solutions can launch in weeks/months instead of years.<br> 2. <strong>Lower costs</strong>: Training requires massive infrastructure; orchestration and retrieval run on standard systems.<br> 3. <strong>Always up to date</strong>: Retrieval ensures the AI can use new documents or policies immediately.<br> 4. <strong>Trust and safety</strong>: Rules and checks prevent errors and misuse.<br> 5. <strong>Scalability</strong>: Companies can swap models, add data, or integrate tools without starting over.</p><h3>A simple example</h3><p>Imagine you’re building a chatbot for an HR team. You don’t train a new model. Instead, you connect an existing one to the HR policy documents.</p><p>Here’s how it works:<br> 1. HR policies and FAQs are split into small, searchable sections.<br> 2. When someone asks, “How many days of parental leave do we get?”, the system finds the most relevant policy. (Retrieval)<br> 3. That piece of text is added to the original question before sending it to the model. (Augmentation)<br> 4. The model reads both and generates a response:<br> “According to section 5.2 of the HR policy, employees are entitled to 30 days of parental leave.” (Generation)<br> 5. Orchestration ensures guardrails double-check the response before showing it, for example, confirming the number is from an official source.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/0*hukCpVjghyCnWBnZ.png" /></figure><p>The model isn’t memorizing policies, it’s reading them at runtime. This makes the approach accurate, scalable, and easy to maintain. When HR updates a document, the AI automatically uses the latest version.</p><p>Enterprise AI isn’t necessarily about building new models, it’s about making the most of the ones we already have. Pre-trained models provide the intelligence, while retrieval and orchestration provide the structure that makes them reliable for business use.</p><p>By focusing on these layers, enterprises can:<br> * Launch solutions quicker.<br> * Keep answers accurate and current.<br> * Ensure trust and compliance.<br> * Scale easily as technology evolves.</p><p>The real measure of success in enterprise AI won’t be who trains the largest models, but who integrates existing ones into workflows that are accurate, safe, and scalable.</p><p><em>Originally published at </em><a href="https://tallysolutions.com/technology/enterprise-ai-isnt-just-about-training-models/"><em>https://tallysolutions.com</em></a><em> on April 28, 2026.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b89959e0ce00" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What Makes Tally’s Automation Testing Special]]></title>
            <link>https://medium.com/@techattally/what-makes-tallys-automation-testing-special-3066e2077cf5?source=rss-996d4db5c7dc------2</link>
            <guid isPermaLink="false">https://medium.com/p/3066e2077cf5</guid>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[automation]]></category>
            <category><![CDATA[tally]]></category>
            <category><![CDATA[testing]]></category>
            <dc:creator><![CDATA[Tech @Tally]]></dc:creator>
            <pubDate>Fri, 17 Apr 2026 09:12:57 GMT</pubDate>
            <atom:updated>2026-05-04T06:50:25.199Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>30 second summary |</strong> TallyRobot is Tally’s purpose-built, AI-powered testing framework designed specifically for Tally Prime’s unique architecture. Unlike generic tools, it uses a custom scripting format and a Map File + Keyword-driven approach to create clean, maintainable, and context-aware automation. It combines functional, performance, and visual validation within a single framework, while scaling seamlessly through a Jenkins-based infrastructure that can execute millions of tests daily. Powered by FailSense AI, it also simplifies result analysis by intelligently grouping failures and highlighting real issues — helping teams deliver high-quality software faster and more reliably.</p><p>Software delivery today demands both speed and quality. Traditional automation tools, though effective in some areas, often stumble when applied to products with unique architectures like Tally. That’s why we created <strong>TallyRobot</strong> — a purpose-built automation testing framework that understands Tally Prime’s DNA.</p><p>It’s not just another automation tool. TallyRobot acts like an <strong>AI-powered testing agent</strong>, blending precision, adaptability, and intelligence into every test run.</p><h3>Why TallyRobot is different</h3><h3>1. Custom scripting format</h3><p>Instead of generic scripting languages, TallyRobot uses a <strong>purpose-built scripting format </strong>for <strong>TDL-based UIs </strong>. This ensures automation is tightly aligned with Tally’s interface behavior, making scripts:</p><ul><li>Cleaner</li><li>Faster to write</li><li>Easier to maintain</li></ul><h3>2. Map File + Keyword driven approach</h3><h3>TallyRobot uses a <strong>Map File + Keyword driven</strong> structure that enables context-aware navigation.</h3><ul><li><strong>Centralized UI map:</strong> Scripts don’t hardcode steps but reference a shared map file.</li><li><strong>Context verification:</strong> Before executing any keystroke, TallyRobot confirms it’s in the correct UI context.</li><li><strong>Effortless maintenance:</strong> If the UI changes, only the map file needs updating — all scripts adapt automatically.</li></ul><h3>3. One script, Multiple capabilities</h3><h3>TallyRobot eliminates duplication by allowing a <strong>single script</strong> to validate multiple scenarios, including:</h3><ul><li><strong>Remote functionality testing</strong></li><li><strong>Multi-lingual testing</strong></li></ul><p>This saves time, ensures consistency, and expands test coverage without additional scripting overhead.</p><h3>4. Built-in performance testing</h3><p>TallyRobot isn’t just about functional testing — it also supports <strong>dedicated performance testing</strong> without needing a separate tool.</p><p>The scripter adds <strong>perfstart</strong> and <strong>perfend</strong> commands in the script.</p><p>Between these markers, TallyRobot captures performance metrics such as:</p><ul><li>CPU usage</li><li>I/O operations</li><li>Execution duration</li><li>CPU time</li><li>Memory usage</li></ul><p>This approach makes performance testing lightweight, integrated, and accessible within the same framework.</p><h3>5. Human-like validation with screen capture</h3><p>TallyRobot also brings a unique validation capability that mimics how a human tester works:</p><ul><li><strong>Tally screen capture feature:</strong> During execution, TallyRobot captures actual Tally product screens.</li><li><strong>Visual validation:</strong> These captures are compared against expected screens to confirm correctness.</li><li><strong>Closer to human testing:</strong> This approach ensures validation isn’t limited to backend checks — it reflects how a real tester would verify the UI.</li></ul><h3>Seamless integration with scalable test deployment</h3><p>TallyRobot integrates with a <strong>scalable Jenkins-based deployment model</strong> for massive test execution.</p><p>Key features of this model:</p><p><strong>1. Dev &amp; production server setup</strong> — to streamline the automation lifecycle (Plan → Define → Build → Test → Deploy).</p><p><strong>2. Centralized repository (AWS S3)</strong> — for versioned test data and results, accessible from anywhere.</p><p><strong>3. Scalable server orchestration</strong> — auto-deploys multiple Jenkins servers &amp; nodes for heavy workloads.</p><p><strong>Intelligent infra deployment</strong> –</p><ul><li>Auto-scales test infra (AWS/OCI) based on queue size.</li><li>Chooses alternate instance types on creation failure.</li><li>Self-heals connections with Jenkins servers.</li><li>Auto-terminates idle test infra to save cost.</li><li>System health monitor detects stuck scripts (CPU/memory/disk stalls) and aborts them, so the next script in the queue can continue without delay.</li></ul><p><strong>4. Result tags</strong> — store project-wise results of the same scripts against multiple baselines.</p><p><strong>5. AWS DynamoDB</strong> — to store test suite result metadata.</p><p><strong>6. AWS Lambda</strong> — for baseline lifecycle (creation, viewer, update, rollback).</p><p><strong>Impact: </strong>With this scalable model, Tally validates 1.3 million test results in a single day.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BPK5CLXMIZj3RyhglefnBQ.png" /></figure><h3>Smart test results with FailSense AI</h3><p>Bringing it all together is theTest Result Framework, enabled by our in-house FailSense AI.</p><ul><li><strong>Failure patterning:</strong> Groups failures by common root causes.</li><li><strong>Self-learning:</strong> Learns from past results and validates automatically.</li><li><strong>Insights at a glance:</strong> Highlights recurring failures, reopened issues, false positives, and genuinely new bugs.</li><li><strong>Visual analysis:</strong> Provides diff images, OCR checks, and HTML reports with side-by-side screenshots.</li></ul><p>This means testers spend less time triaging noise and more time focusing on real product issues.</p><p>TallyRobot is not just automation — it’s a complete intelligent testing ecosystem. From context-aware scripting to built-in performance capture, from scalable deployment to AI-powered failure analysis, Tally has built a framework that ensures every release is fast, reliable, and customer-ready.</p><p>With 1.3 million validations a day, Tally’s automation testing is proof that quality and speed can go hand in hand — if you build the right system.</p><p><em>Originally published at </em><a href="https://tallysolutions.com/technology/what-makes-tallys-automation-testing-special/"><em>https://tallysolutions.com</em></a><em> on April 17, 2026.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3066e2077cf5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI, Architecture, and the Human in the Loop]]></title>
            <link>https://medium.com/@techattally/ai-architecture-and-the-human-in-the-loop-c0fa41c4db8b?source=rss-996d4db5c7dc------2</link>
            <guid isPermaLink="false">https://medium.com/p/c0fa41c4db8b</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[architecture]]></category>
            <category><![CDATA[prompt-engineering]]></category>
            <dc:creator><![CDATA[Tech @Tally]]></dc:creator>
            <pubDate>Thu, 02 Apr 2026 07:06:56 GMT</pubDate>
            <atom:updated>2026-04-20T08:46:15.668Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>30 second summary |</strong> AI acts like a mirror of your thinking — it amplifies clarity if your mental model is strong, and confusion if it’s weak. It excels at execution and repetitive tasks (heavy lifting) but should never be trusted with architecture, intent, or long-term decisions (high art). As systems evolve from prompting to agentic AI, the role of engineers shifts from doing to defining goals, constraints, and boundaries. While AI accelerates development, it also accelerates technical debt and architectural entropy if not guided properly. The real advantage in the AI era is not speed, but deep thinking, judgment, and ownership, which remain uniquely human.</p><h3>Part 1: The Mirror Effect — Why AI is only as good as your mental model</h3><p>I’ve spent close to 25 years building, using, and watching software tools evolve — from text editors to IDEs, debuggers to profilers, build systems to automation pipelines. Over the last few years, <strong><em>that evolution has taken a sharp turn </em></strong>, with chat-based, assistive, and now agentic AI tools becoming part of everyday engineering work.</p><p>After living with these tools, deeply, not casually; one uncomfortable truth has become clear to me.</p><p><strong><em>“AI hasn’t eliminated technical debt caused by weak architectural decisions.</em></strong> <br><strong><em>It has accelerated how quickly that debt accumulates”</em></strong></p><p>This isn’t a cynical statement. It’s an architectural one.</p><h3>AI is not a magic box</h3><p>AI is often spoken about as if it were a new form of intelligence. It isn’t. AI does not <strong><em>understand </em></strong>systems, <strong><em>intend </em></strong>outcomes, or <strong><em>care </em></strong>whether a design will survive its second year in production. What it does exceptionally well is <em>pattern completion </em>, <em>language synthesis </em>, and, most importantly, the <strong><em>plausible expansion </em></strong>of incomplete intent.</p><p>That last point is critical. AI does not create intent. It amplifies whatever intent already exists.</p><h3>The Mirror Effect</h3><p>I’ve come to think of modern AI as a high‑fidelity mirror. If your mental model of the system is clear, grounded in first principles, and shaped by experience, the AI reflects that clarity back to you — often at breathtaking speed. If your mental model is fuzzy, fragmented, or internally inconsistent, the AI reflects that instead — confidently, fluently, and at scale.</p><p>This explains a phenomenon many teams are observing but struggling to articulate:</p><p><strong>Two people can use the same AI tool and walk away with wildly different outcomes. The tool didn’t change. The person did.</strong></p><p>This is a very important aspect to inspect — while using these tools. The ‘Context’, deep structural, first principle based, becomes very important for the person. AI’s have extensive knowledge of the world of technological domains, solutions and patterns. But they have zero clue of what the person intends to do. Or what the design behind the code pointed to it. — Designs are always something invisible parts of a code. They are the expert compression of years of experience, emotional intelligence, pains, failed approaches, all compressed to result in principles, patterns, constraints.</p><h3>The master craftsman and the apprentice</h3><p>This metaphor is personal for me. After 25 years in engineering, I still see myself as an apprentice — not because mastery is unreachable, but because craftsmanship demands lifelong pursuit, not arrival. AI, in this framing, is the world’s fastest apprentice: tireless, well‑read, and capable of instant execution.</p><p>But like every apprentice in history, it still depends on a master craftsman — someone who understands the blueprint, knows where the load‑bearing walls are, can tell the difference between elegance and coincidence, and has lived through failure modes rather than just reading about them. Without that mastery, the apprentice doesn’t become autonomous. It becomes dangerous.</p><h3>Knowing where the Sun is</h3><p>Every non‑trivial system has a <strong><em>center of gravity </em></strong>— a core idea, an organizing principle, a ‘Sun’ around which everything else orbits. Good architects know where that Sun is. They may disagree on implementation details, but they are rarely confused about what the system is fundamentally about, what must never be compromised, and which trade‑offs are acceptable.</p><p>If you don’t know where the Sun is in your system, AI will not find it for you. It will simply give you a darker sky — filled with more stars.</p><p><strong>The real question, then, is not ‘What can AI do?’ but ‘What do you bring to the table that AI can amplify?’</strong></p><p>Depth matters again. Architecture matters again. First principles matter again. And paradoxically, that is the most optimistic thing about the AI era.</p><h3>Part 2: Heavy lifting vs high art — What to delegate, what to guard religiously</h3><p>One of the most dangerous mistakes we can make with AI is to use it everywhere equally. Not all work is equal. Not all thinking is equal. And not all delegation is harmless.</p><p>After working deeply with AI across code, architecture notes, documentation, and reviews, one principle has become clear to me: if you’re using human cognition for boilerplate, you’re wasting time. If you’re outsourcing reasoning, you’re accumulating debt. The art is knowing the difference.</p><h3>Heavy lifting: Where AI Is unambiguously good</h3><p>There is a large class of work where AI is not just useful — it is objectively better than humans. This includes <strong><em>repetitive, mechanical, well‑bounded tasks </em></strong>with <strong><em>low ambiguity </em></strong>and <strong><em>high effort </em></strong>: boilerplate code, migrations, unit tests once intent is clear, refactoring for consistency, and converting scattered notes into structured documents.</p><p>This is heavy lifting. Using senior engineering time for this kind of work is not craftsmanship; it is inefficiency. Delegating it to AI is not laziness — it is focus.</p><h3>High art: Where AI must never be left alone</h3><p>There is another class of work that looks similar on the surface but is fundamentally different underneath. This work involves <strong><em>intent, judgment, first principles, trade‑offs, and long‑term consequences </em></strong>: choosing architectures, defining boundaries, establishing invariants, and <strong><em>owning </em></strong>the ‘why’ behind decisions.</p><p>This is high art. Here, AI can assist, but it must never decide. It optimizes for plausibility, not truth, and has no instinct for maintenance, entropy, or future pain. Outsource this, and you don’t get leverage — you get a very confident form of technical debt.</p><h3>The fast car analogy</h3><p>I often visualize AI in the hands of an apprentice like an F1 car used to learn driving skills on a racetrack. Speed is intoxicating — it feels powerful and progressive. But the car does not choose the destination, understand the terrain, or know who is holding the steering wheel. <strong><em>It simply accelerates faster </em></strong>.</p><p>Garbage in, garbage out. If you point it in the wrong direction, it won’t correct you — it will just help you arrive there faster.</p><p><strong><em>AI gives us feedback loops faster, which is a gift, but only if someone is still holding the map.</em></strong></p><h3>Why this matters for architects</h3><p>As AI takes over more surface‑level work, something interesting happens: <strong><em>the center of gravity shifts </em></strong>. What becomes scarce is not code, but clarity. Architects benefit disproportionately because they can <strong><em>separate signals from noise </em></strong>, recognize bad abstractions early, and understand the cost of being ‘almost right’ or ‘chasing perfection’. AI doesn’t replace this. It rewards it.</p><h3>Part 3: From prompting to agency — When tools start feeling like teams</h3><p>For a long time, our interaction with AI looked deceptively simple: you typed a prompt, the system responded, and you edited the output. But as <strong><em>systems began holding context, executing sequences, consuming feedback, and revisiting decisions </em></strong>, we crossed an invisible line — from prompting tools to directing agents.</p><p>Prompting is transactional and local. Agency is systemic. You define goals, constraints, boundaries, and evaluation criteria, and the system figures out how to move within that space. This shift quietly but fundamentally changes the architect’s role.</p><h3>Prompting is local. Agency is systemic.</h3><p>Prompting is transactional: You ask a question. You get an answer. The interaction ends.</p><p>Agency is systemic: You define goals, constraints, boundaries, evaluation criteria. And the system figures out to move within that space.</p><p>This difference matters.</p><p>A prompt is about what to say next. Agency is about what to do next</p><h3>Why agentic systems feel uncomfortable (At first)</h3><p>When people first encounter agentic AI, the reaction is often uneasy.</p><ul><li>“It’s doing too much.”</li><li>“I don’t know what it’s thinking.”</li><li><strong>“I’ve lost control.”</strong></li></ul><p>This discomfort is not accidental. Agentic systems expose something we often avoid confronting:</p><p><strong>“Control does not come from micromanagement. It comes from clear intent, first principles, direction and strong boundaries”</strong></p><p>When intent is vague, agency feels dangerous. When boundaries are weak, autonomy feels chaotic.</p><p>This is not an AI problem. It’s an architectural one.</p><h3>The architect as orchestrator</h3><p>Architects were always doing deep work — exploring diversity, testing patterns, running proofs, and converging slowly on decisions that would shape systems for years.</p><p>Agentic systems don’t change work. They accelerate this. They change <strong>where the effort goes </strong>.</p><p>Instead of personally traversing the entire solution space, architects now explicitly define: the intent, constraints, invariants, and what success must look like.</p><p>The system explores faster; the architect decides more deliberately. That is the shift.</p><p>Autonomy without this orchestration is not intelligence. It is fast chaos.</p><p>Experience matters here because knowing <em>what to explore </em>, <em>what to forbid </em>, and <em>when to stop </em>is not theoretical knowledge. It is earned judgment.</p><h3>Part 4: AI and architectural entropy — Why speed without structure destroys systems</h3><p>Most systems fail not because of lack of intelligence, but because of entropy — small, compounding decisions whose cost is deferred.</p><p>AI does not change this law. It accelerates it.</p><h3>Creation is easy. Maintenance is brutal.</h3><p>AI is exceptionally good at creation. It can generate code, assemble architectures that look coherent, and produce systems that run and scale — initially. But maintenance is where truth emerges.</p><p>Maintenance is not about changing code; it is about preserving invisible design truths: intent, invariants, and expert compression that never fully lives in documentation. AI can read design rationales, but it cannot grasp their spirit.</p><h3>The most dangerous phrase: ‘AI built it’</h3><p>This is a dangerous phrase. AI builds nothing on its own. Every system it produces reflects the constraints it was given, the intent it inferred, and the architecture it was allowed to assume.</p><h3>A common misunderstanding</h3><p>We often cite examples where AI has “built” impressive things — for instance, a C compiler capable of compiling a large system like Linux.</p><p>But let’s be honest about what really happened.</p><p>The C language is decades old. Its semantics are deeply understood. Its design trade-offs have been explored, debated, and paid for by generations of human experts.</p><p>Enormous expert compression, decades of pain and operational learning, mature, battletested compiler codebases, a rich ecosystem of prior art, are all already existed.</p><p>AI did not invent that knowledge. It understood it and stood on top of it.</p><h3>The real question</h3><p>is not whether AI can generate a compiler.</p><p>The real question is this:</p><ul><li><em>Would you trust that compiler for your product that millions of customers are already using?</em></li><li><em>Would you bet your system’s correctness, performance, and longterm maintainability on it?</em></li><li><em>What was the architectural intent of building that C compiler? To replace any existing ones? Or to showcase the power of the tool that created it?</em></li></ul><p>That hesitation — the pause before answering — is the point.</p><p>Architecture is not about producing artefacts. It is about <strong>owning consequences </strong>.</p><p>When systems collapse under maintenance load, the cause is almost never the tool. It is missing architectural ownership, unclear invariants, unexamined assumptions, and deferred decisions disguised as flexibility.</p><p><strong><em>“AI can reproduce the letter of past success. It cannot yet carry the responsibility of the future”</em></strong></p><p>AI can help us build faster than ever before. It can also help us decay faster than ever before. Both outcomes are possible. Both will happen.</p><p>The difference will not be the model. It will not be the tooling. It will not even be talent.</p><p>It will be <strong>architecture </strong>— quiet, opinionated, and deeply human.</p><h3>Part 5: The human edge — Growing 10× better (Not faster) in the age of AI</h3><p>The most important question is no longer: <em>What can AI do?</em></p><p>The real question is: <strong>What should humans become in an AI accelerated world?</strong></p><p><strong>“Paradoxically, as AI becomes more autonomous, the value of deeply human qualities increases — not decreases”</strong></p><p>Its <strong><em>creativity, judgment, taste, and responsibility </em></strong>. The opportunity is not to move faster, but to grow better — in depth, clarity, and accountability.</p><h3>A note to students and new‑age engineers</h3><p>Do not compete with AI on execution — it will always be faster. Instead, invest in mental models, chase responsibility early, and learn to own systems over time. Use AI as a multiplier, never as a substitute for understanding. That responsibility — and the growth that comes with it — remains deeply human.</p><h3>A satisfying, durable career</h3><p>A career built on shallow wins, tool or programming language ( <em>will they exists? </em>) hopping, output without ownership; Looks good early — and collapses later.</p><p>A career built on judgment, taste, responsibility; compounds quietly — and lasts.</p><h3>Closing note</h3><p>Every generation of engineers is shaped by their tools. But remembered engineers are shaped by something else:</p><ul><li>The systems they stood behind</li><li>The consequences they owned</li><li>The judgment they exercised when it mattered most</li></ul><p><strong><em>AI will change how we build. It will not change who must answer when things go wrong.</em></strong></p><p>That responsibility — and the growth that comes with it — remains deeply, stubbornly human.</p><p><em>Originally published at </em><a href="https://tallysolutions.com/technology/ai-architecture-and-the-human-in-the-loop/"><em>https://tallysolutions.com</em></a><em> on April 2, 2026.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c0fa41c4db8b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Revolutionizing Development: How AI-Powered Code Generation Built a Marketplace in 24 Hours]]></title>
            <link>https://medium.com/@techattally/revolutionizing-development-how-ai-powered-code-generation-built-a-marketplace-in-24-hours-481626cb8fba?source=rss-996d4db5c7dc------2</link>
            <guid isPermaLink="false">https://medium.com/p/481626cb8fba</guid>
            <category><![CDATA[innovation]]></category>
            <category><![CDATA[testing]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Tech @Tally]]></dc:creator>
            <pubDate>Mon, 16 Feb 2026 10:21:31 GMT</pubDate>
            <atom:updated>2026-02-23T10:21:32.665Z</atom:updated>
            <content:encoded><![CDATA[<p>Imagine building a fully functional, production-ready marketplace platform-not in weeks, but in just 24 hours. Welcome to the new era of software development, where AI-powered code generation is rewriting the rules. Gone are the days of lengthy requirements gathering, endless coding sprints, and tedious manual testing. Today, artificial intelligence is transforming traditional workflows, delivering robust systems with comprehensive testing and enterprise-grade quality at lightning speed.</p><p>We experienced this paradigm shift when we built a comprehensive marketplace system using AI for Tally, enabling unprecedented testing coverage and quality assurance.</p><p>As this was a standalone system, there was significant effort to integrate it with our CRM/PRM systems, which have user management, payment gateways, deployment modules, to name a few. But with the help of AI, we had the framework ready, and with that, 60–70% of the work was complete. This article takes you behind the scenes of a real-world project that harnessed AI to deliver a sophisticated marketplace-complete with multi-role user management, GST compliance, secure payments, and real-time analytics-in less than a day.</p><h3>The challenge: Enterprise marketplace platform</h3><p>We set out to build a sophisticated add-on marketplace with complex requirements:</p><ul><li>Multi-role user management (Customers, partners, administrators)</li><li>Comprehensive addon upload and validation workflows</li><li>Product ownership verification system</li><li>Indian market localization with GST compliance</li><li>Real-time analytics and monitoring</li><li>Secure payment processing</li></ul><p>Traditional estimation: <strong>6–8 weeks</strong> <br>AI assisted delivery: <strong>Under 24 hours</strong></p><h3>AI development methodology</h3><h3>Structured requirements engineering</h3><p>AI introduced a systematic approach to requirements gathering using the Easy Approach to Requirements Syntax (EARS):</p><p>WHEN a partner uploads an addon THEN the system SHALL validate the addon package format and metadata</p><p>WHEN an addon is submitted THEN the system SHALL perform security scanning and compatibility checks</p><p>IF an addon fails validation THEN the system SHALL provide detailed error messages</p><p>This precision eliminated ambiguity and ensured complete requirement coverage from day one.</p><h3>Architecture-first development</h3><p>AI immediately established a robust technical foundation:</p><p><strong>Frontend architecture:</strong></p><ul><li>React 18 with TypeScript for type safety</li><li>Material-UI design system for consistency</li><li>Redux Toolkit for predictable state management</li><li>Responsive design with mobile compatibility</li></ul><p><strong>Backend architecture:</strong></p><ul><li>FastAPI for high-performance Python backend</li><li>PostgreSQL with proper indexing and relationships</li><li>JWT authentication with role-based access control</li><li>Comprehensive error handling and logging</li></ul><p><strong>High level architecture:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nbhlKUYC3sG0bXa2.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*K8R2kUAr2dv3orVe.png" /></figure><p><em>Homepage showing featured addons with Indian pricing and advanced filtering</em></p><h3>The 24-hour development journey</h3><h3>Phase 1: Foundation and core infrastructure (Hours 0–6)</h3><p>AI generated the complete project structure and core infrastructure:</p><pre># Database schema design by AI CREATE TABLE users ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), email VARCHAR(255) UNIQUE NOT NULL, user_type VARCHAR(20) NOT NULL CHECK (user_type IN (&#39;customer&#39;, &#39;partner&#39;, &#39;admin&#39;)), owned_products TEXT[], product_flavors TEXT[], created_at TIMESTAMP DEFAULT NOW() );</pre><p>AI automatically implemented:</p><ul><li>Authentication system with secure password hashing</li><li>Database models with proper relationships</li><li>API endpoints with validation</li><li>Basic UI components</li><li>Initial test suites with 85%+ coverage</li></ul><h3>Phase 2: Core features with integrated testing (Hours 6–12)</h3><p><strong>Customer experience implementation</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*TOI-SlZJFtxXlYFS.png" /></figure><p><em>Shopping cart showing Indian pricing with 18% GST calculation</em></p><p><strong>AI-generated test examples:</strong></p><pre>// Comprehensive component testing describe(&#39;AddonCard Component&#39;, () =&gt; { it(&#39;should display addon information correctly&#39;, () =&gt; { const mockAddon = { id: &#39;test-addon&#39;, name: &#39;Test Addon&#39;, price: 2490, rating: 4.5, downloadCount: 150 }; render(); expect(screen.getByText(&#39;Test Addon&#39;)).toBeInTheDocument(); expect(screen.getByText(&#39;₹2,490&#39;)).toBeInTheDocument(); expect(screen.getByText(&#39;4.5 ★&#39;)).toBeInTheDocument(); expect(screen.getByRole(&#39;button&#39;, { name: /add to cart/i })).toBeInTheDocument(); }); });</pre><h3>Phase 3: Advanced capabilities with comprehensive testing (Hours 12–18)</h3><p><strong>Admin interface development:</strong></p><ul><li>System monitoring dashboard</li><li>User management with advanced filtering</li><li>Content moderation workflows</li><li>Real-time analytics</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*OW6_C5eUP2bYeCO_.png" /></figure><p><em>Admin dashboard showing system health and pending approvals</em></p><p><strong>Backend integration testing:</strong></p><pre># AI-generated API integration tests class TestAddonEndpoints: def test_create_addon_as_partner(self, authenticated_partner_client): &quot;&quot;&quot;Test addon creation by authenticated partner&quot;&quot;&quot; addon_data = { &quot;name&quot;: &quot;Test Addon&quot;, &quot;description&quot;: &quot;A test addon&quot;, &quot;price&quot;: 99.99, &quot;required_product&quot;: &quot;base_product&quot;, &quot;product_flavors&quot;: [&quot;silver&quot;, &quot;gold&quot;] } response = authenticated_partner_client.post( &quot;/api/v1/addons&quot;, json=addon_data, files={&quot;package_file&quot;: (&quot;test.zip&quot;, b&quot;fake content&quot;, &quot;application/zip&quot;)} ) assert response.status_code == 201 assert response.json()[&quot;name&quot;] == addon_data[&quot;name&quot;] assert response.json()[&quot;status&quot;] == &quot;pending&quot;</pre><h3>Phase 4: Polish, integration, and test optimization (Hours 18–24)</h3><ul><li>Indian market localization (₹ pricing, UPI payments)</li><li>Performance optimization</li><li>Error handling refinement</li><li>Documentation generation</li><li>Comprehensive test execution and coverage analysis</li></ul><h3>Comprehensive API documentation</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*oBi3UjXpFuYlboV5.png" /></figure><p>With the help of these, we can plug and play the APIs from our systems.</p><h3>Revolutionizing testing: AI-powered quality assurance</h3><h3>Automated test generation at scale</h3><p>AI transformed our testing approach by automatically generating comprehensive test suites alongside feature development. It understood the business logic and generated meaningful tests that covered both happy paths and edge cases.</p><p><strong>Testing pyramid implementation:</strong></p><p>1. <strong>Unit tests</strong> — Component and service testing</p><pre># AI-generated authentication tests def test_user_registration_success(): &quot;&quot;&quot;Test successful user registration&quot;&quot;&quot; user_data = { &quot;email&quot;: &quot;test@example.com&quot;, &quot;password&quot;: &quot;SecurePassword123!&quot;, &quot;user_type&quot;: &quot;customer&quot;, &quot;first_name&quot;: &quot;John&quot;, &quot;last_name&quot;: &quot;Doe&quot; } response = client.post(&quot;/api/v1/users/register&quot;, json=user_data) assert response.status_code == 201 assert response.json()[&quot;email&quot;] == user_data[&quot;email&quot;] assert &quot;password&quot; not in response.json()</pre><p>2. <strong>Integration tests </strong>— API endpoint and database testing</p><pre>def test_commission_calculation_edge_cases(): &quot;&quot;&quot;Test commission calculation with various edge cases&quot;&quot;&quot; # Test zero sale amount result = commission_service.calculate_commission(&quot;addon_1&quot;, &quot;partner_1&quot;, Decimal(&#39;0&#39;)) assert result[&#39;commission_amount&#39;] == Decimal(&#39;0&#39;) # Test tiered commission with threshold boundaries partner_stats = {&#39;total_sales&#39;: 99, &#39;total_revenue&#39;: Decimal(&#39;9999.99&#39;)} result = commission_service.calculate_commission_with_stats( &quot;addon_1&quot;, &quot;partner_1&quot;, Decimal(&#39;100&#39;), partner_stats ) assert result[&#39;applied_rule&#39;] == &#39;Standard Rate&#39;</pre><p>3. <strong>End-to-end tests</strong> — Complete user workflow testing</p><pre>// AI-generated E2E test for purchase workflow describe(&#39;Complete Purchase Workflow&#39;, () =&gt; { it(&#39;should allow user to browse, add to cart, and purchase addon&#39;, () =&gt; { // Login as customer cy.visit(&#39;/login&#39;); cy.get(&#39;[data-testid=&quot;email-input&quot;]&#39;).type(&#39;customer@example.com&#39;); cy.get(&#39;[data-testid=&quot;password-input&quot;]&#39;).type(&#39;password123&#39;); cy.get(&#39;[data-testid=&quot;login-button&quot;]&#39;).click(); // Browse and purchase cy.visit(&#39;/marketplace&#39;); cy.get(&#39;[data-testid=&quot;addon-card&quot;]&#39;).first().click(); cy.get(&#39;[data-testid=&quot;add-to-cart-button&quot;]&#39;).click(); cy.get(&#39;[data-testid=&quot;checkout-button&quot;]&#39;).click(); // Verify success cy.get(&#39;[data-testid=&quot;purchase-success&quot;]&#39;).should(&#39;be.visible&#39;); }); });</pre><h3>Intelligent test execution and coverage analysis</h3><p>AI didn’t just generate tests-it executed them and provided real-time coverage analysis:</p><p><strong>Coverage metrics achieved:</strong></p><p><strong>Backend API coverage:</strong> 92%</p><p><strong>Frontend component coverage:</strong> 88%</p><p><strong>Business logic coverage:</strong> 95%</p><p><strong>Error scenario coverage:</strong> 85%</p><pre># AI-generated test coverage report &quot;&quot;&quot; Test Coverage Report =================== Backend Services: ✅ User Authentication: 94% coverage ✅ Addon Management: 91% coverage ✅ Payment Processing: 89% coverage ✅ Commission Calculation: 96% coverage Frontend Components: ✅ Addon Marketplace: 87% coverage ✅ Shopping Cart: 90% coverage ✅ Partner Dashboard: 85% coverage ✅ Admin Interface: 86% coverage &quot;&quot;&quot;</pre><h3>Edge case and error scenario testing</h3><p>AI excelled at identifying and testing edge cases that human developers might overlook:</p><pre># AI-generated security testing def test_file_upload_security(): &quot;&quot;&quot;Test that malicious file uploads are blocked&quot;&quot;&quot; # Attempt to upload executable file disguised as ZIP malicious_file = (&quot;malicious.exe&quot;, b&quot;malicious content&quot;, &quot;application/zip&quot;) response = authenticated_client.post( &quot;/api/v1/addons/upload&quot;, files={&quot;package_file&quot;: malicious_file} ) # AI test caught security vulnerability in initial implementation assert response.status_code == 400 assert &quot;invalid file type&quot; in response.json()[&quot;detail&quot;]</pre><h3>Key features delivered with comprehensive testing</h3><h3>Comprehensive addon management</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*VIj9iqYjxqGMDhMb.png" /></figure><p><em>4-step addon upload process with documentation requirements</em></p><p>The system enforces rigorous documentation standards with full test coverage:</p><ul><li>Product schema documentation</li><li>API documentation</li><li>Security documentation</li><li>Help and FAQ documents</li><li>Integration guidelines</li></ul><h3>Product validation system with test coverage</h3><pre>// AI-generated validation logic with tests const validateProductOwnership = async (addonId: string) =&gt; { const validation = await apiClient.get(`/users/validate-product/${addonId}`); return { canPurchase: validation.hasProduct &amp;&amp; validation.hasFlavor, requirements: validation.requiredProduct, userProducts: validation.userProducts }; }; // Corresponding AI-generated tests describe(&#39;Product Validation&#39;, () =&gt; { it(&#39;should allow purchase when user has required product and flavor&#39;, async () =&gt; { const validation = await validateProductOwnership(&#39;addon_123&#39;); expect(validation.canPurchase).toBe(true); }); it(&#39;should block purchase when user lacks required flavor&#39;, async () =&gt; { const validation = await validateProductOwnership(&#39;addon_456&#39;); expect(validation.canPurchase).toBe(false); expect(validation.requirements).toContain(&#39;Gold&#39;); }); });</pre><h3>Indian market localization with validation</h3><ul><li><strong>Currency</strong>: Complete ₹ formatting with Indian number system</li><li><strong>Tax compliance</strong>: 18% GST calculation and display</li><li><strong>Payment methods</strong>: UPI, Net Banking, Cards, Digital Wallets</li><li><strong>Address format</strong>: Indian states, PIN codes, and cities</li></ul><p><strong>Localization testing:</strong></p><pre>def test_indian_currency_formatting(): &quot;&quot;&quot;Test Indian currency formatting and GST calculation&quot;&quot;&quot; subtotal = Decimal(&#39;1000.00&#39;) gst_rate = Decimal(&#39;0.18&#39;) gst_amount = subtotal * gst_rate total = subtotal + gst_amount assert gst_amount == Decimal(&#39;180.00&#39;) assert total == Decimal(&#39;1180.00&#39;) # Test formatting formatted_total = format_currency(total, &#39;INR&#39;, locale=&#39;en_IN&#39;) assert formatted_total == &#39;₹1,180.00&#39;</pre><h3>Continuous testing integration</h3><p>AI integrated testing into our development workflow seamlessly:</p><h3>Pre-commit hooks and CI/CD pipeline</h3><pre># AI-configured GitHub Actions workflow name: Marketplace CI/CD on: [push, pull_request] jobs: test: runs-on: ubuntu-latest steps: - name: Run backend tests run: | pytest tests/unit/ --cov=app --cov-report=xml --cov-report=html - name: Run frontend tests run: | npm test -- --coverage --watchAll=false --ci - name: Upload coverage reports uses: codecov/codecov-action@v3 with: file: ./coverage.xml</pre><h3>The testing transformation: From afterthought to foundation</h3><h3>Traditional testing challenges overcome</h3><ul><li>Time-consuming: Manual test creation reduced from 30–40% to 5% of development time</li><li>Incomplete coverage: AI identified and tested critical edge cases automatically</li><li>Maintenance burden: Tests self-updated with code changes</li><li>Late feedback: Issues caught within minutes of introduction</li></ul><h3>AI-powered testing advantages realized</h3><ul><li>Instant test generation: 200+ tests created automatically</li><li>Comprehensive coverage: 90%+ coverage across all critical paths</li><li>Security validation: Automated security testing caught 3 critical vulnerabilities</li><li>Performance assurance: Load testing for 100+ concurrent users</li></ul><h3>Testing impact and results</h3><h3>Quality metrics achieved</h3><p><strong>Test Effectiveness:</strong></p><ul><li><strong>Bug detection rate:</strong> 94% of critical issues caught by AI-generated tests</li><li><strong>Regression prevention:</strong> 100% of existing functionality protected</li><li><strong>Performance assurance:</strong> All performance requirements validated</li><li><strong>Security validation:</strong> Comprehensive security testing coverage</li></ul><p><strong>Development Velocity Impact:</strong></p><ul><li><strong>Test creation time:</strong> Reduced from hours to minutes per feature</li><li><strong>Bug fix cycle:</strong> Shortened by 75% through immediate test feedback</li><li><strong>Confidence in releases:</strong> Increased from 70% to 95%</li><li><strong>Manual testing effort:</strong> Reduced by 85%</li></ul><h3>The development advantage with AI</h3><h3>Rapid iteration and refinement</h3><p>AI enabled continuous improvement through:</p><ul><li>Instant code generation: From specification to implementation in seconds</li><li>Automatic best practices: Built-in security, performance, and maintainability</li><li>Comprehensive testing: Generated unit tests with 90%+ coverage</li><li>Documentation sync: Automatic technical documentation updates</li></ul><h3>Quality and consistency</h3><p>The AI-assisted approach ensured:</p><ul><li>Consistent code style: Across all components and services</li><li>Proper error handling: Comprehensive error scenarios covered</li><li>Security compliance: Enterprise-grade security patterns</li><li>Performance optimization: Built-in optimization from day one</li></ul><h3>Reduced cognitive load</h3><ul><li>Developers could focus on:</li><li>Business logic refinement</li><li>User experience optimization</li><li>Integration testing</li><li>Deployment configuration</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aVrHUHG-s4DMG_OHFHiFNg.png" /></figure><h3>Business impact</h3><h3>Accelerated time-to-market</h3><p><strong>Traditional approach:</strong></p><ul><li>Requirements gathering: 1 week</li><li>Design and architecture: 1 week</li><li>Development: 3–4 weeks</li><li>Testing and refinement: 1–2 weeks</li><li>Total: <strong>6–8 weeks</strong></li></ul><p><strong>AI approach:</strong></p><ul><li>Complete development cycle: 24 hours</li><li>Time savings: 96%</li><li>Cost Efficiency and Quality</li><li>Reduced development costs by approximately 85%</li><li>Eliminated requirement misinterpretation costs</li><li>Minimized bug-fixing and rework cycles</li><li>Accelerated revenue generation</li><li>Achieved enterprise-grade quality from day one</li></ul><h3>The future of software development</h3><h3>Paradigm shift</h3><p>Our experience demonstrates that AI-assisted development represents a fundamental shift:</p><ul><li>From manual to guided development</li><li>From sequential to parallel implementation</li><li>From error-prone to quality-assured</li><li>From time-consuming to instantaneous</li><li>From testing as afterthought to testing as foundation</li></ul><h3>Developer evolution</h3><p>The role of developers is evolving from:</p><p>Code Writers → System Architects</p><p>Bug Fixers → Quality Validators</p><p>Implementation Focused → Business Value Focused</p><p>Test Creators → Test Strategy Designers</p><h3>The new development reality</h3><p>The successful delivery of a sophisticated marketplace system in under 24 hours using</p><p>AI marks a watershed moment in software engineering. This isn’t just about faster development-it’s about better, more reliable development with built-in quality assurance.</p><h3>Key takeaways</h3><ul><li>AI Assistance is production-ready: Multiple AI are delivering real business value today</li><li>Quality and speed can coexist: Properly implemented AI assistance improves both development velocity and code quality</li><li>Testing transformation: AI-powered testing moves quality assurance from bottleneck to accelerator</li><li>Human expertise still crucial: AI augments rather than replaces developer expertise</li><li>Systematic approach wins: Structured development methodologies combined with AI deliver exceptional results</li></ul><h3>The path forward</h3><p>As we move forward, organizations that embrace AI-assisted development will gain significant competitive advantages:</p><ul><li>Faster innovation cycles</li><li>Reduced development costs</li><li>Higher quality outputs</li><li>Improved developer satisfaction</li><li>Comprehensive test coverage from day one</li></ul><p>The 24-hour marketplace delivery with 90%+ test coverage isn’t an anomaly-it’s the new benchmark for what’s possible in modern software development. The age of AI-powered code generation has arrived, and it’s transforming our industry at breathtaking speed while raising quality standards to unprecedented levels.</p><p>*This article was inspired by actual development experience with an AI CodeGen tool. All screenshots and technical implementations are from the delivered marketplace system. The testing metrics and coverage results reflect actual achievements during the 24-hour development cycle.*</p><p><em>Originally published at </em><a href="https://tallysolutions.com/technology/how-ai-powered-code-generation-built-a-marketplace-in-24-hours/"><em>https://tallysolutions.com</em></a><em> on February 16, 2026.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=481626cb8fba" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fortifying the Future: A Blueprint for Security in Emerging Projects — The OCI Way]]></title>
            <link>https://medium.com/@techattally/fortifying-the-future-a-blueprint-for-security-in-emerging-projects-the-oci-way-4c41289076d2?source=rss-996d4db5c7dc------2</link>
            <guid isPermaLink="false">https://medium.com/p/4c41289076d2</guid>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[architecture]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[oracle-cloud]]></category>
            <dc:creator><![CDATA[Tech @Tally]]></dc:creator>
            <pubDate>Mon, 15 Dec 2025 10:47:54 GMT</pubDate>
            <atom:updated>2025-12-24T06:46:21.341Z</atom:updated>
            <content:encoded><![CDATA[<h3>Fortifying the Future: A Blueprint for Security in Emerging Projects — The OCI Way</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iZRE9o690TMCOuBX-2-waA.png" /></figure><p>As cloud-native architecture continues to evolve, securing workloads on Oracle Cloud Infrastructure (OCI) has become more critical-and more complex-than ever. A layered, principle-driven security approach is essential to protect modern cloud environments from emerging threats. In this article, we unpack practical, actionable strategies across design, development, infrastructure, and deployment phases. Whether you’re an architect, developer, security engineer, or operations professional, this guide provides the insights you need to build resilient, compliant, and future-ready cloud systems that stand strong in today’s dynamic landscape.</p><h3>Security principles in design</h3><p>Designing security principles in Oracle Cloud Infrastructure (OCI) and Kubernetes (OKE) means thinking beyond compliance. The goal is to build cloud-native systems where security is intentional, repeatable, and integrated into every decision.</p><ul><li><strong>Multi-tenancy architecture: </strong>Multi-tenancy in OCI enables secure and isolated sharing of cloud resources across multiple tenants using compartments and IAM policies. It supports centralized management while maintaining strict separation between tenants.</li><li><strong>Workload identity for authorizing Kubernetes workloads</strong>: Design IAM roles so that Kubernetes workloads only access required OCI services, in other words, use namespace basis authentication through OCI Policies — gives greater control over access management.</li><li><strong>No privileged workload</strong>: Assume no trust between components. Isolate workloads via Kubernetes namespaces and pod security policies (via Kyverno).</li><li><strong>Network security: </strong>Use Virtual Cloud Networks (VCNs) and Network Security Groups (NSGs) to effectively isolate and control traffic within OCI. Including Dynamic Routing Gateways (DRGs) and Service Gateways help ensure secure, private connectivity to on-premises networks and Oracle services. This approach creates a strong, layered network security framework for cloud environments.</li></ul><h3>Security inbuilt into code</h3><p>Security is strongest when embedded into development pipelines, not bolted on afterward.</p><ul><li><strong>Shift-left scanning</strong>: Integrate OCI’s Vulnerability Scanner into CI/CD pipelines to detect image vulnerabilities before deployment, aided with CloudGuard security scoring</li><li><strong>Secrets lifecycle management</strong>: Store sensitive values in OCI Vault and inject them into pods via workload identity. Developers never see raw secrets.</li><li><strong>Automated remediation:</strong> Use CloudGuard’s detector and responder recipes to identify misconfigurations and automatically or semi-automatically fix issues before they impact production.</li></ul><h3>Cloud infrastructure security principles</h3><p>OCI’s infrastructure is the foundation for securing Kubernetes workloads.</p><ul><li><strong>Regularize CIS compliance checker</strong> : Oracle has created an assessment script that can be run against any OCI tenancy to evaluate the compliance with the CIS OCI Foundations Benchmark and Oracle Best Practices.</li><li><strong>Network security controls:</strong> Define security rules using Network Security Groups (NSGs) and Kubernetes Network Policies (e.g., Calico) to enforce pod-level and host-level network segmentation.</li><li><strong>Private cluster deployment:</strong> Deploy private OKE clusters with Kubernetes API endpoints in private subnets to limit access within OCI networks.</li><li><strong>Perimeter defense:</strong> Enable OCI Web Application Firewall (WAF) to protect APIs and apps from OWASP Top 10 threats, bots, and DDoS attacks.</li><li><strong>Continuous monitoring and auto-remediation:</strong> Use OCI cloud guard’s detector and responder recipes to assess risks continuously and automate remediation, integrated with Event Rules for fast action.</li><li><strong>Multi-layered governance:</strong> Combine OCI Security Zones, Governance Rules (Region and Tag-based), Quota Policies and Kubernetes admission controllers, and Kyverno policies for comprehensive security enforcement.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aeHHCdUYPluJ7pHz9s_KRw.png" /></figure><h3>Cloud deployment security principles</h3><p>Deployments are the last frontier where security decisions meet production workloads.</p><ul><li><strong>IAM boundaries: </strong>Apply quota policies to restrict resource creation limits (e.g., compute instances, node pools) to reduce blast radius from compromised credentials or automation errors.</li><li><strong>Dependency management:</strong> Automatically generate and validate Software Bill of Materials (SBOMs) in build pipelines to track and prevent vulnerable open-source components.</li><li><strong>Secure remote access: </strong>Use OCI Bastion Service to provide secure, auditable access to instances and containers instead of exposing SSH or RDP ports publicly.</li><li><strong>IAM administration: </strong>Enforce least privilege access with IAM and role-based policies at all levels — IAM Administration from the parent tenancy with tenancies with domains synced via OCI integrated application<strong>.</strong></li><li><strong>Configuration management: </strong>Use infrastructure as code (Terraform) with policy-as-code enforcement to ensure security baselines and prevent drift during deployments.</li><li><strong>Continuous monitoring and incident response: </strong>Leverage OCI Cloud Guard, and OCI Event Rule to monitor deployed workloads, alert on suspicious activities, and trigger automated or manual remediation.</li></ul><p>Securing OCI and Kubernetes workloads is not a one-time project-it’s a continuous discipline. By embedding security principles at every stage-from design and code to infrastructure and deployment-organizations can move faster without sacrificing resilience. OCI’s native services like Cloud Guard, Security Zones, Vault, Bastion, and DRG combined with Kubernetes-native tools such as Workload Identity and Kyverno create a strong, adaptive defense posture.</p><p>For security professionals, it ensures compliance and risk reduction. For architects, it offers scalable, repeatable patterns. For developers, it reduces friction through secure defaults. And for operations teams, it delivers guardrails and visibility.</p><p>Ultimately, the blueprint for emerging projects is clear : security must be built in, not bolted on.</p><p><em>Originally published at </em><a href="https://tallysolutions.com/technology/security-blueprint-emerging-projects-oci/"><em>https://tallysolutions.com</em></a><em> on December 15, 2025.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4c41289076d2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Landscape of Concurrent Computing: Workers, Work, and Resources]]></title>
            <link>https://medium.com/@techattally/the-landscape-of-concurrent-computing-workers-work-and-resources-27913b3adcf1?source=rss-996d4db5c7dc------2</link>
            <guid isPermaLink="false">https://medium.com/p/27913b3adcf1</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[operations]]></category>
            <category><![CDATA[synchronization]]></category>
            <category><![CDATA[computing]]></category>
            <category><![CDATA[operating-systems]]></category>
            <dc:creator><![CDATA[Tech @Tally]]></dc:creator>
            <pubDate>Thu, 20 Nov 2025 05:39:09 GMT</pubDate>
            <atom:updated>2025-11-24T08:41:31.151Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*13kz-xEjZ_z1vfN6qzh6ow.png" /></figure><p>In the realm of computing, every operation boils down to a dynamic interplay between three core components:</p><ul><li><strong>Workers (CPU Cores/Execution Units):</strong> These are the computational engines, the physical or logical units within a CPU that execute instructions. Modern CPUs often feature multiple cores, each capable of acting as an independent worker.</li><li><strong>The Work (Tasks/Operations):</strong> This encompasses all the instructions, calculations, and I/O operations that a program needs to perform.</li><li><strong>Resources (Memory, Disk, Network, Data Structures):</strong> These are the passive elements that workers interact with or depend on. This includes system hardware (like memory, storage drives, network interfaces) and shared data structures within a program.</li></ul><p>The fundamental challenge in optimizing performance is that we typically face constraints: a finite number of workers and limited resource bandwidth. Yet, the demand for “work” is often effectively infinite. The universe’s to-do list, it seems, is always longer than our processing power! This inherent imbalance necessitates clever algorithms, and system designs to maximize throughput and responsiveness.</p><h3>Understanding “Workers”: A multi-layered perspective shift</h3><p>The term “worker” in computing can be a source of confusion because it’s meaning subtly changes depending on the layer of abstraction you’re observing. Let’s clarify these distinct perspectives:</p><ol><li><strong>Hardware perspective (Microprocessor/Microcode):</strong></li></ol><ul><li><strong>Worker:</strong> At the very lowest level, the physical CPU core itself is the ultimate worker. It contains execution units (ALUs, FPUs), registers, caches, and control logic driven by <strong>microcode</strong> (low-level instructions that translate complex CPU instructions into simpler, hardware-executable steps).</li><li><strong>Work:</strong> The actual machine code instructions (e.g., ADD, LOAD, STORE) that microcode executes.</li><li><strong>Focus:</strong> This level is about the raw ability to execute instructions as fast as possible, managing instruction pipelines and cache coherence.</li></ul><p><strong>2. Operating System (OS) perspective:</strong></p><ul><li><strong>Worker:</strong> From the OS kernel’s viewpoint, the <strong>physical CPU cores</strong> are its workers.</li><li><strong>Work:</strong> <strong>Threads</strong> (and processes, which contain at least one thread) are the <strong><em>schedulable units of work</em></strong> that the OS manages. The OS scheduler’s job is to efficiently allocate and switch these “work units” (threads) onto the available “workers” (CPU cores). The OS is the grand choreographer, deciding who gets CPU time.</li><li><strong>Focus:</strong> Scheduling, resource allocation, isolation between processes, and providing system services to applications.</li></ul><p><strong>3. Platform perspective (Runtime/Framework Abstractions):</strong></p><ul><li><strong>Worker:</strong> On top of the OS, various programming platforms and runtimes (like Java Virtual Machine, Node.js runtime, Go runtime, .NET CLR) create their own “workers” or scheduling models. These are often <strong>user-space threads</strong> (sometimes called ‘fibers’ or green threads) or asynchronous event loops.</li><li><strong>Work:</strong> The higher-level tasks defined by the platform’s concurrency model (e.g., a JavaScript callback in an event loop, a Go’s goroutine). These platform workers are mapped onto a smaller, managed pool of underlying OS threads.</li><li><strong>Focus:</strong> Providing a simpler, more abstract, and often more efficient concurrency model for application developers, reducing the direct burden of OS thread management.</li></ul><p><strong>4. Application programmer’s perspective:</strong></p><ul><li><strong>Worker:</strong> For someone writing an application, the threads (whether OS threads or platform-managed user-space threads) created within their program are their primary “workers.”</li><li><strong>Work:</strong> The specific logic, functions, and algorithms of the application itself. The programmer designs how these application-level “workers” will collaborate or operate independently to achieve the application’s goals.</li><li><strong>Focus:</strong> Implementing application features, managing shared data within the application, and orchestrating application-specific tasks.</li></ul><p>This layered understanding is crucial because it clarifies that “worker” isn’t a single, monolithic concept but rather an abstraction that changes meaning depending on which layer of the computing stack you’re examining. Another important aspect to think of here is each of these layers add a level of simplicity to upper layers, at the same time, they add their management overhead.</p><p><strong><em>“You have to dig deeper to build higher”</em></strong></p><p>The reason why any applications that ‘go deeper’, can extract performance out of it. This allows an application logic, for e.g., to directly execute on the OS thread/scheduler or even hardware physical directly bypassing the management layers.</p><p>With our understanding of these multi-layer perspective, we will proceed further with the OS perspective (which is the most common thing and easier to relate for our rest of discussions). So, we will be using the word ‘thread’ largely for ‘worker’.</p><h3>Threading models: Single-threaded vs. Multi-threaded</h3><p>The way a program structures its application-level “workers” (threads) significantly impacts its behaviour and performance.</p><ul><li><strong>Definition:</strong> A program or system that has only one main application thread of execution. All tasks are processed sequentially by this single worker (thread).</li><li><strong>How it Works:</strong> The single thread picks up one task, executes it to completion, and then moves to the next.</li><li><strong>Pros:</strong> Simpler to design, implement, and debug due to a straightforward, predictable flow. Eliminates complexities like race conditions that arise from multiple workers accessing shared data, as only one worker exists.</li><li><strong>Cons:</strong> Can lead to unresponsiveness if a single task takes a long time (e.g., waiting for data from a slow network or disk, known as a “blocking I/O operation”). This single thread blocks, halting all other processing for that program. Your program becomes the digital equivalent of a statue, unresponsive to clicks or key presses until it’s done. It cannot fully leverage multi-core processors.</li><li><strong>Use Cases:</strong> Simple scripts, command-line utilities, or applications where responsiveness isn’t paramount and tasks are predominantly short and CPU-bound. Early operating systems like DOS were largely single-threaded.</li></ul><p><strong>2. Multi-threaded systems</strong></p><ul><li><strong>Definition:</strong> A program or system designed to create and manage multiple application threads of execution concurrently (or in parallel on multi-core processors).</li><li><strong>How it Works:</strong> The underlying OS or platform runtime efficiently distributes these application threads across available CPU cores. Each thread can potentially execute independently. On a single-core CPU, the OS rapidly switches between threads (context switching), creating the <em>illusion</em> of parallelism. On multi-core CPUs, threads can run genuinely in parallel on different cores.</li><li><strong>Pros:</strong></li></ul><p><strong>Improved responsiveness:</strong> If one thread blocks (e.g., waiting for I/O), other threads can continue executing, keeping the application interactive.</p><p><strong>Enhanced performance (Parallelism):</strong> Can harness the power of multi-core processors to perform multiple CPU-bound tasks simultaneously, significantly speeding up overall execution.</p><p><strong>Efficient resource utilization:</strong> Threads within the same process share resources like memory, making them more lightweight than separate processes.</p><ul><li><strong>Cons:</strong></li></ul><p><strong>Complexity:</strong> Introduces challenges like race conditions (multiple threads trying to access/modify shared data at the same time, leading to unpredictable results) and deadlocks (where threads get stuck waiting for each other indefinitely).</p><p><strong>Synchronization overhead:</strong> Requires careful management of shared resources using synchronization primitives (like locks) which add performance overhead.</p><p><strong>Debugging difficulty:</strong> Non-deterministic nature makes bugs harder to reproduce and fix.</p><ul><li><strong>Use cases:</strong> Modern operating systems, web servers, database systems, gaming engines, and almost any high-performance, interactive application.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*skHtsAsG_NM5CIal.jpg" /></figure><h3>Concurrency building blocks: The layers of orchestration</h3><p>Concurrency is built up in layers, from the hardware up to the application. Understanding these building blocks helps clarify how it all fits together.</p><p><strong>Layer 1: Hardware/CPU level (The physical workers and atomics)</strong></p><p>At the lowest level, the CPU provides the fundamental capabilities for concurrency.</p><ul><li><strong>CPU cores:</strong> The true physical “workers” that execute instructions.</li><li><strong>CPU caches:</strong> High-speed memory located directly on or very near the CPU cores, used to speed up data access. Managing cache coherence across multiple cores is crucial and handled by hardware.</li><li><strong>Atomic instructions:</strong> These are special CPU instructions (e.g., increment, swap, compare-and-swap (CAS)) that are guaranteed to complete without interruption, even if multiple cores try to execute them on the same memory location simultaneously. They are the bedrock upon which all higher-level synchronization primitives are built. If conflicts arise, the hardware ensures one operation succeeds, and others are notified (implicitly or explicitly via status flags) to retry.</li></ul><p><strong>Layer 2: Operating System (OS) level (Managing work units and resource access)</strong></p><p>The OS acts as the primary orchestrator between applications and hardware.</p><ul><li><strong>Processes:</strong> Independent programs with their own isolated memory spaces. The OS manages their creation, scheduling, and termination.</li><li><strong>Threads (OS threads):</strong> Lightweight units of execution <em>within a process</em>. The OS schedules these threads onto CPU cores. An OS thread is the smallest unit of work that the OS scheduler can manage independently.</li><li><strong>Schedulers:</strong> The part of the OS that decides which thread runs on which CPU core and for how long. It’s responsible for <strong>context switching</strong> (saving the state of a running thread and loading the state of another) to give the illusion of concurrency on single cores or to manage parallelism on multi-core systems.</li></ul><h3>OS-provided synchronization primitives: Mutexes, semaphores, and condition variables</h3><p>These are essential tools provided by the OS to application programmers for coordinating access to shared resources and managing thread interactions. While they all serve to synchronize threads, they have distinct purposes. Let’s use a common analogy: a bank.</p><ul><li><strong>Mutex (Mutual Exclusion Lock):</strong></li></ul><p><strong>Purpose:</strong> To protect a <strong>critical section</strong> of code or a shared resource, ensuring that <em>only one thread</em> can access it at any given time.</p><p><strong>Mechanism:</strong> A thread attempts to “lock” the mutex. If it’s unlocked, the thread acquires it and proceeds. If it’s locked by another thread, the attempting thread blocks (is put to sleep by the OS) until the mutex is released. When released, one waiting thread is woken up by the OS to acquire the lock.</p><p><strong>Bank Analogy:</strong> Imagine a single-occupancy teller window. Only one customer (thread) can be served at a time. If someone is already there, everyone else waits politely in line until the window is free.</p><p><strong>Relationship to Atomics:</strong> Mutexes are built upon lower-level atomic instructions. For instance, the very act of checking if a mutex is free and then marking it as locked typically involves an atomic operation.</p><ul><li><strong>Semaphore:</strong></li></ul><p><strong>Purpose:</strong> A more generalized signalling mechanism used to control access to a limited number of resources or to signal between threads that an event has occurred.</p><p><strong>Mechanism:</strong> A semaphore maintains an internal counter. Threads can perform two operations:</p><ul><li><strong>Wait or acquire:</strong> Decrements the counter. If the counter becomes negative, the thread blocks until another thread increments it.</li><li><strong>Signal or release: </strong>Increments the counter. If there are threads waiting, one is unblocked.</li></ul><p><strong>Bank analogy:</strong> Think of a bank branch that has a limited number of safe deposit box rooms — say, 5 rooms. A semaphore could represent these 5 rooms. acquire is a customer trying to enter a room; signal is a customer leaving a room. If all 5 rooms are occupied, the 6th customer waits outside. The world, instead of solving its bank queues by building infinite counters, intuitively solved it with a queue system and later a token-based “wait” system (like the numbered tokens you get at some service centers) — because even banks understood the value of waiting efficiently rather than letting everyone storm the counter simultaneously!</p><p><strong>Difference from Mutex:</strong> A mutex is essentially a binary semaphore (counter of 0 or 1) used specifically for mutual exclusion. A general semaphore can have any non-negative integer value, allowing more than one thread (up to the counter value) to access a resource concurrently. A thread that signals a semaphore doesn’t necessarily have to be the one that previously waited on it.</p><ul><li><strong>Condition variable:</strong></li></ul><p><strong>Purpose:</strong> Used to make threads wait until a specific condition becomes true, often in conjunction with a mutex. It doesn’t protect data itself but rather facilitates communication and coordination between threads based on shared state.</p><p><strong>Mechanism:</strong> A thread that needs a condition to be true will acquire an associated mutex (to protect the shared condition data), check the condition, and if false, it will call wait on the condition variable. This atomically releases the mutex and blocks the thread. When another thread changes the shared state and makes the condition true, it acquires the same mutex, modifies the state, and then calls signal (or broadcast) on the condition variable to wake up one (or all) waiting threads. The woken thread then re-acquires the mutex and re-checks the condition.</p><p><strong>Bank analogy:</strong> This is the bank manager announcing, “Attention customers: We’ve just restocked on that rare foreign currency you wanted! Everyone who was waiting for it, you can come up now!” You don’t need the teller window key (mutex) to hear this, but you do need to hear it (the condition being met) to proceed with your transaction.</p><h3>The critical concept of “context” and context switching overhead</h3><p>To effectively manage multiple threads or tasks, the OS (or a runtime’s scheduler) must be able to pause one thread and resume another. This requires preserving the “state” of the paused thread, known as its context.</p><p><strong>What is a Thread’s context?<br></strong>A thread’s context is the minimal set of data that fully describes its current state of execution at any given moment. This includes:</p><ul><li><strong>CPU register values:</strong> The contents of all general-purpose registers, special-purpose registers and stack pointers.</li><li><strong>Program Counter (PC):</strong> The address of the next instruction to be executed.</li><li><strong>Stack Pointer (SP):</strong> The current position of the thread’s execution stack, which stores local variables, function call information, and return addresses.</li><li><strong>Process Control Block (PCB) / Thread Control Block (TCB) information:</strong> Data structures maintained by the OS that hold various administrative details about the thread or process.</li></ul><p><strong>The overhead of context switching</strong></p><p>When the OS decides to switch from one running thread to another (a context switch), it must perform the following steps:</p><ol><li><strong>Save the context:</strong> The OS saves the complete context of the currently running thread into its Thread Control Block (TCB) in memory.</li><li><strong>Load the context:</strong> The OS loads the previously saved context of the next thread to be run from its TCB into the CPU’s registers.</li><li><strong>Resume execution:</strong> The CPU then starts executing instructions from the loaded thread’s program counter.</li></ol><p>This process, while essential for multitasking, is not free. The act of saving and restoring context is pure overhead, as no “useful” work (application logic) is being done during this time. The overhead includes:</p><ul><li><strong>Direct CPU cycles:</strong> The time spent executing the OS kernel code to perform the save/restore operations. This involves memory accesses, which are slower than CPU operations.</li><li><strong>Cache invalidation:</strong> When a new thread starts executing, its data and instruction pipelines might not be in the CPU’s fast caches (L1, L2, L3). This leads to cache misses, forcing the CPU to fetch data from slower main memory, significantly impacting performance. Similarly, the Translation Lookaside Buffer (TLB), which caches virtual-to-physical memory address mappings, might need to be flushed or updated, leading to more memory access delays.</li></ul><p>Think of it like this: the CPU, bless its silicon heart, has to essentially “save its brain state” into a digital folder, then open another folder for the next task. These mental acrobatics isn’t free; it’s like stopping a high-speed train, changing all the passengers, and then accelerating again. Therefore, algorithms and system designs aim to:</p><ul><li><strong>Minimize context switches:</strong> Reduce the frequency of switches, especially for performance-critical paths.</li><li><strong>Reduce context size:</strong> Keep the amount of data that needs to be saved/restored small.</li></ul><h3>Synchronization algorithms: Lock-based, lock-free, and wait-free</h3><p>Understanding context switch overhead highlights why different synchronization strategies are chosen.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*OgzKKhBTrl4wIPLo.jpg" /></figure><ol><li><strong>Lock-based algorithms</strong></li></ol><ul><li><strong>Mechanism (with overhead in mind):</strong> When an application thread tries to acquire a lock that’s held, the OS deschedules it and performs a context switch to another runnable thread. This avoids busy waiting (spinning) and frees the CPU core for other tasks.</li><li><strong>Trade-off:</strong> For long waits or low contention, this is highly efficient because it prevents a CPU core from wasting cycles. However, for very short waits or high contention (where locks are acquired and released rapidly), the overhead of frequent context switches can become a performance bottleneck, potentially making the system <em>slower</em> than if it had just spun for a brief moment.</li></ul><p>2. <strong>Lock-free algorithms (Non-blocking progress)</strong></p><ul><li><strong>Concept:</strong> A subset of non-blocking algorithms that guarantees system-wide progress. this means that if multiple threads are attempting to access a shared resource, at least one of them will complete its operation in a finite number of steps, even if other threads are temporarily delayed or suspended. Individual threads might still “starve” (be repeatedly retried/delayed), but the system as a whole always makes progress.</li><li><strong>Mechanism (with overhead in mind):</strong> These algorithms typically rely on atomic operations and retries (spinning). An application thread will <em>not</em> block or trigger an OS context switch if its atomic operation fails. Instead, it immediately retries.</li><li><strong>Trade-off:</strong> Eliminates context switch overhead during contention, which can be a significant advantage for very short critical sections or high contention on multi-core systems, where threads are likely to find the resource free quickly or spinning is brief. However, if contention is prolonged or atomic operations repeatedly fail, threads can busy-wait (spin), wasting CPU cycles. This “wasted work” still impacts system throughput, even if it avoids the context switch penalty for individual threads.</li></ul><p><strong>3. Wait-free algorithms (Strongest guarantee)</strong></p><ul><li><strong>Concept:</strong> The strongest form of non-blocking algorithm. It guarantees per-thread progress, meaning <em>every thread</em> attempting an operation on a shared resource will complete its operation in a <em>finite number of steps</em>, regardless of the execution speed or failures (including pre-emption, crashes, or arbitrary delays) of other threads. It’s not only about a worker picking up “other work” while waiting; it’s about the guaranteed, bounded completion of the specific operation <em>it</em> is performing.</li><li><strong>Mechanism (with overhead in mind):</strong> Wait-free algorithms are exceptionally complex to design. They avoid blocking. They often involve more sophisticated atomic operations, explicit “helping” mechanisms (where a thread might help another thread complete its stalled operation to ensure both make progress), or copying data to allow threads to work on private copies before atomically committing.</li><li><strong>Pros:</strong></li><li><strong>Eliminates starvation:</strong> The ultimate guarantee of progress for every participating thread. No thread will ever be left waiting indefinitely.</li><li><strong>Predictable prformance:</strong> Critical for hard real-time systems where operations must complete within strict, bounded time limits.</li><li><strong>Fault tolerance:</strong> A work failing or being delayed cannot prevent other threads from making progress.</li><li><strong>Cons:</strong></li><li><strong>Extremely complex implementation:</strong> Designing and verifying wait-free algorithms is notoriously difficult, often requiring deep mathematical proofs of correctness.</li><li><strong>Higher overhead (often):</strong> The mechanisms required to provide the wait-free guarantee (e.g., more complex atomic operations, data copying, helping other threads) can sometimes introduce more overhead per operation than simpler lock-free or even well-tuned lock-based algorithms, especially under low contention. This overhead is what you pay for the absolute guarantee.</li><li><strong>Scalability challenges:</strong> Can be harder to scale efficiently to very large numbers of cores due to increased cache coherency traffic or helping overhead.</li><li><strong>Industry use:</strong> Very niche. Primarily found in highly specialized real-time operating systems, mission-critical embedded systems (e.g., in avionics or medical devices), and academic research where absolute guarantees are non-negotiable.</li></ul><p>Anything that feels superior is also extremely complex inside. The wait-free algorithms are not some ‘generalised runtime / frameworks’ that can do the magic. It involves every work to be thought of having breakdowns to do them at different points in time (asynchronous), by different threads/execution units. As per our worker-work-resource analogy, each work to be designed in a way to be executed part-by part by different workers without much context overhead, resources are optimized to be able to allow some work to be paused (when they are in-use), while other work can continue. This allows workers to be always doing ‘some work’ utilizing the overall system processing power. In a simpler way, wait-free is about making worker ‘not wait’ and keep getting occupied with some work, at the cost of some ‘work’ waiting for its dependencies, resources needed to be available to progress. So, you pause / resume the work and not the worker!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZvF-plQqdWLJc6qCF1m7Lw.png" /></figure><h3>Synchronous vs. Asynchronous: Managing flow and latency</h3><p>The choice between synchronous and asynchronous operations is a design decision directly influenced by context switching costs and the desire for responsiveness as well as the trade-off on various resources (computing resources vs. latency heavy IO resources for e.g.)</p><ol><li><strong>Synchronous operations:</strong></li></ol><ul><li><strong>Concept:</strong> “Blocking” execution. When a synchronous operation is called, the initiating thread pauses and waits for that operation to complete before proceeding to the next instruction.</li><li><strong>Bank analogy:</strong> You walk up to a teller (your thread) and initiate a complex transaction, like a large international money transfer (a long-running I/O operation). You then stand there at the window, absolutely still, watching the teller (the I/O device) process it. You cannot move, talk to anyone else, or do anything else until that single transfer is <em>fully completed</em>. If the teller is slow, you’re just stuck.</li><li><strong>Impact on context switching:</strong> When a thread performs a synchronous I/O operation, it blocks. The OS will then perform a context switch to another runnable thread. The initial thread will remain “asleep” until the I/O operation completes, at which point the OS will again perform a context switch to reschedule it.</li><li><strong>Consideration:</strong> If I/O operations are frequent and long-running, synchronous calls will lead to many context switches, consuming CPU cycles for overhead rather than productive work.</li></ul><p><strong>2. Asynchronous operations</strong></p><ul><li><strong>Concept:</strong> A “non-blocking” execution. An application thread initiates an operation and immediately continues with other tasks. The completion of the asynchronous operation is handled later, typically via callbacks, promises, or async/await syntax. The initiating thread is released to do other work while the I/O operation is handled in the background by dedicated hardware or the OS.</li><li><strong>Bank analogy:</strong> You walk up to a teller and initiate that same complex international money transfer. But this time, instead of standing frozen, the teller gives you a receipt or reference number. You then immediately walk away from the window and go do other things — maybe you fill out another form, check your balance at an ATM, or even step outside for some fresh air. You’re productive and free while the transfer churns in the background. Later, your phone buzzes (a callback/event), signalling the transfer is complete, and you can then pick up its confirmation and may be do any activity that is a ‘continuation’ of the money transfer (e.g., informing your beneficiary).</li><li><strong>Impact on context switching:</strong> Asynchronous patterns effectively manage situations where workers would otherwise be idle due to I/O latency. When an asynchronous I/O operation is initiated, the initiating thread does <em>not</em> block. It can continue its own execution or be context-switched to another task by the OS or a runtime scheduler without waiting for the I/O. When the I/O completes, an event is typically put into a queue, and when the application’s main thread (or an event loop thread) processes that event, it can then continue the dependent work. The number of context switches specifically related to the <em>wait</em> for I/O is greatly reduced or avoided from the application’s perspective.</li><li><strong>Consideration:</strong> This leads to significantly higher CPU utilization and overall responsiveness, as workers are not tied up waiting for slow external operations.</li></ul><h3>The overhead of management</h3><p>While we’ve discussed workers, work, and resources, and the algorithms that try to coordinate them, there’s a crucial layer of “management” that also consumes our precious workers. The OS scheduler, the underlying microcode, and even platform runtimes (like the JVM or Node.js) are themselves complex pieces of software that require CPU cycles to perform their duties:</p><ul><li>Deciding which thread runs next.</li><li>Performing context switches.</li><li>Managing memory and caching.</li><li>Orchestrating user-space threads onto OS threads.</li><li>Handling interrupts and system calls.</li></ul><p>The overhead of management can make things ineffective, if not handled well. An example of an engineer’s whose efficiency largely goes into the planning, tracking, reporting of his work to multiple managerial layers can occupy his time significantly for this indirect work and reducing his actual throughput for the actual coding work. Looking at the context switching overhead as well, an engineering work that is distracted too frequently, can lead to loss of intensity and large amounts of time can go into the context switching.</p><p>However, this “management overhead” is a necessary evil. It cannot be zero. Just like in any large organization, having too many managers can become counterproductive. If the managers (OS, runtime) spend too much of the workers’ (CPU) time <em>managing</em> rather than letting the workers <em>do</em> the actual work, the overall efficiency plummets. The goal is for the managers to be so efficient and lightweight that their presence enables far more work to be done than the worker’s cycles they consume in management overhead.</p><p>That’s why in industries, they say, Thinkers are rare, doers are rarer, and managers are everywhere!</p><h3>Summary: Optimizing work in a constrained world</h3><p>Fundamentally, in the world of computing, work is effectively infinite, while the available workers (CPU cores/threads) and resources (memory, disk, network) are inherently limited. The overarching goal of concurrent programming and system design is to maximize the work accomplished: achieving high throughput for backend logic (processing as much data as possible) and ensuring excellent responsiveness for frontend logic (keeping user interfaces fluid and interactive).</p><p>This optimization hinges on the efficient utilization of our workers (threads). The ideal scenario is to ensure threads are constantly engaged in meaningful work, with the least possible unproductive time. This means minimizing:</p><ul><li><strong>Spinning:</strong> Threads actively consuming CPU cycles while repeatedly checking for a resource, wasting system-wide capacity. It’s like impatiently jogging in place while waiting for a bus that’s very late.</li><li><strong>Sleeping/blocking:</strong> Threads being idled by the OS while waiting for a resource or an I/O operation to complete. While the OS can schedule other processes during this time, it is still “waste” for the specific process / thread that is blocked, as its progress is halted and context switch overhead is incurred.</li><li><strong>Context switching overhead:</strong> The unavoidable cost of the OS saving and restoring thread states when switching between tasks. Frequent context switches, even if for “good” reasons, eat into productive CPU time.</li><li><strong>Management overhead:</strong> The CPU cycles consumed by the very systems (OS, runtimes, microcode) that manage concurrency, highlighting the need for efficient “managers” that enable more work than they consume.</li></ul><p>Sometimes, optimizing for concurrency also means intelligently increasing or logically partitioning resources. Examples include:</p><ul><li><strong>Versioning / Write-ahead systems:</strong> Instead of locking, create new versions of data for updates, allowing readers to access older versions without blocking. This can significantly reduce (or bring close to zero) of the reader-writer contentions.</li><li><strong>Increased granularity:</strong> Breaking down large, shared resources into smaller, independently manageable units to reduce contention on any single part.</li><li><strong>Lock-free data structures:</strong> Designing data structures that allow concurrent access without traditional locks, often leveraging atomic operations.</li><li><strong>Wait-free execution models: </strong>Avoiding the worker (who is the costliest resource) needing to halt and still do some other productive work without blocking on an operation to be completed.</li></ul><p>The pursuit continues for the ultimate algorithms. What concurrency algorithms can truly achieve a “holy grail” system that is simultaneously spin-free, lock-free, wait-free, and incurs zero/low context-switching and management overhead, thereby fundamentally changing the landscape of how we build high-performance and highly responsive systems? This remains a vibrant area of research and innovation in computing.</p><p><em>Originally published at </em><a href="https://tallysolutions.com/technology/concurrent-computing-guide/"><em>https://tallysolutions.com</em></a><em> on November 20, 2025.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=27913b3adcf1" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>