<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Charangan Vasantharajan on Medium]]></title>
        <description><![CDATA[Stories by Charangan Vasantharajan on Medium]]></description>
        <link>https://medium.com/@charangan?source=rss-c3d6dc76f97c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*nYsOs5vQVzY2DCi8b9b-bw.jpeg</url>
            <title>Stories by Charangan Vasantharajan on Medium</title>
            <link>https://medium.com/@charangan?source=rss-c3d6dc76f97c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 06:54:42 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@charangan/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Stopping AI agents from skipping steps, with one file that runs anywhere]]></title>
            <link>https://medium.com/swlh/stopping-ai-agents-from-skipping-steps-with-one-file-that-runs-anywhere-99135d2fca2d?source=rss-c3d6dc76f97c------2</link>
            <guid isPermaLink="false">https://medium.com/p/99135d2fca2d</guid>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[agent-guardrails]]></category>
            <category><![CDATA[mcp-server]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Charangan Vasantharajan]]></dc:creator>
            <pubDate>Sun, 27 Sep 2026 00:00:39 GMT</pubDate>
            <atom:updated>2026-09-28T02:00:31.432Z</atom:updated>
            <content:encoded><![CDATA[<p>An AI agent given a procedure as text decides how much of it to follow. It drops a step, calls the task done, and leaves no record of what it ran. In τ²-bench’s single-control domains, a claimed success the environment does not show accounts for 45% to 48% of failures, and no LLM judge in that study scored above 0.65 AUROC at catching it ( <a href="https://arxiv.org/abs/2606.09863">Advani, 2026</a>). Another study found that 27% to 78% of the success benchmarks report hide a procedural violation ( <a href="https://arxiv.org/abs/2603.03116">Cao, Driouich and Thomas, 2026</a>).</p><p><a href="https://github.com/Chaarangan/stepgate"><strong>Stepgate</strong></a> is the MCP server I built to stop that, and the name is the design: a gate at every step. The procedure is a YAML stepfile. When it runs, Stepgate shows the client’s agent only the current step, makes every API call itself, and moves on only when that step’s gates pass. The agent cannot skip ahead, because it is not told the next step exists until the current one passes, and it cannot declare a step done, because a step passes only when its gates pass over what it submitted.</p><p>A gate is a mechanical check (JSON Schema, a JSONLogic rule, an HTTP verifier, or a person’s approval) and never a model’s opinion. Because Stepgate makes the calls, a gate can compare what the model submitted with what the API returned, which is how a fabricated value gets caught. That makes the path through a procedure deterministic, within limits worth being exact about. Given the same inputs, the same submissions, and the same API responses, a run takes the same path: the steps run in the same order, and every gate gives the same verdict. What the model submits still varies, and so does anything a person approves.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SWhQo1kBCBvfGbJLqwqBtg.png" /></figure><p>The second goal was portability. I wanted an agent I could hand to anyone: one file, with no packages, runtime, or versions to install, that runs in whatever AI client the other person already uses, with the model they already pay for. MCP, the Model Context Protocol that clients such as Claude Code and Cursor use to call tools, makes that possible, because a server that speaks it plugs into all of them. So the stepfile is the whole agent. It names no model, provider, or framework, calls only remote APIs, and declares the credentials it needs without saying where they live, so whoever runs it supplies their own keys.</p><p>That is also what separates a stepfile from a workflow engine such as n8n. An n8n workflow runs inside an n8n server, with the model and keys configured there. A stepfile travels to the agent: anyone runs it with npx -y stepgate in the client they already use, and its gates come with it.</p><p>Two more properties come from Stepgate making every call. The model never holds an API key, because Stepgate attaches credentials itself and sends requests only to the hosts the stepfile declares. And every run writes a hash-chained ledger of each step, tool call, gate verdict, and retry, so editing a record afterwards breaks the chain, which stepgate --verify detects.</p><h3>Half of every stepfile was checking</h3><p>Stepgate ships with a catalog of eighteen stepfiles: CVE triage, a 10-K ratio extraction, a regulatory monitor, a Zendesk-to-Jira escalation and so on. I wrote a script to count where their lines go, and the answer was uncomfortable:</p><pre>#### THE CATALOG, AS FIRST WRITTEN ####<br>Lines, counted as block-style YAML: 16247<br>Share of lines: {&quot;gates&quot;:&quot;53%&quot;,&quot;produces&quot;:&quot;15%&quot;,&quot;tools&quot;:&quot;14%&quot;,&quot;instructions&quot;:&quot;8%&quot;,&quot;other&quot;:&quot;11%&quot;}<br>Filters over calls by tool: 145<br>Predicates reading: {&quot;calls&quot;:106,&quot;steps_or_inputs&quot;:110,&quot;output_only&quot;:45}</pre><p>Gates were 53% of the catalog and instructions were 8%. The same filter (“the successful results of this operation”) was written out by hand 145 times, and 110 of the 261 predicate gates compared the output with the inputs or an earlier step. Reading through them, most were not checking judgment. They checked that the model had copied a value correctly, counted a list correctly, or done a sum correctly.</p><p>The clearest case checks the vehicle on an insurance claim against its VIN. It decodes the VIN with NHTSA’s vPIC service and compares the claimed make and model with the decode, ignoring letter case. The model did the comparison, and a gate redid it by lowercasing both strings with a 26-branch ifone branch per letter of the alphabet. Four agent steps and 372 lines, mostly proving that the model could copy fields out of JSON and compare two strings.</p><p>That is backwards. A model is slow at copying and unreliable at arithmetic, and each of those steps cost tokens, sometimes a retry, and a gate few people could review. None of it needed judgement.</p><h3>The model judges, the server computes</h3><p>A stepfile has two kinds of steps. An agent step has instructions, and the model does it. A mechanical step has do instead: Stepgate makes the listed calls, with arguments built from the inputs and earlier outputs, and computes the step&#39;s output from a template. The client never sees a mechanical step. The VIN decode is one:</p><pre>- id: decode<br>  do:<br>    calls:<br>      - { id: decode, operation: decodeVin, arguments: { vin: { var: inputs.vin } } }<br>    output:<br>      make: { var: responses.decode.Results.0.Make }<br>      model: { var: responses.decode.Results.0.Model }<br>      model_year: { var: responses.decode.Results.0.ModelYear }</pre><p>When only part of an agent step is a computation, such as a count, a derived field covers it: Stepgate fills the field in after the model submits, so the model is never asked for it, and no gate has to check it.</p><p>The VIN stepfile is now four mechanical steps (decode, compare, recalls, verdict) and one agent step, the summary for the claims handler, in 242 lines. Against the live NHTSA APIs, the VIN decodes to a 2009 Toyota Prius with six recalls; the claim says a 2010 Camry, and the verdict isrefer, with no field touched by the model. Its one job is the prose, and the gates on that prose check it against those values. A summary citing a recall campaign that was never returned is rejected by name.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*25_KhLO5NoB5a7PbBIpLtw.png" /></figure><p>The whole catalog works this way now:</p><pre>#### THE CATALOG NOW ####<br>Lines, counted as block-style YAML: 12397<br>Share of lines: {&quot;gates&quot;:&quot;12%&quot;,&quot;mechanical&quot;:&quot;30%&quot;,&quot;produces&quot;:&quot;19%&quot;,&quot;tools&quot;:&quot;18%&quot;,&quot;instructions&quot;:&quot;4%&quot;,&quot;other&quot;:&quot;17%&quot;}<br>Filters over calls by tool: 2</pre><p>This did more for determinism than any gate. A mechanical step returns the same output for the same API responses every time, so each step moved out of the model is one less place for a run to vary.</p><p>The files went from 8,547 lines to 6,321, and gates from 53% to 12%. I trust that second number less than it looks, because much of the gate logic did not disappear. It moved into the do templates, which are JSONLogic too, and gates and templates together are still 42% of the catalog. The model does far less. The stepfile is only somewhat easier to read.</p><h3>Writes wait for a person</h3><p>Six of the stepfiles write something: a Jira issue, a Gmail draft, a monday.com item, a note on a Zendesk ticket, or a ServiceNow incident. Each follows the same shape. An agent step drafts exactly what will be written, a person approves it, and a mechanical step performs the write from the approved output. The write step can read only inputs and earlier outputs, never a fresh API response, so what reaches Jira is what the person saw.</p><p>The approval travels as an MCP elicitation, a form the client shows. Not every client shows it. Testing from the Claude Code extension in VS Code, every approval came back declined within a millisecond of the submission, which is not a human reaction time: the extension declares the capability and then declines without drawing the form (<a href="https://github.com/anthropics/claude-code/issues/79174">#79174</a>). The same test in a terminal session came back approved 5.8 seconds after the submit. Stepgate cannot tell those two declines apart, so it reports only that the approval was declined, and the docs name the clients that work.</p><h3>Where this stops being true</h3><p>Running a stepfile still needs Node.js 22.18 or later wherever Stepgate itself runs, so “no dependencies” is true of the file and not of the machine. Every number here comes from one catalog of eighteen stepfiles that I wrote, so the 53% says as much about how I wrote gates as about gates in general. The eleven public-data stepfiles pass against their live APIs with qwen3.6–35b-a3b through OpenRouter. The six that were written were checked only against recorded cases, because I do not have accounts on those services.</p><p>The bigger limit is JSONLogic. Inside a map or filteran expression sees only the current item, so anything that needs outer data turns into a reduce carrying context or a join against a one-element list. Sorting and date arithmetic are both awkward today. Those gaps are open as proposals in the <a href="https://github.com/Chaarangan/stepgate/issues">issue tracker</a>, with the largest being CEL expressions as strings beside JSONLogic. CEL&#39;s specification guarantees that an expression terminates, which matters when the file is untrusted.</p><h3>Try it</h3><p>Stepgate is on npm and the MCP Registry, and any MCP client can run it:</p><pre>npx -y stepgate --list                 # the catalog<br>npx -y stepgate auto-claim-vin-validation</pre><p>If you already have a procedure written down as a skill or a runbook, stepgate_outline turns it into a skeleton stepfile, with each rule it states (&quot;never&quot;, &quot;must&quot;) listed as a gate still to write. The <a href="https://github.com/Chaarangan/stepgate"><strong>repository</strong></a> has the format reference, the catalog, and the script behind both measurement blocks.</p><p>Moving work out of the model also made the agent more portable. A step Stepgate computes behaves the same whichever model the client runs, so the fewer steps a model does, the less a stepfile’s behaviour depends on where it lands. The path through a stepfile and every step that needs no judgement are deterministic, as long as the APIs return the same data. Only the judgement travels with the model.</p><p>The 53% was my own doing, but I doubt the pattern is mine alone. Ask a model to do mechanical work, and you end up writing checks to catch it, and each of those checks needs a retry budget and a reviewer who can read it. If I were reviewing a team’s agent design, the first question I would ask is which values the model produces that code could compute. Every one of those is cheaper to move into code than to guard. Most of mine should never have been the model’s job.</p><p><strong>Stepgate GitHub: </strong><a href="https://github.com/Chaarangan/stepgate"><strong>https://github.com/Chaarangan/stepgate</strong></a></p><p><em>Originally published at </em><a href="https://chaarangan.github.io/blog/stopping-ai-agents-from-skipping-steps/"><em>https://chaarangan.github.io</em></a><em> on September 27, 2026.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=99135d2fca2d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/stopping-ai-agents-from-skipping-steps-with-one-file-that-runs-anywhere-99135d2fca2d">Stopping AI agents from skipping steps, with one file that runs anywhere</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A per-token cost of zero is not a cost of zero]]></title>
            <link>https://medium.com/swlh/a-per-token-cost-of-zero-is-not-a-cost-of-zero-charangan-vasantharajan-c24a447137f2?source=rss-c3d6dc76f97c------2</link>
            <guid isPermaLink="false">https://medium.com/p/c24a447137f2</guid>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[ai-engineering]]></category>
            <category><![CDATA[local-llm]]></category>
            <category><![CDATA[finops]]></category>
            <category><![CDATA[llm-inference]]></category>
            <dc:creator><![CDATA[Charangan Vasantharajan]]></dc:creator>
            <pubDate>Sun, 06 Sep 2026 00:00:05 GMT</pubDate>
            <atom:updated>2026-09-07T02:06:00.581Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Memory, evaluation, and guardrail calls took 53% to 61% of the GPU across four models but only 27% to 37% of the output tokens. On hardware you own, the first number is the one that predicted the throughput I measured one session at a time.</strong></p><p>An agent that answers questions over your documents makes more model calls than the ones you wrote. Memory compaction, an LLM-as-judge score, a guardrail check on the way in and another on the way out. Across four models and two families, those unwritten calls took 52.8% to 61.4% of the GPU. The same calls were 27% to 37% of the output tokens, and that is the number most dashboards show you.</p><p>Debmalya Biswas recently put a number on this. Non-functional aspects, memory, evals and guardrails, “can lead to 2–3 times more LLM invocations than those invoked to directly execute the agent functionality” (<a href="https://aiadvances.org/tokenomics-for-the-agentic-harness-41dbaa822f7f">Tokenomics: FinOps for the Agentic Harness</a>). The article does not stop at the invoice either: it sizes the self-hosted case too, down to whether the model fits in one GPU and how batch size trades against latency, and proposes an OpenTelemetry attribute set so the calls can be costed.</p><p>What it left me wanting was the conversion. A count of invocations, and the token totals the standard attributes record, are both things I can log. Neither told me what those calls would cost on a box I had already bought, where the marginal token is free and the overhead lands on a budget no finance process watches: how many people you can serve before the machine falls over.</p><p>This post builds a harness and measures that on a 36 GB M3 Max.</p><ul><li>Why output-token accounting reports the overhead with the wrong sign, and why input-token accounting is no better</li><li>Whether the result survives a change of model family and a 20x span in weights</li><li>What happens to the overhead share as a session gets longer</li><li>What rewriting the front of your context costs: 31x on every later call</li><li>How continuous batching hides the harness until the box saturates</li><li>Which of weights and KV reservation actually sets resident memory</li></ul><h3>Two accounting methods, opposite conclusions</h3><p>The harness runs a toy agent over a synthetic corpus and tags every invocation as either <em>functional</em> (plan, answer) or <em>overhead</em> (guard in, compact, judge, guard out). Ollama reports prefill and decode durations separately, so a call can be charged for occupancy instead of merely counted.</p><p>Two shortcuts are worth declaring up front. The judge and the outbound guard get a placeholder where the generated answer would go, a short string naming its length, because this harness measures occupancy and never reads what the model wrote. And what the compaction and judge steps read as the session state is the question plus the retrieved documents, not an accumulated transcript with the plan and the answer in it. Both shortcuts shrink prompts that a real system would grow, so both understate the overhead share below.</p><p>Prefill and decode carry the whole argument. Prefill is the model reading the prompt, and its cost scales with how many tokens you hand it. Decode is the model writing the reply, one token at a time, and its cost scales with how many tokens come back. A hosted API bills for both and weights the bill toward decode. On a GPU you own, both occupy the same hardware and the weighting is irrelevant.</p><pre>@dataclass<br>class Call:<br>    cls: str          # &quot;functional&quot; | &quot;overhead&quot;<br>    step: str<br>    ...<br>    prompt_tokens: int<br>    output_tokens: int<br>    prefill_s: float<br>    decode_s: float<br><br>    @property<br>    def gpu_s(self) -&gt; float:<br>        return self.prefill_s + self.decode_s</pre><p>Context length is pinned with num_ctx for every model, because the daemon default of 131,072 would not fit a 23 GB model on this machine, and because a comparison across model sizes is meaningless if each one gets a different reservation. Three sessions on qwen3:4b:</p><pre>#### OUTPUT ####<br>  step        class       calls  in tok  out tok  prefill  decode   gpu s  % gpu<br>  ------------------------------------------------------------------------------<br>  answer      functional      3    4956      384     2.2s    5.3s    7.5s  30.6%<br>  compact     overhead        3    4974      192     5.0s    2.5s    7.5s  30.3%<br>  judge       overhead        3    5025       48     5.1s    0.6s    5.7s  23.3%<br>  plan        functional      3     116      192     0.2s    2.2s    2.5s  10.1%<br>  guard_in    overhead        3     128       48     0.2s    0.5s    0.7s   2.9%<br>  guard_out   overhead        3     108       48     0.1s    0.6s    0.7s   2.8%<br><br>  overhead:functional  out-tok 0.58x   gpu 1.46x   overhead = 59.3% of gpu</pre><p>The 2:1 call ratio is a design choice, not a discovery: I wrote four overhead steps and two functional ones. The divergence in the last line is the part I did not choose. Counted in output tokens the overhead is 0.58x the functional work, barely half. Counted in GPU-seconds it is 1.46x. The two methods put the overhead on opposite sides of break-even.</p><h3>The call that writes 16 tokens and takes a fifth of the box</h3><p>Look at judge. Across three sessions it produced 48 output tokens and consumed 23.3% of the GPU. That is 76% of what the answer call cost while producing one eighth of the output.</p><p>48 tokens over three calls is 16 each, which is exactly the num_predict ceiling the harness sets for that step, so on this model the judge was cut off before it finished. On the 1.7B and the 35B it stopped on its own after two tokens.</p><p>The explanation is the prefill column. judge spent 5.1s reading and 0.6s writing. answer spent 2.2s reading and 5.3s writing. The two expensive overhead steps are prefill-bound because they re-read the session: compaction reads the transcript and the judge reads it again. The two guardrails read only the question, which is why they sit at the bottom of the ledger at 2.9% and 2.8%.</p><p>That is where the harness parts company with the usual rule of thumb. Biswas cites the standard split, that “in most requests prefill takes less than 20% of the end-to-end latency, while decoding takes more than 80%”. My plan call obeys it exactly, 0.2s reading against 2.2s writing, 8% prefill, and answer is close enough at 29%. The two expensive overhead calls inverted it: compact is 67% prefill and judge is 89%. Across the whole harnessed session prefill took 52% of the GPU. The 20/80 rule describes a short prompt and a long answer, which is what the functional calls are. The non-functional calls are the other shape.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0NAml2sYk3FJ8at2ZBLiTw.png" /></figure><p><em>Compaction cost the same as answering the question, and wrote half as much.</em></p><h3>The same three sessions on three more models</h3><p>One model family proves nothing, and a 4B model is small enough that its fast decode could be flattering the overhead. So the same three sessions ran on four models: two sizes of Qwen 3, a Gemma 3 of matching size for a family check, and a 35B mixture-of-experts with 3B active parameters for a scale check.</p><pre>#### OUTPUT ####<br>  model               resident     decode  out-tok     gpu  overhead % of gpu<br>  ----------------------------------------------------------------------------<br>  qwen3:1.7b             2.4GB     147t/s    0.52x   1.45x              59.1%<br>  qwen3:4b               3.9GB      78t/s    0.58x   1.46x              59.3%<br>  gemma3:4b              3.9GB      80t/s    0.60x   1.59x              61.4%<br>  qwen3.5:35b-a3b       23.3GB      60t/s    0.38x   1.12x              52.8%</pre><p>The overhead share lands between 52.8% and 61.4% across two independent model families. The parameter span is 20x by weights, which is what sets the memory footprint, but the 35B activates only 3B of those per token, so in compute terms the spread is nearer 2x. It is a stronger check on architecture than on scale.</p><p>The 35B does have the lowest share, though not for the reason I first wrote down. I had it as slower decode making the decode-heavy answer call dearer. The rates say otherwise. Across the whole ledger the 35B prefilled at 727 tokens a second against the 4B’s 1,196, a slowdown of 1.65x, while its decode slowed by only 1.3x. On those numbers a prefill-bound overhead share should go up, not down.</p><p>Part of the answer is the output column. The 35B’s judge and outbound guard each stopped after two tokens where the 4B ran into its 16-token ceiling, so the overhead wrote 210 tokens where the 4B wrote 336. Had it written the 4B’s total at its own decode rate the ratio would land at 1.26 instead of 1.12, which recovers about 40% of the drop from 1.46. The rest sits on the functional side, whose prefill rose 2.9x against the overhead’s 1.4x. I have not isolated why.</p><p>The column that matters is out-tok. On every model tested it sits between 0.38x and 0.60x while GPU-seconds sit between 1.12x and 1.59x. Output-token counting puts the overhead below break-even on all four; GPU time puts it above on all four.</p><p>Counting input tokens instead does not rescue the method, it only moves the error. The overhead reads 2.02x what the functional calls read, identical to two decimal places on all four models, against a true GPU cost of 1.12x to 1.59x. So the output column reports between a third and two fifths of what the box charges, and the input column reports 1.3x to 1.8x of it. Neither lands on the real number. That matters for anyone instrumenting with the standard attributes, because input_tokens happens to sit near the 2-3x range the invocation count implies, which makes it look like a confirmation when it is a coincidence.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M6b_xnvQb9bVxSYb8A96LQ.png" /></figure><p><em>Every model puts the two measures on opposite sides of break-even.</em></p><h3>The share I expected to climb, and it did not</h3><p>Before running it I wrote down a prediction: since prefill scales with context and an agent session only grows, the overhead share should climb as the session ages. Eight turns of a growing transcript on qwen3:4b:</p><pre>#### OUTPUT ####<br>   turn  ctx tok  func gpu  over gpu  overhead % of gpu<br>  ------------------------------------------------------<br>      1      838      2.5s      2.6s              51.3%<br>      2     1033      1.8s      1.5s              44.5%<br>      3     1221      1.9s      1.5s              44.7%<br>      4     1409      1.9s      1.5s              43.9%<br>      5     1610      1.9s      1.5s              44.8%<br>      6     1792      1.9s      1.5s              44.5%<br>      7     1987      1.9s      1.5s              45.3%<br>      8     2175      1.9s      1.6s              45.2%</pre><p>Wrong. The context grew 2.6x and the share fell after turn one, then sat flat at about 45%. Turn one costs 5.1 GPU-seconds because nothing is cached yet. Every turn after it costs about 3.4, and that figure does not grow as the transcript does.</p><p>The reason is prefix caching. When a session grows by appending, each call prefills only the new tokens and reuses the rest of the KV cache. So the obvious mitigations, shorter sessions and more aggressive trimming, do not help here.</p><p>Note how little ground this covers. Eight turns took the transcript to 2,175 tokens, well short of the 8,192 the run was pinned to and shorter than the 5,000 token prompts in the ledger above. What I measured is that the share stays flat over the first couple of thousand tokens.</p><h3>Compaction pays twice</h3><p>If append-only growth is cheap because the cache holds, then the expensive thing is invalidating the prefix, not reading a lot. To check that, I ran the same judge call five times over an identical body of session state. In one arm the only text that changed between calls sat at the end of the prompt. In the other, a random nonce was pasted at the front as well.</p><pre>#### OUTPUT ####<br>   rep   append-only prefill   rewritten-prefix prefill<br>  ------------------------------------------------------<br>     0                 1.71s                      1.88s<br>     1                 0.06s                      1.89s<br>     2                 0.06s                      1.83s<br>     3                 0.06s                      1.87s<br>     4                 0.06s                      1.84s<br><br>  median prefill after first call: append-only 0.06s   rewritten 1.87s<br>  penalty 30.9x</pre><p>The session state and the work asked for are identical in both columns; the second carries the nonce on top, about fifteen tokens. Appending lets prefill collapse from 1.71s to 0.06s once the cache is warm. Prepending costs 1.87s on every call for the life of the session.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jlarHM41XJkytLUaOpHzvg.png" /></figure><p><em>The only difference between the two lines is where a few tokens were inserted.</em></p><p>A lot of ordinary agent code sits on the wrong side of that line. Compaction that rewrites the transcript buys memory and forfeits the cache for every later call, which is why it pays twice. Retrieved documents placed at the top of the prompt, the default layout in most RAG code, change every turn and guarantee the cache never hits. Anything rotating in the system prompt, a timestamp or a session ID, costs the entire prefix for the sake of a few tokens.</p><h3>Batching hides the harness until the box saturates</h3><p>The first version of this measurement ran with OLLAMA_NUM_PARALLEL unset, so the box was serialising instead of batching. Rerunning the concurrency sweep at 1 and at 4 says how much of the penalty was queueing. The two settings are separate runs in the raw log, so unlike every other table here this one is merged by hand and the tail column is renamed:</p><pre>#### E4, BOTH RUNS, MERGED BY HAND ####<br>  concurrent  harness   parallel=1  parallel=4   slow p1   slow p4<br>  ------------------------------------------------------------------<br>           1    False        16.17       17.89       3.6       3.3<br>           1     True         6.00        7.28      10.0       8.7<br>           2    False        14.70       18.66       8.4       7.0<br>           2     True        10.15       12.94      11.9       9.5<br>           4    False        16.31       20.24      12.5      11.8<br>           4     True        11.03       17.18      21.5      14.0<br>           8    False        18.02       27.26      24.2      17.6<br>           8     True        10.62       11.49      45.0      41.8</pre><p>Batching works, up to a point. At four concurrent sessions it lifts the harnessed path from 11.03 to 17.18 sessions per minute and the penalty against an unharnessed box nearly disappears, from 1.48x down to 1.18x. At eight sessions the unharnessed path keeps scaling to 27.26 while the harnessed path falls back to 11.49, the penalty reopens to 2.37x, and the slow tail sits at 41.8s.</p><p>Two caveats on that table. Every cell is a single run of four sessions, eight at the top step, with no repeats, so the flat no-harness line at parallel=1 (16.17, 14.70, 16.31, 18.02) is inside the noise and is not a trend. And the last two columns are the second-slowest session in each run, which is all a sample that size supports. They are not a p95, whatever my harness called them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LdMQPzcFE2lu9R1EY6HNtw.png" /></figure><p><em>The harnessed path tracks the clean one to four sessions, then falls back while the clean one keeps going.</em></p><p>So the harness costs headroom, not a fixed fraction of the box. It lowers the concurrency at which the machine stops scaling, so a capacity number measured with the guardrails switched off will be higher than the one you can operate at.</p><h3>Does the ledger predict anything?</h3><p>The whole argument rests on GPU-seconds being the number worth keeping, so it is fair to ask whether it forecasts anything. It does, and the table above is the check.</p><p>The ledger for qwen3:4b puts overhead at 1.46x the functional work, so a harnessed session should cost 2.46 times an unharnessed one, and a box running them one at a time should turn over 2.46x fewer. Measured at OLLAMA_NUM_PARALLEL=4, concurrency 1: 17.89 against 7.28, a ratio of 2.46. With batching off it came out at 2.70, and at eight concurrent sessions 2.37.</p><p>Run the same prediction from the output-token column and you get 1 + 0.58 = 1.58x against the same measured 2.46, an undershoot of about a third on the throughput you give up.</p><p>One honest limit on that comparison. Both predictions are single-session accounting, so the serial column is the only place either of them claims to forecast anything, and away from it neither does. The eight cells of the batching sweep give penalties of 2.70, 1.45, 1.48 and 1.70 with batching off, and 2.46, 1.44, 1.18 and 2.37 with it on. At concurrency 2 and 4 the penalty collapses to between 1.18x and 1.48x, and the 1.58x token figure lands nearer to those than the ledger’s 2.46x does. That is not the token column working. It is a smaller wrong number sitting near a region where the penalty is set by how much queue the batcher can absorb rather than by any per-session ledger.</p><p>Four sessions per cell does not earn a match as close as 2.46 against 2.46, and the workload mix is not quite identical between the two experiments, so the third significant figure is luck. What the ledger buys is the serial number, and that is the one that sets a floor under capacity.</p><h3>Resident memory is weights plus a reservation you chose</h3><p>The first run of this experiment produced an oddity I could not explain: a model whose weights are about 2.5 GB on disk sat at 22 GB resident. Varying num_ctx explains it. Resident figures here are what Ollama reports through /api/ps, in decimal GB, and the sweep ran with four parallel slots.</p><pre>#### OUTPUT ####<br>  model               num_ctx   resident   over weights<br>  ------------------------------------------------------<br>  qwen3:4b               2048      3.9GB          1.4GB<br>  qwen3:4b               8192      7.5GB          5.0GB<br>  qwen3:4b              32768     22.0GB         19.5GB<br>  qwen3:4b             131072     80.6GB         78.1GB</pre><p>The KV reservation is linear in context length and it dwarfs the weights well before you reach the default. It is also per parallel slot, so the num_ctx column above is really four times that many tokens of reservation: 32,768 across four slots produced the same 22 GB that 131,072 on a single slot produced in the earlier run, which is where the original oddity came from. By the last row this 2.5 GB model is asking for 80.6 GB on a machine with 36, a number Ollama will report and the hardware cannot honour.</p><p>Those four rows look like they say resident memory is set by the context window and not by the parameter count. I believed that until I ran the next model.</p><h3>The 35B with more room for context than the 4B</h3><p>Running the same sweep on the mixture-of-experts model inverts the conclusion.</p><pre>#### OUTPUT ####<br>  model               num_ctx   resident   over weights<br>  ------------------------------------------------------<br>  qwen3.5:35b-a3b        2048     23.1GB          0.1GB<br>  qwen3.5:35b-a3b        8192     23.3GB          0.3GB<br>  qwen3.5:35b-a3b       32768     23.8GB          0.8GB</pre><p>Going from 2,048 to 32,768 tokens costs the dense 4B model 18.1 GB and costs the 35B model 0.7 GB. That is 30,720 extra tokens of num_ctx across four slots, so per thousand tokens of num_ctx the 4B pays 589 MB and the 35B pays 23 MB, a factor of 26.</p><p>The generalisation is duller than either sweep suggests: resident memory is weights plus a KV reservation, and which term dominates depends on the model’s attention geometry and on a context setting you chose. The consequence is more surprising. On this 36 GB box the 35B model can hold a longer context than the 4B one. If your instinct is that a smaller model leaves more room for context, that instinct is backwards for a large class of current models. I only measured the 35B out to 32,768, so the size of the gap beyond that is extrapolation. The ordering is not.</p><h3>Where these numbers stop being true</h3><p>One machine, one serving runtime, one synthetic corpus, and a harness whose shape I chose. Each of those bounds the result, and it is worth being specific about which way each one cuts.</p><p>A server built for throughput, as against a laptop daemon, would push the saturation point in the batching section further right, though the shape of the curve should hold, because what saturates is memory bandwidth against reserved KV. A production agent with longer tool outputs and shorter judged answers would move the overhead share up, not down, since every one of those changes adds tokens to be read and not written. Both of the shortcuts I declared at the top cut the same way: feeding the judge and the outbound guard the real answer, and handing compaction a transcript that carries the plan and the answer rather than just the question and the documents, would each move the share up. The prefix-cache result is the most portable of the set, because it depends on a property every serving runtime with a KV cache shares, and the least portable is the exact 59% figure, which is a property of the harness I wrote.</p><p>If you want the number that applies to your system, copy the method and not the harness. Tag each call as functional or overhead, charge it for prefill and decode, then read the ledger. If you are instrumenting with OpenTelemetry, the attribute set Biswas proposes already carries input_tokens, output_tokens and cost for every model invocation, which is three fields that cannot answer this question on hardware you own. Two more, prefill and decode duration, would make occupancy computable on a box where cost is zero.</p><h3>What it cost to find out</h3><pre>#### OUTPUT ####<br>  item                                          value<br>  --------------------------------------------------------<br>  hardware                        Apple M3 Max, 36 GB<br>  models downloaded                           30.2 GB<br>  model invocations                         about 450<br>  experiments                                       5<br>  wall clock, measurement runs                12m 40s<br>  daemon restarts, to vary NUM_PARALLEL             2<br>  hypotheses written down in advance                2<br>  hypotheses that survived                          0</pre><p>Neither prediction survived. The overhead share did not climb with session length, and resident memory is not governed by the context window in the way four convincing rows suggested. Adding a model corrected both. Thinking harder about the ones I already had would not have.</p><h3>What I would cut first</h3><p>I set out to check whether an agent harness is worth what it costs when nobody is billing you per token. On four models it took between 52.8% and 61.4% of the GPU, and on every one of them the output-token column pointed the other way while the input-token column overshot.</p><p>The routing fix people reach for first does not work. Sending the overhead to a smaller model looks appealing until you notice that qwen3:1.7b carries essentially the same overhead share as the 4B, 59.1% against 59.3%, so the shape of the problem is unchanged, and that a second resident model costs its full weights out of a memory budget you have already spent.</p><p>The measurements point at prefill instead, because that is where the overhead sits.</p><ul><li>Keep the context append-only. Prepending anything costs 31x on every subsequent call, the largest single number in this post.</li><li>Move retrieved documents below the stable part of the prompt, not above it, so the cache survives a change in retrieval results.</li><li>Keep the guardrails off the transcript. Mine only ever saw the question, and they were the two cheapest rows in the ledger at 2.9% and 2.8%. I never ran the expensive version, so take this as what the ledger implies and not as something I measured.</li><li>Judge on a sample. The judge wrote at most 16 tokens a call and took 23.3% of the box.</li><li>Size capacity with the harness switched on, since it costs headroom and not a fixed percentage.</li></ul><p>Both wrong predictions came from the same mistake. I counted model invocations as units of work, when the box is charging for tokens read and memory reserved. So count what your agent re-reads, and check what your serving runtime has reserved on your behalf.</p><p>The harness is <a href="https://chaarangan.github.io/assets/code/agent-capacity-harness.py">466 lines of standard library</a> and needs a running Ollama daemon and the four models pulled; the <a href="https://chaarangan.github.io/assets/code/agent-capacity-results.txt">full output of every run is here</a>.</p><p><em>Originally published at </em><a href="https://chaarangan.github.io/blog/a-per-token-cost-of-zero-is-not-a-cost-of-zero/"><em>https://chaarangan.github.io</em></a><em> on September 6, 2026.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c24a447137f2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/a-per-token-cost-of-zero-is-not-a-cost-of-zero-charangan-vasantharajan-c24a447137f2">A per-token cost of zero is not a cost of zero</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Tesseract OCR With JNA Wrapper]]></title>
            <link>https://medium.com/swlh/tesseract-ocr-with-java-jna-8b046dafcc85?source=rss-c3d6dc76f97c------2</link>
            <guid isPermaLink="false">https://medium.com/p/8b046dafcc85</guid>
            <category><![CDATA[tesseract]]></category>
            <category><![CDATA[ocr]]></category>
            <category><![CDATA[tess4j]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[maven]]></category>
            <dc:creator><![CDATA[Charangan Vasantharajan]]></dc:creator>
            <pubDate>Thu, 03 Dec 2020 17:37:11 GMT</pubDate>
            <atom:updated>2021-01-19T08:56:46.947Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ElBZi6veyEg-YYqspGmZ_Q.jpeg" /></figure><h4><strong>What is OCR?</strong></h4><p>Before going to study Tesseract, we need to know what OCR(Optical Character Recognition) is. In simple words, OCR is the process of extracting the plain text or hOCR from the given document, such as images(like JPG, JPEG, TIFF) and PDFs. After locating the text in the file, OCR will read and recognize the text character in the subsequent steps by using OCR engines and return it. This is how OCR helps machines to identify the text from the given file.</p><h4><strong>Introduction to Tesseract</strong></h4><p>Tesseract is an open-source <a href="https://en.wikipedia.org/wiki/Optical_character_recognition"><strong><em>text recognition (OCR)</em></strong></a> Engine written in c/c++ and works on Windows, macOS, and Linux, and comes under <a href="http://www.apache.org/licenses/LICENSE-2.0"><strong><em>Apache 2.0 License</em></strong></a>. It was initially designed by Hewlett Packard in 1985 then later released as an Open Source in 2005. After that, Google sponsored to develop and maintain Tesseract from 2006.</p><p>It can be used directly using an API to extract printed text from files, has Unicode(UTF-8) support, and recognizes more than 100 languages which you can refer to <a href="https://tesseract-ocr.github.io/tessdoc/Data-Files-in-different-versions.html"><strong><em>here</em></strong></a>. Tesseract project does not have a built-in GUI application, if you need GUI, find one from several available 3rdParties such as VietOCR, OCR2Text, dpScreenOCR, NeOCR, etc. You can get more about 3rdParties <a href="https://tesseract-ocr.github.io/tessdoc/User-Projects-%E2%80%93-3rdParty.html"><strong><em>here</em></strong></a>.</p><p>Developers can use the Tesseract API to build their own application under a provided license. If you need to use Tesseract with other programming languages, you need to use Tesseract <a href="https://tesseract-ocr.github.io/tessdoc/AddOns.html#tesseract-wrappers"><strong><em>wrappers</em></strong></a>. I am going to bind Tesseract with Java in my example, so I prefer to use <a href="http://tess4j.sourceforge.net/"><strong><em>Tess4J</em></strong></a> as my JNA wrapper.</p><h4><strong>Latest Release — Tesseract 4.1.1</strong></h4><p>Migrating from version 3 to 4.0x+, Tesseract added a new OCR engine based on <a href="https://en.wikipedia.org/wiki/Long_short-term_memory"><strong><em>LSTM</em></strong></a> neural networks for text recognition, which provides better working experience on x86/Linux, improved accuracy with higher speed, and provides output in different formats such as, plain text, hOCR (HTML), PDF, invisible-text-only PDF, TSV.</p><p>The latest version of Tesseract source code is available from the <a href="https://github.com/tesseract-ocr/tesseract/tree/master"><strong><em>master branch</em></strong></a><strong><em> </em></strong>on GitHub.</p><h4><strong>Tesseract Wrapper — Tess4J</strong></h4><p><a href="http://tess4j.sourceforge.net/"><strong><em>Tess4J</em></strong></a> is a Java JNA wrapper for Tesseract OCR API released and licensed under <a href="http://www.apache.org/licenses/LICENSE-2.0.html"><strong><em>Apache 2.0</em></strong></a> and is also available from SourceForge(<a href="http://mvnrepository.com/artifact/net.sourceforge.tess4j"><strong><em>Maven Central Repository</em></strong></a>). The library provides OCR support for TIFF, JPEG, GIF, PNG, and BMP image formats, Multi-page TIFF images, and PDF document format.</p><p><a href="http://tess4j.sourceforge.net/docs/docs-4.4/"><strong><em>See</em></strong></a> Tess4J Documentation — Link to Tess4J v4.4 which I am going to use in my example.</p><h4><strong>Tesseract Competitors</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/265/1*UTIrHJ9s4zEb0ggqa6JjzA.png" /><figcaption>source: <a href="https://images.app.goo.gl/R9eo161fYLs3Z3uz5">https://images.app.goo.gl/R9eo161fYLs3Z3uz5</a></figcaption></figure><p><strong>ABBYY FineReader - </strong>It was developed at ABBYY in 1993</p><p>Features:</p><ol><li>Supports more than 190 languages</li><li>Outputs to MS Office, RTF, HTML, searchable PDF, and plain text</li><li>Can integrate through Java SDK</li><li>Custom training supported</li><li>Accuracy 0.9 out of 10</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*L_dlRGqHJTRizORoHE8WYQ.jpeg" /><figcaption>source: <a href="https://images.app.goo.gl/WbX14NhQuqhajMUk7">https://images.app.goo.gl/WbX14NhQuqhajMUk7</a></figcaption></figure><p><strong>Google Cloud Vision API -</strong>This API was launched in 2016 by Google</p><p>Features:</p><ol><li>Supports more than 50 languages</li><li>Google Cloud Vision API renders their outputs as JSON</li><li>Can integrate through REST API, and can integrate with Google Images and Google SafeSearch</li><li>Custom training not supported</li><li>Accuracy 0.8 out of 10</li></ol><h4><strong>Setup a Linux Environment for Tesseract</strong></h4><p>There are two steps to set up the environment. First, we want to install the Tesseract OCR engine and trained data files installation.</p><h4><strong>Tesseract Installation</strong></h4><p>The Tesseract OCR engine package is generally called “<strong>tesseract-ocr”</strong> Thus you can install the latest version of Tesseract 4.1.1 through the terminal and its developer tools on Ubuntu Focal 20.04.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5121951b113b83ce73df1679004d1edf/href">https://medium.com/media/5121951b113b83ce73df1679004d1edf/href</a></iframe><p>And also we need to install the command line program “<strong>libtesseract-dev”</strong> to work with the Tesseract OCR engine.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1f6e03c65ee664adc4857398236fcccb/href">https://medium.com/media/1f6e03c65ee664adc4857398236fcccb/href</a></iframe><p>Then we need to install tesseract built binaries(supported languages and scripts) that are available directly from the Linux distributions through <strong>snapd</strong> by running the following command. If you do not install <strong>snapd</strong>, you have to run the below command before installing tesseract built binaries.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a395430dfff9640b154b6d4290b18452/href">https://medium.com/media/a395430dfff9640b154b6d4290b18452/href</a></iframe><p>Then run</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/420fcb6b7eeb051ba3d20b133e8a3644/href">https://medium.com/media/420fcb6b7eeb051ba3d20b133e8a3644/href</a></iframe><p>You can find other installation methods for various Operating systems <a href="https://tesseract-ocr.github.io/tessdoc/Installation.html"><strong><em>here</em></strong></a>.</p><h4><strong>Traineddata Installation</strong></h4><p>Tesseract contains a new neural network-based recognition engine that requires significantly more training data to deliver significantly higher accuracy contents. So we have to train our neural network application to return a better engine with higher accuracy. Normally, it takes a few days to a couple of weeks. So, I am going to choose to use an existing traineddata set that is trained on about 400000 text lines spanning about 4500 fonts.</p><p>There are three sets of traineddata files compatible with Tesseract 4.0x+, but I choose <strong>tessdata</strong> for my installation due to its support for the legacy recognizer. This traineddata is faster than other traineddata sets and has better accuracy. Below I provide some language links to get their traineddata.</p><p>English: <a href="https://github.com/tesseract-ocr/tessdata/raw/4.00/eng.traineddata"><strong><em>Download</em></strong></a></p><p>Tamil: <a href="https://github.com/tesseract-ocr/tessdata/raw/4.00/tam.traineddata"><strong><em>Download</em></strong></a></p><p>Sinhala: <a href="https://github.com/tesseract-ocr/tessdata/raw/4.00/sin.traineddata"><strong><em>Download</em></strong></a></p><p>You can also get other languages’ traineddata sets from <a href="https://tesseract-ocr.github.io/tessdoc/Data-Files.html"><strong><em>here</em></strong></a>.</p><p>We have just finished the installation part!</p><h4><strong>Let’s create a simple Java Project in IntelliJ IDEA</strong></h4><p>Create a new project based on Maven and create an empty class. After creating the class my folder structure will be like this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*29n3Zdsu66igEGOequESzw.png" /><figcaption>Creating a Project based on Maven</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/488/1*m0MrMDjrc59bpIPc90ExgA.png" /><figcaption>Folder Structure after creating the project</figcaption></figure><p>Modify your project’s pom.xml file to add the below dependency under dependencies element in pom.xml file to enable Maven.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/84edfcdfcb448fd2419b31763aa468a9/href">https://medium.com/media/84edfcdfcb448fd2419b31763aa468a9/href</a></iframe><p>After adding the above dependency, my pom.xml file will be like this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/948/1*F8s5JUvc_8DE5P2iTzEBBQ.png" /><figcaption>Modified pom.xml</figcaption></figure><p>Once we have an empty class, we can start adding some code to it. Here is the import statement for the instantiation of the Tesseract object.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3dab5cbafb91b6cf264a79cb3b4ccde4/href">https://medium.com/media/3dab5cbafb91b6cf264a79cb3b4ccde4/href</a></iframe><p>and also we need to import the below statement to avoid exceptions rising while recognizing the text.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9c71f8f1cd5dfa49e1fb9332133b23cd/href">https://medium.com/media/9c71f8f1cd5dfa49e1fb9332133b23cd/href</a></iframe><p>I am going to use files for recognition which are on my local computer. So I need to import the below statement to handle files.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b5a8f8e758a45dbad4511a28ed9300fb/href">https://medium.com/media/b5a8f8e758a45dbad4511a28ed9300fb/href</a></iframe><p>Now, my IntelliJ IDEA view is</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/526/1*oryWigNxn9VbOUe1ibS9Pg.png" /><figcaption>After all, packages imported</figcaption></figure><p>I will define a static method for Tesseract inside the class and Inside this method, make a new instance of Tesseract from the Maven library.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f37ea79fde7651128dcda91d97f72961/href">https://medium.com/media/f37ea79fde7651128dcda91d97f72961/href</a></iframe><p>Next, I will add traineddata details to this instance like where the training data for this library can be found. As I mentioned earlier, I have downloaded the traineddata for English and stored it on my Desktop. So add the below two lines of code after instance creation inside the method.</p><p><strong>Note</strong>: According to your traineddata file’s directory, the path will change.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c099c09507a603c41b1fb77bb230df8d/href">https://medium.com/media/c099c09507a603c41b1fb77bb230df8d/href</a></iframe><p>Finally, return the instance. Now my screen will be:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/533/1*LU0YMmuXCyc_6urpkRmi-w.png" /><figcaption>Return Plain Text</figcaption></figure><p>The above method returns the text as plain text from the resource. If you want to return the result as an HTML, you need to tell Tesseract that the output we need is in the format something called the hOCR(HTML). Basically, the hOCR format is a simple XML-based format.</p><p>We can make it an hOCR format by adding the below statement above the return statement.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/77776bc5160acd6a0d2d496159a8f1f0/href">https://medium.com/media/77776bc5160acd6a0d2d496159a8f1f0/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*7FiGW4O94rhwK3Cu5LNt4g.png" /><figcaption>Return hOCR Format</figcaption></figure><p>Finally, I put the main function below to make it usable and call the static method inside it.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c8aaebe430e02f49c7360202c9887e60/href">https://medium.com/media/c8aaebe430e02f49c7360202c9887e60/href</a></iframe><p>Now, what we have to do is provide a file to Tesseract which it can parse and read its Text. In this example, I am going to try with an image that is stored in my Ubuntu Desktop.</p><p>So, after knowing about the input file path, creating a new file instance, and setting the file path, I also need to pass the file to tesseract for recognition and write a java print statement to get the output. Finally, your code view will be:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/788/1*KlxR5XtBKK3DubJWuuLAvA.png" /><figcaption>With the main function</figcaption></figure><p>Since I settled all things correctly, still IntelliJ IDEA shows some errors in my files. It’s nothing but, we have to reload the project to load all dependency packages and sources.</p><p>Right-click on<strong> <em>pom.xml file -&gt; Maven -&gt; Reload Project</em></strong></p><p>Now, the problem is solved.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/787/1*nLa14usno9kixGJ5WbROTQ.png" /><figcaption>No errors after reloading the project</figcaption></figure><p>If we look closely, there is nothing we did much. That is the power of this wrapper for the Tesseract library I am provided with. Now we are ready to run our sample OCR program.</p><p><strong>Note</strong>: Confirm that your project is set to the latest version of Java like Java 11. If you get an error regarding the Java version, you can simply set it to the latest version.</p><p><strong><em>Go File -&gt; Project Structure</em></strong>.</p><p>You will see a dialog box given below, Make sure that the Java version under Project and Modules section is set to the latest version of Java like Java 11.</p><p><strong><em>Project Setting -&gt; Project</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iOFIYTZw0OQt02OkyV7Wig.png" /><figcaption>Project</figcaption></figure><p><strong><em>Project Setting -&gt; Modules</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1014/1*8ZpxRtsFLcJnQY3Iylxp7w.png" /><figcaption>Module</figcaption></figure><p>And also make sure that the Java Compiler setting also by pressing Ctrl + Alt + S. You will popup a dialog box, then navigate to Java Compiler as given below and check if the target bytecode version is set above to 1.5 for your project.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pF1j9nWB__b4XMm3xbzt7w.png" /><figcaption>Target bytecode</figcaption></figure><p>Fine. Now the time to take a test drive through our sample application, so I am going to use google’s logo as my sample input.</p><p><strong>Note</strong>: Try to avoid PNG images and use JPG/JPEG images instead if you’re working with images at all. This is because Tesseract is very bad at reading PNG images due to its compression techniques.</p><figure><img alt="charangan tesseract ocr project" src="https://cdn-images-1.medium.com/max/385/1*DtzoYYdEz9cDYbRVta1SaQ.jpeg" /><figcaption>source: <a href="https://www.putnamlib.org/images/google.jpg/@@images/image.jpeg">https://www.putnamlib.org/images/google.jpg/@@images/image.jpeg</a></figcaption></figure><p>Once I run the project, I get this simple output as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/539/1*HX3Syfd9X9fAyabi_JhWwQ.png" /><figcaption>Output</figcaption></figure><p>For your reference, all the source code of the sample application is available on my <a href="https://github.com/Chaarangan/TesseractDemo"><strong>Github</strong></a>.</p><h4>Limitations of Tesseract</h4><p>Since Tesseract is an OCR engine, it works well for only clean foreground text. For an instance, if we take an image which is having noise in the background, then we could not get an accurate result in the OCR test. So, there are many sorts of reasons for not getting good quality output from Tesseract. The image which has minimum requirements for its size, contrast, and lightning, recognizes better in the test. The lower quality images require preprocessing to improve the recognization results, such as scale appropriately, convert it to much contrast as possible, and the converted text to be horizontally aligned. Except for the following limitations, Tesseract OCR is a powerful OCR engine.</p><ul><li>Lower quality recognizations may output poor quality OCR results.</li><li>The results do not have information about the text font family.</li><li>It doesn’t work with files that having artifacts including partial occlusion, distorted perspective, and complex background.</li><li>Poor in recognize images which having handwriting letters.</li><li>When the input file contains languages other than the given in set function as an argument(shown below in the example), results in poor quality results.</li><li>Two-column documents fail in the OCR test. That’s why it is not always best at recognizing the natural follow of input files and the result text will seem like the joined text across columns.</li></ul><p><strong>Conclusion</strong></p><p>In this article, we have studied a lot of things about <strong>Tesseract</strong>, <strong>Tess4J</strong>, and some other sort of things and we have made a very simple Tesseract OCR engine which allows us to read text from various format files like PDF and image files(except PNG).</p><p>In addition to this, we have to pre-process the file before recognizing the input file to overcome the <strong>Tesseract Limitations</strong> and get better results. So, I will write about Tesseract Pre-Processing in my next article.</p><h4><strong>References:</strong></h4><ol><li><a href="https://tesseract-ocr.github.io/tessdoc/Home.html"><em>Tesseract Official Page</em></a></li><li><a href="http://tess4j.sourceforge.net/changelog.html"><em>Tess4J Official Page</em></a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8b046dafcc85" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/tesseract-ocr-with-java-jna-8b046dafcc85">Tesseract OCR With JNA Wrapper</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[THINGS TO CONSIDER WHEN CHOOSING A CAREER PATH IN IT]]></title>
            <link>https://charangan.medium.com/things-to-consider-when-choosing-a-career-path-in-it-d82b6ad9b3c7?source=rss-c3d6dc76f97c------2</link>
            <guid isPermaLink="false">https://medium.com/p/d82b6ad9b3c7</guid>
            <category><![CDATA[ict]]></category>
            <category><![CDATA[network]]></category>
            <category><![CDATA[careers]]></category>
            <category><![CDATA[cybersecurity]]></category>
            <category><![CDATA[it]]></category>
            <dc:creator><![CDATA[Charangan Vasantharajan]]></dc:creator>
            <pubDate>Wed, 18 Nov 2020 05:11:33 GMT</pubDate>
            <atom:updated>2020-11-18T05:14:14.414Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*FvCZLQqQh2DJSrueqlcieA.png" /></figure><p>In the current era, everyone must think about choosing a career path in IT in order to make their life better.</p><p>I have contributed to the below article along with <a href="https://medium.com/@sanjeepan23"><strong>Sanjeepan Sivapiran</strong></a>, and <a href="https://medium.com/@thivvyank"><strong><em>Thivvyan Karuneswaran</em></strong></a><strong><em> </em></strong>which was published through Youth Govern SL.</p><p>Link</p><p><a href="https://youthgovernsl.medium.com/things-to-consider-when-choosing-a-career-path-in-it-50e635e9f935">https://youthgovernsl.medium.com/things-to-consider-when-choosing-a-career-path-in-it-50e635e9f935</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d82b6ad9b3c7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Send an email using Nodemailer and Gmail in Node.js Express.js]]></title>
            <link>https://medium.com/swlh/send-an-email-using-nodemailer-and-gmail-in-node-js-express-js-34523d5e0aa4?source=rss-c3d6dc76f97c------2</link>
            <guid isPermaLink="false">https://medium.com/p/34523d5e0aa4</guid>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[nodemailer]]></category>
            <category><![CDATA[email]]></category>
            <category><![CDATA[expressjs]]></category>
            <category><![CDATA[gmail]]></category>
            <dc:creator><![CDATA[Charangan Vasantharajan]]></dc:creator>
            <pubDate>Tue, 14 Jul 2020 21:12:05 GMT</pubDate>
            <atom:updated>2020-07-14T21:12:05.921Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ovGD0BgtbRTncM4luzTNlQ.jpeg" /></figure><h3><strong>Nodemailer</strong></h3><p>Nodemailer is a module for Node.js applications to send emails between different email hosts using different transport methods. It is a single module with zero dependencies with a heavy focus on security (TLS/STARTTLS). It supports Unicode and Windows. We can also send text and HTML content.</p><p>Migrating Nodemailer from v2 to v3 has dropped some features that were available in v2 and introduced some new ones.</p><ul><li>All dependencies were dropped</li><li>All templating is gone</li><li>OAuth2 authentication</li><li>Delivery status notification</li><li>Calendar Events</li></ul><p><strong>Requirement</strong></p><p>Node.js v6.0.0 or newer.</p><p>More about Nodemailer - refer: <a href="https://nodemailer.com/about/">nodemailer.com</a></p><h3><strong>SMTP</strong></h3><p><strong>What is SMTP</strong>? <strong>SMTP</strong> is part of the application layer of the TCP/IP protocol. Using a process called “store and forward”, <strong>SMTP</strong> moves your email on and across networks. It <strong>works</strong> closely with something called the Mail Transfer Agent (<strong>MTA</strong>) to send your communication to the right computer and email inbox.</p><p>It is used as a transporter in Nodemailer for delivering messages between different email hosts as a protocol. Even if every Gmail delivery provider mainly focused on their API based sending, but they also support SMTP based sending. APIs might have more features but using the SMTP method, we only need to change the configuration options to replace one provider with another. It is easy to implement compared to other methods.</p><p>More about SMTP — refer: <a href="https://nodemailer.com/smtp/">Nodemailer SMTP</a></p><p><strong>Install Nodemailer from </strong><a href="https://www.npmjs.com/package/nodemailer"><strong>npm</strong></a></p><pre>npm install nodemailer</pre><p><strong>Install SMTP transport module for Nodemailer with npm</strong></p><pre>npm install nodemailer-smtp-transport</pre><p><strong>Require to your script</strong></p><pre>var nodemailer <strong>=</strong> require(‘nodemailer’);</pre><pre>var smtpTransport <strong>=</strong> require(‘nodemailer-smtp-transport’);</pre><p><strong>Create a Nodemailer transport object</strong></p><pre>var transporter <strong>=</strong> nodemailer.createTransport(smtpTransport(options))</pre><p>Here, <strong>options</strong> define connection data and it includes</p><p><strong>options.port</strong></p><p>Port to connect to (defaults to 25 or 465)</p><p><strong>options.host</strong></p><p>Hostname or IP address to connect to (defaults to ‘localhost’). If you are using SMTP with Google, you have to put ‘smtp.gmail.com’ as a host.</p><p><strong>options.secure</strong></p><p>It defines if the connection should use SSL (if true) or not (if false). if you are connecting to port 465 it should be true. For port 587 or 25 keep it <em>false.</em></p><p><strong>options.auth</strong></p><p>It defines authentication data,</p><p><strong>Type</strong> — indicates the authentication type, defaults to ‘login’, other option is ‘oauth2’</p><p><strong>user</strong> — Username</p><ul><li><strong>pass — </strong>Password for the user if normal login is used</li></ul><p><strong>options.ignoreTLS</strong></p><p>It turns off STARTTLS support if true</p><p><strong>options.name</strong></p><p>The optional hostname of the client, used for identifying the server</p><p><strong>options.localAddress</strong></p><p>It is the local interface to bind to network connections</p><p><strong>options.connectionTimeout</strong></p><p>How many milliseconds to wait for the connection to establish. (default is 2 minutes)</p><p><strong>options.authMethod</strong></p><p>It defines the preferred authentication method, eg. ‘PLAIN’</p><p><strong>options.tls</strong></p><p>It defines additional options to be passed to the socket constructor, eg. <em>{rejectUnauthorized: true}</em></p><blockquote><strong>Before you use your SMTP Gmail server, make sure that ‘less secure’ is on in your Gmail settings.</strong></blockquote><p>Refer: <a href="https://support.google.com/accounts/answer/6010255?hl=en">Google Supports</a></p><ol><li><strong>Setup Nodemailer</strong></li></ol><pre>let transporter = nodemailer.createTransport(smtpTransport({<br>     service: ‘gmail’,<br>     host: ‘smtp.gmail.com’,<br>     auth: {<br>         user: ‘user.gmail’,<br>         pass: ‘user.password’<br>     }<br>}));</pre><p><strong>2.1 Sending plain text</strong></p><pre>const mailOptions = {<br>     from: ‘sender.gmail’,<br>     to: ‘receiver.email’,<br>     subject: ‘subject’,<br>     text: ‘message‘<br>};</pre><p><strong>2.2 Sending HTML content</strong></p><pre>const mailOptions = {<br>     from: ‘sender.gmail’,<br>     to: ‘receiver.email’,<br>     subject: ‘subject’,<br>     html: ‘message‘<br>};</pre><p><strong>3. Sending mail and get sent message</strong></p><pre>transporter.sendMail(mailOptions, function(error, info){<br>     if (error) {<br>         console.log(error);<br>     } else {<br>         console.log(‘Email sent: ‘ + info.response);<br>     }<br>});</pre><p><strong>Example Code</strong></p><ol><li>Setup a form to get details</li></ol><pre>&lt;form role=&quot;form&quot; action=&quot;/send-mail&quot; method=&quot;POST&quot;&gt;<br>    &lt;div class=&quot;form-group&quot;&gt;<br>        &lt;label for=&quot;to&quot;&gt;To&lt;/label&gt; <br>        &lt;input type=&quot;text&quot; class=&quot;form-control&quot; id=&quot;to&quot;    <br>        name=&quot;to&quot; placeholder=&quot;Receiver Email&quot; required&gt;<br>    &lt;/div&gt;<br>    &lt;div class=&quot;form-group&quot;&gt;<br>        &lt;label for=&quot;subject&quot;&gt;Subject&lt;/label&gt;<br>        &lt;input type=&quot;text&quot; class=&quot;form-control&quot; id=&quot;subject&quot;    <br>        name=&quot;subject&quot; placeholder=&quot;Subject&quot; required&gt;<br>    &lt;/div&gt;<br>    &lt;div class=&quot;form-group&quot;&gt;<br>        &lt;label for=&quot;message&quot;&gt;Message&lt;/label&gt;        <br>        &lt;textarea class=&quot;form-control&quot; id=&quot;message&quot;<br>        name=&quot;message&quot; placeholder=&quot;Message&quot; required&gt;&lt;/textarea&gt;<br>    &lt;/div&gt;<br>    &lt;div class=&quot;text-center&quot;&gt;<br>        &lt;button type=&quot;submit&quot; id=&quot;submit&quot; name=&quot;submit&quot; class=&quot;btn <br>        btn-primary btn-sm&quot;&gt;Send&lt;/button&gt;<br>    &lt;/div&gt;<br>&lt;/form&gt;</pre><p>2. <strong>Get ‘send-mail’ route and send email</strong></p><pre>//require<br>var express = require(&quot;express&quot;), <br>    router = express.Router(),   <br>    smtpTransport = require(&#39;nodemailer-smtp-transport&#39;);</pre><pre>//setup nodemailer<br>const nodemailer = require(&#39;nodemailer&#39;);<br>let transporter = nodemailer.createTransport(smtpTransport({    <br>     service: &#39;gmail&#39;,<br>     host: &#39;smtp.gmail.com&#39;, <br>     auth: {        <br>          user: &#39;user.email&#39;,        <br>          pass: &#39;user.password&#39;    <br>     }<br>}));<br></pre><pre>//get route to send mail, from form<br>router.post(&quot;/send-mail&quot;, function(req,res){<br>     var to = req.body.to,<br>         subject = req.body.subject, <br>         message = req.body.message;</pre><pre>     //options<br>     const mailOptions = {<br>          from: ‘example@gmail.com’,<br>          to: to,                   // from req.body.to<br>          subject: subject,         //from req.body.subject<br>          html: message             //from req.body.message</pre><pre>      };</pre><pre>     //delivery<br>     transporter.sendMail(mailOptions, function(error, info){<br>          if (error) {<br>              console.log(error);  <br>          } else {     <br>              console.log(&#39;Email sent: &#39; + info.response);  <br>          }   <br>     });<br>});<br></pre><pre>module.exports = router;</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=34523d5e0aa4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/send-an-email-using-nodemailer-and-gmail-in-node-js-express-js-34523d5e0aa4">Send an email using Nodemailer and Gmail in Node.js Express.js</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The FAQ in Bellman-Ford algorithm, Dijkstra’s Algorithm, and Dynamic Programming]]></title>
            <link>https://charangan.medium.com/the-faq-in-bellman-ford-algorithm-dijkstras-algorithm-and-dynamic-programming-a1c5f4734524?source=rss-c3d6dc76f97c------2</link>
            <guid isPermaLink="false">https://medium.com/p/a1c5f4734524</guid>
            <category><![CDATA[dynamic-programming]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[bellman-ford-algorithm]]></category>
            <category><![CDATA[dijkstra]]></category>
            <category><![CDATA[data-structuresalgorithms]]></category>
            <dc:creator><![CDATA[Charangan Vasantharajan]]></dc:creator>
            <pubDate>Fri, 07 Feb 2020 22:00:37 GMT</pubDate>
            <atom:updated>2020-02-07T22:00:37.413Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/864/1*HNN5i0re16c9BvDCN-GASA.png" /></figure><p><strong>(A) Why the Bellman-Ford algorithm cannot handle negative weight cycled graphs as input?</strong></p><p>Generally, the Bellman-Ford algorithm gives an accurate shortest path in <strong>(N-1)</strong> iterations where N is the number of vertexes, but if a graph has a negative weighted cycle, it will not give the accurate shortest path in <strong>(N-1)</strong> iterations. It acquires more iteration and reduces the cost, but it does not go to end. Therefore we cannot handle the negative weighted cycle by using the Bellman-Ford algorithm.</p><p><strong>(B) What is the purpose of the priority queue in Dijkstra’s Algorithm?</strong></p><p>The priority queue selects the next vertex so as to ensure the shortest paths in a weighted graph by adding all nodes with their distance decorations as the priority. If we use the <strong>FIFO</strong> queue instead, we cannot be able to account for the arbitrary edge weights.</p><p><strong>(C) Key properties an optimization problem should have if the dynamic programming approach is applicable to the problem.</strong></p><p><strong>1. <em>Optimal Substructure</em></strong></p><p>A given problem is said to have the optimal substructure property if an optimal solution of the given problem can be obtained by using optimal solutions of its subproblems. Or we can say that a globally optimal solution can be obtained by combining its local optimal solutions.</p><p><strong>2. <em>Overlapping Subproblems</em></strong></p><p>It happens when an algorithm revisits the same problem over and over. Unlike divide and conquer, dynamic programming store the result of a particular subproblem, and then reuse it when revisit. From this approach of dynamic programming, it runs faster compared to divide and conquer.</p><p><strong>(D) Why greedy solutions would generally run faster compared to dynamic programming solutions.</strong></p><p>The greedy algorithm runs through a one set of subproblems whereas dynamic programming would solve all subproblems and choose one which leads to an optimal solution. Therefore the greedy algorithm takes less time than dynamic programming.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a1c5f4734524" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Running time of Breath-First-Search — Proof]]></title>
            <link>https://charangan.medium.com/running-time-of-breath-first-search-proof-6a1d32aa5dce?source=rss-c3d6dc76f97c------2</link>
            <guid isPermaLink="false">https://medium.com/p/6a1d32aa5dce</guid>
            <dc:creator><![CDATA[Charangan Vasantharajan]]></dc:creator>
            <pubDate>Tue, 21 Jan 2020 20:12:03 GMT</pubDate>
            <atom:updated>2020-01-21T20:12:03.120Z</atom:updated>
            <content:encoded><![CDATA[<h3>Running time of Breath-First-Search — Proof</h3><p>Running time of Breath-First-Search is <strong>(V+E)</strong>, where V — number of vertexes and E — number of edges.</p><p><strong>Pseudocode for BFS.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/635/1*rLH-SCEYEcu5aQXBBLSgyQ.jpeg" /></figure><p>· Enqueue and dequeue operations take <strong>O(1)</strong> time, so the total time for queuing is<strong> O(V)</strong>.</p><p>· Since each adjacency list is explored at most once the sum of all the adjacency list is <strong>O(E)</strong>. So the total time for exploring adjacency lists is <strong>O(E)</strong>.</p><p>Therefore the total running time of BFS is <strong>O(V+E)</strong>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6a1d32aa5dce" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is the advantage of using open addressing over chaining when implementing a Hash Table?]]></title>
            <link>https://charangan.medium.com/what-is-the-advantage-of-using-open-addressing-over-chaining-when-implementing-a-hash-table-2d401d5a403b?source=rss-c3d6dc76f97c------2</link>
            <guid isPermaLink="false">https://medium.com/p/2d401d5a403b</guid>
            <category><![CDATA[hash-table]]></category>
            <category><![CDATA[hashing]]></category>
            <category><![CDATA[data-structures]]></category>
            <category><![CDATA[chaining]]></category>
            <category><![CDATA[collision]]></category>
            <dc:creator><![CDATA[Charangan Vasantharajan]]></dc:creator>
            <pubDate>Wed, 08 Jan 2020 17:07:15 GMT</pubDate>
            <atom:updated>2020-02-20T01:57:14.215Z</atom:updated>
            <content:encoded><![CDATA[<p>There are two types of data structures used to store data differently.</p><ol><li><strong>Direct access table</strong> — The size of an array depends on the size of index values.</li><li><strong>Hash table</strong> — The size of an array does not depend on the size of index values.</li></ol><p><strong>What is a Hash Table?</strong></p><p>A hash table is an unordered data structure to store data uniformly across an array. It maps keys with their corresponding values, where each key has a unique index. It supports insert, search and delete operations with worst-case time complexity of O(1). We can take it out of these data from the array easily if we know the index of the data which we are going to take out. In addition to this, the overflow situation does not occur in hash tables.</p><p><strong>What is Collision?</strong></p><p>A hash table uses a hash function (normally hash functions are defined by division method, multiplication method, and universal hashing method) to determine hash code (Index) of any input record in a hash table.</p><p>While assigning, a hash function computes the same index value for more than one key. It is called hash collisions. Such collisions always handled mainly by two types of collision handling methods.</p><p>1. <strong>Separate Chaining using linked list (Open hashing)</strong></p><p>2. <strong>Open addressing (Closed hashing)</strong></p><p>1. Linear probing</p><p>2. Quadratic probing</p><p>3. Double hashing</p><p>4. Last-Come-First-Served hashing</p><p>5. Cuckoo hashing</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/410/1*m8pm0demNgJnaeN7W9JWew.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/327/1*N4tDbFUETrJam0ZpdbLWsQ.png" /><figcaption>Chaining using linked list vs Open Addressing</figcaption></figure><p><strong>What is the advantage of using open addressing over chaining when implementing a Hash Table?</strong></p><p><strong>Chaining</strong></p><p>Chaining is easy to implement effectively.</p><p>Easily delete a value from the table.</p><p>It uses less memory if the record is large compared to the open addressing.</p><p>Difficult to serialize data from the table.</p><p><strong>Open Addressing</strong></p><p>Open Addressing needs more computation to avoid clustering (better hash functions only).</p><p>Once the table becomes full, hash functions fail to terminate</p><p>Less memory requires for small record size.</p><p>Since open addressing do not use pointers, it can be easier to serialize</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2d401d5a403b" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>