<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Maede Jalali on Medium]]></title>
        <description><![CDATA[Stories by Maede Jalali on Medium]]></description>
        <link>https://medium.com/@theMajesticMj?source=rss-ee4389407aa5------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*6Nbgl-ZpJC6dGCS25AKgZA.jpeg</url>
            <title>Stories by Maede Jalali on Medium</title>
            <link>https://medium.com/@theMajesticMj?source=rss-ee4389407aa5------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 22:49:59 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@theMajesticMj/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[15 Metrics You Should Be Measuring If You Are Building a Conversational AI Assistant]]></title>
            <link>https://pub.towardsai.net/15-metrics-you-should-be-measuring-if-you-are-building-a-conversational-ai-assistant-63252bf4eabf?source=rss-ee4389407aa5------2</link>
            <guid isPermaLink="false">https://medium.com/p/63252bf4eabf</guid>
            <category><![CDATA[conversational-ai]]></category>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[business-metrics]]></category>
            <dc:creator><![CDATA[Maede Jalali]]></dc:creator>
            <pubDate>Thu, 16 Jul 2026 21:01:01 GMT</pubDate>
            <atom:updated>2026-07-16T21:01:01.780Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qZwqA3DtO79BOzxpKp65hg.png" /><figcaption>Image generated with ChatGPT. A chat interface along with a few dashboards each showing a different metric.</figcaption></figure><p>Conversational AI assistants are everywhere now. Customer support, internal tools, sales, onboarding, there is hardly a product category left that hasn’t bolted one on. And yet, for how widespread they have become, most of them are still falling significantly short of what they could be. The reasons vary, but one pattern I keep running into is consistent: <em>teams are not measuring the right things</em>. They ship the assistant, they watch a handful of numbers go up, and they convince themselves the product is working. Meanwhile, users are quietly having a terrible experience.</p><p>I ran into this firsthand with a startup I was consulting with. Their support assistant looked great on paper, ticket closure rate was high, escalations to human agents were low. The dashboard was green across the board. But their NPS scores told a completely different story. When we dug into why, the answer was hiding in metrics they had never thought to track: conversation length, number of back-and-forths, how long it was taking the assistant to actually resolve anything. <em>The bot was closing tickets, but only after exhausting the user with five or six unnecessary exchanges</em>. It lacked context, so it kept asking questions it should have already known the answers to.</p><p>That investigation changed how I think about measuring conversational AI. What follows are the metrics that in my opinion actually matter, the ones that tell you whether your assistant is genuinely working, not just technically responding.</p><h3>1. Net Promoter Score (NPS)</h3><h4><strong>What it is</strong></h4><p><strong>NPS</strong> measures how likely users are to recommend your product after interacting with the assistant. It is collected via a simple post-conversation survey: “How likely are you to recommend us to a friend or colleague?” on a scale of 0 to 10. Scores of 9–10 are promoters, 7–8 are passives, and 0–6 are detractors. Your NPS is the percentage of promoters minus the percentage of detractors.</p><h4><strong>Why it matters</strong></h4><p>NPS is the single metric that ties your assistant’s performance to actual business outcome. A support bot that closes tickets but leaves users frustrated will show up in your NPS before it shows up anywhere else. It is also one of the few metrics that captures the full arc of the interaction, not just whether the assistant answered, but whether the experience was good enough to talk about.</p><h4><strong>How to measure it</strong></h4><p>The model detects when the conversation has naturally ended and triggers an NPS survey dialog in the UI. Segment responses by conversation type, user cohort, and assistant version so you can isolate what is driving scores up or down.</p><blockquote>The following code is a simple demonstration of collecting user feedback based on the conversation’s status. Please avoid using it in production, as it is far from AI engineering best practices..</blockquote><pre>import gradio as gr<br>from openai import OpenAI<br>from pydantic import BaseModel<br><br>client = OpenAI()<br><br>class AssistantResponse(BaseModel):<br>    response: str<br>    conversation_ended: bool<br><br>def chat(message: str, history: list) -&gt; tuple:<br>    messages = [<br>        {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: &quot;You are a helpful support assistant.&quot;}<br>    ] + history + [{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: message}]<br>    completion = client.beta.chat.completions.parse(<br>        model=&quot;gpt-4o&quot;,<br>        messages=messages,<br>        response_format=AssistantResponse<br>    )<br>    parsed = completion.choices[0].message.parsed<br>    show_nps = parsed.conversation_ended<br>    return parsed.response, gr.update(visible=show_nps)<br><br>def submit_nps(score: int, history: list) -&gt; dict:<br>    if score &gt;= 9:<br>        category = &quot;promoter&quot;<br>    elif score &gt;= 7:<br>        category = &quot;passive&quot;<br>    else:<br>        category = &quot;detractor&quot;<br>    nps_result = {<br>        &quot;nps_score&quot;: score,<br>        &quot;category&quot;: category,<br>        &quot;conversation_length&quot;: len(history)<br>    }<br>    print(f&quot;NPS recorded: {nps_result}&quot;)<br>    return gr.update(visible=False)<br><br>with gr.Blocks() as demo:<br>    gr.Markdown(&quot;# Support Assistant&quot;)<br>    with gr.Column(visible=False) as nps_panel:<br>        gr.Markdown(&quot;### How likely are you to recommend our assistant?&quot;)<br>        nps_score = gr.Slider(minimum=0, maximum=10, step=1, label=&quot;Score (0-10)&quot;)<br>        submit_btn = gr.Button(&quot;Submit&quot;)<br>    chatbot = gr.Chatbot(type=&quot;messages&quot;)<br>    gr.ChatInterface(<br>        fn=chat,<br>        chatbot=chatbot,<br>        additional_outputs=[nps_panel]<br>    )<br>demo.launch()</pre><h3>2. Implicit User Satisfaction</h3><h4><strong>What it is</strong></h4><p>Unlike NPS, which you ask for directly, implicit satisfaction is inferred from user behavior during the conversation. Signals like whether the user rephrased the same question, expressed frustration, or dropped off mid-conversation all tell you something about how satisfied they were, without ever asking.</p><h4><strong>Why it matters</strong></h4><p>Not every user will fill in your NPS survey. Implicit signals give you satisfaction data on 100% of your conversations, not just the ones where someone bothered to rate you. It also helps you spot patterns across sessions, which flows consistently frustrate users, which ones resolve cleanly, and where your assistant is quietly losing people.</p><h4><strong>How to measure it</strong></h4><p>Run a nightly batch job over all conversations from the day. For each conversation, send the full transcript to a lightweight model with a carefully designed rubric. The rubric matters more than you’d think, a vague instruction like “was the user satisfied?” will return inconsistent scores. Be explicit: define what satisfied, neutral, and frustrated look like in terms of observable signals in the conversation. You can find the full batch job implementation at the end of this article.</p><h3>3. Conversion to Business Metric / Containment Rate</h3><h4><strong>What it is</strong></h4><p>This is the metric that connects your assistant directly to business outcomes. Conversion measures whether the assistant successfully moved the user toward a meaningful business goal, a completed purchase, a ticket closed without human escalation, a booking confirmed, a payment page reached. Containment rate is the support-specific version: what percentage of conversations were fully resolved by the assistant without involving a human agent.</p><h4><strong>Why it matters</strong></h4><p>This is the metric that exposes the gap between “<strong>the assistant responded</strong>” and “<strong>the assistant was useful</strong>.” A high containment rate with a low NPS, as we saw earlier, means your assistant is technically closing tickets but leaving users exhausted. Tracked together, conversion and containment give you the clearest picture of whether your assistant is actually delivering value or just occupying the space where value should be.</p><h4><strong>How to measure it</strong></h4><p>For conversion, instrument the key downstream actions, payment page visits, booking confirmations, form submissions, and tag every link your assistant sends to users with UTM parameters. Something as simple as ?utm_source=chatbot&amp;utm_medium=assistant&amp;utm_campaign=support on every outbound link makes all the difference when you are trying to attribute what the assistant is actually driving. Without it, those conversions are invisible in your analytics.</p><p>For containment and outcome classification, you do not need a separate pipeline. In the same nightly batch job you are already running for implicit satisfaction, extend the rubric to also ask the model whether the conversation ended in a conversion, a clean containment, an escalation, or an abandonment. One job, one model call per conversation, two metrics out. You can find the full implementation at the end of this article.</p><h3>4. Average Conversation Length</h3><h4><strong>What it is</strong></h4><p>Average conversation length measures the number of turns, a turn being one user message and one assistant response, it takes to reach a resolution. Turns are more useful than time because they are independent of how fast the user types.</p><h4><strong>Why it matters</strong></h4><p>This was the metric that cracked open the problem I described in the intro. The ticket closure rate looked fine, but conversation length told a different story, users were going through five or six exchanges to resolve something that should have taken two. Long conversations are not always bad; some tasks are genuinely complex. But when average length is high on conversations that should be simple, it is almost always a signal that the assistant is missing context, asking questions it should already know the answers to, or failing to understand the user’s intent on the first pass.</p><h4><strong>How to measure it</strong></h4><p>Store each message in your database with a <strong>conversation ID</strong>, <strong>a role field</strong> (user or assistant), and a <strong>timestamp</strong>. From there, measuring average turn count per conversation is a simple query, count user messages grouped by conversation ID, then average across the day. If you are already using <strong>Metabase</strong> or a similar <strong>BI</strong> tool, this becomes a dashboard you can monitor without writing any additional code. <strong>Segment by intent category</strong> if possible, because a billing question and a complex technical issue should not share the same benchmark. The signal is in the per-intent average, not the overall one.</p><p>There is no universal number that defines a normal conversation length. It depends entirely on the category and the type of business you are running. What matters is that you track it consistently over time. After a while a pattern will emerge, and that pattern will either seem rational given the complexity of your use case, or it won’t. If it doesn’t, you investigate and adjust. If it does, that pattern becomes your baseline, and any meaningful deviation from it becomes the signal worth paying attention to.</p><h3>5. Average Number of Tool Calls</h3><h4><strong>What it is</strong></h4><p>If your assistant uses tools, querying a database, calling an API, searching a knowledge base, looking up a user’s order; this metric tracks how many tool calls are made per conversation on average.</p><h4><strong>Why it matters</strong></h4><p>Tool calls are one of the clearest indicators of how efficiently your assistant is operating. Too many tool calls on a simple request means the assistant is either poorly prompted, lacking the right context upfront, or making redundant calls it should not need to make. Each unnecessary tool call adds latency, increases cost, and quietly degrades the user experience. Like conversation length, this metric rarely tells you something is wrong on its own, but tracked alongside turn count and NPS, it becomes a powerful diagnostic. When we looked at this metric alongside conversation length at the startup I mentioned earlier, the two told the same story from different angles.</p><h4><strong>How to measure it</strong></h4><p>If you are already using <strong>LangSmith</strong>, you already have this. LangSmith traces every run and logs each tool call within it, so you can see the exact number of tool calls per conversation, which tools were called, and in what order, without any additional instrumentation. If you are not using LangSmith, log each tool call with a conversation ID and tool name, and run the same kind of daily aggregation query you are already using for conversation length. The same Metabase dashboard can surface average tool calls per conversation alongside turn count, which makes it easy to spot when the two are moving together, a reliable sign that your assistant is working harder than it should.</p><h3>6. Average Cost Per Conversation</h3><h4><strong>What it is</strong></h4><p>Average cost per conversation measures how much each session with your assistant costs in terms of API usage, input tokens, output tokens, and tool calls combined. It is usually expressed as a cost per conversation or cost per resolved conversation, the latter being more meaningful since it accounts for whether the session actually delivered value.</p><h4><strong>Why it matters</strong></h4><p>This is the metric that keeps your unit economics honest. An assistant that costs $0.50 per conversation to run but only resolves 40% of tickets is effectively costing you $1.25 per resolution, and that changes the business case significantly. Tracking cost per conversation also surfaces inefficiencies that are invisible in quality metrics: unnecessarily long system prompts, redundant tool calls, or using a large expensive model for tasks a smaller one could handle just as well.</p><h4><strong>How to measure it</strong></h4><p>Every OpenAI API response includes a usage object with prompt_tokens and completion_tokens. Store both fields alongside the conversation ID and the model used in your messages table. From there, the cost calculation is straightforward: multiply token counts by the per-token price of the model, sum across all turns in a conversation, and you have your cost per session.</p><p>With those fields in your database, Metabase can do the rest. A simple calculated column — (prompt_tokens * input_price + completion_tokens * output_price) / 1,000,000 — gives you cost per turn, and averaging that grouped by conversation ID gives you cost per conversation. You can then cross-reference it with your resolution data from metric 3 to get <strong>cost per resolved conversation, which is the number that actually matters for your unit economics</strong>.</p><p>If you are using multiple models, a larger one for the assistant and a smaller one for the nightly batch jobs, track the model name per message so you can split costs cleanly in your dashboard.</p><h3>7. Fallback Rate</h3><h4><strong>What it is</strong></h4><p>Fallback rate measures how often your assistant fails to handle a request and falls back to a default response, something like “I’m not sure I understand, could you rephrase that?” or “I don’t have enough information to help with that.” It is the percentage of conversations that contain at least one fallback response out of all conversations.</p><h4><strong>Why it matters</strong></h4><p>A fallback is a small failure. One or two in a complex conversation might be acceptable, but a high fallback rate across your sessions is a direct signal that your assistant is encountering requests it was not designed or trained to handle. It could mean your intent coverage is too narrow, your system prompt is too restrictive, or users are coming in with expectations your assistant simply cannot meet. Left unmonitored, a high fallback rate quietly erodes user trust, users who hit a wall once are less likely to try again and it will simply increase your churn rate.</p><h4><strong>How to measure it</strong></h4><p>The most reliable way is to have the model explicitly signal when it is falling back, rather than trying to detect it from the text of the response after the fact. Add a boolean field to your structured response, similar to the conversation_ended field from metric 1 — called is_fallback. When the assistant cannot handle a request confidently, it sets that field to true alongside whatever response it gives the user. Store that field per message in your database and the rest is a simple <strong>Metabase</strong> query: count conversations with at least one is_fallback = true message, divide by total conversations, track over time.</p><p>Segment fallbacks by the user’s message that triggered them. That is where the real value is, not the rate itself, but the list of inputs your assistant keeps failing on. <strong>That list is your next product backlog</strong>.</p><h3>8. Retry Rate</h3><h4><strong>What it is</strong></h4><p>Retry rate measures how often a user rephrases or repeats the same question within a conversation after receiving an unsatisfactory response. It is distinct from fallback rate, a fallback is the assistant explicitly failing, a retry is the user implicitly signaling that the response did not land, even when the assistant thought it did.</p><h4><strong>Why it matters</strong></h4><p>Retries are one of the most honest signals you have. The user is not filling in a survey or triggering a fallback, they are just trying again, which means they have not given up yet but they are not satisfied either. A high retry rate on a specific intent category usually means one of two things: the assistant is misunderstanding the request, or it is understanding it correctly but the response is not useful enough to act on. Those are two very different problems and they have different fixes, which is why tracking the retry rate alongside the actual retried messages is important.</p><h4><strong>How to measure it</strong></h4><p>This one is best detected in the nightly batch job. Add it to the rubric you are already using for implicit satisfaction and outcome classification, ask the model to flag whether the user repeated or rephrased the same request at any point in the conversation, and if so, how many times. Store the result as a retry_count field per conversation in your database. From there, Metabase gives you retry rate over time, average retries per conversation, and most usefully the conversations with the highest retry counts, which are the ones worth reading manually. You can find the full batch job implementation at the end of this article.</p><h3>9. Drop-off Point</h3><h4><strong>What it is</strong></h4><p>Drop-off point tracks where in a conversation users abandon the session without reaching a resolution. Unlike abandonment rate, which tells you that users left, drop-off point tells you where they left, which turn, which topic, which assistant response was the last thing they saw before they gave up.</p><h4><strong>Why it matters</strong></h4><p>Knowing that users are abandoning conversations is useful. Knowing exactly which assistant response is consistently the last one before they leave is actionable. A high drop-off on turn 2 across a specific intent category almost always points to a single broken response, fix that response and the drop-off moves. It is one of those metrics that sounds like a nice-to-have until you actually look at it, and then it becomes obvious why you should have been tracking it from day one.</p><h4><strong>How to measure it</strong></h4><p>Store a timestamp on every message. A conversation is considered abandoned when it ends without a resolution signal and the last message was from the assistant, meaning the user received a response and did not reply. In your nightly batch job, add a field to your rubric asking the model to identify whether the conversation was abandoned and what the assistant’s last message was before the user stopped responding. Cross-reference that with turn count to see at which point in the conversation the drop-off happened. In Metabase, <strong>grouping abandoned conversations by their last assistant message topic will surface the patterns quickly</strong>. You can find the full batch job implementation at the end of this article.</p><h3>10. Return Rate</h3><h4><strong>What it is</strong></h4><p>Return rate measures how often users come back to your assistant after their first session. It is the percentage of users who initiated more than one conversation over a given time window.</p><h4><strong>Why it matters</strong></h4><p>Return rate is a proxy for trust. A user who comes back has decided the assistant is worth trying again, which means their previous experience was good enough to not put them off. A low return rate is not always a problem; some assistants are designed for one-off interactions like booking a flight or filing a complaint. But for assistants built around ongoing engagement, like a support bot, an internal tool, or a product assistant, a low return rate is a quiet signal that the first experience did not leave users with enough confidence to come back. It is also one of the few metrics that captures long-term value rather than just session-level performance.</p><h4><strong>How to measure it</strong></h4><p>Store a user ID alongside every conversation in your database. Return rate is then a straightforward query: count users with more than one conversation divided by total unique users, over whatever time window makes sense for your product. Track it weekly rather than daily to smooth out noise, and segment by acquisition channel if you can — users who came in through different entry points often have very different return patterns.</p><h3>11. Latency Per Response</h3><h4><strong>What it is</strong></h4><p>Latency per response measures the time between a user sending a message and the assistant returning a response. It is usually tracked as an average and a <strong>p95</strong> (the 95th percentile), because averages hide the tail cases that frustrate users the most.</p><h4><strong>Why it matters</strong></h4><p>Latency is one of the few metrics that affects perceived quality independently of actual quality. A correct answer that takes 8 seconds feels worse than a slightly less complete answer that arrives in 2. In conversational interfaces especially, where the expectation is something close to real-time, high latency breaks the flow of the interaction and signals to the user that something is wrong even when it isn’t. Track p95 in addition to average, your average might look fine while a significant portion of your users are waiting uncomfortably long on every message.</p><h4><strong>How to measure it</strong></h4><p>Store a <em>sent_at</em> timestamp on every user message and a <em>responded_at</em> timestamp on every assistant message. Latency per turn is the difference between the two. Average and p95 across all turns gives you your baseline. In Metabase, plotting this over time makes it easy to spot when a model update, a new tool, or an infrastructure change introduced a latency regression.</p><h3>Operational Health</h3><h4><strong>Token Efficiency</strong></h4><p>Token efficiency measures how many tokens your assistant consumes relative to the quality and length of its output. An assistant that produces verbose, padded responses is burning tokens and money without adding value. Track average output tokens per response alongside your resolution rate. If output tokens are climbing but resolution rate is flat or falling, your assistant is getting wordier without getting better. The fix is usually in the system prompt, tightening the instruction around response length and format.</p><h4><strong>Error Rate</strong></h4><p>Error rate tracks how often your assistant fails at the infrastructure level, API timeouts, failed tool calls, malformed responses, or any exception that prevents the assistant from returning a response at all. These are distinct from fallbacks, which are semantic failures. Error rate is operational. Even a small error rate compounds quickly at scale: a <strong>2%</strong> error rate across <strong>10,000</strong> daily conversations means <strong>200</strong> users hitting a broken experience every day, and when you care enough about your users to not to turn them into numbers even a single failure matters. Log every exception with a conversation ID, error type, and timestamp. Track it in Metabase alongside fallback rate so you can distinguish between the assistant not knowing something and the assistant simply not working.</p><h3>Trust Signals</h3><h4><strong>Correction Rate</strong></h4><p>Correction rate measures how often users explicitly correct the assistant mid-conversation, telling it that its response was wrong, that it misunderstood, or that the information it provided was inaccurate. It is a strong signal of <strong>hallucination</strong> or <strong>knowledge gaps</strong>. Detect it in the nightly batch job by asking the model to flag any turn where the user corrected or contradicted the assistant’s previous response. A rising correction rate on a specific topic is usually a sign that your knowledge base or system prompt needs updating in that area. You can find the full batch job implementation below.</p><h4><strong>Override Rate</strong></h4><p>Override rate measures how often users ignore the assistant’s response and take a different action, asking to speak to a human, navigating away from a suggested link, or explicitly saying they will handle something themselves. It is the trust signal that is hardest to detect but most valuable when you do. A user who overrides the assistant has made a judgment call that the assistant cannot be trusted with this particular request. Detect it in the nightly batch job by looking for explicit override signals in the conversation, requests for human handoff, expressions of intent to act independently, or abrupt topic changes after a recommendation. You can find the full batch job implementation below.</p><h3>The Nightly Batch Job</h3><p>All the metrics flagged for nightly detection throughout this article, implicit satisfaction, outcome classification, retry rate, drop-off point, correction rate, and override rate, can be computed in a single batch job. One model call per conversation, all metrics out.</p><blockquote>The following code is a simple demonstration of collecting user feedback based on the conversation’s status. Please avoid using it in production, as it is far from AI engineering best practices. In production, instead of measuring all metrics in a single request, you may want to better engineer your context and divide the analysis into multiple separate requests. This is usually a tradeoff between cost and precision, there is no universal answer, and you should experiment on your own data, evaluate the results, and decide what works best for your use case. Additionally, the rubric included in the prompt below is not complete and is purely for demonstration purposes. In practice, rubrics vary significantly from business to business and should be carefully designed and validated against your specific data and goals.</blockquote><pre>from openai import OpenAI<br>from pydantic import BaseModel<br>from enum import Enum<br><br>client = OpenAI()<br><br>class SatisfactionLevel(str, Enum):<br>    SATISFIED = &quot;satisfied&quot;<br>    NEUTRAL = &quot;neutral&quot;<br>    FRUSTRATED = &quot;frustrated&quot;<br><br>class ConversationOutcome(str, Enum):<br>    CONVERTED = &quot;converted&quot;<br>    CONTAINED = &quot;contained&quot;<br>    ESCALATED = &quot;escalated&quot;<br>    ABANDONED = &quot;abandoned&quot;<br><br>class NightlyAnalysis(BaseModel):<br>    # Metric 2: Implicit satisfaction<br>    satisfaction: SatisfactionLevel<br>    satisfaction_reason: str<br>    satisfaction_confidence: float<br><br>    # Metric 3: Outcome<br>    outcome: ConversationOutcome<br>    outcome_reason: str<br>    outcome_confidence: float<br><br>    # Metric 4: Conversation length<br>    intent_category: str<br><br>    # Metric 8: Retry rate<br>    retry_count: int<br>    retry_detected: bool<br><br>    # Metric 9: Drop-off<br>    abandoned: bool<br>    last_assistant_message_topic: str<br><br>    # Trust signals<br>    correction_detected: bool<br>    correction_count: int<br>    override_detected: bool<br>    override_count: int<br><br>NIGHTLY_RUBRIC = &quot;&quot;&quot;<br>You are analyzing a support conversation. Return a structured analysis<br>covering the following dimensions:<br><br>SATISFACTION: Classify overall user satisfaction as satisfied, neutral,<br>or frustrated.<br>- SATISFIED: Issue resolved, user confirmed positively, no repeated questions<br>- NEUTRAL: Ambiguous outcome, no clear signal<br>- FRUSTRATED: Repeated questions, explicit dissatisfaction, requested human,<br>  abrupt exit<br><br>OUTCOME: Classify the conversation outcome.<br>- CONVERTED: User completed a target business action<br>- CONTAINED: Issue resolved without human escalation<br>- ESCALATED: Handed off to a human agent<br>- ABANDONED: User left without resolution or escalation<br><br>INTENT CATEGORY: Classify the user&#39;s primary intent as one of:<br>billing, technical, account, shipping, general.<br><br>RETRY: Did the user rephrase or repeat the same question after receiving<br>a response? If yes, how many times?<br><br>DROP-OFF: Did the user abandon the conversation without resolution?<br>If yes, what was the topic of the last assistant message before they left?<br><br>CORRECTIONS: Did the user explicitly correct the assistant at any point?<br>If yes, how many times?<br><br>OVERRIDES: Did the user ignore the assistant&#39;s suggestion and indicate<br>they would handle something themselves, request a human, or navigate away?<br>If yes, how many times?<br><br>Return a confidence score between 0.0 and 1.0 for satisfaction and outcome.<br>&quot;&quot;&quot;<br><br>def run_nightly_analysis(conversation_id: str,<br>                         history: list) -&gt; NightlyAnalysis:<br>    transcript = &quot;\n&quot;.join(<br>        f&quot;{msg[&#39;role&#39;].upper()}: {msg[&#39;content&#39;]}&quot;<br>        for msg in history<br>    )<br><br>    completion = client.beta.chat.completions.parse(<br>        model=&quot;gpt-4o-mini&quot;,<br>        messages=[<br>            {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: NIGHTLY_RUBRIC},<br>            {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: f&quot;Conversation transcript:\n{transcript}&quot;}<br>        ],<br>        response_format=NightlyAnalysis<br>    )<br><br>    return completion.choices[0].message.parsed<br><br>def run_nightly_batch(conversations: list[dict]) -&gt; list[NightlyAnalysis]:<br>    results = []<br>    for conv in conversations:<br>        result = run_nightly_analysis(conv[&quot;id&quot;], conv[&quot;history&quot;])<br>        results.append(result)<br>        print(f&quot;[{conv[&#39;id&#39;]}] &quot;<br>              f&quot;satisfaction={result.satisfaction} | &quot;<br>              f&quot;outcome={result.outcome} | &quot;<br>              f&quot;intent={result.intent_category} | &quot;<br>              f&quot;retries={result.retry_count} | &quot;<br>              f&quot;corrections={result.correction_count} | &quot;<br>              f&quot;overrides={result.override_count}&quot;)<br>    return results<br><br>if __name__ == &quot;__main__&quot;:<br>    conversations = [<br>        {<br>            &quot;id&quot;: &quot;conv_001&quot;,<br>            &quot;history&quot;: [<br>                {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;I can&#39;t log into my account&quot;},<br>                {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Try resetting your password.&quot;},<br>                {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;I already tried that&quot;},<br>                {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Let me look into your account.&quot;},<br>                {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Finally, it&#39;s working. Thanks&quot;}<br>            ]<br>        },<br>        {<br>            &quot;id&quot;: &quot;conv_002&quot;,<br>            &quot;history&quot;: [<br>                {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Where is my refund?&quot;},<br>                {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Refunds take 5-7 business days.&quot;},<br>                {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;That&#39;s wrong, I was told 3 days&quot;},<br>                {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;I apologize for the confusion.&quot;},<br>                {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;I&#39;ll just call support directly&quot;}<br>            ]<br>        }<br>    ]<br><br>    results = run_nightly_batch(conversations)</pre><h3>Closing Thoughts</h3><p>These metrics <strong>will not all matter equally for every product</strong>. A one-off booking assistant has no use for return rate. A support bot with no tool integrations does not need to track tool call counts. Start with the metrics that map directly to your use case, <strong>NPS, implicit satisfaction, </strong>and<strong> conversion rate</strong> are a safe starting point for almost everyone, and add the diagnostic ones as you mature.</p><p>What matters most is that you have a clear picture of what your assistant is actually doing in production, not just what it does in your test suite. The gap between those two things is where most assistants quietly fail. These metrics are how you close it.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=63252bf4eabf" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/15-metrics-you-should-be-measuring-if-you-are-building-a-conversational-ai-assistant-63252bf4eabf">15 Metrics You Should Be Measuring If You Are Building a Conversational AI Assistant</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Your LLM Judge Is Not Measuring What You Think It Is]]></title>
            <link>https://ai.plainenglish.io/your-llm-judge-is-not-measuring-what-you-think-it-is-b61f3ae0a13f?source=rss-ee4389407aa5------2</link>
            <guid isPermaLink="false">https://medium.com/p/b61f3ae0a13f</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[llm-evaluation]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <dc:creator><![CDATA[Maede Jalali]]></dc:creator>
            <pubDate>Wed, 08 Jul 2026 10:06:22 GMT</pubDate>
            <atom:updated>2026-07-08T10:06:22.605Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*v9VDKaZxIvE8y8W1bh9ZRQ.png" /><figcaption>A courtroom scene where an AI judge is judging other AI agents, but the judge is clearly not as fair as it thinks it is.</figcaption></figure><p>Over the past few months I have been consulting with a few small startups building AI-based products. Almost all of them had an evaluation pipeline. Almost all of them were using an LLM as a judge. And almost all of them had shipped with confidence, only to run into failures on the simplest edge cases in production.</p><p>When I dug into their setups, the problem was not the model they chose or the data they used. It was the judge. It had never been calibrated, and nobody had checked whether it was actually measuring what they thought it was.</p><p>The two biases I kept finding were the same ones every time.</p><h4><strong>The Biases</strong></h4><p><strong><em>Verbosity bias</em></strong>. The judge had a consistent preference for longer answers, even when the shorter ones were more accurate and direct. This does not mean every score was wrong. Often the longer answer genuinely was better. But the bias meant length was quietly factoring into the score whether it should have or not. You cannot trust a result you cannot separate from noise.</p><p><strong><em>Position bias</em></strong>. In pairwise evaluations, the judge showed a mild but consistent preference for whichever answer appeared first in the prompt. Again, the first answer was sometimes the right winner. But when I ran the same comparisons with the order flipped and win rates shifted on cases where the underlying quality had not changed, that told me position was influencing scores it should not have touched.</p><p>Both of these biases share the same root cause: the judge was given no real guidance. It received two outputs, a vague instruction to pick the better one, and was left to fill in the blanks with its own preferences. Those preferences turned out to be length and order.</p><h4><strong>The Fixes</strong></h4><ol><li>For <strong><em>verbosity</em></strong>, stop asking for a single holistic score. Break your rubric into specific axes: accuracy, conciseness, relevance, whatever dimensions actually matter for your use case. Add an explicit instruction that response length should not influence the score, and define what an appropriate length looks like for the task. The judge needs the same clarity you would give a new hire doing a review.</li><li>For <strong><em>position</em></strong> bias, make order-swapping mandatory. Every pair gets evaluated twice with positions flipped, and you average the scores. If the two runs produce significantly different results, that is a signal worth investigating rather than a number to smooth over.</li><li>There is a third issue that is less obvious but worth checking: <strong><em>do not use the same model as both the candidate and the judge</em></strong>. A model will subtly favor outputs that match its own style and reasoning patterns. It is not intentional, it just is. Using a separate model for evaluation is the simplest way to remove that conflict.</li><li>One more technique that meaningfully reduces all of these biases: <strong><em>force the judge to reason before it scores</em></strong>. Instead of asking for a number first, instruct it to explain its evaluation step by step and only then give a final score. This chain-of-thought approach makes the judge articulate why it prefers one answer over another, which surfaces flawed reasoning you can then correct in your rubric. It also gives you an audit trail, which is invaluable when you are trying to understand why your evaluation pipeline is behaving in unexpected ways.</li></ol><p>The difference in practice is quite significant. A judge prompted with <em>“score these two responses from 1 to 5”</em> will give you a number with no way to verify what drove it. A judge prompted with “first explain what each response does well and poorly across accuracy, conciseness, and relevance, then give a final score” will show its work and it will make it easier for you to debug and figure out what’s biasing your judge, or what’s causing the unwanted results. When the reasoning is wrong — when it praises a response for being thorough but thorough was not what the task required — you can see exactly where your rubric needs tightening. Without that reasoning, the same flawed score would have gone unquestioned. Over time, iterating on your rubric based on the judge’s written reasoning is one of the fastest ways to close the gap between what your judge is measuring and what you actually care about.</p><p><strong>The Checklist</strong></p><p>I always run my judge through the following checklist to make sure I can trust it:</p><ul><li><strong>Break your rubric into axes.</strong> Score accuracy, conciseness, and relevance separately. Never ask for a single holistic score.</li><li><strong>Explicitly exclude length.</strong> Tell the judge that response length should not influence the score, and define what appropriate length looks like for your task.</li><li><strong>Make order-swapping mandatory.</strong> Every pairwise comparison gets evaluated twice with positions flipped. Average the scores. Mark the outliers, the biases are usually lying there.</li><li><strong>Use a different model as your judge.</strong> Never use the same model as both candidate and evaluator.</li><li><strong>Force reasoning before scoring.</strong> Instruct the judge to explain its evaluation step by step before producing a final score.</li><li><strong>Test across evaluation modes.</strong> Run the same comparisons in both pointwise and pairwise modes and check whether rankings hold.</li></ul><p><strong>One More Thing Worth Checking</strong></p><p>LangChain’s research on LLM-as-a-judge points out something related to position bias but worth calling out separately: the same judge can produce contradictory results depending on whether it evaluates outputs individually or compares them side by side [1]. In <strong>pairwise</strong> mode it can even violate <strong>transitivity</strong>, preferring A over B, B over C, and then C over A. It is the same underlying instability, just showing up in a different form. If you have never stress-tested your judge for consistency across evaluation modes, it is worth doing before you trust it to make shipping decisions. A simple way to start is running the same set of comparisons in both pointwise and pairwise modes and checking whether the rankings hold. Significant divergence is a sign your judge is reacting to format more than quality.</p><p><strong>Why This Matters</strong></p><p>An LLM judge is not a neutral third party you can plug in and trust. It is a system with its own preferences, failure modes, and blind spots, and it needs to be treated like one. <strong><em>Calibration is not optional polish</em></strong> you add after launch. It is part of building an honest evaluation pipeline in the first place.</p><p>In practice, calibration means you have validated your rubric against human scores, stress-tested it across evaluation modes, and know where it fails before it fails in production.</p><p>Launching with an uncalibrated judge does not mean you have tested your product. It means you have tested whether your product can impress a biased reviewer. The cost of discovering that in production, in user trust, in decisions already made on bad signal, is almost always higher than the cost of a calibration pass before you ship.</p><p><strong>References</strong></p><p>[1] LangChain, “LLM-as-a-Judge: A Comprehensive Guide,” <a href="https://www.langchain.com/resources/llm-as-a-judge">https://www.langchain.com/resources/llm-as-a-judge</a></p><h3>Before you go</h3><ul><li>Please take a moment to like the post and follow the writer!</li><li>Did you know that over 400,000 developers share what they’re building, learning, and discovering across our platforms every month? Learn how you can contribute <a href="https://plainenglish.io/write-for-us">here</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b61f3ae0a13f" width="1" height="1" alt=""><hr><p><a href="https://ai.plainenglish.io/your-llm-judge-is-not-measuring-what-you-think-it-is-b61f3ae0a13f">Your LLM Judge Is Not Measuring What You Think It Is</a> was originally published in <a href="https://ai.plainenglish.io/">Artificial Intelligence in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Are You a Passive AI-Native Developer?]]></title>
            <link>https://ai.plainenglish.io/are-you-a-passive-ai-native-developer-b8ada23979ce?source=rss-ee4389407aa5------2</link>
            <guid isPermaLink="false">https://medium.com/p/b8ada23979ce</guid>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[ai-native-development]]></category>
            <category><![CDATA[ai-native]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Maede Jalali]]></dc:creator>
            <pubDate>Sun, 14 Jun 2026 15:29:51 GMT</pubDate>
            <atom:updated>2026-06-14T15:29:51.552Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nQQBHBbjav-LA5PjIbXw1w.png" /><figcaption>Picture of an active AI-native developer</figcaption></figure><p>A few months ago I was writing a query in Metabase at work. Nothing high stakes, just a routine data pull. And I sat there staring at the screen, unable to remember the syntax for a basic SQL query. Not a complex one. A basic one I had written hundreds of times.</p><p>My first instinct was to open a chat window and ask. Then I stopped and thought: when did this become my first instinct?</p><p>I started paying attention after that. I noticed I was handing debugging sessions off to AI before I had really sat with the problem myself. It was getting harder to stay focused through a long error trace or a dense technical document. The kind of slow, deliberate thinking that used to feel natural was starting to feel like effort.</p><p><a href="https://www.linkedin.com/in/marina-wyss/">Marina Wyss</a> wrote about <a href="https://www.gratitudedriven.com/p/is-ai-making-you-dumber">something similar</a> recently, calling it the “mashed potatoes” feeling. Her framing resonated with me, though as a developer what I was losing felt more specific: not just focus, but the active relationship with my own craft. That is not a productivity issue. That is the job.</p><p>There is an active version of being AI-native and a passive one, and I had drifted into the passive one without noticing.</p><p>So I did something simple. Every morning, thirty minutes, no AI for coding. I pick something and work through it by hand. An SQL query. A circuit breaker. A backoff strategy. Sometimes a LeetCode problem. I only use AI in that window to learn something new, never to write the code for me.</p><p>It felt slow and uncomfortable the first week, which told me everything I needed to know. But after a few weeks something shifted. My mind felt sharper, not just during the drills but throughout the day. The knowledge that had gone quiet started feeling active again.</p><p>I think the most valuable thing we risk losing by delegating everything to AI is not our tools or our shortcuts. It is our problem solving instincts. The ability to sit with a hard thing, hold it in your head, and work through it. That is the part that makes us good at this job, and it is the part that quietly atrophies when we stop exercising it.</p><p>Whether it is thirty minute drills like mine, the deliberate friction rules Marina describes, or something else entirely, I think we have to actively choose to keep that sharp. It will not stay sharp on its own.</p><p>I am genuinely curious what others are doing here. If you have found a way to stay active as a developer in an AI-native world, I would love to hear it. Drop it in the comments.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b8ada23979ce" width="1" height="1" alt=""><hr><p><a href="https://ai.plainenglish.io/are-you-a-passive-ai-native-developer-b8ada23979ce">Are You a Passive AI-Native Developer?</a> was originally published in <a href="https://ai.plainenglish.io/">Artificial Intelligence in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Automate Your Android Releases with CI/CD: Build, Sign, and Deploy Like a Pro]]></title>
            <link>https://medium.com/@theMajesticMj/automate-your-android-releases-with-ci-cd-build-sign-and-deploy-like-a-pro-1cbfda2cfa9b?source=rss-ee4389407aa5------2</link>
            <guid isPermaLink="false">https://medium.com/p/1cbfda2cfa9b</guid>
            <category><![CDATA[android]]></category>
            <category><![CDATA[android-app-development]]></category>
            <category><![CDATA[ci-cd-pipeline]]></category>
            <dc:creator><![CDATA[Maede Jalali]]></dc:creator>
            <pubDate>Wed, 28 Feb 2024 07:13:50 GMT</pubDate>
            <atom:updated>2024-02-28T16:03:16.764Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HBDTmJeimXvns3yNcmyCbg.png" /></figure><p>Struggling with manual deployments and testing bottlenecks? Automating your Android CI/CD pipeline can be your game-changer, making your releases smoother and testers happier. Since I’ve been struggling a lot to develop one for our codebase at work, I decided to write an article about it. I hope it saves your day and saves you from manual deployments.</p><p>This article guides you through setting up an efficient pipeline, covering essential stages like code formatting, APK building, signing, and even automatic file uploads to Slack.</p><p>I assume you generally know what is CI/CD, but to make sure we’re all on the same page, let’s have a short review of the concept. According to <a href="https://www.redhat.com/en/topics/devops/what-cicd-pipeline">redhat.com</a>, CI/CD is:</p><p><em>A continuous integration and continuous deployment (CI/CD) pipeline is a series of steps that must be performed in order to deliver a new version of software. CI/CD pipelines are a practice focused on improving software delivery throughout the software development life cycle via automation.</em></p><p>By automating CI/CD throughout the development, testing, production, and monitoring phases of the software development lifecycle, organizations are able to develop higher-quality code, faster and more securely. Although it’s possible to manually execute each of the steps of a CI/CD pipeline, the true value of CI/CD pipelines is realized through automation.</p><p>In this article, we’re going to automate the following steps:</p><ol><li>🧹 Linting: checking if the code format respects KtlintFormat rules or not</li><li>🏗️ Building release APKs: Signing the APKs using CI/CD</li><li>↖️ Uploading to Slack: Uploading APK files from CI/CD to Slack (for QA test and internal releases)</li></ol><p>But before jumping to the stages let’s check some stuff you need to do before actually running your stages. We’ll go through each and every line of it step by step together.</p><h3>1. Installing an image</h3><pre>image: martinsumera/gitlab-ci-android</pre><p>ℹ️ Imagine building an entire kitchen from scratch just to cook dinner. That’s what setting up an Android development environment used to be like: installing various tools, libraries, and SDKs — a tedious and time-consuming process.</p><p>This is where Docker images come in, like pre-built kitchens complete with all the utensils and ingredients you need for Android development. The line image: martinsumera/gitlab-ci-android on the CI/CD file, simply tells GitLab CI to use this specific &quot;kitchen,&quot; ensuring you have everything ready to cook (build) your app consistently and efficiently.</p><p><strong>Think of it this way:</strong></p><ul><li><strong>No more manual setup:</strong> Docker images eliminate the need to install everything yourself, saving you time and effort.</li><li><strong>Consistency guaranteed:</strong> Everyone uses the same “kitchen,” ensuring builds are reproducible across different environments.</li><li><strong>Focus on what matters:</strong> Spend less time setting up and more time coding and innovating.</li></ul><p>Now that you’ve installed the image and your kitchen is ready, it’s time to prepare some ingredients before actually jumping to the cooking part.</p><h3>2. Before Script</h3><p>Imagine your app build as a concert. Before the band takes the stage, there’s a lot to do — tuning instruments, setting lights, and checking sound. That’s what before_script does, preparing your environment for a successful build.</p><p>This section runs <strong>before</strong> any other scripts in your CI/CD pipeline. So before_script is responsible for essential tasks like:</p><ul><li><strong>Installing tools:</strong> Maybe your build needs specific libraries or commands. before_script fetches and installs them, ensuring everything&#39;s in place.</li><li><strong>Preparing variables:</strong> Like backstage crew setting cues, before_script defines and configures variables (like version numbers) your build script needs later. (As you can see in the code, I’m doing the same thing in before_script. I’m reading my Release file and extracting the version numbers)</li><li><strong>Environment Checks:</strong> It can even run checks to make sure the environment meets requirements, guaranteeing a smooth performance for your build.</li></ul><p>🔍 So let’s take a closer look at the code:</p><pre>before_script:<br>  - export VERSION_CODE=$(grep &#39;versionCode = &#39; buildSrc/src/main/java/Release.kt | sed &#39;s/^.*= //&#39;)<br>  - export VERSION_NAME=$(grep &#39;versionName = &#39; buildSrc/src/main/java/Release.kt | sed &#39;s/^.*= &quot;\\(.*\\)&quot;/\\1/&#39;)</pre><p>In this script, I’m creating two variables called VERSION_CODE and VERSION_NAME because these two variables will be used frequently in the next steps. I have a file called Release.kt which contains the actual version code and version name. So, what I’m actually doing here is reading from that file using grep and storing version numbers in those variables.</p><h3>3. Anchors</h3><p>There are always some parts of code that we need to use multiple times; thus, we can define an anchor to avoid repeating the code each time. The way to do it in YAML is to store the repeating code in a container, name it, and then reuse that code block by referring to its name.</p><p>First, recognize the &amp; preceding the name of the (&amp;gradle_setup_anchor). It signifies <strong>an anchor name.</strong> gradle_setup_container is just the container of a code block you can reuse multiple times. We will not refer to it by its name, it’s just a container. Whenever we need to access the content of that container we will use (*gradle_setup_anchor). The * is used to reference the value.</p><pre>gradle_setup_container: &amp;gradle_setup_anchor<br>  - export GRADLE_USER_HOME=$(pwd)/.gradle<br>  - chmod +x ./gradlew</pre><p>🔍 <strong>Under the Hood:</strong></p><p>Now, let’s dissect its commands:</p><ul><li><strong>export GRADLE_USER_HOME=$(pwd)/.gradle:</strong> This defines an environment variable, essentially allocating a dedicated space ($(pwd) refers to the current directory) for Gradle to store its downloaded tools and configurations. Like creating a designated &quot;Gradle Locker&quot; to keep things organized.</li><li><strong>chmod +x ./gradlew:</strong> This grants execution permission to the ./gradlew file, Gradle&#39;s command-line companion. Think of it as handing Gradle the key to open its toolbox and access the necessary tools.</li></ul><p><strong>Calling the Helper:</strong></p><p>Later in your CI/CD script, you’ll see references like *gradle_setup_anchor. This copies the content of gradle_setup_container in its place, prompting it to perform its magic and prepare the Gradle environment just as you defined.</p><h3>4. Cache</h3><p>This section utilizes the power of caching to optimize your build process. Imagine building a house — you wouldn’t download the same bricks and tools repeatedly for each wall. Similarly, caching avoids downloading already downloaded Gradle resources, saving time and bandwidth.</p><p>🔍 <strong>Code breakdown:</strong></p><pre>cache:<br>  key: ${CI_PROJECT_ID}<br>  paths:<br>    - .gradle/<br>    - .gradle/wrapper<br>    - .gradle/caches</pre><ul><li><strong>cache::</strong> This keyword initiates the caching functionality.</li><li><strong>key: ${CI_PROJECT_ID}:</strong> This unique identifier ensures cache isolation for different projects, preventing conflicts. Think of it as labelling each project&#39;s cache box clearly.</li><li><strong>paths::</strong> This section specifies the directories to be cached:</li><li><strong>.gradle/</strong>: Gradle&#39;s main configuration directory, holding essential build settings.</li><li><strong>.gradle/wrapper</strong>: Contains information about the specific Gradle version used, ensuring consistency.</li><li><strong>.gradle/caches</strong>: Stores downloaded Gradle libraries and resources, saving future downloads.</li></ul><p><strong>Here’s how it works</strong>:</p><ol><li><strong>First Build:</strong> During the first build, all specified directories are downloaded and the content of the cache path will be stored at the end of the job.</li><li><strong>Subsequent Builds:</strong> For subsequent builds, the CI/CD system checks the cache first. If the cached versions of these directories match the current project and required Gradle version, they are reused instead of being downloaded again, significantly speeding up the build process.</li></ol><h3>5. Stages</h3><p>One crucial element of a CI/CD pipeline is <strong>stages</strong>, which act like checkpoints or milestones on your app’s journey from code commit to user hands.</p><p>You can define different stages according to your needs. For instance, if you want to run your unit tests before deploying and make sure everything is working smoothly, you can have a stage called unitTest, or if you want to check if your code respects the linter rules and you’re not violating any code style, you can have a stage called formatting; so it’s totally up to you, and you have to define them in a way that meets your needs.</p><p>🔍 <strong>Code breakdown:</strong></p><pre>stages:<br>  - check<br>  - release</pre><ul><li>In the <strong>check</strong> stage, we will only check the code style for the sake of simplicity.</li><li>In the <strong>release</strong> stage, we will build a release APK, sign it, and upload it to Slack.</li></ul><blockquote><em>Keep in mind that defining stages, and what each stage actually does, may differ for various codebases.</em></blockquote><h3>Stages code breakdown</h3><h4>lint (a job in the check stage):</h4><pre>lint:<br>  stage: check<br>  script:<br>    - *gradle_setup_anchor<br>    - ./gradlew  ktlintCheck</pre><p><strong>stage: check:</strong> This indicates the lint task belongs to the &quot;check&quot; stage, where your code goes through various quality checks. Think of it as the first line of defence before diving deeper into testing and deployment.</p><p><strong>script:</strong> This section defines the steps executed during the stage:</p><ul><li><strong>gradle_setup_anchor:</strong> This refers to the anchor we defined earlier. As mentioned before, It will set up the Gradle environment for the next step.</li><li><strong>./gradlew ktlintCheck:</strong> This command utilizes the ktlint tool, a code linter specifically for Kotlin code. It scans your code for stylistic inconsistencies, potential bugs, and adherence to best practices. Imagine it as a sharp-eyed editor checking your writing for grammar and style. If you want to add the linter to your project you have to add the KtLint plugin <a href="https://github.com/JLLeitschuh/ktlint-gradle">from here</a> or anywhere else you are adding it from.</li></ul><h3>playStoreRelease:</h3><p>This code snippet orchestrates the final stage of the CI/CD pipeline: preparing and uploading your app to Slack. Let’s dissect it step by step:</p><pre># Play Store Build<br>playStoreRelease:<br>  stage: release<br>  script:<br>    - *gradle_setup_anchor<br>    - BUILD_VERSION_NAME=$VERSION_NAME BUILD_VERSION_CODE=$VERSION_CODE<br>    - echo $KEY_STORE_BASE_64 | base64 -di &gt; keystore<br>    - export KEYSTORE_FILE=$(pwd)/keystore<br>    - ./gradlew assembleProductionRelease<br>    - mkdir artifacts<br>    - cp &quot;app/build/outputs/apk/production/release/app-production-release.apk&quot; &quot;artifacts/sindibplaystore-release-$VERSION_NAME.apk&quot;<br>    - export APK_FILE=&quot;artifacts/playstore-release-$VERSION_NAME.apk&quot;<br>    - *upload_to_slack<br>  artifacts:<br>    name: &quot;app-playStore-release&quot;<br>    expire_in: 1 day<br>    paths:<br>      - artifacts<br>  tags:<br>    - &quot;docker&quot;<br>  environment:<br>    name: Production<br>  when: manual<br>  allow_failure: true<br>  needs: []</pre><h4><strong>1. Setting the Stage:</strong></h4><ul><li><strong>stage: release:</strong> This defines the current stage as &quot;release&quot; and its task as &quot;playStoreRelease,&quot; indicating it&#39;s responsible for building and releasing the Play Store version.</li></ul><h4><strong>2. Script:</strong></h4><ul><li><strong>gradle_setup_anchor:</strong> This calls upon your previously defined anchor, ensuring the Gradle environment is ready for the build.</li><li><strong>BUILD_VERSION_NAME=$VERSION_NAME BUILD_VERSION_CODE=$VERSION_CODE:</strong> These lines set environment variables, retrieved earlier in the pipeline(on the before_script step), defining the app&#39;s version name and code for the Play Store release.</li><li><strong>echo $KEY_STORE_BASE_64 | base64 -di &gt; keystore:</strong> This decodes a base64-encoded string (your app&#39;s signing key) and saves it as a keystore file named &quot;keystore.&quot; This secure key is essential for signing your app. To do that you have to follow the next steps:</li></ul><ol><li>You have to encode your signing key into a base64-encoded string. Doing that is a piece of cake, you just need to find the right command related to your terminal. In Linux it’s base64 keystore &gt; output.txt and in Mac, it’s openssl base64 -in keystore -out output.txt.</li><li>Then you should store that string in your CI/CD environment variables. You can find them in Gitlab → Settings → CI/CD → Variables</li><li>Add a variable called <strong>KEY_STORE_BASE_64</strong> and store the encoded string there</li></ol><blockquote>Be careful not to check the “File” option.</blockquote><ol><li>Now when you’re running this stage, the script can simply decode that string and regenerate your keystore file.</li></ol><blockquote>Regarding the fact that the keystore is a top-secret file the encoding and decoding part is crucial to keep the keystore file safe.</blockquote><ul><li><strong>export KEYSTORE_FILE=$(pwd)/keystore:</strong> This sets another environment variable pointing to the location of the keystore file.</li><li><strong>./gradlew assembleProductionRelease:</strong> This command triggers Gradle to build the release version of your app, utilizing the previously set environment variables for signing and versioning. However, to be able to sign your APK using the CI/CD you need to add the following signing config to your build.gradle’s <em>android</em> tag:</li></ul><pre>android {<br>  def isCiCd = System.getenv(&quot;CI&quot;) != null<br>  signingConfigs {<br>        signing {<br>            if (isCiCd) {<br>                storeFile file(System.getenv(&#39;KEYSTORE_FILE&#39;))<br>                storePassword System.getenv(&#39;KEYSTORE_PASSWORD&#39;)<br>                keyAlias System.getenv(&#39;KEY_ALIAS&#39;)<br>                keyPassword System.getenv(&#39;KEY_PASSWORD&#39;)<br>            }<br>        }<br>    }<br><br>    buildTypes {<br>        release {<br>            signingConfig signingConfigs.signing<br>        }<br>    }<br>}</pre><ul><li><strong>mkdir artifacts - cp &quot;app/build/outputs/apk/production/release/app-production-release.apk&quot; &quot;artifacts/playstore-release-$VERSION_NAME.apk&quot;:</strong> This creates a directory named &quot;artifacts&quot; and copies the generated APK file with a specific name including the version number.</li><li><strong>export APK_FILE=&quot;artifacts/playstore-release-$VERSION_NAME.apk&quot;:</strong> This sets another environment variable holding the path to the final signed and versioned APK file.</li></ul><h4><strong>3. Artifacts Configuration:</strong></h4><ul><li><strong>name: “app-playStore-release”</strong>: This assigns a name to the artifacts that will be uploaded. This likely shows up in the UI where you access uploaded data, helping with identification and organization.</li><li><strong>expire_in: 1 day</strong>: This indicates that the artifacts will be automatically deleted after 1 day. This helps manage storage space and prevents outdated versions from cluttering the system.</li><li><strong>paths: — artifacts</strong>: This points to the directory named “artifacts” where the generated APK file is stored. This specifies which files from the previous steps are considered artifacts for upload.</li></ul><p><strong>Additional Information:</strong></p><ul><li><strong>tags: — “docker”</strong>: This adds a tag “docker” to the job. This is related to dedicating a runner to the CI/CD pipeline. Ask your DevOps team for the correct tag.</li><li><strong>environment: name: Production</strong>: This associates the job with the “Production” environment. This could be helpful for understanding or segregating artifacts based on their deployment target.</li><li><strong>when: manual</strong>: This specifies that this job (and thus the artifact build and upload) will only run when triggered manually. This likely means it’s not part of the automatic CI/CD pipeline but requires deliberate action to deploy to the Play Store.</li><li><strong>allow_failure: true</strong>: This indicates that even if the script within the stage fails, the pipeline will continue. This might be useful for informational uploads or notifications that don’t necessarily block further progress.</li><li><strong>needs: []</strong>: This is an empty list, suggesting that this stage doesn’t depend on the successful completion of any other stages before it can run.</li></ul><h4><strong>4. Sharing the File:</strong></h4><ul><li><strong>upload_to_slack artifacts:</strong> This refers to another anchor (I’ll explain in detail in the next section) responsible for uploading the artifacts (in this case, the APK file) to Slack, notifying your team about the successful build and upload.</li></ul><blockquote><em>NOTE: At this step, (even without the Uploading to Slack step) we can have access to our built APK through GitLab’s pipeline page</em></blockquote><h3>6. Uploading to Slack</h3><ol><li>First, you have to build a <a href="https://api.slack.com/start/apps">Slack App</a> that helps send the APK file to Slack conversations using Web API and webhooks.</li><li>Then you have to <a href="https://api.slack.com/messaging/files">allow that app the necessary permissions</a> (writing to a Slack channel)</li><li>Finally, upload it to Slack:</li></ol><pre>.upload_to_slack: &amp;upload_to_slack<br>  - |<br>    curl \<br>    --limit-rate 2M \<br>    -F token=&quot;${SLACK_CHANNEL_ACCESS_TOKEN}&quot; \<br>    -F channels=&quot;${SLACK_CHANNEL_ID}&quot; \<br>    -F initial_comment=&quot;:rocket: Hello team! This is the latest update for branch: $CI_COMMIT_BRANCH :rocket:&quot; \<br>    -F file=@&quot;$APK_FILE&quot; \<br>    https://slack.com/api/files.upload</pre><p>Remember anchors? here we define an anchor to help us send the file to Slack.</p><p>SLACK_CHANNEL_ACCESS_TOKEN :</p><ol><li><a href="https://api.slack.com/apps/A068Z7EG092">On this page</a>, you can check all of your Slack apps. In <strong>App-Level Tokens → Generate token and scopes,</strong> you can generate your SLACK_CHANNEL_ACCESS_TOKEN.</li><li>Then, you should store that string in your CI/CD environment variables. You can find them in Gitlab → Settings → CI/CD → Variables. Create the SLACK_CHANNEL_ACCESS_TOKEN variable and store the access token there.</li></ol><p>SLACK_CHANNEL_ID :</p><p>This is the ID of the Slack channel you want to send it to them. When you open the channel on a browser, you’ll see the full address of the channel. This is something like this:</p><p><a href="https://app.slack.com/client/XYZXYZ/**SLACK_CHANNEL_ID**">https://app.slack.com/client/XYZXYZ/<strong><em>SLACK_CHANNEL_ID</em></strong></a></p><p>The last parameter is what you need. Store it in a variable named SLACK_CHANNEL_ID and you’re good to go.</p><p>You can also find it on the “About” page of a channel.</p><p>You can check the whole file here:</p><p><a href="https://github.com/MaedehJJ/gitlab-ci-cd/blob/main/.gitlab-ci.yml">gitlab-ci-cd/.gitlab-ci.yml at main · MaedehJJ/gitlab-ci-cd</a></p><pre>image: martinsumera/gitlab-ci-android<br><br><br>before_script:<br>  - export VERSION_CODE=$(grep &#39;versionCode = &#39; buildSrc/src/main/java/Release.kt | sed &#39;s/^.*= //&#39;)<br>  - export VERSION_NAME=$(grep &#39;versionName = &#39; buildSrc/src/main/java/Release.kt | sed &#39;s/^.*= &quot;\(.*\)&quot;/\1/&#39;)<br><br>gradle_setup_container: &amp;gradle_setup_anchor<br>  - export GRADLE_USER_HOME=$(pwd)/.gradle<br>  - chmod +x ./gradlew<br><br>stages:<br>  - check<br>  - release<br><br>.upload_to_slack: &amp;upload_to_slack<br>  - |<br>    curl \<br>    --limit-rate 2M \<br>    -F token=&quot;${SLACK_CHANNEL_ACCESS_TOKEN}&quot; \<br>    -F channels=&quot;${SLACK_CHANNEL_ID}&quot; \<br>    -F initial_comment=&quot;:rocket: Hello team! This is the latest update for branch: $CI_COMMIT_BRANCH :rocket:&quot; \<br>    -F file=@&quot;$APK_FILE&quot; \<br>    https://slack.com/api/files.upload<br><br>cache:<br>  key: ${CI_PROJECT_ID}<br>  paths:<br>    - .gradle/<br>    - .gradle/wrapper<br>    - .gradle/caches<br><br>lint:<br>  stage: check<br>  script:<br>    - *gradle_setup_anchor<br>    - ./gradlew  ktlintCheck<br>  tags:<br>    - &quot;docker&quot;<br>  allow_failure: false<br><br># Play Store Build<br>playStoreRelease:<br>  stage: release<br>  script:<br>    - *gradle_setup_anchor<br>    - BUILD_VERSION_NAME=$VERSION_NAME BUILD_VERSION_CODE=$VERSION_CODE<br>    - echo $KEY_STORE_BASE_64 | base64 -di &gt; keystore<br>    - export KEYSTORE_FILE=$(pwd)/keystore<br>    - ./gradlew assembleProductionRelease<br>    - mkdir artifacts<br>    - cp &quot;app/build/outputs/apk/production/release/app-production-release.apk&quot; &quot;artifacts/playstore-release-$VERSION_NAME.apk&quot;<br>    - export APK_FILE=&quot;artifacts/playstore-release-$VERSION_NAME.apk&quot;<br>    - *upload_to_slack<br>  artifacts:<br>    name: &quot;app-playStore-release&quot;<br>    expire_in: 1 day<br>    paths:<br>      - artifacts<br>  tags:<br>    - &quot;docker&quot;<br>  environment:<br>    name: Production<br>  when: manual<br>  allow_failure: true<br>  needs: []</pre><p>Reached the end of the CI/CD rabbit hole?</p><p>Hope you enjoyed the dive into these powerful tools! Drop a comment below if it helped, sparked questions, or reminded you of your own CI/CD adventures. And hey, if you spot any missteps, don’t hesitate to correct them — knowledge shared is knowledge amplified!</p><p>Remember, the next stop is deploying straight to the Play Store — stay tuned for that!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1cbfda2cfa9b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How GPT Can Boost Your Efficiency as a Developer]]></title>
            <link>https://medium.com/@theMajesticMj/how-gpt-can-boost-your-efficiency-as-a-developer-6585ed9b7922?source=rss-ee4389407aa5------2</link>
            <guid isPermaLink="false">https://medium.com/p/6585ed9b7922</guid>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[gpt]]></category>
            <category><![CDATA[developer-tools]]></category>
            <category><![CDATA[dalle-2]]></category>
            <category><![CDATA[generative-ai-use-cases]]></category>
            <dc:creator><![CDATA[Maede Jalali]]></dc:creator>
            <pubDate>Tue, 28 Nov 2023 10:05:29 GMT</pubDate>
            <atom:updated>2023-11-28T10:05:29.271Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2IJmHhEYaOeKllH5uiKEgw.png" /><figcaption>The image of a programmer produces by DALL-E</figcaption></figure><p>In the midst of discussions about AI taking over jobs, there’s a different perspective worth exploring. It’s not about AI replacing humans but rather about those who know how to use AI replacing those who don’t. As an Android developer, I’ve made an effort to gain hands-on experience with AI and prompt engineering. In this article, I want to highlight some practical use cases of generative AI in my daily life as a developer.</p><h3>Optimizing Code Reviews</h3><p>During code reviews, I often come across functions with excessive nesting or unnecessary complexity. Unfortunately, time constraints prevent me from refactoring these functions myself. That’s where GPT comes in handy. I delegate the refactoring task to GPT, and then I review the generated code to ensure it hasn’t introduced any issues.</p><h3>Streamlining Repetitive Classes</h3><p>In every codebase, there are certain classes that make regular appearances. For example, Android apps commonly require a <em>PermissionHandler</em> class to manage user permissions. This is a repetitive task that GPT can handle effortlessly.</p><h3>Efficient Unit Test Creation</h3><p>GPT excels at crafting precise unit tests. By providing a clear prompt, keeping function bodies concise, and specifying the desired testing library, GPT can generate effective unit tests.</p><h3>Decoding Legacy Codes</h3><p>Legacy codes in a codebase can be time-consuming to read and understand, often featuring excessive nesting and deprecated code. When faced with such challenges, I turn to GPT. By providing context, I can break down complex legacy code into smaller, more understandable segments. I can even ask follow-up questions, such as suggestions for replacing deprecated functions.</p><h3>Figma Design to Code Transformation</h3><p>While GPT may not provide the exact code for a Figma design, it can offer a helpful starting point. It generates a skeleton code, saving developers from starting from scratch.</p><h3>Effective Debugging and Problem-solving</h3><p>GPT proves to be a powerful tool for debugging and problem-solving. By supplying it with code, context, and error messages, GPT often guides me directly to the root of the problem.</p><h3>Simplified Documentation</h3><p>As a developer, documentation is a crucial task. GPT can assist by rephrasing text to sound more natural and reader-friendly. Even this article has been refined with the help of GPT.</p><p>In summary, GPT is a potent tool for developers. Embracing its capabilities is akin to upgrading from a basic text editor to the powerful IDEs offered by JetBrains.</p><h4><em>Join the Conversation: Comment Below!</em></h4><p>If you found this article insightful, drop a comment below! I’m gearing up to publish a follow-up article on prompt engineering to help you make the most of GPT. Stay tuned for more coding insights!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6585ed9b7922" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>