<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by B2B Product Management Journal on Medium]]></title>
        <description><![CDATA[Stories by B2B Product Management Journal on Medium]]></description>
        <link>https://medium.com/@starmancallin?source=rss-1f217f73e23f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*xaYu1MKHtZ05Auel_Mvsuw.png</url>
            <title>Stories by B2B Product Management Journal on Medium</title>
            <link>https://medium.com/@starmancallin?source=rss-1f217f73e23f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 09:27:41 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@starmancallin/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[What working on AI reliability taught me]]></title>
            <link>https://starmancallin.medium.com/what-working-on-ai-reliability-taught-me-6b3a0c3f48cd?source=rss-1f217f73e23f------2</link>
            <guid isPermaLink="false">https://medium.com/p/6b3a0c3f48cd</guid>
            <category><![CDATA[ai-product-management]]></category>
            <category><![CDATA[ai-reliability]]></category>
            <category><![CDATA[ai-evaluation]]></category>
            <category><![CDATA[ai-code-review]]></category>
            <category><![CDATA[ai-observability]]></category>
            <dc:creator><![CDATA[B2B Product Management Journal]]></dc:creator>
            <pubDate>Thu, 23 Jul 2026 16:34:16 GMT</pubDate>
            <atom:updated>2026-07-23T16:44:03.805Z</atom:updated>
            <content:encoded><![CDATA[<p>After <a href="https://medium.com/@starmancallin/what-shipping-real-ai-features-taught-me-c428fc1899df?sharedUserId=starmancallin">building various AI, agentic, and MCP features</a>, I’ve learnt that asking AI to do the things we want, consistently, with quality, and at scale is actually a lot to ask. What made it harder for us is that the teams don’t naturally have the data to prove any of it. The industry may call it AI quality, AI code review, or AI evaluation. To us, we just wanted to try everything to get the codebase right with the structure we have. And here are my field notes.</p><p>TL;DR</p><ul><li>Part 1 — Observable at scale</li><li>Part 2 — The crucial gate of Tech Lead</li><li>Part 3 — Work with the quirks of your model</li></ul><h3>Part 1 — Observable at scale</h3><p>When we first started working in a fully AI-generated codebase, we bravely and boldly had little visibility into what AI was actually doing with the code. We knew how many tokens were burnt, but which file did it touch or which tools did it call were not being actively monitored. Without that, we couldn’t say whether AI was making our software better, the software that our human developers spent a good 15+ years to build. It was clear to me if AI jeopardises the codebase that’ll be the end of the developer trust.</p><p>So the first investment had to be <strong>observability: capturing all the data points that can help us paint a picture of code quality</strong>, e.g. prompts, tool calls, code changes, per session data, etc. It’s not exactly like a cat cam that shows you what your cat is up to when you’re not at home. But it does show you traces of everything they did, which lays the good visibility foundation. On top of that, there’s the <strong>acceptance rate</strong>, which deserves a closer look.</p><p>By acceptance rate, I mean when Claude suggests an edit, how often does a human developer accept or reject it, be it a simple change or when it becomes a commit. Cuz every codebase has its history, if an experienced developer doesn’t agree with an AI’s suggested edit, that’s enough of a reason to calibre. We don’t want to encourage rejection, but we want to<strong> use these early quality signals to build up the AI model and quality pipeline so one day we don’t have to reject</strong>.</p><p>What this also does is that it asks us both product and engineering to self-evaluate our input, which will come in more detail in part 3.</p><p><em>Limitation as of 2026: by default, Claude counts an accepted edit as indefinitely accepted, even if that code is rejected later, e.g. during review. So this requires diligence of human.</em></p><h3>Part 2 — The crucial gate of Tech Lead</h3><p>Quality must be by design. And that starts from humans who understand the organisation’s standards and the history of the codebase.</p><ul><li>It’s not just about not letting the same model that wrote the code to write the test.</li><li>Or about enforcing PM to raise ticket quality and gate check before spec goes through</li><li>Or about setting up code review skills.</li></ul><p>It’s everything together and with a key figure in the centre. To me, that’s the Tech Lead.</p><p><strong>A diligent Tech Lead challenges the code</strong>. Already in classic software development, we knew to “never submit any code you don’t understand”. That basically meant to not randomly google or copy someone else’s code from Stackoverflow. And it gets down to the Tech Lead to examine and reinforce that. Today, the same rule applies, of course with a new meaning of “don’t take AI generated code at face value”. A Tech Lead with quality in mind will know when to ask: <em>can you take me through what does this line of code do?</em> This flushes out questionable code along with the sloppy minds behind it. But if the Tech Lead lets the code slip, there is no quality gate left, and the AI takes full control.</p><p><strong>What a responsible Tech Lead also does is assigning on-call developers</strong>, especially in real agentic programming. An on-call developer is almost the first responder when something breaks in production. This is someone up for the challenge of all situations, error rates go up, latency spiked, etc. In AI programming, this is someone who makes sure AI is meeting expected behaviours or else they must fix it fast.</p><p>This loops back to the acceptance rate from earlier cuz it’s almost the same underlying principle: human still decides quality.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qOImHjkKnNQ4jewCZp8kJQ.png" /></figure><p>We could include two more areas in AI quality or evaluation:</p><ul><li>Security and compliance: guardrails, detection for e.g. prompt injection or data poisoning.</li><li>Operation: cost per session, load, cost per error, audit logs.</li></ul><p>Though these are important aspects for scalable and true agentic development, the underlying principles are the same: observability and human quality gate.</p><h3>Part 3 — Learn the quirks of your AI</h3><p>You get to know an AI’s quirks the way you get to know a colleague’s, and then you learn to work with them. If you had told me that Claude is not particularly good with <em>negation statements</em> a few months ago, I wouldn’t have an idea what you’re on about.</p><p>In our ticket, we fed Claude the rule of “user name cannot be null”. This simple negating logic turned out to be so confusing to Claude that it built, tested, and let the test past a 180 degree opposite logic. What happened was,</p><blockquote>Intent: username_should_not_be_null</blockquote><blockquote>Test logic: username ≠ null</blockquote><blockquote>Test: assert username = null</blockquote><p>We found out that if the ticket says “validate that username should not be null”, it’s ambiguous to a machine. Instead, leave no room for ambiguity, next time say “reject any record where username is null <em>and accept </em>records where username is a non-empty string”.</p><p>This was curiously unexpected, but we were lucky enough to learn a vivid lesson of prompt hygiene and better spec management. Now there’s a pre-processing layer that scans the ticket for ambiguous phrasing and normalises the wording before feeding into Claude. Essentially, a guard between the many semantic probabilities and the LLM that invents and assumes freely if given the chance.</p><h3>Lessons learnt</h3><p>If there should be one theme joining all three parts: AI quality is built by design and by human:</p><ul><li>Observable dataset</li><li>Acceptance rate</li><li>The Tech Lead diligence</li><li>Working with your model’s particularities</li></ul><p>Thanks for reading and I welcome your comments.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6b3a0c3f48cd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Not knowing how to lose is not knowing how to win]]></title>
            <link>https://starmancallin.medium.com/not-knowing-how-to-lose-is-not-knowing-how-to-win-d5602f1859f5?source=rss-1f217f73e23f------2</link>
            <guid isPermaLink="false">https://medium.com/p/d5602f1859f5</guid>
            <category><![CDATA[leadership]]></category>
            <dc:creator><![CDATA[B2B Product Management Journal]]></dc:creator>
            <pubDate>Sun, 19 Jul 2026 13:44:04 GMT</pubDate>
            <atom:updated>2026-07-20T10:40:40.339Z</atom:updated>
            <content:encoded><![CDATA[<p>Blaming the manager again is the real English football tragedy: scapegoating the most visible person as a way to avoid diagnosing the system. Knowing how to lose is the key to win, in football and in business.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*EO9loJmH2PcmTOZ8" /><figcaption>Source <a href="https://www.theguardian.com/football/2026/jul/15/thomas-tuchel-england-argentina-semi-final-world-cup">https://www.theguardian.com/football/2026/jul/15/thomas-tuchel-england-argentina-semi-final-world-cup</a></figcaption></figure><p>England wraps up the 2026 World Cup with a much disappointed 3rd place medal. On a one-goal lead against Argentina in the semi-final, England’s 20 mins of heartbreaking controversial defence became what all England fans could remember. Another broken English dream; another “oh so near” after 60 years of defeat. Something just doesn’t change.</p><p>What also didn’t change was how quick England fans and media jumped to blame the manager Thomas Tuchel. Same as what they did to Southgate, and Eriksson, and ones before them.</p><p>Growing up watching English football, I’ve had my fair share of this blame game and I understand fully what I was doing:</p><ul><li>Didn’t want to admit that we are not better than the opponent.</li><li>Let the long due victory lead to frustration and aggression.</li><li>Couldn’t even bring myself to blame the heroic and talented Beckham or Gerrard or any of the 11 footballers on the pitch. The collective cause was way too complicated than simply microscoping everything the manager does.</li><li>It washes away the sins of the immense pressure and expectation that the media and us fans put onto them.</li></ul><p>Seeking closure for a major fail, needing the result to be justified to our passion and dream, lost group identity when the team gets knocked out - all the anger and grief are understandable. As someone who has the highest status and being the most visible representative of the result, the manager shall indeed be the nearest possible target to blame.</p><p>There’s so much in common between football and business, in both, the problem about attributing blame is that you are almost bound to get it wrong.</p><ul><li>It distracts the attention away from diagnosing the system and finding a solution;</li><li>It underplays the situation: we lost because of just one factor - the manager/the PM/the bug/that one missing feature;</li><li>It delays/avoids action: if we don’t need to face the reality, we don’t need to fundamentally change for the better.</li></ul><p>Between 1996 and 2018, England didn’t win any penalty shoot-out. It became a frightful self-fulfilling 22 years of curse. Gareth Southgate changed that with a 4–3 against Colombia in the 2018 World Cup. He made history by exactly facing the reality. He did a study of penalty: hired a team of 5 analysts carefully studied over 100 penalty shout-outs, analysed all the acts, mechanics, psychological factors of taking a penalty, and translated that into something coachable and practicable. He fundamentally changed the way England takes penalty. This was sound winning instead of keep losing to conventional wisdom or the fan’s blaming of “choosing the wrong players”, which was not for one bit close to the solution. To this day, Southgate is the record holder of the England manager who won the most major tournament games in history.</p><p>We don’t often do that in business. Yes, we analyse data and strategise. But we hesitate to face reality and make fundamental changes that could make history.</p><p>Now that I&#39;m much older and hopefully just a tiny bit wiser, I&#39;m asking if blaming is the best we can do, perhaps we don’t deserve to win, in football or in business.</p><p>To win</p><ul><li>You must face the reality: we are not good enough to win a major tournament. This means we must still work hard; it means this is not our time; it means we must wait while we train to get better at it. While this has “unacceptable” written all over it, even more so in the business world, it’s exactly this reality that triggers utterly necessary change. How did we lose a game on a one-goal lead? How do we carefully analyse and design a clear path to retain the threat next time around?</li><li>You must take the risks to try and fail. Tuchel rightfully analysed that after leading with one goal, England “suddenly played like we have a lot to lose” while Argentina was “taking on a lot of risks”. When protecting a leading position, some become risk-averse. To their own detriment, any loss at this point looks twice as large… This is the missing of growth mindset paired with generational crooked British media incentive, which even if he wants to, is larger than Thomas Tuchel. This is so true in business: we want to innovate, but let’s stick with what we know.</li><li>You must play to win, not to not-lose. With 60 years apart from the last victory in an international major tournament, the England team might have lost or haven’t learnt the winning mindset. When under pressure, it’s so sadly visible to see how the 11 best footballers of the nation make their decisions based on fear instead of what would lead to more goals. Same with a company missing winning culture, people rather not win than lose.</li><li>You must have luck. As luck cannot be controlled. This one has “unacceptable” written all over it too. But winning in football, or in business or life, requires more than talent. The team with the best statistical chances based on pre-tournament studies rarely won in the end. So what sort of learning is this? It’s part of the growth - don’t linger, move on.</li><li>Another thing that is even less controllable than luck: the leader actually only decides on a small portion of the success. You can sack Tuchel and bring on the #1 football manager in the world, it just still may not come home. There is no new learning for the leaders - we already know too well what’s the real action and what’s to be declined, as Tuchel rightfully put: “If the blame game needs to be played, OK, we can do that. But I have the right to not engage.”</li></ul><p>A true England fan is truly self-critical. So as that’s done, I’m proudly pointing out more than just a little winning quality this England team has demonstrated:</p><ul><li>Team excellence over superstar power: their togetherness is the first I’ve seen in a lifetime (speaking for millennials). It lays out the brilliant foundation for the production of 4 goalscorers with the striker Kane and midfielder Bellingham scoring equally 6 goals. This is a team that truly respect and work well with each other. No frustrated top performer, no questionable individual either.</li><li>The hunger and passion: English football has long created the unintentional “don’t want to win” impression. This team was hungry, even ferocious in times. They’ve not only produced incredible football to watch. It’s undeniable that you could finally see the shape and path to an ultimate victory. To any manager that’s leading a team like this, it’s a true blessing.</li><li>The adaptability and resilience: nearly all comments and projections were betting against England ahead of the game against Mexico at over 2000m altitude: England would not have time and energy to climatise, Mexico had never lost a home game there… And England delivered a historical win.</li></ul><p>Football taught me how to lose, at a very young age. This <em>perk </em>of being an England fan is teaching me how to win. A lesson that hopefully being joined by many others. Keep going, England. See you in 2028.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d5602f1859f5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Empower your team when the ground is shaking]]></title>
            <link>https://starmancallin.medium.com/empower-your-team-when-the-ground-is-shaking-ff5bcb7c7818?source=rss-1f217f73e23f------2</link>
            <guid isPermaLink="false">https://medium.com/p/ff5bcb7c7818</guid>
            <category><![CDATA[leadership]]></category>
            <category><![CDATA[change-management]]></category>
            <category><![CDATA[product-leadership]]></category>
            <category><![CDATA[organisational-culture]]></category>
            <category><![CDATA[ai-transformation]]></category>
            <dc:creator><![CDATA[B2B Product Management Journal]]></dc:creator>
            <pubDate>Wed, 15 Jul 2026 14:39:01 GMT</pubDate>
            <atom:updated>2026-07-15T14:40:09.629Z</atom:updated>
            <content:encoded><![CDATA[<p>A 4-piece series for product leaders, and the PMs who’d like to become one. Written for a talk at ProductTank.</p><ol><li><strong>Change the game </strong>— giving your team something to win when the market won’t let them</li><li><strong>What is she giving me? </strong>— the question your team is silently asking about you</li><li><strong>Anger and tears I can handle </strong>— fear is the one that breaks a team</li><li><strong>The pivot nobody noticed </strong>— how to move a team before they feel the ground shift</li></ol><p>A note before we start: I’m sharing what worked for one team, in one fairly specific corner of hell. Take what’s useful, ignore the rest, and assume I got at least a few things wrong along the way — because I did.</p><h3><strong>1. Change the game </strong>— giving your team something to win when the market won’t let them</h3><p>There’s a graph I wish someone had drawn for me earlier on:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VbNjqP1UbBy-Et4L5CEHhQ.png" /></figure><p>On one axis, ambition; on the other, reality. The company sets a goal. And goals being what they are, sits a little higher than what’s achievable. Every product leader knows how to live in it.</p><p>But now draw one more line. Ask your sales, service, customer success, support what they expect from the product team. Their line doesn’t even sit near the company goal but sits above everything.</p><p>Why? Not because stakeholders want to be unreasonable. They’re under immense pressure too: the pipeline, the bookings, the go-lives, retention and CSAT. They need the product team to be the answer. And <strong>the bigger their challenges, the higher they expect from the product</strong>, under the unwritten conditions of:</p><ul><li>It’s not their job to assess what’s plausible or not</li><li>Among themselves, no alignment on each of their own demands is needed before reaching out to product</li><li>Product must serve the departments fighting at the front-line</li></ul><p>This expectation stays high for as long as their own pressure stays high, which can be rather long in a struggling market.</p><p>A few words about the market. If your opportnites were also wiped out in 2025 too due to macro economical fators, such as the <a href="https://www.london.edu/think/what-the-esg-backlash-reveals">ESG backlash</a>. Please skip. I wouldn’t want to pain you even more. ESG had been the poster child prior to 2024 with some 50,000 companies addressable in EU alone. Momentums kept picking up with investors and regulators. New ESG software continousely popped out and immediately got acquired. It was <em>the </em>renvenue wagen that companies jumped on, including ours. At the turn of 2024, our company acquired a Canadian born ESG software, with it came the product team in question. Inheriting this team was one of the best things that happened to my career, even considering the immediate market freefall in 2025, where we were left with 20% of the original market size in a red sea crampped with software solutions. To make matters worse, the Canadian product and organisation were still yet to be digested by the larger European parent while all this is happening.</p><p>This gap of stakeholder expectation and what we could deliver started to affect the team. When they work hard every day, and nothing they ship seems to move the needle, escalations became more and more frequent and fierce, something breaks inside. <strong>It’s not their performance:</strong></p><ul><li>Highest velocity, localised the Canadian software to EU customer and security needs</li><li>Almost each month, there is a major release so big we ran some 70 enablement sessions that year</li><li>Became the AI pioneer team in the company.</li></ul><p>We still couldn’t fill the void, in contract to our performance, the reception was poor:</p><ul><li>Being asked for more timelines they couldn’t give, on problems they hadn’t caused</li><li>Names appearing in non-stop escalation threads</li><li>Individuals became the nearest available thing to blame when a customer or deal goes wrong.</li></ul><p><strong>So what breaks was their doubt of whether their effort is still connected to outcomes</strong>. This is a painful problem that the usual fixes don’t work: better prioritisation, faster velocity, deeper discovery — none of it closes a gap that we didn’t open.</p><p>I’ll be honest about how that felt without overdoing it: it was the kind of grinding, no-exit pressure where however fast you run, the ground moves the other way. And sitting in yet another weekly de-escalation meeting, I realised the thing that never occured to me: I had only been improving our execution. <strong>But we needed was to change the game: </strong>the team needs a short term direction that’s real enough that they can actually move toward to, inspired by, and can create customer outcomes for. Changing the game doesn’t mean ignoring the commercial reality. Although eventually the market will bounce back, the team needs something, now.</p><p>The useful question for me to ask definitely wasn’t “So we add AI to our product then?” AI became the means, but the truth lived in our own customer knowledge: <strong>look honestly at which factors actually move the needle for our customers in a way that’s more or less independent of the macro environment</strong>. For us, the answer turned out to be “AI-driven intelligence, data quality, and automation”. Surfacing genuinely high-value data insights for decarbonisation, eliminating manual setup and configuration that affect the Total Cost of Ownership, the kind of thing a customer needs whether the market is growing or not.</p><p>The rest was transformation and team empowerment, which I’ll continue to discuss in the next chapters. But the results were satisfactory:</p><ul><li>We shipped the company’s first Agentic AI feature</li><li>Shipped the only cross-product line solution</li><li>By the end of 2025, everyone had worked on their first AI feature(s)</li></ul><p><em>On a side note cuz we are on AI. </em>It wasn’t AI that fixed our problems. The high stakeholder expectations and frustrations are human; an AI chatbot can’t answer questions nobody bothered to define, or resolve a tension that’s interpersonal; lastly, encouraging stakeholders to use the shiny new AI agents you built but you may end up still getting questions tangled in historical factors only human can understand. But AI did give the team something they could discover, build, and prove. The external factor couldn’t take that away from them.</p><p>So if there’s one thing I’d offer from this first piece, it’s this: empowerment doesn’t start with autonomy or budget or better processes. It starts with a direction the team can believe in, chosen by <strong>a leader who’s honest about why the old direction stopped working</strong>.</p><p>If your team is working hard and watching nothing change. Ask yourself: what game can you give them that they can actually win?</p><h3>2. What is she giving me? — The question your team is silently asking about you</h3><p>Every person you lead is running a continuous, silent assessment of you. Not because they’re disloyal, they’re working out whether following you is worth the cost. The good news is that <strong>they especially judge you more when they respect you</strong>.</p><p>Our team grew from 6 to 10, no one left throughout the market challenges and organisational restructurings following the M&amp;A. What each individual base their devotion on is simple: what is she giving me?</p><p>Not necessarily money or title, although I wouldn’t pretend that it doesn’t matter. What they’re really asking is whether following you <strong>makes them more capable, secure, and able to do meaningful work they’re proud of</strong>, including finding work elsewhere in the future.</p><ul><li>Does your mission give them something to aim at?</li><li>Do you build conditions for them to work free from judgetment or fear?</li><li>Do they understand the standard? Do they have examples to look up to?</li><li>When pressure comes, do you absorb and translate it, or pass it straight through to them?</li></ul><p>I’ve asked myself those and many more questions, as I needed to know why do my team follow me through the thick and thin: re-organisations, disporportionate internal aggression, high pace, overlooked achievements, 6hrs timezone difference… but zero attrition. Eventually I was able to find out through <strong>anonoumous 360 feedback</strong>. It was a precious moment of clarity and confirmation. It kept me going the direction the team needs.</p><p>I highly recommend a 360 feedback if you are searching for the same answers.</p><p>I’m going to speak about <strong>authority building</strong> now. First of, please don’t be afraid of the word authority or treat it as a taboo. Here’s a distinction I take rather seriously: I don’t exactly need to be friends with anyone in the team. But I need to be able to have a direct conversation with whoever that needs a serious word without tiptoeing around it. I need to know who quietly needs acknowledgement and can’t ask for it. Who’s performing perfectly well but slowly losing motivation. Who’s stuck and waiting to see if I’d noticed. <strong>And I did this with ultimate respectful directness right from the start that everyone accepted</strong>. They are not afraid of me, but they know I’m serious with my words.</p><p><strong>This needs building. </strong>In each meeting, there’s one consistent message: your requirement for quality, ownership, and work standard are cleary explained and shown witout slipping or cutting slack for even once; And in 1:1s, you’re doing the opposite: listening for everyone and everything the other meeting can’t surface. The two modes only work together. Drop either you become unfair.</p><p>Now, the part where I know I’ll get criticised. I run team meetings more often than many product leaders — 15 hrs a week in team meetings, for some people, we speak 1:1 for more than once a week. I’m aware of how that sounds, especially the scalability side of things when you have not 10 but 20 team members or even more. But this is the way I found best value for money. We can discuss individual matters, shift behaviours, explore technical issues, draw conclusions, etc. Do everything to move progress forward, and fast. On the other hand, <strong>your team’s needs don’t disappear just because your schedule is full</strong>. This matters most during changes. Andin this economy, change is rather constant. So if you can’t be the one offering hands-on mentoring, please, please, please, make very sure someone whose judgement you trust is doing it.</p><h3>3. Anger and tears I can handle — fear is the one that breaks a team</h3><p>I’ve had people show their anger out of frustration and being missunderstood after aggresive stakeholder meetings; I’ve had people cry in front of me due to the build-up of constant public escalations or weeks of crushing defeat. I would be lying if it didn’t break my heart too. But you listen and help them recover. You even jump into issues and clear the way for them. To me, anger or sadness aren’t problems at all. It shows there are energy, passion, and effort still in their system. <strong>But fear is the one I’ve learnt to take seriously.</strong></p><p>Fear doesn’t really announce itself. You hardly notice at the start:</p><ul><li>Quietly setting their Teams status to offline while they’re clearly working, because they don’t want to be found by stakeholders</li><li>Taking a sick day on the day of a difficult alignment</li><li>Not responding to stakeholder enquiries to stay safe.</li></ul><p>They don’t have a performance issue, rather they’ve learnt that engaging produces pain and no resolution. When every complaint becomes a public thread, every bug risks an escalation, and every small improvement is scrutnised by a big group of people, you gradually stop trying things and learn to be safe. <strong>And learning to be safe is a killer of innovation</strong>, or product management in its entirety. That’s where the real damage lands.</p><p>Fear forms long lasting impacts. The person who learnt to dodge conversations or pad timelines because honesty got punished would likely still do so many jobs from now, <strong>unless you help them in time</strong>. Spend enough time somewhere that treats every mistake as a public incident, and you stop believing you’re capable, you stop speaking up, includig calling out problems that can be caught early or ideas that would’ve been useful.</p><p>If you are the leader at this point, don’t give an already frightful person autonomy, it only encourages them to hide even deeper. The fear has to be dealt with first.</p><p>While I value psychological safety, it needs time to build. Customers and stakeholders aren’t going to wait around. You need to go in yourself and solve the problems, but <strong>make sure the team members watch you, so you can show them how this is survivable</strong>. You analyse, conclude, and make mistakes, keep progressing until the issue can be closed. Only afterwards, have the 1:1 conversation in the safest setting to help them refelct, and what they’d need to do the next time in a similar situation. Gague if they feel good enough and have enough tools to be autonomous again. If not, don’t leave them alone.</p><p>Product people need to engage: act, lead, push back act. When someone who used to engage has gone quiet. That could be a signal. Apart from giving encouragement or reassurance, it calls for the product leader to go in and show them the way out.</p><h3>4. The pivot nobody noticed — how to move a team before they feel the ground shift</h3><p>Pivoting to me doesn’t mean an announcement, an offsite workshop, or a shiny new roadmap. They only give you noddings in the meeting room but everyone carries on as before.</p><p>In 2025, we did a series of lasting actions to shift our focus and way of working, without any big ceremony or mandate. We were able to successfully transition to AI-native way of working and shifting gear as said in chapter 1. If I must put words to it, our pivots were <strong>deliberate and natrual enough that no one felt the floor move. By the time they noticed they were already somewhere new and ready</strong>. I don’t mean to say that the classic annouce -&gt; change -&gt; stablise method doesn’t work. It works in scale and perfectly top-down, which has it’s suitable scenarios. But my team was not in a position to meet that sceanrio. Our product organisation also didn’t get the kind of central attention or support that merits a company-wide annoucement for transition.</p><p>From a place of understading each individuals of my team, I know what homework I needed to do before shifting — starting from already solving the puzzles the team would definitely get stuck in</p><ul><li>Do I really understand this new direction enough to stick to it?</li><li>What do I not know yet? What does good look like?</li><li>Was the old way so wrong that we need to change? What do I get out of this new direction?</li><li>What does this mean for my standing in the team?</li><li>Are we protected enough to try new things? etc</li></ul><p>To us, the most effective and immersive way was simply consistently change bit by bit, week by week.</p><p>We have our Product and UX team weekly every Tuesday. During the meeting, there’s always a session where individuals freely share new findings, i.e. customer interview insights, loss deal reasons, and industry news. We share disappointments of further ESG pushback; we share curiosities of new customer needs being undercovered. The key was to <strong>share</strong>, especially for a team that stretches across a 6hrs of time difference. I found this a stable and safe foundation to build on. So every few weeks, I would add a new question or topic:</p><ul><li>Did anyone notice any difference of the new LLM model?</li><li>How do you save tokens?</li><li>What do you put in your Claude.md?</li></ul><p>Only one person would speak, perhaps just the once. But the next week and next, by consistently asking, there’d be more people. Within a month or two, we were spending decent time on sharing new practices and at this point, everyone acquired a good level of understanding of what AI intelligence means, even how to build it. I kept this level of sharing as it ensures that the change lasts. In the meantime, more must be added on top.</p><ul><li>I added something called “Ticket closed” to the sharing session. The aim was to let individuals learn from each other’s experience when someone was able to close a ticket without stakeholders complaining. We receive numerous new feature and bug fixing requests, but not all needed action upon. So we would close the ones that we wouldn’t do with a short explaination in the ticket comment. But when we do, stakeholders would swamp the comment and tagging us non-stop hoping to assert enough pressure for us to agree to develop it. After doing a ticket analysis, one of the individuals in the team has hundreds of open tickets assgined to him. But due to fear of escalation, he kept them that way without closing or reacting. That was enough of a signal for me to act. This “Ticket closed” session became a hands on peer learning experience that eventually brought down the number of tickets assgined to each individuals.</li><li>I further added a 10–15min knowledge presentation session. In our precious 1hr per week meeting, I needed everyone to take away something each week, no time wasted. I used to share how to build with AI or how a competitors software automate tasks. <strong>But it was never as effective as one of their own shares it</strong>. Peer learning has been at the centre of our team growth. It wasn’t easy to make up 10–15min extra time per week. So we shortened regular development progress update by implementing a Kanban board with visible progress and ownership at all times. This freed up more than 15min which now became a stable live presentation of everything: building with Claude, collaborating on merge requests, how to write a new skill, etc.</li></ul><p><strong>This needs building.</strong> You can’t force a sharing session into a team meeting out of the blue that doesn’t look natrual. <strong>But in each team, the leader should be able to find a similar foundation to build on. </strong>The sharing also wouldn’t continue if you stop encouraging every imperfections, even mistakes. In addition, it requires individual nudges during 1:1s to ensure there is at least one candidate who will share per meeting. So I’ll tie this back to chapter 2, where you must operate both group meeting and 1:1s deliberately to make it work.</p><h3><strong>BONUS: When to roll up your sleeves</strong></h3><p>There’s a school of thought that says your job is to create the conditions and then step back. I almost mostly believe it. But there are moments when the most empowering thing you can do is to go in first yourself.</p><p>When we moved toward our first customer-facing AI feature, engineering and product were genuinely discovering together, as neither had done this before. No one in my team had yet the steady hands to kick this off and drive it home. So I built the first PoC myself, including sorting out the problem and outcome statements, journeyes, trees and what-not in Miro. <strong>There was a direction of travel to it.</strong> And it looked feasible enough for proper assessment and analysis. That to me was time to hand over to an individual, with me observing closely and back them up whenever needed. There were indeed hurddles during the execution that I needed to go back in. <strong>But I knew it clearly that the end result was coming — someone in the team shows their work that is not me</strong>, cuz watching a team mate do something imperfectly and iteratively teaches the team far more than watching me do it.</p><p>I’ve personally dealt with bugs and small improvements at times; I’ve also trusted individuals with major regulation driven features without interfering; I’ve done months of 1:1s just about the individual bouncing ideas with me and going forward with more certainty. It really helps to be honest about when going in is right, as either “empower” or “be hands-on” seems meaningless without context. And as a leader, while your judgement is crutial, you cannot be too wrong if you understand the individuals in your team very well.</p><h3>The beautiful moment</h3><p>The moment the team realises they’ve grown, and so natrually without disruption, and not because you told them, but because <strong>they did it</strong>, is the most beautiful moment for a truly empowered team. And it’s a wonderful feeling to be part of this team, for them, and for you.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ff5bcb7c7818" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What shipping real AI features taught me]]></title>
            <link>https://starmancallin.medium.com/what-shipping-real-ai-features-taught-me-c428fc1899df?source=rss-1f217f73e23f------2</link>
            <guid isPermaLink="false">https://medium.com/p/c428fc1899df</guid>
            <category><![CDATA[ai-transformation]]></category>
            <category><![CDATA[ai-product-management]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[product-management]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[B2B Product Management Journal]]></dc:creator>
            <pubDate>Tue, 14 Jul 2026 18:35:09 GMT</pubDate>
            <atom:updated>2026-07-23T16:34:49.536Z</atom:updated>
            <content:encoded><![CDATA[<p>It’s been 1.5 years since we shipped the first agentic AI feature, and now <a href="https://starmancallin.medium.com/what-working-on-ai-reliability-taught-me-6b3a0c3f48cd">building on an AI-generated codebase</a>, while it’s largely still a journey of finding out as we go, there are some field notes to share out of the sheer amount of lessons learnt.</p><p>TL;DR</p><ul><li>Part 1 — <strong>Multi-agent</strong> or not?</li><li>Part 2 — <strong>MCP </strong>or not?</li><li>Part 3 — <strong>Evaluation</strong> or not?</li></ul><p>This article is technical. But technical choices only make sense once the commercial ones are settled. So let me lay out two foundations first.</p><h3>Foundation 1: the quickest, biggest value first</h3><p>This is the principle any product person hold long before AI. With AI, even though customers and investments are pushing to <em>get something out of the door</em>, what we release first is not the most sophisticated AI capability, but the biggest customer value that you can deliver quickest. This decides most of the architecture questions later in this article.</p><h3>Foundation 2: what iteration really is</h3><p>We already don’t want to release an unnecessarily bloated MVP. With AI, this gets harder as there are so many traps where a solution can get over-engineered instantly.</p><p>Here’s a small story inside our team. <strong>We internally use the word “skateboard” for our first release. </strong>It came from the idea illustrated in the image below. (Apologies that I can’t be sure who owns this image. If you are, or know who, please contact me.)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tlPU4V8MccxUJtPNAXbnOg.png" /></figure><p>We go through extensive exercises, debates, trade-offs to properly define and agree on the scope of our <em>skateboard</em>, and the subsequent iterations, i.e. <em>scooter </em>to the final <em>car</em>. This became our internal roadmap guidance and the reminder to keep things valuable, simple, and iterative.</p><p>With those two foundations in place, here are the three cases where I’ve learnt the most.</p><h3>Part 1 — Multi-agent, or not?</h3><p>Our very first AI release was a multi-agent, MCP orchestrated agentic big bang. In hindsight, I’m asking did we actually need that right from the start?</p><p>For clarity, <strong>multi-agent here means several specialised agents working together, with a supervisor agent coordinating them</strong>. In addition, there could be an extended agent that assign further tasks, including evaluating the results, which I’ll discuss more in part 3.</p><h4>What we built</h4><p>At the end of 2024, we built exactly that: a conversational assistant with a <em>supervisor orchestrator</em> sitting in front of a set of <em>specialist agents</em>, each are designed for specific tasks, e.g. text generation, recap, proofreading, and one acted as a catch-all for all invalid inputs. We even set up LLM memory properly: The <em>orchestrator </em>was wired to tooling and to a model that counts and summarise every 10 conversations, so the LLM could preserve conversation memory without running the entire script on each prompt. It was a neat, intelligent, and technically comprehensive multi-agent layer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oHsAicPmdkoP06OAtrAN6Q.png" /></figure><p>But that’s exactly why this is worth examining, because <strong>we did not need to build the full layer to deliver the value.</strong></p><h4>The cheaper option that would have worked</h4><p>There is no doubt that we build products based on solid understanding of customer cases. And our discovery had already told us what customers wanted this assistant for: text generation, recap, proofreading... With this, the decision tree should’ve kicked in:</p><ul><li>We could build a supervisor orchestrator agent to automatically interpret user’s intention and choose which specialist agent to call, or</li><li>We could present some speech bubbles of the actions <strong>we know customers want</strong>, and let this action call the respective agent.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Bta9avToLH7ty0c0K4W2tg.png" /></figure><p>We have certainly gone for the more costly path.</p><ul><li>Without a multi-agent structure where tasks are coordinated, <strong>disconnected single agents that don’t talk to each other could do the job too.</strong></li><li><strong>A multi-agent system without memory can also do the job, if you do not expect customers to hold long conversations</strong> with the assistant.</li></ul><p>A note on memory, since it is the part often gets added reflexively (such as us). The model forgets everything between calls by default. The feeling of “conversation” requires persisting of each turn and re-injecting the history into the next prompt. Once it gets too long, in our case 10+ prompts, there can be a step to summarise previous conversation instead of carrying the whole history. This was our effort to <strong>control tokens and cost</strong>. But again, if your interaction is one question and one answer, you may not need any of it.</p><h3>Chapter 2 — MCP, or not?</h3><p>Once we built our tooling into MCP layer from the start, it became the central orchestrator of everything. We could plug in more capabilities, and it felt genuinely powerful. But this is the decision I would most want to revisit if we get to do it again.</p><h4>Choosing MCP is choosing a business direction</h4><p>Because MCP is less a technical choice than a business one. If you believe that your system have value <strong>beyond your own product</strong> so much so that customers would pay to have their own agents/models call your capabilities, even build other systems on top of it, then MCP might be the right and even necessary investment. It gives customers a real action layer and interoperability.</p><p>If you do not yet have that evidence or a clear enough vision, but simply or urgently want an agent out of the door, then you can afford to delay the decision of MCP to a later stage. In our own market, an exposed MCP layer is by far not the top priority for our customers. Building it early did risk us of carrying its weight before it even pays off.</p><h4>The technical cost</h4><p>I’d like to especially call out one hidden cost of MCP: authorisation and data security handling.</p><p>When you call a tool inside your own codebase, you have full control, as the tool runs inside authenticated requests only. The user’s session, permissions, and database transactions are live in the same memory, without leaving your process. This is the standard our customers trust and rely on.</p><p>But as soon as a call crosses a protocol boundary, it’s now in a separate process: serialised to JSON -&gt; sent over a transport -&gt; executed somewhere else -&gt; results serialised back. This means you must pass identity on every call plus validation, as the external systems don’t know who the user is. With that, latency or even call failures could come too. <strong>In hindsight, this almost deliberate security burden would have needed a lot more concrete business case backup.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mJJvHGrenv84Gb1U3cvkaQ.png" /></figure><h4><strong>Simpler alternative</strong></h4><p>To play the devil’s advocate, if we were to review the AI features we’ve built, some of them can really be exchanged into <strong>linear LLM-driven data pipelines side by side than fancy agentic workflows</strong>. Essentially, an <em>agent</em> is an LLM plus tools plus a “control loop”: the model decides to call a tool, the system runs it, feeds the result back, and the model decides whether to act again, looping until it is done.</p><p>Some use cases could find a simpler solution without such loops, e.g. if your task is “image in, structured JSON out,” that is a single step LLM call. Line up a few of these single calls together, plus system logic, many jobs can be done by LLM pipelines, without needing multi-agent or MCP.</p><p>The test I now apply: do my tools genuinely need to talk to each other and to outside systems, or can everything be done by in-process calls to my own codebase? If it is the latter, you very likely do not need MCP at all.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zzTpDnoUfn7UCnhrMw3wsQ.png" /></figure><h3>Part 3 — Evaluation</h3><p>Evaluating AI produced result has been discussed extensively in the industry. We had our fair share of trial and errors.</p><p>Last year, we built two features to bring operational data into the platform from external sources. For our customers, those data must be accurate enough to stand professional audit. Knowing that LLMs cannot be trusted with accuracy, we started building evaluation and validations.</p><h4>LLM as a judge</h4><p>One method we used is <strong>LLM as a judge</strong>. It puts two LLMs one after another, with the second LLM doesn’t generate anything, but scores the result of the first LLM. The development was straightforward: let the first LLM generate an answer, the second LLM sees that generated answer together with a <strong>provided reference answer that is the gold standard answer</strong>, and asked to rate how close the two on a scale of 1–5. Nothing special. But we still faced a big hurdle, which had not much to do with LLM — what’s the gold standard answer. <strong>AI evaluation is only as good as the reference you give</strong>. In our case, which was all about semantic similarity instead of numerical. We were cautious with the reference answers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wr7xRIkRbHtw25EELO4X4A.png" /></figure><p>That semantic scoring is genuinely useful for free text. But you can reasonably ask how far it takes you when the output is numerical.</p><h4>System validation</h4><p>When data quality is essential, validation shouldn’t be purely done by automation. For numerical values, we rather wanted the strongest validation. <strong>A LLM judge model can be part of that, but so far cannot be as robust as good old plain system validation: data schema and business logic</strong> into a structure of system, then AI, then system.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WEoRKjokd3RkofM0rIBErw.png" /></figure><h4>Human in the loop</h4><p>It was still not safe for us with the double defence layers of LLM judge + validation schema, especially from the customer’s expectations, if we own the validation, we must own data quality, which of course we cannot. So we’ve decided that all data will go to a human approval workflow in the UI, which enables our customers to review the AI extracted data before letting it into the system. And reality told us that our customers gladly accept that responsibility.</p><h3>Lessons learnt</h3><p>All in all, if there should be one theme joining all three parts: do not build the fully fledged AI on day one. Go back to our roots:</p><ul><li>Understand customers thoroughly</li><li>Carefully evaluate the most practical and cost-effective choice</li><li>Ship the quickest biggest value first</li></ul><p>Thanks for reading and I welcome your comments.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c428fc1899df" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Surviving 2025]]></title>
            <link>https://starmancallin.medium.com/surviving-2025-cecbb0103511?source=rss-1f217f73e23f------2</link>
            <guid isPermaLink="false">https://medium.com/p/cecbb0103511</guid>
            <category><![CDATA[product-management]]></category>
            <category><![CDATA[product-leadership]]></category>
            <category><![CDATA[change-management]]></category>
            <category><![CDATA[leadership]]></category>
            <dc:creator><![CDATA[B2B Product Management Journal]]></dc:creator>
            <pubDate>Mon, 23 Feb 2026 08:05:45 GMT</pubDate>
            <atom:updated>2026-07-13T15:05:59.257Z</atom:updated>
            <content:encoded><![CDATA[<p>When you look at the world right now, do you feel fear, or an odd excitement of reinvention?</p><p>I was going to find a less passive word for “surviving”, but considering my industry, location and all circumstances, surviving is the right word.</p><p>Here in Germany, in sync with the near-global economical downturn, industries suffered significantly, including my industry that was once the poster child — ESG (Environment, Social, Governance). With it, bank alliances ceased operation and investment pulled away, regulations stalled, etc.</p><p>All led to the bleak start of the year 2025, a dry sales pipeline, high ROI demands from existing customers, and incredibly stressed internal teams. To add an extra layer, in terms of revenue and team size, as a small Canadian product acquired into the massive European mother company, we were still yet to be fully accepted into the organisation and product portfolio.</p><p>It was a difficult situation to be in, but an oddly exciting one. This article is a survival guide if you ever find yourself in a similar situation.</p><h4>TL;DR</h4><ul><li>Work hard and expect the worst — Whoever dares to lead will get shot down</li><li>There is always control — A bad market doesn’t strip you off everything</li><li>Pivot, then again — Double down on adaptation than making plans</li><li>Look after people on the ground — Be responsible for the environment</li></ul><h3>Work hard and expect the worst — Whoever dares to lead will get shot down</h3><p>In an intensive situation, it could have been anyone who was the product leader, and they would have been shot to sieves. It might as well be you.</p><p>Very early in 2025, I could already feel the headwind just judging by the demeanour changes of internal teams:</p><ul><li>Impatience of collaborating with the individuals of my team. Small improvement requests came straight to myself and higher;</li><li>Feedback replaced by public pressure. We first see criticisms from group chats or email threads than 1-to-1 conversations or tickets;</li><li>Ignored prioritisation and processes. Escalations became frequent and bypassed management levels, forcing senior leadership to mediate ground level bugs without sufficient context.</li></ul><p>Personally, the battering was rough to take. The timeline and commitments were difficult to make. But that was not the issue. For our company, things and matters don’t tend to trouble us, only people. It was clear that this is a people challenge. As systemic as it sounds, this was not a task that the senior leadership took on, especially not for a struggling post-M&amp;A step-child. I had to take this one on even if that’s moving a rock using chopsticks.</p><p>Although a people challenge, the tasks at hand were rather straightforward:</p><ul><li>Be forensic. Pull and show data of hotspot issues</li><li>Relentlessly educate and insist on high work standards</li><li>Set new standards for when prioritisation and clarity aren’t possible</li><li>Review and revise processes, and again</li><li>Discourage non-productive communications but encourage reassurance, again, and again…</li></ul><p>Overtime, we were able to bring issue resolutions back on actionable level. More and more people understood that there is no individual loser who failed a project, only the challenging environment that affects all of us. In the end, will you burn out? No, not if you focus on the solutions. But letting yourself drown in criticism will do all the harm. Will you become a target? Constantly. But the end result is never for the product leader to be a likeable bottleneck. So take the lead.</p><h3>There is always control — A bad market doesn’t strip you off everything</h3><p>Any sailors reading? This one is about sailing against the wind.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FObvBDSuKNmnmy3Ef8wD9A.jpeg" /></figure><p>I’ve used the same analogy when speaking about resilience at our company’s innovation conference last year. Sailing is the one topic that gets brought up in every family gathering. There isn’t anyone in the family who hasn’t been on a sail boat, tied a fender, or pulled a sheet. In fact, the many principles in sailing are perfectly applicable in today’s business.</p><p>Long before diesel engine, humans have developed the technology to sail against the direction of the wind. But there’s no heroic story of fighting the tide or big rowing oars, it was purely done by looking inwardly — adjusting the angel of the boat to zig zag a way forward.</p><p>The market might have swallowed our addressable market, but it could not affect us from analysing internal data (If no data, gather data) to purposefully choose our investment. We were able to re-focus on features that are independent of market headwinds, e.g. identifying essential manual steps to be automated or improving the data quality and auditability. For our customers, this certainly led to higher value for the same cost of ownership, but it’s not to say a certain business metric of ours was going to be moved, such as CSAT or Net Retention. When the demand is declining, better customer experience cannot easily stack up into positivity. At best, it prevents further declining. The goal is rather readjusting the course to best allocate our resource.</p><p>For our teams, something good came out of this too:</p><ul><li>We became a data-driven team. Everyone formed the habit of pinpointing problem and outcome statements with the strong backing of data.</li><li>Everyone in the team either developed or started developing at least one AI feature.</li><li>We started quarterly customer communication webinars.</li></ul><h3>Pivot, then again — Double down on adaptation than a fixed plan</h3><p>In sailing, many conditions need to be met to guarantee an ideal trip: Consistent wind, less than 15 knots, a certain destination to dock or anchor for the night, and enough provisions in the cabin. It’s wishful thinking to plan a trip with ideal conditions in mind, or have all days planned at all. Sailors are experts of uncertainty and pivoting.</p><p>In business, you’ll seldomly be able to sit comfortably with uncertainty if you have plans, thus the making of scenario analysis, predictions, or plan Bs to better prepare for the future. It was our time to revisit the concept of planning. The new goal was to help the organisation realise that a plan/roadmap/process is a vision guiding document, not a delivery schedule. Understandably, customer-facing teams particularly struggle with that. So while we still maintain a roadmap for customer communications, internally, our product backlog is flexible on a weekly basis.</p><p>In November, the benefit was eventually seen, as data showed that nearly 50% of the features we delivered in 2025 weren’t initially planned, but picked up in response to customer relationship improvement.</p><h3>Look after people on the ground — Be responsible for the environment</h3><p>Although in the natural development of industries, opportunities rise and fall. Keep going and don’t dwell is the right mentality. But on the ground, people can experience a bleak reality over a long period of time that takes a lot more energy than they have.</p><p>Right at the beginning of the year, after a few waves stressful internal complaints, it became clear that the team was reaching a tilting point:</p><ul><li>Many began to turn their status to “offline” during working hours</li><li>Increased fear in responding to messages</li><li>A lot of sick days</li><li>Energy loss, missed instructions, becoming forgetful</li></ul><p>Because the external factors were strong — a turbulent market, fierce competition, changing regulation, and a tight economy, even the most competent and experienced customer-facing teams started doubting. Their fear of failing the projects, customer trust, or deals landed on us in an unintended form that’s very different from their original intention:</p><ul><li>Public escalations with names of individuals written all over it, several times a week.</li><li>Issues are flagged via group emails without checking directly with individuals or via tickets.</li><li>Conflicting requests from different stakeholder groups flooded into the same channel, with little awareness or acceptance of prioritisation.</li></ul><p>It was the least productive environment for my team, or any team, to thrive. Being asked for timelines they couldn’t give or explain issues they did not cause, while working at full speed to develop the product, they were in no position to turn the situation around. It’s my job to get them out.</p><p>There was everything to be done, but the starting point was always clear: they never skipped any 1:1 with me. Anger, disappointment, sadness were able to came out during our 1:1s. I appreciate those emotions as it means there’s still energy and drive in them. What I also must admit is that there is a strong sense of fear in the system. With their trust, I was able to re-build their security, confience, and courage, week in, week out, with each individual, over each challenge. Eventually, I was able to see again their ability to shoulder the pressure coming from a group of people, resit the urge to commit when being put on the spot, seek clarity of the customer’s needs, and lead discovery conversations instead of being pushed about. This itself is a chapter worths discussing about, which I’ve planned to speak about in an upcoming talk (stay tuned).</p><p>Throughout this process, the team also taught me significant lessons: humility, facing stress, vulnerability, and openness. While this was a growth opportunity for the team, and myself. There are non-negotiable terms of how teams collaborate that I must implement.</p><p>At university, there were many strategy courses that often felt dry. But one case stayed with me. We were shown a video of how the senior management of VW (Volkswagen) interact with each other, analysed how truth and standard were not possible in that organisation, and how it eventually led to the “Diesel Gate” that ended VW’s prestige. If it can happen to VW, it can happen to any company. It’s called “<strong>culture will eat your strategy for breakfast</strong>.”</p><p>I believe that leaders don’t just draft strategies or plans. They build cultures too. I wasn’t going to let the unproductive culture eat us up, so while implementing processes and changes in my team, I took it as my job to bring the cultural concern to the highest level of this company. While there’s a million reasons why I shouldn’t have done so, I think about this — when you reflect on your career, the toughest years that were so painful they changed you, would you rather try to let some good come out of it, or become a changed person who can’t even regret not saying the words while you should have?</p><p>2025 wasn’t easy. But evolution brings unstable situations all through history. If the situation is chaotic, it means it’s full of unformed possibilities too.</p><p>The question is not really “where are we going?” but “what are we becoming?” I hope that some of the things in this article resonates with you, reassure you, triggers you, and I welcome your comments.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cecbb0103511" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Lessons from implementing Agentic: It’s not just about AI]]></title>
            <link>https://starmancallin.medium.com/lessons-from-implementing-agentic-its-not-just-about-ai-d048cb1346e8?source=rss-1f217f73e23f------2</link>
            <guid isPermaLink="false">https://medium.com/p/d048cb1346e8</guid>
            <category><![CDATA[agentic-ai-case]]></category>
            <category><![CDATA[product-management]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <dc:creator><![CDATA[B2B Product Management Journal]]></dc:creator>
            <pubDate>Sat, 06 Sep 2025 09:14:30 GMT</pubDate>
            <atom:updated>2025-09-06T09:21:18.631Z</atom:updated>
            <content:encoded><![CDATA[<h3>Problem</h3><p>As a sustainability software, we deal with a lot of data. Since the foundation of our software is emission calculation of daily activities, such as electricity, water, and fuel consumptions, our customers must give us data points of their business facilities. Among many options, they could enter those data into our system by hand. Once they do, it’s obviously not the most interesting job. But now with Agentic AI, we can extract those data directly from their utility bills without human effort.</p><p>Designing this feature has not been the most straightforward, but not really due to navigating technology, but the common product management challenges.</p><h3>AI</h3><p>Technically, extracting numbers from an image or PDF has always been possible through OCR (Optical Character Recognition). That’s what iPhone uses to identify the texts in our photos for example. However, it’s also widely known that OCR is not 100% accurate. So what’s better than having an AI to make sense of the OCR-extracted data, and validate the quality?</p><p>That’s where we pulled in Agentic AI. Once the bills land in our system, the AI detects file formats, tell if any format conversion task is needed, passes on to the OCR to consume, produces unstructured data, for LLMs to put into structured data, evaluate the quality, and turn into final output. This means no more data entry by hands.</p><p>However, putting an LLM infront of the data output doesn’t guarantee accuracy. Not when LLM is still experiencing cross contamination and hallucination challenges. Letting tech people do their work is the product motto that also applies in AI development. I did however revisit and insisted on this piece, that is the AI should learn over time to improve accuracy. Cuz without learning, it will take no time for our customers to find out that the AI isn’t improving. They will unconsciously account the human correction effort into their ROI. It’s detrimental for the business in the long-run. And since AI doesn’t just learn overtime if you don’t actively design that into the solution, we’ve extended the scope.</p><h3>User</h3><p>In the AI architecture, we didn’t involve more than one Agents as we want human to be in the loop and have the final say. That’s when non-AI common product challenges started to kick in.</p><p>For companies to responsibly disclose their emissions, professional audit is required. What auditors want to see is the evidence of data quality, especially how data has been validated by the company. Our customers are also generally willing to perform the validation themselves for a peace of mind.</p><p>I knew I must chuck in a validation screen in this Agentic user journey. However, no matter what step or component I place and replace, I ended up needing major front-end development. From the get-go, my intention has been very clear — with already the upfront tech investment, complete this feature with minimum front-end effort, before it becomes too humongous to see the light of the day.</p><p>At this point, there was not much thinking of LLM, RAG, MCP, rather re-evaluating the user journey. Eventually, for all the moments in the journey that can lead to significant UI or system logic changes, I replaced or eliminated them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ar7vULnDS08pH8B8lBz9yg.png" /><figcaption>Red cards are scoped out. Rectangular workflow is replaced.</figcaption></figure><p>Replacing the workflow gave me the most uncertainty on user experience. So I threw the requirements into MS Copilot to generate a series of mock-ups. After seeing it in action, it looks to be worth a go.</p><h3>Risk</h3><p>Eliminating every risky assumption on the business case and customer value applies to AI development too. AI is essentially a software solution that people pay for. It wouldn’t have been safe for me to swap out any step in the user journey without evidence. And those evidences don’t live in Agentic architecture, but in prior analysis, user research, and eliminating risky factors before anything had been put to investment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iyEXJX2Hb3nsNehdbyIgUA.png" /><figcaption>Risk hotspots in opportunity solution tree.</figcaption></figure><h3>Emission</h3><p>Lastly, because sustainability is our business, I asked the Copilot how much electricity did I consume by generating those mock-ups. It amounted to 1.5 kWh, that is 0.45 kg in Co2 equivalent.</p><p>Use AI, but be clear of your goal before drafting the prompt and be skilled in prompt engineering.</p><p><strong>In summary, AI is the feature, not the product. Fitting AI solution in the product still requires traditional product skills — beware of scope creep, validate values first.</strong></p><p>How’s your AI implementation journey? I’d like to hear from you.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d048cb1346e8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How not to use AI to solve a problem]]></title>
            <link>https://starmancallin.medium.com/how-not-to-use-ai-to-solve-a-problem-0bf0aabbb1fe?source=rss-1f217f73e23f------2</link>
            <guid isPermaLink="false">https://medium.com/p/0bf0aabbb1fe</guid>
            <category><![CDATA[ai-product-management]]></category>
            <category><![CDATA[ai-product-design]]></category>
            <category><![CDATA[lovable]]></category>
            <category><![CDATA[writing-prompts]]></category>
            <dc:creator><![CDATA[B2B Product Management Journal]]></dc:creator>
            <pubDate>Thu, 14 Aug 2025 18:01:18 GMT</pubDate>
            <atom:updated>2025-08-14T18:04:11.158Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Tools used: ChatGPT, Lovable, MS Copilot Studio, MS Forms</strong></p><p>Like anyone whose responsible for a software product, I face the challenge of managing customer escalations that arise unexpectedly, with little time to prepare, plentiful of expectations. The implementation teams can’t address these escalations alone, as customers typically demand direct conversations with someone reponsible for the product. However, it’s not common that I receive a helpful summary of the project without excessive emailing, messaging or meetings. As a result, preparing for customer de-escalation has been stressful and inefficient.</p><p><strong>This has got to be a problem that AI can solve I thought.</strong></p><p>After consulting with ChatGPT, I set off to develop an application that can help me gather essential information about customers and projects. (The prompt that I used is at the end of this article for feedback and your human ideas.)</p><p>Added ChatGPT’s suggestion to my original prompt, I gave it to MS Copilot Studio. The Copilot Studio successfully failed the job by delivering a lazy chatbot that doesn’t want to do the job.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F0M3Lxm3S8adMyBksh1ItA.png" /></figure><p>I went to Lovable. Lovable expectedly delivered an app in lighting speed . But to my great disappointment, it’s practically a survey.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-2y-3gyxXKSWqyknuksoXQ.png" /></figure><p>This is someting that MS Forms can build too. Since MS infrastructure is hosted by my company, I don’t even need to sort out data and security myself. So eventually I did it in MS Forms.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/987/1*HwIq2TlpA0p3U9o8UwoRLw.png" /></figure><p>In the end, I overcomplicated the problem. My disappointment came from the false expectation that everything should be solved by AI.</p><p>Lovable and Copilot Studios are incredible tools. But they weren’t suitable to solve my simple problem.</p><p>Finally, here’s the prompt and a reminder for the product folks.</p><blockquote>As the Head of Product for an ESG software product, the software implementation projects are usually in an unknown state to me but till the point of customer escalation, where the customer demands resolution within a defined short timeframe. With little time to prepare and customer pressure, the information given to me are often not effective to action on. As a result, preparing for customer de-escalation has been stressful and inefficient. For the implementation teams, they cannot respond to the escalation alone, as the customer usually demands meeting someone responsible for the software. They also do not have the capacity or guidance to give a helpful holistic view of the implementation project at this point. They need the Head of Product to take their pains away.</blockquote><blockquote>I want to build an agent to help me prepare for the de-escalation by gathering information that I need about the customer. The manner this agent gathers information should be professional, solution-driven, and helpful. After gathered all the information, I need the agent to summarise them and feedback to me.</blockquote><blockquote>The specific information that the agent should gather: implementation project status, planned or actual go-live date, contract package that the customer bought, escalation details, project risks, exact software issues that customer raised (provide link to tickets), information on past escalation if any, who else is involved in interfacing this customer, and exactly what do they need from me and when.</blockquote><p>This prompt came after working through the identification and analysis of my challenge, by firstly putting together the mindmap and Opportunity Soution Tree. Especaily at the age of AI, don’t take shortcuts in the problem space.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*L9NX9qCsGQnqvX2YbwzhPQ.png" /></figure><p>Any thoughts? I’d like to hear your views on the approach, the prompt, and your human ideas.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0bf0aabbb1fe" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Book Reviews: The SaaS Playbook, No Bullsh*t Strategy, Hacking SaaS]]></title>
            <link>https://starmancallin.medium.com/book-reviews-the-saas-playbook-no-bullsh-t-strategy-hacking-saas-70131285f355?source=rss-1f217f73e23f------2</link>
            <guid isPermaLink="false">https://medium.com/p/70131285f355</guid>
            <category><![CDATA[enterprise-saas]]></category>
            <category><![CDATA[saas-metrics]]></category>
            <category><![CDATA[b2b-saas-business-model]]></category>
            <category><![CDATA[saas-product-development]]></category>
            <category><![CDATA[enterprise-software]]></category>
            <dc:creator><![CDATA[B2B Product Management Journal]]></dc:creator>
            <pubDate>Mon, 21 Apr 2025 11:40:36 GMT</pubDate>
            <atom:updated>2025-04-21T11:40:36.935Z</atom:updated>
            <content:encoded><![CDATA[<p>Working in SaaS business day in day out, I’m becoming concerningly comfortable of what I know. Thus since the beginning of 2025, I’ve started reading books that discuss SaaS products and businesses, hoping to learn something new or identify knowledge gaps. Here are 3 books that I’ve read and how I would recommend them.</p><h3>The SaaS Playbook: Build a Multimillion-Dollar Startup Without Venture Capital by Rob Walling</h3><p>This is a quick and easy read of everything foundationally SaaS.</p><p><em>The SaaS Playbook </em>gives all the fundamental knowledge that a founder would need in order to start a SaaS business from scratch. Whatever your background is — a software developer or a marketing specialist, this book will give you enough knowledge to start building a SaaS business.</p><p>This book is written from a VC’s perspective of how you wouldn’t need VC to kick off and grow a SaaS business, which makes it quite special. Although it’s written for entrepreneurs, I can see how this book becomes educational for anyone wishing to break into SaaS that are not founders. If you’re looking for one book that introduces all the foundations of Saas, this is the one.</p><h4>Who is this book not for:</h4><p>If you’re already working in a SaaS business, looking to dig deep into specific areas, this book wouldn’t be helpful.</p><h3>No Bullsh*t Strategy: A Founder’s Guide to Gaining Competitive Advantage with a Strategy That Actually Works by Alex M H Smith</h3><p>As promised in the book title, no BS.</p><p>It’s a solid 152-pager of well told strategy. A classic example of big ideas conveyed in simply stories, practical and straight-to-the-point. Considering the length of the book, you’d be surprised of how many examples and principles are introduced and analysed.</p><p>The side benefit of reading this book is learning the simplicity and clarity of language and expressions. If you struggle to shorten your email or simplify your communication, give this book a read. But in general, it’s a book for everyone.</p><h3>Hacking SaaS: An Insider&#39;s Guide to Managing Software Business Success by Eric Mersch ★ ★ ★ ★ ★</h3><p>This is the best book about SaaS that I’ve ever read.</p><p><em>Hacking SaaS</em> is written from a CFO’s perspective, analysing product and sales strategy from metrics level. Perhaps that’s why this book stands out so differently than others. The metrics it introduces go well beyond ARR (Annual Reoccurring Revenue) and Booking, but an extensive list of sales and financial metrics, from Dollar-based Retention Rate (DBRN), Expansion Rate, to the Contribution Margin.</p><p>What also makes is special is that this book discusses success factors of a SaaS business that are so essential that it seems rather surprising that other books don’t mention: Professional Service, integration, new account setup, technical pivoting, etc. For the fellow SaaS folks who’s curious about entrenched big SaaS businesses, or in a more challenging position — working in an entrenched SaaS business, this is the book for us.</p><p><strong>It’s really a book of economics of SaaS business</strong> — sales, marketing, product, engineering, professional service, and everything else. You can expect plenty of examples of other B2B SaaS companies that you normally don’t see in B2C focused books.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/523/1*JvK93zQaPTQXkxF58NXDRQ.png" /></figure><h4><strong>Who is this book not for:</strong></h4><p>Beginner SaaS people should best read books such as <em>The SaaS Playbook</em> first, as you won’t find enough educational material for the fundamental concepts of SaaS.</p><p>People who find financial metrics boring will not enjoy this book. I’ve studied finance and accounting in school, and even I found a couple of pages incredibly dry.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=70131285f355" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why and how do you carry out a customer field study]]></title>
            <link>https://starmancallin.medium.com/why-and-how-do-you-carry-out-a-customer-field-study-f45ddc4dc360?source=rss-1f217f73e23f------2</link>
            <guid isPermaLink="false">https://medium.com/p/f45ddc4dc360</guid>
            <category><![CDATA[b2b-product-management]]></category>
            <category><![CDATA[product-management]]></category>
            <category><![CDATA[field-study]]></category>
            <category><![CDATA[customer-research]]></category>
            <category><![CDATA[user-research]]></category>
            <dc:creator><![CDATA[B2B Product Management Journal]]></dc:creator>
            <pubDate>Sun, 28 Apr 2024 20:07:49 GMT</pubDate>
            <atom:updated>2024-04-28T20:07:49.041Z</atom:updated>
            <content:encoded><![CDATA[<p>If you’ve explored all methods to gain a solid understanding of your customers, i.e. 1:1 interviews, group interviews, and workshops, but still fell short, perhaps it’s time to try field study.</p><p>In this context, field study means visiting the customer’s proprietary business area for a few days and observing their activities in their authentic environment. Without disturbing any work, have conversations with every type of persona who uses your product.</p><p>This article covers the motivation, research preparation and process. Feel free to scroll to the parts that interest you the most.</p><h3>Why are usual research methods not enough</h3><h4>Comprehension of the entire customer journey</h4><p>It’s common for B2B SaaS software to solve complex issues and run massive processes, which often involve different personas collaborating on the same task. For Product Managers who must think through the end-to-end user journey in order to design suitable product logic,<strong> it’s paramount to have a solid understanding of the entire journey and the people involve</strong>d.</p><p>What we usually do is interview users, exploring their perspectives, pains, and needs. <strong>However, some users are easier to get in touch with, and some nearly impossible. </strong>So we end up knowing one or two, but now all sides of the story.</p><p>For example, incident management software such as ours offers ultimate visibility into what’s unsafe in the production area, so that the safety managers can make data-informed decisions to prevent incidents from happening. (This is only one aspect of it, but it’s enough for the context of this article.) Unsurprisingly, safety managers don’t operate machinery or sit in the production area, but rather in an office or the head office of a big corporation. Thus they need the software to be their eyes and window. But safety data don’t just pop into the KPI dashboard by itself. It takes every end user – worker, to record any concern they see and any injury that’s happened. And that’s just the beginning. Afterwards, there’s a chain of people responsible for the follow-ups— ensuring data quality, assigning tasks, implementing protective measures, etc<strong>. So every dot and line on the KPI chart is a result of the effort of many people. That’s why we want to learn everyone’s individual needs, to ensure the software is convenient for everyone to do their parts</strong>.</p><p>And that had been impossible. Interviewing the safety managers is straightforward— they are typically our points of contact; Interviewing the next persona down the journey is challenging, but not impossible. About 10% of our interviews involve the safety representatives who sit close to production areas and workers. But neither of them is the main contributor to data collection. <strong>Those who are, workers and line managers, would never hop on a call with us. We must shift ourselves to their territory</strong>.</p><h4>Break away from theoretical understanding</h4><p>There was another motivation that drove us to carry out a field study. The industries that we solve problems for are not the apparent ones you get to experience in daily life, e.g. food delivery, eCommerce, or social media marketing. They are rather manufacturing, construction, or utilities, which you don’t get to experience just by clicking away on the internet or carrying on with your usual days. Despite the vivid descriptions during customer interviews, without actually seeing the frontline in action, our understandings were inevitably on a rather theoretical level.</p><h4>A wonderful chance to work with the customer</h4><p>Behind every enterprise B2B sale, there is a buyer who not only committed to our software but also promised their internal organisation a reliable product and a smooth transition. That’s an enormous job for them to take on. We can ease their burden by ensuring their organisation opposes the change as little as possible. And that means improving user experience for all types of users.</p><h3>Field study planning</h3><p>Planning the field study primarily means working out a plan together with the customer. But prior to that, determining which customers are most suitable to visit is vital.</p><h4>Which customers are most suitable</h4><p>The ideal customers to visit can vary per product. But firstly, I’d strongly recommend that you <strong>only visit those customers that you already know</strong>, meaning you’ve already had an interview or workshop with. Because an understanding of their processes and main challenges is the foundation for effective field study. Frontline is overwhelming. There are a lot of things and people to observe, and a lot of conversations to be had. You won’t be able to make the best out of it by going in blindly and asking generic questions.</p><p><strong>Secondly, they should fall into your ideal customer profile (ICP)</strong>. We know that it’s not 100% the case that our product is sold to an ideal customer. So take the step to qualify the customer pool, making sure the study focuses on the people and jobs that our product is designed for.</p><p>Lastly, the customer is in a relaxed state with your business. In B2B, account relationships are long-term, meaning they go up and down over time. <strong>Work with your Account Management Team to make sure that the current relationship is gree-lighted</strong> and your visit does not add to any unnecessary complication.</p><h4>How do you ask the customers about field study</h4><p>It’s extremely unlikely that you’re their first visitor, so simply ask. They will give the go-ahead or no-sorry very quickly, given that you have communicated all the background.</p><p>Prior to asking a customer, prepare a one-pager that states the full purpose and goal of the visit, so that you’re always ready to convey the messages, and for the customer to pass on internally when needed. Especially for those customers who have multiple locations or subsidiaries, they need an idea of your research purpose to tell which location is the most suitable for you to visit.</p><p><em>On the same note, it’s absolutely fine to be uncertain of all the activities and scenarios you’d like to observe upfront. After all, you’ve never seen the area, and cannot plan what you’ll be encountering. In my case, I had written in the one pager that I’d like to observe how people report safety concerns on their mobile App, and how they conduct safety training and inspections, etc. But in reality, these events cannot be planned, unless you’d like the customer to stage it, which will lose the meaning of observing the authentic environment.</em></p><h4>How do you plan it with the customer</h4><p>Your visit will be official, obviously not sneaking in and causing suspicion, meaning you’re registered as a visitor in their system, and you’ll be wearing a proper visitor badge or name tag throughout the day. And all the people whose activities you’ll be observing will already have been briefed.</p><p>So when a customer agrees to have you, it’s an enormous time commitment that they make. Appreciate that they’d have to plan a schedule that works for all the key personas, brief the entire frontline area that there will be some “software people” visiting and observing their work, tell them not to worry and that we won’t be grading or judging their performance, even encourage them to talk to you freely, etc. <strong>Try to make it as easy for them as possible</strong> by communicating all information clearly and consistently from the first meeting. When they come back with a date, adhere to their proposal and follow their schedule, e.g. some plants start working and close very early.</p><p>Once again, it’s unrealistic to plan everything perfectly before actually seeing the place. So keep the goal in mind and fill in the details as you go. That’s also partially the reason the field study should last a few days. Another major reason is that it takes a long time to find plenty of opportunities to talk to all the personas.</p><p>This is the first field study which we did purely from a customer research perspective. All I can say is that the efforts were fully paid off. It’s not only all the user knowledge that we gained, but also the role of our software — looking at everyone’s daily business, all the software and hardware they operate, where and when our software comes into play. When speaking of user experience, we rarely consider the world outside our software bubble, while in reality, the users may have completed various other activities until making a stop in our system, before moving on to the next one. It was a fantastic experience to witness the first-hand perspective and begin to consider improving the user experience from before they landed in our system.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*IsbIQ9dbtT5qnst1a7ygYg.jpeg" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f45ddc4dc360" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Balancing Discovery and Rapid Iteration: A Real-World Example]]></title>
            <link>https://starmancallin.medium.com/balancing-discovery-and-rapid-iteration-a-real-world-example-748c2a037edd?source=rss-1f217f73e23f------2</link>
            <guid isPermaLink="false">https://medium.com/p/748c2a037edd</guid>
            <category><![CDATA[iteration]]></category>
            <category><![CDATA[product-management]]></category>
            <category><![CDATA[lessons-learned]]></category>
            <category><![CDATA[qa-testing]]></category>
            <category><![CDATA[product-discovery]]></category>
            <dc:creator><![CDATA[B2B Product Management Journal]]></dc:creator>
            <pubDate>Tue, 23 Apr 2024 07:00:50 GMT</pubDate>
            <atom:updated>2024-04-23T07:01:11.990Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This article is originally written for </em><a href="https://airfocus.com/blog/balancing-discovery-iteration/"><em>airfocus blog</em></a><em>.</em></p><p>Who hasn’t made mistakes as a product manager? It’s not just that time an “easy fix” exploded into a multi-sprints implementation; or when you had bet on one pain point over the others and didn’t gain enough traction after <a href="https://airfocus.com/glossary/what-is-release-roadmap/">feature release</a>; or the over-developed feature that should’ve been released months earlier… It’s in many things we do. This story is about mistakes and learning.</p><h3>Hardcore discovery vs. ship it early</h3><p>As a rule of thumb, we should mitigate possible risks when defining <a href="https://airfocus.com/glossary/what-are-features/">product features</a>.</p><ul><li>How many customers will pay for it?</li><li>Does it work for our overall business strategy?</li><li>Can our teams build it without stacking nasty <a href="https://starmancallin.medium.com/why-is-it-difficult-to-reflect-tech-debt-on-the-balance-sheet-7bc4d8af5fbe">technical debt</a>?</li><li>Will end users find it useful? etc.</li></ul><p>This can easily lead to analysing forever to find that absolute certainty and perfection, but not developing anything, the so-called analysis paralysis. At the other end of the scale is <a href="https://airfocus.com/glossary/what-is-an-iteration/">iteration</a> — you’ll never have all the facts, especially not before a feature sits vividly in front of you and the customers, so ship early and find that balance as you go. Which one is better? I would have said “The truth lies somewhere in between” if not for this failure.</p><p>We recently shipped a new feature aimed at protecting the identity of vulnerable users. What it does is it anonymises the involved person’s full name on a frequently visited page, where comprehensive and sensitive information about an occupational incident is listed.</p><p>With the help of the new functionality, the identities of those involved in incidents don’t have to be compromised when people inform themselves about the incidents. This is not only the desire of our customers, but also seen as a strong reason to start properly using the software without data privacy concerns. We’ve had an internal release, where I explained and demo-ed in detail. The first round of feedback was positive. Until I was contacted by the Data Security Manager at our company.</p><p>It had turned out, <em>ironically</em>, that this data protection feature could pose a potential data privacy issue.</p><p>What could happen is, some users could attach files to the page where we anonymised the full name. And in those files, the full name can be compromised. This was an inconvenient fact. No time to dwell. It needed to be fixed. The immediate response was to define a solution, talk to the team, disturb their lunch break, and implement a hotfix, which was released the next day. Since it was an internal release, no customer data was at risk. But while things were in motion, this guilty <a href="https://airfocus.com/resources/ebooks/product-management/">product manager</a> had only one thing in mind — I had thought of the attachment during the ideation phase, and <strong>consciously eliminated it from the first iteration</strong>.</p><p>My consideration was simple — very few people use the attachments, and there’s no reason for the attached files to be passed on, especially to any external party. So the plan was to focus on the main functionalities, release, and review when real-world feedback come in. Common product management. Apart from the fact that <strong>it’s not a good idea to iterate data privacy, which is either watertight or nothing</strong>. A valuable lesson that I have learnt.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/967/1*oR4qMHJLtzi_XP_OGoWaIg.png" /></figure><p>So should you only develop with perfect certainty, or do something and make it better? I would still say, <strong>don’t analyse forever and chase the <em>Holy Grail</em>, but be conscious of the product’s nature. If it’s as serious as privacy, health, finance, etc, rather be a bit conservative.</strong></p><h3>Iteration for a mature product</h3><p>There’s one more catch in the story. During the QA phase, we had actually captured the issue with the attachments. Strictly speaking, one of the many ways of attaching files. We fixed exactly that one, and that was it. In hindsight, I should’ve slammed the break and analysed the other ways of attaching files.</p><p>This product is a complex one, you may have noticed. It is. The labyrinth in logic makes it incredibly difficult to maintain, let alone innovate. It’s often the case that we have to scope out certain cases to be able to move forward.</p><p>Back in when I communicated the concept of this feature internally, I received several 😮and ❤️ emoji reactions. No one had hoped or thought that this concept could be done. In all honesty, perhaps myself too. When I first presented the tickets to the <a href="https://starmancallin.medium.com/pm-devs-and-the-strategic-bits-and-pieces-e35ca020562a">Dev team</a>, I had the plan of pulling them back any second if the team posed doubt on feasibility. But the truth that mattered is, that the main functionality had already taken up too much mind space, and required a whirlwind of brain activities, since I hadn’t specified the behaviours of every attachment in the <a href="https://airfocus.com/glossary/what-are-acceptance-criteria/">acceptance criteria</a>, it was difficult for the team to be the expected gatekeeper or the last line of defence.</p><p>There won’t be a big talk on not to overcomplicate your product. I believe it can even be an inevitable step for a mature business.</p><p><strong>But a reminder to all the Product Managers who manage a complex product, think carefully about what agility means to you, especially considering the level of complexity and interdependency in the product. Moreover, whether shipping the feature is the only way to get user feedback.</strong></p><h3>Consider tools to aggregate documentation</h3><p>In the then upcoming <a href="https://airfocus.com/glossary/what-is-a-sprint/">sprint review</a>, I shared my personal reflections on this event and had a follow-up chat with our Data Privacy Manager. We immediately came to the same conclusion that we shall identify privacy-related features and enable him to assist early. But only to realise how scattered our documentation is. To follow the discovery and ideation of any feature, he would have to go through four different tools, i.e. Miro, Figma, Confluence, and Jira. To an extent, it is incredibly difficult for a supporting role to stay on top of things.</p><p>So once again, it comes down to the product manager to spot early signs and proactively involve others when needed. <strong>For anyone who doesn’t have too much time and attention to spare, it’s safe to assume tools that are tailored for PMs (</strong><a href="https://airfocus.com/"><strong>like airfocus</strong></a><strong>) make the product manager’s life easier and make these processes less prone to mistakes.</strong></p><h3>Conclusion</h3><p>Mistake happens. When it does, be solution-driven and get it out of the door fast. Afterwards, make time to reflect and share your learnings. Don’t forget to <a href="https://airfocus.com/blog/smart-goals-stress-management/">shield the team from the stress</a>, they only need to focus on implementing the solution. Finally, never underestimate the importance of data privacy.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=748c2a037edd" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>