<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[The TestFairy Blog: Mobile Development, Testing, Continuous Integration &amp; deployment. - Medium]]></title>
        <description><![CDATA[TestFairy&#39;s Blog - Medium]]></description>
        <link>https://medium.com/the-testfairy-blog?source=rss----576ce349b86a---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>The TestFairy Blog: Mobile Development, Testing, Continuous Integration &amp;amp; deployment. - Medium</title>
            <link>https://medium.com/the-testfairy-blog?source=rss----576ce349b86a---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 20:05:06 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/the-testfairy-blog" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How to manage last minute product prioritization and not go crazy]]></title>
            <link>https://medium.com/the-testfairy-blog/how-not-to-go-crazy-with-last-minute-prioritization-a23cd176de66?source=rss----576ce349b86a---4</link>
            <guid isPermaLink="false">https://medium.com/p/a23cd176de66</guid>
            <category><![CDATA[prioritization]]></category>
            <category><![CDATA[ios-app-development]]></category>
            <category><![CDATA[android-app-development]]></category>
            <category><![CDATA[product-management]]></category>
            <category><![CDATA[mobile-app-development]]></category>
            <dc:creator><![CDATA[yuvalgo]]></dc:creator>
            <pubDate>Tue, 17 Oct 2017 07:24:30 GMT</pubDate>
            <atom:updated>2017-10-25T09:54:30.117Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FDh3uwkf0EgTXhVBN3ejkQ.png" /></figure><p>Hey mobile PMs !<br>I’d like to share a few thoughts about last-minute prioritization with you.<br>Last minute, you know... The features are already baked and tested. You ran a usability test with a few potential users and also asked your peers at work to review it (<a href="https://medium.com/the-testfairy-blog/quality-is-a-struggle-heres-one-small-battle-mobile-product-managers-can-win-today-46d0764c8e8a">see previous post about that</a>). All gears are set to release the new version, but, there are still a few major problems…</p><p><strong>At this point, you’ll have a list with these types of feedback<br></strong>(we’ll go over each type and discuss, in a moment…)</p><ol><li>New feature/improvement suggestions.</li><li>Performance issues.</li><li>First time use issues.</li><li>Hardcore functional bugs / crashes.</li><li>I love / hate the new design kind of comments</li></ol><blockquote>Now what do you do with the precious feedback you received? how do you decide what to fix now and what’s not important?</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*lhI6IRG1WPe1ScAb1Ir8pA.png" /><figcaption>Bugs, Bugs, Bugs, Bugs!</figcaption></figure><p><strong>Before we prioritize, let’s make sure what we identify real issues.</strong></p><p>Some of the feedback you may receive, especially from people who are not professional testers, will be hard to understand. So, when sending your “New version is out, please help us test it” message, make sure to ask for the following information in the feedback:</p><ol><li>Add a screenshot, so we can understand exactly what were you doing and which button you were tapping.</li><li>Make sure to include your email, so we can get back to you if we can’t understand something you wrote.</li><li>Indicate the device and iOS or Android version you are using.</li></ol><p>If you are running a usability session, make sure this information is collected and saved.</p><p>In order to draw action items you’ll need to verify that each experience described in a feedback indeed reproduces, and understand the exact conditions under which it reproduces (device, user status, etc). I usually ask the awesome people in the QA team to help me out with that.</p><p><strong>Last minute prioritization buckets</strong></p><p>After verifying that each problem is real, we’ll need to sort them all into these buckets:</p><ol><li>Hold the release, we’re fixing this one.</li><li>This goes into the next version feature backlog.</li><li>Not enough information: to be tested during staged rollout.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/736/1*zNxOMtDyx__PBeMxpjOrrQ.png" /><figcaption>You’ll need a bucket and a monkey for this part.</figcaption></figure><p>So, let’s talk about each type of feedback and how to sort out it’s priority bucket.</p><blockquote><strong>New feature/improvement suggestions</strong></blockquote><p>These ones are probably the easiest. This is not the time to add a new feature, but you’ll add it to your next version feature backlog and prioritize it later on with the content of the next major release. Be polite and explain the situation, then move along.</p><blockquote><strong>Performance issues</strong></blockquote><p>You may receive feedback such as: “It‘s loading forever’”, or “I’m seeing this message about the app not responding, suggesting me to close it”<br>Performance issues are always tricky.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/187/1*Pyos0sVH7oLVY7tu50gqAQ.png" /><figcaption>Give it a moment, it’s going to space!</figcaption></figure><blockquote>Often (and rightfully so) developers will tell you they don’t have a clue about the source of a performance issue, and that they just don’t know how much time will it take to understand and fix the issue.</blockquote><p><strong>So… should we take the time to get to the bottom of the issue?</strong></p><p>Consider the following aspects:</p><p><strong><em>Severity<br></em></strong>The longer it takes to complete an action, the bigger the problem is. If something takes 3 seconds or more, you’re risking users abandoning your app.</p><p>In some situations (when your app is busy for more than 5 seconds), the Android operating system will show the user a dialog saying “your app doesn’t respond” and suggest to close it. I strongly advise you to fix this kind of problem, since users will not be very forgiving towards such an experience.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/178/1*N83b76o3-IvORZngbfYfXQ.png" /><figcaption>App Not Responding, yikes!</figcaption></figure><p><strong><em>Frequency</em></strong><br>When the slow action in question affects the user several times during one app use, consider fixing it right now.</p><p><strong><em>Location in user journey/funnel<br></em></strong>Users may be more tolerant to performance issues further down the funnel. It’s probably not a disaster if the purchase completion takes 3 seconds, but it’s going to be a problem if an app takes 4 seconds to load.</p><p>There’s a caveat though: checkout/payment/share to friends screens should load fast to perform well.</p><p><strong><em>Affected user segment</em></strong><br>If a problem affects a small number of users for an old Android version, you may decide not to fix the issue right away, but if it affects 5% or more of your users, you may have to delay the release for this fix (depending on severity, frequency etc).</p><p><strong><em>Cost of fix</em></strong><br> Some issues will take 2 days to fix, and some will require help from two other teams and take two weeks to complete. You’ll have to take cost into account to understand the ROI of the fix: The value it provides to your users under the specific time constraints you have.</p><blockquote>Let’s stop for a moment and talk about staged rollouts</blockquote><p>If you have strong doubts about the the user segment that will be affected by an issue, you can stage your roleout to the store and allow a small number of users to download the app first. In the <a href="https://support.google.com/googleplay/android-developer/answer/6346149?hl=en">Android</a> Play Store, you’ll be able to set specifically who can get the app, and target specific device or OS version holders. <a href="https://itunespartner.apple.com/en/apps/faq/Managing%20Your%20Apps_Submission%20Process">iOS</a> apps will have less options, though.</p><blockquote>Regardless of the problems you anticipate, it’s an excellent idea to use staged rollout as you’ll also find problems you were not aware of in your app, before it gets to 100% of your users.</blockquote><p>Once a small portion of users uses your app, you’ll need to understand whether and to what extent are they affected by performance issues. In addition to your usual analytics tool, I’ll list a few popular solutions for gaining insights on your real live app.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hsyF0hhRWnTkFOl9E-R8qw.png" /></figure><blockquote><strong>I love/hate the new design</strong></blockquote><p>These are both real feedbacks I received from two co-workers, on the same new version:</p><ol><li>“I just hate this new purple/pink color”</li><li>“I absolutely love this elegant fuchsia element”</li></ol><p>I won’t deny it, I’m thrilled to hear that people love the new design and feel frustrated when they hate it. How can they not love it?</p><p>But, emotions aside, let’s consider our options and ask the right question:</p><p><em>“Is this an area we just neglected and under-designed, or did we purposely design it this way?”</em></p><p>Discuss the issue with your design/UX partners. If they understand the feedback and have good reason to stick with the current design, then go ahead and release the feature and observe real user reaction.</p><p>If not, you can employ the same decision framework from the performance section and evaluat:</p><ul><li>Is it bad? how much it affects users ability to complete important actions?</li><li>How frequent/critical is the area at questions,?</li><li>Which segment of users will be affected?</li><li>How far is it down the user journey/funnel ?</li><li>How long will it take us to fix this?</li></ul><blockquote><strong>Issues related to First Time Experience</strong></blockquote><p>You’ve added a cool new option in your app to save credit card details for the next purchase, and asked your co-workers to try out the new version. Then, you received a lot of feedback from people who just couldn’t find the new addition. The wrong first time experience can make or break your feature KPIs…</p><p>There are usually three common problems at this stage: confusing instructions / call-to-action text, discoverability, no call-for-action for the “empty state”. So make sure to follow up with the people who provided the feedback and understand this point.</p><p>Feature discovery is very important, especially in apps that provide a lot of options and actions.</p><p>I think the golden rule of feature discovery is to prompt the feature when your user is mostly likely to want it ,and avoid interrupting the user when she is doing something else. So, in our imaginary examples, offering the user to save their credit card details before they completed the purchase may be the wrong time, but prompting them to do so after the purchase may be the right one.</p><p>Another common problem is empty state scenarios with no clear call-to-action button or instructions. It’s essential to have the “Add” button at the bottom menu, but it makes sense to add a clear “Add your first _____” button or call to action at the center of the screen as well.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/185/1*mgWwPJasHkbebufM0Fm_zg.png" /></figure><p>It’s difficult to balance feature discoverability with the UX of other features in your app, but if you’re introducing a key capability that holds real value, you’ll need to find that sweet spot. If you receive feedback regarding feature discovery for the most important feature of this release, hold your release for it. If you’re not sure about it, remember that you can always test user behavior during a staged rollout.</p><blockquote><strong>Hardcore functional bugs / crashes</strong></blockquote><p>In some cases, there are still major bugs at this point in the app’s life. We’ll use the same criteria we’ve mentioned before to evaluate bugs and crashes:</p><ol><li>How bad is it? (how badly does it affect the value a user derives from using this feature, or affect your target KPI)</li></ol><p>2. How frequently/critically will the user experience the bug while using the app?</p><p>3. Which segment of users is affected by this bug?</p><p>4. How far is it down the user journey/funnel?</p><p>5. How long will it take us to fix this?</p><p><strong>Making choices<br></strong>It boils down to fixing the right problems with the little time you have an putting aside the no-important issues.</p><p>When you check each problem against the criteria I listed, you get a clearer picture of what needs fixing right now and what can wait for the upgrade release. You’ll also need to inject a nice amount of common sense to make the right decisions.</p><p>And if you’re not sure, take advantage of staged rollouts and collect the right information about each problem.</p><p><strong>The right tools for the job<br></strong>I’ll mention several popular tools and how each one of them can help you gather feedback &amp; understand problem and their severity through the QA and staged rollouts.</p><p><a href="https://try.crashlytics.com/">Fabric</a> (Crashlytics): Best used for detecting crashes during testing and staged rollouts, also good collecting basic usage stats.</p><p><a href="http://www.testfairy.com/">TestFairy</a>: Helps collecting feedback and shows you videos &amp; and insights about problems during QA, dogfooding and staged rollouts.</p><p><a href="https://www.buddybuild.com/">BuddyBuild</a>: Best used for managing app builds and gathering feedback when the app is being tested.</p><p>That’s all folks. I hope some of my thoughts can help you structure your decision process the next time you face those tough choices. <br>Hit the CLAPS to show your love.</p><p>Yuval.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a23cd176de66" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-testfairy-blog/how-not-to-go-crazy-with-last-minute-prioritization-a23cd176de66">How to manage last minute product prioritization and not go crazy</a> was originally published in <a href="https://medium.com/the-testfairy-blog">The TestFairy Blog: Mobile Development, Testing, Continuous Integration &amp; deployment.</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Quality is a struggle, here’s one small battle mobile product managers can win today]]></title>
            <link>https://medium.com/the-testfairy-blog/quality-is-a-struggle-heres-one-small-battle-mobile-product-managers-can-win-today-46d0764c8e8a?source=rss----576ce349b86a---4</link>
            <guid isPermaLink="false">https://medium.com/p/46d0764c8e8a</guid>
            <category><![CDATA[product-management]]></category>
            <category><![CDATA[android-app-development]]></category>
            <category><![CDATA[testfairy]]></category>
            <category><![CDATA[ios-app-development]]></category>
            <category><![CDATA[mobile-app-development]]></category>
            <dc:creator><![CDATA[yuvalgo]]></dc:creator>
            <pubDate>Wed, 04 Oct 2017 11:32:40 GMT</pubDate>
            <atom:updated>2017-10-04T11:32:39.961Z</atom:updated>
            <content:encoded><![CDATA[<p>Hey mobile product managers, I’d like to ask you one important question:<br><strong>How confident do you feel before *launching* a new app version?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*m09_NhrSZmi5SWE77vN8UQ.png" /></figure><p>I’m Yuval, a product manager from TestFairy. If you are anything like me, you’re probably pretty excited and anxious to see your users respond to new features. You want those AB tests to go extremely well and you… hate surprises (at least those of the bad kind).</p><p>For the past year I’ve been working with some of the greatest mobile product managers and engineering teams in the world, building apps for e-commerce, dating, transportation, insurance, customer service and personal finance. <br>Many of these highly successful teams sleep better at night. They run successful dog food sessions (Dogfooding) that dramatically improve the quality of their app and its user experience. Wait, what?</p><h3><strong>Dogfooding? What’s that?</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/425/1*VPfrbngkatxahMo9etpmaQ.png" /></figure><p>In our world of mobile app development, “eating your own dog food” means sharing the latest app version with the entire company and receiving meaningful feedback, as a last check just before the app is released to the store.</p><p>It usually works like this:</p><ol><li>The new version passes QA, then you send an email or Slack message to <strong>everyone</strong> in your group or company, asking them to try it out and provide feedback if they see problems (they’ll also suggest improvements).</li><li>You collect the feedback, sort it out and prioritize it (coming up — a separate post all about this process).</li><li>You fix the essentials and launch your new app version.</li></ol><h3><strong>Isn’t that testing? the job for the people from QA?</strong></h3><p>Actually, it’s not. Dogfood is a process complementary to running your own methodological QA testing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*B7zYp3zfXSz1UihSCswfEQ.png" /></figure><p>Here are several things you can expect from running an effective dogfood program:</p><p><strong>#1 <em>Design comments:</em></strong><em> </em>I remember one time when we used an icon of a funnel for some, well, data filtering action. Well, I received a lot of comments from people not finding it only to understand that a button with text would be a better choice for that specific screen. You’ll receive a lot of feedback regarding the design and user experience of the product. Some of this feedback will be extremely useful. If people can’t find a critical piece of functionality, the icon is blurry or the micro-copy is confusing, you’ll hear about it. You’ll be able to make those small changes right away before your customers notice it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/529/1*x_Hq6mvdBjfHN4XmpG9z3Q.png" /></figure><p><strong><em>#2 Blind-spots:</em></strong> Super smart people from all around the company will find you some bugs that are really tricky to trace. These bugs are not covered in the test-plans, but the genius from floor 6 will find them. Dogfooding helps you find those blind-spots.</p><p><strong><em>#3 Device coverage:</em></strong> Ron, an Android dev lead that we’re working with recently told me about a critical feedback they received. They found out that several of their call to action buttons just drop below the initial fold on some Android devices. When dogfooding, you get a nice device coverage. Everybody has a different phone, with a different version of iOS or Android. You’ll get feedback about problems that only happen on certain devices.</p><p><strong><em>#4 Second opinion:</em></strong> If you are already aware of certain problems with the new version, dogfooders will help you understand how bad the situation really is. How many of them will complain (and how loudly)?. You’ll get a second chance to evaluate your last minute priorities.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/625/1*6hnOYMZ9HGswvMcyhk23WA.png" /></figure><p><strong><em># Get everyone on the same page:</em></strong> Everybody will get to know the new amazing feature your team created. The people from sales will get the chance to learn about it and the customer support team will experience it before customers start asking questions. It’s a great opportunity for them to understand what is going on and also raise important questions that will help them do their job.</p><blockquote>The bottom line is simple: done right — dogfooding provides useful and varied feedback in addition to your traditional testing effort in a very short time, just before your app hits the store.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/425/1*uAQifoYEBmqiqlaAMkwiKQ.png" /></figure><h3><strong>The Hardest thing about Dogfooding</strong></h3><p>Nobody really likes to learn a new process, so your biggest challenge will be to actually jump start the dogfooding. When you launch a new product user acquisition is the key, and the same applies to instilling new team habits.</p><p>Let’s talk about the first time you’re running dogfood and what you can do to get people to join in on the effort.</p><p>Here are some useful tactics teams are successfully using to get people on board:</p><p><strong><em>#1 Use a funny call to action:</em></strong> When it’s time to tell the team about the new version and ask for their help, create an engaging call to action. Send out an email with a funny or intriguing subject to attract the attention of people in the company. Tell them about the new version and ask them to try it out for themselves. Add photos or videos to spice things up and engage your audience.</p><blockquote>Here a couple of subject lines you can borrow:</blockquote><blockquote>1. “Our new app version makes cocktails, click now to order”.<br>2. “[HELP] us out test the new version? (bunnies will suffer if you don’t)”.<br>3. “Gil from marketing tried the new app, see what happened to him”.<br>4. “Don’t open this email, I dare you.”.</blockquote><p><strong><em>#2 Help the team help you: </em></strong>People receiving your call to participate need to decide to put their tasks aside and follow your request, and that’s hard to do. Send out a calendar invitation so people feel comfortable with having a dedicated 20-minute time slot to try out the new version.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/350/1*IYpLgjfxKyIZPdgRM-jI8g.png" /></figure><p><strong><em>#3 Rewards:</em></strong> Offer a cool reward for whoever contributed the most valuable feedback. A funny gift card, a goofy hat or even a huge bowl of ice-cream. Celebrate the reward with a nice photo or video and share it with everyone.</p><p><strong><em>#4 Get senior management sponsorship: </em></strong>Quality is everyone’s job, but we all need a little nudge. You need to make sure everybody knows that dogfooding is part of their job, and no-one is better for carrying that message than your company’s CEO. Speak with your CEO (or ask your manager to do it) and explain the many benefits of dogfooding. It would make sense to her (or him) to pitch in. Ask the CEO to re-iterate the importance of dogfooding at the next all-hands meetings and send a follow up email to your “New Version is out, please try it out” email.</p><p><strong>#5 Send Reminders:</strong> Reminders work. Really. Ask salespeople (they can send even four or five reminders, but you can settle for one).</p><h3><strong>Shameless promotion</strong></h3><p>There are some great tools out there that can really help you share new app versions, collecting fast feedback and improve your apps user experience and quality.</p><p><a href="http://www.testfairy.com/">TestFairy</a> is the best one, but you can also check out <a href="https://get.fabric.io/">Fabric (Beta)</a> and <a href="https://www.buddybuild.com/">Buddy-Build</a>.</p><h3><strong>So, is that it?</strong></h3><p>Almost. We need to discuss how to effectively collect feedback and prioritize all the problems you’ll discover, which is something I’ll cover next time. In the meantime, you can start winning your quality struggle today and start your own dogfood program.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/540/1*yKTatEvHo8FeEera6uGs0Q.png" /></figure><h3>One small word of advice for the road</h3><p>When I was a younger product manager, I used to get a bit frustrated to discover new bugs towards big releases. “How come we have this bug?!, how come we didn’t find it earlier?” . I got a word of advice from an old and wise product leader that kind off changed my mind set about the issue:</p><blockquote>He said (read in Yoda voice): “Quality is a battle you will have to lead as a PM and it’s a never-ending battle. It is never going to be easy to produce an app or a service with great experience and no bugs. But at the end, the struggle pays off because the little things you do now will amount to a great users experience. Stop complaining and go win.” He ended by smiling his big smile and went off before I could think of an answer.</blockquote><p>Bon Appetit!<br>Yuval.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=46d0764c8e8a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-testfairy-blog/quality-is-a-struggle-heres-one-small-battle-mobile-product-managers-can-win-today-46d0764c8e8a">Quality is a struggle, here’s one small battle mobile product managers can win today</a> was originally published in <a href="https://medium.com/the-testfairy-blog">The TestFairy Blog: Mobile Development, Testing, Continuous Integration &amp; deployment.</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>