<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Guilherme Coelho on Medium]]></title>
        <description><![CDATA[Stories by Guilherme Coelho on Medium]]></description>
        <link>https://medium.com/@iamguico?source=rss-94a75f33915a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*HlS9a1hBxFXYu6-o3pER1w.jpeg</url>
            <title>Stories by Guilherme Coelho on Medium</title>
            <link>https://medium.com/@iamguico?source=rss-94a75f33915a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 21:29:46 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@iamguico/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Letting “data” make decisions is plain lazy]]></title>
            <link>https://medium.com/@iamguico/letting-data-make-decisions-is-plain-lazy-a3489deb605e?source=rss-94a75f33915a------2</link>
            <guid isPermaLink="false">https://medium.com/p/a3489deb605e</guid>
            <category><![CDATA[product-design]]></category>
            <category><![CDATA[product-management]]></category>
            <dc:creator><![CDATA[Guilherme Coelho]]></dc:creator>
            <pubDate>Sun, 28 Apr 2019 18:32:12 GMT</pubDate>
            <atom:updated>2022-06-09T08:52:31.418Z</atom:updated>
            <content:encoded><![CDATA[<p>Creating a product is hard work. It’s the hard work of empathizing with people, of going through the catalog of emotions they feel or could feel. It’s the act of doing all you can to get just a little closer to them, to understand them a bit better (including yes, testing things and looking at data). It’s emotional, psychological and sociological craftsmanship, if you may.</p><p>And then it’s the process of taking all that understanding and create something which wasn’t there, using your own hands and skills perfected over time. It’s refining, improving, throwing out and starting over for as long as necessary to end up with something that’s good, that works, that people like even if they didn’t know they did.</p><p>Let’s stop the “let the data decide” excuse and get to work.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*glt_Fvkx2rXYCMN4W24A_w.jpeg" /><figcaption>Source: Unsplash</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a3489deb605e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[We urgently need more Luxury products.]]></title>
            <link>https://medium.productcoalition.com/we-urgently-need-more-luxury-products-5f29bf3c4c7b?source=rss-94a75f33915a------2</link>
            <guid isPermaLink="false">https://medium.com/p/5f29bf3c4c7b</guid>
            <category><![CDATA[digital-marketing]]></category>
            <category><![CDATA[product-design]]></category>
            <category><![CDATA[product-management]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Guilherme Coelho]]></dc:creator>
            <pubDate>Wed, 29 Nov 2017 14:16:36 GMT</pubDate>
            <atom:updated>2022-01-21T08:35:18.175Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sAnb2fITlIjgr4eFlf5LVA.jpeg" /><figcaption><em>Image taken from </em><a href="https://freestocks.org/"><em>here</em></a><em>.</em></figcaption></figure><p>Of course, I’m not saying we need more overpriced leather bags made at the expense of animal lives. We need absolutely none of those.</p><p>What we need are more companies genuinely invested in making the best product they possibly can. Companies who literally bet their business on the sole belief that, if they focus on making the best product for you, everything else will work out and they’ll be successful.</p><p>Perhaps not surprisingly, for those companies, two things tend to hold true. They do make superior products according to what their customers value, and they’re not the cheapest brand in the marketplace.</p><p>For the privileged minority able to afford them (that’s us), I’d argue that buying a few great products is a much better idea than buying a ton of crappy ones. Not only will you end up with objectively better products, but “better” often also correlates with “as durable as we could make it,” which gives meaning to the saying “I’m too poor to buy cheap things.” And of course, it reduces the amount waste we add to our planet.</p><blockquote>I’m too poor to buy cheap things.</blockquote><p>Nevertheless, we seem uninterested in using good, useful, functional products. And that’s particularly true in the digital realm, which now hosts the latest form of brainless consumerism.</p><p><em>Hey! Thanks for reading this far. My name is Guilherme. I’m a Product Manager by trade, and more recently became co-founder of </em><a href="https://www.betasort.com/"><em>Betasort</em></a><em> , where we’re building an all-in-one toolset for running world-class beta programs. Hope you enjoy the rest of the article!</em></p><h3>We’re addicted to free digital stuff</h3><p>Almost everyone assumes digital products must be free, to the point that we even get a bit mad when a company has the outrageous idea of asking to us to pay for their work.</p><blockquote>Look, the trial already expired. These money-suckers only want my dollars…</blockquote><p>I bet you’ve had some version of this thought cross your mind. I certainly have.</p><p>Free products used to be uncool and useless (who’s using that 64mb USB-stick we all got at an event). And they had an explicit name — Promotional Merchandise — explaining why they existed. Now they’re the only thing we’re willing to use, and we even got used to their lack of quality.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*hW2azFf-TZVmdApl.jpg" /><figcaption><em>Image taken from </em><a href="http://www.business-thankyou-cards.com/"><em>here</em></a><em>.</em></figcaption></figure><p>As a result, our lives are filled with digital products made by companies which aren’t even trying to make a great product for us. If we’re unwilling to pay, we’re not their customer. So they do as they should and focus on their real customers — the advertisers — and on keeping them happy.</p><p>To that goal, the product only has to be good enough to keep us busy while we’re presented with ads. And of course, the longer we remain busy, the longer we’ll see ads, the more money they’ll make, and so on. You see where this is going and is not on our best interest. For them, we’re a commodity.</p><p>This dynamic, in summary, is the reason why we feel technology takes too much of our time. It’s not by chance! Some of the most powerful companies ever are doing all they can to achieve exactly that.</p><h3>Time is the new Luxury</h3><p>There’s no shortage of quotes you can frame and put up on your wall stating how “time is the new gold” or how it is “the only thing you can’t make more of.”</p><blockquote>Time is the only thing you can’t make more of.</blockquote><p>And free digital products are cashing in on this gold. Which wouldn’t be a problem if you had decided to give it to them, but more often than not, you haven’t. Digital advertisers use evil manipulation techniques to force you to give your time away, to stay on their product and to watch their useless content.</p><p>Notable exceptions are, of course, all free products that don’t have any advertising business-model behind, like anything open-source.</p><p>So, to finalize, here’s my strategy to dealing with this issue, in three steps:</p><p><strong>1.</strong> Do an audit of all free products you use and re-consider if they add anything positive to your life.</p><p><strong>2.</strong> If they do, consider finding an equivalent paid product (or simply upgrading) so you can get even more of that good stuff into your days (for me that was Spotify, for example). If they don’t, then close your account, delete the app, do whatever you have to do to remove that product from your life (for me that was Facebook, Instagram, Twitter, 9gag, etc).</p><p><strong>3.</strong> Next time you consider buying a digital product and feel like it’s not worth the money, just remember you’ll pay that value anyway. So your only choices are to pay dollars and get a great product, or pay in time and get a mediocre, manipulative one.</p><p>Thanks for reading.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5f29bf3c4c7b" width="1" height="1" alt=""><hr><p><a href="https://medium.productcoalition.com/we-urgently-need-more-luxury-products-5f29bf3c4c7b">We urgently need more Luxury products.</a> was originally published in <a href="https://medium.productcoalition.com/">Product Coalition</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Non-disposable team]]></title>
            <link>https://medium.productcoalition.com/non-disposable-team-3280c0c5fb82?source=rss-94a75f33915a------2</link>
            <guid isPermaLink="false">https://medium.com/p/3280c0c5fb82</guid>
            <category><![CDATA[entrepreneurship]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[team]]></category>
            <dc:creator><![CDATA[Guilherme Coelho]]></dc:creator>
            <pubDate>Sun, 26 Mar 2017 16:03:39 GMT</pubDate>
            <atom:updated>2022-01-21T08:38:36.863Z</atom:updated>
            <content:encoded><![CDATA[<h4>An alternative to setting your team up for self-destruction.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*K7571sVYrZkjjwAtUHvEnQ.jpeg" /></figure><p>Everyone who’s done it says that hiring a great team is the hardest and most important part of building a company. Find the right people and everything else will follow, they say. Which is why sooner or later it becomes the CEO’s main task and why entire companies are bought only to bulk-hire their employees.</p><p>This emphasis makes total sense. If you’re lucky to have been part of a healthy team, you’ll also know intuitively how its magnifying effect feels like. Everything you create ends up being much better than you could have done alone and the process of getting work done is immensely more enjoyable and fulfilling. Working in a team feels natural and empowering.</p><p>What’s surprising is that while these realizations are more or less universal, a vast majority of companies are still set up as if teams were easy to assemble and not all that valuable. This is particularly true if looking at those companies heavily covered by press and therefore taken as examples.</p><p><em>Hey! Thanks for reading this far. My name is Guilherme. I’m a Product Manager by trade, and more recently became co-founder of </em><a href="https://www.betasort.com/"><em>Betasort</em></a><em> , where we’re building an all-in-one toolset for running world-class beta programs. Hope you enjoy the rest of the article. If you’d like to read more from me, you could </em><a href="https://twitter.com/iamguico"><em>follow me on Twitter</em></a><em>.</em></p><p>If not, let’s look at what could be a typical startup story on your favorite news site. It starts with two founders who’ve built the first version of a product and then manage to hire a great team and scale the company. The ending of the story is still unknown at this point, but one thing is already set: the team will not survive it. If the product fails, the company goes bust and the team is gone. If the company succeeds and gets sold, the team is disfigured forever. If the company succeeds and stays independent, it will try to scale that one idea that worked — in that case the initial team is likely to be diffused by boredom.</p><p>There’s a major paradox in this popular definition of success because companies built this way don’t preserve their most valuable and hardest earned asset — the team. And although there are many examples of companies not built this way, both in tech and in other spaces, they are definitely not celebrated enough.</p><p>What if instead of companies being built around an idea (of which there is no shortage), or around a founder or two (of which there is also no shortage), they were built around the team itself?</p><p>That would mean the company’s main goal is to empower its team to thrive. Not to ensure the success of a particular idea, nor to make the founders rich.</p><p>It would also mean that the subject of the work is secondary and likely to change often. Because what keeps the team together is their ambition to do great work and a belief that sticking together is the best way to do just that.</p><p>Indirectly it also means that although it’s important that everyone work on topics they like, it’s even more important that everyone get to produce quality work and enjoy the process of doing so. Which is the same as to say it’s most important that everyone get to enjoy life, since work occupies such a big slice of it.</p><p>In the end, the team builds products and services but doesn’t cease to exist if one of its creations fails or massively succeeds. They’ll just stick together, keep creating and have a good time.</p><h3>Did you like this post?</h3><p><em>You could check out </em><a href="https://www.betasort.com/"><em>Betasort</em></a><em>, the tool I’m building to help Product Managers run world-class beta programs, or </em><a href="https://twitter.com/iamguico"><em>follow me on Twitter</em></a><em>. See you soon!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3280c0c5fb82" width="1" height="1" alt=""><hr><p><a href="https://medium.productcoalition.com/non-disposable-team-3280c0c5fb82">Non-disposable team</a> was originally published in <a href="https://medium.productcoalition.com/">Product Coalition</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to use Dark Magic to generate Product Ideas]]></title>
            <link>https://medium.productcoalition.com/use-dark-magic-to-generate-product-ideas-b4a2f60c1e07?source=rss-94a75f33915a------2</link>
            <guid isPermaLink="false">https://medium.com/p/b4a2f60c1e07</guid>
            <category><![CDATA[product-management]]></category>
            <category><![CDATA[startup]]></category>
            <dc:creator><![CDATA[Guilherme Coelho]]></dc:creator>
            <pubDate>Thu, 19 Jan 2017 14:15:32 GMT</pubDate>
            <atom:updated>2022-06-17T14:10:01.314Z</atom:updated>
            <content:encoded><![CDATA[<h4>… that aren’t the same old boring ideas.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1015/1*o-o9i75WxELl7vyk9sZfpQ.jpeg" /></figure><p>It’s not easy to generate good ideas, particularly not when trying hard to do so. In those moments, thinking harder doesn’t usually get me very far but here’s a a technique I’ve found to be quite useful instead.</p><p>The “<strong>Dark Magic method”</strong> is certainly not something new but rather an ensemble of several interesting strategies I came across (from guys like Tim Ferris or Jason Fried) applied to product ideation. More specifically, Tim has suggested thinking of crazy stuff (and rejecting any idea that doesn’t seem crazy at first) as a way to get to novel directions and Jason is known for stating that “solving problems that customers actually have<strong>”</strong> is the most important factor in a successful product.</p><p>That being said, more than generating just <em>any product idea</em>, we want to get as many ideas as possible that fit the following criteria:</p><ul><li>Are real customer pains</li><li>Are somewhat novel (or at least are not “the problem everyone is solving”)</li></ul><p>It was while trying to get those two outcomes that I ended up using some variations of this method and refining it along the way. Here’s how it goes.</p><h4><strong>Step 1 — Who’s your User?</strong></h4><p>Start by identifying the people you’re building the product or feature for. You could do this in any way you find appropriate, like using personas or simply writing down a high-level definition of your customer.</p><p>If you don’t know who to build a product for, I’ve found it useful to list a bunch of interest groups you are part of yourself. In my case it would be something like this:</p><ul><li>Product Manager</li><li>Startup employee</li><li>Remote worker</li><li>Freelancer</li><li>Weightlifting enthusiast</li><li>Self-improvement enthusiast</li><li>Campervan enthusiast</li><li>….</li></ul><p>The important part for now is to be able to picture clearly who is on the receiving end of the product you’re creating. That’s the person who’s supposed to like it and pay for it.</p><p>For the sake of providing an example, I’ll pick one of my interest groups above: <em>(rather-weak-)Weightlifting enthusiast.</em></p><h4>Step 2 — <strong>Incarnate your User</strong></h4><p>After you know who you’re building for, the next step is to put yourself in that person’s shoes. If you’re not a target User yourself, it’s usually a good idea to picture someone you know who is.</p><p>Take some time to think of this person (or this part of yourself), how his or her routines are, how a normal day looks like. That will help you create empathy and more easily <em>think like that person.</em></p><p>Since I took <em>Weightlifting-enthusiast-Guilherme </em>as the customer I’m thinking about how my normal training routine looks like, from the moment I prepare my bag at home, walk to the gym, warmup, start the first exercise, …</p><h4>Step 3— <strong>The Dark Magic</strong></h4><p>Now that you’re living in your target User’s skin, complete the following sentence:</p><blockquote><strong><em>“It would be nothing less than dark magic if I could … ”</em></strong></blockquote><p>And make sure to come up with real magic-level stuff! Don’t bother thinking of concrete products or features for now. Try to focus on outcomes that would be so great you can hardly believe they’ll ever be possible.</p><p>And that’s it. The expectation is that if you follow this process you’ll end up with a few ideas that simultaneously contain immense value to your target User and are none of the usual things everyone is solving already.</p><p>Hope you find this useful and definitely <a href="http://www.iamguico.com/">let me know</a> if you use the method — I’d love to hear!</p><p><em>Hey! Thanks for reading this far. My name is Guilherme. I’m a Product Manager by trade, and more recently became co-founder of </em><a href="https://www.betasort.com/"><em>Betasort</em></a><em> , where we’re building an all-in-one toolset for running world-class beta programs. Hope you’ve enjoyed this article!</em></p><p>PS: Here are a few dark-magic-level wishes I came up with as <em>weightlifting-enthusiast-Guilherme. </em>I’d definitely pay money for products that deliver (some of?) them:</p><ul><li>… be 100% sure of what exercise routine works best to achieve my goal in the shortest time.</li><li>… be 100% sure the gym is kind of empty (so I don’t have to wait any equipment) but still with enough people (so I don’t feel alone there).</li><li>… get one final and unbiased view of all Nutrition “knowledge” in the world and know exactly what’s the best diet possible, all things considered: health, physique, environmental impact, etc.</li><li>… see what’s happening to my body in real time, on a molecular level, so I know if my training is doing what it should.</li><li>… always get real-time tips during my workouts so I know instantly how to improve my technique.</li><li>… input my desired fat percentage into a machine and simply be at that level ~30min later.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b4a2f60c1e07" width="1" height="1" alt=""><hr><p><a href="https://medium.productcoalition.com/use-dark-magic-to-generate-product-ideas-b4a2f60c1e07">How to use Dark Magic to generate Product Ideas</a> was originally published in <a href="https://medium.productcoalition.com/">Product Coalition</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Serenity]]></title>
            <link>https://medium.com/life-tips/serenity-2c72f8a3429f?source=rss-94a75f33915a------2</link>
            <guid isPermaLink="false">https://medium.com/p/2c72f8a3429f</guid>
            <category><![CDATA[minimalism]]></category>
            <category><![CDATA[life-lessons]]></category>
            <category><![CDATA[self-improvement]]></category>
            <dc:creator><![CDATA[Guilherme Coelho]]></dc:creator>
            <pubDate>Mon, 19 Dec 2016 21:47:08 GMT</pubDate>
            <atom:updated>2016-12-20T09:32:37.656Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QKtmAtsKM-m9ITP5Ov7mwA.jpeg" /></figure><p>We tell our kids they should aim for great things. Be the best student, make it to team captain, get the main role in the theatre play. We celebrate when they succeed and offer comfort when they fall short. We think we make their goals our own, when in truth they are often fighting for ours.</p><p>It’s scary how excess has become the norm in almost all we do, but it’s not exactly surprising given that most human lives start like this. These are the children who will grow up without a hint of how to live happily. Without knowing the value of balance, of relationships and of self-appreciation.</p><p>Instead, they know only one strategy: maximize. They’ll follow it and get that high-paying job, the right set of friends and the right set of things (usually a lot of them). They will “do well”, or at least believe so.</p><p>But maximization is a direction, not a place. It can never be reached. So these same kids who were trained to achieve are now running to catch the air. Good is never good enough, less is boring, the mouse-wheel keeps turning and they run faster and faster.</p><p>Once a friend told me that the only way to be happy is to be happy. I didn’t take him very seriously but now I get it. He meant that if you make happiness depend on something that didn’t happen yet, then you’ll be unhappy until then. And since there’s always something more you can wish for, it’s likely you’ll spend most of your life unhappy if you think that way.</p><p>It follows that appreciating where you already are is the only way to feel good and satisfied. It’s crucial to reflect on what you need, and even more on what you don’t need, in order to realize that sometimes good is good enough. Doing this can certainly be hard, but reaching what you want gets significantly easier when you narrow it down to what really matters to you. Be it objects, goals, tasks or rules, getting rid of what is not essential gives me a sense of clarity and rest. Whatever is left feels important, yet manageable.</p><p>We humans haven’t done this very well so far. As a matter of fact, after a few centuries of living for more, we’ve become extremely good at it. We’ve built a whole society that thrives on continuous growth, therefore perpetuating it with tremendous efficacy. What we don’t collectively know yet is that we’re fooling ourselves. That we’re betting on a strategy that is fundamentally flawed. That we need to recycle this ability of ours and point it in a more promising direction.</p><p>In the meantime this noisy and accelerated way of living is taking its toll. It’s harming our bodies, our happiness and our relationships but also the planet we live in, which only knows one rhythm and can’t keep up with ours.</p><p>Therefore I think it’s worth reconsidering this strategy. It’s worth taking a few steps back to reflect on how all this is affecting our lives. To think of how it’s affecting our physical health but also our relationships with our parents, our children, our friends and our lovers. To notice how it’s preventing us from doing what we like, from being where we want to be and from acting how we want to act. It’s worth imagining how it would feel like if we’d jump out of the mouse-wheel.</p><p>Living a sustainable life is not about sorting trash and efficient light bulbs, although those things may be part. More importantly, it’s about control and intentionality. It’s having the freedom to decide what gets to be part of your life and what doesn’t. It’s about realizing we’re on that goddamned wheel and that we can jump out.</p><p>For our own good, we need to collectively learn to appreciate what we already have. At the end of it awaits a rare reward: serenity.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2c72f8a3429f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/life-tips/serenity-2c72f8a3429f">Serenity</a> was originally published in <a href="https://medium.com/life-tips">Be Yourself</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why (good) software matters]]></title>
            <link>https://medium.productcoalition.com/why-good-software-matters-e2870f2801ce?source=rss-94a75f33915a------2</link>
            <guid isPermaLink="false">https://medium.com/p/e2870f2801ce</guid>
            <category><![CDATA[purpose]]></category>
            <category><![CDATA[society]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Guilherme Coelho]]></dc:creator>
            <pubDate>Thu, 28 Apr 2016 07:17:05 GMT</pubDate>
            <atom:updated>2016-09-10T14:22:47.993Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zQo75hVeX3H9OW2hAr4WdA.jpeg" /></figure><p>Lost in day-by-day activities I recently felt I didn’t know why I was working. Don’t get me wrong, my job is about building digital products — such as websites and mobile apps — and I knew exactly what I should do. What I didn’t know was why I was doing this work in particular and not something else. So I tried to get an answer by thinking of what made me get started with this in the first place, and hoping there was more to it than simple chance.</p><p>It turns out I’ve always built things. As a kid I loved carpentry and playing with tools on a workbench. As a teenager I built websites for my hobbies. Now as an adult, I like to cook my own food, build my own furniture and sew my own clothes. Building things makes me feel good and that’s the first reason why I have this job. The remaining question was then why, out of all things, was I building websites and mobile apps. I think the answer is part chance and part choice.</p><p>When I was born (some) kids had easy access to computers and not so easy access to the tools of other crafts. Furthermore, computers with internet meant we could easily learn more about them. That’s why I learned how to code and put pixels together, but never learned much about woodworking.</p><p>If I got started by chance, I definitely stayed by choice. I used to think it wouldn’t be fair to make this hobby my job, because it came so naturally to me that it didn’t feel like real work. After a couple of small detours, however, I made peace with myself and took the conscious decision to go do what I really liked. I started building things for a living, using the tools I already knew.</p><p>Today I keep building all kinds of things but I remain mostly interested in software. There’s no other thing you can create that’s as easy to give to other people. If what you pass around is good, people will use it. And as people use more software products, more and more of our daily things are recorded in digital devices. In order words, the record of our whole society is going from paper and spoken word into bits. <strong>Everything is becoming searchable, shareable and nothing can be effectively hidden. The truth is getting harder to violate.</strong> Software is making our society more transparent and fair.</p><p>This kind of impact can’t, however, be credited to all kinds of digital products. It’s a condition that the software is actually used by people, which means it should be built with the explicit purpose of serving well those who are going to use it. A process of designing products with such purpose in mind is usually called user-centric. It basically means software follows the User — with his or her personality, life circumstances, expectations and needs — and never the opposite.</p><p>Turns out my job is not so much about building products after all. It’s much more about getting to know real people.</p><h3>Keep reading:</h3><p><a href="http://blog.iamguico.com/its-ok-dont-be-afraid-you-can-stop-micro-managing/">It&#39;s OK. Don&#39;t be afraid. You can stop micro-managing.</a></p><p>Or <a href="https://tinyletter.com/guico"><em>get my articles directly on your inbox</em></a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e2870f2801ce" width="1" height="1" alt=""><hr><p><a href="https://medium.productcoalition.com/why-good-software-matters-e2870f2801ce">Why (good) software matters</a> was originally published in <a href="https://medium.productcoalition.com/">Product Coalition</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Great Teams, Great Products]]></title>
            <link>https://medium.productcoalition.com/great-teams-great-products-8842b73d3f2c?source=rss-94a75f33915a------2</link>
            <guid isPermaLink="false">https://medium.com/p/8842b73d3f2c</guid>
            <category><![CDATA[management]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[product-management]]></category>
            <dc:creator><![CDATA[Guilherme Coelho]]></dc:creator>
            <pubDate>Wed, 06 Apr 2016 19:55:47 GMT</pubDate>
            <atom:updated>2016-09-10T14:18:11.782Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*863D6LDnDaZohCS0CPIqHg.jpeg" /></figure><p>Some of the things we use are great, other things not so much — we’ve all experienced both and can easily tell the difference. What’s not so easy though, is to state <strong>what exactly makes great products great</strong> and how can one make more of those.</p><p>It is no exact science, and <em>great</em> is indeed a vague word, but one thing that I’ve found to hold true in most cases is that great products are created by a team of people. Yet, clearly, <strong>not all teams yield great products</strong> (or product features) — so I decided to focus on what makes those teams special.</p><p>This is far from being a finished process, but here are a few things I’ve found to work well:</p><h3>One ridiculously clear goal</h3><p>It’s tempting to think of teams as machines, with a number of parameters that can be adjusted to fine-tune the end product. In reality, however, teams are living systems of a very complex kind (I don’t mean it metaphorically), which get stimuli from the environment and react in ways that are not easy to predict.</p><p>Asking the team to meet a long list of goals in a certain priority is one of those ideas that make perfect sense in the <em>team-as-machine</em> model, but don’t have place in reality. Providing such complex input to the team will, not surprisingly, lead to a rather chaotic reaction and ultimately to slow progress. It’s just too much to accommodate for, too much to coordinate, too much information to be in everyone’s head at all times.</p><p>What works well is a very precise impulse: <strong>one</strong> well-defined goal for the team to pursue.</p><p>Instead of chaos, this time the team will be able to quickly morph into whatever shape is ideal to react to that stimulus or, in other words, achieve that goal. Each team member will have <strong>a clear principle to base his or her decisions on</strong> (which means less communication is needed) and the limitation in breadth will pay-off in terms of depth of the final solution.</p><h3>Reduce the scope</h3><p>A necessary consequence of having one clear goal is a certain degree of independence of each team: you can’t have a single goal if you also have to care about everyone else’s goals.</p><p>This is particularly relevant for startups, which tend to stay away from well-defined hierarchies of power, and often fail to fill the resulting void and establish a <strong>workable division of goals</strong>. That leads to a “everyone has to worry about everything” kind of feeling, which in turn translates into a stressful workplace and removes any sense of accomplishment. It’s impossible to have impact on everything, particular while being unable to go deep on anything.</p><p>So, make sure everyone gets a slice of the pie and can do his or her job without having to worry about everyone else’s.</p><h3>Size does matter</h3><p>There seems to be an ideal size for a team which reflects basic aspects of human nature. That’s <strong>3 to 6 people</strong> and here’s why:</p><p>Stay under 3 people and you’re likely to converge too quickly (and under-explore the problem) or simply lack the necessary skills to build the full solution without involving people from the outside. Therefore, adding more people will often lead to a more thorough result and reduce communication overhead, which means faster implementation.</p><p>Go over 6, however, and people will naturally form sub-teams — after all, how many times did you throw a party for 10 people and everybody was engaged in one big conversation? The answer is probably never. And that’s because it doesn’t work, not even in your free time with friends. With sub-teams comes communication overhead, diverging goals, slow progress and basically all the drawbacks of <em>separate-yet-not-independent</em> teams already mentioned above.</p><h3>No Rockstars</h3><p>Ninjas, rockstars, A-players, you name it — companies are obsessed with getting overachievers onboard, hoping their individual performance will add up to the team’s.</p><p>In reality, however, people who can significantly contribute more than the rest also prevent the rest from contributing as much as they could. They have to push someone else down in order to standout.</p><p>What’s more interesting though, is that at the end of the day, a balanced team will systematically deliver better results than an unbalanced one. It’s the difference between Brazil’s team of overachievers and Germany’s Mannschaft in which everyone can shine. This is not new and there’s a great <a href="http://www.ted.com/talks/margaret_heffernan_why_it_s_time_to_forget_the_pecking_order_at_work">TED talk</a> about the topic, in case you’re interested.</p><h3>Embrace the conflicts</h3><p>Conflicts are a sign of a healthy team, one which won’t settle for just any solution but will strive for the best one. Conflicts are the very mechanism for collectively shaping ideas that are more powerful than any individual input.</p><p>Avoid conflicts and you’ll end up with a dead team and a mild product that’s unlikely to make any difference.</p><p>That’s why conflicts should be welcomed with open arms and, here’s the catch, a clear strategy on how to resolve them. A strategy that’s optimized to help ideas crystallize.</p><p>How to tie-break? How to move forward when it’s unclear which way will best serve our goal? These questions must be answered by the team before any work is started. They could agree that someone will have the responsibility of breaking ties, or to use a more formal <a href="https://en.wikipedia.org/wiki/Conflict_management#Models_of_conflict_management">conflict management framework</a> — the important thing is to have a plan before the first clash happens.</p><p>Thanks for reading.</p><p><em>This piece was first </em><a href="http://blog.iamguico.com/great-teams-great-products/"><em>published on my blog</em></a><em>.</em></p><p><em>If you’ve enjoyed reading it, you could </em><a href="https://tinyletter.com/guico"><strong><em>subscribe to my newsletter</em></strong></a><em> and get a regular digest of articles, job roundups and interviews about the for-purpose organizations that are </em><strong><em>reinventing how and why we work</em></strong><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8842b73d3f2c" width="1" height="1" alt=""><hr><p><a href="https://medium.productcoalition.com/great-teams-great-products-8842b73d3f2c">Great Teams, Great Products</a> was originally published in <a href="https://medium.productcoalition.com/">Product Coalition</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Not-for-profit, Yes-for-purpose]]></title>
            <link>https://medium.com/@iamguico/not-for-profit-yes-for-purpose-1462c49da25?source=rss-94a75f33915a------2</link>
            <guid isPermaLink="false">https://medium.com/p/1462c49da25</guid>
            <category><![CDATA[organizational-culture]]></category>
            <category><![CDATA[freedom]]></category>
            <category><![CDATA[nonprofit]]></category>
            <dc:creator><![CDATA[Guilherme Coelho]]></dc:creator>
            <pubDate>Thu, 17 Mar 2016 14:07:10 GMT</pubDate>
            <atom:updated>2016-03-25T13:27:46.262Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Sebn25WMR32lGg-M_QxO-Q.jpeg" /><figcaption>Source: Unsplash</figcaption></figure><p>Organizations have been part of human life for a long time. To a certain degree, they are a reflection of our intrinsically social nature or, in other words, an expression of our necessity to interact with others in the never-ending pursuit of happiness and meaning.</p><p>We usually get together around a certain purpose and, because each of us commonly relates to a number of purposes, we’re often part of various groups: the childhood friends, the soccer team, the political party, the company, you name it.</p><p>Each of these groups has its <em>raison d’être</em> and a number of ingredients which are necessary to make it come to life. The soccer team, for instance, might exist so that its members can play together and, for that to happen, they need 11 people, identical jerseys, a ball and an opposing team.</p><p>Money is, not surprisingly, a common ingredient for most organizations because it can be transformed into a variety of other necessary ingredients — the material ones, the ones you can buy.</p><p>Having said that, it’s kind of odd that despite the richness of purposes people find to get together in organizations, we still put them in just two boxes: the ones striving to accumulate money just for the sake of doing it and, well, “the rest”.</p><p><strong>How disrespectful for the rest.</strong></p><p>Yes, the rest is not-for-profit. But it is also for-happiness, for-health, for-passion, for-equality, for-fairness, for-sustainability. The rest is the only way out of this rat race that’s destroying us and our planet with it.</p><p><em>This piece was first </em><a href="http://blog.iamguico.com/not-for-profit-yes-for-purpose/"><em>published on my blog</em></a><em>.</em></p><p><em>If you’ve enjoyed reading it, you could </em><a href="https://tinyletter.com/guico"><strong><em>subscribe to my newsletter</em></strong></a><em> and get a regular digest of articles, job roundups and interviews about the for-purpose organizations that are </em><strong><em>reinventing how and why we work</em></strong><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1462c49da25" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[It’s OK. Don’t be afraid. You can stop micro-managing.]]></title>
            <link>https://medium.productcoalition.com/it-s-ok-don-t-be-afraid-you-can-stop-micro-managing-9ba3664ab408?source=rss-94a75f33915a------2</link>
            <guid isPermaLink="false">https://medium.com/p/9ba3664ab408</guid>
            <category><![CDATA[organization-design]]></category>
            <category><![CDATA[organizational-culture]]></category>
            <category><![CDATA[product-management]]></category>
            <dc:creator><![CDATA[Guilherme Coelho]]></dc:creator>
            <pubDate>Wed, 24 Feb 2016 18:36:46 GMT</pubDate>
            <atom:updated>2016-03-25T13:26:52.696Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EdmK02tHDYJ7IWy9yah56w.jpeg" /><figcaption>Photo: Unsplashed</figcaption></figure><p>Let’s face it: <em>“Hey, can you give me a status update?”</em> is just another set of words for saying <em>“I don’t trust you enough to do your work, so let me look over your shoulder to ensure everything is like I expected”</em>.</p><p>Put this way it might sound exaggerated, but take a second to think about all those times you looked into someone’s work before that person chose to show it to you (or make it visible for everyone). All those times you were afraid.</p><p>I don’t have to emphasize that fear is not a good standpoint from which to make decisions or collaborate with others. Humans are known to act by impulse when they are afraid — after all, that bear is running after you, what else can you do but <strong>run</strong>?</p><h3>But if there’s no bear, then why are we afraid?</h3><p>An easy explanation is that we all have this (not so) little thing inside us called Ego, which at times can prevent us from seeing other people’s perspectives and lead us to believe we can do anyone’s work better than them. So we’re afraid because everyone will always, by default, do a worse job than we would if only we were in their shoes. We’ve got to jump in! We’ve got to get them back on track! Now!</p><p>Perhaps more often though, we’re afraid because we feel insecure about how good of a job we’ve done communicating to the team what the point of their work is. And because we’re personally responsible for the content of the team’s work, we want to get the chance to steer them into the “right direction” in case they drift away.</p><h3>It’s all good, daddy’s here</h3><p>Here’s some encouraging news: none of us is better than everyone else. Sounds crazy, I know.</p><p>Now that we’ve got that out of the way, I really don’t think most micro-managers are simply selfish people with bloated Egos. Pretty much on the contrary, I’ve observed over and over again that all of us together are creating a culture, and companies thereof, that prize and foster micro-management. From heroic founders “obsessed” with detail (hey, Steve!) to organizational structures that systematically concentrate power into a small set of people, we shouldn’t be surprised that our lives are cramped with micro-managers in one side, and unfulfilled, unempowered colleagues on the other. <strong>We’ve designed this system</strong>!</p><p>The important detail though, is that we’ve designed it a long time ago.</p><p>Let’s take some time to think that one through. We’re managing our companies based on ideas created in a time before we even had managed to figure out that, for example, men are not more intelligent than women. So, to a large extent, we’re still managing our lives from paradigms we don’t believe in anymore.</p><h3>What happens if we don’t stop</h3><p>If we keep putting people in boxes, doubting about their value and, fundamentally, not letting them use their talents and do their jobs, we’re directly achieving two things:</p><p><strong>We’re limiting the team’s potential by that of one person.</strong> The team will stay inside the little box of things the manager understands now, and the company will effectively miss on the potential all those people could add. Think about it this way: for the company, the only difference between a single empowered employee and a micro-managed team of 10 is that the second option costs 10x.</p><p><strong>We’re actively making the biggest part of our awake time into the burden we’ve come to accept as work.</strong> Stop the illusion that work and life are different things — for most practical reasons our work is our life. So don’t ditch on “life things” while on the job. At work, as in life, people need to be respected, have meaningful relationships and, generally, feel good and happy.</p><h3>So, how can we create better companies?</h3><p><strong>Assume people are good.</strong> Get excited by the variety of ideas and approaches they bring in because they are different, have different skills and different talents.</p><p><strong>Shift to mentoring.</strong> There’s nothing bad about sharing your own talents and the things you know a lot about. When mentoring someone, see your task as that of adding something to the other person, not of replacing something in them. Remember it’s a privilege that someone is opening his or her mind to your ideas.</p><p><strong>See leaders as facilitators</strong>. Leaders are trusted members of the team who keep it together, facilitate the work and help solve conflicts. They are neither personally responsible for the content of the work the team produces nor do they have any power over the team members.</p><p><strong>Move to distributed decision-making.</strong> If everyone in the company has good intentions and no one is smarter than the rest, then why should a small set of people always be given the task to “decide”? Deciding is not a job, it’s part of everyone’s work.</p><p>Once this is changed, and decisions are made by whoever has the best context to do so, then reporting itself starts looking a lot like that old DVD player most of us still own — a tool from the past that we don’t need anymore.</p><p><em>This piece was first </em><a href="http://blog.iamguico.com/its-ok-dont-be-afraid-you-can-stop-micro-managing/"><em>published on my blog</em></a><em>.</em></p><p><em>If you’ve enjoyed reading it, you could </em><a href="https://tinyletter.com/guico"><strong><em>subscribe to my newsletter</em></strong></a><em> and get a regular digest of articles, job roundups and interviews about the for-purpose organizations that are </em><strong><em>reinventing how and why we work</em></strong><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9ba3664ab408" width="1" height="1" alt=""><hr><p><a href="https://medium.productcoalition.com/it-s-ok-don-t-be-afraid-you-can-stop-micro-managing-9ba3664ab408">It’s OK. Don’t be afraid. You can stop micro-managing.</a> was originally published in <a href="https://medium.productcoalition.com/">Product Coalition</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Start here: Statistics for A/B testing]]></title>
            <link>https://medium.productcoalition.com/start-here-statistics-for-a-b-testing-5f5c7e02ce1e?source=rss-94a75f33915a------2</link>
            <guid isPermaLink="false">https://medium.com/p/5f5c7e02ce1e</guid>
            <category><![CDATA[product-management]]></category>
            <category><![CDATA[growth-hacking]]></category>
            <category><![CDATA[online-marketing]]></category>
            <dc:creator><![CDATA[Guilherme Coelho]]></dc:creator>
            <pubDate>Thu, 04 Feb 2016 13:44:11 GMT</pubDate>
            <atom:updated>2022-01-21T08:25:31.292Z</atom:updated>
            <content:encoded><![CDATA[<p>We’re Agile, we think lean, we’re data-driven. If you live in the new economy and work in some sort of digital product you hear some of this buzzwords (or perhaps all of them) on a daily basis.</p><p>You can argue they don’t all mean the exact same thing but they do carry one common fundamental principle:</p><blockquote>Blind guessing is not a desirable way of making decisions, but rather a necessity when nothing more reliable exists to inform our reasoning. In developing digital products, however, that necessity rarely exists.</blockquote><p>Testing hypothesis with some sort of A/B testing is one of the ways to avoid blind-guessing and “hope-for-the-best” approaches. Below you’ll find a collection of statistics principles and best-practices that I consider fundamental for anyone wanting to use A/B testing.</p><p>Hope you’ll find it useful!</p><p><em>Hey! Thanks for reading this far. My name is Guilherme. I’m a Product Manager by trade, and more recently became co-founder of </em><a href="https://www.betasort.com/"><em>Betasort</em></a><em> , where we’re building an all-in-one toolset for running world-class beta programs. Hope you enjoy this article!</em></p><h3>What is an A/B test?</h3><p>An A/B test consists of taking two comparable groups of users and exposing them to two different versions (the control and a variation) of a software experience. By doing that and measuring how each of the groups interacts with the software we hope to infer which of the two versions best serves its purpose.</p><h3>Some relevant statistics concepts</h3><h4>Overall Evaluation Criterion (OEC)</h4><p>Also called Primary Goal (in Optimizely and other tools) or Dependent variable, in statistics terminology. It’s a quantitative measure that defines the experiment’s objective. <em>Ex. Conversion rate.</em></p><h4>Null hypothesis (Ho)</h4><p>The hypothesis that the OECs for the variants are not different in fact, and that any observed differences during the experiment are due to random fluctuations. The goal of our split-test is to reject this null hypothesis, and therefore conclude with statistical significance that an observed difference in the OECs is not due to random fluctuations. Failing to reject the Null hypothesis doesn’t mean it is true, means no conclusion can be drawn.</p><p>(Most of the statistics used in A/B testing come from the field of <a href="https://en.wikipedia.org/wiki/Statistical_hypothesis_testing">Statistical hypothesis testing</a>.)</p><h4>P-value</h4><p>In the context of Null hypothesis testing, the p-value for a given observed result represents the probability obtaining such result (or any result more extreme than that) only due to random fluctuations, assuming the Null hypothesis is true.</p><p>For the purpose of A/B testing, we can say the p-value represents the probability of observing a given OEC difference (or any difference more extreme than that) between variants due only to random fluctuations, assuming the OECs are in reality the same for both.</p><p>Also:</p><ul><li>The P-value is not the probability of the null-hypothesis being true — it is calculated assuming the null hypothesis is true to start with.</li><li>On Optimizely p-value is not made visible and instead the concept “Statistical Significance status” is mentioned. This value represents the probability of a given observed difference not being due to chance.</li></ul><h4>Significance level (SL)</h4><p>Set in the beginning of the experiment, the significance level (traditionally set to 1% or 5%) is the probability threshold under which the null hypothesis can be rejected. In other words, if a given OEC difference has p-value smaller than the significance level, we’ll conclude the OECs are not the same for the variants and, therefore, believe there’s causality between the change and the effect. In other words, we can say variant B’s OEC is statistically significantly different from the control’s OEC.</p><p>Significance level is also the probability of rejecting the Null hypothesis when this hypothesis is true (called Type I error or false positive).</p><p>As an example, setting the SL of an A/B test to 0.05 (or 5%) means that even for statistically significant results, there’s a 5% risk of concluding there’s a difference between the variations when in fact there is not.</p><p>Note: Optimizely uses different terminology here and allows an <em>Optimizely-significance </em>value to be set for each project (default 10%). According to the documentation, the <em>Optimizely-significance</em> value represents 1-pvalue, which means that to set the SL to 0.05 this value must be chosen to be 95%.</p><h4>Power</h4><p>On the contrary of SL which focuses on assessing the down-side of the experiment (probability of error and no-error), Power represents the probability of correctly rejecting the null hypothesis or obtaining a true-positive.</p><p>In other words, statistical Power is the likelihood that an experiment will detect an effect if there is in fact an effect to be detected.</p><p>The power of an experiment is influenced by a number of factors (such as sample size) and 80% is a typical desired value. Experiments with low power will <em>never</em> be conclusive in practical terms.</p><h4>Standard deviation</h4><p>The standard deviation (σ — greek ‘sigma’)<strong> </strong>is a measure that is used to quantify the amount of variation or dispersion of a set of data values, in other words it represents how much ‘spread-out’ the data points are. In the context of A/B testing, we look at the distribution of observed OECs.</p><p>Conversion events, such as purchases or registrations, can be modelled as a Bernoulli trial with probability (p)= conversion rate. In that case, the standard deviation (σ) is given by:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/172/1*F3onIMmYwGaEnP80lHCkDw.png" /></figure><h4>Minimum sample size</h4><p>For a desired Power (probability of detecting a true-positive if it exists) and a sensitivity Δ (the amount of change we want to detect, ex. 5% of control value) we can estimate a minimum sample size that‘s needed to achieve that. A useful formula, for a Power = 80% is the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/155/1*vWZWfBOCtXkkvfCRNZK5xA.png" /></figure><p>where n is the number of users in each variant. That means that to detect a change Δ with 80% probability, we’ll need n users in each variant.</p><p>Such estimation is interesting to understand how long a given experiment we’re planning is likely to take in order to detect the change we’re looking for, if it exists. As can be seen from the formula above, n depends on the square of σ which means we can reduce the minimum sample size by choosing an OEC with lower variability (ex. <em>conversion rate</em> instead of <em>purchase value</em>).</p><p>Perhaps even more relevant, this formula tells us that small changes are harder to detect. Which means that great ideas will hardly be missed but also that A/B testing is often not a suitable/efficient way to make small incremental improvements.</p><h4>Confidence intervals</h4><p>The Confidence interval is a range of values that is likely to contain the actual effect of the variant on the OEC of the control. Just as an example, a 95% confidence interval (that is, an interval that has 95% probability of containing the actual effect) could be calculated as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/269/1*mmeO6ov9cQX1JdRj--ntTg.png" /></figure><p>where OB and OA are the average value of the observed OECs and sigma-d is an estimation of the standard deviation of the difference. In practice though, tools such as Optimizely calculate this interval automatically.</p><p>Looking at the confidence interval gives us an insight of potential up and down-side of the experiment. It allows us to say, with a given probability (in the case above at least 95%), that the true difference between OECs is not higher than the higher bound of the interval. That can support, the decision of stopping the test because the upside is very likely lower than what we were hoping to achieve.</p><p>However, we might not be able to tell significantly if the test represents, or not, an improvement for the OEC — that conclusion can only be made if the whole interval is positive and does not contain 0.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/372/0*lUQ_UZCFMiCTtg5S." /></figure><p>Fig 1. Inconclusive confidence interval as seen on Optimizely</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/296/0*QbigfmR2evoo6oZ6." /></figure><p>Fig 2. Conclusive confidence interval as seen on Optimizely</p><p>Note:</p><ul><li>Optimizely automatically sets the Confidence interval to the same value as <em>Optimizely-significance</em>. Which means, if you set <em>Optimizely-significance</em> to 95% in your project, you’ll see 95% confidence intervals.</li><li>Optimizely only marks an experiment as conclusive if this last condition is met, which means that depending on the OEC goal we’ve set for the experiment, we are likely to be able to make a negative conclusion earlier. To be more precise, we can make a conclusion that our goal was not met as soon as the confidence interval is inferior and doesn’t contain our OEC goal.</li></ul><h3>Good practices</h3><h4>Choose OEC in advance</h4><p>Define the success metric for your test in advance and define a minimum improvement you’d consider a success. Those two decisions must take into account the following factors:</p><ul><li>Slow variant OEC (lower 𝝈) need smaller sample sizes (and, therefore, less time) to detect changes. In an e-commerce setup, a <em>conversion rate</em> is a good example of a slow variant OEC, as opposed to <em>basket size.</em></li><li>Large differences are easier and faster to detect. Small improvements take big sample sizes, a lot of time and, therefore, might never reach a conclusion. So make big changes and set big goals.</li></ul><h4>Aim big</h4><p>Be ambitious not only on the goals you set but also on the variations you test. For example, try two completely new designs or split users between two completely different pages. Stay away from little incremental changes since A/B testing is unlikely to be the right tool to support your decision on that situation.</p><h4>Traditional A/B testing vs. Multivariable testing</h4><p>Multivariable testing is a generalization of A/B testing in which several factors are tested simultaneously — in its most standard form (the one supported by Optimizely — also called full factorial) there’s one variant for each possible combination of the different factors.</p><p>This kind of testing is ideal if you suspect several factors will interact strongly. It should also be used if you have, or can have, several factors (called sections on Optimizely) implemented and tested straight away because you’ll reach a conclusion faster than with consequent A/B tests of a single factor each.</p><h4>Minimum sample size for Multivariable experiments</h4><p>The formula given above for Minimum sample size of a traditional split test with n variants can be generalized to the case of Multivariable experiments, with several factors and several variants per factor:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/163/1*S_npEn-0c1Kuucf7zX_C3w.png" /></figure><p>where r stands for the total number of variants and n is still the number of users per variant.</p><h4>Traffic allocation</h4><p>It’s a good strategy to do a traffic ramp-up, that is: start by serving variant B to only 10% of the users to ensure there are no implementation problems or catastrophically losing variations.</p><p>To maximize the Power of the experiment, however, you should increase the traffic allocation to 50%/50% as soon as possible. As an example, an experiment running at 1%/99% will have to run 25 times longer than the same experiment at 50%/50%. This effect is particularly relevant for multivariable testing, as any decrease in the traffic allocated to one of the variations will cap the sample size for any comparison within the test.</p><p><em>EDIT: As mentioned by Chris Stucchio, it’s fundamental throw away all the data and start collecting from scratch once you’ve ruled out any implementation bugs and scaled traffic allocation to 50–50. In his words “Since conversion rates change during the week (i.e., sat != tues), keeping the data during the ramp-up period is a great way to get wrong results due to Simpson’s Paradox.”</em></p><h4>Always have to wait until test is “conclusive”?</h4><p>No, it’s wise to assess each case since in some cases — particularly if minimum sample size or magnitude of change were not properly estimated — significance will never be reached because the power of the experience is too low.</p><p>In the majority of the cases, however, you will reach significance at some point. For these experiments, as described above, you might still be able to tell if your experiment didn’t reach its goal if the Confidence interval is lower and doesn’t contain the goal.</p><h4>Estimate minimum sample size</h4><p>Estimating the sample size helps you have an idea for how long the test will likely have to run until you have a conclusion. Tests taking longer than a couple of months are not recommended because several external factors, such as cookie churn, start becoming relevant and impacting the experiment.</p><p>You can play with factors such as the number of variations, magnitude of the expected change and the OEC you’ve chosen in order to tune and reduce the minimum sample size</p><h4>Examples</h4><p>Calculate Minimum sample size for a traditional A/B test (1 control, 1 variant) for detecting a 5% change in the conversion rate of an e-commerce checkout page with baseline conversion of 3%:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/639/1*U3SKirrIwwR9J14xyoSuFA.png" /></figure><p>Note that Δ is given by 0.03*0.05 because the magnitude of the change we’re trying to measure is 5% of the 3% baseline.</p><h3>Further reading and sources</h3><ul><li><a href="http://link.springer.com/content/pdf/10.1007/s10618-008-0114-1.pdf">Kohavi, Ron, et al. “Controlled experiments on the web: survey and practical guide.” Data mining and knowledge discovery 18.1 (2009): 140–181. APA (Published by Springer)</a></li><li><a href="https://en.wikipedia.org/wiki/P-value">https://en.wikipedia.org/wiki/P-value</a></li><li><a href="https://en.wikipedia.org/wiki/Type_I_and_type_II_errors#Type_I_error">https://en.wikipedia.org/wiki/Type_I_and_type_II_errors#Type_I_error</a></li><li><a href="https://www.optimizely.com/resources/multivariate-testing/">https://www.optimizely.com/resources/multivariate-testing/</a></li><li><a href="https://help.optimizely.com/hc/en-us/articles/200039895">https://help.optimizely.com/hc/en-us/articles/200039895</a></li></ul><h4><em>Did you like this post?</em></h4><p><em>You could check out </em><a href="https://www.betasort.com/"><em>Betasort</em></a><em>, the tool I’m building to help Product Managers run world-class beta programs, or </em><a href="https://twitter.com/iamguico"><em>follow me on Twitter</em></a><em>. See you soon!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5f5c7e02ce1e" width="1" height="1" alt=""><hr><p><a href="https://medium.productcoalition.com/start-here-statistics-for-a-b-testing-5f5c7e02ce1e">Start here: Statistics for A/B testing</a> was originally published in <a href="https://medium.productcoalition.com/">Product Coalition</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>