<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Dave Collins on Medium]]></title>
        <description><![CDATA[Stories by Dave Collins on Medium]]></description>
        <link>https://medium.com/@radcollo?source=rss-889219a6365------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*RgMkRJyzP7EGTCec.jpeg</url>
            <title>Stories by Dave Collins on Medium</title>
            <link>https://medium.com/@radcollo?source=rss-889219a6365------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 17:02:15 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@radcollo/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[9 Tips to help find the right path as a Product Manager]]></title>
            <link>https://blog.prototypr.io/9-tips-to-help-find-the-right-path-as-a-product-manager-c53ee8b51901?source=rss-889219a6365------2</link>
            <guid isPermaLink="false">https://medium.com/p/c53ee8b51901</guid>
            <category><![CDATA[lean-startup]]></category>
            <category><![CDATA[agile]]></category>
            <category><![CDATA[product-management]]></category>
            <category><![CDATA[design-thinking]]></category>
            <dc:creator><![CDATA[Dave Collins]]></dc:creator>
            <pubDate>Tue, 09 Jan 2018 09:16:49 GMT</pubDate>
            <atom:updated>2018-01-19T13:30:25.676Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kYeJmi5q1kHMTUxuMzOXHA.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@cristina_gottardi">Cristina Gottardi</a></figcaption></figure><p>Product management can be a tough gig. The role covers a broad range of disciplines and responsibilities, probably more than any other job in the tech scene . So it’s a challenge knowing which path to focus on, which tends to vary from one company to another. Because of this general role ambivalence, it’s not unusual for product managers to assume responsibilities peripheral to the role, which can lead you down the path of limited effectiveness.</p><p>With this in mind, here’s some tips to make the best use of your precious time and to help ensure you become the impactful product manager that you want to be.</p><h3>1. Lead with a vision, objectives and empathy for your customers</h3><p>Know the company vision. You don’t have to come up with one, it should already exist. Any idea that you and your team come up with needs to support the vision. There’s plenty of <a href="http://www.themarketingblender.com/vision-mission-statements/">vision examples</a> out there, like the below one from Ikea.</p><blockquote>Ikea vision “To create a better everyday life for many people. Our business idea supports this vision by offering a wide range of well-designed, functional home furnishing products at prices so low that as many people as possible will be able to afford them.”</blockquote><p>A vision statement is nice, but supporting objectives (the what) and key results (the how) are needed to provide a better sense of alignment and progression towards living it. Again, your company objectives and key results (<a href="https://medium.com/startup-tools/okrs-5afdc298bc28">OKR’s</a>) <em>should</em> already exist (in some shape, way or form) and usually align to key customer journeys. See Uber’s objectives example below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*COgeNNIrtUA24v1hnRiX6g.jpeg" /><figcaption>The objectives of Uber.</figcaption></figure><p>Each of the above objectives needs to have supporting key results. For example the objective of ‘MORE DRIVERS’ needs to have the key results of</p><ul><li>Increase driver base in each region by 30%</li><li>Increase driver average session to 30hours / weekly in all active regions</li></ul><p>With the company vision and OKR’s down pat, you have a basis to lead with. You also need to have this direction in place to start discovering ideas that will shift you there.</p><h3>2. Product discovery is your core competence</h3><p><a href="https://svpg.com/">Marty Cagan</a> put’s it nicely “the aim is to prove that your ideas are valuable, usable, feasible and ethical with customers”.</p><p>Getting this validation requires you to get some form of a prototype in front of users as quickly and cheaply as possible. You’re essentially <em>listening</em> to customers and <em>looking</em> for early qualitative insights to filter out the good ideas from the bad ones.</p><p>To prepare a prototype fast you need tight collaboration with your designers. Co-locate them with you and your delivery team to ensure this happens. Regular feasibility checks with engineering of course are essential, they can let you know what’s technically possible because they’re neck deep in the domain everyday. Involve them early. Involve them often.</p><p>The more user testing you do the more you’ll discover, so be pragmatic about it. The best discovery processes are optimised for multiple testing sessions a week, imagine that!</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fd5_h1VuwD6g%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dd5_h1VuwD6g&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fd5_h1VuwD6g%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/9162bacd727dc13a5456b686756424f6/href">https://medium.com/media/9162bacd727dc13a5456b686756424f6/href</a></iframe><p>Quantitative <a href="https://medium.com/startup-snacks/10-essentials-for-a-b-testing-ce29a6e8d4ef">a/b testing</a> should be another important tool in your product discovery tool kit. It usually comes after you’ve started to build the best ideas (confirmed by your user testing). It helps you answer a different set of questions, like what quantitative impact is this initiative going to make on converting users through the product? Will this initiative make a positive impact on our OKR’s? People tend to be in one camp or the other when it comes to user testing vs. a/b testing. It’s your job to realise they are both important for different reasons.</p><h3>3. Over-communicate, but stick to the point</h3><p>Be on the front foot with communicating the results of your product discovery work. To peak engagement, lead with insights and results and articulate how they address user needs and impact OKR’s. Try to respond with encouragement when other product mangers are doing the same, they’ll really appreciate it.</p><p>Be transparent with your delivery team. Keep them across your plans with other areas of the business — most importantly let them know the <em>why </em>for these dealings. Keep them across the performance of the business, your product, the performance of a/b tests, product discovery insights and customer feedback trends. This provides context around what they could be working on next.</p><p>Your work can help make or break other departments in the business so you need keep be in sync with them constantly. It could be GTM managers, marketing, legal, customer care, sales, product managers from other departments, your manager, other managers — anyone. The point is, lines of communication should be constantly flowing. Reoccurring catch ups are an obvious way to do this, but keep them structured. Think about your agenda items and send them through in advance (if you have nothing, it’s okay to postpone), write and assign actions and ask for updates in the next catchup. This sounds obvious, but it doesn’t happen that often.</p><h3>4. You’re not a project manager…</h3><p>You’re not hired to keep work flowing smoothly through a delivery team, or to ship features on time. So, If you find yourself being solely responsible for things like release planning, running retros, showcases, stand-ups or grooming sessions; your time and attention in the areas that matter the most will take a hit. Don’t be lulled by the sense of satisfaction felt by keeping a team running well. You’ll probably end up being one of those well-run teams, working towards a product that no-one wants, because your discovery investment was too light on.</p><h3>5. You’re not a tester either…</h3><p>Be careful that the task of product verification is not misinterpreted as product testing. For me product verification means checking a (dev and test) complete story to ensure the acceptance criteria (that you should have defined) are met. If you find yourself spending excessive amounts of time, not only verifying, but also doing things like explorative testing, regression testing, testing on multiple phones, tablets, orientations or internet strengths — don’t! It’s not the best use of your time and the delivery team are there to ensure quality standards are met, via automated and manual quality assurance measures.</p><h3>6. You’re the master of scope</h3><p>I try to keep a simple rule; If a task is scope related i.e. backlog prioritisation, writing acceptance criteria, grooming a story with engineers in preparation for development or story verification. This is your area to keep on top off to help ensure the team runs smoothly. Own these tasks with passion!</p><h3>7. Know what tech innovation mostly means</h3><p>A lot of us tend to think of tech innovation as the most currently marketed capabilities like Machine Learning, Augmented/Virtual Reality and autonomous cars. Innovation however, more commonly comes from the fundamental ideas engineers have that allow for the improvement and scaling a features for full production use.</p><p>These are ideas around making code more intuitive and sustainable for future engineers to work with, ideas that allow capability to be reused, that allow products to load faster, that improve uptime, that make products more robust, secure and reliable. You need to understand the immense value of this work and make way for it. Denying engineers the opportunity to innovate in this regard, will come back to bite you big time further down the track. This is particularly important when scaling a successful a/b test variant. I’m all for faking it until you make it, just as long as you don’t fake it anymore, once you’ve mad it.</p><h3>8. What kind of analyst do you work with?</h3><p>You may come across different kinds of business analysts during your career, such as analysts responsible for defining low level detail and analysts responsible for data.</p><p>When it comes to matters of lower level scope definition, it should be lead be product, in consultation with design and engineering— if your going to manage a product you need to know how it works intimately, right? If you have an analyst in the team trying to do the same thing, there’ll be too much overlap on this task, creating inefficiencies due to role confusion and competition.</p><p>Working with an analyst responsible for data on the other had, can be hugely beneficial. They should be there to enable you and the team with the right tools (for reporting, a/b testing and what ever else you need to measure), the right processes (to design, implement and test tracking code and experiments) and to be able to interpret data correctly. This leads me to the next tip…</p><h3>9. Own your data</h3><p>I’ve recently come to the conclusion that product managers should also be responsible for tracking details, right down to the definition of goals, events and dimensions used for tracking and a/b testing tools. These things are really important extensions of the core products definition, it’s scope for you to own.</p><p>There’s on overhead to adopting this skill initially but it pays of, as you’ll soon have a more thorough understanding of, and confidence in the data that your product and customers produce. This will show in the way you communicate findings and impacts to stakeholders and the business.</p><h3>Closing thoughts</h3><p>The main aim for product managers it to know your company direction and customers intimately. Use this to direct your product discovery efforts, as this is absolutely your main competence and responsibility.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/646/1*M0Gks4S7lgLfzBLp-eT5_g.png" /><figcaption>Marty Cagan linking OKR’s, product discovery and delivery to prove Product/Marker Fit.</figcaption></figure><p>Ultimately you want to create an environment that allows you to pull the great ideas into production once the validation is there. This works best when you commit to solving customer problems and positive impacts to OKR’s, and not to features on a road map. Understanding and adopting the processes of Deign Thinking, Lean and Agile are critical in setting up this way of operating. This sets the best companies apart from most companies. <a href="https://medium.com/u/587293220115">Jonny Schneider</a> wrote a fantastic book to help in these areas, aptly named “<a href="https://medium.com/@jonnyschneider/understanding-how-design-thinking-lean-and-agile-work-together-fe65fd854407">How Design Thinking, Lean and Agile Work Together</a>”.</p><p>In order to achieve the above, be mindful about how you contribute to peripheral areas. You can still <em>lead</em> delivery teams without being a project manager. After all delivery teams are only as good as the products the deliver and it’s your job to ensure they deliver the right ones.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c53ee8b51901" width="1" height="1" alt=""><hr><p><a href="https://blog.prototypr.io/9-tips-to-help-find-the-right-path-as-a-product-manager-c53ee8b51901">9 Tips to help find the right path as a Product Manager</a> was originally published in <a href="https://blog.prototypr.io/">Prototypr</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[10 Essentials for A/B Testing]]></title>
            <link>https://medium.com/startup-snacks/10-essentials-for-a-b-testing-ce29a6e8d4ef?source=rss-889219a6365------2</link>
            <guid isPermaLink="false">https://medium.com/p/ce29a6e8d4ef</guid>
            <category><![CDATA[startup-snacks]]></category>
            <category><![CDATA[marketing]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[growth]]></category>
            <category><![CDATA[a-b-testing]]></category>
            <dc:creator><![CDATA[Dave Collins]]></dc:creator>
            <pubDate>Fri, 29 Dec 2017 12:27:41 GMT</pubDate>
            <atom:updated>2018-01-03T09:54:35.610Z</atom:updated>
            <content:encoded><![CDATA[<p>At <a href="https://www.immobilienscout24.de/unternehmen/jobs-karriere.html">Immo Scout24</a>, hypothesis driven development and a/b testing are core parts of our operating model. For product managers getting the process right can be tricky at first, but once you’re on top of your a/b testing game, it’s an incredibly enjoyable and fulfilling way to work, because you know exactly what impacts you and your team are making.</p><p>From my time operating this way, here’s a list of 10 essential things to consider when setting up, running, stoping and interpreting the results of your a/b tests.</p><h3><strong>1. Always start with a hypothesis</strong></h3><p>Hypothesis driven development, should be a standard thing for best practising product managers. You need to have your hypothesis well thought out and communicated before creating your test. Below is a <a href="https://www.thoughtworks.com/insights/blog/how-implement-hypothesis-driven-development">ThoughtWorks</a> example template used to construct your hypothesis. The ‘capability’ is essentially the scope of your test. The ‘outcome’ is the longer term, overarching impact you want to make (usually an operational or strategic business KPI ). The ‘Measurable signal’ is something you can accurately measure within the test, that (in theory) rolls up to the overarching outcome.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/585/1*etLz_VkUCtO2-DAtWI0Onw.png" /></figure><h3><strong>2. Goal selection</strong></h3><p>Goal selection (or measurable signal, as sated above), tends to divide opinions. In one camp you’ll find those that want to simplify &amp; focus things by restricting goals to the bottom line of a business, the KPI’s. If your initiative doesn’t positively impact KPI’s then why would you fully release it, right?</p><p>Well…sort of. This type of scenario is definitely a desired longer term ‘outcome’. But the reality is, you can’t always prove this type of over-arching impact within a time constrained test, which leads me to the other ‘micro-conversion’ opinion camp for goal selection (that I tend to agree with).</p><p>Micro conversions are essentially the lower levels of your Goal Tree (like the below example from <a href="https://help.optimizely.com/Ideate_and_Hypothesize/Improve_metrics_that_matter_with_your_optimization_program">Optimizely</a>). If you set your test up right, it should be easy to prove an impact. Using the below example, the micro-conversion of ‘add to cart’, is a ‘measurable signal’ that rolls up to the over-arching ‘outcome’ of increased revenue.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bYSFboEnYQk6BkMG9ja7Qw.png" /><figcaption>Goal tree example from <a href="https://help.optimizely.com/Ideate_and_Hypothesize/Improve_metrics_that_matter_with_your_optimization_program">Optimizely</a></figcaption></figure><p>Sometimes when the planets align, you’ll find that the measurable micro-conversions within a test can be equal to a business KPI. Say for example you’re conducting an a/b test on a new push notifications topic. A measurable micro-conversion for such a test, could be the additional app entires via the new push topic. If the business has a KPI around sessions or user retention, in this case the micro-conversion push notification app entry == the sessions overarching outcome.</p><h3><strong>3. Getting the scope of the test right</strong></h3><p>Setting up a test to see what colour submit button converts better on your enquiry form can be interesting, however this approach would require considerably more more effort to set up the test and ensure clean results, then the actual scope change. A testing strategy where the effort to set up the test outweighs the scope being tested can be unsustainable. The other thing to be careful of with such an approach is analysis paralysis, where too many tests, too much data and not enough progress could eventuate. It’s more that morale and patience are tested in such circumstances.</p><p>My current preference is test scope at more of macro level. At an initiative or feature level. It’s pretty standard these days to develop with feature toggles, so that your <a href="https://en.wikipedia.org/wiki/Minimum_viable_product">MVP</a> can be tested first and foremost. This allows you to validate the overall <em>concept</em> of your idea as quickly as possible. If this lives up to the expectations you set in your hypothesis, then perhaps over time more fine grained ‘coloured submit button’ tests will become more warranted.</p><h3>4. Selecting users and a duration for your test</h3><p>Generally speaking you should only need 2–4 weeks to get results the results you need. User selection is key for ensuring your test only needs to run for this duration. The percentage of users needed will be lower when testing upper funnel scope that most, if not all users will encounter, like a landing screen, a far reaching push notifcation topic, or a new navigation style.</p><p>For lower funnel scope like the conversion of a lead or purchase, you’ll struggle to get statistically significant results if you don’t filter out the users that never enter the specific area you’re testing. This can be done by creating;</p><ul><li>Pre-exiting audiences e.g. users that entered the test via a specific campaign that deep-links to a specific landing screen.</li><li>Segmenting users e.g. show results just for users that are logged in or have agreed to the global push permission.</li><li>Starting the test at specific points in the product e.g. initiate test only when users enter the specific landing screen or area of the product you’re testing.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t0TklosjfNWOs6wktIsqgQ.png" /><figcaption>Segmenting users in Optimizely X</figcaption></figure><h3><strong>5. Keeping tests mutually exclusive</strong></h3><p>You might work in an organisation where a/b testing becomes the norm and multiple product mangers form different areas of the business are all running tests at the same time in the app. This is a good problem to have and can be easily solved by using features like <a href="https://help.optimizely.com/Build_Campaigns_and_Experiments/Mutually_exclusive_experiments_in_Optimizely_X_Web">Exclusion Groups</a> from Optimizely X, which prevents users from overlapping into multiple tests and muddying up your results. This is the last thing you need!</p><p>Setting up a shared a/b testing roadmap that specifies when tests will run, what the goals are, where the test will start, how many users you’ll need, etc is a good way to plan for simultaneously running tests in advance. With this in place, you might find that due to test scope and user selections, simultaneous tests might not even need to have mutually exclusive users e.g. if you have simultaneous tests running on independent lead conversion funnels within separate parts of the the app.</p><h3>6. Testing your test</h3><p>Testing your test is crucial. If not done as part of the set-up phase, you won’t know about issues that prevent clean test results until it’s too late. Things like uneven distributions between the control and experimental groups (that can be caused by caching issues) or syntax errors in experiment keys that prevent test group data being sent from your a/b testing tool to other reporting tools are common, especially when you’re just starting out with a/b testing. If you don’t catch these issues during the test set-up phase, the opportunity cost is high, in terms of the time it takes to restart the process again, particularly if your running your tests in apps that will require another submit, approval and roll-out with Apple or Android.</p><h3><strong>7. Linking test user groups to other analytical tools</strong></h3><p>Most a/b testing tools have their own analytics that are great at overlaying things like statistical significance calculations, but can be limited in their coverage.</p><p>For a while now we’ve been sending custom dimensions for test user groups to Google Analytics. This allows us to augment the performance of a control vs. experimental group against any metic you want in GA, like user session frequency, active user rates and user retention rates, to name but a few. These ‘mid-level’ goal-tree metrics typically can not be assessed in a/b testing tools like Optimizely (at the time or writing).</p><p>For extra points you can send your test user group data to tools like <a href="https://firebase.google.com/docs/analytics/">Firebase Analytics</a>, where app uninstalls are measured (Android only at the time of writing). This is a partially important (negative) signal you want to assess when testing things like push notifications.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rt_dTJbZ_DpNj4s5cRjOyg.png" /><figcaption>Comparing Android app uninstalls between control and experimental groups in Firebase Analytics</figcaption></figure><h3><strong>8. Interpreting results</strong></h3><p>If you don’t have an even split between your control and experimental groups (by bug or design) be careful about interpreting (and communicating) any results of your test. Chances are the results would be different if the split was even.</p><p>Assuming you achieve an even split with your user groups, the below data points around Conversation Rate and Improvement are self explanatory. Where some get confused is the interpretation of the Confidence Interval and Statistical Significance.</p><p>The below example would be communicated as there being a 99% likelihood of a conversion rate improvement being between +2.53% and +12.18%, with the mid point of this confidence interval being +7.29%.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7SwCIRH_ixmUuEeRG55gbw.png" /><figcaption>Interpreting test results in Optimizely X</figcaption></figure><h3><strong>9. Creating holdout groups</strong></h3><p>Sometimes you might really want to test an initiatives impact against an overarching outcome like the retention of more active users. It can take an extended amount of time to get a measurement here(much longer then the standard 2–4 week a/b test duration), as you’ll need the accumulation of lots of users, all travelling through their user journeys which can take months, if not years (depending on the type of product you have)! Holdout groups are essentially a long-term control group that may never see your new landing screen, new push notifcation topic or new navigation style. They are typically a smaller percentage of your users (maybe 5%) and If the holdout group persists for say 6 months, you should be able to accumulate enough users to get a good read on longer term retention rates (for example) between 5% of the do haves and 5% of the don’t haves for the new(ish) initiative in question.</p><h3><strong>10. Stopping tests</strong></h3><p>So you’ve successfully set up and run your test. The results were well interpreted and you have good insight about what your next step should be (a great article about whether to scale, iterate or kill can be found <a href="https://mobilegrowthstack.com/scale-iterate-or-kill-f4a8e8987d4e">here</a>).</p><p>In the situation where you want to fully roll out your tested feature (to scale it), be careful that you don’t to stop the test and switch off the new functionality for your experimental group, thus leaving them confused and potentially annoyed. To avoid this, make sure you pause the test so that things stay the same for users already in the test and stope it from including more new users in the test over time (to save on credits and users for other tests). Once the version of your app with the fully rolled out new functionality is fully adopted by your active user base, it’s safe to stop your test and archive the results.</p><p>Happy testing!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ce29a6e8d4ef" width="1" height="1" alt=""><hr><p><a href="https://medium.com/startup-snacks/10-essentials-for-a-b-testing-ce29a6e8d4ef">10 Essentials for A/B Testing</a> was originally published in <a href="https://medium.com/startup-snacks">Startup Snacks</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Smarter push notification frequencies]]></title>
            <link>https://medium.com/bonza-product/smarter-push-notification-frequencies-79d97fbf6ce2?source=rss-889219a6365------2</link>
            <guid isPermaLink="false">https://medium.com/p/79d97fbf6ce2</guid>
            <category><![CDATA[mobile-apps]]></category>
            <category><![CDATA[engagement]]></category>
            <category><![CDATA[push-notification]]></category>
            <category><![CDATA[mobile]]></category>
            <category><![CDATA[growth]]></category>
            <dc:creator><![CDATA[Dave Collins]]></dc:creator>
            <pubDate>Mon, 13 Nov 2017 11:14:33 GMT</pubDate>
            <atom:updated>2017-11-13T11:14:33.747Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*9ngAZRzpmd6G3BBFAGgRXw.jpeg" /></figure><h3>Solve the user need and grow the app</h3><p>Notifications are powerful. As <a href="http://www.phiture.com/">Andy Carvell</a> puts it, if you create an activity notifications topic that’s highly relevant to users, has extensive reach across your user-base, and is delivered at a healthy frequency — then expect postive impacts to user engagement and retention as shown in his <a href="https://mobilegrowthstack.com/what-is-the-mobile-growth-stack-426c6e474329">Mobile Growth Stack</a>.</p><p>In this article I wanted to dive into the topic of notifcation frequency, for even if you’ve nailed the relevance and reach with your topic, the frequency at which notifications are pushed, can make or break the whole topic.</p><h3><strong>Why move to a smarter push frequency</strong></h3><p>If your app pushes activity notifications at set intervals to subscribers, say twice daily for when new content is listed in a favourite search, our assumption is that highly engaged users, will appreciate being informed about a newly listed item in as-close-to-real-time as possible. This is because it’s <em>likely</em> these users are under pressure to enquire or transact and need to beat the clock and the competition coming from other users .</p><p>Towards the other end of the engagement spectrum, you might have a lot of users going through their initial, and sometimes prolonged discovery phase, perhaps day-dreaming about what could be, exploring multiple options and not under any significant time pressure to purchase or transact.</p><p>The challenge for apps that fit this bill, is to ensure that push notifications frequencies are altered to suit the different engagement levels of users at various stages of this spectrum.</p><h3>Approach</h3><p>Of course you could always add a notifications frequency controller within your notifications settings area, however this might not give you the desired network effect, given the awareness of settings areas is usually pretty low. An alternative is to go down the path of automatically altering notification frequencies based on user engagement levels . This type of approach of making your notifcation frequencies a little smarter, will probably have a more wide-spread effect on your user-base.</p><h3>Establishing a baseline</h3><p>First up if it could be interesting to understand the baseline impacts of straight up real-time notifications (assuming your push server and back-end systems allow real-time push), without any rules to alter the frequency depending on users engagement levels. You could achieve this by setting up an A/B test (<a href="https://blog.optimizely.com/2017/01/24/new-mobile-sdks-available-optimizely-x/">Optimizely X</a> works great for native apps) to understand this effect, compared to a control group that continues to receive notifications at your current set frequently (twice daily, this is pretty common).</p><p>Some goals for the test could be;</p><ul><li><strong>Notification volume and conversion rate</strong></li><li><strong>App sessions / app sessions per user</strong></li><li><strong>Customer enquiry</strong></li><li><strong>Notification disables (in-app topic and notification permission levels)</strong></li><li><strong>App uninstalls</strong></li></ul><p>To summarise your baseline test, you should expect to see an increase in push volume, and despite a negative impact to the notification conversion rate, you’ll most likely start to see an immediate increase in sessions (more from the same users). But of course, you’ll probably be brining user into the app, too frequently, so expect to see a negative impact on key engagement metrics like customer enquiry rates, compared to the control group. You may also see a spike in notifications opt-outs and app uninstalls. Don’t be to alarmed here, remember that these impacts should be quarantined within the safety of your experiment. The value in doing this is that it give you important insights to work with and improve.</p><h3>Now to try introducing some smarts</h3><p>Not every mobile developer has access to their own Machine Learning capability to automatically segment users who should receive notifications at different frequencies (maybe <a href="https://firebase.google.com/docs/predictions/">Firebase Predictions</a> could change the game here?). So start with some simple logic to change the real-time frequency back to your set internal. This could be triggered by certain postitive or negative user events . For example, say a user swiped away the notification or they didn’t enter the app after sending a certain amount of notifications. You may find that despite altering notification frequencies based on these fairly simple signals, the impact to your goals could be significant.</p><p>Relative to the raw real-time experiment results, your slightly smarter notifications should improve the notification conversion rate, improve customer engagement metrics and have an even higher potential for sessions growth. This is because you should see less notification opt outs and less app uninstalls, due to the notifcation frequency backing off for the less engaged segment of users, who’d be the highest risk churners. You’d essentially be retaining more of your audience as your notifications would be responding appropriately to their engagement levels, albeit basically.</p><h3>Getting the balance right</h3><p>The long-term goal is to send notifications at a frequency relative to user engagement levels. If you get this right you should be able to maximise sessions growth, whilst not impacting customer enquiry rates, notification opt outs or app uninstalls — you might even be able to reduce the latter metrics and really get the best from both worlds.</p><p>Achieving this would require ongoing commitment and fine-tuning of your rules. Perhaps aim for a method that allows for server-side configuration to minimize your overheads and to prevent being dependant on app versions to deploy updates. The other advantage with this approach would be consistent behaviour for your iOS and Android notifications (if you have apps for both platforms).</p><p>Perhaps the best-case scenario is that the insights gained from your experiments help sell the even greater growth potential that could be realised through an investment in an approach involving more advanced Machine Learning capability.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=79d97fbf6ce2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/bonza-product/smarter-push-notification-frequencies-79d97fbf6ce2">Smarter push notification frequencies</a> was originally published in <a href="https://medium.com/bonza-product">BONZA</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The perils of permissions priming]]></title>
            <link>https://medium.com/bonza-product/the-perils-of-priming-for-permissions-2156ac5cbda1?source=rss-889219a6365------2</link>
            <guid isPermaLink="false">https://medium.com/p/2156ac5cbda1</guid>
            <category><![CDATA[mobile]]></category>
            <category><![CDATA[engagement]]></category>
            <category><![CDATA[retention]]></category>
            <category><![CDATA[apps]]></category>
            <category><![CDATA[ios]]></category>
            <dc:creator><![CDATA[Dave Collins]]></dc:creator>
            <pubDate>Fri, 11 Aug 2017 09:46:56 GMT</pubDate>
            <atom:updated>2017-08-11T15:08:57.619Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Hypothesis:</em> <em>Adding on-boarding priming screens for iOS system permission requests, will increase the rate of users allowing them. This is because the priming screens will educate new users about their benefits, before being presented with the dialogues to do so e.g. Allow push notifications to be alerted about relevant content updates.</em></p><p>Results have revealed that the above assumption is false for some apps. It’s been observed that placing a permission priming screens in front of permission dialogues can impact permission allowing <strong>by up to 100%. </strong>There are a couple of theories as to why this happens.</p><h4><strong>Bypassing permissions</strong></h4><p>Firstly most permission priming screens give users the option of by-passing the actual permission dialogue, like the <a href="https://itunes.apple.com/us/app/domain-real-estate-and-property/id319908646?mt=8">Domain</a> app below, where users can tap ‘Not now’ on and completely bypass each permission dialogue, where not allowing is also an option.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/375/1*4PrO0O0RzOGIJo0742LBfg@2x.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/375/1*qIomVBWk4WBxeiBxGdDR_A@2x.png" /></figure><p>We’re now seeing more apps (particularly more popular ones like the car sharing app <a href="https://itunes.apple.com/us/app/drivenow-carsharing/id435719709?mt=8">DriveNow</a>, below) doing away with permission priming screens. As a result their on-boarding experience is fairly terse, but no doubt optimised towards conversion of the their iOS permissions. One could also presume that popular apps have stronger brand, and therefore trust attributes. This may reduce their need to essentially try and convince new users to allow permissions with primings screens.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*LhCZMbga-2oz7dSkzwK_tQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*c0TTqKa1F_DYsv9WlBYpLw.png" /><figcaption>DriveNow have done away with permission priming screens for their iOS on-boarding experience</figcaption></figure><h4>Contextual placement</h4><p>Another potential factor contributing to the poor performance of placing priming screens in the on-boarding flow, is that they aren&#39;t contextually placed. Some users may not get why allowing permissions is useful, despite your best efforts to explain why. During on-boarding users might be more interested at generally activating in the app e.g. doing their first search or transaction.</p><p>Perhaps a better tactic is to present permissions (for those that are yet to allow them) at the exact point in which they can be useful within the app’s experience. This way certain functionality can serve as the priming e.g. Conducting a current location search as <em>priming</em> to present the iOS location services permission. Allowing the permission in this case is actually needed to complete the task so the conversion should be much higher.</p><h4>In summary</h4><p>When permission priming screens are used in the on-boarding flows, impacts to other important conversions, like account signup and new user activation rates have also been observed.</p><p>So it is worth considering how popular your app is (active user and install base), as the more popular it is, the less likely they are needed. If not already done, considering how to contextually place permissions, to catch users who didn’t allow during on-boarding is also a worthy exercise.</p><p>If your app currently does use permission priming screens it really does pay to understand if they are impacting the rate of users allowing permissions, and other adjacent events within the on-boarding flow. Form my experience the opposite to desired effect can occur which is never good. Knowing where you stand here can be achieved by running an A/B test, with tools like <a href="https://www.optimizely.com/">Optimizely</a>, where the statistical significance of impacts are calculated on the fly.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2156ac5cbda1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/bonza-product/the-perils-of-priming-for-permissions-2156ac5cbda1">The perils of permissions priming</a> was originally published in <a href="https://medium.com/bonza-product">BONZA</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>