<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Priscilla Cheung on Medium]]></title>
        <description><![CDATA[Stories by Priscilla Cheung on Medium]]></description>
        <link>https://medium.com/@prischcheung?source=rss-667cd6aa2b4d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*CPQPOi6xc9nNcngBgbKYQg.jpeg</url>
            <title>Stories by Priscilla Cheung on Medium</title>
            <link>https://medium.com/@prischcheung?source=rss-667cd6aa2b4d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 20:37:41 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@prischcheung/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Experimentation: Backing It Up]]></title>
            <link>https://medium.com/data-science/experimentation-backing-it-up-b7b9aa757b72?source=rss-667cd6aa2b4d------2</link>
            <guid isPermaLink="false">https://medium.com/p/b7b9aa757b72</guid>
            <category><![CDATA[experimentation]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[data-infrastructure]]></category>
            <category><![CDATA[conversion-optimization]]></category>
            <category><![CDATA[data-product-creation]]></category>
            <dc:creator><![CDATA[Priscilla Cheung]]></dc:creator>
            <pubDate>Sun, 31 Oct 2021 13:43:24 GMT</pubDate>
            <atom:updated>2021-10-31T13:43:24.201Z</atom:updated>
            <content:encoded><![CDATA[<h4>Setting up the infrastructure to drive a culture of effective analysis</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*MlCGXBot1ganipP8" /><figcaption>Photo by <a href="https://unsplash.com/@rgaleria?utm_source=medium&amp;utm_medium=referral">Ricardo Gomez Angel</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><em>In a </em><a href="https://prischcheung.medium.com/experimentation-to-test-or-not-to-test-547b10385939"><em>previous post</em></a><em> I outlined the drive for a culture of experimentation and the principles of analysis required to entrench this mindset shift across a business. However, it’s time to shift gears — before propelling continuous learning through experimentation, we need to go back to the very foundations of data capture and transformation.</em></p><p>You would think that as such an integral component of product development, there would be more emphasis on integrating experimentation with data infrastructure. Sure, you get the likes of <a href="https://netflixtechblog.com/its-all-a-bout-testing-the-netflix-experimentation-platform-4e1ca458c15">Netflix</a> and <a href="https://content.linkedin.com/content/dam/engineering/site-assets/pdfs/ABTestingSocialNetwork_share.pdf">LinkedIn</a> leading the way with their own experimentation platforms — but what about the small fish? I want to explore how even smaller companies can overcome the limitations of out-of-the-box experimentation solutions and progress towards building similar in-house platforms.</p><h4><strong>The old buy vs build adage</strong></h4><p>Why is it, that for many applications of prescriptive analysis, we’re happy to engineer data from raw ingestion through to transformation — but for experimentation (arguably the crux of combining descriptive and predictive analytics into intelligent decisioning), we’re content with plug and play solutions? To realise the power of integrating experimentation with internal data infrastructure we need to firstly understand the limitations of the optimisation tools that are readily available.</p><p>Plugging into readily available experimentation solutions such as Google Optimize or Optimizely has typical out of the box advantages: ease of setup, implementation and insight. Consider, however, these limitations:</p><ul><li>Data inputs— out of the box solutions are built for digital experiments on customers’ experience across websites and apps. Once you include any touchpoints with customers that happen offline (for example, applications converted over the phone), then implementation is not as simple. You can upload offline touchpoints as conversions through API configurations, though this starts to take away from the ease of implementation these solutions offer in the first place;</li><li>Data outputs— existing experimentation platforms thrive on the insights they make available, i.e primarily revolving around (and rightly so) experiment analysis. However, querying the experiment data for other forms of advanced analysis and visualisation is only available if you export the event data from the platform.</li></ul><p>Put frankly, with the effort required to set up your experimentation solution with <em>inputs</em> from your other data sources, then to export these data <em>outputs</em> for further analysis — you might as well build experimentation into your own data stack. Not only that — integrating your experiment data with your customer’s data end-to-end doesn’t just benefit your experimentation culture, but enriches your view of your customer overall; enabling more agile analysis and personalisation at scale.</p><h4><strong>Building the experimentation data infrastructure</strong></h4><p>So if we are to foster an iterative, agile environment for effective experimentation culture, we need a framework for integrating an experimentation solution into our data infrastructure. Albeit this framework needs to be considered alongside the backend that powers random assignment, running concurrent experiments, and variant retrieval requests — given this has been more extensively explored, I want to focus on the data-related components in this framework. There are five key components:</p><ul><li>Digital analytics measurement plan— mapping out the key user parameters (e.g acquisition sources: are they new or existing customers? Have they been driven by specific marketing channels?) and event touchpoints that occur across your digital landscape. Make sure you are tracking a user ID that can connect the events for an individual user across multiple sessions;</li><li>End-to-end data blueprint — once your measurement plan is implemented, the next step is to connect your digital landscape with customer touchpoints that occur offline, with a person ID* or the like that can be used as a unique key across your whole <a href="https://www.lucidchart.com/blog/what-is-a-service-blueprint">service blueprint</a>. This will allow you to map out the entire journey for each individual person, enabling a single view of your customer;</li><li>Experiment data model — outlining the data to be collected to understand every experiment and variant a person is exposed to, and whether they converted on both <a href="https://neilpatel.com/blog/secondary-goals/">primary and secondary metrics</a>. Then mapping this to the rest of your data blueprint and automating the ingestion and transformation of this data for analysis;</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EzHAgU5HjunDWfKHbUWstQ.png" /><figcaption>Relationship diagram outlining experiment data to be collected and mapped [Image by author]</figcaption></figure><ul><li>Frontend experimentation interface— developing a tool that analyses statistically significant difference between your variants. Part of this includes surfacing the <a href="https://www.linkedin.com/pulse/making-product-decisions-bayesian-analysis-john-ostrowski/">principles of Bayesian analysis</a> when sample sizes are lower and statistical significance is harder to achieve. Visualising this analysis effectively is also crucial to driving a culture of experimentation, so that anyone is able to understand the results and take action;</li><li>E2E experimentation strategy — as with any other product, data products are not successful unless they are adapted by their end-users. Training teams on how to use the experimentation interface, and developing and maintaining an ideas registry are all part of fostering momentum around cross-functional experimentation.</li></ul><p>Successful experimentation analyses that paves the way for a thriving test and learn culture depends on a framework to integrate your experiment data with the rest of your data infrastructure. Integrating the collection, transformation and analysis of your experiment data with the rest of your customer data ultimately enables experimentation to be engrained into the culture of your organisation, to become pivotal to your product strategy.</p><ul><li>For the purposes of this post, we are dealing with an actual person, i.e with a heartbeat. This could look different if you’re dealing B2B.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b7b9aa757b72" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/experimentation-backing-it-up-b7b9aa757b72">Experimentation: Backing It Up</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Squeezing the Most Out of Segmentation]]></title>
            <link>https://medium.com/data-science/squeezing-the-most-out-of-segmentation-466db9d08573?source=rss-667cd6aa2b4d------2</link>
            <guid isPermaLink="false">https://medium.com/p/466db9d08573</guid>
            <category><![CDATA[product-data]]></category>
            <category><![CDATA[clustering]]></category>
            <category><![CDATA[tableau]]></category>
            <category><![CDATA[segmentation-analysis]]></category>
            <category><![CDATA[data-analysis]]></category>
            <dc:creator><![CDATA[Priscilla Cheung]]></dc:creator>
            <pubDate>Mon, 26 Jul 2021 12:57:57 GMT</pubDate>
            <atom:updated>2021-07-27T13:03:28.234Z</atom:updated>
            <content:encoded><![CDATA[<h4>Is there any point to this analysis — or will we remain divided?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ce4Cd8bXmqo3kcgQ" /><figcaption>Photo by <a href="https://unsplash.com/@thepaintedsquare?utm_source=medium&amp;utm_medium=referral">Jessica Lewis</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><em>Every time someone asks to segment our customers, then proceeds to offer predetermined attributes to group them by, I immediately think — do you just want me to filter the data? Is this segmentation at its finest, or simply audience creation?</em></p><p>If we refer to the definition of segmentation — that is, to collate individuals based on their similarities <em>within</em> groups, and distinctness <em>between</em> groups — then the above requests seem perfectly reasonable. Why then, do they leave such a disappointing aftertaste?</p><p>My dissatisfaction with this type of segmentation stems from the richness that, in fact, segmentation has to offer. There is so much to be extracted (double pulp!) from segmenting customers beyond their demographic attributes, or descriptive characteristics of the like. Take myself as an example — as a 30-something female, I could not care less about hair products (my version of a good hair day is when I wash my hair). If you were to assign me to a segment based purely on demographics, you’d be swiftly saying goodbye to your aerodynamic, Airwrap ad spend.</p><p>The beauty, the richness of segmentation, lies in two points of view that don’t just warrant segmentation itself, but segmentation <em>analysis</em>. Firstly, how we view the funnel — the consideration of individuals’ <em>behavioural</em> tendencies pre and post conversion, to enhance how we group them. Secondly, how we view scalability — integrating segmentation analysis as an ongoing measure of marketing and product initiatives. It’s this mindset to clustering individuals into groups that allows for rich, (somewhat ironically) personalised user experiences.</p><h4><strong>A Funnel View: Far from the shallow…</strong></h4><p>Whilst the amount of behavioural, first-party data you can collect for segmentation analysis is often dependent on how far down the conversion funnel individuals are, it is certainly not limited to only once they become customers. In fact there is behavioural data that can enhance segmentation in both directions of the funnel, pre and post conversion.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lXFVEAzKX0FU_jsL7fxWXw.png" /><figcaption>Behavioural data is not limited to the revenue stage of the conversion (AARRR) funnel [Image by author]</figcaption></figure><p>If you look towards the upper funnel there are behavioural attributes you can employ starting at the awareness stage. Prospective customer behaviour can be unearthed through the differences in the way website users search information, consume content, and make comparisons. These behaviours indicate users’ propensity to move further through the funnel, consequently more effective strategies to be adopted than those through the segments that using only demographic characteristics enable.</p><p>Traversing further through the funnel, you’ll then find valuable behavioural attributes beyond the revenue (or transaction) stage. Utilisation behaviour delves past customers’ behaviour at their point-of-purchase — revealing ongoing tendencies that highlight a truer product-market fit, a stronger tell of their value. As a simple example, Customer A who purchases a $100 product but only uses 50% of it may be less valuable than Customer B, who purchases a product valued at $60 but utilises 100%. These behaviours, along with referral behaviours such as promoter reviews, divulge opportunities for optimising customers’ lifetime value; more than their demographic or even transactional attributes disseminate about customers’ fit for product.</p><h4><strong>A Scalable View: Running lean v scaling up</strong></h4><p>By now I’ve hopefully sold you on the richness of segmentation through adopting a full-funnel view. How then, can you cash in on this approach? Good news is that the benefits are both immediate and long term — from gaining quick insights, through to building out segmentation as a scalable data product using your customer database. Moreover — segmenting individuals (beyond deciding arbitrarily how to group individual data points) through cluster analysis applies statistical rigour that can be both lean and scaled.</p><p>Let’s start with the quick insights — I’ve been able to tinker more recently with Tableau’s relatively lean yet insightful clustering <a href="https://help.tableau.com/current/pro/desktop/en-us/clustering.htm">solution</a>. Its feature offering is great for when you want both a straightforward and robust approach, as it enables:</p><ul><li>Automated scaling — straightforward but much needed for normalising input variables of various scales;</li><li>Automated optimal number of clusters — using k-means to delineate groups for the given data points. Instead of having to manually chart out the optimal number of clusters with the <a href="https://www.scikit-yb.org/en/latest/api/cluster/elbow.html">elbow method</a> or similar means, Tableau employs the <a href="https://towardsdatascience.com/cluster-analysis-in-tableau-1f19acd0c647">Calinski-Harabasz index</a> to automatically output <em>k </em>clusters. Alternatively, you can customise your desired number of clusters (less preferable but available if you can’t serve as many different segments based on budget limitations);</li><li>Exploratory analysis — being able to quickly visualise the statistically-based clusters across multiple axes (even geographically, if available in your dataset).</li></ul><p>When it comes to <em>scaling</em> segmentation as a data product, this is where Tableau hits its limitations as a tool primarily for visualisation. Although there is more upfront effort, you’re better off using a combination of implementing statistical analysis in <a href="https://www.rdocumentation.org/packages/stats/versions/3.6.2/topics/kmeans">R</a> or <a href="https://realpython.com/k-means-clustering-python/#how-to-perform-k-means-clustering-in-python">Python</a> (there are various packages available for clustering), and automating it to run through Airflow (or the like) on your customer base instead. This approach is far more scalable compared to an out-of-box solution like Tableau, through both customising and automating:</p><ul><li>Data cleaning and transformation — coding code categorical variables into dummy variables, and recoding null values into mean or median values. Tableau’s exclusion of data points with null values from its clustering (simply assigned ‘Not clustered’) can skew your segmentation, so recoding data points with null values is more ideal;</li><li>Variable selection — identifying variables to be used in cluster analysis. Although Tableau offers an intuitive drag and drop function to interchange between the variables used for clustering, this doesn’t tell you <em>which</em> variables should be used. Rather than arbitrarily determining which variables to use for clustering, you can incorporate feature importance analysis prior to your segmentation analysis to determine input variables;</li><li>Cluster assignment — allocating clusters to new data points based on the cluster centre closest to each point. In Tableau, cluster assignment on new data is manual — by physically clicking the ‘Refit’ option on your clusters (saved as groups) in the Dimensions pane. If you built a dashboard with your cluster analysis (as I did), you wouldn’t be able to automate this refitting — even if the data source is scheduled for refresh. Cluster automation can also include scheduling new cluster analysis based on new data points and updates to their behaviour. This can be useful, for instance, when you introduce a new product to market; but proceed with caution — because you also want to be able to benchmark and compare the trending movement of clusters as you implement new initiatives. Running new cluster analysis on a daily basis won’t allow you to make a grounded, historical comparison (don’t forget the set.seed!).</li></ul><p>Let’s dispel the notion that segmentation is merely grouping customers by demographic or descriptive variables. In fact, if used with a full-funnel and build-for-scale mindset — then segmentation provides a much richer understanding of our customers, ultimately powering a sustainable, personalised experience.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=466db9d08573" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/squeezing-the-most-out-of-segmentation-466db9d08573">Squeezing the Most Out of Segmentation</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Rules of (Content) Engagement]]></title>
            <link>https://prischcheung.medium.com/the-rules-of-content-engagement-5f958fa23236?source=rss-667cd6aa2b4d------2</link>
            <guid isPermaLink="false">https://medium.com/p/5f958fa23236</guid>
            <category><![CDATA[vanity-metrics]]></category>
            <category><![CDATA[content-marketing]]></category>
            <category><![CDATA[data-driven-marketing]]></category>
            <category><![CDATA[web-analytics]]></category>
            <category><![CDATA[digital-analytics]]></category>
            <dc:creator><![CDATA[Priscilla Cheung]]></dc:creator>
            <pubDate>Mon, 31 May 2021 06:28:08 GMT</pubDate>
            <atom:updated>2021-05-31T06:30:03.463Z</atom:updated>
            <content:encoded><![CDATA[<h4>Is all vanity with analytics for content marketing?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ulyObCl0kTjLwxBr.jpg" /><figcaption>Image by Tom Fishburne on <a href="https://marketoonist.com/">Marketoonist</a></figcaption></figure><p><em>When it comes to evaluating the performance of content marketing, we often find vanity metrics such as followers, reach and pageviews being used. Heck, even Medium uses stats such as fans and read ratio to measure engagement.</em></p><p>True to their namesake vanity metrics hype up performance, but don’t actually drive business growth. With content engagement these metrics become even more prevalent, as content marketing often isn’t linked to hard metrics such as revenue. How then, asks the analyst, do we spend less time on metrics that make a business <em>temporarily</em> look good, and rather — set measures of success within the realms of content marketing that grow a business long term?</p><p>Let me count thy ways.</p><h4>1. Action-biased measures</h4><p>Meaningful success metrics stem from a genuine understanding of what actions will be taken as a result of their measurement. It comes from probing: what does success look like? Will knowing how many followers you have drive this success? More explicitly — what actions will you take knowing how many followers you have?</p><p>In fact, a tried and true approach I’ve used to derive actionable, <a href="https://review.firstround.com/im-sorry-but-those-are-vanity-metrics">clarity metrics</a> from content marketers is an adaptation of the five interrogating whys used in root cause analysis.</p><p><em>Why</em> do you want someone to read your blog?</p><p>— To learn more about small businesses.</p><p><em>Why</em> do you want your readers to learn more about small businesses?</p><p>— To support them with their own small businesses.</p><p><em>Why</em> would you to support your readers with their small businesses?</p><p>— So they trust us to support them with more than what we post on our blog.</p><p><em>Why</em> do you want your readers to trust you?</p><p>— So that they return to our website.</p><p><em>Why</em> do you want readers to return to your website?</p><p>— To ultimately win them over as customers.</p><p><em>Bingo.</em></p><p>Eventually, you’ll unearth the point of value your marketers are trying to create. In this case, to win over more customers — worthwhile success metrics include the % users who return to the site after engaging with the blog, and subsequently become customers (hello, attribution!)</p><p>Contrary to popular belief, content marketers shouldn’t be exclusively concerned with driving upper funnel metrics such as followers or pageviews. Understanding how their influence extends further down the funnel delivers measures that bias towards action, and spurs longer term impact.</p><h4>2. Multifaceted measures</h4><p>Metrics used in isolation can also prove inconsequential at driving growth and success. With content engagement, oft-used metrics such as scroll depth and time on page are futile when observed individually — what difference does it make if someone has scrolled 100% down the page, if they haven’t actually consumed the content?</p><p>However — by their powers combined, these metrics can be valuable. Simo Ahava outlines in great detail how to implement a combined metric of <a href="https://www.simoahava.com/analytics/fire-tag-upon-scroll-depth-and-time-spent/">scroll depth and time spent</a> as a better measure of engagement. Rather than adopting his example of 50% scroll depth and 30 seconds spent, I’ve found analysing your users’ engagement first through both measures allows them to be more effective to your specific use case.</p><p>In our case, a simple cross-tab demonstrates that users spend over 2 minutes on a post, scrolling through at least 50%. As a result we opted to combine a longer amount of time than the suggested 30 seconds with 50% scroll depth before determining a user as successfully engaged.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XVAUbJkwr92M4fTywLJQGg.png" /><figcaption>% Posts by Scroll Depth x Time Spent (sec)</figcaption></figure><p>We can see therefore that measuring engagement with multiple metrics paints a better picture of behaviour than when each are used individually. Besides the combination of scroll depth and time spent, other multifaceted metrics can further depict key components of content engagement:</p><ul><li>Relevance: search queries + articles read</li><li>Advocacy: fans + shares</li></ul><p>There’s bound to be many more…let me know if you have any in the comments!</p><p>Let’s stop using vanity metrics to suit pre-determined theories of success — these ultimately don’t propel a business to success. Rather, let’s question: will this metric inform a decision on how to grow the business, or to be more efficient? Measurement of engagement (and beyond) doesn’t have to be in vain!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5f958fa23236" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Are Two Better Than One?]]></title>
            <link>https://prischcheung.medium.com/are-two-better-than-one-a0a91e43cff6?source=rss-667cd6aa2b4d------2</link>
            <guid isPermaLink="false">https://medium.com/p/a0a91e43cff6</guid>
            <category><![CDATA[digital-analytics]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[data-analytics-tools]]></category>
            <category><![CDATA[analytics-platforms]]></category>
            <dc:creator><![CDATA[Priscilla Cheung]]></dc:creator>
            <pubDate>Mon, 22 Mar 2021 21:12:53 GMT</pubDate>
            <atom:updated>2021-07-12T01:54:59.676Z</atom:updated>
            <content:encoded><![CDATA[<h4>Pros and cons of having multiple digital analytics platforms</h4><p><em>We’ve all been there: starting a new job, ready to get stuck into the data. You begin to get the lie of the land when, hold up — where is this data the other teams are using?</em></p><p><em>Oh. It’s in an entirely separate digital analytics platform.</em></p><p>You might as well cue the music (think Eye of the Tiger: duh, duh duhduh) — because it’s about to go down. Ding ding ding, it’s a battle of the digital analytics platforms; and the question ringing at the back of everyone’s mind: which will be the reigning champion?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*27ItxYzEiQ25FFFu" /><figcaption>Photo by <a href="https://unsplash.com/@hypefilms?utm_source=medium&amp;utm_medium=referral">David Guliciuc</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Survival of the Fittest…or Not</h3><p>Once you step into the ring, though, an alternative question rises to the fore — is having multiple digital analytics platforms, in fact, more beneficial than relying on a sole platform? Is it better, rather, to separate than consolidate?</p><p>In one corner, we have the pros:</p><ol><li><strong>Metric comparison</strong>: collecting data across multiple platforms allows for similar events and parameters to be compared across each. It’s naive to presume that any single platform offers perfect measurement, so comparing nuances across platforms is valuable. I found this to be the case particularly when defining the simplest metrics — take sessions, for example: when <em>should</em> they expire?</li><li><strong>Case-specific application</strong>: despite adamant, sales-driven claims to do it all, we all know that platforms are built for specific use cases, and thus better suited to those needs. Google Analytics is renowned for its capabilities to deliver insights on marketing performance, whereas the likes of Heap and Amplitude are more intuitive for product insights.</li><li><strong>Data backup</strong>: in case (gasp!) your tracking ever breaks, or events are not defined prior to deployment. Products such as Heap that offer auto-capture allow for events to be defined retrospectively, often advantageous to a build-fast, learn-fast approach.</li><li><strong>Vendor agnosticism</strong>: with the idea again that no individual platform is perfect, and given the revolving door of updates to digital data collection and privacy standards — it can be immensely valuable not putting all your eggs into the same vendor basket (ecosystem).</li></ol><p>In the opposing corner, we have the cons:</p><ol><li><strong>Metric comparison</strong>: the flip side of having the same metrics reported across multiple platforms is that this can lead to metric confusion; where stakeholders are paralysed by the variation reported between the platforms, rather than the insights they provide.</li><li><strong>Use case ambiguity</strong>: having multiple platforms can generate further confusion as to when best to use each, especially if the same metrics are tracked across both. Often, this leads to stakeholders selecting a platform based on experience, ease of use, or worse — whichever provides a better view of their performance.</li><li><strong>Cost</strong>: put simply: the more you have, the more it’ll cost. I’ll say it here loud and clear — <em>no</em> platform is free (especially<em> </em>open source software). Upfront costs aside, there’s also the underlying time and resources required of your tech and analytics teams to maintain data quality and accuracy. On top of this, prioritisation of tracking implementation and reporting management across platforms presents its own challenges.</li><li><strong>Skill specificity</strong>: there will always be a number of skill requirements unique to each platform, albeit some are transferrable across the board. Of course these can be learnt, though keeping up to date with best practices across multiple platforms is time intensive. Adobe Analytics implementation requires greater expertise in technical infrastructure, which differs from the expertise afforded by Google Analytic’s relative ease of implementation. Though now in the midst of migrating our analytics configurations to Google Analytics 4 properties, I might end up eating my words!</li></ol><h3>One Size Does Not Fit All</h3><p>If you were hoping to reach this point with a definitive answer as to whether having single or multiple digital analytics platforms is better, the bad news is: there is no <em>one</em> solution that fits all. The good news: let me share an evaluation process to help you effectively weigh up what will work best for you.</p><h4>It’s a Numbers Game</h4><ol><li>Size matters: evaluate the size of your company before selecting your platform(s). Larger companies may benefit from varying platforms for specific use cases, and better resourced to implement and maintain multiple platforms effectively.</li><li>Three’s a crowd: there’s rarely been an instance when I’ve found the benefit of having more than two platforms to outweigh the costs. If you must, three platforms is the absolute limit — though I would be sure to be clear on the benefits of implementing that extra platform.</li></ol><h4>If the Shoe Fits</h4><ol><li>Be specific: ensure that the uses cases for each platform are clear, and set these objectives as part of their ongoing assessment. If you find a platform is no longer providing any added benefit, it’s <em>okay</em> to cut your losses.</li><li>Assess for the best: when choosing the optimal digital analytics platforms for your needs, there are several key capabilities I compare on:</li></ol><ul><li><strong>Analytics intelligence</strong>: how does each platform allow you to gain insights into your specific use cases? In our instance, the key use cases were to understand marketing performance and attribution, as well as user behaviour and product experience. For this reason, we found Google Analytics and FullStory to be complementary as they each had strengths in meeting our respective needs.</li><li><strong>Data democratisation</strong>: are data and insights made available and actionable by both technical and non-technical users? The key here is how intuitive the platform is, as this equips users to self-serve, as well as easily share their findings.</li><li><strong>Data governance</strong>: is data accessibility, quality and privacy built into the way the platform collects and presents data? Though often overlooked for its implicit value, platforms that prioritise data governance are more effective at driving a source of truth in data across business units.</li><li><strong>Data activation</strong>: how well does each platform integrate with your current stack? There is little value in implementing a new platform if it does not connect into the rest of your data ecosystem; including offline behaviour, marketing and communications, and experimentation considerations.</li></ul><p>Regardless of whether you go down the path of having single, vs multiple, digital analytics platforms, the solutions you opt for will only be as good as their data quality. Processes that are set up to maintain stability in tracking, including the continuous monitoring for anomalies, will ensure ongoing quality in data.</p><p>Deciding on digital analytics platforms is also more than just a plug-and-play effort: the value each adds should be assessed on their suitability for long term scalability and efficiency. Beyond just their initial assessment, these platforms should be reviewed on an ongoing basis as to whether each remains the most suitable for their required use cases.</p><p>Ultimately, the success of your digital analytics platforms of choice will come down to their adoption and ongoing usage. Bringing your stakeholders (the key data consumers) onboard the process of analytics implementation is not overkill, it is essential. Data needs to be trusted. It should consolidate and integrate your understanding of users’ journeys — these platforms are simply just the tools that enable you to do so.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a0a91e43cff6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Experimentation: To Test or Not To Test?]]></title>
            <link>https://prischcheung.medium.com/experimentation-to-test-or-not-to-test-547b10385939?source=rss-667cd6aa2b4d------2</link>
            <guid isPermaLink="false">https://medium.com/p/547b10385939</guid>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[product-analytics]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[conversion-rate-optimizer]]></category>
            <category><![CDATA[experiment]]></category>
            <dc:creator><![CDATA[Priscilla Cheung]]></dc:creator>
            <pubDate>Mon, 25 Jan 2021 05:01:43 GMT</pubDate>
            <atom:updated>2021-03-22T06:30:28.826Z</atom:updated>
            <content:encoded><![CDATA[<h4>A tale of two priorities</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7Hjcb-XRl6bQk9co" /><figcaption>Photo by <a href="https://unsplash.com/@lox?utm_source=medium&amp;utm_medium=referral">Lachlan Donald</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><em>In recent years I’ve experienced marketing and product teams hungrier than ever to test their initiatives with experimentation — an analyst’s dream, right?</em></p><p>You would expect any analyst’s nirvana to be their stakeholders’ increasing appetite for being data-driven. In reality, this often leads to a crossroads between two conflicting priorities: driving a genuine test and learn culture, versus being asked to test everything under the blazing sun.</p><p>When it comes to driving true experimentation culture across a business, the key is to discern what is actually worth testing. You need to <em>move</em> the needle, not <em>find</em> the needle. That is, experimenting on marketing and product developments that drive high impact, rather than arbitrarily finding a needle in the haystack of ideas to test.</p><p>A common example of this is found in the practice of conversion rate optimisation (CRO). Marketers will want to test whether varying images, or the colour of a call-to-action button, will increase the conversion rate. More often than not, these are “find the needle” tests: there is no statistically significant difference between the variations in optimising conversions. What’s worse, these tests consume a great deal of resources— squandering valuable time and effort from your engineering and design teams, even analysts — for no added value.</p><p>To combat this, I’ve adapted two common analysis principles specifically for experimentation to optimise effectiveness in testing.</p><h3>Analyses Before Hypotheses</h3><p>Whilst the <a href="http://www.moekiss.com/2016/09/analysis-competing-hypotheses-digital-analytics/">analysis of competing hypotheses</a> (ACH) methodology is structured to reduce bias inherent even in a seasoned analyst’s critical thinking — one of the main critiques of the method is the rigour given to the initial hypotheses generation. If we are to be exhaustive in generating hypotheses (thereby limiting biases), there needs to be exploratory analysis even prior to their proposal.</p><blockquote>Experiments should be founded on analyses, with recommendations for tests spearheaded by data-led hypotheses that move the needle to a notable difference.</blockquote><p>Answering what would improve the quality of our online leads was initially met with several hypotheses — marketing channels, landing pages and content all anticipated to be drivers of lead quality.</p><p>Through exploratory analysis, I discovered leads who gave their contact details immediately before undergoing multiple steps of a form tended to be lower quality leads than those who completed all the steps. Leads like Bradley, who were hoping to start a business, happily complied with providing their details at the beginning of a form to source further information; whereas quality leads like Jane who urgently needed a loan to expand her business — were more willing to go through the whole form before submitting their details.</p><p>This analysis rendered an additional hypothesis to those initially ideated: that <em>where</em> in a form customers submitted their details (at the beginning versus the end of a form) was a driver of lead quality. The ensuing experiment resulted in a lower volume of leads when customers submitted their details at the end, but statistically higher lead quality.</p><p>Since the generation of hypotheses was founded on analyses, we were able to extend the hypotheses beyond the limitations of our existing perspectives and collective experience. We were then able to test for, and subsequently impact, behaviour that moved the needle and drove notable growth for the business.</p><h3>Analysis Before Paralysis</h3><p>Experimentation also inevitably warrants instances of when there is <em>no statistical difference</em> — with this, you will be at a crossroads of knowing whether there ever <em>will</em> be, or <em>should</em> be, a clear winner.</p><blockquote>As analysts, you are often required to make decisions on what success looks like. Taking the stance of “moving the needle” means that you need to overcome analysis paralysis by being simultaneously data-driven and agile when determining the success of experiments.</blockquote><p>We had been testing two alternate forms on our website for six weeks with no statistically significant difference between the variations. Our wait for a winning variation was beginning to impede the implementation of other experiments. Rather than let this paralyse our progress, I advocated for the experiment to be discontinued — and to keep the variation with the simpler flow.</p><p>The basis of this recommendation was by no means driven without being informed. In this case, no statistical difference indicated that there was not enough to evidence that either form was worse off than the other. With enough data, equivalence testing can confirm the null hypothesis <a href="https://www.integral-concepts.com/wp-content/media/What-is-Equivalence-Testing-and-When-Should-We-Use-It.pdf">beyond reasonable doubt</a> — that there is no worthwhile difference between variations. In the absence of this at six weeks, I drove the recommendation to go ahead without a winning variation by using a significance estimator (built as part of our experimentation tracker*).</p><p>Using an automated ingestion of the experiment data, the significance estimator gauges the additional duration required for statistical difference between variations in an experiment. The estimation is on the grounds that all other inputs to the experiment remain constant (in this case the resulting conversion rates and sample sizes from the website traffic). Given the additional amount of weeks estimated, this gave way to moving on from the experiment without statistical difference.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-FNfyRcP49Y4lFp0XGaorw.png" /><figcaption>Experiment Significance Estimator</figcaption></figure><p>Being agile in experimentation means letting the paralysis of needing “more time” or “more data” step aside. I’ve been guilty of wanting more time to perfect my analyses — but you are not always afforded this luxury of time. A thriving experimentation culture requires making timely decisions to continuously test and learn; yet this does not have to come at the expense of informed analyses. The right analysis can in fact support effective experimentation of feature development, with conversion rate optimisation at the core.</p><p>So — to test or not to test? Whether to pursue a culture of experimentation as an analytical mindset across your teams involves, in its own (Nolan’s) Inception-esque way — analysis.</p><p>’Tis <em>not</em> nobler in the mind to suffer from running experiments for the sake of doing experimentation.</p><p>‘Tis nobler to take arms with analyses — proactive analysis that will inform your hypotheses, as well as pre-emptive analysis to determine the success of your experiments. With this approach in mind, you may well make it to nirvana.</p><p>*Stay tuned — this experimentation tracker deserves its own post!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=547b10385939" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[2020 Hind-Vision: Changes I Didn’t See Coming]]></title>
            <link>https://prischcheung.medium.com/2020-hind-vision-changes-i-didnt-see-coming-3285b263ac91?source=rss-667cd6aa2b4d------2</link>
            <guid isPermaLink="false">https://medium.com/p/3285b263ac91</guid>
            <category><![CDATA[data-analyst]]></category>
            <category><![CDATA[social-media]]></category>
            <category><![CDATA[society]]></category>
            <category><![CDATA[work-life-balance]]></category>
            <category><![CDATA[2020]]></category>
            <dc:creator><![CDATA[Priscilla Cheung]]></dc:creator>
            <pubDate>Wed, 30 Dec 2020 23:12:19 GMT</pubDate>
            <atom:updated>2021-07-26T04:38:09.907Z</atom:updated>
            <content:encoded><![CDATA[<h4>What I don’t actually want to leave behind.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*yiYX4caJCrUyiI-6" /><figcaption>Photo by <a href="https://unsplash.com/@joshcala?utm_source=medium&amp;utm_medium=referral">Josh Calabrese</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><em>As a data analyst and someone who thrives better with some semblance of a plan, predictability is high on my radar. 2020 took any ounce of that predictability and kicked it to the curb. To the gutter, rather.</em></p><p>With the year drawing to a close, many would prefer to cancel 2020 altogether like it didn’t happen. I don’t discount for a moment anything that anyone has gone through this year they’d rather forget (health, loss of income and strained relationships were some of the things I grappled with). These have been times so devastating and unrelenting you couldn’t have planned for it if you tried.</p><p>Yet from all this — wouldn’t the year (the sweat and tears) — be wasted if we just “leave 2020 behind”, and not look back in hindsight to recognise how 2020 has changed us?</p><p>These are the changes I won’t be leaving behind.</p><h3><strong>Sanity through structure</strong></h3><p>When I took on my most recent job, I was able to work four days a week — keeping Mondays free to prioritise my wellbeing and pursue creative interests (it’s rather hard to combine my pursuit of tablescaping with my love of data). What I didn’t anticipate, is that it would be <em>these</em> Mondays that would help me adjust when we, like a lot of companies, shifted to working from home.</p><p>Being a fintech, our company transitioned quickly to working from home by mid-March. Working completely remote for many meant that the structure of 9 to 5 changed drastically. It allowed for numerous perks of flexibility (hello, midweek laundry); though also obscured the lines between work and home life. Consistent productivity became a challenge.</p><p>My Mondays off could easily have spiralled into an extension of doing what I enjoy on weekends — a whole lot of <em>nothing</em>. However, when I was first afforded this luxury, I committed to making the most of these days and not wasting them away to extra sleep. I established structure for myself; getting out of bed the same time as any other working day, reading for an hour with a coffee in hand, before taking on the day’s goals I set the previous night. This did not mean that I didn’t allow for flexibility — being kind to myself when I <em>did</em> need to catch up on rest, and shifting plans to help out friends when they needed.</p><p>When we transitioned to working remotely, I kept this structure. Waking up the same time as I would have if traveling to work, spending the extra time reading. Breaking up the end of an arduous day of coding with exercise, mentally unblurring the lines between work and home. In a year where our ways of working were entirely disrupted, maintaining structure whilst working at home kept me productive, and more importantly — my sanity in check.</p><h3><strong>An appetite for small wins</strong></h3><p>2020 was meant to be a year of travel for me; including a three week escape to the rolling hills of Tuscany, lazy summer days and delicious chianti no doubt involved. In anticipation, I started using <a href="https://www.duolingo.com/">Duolingo</a> to pick up a few Italian phrases. It became increasingly apparent that with COVID-19 I wasn’t going to make it to Italy, but I decided to keep up the Duolingo — attributing my newfound addiction to the app’s gamification experience (I’m a sucker for positive reinforcement). They say you peak at learning new languages by the time you reach 10 years of age (of which I’m well past) — but at a 287 day streak, I’ve impressed nobody more than myself. Besides the language, I also had other Italian-themed wins — perfecting the ultimate pizza base, and how to make a proper caffè e latte (latte art still pending).</p><p>My non-Italian endeavours were more short-lived — in my attempts to pick up my guitar again, I only managed to extend my repertoire by a few extra chords. I don’t see this as a failure though. However brief my tinkering, those extra chords were worth it — not only for the additional songs I’m now able to play (four chord progression for the win), but for rekindling my appetite in the small wins, and the joys they’ve afforded.</p><h3>Essential giving</h3><p>As <a href="https://www.abc.net.au/news/2020-04-07/coronavirus-health-modelling-scott-morrison-national-cabinet/12128042">Scotty started putting the call out for Australians to stay home</a> early in the year, for some reason (still inexplicable to me) toilet paper became hot property. Out of sheer laziness (rather than panic) and from the thrifty genes I’ve inherited, I’ve always made an annual bulk purchase from <a href="https://au.whogivesacrap.org/">Who Gives a Crap</a> — so this year, I could have been <em>queen</em> of the toilet paper economy. Rather than keep my stash as I usually do at my parents’ place (my old room has now become a free storage facility for everything that doesn’t fit into my millennial-sized apartment), I got to experience the same rush that Oprah must have had during her giveaways — rolling out however much of the 480 (I did say <em>bulk</em>) precious, pre-purchased TP my friends needed.</p><p>I was also on the receiving end of others’ generosity, at times I needed it the most. Most recently I had to self-isolate, not knowing initially whether this would extend through to Christmas and beyond. During this time I had countless people offer contactless drop offs of necessities and non-necessities (though you could argue wine <em>is</em> essential). I ended up eating better than I usually do when I shop for the house (thanks for the mangoes and falafel, Lorraine).</p><p>More valuable a commodity than toilet paper this year has been the gift of time. In a year where connecting with others became paramount, the few minutes it took to reach out to someone were low in effort but high in value. I was able to connect with others in this season where distance otherwise wouldn’t have enabled, but made possible with us all being stuck at home and masters of Zoom. With so much taken away from us this year, time was something I was still able to give.</p><p>There’s no denying this year was hard. Despite every marketing mantra permeating our social feeds right now, we can’t wrap up the year neatly with a bow and simply move on. We won’t miraculously be going back to “normal” when the clock ticks over into 2021.</p><p>But to recognise how we’ve adapted, to look back on the changes that have helped us survive — that’s the most we can give ourselves to help us through whatever’s around the corner.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3285b263ac91" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>