<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by António Lima on Medium]]></title>
        <description><![CDATA[Stories by António Lima on Medium]]></description>
        <link>https://medium.com/@limata?source=rss-94730e4c6f24------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*frhi-ZAF0hjYO9j2FnuLTw@2x.jpeg</url>
            <title>Stories by António Lima on Medium</title>
            <link>https://medium.com/@limata?source=rss-94730e4c6f24------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 03:15:46 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@limata/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Marketing Attribution Modelling at Cazoo]]></title>
            <link>https://medium.com/cazoo/marketing-attribution-modelling-at-cazoo-ef3a31ceb591?source=rss-94730e4c6f24------2</link>
            <guid isPermaLink="false">https://medium.com/p/ef3a31ceb591</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[growth-hacking]]></category>
            <category><![CDATA[digital-marketing]]></category>
            <category><![CDATA[ecommerce]]></category>
            <category><![CDATA[analytics]]></category>
            <dc:creator><![CDATA[António Lima]]></dc:creator>
            <pubDate>Wed, 15 Dec 2021 15:19:25 GMT</pubDate>
            <atom:updated>2021-12-15T15:19:25.053Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k7DSnwst3eAnwot2wUuxWQ.jpeg" /></figure><h4>Why do we need an attribution model</h4><p>Marketing Attribution Modelling is a hot topic — no doubt about it. A quick search on Google returns 7 ads from technology providers to marketing agencies and content generators. These companies are paying for ads because they know there’s enough appetite from marketers, data professionals and senior leadership to successfully implement an attribution model for their companies. Why is that though? These companies are spending, thousands, hundreds of thousands and sometimes millions of pounds in advertising and they see attribution (<em>I’ll start to use attribution to refer to marketing attribution modeling from now on</em>) as a way to:</p><ul><li>Understand their ROAS (Return On Ad Spend) and prove they are making profitable and sensible investments with their marketing budget;</li><li>Inform how they can optimise their marketing spend by shifting ad spend from the least performing channels and campaigns into the better performing ones.</li></ul><h4>How does attribution work</h4><p>Attribution is the process of distributing the credit for sales/revenues/leads (<em>I’ll use conversions from now on to refer to all of these</em>) amongst the different channels that customer interacts with from the start of their journey through to when they convert.</p><p>An attribution model will distribute no more and no less than 100% of total conversions across all of the channels. This is key and one of the main reasons why one shouldn’t take attribution within a single advertising platform at face value. As multiple advertising platforms will claim credit for the same conversions you’ll soon see that the total conversions attributed will be more than 100% of your total conversions.</p><p>Attribution is also a deterministic process in the sense that it assumes that all of a customer’s touchpoints explain all of the reasons that led them to eventually convert — it assumes causation equals correlation (later in the article I’ll explain some of the pitfalls that might come with this and some complementary methods and techniques to attribution but for now we’ll accept this).</p><p><em>*Note: Whilst touchpoints are mostly clicks to website/app, they can also be other interactions like calls, emails, in-person visits, impressions, etc.)</em></p><p><strong>Now, how do we distribute those conversions amongst the different channels?</strong></p><p>Here’s where we introduce different types of attribution modelling.</p><p>At Cazoo we use first touch, last touch, linear and position based attribution. There are others including time-decay, markov chains, algorithmic, etc. but I will only dive into the ones we use here at Cazoo:</p><ul><li>First touch gives 100% of credit to the channel responsible for the customer first touchpoint.</li><li>Last touch gives 100% to the last touchpoint.</li><li>Linear attribution distributes credit equally amongst all touchpoints.</li><li>Position-based attribution gives 40% of the credit to the first touchpoint, 40% to the last touchpoint and 20% to all touchpoints in between.</li></ul><figure><img alt="Marketing Attribution Models" src="https://cdn-images-1.medium.com/max/733/1*PjrEedeFvk89ar2M0A9RWQ.png" /><figcaption>Cazoo’s Attribution Models</figcaption></figure><p>In addition to these 4 models in their purest form we created variations to specifically meet the Cazoo needs — we sell cars (usually anyone’s 2nd biggest purchase after buying a house) so we have longer customer journeys than most businesses. For that reason we have modified these models in two ways.</p><ul><li>Lead generation based:</li></ul><p>In the <em>‘to lead</em>’ attribution we only distribute credit to the channels that played a part in turning a customer into a lead (we generally consider a lead someone we have contact details for). In these <em>to-lead</em> variations everything that happens between us getting the contact details and the user placing an order is disregarded from an attribution perspective as the marketing objective of generating a lead has been met.</p><ul><li>Non-direct touchpoints based:</li></ul><p>In the ‘non-direct’ attribution direct touchpoints are not given attribution credit. This is because we have long customer journeys so a customer will visit us directly multiple times before converting. And if we attributed credit to the direct touchpoints that would steer considerable credit away from the marketing channels. Often marketing channels are still responsible for driving direct interactions, they just happen in different follow-up sessions. The only exception to still give credit to direct touchpoints in this variation would be when Direct is your first touchpoint as in such a case it denotes an absence of marketing channel responsible for your customer journey.</p><p>This means we end up with a total of 13 attribution models: First touch is always first touch regardless, and then 4 variations (<em>“to order”</em>, <em>“to lead”</em>, <em>“to order non direct”</em>, <em>“to lead non direct”</em>) for last touch, linear and position based.</p><p>I like to think of <em>position-based to lead non-direct</em> as a good candidate for a company default model, but the truth is we build it in a way we can be flexible about it and flick between attribution models as one wishes. Let’s actually take a look into a real-life example at Cazoo (PII protected obviously):</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5bf82f4bb93f5e27c13aa3ee8b392be7/href">https://medium.com/media/5bf82f4bb93f5e27c13aa3ee8b392be7/href</a></iframe><pre><em>Touchpoint is Lead = True</em> means the customer became a lead on its 2nd touchpoint (after clicking on a FB Ads ad); <em>Touchpoint is Order = True </em>means that the customer converted (ordered a car) on its 21st touchpoint (on a session originated by an Organic Search). </pre><p>So, given this customer journey how is credit distributed amongst the different channels?</p><ul><li>Generic Paid Search being the first touchpoint will naturally be credited 100% of the conversion on a first touch basis and Organic Search being the last touchpoint will be credited 100% on a last touch basis;</li><li>However if we use the variation last touch to lead it will be Facebook ads receiving 100% of the credit as it was the last touchpoint before the customer became a lead;</li><li>As for linear attribution, in its original form Email gets most of the credit (most of the touchpoints in the journey are originated by email) and Direct gets ~14% of the credit whereas the other 3 channels only get ~5% of it. Although, on the <em>to lead</em> variation Facebook Ads and Generic Paid Search get 50% of the credit each and I think this illustrates why the <em>to lead</em> variation can be so useful from a marketing point of view (Email was just mostly working with the lead that FB and Google “gave” to it, so it makes more sense to credit the channels responsible for the acquisition of this customer);</li><li>Finally, the <em>non-direct</em> variations strip Direct from any credit (still on the linear attribution example, the <em>linear attribution to order non direct</em> gives 0% credit to Direct when its original variation was attributing ~14% of the credit to Direct);</li><li>The table below summarises how credit is distributed amongst the different channels for each of the attribution models.</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/53f2f51c4971ddd8c1c7c08d0f3857be/href">https://medium.com/media/53f2f51c4971ddd8c1c7c08d0f3857be/href</a></iframe><p>If you’ve reached this far you can now understand how attribution solve for the problems mentioned at the beginning; we now know much credit is due to each channel and we also know how much are we spending on those channels (not only in £ ad spend but also in terms of effort/people) so we can understand if the return is worth the investment. Now, if you were mostly interested from a business point of view you can go directly into the last bit of the article (i.e. <em>final thoughts</em>), however if you are interested in learning about the data and tech parts of this project and how to implement an attribution model in real-life in a world of imperfect data and tracking then stick around for the next chapter.</p><h4>Actually implementing attribution modelling in your organisation</h4><p>Building an attribution model involves a wide range of technical challenges which at the same time are important to mention but some could also take its own article to dive deep into them, so in the interest of reading time I’ll try to be thorough and succinct at the same time. These are roughly the steps we had to take in order to make it happen here at Cazoo:</p><ol><li><strong>Event Tracking</strong> — This is not necessarily part of an attribution modelling project, but it’s definitely a pre requisite. When above we talk about touchpoints, in order for us to actually capture them we need some front end event tracking solution. There are several solutions and possibilities out there but our setup is quite simple yet really powerful. We use Google Tag Manager to track events into Segment which in turn disseminates those events into several other platforms, namely google analytics which we then seamlessly connect to our Big Query data warehouse (as they are both Google products).</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*TVoy7sRl9bgIybGw8DMBAw.png" /><figcaption>Cazoo’s Font-End Event Tracking</figcaption></figure><p>2. Put together a <strong>holistic customer journey</strong> — customers <strong>cross device tracking</strong>. This one is particularly challenging for Cazoo, although a key step in the whole process. As our customers are not logged in when they are browsing our website we don’t have the same weapons to tackle identity resolution as other companies do, hence we need alternatives. We have mostly two ate the moment: 1) Our best one leverages Segment’s personas product: when you submit your email (or click on an email that takes you to cazoo website) on multiple devices we know that those two devices belong to the same customer; 2) We use some internal Ids to tie together cross device journeys — whenever you submit a finance application we attribute you a finance checkout id. Then if you happen to resume that journey on a different device by for example clicking on an email that leads you to that same finance checkout id we would understand that those two devices belong to the same customer.</p><p>And why is this important? Take a look into the image below, we see a customer who did most of its research and getting familiar with the website on a mobile device but eventually places an order on a desktop. If we were not to consider this cross device behaviour we would have wrongly assumed that this customer’s journey was a single session one originated by an Organic Search when in fact their journey started 43 days earlier via Generic Paid Search.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*elKhJ0yInmM_SF38bBIBIw.png" /></figure><p>3. Put together a <strong>holistic customer journey</strong> — customers <strong>cross platform tracking</strong>. This one is also very particular to Cazoo and related to the fact of us advertising on aggregator websites like Autotrader. It is also probably the one that made the biggest difference in terms of fully understanding the impact of a channel — without it we’d have a completely different read on the importance of these aggregators into our marketing mix. When you are browsing Autotrader and you come across one of our ads/listings you have three options to reach Cazoo: Click on a link into our website (that one will be tracked like any other channel), email us directly from the Autotrader platform or call us. When you email us we can then compare that against any email that you submitted on our website (for example: when you submit a finance application with us or subscribe to our stock alerts). Finally, when you call us about an Autotrader listing we will at a later stage try to match against the phone number you submit on our website when placing an order.</p><p>Again, let’s look into the image below to understand why this is important. If we were not to attempt to put the customers contacting us in aggregator websites together with their journeys on Cazoo’s website we would assume that in this particular case their journey had started via email in our website (which is impossible because you can only receive an email from Cazoo if you have been in touch with us). However, when we understand that you first reached out to us via the email form in Autotrader your whole journey suddenly makes sense and we can give credit where it’s due: to the aggregator listing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hvlUJEbNJyf8BNIa9thGTQ.png" /></figure><pre><strong>Note: we do this in a completely anonymised way, we never actually look into users&#39; emails or phone numbers but an hashed version of it</strong></pre><p>4. <strong>Data Manipulation, ETLing and lots of SQL code. </strong>In this stage we start by basically take the work that has been done with putting together holistic customer journeys and we collate it in one big table we will call <em>customer_touchpoints</em> (i.e. a place where we store all relevant customer interactions from a marketing point of view — for us at Cazoo this is mostly website sessions and form submissions in aggregator websites like autotrader). From here, the goal is to reach a version in which such touchpoints have been attributed its due credit for each conversion but in the middle there is lots of data manipulation, window functions and ranking of touchpoints (<em>This is the way a computer reading SQL in an automated way has to make sense of touchpoints’ order vs when a human looks into them in a spreadsheet manually</em>) and ETLing using <a href="https://www.getdbt.com/">DBT</a> (massive time saver, you should check it out!). In case you are interested in the more technical details of it you can find some code snippets below using the same example in the intro chapter but now working it out in a more SQLish way.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e7c6997ee9f82681ac8d119c1b9a110c/href">https://medium.com/media/e7c6997ee9f82681ac8d119c1b9a110c/href</a></iframe><p>The output table (<em>attributed_conversions) </em>structure is basically still a big collection of touchpoints with each of them referencing the due credit they’re owed for their influence on the path to conversion. Such table is quite granular which allows us to then calculate an aggregate sum of attributed conversions at whichever levels we need to (by channel, date, device type, campaign, etc.).</p><p>5. <strong>Democratise it across the organisation</strong> — The final stage is both a critical but a relatively simple step in the process. It’s critical because without it our stakeholders in the marketing teams wouldn’t benefit from the project; simple because the heavy lifting of data manipulation and tracking has already been done by now. The last step is to make your <em>attributed_conversions</em> table available in a visualisation tool (we use Looker at Cazoo) so that the marketing teams can explore this data without the need to run any SQL queries at all.</p><h4>Final Thoughts</h4><p>Ok great we built an attribution model, so what? What is the business impact so far?</p><ol><li><strong>Knowledge is power, you can’t optimise what you don’t measure</strong> — The first impact has to be the fact that all our marketing teams can now better understand the impact that their channels and campaigns are having into the bottom line. In addition, they can monitor this in nearly real time in charts similar to the one below (here we are comparing channels monthly, but we could also be looking into totals per channel, campaign, device type, we could be looking into it daily, weekly, etc.)</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/877/1*6cxVYmAGLE4Tjx-lqQylAA.png" /><figcaption>Attributed Orders per Channel in Looker</figcaption></figure><p>2. The <strong>Performance Marketing team has already made some investment decisions from the back of it</strong>. We have combined the attributed conversions data with the advertising platforms data (things like impressions, clicks, spend) and by doing it so we can get to a cost per attributed conversion per channel (or per campaign, ad group, etc.). Once we have that, we were able to understand which channels had higher cost per attributed conversion and move some of its budget into other channels and it also provide evidence to negotiate a contract with one of the aggregator platforms we list on (the leads they were sending us were of much lower quality after all and they were not resulting in many conversions leading to an extremely high cost per attributed conversion when compared with other aggregators)</p><p>I want to leave a final paragraph just to say that although attribution has been a major enhancement to the way we measure our marketing activity at Cazoo, that it shouldn’t be the end of it. As I mentioned earlier, attribution is deterministic and in real life that is not always the case, particularly when we do so much Offline marketing activity like TV. We are now a well recognised brand so we must know that some of the conversions we get from some digital channels would potentially still happen had we not run that digital activity. How much, that we don’t know for sure, and that’s why we should include in our roadmap of marketing measurement some incrementality studies (basically running some tests to understand how many conversions would we still get were we not to run activity in a given digital marketing channel at all). Finally, another (less obvious) caveat is that attribution measures average cost per conversion but it tells us nothing about our marginal performance (i.e. it doesn’t tell us by how much our cost per attributed order would have been had we invested 10% more or less in a given channel — and the answers would likely not be by 10% either direction. Curious about this topic? More in <a href="https://towardsdatascience.com/paid-search-incrementally-adc5ea284279">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ef3a31ceb591" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cazoo/marketing-attribution-modelling-at-cazoo-ef3a31ceb591">Marketing Attribution Modelling at Cazoo</a> was originally published in <a href="https://medium.com/cazoo">Cazoo Technology Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Paid Search Incrementality]]></title>
            <link>https://medium.com/data-science/paid-search-incrementally-adc5ea284279?source=rss-94730e4c6f24------2</link>
            <guid isPermaLink="false">https://medium.com/p/adc5ea284279</guid>
            <category><![CDATA[marketing-automation]]></category>
            <category><![CDATA[sem]]></category>
            <category><![CDATA[paid-search]]></category>
            <category><![CDATA[incrementality]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[António Lima]]></dc:creator>
            <pubDate>Mon, 27 Apr 2020 17:48:59 GMT</pubDate>
            <atom:updated>2020-05-07T15:47:05.733Z</atom:updated>
            <content:encoded><![CDATA[<h4>Rethinking your targets and finding the optimal bidding levels — A business explanation and a <em>how-to</em> guide.</h4><p>Many advertisers and digital marketing professionals often work towards achieving a target cost per sale while maximising the number of customers or sales they bring to the business. Today we will challenge the best way to achieve those goals in the paid search world introducing the concept of incrementality. We will do it so by not only explaining the business rational and theory behind it but also proposing actual practical steps to operate in an incrementality framework as well as offering code samples to tackle the challenge in an efficient manner.</p><p>Let’s then start by defining incrementality as the impact that a specific action has in changing total output (that being cost, sales, revenue, profit or other). To understand <strong>incrementality</strong> we have to realise that there is always a <strong>baseline situation</strong> against which you want to <strong>measure</strong> the impact of any given action:</p><ul><li>Scenario 1 — <em>An action is totally incremental</em> — Ultimately if the baseline scenario is a state of absence of almost everything then all actions are 100% incremental. For instance, having a live website for an e-commerce company is 100% incremental. No sales or revenue can be made without it.</li><li>Scenario 2 —<em> An action is partially incremental </em>— Not all actions are as critical as the one above. Consider for example a company that buys advertising space regarding their own brand on Google, it is expectable that if they didn’t do it so then some potential customers would click on their competitors’ paid ads links; however, they cannot say that the revenue originated by people who clicked on their own ads is 100% incremental because some people could have come to their website in any case via the SEO organic link on their own brand, skipping through the competitors’ paid ads. (Note: not advocating for not bidding on your brand, many companies do it because they’re usually cheap clicks and they’re afraid competition will steer away some of their natural customers)</li><li>Scenario 3 — <em>An action is partially incremental and you want to find out how much of doing that action is optimal</em> — Imagine a restaurant, for example, trying to figure out how many options they should have available on their menus. Surely the answer is often more than one or two for increased choice and pleasing different subsets of customers, however, at which point the benefit of one extra menu item for increased variability is surpassed by the cost of having to manage extra food inventories, train their chefs to make such recipes and increased difficulty in kitchen coordination to jump between different dishes at one given service? Probably many restaurants would like to have a tool at their disposal that would help them solve this optimisation problem.</li><li>Scenario 4 — <em>An action is not incremental at all</em> — A simple example would be giving your employees white notebooks instead of black notebooks: this action is hardly ever incremental on employees’ productivity at any given point. Sometimes, however, non-incremental actions are harder to identify. For example, you show some Facebook ads to your existing customers and they eventually at some point purchase your items again (there’s a correlation). Although, it could be the case that they would buy your items in any case (correlation not implying causation) or even that some of them would eventually buy again from you but they became upset with your company over-communicating and ended up not returning to your website or shop (in this case your action would have been negatively incremental).</li></ul><p><em>A quick note before proceeding: A reasonable expectation regarding the meaning of the expression paid search incrementality could be the pursuit of understanding the incremental value of the channel as a whole (i.e. what would happen if we turned off ppc at all? Would some potential customers still find us via other marketing channels?). Although this can be an immensely insightful study for some companies, this is not what this article will be exploring today. Instead, this article will be focusing on what we described in scenario 3 and its application to the Paid Search world. Hopefully, this could be useful to help you think about your PPC goals and targets, and how to measure what are your optimal bids for achieving them.</em></p><p>So like in the example of restaurants optimising the number of menu items they should have, chances are that some google ads advertising will be beneficial and profitable for your business and that too much google ads advertising starts to become marginally loss making for your company. This article proposes a framework for finding that optimal amount of google ads advertising while bearing an <em>incrementality </em>mindset and steering away from an <em>averages</em> way of thinking. This means making the transition between optimisation maturity 2 and 3 described below (which can be as important and fundamental as making the transition between optimisation maturity 1 and 2):</p><ul><li>Optimisation maturity 1 — <em>Company Target — </em>The SEM channel manager ensures that the account is roughly achieving an overall target per conversion and that the return on ad spend is in line with the expectations of upper management or clients. The diversity of campaigns is either limited or the span in actual cost per conversion across campaigns is really big.</li><li>Optimisation maturity 2 — <em>Averages — </em>The SEM manager still ensures that the account’s return on ad spend is in line with expectations, but at the same time ensures that it has a robust account structure with a multitude of campaigns achieving similar costs per conversion across all of them. The rationale behind it is that firstly campaigns with an average cost per conversion above target are unprofitable for the company and secondly that volumes of conversions are maximised when all campaigns are on target vs having some campaigns below and others above target (even if at the end of the day both scenarios lead to an overall account on target). As a second step after ensuring the above, the SEM manager will also often ensure that other ways of “slicing” the account other than on a campaign level will also be on par with each other in terms of the average cost per conversion (examples being device type, major locations, ad group types that repeat themselves across many campaigns, etc.). Most companies operate on this level of ppc optimisation maturity.</li><li>Optimisation maturity 3 — <em>Incrementality — </em>Companies operating on this framework understand that their average targets and metrics are just the tip of the iceberg. The critical mind shift is that they redefine the question from “<em>what’s my average cost per conversion?</em>” into “<em>what is the maximum I am willing to spend on any given conversion?</em>”. They also understand diminishing returns and that they vary depending on many factors: They know that two campaigns with the same average cost per conversion can have very different incremental costs per conversion and if they have a pound or a dollar more to spend they won’t distribute evenly across these two campaigns but they will instead apply it on the one with lower incremental cost per conversion. However, to put such framework in practice the very first thing they do differently is to generate their own datasets as incrementality metrics are not readily available and this usually involves a lot of cross-team collaboration (and they don’t by any means underestimate the importance of communication, alignment and buy in across all teams involved). Without such an effort, this framework won’t ever be nothing more than a theoretical exercise. Finally, once that’s unlocked, they go from theory to practice, from averages to incrementality, by using their new metrics, reports and marketing business objectives.</li></ul><p>But what do we mean by campaigns with the same averages but different incrementality economics? Let’s consider the following example: For both campaigns ABC and XYZ, you have set a target cpa of 6£ (assuming smart bidding here but should be no different if you’re running keyword level manual bidding) and Google is able to get you 100 conversions at that average cost for each campaign. Without taking incrementality into consideration you might have no reason to take different actions when optimising each campaign. However, what might be happening in the background is that campaign ABC would be able to get you 80 conversions at an average price of £5.8 and extra 20 conversions at an average price of £6.8 (average cpa for the total 100 conversion still being £6/conversion) whereas campaign XYZ would be able to get you 80 conversions at an average price of £4 and then finds some slack to get you an extra 20 conversions at an average price of £14 (average cpa for the total conversions still being £6 pounds — and yes this can happen and it does happen all the time in so many occasions!). Basically, in one case you are acquiring extra 20 conversions at a price that’s still reasonable and in the other at a price that probably makes you lose money and this sums up the idea of why averages often hide half of the picture. We finish this section with a chart that illustrates this idea:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_jIoRFZVx3zDsFvucj4mgA.png" /><figcaption>Marginal Vs. Average Cost per Conversion</figcaption></figure><p>At this stage, we have laid the theoretical framework in regards to why PPC Incrementality is important, but we haven’t yet proposed a practical way to operate under this level of optimisation maturity and that is probably the main point of this article. It is harder though than operating on an <em>averages</em> framework hence fewer companies adopt it. The good thing about working with averages is that the data is usually readily available for consumption and it does not require extra steps for generating it. Google ads will provide you with detailed spend and conversion data from which you can easily calculate your average cost per conversion on the account itself or on almost whichever <em>“slice” </em>of the account (per campaign, ad group, device, a combination of device and campaign, location, audience, etc.). If you want to operate on an incrementality framework you will have to figure out which data you want to generate, do generate it and ensure its quality and accuracy.</p><p>So, how do you run an incrementality test on your ppc account? And yes I say a test because on the one hand for many people ppc incrementality might have not been a thing they have yet implemented and on the other ppc incrementality does require testing per se. When we earlier gave an example in which out of you 100 conversions you would get at a total cost of £600 you could potentially get 80 of them at a cost of £4 and extra 20 at a cost of £14, we did not mention that there is actually a way to know that is the case (or not) <em>for sure </em>(at least statistically speaking). And the vehicle for that is the very likely known to you google ads’ <em>Drafts and Experiments</em>. This tool will allow you to A/B test any campaign in your account splitting search traffic equally among the control and variant (or original and draft campaign in google ads jargon) and peek into what would have happened to your metrics (spend and conversions) had you have the draft instead of the original in place. The difference between your draft and original campaign will give you your incremental spend and incremental conversions. Careful though, if we don’t equate statistical significance into our test, it will be as informative as doing nothing. Moreover, chances are that you will hardly ever get statistical significance at the campaign level and you will have to get less granular in your test (i.e. consider starting at account/group of campaigns/portfolio level).</p><p>Thence it eventually boils down to the following steps (and now we enter into the practical application of the article):</p><ol><li>Scenario Analysis</li><li>Create multiple drafts and experiments at scale</li><li>Monitor the integrity of the test</li><li>Collect the data in a suitable manner and get it ready for measurement.</li><li>Evaluate the Test</li><li>Exploratory Analysis</li><li>Wrap up and prepare for future incrementality tests.</li></ol><h4>1) Scenario Analysis</h4><p>This is planning, playing with numbers and forecasts phase. You should start with a hypothesis as in any good experiment and that hypothesis should be actionable (i.e. its confirmation or not should make you take either action A or action B) and measurable with a good degree of accuracy (i.e. you must be able to generate enough data to reach a statistically significant conclusion and there shouldn’t exist any technical, commercial or business blockers preventing you from generating such data). Let’s get back to our example in which we had a ppc account generating a total of 100 conversions with an average cpc of £6. An example of a good hypothesis, in this case, could be:</p><ul><li>You want to find out if your incremental conversions cost you more than £15 (because you breakeven at £15 or because you want a marginal ROAS of at least 2 times to compensate for all other costs you have for example).</li><li>If you confirm your hypothesis (incremental conversions &gt; £15 per conv) the action you will take is to decrease your bids up to the same bidding level you tested in this experiment.</li><li>If you disprove your hypothesis (incremental conversions &lt; £15 per conv) the action you will take is to maintain your bid levels.</li></ul><p>With this in mind you’ll now run some scenarios playing with the variables of the test:</p><ul><li>Potential metrics of the test: lower funnel conversions may provide you with the answer you’re most interested in, but upper funnel conversions are more frequent hence meaning shorter test durations. The more suspicious you are in the effect that varying bids might have into the type of visitors that you ppc account attracts the more reason to make the effort to have lower funnel conversions as your test metric, otherwise upper funnel conversions might just do it;</li><li>Variation in bidding levels: it’s more interesting to test bidding levels closer to each other like a 10% change, but you might not have enough data to accurately validate such small difference;</li><li>Assumptions to what happens to spend and conversions when changing your bids by the levels defined above: if you are decreasing you bids naturally your spending levels will decrease as well, but by how much one might wonder? You should put forward a few assumptions in advance and check what impact in your estimates and confidence levels they will have “if you were to observe such difference at the end of the test”. You want to ensure beforehand that the design of the test is robust enough to withstand different outcomes and still be able to provide you with the answer to your initial question: is my incremental cost per conversion above or below £15? And for that, you will want that your confidence intervals don’t overlap £15.</li><li>Time to run the test for: the longer it is the more data you gather and your confidence intervals become smaller, however running a test has always a cost associated with it being it at least number of conversions sacrificed or opportunity cost of not running other tests. The more important is the answer to your question the more willing you will be to extend the test duration.</li></ul><p>The more options you consider for the 4 variables described above the more scenarios you will end up with so we want to strike a balance between a comprehensive mix of scenarios and not getting overwhelmed by them. Back to our example, we have created some scenarios ourselves as well. We have considered two metrics; two variations in bidding levels; two different assumptions to what happens to spend and conversions when varying each of the bidding levels; and two test durations. All these combinations resulted in a total of 16 scenarios as per the table below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HFtF0WRARZCp0MY7ldyF6Q.png" /><figcaption>Scenario Analysis</figcaption></figure><p>The first thing that is important to explain is that you will generate a lot more “spend” data than “conversions” data, hence your spend confidence interval will usually be very close to its point estimate meaning that your test accuracy will mostly depend on your conversions data.</p><p>Secondly, the confidence intervals calculation depend not only of your assumptions but also on how much data you will generate (you should look into your account’s historic to have an idea of how much that is in your case). In the <em>“Evaluate the Test”</em> section we dive deeper into confidence intervals calculations which you can then also apply when calculating the expected CIs in the scenario analysis.</p><p>So, once we have all these it’s time to make decisions regarding the final design of our experiment and looking at the scenarios in the example it seems that the following are not possible:</p><ul><li>A 10% variation in bids doesn’t seem to be an option. Even if we were to run it for 2 months and used leads as the test metric, the confidence interval in the scenario in which we assumed spend to decrease by 15% and conversions by 7% would overlap £15. We could end up in a situation by the end of the test in which we wouldn’t have either confirmation or disproval of our hypothesis hence unsure of whether to take action A or action B.</li><li>Running a test based on purchases for the duration of 1 month seems to be out of the table too (in our spend 50% down and conversions 25% down scenario the confidence interval overlaps £15 by a lot — [£8.8 ; £18.8]; and using the other assumptions the confidence interval just touches £15)</li></ul><p>Hence our options would be either:</p><ul><li>Run the test for 1 month and evaluate it on Leads (both spend/conversions assumptions end up in a confidence interval clear of overlapping £15)</li><li>Extend the test duration to two months and turn Purchases into your metric (again CIs clear of overlapping £15)</li></ul><p>Which one you end up choosing depends on what’s more important for you in the tradeoff of test duration vs lower funnel conversions.</p><h4><em>2) Create multiple drafts and experiments at scale</em></h4><p>At this point you have chosen the design of your test and here is where we need to get a bit more technical as we will be introducing code samples to tackle this step. Python being the data language of election will be what will be using, plus we will be interacting with the adwords API. So to start with you will need to connect to your account’s AdWords API. If you haven’t used the API before, the first thing you have to do is to get a Developer Token; a Google Client ID (this is different from the Client Customer Id — the one you actually see in google ads user interface with a number similar to this one: 562–425–3085); Google Client Secret; and Google Refresh Token. There’s very useful documentation in how to get started with the adwords API and generate all of the above in <a href="https://developers.google.com/adwords/api/docs/guides/first-api-call">here</a>.</p><p>If you have succeeded on the above then the following script should connect you to your adwords API:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a6c3e5129b4e0dec165c2d4f772304da/href">https://medium.com/media/a6c3e5129b4e0dec165c2d4f772304da/href</a></iframe><p>The next thing you will probably have to do relates to budgets. If you are not using any shared budgets in your account you can ignore this next bit, but my bet is that at least some of your campaigns are sharing budgets. If that is indeed your account’s case then the next bit sets to address one of the drawbacks of using Drafts and Experiments: you can’t create a draft for a campaign that is using a shared budget. This means that you will have to create individual budgets and assign each of them to your campaigns. Fortunately, you can also do this programmatically, which will save you lots of time when doing it for all the campaigns in your account that you proposed to enter the incrementality test. Once you decide what is a sensible amount for each campaign’s individual budget, the following scripts will help you create them:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d329a28a0af06d03cd5e502e142aa147/href">https://medium.com/media/d329a28a0af06d03cd5e502e142aa147/href</a></iframe><p>Now is a good time to introduce why google has probably named this feature Drafts and Experiments and not juts Drafts or just Experiments, since creating a draft is next on the list. In a nutshell, a draft when created is a duplication of an existing campaign that will always remain linked with its original campaign (i.e. can’t run on its own independently) and that will see some component of it get changed vs the original campaign (in theory you don’t have to but that would seem pointless); whereas an experiment is the settings that will determine in which circumstances and for how long a test between a draft and an original campaign will happen (what is the split of traffic, if this split is made on a cookie or search basis, what’s the start date, etc.). A draft is a campaign, an experiment are settings.</p><p>So next on the list is to create the Draft, which can be accomplished by running the script below:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dab2ae4494f72fd11c3f855bcc5b71ea/href">https://medium.com/media/dab2ae4494f72fd11c3f855bcc5b71ea/href</a></iframe><p>And surely the most important thing to note is the information returned by this create_draft() function as we will need it afterwards to make changes to the draft campaign and also to create the experiment. And that is exactly what we will be doing next: changes to the draft campaign.</p><p>As in the examples earlier above we will consider that you are operating on a target_cpa smart bidding strategy and that you want to find out what happens when you drop your targets by 30%. The code below will help you with that.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c6a7ea610bdce03e89d292b01c0f5f80/href">https://medium.com/media/c6a7ea610bdce03e89d292b01c0f5f80/href</a></iframe><p>The code above is doing two things, firstly it creates a target cpa portfolio with a target that is 30% less than the portfolio that is assigned to the original campaign, and secondly, it assigns that newly created portfolio to the draft campaign we created in the step before. There are a few additional things though that we should bear in mind:</p><ul><li>If a multitude of your campaigns is assigned to the same portfolio then you wouldn’t want to create a single portfolio for each draft campaign. Instead, a more sensible approach would be to create one portfolio for each group of draft campaigns that share the same target cpa and assign that portfolio to all the drafts with the same target. The exact code to do it would be slightly different but would follow a similar logic.</li><li>You might have ad groups within some campaigns that have their own individual targets. If that is the case you will have to ensure that the draft campaign corresponding ad groups will have their targets dropped by 30% too (in this example). Otherwise, you won’t have a fair test as some ad groups won’t be dropped in the same order of magnitude as others. You would need an additional script to do this operation.</li></ul><p>Once the above is done, we’re off to the final step of making an experiment live using Drafts and Experiments. Not a complicated one, you just need to grab the draft_id an campaign_id returned by the create_draft() function, then chose an experiment name (ideally something along the lines of a bit that identifies the original campaign and a bit that identifies the whole group of experiments that are part of this test like a suffix: <em>campaign_123-decrease_30_test</em> for example), a split percentage (recommend 50%) and a split_type (recommend Cookie). The following code will then handle the final operation of this section:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/495d6e0b5a146621511e0910300c7c85/href">https://medium.com/media/495d6e0b5a146621511e0910300c7c85/href</a></iframe><h4>3) Monitor the integrity of the test</h4><p>When you are running a test there is nothing worse than seeing it being invalidated, particularly for totally unavoidable reasons. This section is about the watchpoints of things that can go wrong if we are not careful and that ultimately would invalidate this test’s data (these incrementality datasets) that is so precious for our end goal. The crux of the matter is that the only setting that should be different between your original and draft campaigns is the bidding levels. So:</p><ul><li>If you have any team member or tool that updates ad copy in a systematic way across the account, make sure the same changes are applied to original and draft.</li><li>If your landing pages are changing by any reason, make sure that the end of the day original and draft have the same landing pages.</li><li>If you’re adding bid modifiers to your campaigns, you need to add those to the drafts too.</li><li>As previously mentioned, if your campaigns have ad group targets, they’ll have to be reduced/increased by the same proportion the main campaign targets has been and you need to ensure they keep the same way throughout the test. I have added some example code of how you can quickly make this comparison, in particular, leveraging the adwords API:</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f511328f0eb67c24d38fa1369ad482bd/href">https://medium.com/media/f511328f0eb67c24d38fa1369ad482bd/href</a></iframe><ul><li>I think you got the point already, but for comprehensiveness here are some other settings you might want to bear in mind: Budgets, Location targeting, audiences, networks, ad rotation, conversions, new ad groups or keywords, negative keywords, extensions.</li></ul><p>Ultimately it’s a balance of either trying to avoid making many changes to the campaigns that you have created a draft and experiment for; or having a robust system in place that will ensure changes applied to original are also applied to draft.</p><p>Finally, one extra note to close this section: Did your portfolios in the draft campaigns went into learning mode and behave «funny» in the first few days? If yes, consider excluding these days from your incrementality dataset.</p><h4>4) Collect the data in a suitable manner and get it ready for measurement</h4><p>So you’ve launched your test and you made sure it’s running on fair terms, the next thing you want to do is to gather and prepare the test data in a manner that’s suitable for analysis and evaluation. In line with what we have done so far we will resort to adwords api to tackle this step of the process as well.</p><p>But first let me ask you, have you heard about <a href="https://fivetran.com/directory/google-ads">fivetran</a>? If not please do yourself a favour a go implement it straight ahead! Assuming you currently use fivetran or you went on implementing it and came back to the article, we will use the google ads fivetran connector to get ourselves the data we need. The alternative would be to use the AdWords Query Language (AWQL) directly, but fivetran handles all of that for us with just a few clicks.</p><p>So, using fivetran what we need to do is to create a new google ads connector and sync a Campaign Performance Report into your database of election (<em>psst… </em><a href="http://www.snowflake.com/"><em>snowflake</em></a><em> is the best one) </em>choosing the following fields as per the image below:</p><ul><li>AccountDescriptiveName</li><li>CampaignId</li><li>CampaignName</li><li>AdNetworkType2 <em>(Not needed for test analysis but useful for deep dives)</em></li><li>Date</li><li>Device <em>(Not needed for test analysis but useful for deep dives)</em></li><li>Clicks <em>(Not needed for test analysis but useful for deep dives)</em></li><li>Conversions</li><li>ConversionValue</li><li>Cost</li><li>Impressions <em>(Not needed for test analysis but useful for deep dives)</em></li><li>SearchImpressionShare <em>(Not needed for test analysis but useful for deep dives)</em></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/425/1*1NIi4wIKdgiBDOtI4Re6lA.png" /><figcaption>Fivetran Google Ads Connector</figcaption></figure><p>And finally, once this data has made its way into your database, the following query will organise it in a way that’s suitable for test evaluation (Such query will serve as the backbone for the evaluation of the test):</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5d1f207e4753997aac54b623b7c0d0ff/href">https://medium.com/media/5d1f207e4753997aac54b623b7c0d0ff/href</a></iframe><h4>5) Evaluate the test</h4><p>At this stage, your test is coming to an end or is already finished. You have collected and prepared your data in a way that is now suitable for analysis and that’s exactly what you’re going to do now. But how? There’s not one single right to way to it, but we propose that we use a statistical technique called bootstrap resampling (which is actually not so different from what google itself uses to report their confidence intervals — they use jackknife resampling though). The reason why we need to do this has to due with google only reporting confidence intervals on individual campaigns’ experiments, so if you are running an experiment on a multitude of campaigns you will have to calculate such confidence intervals yourself as if they were one big campaign because they are in fact all part of the same experiment.</p><p>The end goal of this stage is not just to find the difference in spend and in conversions between the test and control groups (because that is relatively easy — we just need to look at total spend and conversions on both groups of the experiment) but also to find a confidence interval for that difference hence the resampling. By resampling we will be taking a sample (pass the redundancy) with replacement of n observations of the performance of different campaigns on different days and compare the differences between test and control (n being the total number of rows yielded by the query provided in section 4 — i.e. total combinations of campaigns/days). We will be repeating this process a multitude of times (usually between 1,000 and 10,000) and for each of them comparing the difference between test and control. Having this set of samples we are now able to calculate the point estimate (the mean of the resample’s means) and the variance of performance amongst this set of samples and hence calculating our statistics’ standard deviation as:</p><pre><strong>σ</strong>x<strong> = ( ∑ ( (xi -x) ^ 2 ) / j) ^ (½)</strong></pre><pre>xi, being the mean of each resample<br>x,  being the point estimator (the mean of all resamples&#39; means)<br>j,  being the number of resamples</pre><p>And now having both the variance and the observed difference between control and test we are able to calculate the so much wanted confidence interval as:</p><pre><strong>CI = x ± 1.64 * ( σ</strong>x<strong> / n ^ (1/2) )</strong></pre><pre>x,    being the point estimator (the mean of all resamples&#39; means)<br><strong>σ</strong>x,   the standard deviation of the samples&#39; means<br>n,    the size of the sample (i.e. combinations of campaigns/days)<br>1.64, Zα for a 90% Confidence Interval</pre><p>Finally, we can say with 90% of confidence that the true difference in performance at different bidding levels in our ppc account lies between the lower bound and the upper bound of the confidence interval and make business decisions accordingly — back to our example in the scenario analysis section this means that if the confidence interval is below £15 we will maintain our bidding levels, however, if it is above £15 we will decrease our bids by 30% (as the marginal cost of the marginal conversions would be above our threshold of profitability)</p><p>The actual code to do make such calculations using python and a dummy dataset mimicking one prepared accordingly to the methodology proposed in step 4 are as per below:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ca90c69015c3cd279bc4a844116507ea/href">https://medium.com/media/ca90c69015c3cd279bc4a844116507ea/href</a></iframe><p>If you are interested in finding out more about this resampling technique and the theory behind it I really recommend this <a href="https://towardsdatascience.com/an-introduction-to-the-bootstrap-method-58bcb51b4d60">article</a>.</p><h4>6) Exploratory analysis.</h4><p>This is probably one of the most interesting and joyfully stages of this whole journey. You have successfully identified what’s the incremental value of going from bidding level <em>x</em> to bidding level <em>y</em> and gave your business crucial information to make a decision on whether to keep the biding levels or change them depending on the incremental value of your incremental spend. You have done this for the overall account potentially if your doing it for the first time or if you don’t generate a lot of conversion data; or you have gone further and did it for multiple portfolios, groups of campaigns or others (meaning that your test was actually a set of tests and you took actions on each of them accordingly) — for simplicity though, in this section we will assume you did an overall account test.</p><p>Nonetheless having a clear hypothesis that we want to answer being the main goal of this venture, a ppc incrementality test should also create many byproducts that you won’t want to ignore. The byproducts might need to be planned (for example if you purposely breakdown your campaigns into targeting locations where it is raining at the moment and locations where it isn’t raining — farfetched in most cases but for instance if you have a windshield business or you sell snow chains this might be important for you) but usually they will be available in on or more of the many <a href="https://developers.google.com/adwords/api/docs/appendix/reports">adwords report types</a> and in many cases they will even be unintentional and only found after careful post test analysis. So let’s dig more into this second option: I will give a couple of examples in bigger depth and in addition suggest a list of factors that might have an interesting impact into different incrementality levels within your account — your job is to look at all of them and find out which ones are more decisive for your business.</p><p>Example 1: Impression share can play a critical role in incrementality. Imagine you sell two types of t-shirts in your website — A blue and a red one (and you have a ppc campaign for each of them). They’re both sold at the same price but the blue T-shirt’s conversion rate is 40% and the red’s is 10%, hence you’re willing to pay 4x more for a blue T-shirt ppc click than a red T-shirt click. In this case you’ll end up with the same cost per conversion on both campaigns but you’ll likely have a higher impression share on the blue campaign (because you’re paying more per click). What’s important about this is that usually, it’s difficult to go after the extra bit of potential customers that you’re not showing ads to when you have 80% impression share than going after a few more of the yet so many available potential customers when your impression share is 20%, hence (rule of thumb) the higher the impression share the higher is your marginal cost per conversion and this is particularly noticeable on both extremes of impression share. Below is a real life based example of the role of impression share in marginal cost per conversion:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*__6lVUaHH_ZmEMtY65RL6A.png" /><figcaption>Marginal Cost &amp; Search Impression Share</figcaption></figure><p>Example 2: Your main competitor offers their services both via web and via an app and to attract customers they use both google search ads and universal app campaigns. This means that your competitor will have a wider range of possibilities and a higher probability of competing for the same search keywords in a mobile device than in a computer. The competitive landscape will be more favourable in computer devices and all things equal you would expect your marginal cost per conversion to be higher on mobile devices than in computers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/425/1*VDorW_1sTbfnRGV7JKiBsg.png" /><figcaption>Marginal Cost &amp; Device Type</figcaption></figure><p>List of factors potentially playing an important role into incrementality:</p><ul><li>Impression Share (and other measures of competitiveness like average position, absolute top of page, etc.)</li><li>Device</li><li>Location</li><li>Audiences (People who have interacted with your website vs. people who haven’t registered any activity with you; Gender; Demographics; etc.)</li><li>Campaign Type / Ad group type</li><li>Keyword Match Type</li><li>Time of Day &amp; Day of Week</li><li>Network</li><li>Demographic factors like age and gender</li><li>Others like weather (mentioned in example above); occurrence of significant event (for newspaper business); etc. As many as your creativity can get you too</li></ul><p>Finally, what’s still most important about these analyses is that they should and most likely will inform which tests you will run next. Let’s say again this was the very first test you ran and you had no preconceived idea of which factors would most contribute to incrementality within the account. After you run such exploratory analysis you will be able to formulate a more robust hypothesis and ensure you design a test to statistically prove (or not) that. For example, you might be able to group campaigns into four groups because you observe a pattern and design a test that will ensure you find with precision their incrementality levels and shift spending across them according to the test results; or you might design a test-focused in confirming (or not) that campaigns/ad groups / or devices with higher impression share will be less incremental hence shifting budget to the ones with lower levels of IS.</p><h4>7) Wrap up and prepare for future incrementality tests.</h4><p>If you have made it this far then congrats, you’ve done it — you have optimised your paid search account in an incrementality framework. Nevertheless, now it’s also the time to think about the future and how you are going to keep operating in an incrementality world. Below a list of the 3 most important things you want to consider at this wrapping up stage:</p><ul><li>We hope that the test has turned out to be conclusive and that has led you to take either one or another action, but even if this is the case these findings might not hold true for long: Auctions’ competitive landscape is ever-evolving and hence marginal cost per conversion will keep changing. You need to plan in order to keep a reasonably constant level of testing inbuilt into your accounts so that you can have accurate and updated answers to guide your optimisation goals.</li><li>You need to decide if the next set of tests you will run will be made on the same dimensions as the first one (i.e. account/portfolio/campaigns grouped by a given logic, etc.). If this is the case you still want to sense check if there’s anything you want to change about the test setup (could you run it for a shorter period of time? Given what you now know, should you change the test metric? Etc.). Alternatively, you might have found something interesting in the exploratory analysis that you want now to confirm (statistically speaking). For example, in the next test, you might group your campaigns into low/medium/high impression share and test increasing/decreasing different bidding levels for the different groups; or you might want to test different device multipliers for mobile/tablet/computer instead of changing campaign’s bids.</li><li>Finally, at some point, you will also want to consider some sort of automation in terms of implementing and measuring these tests.</li></ul><h4>Conclusion</h4><p>This article covered the reasoning behind paid search incrementality, why you should implement it in your marketing optimisation routines, and a practical way to indeed implement it; but instead of using this section to stress any of those points, I will save it to talk about the <strong>impact</strong> which at the end of the day is what really matters. Hope this will be the extra bit of motivation you need to actually go for it and implement it on your paid search account. So let me end talking about a startup in which we implemented this methodology and what it meant for the company:</p><ul><li>Context: This was a company in a high growth phase, hence historically, they were willing to break even on cost per conversion and revenue per conversion (on average). On a given period the company started to feel the need to show signs of profitability so google ads could no longer run at break-even cost (on average). Hence the challenge was defined as: Moving from breaking even on average to breaking even marginally.</li><li>The methodology was to run a series of tests from bidding level x to y to z (each of them following the 7 practical steps defined above) to find the point in the bidding level curve where the marginal cost per conversion was the same as the up to the moment average cost.</li><li>Results were: 1) For each 10% we saved in total spend in paid search we only sacrificed 4.2% of the total conversions generated by paid search. 2) <strong>From breaking even in paid search to a return on ad spend of 233%</strong>. 3) No single conversion turning a loss to the company.</li></ul><h4>Additional Thoughts</h4><p>Leaving a note on potential alternative ways to tackle this challenge other than the Drafts &amp; Experiments route. The first one is to use google ads bid landscapes — these are basically estimates that google makes about spend, conversions and conversion value were you to have different target cpas or target road. You can use these in the google ads UI for a one-off or programmatically via the <a href="https://developers.google.com/adwords/api/docs/guides/bid-landscapes">API</a>: The main drawback is that is only available on an ad-group level which may not generalise well for your use case and in addition the ad group needs to meet certain criteria. If this becomes available via the API on campaign or portfolio level this can become really powerful though. The second alternative is to use the <a href="https://developers.google.com/adwords/api/docs/guides/traffic-estimator-service?hl=pt">trafficEstimator API</a> — Haven’t used it myself because I have been working with smart bidding mostly but if you use manual bidding this tool should help you calculate how many clicks you’d generate for different bidding levels(max cpcs in this case) and if you assume a flat conversion rate from click to conversion you will be able to calculate you marginal cost per conversion from bidding level <em>x</em> to bidding level <em>y</em>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=adc5ea284279" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/paid-search-incrementally-adc5ea284279">Paid Search Incrementality</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>