<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Getintent on Medium]]></title>
        <description><![CDATA[Stories by Getintent on Medium]]></description>
        <link>https://medium.com/@GetIntent?source=rss-11e1dd11d1aa------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*SKzqF78GQ8uaMzKezluN9A.png</url>
            <title>Stories by Getintent on Medium</title>
            <link>https://medium.com/@GetIntent?source=rss-11e1dd11d1aa------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 19:38:05 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@GetIntent/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Excel at Reporting: How to Import Data into Google Spreadsheets Automagically]]></title>
            <link>https://medium.com/getintent/excel-at-reporting-how-to-import-data-into-google-spreadsheets-automagically-1e3ceb30097b?source=rss-11e1dd11d1aa------2</link>
            <guid isPermaLink="false">https://medium.com/p/1e3ceb30097b</guid>
            <category><![CDATA[reporting]]></category>
            <category><![CDATA[programmatic]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[google-sheets]]></category>
            <category><![CDATA[api]]></category>
            <dc:creator><![CDATA[Getintent]]></dc:creator>
            <pubDate>Thu, 21 Jun 2018 07:01:02 GMT</pubDate>
            <atom:updated>2018-06-21T11:30:23.411Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vUmskVY5hBFyt0uwGG7bOg.jpeg" /><figcaption><a href="https://www.shutterstock.com/ru/g/amy%20huish">cjmacer</a> / <a href="https://www.shutterstock.com/">Shutterstock.com</a></figcaption></figure><blockquote>“Data! Data! Data! I can’t make bricks without clay.”</blockquote><blockquote>― Arthur Conan Doyle, The Adventure of the Copper Beeches</blockquote><p>Just as Sherlock Holmes needed information to solve crime cases, so do marketers rely on data to understand what is working and what can be improved about their efforts. As a marketer managing programmatic campaigns, you must heavily depend on reporting data to improve your company’s programmatic buying strategy.</p><p>For this purpose, you most likely want to have all the campaign data in one place in an easy-to-read form. The UI built-in dashboard seems to be the right place to track key metrics, but what if it’s missing specific analytics or customization features? Or have you ever wondered how to enable other team members to view campaign data in real time without having to login? Say ‘hello’ to the Reporting API!</p><h3>Why use Reporting API</h3><p>The Getintent Reporting API is an effective tool for smart dashboarding and data analytics. It automatically imports campaign stats from the Getintent UI into a Google Sheet, turning raw data into actionable insights. You can use Reporting API to create pivot tables, graphs, dashboards, and other real-time reports showing campaign and creative statistics, spend numbers, or impressions.</p><p>Once you take time to construct an appropriate Google Sheet, you get those neat dashboards for quick ad hoc analysis that help you trace up-to-the-minute parameters and see trends unrecognizable otherwise. See an example of the link on the screenshot below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7aCiWeh1cMYqiY_LMs8vFQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/937/1*9iPKm-ej5nSBJUU07vtnIw.png" /></figure><p>Here are three steps to parse the Getintent UI data into Google spreadsheets.</p><h3>1. Access API and choose variables</h3><p>To enable reporting API, get your API key. Find the access token in your profile settings:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/831/1*Y7OkHVkuEdTsgzsqhlJrpw.png" /></figure><p>and paste it in the link below instead of &lt;token&gt;:</p><pre>https://reporting.getintent.com/api/v2/metadata?token=&lt;token&gt;</pre><p>The link provides access to the metadata that contains all the report forms and variables available for you through API:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/799/1*ZZY9HrowPn9ffCkaXzY0JQ.png" /></figure><p>Here is the most important information to get from a metadata report:</p><ul><li><strong>report_name<em> </em></strong>— the name of the report available for your API key;</li><li><strong>key_fields <em>— </em></strong>usually used as filters and rows to group values;</li><li><strong>value_fields </strong>— usually used as values in columns.</li></ul><h3>2. Choose the report and variables</h3><p>Any report from the UI can be acquired via a Reporting API link like this one:</p><pre>https://reporting.getintent.com/api/v2/reports?token=&lt;token&gt;&amp;dataset_name=&lt;dataset&gt;&amp;start=&lt;start_date&gt;&amp;end=&lt;end_date&gt;</pre><p>Here is a list of attributes necessary or optional for your API request:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dccb1efae75c75d501235ca335dcc6ed/href">https://medium.com/media/dccb1efae75c75d501235ca335dcc6ed/href</a></iframe><p><strong>You can build the link without all those manipulations in steps 1 and 2 by using the key <em>Link to API</em> <em>request</em> that is available in your UI Reporting after a report is built:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/561/1*Y0Tc7m3c_-JhfWlMVdl3JQ.png" /></figure><p>Here are a few examples of Reporting API requests viewable as comma-separated-values tables that can be pasted into Google spreadsheets:</p><ul><li>The report on clicks, impressions and average CPM with advertiser, month and campaign breakdowns:</li></ul><pre><a href="https://reporting.getintent.com/api/v2/reports">https://reporting.getintent.com/api/v2/reports</a>?token=&lt;token&gt;&amp;dataset_name=hourly_budget&amp;keys=advertiser_id,month,campaign_id&amp;timezone=UTC&amp;start=2018-05-01&amp;end=2018-05-18&amp;values=imps,clicks,cpm&amp;filter[]=eq:campaign_currency:USD&amp;filter[]=eq:&amp;csv_separator=,</pre><ul><li>The report on impressions and spend budget by countries:</li></ul><pre><a href="https://reporting.getintent.com/api/v2/reports">https://reporting.getintent.com/api/v2/reports</a>?token=&lt;token&gt;&amp;dataset_name=browser_traffic&amp;keys=country&amp;timezone=UTC&amp;start=2018-05-01&amp;end=2018-05-18&amp;values=imps,budget&amp;filter[]=eq:campaign_currency:USD&amp;filter[]=eq:&amp;csv_separator=,</pre><ul><li>The report on impressions and spend budget by SSPs:</li></ul><pre><a href="https://reporting.getintent.com/api/v2/reports">https://reporting.getintent.com/api/v2/reports</a>?token=&lt;token&gt;&amp;dataset_name=browser_traffic&amp;keys=ssp&amp;timezone=UTC&amp;start=2018-05-01&amp;end=2018-05-18&amp;values=imps,budget&amp;filter[]=eq:campaign_currency:USD&amp;filter[]=eq:&amp;csv_separator=,</pre><h3>3. Paste the data into Google spreadsheets and construct the report</h3><p>Once you construct a link, paste it into a Google spreadsheet using Excel <strong>=IMPORTDATA()</strong> function:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hMtXRlywrFiIVoH5x7q_3g.png" /></figure><p>By combining the API report link with <strong>=CONCATENATE()</strong> and <strong>=TEXT()</strong> function, you can change API report query parameters just by altering the numbers in the cell.</p><p>For example, you can change the date in your API link by concatenating your link with certain dates or write the <strong>=TODAY()</strong> function in the concatenated cell so the report is always up-to-date.</p><p>You can also insert a new timezone, API key or any other parameter without rewriting the API link:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b8MCylTydOZHRkcEHTgbKQ.png" /></figure><p>After you get the report, you can build custom tables or graphs that will be updated automatically in real time:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/876/1*nYg_mUEfPgWB_qbmI1JYiA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pK0P-kOKQ0Sy-bOVodaaHA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dSoTneeAY24FO-ko8PFI7A.png" /></figure><h3>Turn insights into action</h3><p>Now that your team has a clearer picture of how your programmatic campaigns are performing, make sure to turn those insights into actions. You may get carried away with building fancy reports and dashboards, but don’t forget analytics provides no value if that data is not acted upon.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1e3ceb30097b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/getintent/excel-at-reporting-how-to-import-data-into-google-spreadsheets-automagically-1e3ceb30097b">Excel at Reporting: How to Import Data into Google Spreadsheets Automagically</a> was originally published in <a href="https://medium.com/getintent">Getintent</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Empire State Bidding: RTB Auctions Research 2.0]]></title>
            <link>https://medium.com/getintent/empire-state-bidding-rtb-auctions-research-2-0-17d72f35597?source=rss-11e1dd11d1aa------2</link>
            <guid isPermaLink="false">https://medium.com/p/17d72f35597</guid>
            <category><![CDATA[auction]]></category>
            <category><![CDATA[digital-marketing]]></category>
            <category><![CDATA[advertising]]></category>
            <category><![CDATA[programmatic]]></category>
            <category><![CDATA[rtb]]></category>
            <dc:creator><![CDATA[Getintent]]></dc:creator>
            <pubDate>Fri, 20 Apr 2018 09:03:34 GMT</pubDate>
            <atom:updated>2018-05-03T14:40:59.724Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x8NmIR14UEtq9p9kgqyD5w.jpeg" /><figcaption><a href="https://www.shutterstock.com/ru/g/lewistse">Lewis Tse Pui Lung</a> / <a href="https://www.shutterstock.com/">Shutterstock.com</a></figcaption></figure><p>Historically, programmatic has rested on a second-price auction model, where the second highest bid determines the price the winner pays (one cent more than the second highest bidder offers). As a result, buyers usually paid far less than what they were bidding. The gap between the first and second highest bid enticed some exchanges and SSPs to make up for the missed profit. The strategic-minded ones secretly resorted to tricks like phantom bidders (to create the appearance of competition) or soft price floors, where they would raise the floor without buyers’ knowledge. This created the climate of mistrust in the industry — the issue so much publicized about.</p><p>The situation got even worse with the rise of <a href="https://blog.getintent.com/scraping-through-wrappers-overview-of-header-bidding-solutions-implemented-by-publishers-1007bff01780">header bidding</a> — a complex, multi-tier auction model where several SSPs bid on the same inventory simultaneously. The header bidding setup implies the bid that wins in an SSP’s second-price auction is to compete in another auction, a first-price one, in the publisher’s ad server. So if you’ve won in the first round of auction, but your second bid passed from the SSP to the publisher’s ad server is not the highest across all SSPs, you will lose the final auction, in other words, your win rate will drop.</p><p>What happens to the law-abiding SSPs that hold fair second-price auctions in the header bidding environment? They find themselves with low win rates, which means low revenue. So they face a choice — to miss out on valuable impressions and lose money, or to hustle to survive.</p><h3>Survival of the swiftest</h3><p>To ensure higher revenue for themselves, SSPs either have to cheat or move to the first-price auction model, where the winning bid is the amount paid. Moving to first-price auctioning, advocates say, is the route to transparency and a win-win game for all the parties involved: advertisers pay what they’re ready to pay, publishers get a better price for their inventory, the marketplace becomes more reliable. Yet, things aren’t always what they seem.</p><p><a href="https://digiday.com/marketing/programmatic-advertising-readying-first-price-auction-era/">Some major SSPs</a> are playing fair and square. For example, OpenX informs DSPs on the final auction type in the bid request. Rubicon Project automatically suggests the most appropriate auction dynamic for each impression. <a href="https://www.emarketer.com/content/how-index-exchange-switched-to-first-price-auctions">Index Exchange</a> also notifies DSPs about its shift to first-price.</p><p>But some SSPs mislabel the type of auction they run, leaving advertisers in the dark to play the guessing game. As a result, programmatic buyers can’t design a proper strategy to optimize their media spend. If buyers think they participate in a second-price auction while it isn’t, they overpay. On the other hand, if they think they are playing according to the first-price rules while in fact it’s a second-price auction, buyers bid too low and miss out on the inventory they want.</p><p>The issue has been highlighted <a href="https://www.emarketer.com/content/understanding-programmatic-auction-pricing-is-a-major-priority-for-marketers">in a recent survey</a>. More than 60% of marketers worldwide prioritize understanding auction pricing in 2018, and about a quarter of respondents identify it as a major priority. So the present RTB landscape is well worth investigating.</p><h3><strong>Bad guys and good guys in profile</strong></h3><p>It’s been more than six months since our <a href="https://blog.getintent.com/rtb-auctions-fair-play-3b372d505089">first research</a> into auction dynamics. Last October we witnessed a prevalence of second-price auctions with anomalies (52%) over fair second-price (35%) auctions across 39 SSPs from Getintent’s inventory. The share of first-price auctions was 13%.</p><p>Now that on media there are even more theoretical debates over auction pricing, it’s interesting to know how actually the situation has changed in the programmatic market.</p><p>This time we have examined the auctions run on the same 39 US-based SSPs in March, 2018. Again, for visualization purpose, we plotted normalized histograms that illustrate the win/bid prices ratio. This helps to analyze the nature of the bidding logic and spot deviations.</p><p>How do we tell what type of auction this or that SSP applies? A fair second-price auction on the Getintent platform is usually characterized by a more or less balanced distribution of the bid groups on the win CPM/bid CPM axis. This looks pretty much like Chicago skyline — dense buildings, with spikes of skyscrapers now and then.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1003/1*FM63OEOIVu4zw_Ve59kGlQ.png" /><figcaption>Fair second-price auctions</figcaption></figure><p>In a case where the win CPM is very close to the bid CPM, we assume it’s either a first-price auction (if the SSP has stated so in a special field on the OpenRTB protocol) or a second-price auction with an anomaly (if the auction type is declared as second-price). The profile of these two resemble a timeline that roughly depicts how the city of Dubai was built: a continuous desert with a lonely Burj Khalifa tower on the far-right.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/993/1*FW4ULUp6ZMhP501-xVZAoA.png" /><figcaption>Auctions with win CPM very close to bid CPM</figcaption></figure><p>Here we’ve spotted a cohort of SSPs with a weird distribution where all Getintent win CPMs lie within ratio 0.9–1 or 0.67–0.97:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/916/1*BSbQFDs5cPWcGknVnDIVKw.png" /><figcaption>Auctions where win CPM/bid CPM ratio lies within some interval</figcaption></figure><p>Another example, not as impressive as the cases above, but still a very suspicious distribution:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/914/1*Zb5Ij-0W9fgUOdhOF92jyA.png" /><figcaption>Suspicious auctions</figcaption></figure><p>To put it all together, here’s what the split of RTB auctions across 39 SSPs by pricing looks like in March:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DcLbluDeQFnLhckQbpFdVg.png" /></figure><p>The share of SSPs conducting first-price auctions has increased by 7% since October 2017, which might seem not that significant. But you’ll see the first-price auctioning is a trend indeed if you look at the auction split at the impression level.</p><p>Our analysis of more than 336 billion US impressions over a study period in March has shown the share of the impressions available via a first-price auction is over 43%:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jwplhuZc7x5HdEvudXVVQg.png" /></figure><p>Compare it with the picture for December 2017 — the share of the impressions available via a first-price auction hardly made 6%. The growth of the first-price share over the past four months is dramatic!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4AiRUtCBy77YU7Y6Vu1liA.png" /></figure><p>We can suggest that many of those SSPs that offered inventory via a second-price auction in December have fairly switched to the first-price, which resulted in an increased share of the first-price inventory in March. Meanwhile, the share of the impressions offered in auctions with anomalies has not changed significantly over this period.</p><p>The following diagrams show auction type dynamics across a few of Getintent’s key SSPs. All of them have increased the share of first-price auctions in their traffic, and we can expect the trend to pick up the pace in the future.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*h_uQ8gtvV7wbsB3UB-WMpA.png" /><figcaption>Rubicon Project auction dynamics</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*H1fAlepicwqlXKU0s1cW8w.png" /><figcaption>OpenX auction dynamics</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*R2sfSis9ZEAILR_DbR3BNA.png" /><figcaption>Pubmatic auction dynamics</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*eYgwIqzNrpnO2mbghAlZBQ.png" /><figcaption>Sovrn auction dynamics</figcaption></figure><p>We can suggest that the rise of first-price auctions is driven by steady growth of <a href="https://dev.serverbid.com/v1.0/docs/hbix-march-2018">header bidding adoption</a>. With the number of publishers embedding header bidding tags growing on a monthly basis, the share of SSPs that have at least partially moved to first-price increases too. Yet the hypothesis is hard to check since the OpenRTB protocol does not specify whether the auction takes place in the publisher’s header or it’s a good old waterfall.</p><h3><strong>Expert note</strong></h3><p>“First-price auctions indeed can bring more transparency for DSPs, provided the auction type is clearly communicated to buyers,” — says Kirill Solokhov, Data Team Lead at Getintent, who performed the analysis. — “However, we can’t say it’s a step forward for the industry. We shouldn’t dismiss the auction dynamics driven by common sense and business interest — buyers’ bidding strategy won’t remain the same”.</p><p>“It’s natural that DSPs want to win the auction at a lowest bid,” — he explains. “Sooner or later buyers might get a feeling they are paying more than necessary and start testing lower bids, which means practicing <a href="https://en.wikipedia.org/wiki/Bid_shading">bid shading</a>. Such logic debunks the wide-spread argument that publishers benefit from first-price auctions. Publishers may make more money from first-price auctioning, but only in the short run. In the long-term, first-price will actually lower CPM. As another side-effect, it will be harder for them to understand the true value of their inventory”.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=17d72f35597" width="1" height="1" alt=""><hr><p><a href="https://medium.com/getintent/empire-state-bidding-rtb-auctions-research-2-0-17d72f35597">Empire State Bidding: RTB Auctions Research 2.0</a> was originally published in <a href="https://medium.com/getintent">Getintent</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[RTB Auctions: Fair Play?]]></title>
            <link>https://medium.com/getintent/rtb-auctions-fair-play-3b372d505089?source=rss-11e1dd11d1aa------2</link>
            <guid isPermaLink="false">https://medium.com/p/3b372d505089</guid>
            <category><![CDATA[advertising]]></category>
            <category><![CDATA[programmatic]]></category>
            <category><![CDATA[rtb]]></category>
            <category><![CDATA[digital-marketing]]></category>
            <category><![CDATA[auction]]></category>
            <dc:creator><![CDATA[Getintent]]></dc:creator>
            <pubDate>Thu, 26 Oct 2017 15:39:48 GMT</pubDate>
            <atom:updated>2017-10-26T15:39:48.414Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*21ABHsI50keyXFN10bkp1w.png" /><figcaption>Photo credit: <a href="https://www.flickr.com/photos/wwarby/11734164904/">wwarby</a> via <a href="https://visualhunt.com/re/dab76d">Visualhunt.com</a> / <a href="http://creativecommons.org/licenses/by/2.0/">CC BY</a></figcaption></figure><h3>All hail Vickrey!</h3><p>Despite being well researched, auctions remain quite tricky. Lots of papers have been written on the topic. William Vickrey was even <a href="https://www.nobelprize.org/nobel_prizes/economic-sciences/laureates/1996/press.html">awarded the Nobel Prize in Economics</a> for his work on the theory of auctions in 1996. He proved that the second-price auction (now often called the Vickrey auction) reveals the true value of the object, because it is in the individual’s best interest <a href="https://en.wikipedia.org/wiki/Revenue_equivalence">to pay a fair price for the item</a>. For this reason, the second-price auction was adopted by the adtech industry as a default auction in the OpenRTB 2.0 protocol released in January, 2012. This means, unless specified otherwise, media buyers and sellers deal with the second-price auction.</p><p>However, the theory shatters on the rocks of reality. In real life publishers face a lot of auction-related issues and ask themselves: <em>What if my inventory is not very good and there’s only one bidder? What if the fair price for my inventory is much lower than what I want?</em></p><p>And here is when hard and soft floor prices come to the rescue. Unfortunately, floor pricing is not covered in the OpenRTB specification, so you need to read the fine print in the contract with an SSP. SSPs may also take advantage of dynamic floor pricing, which means in case you are the only auction participant, SSP will try to set the floor price equal or close to your bid. Tricky, isn’t it?</p><blockquote>But floor pricing is a relatively fair way to squeeze profit out of the second-price auction. In private auctions, SSPs can resort to some really dirty tricks, until they get caught with the hand in the cookie jar and <a href="https://adexchanger.com/online-advertising/rhythmone-dataxu-tussel-unpaid-bills-hidden-fees/#more-114534">another lawsuit breaks out</a>.</blockquote><h3>Everybody lies</h3><p>At Getintent, we have analyzed over than 1 billion impressions in the US for suspicious auction mechanics. Specifically, we looked into how often SSP reported the auction type in the OpenRTB protocol and what the win/bid price ratio was.</p><p>First, we divided all auctions into three types:</p><ul><li>First price auction;</li><li>Fair second price auction that looks like it is supposed to;</li><li>Second price auction with anomalies.</li></ul><blockquote>Your expectations is your problem. So take time to study the contract carefully, because you will probably find a clause that says the SSP can change the auctions rules as it wants.</blockquote><p>Here is what we got:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nL5rqs9h_o8r05z9Nk3dkw.png" /></figure><p>Then we plotted normalized histograms that illustrate the ratio between win and bid prices and distribution of the auction groups with approximately equivalent values.</p><p>The findings show 52% of the auctions have some anomalies. Below is the list of deviation scenarios.</p><p>Have a look at these guys. The SSP claims it applies the second price auction model, but the win bids can be divided into three buckets with values 0.8, 0.9 and 1.0.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/743/1*qPZO1RPfpA4Eodr_pvs31g.png" /></figure><p>The right most bar with the ratio of the bid/win price 1.0 signals that almost 30% of all impressions in the auction are bought at the first price.</p><p>As we move further, we see other SSP guys who take it easy and just break their clearing prices into 6 almost identical buckets — 0.89, 0.90, 0.91, 0.92, 0.93, 0.94 — with each roughly equal to 16.6%. Nowhere in the galaxy can this be called a fair auction.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/743/1*rLBIp6VDThjUDCUycr-lPg.png" /></figure><p>The case is not a one-off. Below we see an auction with 10 buckets.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/743/1*aKfXw4hw2OkgXHGSdlGjLA.png" /></figure><p>It gets worse and worse as it goes on. On the following SSP, despite the fact the second price auction is declared in the bid request, the win and bid prices are equal for more than 95% impressions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/743/1*clTq7Ir6jmhnGqzuNwcWMQ.png" /></figure><p>Interestingly, there is also an opposite situation. Even though the first price auction was intended, Getintent bidder won with a lower bid. This looks like a software bug at the SSP level or missconfiguration on the publishers side.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/743/1*JzWoR43I8ke0wpWj-z9wdw.png" /></figure><p>So these have been the bad guys. But what should good SSP guys look like? Here is the profile of a fair auction:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/743/1*Tm6end34i8xoH9CKIW31zg.png" /></figure><p>“You see the buckets are continuously distributed, with some random spikes,”— explains Kirill Solokhov, Data Team Lead at Getintent. “These spikes can be accounted for by the big campaigns running on the Getintent platform, but there is only one bucket with the value greater than 3%. Compare it with the buckets on other histograms.”</p><h3>The truth is out there</h3><p>All these schemes cast a shadow on the second price auction. No wonder there has been a tendency to switch to the first price auction mechanics recently. The key programmatic exchanges and SSPs like OpenX, Index Exchange, AppNexus, Rubicon Project, and PubMatic have already admitted starting experimenting with the first price auctions.</p><p>William Vickrey would say that the revenue would fall. Meanwhile the adtech community believes letting buyers pay what they bid will bring more transparency to digital advertising.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3b372d505089" width="1" height="1" alt=""><hr><p><a href="https://medium.com/getintent/rtb-auctions-fair-play-3b372d505089">RTB Auctions: Fair Play?</a> was originally published in <a href="https://medium.com/getintent">Getintent</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Lawyer Broflovski’s Principle, or DIY: Cloud Load Balancing]]></title>
            <link>https://medium.com/getintent/lawyer-broflovskis-principle-or-diy-cloud-load-balancing-a1fd7f818392?source=rss-11e1dd11d1aa------2</link>
            <guid isPermaLink="false">https://medium.com/p/a1fd7f818392</guid>
            <category><![CDATA[load-balancing]]></category>
            <category><![CDATA[web]]></category>
            <category><![CDATA[system-administration]]></category>
            <category><![CDATA[nginx]]></category>
            <category><![CDATA[programmatic]]></category>
            <dc:creator><![CDATA[Getintent]]></dc:creator>
            <pubDate>Fri, 20 Oct 2017 08:01:00 GMT</pubDate>
            <atom:updated>2017-10-20T08:01:00.913Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*sb8eSHZTkU5CJBUrWKYu4A.jpeg" /></figure><blockquote>Representing the side of “Everyone” is Gerald Broflovski, the lawyer from South Park who plans to make quite a commission. Representing the side of “Everyone Else” is Gerald Broflovski. So whatever the outcome, things look very bright for Kyle’s dad.</blockquote><blockquote>South Park, Episode 306, Panda Sexual Harassment</blockquote><h3><strong>Smug Alert!</strong></h3><p>If you have already launched projects with multiple frontends, you probably know how to distribute traffic. Hosting providers like Amazon AWS or Google Clouds have plenty of tools. But what if you are still not happy with them? Or what if you like working with bare metal servers, not clouds? At <em>Getintent</em>, we love bare metal equipment and can talk about the advantages for hours. Check out some highlights below:</p><ul><li><strong>No traffic rip-off. </strong><em>Pay per gigabyte </em>is the most popular revenue structure for cloud providers that have been spoilt by the start-ups’ demand. By contrast, hardware providers compete fiercely for the tight-fisted customer, and their common practice is to offer big packages of traffic for free. There are also clouds with pre-packaged traffic such as Linode, but you will be disappointed by the power of the instances provided. You might find out that to build some custom service you need a lot of instances. This means you end up paying the cloud provider for traffic all the same.</li><li><strong>You can buy servers predictably.</strong> For example, the AWS Service Limits description<a href="http://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html"> takes up more than 30 printed pages</a>. Each time you bump into a limit, you’ll have to explain use case to Amazon, and the latter will decide whether to give it to you. The requests from small customers are being processed within 3 days, but more often are not resolved for a full week. Meanwhile, reliable bare metal providers arrange everything immediately or the next day, and their only concern is how law-obedient you are and how promptly you pay the bills.</li><li><strong>Hardware life makes you use proven open source</strong>, not proprietary solutions with vague features that cannot be changed if inappropriate or fixed when broken down.</li><li><strong>Some things work properly only on the carefully adjusted equipment. </strong>Take Aerospike DBMS for example. The number of problems with it is directly proportional to the age of SSDs. That is why we do not only require bare metal SSDs, but also brand new ones. Moving to NVMe interfaces where possible reduced the load average fivefold. Those using Aerospike on cloud have nothing to do but smile sadly and go on fixing yet another cluster problem.</li></ul><p>A lot of dedicated providers have implemented virtual networks feature so server farm expansion in them is no problem. A popular exception is Hetzner where you still need to reserve units and allocate space. Let us hope it will soon come up with virtual networking too.</p><p>Where it even makes sense to discuss balancing with the provider, they will most likely offer a modern version of this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/374/1*C_tIIBnCuLRR5b7Hi9knXA.gif" /></figure><p>This hardware is pretty rare, therefore costs a fortune and requires specific expertise for configuration.</p><p>And then it is time to think about …</p><p><strong>Chef, what’s for lunch today?</strong></p><p>Your farms must already be using some kind of automation, which means making changes at scale is cheap. As an HTTP front-end, you are most likely using <em>nginx</em> on the same servers that serve the application. This makes it easy to build a balancing scheme, where each server is a balancer and a host for the app that responds to the request.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/732/1*NzYmGvf4nRhDCji83jsBpA.png" /></figure><p>First, configure the server pool in <em>gdnsd</em>:</p><pre>; <strong>Let’s check with a command whether the server <br>; with a specific IP-address is alive</strong><br>service_types =&gt; {<br>   adserver =&gt; {<br>      plugin =&gt; extmon,<br>      cmd =&gt; [&quot;/usr/local/bin/check-adserver-node-for-gdnsd&quot;, &quot;%%IPADDR%%&quot;]<br>      down_thresh =&gt; 4,  ; <strong>Failed to respond to the test fourfold? <br>                         </strong>; <strong>It’s dead.</strong><br>      ok_thresh =&gt; 1,    ; <strong>Responded at least once, it’s alive.</strong><br>      interval =&gt; 20,    ; <strong>Check every 20 seconds.</strong><br>      timeout =&gt; 5,      ; <strong>Consider a failure to respond <br>                         ; within 5 seconds as no-response.</strong><br>   } ; adserver<br>} ; service_types</pre><pre>plugins =&gt; {<br>   multifo =&gt; {<br>      <strong>; If less than one third of the hosts are alive, <br>      ; DNS will return the whole list of the servers <br>      ; as there must be some problem, e.g. an application overload.<br>      ; Disabling the broken servers can only make things worse.  </strong><br>      up_thresh =&gt; 0.3,<br>      adserver-eu =&gt; {<br>         service_types =&gt; adserver,<br>         www1-de =&gt; 192.168.93.1,<br>         www2-de =&gt; 192.168.93.2,<br>         www3-de =&gt; 192.168.93.3,<br>         www4-de =&gt; 192.168.93.4,<br>         www5-de =&gt; 192.168.93.5,<br>      } ; adserver-eu<br>   } ; multifo<br>} ; plugins</pre><p>If less than one third of the hosts are alive, DNS will return the whole list of the servers as there must be something wrong, like an application overload. Disabling the broken servers can only make things worse.</p><p>Use this configuration in the DNS zone file:</p><pre>eu.adserver.sample. 60 DYNA multifo!adserver-eu</pre><p>Then create a pool of <em>nginx</em> servers and deploy the configuration across the cluster:</p><pre>upstream adserver-eu {<br>    server localhost:8080 max_fails=5  fail_timeout=5s;<br>    server www1-de.adserver.sample:8080 max_fails=5 fail_timeout=5s;<br>    server www2-de.adserver.sample:8080 max_fails=5 fail_timeout=5s;<br>    server www3-de.adserver.sample:8080 max_fails=5 fail_timeout=5s;<br>    server www4-de.adserver.sample:8080 max_fails=5 fail_timeout=5s;<br>    server www5-de.adserver.sample:8080 max_fails=5 fail_timeout=5s;<br>    keepalive 1000;<br>}</pre><p>If there are more than 25 servers in the rotation, make sure to<a href="http://stackoverflow.com/questions/6794926/how-many-a-records-can-fit-in-a-single-dns-response"> split them into groups of 25 under one name</a> and dynamically select synonyms (DYNC) for <em>gdnsd</em> instead of A-records.</p><p>The list of servers can be filled in automatically based on the data from <em>gdnsd</em> or automation system.</p><p>At <em>Getintent</em>, we use Puppet software and a special cron script keeps the lists of the several types of servers registered on the puppetmaster up to date. The localhost line keeps us safe in case the list is mistakenly empty. As long as each <em>nginx</em> serves at least itself, ­no crash should occur.</p><h3><strong>Bring it on, boys, bring it on!</strong></h3><p>So, what we’ve got:</p><ul><li><strong>Excellent fault-tolerance.</strong> Risks are limited to network and configuration issues. Single point of failure in the connectivity can be solved by distributing the application across several regions with independent power and network. As for the configuration errors, you can make <em>nginx</em> or <em>gdnsd</em> changes in the configuration and break everything down, but luckily there are dozens of ways to fix this problem. For example, force delay between updating the servers and update each region independently.</li><li><strong>No problems like a pool warm-up or lack of server capacity. </strong>Surely your web application will die from the load first.</li><li><strong>Even traffic usage and network load distribution.</strong> Take traffic at 5 Gbit/s on 10 servers with gigabit? No problem! Just don’t forget to direct the traffic between the balancers and applications to the internal network. Also, make sure the provider does not sell you a switch with 1 Gbit/s of total uplink to the Internet for these 10 servers.</li><li><strong>Smooth deploys.</strong> When we used DNS-balancing only, our SSP partners would get pissed off each time our servers dropped out from the pool one by one. Furthermore, during initialization our Java applications load a lot of data into the memory, which takes time and would make the situation even more jumbled. Cross-balancing has made the process almost unnoticeable for the other side.</li></ul><h3><strong>Mr Twig gets offended</strong></h3><p>There is no such thing as a free lunch. So you have to pay for free balancing, specifically for sticky sessions. However, a free version of <em>nginx</em> offers<a href="http://nginx.org/en/docs/http/load_balancing.html#nginx_load_balancing_with_ip_hash"> IP hash balancing</a>, while modern NoSQL databases like Aerospike can be used as a reliable storage for shared sessions with fast response. Finally, you may switch to HAProxy that<a href="http://www.haproxy.com/blog/load-balancing-affinity-persistence-sticky-sessions-what-you-need-to-know/"> can route users on multiple balancer sheets by a single rule</a>. So when we need sticky sessions, we will use these tools available.</p><h3><strong>Numbers</strong></h3><p>Now each of our 20-core servers processes up to 15,000 requests per second, which is close to the upper limit for our Java application. Across all clusters, we balance and process in peak more than half a million requests per second, 5 Gbps of incoming and 4 Gbps of outgoing traffic excluding CDN. Yes, the specifics of DSP is that incoming traffic surpasses the outgoing one, as there are more bids than relevant responses to them.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a1fd7f818392" width="1" height="1" alt=""><hr><p><a href="https://medium.com/getintent/lawyer-broflovskis-principle-or-diy-cloud-load-balancing-a1fd7f818392">Lawyer Broflovski’s Principle, or DIY: Cloud Load Balancing</a> was originally published in <a href="https://medium.com/getintent">Getintent</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Making Headway. Header Bidding Research 2.0]]></title>
            <link>https://medium.com/getintent/making-headway-header-bidding-research-2-0-ca25303b6186?source=rss-11e1dd11d1aa------2</link>
            <guid isPermaLink="false">https://medium.com/p/ca25303b6186</guid>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[programmatic]]></category>
            <category><![CDATA[advertising]]></category>
            <category><![CDATA[digital]]></category>
            <category><![CDATA[marketing]]></category>
            <dc:creator><![CDATA[Getintent]]></dc:creator>
            <pubDate>Mon, 25 Sep 2017 13:01:01 GMT</pubDate>
            <atom:updated>2017-09-25T13:01:01.904Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*RdxCGWxa2HNIr0p4nnsVzA.jpeg" /><figcaption>Photo credit:<a href="https://www.flickr.com/photos/kalexanderson/6140989346/"> Kalexanderson</a> via<a href="https://visualhunt.com/re/179f09"> Visual Hunt</a> /<a href="http://creativecommons.org/licenses/by-nc-nd/2.0/"> CC BY-NC-ND</a></figcaption></figure><p>At the beginning of 2017 there was so much talking about header bidding in the adtech, that the technology would be called a synonym of programmatic advertising. But as time passed, the headlines changed, and for the last six months it’s been blockchain and M&amp;A deals that have been causing most of the buzz.</p><p>The topic, however, has recently been stirred up again as PubMatic published a white paper highlighting that header bidding is thriving, specifically on mobile. This brought us back to March 2017, when Getintent performed <a href="https://medium.com/@GetIntent/scraping-through-wrappers-overview-of-header-bidding-solutions-implemented-by-publishers-1007bff01780">a research on header bidding implementation</a> among Alexa Top 1,000 websites.</p><h3><strong>Previously on</strong></h3><p>The findings drawn by Getintent’s scraper that spring failed to prove the assumption that publishers were massively adopting the technology.</p><p>“For many it came as a surprise that only 121 publishers out of Alexa Top 1,000 websites were using header bidding,” comments Kirill Solokhov, Data Team Lead at Getintent, who conducted the research. “However, the figures are consistent if you mind the sampling frame and recognize that Alexa lists the highest-performing websites (according to Alexa traffic rank algorithm), many of which are governmental and corporate sites that don’t have ads by design, so they aren’t publishers per se.”</p><p>Today, six months later, Getintent fires up the scraper once again to run it through the same Alexa list and check how the situation has changed.</p><h3><strong>The moment of truth</strong></h3><p>As of September 4th, the findings show there’s a slight growth of the websites with the header bidding code plugged in, from 121 to 159 publishers (or from 12% to approx. 16%). This might seem hardly impressive at the scale of the Alexa thousand, but in the ‘publishers only’ context gets a more significant value.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QGfbVqh0DGqhEhZqiJBjfg.png" /></figure><p>Meanwhile, the split for header bidding solutions — 223 altogether detected (since some publishers use several solutions at once)— is now even more diverse. The open-source Prebid.js solution (79 wrappers, or 36%) is a little bit ahead of the custom solutions (66 use cases, or 30%).</p><p>“Such an evident rise of the custom-built containers demonstrates that publishers are starting to choose an independent approach and build their own tech,” Kirill Solokhov explains. “With its own wrapper, a publisher gets more control in an ever-changing ecosystem, and, more importantly, eliminates the risk of unplanned fees imposed by the tech provider.”</p><p>The Amazon solutions show a timid growth (43 use cases, 19%), which might suggest that the company has shifted focus and is encouraging its programmatic partners to switch to the server-to-server setup. The remaining solutions are represented by Criteo and OpenX (identified in 25 and 10 cases respectively).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YDhh9VPh01JRmhk8lyqVEQ.png" /></figure><p>As for the adapters, the findings show a greater variety of solutions from different vendors. On 159 publishers’ websites, the scraper identified 400 adapters delivered by 25 tech providers. Criteo, AppNexus, and Rubicon are in the lead having planted their solutions on more than 40 websites. Index Exchange, PubMatic, OpenX, AOL, and Sharethrough have also proved popular, with Index Exchange showing an impressive upsurge in comparison with its position in March.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lBMyCLHzNiOhcUmsXKC2IA.png" /></figure><p>This time Getintent has extended the research methodology and scanned its top supply inventory for header bidding solutions. The side research shows, 96 websites among Getintent’s top 200 publishers have header tags, which makes 48% of the sampled group of websites.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Gg87Iey3BCpg9Rkz4XehQg.png" /></figure><p>The breakdown for the wrappers and adapters providers mostly replicates the ratio of the Alexa findings, yet with little reshuffling. Among 158 solutions identified, Prebid.js obtains 36% (56 wrappers), while Amazon outweighs custom solutions (42 solutions, or 27%, and 37 wrappers, or 23%, respectively). 18 containers, or 11%, are those developed by OpenX, and the rest are PubFood and Criteo solutions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*668RdpIcPSZsPtzJoMTNgA.png" /></figure><p>When it comes to adapters, the most common ones are those offered by (in descending order) AppNexus, OpenX, Rubicon, Index Exchange, Sovrn, AOL, Sonobi, and PubMatic. Altogether, the scraper was able to identify 365 adapters from 25 vendors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SC0PU3xePWTxS7-Joe-azg.png" /></figure><h3><strong>On the next episode</strong></h3><p>The fact that header bidding has gained momentum is unquestioned. The Getintent research findings prove the technology continues to grow and evolve, the number of players increases, while its profile across the programmatic landscape gets more diverse.</p><p>Even though a lot of publishers still rely on other vendors’ solutions, more publishers start building their own wrappers, usually on top of the open-source container script. For example, PubMatic and AOL solutions are built on the Prebid.js architecture.</p><p>By bringing a unified auction to the web page header, header bidding has given publishers more control over their ad inventory and ability to optimize yield. As more publishers recognize the benefits, the technology expands and continues to be modified. Programmatic players look for the integrations both functional and efficient across all formats and devices, so we are going to see even greater offering of header bidding solutions.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ca25303b6186" width="1" height="1" alt=""><hr><p><a href="https://medium.com/getintent/making-headway-header-bidding-research-2-0-ca25303b6186">Making Headway. Header Bidding Research 2.0</a> was originally published in <a href="https://medium.com/getintent">Getintent</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Ads.txt Research: Not a Big Clean-up Yet]]></title>
            <link>https://medium.com/getintent/the-ads-txt-research-not-a-big-clean-up-yet-96cf138a5e85?source=rss-11e1dd11d1aa------2</link>
            <guid isPermaLink="false">https://medium.com/p/96cf138a5e85</guid>
            <category><![CDATA[advertising]]></category>
            <category><![CDATA[digital]]></category>
            <category><![CDATA[programmatic]]></category>
            <category><![CDATA[marketing]]></category>
            <category><![CDATA[tech]]></category>
            <dc:creator><![CDATA[Getintent]]></dc:creator>
            <pubDate>Tue, 15 Aug 2017 13:47:43 GMT</pubDate>
            <atom:updated>2017-08-15T13:47:43.212Z</atom:updated>
            <content:encoded><![CDATA[<h3>The State of Ads.txt: Not a Big Clean-up Yet</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vIhMQKFfhXzni_c35qb9-g.png" /><figcaption>Photo credit: <a href="http://foter.com/re/a4eb30">Foter.com</a></figcaption></figure><p>Ad tech is not immaculate, let’s face it. The ecosystem is inhabited by bad actors and digital transactions are intervened by shady intermediaries. Tricky fraudsters like Methbot generate invalid traffic and get away with it.</p><p>Programmatic has long lacked a mechanism for buyers to verify whether a supplier is legitimate to sell a publisher’s ad inventory. On the open exchange, buyers think they’re buying inventory from reputable publishers like NBCUniversal but often end up purchasing from impostor sites instead. As a result — the advertiser’s money wasted, the brand image misrepresented, the publisher blacklisted, and the credit to programmatic undermined.</p><h3><strong>Magic bullet</strong></h3><p>To address this big problem of domain spoofing — that’s the name for the worst-case scenario given above, — the IAB Tech Lab got inspired by the <em>robots.txt</em> and has come up with an <em>ads.txt</em> standard. It’s a publicly available text file where a publisher lists partners authorized to sell and resell its digital inventory.</p><p>The idea is simple and its implementation requires no extra development efforts. Publishers simply make a list of their account IDs on the SSPs, networks, and exchanges they work with and upload this text file to their root domain. Buyers can then crawl the web for those lists to check the validity of the inventory they purchase. DSPs can set filters to assure they transact with the legitimate exchanges.</p><p>After the public comment period the final <em>ads.txt</em> spec was released on June 27th and is now ready for wide adoption. The standard bodes unprecedented transparency, and the programmatic community — marketers, publishers and adtech vendors — have sounded strongly supportive of the initiative so far. But how many of them are really ready to adopt it?</p><h3><strong>Headcount</strong></h3><p>Getintent has done a research to check how many publishers have already hopped aboard the <em>ads.txt</em> bandwagon. The findings show it’s early to speak about widespread implementation. It’s been a month since the specification was released, and as of July 26th we have identified only 13 websites among top 1,000 domains on Getintent’s available inventory that have put up <em>ads.txt</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YjKqyfdXEDz1xkvTX9_j0w.jpeg" /></figure><p>Interestingly, only two out of <a href="https://adexchanger.com/publishers/top-10-programmatic-publishers/">top ten programmatic publishers</a> — Business Insider and The Washington Post — are on the list of the early adopters. Another big-league publisher that has joined the transparency club is The New York Times. The rest of the pioneers come mostly from the entertainment legion.</p><p>Since the text file lists identifiers associated both with seller and reseller accounts within the advertising system, now it’s publicly available how much of the publisher’s ad space is being resold via authorized partners.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WVOwk613taoDqoEIk4PBeg.jpeg" /></figure><p>The most quoted adtech companies on the text files are Index Exchange, AppNexus, and OpenX. Others include AOL, Pubmatic, Rubicon Project, Sovrn, Taboola, Teads, TripleLift, and Yieldmo. Many of the publishers partner with Amazon, Google, and Facebook.</p><h3>Why so humble?</h3><p>It seems like everyone should benefit from the initiative. It promises a more trusted marketplace and should reassure the advertisers who have steered clear from programmatic over brand safety concerns. Moreover, weeding out unauthorized sellers adds value to the supply chain, which means higher conversions for advertisers and bigger revenue for publishers.</p><p>However, there’re still some concerns. The current spec fails to cover the whole supply chain. First of all, <em>ads.txt</em> doesn’t specify the authorized ad format yet. This leaves a loophole for an arbitrage scheme where display inventory is disguised as video inventory. Secondly, the standard is not applicable to mobile apps, anonymous inventory or syndicated content. Finally, there are a lot of influential players who rely greatly on undisclosed reselling and are unlikely to give up on arbitrage as one of the pillars for their business.</p><p>Whether <em>ads.txt</em> will take over depends on the network effect. The digital industry recognizes that the standard will work only when adopted collectively. So, we’ll probably see more peer pressure put on the programmatic players to join the game.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=96cf138a5e85" width="1" height="1" alt=""><hr><p><a href="https://medium.com/getintent/the-ads-txt-research-not-a-big-clean-up-yet-96cf138a5e85">The Ads.txt Research: Not a Big Clean-up Yet</a> was originally published in <a href="https://medium.com/getintent">Getintent</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What’s Behind the Buzzword of ‘Programmatic’]]></title>
            <link>https://medium.com/getintent/whats-behind-the-buzzword-of-programmatic-d97e8aea61db?source=rss-11e1dd11d1aa------2</link>
            <guid isPermaLink="false">https://medium.com/p/d97e8aea61db</guid>
            <category><![CDATA[advertising]]></category>
            <category><![CDATA[marketing]]></category>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[programmatic]]></category>
            <category><![CDATA[digital]]></category>
            <dc:creator><![CDATA[Getintent]]></dc:creator>
            <pubDate>Fri, 24 Mar 2017 06:02:00 GMT</pubDate>
            <atom:updated>2017-03-24T09:00:46.429Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/791/1*POe0pdQAVnUQiAohEGyC2A.jpeg" /><figcaption><em>Photo credit: </em><a href="https://www.flickr.com/photos/wwarby/11457279916/"><em>wwarby</em></a><em> via </em><a href="http://foter.com/re/b091c6"><em>Foter.com</em></a><em> / </em><a href="http://creativecommons.org/licenses/by/2.0/"><em>CC BY</em></a></figcaption></figure><p>Have you ever caught yourself thinking that your Instagram seems to know something about you? It must have overheard your new year resolution to switch to a healthy lifestyle, you think, and now reminds you of that by offering fresh food delivery. Meanwhile on Twitter, another promoted tweet from a sportswear brand pops up. And even when you’re jogging, that motivating <em>Just do it</em> billboard in the local park seems to be speaking to you.</p><p>At such a moment you get that anxious feeling of Big Brother watching you. Yet, on the other side, you don’t mind being offered something that is <em>just right</em> for you, because it’s actually saves your time and searching efforts.</p><p>Speaking about online searches, before you even click, Google tailors the content based on the data it gathers about you — your name, gender, location, device, and browsing history. Facebook has an algorithm that, by measuring what you click on most, tailors your feed to what it thinks you like. And it’s not only about Facebook and Google. The same is true for any web source you browse online — The Washington Post, or Amazon, or even Netflix.</p><blockquote>These days you can hardly find content online that has not been tailored specifically for you. We’re living in the era when algorithms decide what you should see. We’re living in the digital environment that is becoming programmatic.</blockquote><p>Love it or hate it, programmatic is everywhere.<strong> </strong>Even though overused, the buzzword from the advertising industry is still confusing for most people. But this blog post will show that programmatic is no rocket science and can be explained even to your granny.</p><h3><strong>Programmatic 1.0</strong></h3><p>Proclaimed by the media ‘the future of digital advertising’, programmatic is an algorithmic technology that automates and optimizes media buying process. It all started from buying and selling banner ads online more than 13 years ago. But by now programmatic has evolved to include desktop and mobile display ads, video, audio, TV, and even out-of-home. See? That overlooking billboard on your jogging route wasn’t an accidental thing, but a carefully measured marketing campaign with a locationally pinned down brand message.</p><p>So how does programmatic work? Let’s start with defining the ad space and its key players.</p><p>The programmatic ecosystem encompasses the following constituents:</p><ul><li><strong>Ad inventory</strong> — real estate to show ads;</li><li><strong>Ad networks </strong>— groups of several like-minded publishers (media owners);</li><li><strong>Ad exchanges</strong> — a ‘stock exchange’ where publishers and advertisers connect for an auction;</li><li><strong>Data management platforms (DMPs)</strong> — data warehouses that collect multiple user data (cookies, search engine data, on-site data, CRM data, mobile device IDs) and create relevant audience segments;</li><li><strong>Demand-side platforms (DSPs)</strong> — buying platforms (software) that help advertisers automatically purchase ad inventory;</li><li><strong>Supply-side platforms (SSPs)</strong> — selling platforms used by publishers to monetize their ad inventory;</li><li><strong>Trading desks</strong> — layered on top of DSPs, management platforms used by ad agencies.</li></ul><p>This network rests on the paradigm that data is the most valuable asset and is governed by the principle of real-time bidding (RTB) — a computer run auction whereby advertisers compete against each other online over the opportunity to serve ads to a specific target audience based on the user cookies.</p><p>How does the process go? Publishers generate content — such as news, music, or other information or entertainment — to draw an audience. With money in mind, publishers sell their ad space to advertisers who want to reach that audience. When you visit a webpage, the browser connects to the publisher ad server that, guided by built-in logic, chooses what content to show.</p><p>If the ad space isn’t reserved for any specific advertiser via <em>direct deals</em> (previously negotiated with the sales team), the ad server considers what it knows about you to decide what advertisers on the <em>private marketplace</em> (a select number of invited advertisers) might be interested in the opportunity to show you their advertisements.</p><p>If there’s no demand either through direct or private deals, the publisher ad server may decide to put the opportunity out on the open ad market. To do so, it connects to a supply-side platform. The SSP, in its turn, checks whether it has seen you before, requests additional information from its data provider (DMP), and then sends the ad request to an ad exchange.</p><p>The ad exchange is a place where potential buyers — demand-side platforms (DSPs), ad networks and other exchanges — gather to make precast bids. Just like Wall Street stockbrokers, they call out that, for example, they will pay a dollar per thousand ads displayed to males aged from 18 to 35 years old living in the Pacific Northwest — the segment you fit well into.</p><p>If there are no precast bids, the exchange offers its partners to show an ad to the audience like you and asks them to come back with their bids within 10 milliseconds. The buyers respond and the exchange selects a winning bid — the one that is willing to pay the highest price.</p><p>Then the winning DSP tells the exchange to retrieve the ad, the exchange sends the request to the SSP, the SSP passes the instructions to the publisher ad server, and finally the latter tells your browser to retrieve the ad from the agency ad server — ta da! The web browser shows you the ad that most appropriately matches your preferences. The entire process happens before the web page loads and takes up about 100 milliseconds, which is three times quicker than you blink!</p><blockquote>To put the whole story even shorter, programmatic is about marketing, data, and technology. The technology leverages big data sourced from your web surfing history and gleaned with the information you volunteered on various surveys or websites you’ve subscribed to — all for the sake of serving you an advertisement that is most relevant to you.</blockquote><p>So, in case your granny asks about programmatic, tell her it’s a computer technology that displays a message saying that she might as well like <em>The Game of Thrones Cookbook</em> after she has just finished watching the latest episode of the same-name series.</p><h3><strong>Why programmatic matters</strong></h3><p>You don’t have to be a marketing guru to understand that with billions of websites it’s extremely tough for advertisers to get the right people’s attention. In the old days of online advertising, buying ads looked pretty much like playing <em>Where’s Wally?</em> game: advertisers couldn’t tell where their customer was and how their brand message performed.</p><p>To serve ads on a publisher’s website, advertisers had to buy ad inventory directly, so they had to contact the publisher. This involved negotiating details like ad sizes, time, and rates. What is more, the ad campaigns had to be launched and run across sites one by one, and advertisers couldn’t track performance on a single platform. It was a sluggish and pricey process dependent on human interfacing and entailing a lot of manual operations.</p><blockquote>The advent of programmatic has transformed the industry. Today we interact with brands on different devices and via different channels. Programmatic successfully lives up to this shift in customer communication: the technology shows brand messages that are most relevant for the customer’s stage of the journey across screens.</blockquote><p>For advertisers, programmatic is a powerful technology that enables enormous audience reach, precise targeting, and one-on-one engagement. Marketers finally get the ability to deliver a customized message to the right person at the right moment. What is more, the technology consolidates user interactions across multiple channels into a single dashboard, enabling a 360-degree view of the target audience. As a result, brands and agencies get a more complete view of the customer behaviors and preferences.</p><p>To crown it all, programmatic helps advertisers get more value for each dollar spent, while publishers get real benefits from their audience data. Advantageous for both publishers and brands, programmatic also keeps customers happily engaged with the brands that excite them. To make ad campaigns even more relevant and resonant, advertisers are exploring interactive and innovative formats like audio, video, virtual and augmented reality to engage us. Thus, programmatic turns once irrelevant, obtrusive advertising into compelling art.</p><p>Having recognized the prospects programmatic brings to the ad tech industry, forward-thinkers have already invested heavily in the technology. According to <a href="http://www.dnb.com/content/dam/english/dnb-solutions/sales-and-marketing/dnb-adweek-report-2017.pdf">Dun &amp; Bradstreet Report</a>, almost 70% of the U.S. B2B marketers plan to increase programmatic ad spend in 2017. And as programmatic is becoming mainstream, the buzz around the term won’t come to rest.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d97e8aea61db" width="1" height="1" alt=""><hr><p><a href="https://medium.com/getintent/whats-behind-the-buzzword-of-programmatic-d97e8aea61db">What’s Behind the Buzzword of ‘Programmatic’</a> was originally published in <a href="https://medium.com/getintent">Getintent</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Scraping Through Wrappers: Overview of Header Bidding Solutions Implemented by Publishers]]></title>
            <link>https://medium.com/getintent/scraping-through-wrappers-overview-of-header-bidding-solutions-implemented-by-publishers-1007bff01780?source=rss-11e1dd11d1aa------2</link>
            <guid isPermaLink="false">https://medium.com/p/1007bff01780</guid>
            <category><![CDATA[programmatic]]></category>
            <category><![CDATA[advertising]]></category>
            <category><![CDATA[digital]]></category>
            <category><![CDATA[marketing]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Getintent]]></dc:creator>
            <pubDate>Thu, 16 Mar 2017 12:42:04 GMT</pubDate>
            <atom:updated>2017-09-03T13:07:02.002Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Scraping Through Wrappers: Getintent Researches Header Bidding Solutions</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q6HhyOSGIXiPfnqn_wdC8Q.jpeg" /><figcaption><em>Photo credit: </em><a href="https://www.flickr.com/photos/ntr23/4545972659/"><em>ntr23</em></a><em> via </em><a href="http://foter.com/"><em>Foter.com</em></a><em> / </em><a href="http://creativecommons.org/licenses/by-nc-sa/2.0/"><em>CC BY-NC-SA</em></a></figcaption></figure><p>The year of 2016 was marked by the rise of header bidding over the digital landscape. Media have been brimming with headlines dubbing the technology a driveway to the promised land for publishers. The latter one by one have been reporting maximized yields from adopting header bidding.</p><p>Curious about what figures back up such claims, Getintent, a New York based ad tech company developing programmatic solutions, embarked on its own research.</p><h3><strong>Setting</strong></h3><p>“We wrote a PhantomJS scraper that is able to identify header bidding technology used on the website and see what demand sources the page addresses when loading,” said Vladimir Klimontovich, CTO at Getintent.</p><p>“For the sake of research, we scraped through Alexa top 1,000 websites to analyze how many publishers have shifted to header bidding and what solutions — specifically, wrappers and adapters — they are implementing,” Klimontovich explained.</p><p>In a nutshell, omitting the details of <a href="https://medium.com/@GetIntent/header-bidding-disruptive-innovation-most-publishers-welcome-11f3e745c34e#.7uwi1bvfv">how header bidding works</a>, wrappers, or containers, help publishers manage multiple header bidding solutions. A <em>wrapper</em> gathers bids from demand sources and implants them directly into a publisher’s ad server to compete with the bids from direct sales partners. Inside the wrapper, there are <em>adapters</em> that translate bids into a common key value for the ad server.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7RKIVSay2PYqjGHamsq5iA.jpeg" /></figure><p>As a result of the scraping process, header bidding solutions were spotted on 121 unique web pages out of Alexa top 1,000 websites, which makes 12%.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l_VWxPr_uglgsOrA5s1xog.jpeg" /></figure><h3><strong>Wrappers</strong></h3><p>The research revealed that among these 121 publishers there are 164 cases of header bidding solution usage. The most commonly used header bidding solutions are those provided by Criteo (in 69 cases, which is approx. 42.1%) and Amazon (in 40 cases, which makes up 24.4%). In 28 cases (17.1%) publishers use the open-source and free Prebid.js container. Not without reason, some publishers take advantage of using several wrappers together.</p><p>According to Getintent, if the scraper observed page calls for adapters but was not able to identify the wrapper, it’s highly probable that the publisher is using some custom solution. Thus, in the rest 27 cases (16.5%) publishers fall back on self-made containers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*L1TCnGak7gn8SDSN55lbdQ.jpeg" /></figure><p>“Some publishers make their own header bidding containers,” commented Maxim Kraynov, Lead Software Engineer at Getintent responsible for the research. “Sometimes it’s a fully proprietary technology, sometimes it is a wrapper built upon some already existing container,” he specified.</p><p>“Interestingly, the recognized leaders in the header bidding domain, Amazon and Criteo, are not wrappers per se,” Kraynov added. “Our scraper managed to detect the Amazon technology as a container, but the company itself doesn’t label its header bidding product a wrapper, because it’s a cloud-based solution. Similarly, the Criteo technology is not a container as such. Nevertheless, for simplicity reasons we call such an approach header bidding.”</p><h3><strong>Adapters</strong></h3><p>As part of the research Getintent was able to review what bidder adapters are implemented by the publishers for Prebid.js.</p><p>“It should be noted that, a cloud-based solution, Amazon has no visible adapters,” Maxim Kraynov highlighted. “Neither does Criteo, as in this case the publisher embeds the code directly onto the page, so there’s no need for an adapter.”</p><p>The scraper spotted 126 cases of adapter usage on the selected 121 websites, which verifies that some publishers leverage technologies of multiple vendors simultaneously.</p><p>As for the tech providers, it turned out that the AppNexus solution is most common (approx. 24%), followed with the Rubicon, Sonobi, and Sovrn adapters (approx. 10% each).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aUZZDCzzXMmZSF3g-N0w9Q.jpeg" /></figure><h3><strong>Summary</strong></h3><p>“The research shows that the header bidding market has become more diverse and competitive with vendors coming up with browser-based, server-side, and cloud products,” summed up the analysis Vladimir Klimontovich. “The Criteo and Amazon solutions (that aren’t actually wrappers) are currently most favored with publishers. Yet, we can’t say for sure which solutions will gain mass adoption,” he commented.</p><p>The Getintent analysis has also shown that despite the reportedly growing number of publishers adopting header bidding, most publishers (more than 80% of Alexa top 1,000 websites) are still to discover the game-changing technology.</p><p>“The scraper is not a faultless tool though, and the findings don’t claim a monopoly of absolute accuracy,” Klimontovich remarked. “Nevertheless, the Getintent research results are not a far cry from the data reported by similar studies.”</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1007bff01780" width="1" height="1" alt=""><hr><p><a href="https://medium.com/getintent/scraping-through-wrappers-overview-of-header-bidding-solutions-implemented-by-publishers-1007bff01780">Scraping Through Wrappers: Overview of Header Bidding Solutions Implemented by Publishers</a> was originally published in <a href="https://medium.com/getintent">Getintent</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Header Bidding: Disruptive Innovation Most Publishers Welcome]]></title>
            <link>https://medium.com/getintent/header-bidding-disruptive-innovation-most-publishers-welcome-11f3e745c34e?source=rss-11e1dd11d1aa------2</link>
            <guid isPermaLink="false">https://medium.com/p/11f3e745c34e</guid>
            <category><![CDATA[digital]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[advertising]]></category>
            <category><![CDATA[adtech]]></category>
            <category><![CDATA[programmatic]]></category>
            <dc:creator><![CDATA[Getintent]]></dc:creator>
            <pubDate>Fri, 03 Feb 2017 14:01:05 GMT</pubDate>
            <atom:updated>2017-02-03T14:01:05.348Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/873/1*qumuYCKD_z3dTLTc2JOm5Q.jpeg" /><figcaption>Photo credit: <a href="https://www.flickr.com/photos/brieuc/4225881624/">brieuc_s</a> via <a href="http://foter.com/">Foter.com</a> / <a href="http://creativecommons.org/licenses/by/2.0/">CC BY</a></figcaption></figure><p>Header bidding has created quite a buzz in the advertising industry recently. But it was as far back as 2009 when the technology first showed up in the house of programmatic. The solution was originally conceived as an alternative to bypass the waterfall auction, but it’s becoming a common practice among publishers. Why are publishers rapidly shifting to header bidding, and what ‘waterfalls’ are we talking about?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/568/0*x-45EA-_MwfalONm." /></figure><h4><strong>It all started with a waterfall</strong></h4><p>Suppose, you’re a publisher that needs to monetize the ad space on its website. To find buyers, first of all, you refer to your ad server for the direct-sold ads. Then, you’ll most likely push the rest of the inventory to open exchanges. And here’s where the ‘waterfall’ starts.</p><p>The unsold ad space is offered to demand partners according to their buying volume, so that the top-rated ones can cherry-pick the best spots before others have even a chance to bid. If unsold, the impression is passed on to the next lower-ranked exchange until it is finally sold. Thus the big league buyers are privileged while the lower-ranked bidders — even those who would have paid more — are left on the sidelines.</p><p>Such a buying model is a three-way deadlock. For demand-side platforms, a publisher partnering with DoubleClick for Publishers (Google’s ad server) means Ad Exchange (Google’s real-time marketplace) will always outbid any other competitor. As a publisher, going from one advertiser to another and losing impressions, you miss out on a lot of programmatic revenue. Neither is user left unaffected: the longer the waterfall is, the slower the page loads.</p><p>This is why the waterfall auction in a Google dominated ecosystem has been proclaimed complicated, inefficient, and short on revenue. Nevertheless, it has long been a standard way of buying and selling ads. Until developers came up with a header bidding solution.</p><h4>How header bidding works its magic</h4><p>The header bidding gimmick lies in a piece of JavaScript code embedded in the publisher’s website header (hence the name). When a user visits the website and the page is loading, before calling the ad server for direct sales, the publisher refers to demand-side platforms for bids. The demand source — SSP, DSP, or ad exchange — gets a glimpse of the page inventory and sends back a dynamic bid with a price a buyer is willing to pay for a specific impression. If the value in the bid corresponds with the publisher’s preset requirements, the network with the highest bid wins and the ad is served up. The mechanism sounds complex, but the whole process hardly takes 500 milliseconds.</p><p>So what’s the essential difference between the waterfall sequence and header bidding? First of all, in contrast to the waterfall auction, header bidding allows simultaneous calls to multiple demand sources. Secondly, the latter are bidding against each other and sometimes even against the winner from the direct sales. Finally, demand side platforms are bidding for all available impressions, not just the leftovers.</p><h4><strong>How header bidding reshapes ad buying</strong></h4><p>Disclosing ad opportunities to multiple buyers at a time, publishers can see the true value of their inventory, estimate the demand and maximize revenue. Advertisers, in their turn, get better access to the right inventory and enjoy greater transparency. The solution also puts a stop to Google’s monopoly.</p><p>As a result, header bidding creates a competitive environment for guaranteed, RTB buyers, and exchanges, levelling the playing field for both demand sources and publishers while increasing the yield for the latter. Last but not least, users get ads that mean something to them.</p><p>Looks like a happy publisher-advertiser-consumer engagement. No wonder <a href="http://www.businessinsider.com/header-bidding-gains-momentum-drives-up-publisher-ad-revenue-2016-5">up to 70% publishers</a> have already switched to header bidding.</p><p>MailOnline, CNN International, Trinity Mirror have all claimed their programmatic revenues have risen with the shift to header bidding. The Telegraph has increased its programmatic ad team fourfold due to a 20-percent boost of its programmatic revenue. Some publishers mix approaches selling impressions through header bidding in the first place, while resorting to the waterfall as a Plan B.</p><p>The daisy chain days are not gone yet with colossuses like The Financial Times, The Economist, and BBC abstaining from header bidding. Because however advantageous, the solution comes with its drawbacks.</p><h4><strong>The flipside of header bidding</strong></h4><p>A major concern that keeps some publishers away from adopting header bidding is that stuffing the page with multiple header bidding tags means greater risk of latency. With a slower page load, the number of impressions delivered decreases, viewability and user experience are hindered.</p><p>As a workaround for this issue some publishers reduce the number of partners or implement timeout thresholds to weed out slowpokes that fail to file the bid within the set time limit. Some publishers move to a single-request architecture, wherein every header bidder gets one request.</p><p>Another solution to the latency woe developers have come up with is a header bidding container, aka framework, or a wrapper.</p><h4>Unwrapping the wrapper</h4><p>Header bidding is not a plug-and-play gizmo: implementing the solution and managing multiple partners involves technical and ad operations resources. Every new partner a publisher takes on means adding extra code, whereas not all header bidding technologies perform smoothly together. Integration, adjustment, measurement — all that comes as headache for ad ops and engineering teams. Especially, when the publisher is a large media company with dozens of properties.</p><p>Wrappers were designed to solve these challenges. Instead of embedding a separate tag for each demand source, a container tag — another string of JavaScript — is placed in the header, so that publishers can update header tags without changing the website code. Containers can include tags to track analytics and viewability or set auction timeout thresholds. As a result, apart from reducing latency, wrappers enhance overall security and transparency. Altogether this facilitates the implementation of the technology at scale.</p><p>As more publishers embrace header bidding, more tech companies develop their own frameworks. Some frameworks are open-source, which means a publisher can adjust or upgrade them, but with no support provided. Some vendors charge a fee for their wrappers, some ad tech companies offer customizable out-of-the-box solutions.</p><p>Time Inc. was the first publisher to report using a header bidding wrapper across its websites including People, Sports Illustrated and Fortune. In December 2016 Amazon announced the launch of a cloud-based wrapper. Rather than handle all ad requests in the users’ browsers, the solution will move the auction to the cloud, which means less lag time loading web pages. AOL, another mass media giant, is also building its own wrapper.</p><h4><strong>Where header bidding is heading</strong></h4><p>Originally perceived as a temporary fix to the daisy chains, header bidding has quickly evolved into a must-have tool for publishers. Promoting a unified auction in a transparent and competitive marketplace, the solution offers a refined ad buying model.</p><p>The technology is changing the industry. However, it might be facing an existential challenge as more publishers start experimenting with server-to-server solutions for their video ads. Skeptics assume that server-to-server connection is actually killing the idea header bidding was primarily intended for.</p><p>Whatever the future holds for header bidding, with the disruption it has brought about, digital advertising will never be the same.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=11f3e745c34e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/getintent/header-bidding-disruptive-innovation-most-publishers-welcome-11f3e745c34e">Header Bidding: Disruptive Innovation Most Publishers Welcome</a> was originally published in <a href="https://medium.com/getintent">Getintent</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Docker to the Rescue: Writing Lightweight Integration Tests]]></title>
            <link>https://medium.com/getintent/docker-to-the-rescue-writing-lightweight-integration-tests-f4665c8847aa?source=rss-11e1dd11d1aa------2</link>
            <guid isPermaLink="false">https://medium.com/p/f4665c8847aa</guid>
            <category><![CDATA[testing]]></category>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Getintent]]></dc:creator>
            <pubDate>Fri, 13 Jan 2017 06:02:01 GMT</pubDate>
            <atom:updated>2017-01-13T06:02:01.536Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1011/1*z3cpxD40E7etXN74lugk9g.jpeg" /><figcaption>Photo credit: <a href="https://www.flickr.com/photos/134416355@N07/31775353301/">maijou2501</a> via <a href="http://foter.com/">Foter.com</a> / <a href="http://creativecommons.org/licenses/by-sa/2.0/">CC BY-SA</a></figcaption></figure><p>To show your ads properly, we at <a href="http://getintent.com/">GetIntent</a> ensure that our ad platform is reliable and trouble-proof. The reliability of the system depends on such things as the hardware type, system or network configuration, and application architecture.</p><p>Introducing changes to complex, distributed applications always entails some risk. To minimize these risks, developers write unit and integration tests. While there’s usually no difficulty writing unit tests, creating integration tests, especially sophisticated ones, can be rather demanding.</p><p><strong>When challenges arise</strong></p><p>Using Tomcat or Jetty for testing is no problem since, written in Java, these servers can be easily embedded. But dealing with <a href="http://www.aerospike.com/">Aerospike</a>, which is our case, the database integration testing implies the following challenges:</p><ul><li>Not written in Java, Aerospike can’t be integrated into our application effortlessly.</li><li>It is desirable that a developer run tests on all popular platforms including Windows, OS X, or Linux. Aerospike, however, provides binaries for Linux only.</li><li>Tests can be run synchronously, so do we need more servers?</li><li>For each test there should be a separate database instance.</li></ul><p>In such a case, to run integration tests, shared test environment (e.g. a remote server with a configured Aerospike) is often used. Nonetheless, this approach has certain drawbacks:</p><ul><li>Running tests on a remote server takes more time than running a local test. This is particularly noticeable when dealing with slow internet.</li><li>There might be data isolation issues in case several developers decide to test the application simultaneously.</li><li>Quite often, to comply with the information security policy, a company shall hide the server behind the corporate VPN. The idea of setting up a VPN client doesn’t sound encouraging when you are working from home, does it?</li></ul><p>An alternative to this method would be preparing the local environment for testing, writing long instructions on installing and configuring Aerospike for Windows, Mac, and Linux. But there is another option — using automation tools, such as Docker.</p><p><strong>Docker comes to help</strong></p><p>Docker is a system used to deploy and manage applications in an isolated environment (containers). It is built on the client server architecture principles, with Docker Client available for all major operating systems and Docker Daemon run only on Linux. But the latter is no trouble: using Docker Machine you can run Docker on Windows or OS X as well, though on a virtual host.</p><p>So, to start the test, a developer needs a machine with a configured Docker Client, which is checked with a command: <em>docker run hello-world</em>. For Windows and OS X, <a href="https://docs.docker.com/">Docker Machine</a> should be installed.</p><p><strong>Embedded Aerospike</strong></p><p>To use Aerospike in integration tests, we created <a href="https://github.com/getintent/embedded-aerospike">a wrapper for Docker and Docker Machine</a> that can:</p><ul><li>Start and stop containers;</li><li>Mount the configuration file aerospike.conf inside the container;</li><li>Bind a container port to the free host port;</li><li>Start and stop Docker Machine if testing is run on Windows or OS X.</li></ul><p>To manage the containers, we use a popular <a href="https://github.com/docker-java/docker-java">Java Docker API Client</a>. To start an Aerospike server within a Docker container, to configure port forwarding and mount the configuration file, we use the following command:</p><pre>docker run -d -P -p 3000:3000 -v<br>path/to/aerospike.conf:/etc/aerospike/aerospike.conf — name aerospike aerospike</pre><p>And here is the code to do the same thing using Docker Remote Api:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/03341b391a8e87b30975dbc1e1f2639b/href">https://medium.com/media/03341b391a8e87b30975dbc1e1f2639b/href</a></iframe><p>More details can be found in the <a href="https://github.com/getintent/embedded-aerospike/blob/master/src/main/java/com/getintent/aerospike/embedded/AerospikeServer.java">AerospikeServer</a> class.</p><p><strong>Writing a test</strong></p><p>Let’s take an integration test using the embedded Aerospike as an example. Suppose we have SimpleAerospikeClient class that is able to store and retrieve user segments by ID.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ec971df8df3e40002de0a6ab832ad769/href">https://medium.com/media/ec971df8df3e40002de0a6ab832ad769/href</a></iframe><p><a href="https://github.com/getintent/embedded-aerospike/blob/master/src/test/java/com/getintent/aerospike/client/SimpleAerospikeClient.java">View</a> the full class.</p><p>To write a test that will check the proper implementation of these methods, we need to set up and start the server first:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/179d21415199f4d42a11677b4a8beb46/href">https://medium.com/media/179d21415199f4d42a11677b4a8beb46/href</a></iframe><p>Check that the data is written to and read from the database correctly.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/793e82f1e72f0cd26be7663393cd8cb2/href">https://medium.com/media/793e82f1e72f0cd26be7663393cd8cb2/href</a></iframe><p>Do not forget to stop and remove the containers created during the tests.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7edaf81a5536df8723c2b68613e5854c/href">https://medium.com/media/7edaf81a5536df8723c2b68613e5854c/href</a></iframe><p><strong>Conclusion</strong></p><p>Such an approach allows running integration tests on a developer’s machine, which, in the first place, means no maintenance spending on test servers. Secondly, you can develop and test applications without access to the infrastructure.</p><p>We have reviewed a case with Aerospike, but integration tests for other services can be run in a similar way.</p><p><strong>UPD</strong></p><p>As a more recent and optimized way of writing integration tests we would also recommend paying attention to <a href="https://github.com/testcontainers/testcontainers-java">testcontainers-java</a>. It’s a Java 8 library that supports JUnit tests providing lightweight, throwaway instances of common databases and Selenium web browsers.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f4665c8847aa" width="1" height="1" alt=""><hr><p><a href="https://medium.com/getintent/docker-to-the-rescue-writing-lightweight-integration-tests-f4665c8847aa">Docker to the Rescue: Writing Lightweight Integration Tests</a> was originally published in <a href="https://medium.com/getintent">Getintent</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>