After an awesome three years at Uber that I will forever be grateful for, I have just started a new job leading the Product, Data Science and Design orgs at KeepTruckin. In the course of deciding upon this move, I have learnt more than I ever expected to know about the long-haul trucking space and would love to share why I made this choice.

Without exaggeration, trucking is the life-blood of our modern economy. Trucks move over 70% of the nation’s freight by tonnage, moving nearly 11 billion tons of freight annually. This represents 200+ pounds per person per day! While trucks represent 10% of miles traveled on our highways, given that an average truck runs at 6 mpg as opposed to 18+mpg for a car, they also represent a disproportionate share of fuel consumption in our country. Finally, trucking is a critical source of employment for over 3.6 million truckers, and more than 8 million folks who support the long-haul industry.

For various structural reasons that I won’t go into in this post, the long-haul freight industry is highly fragmented with thousands of small fleets. Due to this fragmentation, the ability to create a holistic picture and deeply understand how the industry operates has been limited.

Enter the ELD Mandate. Trucking has always operated with a set of safety rules around hours of service and other aspects that ensure drivers have enough rest when they are on the road and can drive safely at all times. Historically, this has been self-reported on paper. But at the end of 2017, a new mandate required the use of an Electronic Log Device to collect moving times and other data from truck onboard systems. Now, most trucks have such a device installed and by the end of next year, all trucks built within the last 20 years will have one as well.

This is a crucial opportunity for the freight industry as a whole. Rich data from these devices gives us our deepest insight ever into trucking. Simultaneously, the progress in Machine Learning allows us to extract maximal optimization from this data.

What does this mean in terms of practical applications? We can use this data to:

• Drive improvements in safety, saving lives while driving insurance costs down
• Maintain vehicle health and reduce fuel consumption
• Improve planning and routing, saving time and fuel
• Reduce the number of miles driven empty (either going to pick up a load or coming back from dropping off a load)
• and many, many more…

All of these translate to ->

1. Reduced accident rates on the highways
2. Lower costs for shipping which translates to cheaper goods for all of us
3. Improved pay and lifestyle for truckers — one of the hardest working professions in the country

Talk about an exciting mission!

KeepTruckin saw this opportunity and built a modern Electronic Log Device (ELD) to serve the need of truckers as the mandate came into effect. Over the course of the last two years, it has established itself as the leading player in the space of Fleet Management. In fact, it is the most popular device amongst the small to mid sized fleets that dominate trucking. At this point, KeepTruckin ELDs are installed on hundreds of thousands of trucks across the United States.

KeepTruckin has also continuously expanded its offering — with cameras for tracking safety, fleet visibility to operators, document transfer and many other capabilities for fleet management.

The end result is one of the most comprehensive and rich datasets on how trucks operate in the United States.

Our goal is to solve all the problems I discussed above. Our current (and growing) footprint in the trucking industry means that we are incredibly well positioned to do so with richer data than any other participant in the trucking ecosystem.

I’m super excited to work on such a rewarding set of problems in a company that is leading the revolution at an opportune moment in the history of trucking.

Made it all the way through? Still interested? We’re hiring in every possible role — including engineering, data science, product management and others. Reach out to me!

For more interesting statistics on freight transportation, go here.

Over the last year and a half, I’ve been immersed in performance marketing as part of my role in building out the tech organization that supports our performance marketers at Uber. As a neophyte in the area, the whole experience has been very educational. The supply chain for marketing is endlessly interesting given both the rapid evolution in the space and the parallels one can draw with the stock market. But that is a story for another day.

Today, I want to share some lessons learned on how to think about valuing money spent on performance marketing.

Early acknowledgements — Kim Larsen, Kevin Frisch and Irfan Ganchi have been my patient tutors on the topic during my time at Uber. Apologies to them if I misrepresent their wisdom in anyway. Also, this will not be focused on what we do at Uber and is intentionally meant to be a high level overview rather than anything specific to my day job.

Establishing Goals — What are we trying to accomplish?

Before talking about analytics and measurement, we need to understand the key business decisions we wish to drive from a deeper understanding of our marketing spend.

There are three key levels of decision making that require support from attribution models.

• Marketing Budget Allocation and Growth Projections

At the highest level, each dollar you spend on marketing should drive incremental LTV for your business. The meta question each CFO needs to ask is what is the right level of budget allocation for marketing given a desired growth goal

This happens at various time grains — annual budget processes, as well as quarterly decisions. In general, due to planning considerations, marketers rarely try to make budget level decisions at smaller than quarterly grains.

• Per Channel / Channel Group Budget Allocation:

A marketing machine that leverages multiple channels needs to decide how much money to spend on each channel (Facebook, Google, Amazon, Display networks, etc).

Channels can differ wildly in performance based on the type of product on offer as well as other limiting attributes (such as scale efficiency, intent, etc). The goal of every marketer is to invest in channels so that each channel is producing at the same level of efficiency.

• Intra-Channel Optimization:

At the most granular levels, channel managers have to make weekly and daily decisions on how best to spend their money within their channel. There are many decisions being made constantly, both within a specific channel and across channels (for instance, creative selection, audience selection, etc) that drives optimization at the individual channel level.

Depending on the business goals and the type of channels being leveraged, different methods may be best suited for better business decisions. Most of what is described below will work equally well for acquisition and re-engagement though metrics may need to be adjusted accordingly.

A Note on Incrementality — the holy grail of marketing measurement

Lets first come to terms with some basic and obvious expectations around marketing. The process of understanding the true value of marketing is very hard for some simple reasons.

When we show an ad on TV, our impact on user intent is subtle and hard to measure. When someone clicks on an ad, they may be reacting to an amazing ad that convinced them to use our product OR they may have been intending to use the product anyways and happened to click on the ad to get onboard. Interpretation of that human intent is hard with the signals we get.

The true effect of marketing is the # of people signed up due to our efforts after we subtract people who would have signed up anyway. Each of these is hard to measure and the combination is even harder. This is what marketers refer to as incrementality and is key to making sound financial decisions.

Incremental Users = Users <acquired | engaged> — Users who would have joined organically anyway

Marketing budget allocation and growth estimation

In order to decide on macro-budget decisions for marketing, one needs a clear picture of the incremental value of marketing and potential growth impact of investment

If you can understand this curve for your business, you can identify the level of spend based on your desired growth objectives and willingness to pay a certain incremental cost per acquisition.

The usual constraints here would be spend no more than expected LTV of the user on an incremental basis while maintaining marketing spend below some %-age of revenue.

For example: Spend less than 20% of revenue on marketing at incremental cost per acquisition < 0.9% of LTV

So the key to marketing budget allocation would be understanding the curve above. There are a few standard methods to do this effectively:

• Switchback experimentation

Turn on and off marketing over periods of time (off for 2 weeks, on for 2 and repeat a few times). See how your conversions are impacted.

This will give you a good sense of value but is hard to do because of the effort involved in turning on and off channels, the potential business impact while marketing is turned off, and the challenges of things like seasonality affecting your results (example: one of the weeks had a holiday that throws off the numbers or it rained a lot one week and customers behave differently during rain)

• Synthetic control experiments

Identify two “similar” groups, such as two comparable markets (lets say Dallas and Houston) and turn off marketing on one, while keeping it on in the other and see how they behave.

The difficulty here lies in constructing the comparable markets based on your business metrics and requires thoughtful analytics to correctly setup. But this is often easier than doing full switchbacks.

• Media Mix Modeling

We will talk more about this in a later section but this method tries to leverage the inherent variation in spend across time and extract signal from your existing data. This is a very tough analytical problem and requires sufficient data to be effective. But this method has some very nice properties if you can pull it off.

Smaller businesses will likely be best off doing a simple set of synthetic control experiments periodically based on their business understanding to establish a reasonable baseline while companies with large marketing budgets may find value in developing a multi-pronged approach to measuring marketing effectiveness.

With these numbers in hand, and a measure of the long-term value, your CFO can now reasonably start planning financial outlays for marketing with an understanding of ROI.

Note that for non-digital channels, the methods above are usually the only ways to measure success of advertising at any granularity.

Per Channel Budget Allocation

Marketers can use many different channels such as TV / Radio, Facebook (low intent, high scale), Google (high intent), Display, CRM, and other more specialized offerings. After deciding how much to spend on ALL of marketing, the next decision is how much to spend in EACH channel / channel group (search, social, display, etc).

Each channel has its own curve similar to the diagram above and in an ideal world, we would pick points in the marketing spend on each curve that results in the same incremental cost per acquisition.

There are a few ways to do channel group attribution at this level:

• First / Last / <your favorite stage of the funnel> Touch Attribution

Far and away the most common method to attribution, this is usually the EASIEST measure and is also often needed for granular optimization since it is also the FASTEST way to measure effectiveness. The basic idea is that if someone clicks on an ad, we can say that the ad was responsible for converting the user.

All marketers should at least have a view based on Last Touch or such like attribution. Note that this only works for digital channels (there is no last touch in TV, for example).

A note on attribution — one of the easiest wins you can get on your touch based attribution models is fine-tuning the window for credit. For instance, an ad that was shown 6 months ago getting credit for a sale now is probably not reasonable. But depending on your business, even an ad from a week ago may be too old to deserve credit. Fine-tuning this “attribution window” to your business is very important.

Data required: Log of each conversion event and source as well as granular spend information.

• Multi-touch attribution (MTA)

Often spoken about with reverence but rarely implemented, this uses statistical rigor to the intuitive notion that each time a user sees an ad, they are forming an impression of the product and each step contributes to the overall conversion. Thus, such attribution methods try to give partial credit to each view using various approaches.

This can range from super simple models where every impression gets equal credit to more complex ones that decay older impressions or even sophisticated ML-based MTA where weights are based on data-driven models that predict conversion.

Collecting the full impression dataset to build such models is close to impossible (due, in part, to walled gardens of Google and Facebook which do not share data about impressions with you) and the underlying models have to measure very small effects (i.e., the model is measuring small lifts at a individual level on top of a small base conversion). Many vendor solutions in market are hokie at best.

Data required: Every impression event + all data required for first / last touch attribution.

• A special note on the asinine walled gardens:

Facebook and Google refuse to share impression data with advertisers (unlike pretty much everyone else — yay, power of oligopolies). Instead they offer “Switzerland” databases which allow you to import your data into their systems and then run SQL or write some basic scripts to do various analyses including MTA.

Of course, this isnt super useful for MTA because guess what — they cant share with each other. So you cannot get a complete picture.

Media Mix Models: Media Mix Models are statistical models that try to derive signal from variations in spend across channels over time. If we have enough such signals, we can start inferring the effectiveness of each channel, thus allowing us to construct the acquisition curve purely through spend signals.

This approach a few big advantages:

• Does not need any granular click data — only operates based on spend levels per channel group and overall conversion rates across markets
• Only method capable of handling media such as TV and Radio which do not have impression and view data at a per user level.
• Can capture cross channel effects (show that one channel is supporting the effectiveness of another channel similar to multi-touch attribution).
• If done right, and assuming you have good historical data with decent experimentation, these models can capture the causal impact of marketing as well as the rate of diminishing returns as spend is increased.

The difficulty here is that the statistical modeling involved is fairly sophisticated. This is not a silver bullet and it’s easy for opponents of the model to dismiss results unless you have robust experiments to prove the findings. Also, getting to a fine level of channel-granularity (e.g., message or campaign) is not feasible. Companies such as Neustar provide solutions in this space that may be of interest.

In my view, a combination of Media Mix Modeling and a <First | Last > touch attribution often provides two alternative views that can be used to validate each other and is really the best combination to leverage for sound decisions at this granularity.

Data required: Spend by channel by city over time, conversions by channel by city over same time

Intra-Channel Optimization:

Within a channel, there are constant needs to optimize your running campaigns, remove bad creatives or target audiences, reposition money to areas that are working, etc. Most of these decisions are made weekly, daily or even multiple times a day to get maximum value from your campaigns.

Only Last / First / Multi touch attribution methods provide data quickly enough to allow such dynamic responses and are the preferred method for marketers. Having said that, if your organization has a media mix model in place, the multipliers from such a model can be inserted into the last / first touch attribution to provide a more accurate measure of value as well.

At this level of grain, it is important to capture every aspect of a campaign so that marketers can quickly cross validate what is the source of success or failure. We should be able to tell if there is a problem with the bid strategy for a particular FB campaign or the lookalike audience or the creative or the message easily. The more data we capture, the better decisions we can make but this does require appropriate dash-boarding to make accessing this data super easy.

Additionally, every campaign of sufficient scale should consider some form of experimentation / hold-back so that you can measure campaign level incrementality as well. This can be done through straight A/B methods and are much easier to implement. Many vendors provide out of the box solutions for this as well.

Sequencing your attribution plans

There are two must haves for attribution that all marketers should do:

• Experiment, Experiment, Experiment — Good experimentation is the only reliable way to measure effectiveness at every level of granularity. Develop good experimentation practices, whether A/B tests at the campaign level or synthetic control tests at the overall level. Run experiments all the time!
• Last / First touch attribution — Bring the data in and start doing some form of attribution like this. This is the easiest way to drive tactical decision making and is critical to running a tight marketing organization. Leverage tools at your disposal to do this (Google Analytics, etc).

After you get these basics and have a couple of years of marketing under your belt, you can start looking at more advanced methods such as Media Mix Models.

More where this came from

This story is published in Noteworthy, where thousands come every day to learn about the people & ideas shaping the products we love.

Follow our publication to see more product & design stories featured by the Journal team.