In a large, multi-year engagement, Strong Analytics provided strategic consulting, experimental strategy, and machine learning research support to a large consumer-facing real estate business. The company had reached a point in their marketing strategy where product awareness reached near-total market saturation.

We worked with the client to design and implement a new strategy to reach their next phase of venture development. These goals included personalizing nurture outreach through micro-targeting and personalized content, measuring incrementality through A/B testing, and leveraging uplift modeling and next-best-action optimization for continuous, automated improvement.

The Problem

As product awareness increased, our client faced diminishing returns from their marketing efforts. A one-size-fits-all marketing approach was no longer effective in capturing the attention and interest of their target audience. To continue growing their business, the company needed to pivot towards more personalized marketing campaigns at scale.

However, doing so presented several challenges. The company lacked the infrastructure and capabilities to effectively collect, analyze, and utilize customer behavioral data to tailor their marketing messages. They also needed a way to measure the impact of their campaigns and continuously optimize their outreach strategies based on the insights gained from data analysis. They needed a more comprehensive solution that could automate the execution of micro-targeted campaigns, ensuring that the right content reached the right audience through the most effective channels.

Our Solution

As we have done in other marketing personalization projects, we took a holistic approach to building a production-grade marketing platform that could facilitate personalized marketing campaigns at scale.

The key objectives of the solution that we developed were as follows:

Automating Micro-targeted Marketing Campaigns

We developed a bespoke marketing platform that could seamlessly ingest customer behavioral data and user-level outcomes across lines of business. This platform enabled our client to automatically execute micro-targeted marketing campaigns based on individual customer preferences and behaviors.

Measuring Campaign Impact and Optimization

The marketing platform was designed to measure the incrementality of each campaign through rigorous A/B testing. By comparing the results of different content and channel strategies, our client could determine the effectiveness of their outreach efforts and make data-driven decisions to optimize future campaigns at the audience-, channel-, and message-levels

We paired this new, bespoke marketing platform with several strategic marketing initiatives to enable our partner to derive maximum value from the platform. Some of those initiatives included the following:

Campaign Verticals

We designed several campaign “verticals” focused on specific lines of business within the platform. Each campaign vertical had its own creative content and channel options, tailored to appeal to the unique interests and preferences of different customer segments.

Audience Segmentation

We segmented the company’s customer base based on their affinity for each campaign vertical. Within each segment, we began broadly experimenting with content and channel strategies using fully-autonomous experimentation. Our new platform would randomly experiment by sending different kinds of content and selecting different channels for users while, critically, maintaining matched control groups for each action to fuel subsequent analysis/optimization. This allowed for targeted messaging and tailored content delivery to specific audience segments, increasing the relevance and impact of the marketing campaigns.

Next Best Action Optimization

Using uplift modeling and next-best-action sequence optimization, we analyzed the experimental results to understand the lift generated by different campaigns on various audience segments. By identifying the most effective content themes and channels, the company could optimize their marketing strategies and deliver personalized messages that resonated with their customers.

Campaign Optimization

Based on the insights gained from audience segmentation and next-best-action optimization, we continuously fine-tuned and optimized the marketing campaigns. This iterative approach allowed the company to achieve significant business impact by efficiently reaching a smaller but highly influenceable population with tailored messages and offers.

Performance Impact

The implementation of the personalized marketing strategy and associated marketing platform had a significant impact on our client’s business. The multidisciplinary effort involving data science, AI, marketing, and product teams resulted in millions of dollars of incremental revenue. The personalized campaigns not only drove greater efficiency in customer acquisition and retention but also fostered stronger customer engagement and loyalty.

Moreover, the strategy laid the foundation for an evergreen approach to marketing personalization. The insights gained and the lessons learned from this project can be applied across the entire business to optimize marketing impact through ongoing personalization efforts. The real estate company now has a scalable and data-driven marketing framework that enables them to adapt to changing customer preferences and market dynamics, ensuring continued growth and competitive advantage.

*This article was first published on the Strong Analytics blog here.

Strong Analytics builds enterprise-grade data science, machine learning, and AI to power the next generation of products and solutions. Our team of full-stack data scientists and engineers accelerate innovation through their development expertise, scientific rigor, and deep knowledge of state-of-the-art techniques. We work with innovative organizations of all sizes, from startups to Fortune 500 companies. Come introduce yourself on Twitter or LinkedIn, or tell us about your data science needs.

Casino gaming is a fast-paced industry, often pursuing state-of-the-art technology to improve the player experience. However, when it comes to their loyalty marketing programs, most casino gaming providers have stuck to traditional tier-based direct marketing approaches.

Specifically, following the traditional approach, patron data are analyzed on a regular (typically, monthly) cadence to determine each player’s value based on how much patrons tend to spend when they are in the casino. Players are then “tiered” into low- to high-value buckets based on those values. Finally, promotional offers such as free-play and restaurant and hotel discounts are allocated to tiers, with players receiving notifications about their new offers via direct mail or email.

There are several disadvantages associated with this traditional approach. First, the process is very slow, as promotions are only being sent out about once per month. Second, the process is often very manual, requiring a lot of effort on the part of those involved in the promotional distribution process. Third, the process is rather static in the sense that it is not responsive to recent changes in player behavior. If promotional messaging is only sent out monthly, it is likely that the player information is outdated by the time the monthly mailer is finished being prepped. Finally, the process lacks personalization since users are broken down into discrete buckets rather than individual players getting customized offers.

In a recent project, Strong worked with a leading casino gaming provider to build a one-to-one marketing engine for their players. This platform, the most ambitious of its kind, continuously and automatically learns from players behaviors both online and in-casino to build and nurture player relationships through providing personalized, highly-responsive offers.

Separating Signal from the Noise to Optimize Actions

In a world driven by the excitement of random chance, separating the signal from the noise is no easy task. Thus, to build an engine capable of learning how to engage players at the individual, one-to-one level, we need to consider the many facets of player engagement.

Moreover, while many projects are interested in merely predicting what a player will do in the future, we needed to go on mere observation to optimize action. Given everything we know in general and about this particular player, how should we incentivize and communicate to them in order to best nurture the relationship?

Our Solution

We built a custom solution using our end-to-end platform for distributed reinforcement learning at scale, Strong RL. This multi-channel “next best action” platform streams in data from multiple sources, builds a quantified snapshot of each player in real-time, and streams out optimal actions that maximize player engagement — including incentives and personalized communication across email, direct mail, online, in-app, and SMS channels.

The results have been impressive. Not only did the system enhance players’ experience through a highly-personalized offers that responded to quick changes in players’ play, experimental testing revealed that it improved player profitability for both new players and existing players migrated from the traditional approach.

*This article was first published on the Strong Analytics blog here.

Strong Analytics builds enterprise-grade data science, machine learning, and AI to power the next generation of products and solutions. Our team of full-stack data scientists and engineers accelerate innovation through their development expertise, scientific rigor, and deep knowledge of state-of-the-art techniques. We work with innovative organizations of all sizes, from startups to Fortune 500 companies. Come introduce yourself on Twitter or LinkedIn, or tell us about your data science needs.

In a recent engagement, Strong Analytics worked with a leading partner in foodservice innovation that develops and brings user-centered solutions to market. Our partner company is always innovating to stay ahead of the curve, leveraging digital technology to modernize the food industry.

We worked with this leading foodservice innovation partner to improve their understanding of how product sales for certain ingredients differ by geographic market. This would enable them to optimize their deployment of ingredients based on where consumers are located.

The Problem

Our partner came to Strong Analytics with the goal of ​​better understanding which of their products sells best in different markets. To address this, they aimed to build a dashboard for general managers and sales leaders across the USA and Canada to help them understand local market conditions and trends. They wanted the dashboard to integrate multiple potential data sources and enable data-driven decision making for products and sales.

The Solution

We designed and built the MVP version of a market health tool for our partner. This tool leverages restaurant purchasing data and Yelp restaurant review data to understand what ingredients sell best at different restaurants and what people have to say about it online.

The MVP version of the tool incorporates a high-level overview of the popular ingredients nationwide. The dashboard we built also enables key stakeholders at the firm to understand how ingredient demand differs by geographic market.

Performance Impact

Our work enables the partner’s key stakeholders to explore the trends for themselves, allowing them to extract critical insights that drive strategic business decisions. Given the success of the MVP, we are currently in the process of converting the MVP version of the market health tool to a fully functional app that will be leveraged by a broader user base.

*This article was first published on the Strong Analytics blog here.

Strong Analytics builds enterprise-grade data science, machine learning, and AI to power the next generation of products and solutions. Our team of full-stack data scientists and engineers accelerate innovation through their development expertise, scientific rigor, and deep knowledge of state-of-the-art techniques. We work with innovative organizations of all sizes, from startups to Fortune 500 companies. Come introduce yourself on Twitter or LinkedIn, or tell us about your data science needs.

In a recent engagement, Strong Analytics had the opportunity to partner with a leading firm in the agricultural sector. Our partner company is at the forefront of agricultural innovation, focusing on the development of autonomous robots to mechanize the process of weeding fields. The objective of these robots is to mitigate the need for expensive manual labor and the use of potentially harmful pesticides, which can negatively impact the environment and human health.

The Problem

In our collaborative efforts with our partner, we aimed to enhance their crop object detection model, a critical component of their robotic system. The vision system in the robots is designed to distinguish between crops and weeds accurately. The mechanical component of this robot is programmed to selectively eliminate weeds while leaving the desired crops undisturbed, thereby ensuring efficiency and successful harvest with optimal crop yield.

Our partner’s initial model faced challenges in meeting requirements on low powered edge hardware, and the team at Strong was tasked with the responsibility of improving the overall accuracy and speed of the existing model.

Our Solution

The fundamental challenge at the beginning of this project was to balance the traditional trade-off between speed and accuracy. This required us to go beyond conventional methods to leverage advanced techniques and approaches in machine learning to retain accuracy with a substantially smaller model meant to run on edge.

By integrating several state-of-the-art approaches, we succeeded in developing a more efficient object detection model that significantly outperformed the original model in terms of speed and maintained (and exceeded) the original accuracy. With the new model, we managed to process images faster than real-time and exceed the original accuracy, which was a substantial improvement over the previous model.

Performance Impact

Our proposed model has resulted in substantial improvements in performance and speed and has been integrated into the partner’s robotic platform. Since the launch of our initial model, we’ve broadened the scope of work to encompass a variety of additional impactful research areas, including the following:

Multi-crop detection models. Transitioning from a single-crop detection approach to a multi-crop detection methodology presented several challenges, but ultimately unlocked significant advantages. This shift greatly simplified the workflow for end users as well as the deployment process. Now, a single model can be employed across different fields, irrespective of the crop type, enabling indiscriminate weeding. This means that robots can be deployed to weed any field at any time, significantly simplifying operations.

Unannotated image search. As part of our goal to improve model performance in an efficient manner, we introduced an unannotated image search feature. This feature enables automatic exploration of extensive unannotated image datasets to identify complex crop and weed combinations.

Generative image training. Adapting models to variable field conditions, such as different camera settings, lighting conditions, and soil characteristics, was a major challenge for our partner and for agricultural applications of ML in general. To address these challenges, we turned to the power of generative AI and implemented a solution utilizing a state-of-the-art algorithm in image translation. The ability to simulate these varying conditions in a controlled manner, without the need to collect new data or perform additional costly annotations has been critical to the model’s adaptability.

Conclusion

Through partnering with this leading agricultural company, we were able to enhance their crop object detection model, which is an important component of their robotic system. Through the integration of advanced machine learning techniques, we were able to develop a more efficient model that significantly outperformed the original model in terms of speed and accuracy. The proposed model has resulted in substantial improvements in performance and speed and has been integrated into the partner’s robotic platform.

Overall, this work showcases the power of machine learning in addressing real-world challenges and improving efficiency and sustainability in agriculture. Interested in learning more about how artificial intelligence and machine learning can help you streamline operations and push the boundaries of innovation in your own business? Reach out to our team of PhD-trained researchers and data scientists to learn how strategically investing in technology development can help you take your business to the next level.

*This article was first published on the Strong Analytics blog here.

Strong Analytics builds enterprise-grade data science, machine learning, and AI to power the next generation of products and solutions. Our team of full-stack data scientists and engineers accelerate innovation through their development expertise, scientific rigor, and deep knowledge of state-of-the-art techniques. We work with innovative organizations of all sizes, from startups to Fortune 500 companies. Come introduce yourself on Twitter or LinkedIn, or tell us about your data science needs.

After years of hard work and several pivots, your startup is finally seeing some product traction! Now, you’re looking to double-down on your current product vision and grow faster than you ever have before.

This requires making several important strategic decisions. Where should you spend your marketing budget? What parts of your product should you focus on in the next release? And how should you tailor your company’s support to best address customers’ frustrations?

One way to address these questions is to use your intuitions. After all, you know your product better than anyone, right? The problem with this approach is that intuitions aren’t always correct, and startups don’t have the time and resources to waste time on bad decisions. Customers don’t always see products the same way as their inventors.

A better way to address these questions is to ask more objective questions about your data. What marketing efforts have provided the greatest returns on investment? What product features are customers spending the most time using? And which features do they avoid, or use least effectively?

Time to turn to the data! If you’re like most early startups, you have all the basics covered. You’re using Google Analytics for marketing. You have Stripe for payment processing. You use Intercom for customer communication. And all your app data is stored in a variety of S3 and RDS instances over at AWS.

The problem is that these products — while great independently — aren’t going to give you the answers you need. They’ll each answer a couple basic questions, but none will offer the kinds of answers you need to make the biggest decisions facing your startup’s growth. For example, you might learn in Google Analytics that Facebook ads are really good at generating trial sign-ups, but do these trial sign-ups ever become paying customers? Moreover, maybe your product is usage-based — are these sign-ups high-usage (i.e., high-paying) customers? Or are they low-usage customers, who are taking all of your support team’s time without paying any of their salaries?

What you need is an integrated, scaleable data strategy that you can turn to in situations like this to get answers to your most important questions — the kinds of questions that span every aspect of your business. If you don’t already have a data strategy, I offer five simple points below that will guide you in preparing your company to use data — and data science — most effectively.

1. Track Everything

Storage is cheap, and it’s been cheap for a long time. This means that there is no excuse for not tracking every user action and interaction possible. It’s easy to filter out what you don’t need (or learn isn’t valuable) later, but it’s impossible to retrieve what you never stored in the first place.

Lots of companies are going to get a good amount of data tracked for free simply by using popular services like those listed above. But what they’ll likely miss is the app-specific data (unless they have a proper data strategy). And if you’re bringing a new product to market, this is the data that matters most!

But some companies will miss even the basic stuff. For example, I once had to rebuild a Software-as-a-Service (SaaS) application’s entire revenue history from raw payment and user information just to determine their historical churn rate. Something that would have taken a single, 1 second query instead took a day of work building and validating a data re-creation setup.

The bottom line: track everything you can, whether you see it as being immediately relevant or not.

2. Link Everything Together

Let’s say you have all these external and internal services tracking lots of data. Your work isn’t done! What you really need is a way to integrate them easily or, preferably, instantly. Your customers interact with each of your business’s departments (e.g., marketing, product development, billing). Unsurprisingly, then, your questions about how to make them spend more, use the product more, and generally be happier often require you to integrate their experience across these departments.

This is where something like Segment (or a similar/custom service) can come in handy. It will help you integrate data from your standard web services (e.g., Google Analytics, AWS, Stripe) into a single data warehouse alongside your custom data. Then, if you know what you’re doing, you can query and model these data to help you make decisions and improve your product. Which leads me to…

3. Build Models to Predict and Understand Key Customer Actions

With the data properly stored and integrated, you can get to the really fun stuff: predicting and understanding your customers’ actions.

The benefits of prediction are clear. By identifying an action a customer will take before they take it, you can tailor your interactions with them to maximize their value. For example, by identifying which free trial users are going to convert (and conversely, which are not), you might decide that unlikely-converters should receive a coupon and likely-converters should not (they might just pay full price!). Or by determining who is at risk of churn (leaving your company), you can trigger a series of helpful emails or a real-life phone call that might be enough to extend their time as your customer.

Conversions and churn are the most commonly predicted actions. But with the proper data, you can predict almost anything: Who is going to have the most trouble using your product? Who will recommend your product most? Who is going to take the most support time? Each of these is useful in their own way, and can turn your business from one that is primarily reactive to one that is proactive.

Prediction models typically rely on a variety of machine learning approaches, including gradient boosting, random forests, neural networks, or some Frankenstein-like ensemble of many models.

Beyond prediction, understanding the underlying factors that drive customer actions is also important and, from a decision-making perspective, often more rewarding. At the heart of understanding is explanation — why did a customer leave? Why did they sign up? Why are they having trouble? Identifying these causal factors underlying an action give you the potential for intervention: manipulating some factor in order to increase the likelihood of a desired outcome, or reduce the likelihood of an undesired outcome.

Common statistical models used for understanding actions include decision trees, linear models, principal components analysis, and Bayesian models. But true causal inference will only be licensed when these models are applied to experimental data (see point #5 below).

Building and implementing statistical models like this isn’t easy and, once built, you’ll want to have a process for ongoing model integration, updating, and validation of the models. But doing so can be tremendously helpful to a startup looking to maximize efficiency and effectiveness for fast growth.

4. Communicate Key Metrics and Models

Things are looking up! You have your data stored and integrated, and if you’re lucky (or you hired us), you have killer data models offering live predictions and causal models of your customers’ key actions. Quick question: If a model makes a prediction, and no one is there to see it, did it ever really make a prediction? Answer: It doesn’t matter. Your model is as useless as that question.

Data science for business is like any science: it’s only as good as it’s communicated to the people who need to know about it. Historically, this might have meant a project report presented at a company meeting. But now, this might mean a daily, automated email to your team with key organizational metrics and historical trends, combined with a live dashboard to see exactly how your company is doing and how the models are performing. Or maybe you want to go even further, perhaps with live prediction metrics and auto-segmentation available to your sales and support staff in your CRM? Or automatically-triggered interactions with your customer?

This is where your data strategy really pays off. In fact, this is where it even becomes apparent that you have a data strategy — when what you learn and predict is integrated back into your operations to make your business more efficient.

5. Experiment

The biggest criticism about the ‘big data’ movement is that no amount of data can overcome the issue we all learned about on the first day of our statistical training: correlation does not equal causation. In some cases, like when you are generating predictions, this issue can be ignored. You just want to know whether someone is doing something — you don’t care about the underlying causes.

But if you want to truly understand why something is happening in your organization or with your customers — which is typically the case — you’ll ultimately need to run some experiments, the simplest of which are called ‘A/B tests.’ This is the only way to determine whether some factor (e.g., seeing an advertisement, going through an onboarding process, interacting with your support team) actually causes someone to do something (e.g., subscribe longer, pay more, send you an angry email).

This is why we always recommend having a platform for experimentation setup, and a workflow for easily running valid analyses. (Believe me: if it’s not easy, you won’t run the analyses when it really matters.) For simple website-based experiments, Optimizely might be your answer. For more complex experiments, you may need a custom platform. But setting up an experimentation platform is critical to turning your best guesses into valid answers.

What’s Next?

These are the five most important components of any data strategy — and things we recommend any startup have in place as soon as possible. Not doing so well with your data strategy? Get in touch with us and we’ll show you how we can help make your next big decision easier than any to this point.

*This article was first published on the Strong Analytics blog here.

Strong Analytics builds enterprise-grade data science, machine learning, and AI to power the next generation of products and solutions. Our team of full-stack data scientists and engineers accelerate innovation through their development expertise, scientific rigor, and deep knowledge of state-of-the-art techniques. We work with innovative organizations of all sizes, from startups to Fortune 500 companies. Come introduce yourself on Twitter or LinkedIn, or tell us about your data science needs.

Deep Learning (DL) has made remarkable strides this past year, particularly in computer vision and natural language processing, where we saw tools like Stable Diffusion creating beautiful surrealistic images from a prompt, others solving complex protein-folding problems, to more recently where one can write a short story or create a working app from a text prompt and ChatGPT. Exciting times indeed!

Then again, perhaps we shouldn’t be surprised as it’s these domains where DL has shone brightly in the past also. Interestingly, when we switch to tabular data — the kind of data more ubiquitous in data science — deep learning has not been as successful.

We can think of data in terms of general types. A popular distinction is structured vs. unstructured. A common example would suggest text as unstructured data and tabular data as structured, or even more generally as tabular (structured) vs. anything else (unstructured). This distinction does not seem evenly applied, and unstructured data can and often is given a ‘structure’ in order to be able to use it for modeling. Perhaps a better distinction for our purposes is homogeneous vs. heterogeneous, where the data inputs are all of the exact same type or source (homogeneous) or of varying types (heterogeneous). Homogeneous would include images, text, and even some structured data settings (e.g., multivariate times series, like financial data). Heterogeneous data is usually more complex, and the kind we typically observe with tabular data. Even in homogeneous tabular data settings, it’s often trivial to add additional useful features from other sources (e.g. adding seasonal or firm characteristics to the financial data).

Tabular data is characterized by feature heterogeneity- mixed input types coming from possibly disparate sources, and often varying wildly in reliability and integrity. For example, we might have observational data on individuals and their interactions with a website or app. For these individuals we could have demographic information (e.g., age, what state they live in), information on how often they visit or interact with a website (e.g., how many times they clicked on an ad in a month), date-related information about when they visited, and more. Some features are expressed in categories, others are numeric but highly skewed, some features come in clusters (e.g., multiple survey items about a specific topic), some may be unreliable (e.g., asking how favorably they view a particular topic).

Among the challenges with tabular data are the mixed data types, feature inter-correlations and redundancy, missing values, poor scaling, and noisy measurement. These challenges often lead to data situations with low signal-to-noise ratio, making it difficult to devise a model that will easily perform well.

On the surface, one might expect that DL, a special case of the universal function approximators better known as neural networks, should make quick work of tabular data, given their immense successes in other domains. After all, these are the kind of data analyzed by millions each day in spreadsheets around the world. Yet this type of data has proven consistently challenging for DL due to the challenges inherent in such data.

Techniques for analyzing tabular data

Aside from deep learning, what are some typical ways we might analyze this type of data? We can start with linear/logistic regression or anything along those lines (generalized linear/additive models). These approaches will be easiest to conduct, allow for relatively easy interpretation, have common methods to understand the uncertainty in the results, and are easily applied to the small data situations that are common among tabular data. However, these techniques are not usually very performant compared to other tools we have at our disposal. Why? Because in most data situations, we usually have interactions among features, and nonlinear relationships between features and outcomes, which these tools usually require additional steps to address. We also need to add penalization and/or other techniques to make better predictions beyond the data we have, also known as “generalization”. Some of the issues are easily and typically addressed, like adding some regularization to the model fitting process, but others can be more difficult- for example, handling interactions and nonlinearities becomes tedious and problematic as the number of features grows. In addition, standard statistical methods become more difficult to apply in very large data situations without compromising some of the inherent benefits (e.g. easy uncertainty estimation).

More complex models for tabular data

If we’re primarily interested in predictive performance, a standard approach to tabular data is via boosting, with popular libraries such as lightgbm, xgboost, catboost, all of which are available in Python and R. Boosting is fast and will generally do much better than linear model approaches out of the box. When appropriately tuned, they are hard to beat, and differences between the specific boosting implementations mentioned above are minor in practice. These tools are also applicable to large data situations with possibly dozens of features, where standard statistical methods will struggle or fail. In addition, we can also get measures of feature importance, and use other approaches to aid interpretability.

As we increase the complexity of the model, other aspects of the endeavor start to suffer. It becomes more difficult to get straightforward interpretations of feature-target relationships due to complex interactions and feature inter-correlations. It also becomes more difficult to get an understanding of the uncertainty in the results without adding to computational complexity, though there are methods there as well. This is just to say that just because we get a good performing model, doesn’t mean that we get everything we’d want.

Deep learning for analyzing tabular data

Various methods for Deep Learning have been applied to tabular data, from standard multi-layer perceptrons to complex architectures and ensembles of those (some examples are provided in references below). Unfortunately none consistently beat boosting methods, and on closer inspection, the performance is actually worse, or boosting is better, than what is touted in the initial papers. Many papers are relying on several decimal place differences in performance (e.g., in accuracy or RMSE) to report a win for the DL model, which is neither a practical or statistical difference. I have even seen a case where someone was able to produce better results for boosting than a DL paper reported right after it was published, even without hyper-parameter tuning, and those results beat the paper’s reported DL model performance! Unfortunately, all too often the hype is just that.

However, we don’t want to fault the application of DL to tabular data in general just for some poor demonstrations of it. And beyond the hype, where DL seems to excel is with strictly numeric data with notable nonlinear relationships, including interactions (see, for example, my meta-anaysis of several paper results). Consider the case of an experimental data situation with a simple treatment condition (control vs treatment) and another two group feature (e.g., pre-post setting). Assuming good experimental design and control, the simplest model we could run is just y ~ trt + post + trt*post. A DL model can’t do much with that simple feature setup, though it is a valid and useful model applicable to many experimental settings. However, in even slightly more complex settings, for example, 10 features with notable interactions and nonlinear relationships with the outcome, standard statistical models will have difficulty modeling the complexity without a lot of work, and often will fail to converge in the estimation process. We could apply DL techniques in such a case, though even then, performance may only slightly improve upon boosting results, which also can model those complexities.

Summary & Recommendations

At present, deep learning for tabular data may have a place, but there has been no major success with tabular data, and it’s not clear what remaining tricks there are to pull off. But time will tell, and given recent DL advances in other domains it seems anything is possible. As such, for now our recommendation is to pursue deep learning for tabular data when you have a large matrix of homogeneous data, want to take advantage of the tools’ ability to handle large data situations, or want to use it as part of an ensemble with other methods. You may also want to use a DL approach because part of your problem includes NLP or computer vision, and want to use consistent tools or even use them together to predict some target.

The following can serve as some general guidelines for working with tabular data.

Consider using deep learning if:

• Your tabular data are homogeneous
• You want to use a standard model (like linear regression) with very large data
• You’d like to combine your tabular data with other DL models/tools (e.g. NLP)

If the goal is performance with tabular data:

• Boosting should still be the go to, and should remain a baseline benchmark for any deep learning approach
• You should consider ensembling approaches, including deep learning

If goal is interpretability:

• Consider using GLM/GAMMs
• Realize that if the data relationships are complex, easy interpretability may not be achievable

References

The following sources can provide a good overview of the tools:

A Short Chronology Of Deep Learning For Tabular Data

This is definitely not all you need

Deep Learning for Tabular Data

Tabular Data: Deep Learning is Not All You Need

A couple recent specific implementations of DL models for tabular data:

TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second

SAINT: Improved Neural Networks for Tabular Data

TabDDPM: Modelling tabular data with diffusion models

Contact Us

Contact Strong to learn more about our expertise and how we help our customers build robust statistical solutions for complex business problems.

*This article was first published on the Strong Analytics blog here.

Strong Analytics builds enterprise-grade data science, machine learning, and AI to power the next generation of products and solutions. Our team of full-stack data scientists and engineers accelerate innovation through their development expertise, scientific rigor, and deep knowledge of state-of-the-art techniques. We work with innovative organizations of all sizes, from startups to Fortune 500 companies. Come introduce yourself on Twitter or LinkedIn, or tell us about your data science needs.

Object tracking involves a distinct set of challenges and trade-offs that make it one of the most demanding specialties within computer vision. Video data is typically heavier than static images used in classification models. Speed is often paramount, as many models are deployed to track objects in real time, necessitating tight optimizations for latency. And unlike other computer vision models, object trackers often have to keep their eyes on more than one object while maintaining those objects’ unique identities.

The complex balancing act has meant that object tracking was relatively unreliable until recently. But the combination of advances in deep learning and the proliferation of edge devices has meant not only that object tracking models have grown more robust, but that there are now viable roads to deployable applications. From traffic to security to inventory management, the sophistication of newer models and hardware has led the discipline to an inflection point and ushered in scores of engineers to tackle problems in object tracking.

If you’re thinking about implementing object tracking into your application, there’s a lot to consider. Here are a five high-level points to get you oriented and hopefully refine your product road map.

Decide whether you need to track objects or just detect them

Unless you’re tracking objects aerially on an ice rink, chances are that the line of sight between your sensor and the objects being tracked will suffer from interference. Whether or not you build a model to continually maintain the identity of those objects depends on your application.

If you’re tracking horses in a race, for instance, where jockeys pull ahead and fall behind one another throughout the race and persistently occlude each other, maintaining the horses’ identities is central to the application — it doesn’t matter that any horse is in third, but particularly that Bag O’ Beans is. On the other hand, a system built to monitor the availability of parking space doesn’t need to know which cars are occupying a lot, just how many of them there are.

If possible, determine how many objects you need to track ahead of time

A self-driving car can’t know ahead of time how many objects will be in its line of sight at a given moment, and a surveillance system probably won’t be deployed to monitor a predetermined number of people. But if you do know how many objects you’ll be tracking, you’ll reduce the need for algorithmic guesswork, greatly simplifying the application.

Consider a model built to track people in a soccer game. The model will try to identify that there are 23 people total, and no more: 11 players from each team, and one referee. By constraining the system, the model avoids false positives (since the unruly fan on the field can be identified as a twelfth player on a team), and mitigates false negatives. And in so doing, it enhances the accuracy of identification: even if your model accidentally assigns high probabilities of being a referee to two separate people, only one of them can be, which means that second non-referee has to be someone else.

In other words, by knowing how many of each object you need to identify, the very identification of an object assists in identifying the others in a way that’s not quite true when the quantities are more ambiguous.

Determine if your app needs to run in real-time or offline

Implementing real-time tracking into your application is sometimes non-negotiable. Self-driving cars need to know whether those things in front of them are people, and security systems have to identify threats before, not after, they happen. But tracking objects in real time comes not only with the typical challenges of minimizing latency (i.e., cost and resources), but also, in some object-tracking applications, with the added hurdle of having to perform the task in an usual, localized setting, like on a moving vehicle or a robot that might require specialized edge devices.

In cases where the advantages of real-time tracking are more ambiguous — say, in tracking shopper behavior to figure out how to redesign the floorplan — running models offline can be enormously beneficial. Since speed becomes less paramount, you’re more easily able to run models that would be less feasible if used in real-time apps without enormous outlay. But leveraging offline data is analytically useful as well. For historical data, time no longer moves in a straight line — the object can be tracked from second 0 to second 10, and in reverse, from second 10 back to second 0. If your application is heavily reliant on identification, then allowing the tracking to move in both directions can help mitigate problems with object occlusion, since you have more information with which to triangulate the specific object.

But even if running offline is ideal, the choice isn’t all-or-nothing. There might be an imperative for part of your application to track objects as they happen and then run less time-sensitive tracking analytics afterwards for more accurate and/or complex measurements — whether a car was driving at 100mph or 115mph through a race, and all the instantaneous variances of velocity, are much easier to answer offline, and they’re probably less interesting to lay viewers than to the car’s owner.

Minimize the efforts spent on annotation wherever possible

If you’re at the very beginning of road mapping your application, you probably already know that object tracking presents some of the greatest annotation challenges in all of machine learning. Not only do video frames have to be annotated with bounding boxes, identities, and sometimes image segments, but sequences between frames often have to be annotated as well. Depending on your application, you could wind up with a dozen annotations on a single frame to train your model. The fact that annotators have to look at mostly minor variations of the same image over and over doesn’t exactly boost morale.

Sometimes, doing it all frame by frame, one annotation at a time, is the prerequisite for your application. But things aren’t always as dire as they may seem. Consider an application that tracks objects in an industrial setting, like boxes moving on a conveyor belt. The belt will typically move at a steady speed, and so boxes will move at predictable velocities in different segments of the data. In such cases, when you can expect your real-world use case to not involve objects that behave in substantially different ways, you can augment your data by replicating it and slightly altering the replicants, thereby capturing the most essential features needed to train your tracking model while avoiding overfitting — all while reducing the amount of manual annotation required.

And some tracked objects within the same dataset may require less detailed annotation than others. In training a self-driving car, the car should probably be able to recognize objects like pedestrians with pinpoint accuracy, which can involve annotating those objects not just with bounding boxes, but also with more (often more effortful) image segmentations. But inanimate objects, like signage and trees, may not prompt the same concerns for accuracy. Identifying when maximum accuracy is actually critical and when it’s not can save substantial time on annotation.

Adding more sensors can help, but they come at a cost

Increasing the amount and type of data used by your model will, generally and predictably, increase its accuracy and address common issues including occlusion. Using multiple cameras to track objects makes it more likely that if a line of sight is cut off in one camera, it’s picked up by another. Adding sensors can also help moving objects localize themselves in space, either by multi-view geometric approaches from multiple cameras, or by using data from different sources, like depth information from LiDAR.

Adding sensors can significantly boost the robustness and performance of your application. But that robustness comes at a cost: those additional sensors introduce algorithmic complexity, heavier hardware burdens, and compute cost.

Deciding if the trade-offs are worth it depends entirely on your use case. If integrating new sensors increases the safety of a self-driving car, more sensors are probably warranted. But if they simply add marginal accuracy to velocity measurements of shipping containers, that threshold has to be measured against obvious cost/benefit ratios that are specific to your organization, and which can sometimes be achieved in other less costly, more artful ways, like thoroughly analyzing offline data.

Contact Strong

Contact us to find out how we can help build and integrate state-of-the-art object tracking into your products and solutions.

*This article was first published on the Strong Analytics blog here.

Strong Analytics builds enterprise-grade data science, machine learning, and AI to power the next generation of products and solutions. Our team of full-stack data scientists and engineers accelerate innovation through their development expertise, scientific rigor, and deep knowledge of state-of-the-art techniques. We work with innovative organizations of all sizes, from startups to Fortune 500 companies. Come introduce yourself on Twitter or LinkedIn, or tell us about your data science needs.

The pharmaceutical industry is constantly evolving and advancing, with new discoveries and innovations that are transforming the way we approach healthcare. One of the most significant drivers of this transformation is machine learning (ML), which is changing the way we discover and develop drugs, optimize clinical trials, and personalize treatments.

Machine learning algorithms are capable of analyzing vast amounts of data and identifying patterns and relationships that would be impossible for humans to detect. As a result, ML is accelerating drug discovery, enabling precision medicine, improving drug safety, and reducing costs.

Here are five ways machine learning is changing pharmaceuticals:

Drug Discovery and Development

The process of drug discovery and development is a long and expensive process that can take up to 15 years and cost billions of dollars. Machine learning has the potential to accelerate this process and reduce costs. ML algorithms can analyze large amounts of data and identify patterns that humans might miss. This can help researchers identify new drug targets, predict drug efficacy and toxicity, and design more effective clinical trials.

For example, ML algorithms can analyze the genetic data of patients to identify potential drug targets. By analyzing the gene expression profiles of patients, researchers can identify the genes that are overexpressed or underexpressed in disease states. ML algorithms can then be used to identify potential drug targets that can be targeted to treat these diseases.

Precision Medicine

Precision medicine is an approach to healthcare that takes into account the unique genetic, environmental, and lifestyle factors of each individual to develop personalized treatments. Machine learning can play a critical role in precision medicine by analyzing large amounts of patient data and identifying patterns that can help predict patient outcomes and response to treatment.

For example, ML algorithms can analyze patient data to identify genetic variations that may affect drug metabolism and efficacy. This information can be used to develop personalized treatment plans that take into account a patient’s genetic makeup and other factors.

Drug Safety and Pharmacovigilance

Drug safety and pharmacovigilance are critical components of the drug development process. Machine learning can help identify potential adverse drug reactions (ADRs) and improve drug safety. ML algorithms can analyze large amounts of patient data to identify patterns that may indicate a potential ADR.

For example, ML algorithms can analyze social media posts and other online forums to identify potential ADRs that may not have been reported to regulatory authorities. This information can be used to improve drug safety and prevent harm to patients.

Clinical Trial Design and Optimization

Machine learning can help optimize clinical trial design by predicting patient outcomes and identifying optimal dosing regimens. By analyzing large datasets of clinical trial data, machine learning algorithms can identify factors that affect patient outcomes and predict which patients may benefit most from a specific treatment.

For example, machine learning algorithms can analyze clinical trial data to identify patient subgroups that respond differently to a particular treatment. This information can be used to design more efficient and effective clinical trials.

Supply Chain Management

Machine learning can also help optimize pharmaceutical supply chain management. By analyzing large sets of supply chain data, machine learning algorithms can identify inefficiencies and predict future demand for drugs and medical supplies.

For example, machine learning algorithms can analyze data on drug shortages to predict future shortages and help prevent them by optimizing production and distribution.

Conclusion

In conclusion, machine learning is changing the pharmaceutical industry in many ways. By analyzing large sets of biological, chemical, and clinical data, machine learning algorithms are helping researchers and healthcare professionals make better decisions and improve patient outcomes. As this technology continues to advance, it has the potential to transform healthcare and improve the lives of millions of people around the world.

Pharmaceuticals is one of the industries we specialize in at Strong Analytics. We’ve done work with some of the largest enterprises in the space to build and deploy an artificially intelligent multi-channel promotional outreach engine across multiple strategic brand portfolios. We’ve also used one of our key platforms, Strong RL, to demonstrate how reinforcement learning can be applied to drug dosing.

Interested in learning more about how we can help you leverage machine learning to reduce costs and enhance the level of innovation in your current business? Reach out to our team of machine learning experts to get started.

*This article was first published on the Strong Analytics blog here.

Strong Analytics builds enterprise-grade data science, machine learning, and AI to power the next generation of products and solutions. Our team of full-stack data scientists and engineers accelerate innovation through their development expertise, scientific rigor, and deep knowledge of state-of-the-art techniques. We work with innovative organizations of all sizes, from startups to Fortune 500 companies. Come introduce yourself on Twitter or LinkedIn, or tell us about your data science needs.

No matter how ambitious or moderate your plans for your Machine Learning product, you need to make sure that what you’re building works under the most minimally basic conditions for your application. This first prototyping step is the product’s proof-of-concept (PoC), and when executed thoughtfully and thoroughly, it paves the way for successful productization and minimizes unanticipated headaches.

Think of your PoC as the first step towards the Minimally Viable Product, your initial move from the whiteboard to the real world to ensure that the core components that might go into your MVP are even feasible. While your MVP might have a user-facing front end or be integrated within your larger application, your PoC could involve a standalone component no more complex than a command line interface (CLI). And PoCs can entail different steps in different contexts. If you plan on deploying a product using a novel deep learning algorithm you’ve implemented, your PoC could be a test to make sure that the algorithm produces acceptable results on simulated data. If you’re using a well known and validated approach, your PoC could apply the algorithm to a small subset of real-world examples specific to your product to make sure they work in their intended context.

However it might manifest, a PoC is above all a demonstration that the foundational ML elements to a solution can in fact be built and deployed in the way your intended product demands — and in the process, maps how to get there. By the time you’ve completed your PoC, how to build what you want should be totally unambiguous.

Before you jump into building your PoC, here are a few considerations you should reflect on to ensure that your project is as successful as it can be.

Understand why you’re building

As engineers, it’s tempting to begin tinkering with abandon, guided by an impulse to innovate first and justify later. But it’s important to always keep in mind that your PoC isn’t there to exhibit a vague theoretical concept to satiate curiosity for its own sake. Rather, it’s one of the first of many investments of time, money and manpower towards a real thing used by real people.

In this sense, building a PoC in enterprise starkly contrasts with the purely conceptual research that comes out of labs and universities. Instead, organizations have to take a deeply human-centered approach from the beginning when designing ML. The illegible Jupyter Notebook that produced a state-of-the-art algorithm to increase a metric by 0.05 over the baseline usually should be avoided.

By never losing sight of why you’re building, you’ll avoid turning your PoC into a hobby horse that, while perhaps impressive, quickly hits a dead end.

Start simple

Once you turn from pure ideation (where things always seem to fit together so neatly) to the first lines of code, it’s tempting to feel slightly discouraged when confronting the distance between your PoC and your vision for the fully deployed product.

However, as you build out your PoC, keep in mind its goal is to demonstrate the feasibility and viability of your product or solution. Maybe your initial algorithm’s accuracy isn’t as high as it needs to be for production, or it runs more slowly than you’d like. Between your PoC and the end product, there’s substantial effort in optimization, re-training, responding to edge cases, and more — all of which will substantially improve the product.

We’ve run this distance many times. One client wanted to implement a computer vision solution that could identify objects in 3D using a multi-channel video setup on vehicles in real time. Rather than focus on building the production solution, we started with a basic PoC that used historical data run offline at a few frames per second. What mattered in this scenario wasn’t polish and perfection, but rather to test and explore how the solution’s various algorithmic components could integrate in a coherent way. After proving out the PoC, we continued to build out the MVP and optimized our models for edge deployment which allowed us to achieve real-time performance.

In another recent project, we worked with a gaming client that wanted to build out a marketing engine to automatically send offers to players to effectively engage them at the right time. In that case, the PoC utilized a small subset of players in a real-world environment run in an A/B-test setting that allowed us to determine whether our approach would work. It did, and the PoC was ultimately fully productized and deployed. Starting small allowed us to prove out the concept, minimize risk, and lay the groundwork for successful productionization.

Determine requirements and constraints early

Both before and during the PoC development process, it’s critically important to keep in mind the business and user requirements. Some of those you’ll foresee before you even begin, and some of them will reveal themselves as the PoC is built.

What data will you need, and is there sufficient historical data collected? How will you access these data in the real application? What are the budgetary constraints you’re working within? Do you know how fast the application will need to run? Who are the end users, and how will this improve their experience? What timelines and constraints are you working with?

On a somewhat optimistically cautionary note, although you should be realistically estimating what you’ll need, your worst case scenario often isn’t the most accurate. The common misconception that companies need to be the size of Amazon to utilize Machine Learning is just that: a misconception. A great deal can be achieved in low data and resource environments. Even cost-wise, your early expectations may be grimmer than reality. With one client, we were able to reduce cloud costs, which had reached over $1 million per year, in half with optimization of their models and inference pipelines.

Finally, and more intangibly, make sure you’re aware of any organizational and human constraints. Do you have buy-in from key stakeholders in the project? Is this aligned with the organization’s overarching purpose? These sorts of considerations might not be as easily quantified, and discovering them certainly requires different methods than cost rundowns, but they’re no less important.

Determine who will build it

Once you’ve settled on building your PoC, you have to figure out who is going to actually do the building. Having the proper resources in place to execute an idea can make the difference between years of toil with no results and deploying v2 based on user feedback in a few months. Personnel-wise, there are four potential paths to building a PoC:

Buy off the shelf

This usually involves taking an existing SaaS platform and implementing it to your uses. If an off-the-shelf solution turns out to be close to perfect for you without much tweaking, that’s great. But many of our clients have turned to us after trying and failing to make an existing solution bend to their wills. Off-the-shelf options are inherently not customizable, often pose integration difficulties, and can be acquired by larger companies that render them unusable for you, leaving you scrambling.

Hire for internal roles

While it’s tempting to hire full-time staff, building a team from the ground up is often incredibly challenging. Additionally, you may discover in the course of developing your PoC that your original product is less viable than you’d originally imagine, either demanding more personnel to complete or rendering it infeasible altogether and leaving you with newly onboarded data scientists with nothing to do, or having to rapidly re-downsize after making fresh hires, which can be demoralizing within the organization.

Hire an independent contractor

If you’re building a PoC with an existing team in place and you need to augment it with one or two additional seats, using independent contractors can make a lot of sense. But relying on a lone contractor to build out your entire PoC is usually too tall a task for any one individual, especially if they lack deep ML experience. And most contractors don’t have the range of expertise to not only build the algorithmic components, integrate the components with an existing service, perform data engineering, and so on, all of which are best performed by various individuals capable of doing each.

Hire a team

Bringing on an independent team strikes the right balance between all the other three options. By relying on an outside team, organizations get the contractual flexibility of working with a freelancer, the diverse expertise of a cohesive group, and the turnkey convenience associated with off-the-shelf options. We’ve seen firsthand how companies that struggled with any of those other options have watched the skies clear when bringing on external teams.

As Strong, we’ve built PoC applications that incorporate computer vision, deep learning, natural language processing, reinforcement learning, and other state-of-the-art approaches for Fortune 500s, seed-stage startups, and everyone in between. Along the way, we’ve developed an acute sense of what it takes to build an ML PoC that can successfully pave the way for a full-fledged data product.

Contact us to find out how we can help you take your ideas from the whiteboard to the real world.

*This article was first published on the Strong Analytics blog here.

Strong Analytics builds enterprise-grade data science, machine learning, and AI to power the next generation of products and solutions. Our team of full-stack data scientists and engineers accelerate innovation through their development expertise, scientific rigor, and deep knowledge of state-of-the-art techniques. We work with innovative organizations of all sizes, from startups to Fortune 500 companies. Come introduce yourself on Twitter or LinkedIn, or tell us about your data science needs.

Applying machine learning to natural language is a uniquely challenging task. If computer vision can be thought of as opening a window onto the physical world, the field of Natural Language Processing (NLP) opens a window onto a speaker’s mind. One could also argue it’s less of a window and more of an abstract code to be deciphered by the listener.

We know it’s possible to learn language; after all, babies learn to speak just as they learn to see. Most babies around the world learn even more than one language at a time!

Nevertheless, it is for good reason that babies’ vision and physical reasoning proceeds more quickly than their language development: language is tricky. The same sentence can mean different things depending on context. Words evolve over time as speakers fundamentally change the meanings of words and how they are used (literally!). And, more often than you would think, speakers play with language in jokes or hyperbole, or even say the exact opposite of what they believe in sarcasm.

We believe that understanding and addressing these challenges through strong NLP research and engineering can lead to more robust, generalizable language models. Here we offer five tips for NLP engineers — or organizations considering a project in NLP — to consider in their approach.

Try to break your model

The best-case scenarios you present your model in training often aren’t the most likely ones it will encounter out in the wild. This is truer in some domains than in others. If you’re simply processing New York Times articles to determine if they’re about Canadian politics, you can rely on mostly similar input formats from document to document. But if, say, you’re classifying Facebook posts for whether a user is depressed, or you’re analyzing data from multiple corpora, things get less predictable.

First, the input might break your model entirely by throwing an error. If you’ve trained your model on nothing but social media posts of 200–300 words and it suddenly force it to process a document of 5,000 emojis (this is not a hypothetical; we’re speaking from experience), your model could be unable to process such a large array or deal with one made up of characters that it has no familiarity with.

Second, your input could be unfamiliar in content but similar in structure such that the model produces a result, but an undesirable one. For example, if you have a topic modeling algorithm that sorts every single document into financial, sports, or political news, and then feed it a piece of celebrity gossip, the topic assigned will almost necessarily be incorrect.

To guard against both, NLP engineers should think similarly to information security architects who conduct penetration testing by trying to break what they build. It’s important to regularly stress-test your model, both in the initial training and after deployment, by feeding it unexpected inputs. In some cases, we’ve gone as far as writing quick scripts to generate gibberish to adjust the model accordingly. Additionally, we’ve often found it helpful to swap out words for their synonyms and see if our application still works as intended. For example, when classifying violent text, if “hit” is swapped out for “strike,” do we still get the result we want?

These are just a couple possible small ways of many to make your model more robust. But regardless of exactly how you disrupt your model’s sense of regularity, you should never let your application get too comfortable.

As language changes, so should your model

Much of language’s ambiguity owes to its shifting usage and meaning over time. For NLP engineers, this means that applications are doomed if the models they rely on are too static and aren’t able to adapt to new words, syntax, and constructions.

The past year provides ample examples. A model trained on text from 2018 and deployed to classify text from 2019 would have reasonably situated a document with the word “lockdown” in a vector close to the somewhat threatening, which could have a number of implications depending on the application. By mid-2020, however, “lockdown” had become semi-synonymous with “quarantine”; a model built to classify violent text would be spinning out of control with false positives.

In cases like that, where the word is uncommon enough as to not have previously been a part of everyday speech, training it on a few additional examples is a feasible way to adjust the model’s weights. In such cases, you can even sometimes simply make manual, rule-based adjustments (e.g., if a text contains “lockdown” and also contains “coronavirus,” classify it as nonviolent).

In other situations, where an extremely common word changes meaning (say, if there was a new $100-billion company called Water) or where entirely new words and slang become unignorable, more comprehensive re-training is required. In fact, it’s ideal to introduce new data to your model on a fairly regular basis to train it on words you might not even be aware of (at this point, your application should be able to process “sus”). In any event, use your instinct — if you sense that your model is producing results significantly disproportionate to what you expect, it might be time for an adjustment.

Context matters

As many NLP engineers know, a rose is not always a rose. The context of a text matters, and this is what makes detecting traits like sarcasm and humor one of the field’s harder problems. It also means potentially incorrect results in a classifier. In the case of our example model that detects violent language, this means, for instance, taking care that “i want to kill him” and “Sarah thought to herself, ‘I want to kill him’” aren’t analyzed in a way that elides their grammatical and syntactical nuances. As humans, the first sentence is more obviously the product of day-to-day speech patterns that make it more likely a person actually typed it when communicating with someone else. In the second, the sentence bears a clear literary quality that may be less concerning.

It’s easy to wind up with models that zero in on a single word or ngram without adjusting for context. Some preprocessing is designed to minimize the sheer number of distinct words or characters your model has to remember, and that can mean removing punctuation and rendering all words either lowercase or uppercase. In our example, it’s precisely the kind of information that’s often processed as extraneous that would help to negatively classify the latter sentence.

Other information can factor into contextualization as well. Is there an unusual degree of italicization and bolding? Is there preceding or succeeding information, for example a reply, that modulates certain qualities that might seem like otherwise clear signals? Contextualizing a document is one of the perennially difficult tasks in NLP, not least of which because there are so many possible ways to do so (and just as changes in language necessitate adjustments to the model, so do new contexts emerge and shift). But taking them into consideration is necessary to minimize sloppiness and maximize accuracy and precision when your application is ultimately deployed.

Don’t necessarily rely on just one model

Sometimes the best way to improve a model is to build a new one entirely and integrate them together in a single pipeline. This can be particularly helpful when trying to contextualize a document. For example, one model can classify a document as violent or non-violent looking just at the vocabulary itself. If it’s labeled as violent, it can then be passed through another model that examines all the data clipped from the first, like punctuation, style, and so on, to determine if the document’s context changes the text’s intent.

By divvying up jobs among multiple models, your application benefits in two ways. First, each model can focus more granularly on its task and perform better than a single model trying to do everything — Machine Learning division of labor. Second, it prevents runaway complexity in your model that makes adjustments more difficult (and resource intensive) and causality more difficult to pinpoint.

Deploying an additional model is especially beneficial when it can be trained on open or existing data that’s already labeled (or is very easy to label) so that building it doesn’t require you to effectively double the time spent labeling data. If, say, you wanted to include a step in a pipeline that first determines the language of the document, there’s no shortage of pre-labeled corpora to begin with. But even less straightforward examples abound, and it’s worth doing a search as you build to see if an existing dataset is available for you to add an additional model more easily than you might initially expect.

Keep humans in the loop

Lastly, never forget what you’re ultimately analyzing is the output of humans — as such, humans themselves are necessary to ensure your model is doing what it’s supposed to, flag false positives, and overall provide quality assurance.

If your application is processing millions of documents every day, you obviously won’t be able to review every single classification or output. But having a few people regularly read through a sample of the results goes a long way to spotting any red flags. In certain applications, you might also pass every single document that’s assigned a particular label to human review so that you don’t end up mistakenly automating an action that deserves rigorous discretion. As a rule of thumb, the more consequential your model’s results are on people’s lives, the more critical human review is.

Contact Strong

Contact us to learn more about how we help top organizations build, deploy, and maintain state-of-the-art NLP systems.

*This article was first published on the Strong Analytics blog here.

Strong Analytics builds enterprise-grade data science, machine learning, and AI to power the next generation of products and solutions. Our team of full-stack data scientists and engineers accelerate innovation through their development expertise, scientific rigor, and deep knowledge of state-of-the-art techniques. We work with innovative organizations of all sizes, from startups to Fortune 500 companies. Come introduce yourself on Twitter or LinkedIn, or tell us about your data science needs.