By Lauren Walton, Director, Data Science

Did you know that measurement doesn’t always require active participation? Yes, traditional survey methods involve asking people questions and logging their responses. But that’s not the only way we measure, and asking questions doesn’t always uncover everything there is to learn.

Passive measurement methods involve getting answers without asking questions directly. These methods attempt to reduce or eliminate measurement error from a person or an instrument being “measured.” Think of it this way: Traditional surveys measure attitudes and behaviors that reside in a person’s consciousness. People aren’t always aware of their underlying feelings or preferences, and passive measurement has the potential to measure these.

At Nielsen, we’re using passive measurement to improve our media research. Our goal is to make participating in a Nielsen measurement service or panel the best experience we can for those being measured. That said, we don’t view passive measurement as a replacement for asking questions about media consumption behaviors. Rather, passive measurement methods are complementary to traditional methods that ultimately become part of a larger data set that provides a more holistic measurement outcome. Understanding how participants engage with our materials helps ensure that our panelists continue to generate great insights for our clients.

Here are a few ways in which we’re using passive measurement at Nielsen.

The Eyes Have It

Eye tracking is a way to measure whether objects that attract the visual attention of people without asking them. It’s been used in market research for years to understand which products study participants are drawn to in an array of scenarios, ranging from retail store shelves to websites. If I ask you to tell me where you look first at this article, you might have difficulty telling me. But, if you had eye-tracking glasses on while reading this article, I would know the order you looked at this article and which areas you spent more time looking at.

At Nielsen, we’ve been using eye tracking for years as a way to understand how people engage with panelist-facing materials ranging from print materials to smartphone apps. Within survey design, we use eye tracking to study how respondents attend to questions, response options and images within the self-administered survey mode.

The real benefit of eye tracking is being able to see what your participant is seeing. Survey tools and other qualitative methods measure what participants can — or want to — tell you, but eye tracking allows researchers to pair gaze data with research participant verbalizations to gain a more robust understanding of experience. From a usability research perspective, we use the technology to understand if panelists are able to understand the directions we provide them that guide them on how to use the measurement technology provided.

Eye-Tracking Use Cases

Example 1. Using Multilingual UX Testing to Develop an Online Computer and Mobile Panel

We wanted to make participating in our Nielsen Computer and Mobile Panel a better experience. The Nielsen Computer and Mobile Panel is an opt-in convenience panel that uses a meter to measure what consumers do on their computers and mobile devices. We conducted iterative rounds of user experience testing in both English and Spanish for the U.S. market as part of our website development.

This is an aggregate heat map for Spanish language Hispanic participants. We see that the call to action button “Empice” or “Get Started” was not fixated on as much compared to the English language participants. The images in the English version use the model’s gaze to direct attention to “Get Started.” We were able to change the Spanish images to do the same after this testing.

Example 2. Paper Diary Eye Tracking

Back in 2012, the data science team sought to understand why some newly re-designed communications materials did not perform as well as previous materials. To understand why, the team conducted an eye tracking usability study and found that many participants failed to see the Nielsen logo in the bottom left portion of the materials. Once the logo was re-positioned, the materials were able to better engage with respondents because they could more quickly identify the Nielsen brand.

Partnering with Nielsen Neuro

Consumer neuroscience methodologies capture non-conscious, unbiased responses to stimuli (like advertising) in a way that doesn’t rely solely on what viewers are able to tell us. In order to leverage this type of research, Nielsen data scientists partner with Nielsen Neuro to use EEG (electroencephalography) and eye tracking to reveal how our target consumers engage with an array of recruitment materials, including recruitment ads on the internet.

EEG involves having study participants wear an EEG “cap” that is embedded with 32 medical grade electrodes, which capture electrical signals occurring between neurons in the brain. From this data, the Nielsen Neuro team derives three primary metrics, each of which help Nielsen’s data scientists to understand how to better engage current and prospective panelists.

The Nielsen Neuro team relies on three primary metrics when conducting its research, each of which help Nielsen’s data scientists better engage with current and prospective panelists:

• Emotional Motivation: The intensity and extent of being drawn to the experience emotionally (a measure of lean in)
• Memory Activation: The formation of connections — with new and past experiences (a measure of relevance)
• Attention Processing: Measures sustained focus and shifts in focus over time (a measure of cognitive effort

Example 3 Digital Recruitment Advertisement Testing for Nielsen Computer and Mobile Panel

To enlist prospective panelists to join our panel, we use online advertisements, and those ads have changed over the years.

To make sure our ads resonate with prospective panelists, we use Nielsen Neuro methods to examine responses to images, taglines and typefaces. Through our testing across English and Spanish advertisements, the team learned that messaging and imagery that emphasizes personal benefit and perspective resonates well with people. We wanted to look at what types of headline messages are most effective.

Across our advertisement study in English and Spanish, we saw that headlines that emphasize a personal benefit work well to engage viewers. “For me” messaging with clear reward or benefit language outperform those that are less “me-centric.” And imagery that closely matches a first-person perspective (e.g., hands on smartphone close up) also drives high engagement. The results from testing the Spanish ads really reinforced this idea: “It’s rewarding being me” resulted in significantly higher levels of EEG engagement than the alternative headline (“it’s rewarding being us”). EEG engagement combines measures of Emotion, Memory and Attention, with an emphasis on Emotion.

Word choice and positioning really matter and can drive large differences in non-conscious response to the creative. This can often be very difficult or near impossible to tease apart using more traditional research methods.

New Technologies Improve our Panels and Our Measurement

The use of emerging technology has broad implications across Nielsen. While we’re using artificial intelligence and machine learning to improve the ways in which we measure consumer behavior, we’re also interested in using them to improve the experiences of our panelists. Not only does this improve our panels, it improves our measurement — which is how our clients truly benefit.

Passive Measurement Sheds Light on Unconscious Insights was originally published in Nielsen Forward/ on Medium, where people are continuing the conversation by highlighting and responding to this story.

By Kevin M. Alvero, CISA, CFE, SVP, Nielsen Internal Audit

Artificial intelligence (AI) represents a huge opportunity for businesses and organizations everywhere, and those that tap into it as it sooner rather than later will be much better positioned to address the challenges and opportunities of tomorrow.

At Nielsen, we continue to lean on AI more heavily to harness the power of data, but that’s just one way in which AI is helping our organization. As companies explore new ways to leverage AI, they need to ensure that they’re managing any associated risks along the way. That’s why our internal audit function has embraced the process of auditing AI. Organizations have to be vigilant to ensure that they are investing the right resources in the right places to capitalize on the opportunities AI represents, that they’re accurately assessing AI-related risks, and that AI projects remain consistent with the organization’s mission, values, culture and larger technology strategy.

Internal audit should be a valuable ally in this effort, but AI poses a number of challenges from an auditing perspective. First, the very definition of AI is somewhat ambiguous. Meanwhile, organizations are aggressively seeking to develop or acquire AI capabilities for a broad range of activities, and AI programs and processes can be highly complex and technical. Finally, there is a lot of hype out there about what AI is and what it can do.

The practice of auditing AI is still being developed. With regard to advanced AI technologies, such as machine learning, there are no universal standards governing AI. The standards, along with frameworks to audit against them, are still being written. ISACA (the Information Systems Audit and Control Association) has issued guidance on applying its existing COBIT framework (use to govern and manage enterprise IT) to AI, and the Institute of Internal Auditors has proposed an approach as well. Meanwhile, an array of organizations, including NIST, ISO and ASTM, are drafting standards for AI. As the practice of auditing AI continues to evolve, however, it has become clear that internal audit needs to remain clearly focused on strategy and governance — ensuring that the organization has a sound AI strategy and a robust governance structure in place to support the execution of that strategy.

Defining AI

Without universal agreement about what AI is and what types of technologies should be considered artificial intelligence, it’s difficult to establish standards for auditing AI. Not only do opinions about what AI is vary, there are varied beliefs about how long it’s been around. Some take a broad view and insist that AI has been around far longer than most people think. Others define it more narrowly, which fuels their stance that AI doesn’t really exist yet and that so-called AI technologies are actually misunderstood as such.

Technologies that are commonly considered to fall into the realm of AI include: deep learning, machine learning, image recognition, natural language processing, cognitive computing, intelligence amplification, cognitive augmentation, machine augmented intelligence, augmented intelligence, and more, but sources disagree about how to group and describe them. For example, some consider robotic process automation (RPA) to be a form of AI because it can execute highly complex algorithms across large and diverse data, which seemingly simulates intelligence because of the way it is programmed to make decisions. On the other hand, others, including ISACA, disagree, stating that “AI does not operate based on a set of predetermined rules,” and that such rule-based functionality is “a hallmark of traditional software engineering.”

To take it a step further, AI itself can be grouped and sub-grouped by technology type. For example, when the Institute of Internal Auditors published its Global Perspectives and Insights on the subject of auditing AI in 2017, it cited the four-type categorization of AI (i.e., I-Reactive Machines, II-Limited Memory, III-Theory of Mind, IV-Self Awareness). Protiviti, meanwhile, chose to group machine learning, deep learning and natural language processing for a 2019 study and termed those technologies collectively as “advanced artificial intelligence.”

The topic of what, exactly, AI is might seem like something for sticklers to haggle over, but it is actually very relevant to internal audit of AI and the audit function’s ability to deliver value and meet stakeholder expectations. When considering what technologies fall under the umbrella of AI for internal audit purposes, the most important factor is understanding how an organization defines AI for itself. By being proactive and initiating open conversations, internal audit can answer the question, What do we, as an organization, mean when we say “AI”? Obviously, this alignment is important to manage stakeholder expectations as it relates to the scope of auditing of AI. However, it also may provide internal audit insight into whether an organization’s definition of AI is broad enough — or narrow enough — for it to be able to perceive risk in the marketplace.

Auditing AI

Despite the varying notions of the nature of AI itself, when it comes to the practice of auditing AI, sources of prevailing guidance generally agree that internal audit should remain focused on strategy and governance.

Strategy

Without a clearly articulated and regularly reviewed strategy, investments in AI capabilities will yield disappointing results, and worse, they could result in financial and/or reputational damage to an organization. Internal audit should be interested in confirming the existence of a documented AI strategy and assessing its strength based on the following considerations:

• Does it clearly express the intended result of AI activities?
• Does it include plans to identify and address AI threats and opportunities?
• Was it developed collaboratively between business and technology leaders?
• Is it consistent/compatible with the mission, values and culture of the organization?
• Is it reviewed and updated?
• Does it consider the supporting competencies needed to leverage AI?

Organizations need their internal audit departments to ask these types of questions, and not just once, but repeatedly. Research continues to show that organizations want their internal audit departments to be more forward-looking and provide more value as it relates to assessing strategic risks. As it relates to supporting competencies, it should be noted that the top concern among board members and C-level leaders for 2019 is that their existing operations and infrastructure will be unable to adjust to meet performance expectations among “born digital” competitors. As such, internal auditors can and should address the question — on an ongoing basis — of whether the organization’s AI strategy is appropriate and can realistically be carried out.

Governance

As with any other major system, proper governance structures need to be established for AI initiatives to ensure that there is proper control and accountability and to determine if AI projects perform as expected and accomplish their objectives.

Again, there is no tried-and-true template to follow to manage AI governance. “The playbook,” writes one expert, “has yet to be written.” Nevertheless, internal auditors should explore the care that business leaders take to develop a robust governance structure in support of AI applications, beginning with big data.

Big data forms the foundation of AI capability, which means that organizations should pay special attention to the governance structures for their data. Internal audit should understand how organizations are ensuring that their data infrastructure has the capacity to accommodate the size and complexity of AI activity set forth in the AI strategy. At the same time, internal audit should understand how the organization manages risks to data quality and consistency, including controls around data collection, access rights, retention, taxonomy (i.e., naming) and editing and processing rules. Internal audit should also consider security, cyber resilience and business continuity and assess the extent to which the organization is prepared to handle threats to the accuracy, completeness, and availability of data.

AI value and performance is also dependent on the quality and accuracy of the algorithms that define the processes that AI performs on big data. Documented methodologies for algorithm development, as well as quality controls, must exist to ensure that these algorithms are written correctly, free from bias and use data appropriately. Internal audit should also understand how AI system decisions are being validated and whether those decisions could be defended if they were to be challenged.

In addition to governance around data and AI algorithms, governance structures should also be examined to determine whether:

• Accountability, responsibility and oversight are clearly established.
• Policies and procedures are properly documented and followed.
• Those with AI responsibilities have the necessary skills and expertise.
• AI activities and AI-related decisions and actions are consistent with the organization’s values, and ethical, social and legal responsibilities.
• Third-party risk management procedures are performed around any vendors.

The Audit Department

In order to audit AI effectively, internal audit functions must ensure that they have (or can acquire) sufficient resources, knowledge and skill to perform audit procedures around AI, even if they do not necessarily have expert-level knowledge on staff. This has proved to be a challenge. According to the 2018 North American Pulse of Internal Audit, 78% of chief audit executives indicated it was extremely or very difficult to recruit individuals with data mining and analytics competencies. Nevertheless, internal audit functions should work to steadily increase their AI expertise through training and talent recruitment, because organizational dependence on AI is only going to increase in the coming years.

Success in auditing AI is not directly dependent on technical expertise. Success depends on assessing strategy, governance, risk and process quality — all of which internal audit excels at from an independent, cross-departmental point of view. These are things that internal audit should remain focused on when it comes to AI.

The Time to Focus on Auditing AI is Now

Despite a rapidly changing marketplace and the sometimes hazy nature of AI, internal auditors can provide valuable, fundamentally sound assurance that the organizations they serve are pointing their AI investments in the right direction, accounting for risks and opportunities, and executing on their business objectives as it relates to AI. The sooner internal auditors do this, the better, simply because AI, in all its various forms, is not going away. It’s gaining momentum. Internal audit functions should become as accustomed to auditing and delivering assurance around AI as they are other business processes because in the coming years, it will become increasingly difficult to find an area of the business that does not leverage AI in some way. Focusing on strategy and governance will enable auditors to do this in spite of rapidly emerging technologies and risks.

Don’t Let the Race to Embrace AI Overshadow the Needs to Audit Your Advancements was originally published in Nielsen Forward/ on Medium, where people are continuing the conversation by highlighting and responding to this story.

By Chris Ballard, Machine Learning Research Leader, Nielsen Connect Data Science

At the supermarket shelves, consumers make their purchase choices based on an array of information, including pack design, market positioning and product claims. But what if we remove the physical shelf from the equation?

In an online environment, the options are far less finite and package information becomes that much more critical in driving purchase decisions. Think about it: If a consumer can’t pick up the product, hold it in their hands and inspect it, every aspect of a product’s packaging has to work that much harder.

Product pack information isn’t just needed for online retailers, but the acceleration of digital channels and the growing wealth of choice in the retail market shines a spotlight on a historic efficiency impediment: Manually converting this type of information into usable data is painstakingly laborious. It’s also encumbered by the need for consistent human involvement. Specifically, human “coders” identify product information and then enter it into a dedicated data field in the data schema used to record this information.

While recording product information manually might be an option for small product groups, it’s far less feasible at scale across the entire consumer packaged goods (CPG) space — dense with variety, myriad categories and frequent new product offerings. Until now.

As the source for one consumer truth, Nielsen helps clients understand consumers and the decisions they make along their paths to purchase, and product packaging plays a critical role in those decisions. To help retailers track this information, Nielsen Brandbank collects images of product packaging, which it turns into structured data and syndicates to retailers for use in their e-commerce grocery sites. This information includes ingredient listings, nutrition tables, manufacturer branding and standard industry logos, such as whether the packaging comes from recycled material and if the product meets specific dietary needs (e.g., vegan).

Given our footprint in the space, in addition to understanding the arduous process of converting package information into usable data, the Nielsen Connect AI R&D team decided to tackle the challenge of corralling this data in a better way.

The team’s goal was to develop an automated approach to recognize specific sections of product packaging based on section type, such as ingredient listing and nutrition table. Since we have many thousands of images of product packaging and the corresponding structured data readily available, we wanted an image recognition algorithm that would be able to locate the text for the information on the image, and convert it to structured data. Doing this would reduce the amount of manual effort that the current process requires.

At the onset, we needed to assess which tools were already available that we might be able to leverage. So we started analyzing convolutional neural networks (CNNs) as a possible option, given their frequent use in object detection algorithms. However, as we explored the suitability of CNNs, we began to realize that they had limitations for our use case. Since our goal is to capture sections on the packaging that typically contain text, we needed to incorporate other information in addition to the image to help the algorithm more differentiate between different sections more precisely.

Why the need to be so precise? Now more than ever, consumers are focused on ingredients, product claims and transparency. And that means that product labeling can make or break a sale. It can also have much more serious implications. If a consumer has an allergy, for example, and a product’s labeling is incorrect, the consequences could be much worse than just a lost sale.

Object Detection 101

To get started, let’s look at object detection, which is the task of identifying specific objects and their location within an image. In a street scene, for example, we might be tasked with identifying the pedestrians, which could involve sorting through any number of other objects in the scene. If we’re designing AI for a self-driving car, we need to be able to identify the location of any objects to avoid, such as pedestrians, other cars and bicycles. So object detection algorithms typically classify object type as well as location.

So how does this relate to detecting information on product packaging?

Unlike in a street scene, the objects that need to be detected on a product package are specific regions that contain specific information. The trick with product packaging, however, is that there’s no uniform structure across products. Manufacturers need to distinguish their products from others, so it’s common to see variances in how different brands style and position the information on their packs. The degree of overlap from region to region further complicates the issue. The ingredients, for example, may “flow” around the nutrition or be entirely independent from it. For these reasons, using a naïve approach to extract all of the text from an image will yield unstructured text with no context. This diminishes the quality of the extracted data, given the “noise” of the surrounding information.

To solve for the lack of consistency across products, we use an object detection algorithm to both classify and locate specific regions on the packaging before we extract the text. For example, as well as locating the ingredient listing on the packaging, we also label it as an ingredient list to differentiate it from other types of section. That way, we ensure that we extract the text in context. This has a number of benefits. If we know that a portion of text is an ingredient listing, we can clean the detected text using specific ingredient dictionaries, which will yield much higher-quality ingredients than if this process was done without knowing the context.

The Need for New Object Detection Techniques

Certainly we aren’t pioneers in object detection. That said, however, standard object detection algorithms, including Faster R-CNN, aren’t a match for our goals. Networks like Faster R-CNN, which combine region proposal and classification into a single deep network architecture, work well when they’re trained to recognize regions that can be identified with visual cues learned through convolutional layers. For example, there are strong visual indicators for the presence of people in a street scene.

Additionally, when we’re training a Faster R-CNN network to recognize regions on packaging, we usually need the network to locate textual regions, such as ingredients and allergen information. These regions visually look very similar, and we have found that the standard Faster R-CNN architecture does not perform well in these instances.

Appearance and Text Information in a CNN

When training a network to recognize regions on product packaging, it’s not feasible to rely on appearance alone. Textual information plays a key role in differentiating different content regions, and it’s important to consider how humans recognize the differences between different regions of text, such as nutritional content and ingredients.

Importantly, when we read ingredient information, we expect to see listings of included ingredients. Comparatively, we expect that nutritional content will be presented in sentence structure format, complete with standard headings alongside weights and percentages. To address this challenge, we developed a method that generates a text map to inject textual information into the network. A text map is basically a heat map whereby the intensity of the color of the word varies depending on its presence in a region.

To build text maps, we use optical character recognition (OCR) to capture text on images. We then color the text according to the probability of it being present in the region. For example, the word “ingredients” and commas will appear in greater intensity than other words because they are much more likely to be present within the ingredient section. Similar text maps can also be generated for other sections, such as nutritional facts.

Standard convolutional networks typically ingest three channels of information: one for each of the RGB channels containing visual information. To inject text maps, we provide this information as additional channels, which allows the convolutional layers to generate feature maps that account for both the visual and textual appearance combined. We create three additional channels for each text map, which encodes word-level information by color according to predefined metrics, such as punctuation, word occurrence and Bayesian distance between words.

In testing, we have found that our automated recognition system performs significantly better than standard Faster R-CNN for detecting ingredient and nutritional information.

While we’ve always provided retailers and other clients with product characteristics that allow them to segment markets (e.g., by fat percentage), integrating ingredient and nutrition detection will provide much richer data sets. And what’s more, the benefit isn’t limited to our clients. By incorporating ingredient data into our sales analytics, our clients will be able to see how the presence — or absence — of certain ingredients factor into consumer purchase decisions. And that will, in turn, drive the innovation that will feed consumer demand.

Notes

The Nielsen Connect AI team first presented this research at CVPR 2019 in the Language and Vision Workshop. The team also presented the research in a paper dated May 2019 (authors: Arroyo, Roberto & Tovar, Javier & J. Delgado, Francisco & J. Almazán, Emilio & G. Serrador, Diego & Hurtado, Antonio).

How We Use Machine Learning to Turn Product Packaging into Structured Data was originally published in Nielsen Forward/ on Medium, where people are continuing the conversation by highlighting and responding to this story.

Truth in measurement has never been more important than it is today. Therefore, truth is our only agenda. But arriving at that truth has never been more complicated. While many view big data as a panacea for measurement in a digitally rich world, we know it’s not that simple.

Nielsen’s panels have been the foundation of person-level measurement for decades, and they remain so today. The growth of big data, however, can’t be ignored as a source of valuable information. But big data alone isn’t suitable for representative measurement. Think about when you change the channel on your TV. That change becomes part of big data, but there is no record of who made the change or who witnessed it.

To highlight the shortcomings of big data from a measurement perspective, we conducted an analysis in the U.S. earlier this year that compared set-top box data with set-top box data that we calibrated with Nielsen panel data. The analysis found that the uncalibrated data is inherently biased and underrepresents minority audiences.

That’s not to say, however, that big data has no value. Quite the opposite. But it does need to be grounded in a foundational truth set. That’s where our panels and artificial intelligence (AI) come into play. Our panel data — the key to persons-level measurement — is the perfect truth set for training big data.

Through the application of AI, we use big data to dramatically broaden our measurement capabilities while preserving quality and representativeness. Today, AI is integral in our measurement methodologies. For example, it played a pivotal role in the development of our enhanced measurement capabilities for local TV markets, which combines the scale of big data (return path data {RPD} from TV sets) with fully representative in-market panel data.

As we sought to integrate RPD into our local measurement, we identified four key uses of AI.

Recognizing Data Patterns

As we researched ways to integrate RPD into our measurement, we identified limitations associated with the RPD through what we refer to as “common homes analyses.” For these analyses, which continue today, we compare tuning data from Nielsen meters with RPD tuning data. These analyses cover more than 5,000 homes (12,000 TVs) each month and have found that RPD misses some tuning.

To address this shortcoming, we developed a patent-pending technique that uses classifiers to recognize the patterns associated with the missing tuning in RPD homes. From there, AI algorithms remove these homes from use in measurement.

Knowing When Set-Top Boxes are On and TV Sets Are Off

Nielsen’s common home analyses analyze more than 77 million minutes of tuning in a given month, which provides powerful insights. That tuning, however, is not always accurate. For example, people don’t always turn off their set-top-boxes when they turn off their TV sets. The RPD presents these situations as TV viewing even though no one is watching.

We can overcome this limitation by employing deep learning classifiers to identify situations where the set-top-box is on while the TV is off. The algorithm then removes the invalid tuning from the RPD.

Identifying Household Characteristics and Demographic Information from RPD

RPD is nameless and faceless, and it can’t provide demographic information. Demographic information is critical in correctly representing all segments of a population. And beyond that, accurate measurement means being able to measure people, not just households.

So to unlock the powerful information within RPD homes, we calibrate RPD with the known characteristics, demographics and tuning information from more than 45,000 Nielsen metered homes and third-party assigned characteristics and demographics. We then add these inputs into a patent-pending recurrent neural network and mixed integer programming technique that accurately identifies the characteristics and demographics of RPD homes. This AI algorithm allows us to accurately report demographic characteristics of the persons and households.

Determining Set-Top-Box Room Location

Nielsen panel information provides viewer information and viewing location. RPD does not provide either. We can, however, obtain that information from RPD through AI. We use a scientifically-proven methodology to identify which household member is watching and where viewing is occurring within the home.

Research has found that room location is one of the key predictors of which household members are in a viewing audience. So we use a classifier to identify room location of the set-top-box where tuning is happening in RPD homes. That way, we can use this variable in the viewer assignment process.

With so much information available today, it’s tempting to view big data through rose colored glasses. Without a connection to persons, however, big data is far from accurate. We found AI as a powerful way to address bias in the big data and delighted to bring the innovation to the clients. This innovation benefits from both the wealth of information that big data sources provide and a truth set that ensures that they can plan, activate and measure based on data that is representative and accurate.

Artificial Intelligence is the Link Between Big Data and Persons-Level Measurement was originally published in Nielsen Forward/ on Medium, where people are continuing the conversation by highlighting and responding to this story.

If, as science fiction writer Arthur C. Clarke posited, “any sufficiently advanced technology is indistinguishable from magic,” then it sometimes feels like data science is viewed as pulling rabbits out of hats. This narrative is, of course, misguided. It’s the perception of what I call “data magic,” where people believe data can be pumped into one end of the “data-science machine” and the perfect widget (the solution to everyone’s problems) will emerge on the other side.

To a considerable degree, this is because the vast majority of people don’t understand the workings of data science, and when you get into the more advanced areas, such as deep learning, even many data scientists will acknowledge they don’t understand the many levels of complexity. But if you’re a data scientist, you (mostly) know enough to know when you’re out of your depth. Outside the data science community, however, that’s not always the case. And that’s understandable.

In part, it’s our own fault. It’s a tough club; historically difficult for “outsiders” to penetrate, much less understand. The languages of data science have been held close to the chest. Just as the Roman Catholic Church selected Ecclesiastical Latin as the core communication language to control messaging, particularly through the Middle Ages and into the early-Modern period, we data scientists could be accused of similar actions — though obviously not on the same scale, nor having such a direct impact on entire populations. But, just as the Reformations of the 16th century led to unshackling language, making the tenets of the various Christian churches more available to the masses, data science must now further extend its vernacular.

A shared language would allow us to move beyond believing that data scientists have mystical capabilities to solve any problem, by running data through an AI environment to produce the desired results, as if by magic. It would help people understand that data science isn’t a magical panacea.

In fact, if you truly want advanced data science, one of the worst things to do is assign a data scientist to solve isolated or ad hoc problems, as this will silo communication by keeping data science in the back room.

Rather, the best way to proliferate data science is to expose enterprise-level problems, understanding that, if done right, data science is a team sport. Having multi-disciplinary teams dedicated to products or customers yields superior business results and develops cross-functional understanding. A cohort including a commercial associate, a product manager, an engineer, a data scientist and representatives from other key functional organizations should be locked in an uninterrupted room for a meeting of the minds focused on the biggest needs and opportunities. This is where the true magic happens.

Still, a traveler on this journey should be cognizant of the warning signs. If, in working with clients or other third parties in the spirit of collaboration, you begin searching for solutions explainable to absolutely everyone, take pause. Just as a magician does not limit a performance to elementary tricks, the audience understands that data scientists should not default to easily explainable solutions. Deliberately watering down the process may have the collateral effect of delivering a less-than-optimal solution for the problem. It’s a balance.

That balance relies on companies trusting in the capabilities of their data scientists. Trusting data scientists will share our language as much as we can, but not dilute solutions when things turn too technical, and that we’ll always remain true to our discipline This is the kind of trust and balance enabling technologically advanced companies to get at their respective truth sets.

For more insights from Nielsen, visit nielsen.com.

The True Magic of Data Science was originally published in Nielsen Forward/ on Medium, where people are continuing the conversation by highlighting and responding to this story.