New customer data platforms make it possible for marketers and analysts to combine these inter-enterprise insights in a trusted and transparent way, without losing control of the data. This enables organisations to securely collaborate over data and make more informed and data-driven decisions.

First, second or third party data?

First party data is the data collected by a company about their customers, whereas second party data is another company’s first party data. The combination of the two creates unique insights that help organisations to produce more detailed and accurate audience profiles.

Conversely, third party data is usually acquired from a data broker and is widely available, so competitors are likely using the same external information and are making the same assumptions. As Forrester notes, “using ill-gotten or badly aggregated data can be riskier than sharing data carefully and securely with select trusted business partners”.

Understand your audience

By tapping into second party data sources, organisations can double the impact of their data by more accurately targeting customers, validating the benefits of a partner relationship or understanding the knowledge of joint users. Gaining collaborative business intelligence is the premise behind data cooperatives, where companies pool their data into a centralised location where is can be integrated and analysed together.

According to a study by Criteo and Forbes, 60% of brands and retailers are currently participating as a part of a data cooperative and 72% of marketers cited “increased revenue” as a key benefit from the expanded customer understanding. These partnerships help to level the playing field, as they enable SMEs to have the same level of customer understanding as the larger and more diversified corporations.

Retain data ownership

Pooling data to combine insight is more complex under the GDPR, as if a customer withdraws consent, then it is the responsibility of the company that collected the data to ensure all other parties have also deleted it. Decentralised customer data platforms can provide the same end result as data cooperatives but, crucially, without the loss of data ownership and the privacy, security and commercial risks.

By using decentralised customer data platforms to securely partner with second party data sources, companies are able to “see the bigger picture” to further understand their audience and tailor campaigns based on detailed and accurate customer personas. They can learn more about demands to answer strategic questions, drive revenue and direct the future of the organisation.

The second party data opportunity was originally published in InfoSum on Medium, where people are continuing the conversation by highlighting and responding to this story.

Digital bridges are built between the datasets and only the query moves, meaning data is never at risk. The query matches identities and returns aggregated statistical answers, making it the most privacy-preserving way to connect customer data.

Data collaboration can be used internally to combine siloed datasets, and externally to work with other parties without sharing sensitive data. It enables users to co-operate with colleagues and partners in a trusted environment.

Generate new insights

The competitive edge gained through connected data can be seen through subscription services. These disruptive businesses are able to demonstrate a higher level of customer understanding, as they know exactly who their customers are, what products they like and how they can be up-sold.

For a long time, Gillette were the clear market leaders in the male grooming industry, but even they have been disrupted by Harry’s and Dollar Shave Club, with their less expensive products, innovative subscription services, convenient home delivery, and quirky ad campaigns.

In this instance, success has been achieved by providing a data-driven customer experience. For companies with different business models, that aren’t able to create a 360 degree customer view, data collaboration offers a solution to enrich and augment customer understanding and gain an advantage over competitors.

By collaborating over data with a colleague or strategic partner, you can access broader insights about your customer and audiences, without the commercial, privacy and security risks associated with data sharing. This creates the opportunity to achieve more with your data and translate this knowledge to provide personalised services and relevant product offerings.

What is data collaboration? was originally published in InfoSum on Medium, where people are continuing the conversation by highlighting and responding to this story.

What is dark data?

Gartner defines dark data as “the information assets organisations collect, process and store during regular business activities, but generally fail to use for other purposes”. For example, this includes “text messages, documents, email, video and audio files, and still images”.

Dark data gained it’s rather unfortunate name from being difficult to process and analyse using traditional methods. It is generally unstructured, often hoarded in one large repository and then left untouched — so it usually gets forgotten about like the bummock of an iceberg.

This hidden data can contain confidential or sensitive information, such as “log archives and other untagged, non-inventoried data” — posing a range of risks if it ends up in the wrong hands. Crucially, dark data is subject to the same data privacy and security laws as it’s lighter alternative (or the hummock). As it has, after all, been generated from people and so some of it may be personally identifiable data.

Time to shine

In a similar way to integrating external data, harnessing previously unanalysed dark data gives companies the opportunity to unearth new insights and inch ahead of competitors. According to the International Data Corporation, dark data makes up a whopping 90% of the data collected by companies — and is growing at an exponential rate.

The technology to find the gems in dark data is out there. Earlier this year, Apple interestingly spent $200 million to buy Lattice Data — essentially to get hold of their machine learning which makes dark data useable. Although shining a light on dark data will require some time and investment, the benefits will be worth it in the long term. There is also the risk that competitors will invest in processing dark data, so companies may end up paying the cost through lost opportunities anyway.

In the same way that only the right big data is useful, only the right dark data is actionable and the rest should be deleted. Finding that diamond ring does require some creative thinking around asking the right questions. For example, as pointed out by Steven Dough in a recent piece for Ad Age, in the context of e-commerce:

Using data collected during an interaction with the customer can also help markers infer and test different types of behaviours for which they don’t have direct answers.

Take hover data. Consumers who hover their cursor over a link for a while but do not click it, or hover over a product but never put it in their cart, might have a totally different intent than someone who clicks and buys or simply scrolls on through.

This “hover insight” can be the basis of exploratory re-marketing campaigns to see if a consumer didn’t take action because of pricing, product details or simply bad timing.

Dough also references the insight which can be derived through examining how many comments a customer reads, and whether they read more comments with positive or negative connotations. This information would enable an e-commerce retailer to take actions and encourage customers to make that purchase.

Of course this is only one use case, but there are plenty more: from improving health care and travel, to making scientific discoveries and preventing fraud. So the choice is up to companies: stick to the norm and risk becoming a laggard, or explore something new and potentially find a diamond.

Why it’s time dark data came out of the shadows was originally published in InfoSum on Medium, where people are continuing the conversation by highlighting and responding to this story.

Day one, 20 June:

• The event opened with a robot, Sophia, on stage. Built by Hanson Robotics, Sophia featured amazing aesthetic detail and was able to interact and showcase an ability to learn from people. But whilst robots might be people’s assumed embodiment of AI, less obvious silent AI transformations are also taking place without us even knowing it — the integration of chat bots into the online customer experience is a great example of this.
• A common theme across all the panels, and articulated especially well by Dr. Pippa Malmgren, founder, H Robotics, was that: “forward-looking human creativity cannot be replaced by AI — only enhanced.”
• Data and AI have a complex relationship. The world of AI needs more data to train models and grow — yet at the same time need to ensure more robust frameworks are being used as more and more data is implemented. Interestingly AI is not overtly covered by the upcoming GDPR, which the ‘Data privacy, data ethics and GDPR’ panel noted could become an issue. Aggregated data is the way forward as it allows companies to analyse and segment customers without needing to be focused on the individual, therefore keeping privacy intact.
• Matthew Hancock, Minister for Digital, delivered the keynote of the day and emphasised the need for a forward looking and optimistic framework for AI. Indeed the panel about the impact on society, felt that AI could create a fulfilling Star Trek economy — where robots do the boring jobs and humans continue to explore new frontiers would be the ideal. AI presents so many opportunities for society, but at the same time it needs to be put in the hands of more people.

Day two, 21 June:

• Lord Young addressed the audience on the second day. He tellingly said that “Mankind needs a job, a purpose.” AI, he claimed, would only enhance this ambition not destroy it. The question is whether government and policies can keep up with the rate of change in order to balance the job losses that come from AI with job creation.
• The insurance sector is ripe for technical disruption, but with only three people in the audience for the insurance and AI panel, what does that say about its appetite for such change? Helen Crooks, Chief Data Officer, Lloyds of London, revealed it has developed a cognitive AI capability. Brooks said that in a regulated environment, AI requires work and testing as you can’t afford to get it wrong but that she was keen the insurance industry changed how it thought about and applied AI. However, when asked to predict the relationship that insurance would have with AI in a year, Matthew O’Kane of Accenture felt the timeframe was too short for any tangible change to be evident.
• Another reference to Star Trek came in the customer experience panel, with William Tunstall-Pedoe, founder of Evi (now Amazon Alexa) saying that the technology has now passed a threshold of usefulness making the computers of Star Trek possible. The key conclusion of this group was that transparency will be key to AI’s success — people do not like to feel duped. Paul Ryan, IBM’s UK Head of Watson, gave the example of Vodafone, which created an avatar for its AI service and people were happy with this because they knew it was not a person. When that line isn’t clear, it was felt the potential to cross into the realms of creepy increased significantly.

After two days of discussion, the potential applications of AI in different industries and its impact are somewhat clearer. No-one shied away from difficult topics such as job losses but what’s less clear is the path forward. Many questions remain around the guidelines and frameworks that will be needed to oversee the application of AI and if, as a society, we’re ready for the next frontier.

CogX: How AI could shape the world was originally published in InfoSum on Medium, where people are continuing the conversation by highlighting and responding to this story.