A data silo is an isolated dataset that cannot be analysed with other datasets held by an organisation. This fragmentation, means that it’s not possible to run analysis across the entire business and gain a true picture of customer behaviour. Flying half blind has a dramatic impact on business success and enables those who have a complete picture to succeed.

“Any enterprise CEO really ought to be able to ask a question that involves connecting data across the organization, be able to run a company effectively, and especially to be able to respond to unexpected events. Most organizations are missing this ability to connect all the data together.”
Tim Berners Lee*

Where do these data silos come from?

The most common example of siloed datasets is where it is inherent in the business structure. With so many companies now diversifying into other products, it is natural that these data silos would appear. For example, some of the larger supermarkets in the UK have spread their wings into loyalty programmes, banking, insurance, and other retail ventures. It is common that the customer data for these different divisions is held separately.

With this customer data being held separately, it is not possible for the group level leadership to quickly and easily run analysis across their entire customer base. They may be able to quickly take a look at their supermarket customers, but can’t then easily determine if a large proportion of those customers also bank with them, and therefore what sort of cross-promotional campaigns they could be running.

Why don’t they simply consolidate their data silos?

Consolidating siloed datasets has been one option for those looking to get a complete understanding of their customers. However, the increased focus on data protection in GDPR makes this increasingly less attractive for a variety of reasons. First, the additional processing of personal data requires a solid legal basis, perhaps bringing together several data sets themselves run on different grounds. Secondly, the creation of a single data pool is an additional point of risk, with one exploit potentially leading to a much bigger breach than if it occured in one silo. Finally, there may be solid business reasons for keeping the data from the different divisions separate and not having entanglements which would, for example, prevent rapid re-organisations or spin offs in the future.

So, continuing with the above example, while the supermarket would likely have a clear legal basis to merge its customer data from its grocery and financial services divisions, the risks and logistics of creating a single data pool are problematic.

The second option, that has been more popular, is to use a Business Intelligence Platform, that brings together various datasets and provides a complete picture of the data. However, the majority of these platforms require the raw data to be physically moved (through the cloud) and combined.

So what’s the answer?

Share knowledge, not data

The answer sounds simple, but the technology to achieve it is far from it. The answer is to take anonymised statistics from one dataset and make a statistical comparison with another. Our Quantum Technology allows data to be analysed across completely disparate datasets, without sharing or pooling the raw data. This will allow any business to run an aggregate analysis across any dataset within their company, unlocking knowledge that can help them to:

• Improve customer knowledge across all aspects of your business
• Enhance customer service
• Understand the wider implications and impact of marketing activity
• Increase productivity and reduce operational costs

➡️ See this technology in action in our new video that demonstrates our first tool, Link.

*Source Q&A with Tim Berners-Lee — Bloomberg

Unlocking knowledge from internal data silos without drowning in a data lake was originally published in InfoSum on Medium, where people are continuing the conversation by highlighting and responding to this story.

New customer data platforms make it possible for marketers and analysts to combine these inter-enterprise insights in a trusted and transparent way, without losing control of the data. This enables organisations to securely collaborate over data and make more informed and data-driven decisions.

First, second or third party data?

First party data is the data collected by a company about their customers, whereas second party data is another company’s first party data. The combination of the two creates unique insights that help organisations to produce more detailed and accurate audience profiles.

Conversely, third party data is usually acquired from a data broker and is widely available, so competitors are likely using the same external information and are making the same assumptions. As Forrester notes, “using ill-gotten or badly aggregated data can be riskier than sharing data carefully and securely with select trusted business partners”.

Understand your audience

By tapping into second party data sources, organisations can double the impact of their data by more accurately targeting customers, validating the benefits of a partner relationship or understanding the knowledge of joint users. Gaining collaborative business intelligence is the premise behind data cooperatives, where companies pool their data into a centralised location where is can be integrated and analysed together.

According to a study by Criteo and Forbes, 60% of brands and retailers are currently participating as a part of a data cooperative and 72% of marketers cited “increased revenue” as a key benefit from the expanded customer understanding. These partnerships help to level the playing field, as they enable SMEs to have the same level of customer understanding as the larger and more diversified corporations.

Retain data ownership

Pooling data to combine insight is more complex under the GDPR, as if a customer withdraws consent, then it is the responsibility of the company that collected the data to ensure all other parties have also deleted it. Decentralised customer data platforms can provide the same end result as data cooperatives but, crucially, without the loss of data ownership and the privacy, security and commercial risks.

By using decentralised customer data platforms to securely partner with second party data sources, companies are able to “see the bigger picture” to further understand their audience and tailor campaigns based on detailed and accurate customer personas. They can learn more about demands to answer strategic questions, drive revenue and direct the future of the organisation.

The second party data opportunity was originally published in InfoSum on Medium, where people are continuing the conversation by highlighting and responding to this story.

Too often, critical M&A decisions are made using secondary research or sample data sources, which can provide an unreliable analysis of the potential synergy between the two customer bases.

When entering into negotiations, both parties in an M&A deal should be able to analyse and evaluate what additional value would be achieved through the merger or acquisition, without sharing or pooling their data assets.

Synergy, InfoSum’s new M&A focused solution enables the pre-deal analysis of real customer data, without sharing or pooling it.

Built on our core Quantum technology, it enables both parties to get a real view of the overlap in their customer bases, earlier in the process than ever before.

How it works

Quantum connects and analyses the spatially separated and disparate datasets without any exchange of raw data occurring, in a trusted software environment.

Each party uploads their data to a secure and isolated server, where it is normalised, encrypted and the personal identifiers removed.

During the normalisation phase, Quantum adapts each party’s data into the same format, so they can be compared and analysed together.

Identities are also matched at an individual level and the results are collated at an aggregate level — without bringing the datasets together.

This process creates the optimal match of unique identities and attributes allowing the datasets to be statistically compared, and insight unlocked.

The benefits

Both parties in an M&A deal are able to quickly and easily identify the overlap in their two customer bases earlier in the process; enabling them to make informed business decisions around the value of the acquisition or merger.

Synergy can also be used during the latter integration phase, to run reports across the datasets prior to full integration, or to unlock the knowledge where data is to remain siloed.

Gaining this early intelligence will allow businesses to move forward in a deal with a genuine understanding of not only business capabilities unlocked through the merger or acquisition, but also the new customer base they will be servicing.

➡️ Find out more about Synergy and InfoSum’s Quantum technology.

Mergers & Acquisitions: How InfoSum enables pre-deal data analysis was originally published in InfoSum on Medium, where people are continuing the conversation by highlighting and responding to this story.

Digital bridges are built between the datasets and only the query moves, meaning data is never at risk. The query matches identities and returns aggregated statistical answers, making it the most privacy-preserving way to connect customer data.

Data collaboration can be used internally to combine siloed datasets, and externally to work with other parties without sharing sensitive data. It enables users to co-operate with colleagues and partners in a trusted environment.

Generate new insights

The competitive edge gained through connected data can be seen through subscription services. These disruptive businesses are able to demonstrate a higher level of customer understanding, as they know exactly who their customers are, what products they like and how they can be up-sold.

For a long time, Gillette were the clear market leaders in the male grooming industry, but even they have been disrupted by Harry’s and Dollar Shave Club, with their less expensive products, innovative subscription services, convenient home delivery, and quirky ad campaigns.

In this instance, success has been achieved by providing a data-driven customer experience. For companies with different business models, that aren’t able to create a 360 degree customer view, data collaboration offers a solution to enrich and augment customer understanding and gain an advantage over competitors.

By collaborating over data with a colleague or strategic partner, you can access broader insights about your customer and audiences, without the commercial, privacy and security risks associated with data sharing. This creates the opportunity to achieve more with your data and translate this knowledge to provide personalised services and relevant product offerings.

What is data collaboration? was originally published in InfoSum on Medium, where people are continuing the conversation by highlighting and responding to this story.

This is because the new legislation tightens the rules around businesses’ ability to process personal data — making it more risky and complicated to share and purchase raw customer data.

This legislation comes at a time when businesses, from the likes of the big tech giants to SMEs, increasingly rely on generating insight from customer data to improve their offering and the way they engage with consumers. Also, consumers increasingly worry about data privacy and their information being misused by brands. Our recent Exploring Data research found that 61% are worried about their data being stolen or sold to spam marketers.

So how can organisations continue to successfully derive insight from their customer data in a GDPR world, yet keep their customers happy in the knowledge that they are keeping their data secure when sharing insight with others?

Aggregated data

The answer is to use aggregated data. This is information gathered and expressed in a summary form for purposes such as statistical analysis, and so is not personal data for the purposes of data protection law, such as GDPR.

By using statistical insights from anonymised personal data it’s possible, for example, to build look-a-like audiences for advertising and spot previously unknown trends or patterns that open up opportunities to create new product lines, upsell and improve customer engagement.

It offers brands a risk-free way to deliver an exceptional level of insight on their customers to help drive highly targeted marketing activity in a GDPR world.

Consumers prefer brands to use aggregated data

From our Exploring Data study, we found that consumers are much more comfortable with organisations using aggregated data, because they get the benefits of businesses sharing their data — including better products services and communications — without the privacy concerns of individual targeting.

We found that:

• ‍80% of consumers are comfortable with brands using aggregated data to create products, in comparison to 68% for personal data
• 78% of consumers are happy for aggregated data to be used by businesses to recommend products, in comparison to 62% for personal data
• ‍74% of consumers are comfortable with brands using aggregated data in marketing, in comparison to 55% for personal data
• ‍71% of consumers are comfortable with aggregated datasets being connected, in comparison to 49% for personal data
• ‍58% of consumers are happy for aggregated data to be sold to other companies, in comparison to 31% for personal data

A good example of an increase in the comfort level amongst consumers from brands using aggregated data comes from some questions we asked in our research about how they would feel about their automated vacuum cleaner collecting room layout data to be sent back to the manufacturer. Only 32% said this was acceptable. However, when they were told that this data would be stored separately and anonymously, and aggregated together with other customer data to create better furnishings, 57% found this acceptable.

Consumer Attitudes — Shaping Insight Together

Additional security

When looking to reap the rewards of using aggregated data brands should look to use a SaaS technology such as ours. We allow marketers to share insight without moving the actual data — protecting the customer data. This means there’s no need to worry about third parties being able to copy, or worse, leak it. Whilst at the same time it enables brands to deliver an exceptional level of customer insight from aggregated data to help drive their business forward in a GDPR compliant way.

So, whether it’s an improved customer experience, improved retention and loyalty, bigger share of wallet, new products and services, or strategic partnerships you are looking to deliver and our new data collaboration technology with privacy and security at the core of its design will give best results. Your customers will thank you for it.

Why aggregated data benefits everyone in a GDPR world was originally published in InfoSum on Medium, where people are continuing the conversation by highlighting and responding to this story.

A few years ago, I attended a networking event in Berlin where the topic of anonymised data collection was discussed.

One particular speaker told a story of how Jawbone wearers in the Bay Area reacted to the Napa earthquake in 2014. By collating the data received from the Jawbone devices, they were able to draw a picture of how widespread the shock of the earthquake was felt, and thus revealing the spread of the event and how people were affected.

They were able to see that 93% of Jawbone wearers living within 15 miles of the epicentre of the ‘quake woke up at 3.20am, and 45% didn’t go back to sleep for the rest of the night.

The reaction to this story divided the room immediately!

Half of the attendees were outraged at what they saw was a violation of intimate information (this was data that was collected directly from the wearers beds after all I suppose!), and the other half of the room marvelled at the wonder of this kind of insight, and brains whirred with the practical application of this level of data analysis.

I was in the latter category.

After all, what was really taken from the Jawbone users that would leave them feeling vulnerable?

There was no personal information, nothing identifiable, and nothing that can be used against them. I’m taking an educated guess, but I would think that the only data that was used to produce these remarkable figures were ‘heartrate ’ and ‘movement’ (although technically, any physics fan will tell you that you can’t accurately measure both of these things at the same time without breaking the uncertainty principle…)

People are right to be guarded about their personal information, absolutely. Most of it is treasured, valuable and, well, personal.

But would you have felt violated if you had been one of the Jawbone wearers in California that night? Knowing that without giving anything away, you have inadvertantly helped scientists gain a better insight into natural disasters?

I sure wouldn’t have minded.

This is just one example, but there are countless others which show how mediated, anonymised personal data can potentially unlock new insights and tell us more about how we, as people, act within and react to, the world around us.

Google Maps, is another great example. They use anonymised data collected from their iOS app on iPhones, and location data from Android devices, to warn drivers of upcoming traffic problems and suggest a better route. I would wager that more than half of us have benefitted from this kind of service in the past, and it wouldn’t be possible unless your data was anonymously shared.

Jawbone and Google are not the only tech companies who collect this kind of data. Most, if not all do. Often, this data is collected as a by-product of another function; location information from an iOS app, shopping trends for an online store etc. This data is then stored and then left unused.

The real power comes however, when these data sets are combined. What if the Jawbone data was compared with that from energy companies, medical centers or public transport operators?

We could get an even more holistic view of how people respond in this kind of scenario, and be able to plan and react better in the future across the board.

Bringing together different anonymised data sets, can provide us with real insights, with real world applications being the benefit.

Maybe it’s time for us as individuals to rethink the anxiety we feel about how some of our data is shared, but maybe it’s also time for more companies like Jawbone and Google to use the rich pool of data that they collect for the greater good.

Shared, anonymous knowledge, when used in a responsible manner, can only lead to great things.

Sharing data from your bedroom. Problem or progress? was originally published in InfoSum on Medium, where people are continuing the conversation by highlighting and responding to this story.

Sharing and moving data into a centralised pool has become an outdated solution. As shown by last week’s enormous Equifax breach, combining large amounts of data is far too attractive for hackers and risky for companies, even with the most robust cyber security.

With the arrival of more stringent data protection laws, such as the GDPR increasing fines for data breaches up to 4% of global annual turnover, centralising data is no longer enough to protect customer privacy.

But this shouldn’t slow down innovation.

I founded InfoSum as I wanted to create a new and better way for all companies, regardless of size, to collaborate over consumer insight and become smarter together — without the privacy or security risks that come with centralising data or sharing it with third parties.

I wanted to make it possible for every company to understand more about their customers and quickly react to market changes. To take the level of intelligence held in monolithic centralised databases and make it universally accessible — all whilst maintaining individual data privacy and GDPR compliance.

Our technology

At InfoSum, we have developed a new and unique form of decentralised technology which can analyse data without moving it.

We have solved the problem of companies storing data in different formats and invented a way that they can all be treated as one. For example, different companies may store age in various ways — such as date of birth, exact age or as an age range — but our AI powered technology can understand and standardise this information.

So even though no two data sets are the same, our technology makes them compatible.

Our vision is to use this capability to help companies understand a broader perspective of their customers’ lives and analyse previously incompatible data sets. So every company can discover new sources and dimensions of data whilst meeting all international data protection regulations, and become more in tune and relevant to their customers than currently possible.

For example, a retailer knows how customers shop with them but knows little else about their lifestyle, behaviour and interests. Using our technology, the retailer can access deeper insights based on the characteristics of their top customers. This will enable insight driven innovations, like recognising an interest for and then introducing an activewear line, which increase sales and satisfy consumer demand.

With the support of our innovative VCs, we have built a world-class team and are busy working on bringing our first product to market. We also recently announced an additional $5 million seed funding, taking the total to $8 million. Investors include Mark Suster (Upfront Ventures), Roger Ehrenberg (IA Ventures), Saul Klein (LocalGlobe), and Mike Chalfen (Mosaic Ventures).

InfoSum: Pioneering the next generation of cross-company insights was originally published in InfoSum on Medium, where people are continuing the conversation by highlighting and responding to this story.

The announcement earlier this month by the UK government of an overhaul of data protection laws sparked headlines heralding the new rights of citizens to control their data.

To those who have been paying close attention to data laws, this came as no surprise. The new provisions around consent and the ability to withdraw consent are in the EU’s General Data Protection Regulations, due to come into force in May 2018, and published many years ago, and 2017 manifestos for both the Conservative and Labour Parties included such a pledge.

The most eye catching announcement for me was the news that the government would create a new criminal offence: “of intentionally or recklessly re-identifying individuals from anonymised or pseudonymised data” punishable by an unlimited fine.

Whilst I welcome this unexpected legal protection for anonymised and pseudonymised data, I fear that it signals a shift in the burden to protect such data away from the original data controller, whose responsibility it is to ensure that data cannot be re-identified.

Anonymised data is simply data about people where the individuals are no longer identifiable, usually, because the results have been aggregated. At a very basic level, pseudonymisation maintains the data in a non-aggregated form but removes or hashes identifying features, such as a name. Both have the advantage of making the data “non-personal” and taking it outside the scope of data protection laws, making it easier to store, move, share and analyse.

Poorly pseudonymised data is vulnerable to re-identification — sometimes even by people unwittingly doing so, simply because they have some special knowledge of the individuals involved and can recognise them from the details supplied. On a mass scale, re-identification can sometimes be possible when information disclosed can be matched to other information in the public domain.

One of the most notorious examples involved Netflix, which 10 years ago released “anonymised” ratings data as part of a competition to improve its recommendation algorithm. Researchers were quickly able to match ratings given on Netflix with other sources such as IMDb, and identify individuals. This led to a US class action law suit, which was joined by an “in-the-closet lesbian mom”, who claimed she risked being outed by Netflix.

Earlier this month, it was announced that German researchers had been able to purchase “anonymous” browsing histories of 3 million Germans and were identify most of them — as many of these records included a social media handle, which could then be linked to a real person.

In both cases, the re-identification risk was highlighted by the data equivalent of an ethical hacker and was undoubtedly a good deed, as it called out the irresponsible actions of others. Even though the government says it will protect whistleblowers, the threat of criminal sanctions may put off many similar ethical projects and give a free pass to poorly anonymised data.

The protection of whistleblowers should go further and the government should implement a notification system to the ICO so that researchers can easily receive advance clearance for re-identification projects and be shielded from the risk of prosecution.

Of course, there are many non-ethical examples of re-identification, at which this law will be aimed. In its Anonymisation Code of Practice, the ICO gives some examples of the reasons someone might want to do this:

• finding out personal data about someone else, for nefarious personal reasons or financial gain;
• the possibility of causing mischief by embarrassing others;
• revealing newsworthy information about public figures;
• political or activistic purposes, eg as part of a campaign against a particular organisation or person; or
• curiosity, eg a local person’s desire to find out who has been involved in an incident shown on a crime map.

It is quite hard to envisage a real life example of how the crime of re-identification would be the subject of a prosecution, without another crime being committed at the same time, for example fraud. A more sensible approach would perhaps be to make re-identification an inchoate offence, such as conspiracy, which relies on the ultimate aim being the commission of another offence.

I await with interest how the government intends to define “anonymised” and “pseudonymised”, and whether it will encompass even the simplest forms. For example, would it be a criminal offence to identify the individual behind a Twitter handle, or the person who leaves the abusive comment on a blog post?

More importantly for data sets, there needs to be a minimum standard of pseudonymisation that will be required to earn the protection of this new law. The current guidelines are incredibly vague and loose, with the ICO stating that data owners should consider various factors such as who might have special knowledge allowing them to re-identify data (and whether they are likely to attempt it), what other data is in the public domain, the availability of the required computing power, and the consequences for the data subjects.

Of course, releasing badly anonymised data is already going to fall foul of existing data protection but surely this is where the ICO needs to tighten up and lay out clear tests that anonymisation must pass before it can be legally considered non-personal data.

As much as new technology can help attackers re-identify data, new technology is available that prevents it, or — as we are developing — ensures that the data can be used for analysis without the need to release it in the first place and risk re-identification. The wide understanding of re-identification techniques and the availability of technical solutions mean that poorly anonymised data should become a thing of the past, and its release without the appropriate technical protection should also be criminal act.

A shorter version of this article first appeared in CityAM

Shining a spotlight on re-identification was originally published in InfoSum on Medium, where people are continuing the conversation by highlighting and responding to this story.

Few companies actually know what happens to customer data when they share or sell it to third parties. They have no control over where the data goes after it leaves the organisation and where it could be moved onto next.

However, under GDPR, if a customer withdraws consent, it will be the responsibility of the company to ensure all third parties also stop processing this data. To the extent that the original company will be hit with the same non-compliance fine (up to €20 million or 4% of annual global turnover, whichever is higher).

A topical example is the controversial partnership between Google’s DeepMind and Royal Free London NHS Foundation — all for not gaining explicit consent to share health records. Although the possibilities for advancing medical research are vastly increased by being able to share medical data, this data is very sensitive and it is vital that best practices are being used.

To tackle this, as I said during the panel, we need new frameworks for secure data collaboration.

I’m a big believer in aggregated data, as it enables companies to analyse and segment customers without needing to be focused on the individual, therefore keeping privacy intact.

The decentralisation of models is set to completely disrupt how data is handled and will be a game-changer for increasing data privacy.

In addition, the panel’s chair, Ian West, posed the question of whether GDPR would stifle innovation and the consensus was no. I believe that whilst GDPR will bring short-term pain, it will also bring long-term competitive advantage.

The legislative changes will build trust between customers and businesses, which will result in closer relationships and increased loyalty. In addition, GDPR has spurned a whole host of innovative startups in Europe looking to help companies tackle this very problem — who are now ahead of companies in the US and Asia.

Lastly, the world of AI needs more data to train models and grow — but we need to ensure robust frameworks are being used as more and more data is implemented. I firmly believe that the best practice for obtaining and analysing higher quality datasets is through decentralised and aggregated data models.

Enjoy reading this? Join my weekly data digest. 💌

Frameworks needed for secure data collaboration was originally published in InfoSum on Medium, where people are continuing the conversation by highlighting and responding to this story.

Day one, 20 June:

• The event opened with a robot, Sophia, on stage. Built by Hanson Robotics, Sophia featured amazing aesthetic detail and was able to interact and showcase an ability to learn from people. But whilst robots might be people’s assumed embodiment of AI, less obvious silent AI transformations are also taking place without us even knowing it — the integration of chat bots into the online customer experience is a great example of this.
• A common theme across all the panels, and articulated especially well by Dr. Pippa Malmgren, founder, H Robotics, was that: “forward-looking human creativity cannot be replaced by AI — only enhanced.”
• Data and AI have a complex relationship. The world of AI needs more data to train models and grow — yet at the same time need to ensure more robust frameworks are being used as more and more data is implemented. Interestingly AI is not overtly covered by the upcoming GDPR, which the ‘Data privacy, data ethics and GDPR’ panel noted could become an issue. Aggregated data is the way forward as it allows companies to analyse and segment customers without needing to be focused on the individual, therefore keeping privacy intact.
• Matthew Hancock, Minister for Digital, delivered the keynote of the day and emphasised the need for a forward looking and optimistic framework for AI. Indeed the panel about the impact on society, felt that AI could create a fulfilling Star Trek economy — where robots do the boring jobs and humans continue to explore new frontiers would be the ideal. AI presents so many opportunities for society, but at the same time it needs to be put in the hands of more people.

Day two, 21 June:

• Lord Young addressed the audience on the second day. He tellingly said that “Mankind needs a job, a purpose.” AI, he claimed, would only enhance this ambition not destroy it. The question is whether government and policies can keep up with the rate of change in order to balance the job losses that come from AI with job creation.
• The insurance sector is ripe for technical disruption, but with only three people in the audience for the insurance and AI panel, what does that say about its appetite for such change? Helen Crooks, Chief Data Officer, Lloyds of London, revealed it has developed a cognitive AI capability. Brooks said that in a regulated environment, AI requires work and testing as you can’t afford to get it wrong but that she was keen the insurance industry changed how it thought about and applied AI. However, when asked to predict the relationship that insurance would have with AI in a year, Matthew O’Kane of Accenture felt the timeframe was too short for any tangible change to be evident.
• Another reference to Star Trek came in the customer experience panel, with William Tunstall-Pedoe, founder of Evi (now Amazon Alexa) saying that the technology has now passed a threshold of usefulness making the computers of Star Trek possible. The key conclusion of this group was that transparency will be key to AI’s success — people do not like to feel duped. Paul Ryan, IBM’s UK Head of Watson, gave the example of Vodafone, which created an avatar for its AI service and people were happy with this because they knew it was not a person. When that line isn’t clear, it was felt the potential to cross into the realms of creepy increased significantly.

After two days of discussion, the potential applications of AI in different industries and its impact are somewhat clearer. No-one shied away from difficult topics such as job losses but what’s less clear is the path forward. Many questions remain around the guidelines and frameworks that will be needed to oversee the application of AI and if, as a society, we’re ready for the next frontier.

CogX: How AI could shape the world was originally published in InfoSum on Medium, where people are continuing the conversation by highlighting and responding to this story.