In geoscience, we regularly talk about looking for anomalies in data as indicators of potential mineralisation. But what do we define as an anomaly, and how can data scientists and geologists collaborate to generate new insights?

Geologist Holly Bridgwater and data scientist, Dr Jess Robertson recently caught up for a chat about how to describe anomalies in way that would make sense for someone looking at geoscience data from a generalist data science perspective.

Read on to see their take — or watch the original video here.

Note: this discussion was prompted by the Explorer Challenge, a global competition to use data to improve the minerals exploration process — the video makes some specific references to that, but this article generalises a bit more.

Holly:

Today we are going to host a session to address the question ‘What is an anomaly and how do you detect it in data?’ Our aim is to help data scientists get a handle on geological data. I’m going to play the role of the geologist, asking frustrating, poorly framed data science questions, and Jess will be able to translate that for you, hopefully in a meaningful way. Fingers crossed!

So what we thought we’d do is focus on three specific types of anomalies which are really common to see in geological data.

Physical characteristics

Holly: First we’re going to talk about the physical characteristics of rocks, and specifically geophysical data. As a geologist what I’m looking for in this type of data is something that indicates to me a physical differences in the rock. As an example, certain types of ore bodies, like sulfide, are more dense and more conductive than the surrounding rocks — so I go and look for signals that might indicate to me more dense or more conductive rocks.

How I do that is firstly I get someone to collect data, usually an airborne geophysical survey, probably magnetics and gravity. Then, I get someone much smarter than me to do some nice maths on the data (inversions) and generate a number of gridded images that are typically coloured; red as high magnetic intensity, blue as low.

Still Holly:
I look in those images for structures and trends that may indicate something different is going on. I might be looking for a particular change in rock type or physical characteristics, or I might be looking for a structure or chemical change which is indicated by a difference in density.

Jess:

I guess I’d start by noting that you just talked about five or six different steps with this data.

There’s the raw data that you get off the instrument.

There’s some processing of that data — for example, if it was magnetics, you want to remove the effect of the Earth’s magnetic field so that you’re just looking at what the local geology is contributing to that field. Then, you mentioned inversion and you also mentioned pretty images which I think are two separate steps.

Holly:

I guess a geologist’s basic description of why we do inversions is that we’re recording raw measurements from the Earth which are causing the signal to come back in a certain way — but what we actually want to know is what it’s telling us. During the inversion process we apply some prior knowledge to help us determine that.

A savvy exploration geo, before doing a survey would provide the geophysicists with some characteristics of the rock that you already know. So if you already know about the rock types and some of their characteristics you can give those to geophysicists and they can generate a better model that honours your known geology.

Jess:

So, coming at this from a data science perspective, essentially the thing that we don’t do very well in exploration [note, Jess also worked as a geologist], which we should do better at, is understanding the background as well as trying to understand the anomalies.

Obviously, we know we’re looking for mineralization. We want to find that big spike of sulfide that’s going to make us all rich. But sometimes we don’t spend enough time actually understanding what that background actually looks like and what the natural variability of the rock is — so that then we can threshold properly and say “this thing is actually an anomaly”.

One of the advantages that we think data science will be able to bring to exploration is potentially taking a larger look at some of all public data sets and trying to understand what natural variability there is. Then when you’re presenting your anomalous target for minerals you can say, well actually we looked at all of the greenstones in South Australia and found they have these kind of geophysical signals and this thing is just different to that. So being able to provide that kind of bigger data or higher level view is a bonus.

Holly:

How difficult is it for data scientists when giving them raw geophysical data with no context?

Jess:

Part of a data scientist’s role is to create the context from the data! The things that we would identify as ‘useful context’ might not be quantitative enough or scale to all of the data. So a data scientist is going to take a ‘more is more’ approach, where they try to think of a superset of all possible things that might be ‘useful context’, and then learn which ones are actually useful from the data.

There’s still a lot of things that you can kind of pull out of that data without having to do a full inversion.

Just to clarify some of the language, when I would talk about an inversion, coming from a physics background I would expect that we have some governing equations that describe how the physical properties of the rock lead to the signals that we see in the field. When I’m doing an inversion I’m basically trying to go from the field back to those physical properties.

This works well when you’re just looking at one or two fields. Where it becomes really hard is when you’re trying to actually stack a whole bunch of other data on top of there as well. For example, we don’t have any governing equations that tell us simultaneously how rock’s magnetic and density properties plus the geochemistry plus the mineralogy plus the structure invert. You put all of those things together and that system of equations is just too complicated to invent properly.

This is where there’s a specific advantage for machine learning, because potentially with machine learning we don’t need to solve that inversion problem — but we can actually look for patterns instead. This is kind of a data fusion approach. Or, you can do that conversion and you can start to constrain the inversion with some other data sets; that’s also a really valuable way to go and can actually then feed into a machine learning model as well.

So, again lots of different ways to apply machine learning in geology.

Geochemistry

Holly:

So, let’s move on to my favourite thing — geochemistry!

The first thing to point out with regards to geochemistry is that when we talk data we’re talking about a lab assay result or some real-time chemical reading or indicator of what the elements are within your rocks.

Often geochemistry databases are from drilling, but the density and spacing of drillholes will depend on if you‘re drilling out a resource or doing regional exploration. So due to the possible sparsity of the data I imagine it’s really quite difficult from a data science point of view.

What we look for in a geochemical anomaly is any chemical change that is indicative of potential mineralization.

Still Holly:
My process of going through geochemical data is firstly to segment that data.

I know that if I look at the data in bulk I’m unlikely to see any meaningful trends; I’m probably going to see different rock types but I need to separate those different rock types out into separate populations so that I can see if the actual anomalies stand out within those separate datasets. So, I segment that data out into different rock types and even into potential subsets for alteration types. Then I can really clearly see what are the anomalous points in there

If we’re looking for a new deposit, we know we’re looking for something different but we don’t know exactly what the chemical signature might be. It could be an enrichment of some elements, but it could also be a depletion, so really we’re just looking for something that stands out as different in a certain area.

Jess:

This is this is where there’s a real opportunity to connect the chemistry data that you’ve got directly to a geological process actually forming the deposit.

Holly hasn’t mentioned this yet, but what a lot of geochemistry is about is trying to understand how different elements behave under each deposit-forming process.

For example, you might have a bunch of hot fluid moving through some rock and what that’ll do is change all of the minerals in that rock and you will see some elements enriched — like silica. Minerals like plagioclase are altered into clay minerals, or lose the silica, which is transported elsewhere by that hot moving fluid. When you can see that kind of operation that’s an indication that there’s been a hydrothermal process acting there. You might be thinking about hydrothermal deposits like gold or some kind of copper.

Holly:

As geologists we learn through our education of what unaltered and altered rocks look like, chemically-speaking.

Breaking it down really basically, we can consider sedimentary, metamorphic, and igneous rock. Geologists understand the different chemistries of each in their natural state. They’ll also know what elemental indicators can potentially show up if they’ve been altered. We’ve learned about those physical systems. So we apply that knowledge when we’re looking for evidence of economic mineralization.

Jess:

There can be such a broad range of elements to look at. Each element behaves differently in these alteration processes, but groups of elements behave similarly, so there’s actually a lot of information that you can get about the kind of processes that are happening by being able to compare groups of similar elements together.

Geologists have a whole bunch of language around things like hydrothermal elements. Often what you can start to do is take ratios of elements — and it’s actually the ratio of the elements that changes, giving you this real big indication of… things.

Just as a technical point, you don’t want to be just using the raw element of data outside of the chemistry. The key thing about all of those results is that each result needs to add to 100%, you can’t have more than 100% silica in a rock for example. So you need to basically be looking at using log ratio transforms or something similar to actually transform that data into a space where you would start to apply your traditional clustering methods.

Geologists use tools like ioGas to do that. If you’re doing this in machine learning you’d probably want to look at implementing this yourself. It’s actually really straightforward to do.

Structures

Holly:

Our last anomaly type to talk about is structures.

What we’re talking about in structures are features in the crust that may have been involved with mineralization. These are things like faults, breccias, veins, which may have facilitated mineralization occurring.

You might think of them like traps or sinks. Basically there’s a reason why this certain feature has allowed fluid to flow along it or has created a chemical or physical barrier — so we look for those kind of features in the data.

Jess:

I mean, if you think about it you’re really looking for places in the Earth’s crust where elements can be concentrated and there’s a few ways that they can do that.

We can have sort of transport pathways. With those hot flowing fluids we mentioned earlier, they can pass through faults that are acting as preferential pathway. In those cases, you’re more likely to be bringing all of those elements together around that kind of fault, so that makes it a good target for trying to find some kind of mineralization.

Imagine for example that you start off with a fluid deep in the crust and you bring it up along a fault. While deep in the crust it’s sitting in chemical equilibrium with all the rocks around it, but as you bring it up along that fault it comes into contact with other rocks and then you can have chemical reactions and as those chemical reactions occur that will cause precipitation of certain elements.

This makes it useful to look for places where we have these kind of guiding structures, or transport structures, that are going to be bringing material from a broad area into a really narrow area.

It’s not always the case that every deposit that you’re looking at will have formed this way though. Sometimes you’re looking at variations in groundwater that are actually causing things to precipitate out.

Holly:

It’s worth mentioning that often certain types of ore bodies can be fairly vertical so if you looked from above it would look like a fairly linear straight thin line; this is a kind of feature we look for in the data. Often in geophysical data because there’s often a contrast across that structure either in density or conductivity or resistivity which enables us to pick that up.

Jess:

I would say that if you’re taking a data science approach to geology and you’re looking for changes in these faults then you should be thinking about trying to do some kind of spatial filtering, particularly on the potential field data or the magnetics and gravity.

Mainly because what you’re looking for are high frequency changes, so generally when you look at, say, the magnetic map, you’ll see broad features that are you know changes in cover thickness or changes in the bulk behaviour of the rock. Then you’ll see really fine scale lumps and bumps; some of them are just noise, but some of them are actually these really fine-scale fault features.

South Australian Government Resources Information Gateway has just added some fresh magnetotellurics data, which is deep crustal stuff. If you’re interested in trying to understand where deep fluids might be coming from the crust, then it’s probably worth taking a look at that set.

Geology can provide context for what an anomaly is. What we can do in data science is extract all of those as covariates to our predictor — essentially trying to say that given we know what all of the background rocks look like, tell me whether this is anomalous.

If you’re trying to convince a geologist that your model is worth something, you’re trying to say this is the context that my model is adding to your interpretation of this data. So you can say, well we think we think this is anomalous because of all of these reasons, does that actually link really well to the geological processes?

Holly:

And when you are working as a geologist for a company there are additional constraints applied on you. So, we don’t just get asked, ‘just go and find anything, go and look for anomalies’. Usually there are other things that are driving what we’re doing — for instance if a deposit of some type has been found nearby, we’ll often be looking to use that model for that style of mineralization to find other similar deposits.

This makes us very driven by a particular model looking for particular styles of anomalies, and it can be easy to miss the other types of anomalies in the data which weren’t expected.

Also, most exploration departments, whether they be small within large miners have a focus on a particular commodity or a particular style. This focuses us and allows us to apply our funds in a particular way and but it does mean that we might be too biased, looking for ore bodies that the department already processes or trades in.

And that’s a wrap!

Many thanks to Holly and Jess for geeking out for 20 minutes to create this. It’s our hope that it will introduce some more data scientists to the wonderful world of geology — because we need to find more resources to produce all the things society needs, but it’s a difficult game!

One of these things is not like the others: geological anomalies and anomaly detection with data was originally published in TDS Archive on Medium, where people are continuing the conversation by highlighting and responding to this story.

Written by Holly Bridgwater and Jesse Robertson, compiled by Yhana Lucas for the Explorer Challenge

Over the life of a project, exploration geologists collect a vast array of different datasets, all aimed at providing them with different information on rock characteristics. They use this information to assess an area of land for the potential to host economic quantities of the mineral or element they are looking for, like copper or gold. Economic mineral deposits are rare, and over time are becoming harder to find (all the easy ones have been found), so geologists are continuously looking for new data sources and techniques to give them an additional edge.

These datasets can feel a bit intimidating at times, as it may appear that you need an unattainable amount of specialist knowledge to interpret them. Here, we give an overview for those new to geology data — of some commonly collected datasets, how they are obtained, how geologists typically use them and what to watch out for.

The Exploration Lifecycle

Before we jump into the datasets, it’s useful to get a snapshot view of how exploration geologists approach looking for deposits. A really great infographic on mineral exploration is provided here by Visual Capitalist and Orix Geoscience. It’s a great place to get started to get a feel for the overall process.

Planet Earth itself

If you’re totally totally new, here’s an exciting (we think) video on how earth was formed.

Okay, so onto the data.

Geological Reports

In Australia, exploration and mining companies have been required by the government to submit annual reports of their activities back into the 1970’s. In the last 15 years or so, they have also been required to submit all their datasets digitally, so you can imagine there is a wealth of knowledge in these historical exploration reports! The government have done their best to digitise most of this data, but geologists will always review the historical reports to look for extra clues that might have been missed!

The reports that companies have submitted, will relate to the land which they have been exploring over. Companies are granted a permit to explore over a specific area of land, which is called a tenement. Each report is associated with a tenement, or multiple tenement numbers.

Geophysics

As the name implies, geophysics is the study of physical properties of the earth. Ore bodies often (but frustratingly not always!) have distinctly different physical characteristics to the rocks around them. They may be more dense, more magnetic, more conductive or resistive. Geologists use a suite of different airborne and ground surveys to measure these characteristics both at surface and at depth. The needle in a haystack analogy is pretty applicable in this case, considering the very different physical properties of the needle and the hay.

Common geophysical surveys include; seismic, magnetics, radiometrics, gravity, IP (induced polarisation), resistivity, EM (electromagnetic induction) and magnetotellurics.

Geophysical surveys are carried out on a range of different scales. You’ll find merged Australia wide and state surveys at Geoscience Australia and the SARIG portal, and small scale surveys carried out over project areas.

When a geophysical survey is conducted, we are typically putting energy into the ground and recording the response. That response is caused by the distribution of physical properties in the rocks. What we really want to know though, is the physical properties that gave rise to the response, not the response itself. The process to get to that is called inversion, which is a mathematical process applied to the raw data based on prior knowledge about the region. Several methods effectively measure the same physical property but at different scales to trade off between the penetration and resolution of the method (generally speaking, longer wavelengths penetrate deeper in the earth but at lower resolution). In the various datasets you will notice you have access to a combination of the raw and inverted results, and gridded images of those inversions.

Remote Sensing

Remote Sensing as a general term refers to the acquisition of data without physically touching the object you are acquiring data from. In geology, this is largely satellite data, or data captured via airborne surveys. Although remote sensing would also include some of the geophysical surveys mentioned above, geologists are usually referring to a couple of types of data: Landsat (satellite imagery) and spectral data like ASTER that give us information about the earth’s surface, usually at a 10–50 metre resolution (for currently-available public data). Geologists frequently use spectral data to indicate what minerals are present on the surface and subsurface. Different minerals have distinct emission and reflection spectra. This type of data is most regularly used to identify alteration minerals around deposits, which may extend many 100’s of metres away from the deposit itself, but can also be used to provide predictive covariates for surface geochemical samples.

Surface Geochemistry and Geological Mapping

From the early phases of exploration geologists will often take field surface samples, from rocks or soils. Surface geochemistry results will indicate if anomalous levels of an element are present, be that the ore element itself or proxies/vectors. This is a fairly cheap and quick way to assess an area of land. Geological mapping often occurs in conjunction with surface geochemistry surveys. Geologists record their observations in the field to create geological maps which can be used to identify prospective features for mineralisation locations.

At many sites, the majority of the tenement is undercover, by this we mean that the older, prospective rock units are covered by more recent sediments. There is therefore little surface geochemistry or field mapping for these projects.

Drillhole Database

This is where you get to dig a hole!

Geologists often referring to drilling as the truth machine. It is the one surefire way to demonstrate if mineralisation is present under the surface, through collecting subsurface samples. Drilling is expensive, so geologists will only carry this out once they have used some of the techniques above to identify prospective areas.

Drillhole databases (DHDB) include all the information that was recorded about the samples collected and the drilling process including; chemical assays, geology logs, surveys and often much more.

For chemical assays, when drillhole samples are submitted to a laboratory for analysis, the geologist will select what elements they want to know the concentrations of and what analysis method they would like used. Different analysis methods have different limits of detection (the lowest concentration detectable) for different elements. This has also changed through time, so historic results will usually have higher detection limits than more modern results, where the detection limits have improved with new technologies and methods.

Assay results are almost never normalised for a project, so always look out for different analytical method codes and detection limits when you are comparing results. The is particularly obvious in gold exploration, where different methods may be used for surface sampling compared to drilling samples.

It’s also worth noting that drillholes rarely go as smoothly and straight as we’d like — you can read more about how coordinates are computed to allow for this here.

Want to have a crack at picking where to dig? Join the Explorer Challenge, main or Data Science stream.

What else?

We’ve just gone through the major dataset types — but this is absolutely not an exhaustive list.

To generate new insights, we encourage the use of other datasets that are commonly not used by geologists in their exploration programs.

Here’s a quick hitlist of public datasets to get started with:

• The South Australian Resources Information Gateway (SARIG)
• National Computational Infrastructure (NCI) Geoscientific Datasets
• NCI Remote Sensing
• Geoscience Australia data and publications portal
• Commonwealth Scientific and Industrial Research Organisation (CSIRO) general data
• CSIRO Hydrogeochemistry data
• Other CSIRO Geochemistry data
• NationalMap

So you want to dig a hole: How geologists use data to make decisions was originally published in Unearthed Community on Medium, where people are continuing the conversation by highlighting and responding to this story.

Multi-disciplinary approaches have been shown time and time again to produce the best solutions for difficult problems, in a variety of instances.

We’re encouraging Explorers to test this out for minerals exploration!

In medicine, multi-disciplinary teams have been shown to improve outcomes for patients with chronic or complex health challenges.

In reality television, the challenges are always won quickest when the Beauties and the Geeks combine their knowledge — and the Survivors don’t stab each other in the back.

In nature, egrets and cattle team up; the birds get a free ride and meals of tasty insects, the cattle get picked clean of fleas and ticks and have an early warning system for approaching predators.

This is relevant!

See, the Explorer Challenge combines two disciplines each with advanced technical knowledge; data science, and geoscience. This is the kind of project where multi-disciplinary teams have a clear opportunity to excel; the breadth and depth of knowledge doesn’t sit neatly in any domain, and it would take some kind of wizard to know everything that’s going to be relevant.

Though we have poked the geoscience bear somewhat with that Sheldon Cooper meme, amongst other allusions, we 100% rate the importance of geoscience for the challenge. Having sound geological assumptions will be crucial for model development.

Likewise, we think geologists will benefit from the skills of data scientists: yes, we know you use modelling software and can handle the maths, but if geologists could solve this problem alone, we wouldn’t need to hold this challenge at all.

There are more benefits though. Not only do musti-disciplinary teams give you a much greater breadth of technical knowledge, and a multitude of ways that said knowledge can be combined, but they also offer different perspectives and thinking and working styles.

Whilst different thinking and working styles also exist within single domains, there tend to be dominant styles of thinking and ways approaching problems found in every field. It can be argued that this is a natural, justified state — that different personalities just suit different careers better; but also, it’s a state that’s compounded by both conscious and unconscious bias, hiring and promoting similar kinds of people as past staff and leaders.

Regardless of the cause though, combining fields helps gather a diversity in views, which can generate previously untested ideas.

Our resident geologist-turned-data-scientist described the two dominant thinking styles as “splitting” and “mashing”, while speaking at the Perth Machine Learning Group meetup.

“Speaking as someone who has sat on both sides of the fence… when it comes down to it, data scientists and geologists think in fundamentally different ways,” he said.

“Machine learning is really good at doing splitting — categorising things, pulling out trends… Geologists are really good at taking information and mashing it back together… the reason why mashing is important is that it’s not always clear from the data alone what it is we’re actually looking for.”

“We think that the winning teams are going to be the ones that take a combination of approach.”

So, how can you form a team?

If you’re ready to try this multi-disciplinary thing, but need to make some new connections to do so, head on over to the #explorer channel on Slack (once you create an account on our platform, you can join our Slack workspace).

You can post a detailed description of yourself and your ideal teammates, worthy of eHarmony, or just stick up a simple “seeking geos, apply within.”

Slack has private chat functionality, so you can take conversations off the public channel once you find some potential collaborators.

Any tips for teamwork?

We’d love to hear any tips for working successfully in multi-disciplinary technical teams! Comment below or shoot us an email if you’d like to share!

By Your Powers Combined: the potential of multi-disciplinary teamwork was originally published in Unearthed Community on Medium, where people are continuing the conversation by highlighting and responding to this story.

Can data scientists make geology a real science?

Geologists are often given a pretty hard time by other scientists (and TV characters) about geology being more of an art than a science; unpacking why unveils a huge opportunity for data scientists.

A mind-boggling 4.5 billion years of chaos has created Planet Earth as we know it today — so the locations and types of mineral deposits are due to epic space collisions, massive moving plates of rock and magma, and a dynamo in the centre as hot as the sun somehow producing a stabilizing magnetic field.

Going back to basics, most modern sciences follow the scientific method to test and modify hypotheses; observe, measure, experiment, analyse, report and so on. Well, in geology, it’s just not that easy. Mineralization processes, including mountain building, earthquake activity and groundwater flow for example, are a tightly-coupled web of causes and effects spanning from the nanoscale to the radius of the earth, over timescales from microseconds to billions of years. This makes it really hard to build a rigorous, quantitative physical model for ore body formation (beyond the one-hit-wonder that is the planet we live on). Geologists are, in a way, passive scientists, mere witnesses of the amazing processes that happen around our one Earth.

Since we can’t use physical models to easily explain orebodies at the scales we need to, there’s a huge opportunity for data scientists to explain it with data instead!

The current standard for mineral exploration is; an iterative process of collecting different datasets and subjectively applying descriptive geological interpretation to known deposit styles, piece-by-piece, with valuable results that are few and far between. It’s rather like blind men encountering their first elephant, and each investigating only one part of the creature, but claiming to have a whole picture. And because the interpretation of the data is carried out subjectively by geologists, there are plenty of examples (e.g. Mt Remarkable in Western Australia) where explorers have missed things that were staring them in the face, because they were looking for a different deposit type, or have drilled in the wrong place because they’ve interpreted the geophysics incorrectly, or simply because geologists are better at visualising ore deposit models in two dimensional slices rather than a three dimensional volume.

This status quo of holistic interpretation and testing, guessing at secrets held within the earth’s depths, has contributed to the extended periods of time spent on discovering any deposit.

All of this makes mineral exploration an intriguing realm for data scientists, while making data scientists an invaluable asset to exploration companies.

With such large amounts of ground to cover, data to work with, and stories to tell, it is a perfect area for machine learning to once again prove itself.

Machine learning has catapulted us forwards in several areas, which aren’t unlike minerals exploration in their challenges and opportunities — for example, remote sensing and agriculture.

Deep learning and deep recurrent networks have been applied in remote sensing to perform image processing, interpretation, data fusion and time-series analysis — such as removing cloud cover, classifying objects, detecting and tracking targets.

Remote sensing advancements have even affected the geology world in recent years; for instance, Geoscience Australia, in collaboration with CSIRO (the Commonwealth Scientific and Industrial Research Organisation, Australia’s peak public research-body), has developed algorithms to generate cover depth predictions.

In agriculture, the same machine learning methods have applications for detecting weeds and disease, predicting yield and crop quality, and manage fauna, water, and soil.

But even with the leaps and bounds achieved in these fields, there are still many opportunities where data science could be applied to mineral exploration.

So, how and why?

As you might have gathered from the “Geology 101” first-half of this article, it’s not easy to identify economic mineral deposits. Because of the rarity of positive results, the application of data science isn’t as straightforward as simple cut-and-paste, legacy machine learning algorithms — and physical models for inversion just aren’t available. The field requires more complex deep learning applications, not simple AI or rudimentary machine learning.

There are a few particular challenges which the right approaches could overcome.

For instance, whilst recognising different geological data markers is easy enough for a machine, every deposit is different. So, deposit-finding isn’t as simple as training based on defined positives and negatives in a large training dataset — because each recognisable past scenario will likely never occur again. However, machines can be guided by strong priors and identify useful information in datasets which may come together to suggest a deposit.

Geological data is also complex and highly dimensional, but it’s also typically quite constrained. For example, we know that certain minerals will always form together based on rock composition, temperature and pressure. Manifold learning, which seeks to reduce dimensionality of non-linear datasets, could help solve the challenge of separating large-scale geological processes present in the dataset (e.g. regional metamorphic gradient) from smaller-scale processes indicative of a deposit (e.g. local chemical changes around veins which host gold).

Warwick Anderson, founder of OreFox who is using artificial intelligence techniques in mineral exploration agrees, telling us “machine and deep learning are showing massive potential.”

“I think where geologists and data scientists need to work together is on things like normalization, transforms and outlier detection — or removal!” Anderson said.

“Geological maps can be massively extrapolated, so what value can we place on data from it?” Multiscale approaches could be brought in here to handle data variety and sparse aspects — to construct useful models in mineral exploration despite huge, noisy datasets.

As an added bonus, applying data science will also move the field away from descriptive, subjective analyses, towards an empirical, recreatable science.

“Geology is chaos,” according to Anderson. “[It] can be very biased… [using] visual methods to determine rock types. Rock types can change to other similar types in space of a metre; granite to granodiorite for instance, which may have a statistical difference.”

But with the creation of feedback loops in data science methods, it will ensure that no knowledge goes forgotten or ignored when the right geologist doesn’t happen to be looking at the right data at the right time.

Data science isn’t a panacea of course, and no method is able to solve the problem of resource discovery in one go. To move forward, domain reduction is crucial, to progress towards a manageable problem and improve on the status quo.

To this end, an Australian project is opening up years of data on a large exploration project for data scientists to have at it, as part of a crowdsourcing competition with a A$ one million dollar prize pool.

The reward for geologists is greater than just financial recompense though. The full history of earth’s structures has puzzled man for centuries, and minerals and metals are in need of communications devices, transport, energy, and medical technology.

Finding economic deposits of much-needed materials is exactly like looking for a proverbial needle in an equally proverbial haystack.

It’s a bit overused in the data science space, the needle-haystack analogy, but it fits. And it fits just as much in geoscience; we could actually cut the whole hay and needle example and say that data science is often like looking for economic mineralizations of needed materials, and vice versa, but it’s not quite as pithy.

Data science moves much faster to find the needle than mineral exploration, and finds far more lost treasures amongst the hay in the process. And the geo world has many literal treasures yet to be uncovered!

Thank you to Holly Bridgwater, Jess Robertson, and Warwick Anderson for their input on this article.

Digging on deep data: a real-world global treasure hunt was originally published in TDS Archive on Medium, where people are continuing the conversation by highlighting and responding to this story.