<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Antoine Aubay on Medium]]></title>
        <description><![CDATA[Stories by Antoine Aubay on Medium]]></description>
        <link>https://medium.com/@antoine.aubay_12521?source=rss-aa8e4536eda9------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*g4t2a09V_fF_fngbevlw4Q.png</url>
            <title>Stories by Antoine Aubay on Medium</title>
            <link>https://medium.com/@antoine.aubay_12521?source=rss-aa8e4536eda9------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 15:04:08 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@antoine.aubay_12521/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Scoring Customer Propensity using Machine Learning Models on Google Analytics Data]]></title>
            <link>https://medium.com/artefact-engineering-and-data-science/scoring-customer-propensity-using-machine-learning-models-on-google-analytics-data-ba1126469c1f?source=rss-aa8e4536eda9------2</link>
            <guid isPermaLink="false">https://medium.com/p/ba1126469c1f</guid>
            <category><![CDATA[google-analytics]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-marketing]]></category>
            <dc:creator><![CDATA[Antoine Aubay]]></dc:creator>
            <pubDate>Thu, 09 Dec 2021 09:37:57 GMT</pubDate>
            <atom:updated>2021-12-09T12:36:32.973Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VZCc9shEdhsBHBclqQJ68g.jpeg" /></figure><p><em>A deep-dive on how we built state of the art custom machine learning models to estimate customer propensity to buy a product using Google Analytics data.</em></p><h3>TL;DR</h3><ul><li>Propensity modeling can be used to<strong> increase the impact of your communication</strong> with customers and <strong>optimize your advertising budget spendings.</strong></li><li>Google Analytics data is a well structured data source that can easily be transformed into a<strong> machine learning ready dataset.</strong></li><li><strong>Backtest</strong> on historical data and <strong>technical metrics </strong>can give you a first sense of your model’s performance while <strong>live test</strong> and <strong>business metrics </strong>will allow you to confirm your model’s impact.</li><li>Our custom machine learning model outperformed existing baselines: during live tests in terms of <strong>ROAS</strong> (Return on advertising spend): <strong>+221% vs rule based model</strong> and +<strong>73% vs off-the-shelf machine learning</strong> (Google Analytics session quality score).</li></ul><p>This article assumes basic fundamentals in machine learning and marketing.</p><h3>What is propensity modeling ?</h3><p>Propensity modeling is <strong>estimating how likely a customer will perform a given action</strong>. There are several actions that can be useful to estimate:</p><ul><li>Purchasing a product</li><li>Churn</li><li>Unsubscription</li><li>etc …</li></ul><p>In this article we we will focus on estimating the propensity to purchase an item on an e-commerce website.</p><p>But why estimate propensity to purchase ? Because it allows to<strong> adapt how we want to interact with a customer</strong>. For exemple, suppose we have a very simple propensity model that classify the customers in “Cold”, “Warm” and “Hot” for a given product (“Hot” being customers with highest chance of buying and “Cold” the least):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/501/1*qLX-4LMYX70IpLMml7AOMg.png" /><figcaption>Simple Rule Based Propensity Model</figcaption></figure><p>Well, based on this classification<strong> you can have a specific targeted response for each class</strong>. You might want to have a different marketing approach with a customer that is very close to buying than with one who might not even have heard of your product. Also if you have a limited media budget , you can focus it on customers that have a high likelihood to buy<strong> </strong>and not spend too much on the ones that are long shots.</p><p>This simple type of rule based classification can give good results and is usually better than not having any but it has<strong> several limitations</strong>:</p><ul><li>It is likely <strong>not exploiting all the data </strong>you have at your disposal whether it be more precise information on the customer journey or your website or other data sources you may have at your disposal like CRM data.</li><li>While it seems obvious that customers classified as “Hot” are more likely to purchase than “Warm” which are more likely to purchase than “Cold”, this approach does not give us any specific figures on<strong> how likely they are to purchase</strong>. Do “warm” customers have 3% chance to purchase ? 5%? 10% ?</li><li>Using simple rules, the number of classes you can obtain is limited, which <strong>limits how customized your targeted response</strong> can be.</li></ul><p>To cope with those limitations we can use a more data driven approach: use <strong>machine learning </strong>on our data to <strong>predict a probability of purchase</strong> for each customer.</p><h3>Understanding Google Analytics data</h3><p>Google Analytics is an <strong>analytics web service </strong>that tracks usage data and traffic on website and applications.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5jg1bozlAVclnfcG6LQu5g.png" /><figcaption>Example of GA Dashboard (Google Analytics Demo Account)</figcaption></figure><p><a href="https://support.google.com/analytics/answer/3416092?hl=en#zippy=%2Cin-this-article">Google Analytics data can be<strong> easily exported to Big Query</strong> </a>(Google Cloud Platform fully managed <a href="https://cloud.google.com/bigquery/?utm_source=google&amp;utm_medium=cpc&amp;utm_campaign=emea-gb-all-en-dr-skws-all-solutions-trial-b-gcp-1010042&amp;utm_content=text-ad-none-any-DEV_c-CRE_335630920539-ADGP_Hybrid+%7C+SKWS+-+BMM+%7C+Txt+~+Data+Analytics+~+BigQuery%23v1-KWID_43700053279032269-kwd-47616964923-userloc_1006094&amp;utm_term=KW_%2Bbigquery-NET_g-PLAC_&amp;gclid=CjwKCAjwiY6MBhBqEiwARFSCPqzx1ubPaHp-g3MMEY8zES0fgiSrD3RYgcBGjQeNRRcV_EiS10fZ_RoCUgcQAvD_BwE&amp;gclsrc=aw.ds">data warehouse service</a>) where it can be accessed via an SQL like syntax:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oh_x-JcDMYPO9N6ceWQGJA.png" /><figcaption>Accessing GA data on Big query (GA360 Sample Data)</figcaption></figure><p>Note that the Big Query export table with Google Analytics data is a<strong> nested table at session level</strong>:</p><ul><li><a href="https://support.google.com/analytics/answer/2731565?hl=en#zippy=%2Cin-this-article">Sessions</a> are a list of actions a specific customer does within a given timeframe. They start when a customer visit a page and end after 30 minutes of activity.</li><li>Each customer can have several sessions.</li><li>Each session can be made of severals hits (i.e. events) and each hit can have a several attributes or custom metrics (this is why the table is nested, for instance if you want to look at a the data at hit level you will need to flatten the table).</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/922/1*KA6MqzhshsCEyuoQBnS9Ew.png" /><figcaption>GA360 Data Model</figcaption></figure><p>For example in this query we are only looking at <strong>session level features</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/595/1*dJSwEoqC5Ktjz3oMlD1ppA.png" /><figcaption>Query Session Level</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SiAGMmGUUbDeoHGIo1kekQ.png" /><figcaption>Session Level Features (GA360 Sample Data)</figcaption></figure><p>And in this query we have used an Unnest function to query the same information at<strong> hit level</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/597/1*qxDxW-QhPKh88000gsGSmQ.png" /><figcaption>Query Hit Level</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RspbbC1_hL2znOmE4Hmw_w.png" /><figcaption>Hit Level Features (GA360 Sample Data)</figcaption></figure><p>For more information on GA data check the <a href="https://developers.google.com/analytics">documentation</a>. Note that our project was developed on GA360 so if you are using the latest version, GA4, there will be some slight differences in data model, especially the table will be at event level. There are public sample tables of <a href="https://console.cloud.google.com/marketplace/product/obfuscated-ga360-data/obfuscated-ga360-data?project=lexical-script-761">GA360</a> and <a href="https://support.google.com/analytics/answer/10937659?hl=en&amp;ref_topic=9359001#zippy=%2Cin-this-article">GA4</a> data available on Big Query.</p><p>Now that we have access our raw data source we need to perform feature engineering before we can feed our table to a machine learning algorithm</p><h3>Crafting the right features</h3><p>The aim of the feature engineering step is to transform the raw Google Analytics data (extracted from Big Query) into a <strong>table ready </strong>to be used for<strong> Machine Learning</strong>.</p><p>GA data is very well structured and will require minimal data cleaning steps. However there are still a lot of information present in the table, many of which are not useful for machine learning or cannot be used as is so selecting and crafting the right features is important. For this we built features that seemed to be th<strong>e most correlated with buying a product.</strong></p><p>We crafted 4 types of features:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/842/1*shiin_SbsaPP8btmh8E3Mg.png" /><figcaption>Features used in the model</figcaption></figure><p>Note that we are computing all those features at a customer level which means that we are aggregating information from multiple sessions for each customer (using fullVisitorId field as a key)</p><h4>General Features</h4><p>Global features are <strong>numerical features</strong> that give general information about the session.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/694/1*xCVkEed8OB5qC3cNMJWRSQ.png" /><figcaption>General Features</figcaption></figure><p>Note that bounce rate is defined as % of times the customer only visited only one webpage during a session.</p><p>It was also important to include information on the<strong> recency of events</strong>: for instance a customer that just visited your website is probably more keen to purchase than one that visited it 3 months ago. For more information on this topic you can check the theory on <a href="https://en.wikipedia.org/wiki/RFM_(market_research)">RFM (recency, frequency monetary value)</a>.</p><p>So we added a feature<em> Recency since last session</em> = <em>1 / Number of days since last session </em>which allows the value to be normalized between 0 and 1</p><h4>Favorite features</h4><p>We also wanted to include some information on <strong>the key categorical data </strong>available such as<strong> browser </strong>or<strong> device</strong>. Since that information is at session level, there can be several different values for a single customer so we only take the one that occurs the most per customer (i.e. the favorite). Also, to avoid having categorical features with too high cardinality, we only keep the 5 most common values for each feature and replace all the other values with an “Other” value</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/697/1*P5k4xd-MqcVzehyYc-wvBA.png" /><figcaption>Favorite Features</figcaption></figure><h4>Product features</h4><p>While the first two types of features are definitely useful in helping us answer the question “Is a customer going to buy on my website?”, they are not specific enough if we need to know<strong> “Is the customer going to buy a specific product?”</strong>. To help answer this question we built product specific features that only include the product for which we are trying to predict the purchase:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/691/1*uvRwStmn2I7vGpTkMa1dzA.png" /><figcaption>Product Features</figcaption></figure><p>For<em> Recency since last session with at least one interaction with this product, </em>we use the same formula than for the <em>Session Recency</em> in the General Features. However we can have cases where there is 0 session with at least one interaction with the product, in which case we fill with 0. This makes sense from a business perspective since is our highest possible value is 1 (when the customer had a session since yesterday).</p><h4>Similar Product features</h4><p>In addition to looking at the customer’s interaction with the product for which we are trying to predict the probability to purchase, knowing that the customer interacted with<strong> other products with similar function and price range </strong>can definitely be useful<strong> </strong>(ie substitute product). For this reason we added a set of Similar Product features that are identical to the Product features except that we also include similar products in the variable scope. The similar products for a given product were defined using business inputs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/693/1*3QuwNxyYBRjdlkt3Jm0pSg.png" /><figcaption>Similar Product Features</figcaption></figure><p>We now have our <strong>feature engineered dataset</strong> on which we can train our machine learning model.</p><h3>Training the model</h3><p>Since we want to know whether a customer is going to purchase a specific product or not, this is a <strong>binary classification problem.</strong></p><p>For our first iteration, we did the following to create our machine learning dataset (which was 1 row per customer):</p><ul><li>Compute the<strong> features using the sessions in a 3 months time window</strong> for each customer.</li><li>Compute the <strong>target using the sessions in a 3 weeks time window</strong> subsequent to the feature time window. If there is at least one purchase of the product in the time window, Target it equal to 1 (defined as Class 1), else Target is equal to 0 (defined as Class 0)</li><li>Split the data between a Train set and a Test using 80 / 20 random split.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/658/1*BRUHxHe19ItwQbqEl3vGMQ.png" /><figcaption>Fixed Window Test / Train split</figcaption></figure><p>However some first data exploration quickly showed that there was a <strong>strong class imbalance issue</strong>: Class 1 / Class 0 ratio was over 1:1000 and we did not have enough Class 1 customers. This can be very problematic for machine learning models.</p><p>To cope with these issues we made several modifications in our approach:</p><ul><li>We<strong> switched the target variable</strong> from making a<strong> purchase</strong> to making an <strong>add to cart</strong>. Hence, our model looses a bit in terms of business signification but increasing the volume of Class 1 more than compensates.</li><li>We<strong> trained the model on several shifting windows, </strong>each of 3 months + 3 weeks, instead of a single one. In addition to increasing our volumes of data, this improves the generalization capacity of the model by training on various time of the year where the customers can have different purchase behaviors. Note that due to this, the same customer will be present several times in the dataset (at different periods). To avoid data leakage we make sure that he is always either in the training or the test dataset.</li><li>We<strong> undersampled our Class 0 so that the Class 1 / Class 0 ratio is 1</strong>. Undersampling is a good solution to deal with the class imbalance issue, compared to other options such as oversampling or <a href="https://machinelearningmastery.com/smote-oversampling-for-imbalanced-classification/">SMOTE</a>, because we were already able to increase the volume of Class 1considerably with the first two changes. <strong>Only the training set is rebalanced </strong>since we want the test set to have the same class ratios than the future data we will test it on. Note that we tested with higher ratios such as 5 or 10 but 1 was optimal in model evaluation.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/921/1*Uagm6cCoQWeJaTDLQPx33w.png" /><figcaption>Sliding Window Test / Train Split</figcaption></figure><p>Using this dataset we tested with several classification models: Linear Model, Random Forest and XGboost, finetuning hyperparameters using grid search, and ended up selecting an<strong> </strong><a href="https://xgboost.readthedocs.io/en/stable/"><strong>XGboost model</strong></a><strong>.</strong></p><h3>Evaluating our model</h3><p>When evaluating a propensity model there are two main types of evaluations that can be performed:</p><ul><li><strong>Backtest Evaluation</strong></li><li><strong>Livetest Evaluation</strong></li></ul><h4>Backtest Evaluation</h4><p>First we performed <strong>backtest evaluation</strong>:<strong> </strong>we applied our model to<strong> past historical data</strong> and checked that our model is correctly identifying customers that are going to perform an add to cart. Since we are using a binary classifier, the model produces a probability score between 0 and 1 of being Class 1 (Add to cart).</p><p>A first step in evaluating a binary classification model is create a <a href="https://en.wikipedia.org/wiki/Confusion_matrix"><strong>confusion matrix</strong></a><strong> </strong>and compute the <a href="https://en.wikipedia.org/wiki/Precision_and_recall"><strong>precision / recall</strong></a> (or their combined form in the<a href="https://en.wikipedia.org/wiki/F-score"><strong> f1 score</strong></a>). However there are two issues with these simple metrics:</p><ul><li>Some can be <strong>hard to interpret</strong> because the dataset is imbalanced <em>(for instance the precision metric will generally be very low because we have so few Class 1)</em></li><li>They require to decide on a <strong>probability threshold</strong> to discriminate between <strong>Class 0 and 1</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fDjXZkduHolq_Jt75Q2fcA.png" /><figcaption>Confusion Matrix Example for our Class Imbalanced problem</figcaption></figure><p>So we decided to use two metrics that were more <strong>interpretable</strong>:</p><ul><li><strong>PR AUC</strong>: <strong>Area under the curve of precision by recall graph </strong>(<a href="https://neptune.ai/blog/f1-score-accuracy-roc-auc-pr-auc">see this explanation for more details</a>). Essentially this metric allows us to get a<strong> global evaluation on every possible threshold. </strong>This metric is well suited for unbalanced dataset where the priority is to maximize precision and recall on the minority class : Class 1(contrary to its cousin the ROC AUC)</li><li><strong>Uplift</strong>: we sort customers by their probability score and we divide our results into 20 ventiles. Uplift is defined as the <strong>Class 1 Rate in the top 5% / the Class 1 Rate across all the dataset</strong>. <em>So for instance if we have 21 % Add to Cart in the top Top 5 % of the dataset vs 3 % Add to cart Rate in whole dataset we have an uplift of 7 which means our model is 7 times more effective than a random model.</em></li></ul><p>Results on those metrics were rather positive, especially, <strong>Uplift </strong>was around <strong>13.5</strong>.</p><p>Backtest evaluation is a risk free method for a first assessment of a propensity model but it has several limitations:</p><ul><li>Since it is only done on the past, the model output is not actually being used to<strong> impact the media budget strategy</strong>.</li><li>With our metrics, we only assessed if the model was able to correctly identify customers who would make an add to cart but we did not assess<strong> how the identification of those customers would generate a sales uplift</strong>.</li></ul><h4>Livetest Evaluation</h4><p>So to get a better idea of our model’s business value we need to perform <strong>live test evaluation. Here we activate our model and use it to prioritize advertising budget spendings :</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tTGURYzBGFnkZ78ORlB25w.png" /><figcaption>Livetest Evaluation Protocol</figcaption></figure><p>Results we obtained on the livetest were very solid:</p><ul><li>Compared to a <strong>simple rule based approach</strong> for evaluation propensity,<strong> our model’s ROAS was +221 %</strong></li><li>Furthermore we also compared our performance to a strong contender competition in the form of <strong>Google’s Session Quality Score</strong>: a score provided by Google in the Google Analytics dataset, and in that case <strong>our model was still at +73 % ROAS</strong>. This shows how a custom ML approach can bring considerable business value.</li></ul><h3>Conclusion</h3><p>In addition to reaching solid performance, a strong side benefit of our approach is that our feature engineering is very generic. Almost <strong>none of the feature engineering steps need to be adapted</strong> to apply our model to a<strong> different country scope or product scope</strong>. In fact following our first success in the livetest, we were able to<strong> roll out our model to multiple countries and products in a very efficient manner.</strong></p><p>Thank you for reading. I would be glad to hear your comments on this approach. Have you ever build propensity models? If so, what did you do differently?</p><p><em>Thanks to Bruce Delattre, Rafaëlle Aygalenq, and Cédric Ly.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ba1126469c1f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/artefact-engineering-and-data-science/scoring-customer-propensity-using-machine-learning-models-on-google-analytics-data-ba1126469c1f">Scoring Customer Propensity using Machine Learning Models on Google Analytics Data</a> was originally published in <a href="https://medium.com/artefact-engineering-and-data-science">Artefact Engineering and Data Science</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Leveraging satellite imagery for machine learning computer vision applications]]></title>
            <link>https://medium.com/artefact-engineering-and-data-science/leveraging-satellite-imagery-for-machine-learning-computer-vision-applications-d22143f72d94?source=rss-aa8e4536eda9------2</link>
            <guid isPermaLink="false">https://medium.com/p/d22143f72d94</guid>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[sentinel]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[satellite-technology]]></category>
            <dc:creator><![CDATA[Antoine Aubay]]></dc:creator>
            <pubDate>Wed, 05 May 2021 09:05:12 GMT</pubDate>
            <atom:updated>2021-08-20T15:14:10.368Z</atom:updated>
            <content:encoded><![CDATA[<h4>A step by step guide on how to collect and preprocess satellite imagery data for machine learning algorithms</h4><p><em>This article is the first part of a 2 part series which explores the uses of machine learning algorithms on satellite imagery. Here we will focus on the data collection and preprocessing steps while the second part will showcase the</em><a href="https://medium.com/artefact-engineering-and-data-science/applying-machine-learning-algorithms-to-satellite-imagery-for-agriculture-applications-d505eb8df1b1"><em> use of various machine learning algorithms</em></a><em>.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MMSI4M5rdIJa-n8lsNnrGg.jpeg" /></figure><p>Computer vision is a fascinating branch of machine learning that has been quickly developing in past years. With public access to many off the shelf algorithm libraries (<a href="https://www.tensorflow.org/">Tensorflow</a>, <a href="https://opencv.org/">OpenCv</a>, <a href="https://pytorch.org/">Pytorch</a>, <a href="https://www.fast.ai/">Fastai</a> …) as well as open source datasets (<a href="https://www.cs.toronto.edu/~kriz/cifar.html">CIFAR -10</a>, <a href="https://www.image-net.org/">Imagenet</a>, <a href="https://www.imdb.com/interfaces/">IMDB</a> …) it’s very easy for a data scientist to get hands on experience on this topic.</p><p>But did you know that you could also access open source satellite imagery ? Those types of imagery can be used in a wealth of use cases (Agriculture, Logistics, Energy …). However, they provide additional challenges for a data scientist due to their size and complexity. I have written this article to share some of my key learnings working with them.</p><h3>TL;DR</h3><p>This article will:</p><ul><li>Give you a better understanding of satellite imagery</li><li>Show you what type of imagery can be collected open source</li><li>Guide you through the necessary steps to have “machine learning ready” images</li></ul><p>We will use the detection and classification of agriculture fields as an example. Python and jupyter notebook are used for the preprocessing steps. <a href="https://github.com/artefactory/medium_satellite_imagery">The jupyter notebook for reproducing the steps is available in github.</a></p><p>This article assumes basic fundamentals in data science and python.</p><h3>Step 1 — Select the satellite open data source</h3><p>First, you need to choose which satellite you want to use. Key features to watch out for here are :</p><ul><li><strong>Spatial Resolution</strong>: how many meters are covered per pixel, higher resolution will unlock some use cases and give better performance for most algorithms.</li><li><strong>Instruments available</strong>: the instruments will capture different spectral bands, both from the visible and invisible spectrum. Bands usefulness will vary depending on the use case.</li><li><strong>Temporal resolution</strong> : time between two visits on a given spot on earth. Note that many satellite systems actually include several satellites and in that case the global temporal resolution is usually equal to that of one of its satellites divided by the number of satellites.</li><li><strong>Radiometric resolution</strong> : the number of possible values the instrument captures. The higher, the more precise the measurements are.</li></ul><p>Here are satellites whose data are available free of charge :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VDXhX8hNa3Ue5O6DZ1QLhQ.png" /><figcaption>Benchmark of satellite systems</figcaption></figure><p>Since our goal is to detect and identify agricultural crops, the visible spectrum (which is used to provide standard color images) should be quite useful. Additionally there are correlations between nitrogen level of crops and NIR (Near-infrared) spectral bands. As far as free options go, Sentinel 2 is currently the best in terms of resolution with 10 m for those spectral bands, so we are going to use this satellite.</p><p>Of course, 10 m resolution is much lower compared to state of the art in satellite imagery with commercial satellites reaching a 30 cm resolution. Access to commercial satellite data can unlock many use cases, but despite increasing competition with new players entering this market, they remain highly priced and hence difficult to integrate in a scaled use case.</p><h3>Step 2 — Collect the satellite imagery data from Sentinel 2</h3><p>Now that we have selected our satellite source, the new step is to download the images. Sentinel 2 is part of the Copernicus Programme from the <a href="http://www.esa.int/">European Space Agency</a>. We can access its data using the <a href="https://pypi.org/project/sentinelsat/">SentinelSat python API</a>.</p><p>We first need to create an account at the <a href="https://scihub.copernicus.eu/dhus/#/home">Copernicus Open Access Hub</a>. Alternatively note that this website also provides a way to query images with a GUI.</p><p>A way to specify the zones you want to explore is to provide the API with a geojson file, which is a json containing GPS coordinates position (latitude and longitude) of a zone. Geojson can be quickly created on this <a href="https://geojson.io/#map=2/20.0/0.0">website</a>.</p><p>This way you can query all the Satellite images intersecting with the zone provided.</p><p>Here we list all images available on a for a given zone and time range :</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/74d026cc5946d0072f39f9453eb41d76/href">https://medium.com/media/74d026cc5946d0072f39f9453eb41d76/href</a></iframe><p>Note the use of arguments:</p><ul><li><strong>date</strong>: this range should be set keeping in mind the 6 day temporal resolution. Since we are going to detect crops we set it at a pre-harvest period</li><li><strong>processinglevel</strong> : Level-2A is the most advanced level of preprocessing. For instance atmospheric corrections have already been performed on the images (<a href="https://sentinel.esa.int/web/sentinel/user-guides/sentinel-2-msi/processing-levels">to go more in depth on processing levels</a>)</li><li><strong>cloudcoverpercentage</strong> : Clouds will make some images difficult to exploit. We filter out images with more than 30 % as they are likely unusable</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/927/1*BHBWNBc35fGb_5hodmaiMQ.png" /><figcaption>images_df</figcaption></figure><p>Images_df is a pandas dataframe containing all images that match our query. We select one with a low cloud cover and download it using the api :</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/04038acf5182b75ead9217208da42d53/href">https://medium.com/media/04038acf5182b75ead9217208da42d53/href</a></iframe><h3>Step 3 — Understand which spectral bands to use and manage quantization</h3><p>Once the zip file is downloaded we can see that it actually contains several images.</p><p>This is because as we said in step 1, Sentinel 2 captures many spectral bands and not only Red Green and Blue and each band is captured in a single image.</p><p>Also its instruments do not all have the same spatial resolution, so there is a subfolder for each different resolution 10, 20 and 60 meters.</p><p>If we check the <em>GRANULE/&lt;image_id&gt;/IMG_DATA</em> folder we see that there is a subfolder for each different resolution: 10, 20 and 60 meters as the spectral bands. We are going to use those with max resolution 10m as they provide Band 2, 3, 4 and 8 which are Blue, Green, Red and NIR. Band 7, 8, 9 which are vegetation red edge (spectral bands between red and NIR that mark a transition in reflectance value for vegetation) could also be interesting for detecting crops but since they are lower resolution we are going to leave them aside now</p><p>Here we load each band as a 2D numpy matrix:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/30abeffae0bb19f02881bfb936fb4795/href">https://medium.com/media/30abeffae0bb19f02881bfb936fb4795/href</a></iframe><p>A quick test to make sure everything is working is trying to recreate the true color image (ie RGB) and display it using opencv and matplotlib:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fa3cb6c912641ae9a94790fdc93f66e0/href">https://medium.com/media/fa3cb6c912641ae9a94790fdc93f66e0/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/333/1*0b8FQrQof7t1mED_Ts5vnw.png" /><figcaption>Satellite Image — Color intensity issue (Copernicus Sentinel data 2020)</figcaption></figure><p>However there is an issue with color intensity. This is because the quantization (number of possible values) differs from the standard for matplotlib which assumes either a 0–255 int value or a 0–1 float value. Standard images often use an 8 bit quantization (256 possible values) but Sentinel 2 images use a 12 bit quantization and a reprocessing converts the values to a 16 bit integer (65536 values).</p><p>If we want to scale a 16 bit integer to an 8 bit integer we should in theory divide by 256 but this actually results in a very dark image since the whole range of possible values is not used. The max value in our image is actually 16752 out of 65536 and few pixels actually reach values higher than 4000 so dividing by 8 actually gives an image with a decent contrast</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6fcca11ec131e97d2281e11de6dca22c/href">https://medium.com/media/6fcca11ec131e97d2281e11de6dca22c/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/336/1*1kYP-DJloJXkuBku88Lk7w.png" /><figcaption>Satellite Image — Right color intensity (Copernicus Sentinel data 2020)</figcaption></figure><p>We now have a 3 dimension tensor of shape (10980, 10980, 3) of 8 bit integers in the form of a numpy array</p><p>Note that :</p><ul><li>We could go further by using a non linear scaling approach to improve contrast.</li><li>Casting as integer amounts to rounding the value so we are actually losing some information. This is fine for many use cases and improves processing speed but if precision is key it might be better to convert the reflectance value to a 0–1 float (a format also managed by most ML libraries) rather than a 0–255 integer.</li><li>For now we are leaving the NIR spectral band aside. This will help us test faster since many off-the shelf algorithms expect images with 3 color channels. But it could be interesting to reintegrate it at some point either by replacing one of the other channels or by customizing algorithms to expect 4 color channels.</li></ul><h3>Step 4 — Split the images to machine learning ready sizes</h3><p>We could try applying an algorithm to our current image but in practice that would not work with most techniques due to its size. Even with strong computing power, most algorithms (and especially deep learning) would be off the table.</p><p>So we must split the image into fragments. One question is how to set the fragment’s size.</p><ul><li>Small fragments will improve performance.</li><li>Larger fragments reduce the number of objects that will be hard to detect due to being split into several images. However fragments that are too large can be an issue for some ML algorithms that rescale images or are not able to scale well past a certain number of objects in a given image.</li></ul><p>We decide to go for splitting each image in a 45 * 45 grid (45 being conveniently a divider of 10980)</p><p>Past this parameter fixing, this step is straightforward using numpy array manipulation . We store our values in a dict using (x, y) tuples as keys.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/964b131adebde162001bad82ce5d39f7/href">https://medium.com/media/964b131adebde162001bad82ce5d39f7/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*A3mwEgJOfxobZmapZ_lK8A.png" /><figcaption>Fragment of the satellite image (Copernicus Sentinel data 2020)</figcaption></figure><h3>Step 5 — Link your data to the satellite images using GPS to UTM conversion</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZOLq2_fCWRDvAYEVRCaQLQ.jpeg" /></figure><p>Finally we need to link our images with the data we have on agricultural fields to enable a supervised machine learning approach. In our case we have data on agricultural fields in the form of a list of GPS coordinates (latitude, longitude) , similar to the geojson we used in step 2 and we want to be able to find their pixel position. However, for the sake of clarity, I will start by showcasing a pixel conversion to GPS coordinates and then show the reverse</p><p>The metadata collected while requesting the SentinelSat API provides the GPS coordinates of the corners of the image (footprint column) and we want the coordinates of a specific pixels. Latitude and longitude which are angular values do not evolve linearly within an image so we cannot do a simple ratio using the pixel position. There are ways to solve this using math equations but off the shelf solutions already exist in python.</p><p>One quick solution is to convert the pixel position to UTM (Universal Transverse Mercator) in which a position is defined by a zone as well as an (x, y) position (in meter unit) which does evolve linearly with the image. We can then use a UTM to Latitude / Longitude conversion provided in the <a href="https://pypi.org/project/utm/">utm library</a>. To do this we first need to obtain the UTM zone and position of the upper left corner of the image which can be found in true color image’s metadata.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5b8d574e43d12eef77736c517cb403d2/href">https://medium.com/media/5b8d574e43d12eef77736c517cb403d2/href</a></iframe><p>The GPS position to pixel conversion is done using the reverse formulas. In that case we have to make sure that the zone obtained matches with our image. This can be done by specifying the zone. If our object is not present in the image this will result in an out of bound pixel value.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4d907545c3dc58f77656fd0e5a187248/href">https://medium.com/media/4d907545c3dc58f77656fd0e5a187248/href</a></iframe><h3>Conclusion</h3><p>We are now ready to use the satellite images for machine learning !</p><p>In the next article of this 2 part series :<a href="https://medium.com/artefact-engineering-and-data-science/applying-machine-learning-algorithms-to-satellite-imagery-for-agriculture-applications-d505eb8df1b1"><em> “Applying machine learning algorithms to satellite imagery for agriculture applications</em></a>” we will see how we can use those images to detect and classify fields surfaces using both supervised and unsupervised machine learning.</p><p>Thanks for reading and don’t hesitate to<a href="https://medium.com/artefact-engineering-and-data-science"> follow the Artefact tech blog</a> for more interesting data science articles !</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d22143f72d94" width="1" height="1" alt=""><hr><p><a href="https://medium.com/artefact-engineering-and-data-science/leveraging-satellite-imagery-for-machine-learning-computer-vision-applications-d22143f72d94">Leveraging satellite imagery for machine learning computer vision applications</a> was originally published in <a href="https://medium.com/artefact-engineering-and-data-science">Artefact Engineering and Data Science</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>