<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Andrew Gooday on Medium]]></title>
        <description><![CDATA[Stories by Andrew Gooday on Medium]]></description>
        <link>https://medium.com/@agooday?source=rss-20013b2143c2------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Zu7R2a7odT8Bh7gWnHwuVQ.jpeg</url>
            <title>Stories by Andrew Gooday on Medium</title>
            <link>https://medium.com/@agooday?source=rss-20013b2143c2------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 13:26:35 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@agooday/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[An Introduction to AutoML]]></title>
            <link>https://medium.com/analytics-vidhya/an-introduction-to-automl-8356b6ceb091?source=rss-20013b2143c2------2</link>
            <guid isPermaLink="false">https://medium.com/p/8356b6ceb091</guid>
            <category><![CDATA[google-automl]]></category>
            <category><![CDATA[driverless-ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[emerging-technology]]></category>
            <category><![CDATA[automl]]></category>
            <dc:creator><![CDATA[Andrew Gooday]]></dc:creator>
            <pubDate>Sat, 04 Jul 2020 09:44:12 GMT</pubDate>
            <atom:updated>2020-07-06T16:47:46.720Z</atom:updated>
            <content:encoded><![CDATA[<h4>Automated Machine Learning</h4><h4>Google, Amazon &amp; H2O.ai AutoML</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jUJ-AgrW5uTShiVF4gVdGQ.jpeg" /></figure><h3>Building AI with Machine Learning</h3><p>Perhaps the fastest way to become familiar with a machine learning workflow and start to get a sense of some of the difficulties posed is to take a look at Kaggle.</p><p>Kaggle, owned by Google, hosts a community of machine learning enthusiasts and professionals, lots of useful public data sets, free GPU compute, prize money awarding machine learning competitions and hosted machine learning ‘notebooks’.</p><p>Here’s an excellent example of a machine learning notebook — it combines Python code with readable graphs, comments and explanations.</p><figure><a href="https://www.kaggle.com/janiobachmann/bank-marketing-campaign-opening-a-term-deposit/data"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XAyzGE69v8aJKE0VMr_V1w.png" /></a><figcaption>Kaggle Notebook, by Janio Bachmann — extract showing data exploration</figcaption></figure><p>We’ll be referring back to this specific notebook as we test drive AutoML solutions, so you might want to create a copy and run through it — clicking on the image above will take you directly to the notebook. Even if you don’t know Python, you can read the comments and view the analysis.</p><p>Take a look at any such data science notebook for a given Kaggle challenge and it will likely include many of the following:</p><ul><li><strong>Data Exploration</strong> — tables, and graphs to assess the quality and ‘shape’ of the data. (see the above image)</li><li><strong>Data Preparation</strong> — once the data has been visualised and explored, work to pre-process the data for machine learning can begin. This can include dealing with outliers, scaling and mapping categorical to numerical values, fixing errors.</li><li><strong>Feature Engineering </strong>— creating new features from the original data set. Feature crosses, dimensionality reduction (for example via Principal Component Analysis,) dropping features, consolidation and imputation of missing values. Some of these steps may even include use of machine learning.</li><li><strong>Model Selection &amp; Training </strong>— selecting the appropriate candidate models for the task and data, training them on the engineered data.</li><li><strong>Scoring &amp; Optimisation</strong> — selecting the appropriate scorer (e.g. AUC vs RMSE), scoring and tuning hyper-parameters. A model leaderboard, and selection and optimisation of the ‘winning’ model and transformations.</li></ul><p>However, Kaggle doesn’t provide the full picture. The business problem &amp; use case has been defined for you, the data has been supplied and the focus is on creating a high scoring model, not deploying and maintaining a product inference solution.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3ywXG96Nq1CrmqrmpfGRwA.png" /><figcaption>The Machine Learning Process — End to End</figcaption></figure><p>So, in addition to what we’ve seen on Kaggle, we also need to consider these important machine learning elements:</p><ul><li><strong>Machine Learning Problem Framing </strong>— Ensuring that your use case or business requirement can be framed as a machine learning problem, and that it’s appropriate to use machine learning to address them.</li><li><strong>Data Acquisition </strong>— You’ll need the right user opt-ins, the right systems and products to capture data, stream it to where it’s needed, store it in data lakes, move it to data warehouses. Until recently, machine learning needed big data. With transfer learning, the ability to build machine learning solutions to address new use cases with small data — perhaps only hundreds or a few thousand examples, is emerging. But you still need high quality, labelled data.</li><li><strong>ML Experiments — </strong>Forms an<strong> </strong>iterative cycle which includes<strong> Data Engineering</strong>, <strong>Model Selection &amp; Training</strong>, <strong>Scoring &amp; Optimisation</strong></li><li><strong>Machine Learning Inference Pipeline </strong>— A production machine learning solution typically compromises more than the model. The data transformations created to feed the model need to be included plus.</li><li><strong>Model Documentation </strong>— For many real world applications, interpretability &amp; explainability are required to mitigate risk from unintended discrimination and bias, to meet legal requirements, to ensure trust in the service through understanding of model behaviour.</li><li><strong>Model Deployment </strong>— In general although machine learning deployment can be difficult, but use of containers and cloud endpoints make deploying and scaling inference easier. It’s really important to note that the systems and type of compute used to create the models are not necessarily suited to supporting production inference with the associated low latency and scale.</li><li><strong>Model Management </strong>— The performance of you model and machine learning inference will degrade over time. Capturing the data needed to maintain and update the model, ensuring performance and data drift can be detected, being able to test deploy &amp; new models are all part of the machine learning lifecycle.</li></ul><p>End to end, the process of building a new model and moving it into production can involve many teams — Data Ops and Engineering, Data Science, Development &amp; DevOps, Product.</p><p>Factor in best practices, keeping up with the latest research, models, feature engineering techniques and given the resources involved, the amount of iteration — surely some degree of automation is available?</p><p>The answer is — yes. Automation varies in terms of scope and functionality, but ‘AutoML’ is addressing almost all of the areas above.</p><h3>Defining AutoML</h3><p>I’m going to look at AutoML services provided by Google, Amazon and H2O.ai, and use them to define AutoML. Then, in the next section, I’ll run the banking data, as used by the featured Kaggle notebook, through each system to provide some insight into how the different AutoML solutions vary.</p><h3>Google’s AutoML</h3><p>Google provide a range of AI services, but for the purposes of this discussion, we’ll focus on Google’s <a href="https://cloud.google.com/automl">AutoML</a> offering.</p><p>Google AutoML Video, Google AutoML Vision, Google AutoML Natural Language, Google AutoML Text &amp; Doc Entity Extraction, Google AutoML Sentiment Analysis, Google AutoML Translation effectively allow you to extend the functionality of existing Google AI services, using your data. Transfer learning or neural architecture search run behind the scenes to adapt inference to your use case and based on your data.</p><p>I’m going to refer to this as ‘Closed’ AutoML,</p><p><strong>Google AutoML — An example of ‘Closed’ AutoML</strong></p><ul><li><strong>Data Acquisition </strong>— You supply ‘clean’ data.</li><li>Basic <strong>Data Exploration </strong>— In Google AutoML, you can view fields from your source data, get some basic statistics on missing values, distinct values, ‘nullable’ fields, target correlation.</li><li>Basic <strong>Data Preparation</strong> — You can select the target field, drop columns from training via GoogleML’s basic GUI. Services to help you label that data are available. You can control train/validation/test split.</li><li><strong>ML Experiments</strong> — You can initiate training from a GUI, but you cannot influence model selection. You are given no access to feature engineering, no visibility of training, optimisation, any ensembling or stacking that happens. Basic metrics for the final model/pipeline are made available.</li><li><strong>ML Inference Pipeline</strong>. You cannot access the pipeline components &amp; there is no explainability or other documentation generated.</li><li><strong>Model Deployment</strong>. The resulting model can typically be accessed by RESTful API and may be available for deployment in a container. However, you don’t own the model &amp; you can’t access the model.</li></ul><p>All of that said, the results can still be impressive and achieved quickly.</p><p>Google’s AutoML services generally have a corresponding Google API, where you can’t customise the model with your data, but more features are available. As an example, compare AutoML Vision with the Vision API <a href="https://cloud.google.com/vision">here.</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GI6IBsr9on9QGrs6fdINfA.png" /><figcaption>Google AutoML — ‘Closed’ AutoML</figcaption></figure><h3>Amazon’s AutoML</h3><p>The following Amazon AI services fit my definition of ‘Closed’ ’AutoML. Amazon Comprehend &amp; Amazon Translate have a ‘custom’ feature, allowing the automated building of a Natural Language Processing based on your data in the former case, handling of domain or brand specific language in the latter.</p><p>Amazon Rekognition offers an automated custom label service for image classification and object detection, providing a GUI <a href="https://aws.amazon.com/blogs/machine-learning/announcing-amazon-rekognition-custom-labels/">interface</a> for labelling your image data and some basic evaluation metrics. In my <a href="https://medium.com/swlh/building-a-simple-ai-photo-analytics-app-on-aws-68d0b0f84c87">‘Building a Simple AI Photo Analytics App on AWS’</a> article, I used Amazon Rekognition and this could easily be extended via AutoML and Rekognition’s custom labels.</p><p>However, Amazon also offer SageMaker <a href="https://aws.amazon.com/sagemaker/autopilot/">Autopilot</a>, which extends automation beyond the initial hyper-parameter tuning initially offered. When used via SageMaker Studio, which is not available in all AWS regions yet, a machine learning IDE, Autopilot blends model development automation with direct access to auto-generated Python machine learning notebooks.</p><p>This increase in the ability to view and customise at various stages of the automated machine learning process provides an example of ‘Open’ AutoML.</p><p><strong>SageMaker Autopilot — An example of ‘Open’ AutoML</strong></p><ul><li><strong>Data Acquisition </strong>— You still supply ‘clean’ data.</li><li><strong>Data Exploration </strong>— SageMaker Studio &amp; Autopilot automatically creates a <em>data exploration notebook</em> which includes a dataset sample, column analysis, count statistics (for string type features), descriptive statistics (for numerical features) and some recommended manual check &amp; improvement actions.</li><li><strong>Data Preparation</strong> — SageMaker Studio &amp; Autopilot builds a <em>candidate generation</em> <em>notebook which </em>can be used to view and modify transformations which have been automatically created for each candidate model. Data transformations such as imputation, one-hot-encoding, principal component analysis, standard scalar, are proposed.</li><li><strong>Feature Engineering</strong> — SageMaker Studio &amp; Autopilot’s <em>candidate generation</em> <em>notebook</em> can be modified to customise the experiment.</li><li><strong>Selection &amp; Training</strong> — This happens automatically. The <em>candidate generation notebook </em>can be used to step through the auto generated workflow if desired.</li><li><strong>Scoring &amp; Optimisation</strong> — You can select your scoring algorithm, and adjust the hyper-parameter ranges and optimisation via SageMaker’s <em>c</em>a<em>ndidate generation notebook</em></li><li><strong>ML Inference Pipeline</strong> — You can access SageMaker Autopilot’s model artifacts, data processor models, tuning details and notebooks, which are stored in S3.</li><li><strong>Model Deployment — </strong>You have full control of model deployment &amp; could export and use the model and pipeline where ever you need. Deployment to AWS is semi-automated by SageMaker.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i1TrbMJTPwjn1THQdLPsog.png" /><figcaption>Amazon SageMaker Autopilot — ‘Open’ AutoML</figcaption></figure><p>Like Google AutoML Tables, Autopilot currently only works with structured data.</p><h3>3. H2O.ai’s AutoML</h3><p>With a mission to “democratize AI for everyone” , H20.ai look to be very committed to AutoML.</p><p>Like SageMaker Autopilot, <a href="http://docs.h2o.ai/driverless-ai/latest-stable/docs/userguide/examples/credit_card/credit_card_default.html">H20.ai Driverless AI </a>provides access and control to each part of the automated machine learning process.</p><p>H20.ai provide an intuitive and fully integrated user interface. It’s not quite an IDE, although support for Python and R clients provides control of automation from either environment.</p><p><em>Unlike both Google AutoML and SageMaker Autopilot</em>, Driverless AI can run automation on NLP, Time Series and Structured/Tabular data. Driverless AI supports Tensorflow for deep learning models and can also run automation using your transformations, models or scoring ‘recipes’.</p><p><strong>H2O.ai — Another example of ‘Open’ AutoML</strong></p><ul><li><strong>Data Acquisition </strong>— You still supply ‘clean’ data, with connectivity options to Hadoop hdfs file systems, or S3 bucket, Azure, Hive, JDBC etc.</li><li><strong>Data Exploration </strong>— Extensive visualisation options for your source data are provided via ‘AutoViz’ — Spikie &amp; Skewed histograms, box plots, outliers, correlation graphs, parallel coordinate plots, RADAR plots, data &amp; ‘missing’ heatmaps. Data set details contain summaries of types, statistics, missing values, unique values, and first 20 rows of a column.</li><li><strong>Data Preparation</strong> — Custom recipes can be created to create new data sets via the GUI. Test/train/validation splits can be determined and the target variable set.</li><li><strong>Feature Engineering</strong> — Not only are features generated, but you can track their importance and use in real time as experiments progress.</li><li><strong>Selection &amp; Training</strong> — The range of candidate models varies based on your initial experiment settings — Accuracy, Time, Interpretability and Scorer. You can see exactly which models will be included in the experiment via the GUI, which also provide a real-time dashboard showing performance for each model.</li><li><strong>Scoring &amp; Optimisation</strong> — You can select your scoring algorithm.</li><li><strong>ML Inference Pipeline</strong> — Low latency, Python, R, C++, Java code is generated. Comprehensive auto generated human readable documentation is generated from the experiment. In addition, a large amount of explainability, interpretability information is provided: “K-LIME, Shapley, Variable Importance, Decision Tree Surrogate, Partial Dependence, Individual Conditional Expectation, Sensitivity Analysis, NLP Tokens, NLP LOCO” plus support for LIME and Shapley reason codes.</li><li><strong>Model Deployment — </strong>scoring pipelines are deployed manually.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wllKb8SRXSPtnsKaPai8cA.png" /><figcaption>H20.ai Driverless AI — ‘Open’ AutoML</figcaption></figure><h3>Exploring AutoML</h3><p>I’ll now carry out a simple run through of each of the above AutoML solutions. I’ll be using a subset of the data file that Google use in their AutoML demo. Using this simple data set will allow direct comparison with Janio Bachman’s Kaggle notebook, and hence ‘manual’ machine learning.</p><p>This is definitely not a benchmarking test, rather the purpose is to illustrate the differences between the various AutoML solutions.</p><p>It’s worth noting — this is a small dataset at 17 columns and less than 12,000 rows. There aren’t any missing values, and it’s nicely balanced in terms of the prediction target we’re using, the ‘deposit’ column, split nearly 50:50.</p><p>So it’s not the most difficult of data sets, and therefore a little atypical.</p><p>But this is okay — I’m only looking to highlight some of the differences between approaches and the potential of ‘AutoML’.</p><h3>Google AutoML Tables</h3><p>So let’s get started with Google AutoML tables, and I’ll stay very close to the Google demo.</p><p>Following the instructions <a href="https://cloud.google.com/automl-tables/docs/quickstart">here</a>, but minus any need to do anything billing related, I created a project, a bucket and uploaded the csv file.</p><p>I selected ‘deposit’ as the target prediction variable. All other columns are ‘features’ which can be fed into the model to help make a prediction.</p><p>Visualisations of each of the feature data can be seen by simply clicking on them. The correlation of each feature to the target variable is shown. I can manually drop those columns with low correlation if I choose.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8p2uQJ6xyTLiGMhnE2eDtg.png" /><figcaption>Google AutoML Tables — Selecting the Prediction Target</figcaption></figure><p>Google automatically decides to build a classification model, which is what we need. The goal is to work out whether the customer will make a deposit and it’s a binary choice, yes/no, hence we want to build a ‘binary classifier’.</p><p>Selecting ‘Train Model’ allows me to specify a budget limit of 1 node hour to run the work and uncheck any unwanted feature columns.</p><p>That’s it until I receive an email, which will confirm training is complete.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OaVKZnS583Ey0hNPBhVhnA.png" /><figcaption>Google AutoML Tables — Model Evaluation</figcaption></figure><p>Google has automatically split the data into train (80%), validate (10%) and test (10%) which means the model can be tuned by the validation set and then checked with unseen data (the test set.)</p><p>On the evaluate tab there’s lots of useful information — as this is a classification problem, there are ROC/AUC and Precision/Recall curves, confusion matrix and feature importance. The sort of things you’d likely find on a Kaggle notebook, used to determine how well the model has done.</p><p>In our case, the model seems to be doing well — but the definition of good would depend on the cost of a false negative or positive, so the use case in question.</p><p>We can also see feature importance in the models classification decision — so there’s some basic explainability here too.</p><p>How does the model fair verses the Kaggle notebook?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WWP6Jj1XTwIbKJPb6Z7sog.png" /><figcaption>Kaggle Notebook, by Janio Bachmann — showing model performance</figcaption></figure><p>Not bad — we’ve got a higher ROC/AUC score at 93.9. There’s also agreement on the 1st and 3rd most important features, although very little agreement on any others.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kJpuYK3HBIk9R7K3ETn0vg.png" /><figcaption>Kaggle Notebook, by Janio Bachmann — showing feature importance</figcaption></figure><p>And in terms of accuracy, we’re ahead too with 88.1% vs 84%</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ftAR61F9AnwZCHutMoV1ng.png" /><figcaption>Kaggle Notebook, by Janio Bachmann — showing accuracy</figcaption></figure><p>We could now go ahead and run further tests, batch predictions, place the model online with one click, or export the model as a TensorFlow package to run in a Docker container.</p><p>In summary, I’ve used default values, very few data science skills, and some luck, in particular how well behaved the source data is, to create a production ready model that I can deploy any time I want.</p><p>But I don’t know what the model is — is it tree based? Is it an ensemble, a neural net? How does it make decisions? How would I tune or improve on the model?</p><h3>Amazon SageMaker Autopilot</h3><p>Which brings us on recent updates to Amazon’s SageMaker. By using the SageMaker AutoPilot service, I’ll be able to address some of gaps above. I’ll know what data transformations have taken place, which models are in use, and be able to modify them.</p><p>The SageMaker Studio service is still not available in all regions, so I’ll launch in the US-East ‘Ohio’ region, using a US East based S3 bucket to upload the banking data file.</p><p>SageMaker Studio’s JupyterLab environment provides a machine learning IDE. Selecting ‘Create Amazon SageMaker Autopilot Experiment’, you’ll see there are a few differences from Google’s AutoML already.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JG3uZxqLYl3S2NE7IfStGA.png" /><figcaption>Initiating an AutoPilot Experiment in SageMaker Studio</figcaption></figure><p>The access to automatically generated machine learning notebooks and the resulting visibility is a big difference between Google and AWS’s approaches.</p><p>It’s unfortunate that we don’t have a GUI to select the target and feature columns. That, and just how long it takes to get an error message back — many minutes after initiating the run following my entering an invalid target name. One to address in future releases.</p><p>Autopilot goes through 4 stages: ‘Analyzing Data’, ‘Feature Engineering’, ‘Model Tuning’, ‘Completed’.</p><p>If you remember, I limited Google to 1 hour. This won’t be an entirely fair test because I didn’t limit Amazon in the same way.</p><p>Two hours after initiating Autopilot on the same bank data csv, feature engineering has completed, but model tuning is ongoing (there are 250 tuning jobs by default.)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yhzhBQc24lqt9xL5w9_u1w.png" /><figcaption>Amazon SageMaker — Hyper Parameter Tuning</figcaption></figure><p>On completion, about 30 minutes later, you can take a look at the metrics. Go to experiments, control &amp; click to ‘open in trial component list’ and add metrics. From here, a quick sort shows the best model provided a validation accuracy of 85.964%.</p><p>At first glance — this is bad. It’s better than the Kaggle workbook, but worse than Google AutoML, taking far longer to complete. Plus we don’t see a precision/recall curve, a ROC score, F1 score, confusion matrix, so it’s not possible to immediately compare.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W9jRbcA_WyX2hnIGkisMrw.png" /><figcaption>Amazon SageMaker — Autopilot Completion &amp; Metrics</figcaption></figure><p>But, a click on ‘open candidate generation notebook’ and immediately we can see the target objective, exactly which models and transformation are going to be used.</p><p>** <strong>tip</strong> — import and save the candidate definition and data exploration notebooks in Studio as I had difficulty finding them again later **</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yxgSsJv2aKX-z9bdMOnDjg.png" /><figcaption>Autopilot’s auto-generated Python Notebook.</figcaption></figure><p>The hyper-parameter tuning ranges can also be seen, as can the transformation strategies for each model.</p><p>All of these can be adjusted, and you can step through the full process from the notebook by selecting import notebook and the data science kernel.</p><p>However, things get a little awkward if your Studio session times-out, and I think there’s still some work to make this a truly stand-alone GUI.</p><h3>H20.ai Driverless AI</h3><p>Moving from Google to AWS, we gained an increase in visibility and control at the cost of ease of use. Amazon’s offering in particular is a more developer friendly user interface.</p><p>Moving from Amazon on to H20.ai, we have a much more polished, data scientist friendly user interface. I tried the cloud version — ‘aquarium’ (running on AWS), creating a new project and loading the same banking data set into it.</p><p>Data visualisations on the source banking data csv were immediately available via ‘AutoViz’.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uobySiefXcgB2_NZdxsG6Q.png" /><figcaption>H20.ai Data Visualisations</figcaption></figure><p>Without reading the built in instructions, or following any tutorials, I was able create a train/test split in one click, then select the target ‘deposit’ column.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4CGiDCvsU50B-zrDg17_KA.png" /><figcaption>H20.ai Driverless AI — Setting Up an Experiment</figcaption></figure><p>The training settings are intuitive and on the left you can see the various trade-offs in models, feature engineering strategies that correspond to your choices.</p><p>So it’s fair to say, this is a very intuitive interface.</p><p>It’s also a lot of fun to watch as it immediately kicks off at an AUC score of over 0.90, taking us immediately to a similar performance to Kaggle notebook.</p><p>After about 30 minutes, the experiment completed. With the final scores — AUC 0.9295, Accuracy 0.8646, F1 0.8619, this is lower than achieved via Google AutoML, but higher than SageMaker’s Accuracy and better results than the Kaggle notebook.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1FRIURKpH-kNM8ka-JVBnA.png" /></figure><p>This was by far my favourite interface, providing the simplicity of Google’s AutoML with the power of Amazon’s customisation.</p><h3>Other AutoML Services</h3><p>Microsoft Azure provides many customisable AI services: MCS Speech-to-Text, LUIS, QnA, Speech Translation, Form Recogniser, Anomaly Detector, Personaliser, Speaker Recognition, Custom Vision and Video Indexer.</p><p>I’d categorise all of the above as being ‘Closed’ AutoML.</p><p>Microsoft’s <a href="https://www.youtube.com/watch?v=n2q3vI1NBXA">AutoM</a> is an ‘Open’ AutoML solution which can handle classification, regression and time series forecasting.</p><p>IBM <a href="https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/autoai-overview.html">AutoAI</a> is also an ‘Open’ AutoML solution, which includes support for auto generated Python notebooks.</p><p>In addition to established AI players there are also newer players such as <a href="https://kortical.com/">Kortikal</a> providing ‘Open’ Auto ML.</p><h3>Conclusion</h3><p>AutoML is an emerging technology and as it improves it will have significant implications for how AI is consumed as the second wave of AI adoption reaches new industries and use cases.</p><p>AutoML will have an impact on how businesses recruit for AI, even beyond data and development teams. For example, in my last article, <a href="https://medium.com/digital-diplomacy/ai-product-managers-evolution-or-revolution-729af7e99154">“AI Product Managers — Evolution or Revolution”</a>, I include AutoML as contributing to how Product roles will change.</p><p>I’d expect to see the number of AutoML versions of cognitive APIs growing. I’m also hoping for improvements to the initial AutoML offerings from Microsoft, AWS and others.</p><p>The scope of AutoML may expand also, automating more of the deployment and in-life management on one side, integrating more seamlessly with data sources on the other.</p><p>‘Open’ AutoML looks more likely to appeal to those actively working in machine learning and also seems much more likely to address real-world business needs than ‘Closed’ AutoML.</p><p>It could be the case that if automation becomes sufficiently advanced, the need for the kind of visibility and control provided by ‘Open’ AutoML will lessen and we’ll see ‘Closed’ AutoML being adopted by enterprises.</p><figure><a href="https://www.theactivationfunction.com/"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yRf1mGomD7pV0dN02a_dzQ.jpeg" /></a></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8356b6ceb091" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/an-introduction-to-automl-8356b6ceb091">An Introduction to AutoML</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI Product Managers — Evolution or Revolution?]]></title>
            <link>https://medium.com/digital-diplomacy/ai-product-managers-evolution-or-revolution-729af7e99154?source=rss-20013b2143c2------2</link>
            <guid isPermaLink="false">https://medium.com/p/729af7e99154</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[product-management]]></category>
            <category><![CDATA[ai-product-management]]></category>
            <category><![CDATA[technology-trends]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Andrew Gooday]]></dc:creator>
            <pubDate>Wed, 24 Jun 2020 13:28:39 GMT</pubDate>
            <atom:updated>2020-06-24T13:28:39.061Z</atom:updated>
            <content:encoded><![CDATA[<h4>Product Management &amp; Machine Learning.</h4><h3>AI Product Managers — Evolution or Revolution?</h3><h4>How AI is changing the Product Management role.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YT7dDF1f6NvOham6V--4eA.jpeg" /></figure><h3>AI Everywhere</h3><p>Machine Learning has already transformed the software industry, and AI powered applications are part of everyday life.</p><p>Now, there is a ‘second wave’ of AI, which extend to reach almost every sector and industry, moving beyond today’s software, healthcare, financial services base. According to both PWC and McKinsey, AI will generate $13-$15 trillion in global GDP by 2030.</p><p>As a product manager you may already be using machine learning services, perhaps relying on partners to deliver the functionality your product needs. Maybe you’re using machine learning, but do you understand the implications?</p><p>Perhaps the fraud detection service you’re using is actually biased, with the associated risk to reputation? Or maybe you should push back on your in-house development team’s enthusiastic recommendation to ‘try a <a href="https://www.theregister.com/2020/06/19/generative_adversarial_networks_are_hard/">GAN</a> approach’ so that you can get your product to market on time.</p><p>If you have only a superficial understanding, you may be missing out on key innovation triggers or lack sufficient insight in to how AI will transform your competitors and market.</p><p>In this article, I argue that as a Product Manager you’ll need one of three levels of expertise: “AI Aware”, “AI Enabler”, “AI Expert”, in order to provide the leadership needed to evolve your product and services as AI becomes ubiquitous.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GknANalBGrkrJc_A3SHhYQ.png" /><figcaption>Product Management &amp; Machine Learning</figcaption></figure><h3>The ‘AI Product Chasm’</h3><p>Saleforce’s Director of Product Management, <a href="https://medium.com/u/897e7262ab0d">Mayukh Bhaowal</a>, recently spoke about an ‘AI product <a href="https://www.mindtheproduct.com/how-ai-is-changing-the-product-management-job-description-by-mayukh-bhaowal/">chasm</a>’. The software industry, although leading the way, is lacking Product Managers with the right skills to fully address the AI opportunity.</p><p>Mayukh goes on to list ‘5 new skillsets’ for Product Managers: ‘Problem Mapping’ — in particular knowing whether AI can address your business goal, ‘Data is the new UI’ — product owners used to specifying user experience need to be equally adept at understanding data and setting ML appropriate requirements , ‘Explainability &amp; Ethics’ — topical, and essential, ‘Scaling from Research to Production’ — particularly if you’re relying on your own infrastructure to build and deploy.</p><p>Mayukh is not alone. O’Reilly’s ‘<a href="https://www.oreilly.com/radar/what-you-need-to-know-about-product-management-for-ai/">Radar</a>’ has highlighted the ‘need for an AI Product Manager’, whilst there is an ever increasing number of AI product role vacancies and courses.</p><p>But what exactly is an AI Product Manager?</p><h3>Today’s Product Roles</h3><p>Before we can take a look at how product roles might change, let’s first untangle Product Manager, Product Owner and Product Marketing Manager roles.</p><p>Here’s a very useful framework from <a href="https://www.productfocus.com/product-management-basics/what-is-product-management/">Product Focus </a>—</p><figure><a href="https://www.productfocus.com/product-management-basics/what-is-product-management/"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*K4ONghZUhOtUjThdo_igUw.png" /></a><figcaption>Product Focus’s Framework — Included here with the kind permission of Product Focus</figcaption></figure><h4>Product Manager</h4><p>The ‘Strategic Activities’ above belong to the traditional corporate or enterprise Product Manager. This is a leadership role, creating product direction, vision and strategy. There’s a strong understanding of the product’s market, commercials and the role usually requires strong people &amp; management skills as teams, processes and people are brought together to get things done. This is also an ‘enabler’ role with accountability for product profitability and performance.</p><p>In the software world, the line between Product Owner and Product Manager is often more blurred. Even so, the widely adopted ‘Scaled Agile Framework’ (<a href="https://www.scaledagileframework.com/product-owner/#:~:text=The%20Product%20Owner%20(PO)%20is,or%20components%20for%20the%20team">SAFe</a>), acknowledges that the Product Manager is more market and customer focussed, whist the Product Owner is ‘solution, technology and (delivery) team focused.</p><p>Roman Pichler’s Product <a href="https://www.romanpichler.com/blog/the-t-shaped-product-manager/">blog</a> refers to an ideal product manager — a ‘ T type’, where the horizontal line in ‘T’ represents “transferable product management experience”, the vertical “Expertise to make the right decision for a given product”.</p><p>Given the imminent impact of AI on all industries, sufficient knowledge of AI is going to be required to make that ‘right decision’, and the Product Manager role must evolve accordingly.</p><h4>Product Owner</h4><p>Product Owner is a role mandated by Agile Software development, is delivery focussed, typically with a solid technical understanding. The role corresponds to the ‘Inbound Activities’ shown above, and can include customer trial and in-life activities. The Product Owner is much closer to the software team, with daily contact and direct input into each sprint.</p><p>When I first viewed Andrew Ng’s, Stamford Professor, Google Brain Founder’s 2016 “AI Nuts &amp; Bolts” <a href="https://www.youtube.com/watch?v=wjqaz6m42wU&amp;feature=youtu.be&amp;t=6038">lecture</a> — highly recommended, and still very relevant, I found myself considering his AI Product Manager definition, which in reality described a ‘Product Owner’.</p><p>I’m jumping ahead, but this greater understanding of software lifecycle and technical background means that the first AI Product Managers you meet are very likely to be ‘augmented’ Product Owners — Product Owners with an AI skill set.</p><h4>Product Marketing Manager</h4><p>This role is primarily responsible for getting the product ‘off the shelves’, translating a Product Manager brief into actual sales, the “Outbound Activities” listed by Product Focus. Unless the product itself is heavily differentiated by AI, or is a true AI product, for example a service or API, then it’s unlikely that the Product Marketing manager will need a good understanding of machine learning.</p><p>The above descriptions of Product Roles are simplifications. Depending on the size and culture of your organisation, there’s potential for lots of overlap — I’ve certainly been involved in strategic, detailed technical delivery and marketing work as a Product Manager.</p><p>You’ll see alternate definitions, but as product manager I like <a href="https://www.forbes.com/sites/janakirammsv/2020/03/02/key-takeaways-from-the-gartner-magic-quadrant-for-ai-developer-services/#5d679622e3e5">Forbe’s</a> take on Gartner’s definition — “machine learning is a subset of data science while <em>artificial intelligence is the business outcome of machine learning.</em>”</p><p>Product Manager’s more commercial and market focus means they are closer to the business outcome and hence ‘AI’. Product Owner’s greater technical knowledge and delivery focus puts them nearer to the underlying machine learning.</p><p>But before defining the skills needed, let’s take a quick look at some important trends.</p><h3>Accessing Machine Learning</h3><p>Things are changing — increasingly, you don’t have to build machine learning solutions from the ground up using vast amounts of data.</p><h4>Cognitive APIs — for Standard Use Cases</h4><p>For a while it’s been possible to purchase end to end solutions from suppliers, or else utilise cognitive APIs (or more simply put, regular ‘APIs’) to embed a specific AI capability into your product.</p><p>As an example, in my last <a href="https://medium.com/swlh/building-a-simple-ai-photo-analytics-app-on-aws-68d0b0f84c87">post</a>, I showed how to build a self contained AI image analysis dashboard using Amazon Rekognition’s API. No data needed to create the model, all entirely serverless, and infinitely scalable. We’re simply using a RESTful API to get the results we need.</p><p>This is fine — but you’ve no opportunity to unlock the value of your business’s data, or to address custom use cases. Until recently, the only way to proceed would be to build your own machine learning solutions, with all of the associated complexity.</p><p>What’s changed is ‘AutoML’.</p><h4>AutoML — for Custom Use Cases</h4><p>‘AutoML’ services are emerging that allow you to build on existing machine learning models, whilst also automating a complex and time consuming portion of machine learning development — data analysis, model tuning and selection.</p><p>AutoML solutions are now available from Google, Microsoft, IBM, H2o, Amazon Web Services and others. This is a big step towards democratising access to AI.</p><p>You can extend text comprehension to specific domains, think IBM Watson’s Natural Language Understanding <a href="https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-customizing">customisation</a>, or create more granular image &amp; video classification, for example via Google’s <a href="https://cloud.google.com/automl">AutoML</a>, without needing to bring huge quantities of data.</p><p>The requirement for <em>less data </em>(“small data”)<em> </em>is<em> </em>key, because having sufficient high quality, labelled data in order to run supervised learning and gain the necessary results, is one of the hardest aspects of building AI solutions and has slowed adoption of machine learning solutions beyond large players.</p><p>AutoML makes it much easier for your organisation to use its unique and valuable data to create new business outcomes or to help differentiate your products.</p><p>Going back to my AI app example, I could extend my cloud based image analytics to use Amazon Rekognition’s ‘custom labels’. Perhaps I need finer grained classification — a particular type of car vs ‘a car’. Or, I need higher classifier performance for my use case. I could accomplish this with much less data, using tools to help my team or external teams label the data — In Amazon’s case, ‘GroundTruth.’ Google simply call their equivalent service, ‘human labelling’.</p><p>So, with ‘AutoML’ there’s transfer learning, data transformation, experimentation, model tuning and selection happening in the background — your development teams don’t need to start working with the underlying CNN, LSTM, RNN, or which ever flavour Neural Network model and topology has been used. Even better, the latest and most powerful models are made available — you don’t need to research, select or manage them in-life.</p><h4>Ground Up Machine Learning — for Advanced Use Cases</h4><p>If you’ve got the data, the data teams and the development expertise, and you’re addressing cutting edge use cases, then you may be in a position to develop your own machine learning models. You’ll need robust data processes — AWS propose a modified version of CRISP-DM as best practice, whilst for Microsoft Azure, <a href="https://docs.microsoft.com/en-us/azure/machine-learning/team-data-science-process/overview">Teams Data Science Processes</a>, is the preferred point of reference.</p><p>There’s lots of risk associated with this approach however — there’s a good chance your work won’t even reach production. When you’re building your own models, you need to keep in mind that Machine Learning is ‘Stochastic’. Given the same or similar inputs, you won’t necessarily get the same outputs. Of the 100s of machine learning papers published every day, few will scale to production.</p><p>Where the Machine Learning code will sit — on a device, at the ‘edge’, in the cloud, is an important consideration. The machine learning code itself is only a small portion of the whole needed to deploy and manage a model in production.</p><p>With the emergence of Cloud AI &amp; AI PaaS, this should get easier. An AWS example: your data and development teams can build a custom model on TensorFlow, DevOps deploying as container behind a secure globally available endpoint. Your data team’s Extract Transform &amp; Load (ETL) data work can be run on a fully managed Spark environment.</p><p>Still, building your own solutions requires significantly more knowledge and incurs greater risk.</p><h3>The AI Product Manager</h3><p>Given the trend towards more accessible machine learning, I believe that the Product skills ‘chasm’’ can be bridged — Product Management roles can and will evolve to address the AI revolution.</p><p>It’s likely that Product Managers will fall into either “AI Aware” or “AI Enabler” categories below, whilst Product Owners are more likely to become “AI Experts.”</p><p>Let’s take a look — refer back to the product life-cycle at the beginning of this article for some Product Lifecycle context.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6nN_dm6l0tH87po_c4casg.png" /><figcaption>AI Skills for Product Managers</figcaption></figure><p>The amount of AI skill needed by Product Managers is closely related to the approach taken to AI enable the product. If you’re going to be consuming AI Services and APIs, then the amount of machine learning knowledge needed within the product team is less.</p><p>However, all Product Managers will need to understand how AI will affect product risk, time to market and their product’s market itself. Proper understanding of data, how it is managed, transformed and used, how to correctly phrase requirements will also be essential.</p><p>No matter where you sit, as a member of a product team you will need to be “AI Aware”. The ability to determine whether machine learning can or should be used up front will save considerable time and effort.</p><p>Although small data techniques are <a href="https://www.industryweek.com/technology-and-iiot/digital-tools/article/21122846/making-ai-work-with-small-data">emerging</a>, data sourcing and management is still critical to developing new use cases, and the ability to understand and work with that data is going to become an important product skill. Ensuring the necessary opt-ins, and data systems are defining characteristics of a Product Manager or Product Owner able to act as an “AI Enabler”.</p><p>Product Owners working in software organisations will benefit from much deeper understanding of AI, proportionate to the amount of in-house AI development taking place. They are very likely to be “AI Experts”, with the ability to engage with machine learning teams across the business to effectively lead, whilst checking and challenging delivery decisions taken to bring their product to market.</p><p>In conclusion, whilst the software and internet sectors currently frame AI Product Manager roles as specialist Product Owners, as AI reaches beyond software, Product Managers will adapt and evolve to add key AI skills. This will ensure that the full value of AI can be brought into new industries and sectors.</p><figure><a href="https://www.theactivationfunction.com/"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yRf1mGomD7pV0dN02a_dzQ.jpeg" /></a></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=729af7e99154" width="1" height="1" alt=""><hr><p><a href="https://medium.com/digital-diplomacy/ai-product-managers-evolution-or-revolution-729af7e99154">AI Product Managers — Evolution or Revolution?</a> was originally published in <a href="https://medium.com/digital-diplomacy">Digital Diplomacy</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Simple AI Photo Analytics App on AWS]]></title>
            <link>https://medium.com/swlh/building-a-simple-ai-photo-analytics-app-on-aws-68d0b0f84c87?source=rss-20013b2143c2------2</link>
            <guid isPermaLink="false">https://medium.com/p/68d0b0f84c87</guid>
            <category><![CDATA[aws-glue]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[aws-lambda-functions]]></category>
            <category><![CDATA[amazon-rekognition]]></category>
            <category><![CDATA[amazon-quicksight]]></category>
            <dc:creator><![CDATA[Andrew Gooday]]></dc:creator>
            <pubDate>Wed, 17 Jun 2020 16:24:09 GMT</pubDate>
            <atom:updated>2020-12-03T14:26:12.813Z</atom:updated>
            <content:encoded><![CDATA[<h4>A Step by Step Guide</h4><h4>Using Amazon’s Rekognition, Lambda &amp; Quicksight for near real-time Image Analytics</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*euxhe856Wg2rmnYW8oW3ZQ.png" /><figcaption>Image Analytics via Amazon QuickSight</figcaption></figure><h3>Overview</h3><p>We’re going to use some clever AWS features to build an AI powered image analytics app, providing machine learning derived insights on photos in near real time.</p><p>What’s more, we’re going to use ‘serverless’ components, meaning you should be able to do get this up and running on your images in a couple of hours or less!</p><p>Here’s what you’re aiming for — a dashboard that refreshes with some nice infographics to show you what objects and text are in your photos, immediately that you upload them.</p><h3>Getting Started</h3><p>First things first, you’ll need an <a href="https://aws.amazon.com/">AWS</a> account, ideally with full admin access. <br>Please, use the SAME REGION for everything — S3, QuickSight, Athena, Lambda, Glue and make QuickSight is available in the region you’ve chosen.</p><p>There will be some small charges incurred — check AWS pricing details for your region. If you’re on the 1 year free tier, then unless you do something very wrong (in particular in the “Lambda” section below), you shouldn’t spend more than a couple of dollars maximum. (I didn’t)</p><p>What you will notice as we progress is that we’re:</p><ul><li>NOT building, configuring, managing any servers including any databases. <br><em>That’s fine, because this is all going to be ‘serverless’.</em></li><li>NOT building, tuning, training any neural nets.<br><em>Instead, we’ll be using AWS Rekognition’s built in capabilities.</em></li></ul><h3>How it Works</h3><p>The user (you!) puts 20–30 images into a specific S3 bucket (1.) This triggers an event that launches a lambda function for which I supply a bit of Python code (2.) This is code running in the cloud without you having to think about servers etc. so pretty useful. In fact, 20–30 Lambda functions will fire up, one to handle each image, last about 1 second, then disappear.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EXL7o2cFlQGuJCS7RGKdjw.png" /><figcaption>AI Photo Analytics App</figcaption></figure><p>The Lambda python code calls Amazon Rekognition, which runs 2 algorithms — object detection and text detection on the image (3.) Because the image is sitting in S3, you don’t have to worry about pre-processing it. You can extend the Python code later to add in more Rekognition calls (face detection etc.) if you want.</p><p>The Lambda function stores the resulting label and text descriptions (4), one file each, into their corresponding ‘folders’ in S3. These output files are in JSON format, and ‘flattened’ so that you get some useful columns for analysis later. (Rekognition’s JSON is deeply nested — for example the ‘person’ label may have lots of ‘bounding boxes’ returned if there are numerous people in the image.)</p><p>So, here’s the clever bit step (5), we’ll manually run an AWS Glue crawler which will create the data catalogue and meta data needed so that both Athena and QuickSight can <em>run analysis directly on the JSON files (from step 4) as if you have a database.</em></p><p>You only need to run the crawler once, but based on the data generated from your first upload, there’s enough info there for the “image” and “labels” tables to be constructed automatically. Again, there’s no database behind this analytics app, just your JSON files sitting in S3.</p><p>As we’ve got two tables, if you want to use data from both text and object detection, we’ll need to join them to create a ‘view’. This can be done easily in Athena with a simple command (6).</p><p>Finally, step (7), we’ll connect QuickSight to the “database” and “view” created in the previous step.</p><h3>Setting Up Your S3 Buckets</h3><p>Step 1</p><p>If you’re reading this article, you’ll hopefully be at least a little familiar with S3. S3 is serverless, infinitely scalable storage, where all of your unstructured data can sit, floating around in a fully resilient and secure data lake.</p><p>But for this tutorial, you’ll just be creating 2 buckets.</p><ol><li>create a new empty bucket to store images in. I’d rather you had a specific bucket, used for no other purpose than storing images, as it will make it harder to get things wrong later. I’ve called mine “agooday-images-eu”.</li><li>create a new empty bucket for storing the image and text detection files and give each a separate ‘folder’ (they’re not really folders, but it’s easier to talk about them as such.) I’ve called mine “agooday-athena-test/labels”, “agooday-athena-test/text”. ** please do not put your output files in the same bucket as your source images **</li></ol><h3>Configuring Lambda &amp; Rekognition</h3><p>Steps 2, 3 &amp; 4</p><p>Ok, let’s configure the Lambda function. Here’s the python code:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0634aa0422415f5a5c650f4b15dc696b/href">https://medium.com/media/0634aa0422415f5a5c650f4b15dc696b/href</a></iframe><p>The <em>only </em>thing you need to do before we use this code is to update line 6 with your target bucket name — the name of the bucket where the images/ and text/ JSON will be stored.</p><p>Boto3 is AWS’s python SDK allowing you to control any AWS service. We’re using it to read/write files to S3 and to access Rekognition. Rekognition returns JSON in a Python Dictionary. The script saves this to the target bucket into a format that AWS Glue likes.</p><p>Log in to the AWS console and create a new Lambda function. Select “from scratch”, “python3.7” and “create a new role with basic Lambda permissions”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mrmcsXr1fLyyWKNPSWr8nA.png" /><figcaption>Create a Lambda Function — Author from Scratch</figcaption></figure><p>Now, go to permissions and click on the role that’s just been automatically created, in this example ‘s3lambda-role-4ywsmx9h’…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xFGnRadq729jM_L0YcqNJQ.png" /><figcaption>Lambda Permissions — Click on the Link</figcaption></figure><p>You should be now at the IAM console, where you’ll assign further permissions. Click “Attach Permissions” and search for each of the policies your missing from below. Click the checkbox and add (if you click on the policy you’ll just head off looking at the policy details.)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qmJ7SqW18W2nxrnwQ4EFQA.png" /><figcaption>Updating Lambda Role Permissions</figcaption></figure><p>You’ll need the CloudWatch role to see what’s happening and troubleshoot, S3 to read/write to your buckets, Rekognition — well, to recognise things!</p><p>Now we’re ready to paste the Python code into Lambda.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VcRi1t7nYFteEE2rDT4Y9g.png" /><figcaption>Pasting the Python Code into Lambda</figcaption></figure><p>You’ll see I’ve updated the target bucket in line 6. Once that’s done, save and select + Add trigger.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r4dClj__d_SGhxkWJkQkzQ.png" /><figcaption>Adding S3 Trigger</figcaption></figure><p>This dialogue will configure S3 to send an event and launch the Lambda function. Please select the source bucket in which you’ll be putting your images — and only your images, nothing else unless you want to trigger the Lambda. Disable the trigger for now and click add. Later when the trigger is active, you’ll be able to see it in your S3 bucket, properties, events as an active notification.</p><p>You should now be here:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8FLSwD49sh2XigqQSA9d7A.png" /><figcaption>Lambda — Code &amp; Triggers Configured</figcaption></figure><p>Time to test this is working. Step 1, select Test, navigate to s3-put (which if you get stuck, you can modify to help troubleshoot) and create a test event.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*K_RNMjPxqqEKC-Eu1aeyYA.png" /><figcaption>Configuring a Lambda Test Event</figcaption></figure><p>Run the test and you should see the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vfvG6TSIIJxFQZQv7GMhbQ.png" /><figcaption>Lambda Test Output</figcaption></figure><p>Don’t worry about the “Call Failed …” error.</p><p>Now enable the trigger, and click on your S3 bucket. This will open up the correct bucket (avoiding chance of error.) and upload a file directly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mATT5mSgr9USy2ZNW1QwIg.png" /><figcaption>Upload Photos into S3 via the Console</figcaption></figure><p>Now go back into Lambda, Monitoring and scroll down to logs. Click on the most recent log:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tC81ga_4JOlGzBYhPnBRVg.png" /><figcaption>Lambda Logs List`</figcaption></figure><p>And you should see something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*d2MWnHCWxoxJG7m7weqWhQ.png" /><figcaption>Detailed Logs from Lambda</figcaption></figure><p>If you don’t see any logs, check your permissions. If you need to troubleshoot further, then you’ll have to update the test put with some valid details. I’ve an example <a href="https://gist.github.com/theactivationfunction/a4f9a3035b900b853e84bf36465ef7a6">here</a> —</p><p>Navigate to your target bucket, and labels/ and check for a *.labels.json file.<br>There may also be a *.text.json file in your text/ directory. It depends on whether text was detected. Check the CloudWatch Lambda logs if in doubt as they will report whether labels or text found.</p><p>Well done! You’ve just configured an S3 trigger and Lambda functions to get object and text details, able to scale to handle any amount of photos (your finances allowing!) Note you may need to extend your Lambda default from the default of <a href="https://www.amazonaws.cn/en/lambda/faqs/#:~:text=All%20calls%20made%20to%20AWS,between%201%20and%20300%20seconds.">3s</a> if you see Lambda timeouts in the CloudWatch logs.</p><h3>Cataloguing Your AI Data with Glue</h3><p>Step 5</p><p>It gets easier from here — Firstly, upload 20–30 photos into your S3 source bucket and check that the corresponding labels and text JSON files are generated in your target bucket. Not every image will have a text JSON file, but most if not all should have a labels file. (The Lambda function won’t write empty JSON if nothing is detected.)</p><p>You’ve done this so that we can build a table with a good number of columns in. Some images will have lots of labels, and nested parent categories.</p><p>Navigate to Glue and “Add Crawler”. I’ve expanded the custom classifier tab below to show it’s there, and to point out I wasted a lot of time here trying to get Glue to work with the JSON as it’s sent by Rekognition. Fortunately, your shiny new Lambda function flattens and formats the JSON, so nothing other than naming your crawler needed here — ignore the classifiers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UJP3KlRqSg2yllUHfOXyDg.png" /><figcaption>Creating a Crawler — No need for Custom Classifiers!</figcaption></figure><p>On the next screen select “Data stores”, then “next”. Now add your <em>target </em>bucket as your data store.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u27Gs1Y3OSLlzYuXkw0eUw.png" /><figcaption>Specify the Data Source for your Analytics</figcaption></figure><p>On the next screen, select “no” to add another data store, then “Next”.</p><p>Create an IAM role</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H4_ajtKNH8mWXCSk0Cmy1Q.png" /><figcaption>Setting the Glue Role</figcaption></figure><p>Then “Run on Demand”, “Next”</p><p>On “Configure the crawler’s output”, select Add Database and give it a name “image-db”, “create” and then “Next”, then “Finish”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dszXBFI8v1KEoydO6ZqaGg.png" /><figcaption>Run the Crawler</figcaption></figure><p>It’ll take about 40–50 seconds, and when it’s finished you will have a database “image-db” with two tables, “labels” and “text” in.</p><p>Have a look at the “labels” table. And yes, you can now query this with SQL.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cdVXT-nP9GSGmSdcPmUC6A.png" /><figcaption>A Table Created from your Image Data, automatically by Glue</figcaption></figure><p>Go back to the list of tables, and select Action, “View data”, “Preview data”.</p><p>This will take you to Athena, which is where we need to be next…</p><h3>Creating a View in Athena</h3><p>Step 6</p><p>Nearly there. You should now be looking at a SQL query dialogue:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dDWysEsVNbgrxOj6HD4HiA.png" /><figcaption>Querying your Image’s JSON files Directly, via Athena’s SQL</figcaption></figure><p>So we just now need to create a view so as to join both labels and text table together.</p><p>I’m being selective below, but you could select all columns if you wanted.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2a2afe896905e08d296949f42f34669b/href">https://medium.com/media/2a2afe896905e08d296949f42f34669b/href</a></iframe><p>Run this sql, and you should now have a view that looks like this —</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dDWysEsVNbgrxOj6HD4HiA.png" /><figcaption>Creating a View in Athena</figcaption></figure><p>That’s all of the data catalogue work done — every time you upload an image, the AI derived text and image label data will be available immediately in your database.</p><p>That’s cool — but it would be even better if we could present in a dashboard, which is what we’ll do next.</p><h3>Connecting It All Together — Adding QuickSight</h3><p>Step 7</p><p>Now for the fun stuff — let’s hook up QuickSight and take a look at our image data. Create a QuickSight account, then navigate to “Create a Data Set”.</p><p>Ensure you use the database name we created earlier.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6BxymVeuHjqgefDjpJblfQ.png" /><figcaption>Connecting Athena to Image Data</figcaption></figure><p>Then select “myview” on the next screen.</p><p>Don’t use SPICE, query data directly for now. This ensures that as you add images, you’ll be able to access the image object and text data straight away.</p><p>Then click “Visualize”. You now need to check permissions.</p><p>Select “Manage QuickSight” from your account icon, top right, security &amp; permissions. Click on Add or Remove and ensure IAM, Amazon S3 and Amazon Athena are checked.</p><p>Most importantly, click on Details and ensure your target bucket is checked (the bucket with the JSON files in.)</p><p>That’s it — we’re done!</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fnc7vgXyS8z4%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dnc7vgXyS8z4&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fnc7vgXyS8z4%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/5130ef92326ecb9763b55d0ec619488a/href">https://medium.com/media/5130ef92326ecb9763b55d0ec619488a/href</a></iframe><p>Take a look at this video — I start with empty S3 image and data buckets, so QuickSight shows ‘no data’. I then drop the images back into the source folder. This triggers Lambda, and the logs show the label and text capture. The target folders are populated and a refresh of QuickSight shows the data present and ready for querying.</p><h3>What Next?</h3><p>You could try adding custom labels — to get AWS Rekognition to build on what it can already identify (transfer learning without the hassle.) Or add face recognition, content moderation. You could use Lambda to send images with high text content to Comprehend for further insight.</p><p>Keep an eye on your billing data via the billing console and don’t go crazy with your free tier limits, for example rekognition’s 5000 images per month.</p><p>It’s more work, but you could start working with videos and streams — check out what’s possible <a href="https://aws.amazon.com/rekognition/">here</a></p><figure><a href="https://www.theactivationfunction.com/"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yRf1mGomD7pV0dN02a_dzQ.jpeg" /></a></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=68d0b0f84c87" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/building-a-simple-ai-photo-analytics-app-on-aws-68d0b0f84c87">Building a Simple AI Photo Analytics App on AWS</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>