<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Theodoros Ntakouris on Medium]]></title>
        <description><![CDATA[Stories by Theodoros Ntakouris on Medium]]></description>
        <link>https://medium.com/@zarkopafilis?source=rss-a030fa2f6701------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*hcLDkyzHGS7vda57VHgjNQ.png</url>
            <title>Stories by Theodoros Ntakouris on Medium</title>
            <link>https://medium.com/@zarkopafilis?source=rss-a030fa2f6701------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 16:37:17 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@zarkopafilis/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Image deduplication using OpenAI’s CLIP and Community Detection]]></title>
            <link>https://zarkopafilis.medium.com/image-deduplication-using-openais-clip-and-community-detection-2504f0437e7e?source=rss-a030fa2f6701------2</link>
            <guid isPermaLink="false">https://medium.com/p/2504f0437e7e</guid>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Theodoros Ntakouris]]></dc:creator>
            <pubDate>Sun, 09 Apr 2023 11:56:06 GMT</pubDate>
            <atom:updated>2023-04-09T11:56:06.632Z</atom:updated>
            <content:encoded><![CDATA[<p>A short guide on how to use image embeddings from OpenAI’s CLIP and clustering techniques in order to group near-duplicate images together.</p><figure><img alt="Overview" src="https://cdn-images-1.medium.com/max/1024/1*r81t8fgdtKD9p8T-Mw59WQ.png" /><figcaption>Overview</figcaption></figure><h3>Overview</h3><ul><li>We’re going to use <a href="https://github.com/huggingface/transformers/">huggingface/transformers</a> to quickly load openai/clip-vit-base-patch32 from the hub</li><li>We’re going to perform inference in order to extract image features</li><li>We will use those feature vectors to perform community detection in order to place near-similar images into the same buckets.</li></ul><h4>OpenAI/CLIP</h4><p><a href="https://openai.com/research/clip">CLIP </a>is trained by trying to align image &lt;&gt; text embedding pairs, or “learning visual representations from natural language supervision”.</p><p>You can use it’s text or image embeddings to accomplish a lot of different tasks, such as zero-shot image classification! It’s embeddings are pretty powerful.</p><h4>Dataset &amp; Evaluation</h4><p>For this task, we’re going to use the <a href="https://www.kaggle.com/datasets/barelydedicated/airbnb-duplicate-image-detection">AirBnB Duplicate Image Dataset, available on Kaggle</a>. For ease of use, we’re only going to use one directory which contains 14 images: Test Data/bathroom . Don’t worry, the code easily generalizes to many images.</p><p>We’re only going to qualitatively evaluate this (just by looking into the output images), to get the gist of it. You can design your own evaluation criteria given a dataset, the task at hand and other business-problem related assumptions.</p><h3>Implementation</h3><p>Without further delay, let’s dive straight in to the code. First, we’ll load the models from the hugging face hub.</p><pre>from transformers import CLIPProcessor, CLIPModel<br><br>model = CLIPModel.from_pretrained(&quot;openai/clip-vit-base-patch32&quot;)<br>processor = CLIPProcessor.from_pretrained(&quot;openai/clip-vit-base-patch32&quot;)</pre><p>Then, let’s just load the images under our directory and generate our embeddings.</p><pre>import torch<br>from pathlib import Path<br><br>demo_directory = Path(&quot;.../bathroom&quot;)<br># {&quot;&lt;name&gt;&quot;: &quot;.../bathroom/&lt;name&gt;.jpg }<br>images_to_paths = {image_path.stem: image_path for image_path in demo_directory.iterdir()}<br><br>images = [Image.open(path) for path in images_to_paths.values()]<br>inputs = processor(images=images, return_tensors=&quot;pt&quot;, padding=True)<br><br>with torch.no_grad():<br>    outputs = model.get_image_features(**inputs)<br><br>images_to_embeddings = {image_id: tensor_embedding.detach().numpy() for image_id, tensor_embedding in zip(images_to_paths.keys(), outputs)}</pre><p>Now, it’s time to perform community detection by clustering. There are lot’s of clustering algorithms out there, and one of the popular kids in the block is KMeans. Beware, we will avoid clustering methods that use a predefined number of clusters as an input, as this assumes that we know how many groups of similar images our dataset includes!</p><pre>import numpy as np<br>from sklearn.cluster import DBSCAN<br>from collections import defaultdict<br><br># tune eps to fit your needs<br>clustering = DBSCAN(min_samples=2, eps=3).fit(np.stack(images_to_embeddings.values()))<br><br># postprocess cluster labels into groups of similar images<br>image_id_communities = defaultdict(set)<br>independent_image_ids = set()<br><br>for image_id, cluster_idx in zip(images_to_paths.keys(), clustering.labels_):<br>    cluster_idx = int(cluster_idx)<br>    if cluster_idx == -1:<br>        independent_image_ids.add(image_id)<br>        continue<br><br>    image_id_communities[cluster_idx].add(image_id)</pre><p>Let’s plot the results.</p><pre>len(independent_image_ids) # = 10<br><br>image_id_communities # =<br># {<br># 0: {&#39;berlin_1583556_1&#39;, &#39;berlin_1583556_2&#39;},<br># 1: {&#39;berlin_969200_1&#39;, &#39;berlin_969200_2&#39;}<br># }</pre><pre>for image_id_community in image_id_communities.values():<br>    for image_id in image_id_community:<br>        plt.figure()<br>        plt.imshow(Image.open(images_to_paths[image_id]))<br><br># images that have not got a cluster with similar images assigned to them<br>for image_id in independent_image_ids:<br>    plt.figure()<br>    plt.imshow(Image.open(images_to_paths[image_id]))</pre><p>You can see that from the total 14 images of the dataset, we detect 2 groups that contain 2 similar images each. Quickly skim through all the other images (10 in total) and you’ll see that they are not near-duplicates.</p><p>Here are the plotted similar groups of images: pretty tough examples for a near-duplicate detection system!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/798/1*vnnUPWrUQWzIXea_cw2iVQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/786/1*yKnVkZKMnhqR86OwqLlHBw.png" /></figure><h3>Next Steps</h3><p>For the interested data scientist, here are some take-home exercises or things to think about:</p><ul><li>Use a projection method to plot the 512-d embeddings of the images, and estimate nearest-neighbour distance (`eps` of the clustering algorithm)</li><li>Design a pipeline that evaluates the performance of near-duplicate community detection algorithms</li><li>Experiment with different model embeddings and different clustering algorithms. What’s the best combination to use?</li></ul><p>Thanks for reading!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2504f0437e7e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Python & Maintainability: Keeping track of constants]]></title>
            <link>https://zarkopafilis.medium.com/python-maintainability-keeping-track-of-constants-36f7d82658a7?source=rss-a030fa2f6701------2</link>
            <guid isPermaLink="false">https://medium.com/p/36f7d82658a7</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[web-development]]></category>
            <dc:creator><![CDATA[Theodoros Ntakouris]]></dc:creator>
            <pubDate>Tue, 19 Apr 2022 08:37:32 GMT</pubDate>
            <atom:updated>2022-04-19T08:37:32.361Z</atom:updated>
            <content:encoded><![CDATA[<p>Hard-coded string values that linger around your project impact your ability to change negatively, but there’s an easy way around that.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ksFIrj8C_ATZZTgTkmrkSA.jpeg" /></figure><h4>But first, what’s wrong with <a href="https://softwareengineering.stackexchange.com/questions/365339/what-is-wrong-with-magic-strings">magic strings</a>?</h4><p>We’re going to demonstrate the usage of using static classes to keep track of constants, in 2 scenarios:</p><ul><li>Declaring routes in a FastAPI app</li><li>Keeping track of metrics and artifacts in a machine learning pipeline</li></ul><h3>Dictionaries</h3><p>You simply need to pass data around. It’s so tempting to just instantiate a new dict and assign a few items to it; Python makes this so easy, just {a: b}.</p><p>The superbly optimized Python dict, <strong>is both a blessing, and a curse</strong>, for code readability and maintainability. It’s not proper to have hard coded constants flying around. A string rename in one file can lead to problems in another portion of the program. If the project contains good test code, this mistake is probably going to be caught during testing. If not, your api/data pipeline/you name it, will blow up in prod. We don’t want that.</p><h4>Proposed technique: Static class definitions</h4><p>We’re going to keep all the constant definitions in one place: under a python module, inside classes. Hierarchies like that, can help the reader/reviewer understand the scope of your changes, and there’s a pretty easy way to rename everything by editing a few lines of code.</p><p>Let’s consider a FastAPI app and showcase how we are going to declare routes. There are 2 ways to go around this. Pick your poison:</p><ul><li>A relative structure where FastAPI itself keeps track of prefixes</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9eec01adccc0012982304a758d29e47f/href">https://medium.com/media/9eec01adccc0012982304a758d29e47f/href</a></iframe><ul><li>And an absolute structure where the classes keep track of their hierarchy</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ff33560316aa7ce726f4e2d4103ca895/href">https://medium.com/media/ff33560316aa7ce726f4e2d4103ca895/href</a></iframe><p>Personally, I’d pick the second one. It’s easier to construct URIs by using those classes/lambda functions.</p><p>Let’s consider a simple ML pipeline. You train, evaluate the model and post metrics to some experiment tracking server. Almost all of those tracking solutions accept a very specific structure for metrics, which makes it so tempting to use dicts: str -&gt; value pairs. Those can also contain some iteration index or timestamp.</p><p>One could argue that for each experiment, you can declare some sort of data class or typed dict to keep track of those. You can have lot’s of optional parts there, and an implementation should look like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9850bc6c26c55369691808916effb320/href">https://medium.com/media/9850bc6c26c55369691808916effb320/href</a></iframe><p>You see that things quickly get cluttered: optional fields with default None values, and a lack of over-time representation — you’d have to either have a list of MetricsInstagram or create a new dataclass that contains the history of all metrics for the given experiment. Data Science and Machine Learning projects are bound to change very fast (especially during early stages). This looks like a good match for completely dynamic dictionaries.</p><p>Adding a new experiment variant or a new optional metric in specific use cases can slow the whole project down, just because you have to maintain the metrics package! Not to mention that this needs to pull all the metrics from the tracking server, map those to the dataclass and then pick the one you need!</p><p>We can tackle those problems by using (you guessed it) static class structures and scopes! Here’s what I mean:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2029d0cbb5fc7e3ed236025c9465987c/href">https://medium.com/media/2029d0cbb5fc7e3ed236025c9465987c/href</a></iframe><blockquote><em>That’s just a proof of concept example. I expect the interested reader to refactor and rearrange parts according to his own needs.</em></blockquote><p>This is a really flexible approach that supports arbitrary combinations of scopes and metric names, and that also considers some sort of maintainability. It’s my personal choice for machine learning projects, as of now.</p><p>Thanks for reading!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=36f7d82658a7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Functional and Type-Safe Heterogeneous Data Handling in Rust]]></title>
            <link>https://zarkopafilis.medium.com/functional-and-type-safe-heterogeneous-data-handling-in-rust-cdb62edfa147?source=rss-a030fa2f6701------2</link>
            <guid isPermaLink="false">https://medium.com/p/cdb62edfa147</guid>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[rust]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Theodoros Ntakouris]]></dc:creator>
            <pubDate>Fri, 01 Oct 2021 05:17:28 GMT</pubDate>
            <atom:updated>2021-10-01T05:17:28.027Z</atom:updated>
            <content:encoded><![CDATA[<h4>Safely store and handle incoming heterogeneous data in a functional, type-safe, rusty way.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hkRxQym-GvYy1cjBbudajg.jpeg" /><figcaption>Image by <a href="https://pixabay.com/users/5317367-5317367/?utm_source=link-attribution&amp;utm_medium=referral&amp;utm_campaign=image&amp;utm_content=2672184">5317367</a> from <a href="https://pixabay.com/?utm_source=link-attribution&amp;utm_medium=referral&amp;utm_campaign=image&amp;utm_content=2672184">Pixabay</a></figcaption></figure><h3>Problem Statement</h3><p>You’ve got to make a web API that accepts JSON objects (or anything that <strong>serde</strong> supports, actually). You need some code — first way of declaring the structure of those incoming entities, so that they can be processed accordingly and deserialized into easy-to-use structs and traits. Consider the following example:</p><ul><li>An endpoint receives an ID and an Value that needs to be stored on a database, and retrieved afterwards</li><li>For start, we only need to handle a few cases: strings, integers, floats and booleans</li></ul><p>While this seems pretty straightforward, let’s investigate what happens when we need to incrementally add new types in our app, for example:</p><ul><li>We need to handle captioned images</li><li>We need to handle specific JSON structures differently as payload</li></ul><p>It is fair to assume that the backing database can only efficiently store small values, and you need to redirect larger artifacts (such as media), in a different blob storage service. We’re going to also assume the usage of a Document Database, but you can really use any storage system with proper modifications (ex. flattening your schemas).</p><h3>Functional Treatment</h3><p>There are a lot of ways to face this problem, with different advantages and disadvantages, either on the code-side or the database/infrastructure-side. We’re going to take a completely type safe and functional approach:</p><blockquote><strong>Everything will be typed, and when we need to add a new handled type to our application, every case will need to be reasoned about with code, else the program will not even compile. </strong>This way, no mistakes are going to be made, and we are always going to cover all edge cases of data processing and storing.</blockquote><h3>Proposed Solution</h3><p>Let’s start by defining the schema that our service will consume. We are looking for 2 kinds of HTTP requests: to save entities and to retrieve entities, by id.</p><p>First, for a simple primitive-valued payload:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d9c9c03c07e571e6570d290be870bf95/href">https://medium.com/media/d9c9c03c07e571e6570d290be870bf95/href</a></iframe><p>Then, for other, <em>heterogeneous </em>payloads, such as a captioned image:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b165a0c5224a8d8017b627aafc6cd0de/href">https://medium.com/media/b165a0c5224a8d8017b627aafc6cd0de/href</a></iframe><p>You can easily map those out to a few simple structs . But, we need a functional way of handling this heterogeneity. <a href="https://serde.rs/enum-representations.html">Serde’s (untagged) json enum parsing comes to the rescue.</a></p><p>We can use the following declaration in order to box the value field of the JSON objects as an enum, containing our different payload types:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ec73a518603a5584f613f929747de440/href">https://medium.com/media/ec73a518603a5584f613f929747de440/href</a></iframe><p>I called this entity EventEntity , but you can call this anyhow. <strong>This arrangement of structs and enums is able to parse both required inputs automatically</strong>, and you can still use Option&lt;T&gt; if something is optional.</p><p>Those <strong><em>contract</em></strong><em> </em>objects define our endpoint’s entity parsing signature. It’s a good practise to <strong><em>map </em></strong>those to different data model entities that you are going to actually persist in the database (and in many cases, it is also advised to also encorporate a <strong><em>domain </em></strong>collection of entities, mapping included) — see for example: <a href="https://blog.cleancoder.com/uncle-bob/2012/08/13/the-clean-architecture.html">Clean Architecture</a> or other popular methodologies.</p><p>We’re going to use the functional aspect of the language and the structures we just created, in order to map our EventEntity , to some DataResource , that we are going to ultimately save to disk. This may seem nonesense at first, but as you proceed to add more heterogeneous data that can be uniformly treated by your system (i.e. with great power comes great responsibility: don’t overdo it), it will make sense and you are also going to appreciate the security benefits. <strong>Nobody will forget to implement an edge case because they can’t; the program won’t compile.</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6bb9ac0eca0425510702c7b2da0a85b7/href">https://medium.com/media/6bb9ac0eca0425510702c7b2da0a85b7/href</a></iframe><p>This might seem terribly similar at first glance: and it is, except when you need to treat differently anything that’s different than a primitive value (ex. save an image to a blob storage).</p><p>Before stitching everything together on our controller, we’re going to define a very simple database trait (interface — for the OOP friends out there) that supports 2 operations:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d3243e16489e53a6a272d68c43fcb48a/href">https://medium.com/media/d3243e16489e53a6a272d68c43fcb48a/href</a></iframe><p>There’s a very simple implementation of this under <a href="https://github.com/ntakouris/hopplex/blob/main/src/db/in_memory.rs">gh/ntakouris/hopplex</a>, but all this Mutex/Cell/Box are out of scope for this article, so you can just think of this as a simple hash map.</p><p>Using the popular <a href="https://rocket.rs/">rocket.rs</a> framework, we’re going to scaffold a new project and add 2 endpoints: /retrieve_event and /publish_event . We’re also going to initialize an instance of our trait interface database to use in our endpoint functions:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b6243014772c62800c7e3fcc75770312/href">https://medium.com/media/b6243014772c62800c7e3fcc75770312/href</a></iframe><p>And finally, the missing parts are very, very trivial to fill in (<strong>you can’t do much but handle all the cases anyway!</strong>).</p><p>First, for the insertion part, we’re simply going to map any field that does not need special care. For this example, the CaptionedImage variant needs extra care, and that’s really easy to do:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0682dab3bb741d5666fa926b775be179/href">https://medium.com/media/0682dab3bb741d5666fa926b775be179/href</a></iframe><p>Retrieving the data follows the same pattern with inverse logic: retrieve the image from the blob storage instead of saving it. The caption, or other small structured json objects are saved in the document itself:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ec68a1cdbd4120e2e989a020ade21e33/href">https://medium.com/media/ec68a1cdbd4120e2e989a020ade21e33/href</a></iframe><h3>Conclusion</h3><p>Add your own error and/or response types and you’ve got a very elegant project to work with. Test cases are very easy to reason about as well: one for each action (endpoint), for each declared (supported) entity type.</p><p>In the event where you also need to process a stream of this entities in the same, structured manner, this sort of handling heterogeneous data is very useful as well.</p><p>The sample code is open source, ready to be downloaded and cargo run d, at <a href="https://github.com/ntakouris/hopplex">github.com/ntakouris/hopplex</a>. Please, download it and mess around!</p><h4><strong>Thanks for reading!</strong></h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cdb62edfa147" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Time Series Transformer]]></title>
            <link>https://medium.com/data-science/the-time-series-transformer-2a521a0efad3?source=rss-a030fa2f6701------2</link>
            <guid isPermaLink="false">https://medium.com/p/2a521a0efad3</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[time-series-forecasting]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Theodoros Ntakouris]]></dc:creator>
            <pubDate>Tue, 26 Jan 2021 18:57:53 GMT</pubDate>
            <atom:updated>2021-01-26T20:07:14.506Z</atom:updated>
            <content:encoded><![CDATA[<h4>All you need to know about the state of the art Transformer Neural Network Architecture, adapted to Time Series Tasks. Keras code included.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Bm-dofcbIwl2Wc5_BD8zlw.jpeg" /><figcaption>Image by <a href="https://pixabay.com/users/romnyyepez-12641300/?utm_source=link-attribution&amp;utm_medium=referral&amp;utm_campaign=image&amp;utm_content=4240338">romnyyepez</a> from <a href="https://pixabay.com/?utm_source=link-attribution&amp;utm_medium=referral&amp;utm_campaign=image&amp;utm_content=4240338">Pixabay</a></figcaption></figure><h3>Table of Contents</h3><ul><li>Introduction</li><li>Preprocessing</li><li>Learnable Time Representation (Time 2 Vec)</li><li>Architecture</li><li>Bag Of Tricks (things to consider when training Transformers)</li></ul><h3>Introduction</h3><p><a href="https://arxiv.org/abs/1706.03762">Attention Is All You Need</a> they said. Is it a more robust convolution? Is it just a hack to squeeze more learning capacity out of fewer parameters? Is it supposed to be sparse? How did the original authors come up with this architecture?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/551/1*sJnNyqxX6oMm1bKZ6HXvEg.png" /><figcaption>The Transformer Architecture</figcaption></figure><ul><li>It’s better than RNNs because it’s not recurrent and can use previous time step features without a loss in detail</li><li>It’s the top performer architecture on plethera of tasks, including but not limited to: NLP, Vision, Regression (it scales)</li></ul><p>It is pretty easy to switch from an existing RNN model to the Attention architecture. Inputs are of the same shape!</p><h3>Preprocessing</h3><p>Using Transformers for Time Series Tasks is different than using them for NLP or Computer Vision. We neither tokenize data, nor cut them into 16x16 image chunks. Instead, we follow a more classic / old school way of preparing data for training.</p><p>One thing that is definitely true is that we have to feed data in the same value range as input, to eliminate bias. This is typically on the [0, 1] or [-1, 1] range. In general, it is recommended to apply the same kind of preprocessing pipeline on all of your input features to eliminate this bias. Individual use cases may be exempt from this, different models and data are unique! Think about the origin of your data for a moment.</p><h4>Popular time series preprocessing techniques include:</h4><ul><li>Just scaling to [0, 1] or [-1, 1]</li><li>Standard Scaling (removing mean, dividing by standard deviation)</li><li>Power Transforming (using a power function to push the data to a more normal distribution, typically used on skewed data / where outliers are present)</li><li>Outlier Removal</li><li>Pairwise Diffing or Calculating Percentage Differences</li><li>Seasonal Decomposition (trying to make the time series stationary)</li><li>Engineering More Features (automated feature extractors, bucketing to percentiles, etc)</li><li>Resampling in the time dimension</li><li>Resampling in a feature dimension (instead of using the time interval, use a predicate on a feature to re-arrange your time steps — for example when recorded quantity exceeds N units)</li><li>Rolling Values</li><li>Aggregations</li><li>Combinations of these techniques</li></ul><p>Again, preprocessing decisions are tightly coupled to the problem and data at hand, but this is a nice list to get your started.</p><p>If your time series can become stationary by doing preprocessing such as seasonal decomposition, you could get good quality predictions by using smaller models (that also get trained way faster and require less code and effort), such as <a href="https://github.com/ourownstory/neural_prophet">NeuralProphet</a> or <a href="https://blog.tensorflow.org/2019/03/structural-time-series-modeling-in.html">Tensorflow Probability</a>.</p><p>Deep Neural Networks can learn linear and periodic components on their own, during training (we will use Time 2 Vec later). That said, I would advise against seasonal decomposition as a preprocessing step.</p><p>Other decisions such as calculating aggregates and pairwise differences, depend on the nature of your data, and what you want to predict.</p><p>Treating sequence length as a hyper parameter, this leads us to an input tensor shape that is similar to RNNs: (batch size, sequence length, features) .</p><p>Here is a drawing for all the dimensions set to 3.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/484/1*q-6btrS9S_mcJDdcAW-ItA.png" /><figcaption>Input Shapes</figcaption></figure><h3>Learnable Time Representation</h3><p>For Attention to work, you need to attach the meaning of time to your input features. In the original NLP model, a collection of superimposed sinusoidal functions were added to each input embedding. We need a different representation now that our inputs are scalar values and not distinct words/tokens.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PYxmooqjijARQ2f0PWAW_g.png" /><figcaption>Positional Encoding Visualization from <a href="https://kazemnejad.com/blog/transformer_architecture_positional_encoding/">kazemnejad’s blog.</a></figcaption></figure><p><a href="https://arxiv.org/abs/1907.05321">The Time 2 Vec paper</a> comes in handy. It’s a learnable and complementary, model-agnostic represetation of time. If you’ve studied Fourier Transforms in the past, this should be easy to understand.</p><p>Just break down each input feature to a linear component ( a line ) and as many periodic (sinusoidal) components you wish. By defining the decomposition as a function, we can make the weights learnable through back propagation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/616/1*vlmwMaNqPh6bfQm-3XMtXQ.png" /><figcaption>Time 2 Vec Decomposition Equation</figcaption></figure><p>For each input feature, we apply the same layer in a time-independent (time-distributed layer) manner. This learnable embedding does not depend on time! Finally, concatenate the original inputs.</p><p>Here is a an illustration of the learned time embeddings, which are <strong>different</strong>, for each input feature category (1 learned linear component and 1 learned periodic component per feature).</p><blockquote>This does not mean that each time — step will carry the same embedding value, because the computation of the time2vec embeddings depend on the input values!</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/661/1*SZj-IjJNYkwaoe1jBsUwVw.png" /></figure><p>And, in the end, we concatenate these all together to form the input for the attention blocks.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cf1dcf0485b1109ac5bf7fddc65784f9/href">https://medium.com/media/cf1dcf0485b1109ac5bf7fddc65784f9/href</a></iframe><h3>Architecture</h3><p>We are going to use Multi-Head Self-Attention (setting Q, K and V to depend on the input through different dense layers/matrices). The next part is optional and depends on the scale of your model and data, but we are also going to ditch the decoder part completely. This means, that we are only going to use one or more attention block layers.</p><p>In the last part, we are going to use a few (one or more) Dense layers to predict whatever we want to predict.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TWkmOC8aK_2TgcCIm_mKMg.png" /><figcaption>Our Architecture</figcaption></figure><p>Each Attention Block consists of Self Attention, Layer Normalizations and a Feed — Forward Block. The input dimensions of each block are equal to it’s output dimensions.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/87f9d565f7c1827f443a920923755881/href">https://medium.com/media/87f9d565f7c1827f443a920923755881/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/38a365527fbe59930def0d00bff258f4/href">https://medium.com/media/38a365527fbe59930def0d00bff258f4/href</a></iframe><p><em>Optionally, before the head part, you can apply some sort of pooling (Global Average 1D for example).</em></p><h3>Bag Of Tricks</h3><p>Things to consider when using Transformers and Attention, to get the most out of your model.</p><ul><li><strong>Start Small</strong></li></ul><p>Don’t go crazy with hyperparameters. Start with a single, humble attention layer, a couple of heads and a low dimension. Observe results and adjust hyper parameters accordingly — don’t overfit! Scale your model along with your data. Nevertheless, nothing is stopping you from scheduling a huge hyperparameter search job :).</p><ul><li><strong>Learning Rate Warmup</strong></li></ul><p>A crucial part of the attention mechanism that leads to greater stability is learning-rate warmup. Start with a small learnign rate and gradually increase it till you reach the base one, then decrease again. You can go crazy with exponential — decaying schedules and sophisticated formulas, but I will just give you a simple example that you should be able to understand just by reading the following code out loud:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2cfcf33027b9ea104936970dea1a86d5/href">https://medium.com/media/2cfcf33027b9ea104936970dea1a86d5/href</a></iframe><ul><li><strong>Use Adam (or variants)</strong></li></ul><p>Non-accelerated gradient descent optimization methods do not work well with Transformers. Adam is a good initial optimizer choice to train with. Keep an eye out for newer (and possibly better) optimization techniques like AdamW or NovoGrad!</p><p><em>Thanks for reading all the way to the end!</em></p><h4>References</h4><p>[1] Attention Is All You Need, Ashish Vaswani and Noam Shazeer and Niki Parmar and Jakob Uszkoreit and Llion Jones and Aidan N. Gomez and Lukasz Kaiser and Illia Polosukhin, 2017</p><p>[2] Time2Vec: Learning a Vector Representation of Time, Seyed Mehran Kazemi and Rishab Goel and Sepehr Eghbali and Janahan Ramanan and Jaspreet Sahota and Sanjay Thakur and Stella Wu and Cathal Smyth and Pascal Poupart and Marcus Brubaker, 2019</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2a521a0efad3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/the-time-series-transformer-2a521a0efad3">The Time Series Transformer</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A quick overview of compression methods for Convolutional Neural Networks]]></title>
            <link>https://zarkopafilis.medium.com/a-quick-overview-of-compression-methods-for-convolutional-neural-networks-c651f88d6db5?source=rss-a030fa2f6701------2</link>
            <guid isPermaLink="false">https://medium.com/p/c651f88d6db5</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[efficiency]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[convolutional-network]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Theodoros Ntakouris]]></dc:creator>
            <pubDate>Thu, 10 Dec 2020 16:39:44 GMT</pubDate>
            <atom:updated>2020-12-10T16:39:44.464Z</atom:updated>
            <content:encoded><![CDATA[<h4>Namely: Hyperparameter Search, Convolution Variants, Network-in-Network, Weight Sharing, Pruning, Quantization</h4><p>The following is a very, <strong>very </strong>brief and <strong>non-technical</strong> summary of a chapter in the author’s dissertation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bshjHGybnDtWuJsbWQ2IyQ.jpeg" /><figcaption><a href="https://pixabay.com/photos/stress-tension-pressure-stressing-624220/">Picture by stevepb on Pixabay</a></figcaption></figure><h3>Motives</h3><p>A very deep neural network that is designed to solve hard problems might take a few seconds to run on modern computers with hardware accelerators for linear algebra operations. Similarly, smaller neural networks that do not take that much time to run, still do not meet realtime constraints. Hardware resources and execution time constraints is what drives the research community to investigate different methods of neural network compression. In the next few sections, common compression methods are presented.</p><h3>Hyperparameter Search</h3><p>In this specific problem, we do not only optimize for accuracy (or the metric of your choice), but also for efficiency. <a href="https://keras-team.github.io/keras-tuner/">Typical techniques</a> for hyperparameter search include Random Search, Grid Search, Bayesian Optimization and Bandit/Tourament based methods.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/771/1*p7dD69ZqWpEvT9tNSFnj1g.png" /><figcaption>Hyperparameter Search Visualization</figcaption></figure><p>For CNNs, one of such methods is <a href="https://keras-team.github.io/keras-tuner/">EfficientNet</a> (top of the leaderboards at <a href="https://paperswithcode.com/">Papers With Code</a>). Given the constraint of memory and FLOPS, it scales your typical convolutional network on several dimensions: width, depth, compound and resolution (number of layers, number of filters per layer, convolution dimensions).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lpYRVUUSj-HOYtvQqqqjPQ.png" /><figcaption>Efficient Net Scaling Dimensions</figcaption></figure><h3>Convolutional Variants</h3><p>Besides hyperparameter search methods, there are some hand-made computational schemes that depend on mathematical tricks to reduce the parameter count of each layer while they proportionally keep a larger amount of their computational capacity.</p><h4>Group Convolution</h4><p>Split the convolutional filters of each layer into equal sized groups. Apply to them to equal sized groups. You can end up with a more computationally demanding network if you choose too large group and filter sizes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/668/1*zXvj9I4d0P5J0XXsu4zmqQ.png" /><figcaption>Group Convolution</figcaption></figure><p>It’s not in one’s benefit to keep the output groups aligned as — is. This is a topological constraint that has no reason to exist. Treat it like <a href="https://arxiv.org/abs/1707.01083">ShuffleNets</a> do, and mix the output channel groups.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*z2OHqATvmHF4qSlEgK33Gg.png" /><figcaption>Output Channel Group Shuffling</figcaption></figure><h3>Network-In-Network</h3><p>Hyperparameter search? Special convolution ‘variants’? Empirical network design? Actually, this technique sits right in the middle. Let’s focus on the <a href="https://arxiv.org/abs/1602.07261">Inception</a>-v{1,2,3,4} model family. Those replace the typical convolutional layer stack in classic convoltional networks like <a href="https://arxiv.org/abs/1512.03385">ResNets</a> and <a href="https://arxiv.org/abs/1409.1556">VGG</a>, with layer groups that are aimed to have a greater knowledge capacity with regards to their slight increase in parameters and computational cost.</p><h4>Inception-v1</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/650/1*hNzVI7pP49ZzruEyMi3uyw.png" /><figcaption>Naive Inception v1 Block</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/651/1*FiAPKc79595U4_-YY0osMQ.png" /><figcaption>Inception v1 Block with less input channels</figcaption></figure><h4>Inception v{2,3}</h4><p>The larger convolutional kernels of size (5, 5) are replaced as a cheaper alternative: 2 layers with kernels of size (3, 3). Then, the layers which contain kernel sizes (n, n) with n &gt; 1, are replaced by 2 layers with the first one having filter size of (n, 1) and the second one having filter size of (1, n). In order to reduce the depth of the total network (you typically stack 10s of these layer groups together), the (1, n) and (n, 1) filters are computed in parallel.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/575/1*fwYRUvzWwOO5nzAbsIMIGg.png" /><figcaption>Inception v2 Group</figcaption></figure><p>The main difference of Inception v3 is that it uses greater convolutional filter sizes (7, 7).</p><h4>Inception v4</h4><p>This revision of the Inception family ivnestigated a lot of filter size and arrangement configurations in order to be more efficient. Special steps were taken to reduce computational overhead where less knowledge capacity is needed. In brief, downsize layers were created in order to work better with residual networks.</p><h3>Weight Sharing</h3><p>We can keep a smaller number of the weights required to train a neural network and project them into virtual weight matrices that are used to perform forward pass. Then, on the backward pass we just accumulate (or average) the loss for each real one, based on the virtual ones! <a href="https://arxiv.org/abs/1504.04788">This is typically achieved with some sort of hash function</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/585/1*9eVzyvieyXuz-Qir9HHGAA.png" /><figcaption>Weight Sharing Visualization</figcaption></figure><h3>Sparsity and Pruning</h3><p>Not every neuron is created (or trained) equal. Some may contribute practically zero information to the total output of the network (due to very small weight values). These can be pruned away, leading to a sparse (lots of zeros) network. Then, special matrix storage types (CSC, CSR) and linear algebra subroutines can be used to accelerate computation and reduce storage space by a huge margin. Loading the whole network on on-chip SRAM is a huge advantage. There are lots of techniques to do such thing:</p><ul><li><a href="https://proceedings.neurips.cc/paper/2015/file/ae0eb3eed39d2bcef4622b2499a05fe6-Paper.pdf">Learn the connections along with the weight values</a></li><li><a href="https://openreview.net/forum?id=SyGdZXSboX">Prune the network given some data with labels/data without labels/no data at all</a></li><li>Train a Reinforcement Learning agent to prune filters</li></ul><p>Keep in mind that a high perceptage of sparsity does not always correspond to full hardware utilization. You can get matrices with &gt; 50% sparsity that are so dense in general that can not be multiplied effectively with special hardware. The most easy way to take advantage of pruning is to remove specific filters as a whole when possible.</p><h3>Quantization</h3><p>You don’t need 32 bits to represent weights. You might not even need 16! <a href="https://en.wikipedia.org/wiki/Bfloat16_floating-point_format">Google</a> and nVidia already use less bits for precision (16 bits) for their neural computers. Weights are typically small and need high precision close to zero and one. You can use less bits to do that. Sometimes even 1 (<a href="https://arxiv.org/abs/1909.13863">XNORNet</a>)!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/601/1*OJuUP42gRqopUggz30EiAw.png" /><figcaption>IEEE 754</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/595/1*j0t32eiNt9EfE2tI69ReXw.png" /><figcaption>Google TPUs bfloat16</figcaption></figure><h4>Quantizing less than 16 bits</h4><p>It’s not efficient to stick with the typical sign — exponent — mantissa representation for very low bit width (≤ 8 bits). The usual ‘trick’ employed is to first project the weights to the [0, 1] range, using <em>tanh(.)</em> , then to quantize (map to nearest number) and then to re — project back to [-1, 1]. In the 1-bit scenario you expect the values to be either -1 or 1.</p><p>Due to the non — differentiable nature of quantizers (thresholding operations), we need to somehow calculate the gradient on the backward pass. We just set the gradient to 1 where the quantization intervals are used and 0 otherwise (Straight Through Estimator). For training, we do use shadow/virtual weights that are the ones to get updated during back propagation, while quantization is done on-the-fly while the network executes. Then, when training is complete and we have reached convergence, we can just quantize and store the weights.</p><p><a href="https://arxiv.org/abs/2004.05284">You can use the methodology of knowledge distillation to train networks with multiple bit modes jointly.</a></p><h4>Learnable Quantization</h4><p><a href="https://arxiv.org/abs/1807.10029">You can also learn the quantizer parameters stochastically along with the networks weights!</a> Specific constraints/design considerations can be put in place in order to ensure bit-operation compatible output format. Modern CPUs (even without a signal processing or graphic accelerator module) support these operations in a vectorized manner, being able to process &gt; 64 batches of data at once!</p><p>I won’t dive deeper in this matter, due to the fact that I am going to get way more technical than I originally intended to.</p><p>I hope that this blog post was food for thought and provided enough resources for the interested reader to look into later.</p><p>Thanks for reading all the way to the end!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c651f88d6db5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Advanced Tensorflow Data Input Pipelines: Handling Time Series]]></title>
            <link>https://medium.com/data-science/advanced-tensorflow-data-input-pipelines-handling-time-series-e990717d0089?source=rss-a030fa2f6701------2</link>
            <guid isPermaLink="false">https://medium.com/p/e990717d0089</guid>
            <category><![CDATA[cloud-native]]></category>
            <category><![CDATA[time-series-forecasting]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Theodoros Ntakouris]]></dc:creator>
            <pubDate>Fri, 25 Sep 2020 15:22:03 GMT</pubDate>
            <atom:updated>2020-09-25T17:43:19.061Z</atom:updated>
            <content:encoded><![CDATA[<h4>Extracting labels, windowing multivariate series, multiple TF Record file shards and other useful tips for dealing with sequential data</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OOPQQ15DVNGjdXwfrMmT9g.jpeg" /><figcaption><a href="https://pixabay.com/photos/grow-blossom-time-lapse-sequence-73353/">Image by stux from Pixabay</a></figcaption></figure><p>The <a href="https://www.tensorflow.org/guide/data">tf.data.Dataset</a> API is a very efficient pipeline builder. Time Series Tasks can be a bit tricky to implement properly. In this article, we are going to dive deep into common tasks:</p><ul><li>Windowing Labelled Data</li><li>Windowing Unlabelled Data by Looking Ahead</li><li>Sharding TF Record Files Tips for Efficiency and No Data Loss</li></ul><p>Let’s begin!</p><h3>Windowing Labelled Data</h3><p>With the dataset api this is simple to do. Assume the following configuration. input feature is a and label is b .</p><pre>a, b<br>1, 0<br>2, 0<br>3, 1<br>4, 0<br>5, 0<br>6, 1</pre><p>Each row can be described by a tensor shaped (2,) . So our dataset is of size (6, 2) .</p><p>Now, with a window size of 2 , produce training data. This would look like:</p><pre>1, 2 -&gt; 0 (label of row 2)<br>2, 3 -&gt; 1 (label of row 3)<br>4, 5 -&gt; 0 ...</pre><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6b82cbb93d86722ce7cbe842505f1f28/href">https://medium.com/media/6b82cbb93d86722ce7cbe842505f1f28/href</a></iframe><p>The .window() function actually produces a set of datasets. This is why we need to do a .flat_map(batch) operation to end up with a series of tensors we can treat uniformly.</p><h3>Windowing Unlabelled Data by Looking Ahead</h3><p>Sometimes you just want to predict the next tick of a sequence. This can be done without it being labelled. For an input dataset:</p><pre>a<br>1<br>2<br>3<br>4<br>5<br>6</pre><p>The training pairs would be (again, with a window of size 2 ):</p><pre>1, 2 -&gt; 3<br>2, 3 -&gt; 4<br>4, 5 -&gt; 6</pre><p>Let’s try to take the previous approach, with a window of size 3 , to keep the last element as a label.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dd6abea3dd8772d5c9a4ff78887842fd/href">https://medium.com/media/dd6abea3dd8772d5c9a4ff78887842fd/href</a></iframe><blockquote>Data Loss</blockquote><p>The best — practisey way to do this would be to create 2 similar dataset pipelines with a window size of 2 , but with one of 2, lagging infront.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3cd1792708465fa0b285b5f967338c7b/href">https://medium.com/media/3cd1792708465fa0b285b5f967338c7b/href</a></iframe><h3>Sharding TF Record Files Tips for Efficiency and No Data Loss</h3><p>The quintessential way of large scale deep learning input pipelines is to shard your input data to files in the 1–100MBs range, that are able to be read sequentially and in parallel. This means that the storage server cost is reduced drastically by just using rotational drives instead of SSDs, by keeping up the performance.</p><p>A very common case of this practise is to store TF Records in a Hadoop File System or on bucket — based public cloud solutions like Google Cloud Storage.</p><p>Splitting X-Y pair datasets (like images) to multiple files is trivial. With a time series windowing task at hand, it becomes tricky to maintain sequential integrity and avoid data loss.</p><p>Let’s walk through a simple example.</p><pre>File 1<br>----------<br>a, b<br>1, 1<br>1, 2<br>1, 3</pre><pre>File 2<br>----------<br>a, b<br>1, 4<br>1, 5<br>1, 6<br>1, 7</pre><p>For sure, you can get a file glob and load them up and push em straight to training. But you’ve got many issues.</p><p>Firstly, for maximum performance, order is non-deterministic between 2+ shards. This means that your data will be <strong>out of order, </strong>similarly to position — invariant parallel algorithms like map — reduce.</p><p>You will end up windowing together line 1 from file 1 and line 2 from file 2, while another worker windows together line 2 from file 1 and line 1 from file 2.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a34421681e1623fc81820df22bc05048/href">https://medium.com/media/a34421681e1623fc81820df22bc05048/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9afe433c3c925913063f24a30bfb465a/href">https://medium.com/media/9afe433c3c925913063f24a30bfb465a/href</a></iframe><p>The out — of — order part is now fixed. But there’s another problem that needs careful inspection. Missing <strong>cross-file data</strong>. If file 2 is the sequence continuation of file 1, you need to make sure that your look — back window includes a link between those 2 files. Else, you lose data points.</p><p>This means that while tf.data.Dataset is parsing the sharded files, the final dataset that is generated for your model to train, needs to include (let’s demonstrate this for a window of 2):</p><pre>(a)1,2 and (a)1,3 -&gt; (b)1, 4<br>(a)1,3 and (b)1,4 -&gt; (b)1, 5</pre><p>Sure, you can drop those few datapoints. But keep in mind that the bigger your window size is, the more data points you’re going to lose. Depending on the nature and the size of data, make the correct decision.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/90b2429c2275c899de8c2187dff831d9/href">https://medium.com/media/90b2429c2275c899de8c2187dff831d9/href</a></iframe><p>If you enforce a purely sequential read like we demonstrated on the <strong>Windowing Unlabelled Data by Looking Ahead </strong>section above, you’re going to enjoy no benefit from the parallel read speedup.</p><p>The simple and effective solution is to make multiple datasets, and sequentially concatenate them by 2.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e131b39cd7167d67c3322f3e788842bf/href">https://medium.com/media/e131b39cd7167d67c3322f3e788842bf/href</a></iframe><h3>Conclusion</h3><p>You can even use such optimized pipelines to train with massive amounts of time series data on one or more TPUs.</p><p>By implementing the techniques mentioned in this article, you can create performant pipelines for sequential data by avoiding common pitfalls such as data loss and sequential integrity.</p><p>Congratulations! You’ve achieved time series <strong>excellence</strong>.</p><p>Thank you for reading all the way to the end!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e990717d0089" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/advanced-tensorflow-data-input-pipelines-handling-time-series-e990717d0089">Advanced Tensorflow Data Input Pipelines: Handling Time Series</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Tensorflow Best Practises: Named Inputs and Outputs]]></title>
            <link>https://medium.com/data-science/tensorflow-best-practises-named-inputs-and-outputs-ed80c45eebd4?source=rss-a030fa2f6701------2</link>
            <guid isPermaLink="false">https://medium.com/p/ed80c45eebd4</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[tensorflo]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[best-practices]]></category>
            <dc:creator><![CDATA[Theodoros Ntakouris]]></dc:creator>
            <pubDate>Fri, 25 Sep 2020 11:56:19 GMT</pubDate>
            <atom:updated>2020-09-25T13:01:13.740Z</atom:updated>
            <content:encoded><![CDATA[<h4>Quit depending on positional indices and input value ordering. Start relying on named inputs and outputs. Avoiding data wiring errors</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*reQ1p_RujtcjHVcheeCuAQ.jpeg" /><figcaption><a href="https://pixabay.com/photos/fiber-cable-wire-connection-4814456/">Image by Daniel Dino-Slofer from Pixabay</a></figcaption></figure><blockquote><strong>Named inputs and outputs are essentially dictionaries with string keys and tensor values</strong>.</blockquote><h3>Benefits</h3><ol><li>Defence Against Feature Reordering</li><li>Self — Sufficient Model Serving Signatures and Metadata</li><li>Renaming and Absent Feature Protection</li></ol><p>Most machine learning pipelines read data from a structured source ( database, CSV files/ Pandas Dataframes , TF Records), perform feature selection, cleaning, (and possibly) preprocessing, passing a raw multidimensional array (tensor) to a model along with another tensor representing the correct prediction for each input sample.</p><p><em>Reorder or rename input features in production?</em> → <strong>Useless results or the client — side breaks in production</strong></p><p><em>Absent Features? Missing Data? Bad output value interpretation? Mixing up integer indices by mistake? </em>→ <strong>Useless Results or the client — side breaks in production</strong></p><p><em>Want to know what feature columns were used for training in order to provide the same ones for inference?</em> → <strong>You can’t — Misinterpretation Errors</strong></p><p><em>Want to know what value output values represent?</em> → <strong>You can’t — Misinterpretation Errors</strong></p><h3>Don’t drop column names on the model input layers.</h3><p>The tf.data.Dataset already allows you to do that by default, by treating the input as a dictionary.</p><p>Over the years the above problems have got easier to deal with. Here’s a small overview of available solutions, with the Tensorflow 2.x ecosystem.</p><ul><li><a href="https://www.tensorflow.org/tutorials/load_data/tfrecord">TFRecords and tf.Example</a> is hands down the best data format to use with any scale deep learning projects. Every feature is <strong>named by default.</strong></li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3971af97c7b051a8e9846d959df82883/href">https://medium.com/media/3971af97c7b051a8e9846d959df82883/href</a></iframe><ul><li><a href="https://www.tensorflow.org/tfx/tutorials/transform/simple">Tensorflow Transform </a>uses named inputs and produces named outputs, encouraging you to do the same for your model.</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/291bba300781f76c40421d1c7ad65be6/href">https://medium.com/media/291bba300781f76c40421d1c7ad65be6/href</a></iframe><ul><li>Keras supports dictionaries of layers as inputs *and* outputs</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6e025bf1157b9686ed315f49a88c71de/href">https://medium.com/media/6e025bf1157b9686ed315f49a88c71de/href</a></iframe><ul><li><a href="https://datascience.stackexchange.com/questions/71724/what-is-a-tensorspec-tensorflow-2-0">TensorSpec</a> and Serving Signature definitions support named IOs by default.</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e8afdc579337120109ae28ccb81e5d47/href">https://medium.com/media/e8afdc579337120109ae28ccb81e5d47/href</a></iframe><p>By using this serving_raw signature definition, you can call a <a href="https://www.tensorflow.org/tfx/serving/serving_basic">Tensorflow Serving Endpoint directly by a JSON payload</a>, without serialising to tf.Example .</p><p>Check out the metadata signature on TF Serving, with a sample bitcoin prediction mode I am currently working on:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/92138eceb1761b017ff44d7dad22d3b6/href">https://medium.com/media/92138eceb1761b017ff44d7dad22d3b6/href</a></iframe><blockquote>Lastly, if you are using TFX or got a protocol buffer schema for the inputs, you should use that to send over data for inference, as it is <em>much</em> <em>more</em> <em>efficient</em> and the errors appear in the client — side sooner, instead of the server — side. <em>Even on this case, keep using named inputs and outputs for your model.</em></blockquote><p><strong>Thanks for reading all the way to the end!</strong></p><p>Want to also learn how to structure your next machine learning project properly?</p><ul><li>Check out my<strong> </strong><a href="http://Want to create"><strong>Structuring ML Pipeline Projects article</strong></a><strong>.</strong></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ed80c45eebd4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/tensorflow-best-practises-named-inputs-and-outputs-ed80c45eebd4">Tensorflow Best Practises: Named Inputs and Outputs</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Playing chrome’s dino game by physically jumping and crouching]]></title>
            <link>https://medium.com/data-science/playing-chromes-dino-game-by-physically-jumping-and-crouching-a3d319234727?source=rss-a030fa2f6701------2</link>
            <guid isPermaLink="false">https://medium.com/p/a3d319234727</guid>
            <category><![CDATA[creative]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[gaming]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Theodoros Ntakouris]]></dc:creator>
            <pubDate>Wed, 23 Sep 2020 10:26:06 GMT</pubDate>
            <atom:updated>2020-09-23T15:04:28.470Z</atom:updated>
            <content:encoded><![CDATA[<h4>A creative PoseNet application that runs on your browser and tries to predict if you’re jumping, crouching, or staying still</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/435/1*UoyyI91juQgWn2eLBKjSQQ.png" /><figcaption>Screenshot from <a href="https://chromedino.com/">chromedino.com</a></figcaption></figure><p>You all know what this game is about. This is the best service-offline-sorry page in the world. People have made simple bots that time the dino’s jump to beat the game to reinforcement learning agents with CNN state encoders.</p><p>It’s a game and we’re supposed to have fun. Today, I’ll walk you through how to write some JavaScript code to play the game by jumping around in your room.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FgWQd4n0hGFw%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DgWQd4n0hGFw&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FgWQd4n0hGFw%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/08281a65d26a46bfa577c79fd31ab071/href">https://medium.com/media/08281a65d26a46bfa577c79fd31ab071/href</a></iframe><p><em>This thing is hard to play.</em></p><blockquote>You can try the game <a href="https://ntakour.is/posenet-dino-game/">here</a> and view the full source code <a href="https://github.com/ntakouris/posenet-dino-game">here</a>.</blockquote><h3>Overcoming Tech Barriers</h3><p>Setting up a small webpage with basic javascript support to get a webcam feed and a dino game container is trivial for seasoned developers. All you need is latest chrome, a &lt;video&gt; tag, <a href="https://stackoverflow.com/questions/48049285/webcam-video-stream-not-showing-in-html5-video-tag">some JavaScript snippets to load a webcam feed from stackoveflow</a> and <a href="https://github.com/wayou/t-rex-runner/">the ripped t-rex game</a>.</p><p>Moving on to the interesting part. Movement/Action detection.</p><p>Tensorflow Lite has open — sourced a lot of fine — tuned models for web or mobile usage. I decided to use the full (both lower and upper body) pose detector. You can import tensorflow lite for javascript as well as the model I am using here, by adding these 2 lines inside your &lt;head&gt; .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/13638881a45a85ead22578b812bd88b5/href">https://medium.com/media/13638881a45a85ead22578b812bd88b5/href</a></iframe><p>With every dependency downloaded and ready, we can start by handling the camera element’s loadeddata event.</p><p>Once the webcam is ready, load the model and start a pose prediction loop.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/90c951905868fdf2c9d3158dedd53cee/href">https://medium.com/media/90c951905868fdf2c9d3158dedd53cee/href</a></iframe><p>I’ve also included a small recording play/pause functionality by just adding an on — click event on the video feed container.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/14645e87b49d48831c38ba7d9335eaa3/href">https://medium.com/media/14645e87b49d48831c38ba7d9335eaa3/href</a></iframe><h3>Detecting Actions</h3><p>Posenet outputs a list of each predicted bone positions, along with a <em>score</em> value.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*zlLbpslnXIlgp_Xoc_rhUA.gif" /><figcaption><a href="https://www.tensorflow.org/lite/models/pose_estimation/overview">https://www.tensorflow.org/lite/models/pose_estimation/overview</a></figcaption></figure><p>For the sake of simplicity, our algorithm is pretty simple.</p><ol><li>Pick out the left and right hip bones</li><li>Select the hip bone with the greatest confidence (score)</li><li>If the score is not &gt; 0.6 ( better than a random guess ), go to next frame</li><li>Else, perform simple thresholding to detect actions, on the y axis</li></ol><p>Here’s the first part of our algorithm so far:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/00928dee4f1e527a0c0c01247504ab96/href">https://medium.com/media/00928dee4f1e527a0c0c01247504ab96/href</a></iframe><p>Moving on to <strong>thresholding.</strong></p><p>KISS.</p><ul><li>If the hip is positioned on the lower part (bottom 20% ) of the camera feed, treat this as crouching.</li><li>If the hip is in the middle of the feed (between 20% and 70% of the height of the camera feed), treat this as idle.</li><li>Else, treat this as a jump</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/206be7b4db400cadbe2c219986b81da5/href">https://medium.com/media/206be7b4db400cadbe2c219986b81da5/href</a></iframe><blockquote>Of course, this presumes that the player is standing in the proper position in front of the camera. This is pretty easy to do. Just make sure you are far away so that your hips are in the middle of the viewport, with enough room so that you can crouch and jump.</blockquote><p>The last part, implementing toggleAction is pretty easy. We are just going to emulate key pressed events. Key code 32 means spacebar and key code 40 means bottom arrow.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3dc3029003ae4578245c0219a1d9aa6d/href">https://medium.com/media/3dc3029003ae4578245c0219a1d9aa6d/href</a></iframe><blockquote>You can try the game <a href="https://ntakour.is/posenet-dino-game/">here</a> and view the full source code <a href="https://github.com/ntakouris/posenet-dino-game">here</a>.</blockquote><p>That’s it. Thanks for reading!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a3d319234727" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/playing-chromes-dino-game-by-physically-jumping-and-crouching-a3d319234727">Playing chrome’s dino game by physically jumping and crouching</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Structuring ML Pipeline Projects]]></title>
            <link>https://medium.com/data-science/structuring-ml-pipeline-projects-97c16348be4a?source=rss-a030fa2f6701------2</link>
            <guid isPermaLink="false">https://medium.com/p/97c16348be4a</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[cloud-native]]></category>
            <category><![CDATA[tensorflow-extended]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Theodoros Ntakouris]]></dc:creator>
            <pubDate>Sat, 19 Sep 2020 09:44:55 GMT</pubDate>
            <atom:updated>2020-09-19T16:58:32.983Z</atom:updated>
            <content:encoded><![CDATA[<h4>An organised codebase enables you to implement changes faster and make less mistakes, ultimately leading to higher code and model quality. Read more to learn how to structure your ML projects with Tensorflow Extended (TFX), the easy and straightforward way.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6Whp3OSNl8mkWYMcLqsVow.png" /><figcaption><a href="https://pixabay.com/illustrations/lego-building-game-toy-drawing-3388163/">Image by Francis Ray from Pixabay</a></figcaption></figure><h3>Project Structure: Requirements</h3><ul><li>Enable experimentation with multiple pipelines</li><li>Support both alocal execution mode and a deployment execution mode. This ensures the creation of 2 separate running configurations, with the first being used for local development and end-to-end testing and the second one used for running in the cloud.</li><li>Reuse code across pipeline variants if it makes sense to do so</li><li>Provide an easy to useCLI interface for executing pipelines with different configurations and data</li></ul><p>A correct implementation also ensures that tests are easy to incorporate in your workflow.</p><h3>Project Structure: Design Decisions</h3><ul><li>Use Python.</li><li>Use Tensorflow Extended (TFX) as the pipeline framework.</li></ul><p>In this article we will demonstrate how to run a TFX pipeline both locally and on a <a href="https://www.kubeflow.org/docs/pipelines/overview/pipelines-overview/">Kubeflow Pipelines </a>installation with minimum hassle.</p><h3>Side Effects Caused By Design Decisions</h3><ul><li>By using TFX, we are going to use tensorflow . Keep in mind that tensorflow supports more types of models, like <a href="https://www.tensorflow.org/tutorials/estimator/boosted_trees">boosted trees</a>.</li><li>Apache Beam can execute locally, anywhere kubernetes runs and on all public cloud providers. Examples include but are not limited to: GCP Dataflow, Azure Databricks.</li><li>Due to Apache Beam, we need to make sure that the project code is easily packageable by python’ssdist for maximum portability. This is reflected on the top-level module structure of the project. (If you use external libraries be sure to include them by providing an argument to apache beam. <em>Read more about this on </em><a href="https://beam.apache.org/documentation/sdks/python-pipeline-dependencies/"><em>Apache Beam: Managing Python Pipeline Dependencies</em></a><em>).</em></li></ul><blockquote><strong>[Optional]</strong> Before continuing, take a moment to read about the provided <a href="https://www.tensorflow.org/tfx/guide/cli">TFX CLI</a>. Currently, it is <a href="https://github.com/tensorflow/tfx/issues/2164">embarrasingly slow to operate</a> and the directory structure is much more verbose than it needs to be. It also does not include any notes on reproducibility and code reuse.</blockquote><h3>Directory Structure and Intuition Behind It</h3><ul><li>$project-name is the root directory of your project</li><li>$project-name/ml includes machine learning related stuff.</li><li>$project-name/ml/pipelines includes the actual ML pipeline code</li><li>Typically, you may find yourself with multiple ML pipelines to manage, such as $project-name/ml/pipelines/predict-sales and $project-name/ml/pipelines/classify-fraud or similar.</li><li>Here is a simple tree view:</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c263e8459dd679c50fdfc1c22be6a14e/href">https://medium.com/media/c263e8459dd679c50fdfc1c22be6a14e/href</a></iframe><p>$project-name/ml/pipelines includes the following:</p><ul><li>data → small amount of representative training data to run locally for testing and on CI. That’s true if your system does not have a dedicated component to pull data from somewhere. If this is true, make sure to include a sampling query with a small limited number of items.</li><li>util → code that is reused and shared across $pipeline-name s. It is not necessary to include input_fn_utils.py and model_utils.py . Use whatever makes sense here. Here are some examples:</li></ul><p>In my own projects, it made sense to abstract some parts on the utility module, like building named input and output layers for the keras models.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0bb7710044e29630f871b824ce78efd1/href">https://medium.com/media/0bb7710044e29630f871b824ce78efd1/href</a></iframe><p>Building the serving signature metagraph using Tensorflow Transform output.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/401ecc0c5e30e96922c973596570dbaf/href">https://medium.com/media/401ecc0c5e30e96922c973596570dbaf/href</a></iframe><p>Preprocessing features into groups by using keys.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9df461cbfaafe2906ecc1f6e1c25d03c/href">https://medium.com/media/9df461cbfaafe2906ecc1f6e1c25d03c/href</a></iframe><p>And also other common repetitive tasks, like building input pipelines with the Tensorflow dataset api.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7f537be18838eff2af6e755f0f546406/href">https://medium.com/media/7f537be18838eff2af6e755f0f546406/href</a></iframe><ul><li>cli.py → entry point and command line interface for the pipelines. Here are some common things to consider when using TFX.</li></ul><p>By using <a href="https://github.com/abseil/abseil-py">abseil</a> you can declare and access flags globally. <a href="https://abseil.io/docs/python/guides/flags">Each module defines flags that are specific to it.</a> It is a distributed system. This means that the common flags, like --data_dir=... , --hparam_tuning , --pipeline_root , --ml_metadata_url , --use_cache , --train_epochs is some you can define on the actual cli.py file. Other, more specific ones for each pipeline can be defined on submodules.</p><p>This file acts as an entry point for the system. It uses contents in pipeline.py to set up the components of the pipeline as well as provide the user-provided module files ( in the tree example these are constants.py , model.py , training.py ) based on some flag like --pipeline_name=$pipeline-name or some other configuration.</p><p>Finally, with the assembled pipeline, it calls some _runner.py file, by using a --runner= flag.</p><ul><li>pipeline.py → parameterised pipeline component declaration and wiring. This is usually just a function that declares a bunch of TFX components and returns a tfx.orchestration.Pipeline object.</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5363a5f05af66343c344584c90d30922/href">https://medium.com/media/5363a5f05af66343c344584c90d30922/href</a></iframe><ul><li>local_beam_dag_runner.py → configuration to run locally with the portable Beam runner. This can typically be almost configuration — free, just by using the BeamDagRunner .</li><li>kfp_runner.py → configuration to run on Kubeflow Pipelines. This typically includes different data path and pipeline output prefixes and auto-binds an ml-metadata instance.</li></ul><blockquote>Note: you can have more runners, like something that runs on GCP and just configures more provisioning resource like, TPU instances, parallel AI platform hyperparameter search etc.</blockquote><h4>$pipeline-name</h4><p>This is the user-provided code that makes different models, schedules different experiments, etc.</p><p>Due to the util submodule, code under each pipeline should be much leaner. No need to split it in more than 3 files. It’s not prohibiting to split your code throughout more files though.</p><p>From experimentation, I converged to a constants , model and training split.</p><ul><li>constants.py → declarations. Sensible default values for training parameters, hyperparameter keys and declarations, feature keys, feature groups, evaluation configurations and metrics to track. Here is a small example:</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7ba84bc81a77bb4cdd9249c7c6ae49fe/href">https://medium.com/media/7ba84bc81a77bb4cdd9249c7c6ae49fe/href</a></iframe><ul><li>model.py → Model definition. Typically contains a build_keras_model function and uses imports from util and $pipeline-name.constants . Here’s an example from a recent project of mine:</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f18918303174048ff49a6511bb0295da/href">https://medium.com/media/f18918303174048ff49a6511bb0295da/href</a></iframe><ul><li>Lastly, training.py includes all the fuss required to train the model. This is typically: preprocessing definition, hyperparameter search, setting up training data or model — parallel strategies and tensorboard logs and saving the module for production.</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4c9b4ba980b297c64548db1e22ed40ac/href">https://medium.com/media/4c9b4ba980b297c64548db1e22ed40ac/href</a></iframe><p>That’s it. Thank you for reading to the end!</p><p>I hope that you enjoyed reading this article as much as I enjoyed writing it.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=97c16348be4a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/structuring-ml-pipeline-projects-97c16348be4a">Structuring ML Pipeline Projects</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deep Learning End to End Pipelines made easy with Fluent Tensorflow Extended]]></title>
            <link>https://medium.com/data-science/deep-learning-end-to-end-pipelines-made-easy-with-fluent-tensorflow-extended-2a26f5d5817?source=rss-a030fa2f6701------2</link>
            <guid isPermaLink="false">https://medium.com/p/2a26f5d5817</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[tensorflow-extended]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Theodoros Ntakouris]]></dc:creator>
            <pubDate>Fri, 31 Jul 2020 16:22:05 GMT</pubDate>
            <atom:updated>2020-07-31T16:57:04.205Z</atom:updated>
            <content:encoded><![CDATA[<h4>A quick api overview and a self-contained example of fluent-tfx</h4><p>If this production e2e ML pipelines thing seems new to you, <a href="https://www.tensorflow.org/tfx">please read the TFX guide first.</a></p><p>On the other hand, if you’ve used TFX before, or planning to deploy a machine learning model, you’re in the right place.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6vjxiN8YMZS2fjWmp2c5PA.jpeg" /><figcaption>Image by <a href="https://pixabay.com/users/jarmoluk-143740/?utm_source=link-attribution&amp;utm_medium=referral&amp;utm_campaign=image&amp;utm_content=264778">Michal Jarmoluk</a> from <a href="https://pixabay.com/?utm_source=link-attribution&amp;utm_medium=referral&amp;utm_campaign=image&amp;utm_content=264778">Pixabay</a></figcaption></figure><h4>But Tensorflow Extended is already fully capable to construct e2e pipelines by itself, why bother to use another API ?</h4><ul><li>Verbose and long code definitions. Actual preprocessing and training code can be as lengthy as an actual pipeline component definition.</li><li>Lack of sensible defaults. You have to manually specify inputs and outputs to everything. This allows maximum flexibility on one hand, but on the other 99% of cases, most of the IOs can be automatically wired. <em>For example, your preprocessing component is going to probably read your first input component’s input, and pass outputs to training.</em></li><li>Too much boilerplate code. Scaffolding via the TFX CLI produces 15–20 files in 4–5 directories.</li></ul><h4>The benefits of an easier to use, API layer</h4><ul><li>Fluent and compact pipeline definition and runtime configuration. No more scrolling through endless, huge 300+ line functions that construct pipelines</li><li>No scaffolding, easy to set up by using a few lines of code</li><li>Extra helpful utilities to speed up common tasks, such as data input, TFX component construction and wiring</li><li>Sensible defaults and 99% — suitable component IO wiring</li></ul><blockquote><em>Disclaimer: I am the author of </em><a href="https://github.com/ntakouris/fluent-tfx"><em>fluent-tfx</em></a></blockquote><h3>API Overview through an example</h3><p><em>This is essentially </em><a href="https://github.com/ntakouris/fluent-tfx/blob/master/examples/usage_guide/simple_e2e.py"><em>fluent-tfx/examples/usage_guide/simple_e2e.py</em></a><em>, but please, read on.</em></p><p>Scroll to <strong>Pipeline Building</strong> if you already know the basics of TFX.</p><h4>Data In</h4><p>the file data/data.csv is essentially 4 columns: a, b, c, lbl . a and b are floats sampled randomly, c is a binary feature ( 0 or 1) and lbl is the binary label ( values 0 or 1 ). That’s a toy problem, just for demonstration purposes.</p><h4>Model Code</h4><p>The engineer or ‘user’ provides functions for preprocessing, model building, hyperparameter search and saving the model with proper signatures. We’re going to show how you can easily define these functions in another file (say <a href="https://github.com/ntakouris/fluent-tfx/blob/master/examples/usage_guide/model_code.py">model_code.py</a> )</p><p>We’re going to use all the goodies of TFX to do the bulk of the work.</p><p><a href="https://www.tensorflow.org/tfx/transform/get_started"><strong>Tensorflow Transform for preprocessing</strong></a><strong>:</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/be8da34a31c3709e741baad279365a02/href">https://medium.com/media/be8da34a31c3709e741baad279365a02/href</a></iframe><p><a href="https://www.tensorflow.org/guide/data"><strong>Tensorflow Datasets for feeding input to the model in an effective way:</strong></a></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5737bb9ecd8bdcb0b4035f39a655f051/href">https://medium.com/media/5737bb9ecd8bdcb0b4035f39a655f051/href</a></iframe><p><a href="https://keras-team.github.io/keras-tuner/"><strong>KerasTuner</strong></a><strong> and </strong><a href="https://www.tensorflow.org/tfx/guide/tuner"><strong>TFX Tuner — Trainer for hyperparameter search and model building:</strong></a></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c12bb072c48a316861fd0764c5f784de/href">https://medium.com/media/c12bb072c48a316861fd0764c5f784de/href</a></iframe><h4>Pipeline Building</h4><p>Constructing the pipeline is non trivial: provide some evaluation configuration with <a href="https://www.tensorflow.org/tfx/tutorials/model_analysis/tfma_basic">Tensorflow Model Analysis</a> and just use <a href="https://github.com/ntakouris/fluent-tfx/blob/master/examples/usage_guide/simple_e2e.py"><em>fluent-tfx</em></a>:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5744ef5e82c9543c502c0ff6abac0a0d/href">https://medium.com/media/5744ef5e82c9543c502c0ff6abac0a0d/href</a></iframe><h4>Runners</h4><p>There is no extra effort required to run the pipeline on <a href="https://beam.apache.org/documentation/runners/capability-matrix/">different runners that TFX supports (with Apache Beam)</a>, nor extra dependencies required: PipelineDef produces a vanilla TFX pipeline.</p><p>However, if you are using ftfx utilities inside your pipeline functions, <a href="https://beam.apache.org/documentation/sdks/python-pipeline-dependencies/">be sure to include this package in your requirements.txt beam argument</a>.</p><h3>Appendix: Degrees of Freedom and Limitations</h3><p>Custom components are supported to a large extent, but there will still some edge cases that would only work with the verbose plain old TFX api.</p><p>Assumptions are related to component DAG wiring, paths and naming.</p><h4><strong>Paths</strong></h4><ul><li>PipelineDef needs a pipeline_name and an optional bucket path.</li><li>Binary/Temporary/Staging artifacts are stored under {bucket}/{name}/staging</li><li>Default ml metadata sqlite path is set to {bucket}/{name}/metadata.db unless specified otherwise</li><li>bucket defaults to ./bucket</li><li>Pusher’s relative_push_uri will publish the model to {bucket}/{name}/{relative_push_uri}</li></ul><h4><strong>Component IO and Names</strong></h4><ul><li>An input, or an example_gen component provides .tfrecords (probably in gzipped format) to next components</li><li>Fluent TFX follows the TFX naming of default components for everything. When providing custom components, make sure that inputs and outputs are on par with TFX.</li><li>For example, your custom example_gen component should have a .outputs[&#39;examples&#39;] attribute</li><li>When using extra components from input_builders make sure that the names you provide not override defaults, such as standard tfx component names as snake_case and {name}_examples_provider, user_{x}_importer.</li></ul><h4><strong>Component Wiring Defaults</strong></h4><ul><li>If a user provided schema uri is provided, it will be used for data validation, transform, etc. The generated schema component will still generate artifacts if declared</li><li>If a user did not provide a model evaluation step, it will not be wired to the pusher</li><li>The default training input source are transform outputs. The user can specify if he wants the raw tf records instead</li><li>If hyperparameters are specified and tuner is declared, tuner will still test configurations and produce hyperparameter artifacts, but the provided hyperparameters will be used</li></ul><p>Thank’s for reading all the way to the end! If this production ML pipelines thing seems new to you, <a href="https://www.tensorflow.org/tfx">please read the TFX guide.</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2a26f5d5817" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/deep-learning-end-to-end-pipelines-made-easy-with-fluent-tensorflow-extended-2a26f5d5817">Deep Learning End to End Pipelines made easy with Fluent Tensorflow Extended</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>