<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Rostyslav Neskorozhenyi on Medium]]></title>
        <description><![CDATA[Stories by Rostyslav Neskorozhenyi on Medium]]></description>
        <link>https://medium.com/@slanjr?source=rss-8c7f264e81e5------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*G_xyOhzkp0IGoIefDgZL_A.jpeg</url>
            <title>Stories by Rostyslav Neskorozhenyi on Medium</title>
            <link>https://medium.com/@slanjr?source=rss-8c7f264e81e5------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 00:34:23 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@slanjr/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Integrate Distributed Ray Serve Deployment with Kafka]]></title>
            <link>https://medium.com/data-science/integrate-distributed-ray-serve-deployment-with-kafka-181403f4e194?source=rss-8c7f264e81e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/181403f4e194</guid>
            <category><![CDATA[kafka]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[distributed-systems]]></category>
            <dc:creator><![CDATA[Rostyslav Neskorozhenyi]]></dc:creator>
            <pubDate>Sun, 16 Jul 2023 21:59:18 GMT</pubDate>
            <atom:updated>2023-07-29T00:36:57.627Z</atom:updated>
            <content:encoded><![CDATA[<h4>Learn how to simply combine Ray Serve Deployment with asynchronous Kafka Consumer</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vq3BP6ujpC9DcOlcJoky7Q.png" /><figcaption>Image is generated by <a href="https://www.midjourney.com/">Midjourney</a></figcaption></figure><p><a href="https://github.com/ray-project/ray">Ray</a> is a modern open source framework that allows you to create distributed applications in Python with ease. You can create simple training pipelines, do hyperparameter tuning, data processing and model serving.</p><p>Ray allows you to create online inference APIs with <a href="https://docs.ray.io/en/latest/serve/index.html">Ray Serve</a>. You can easily combine several ML models and custom business logic in one application. Ray Serve automatically creates an HTTP interface for your deployments, taking care of fault tolerance and replication.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0G5tRCWFDkyfwHUerhFApA.png" /><figcaption>Ray ecosystem. Source: <a href="https://docs.ray.io/en/latest/ray-air/getting-started.html">https://docs.ray.io/en/latest/ray-air/getting-started.html</a> (<a href="https://github.com/ray-project/ray/blob/releases/2.6.1/LICENSE">APACHE 2.0</a> licence )</figcaption></figure><p>But there is one thing that <strong>Ray Serve</strong> misses for now. Many modern distributed applications communicate through <a href="https://kafka.apache.org/">Kafka</a>, but there is no out-of-the-box way to connect Ray Serve service to Kafka topic.</p><p>But don’t panic. It will not take too much effort to teach Ray Serve to communicate with Kafka. So, let’s begin.</p><p>First of all we will need to prepare our local environment. We will use a docker-compose file with Kafka and <a href="https://github.com/obsidiandynamics/kafdrop">Kafdrop UI</a> docker containers to start and explore our local Kafka instance (so we assume that you have <a href="https://docs.docker.com/engine/install/ubuntu/">Docker</a> and <a href="https://docs.docker.com/compose/install/linux/#install-using-the-repository">Docker Compose</a> installed). Also we will need to install some Python requirements to get the work done:</p><ul><li><a href="https://github.com/ray-project/ray">Ray</a> itself</li><li><a href="https://github.com/aio-libs/aiokafka">aiokafka</a></li></ul><p>All requirements can be downloaded by this <a href="https://github.com/slanj/ray-serve-kafka/blob/main/requirements.txt"><strong>link</strong></a>.</p><p>Now we will create a <a href="https://github.com/slanj/ray-serve-kafka/blob/main/ray-consumer.py"><em>ray-consumer.py</em></a> file with <a href="https://docs.ray.io/en/latest/serve/key-concepts.html#deployment"><strong>Ray Deployment</strong></a> that will be served with Ray Serve. I will not go into details about Ray Serve concepts, as you can read about that in the <a href="https://docs.ray.io/en/latest/serve/getting_started.html">documentation</a>. Basically it takes the usual Python class and converts it to a asynchronous service <a href="https://docs.ray.io/en/latest/serve/key-concepts.html#deployment">Ray Deployment</a> with <strong>@serve.deployment </strong>decorator<strong>:</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/15cb727bc1f0ce0f00374ccb827c6290/href">https://medium.com/media/15cb727bc1f0ce0f00374ccb827c6290/href</a></iframe><p>Let’s explore this code a little. Here we can see a <em>serve.deployment</em> decorator with several parameters. These parameters will be used by Ray Serve to run our deployment. In our case Ray Serve will run one deployment replica on a local Ray cluster (cluster also may exist on <a href="https://docs.ray.io/en/latest/cluster/getting-started.html"><em>AWS or Kubernetes</em></a><em>)</em>. You can start several deployment replicas with <strong>num_replicas</strong> parameter, Ray Serve will split traffic between these replicas.</p><p>Also we specify the amount of resources which each replica will consume with <strong>num_cpus</strong> and <strong>num_gpus </strong>parameters. Such parameters are useful for example when you have a heavy deployment and it needs a specific number of CPU cores to run. In our case we will not need much, so we will use no GPU, and only 0.1 CPU.</p><p>Each time when deployment starts it will get the running event loop (read about asyncio event loops <a href="https://superfastpython.com/asyncio-event-loop/">here</a>), initialize <a href="https://aiokafka.readthedocs.io/en/stable/consumer.html">AIOKafkaConsumer</a>, connect to Kafka(with <em>topics</em>, <em>bootstrap_servers </em>and <em>group_id</em> parameters) and start <em>consume</em> task for that consumer in the current event loop. Also it will set deployment health status to True.</p><p>By default, the Serve controller periodically <a href="https://docs.ray.io/en/latest/serve/production-guide/fault-tolerance.html">health-checks</a> each Serve deployment replica and restarts it on failure. We defined a custom health-check, by adding a <em>check_health</em> method to our <em>RayConsumer</em> deployment class. Healt-check periods are defined by <strong>health_check_period_s</strong> and <strong>health_check_timeout_s</strong> parameters in <em>serve.deployment </em>decorator<em>.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7AE9FO5pduIRPpGMqkEeGg.png" /><figcaption>Ray Serve Architecture. Source <a href="https://docs.ray.io/en/latest/serve/architecture.html">https://docs.ray.io/en/latest/serve/architecture.html</a> (<a href="https://github.com/ray-project/ray/blob/releases/2.6.1/LICENSE">APACHE 2.0</a> license)</figcaption></figure><p>Our consumer will continue to get messages from Kafka and print a toy ML model prediction to the terminal till Ray Cluster is alive. In our example we just use a toy model to simulate ML prediction, but we can include any logic or ML model inference. We can also use <a href="https://docs.ray.io/en/latest/ray-core/objects.html">Ray object store</a> to store ML models and <a href="https://medium.com/ibm-data-ai/how-to-load-pytorch-models-340-times-faster-with-ray-8be751a6944c">speed up load time</a>.</p><p>In a case when a consumer fails due to some random error — our deployment will set health status to False, <strong>check_health</strong> method will return an error and Ray Serve Controller will automatically restart deployment.</p><p>It goes without saying that before you start consuming messages from Kafka, we need to start Kafka and create a topic in which our messages will be. Go to the folder where you have placed <a href="https://github.com/slanj/ray-serve-kafka/blob/main/docker-compose.yaml">docker-compose.yaml</a> and run the following command in terminal to start Kafka and Kafdrop:</p><pre>docker compose up</pre><p>Now we can go to <a href="http://localhost:9000/">http://localhost:9000</a> and create a new Kafka topic. In our case we name it “ray-topic”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W1K4PtGPIzc3x7Rrw6IRCA.png" /><figcaption>Kafdrop UI. Image by Author</figcaption></figure><p>Everything is ready, so we can start our deployment with the command in terminal (you should run this command in a folder that contains <a href="https://github.com/slanj/ray-serve-kafka/blob/main/ray-consumer.py"><em>ray-consumer.py</em></a>):</p><pre>serve run ray-consumer:deployment</pre><p>This command will set up a local Ray Cluster and start Ray Serve Deployment on it. We can check your deployments status in Ray Dashboard, which is usually accessible by this address <a href="http://localhost:8265/">http://localhost:8265</a> .</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zw0Iv70VWvk4FVFYGsUQ-A.png" /><figcaption>Ray Dashboard. Image by Author</figcaption></figure><p>Now we can send some messages to Kafka, so our deployment can consume them. We can do this with a simple script <a href="https://github.com/slanj/ray-serve-kafka/blob/main/aio-producer.py">aio-producer.py</a>:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7992b6784679f92bfe7b3918cd317c67/href">https://medium.com/media/7992b6784679f92bfe7b3918cd317c67/href</a></iframe><p>This script will use <a href="https://aiokafka.readthedocs.io/en/stable/producer.html">AIOKafkaProducer</a> to send a particular message to a particular topic, for example like this:</p><pre>python3 aio-producer.py --message &quot;From Kafka to Ray&quot; --topic &quot;ray-topic&quot;</pre><p>If everything is ok, our Ray Consumer Deployment will get all those messages from Kafka and just print them to the terminal.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PuKJiAyIPpsq9yKBGYwzIA.png" /><figcaption>Ray Deployment output. Image by Author</figcaption></figure><p>Basically, that’s all. Now you have a working fault tolerant Ray Serve Deployment that consumes messages from Kafka. Of course it is basic, but you can easily add custom business logic, <a href="https://docs.ray.io/en/latest/serve/model_composition.html">compose it with other deployments</a> or even combine it with <a href="https://docs.ray.io/en/latest/workflows/index.html">Ray Workflows</a>. You can add additional logic into <em>check_health </em>method so your deployment will periodically check additional services if needed.</p><p>You can add replicas to your deployment with serve.deployment parameter <strong>num_replicas </strong>or even <a href="https://docs.ray.io/en/latest/serve/scaling-and-resource-allocation.html">set up autoscaling</a> with <strong>min_replicas </strong>and <strong>max_replicas </strong>parameters, so Ray will automatically add replicas to your deployment depending on traffic and load.</p><p>Ray provides you all the tools to build scalable and fault tolerant Python applications, and now knowing how to combine it with Kafka you are able to build some new really powerful distributed apps.</p><p>All the code mentioned in this article can be found by <a href="https://github.com/slanj/ray-serve-kafka/">this link</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=181403f4e194" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/integrate-distributed-ray-serve-deployment-with-kafka-181403f4e194">Integrate Distributed Ray Serve Deployment with Kafka</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to get high score using MMBT and CLIP in Hateful Memes Competition]]></title>
            <link>https://medium.com/data-science/how-to-get-high-score-using-mmbt-and-clip-in-hateful-memes-competition-90bfa65cb117?source=rss-8c7f264e81e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/90bfa65cb117</guid>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[hugging-face]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Rostyslav Neskorozhenyi]]></dc:creator>
            <pubDate>Mon, 27 Sep 2021 13:16:57 GMT</pubDate>
            <atom:updated>2021-09-28T21:46:38.463Z</atom:updated>
            <content:encoded><![CDATA[<h4>Use CLIP as a feature encoder for Multimodal Bitransformer and make MMBT really work with Huggingface Transformers to get surprisingly high accuracy</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uN62waNvz7jjOdR6Ptr8MQ.png" /><figcaption>Image by <a href="https://pixabay.com/ru/illustrations/%d0%bf%d0%be%d0%bb%d0%b8-%d0%bd%d0%b8%d0%b7%d0%ba%d0%b8%d0%b9-%d0%b6%d0%b8%d0%b2%d0%be%d1%82%d0%bd%d0%be%d0%b5-%d0%b2%d0%b5%d0%ba%d1%82%d0%be%d1%80-3295856/">Manuchi</a> on <a href="https://pixabay.com/">Pixabay</a></figcaption></figure><p>The additional stage of <a href="https://www.drivendata.org/competitions/64/hateful-memes/">Hateful Memes Competition</a> from Facebook ended a few months ago. My team was lucky enough to take part in this competition and even get pretty good results (we took tenth place). How we did it and what methods we used — I’ll tell you in this article.</p><h3>Problem description</h3><p>At first glance, the problem that had to be solved in the competition is quite simple — to determine whether a meme is hateful or not using text and image data from it. In reality, the problem is complicated by the many ambiguities inherent in our speech, as well as by the presence of sarcasm and irony, with the definition of which neural networks have problems. You can read more about the competition and the tasks it posed in the corresponding <a href="https://arxiv.org/pdf/2005.04790v2.pdf">paper</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*7xFnnnz_rqN7uybg4IxE4w.png" /><figcaption>Images from <a href="https://unsplash.com/">Unsplash</a></figcaption></figure><h3>Data Overview</h3><p>During the competition, a downloadable zip file was provided. Now competition data can be found at this <a href="https://hatefulmemeschallenge.com/">link</a>.</p><p>The zip file includes a folder with images and several json files with images annotations.</p><p><strong>img/</strong> folder contains all the images of the challenge dataset including train, dev and test split. The images are named &lt;id&gt;.png, where &lt;id&gt; is a unique 5 digit number.</p><p><strong>train.jsonl, dev_seen.jsonl, dev_unseen.jsonl</strong> — json files where each line has a dictionary of key-value pairs of data about the images. The dictionary includes</p><ul><li><strong>id</strong> The unique identifier between the img directory and the .jsonl files, e.g., “id”: 13894.</li><li><strong>img</strong> The actual meme filename, e.g., “img”: img/13894.png, note that the filename contains the img directory described above, and that the filename stem is the id.</li><li><strong>text</strong> The raw text string embedded in the meme image, e.g., img/13894.png has “text”: “putting bows on your pet”</li><li><strong>label</strong> where 1 -&gt; “hateful” and 0 -&gt; “non-hateful”</li></ul><p>For example:</p><p>{“id”:23058,”img”:”img/23058.png”,”label”:0,”text”:”don’t be afraid to love again everyone is not like your ex”}</p><p><strong>test_seen.jsnol</strong> includes mentioned keys, except <strong>label</strong>.</p><h3>Performance metric</h3><p>Model performance and leaderboard rankings were determined using the <a href="https://medium.com/greyatom/lets-learn-about-auc-roc-curve-4a94b4d88152"><strong>AUC ROC</strong></a> or the Area Under the Curve of the Receiver Operating Characteristic. The metric measures how well your binary classifier discriminates between the classes as its decision threshold is varied.</p><p>Another metric was the accuracy of predictions, given by the ratio of correct predictions to the total number of predictions made.</p><h3>Our approach</h3><p>There are many models and frameworks for working with multimodal data, the one that stands out the most is <a href="https://mmf.sh/">MMF</a> from Facebook. <a href="https://mmf.sh/">MMF</a> provides a simple interface for accessing many powerful multimodal models. But we, as big fans of <a href="https://github.com/huggingface/transformers">Huggingface Transformers</a>, decided not to go the easy way. We decided to find out what multimodal models are available in <a href="https://github.com/huggingface/transformers">Transformers</a> and how to get the most out of them. It turned out that only one model of this kind is currently available in Transformers — <a href="https://huggingface.co/transformers/summary.html#multimodal-models"><strong>Multimodal Bitransformer (MMBT)</strong></a>. Making this model work was not so easy, but this made the task only more interesting.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/890/1*wVs2yKRewYHr-3f5CwVL9A.png" /><figcaption>MMBT architecture, from <a href="https://arxiv.org/abs/1909.02950">Supervised Multimodal Bitransformers for Classifying Images and Text paper</a></figcaption></figure><p>MMBT fuses information from text and image encoders. <a href="https://huggingface.co/transformers/model_doc/bert.html">BERT</a> is used as text encoder and <a href="https://pytorch.org/hub/pytorch_vision_resnet/">ResNet</a> as image encoder. We took advantage of MMBT architecture flexibility and replaced ResNet with <a href="https://github.com/openai/CLIP">CLIP</a> for image encoding. <strong>CLIP</strong> pre-trains an image encoder and a text encoder to predict which images were paired with which texts in a dataset. Our assumption was that features from CLIP are more versatile and better suited for a multimodal domain.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*HCqEtbXiWze0t9Sd.png" /><figcaption>Summary of CLIP model’s approach, from <a href="https://arxiv.org/abs/2103.00020">Learning Transferable Visual Models From Natural Language Supervision paper</a></figcaption></figure><p>For text encoding we used <a href="https://huggingface.co/Hate-speech-CNERG/bert-base-uncased-hatexplain">bert-base-uncased-hatexplain</a> model which is available in Huggingface Hub. This model was created for hatespeech detection in English, so in our case features from it are better than from <a href="https://huggingface.co/bert-base-uncased">bert-base-uncased</a> that was used in MMBT initially.</p><p>Final MMBT model was finetuned on a train dataset and validated on dev_seen dataset.</p><p>We also augmented texts in train dataset using controlled <a href="https://huggingface.co/transformers/model_doc/gpt2.html">GPT-2</a> and <a href="https://github.com/dsfsi/textaugment#eda-easy-data-augmentation-techniques-for-boosting-performance-on-text-classification-tasks">Easy Data Augmentation</a> method. This increased the accuracy of our model by an additional few percent. Augmentation is beyond the scope of this article, and I will probably write about it separately if you find this article and our approach interesting.</p><h3>Implementation</h3><p>There will be a lot of code in this section of the article, but I will try to explain all the important parts in detail for a better understanding.</p><p>First, let’s import the required libraries. We will need</p><ul><li>Transformers version &gt;=4.8.2</li><li>Pytorch version 1.8.1</li><li>torchvision 0.9.1</li><li>scikit-learn 0.23.2</li><li>Pillow &gt;=8.2.0</li><li>tqdm &gt;= 4.60.0</li><li>matplotlib &gt;= 3.3.4</li><li>numpy &gt;=1.19.5</li><li><a href="https://github.com/openai/CLIP">CLIP</a> (that can be installed from repository).</li></ul><p>CLIP is now <a href="https://huggingface.co/transformers/model_doc/clip.html">accessible</a> directly in Huggingface Transformers, but at the time of the implementation of our approach it was not there yet. To get the most out of our model, we also used the <a href="https://github.com/facebookresearch/madgrad">MADGRAD</a> optimizer.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ee50233b1e00be19bf27c360b4396ba3/href">https://medium.com/media/ee50233b1e00be19bf27c360b4396ba3/href</a></iframe><p>Create a variable with available device, which will do all needed computations. We will need a GPU, so our device is CUDA.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f097c5994afeaab376a2a693127afe4d/href">https://medium.com/media/f097c5994afeaab376a2a693127afe4d/href</a></iframe><p>Load CLIP model and needed preprocessing. Initialize needed variables.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/76630f35eb9525e473eded25d8a63a59/href">https://medium.com/media/76630f35eb9525e473eded25d8a63a59/href</a></iframe><p>Create a function that will prepare an image for CLIP encoder in a special manner. This function will split image into three tiles (by height or width, depending on the aspect ratio of the image). Finally we will get four vectors after encoding (one vector for each tile and one vector for whole image that was padded to square). With this approach we will get more information from image, since we will create embeddings that describe not only the image as a whole, but also individual parts of the image</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6ae996b3bd98980bf3349ea6bdbb13ce/href">https://medium.com/media/6ae996b3bd98980bf3349ea6bdbb13ce/href</a></iframe><p>Define a function, that will get image features from CLIP.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fb4f2fa84108b21c1b5908df6955e489/href">https://medium.com/media/fb4f2fa84108b21c1b5908df6955e489/href</a></iframe><p>Create JsonlDataset class that will load texts and preprocessed images. Create<strong> collate_fn </strong>that will group data from dataset in a format needed for our pytorch model.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ce7a57c1c5d03a8177719c98ee021f85/href">https://medium.com/media/ce7a57c1c5d03a8177719c98ee021f85/href</a></iframe><p>Define <strong>load_examples</strong> function that will load data described in json dataset into JsonlDataset object. Create functions to load and save model weights.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1ca964e74102c33d7cedcb392bb910c3/href">https://medium.com/media/1ca964e74102c33d7cedcb392bb910c3/href</a></iframe><p>Needed functions and classes are created, so we can load our models.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0c07539096e8c975894407c13bfd7e22/href">https://medium.com/media/0c07539096e8c975894407c13bfd7e22/href</a></iframe><p>Load train and evaluation datasets and create dataloaders for these datasets.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/efc1afb64277bf1ad7b9570b46ed5c9f/href">https://medium.com/media/efc1afb64277bf1ad7b9570b46ed5c9f/href</a></iframe><p>Define model training parameters, optimizer and loss.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/72c1d02d30c267e4a28c304dd62e4e20/href">https://medium.com/media/72c1d02d30c267e4a28c304dd62e4e20/href</a></iframe><p>Define evaluation function that will take evaluation dataloader and calculate prediction AUC, <a href="https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html">F1 score</a> and accuracy.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c0ae37ae33541f306d26c429340edb00/href">https://medium.com/media/c0ae37ae33541f306d26c429340edb00/href</a></iframe><p>Finally we can train our model. We specify minimum needed AUC value in <strong>best_valid_auc</strong> variable, so if the model achieves higher AUC on validation data than it was specified, we will save that model.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/276393c177bfbfe325e3124f2112fbbb/href">https://medium.com/media/276393c177bfbfe325e3124f2112fbbb/href</a></iframe><p>After training is complete, we can visualize results:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7eb4a4ea7dc300fc5f5ca237f1ef7174/href">https://medium.com/media/7eb4a4ea7dc300fc5f5ca237f1ef7174/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/412/1*hp7swfkYCWPFzzcpFlXuCA.png" /><figcaption>Image is created by Rostyslav Neskorozhenyi with <a href="https://matplotlib.org/">Matplotib</a></figcaption></figure><h3>Make predictions for submission</h3><p>Now we can make a prediction for test data. Firstly we will create needed classes and functions for data loading and processing by analogy with the training stage.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cf51e7221c7e64dda92b3c93de758c0c/href">https://medium.com/media/cf51e7221c7e64dda92b3c93de758c0c/href</a></iframe><p>Finally we make a prediction and save it in a specified format.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6b0a1d29adb12861d1e2eefb87551376/href">https://medium.com/media/6b0a1d29adb12861d1e2eefb87551376/href</a></iframe><p>That’s it. In case the competition reopens, this prediction file can be sent there immediately.</p><h3>Conclusion</h3><p>In this article I tried to describe in detail the concept and implementation of the approach that we used in Hateful Memes Competition from Facebook. The tasks that the competition set for us turned out to be extremely interesting and we had a lot of fun developing our approach to solve these tasks. I hope you enjoyed reading this article too.</p><p>I also want to mention that to obtain maximum AUC, we combined the prediction of several variants of this model trained with different loss parameters and different augmentation options. But that’s a topic for an entirely different article.</p><p>All the code described in the article is available as a jupyter notebook by this <a href="https://github.com/slanj/ml-notebooks/blob/main/multimodal-article.ipynb">link</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=90bfa65cb117" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/how-to-get-high-score-using-mmbt-and-clip-in-hateful-memes-competition-90bfa65cb117">How to get high score using MMBT and CLIP in Hateful Memes Competition</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Tile (Slice) YOLO Dataset for Small Objects Detection]]></title>
            <link>https://medium.com/data-science/tile-slice-yolo-dataset-for-small-objects-detection-a75bf26f7fa2?source=rss-8c7f264e81e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/a75bf26f7fa2</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[dataset]]></category>
            <category><![CDATA[yolo]]></category>
            <category><![CDATA[image-dataset]]></category>
            <dc:creator><![CDATA[Rostyslav Neskorozhenyi]]></dc:creator>
            <pubDate>Tue, 09 Mar 2021 10:38:26 GMT</pubDate>
            <atom:updated>2021-03-09T13:12:49.532Z</atom:updated>
            <content:encoded><![CDATA[<h4>Implement a simple script that can cut images and corresponding labels into tiles of a specified size</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mGtMPXhguy4CTInCBtsq_w.jpeg" /><figcaption>Photo by <a href="https://pixabay.com/ru/users/mabelamber-1377835/">MabelAmber</a> on <a href="https://pixabay.com/">Pixabay</a></figcaption></figure><p>In one of my projects I was faced with the task of detecting very small cracks in very high resolution images. I was provided with an already labeled dataset (in YOLO format), and the simplest solution would be to train the object detection model on it as is. But such an approach would require a lot of computing power and a lot of time. The alternative option to reduce the images (say to a size of 416x416) resulted in a serious loss of detail and accuracy (some cracks just disappeared).</p><p>Finally, I came up with a simple solution that allowed me to save computing resources and preserve accuracy. I sliced each image (and its corresponding labels) into small tiles (the same 416x416) and trained the model on the resulting tiles. The resulting detection on a high-dimensional image was collected from several detections on tiles from which this image was composed.</p><p>To cut the dataset, I wrote a small script, which I want to share with you. In this article I will briefly describe the algorithm of the script. You can get the script itself from the <a href="https://github.com/slanj/yolo-tiling">link</a> on github. So let’s get started.</p><h3>Load needed libraries</h3><p>We will need <strong>pandas</strong> to load and save the dataset, <strong>numpy</strong> for working with arrays, <strong>PIL</strong> for working with images, <strong>shapely</strong> for labels slicing, <strong>glob</strong> for listing images in a folder, and <strong>matplotlib</strong> for visualization.</p><pre>import pandas as pd<br>import numpy as np<br>from PIL import Image<br>from shapely.geometry import Polygon, Point<br>from matplotlib import pyplot as plt<br>import glob</pre><h3>Dataset description</h3><p>We assume that we have a dataset in YOLO format, which consists of a folder with images and text files with labels corresponding to these images. For example, if I have an image named im1.JPG, the corresponding labels will be stored in the im1.txt file. The file with labels follows this syntax, for each bounding box the file will contain a line in the format: &lt;object-class&gt; &lt;x_center&gt; &lt;y_center&gt; &lt;width&gt; &lt;height&gt;. We also need a <strong>classes.names</strong> file for human-readable class names, as well as <strong>train.txt</strong> and <strong>test.txt</strong> files to split the dataset into train/test parts. In my case, I have only one class - &quot;crack&quot;. We will implement the train/test split further.</p><p>The structure of our dataset at this stage should look like this:</p><pre>yolosample/<br>    classes.names<br>    ts/<br>        im2.txt<br>        im1.txt<br>        im1.JPG<br>        im2.JPG</pre><p>I recommend reading more about the dataset format, as well as YOLOv4 training on your custom dataset in this <a href="https://towardsdatascience.com/yolov4-in-google-colab-train-your-custom-dataset-traffic-signs-with-ease-3243ca91c81d">excellent article</a>.</p><p>Let’s visualize labels on one of the images. First, let’s open the image and the corresponding labels.</p><pre>imname = &#39;im1&#39;<br>im = Image.open(f&#39;yolosample/ts/{imname}.JPG&#39;)<br>df = pd.read_csv(f&#39;yolosample/ts/{imname}.txt&#39;, sep=&#39; &#39;, names=[&#39;class&#39;, &#39;x1&#39;, &#39;y1&#39;, &#39;w&#39;, &#39;h&#39;])</pre><pre>imr = np.array(im, dtype=np.uint8)</pre><p>Rescale the coordinates from labels according to the dimensions of the image, since they are now normalized from 0 to 1.</p><pre>df_scaled = df.iloc[:, 1:]<br>df_scaled[[&#39;x1&#39;, &#39;w&#39;]] = df_scaled[[&#39;x1&#39;, &#39;w&#39;]] * imr.shape[1]<br>df_scaled[[&#39;y1&#39;, &#39;h&#39;]] = df_scaled[[&#39;y1&#39;, &#39;h&#39;]] * imr.shape[0]</pre><p>We can now display labels on the image.</p><pre>import matplotlib.patches as patches</pre><pre># Create figure and axes<br>fig,ax = plt.subplots(1, figsize=(10,10))</pre><pre># Display the image<br>ax.imshow(imr)</pre><pre>for box in df_scaled.values:<br>    # Create a Rectangle patch<br>    rect = patches.Rectangle((box[0]-(box[2]/2),box[1]-(box[3]/2)),box[2],box[3],linewidth=2,edgecolor=&#39;g&#39;,facecolor=&#39;none&#39;)</pre><pre># Add the patch to the axes<br>    ax.add_patch(rect)<br>    <br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/608/1*VHJsneE7CCp84GIfrHtjSQ.png" /><figcaption>Image is created by Rostyslav Neskorozhenyi with <a href="https://matplotlib.org/">matplotlib</a></figcaption></figure><p>As we can see, the objects are very small and difficult to distinguish. This is where the slicing comes in handy.</p><h3>Tiling script implementation</h3><p>In this script, we slice the image into tiles of <strong>slice_sizeXslice_size</strong> and save these tiles in the <strong>newpath folder</strong>. We use <strong>shapely</strong> to find the bounding boxes that intersect with each specific tile and store the intersection in the labels file for the corresponding tile. We will also save the tiles with no bounding boxes in the <strong>falsepath</strong> folder.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/097dedd00c5eeaac9c7f14b36fec7d1c/href">https://medium.com/media/097dedd00c5eeaac9c7f14b36fec7d1c/href</a></iframe><h3>Visualize obtained dataset</h3><p>Now we can see what we got.</p><pre>imname = &#39;im1_2_10&#39;<br>im = Image.open(f&#39;sampletiled/ts/{imname}.JPG&#39;)<br>imr = np.array(im, dtype=np.uint8)<br>df = pd.read_csv(f&#39;sampletiled/ts/{imname}.txt&#39;, sep=&#39; &#39;, names=[&#39;class&#39;, &#39;x1&#39;, &#39;y1&#39;, &#39;w&#39;, &#39;h&#39;])</pre><pre># rescale coordinates for visualization<br>df_scaled = df.iloc[:, 1:]<br>df_scaled[[&#39;x1&#39;, &#39;w&#39;]] = df_scaled[[&#39;x1&#39;, &#39;w&#39;]] * imr.shape[1]<br>df_scaled[[&#39;y1&#39;, &#39;h&#39;]] = df_scaled[[&#39;y1&#39;, &#39;h&#39;]] * imr.shape[0]</pre><pre># create figure and axes<br>fig,ax = plt.subplots(1, figsize=(10,10))</pre><pre># display the image<br>ax.imshow(imr)</pre><pre>for box in df_scaled.values:<br>    # create a Rectangle patch<br>    rect = patches.Rectangle((box[0]-(box[2]/2),box[1]-(box[3]/2)),box[2],box[3],linewidth=3,edgecolor=&#39;g&#39;,facecolor=&#39;none&#39;)</pre><pre># add the patch to the axes<br>    ax.add_patch(rect)<br>    <br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/587/1*OOx0g-_sCwsKVcsApXFKyw.png" /><figcaption>Image is created by Rostyslav Neskorozhenyi with <a href="https://matplotlib.org/">matplotlib</a></figcaption></figure><h3>Split dataset to train and test subsets</h3><p>Everything looks good and now we just need to split the dataset into train/test parts.</p><pre>import shutil<br>import os<br>import random</pre><pre># get image names and set train/test split ratio<br>ratio = 0.8<br>imnames = glob.glob(&#39;sampletiled/ts/*.JPG&#39;)<br>names = [name.split(&#39;/&#39;)[2] for name in imnames]</pre><pre># split dataset for train and test</pre><pre>train = []<br>test = []<br>for name in names:<br>    if random.random() &gt; ratio:<br>        test.append(f&#39;sampletiled/ts/{name}&#39;)<br>    else:<br>        train.append(f&#39;sampletiled/ts/{name}&#39;)<br>print(&#39;train:&#39;, len(train))<br>print(&#39;test:&#39;, len(test))</pre><p>Save train.txt, test.txt and classes.names for a new dataset. In my case I have only one class. You can specify any needed classes.</p><pre># save train part<br>with open(&#39;sampletiled/train.txt&#39;, &#39;w&#39;) as f:<br>    for item in train:<br>        f.write(&quot;%s\n&quot; % item)</pre><pre># save test part<br>with open(&#39;sampletiled/test.txt&#39;, &#39;w&#39;) as f:<br>    for item in test:<br>        f.write(&quot;%s\n&quot; % item)</pre><pre># save classes names<br>classes = [&#39;crack&#39;]<br>with open(&#39;sampletiled/classes.names&#39;, &#39;w&#39;) as f:<br>    for item in classes:<br>        f.write(&quot;%s\n&quot; % item)</pre><h3>Conclusion</h3><p>Congratulations! We have implemented a tiling script and with its help generated a new dataset on which our model can learn to better detect small objects. As I already mentioned, you can find the final script on github by the <a href="https://github.com/slanj/yolo-tiling">link</a>. Yo can also find this article packed in Google Colab by this <a href="https://colab.research.google.com/drive/1qcPFf7lazas4f0hKM53kgta5Hs5V9Q78">link</a>. If you liked the article and the script was useful for you, then do not forget to clap for this post, I will appreciate it.</p><p>Good luck with Computer Vision!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a75bf26f7fa2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/tile-slice-yolo-dataset-for-small-objects-detection-a75bf26f7fa2">Tile (Slice) YOLO Dataset for Small Objects Detection</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build a bidirectional text generator with XLNet]]></title>
            <link>https://medium.com/data-science/build-a-bidirectional-text-generator-with-xlnet-49d9d37b48a9?source=rss-8c7f264e81e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/49d9d37b48a9</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[nlp]]></category>
            <dc:creator><![CDATA[Rostyslav Neskorozhenyi]]></dc:creator>
            <pubDate>Tue, 18 Aug 2020 15:16:47 GMT</pubDate>
            <atom:updated>2020-08-18T17:38:54.821Z</atom:updated>
            <content:encoded><![CDATA[<h4>Generate text in a new way with Permutation Language Modeling</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2Yn6KqQRtTkn-hymJ6coLw.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@bdchu614?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Brendan Church</a> on <a href="https://unsplash.com/?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>Current <a href="https://arxiv.org/abs/1706.03762">Transformers</a> based models, like GPT-2 or even GPT-3 show incredible achievements in the task of <a href="https://huggingface.co/blog/how-to-generate">text-generation</a> (prediction of the next probable word based on the previous sequence of words). These models can create long, creative and cohesive texts, but usually they can generate text only in one direction, from left to right. I was wondering if there is a way to generate text in both directions and having some start phrase (for example “text generation is cool”) to see what story will unfold around it. <a href="https://huggingface.co/transformers/model_doc/xlnet.html">XLNet</a> was the solution: due to its using of all permutations of the input sequence factorization order this model can help to generate text in any direction.</p><p>In this article we will not study in detail the internal principles of XLNet (excellent brief explanation you can find <a href="https://towardsdatascience.com/xlnet-a-clever-language-modeling-solution-ab41e87798b0">here</a>). Instead, we’ll start experimenting right away: we will practice a little bit in masked word prediction with XLNet, try to implement top-K bidirectional generation, and then implement a more efficient approach that combines beam search and top-K sampling.</p><p>At the end of the article we will get a generator capable of creating such text based on the start phrase (which is highlighted in bold):</p><blockquote><em>Following up on my initial thoughts: </em><strong><em>text generation is cool</em></strong><em>! It works great for creating blog header, title etc. You will need Word 2013</em></blockquote><h3>Install needed modules</h3><p>Let’s begin. We will conduct all our experiments in Google Collab Notebook (with GPU environment), which is available by this <a href="https://colab.research.google.com/drive/1RhHiKTp0os2_q5z6pKS6vQUz0SM1EXrM">link</a>, so the only module we will need to install is the excellent <a href="https://huggingface.co/transformers/">Transformers</a> library. This library provides a simple interface to XLNet, as well as to many other transformers based models.</p><pre>!pip install transformers</pre><h3>Example of masked words prediction with XLNet</h3><p>One of the advantages of XLNet is that this model can perfectly cope with the prediction of several related masked words while taking into account the previous context. For example, I will mention in the text that I gave you three apples, and then ask the model to tell me who now owns some apples by feeding the model a sentence with masked words: “&lt;mask&gt; have &lt;mask&gt; apples in hands”. As a result, we will see that the model perfectly understands who has apples and how many.</p><p>Before we can start communicating with the model, we need to load it, as well as load a tokenizer that processes the incoming text into a digital form understandable for the model. In the basic form, tokenization is splitting of the text into words or subwords, which then are converted to ids. Each model requires text to be tokenized in a specific way. XLNet uses SentencePiece method. You can read more about the tokenization process at the <a href="https://huggingface.co/transformers/tokenizer_summary.html">link</a>.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a4ad6ff5aee89788f9604acad8b0dfc8/href">https://medium.com/media/a4ad6ff5aee89788f9604acad8b0dfc8/href</a></iframe><p>Also we need to add a padding text to help XLNet with short texts as was <a href="https://medium.com/@amanrusia/xlnet-speaks-comparison-to-gpt-2-ea1a4e9ba39e">proposed</a> by Aman Rusia.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a2862e9248d762b98a452bba6fccd925/href">https://medium.com/media/a2862e9248d762b98a452bba6fccd925/href</a></iframe><p>Predict top 5 words for each &lt;mask&gt; token. To make a prediction we need to feed the model with tokenized text, masked words indexes and permutation masks. Permutation masks are needed to disable input tokens to attend to masked tokens. You can read more about model parameters <a href="https://huggingface.co/transformers/model_doc/xlnet.html#xlnetlmheadmodel">here</a>.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0d0e2bbb447e84a389ec5456d56a9ab4/href">https://medium.com/media/0d0e2bbb447e84a389ec5456d56a9ab4/href</a></iframe><p>Output:</p><pre>predicted word: &lt;mask&gt; 0<br>word and logits You -9.070054054260254<br>word and logits I -10.822368621826172<br>word and logits We -12.820359230041504<br>word and logits Now -14.133552551269531<br>word and logits They -14.863320350646973<br>predicted word: &lt;mask&gt; 1<br>word and logits three -23.045528411865234<br>word and logits the -24.3369083404541<br>word and logits these -25.59902000427246<br>word and logits two -25.809444427490234<br>word and logits your -25.947147369384766</pre><h3>Top-k bi-directional generation</h3><p>Now when we know how to predict masked words with XLNet it’s time to create a top-k bidirectional text generator. Its work principles are simple. We will create a loop and at each iteration the model will predict top-k tokens for a masked word on the right or on the left side of start phrase. After that we add random token from topK to the start phrase and repeat iteration for n times.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3809c21c06ae7c9321ead9784105d1b8/href">https://medium.com/media/3809c21c06ae7c9321ead9784105d1b8/href</a></iframe><p>Output:</p><pre>• 1 1 User reviews and says:? The text generation is cool for me to see and the font size seems right</pre><p>Not too impressive. There is a lot of repetitions and whole text looks meaningless. But we will find a better solution.</p><h3>Top-k-beam bi-directional text generation</h3><p>As we can see, it is still quite difficult for the model to generate text right-to-left. We often get a word that does not fit into the context well, which leads to an even less suitable next word. As a result, the generated text becomes incoherent.</p><p>We can increase the chances of finding connected word sequences by generating words not by one on each side of the starting phrase, but by creating a certain number of beams of word sequences and choosing one of the most probable beams of a certain length.</p><p>Thus, we get some kind of combination of top-k sampling and beam search. The principle of the resulting method is shown in the diagram.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F8pqfzFBsyZwPxWIlhyOjQ.jpeg" /><figcaption>Image was created by Rostyslav Neskorozhenyi with <a href="https://draw.io/">draw.io</a></figcaption></figure><p>The bidirectional generation process consists of n iterations. I split each iteration into four steps for better understanding:</p><ul><li>In the first step, we get a start phrase and generate right-to-left on its left side a certain number of beams of a certain length (at each stage of beam search, we select next token candidates with top-K sampling).</li><li>In the second step, we take a random beam from the top-K most probable beams and add it to the start phrase.</li><li>The resulting new phrase serves as a start phrase for the third step, in which we generate a certain number of beams on the right side of the new start phrase.</li><li>In the fourth step, we take a random beam from the top-k beams obtained in the third step and add that beam to the new starting phrase. The resulting phrase serves as the starting point for the next iteration.</li></ul><p>I hope that the description was clear enough and the diagram will help you figure it out. The main thing is that, based on my experiments, this method in most cases allows you to generate quite coherent text bidirectionally.</p><p>Let’s implement the method in code. Firstly we will create a function that will take tokenized start sentence, a sequence of token candidates with their probabilities and generate next n probable sequences of token candidates on the right or on the left side. We will use this function iteratively, so generated token sequences from previous iteration will serve as input on the next iteration.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d4def575d250105b0eb46f5f15150157/href">https://medium.com/media/d4def575d250105b0eb46f5f15150157/href</a></iframe><p>Now we will create <strong>beam_gen</strong> function that will generate a list of token beams of given length (depth) using token candidates proposed by <strong>candidates_gen</strong>.</p><p><strong>beam_gen</strong> function will return final beams list sorted by probability.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/836fe93c22f76425d6b2b6e2a7b24571/href">https://medium.com/media/836fe93c22f76425d6b2b6e2a7b24571/href</a></iframe><p>Let’s gather all parts together in a <strong>bi_gen</strong> function. <strong>bi_gen</strong> will be able to generate text left-to-right (parameter <strong>direction</strong>=’right’), right-to-left (parameter <strong>direction</strong>=’left’), or in both directions (parameter <strong>direction</strong>=’both’)</p><p>If <strong>both</strong> directions are selected, generator will work in the following way: generate <strong>n_tokens</strong> on the left side, after that — n tokens in the right side, then again n tokens on the left side and so on. It will repeat number of times, that is saved in <strong>iterations</strong> parameter.</p><p>We will separately indicate in <strong>first_sample_size</strong> parameter the number of candidates in the first stage of beam search. This number can be higher than the number of candidates in the next stages (specified in the variable <strong>sample_size</strong>), since it is important to get enough candidates for the first token, on which all subsequent sequences will be based. According to my observations, this approach increases the likelihood of generating a coherent and reasonably probable sequence of tokens.</p><p>We will use high <strong>temperature</strong> parameter to lower model confidence in its top token choices. This allows to make the generation more varied and not get stuck with the most likely repeating sequences of tokens.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/40cdcfa4530dd5079a6d37ff7198fa74/href">https://medium.com/media/40cdcfa4530dd5079a6d37ff7198fa74/href</a></iframe><p>And finally we will try our bidirectional text generator with start phrase “text generation is cool”.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/102fc3ca373eb98d6982ef3c77abfcfd/href">https://medium.com/media/102fc3ca373eb98d6982ef3c77abfcfd/href</a></iframe><p>The resulting text:</p><pre>Follow the trend: Graphic design is cool, <strong>text generation</strong> is cool, data manipulation and algorithms are cool, etc.</pre><h3>Conclusion</h3><p>It’s also starting to seem somewhat like we are embarking into a new world largely controlled by <strong>artificial intelligence</strong> based on its ability over a long period to manipulate, manage and adapt our daily lives.</p><p>The entire previous paragraph was generated by our new text generator. The text is pretty convincing, isn’t it? Therefore, please accept my congratulations. We have created almost the first of its kind <strong>transformers based bidirectional text generator</strong>. And while it still makes a lot of mistakes, it can be used to create a lot of interesting and fun stories that will grow around any phrase that comes to your mind.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=49d9d37b48a9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/build-a-bidirectional-text-generator-with-xlnet-49d9d37b48a9">Build a bidirectional text generator with XLNet</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Word embeddings in 2020. Review with code examples]]></title>
            <link>https://medium.com/data-science/word-embeddings-in-2020-review-with-code-examples-11eb39a1ee6d?source=rss-8c7f264e81e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/11eb39a1ee6d</guid>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[word-embeddings]]></category>
            <category><![CDATA[transformers]]></category>
            <dc:creator><![CDATA[Rostyslav Neskorozhenyi]]></dc:creator>
            <pubDate>Fri, 24 Jul 2020 14:33:51 GMT</pubDate>
            <atom:updated>2020-07-24T14:49:08.580Z</atom:updated>
            <content:encoded><![CDATA[<h4>Brief overview of current word embedding methods: from Word2vec to Transformers</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1Kr-mgVDevo63WPYm37gBA.png" /><figcaption>Image is created by Rostyslav Neskorozhenyi with <a href="https://github.com/amueller/word_cloud">WordCloud</a> module</figcaption></figure><p>In this article we will study word embeddings — digital representation of words suitable for processing by machine learning algorithms.</p><p>Originally I created this article as a general overview and compilation of current approaches to word embedding in 2020, which our <a href="http://ai-labs.org/">AI Labs</a> team could use from time to time as a quick refresher. I hope that my article will be useful to a wider circle of data scientists and developers. Each word embedding method in the article has a (very) short description, links for further study, and code examples in Python. All code is packed as <a href="https://colab.research.google.com/drive/1N7HELWImK9xCYheyozVP3C_McbiRo1nb">Google Colab Notebook</a>. So let’s begin.</p><p>According to Wikipedia, <strong>Word embedding</strong> is the collective name for a set of language modeling and feature learning techniques in natural language processing (NLP) where words or phrases from the vocabulary are mapped to vectors of real numbers.</p><h3>One-hot or CountVectorizing</h3><p>The most basic method for transforming words into vectors is to count occurrence of each word in each document. Such approach is called countvectorizing or one-hot encoding.</p><p>The main principle of this method is to collect a set of documents (they can be words, sentences, paragraphs or even articles) and count the occurrence of every word in each document. Strictly speaking, the columns of the resulting matrix are words and the rows are documents.</p><pre>from sklearn.feature_extraction.text import CountVectorizer<br># create CountVectorizer object<br>vectorizer = CountVectorizer()<br>corpus = [<br>          &#39;Text of the very first new sentence with the first words in sentence.&#39;,<br>          &#39;Text of the second sentence.&#39;,<br>          &#39;Number three with lot of words words words.&#39;,<br>          &#39;Short text, less words.&#39;,<br>]</pre><pre># learn the vocabulary and store CountVectorizer sparse matrix in term_frequencies<br>term_frequencies = vectorizer.fit_transform(corpus) <br>vocab = vectorizer.get_feature_names()</pre><pre># convert sparse matrix to numpy array<br>term_frequencies = term_frequencies.toarray()</pre><pre># visualize term frequencies <br>import seaborn as sns<br>sns.heatmap(term_frequencies, annot=True, cbar = False, xticklabels = vocab);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/366/1*uthDlQaXkxocKjEbdKY8Sg.png" /><figcaption>Image is created by Rostyslav Neskorozhenyi with <a href="https://seaborn.pydata.org/">seaborn</a> module</figcaption></figure><p>Another approach in countvectorizing is just to place 1 if the word is found in the document (no matter how often) and 0 if the word is not found in the document. In this case we get real ‘one-hot’ encoding.</p><pre>one_hot_vectorizer = CountVectorizer(binary=True)<br>one_hot = one_hot_vectorizer.fit_transform(corpus).toarray()</pre><pre>sns.heatmap(one_hot, annot=True, cbar = False, xticklabels = vocab)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/366/1*x3tJKM3ssG9NUEg_plox3Q.png" /><figcaption>Image is created by Rostyslav Neskorozhenyi with <a href="https://seaborn.pydata.org/">seaborn</a> module</figcaption></figure><h3>TF-IDF encoding</h3><p>With a large corpus of documents some words like ‘a’, ‘the’, ‘is’, etc. occur very frequently but they don’t carry a lot of information. Using one-hot encoding approach we can decide that these words are important because they appear in many documents. One of the ways to solve this problem is stopwords filtering, but this solution is discrete and not flexible.</p><p>TF-IDF (term frequency — inverse document frequency) can deal with this problem better. TF-IDF lowers the weight of commonly used words and raises the weight of rare words that occur only in current document. TF-IDF formula looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/655/1*tMTbOwNzKx06chETJkZ6ww.png" /></figure><p>Where TF is calculated by dividing number of times the word occurs in the document by the total number of words in the document</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/349/1*y2mxpp2Vl6W7_ZmdUISkXA.png" /></figure><p>IDF (inverse document frequency), interpreted like inversed number of documents, in which the term we’re interested in occurs. N — number of documents, n(t) — number of documents with current word or term t.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/223/1*6PP8jjIoi7Oe3O1Wo4a2vQ.png" /></figure><pre>from sklearn.feature_extraction.text import TfidfVectorizer<br>import seaborn as sns</pre><pre>corpus = [<br>          &#39;Time flies like an arrow.&#39;,<br>          &#39;Fruit flies like a banana.&#39;<br>]</pre><pre>vocab = [&#39;an&#39;, &#39;arrow&#39;, &#39;banana&#39;, &#39;flies&#39;, &#39;fruit&#39;, &#39;like&#39;, &#39;time&#39;]</pre><pre>tfidf_vectorizer = TfidfVectorizer()<br>tfidf = tfidf_vectorizer.fit_transform(corpus).toarray()</pre><pre>sns.heatmap(tfidf, annot=True, cbar = False, xticklabels = vocab)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/366/1*xz3BO60PTBFMGqGftpu5fA.png" /><figcaption>Image is created by Rostyslav Neskorozhenyi with <a href="https://seaborn.pydata.org/">seaborn</a> module</figcaption></figure><h3>Word2Vec and GloVe</h3><p>The most commonly used models for word embeddings are <a href="https://github.com/dav/word2vec/">word2vec</a> and <a href="https://nlp.stanford.edu/projects/glove/">GloVe</a> which are both unsupervised approaches based on the distributional hypothesis (words that occur in the same contexts tend to have similar meanings).</p><p>Word2Vec word embeddings are vector representations of words, that are typically learnt by an unsupervised model when fed with large amounts of text as input (e.g. Wikipedia, science, news, articles etc.). These representation of words capture semantic similarity between words among other properties. Word2Vec word embeddings are learnt in a such way, that <a href="https://en.wikipedia.org/wiki/Euclidean_distance">distance</a> between vectors for words with close meanings (“king” and “queen” for example) are closer than distance for words with complety different meanings (“king” and “carpet” for example).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lzjgo2KaWFRPkV3LCJDr7Q.png" /><figcaption>Image from <a href="https://developers.google.com/machine-learning/crash-course/embeddings/translating-to-a-lower-dimensional-space">developers.google.com</a></figcaption></figure><p>Word2Vec vectors even allow some mathematic operations on vectors. For example, in this operation we are using word2vec vectors for each word:</p><p><strong>king — man + woman = queen</strong></p><pre># Download Google Word2Vec embeddings <a href="https://code.google.com/archive/p/word2vec/">https://code.google.com/archive/p/word2vec/</a></pre><pre>!wget <a href="https://s3.amazonaws.com/dl4j-distribution/GoogleNews-vectors-negative300.bin.gz">https://s3.amazonaws.com/dl4j-distribution/GoogleNews-vectors-negative300.bin.gz</a><br>!gunzip GoogleNews-vectors-negative300.bin</pre><pre># Try Word2Vec with Gensim</pre><pre>import gensim</pre><pre># Load pretrained vectors from Google<br>model = gensim.models.KeyedVectors.load_word2vec_format(&#39;GoogleNews-vectors-negative300.bin&#39;, binary=True)</pre><pre>king = model[&#39;king&#39;]</pre><pre># king - man + woman = queen<br>print(model.most_similar(positive=[&#39;woman&#39;, &#39;king&#39;], negative=[&#39;man&#39;]))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JXGyzf1eXCpHeB3Zh5qZ7w.png" /></figure><p>Check how similar are vectors for words ‘woman’ and ‘man’.</p><pre>print(model.similarity(&#39;woman&#39;, &#39;man&#39;))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/115/1*WXDuF9JkFif2j8WK-GdLSA.png" /></figure><p>Check how similar are vectors for words ‘king’ and ‘woman’.</p><pre>print(model.similarity(&#39;king&#39;, &#39;woman&#39;))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/114/1*oBq_TFjHv3PttrWERuoYdw.png" /></figure><p>Another word embedding method is <strong>Glove</strong> (“Global Vectors”). It is based on matrix factorization techniques on the word-context matrix. It first constructs a large matrix of (words x context) co-occurrence information, i.e. for each “word” (the rows), you count how frequently we see this word in some “context” (the columns) in a large corpus. Then this matrix is factorized to a lower-dimensional (word x features) matrix, where each row now stores a vector representation for each word. In general, this is done by minimizing a “reconstruction loss”. This loss tries to find the lower-dimensional representations which can explain most of the variance in the high-dimensional data.</p><pre># Try Glove word embeddings with Spacy</pre><pre>!python3 -m spacy download en_core_web_lg</pre><pre>import spacy<br># Load the spacy model that you have installed<br>import en_core_web_lg<br>nlp = en_core_web_lg.load()<br># process a sentence using the model<br>doc = nlp(&quot;man king stands on the carpet and sees woman queen&quot;)</pre><p>Find similarity between King and Queen (higher value is better).</p><pre>doc[1].similarity(doc[9])</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/133/1*YooWc7ckFnm_7lzR-IEAog.png" /></figure><p>Find similarity between King and carpet.</p><pre>doc[1].similarity(doc[5])</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/128/1*EdJVh10EafQ_EWA4WbphrA.png" /></figure><p>Check if king — man + woman = queen. We will multiply vectors for ‘man’ and ‘woman’ by two, because subtracting one vector for ‘man’ and adding the vector for ‘woman’ will do little to the original vector for “king”, likely because those “man” and “woman” are related themselves.</p><pre>v =  doc[1].vector - (doc[0].vector*2) + (doc[8].vector*2)</pre><pre>from scipy.spatial import distance<br>import numpy as np</pre><pre># Format the vocabulary for use in the distance function<br>vectors = [token.vector for token in doc]<br>vectors = np.array(vectors)</pre><pre># Find the closest word below <br>closest_index = distance.cdist(np.expand_dims(v, axis = 0), vectors, metric = &#39;cosine&#39;).argmin()<br>output_word = doc[closest_index].text<br>print(output_word)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/81/1*9_iGlCJZV6PkSDtH_2VyAg.png" /></figure><h3>FastText</h3><p><a href="https://github.com/facebookresearch/fastText">FastText</a> is an extension of word2vec. FastText was developed by the team of Tomas Mikolov who created the word2vec framework in 2013.</p><p>The main improvement of FastText over the original word2vec vectors is the inclusion of character <a href="https://en.wikipedia.org/wiki/N-gram">n-grams</a>, which allows computing word representations for words that did not appear in the training data (“out-of-vocabulary” words).</p><pre>!pip install Cython --install-option=&quot;--no-cython-compile&quot;<br>!pip install fasttext</pre><pre># download pre-trained language word vectors from one of 157 languges  <a href="https://fasttext.cc/docs/en/crawl-vectors.html">https://fasttext.cc/docs/en/crawl-vectors.html</a><br># it will take some time, about 5 minutes<br>import fasttext<br>import fasttext.util<br>fasttext.util.download_model(&#39;en&#39;, if_exists=&#39;ignore&#39;)  # English<br>ft = fasttext.load_model(&#39;cc.en.300.bin&#39;)</pre><p>Create an embedding for the word ‘king’.</p><pre>ft.get_word_vector(&#39;king&#39;)</pre><p>Get most similar words for the word ‘king’.</p><pre>ft.get_nearest_neighbors(&#39;king&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/420/1*hXByRootuiR-9igwUHxqAQ.png" /></figure><p>Test model ability to create vectors for unknown words.</p><pre>&#39;king-warrior&#39; in ft.words</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/69/1*JU1cL8cEfhKf_XELsXC0Uw.png" /></figure><p>Get most similar words for unknown word ‘king-warrior’.</p><pre>ft.get_nearest_neighbors(&#39;king-warrior&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/799/1*_a5E3pvSFzSmmOX8Ub2OCw.png" /></figure><h3>ELMo (Embeddings from Language Models)</h3><p>Unlike traditional word embeddings such as word2vec and GLoVe, the ELMo vector assigned to a token or a word depends on current context and is actually a function of the entire sentence containing that word. So the same word can have different word vectors under different contexts. Also ELMo representations are purely character based so they are not limited to any predefined vocabulary.</p><p>Description from the official site:</p><p><a href="https://allennlp.org/elmo"><strong>ELMo</strong></a> is a deep contextualized word representation that models both (1) complex characteristics of the word use (e.g., syntax and semantics), and (2) how these uses vary across linguistic contexts (i.e., to model polysemy). These word vectors are learned functions of the internal states of a deep bidirectional language model (biLM), which is pre-trained on a large text corpus. They can be easily added to existing models and significantly improve the state of the art across a broad range of challenging NLP problems, including question answering, textual entailment and sentiment analysis.</p><pre># use tensorflow 1.x for ELMo, because trere are still no ELMo for tensorflow 2.0</pre><pre>%tensorflow_version 1.x</pre><pre>import tensorflow_hub as hub<br>import tensorflow as tf</pre><pre># Download pretrained ELMo model from Tensorflow Hub <a href="https://tfhub.dev/google/elmo/3">https://tfhub.dev/google/elmo/3</a></pre><pre>elmo = hub.Module(&quot;<a href="https://tfhub.dev/google/elmo/3">https://tfhub.dev/google/elmo/3</a>&quot;, trainable=True)</pre><pre>sentences =  \<br>[&#39;king arthur, also called arthur or aathur pendragon, legendary british king who appears in a cycle of \<br>medieval romances (known as the matter of britain) as the sovereign of a knightly fellowship of the round table.&#39;, <br>&#39;it is not certain how these legends originated or whether the figure of arthur was based on a historical person.&#39;, <br>&#39;the legend possibly originated either in wales or in those parts of northern britain inhabited by brythonic-speaking celts.&#39;, <br>&#39;for a fuller treatment of the stories about king arthur, see also arthurian legend.&#39;]</pre><p>In order to send sentences to the model we need to split them into the arrays of words and pad arrays to the same length. Also we will create ‘mask’ array that will show whether element is a real word or a padding symbol (in our case — ‘_’). We will use ‘mask’ array for visualization later to show only real words.</p><pre>words = []<br>mask = []<br>masked_words = []</pre><pre>for sent in sentences:<br>  splitted = sent.split()<br>  for i in range(36):<br>    try:<br>      words.append(splitted[i])<br>    except:<br>      words.append(&#39;_&#39;)</pre><pre>for word in words:<br>  if word == &quot;_&quot;:<br>    mask.append(False)<br>  else:<br>    mask.append(True)<br>    masked_words.append(word)</pre><p>Create embeddings with ELMo:</p><pre>embeddings = elmo(<br>    sentences,<br>    signature=&quot;default&quot;,<br>    as_dict=True)[&quot;elmo&quot;]</pre><p>Convert Tensorflow tensors to numpy array.</p><pre>%%time<br>with tf.Session() as sess:<br>  sess.run(tf.global_variables_initializer())<br>  sess.run(tf.tables_initializer())<br>  x = sess.run(embeddings)</pre><pre>embs = x.reshape(-1, 1024)</pre><pre>masked_embs = embs[mask]</pre><p>Visualize embeddings using <a href="https://en.wikipedia.org/wiki/Principal_component_analysis">PCA</a>.</p><pre>from sklearn.decomposition import PCA</pre><pre>pca = PCA(n_components=10)<br>y = pca.fit_transform(masked_embs)</pre><pre>from sklearn.manifold import TSNE</pre><pre>y = TSNE(n_components=2).fit_transform(y)</pre><pre>import plotly as py<br>import plotly.graph_objs as go</pre><pre>data = [<br>    go.Scatter(<br>        x=[i[0] for i in y],<br>        y=[i[1] for i in y],<br>        mode=&#39;markers&#39;,<br>        text=[i for i in masked_words],<br>    marker=dict(<br>        size=16,<br>        color = [len(i) for i in masked_words], #set color equal to a variable<br>        opacity= 0.8,<br>        colorscale=&#39;Viridis&#39;,<br>        showscale=False<br>    )<br>    )<br>]<br>layout = go.Layout()<br>layout = dict(<br>              yaxis = dict(zeroline = False),<br>              xaxis = dict(zeroline = False)<br>             )<br>fig = go.Figure(data=data, layout=layout)<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nDE05QGHA7Uils2HEG_5UQ.gif" /><figcaption>Image is created by Rostyslav Neskorozhenyi with <a href="https://plotly.com/">plotly</a> module</figcaption></figure><h3>Transformers</h3><p>At last it’s time for current state-of-the-art approach — Transformers. Famous <a href="https://openai.com/blog/better-language-models/">GPT-2</a>, <a href="https://github.com/google-research/bert">BERT</a>, <a href="https://blog.einstein.ai/introducing-a-conditional-transformer-language-model-for-controllable-generation/">CTRL</a> are all Transformers-based and produce context-sensitive embeddings like ELMo. But unlike ELMo Transformers do not use <a href="https://en.wikipedia.org/wiki/Recurrent_neural_network">RNN</a>, they do not require to process words in sentence sequentially one-by-one. All words in the sentence are processed in parallel, this approach speeds up processing and solves <a href="https://en.wikipedia.org/wiki/Vanishing_gradient_problem">vanishing gradient problem</a>.</p><p>Transformers use the <a href="https://arxiv.org/abs/1706.03762">attention mechanizm</a> to describe the connections and dependencies of each specific word with all other words in the sentence. This mechanism and the main principles of Transformers described in detail in a beautifully illustrated <a href="http://jalammar.github.io/illustrated-transformer/">article</a> by Jay Alammar.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lnxCxwWsz_uUnunF.png" /><figcaption>Image from<a href="http://jalammar.github.io/illustrated-transformer/"> http://jalammar.github.io</a></figcaption></figure><p>For our example we will use brilliant <a href="https://huggingface.co/transformers/">Transformers</a> library which contains the latest Transformers-based models (such as <a href="https://huggingface.co/transformers/model_doc/bert.html">BERT</a>, <a href="https://huggingface.co/transformers/model_doc/xlnet.html">XLNet</a>, <a href="https://huggingface.co/transformers/model_doc/dialogpt.html">DialoGPT</a> or <a href="https://huggingface.co/transformers/model_doc/gpt2.html">GPT-2</a>).</p><p>Let’s make some embeddings with BERT. Firstly we will need to install Transformers library.</p><pre>!pip install transformers</pre><p>Now we import pytorch, the pretrained BERT model, and a BERT tokenizer that will do all the needed work of converting sentences into format appropriate for BERT (tokenizing itself and adding special tokens like [SEP] and [CLS]).</p><pre>import torch<br>torch.manual_seed(0)<br>from transformers import BertTokenizer, BertModel</pre><pre>import logging<br>import matplotlib.pyplot as plt<br>% matplotlib inline</pre><pre># Load pre-trained model tokenizer (vocabulary)<br>tokenizer = BertTokenizer.from_pretrained(&#39;bert-base-uncased&#39;, do_lower_case=True)</pre><p>Enter some sentences and tokenize them.</p><pre>sentences =  \<br>[&#39;king arthur, also called arthur or aathur pendragon, legendary british king who appears in a cycle of \<br>medieval romances (known as the matter of britain) as the sovereign of a knightly fellowship of the round table.&#39;, <br>&#39;it is not certain how these legends originated or whether the figure of arthur was based on a historical person.&#39;, <br>&#39;the legend possibly originated either in wales or in those parts of northern britain inhabited by brythonic-speaking celts.&#39;, <br>&#39;for a fuller treatment of the stories about king arthur, see also arthurian legend.&#39;]</pre><pre># Print the original sentence.<br>print(&#39; Original: &#39;, sentences[0][:99])</pre><pre># Print the sentence splitted into tokens.<br>print(&#39;Tokenized: &#39;, tokenizer.tokenize(sentences[0])[:15])</pre><pre># Print the sentence mapped to token ids.<br>print(&#39;Token IDs: &#39;, tokenizer.convert_tokens_to_ids(tokenizer.tokenize(sentences[0]))[:15])</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IvpCSwGpvsRPcGDTiQCeIg.png" /></figure><p>Note that some tokens may look like this: [‘aa’, ‘##th’, ‘##ur’, ‘pen’, ‘##dra’, ‘##gon’]. This is because of the BERT tokenizer was created with a WordPiece model. This model greedily creates a fixed-size vocabulary of individual characters, subwords, and words that best fits our language data. BERT tokenizer uses vocabulary that contains all English characters plus the ~30,000 most common words and subwords found in the English language corpus the model is trained on. So, if the word is not mentioned in a vocabulary, that words is splitted into subwords and characters. The two hash signs (##) before some subwords shows that subword is part of a larger word and preceded by another subword.</p><p>We will use tokenizer.encode_plus function, that will:</p><ul><li>Split the sentence into tokens.</li><li>Add the special [CLS] and [SEP] tokens.</li><li>Map the tokens to their IDs.</li><li>Pad or truncate all sentences to the same length.</li></ul><pre># Tokenize all of the sentences and map tokens to word IDs.<br>input_ids = []<br>attention_masks = []<br>tokenized_texts = []</pre><pre>for sent in sentences:<br>    encoded_dict = tokenizer.encode_plus(<br>                        sent,                      <br>                        add_special_tokens = True,<br>                        truncation=True,<br>                        max_length = 48,          <br>                        pad_to_max_length = True,                        <br>                        return_tensors = &#39;pt&#39;,    <br>                   )</pre><pre>    # Save tokens from sentence as a separate array. <br>    marked_text = &quot;[CLS] &quot; + sent + &quot; [SEP]&quot;<br>    tokenized_texts.append(tokenizer.tokenize(marked_text))<br>    <br>    # Add the encoded sentence to the list.    <br>    input_ids.append(encoded_dict[&#39;input_ids&#39;])</pre><pre># Convert the list into tensor.<br>input_ids = torch.cat(input_ids, dim=0)</pre><p><strong>Segment ID</strong>. BERT is trained on and expects sentence pairs using 1s and 0s to distinguish between the two sentences. We will encode each sentence separately so we will just mark each token in each sentence with 1.</p><pre>segments_ids = torch.ones_like(input_ids)</pre><p>Now we can call BERT model and finally get model hidden states from which we will create word embeddings.</p><pre>with torch.no_grad():</pre><pre>    outputs = model(input_ids, segments_ids)<br>    hidden_states = outputs[2]</pre><p>Let’s examine what we’ve got.</p><pre>print (&quot;Number of layers:&quot;, len(hidden_states), &quot;  (initial embeddings + 12 BERT layers)&quot;)<br>print (&quot;Number of batches:&quot;, len(hidden_states[0]))<br>print (&quot;Number of tokens:&quot;, len(hidden_states[0][0]))<br>print (&quot;Number of hidden units:&quot;, len(hidden_states[0][0][0]))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/678/1*C6knXyZ1F4ifCBf4Op6_VQ.png" /></figure><pre># Concatenate the tensors for all layers. <br>token_embeddings = torch.stack(hidden_states, dim=0)</pre><pre># Swap dimensions, so we get tensors in format: [sentence, tokens, hidden layes, features]<br>token_embeddings = token_embeddings.permute(1,2,0,3)</pre><p>We will use last four hidden layers to create each word embedding.</p><pre>processed_embeddings = token_embeddings[:, :, 9:, :]</pre><p>Concatenate four layers for each token to create embeddings</p><pre>embeddings = torch.reshape(processed_embeddings, (4, 48, -1))</pre><p>Let’s examine embeddings for the first sentence. Firstly we need to get ids of tokens we need to compare.</p><pre>for i, token_str in enumerate(tokenized_texts[0]):<br>  print (i, token_str)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/176/1*22SANoUAypOA9Wh1AxJ0eA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/54/1*m7_nDa6ZnazaTwdw7bDLVQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/127/1*-PM5ZFBVqQf0b-PwIqjH-w.png" /></figure><p>We can see that word ‘king’ is placed at indexes 1 and 17. We will check distance between embeddings 1 and 17. Also, we will check if embedding for the word ‘arthur’ is closer to ‘king’ then to the word ‘table’.</p><pre>from scipy.spatial.distance import cosine<br> <br>kings = cosine(embeddings[0][1], embeddings[0][17])<br>king_table = cosine(embeddings[0][1], embeddings[0][46])<br>king_archtur = cosine(embeddings[0][2], embeddings[0][1])<br> <br>print(&#39;Distance for two kings:  %.2f&#39; % kings)<br>print(&#39;Distance from king to table:  %.2f&#39; % king_table)<br>print(&#39;Distance from Archtur to king:  %.2f&#39; % king_archtur)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/404/1*gdhM9t-qql7ORgLNNMHJSg.png" /></figure><p>So we see that embeddings for two ‘kings’ are quite similar but not the same, and Archtur is closer to be a king than a table.</p><p>Things may be simplier with <a href="https://github.com/AliOsm/simplerepresentations"><strong>simplerepresentations</strong></a> module. This module does all the work we did earlier — extracts needed hidden states from BERT and creates embeddings in a few lines of code.</p><pre>!pip install simplerepresentations</pre><pre>import torch<br>from simplerepresentations import RepresentationModel<br>torch.manual_seed(0)</pre><pre>model_type = &#39;bert&#39;<br>model_name = &#39;bert-base-uncased&#39;</pre><pre>representation_model = RepresentationModel(<br>  model_type=model_type,<br>  model_name=model_name,<br>  batch_size=4,<br>  max_seq_length=48, <br>  combination_method=&#39;cat&#39;, <br>  last_hidden_to_use=4 <br> )</pre><pre>text_a = sentences</pre><pre>all_sentences_representations, all_tokens_representations = representation_model(text_a=text_a)</pre><p>Check distaces between Archtur, king and table.</p><pre>from scipy.spatial.distance import cosine</pre><pre>kings = cosine(all_tokens_representations[0][1], all_tokens_representations[0][17])<br>king_table = cosine(all_tokens_representations[0][1], all_tokens_representations[0][46])<br>king_archtur = cosine(all_tokens_representations[0][2], all_tokens_representations[0][1])</pre><pre>print(&#39;Distance for two kings:  %.2f&#39; % kings)<br>print(&#39;Distance from king to table:  %.2f&#39; % king_table)<br>print(&#39;Distance from Archtur to king:  %.2f&#39; % king_archtur)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/405/1*5byz3rLd9jf2mSX7n-MHtg.png" /></figure><p>Same results, less code.</p><h3>Conclusion</h3><p>I hope that after reading this article you have formed an idea of the current approaches to word embeddings and began to understand how to quickly implement these approaches in Python. The world of NLP is diverse and there are many more models and methods for embeddings. In my article I focused on the most common and those that we ourselves often use in our work. You can find additional information in the <strong>References</strong> section.</p><h3>References</h3><ul><li><a href="https://mccormickml.com/2019/05/14/BERT-word-embeddings-tutorial/">BERT Word Embeddings Tutorial</a></li><li><a href="http://jalammar.github.io/illustrated-transformer/">The Illustrated Transformer</a></li><li><a href="http://jalammar.github.io/illustrated-gpt2/">The Illustrated GPT-2 (Visualizing Transformer Language Models)</a></li><li><a href="https://towardsdatascience.com/from-pre-trained-word-embeddings-to-pre-trained-language-models-focus-on-bert-343815627598">FROM Pre-trained Word Embeddings TO Pre-trained Language Models — Focus on BERT</a></li><li><a href="https://towardsdatascience.com/make-your-own-rick-sanchez-bot-with-transformers-and-dialogpt-fine-tuning-f85e6d1f4e30">Make your own Rick Sanchez (bot) with Transformers and DialoGPT fine-tuning</a></li><li><a href="https://medium.com/swlh/playing-with-word-vectors-308ab2faa519">Playing with word vectors</a></li><li><a href="https://towardsdatascience.com/light-on-math-ml-intuitive-guide-to-understanding-glove-embeddings-b13b4f19c010">Intuitive Guide to Understanding GloVe Embeddings</a></li><li><a href="https://www.shanelynn.ie/word-embeddings-in-python-with-spacy-and-gensim/">Word Embeddings in Python with Spacy and Gensim</a></li><li><a href="https://medium.com/analytics-vidhya/brief-review-of-word-embedding-families-2019-b2bbc601bbfe">Brief review of word embedding families (2019)</a></li><li><a href="https://towardsdatascience.com/word-embeddings-exploration-explanation-and-exploitation-with-code-in-python-5dac99d5d795">Word embeddings: exploration, explanation, and exploitation (with code in Python)</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=11eb39a1ee6d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/word-embeddings-in-2020-review-with-code-examples-11eb39a1ee6d">Word embeddings in 2020. Review with code examples</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Improve person re-identification with face detection (FaceBoxes)]]></title>
            <link>https://medium.com/data-science/improve-person-re-identification-with-face-detection-faceboxes-4450a93e50c7?source=rss-8c7f264e81e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/4450a93e50c7</guid>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[re-identification]]></category>
            <dc:creator><![CDATA[Rostyslav Neskorozhenyi]]></dc:creator>
            <pubDate>Mon, 29 Jun 2020 13:24:25 GMT</pubDate>
            <atom:updated>2020-06-29T16:11:11.187Z</atom:updated>
            <content:encoded><![CDATA[<h4>Simple method to reduce number of viewpoints and create high-quality embeddings</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ox1NqlBp_O1PRhpciv4bQw.png" /><figcaption>Image from <a href="https://unsplash.com/">Unsplash</a> by <a href="https://unsplash.com/@bullfishfighter">Joël Vogt</a></figcaption></figure><p>Person re-identification is an interesting and not completely solved task. It includes finding (localizing) a person in an image and creating a digital description (vector or embedding) for a photo of a particular person in a way that the <a href="https://en.wikipedia.org/wiki/Euclidean_distance">distance</a> to the vectors for other photos of a particular person is closer than to the vectors generated for photos of other people.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1016/0*9juJaRuR1otZyPQz" /><figcaption>Image from<a href="https://github.com/omoindrot/tensorflow-triplet-loss"> https://github.com/omoindrot/tensorflow-triplet-loss</a></figcaption></figure><p>Person re-identification is used in many tasks including visitor flow analysis in a shopping center, tracking people across cameras, finding a certain person in a huge amount of photos.</p><p>Many effective models and approaches have been created recently to address the re-identification tasks. Full list of those models can be found <a href="https://paperswithcode.com/task/person-re-identification">here</a>. But even the best models are still faced with a lot of problems, such as variations in pose and viewpoints of people because of which the embeddings for a photo of a person from different angles will be too far from each other, and the system can decide that this is a photo of different people.</p><p>The latest state-of-the-art models, such as<a href="https://paperswithcode.com/paper/viewpoint-aware-loss-with-angular"> Viewpoint-Aware Loss with Angular Regularization for Person Re-Identification</a>, are designed to deal with mentioned problems, but we at <a href="http://ai-labs.org/">ai-labs.org</a> came up with a light approach that greatly simplifies the task of re-identification in some situations. I will talk about this approach in more detail.</p><p>Let’s start by explaining how most of the re-id frameworks detect photos of a particular person in the image. The most commonly used object detection models, such as <a href="https://arxiv.org/pdf/1506.01497.pdf">Faster R-CNN</a> or <a href="https://arxiv.org/abs/1911.09070">EfficientDet</a>, are used to create a bounding box for the entire human body. After a photo for the entire human body is extracted, the embedding for this photo will be created.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/556/1*Pz8f2GM6vgUb_gjat3o7ug.png" /><figcaption>Images from <a href="https://www.kaggle.com/twoboysandhats/mars-motion-analysis-and-reidentification-set">MARS: Motion Analysis and Re-identification Set</a></figcaption></figure><p>The problem is that object detection models often work even too well, they find photos of people from a variety of viewpoints and not always of the best quality. Embeddings based on these photos often do not allow correct re-identification of a person or are generated in such a way that embeddings for a photo of a particular person from one viewpoint will be close to embeddings for photos only from the same viewpoint, but not to embeddings for photos of the same person from a different viewpoint and distance.</p><p>The problem can be solved by reducing the number of viewpoints that we will have to work with. This can be achieved if the person re-identification is done not over the whole body image, but only over the face. Face detection model <a href="https://github.com/TropComplique/FaceBoxes-tensorflow">FaceBoxes</a> will help in such case. This model has proven itself in our work, plus it perfectly detects the faces of people in masks, which is especially important now. With face detection we discard a big part of viewpoint variations such as back view but if we use only face photos from low-quality images, then we may not have enough information from a particular photo to distinguish it from thousands of other photos.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/456/1*UfbEDa9Lb6Gj6rrV76z3BQ.png" /><figcaption>Images from <a href="https://www.kaggle.com/twoboysandhats/mars-motion-analysis-and-reidentification-set">MARS: Motion Analysis and Re-identification Set</a></figcaption></figure><p>It is worth mentioning the implementation features of the <a href="https://github.com/TropComplique/FaceBoxes-tensorflow">FaceBoxes</a> model that we use. It does not detect small faces, which in our case turns from a disadvantage into a virtue, since we immediately discard too small (30x30 pixels or less) photos, the information on which will not be enough for correct re-identification.</p><p>We also use a little trick, which will allow us to get more information about a person without increasing the number of viewpoints. We will expand the bounding box of the face according to a certain algorithm, capturing also part of the human torso. Here is an example of the implementation of this algorithm in the form of a function that receives a frame as an input and returns a list of detected people images using the FaceDetector object from <a href="https://github.com/TropComplique/FaceBoxes-tensorflow">FaceBoxes</a>:</p><pre>import numpy as np<br>from PIL import Image<br>from face_detector import FaceDetector<br> <br>MODEL_PATH = &#39;model/facebox.pb&#39;<br>face_detector = FaceDetector(MODEL_PATH, gpu_memory_fraction=0.25, visible_device_list=&#39;0&#39;)</pre><pre>def pad_faces(frame, tres = 0.5):<br>   face_pad = []<br>   width, height = frame.size   <br>   image_array = np.array(frame)<br>   boxes, scores = face_detector(image_array, score_threshold=tres)<br> <br>   for b in boxes:<br>       ymin, xmin, ymax, xmax = b.astype(&#39;int&#39;)<br> <br>       top_pad = int((ymax - ymin) * 0.4)<br>       bottom_pad = int(ymax - ymin)<br>       w_pad = int((xmax - xmin) * 0.4)<br> <br>       ymin = ymin - top_pad if ymin - top_pad &gt; 0 else 0<br>       ymax = ymax + bottom_pad if ymax + bottom_pad &lt; height else height - 1<br>       xmin = xmin - w_pad if xmin - w_pad &gt; 0 else 0<br>       xmax = xmax + w_pad if xmax + w_pad &lt; width else width - 1<br> <br>       ratio = (ymax-ymin) / (xmax-xmin)<br> <br>       if ratio &lt; 1.6:<br>           xmin = xmin + w_pad<br>           xmax = xmax - w_pad<br>       ratio = (ymax-ymin) / (xmax-xmin)<br> <br>       face_pad.append(image_array[ymin:ymax, xmin:xmax])<br>  <br>   return face_pad</pre><pre>imagex = Image.open(&#39;image1.jpg&#39;).convert(&#39;RGB&#39;)<br>padded_faces = pad_faces(imagex)</pre><p>After applying the function, we get images of this format:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/604/1*GBx8RPF0Jp58VvkQAOWd9Q.png" /><figcaption>Images from <a href="https://www.kaggle.com/twoboysandhats/mars-motion-analysis-and-reidentification-set">MARS: Motion Analysis and Re-identification Set</a></figcaption></figure><p>In total, we selected images of higher quality and mainly from one viewpoint. This will simplify the task of re-identification since we receive embeddings for a particular person from one viewpoint, and such embeddings will be close to each other. To generate embeddings, we can use the <a href="https://github.com/michuanhaohao/reid-strong-baseline">Bag of Tricks and A Strong ReID Baseline </a>model, which shows the high quality of embeddings on a variety of datasets.</p><p>This approach is not suitable for all cases, but it can increase the accuracy of your re-identification pipeline in the area where people spend a lot of time and you get a lot of photos of each person from different viewpoints.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4450a93e50c7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/improve-person-re-identification-with-face-detection-faceboxes-4450a93e50c7">Improve person re-identification with face detection (FaceBoxes)</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Make your own Rick Sanchez (bot) with Transformers and DialoGPT fine-tuning]]></title>
            <link>https://medium.com/data-science/make-your-own-rick-sanchez-bot-with-transformers-and-dialogpt-fine-tuning-f85e6d1f4e30?source=rss-8c7f264e81e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/f85e6d1f4e30</guid>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[transformers]]></category>
            <dc:creator><![CDATA[Rostyslav Neskorozhenyi]]></dc:creator>
            <pubDate>Mon, 08 Jun 2020 10:47:03 GMT</pubDate>
            <atom:updated>2020-06-08T19:23:14.762Z</atom:updated>
            <content:encoded><![CDATA[<h4>Teach GPT-2 to write like a mad scientist</h4><p>I am a big fan of ‘<a href="https://en.wikipedia.org/wiki/Rick_and_Morty">Rick and Morty</a>’ and NLP (Natural Language Processing). After watching the last episode of a mad scientist’s and his grandson’s adventures I realized that I have to apply my knowledge in NLP and Machine Learning to create my own Rick (in a form of chat-bot, at least for now).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*t75wxMkidc4GFYxR.gif" /><figcaption>Image from <a href="https://giphy.com/">Giphy</a></figcaption></figure><p>The path to creating virtual Rick was not easy. I tried many models and approaches but virtual Rick did not want to find a common language with me for a long time. Rick’s first response to my greeting was “I am not your buddy” (to tell you the truth, in the beginning, he knew only this phrase). But I did not give up and with the help of excellent Transformers library and Microsoft’s Large-scale Pretrained Response Generation Model (DialoGPT) I managed to talk heart-to-heart with Rick. Here is one of our conversations:</p><blockquote><strong>User</strong>:Hey, Rick, make me a time machine <br><strong>RickBot</strong>: You’re a monster! <br><strong>User</strong>:No, I just want to travel in time <br><strong>RickBot</strong>: To the past. <br><strong>User</strong>:yes <br><strong>RickBot</strong>: No time machine, please. <br><strong>User</strong>:But I need it <br><strong>RickBot</strong>: You don’t need it, Morty. Just do it. <br><strong>User</strong>:I want to see ancient Greece <br><strong>RickBot</strong>: Oh, geez.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*fKL_VMeQ4NjLpjcO.gif" /><figcaption>Image from <a href="https://giphy.com/">Giphy</a></figcaption></figure><h3>A bit of theory</h3><p>In this article, I will tell you how to create a virtual character whose statements will be based on a transcript of my favorite animated science fiction sitcom. You can use characters and sitcoms of your choice. I added code to convert a regular text file with dialogs into a format that the model understands.</p><p>As I already mentioned, the <a href="https://huggingface.co/transformers/">Transformers</a> library, which contains the latest NLP models (such as <a href="https://huggingface.co/transformers/model_doc/bert.html">BERT</a>, <a href="https://huggingface.co/transformers/model_doc/xlnet.html">XLNet</a>, <a href="https://huggingface.co/transformers/model_doc/gpt2.html">GPT-2</a>) will help us in our task. You can read more about transformers in a beautifully illustrated <a href="http://jalammar.github.io/illustrated-transformer/">article</a> by Jay Alammar.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*KM7q7Ybzmmnlz3SV.png" /><figcaption>Image from<a href="http://jalammar.github.io/illustrated-transformer/"> http://jalammar.github.io</a></figcaption></figure><p>Not so long ago, Microsoft’s <a href="https://huggingface.co/transformers/model_doc/dialogpt.html">DialoGPT</a> was added to the Transformers model collection. DialoGPT is a GPT-2 model, trained on 147M multi-turn dialogue from Reddit discussion thread (you can learn more about GPT-2 <a href="http://jalammar.github.io/illustrated-gpt2/">here</a>). This model is ideally suited for creating a virtual character for a fascinating conversation and even in the small implementation option, it can maintain a coherent dialogue, which we will see now.</p><h3>First dialogue with DialoGPT</h3><p>We will conduct all our experiments in <a href="https://colab.research.google.com/">Google Colab</a>, its resources are enough to train the small DialoGPT model. Firstly, we will connect to Google Drive and install the necessary modules.</p><pre>from google.colab import drive<br>drive.mount(&#39;/content/drive/&#39;)</pre><pre>! pip -q install transformers</pre><p>Let’s move to the desired folder in which we will store all our data.</p><pre>import os</pre><pre>os.chdir(&quot;/content/drive/My Drive/Colab Notebooks&quot;)</pre><p>Try to chat with DialoGPT without fine-tuning.</p><pre>from transformers import AutoModelWithLMHead, AutoTokenizer<br>import torch</pre><pre>tokenizer = AutoTokenizer.from_pretrained(&quot;microsoft/DialoGPT-small&quot;)<br>model = AutoModelWithLMHead.from_pretrained(&quot;microsoft/DialoGPT-small&quot;)</pre><p>Let’s chat for 5 lines</p><pre>for step in range(5):<br>    # encode the new user input, add the eos_token and return a tensor in Pytorch<br>    new_user_input_ids = tokenizer.encode(input(&quot;&gt;&gt; User:&quot;) + tokenizer.eos_token, return_tensors=&#39;pt&#39;)</pre><pre># append the new user input tokens to the chat history<br>    bot_input_ids = torch.cat([chat_history_ids, new_user_input_ids], dim=-1) if step &gt; 0 else new_user_input_ids</pre><pre># generated a response while limiting the total chat history to 1000 tokens    <br>    chat_history_ids = model.generate(<br>    bot_input_ids, max_length=1000,<br>    pad_token_id=tokenizer.eos_token_id<br>    )</pre><pre># pretty print last ouput tokens from bot<br>    print(&quot;DialoGPT: {}&quot;.format(tokenizer.decode(chat_history_ids[:, bot_input_ids.shape[-1]:][0], skip_special_tokens=True)))</pre><blockquote><strong>User:</strong>Hi Rick <br><strong>DialoGPT:</strong> Hi Rick <br><strong>User:</strong>How are you? <br><strong>DialoGPT:</strong> I’m good, how are you? <br><strong>User:</strong>I am fine. Where is Morty? <br><strong>DialoGPT:</strong> He’s in the basement. <br><strong>User:</strong>Who is Morty? <br><strong>DialoGPT:</strong> He’s a Morty. <br><strong>User:</strong>Who are you? <br><strong>DialoGPT:</strong> I am a Morty.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*M4GKteqV_Kl-yAaa.gif" /><figcaption>Image from <a href="https://giphy.com/">Giphy</a></figcaption></figure><p>Not bad but not too impressive. We will fix it with fine-tuning.</p><h3>Model initial configuration</h3><p>Let’s train our own Rick chatbot. For a start, we will need basic configuration and a dataset. Configuration and training scripts are mostly based on this <a href="https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_language_modeling.py">script</a> from Huggingface and great <a href="https://nathancooper.io/i-am-a-nerd/chatbot/deep-learning/gpt2/2020/05/12/chatbot-part-1.html">tutorial</a> from Nathan Cooper.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/753a50f5f2996ceeef4a2d089604b36a/href">https://medium.com/media/753a50f5f2996ceeef4a2d089604b36a/href</a></iframe><p>Class Args for conversion of Python script arguments to Colab notebook.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d8f31628f4782e3a5f76cadbd9acd22e/href">https://medium.com/media/d8f31628f4782e3a5f76cadbd9acd22e/href</a></iframe><h3>Prepare Dataset</h3><p>Our dialogues dataset will be based on a dataset used in Andrada Olteanu’s <a href="https://www.kaggle.com/andradaolteanu/sentiment-analysis-rick-and-morty-scripts/">article</a> about Rick and Morty sentiment analysis. Big thanks to her work and also to Gabriel Hernandes, author of original <a href="https://github.com/ghhernandes/rickmorty-gan/tree/master/data">text dataset</a>!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*AuVQEU4bGTvYlKdz.gif" /><figcaption>Image from <a href="https://giphy.com/">Giphy</a></figcaption></figure><p>First of all, we will use a <strong>Kaggle </strong>module to download the needed dataset. You can read in more detail about the module and how to get a Kaggle API Token by this <a href="https://github.com/Kaggle/kaggle-api">link</a>. Or you can just download RickAndMortyScripts.csv file from this <a href="https://www.kaggle.com/andradaolteanu/sentiment-analysis-rick-and-morty-scripts/">article</a> and place this file in your working directory.</p><pre>!pip install kaggle</pre><pre>!mkdir ~/.kaggle<br>!cp kaggle.json ~/.kaggle/kaggle.json</pre><pre>!kaggle datasets download andradaolteanu/rickmorty-scripts -f RickAndMortyScripts.csv <br>!mv datasets%2F506221%2F935855%2FRickAndMortyScripts.csv RickAndMortyScripts.csv</pre><p>Let’s look at the original dataset.</p><pre>all_rick = pd.read_csv(&#39;RickAndMortyScripts.csv&#39;)<br>all_rick.head(10)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/884/1*cdMw77_y34WJdxccvguL5g.png" /></figure><p>We will convert this dataset in a way that every response row will contain <strong>n</strong> previous responses as a context. For our purposes, seven previous responses will be enough.</p><pre>contexted = []</pre><pre>n = 7</pre><pre>for i in range(n, len(all_rick[&#39;line&#39;])):<br>  row = []<br>  prev = i - 1 - n # we additionally subtract 1, so row will contain current response and 7 previous responses  <br>  for j in range(i, prev, -1):<br>    row.append(all_rick[&#39;line&#39;][j])<br>  contexted.append(row)</pre><pre>columns = [&#39;response&#39;, &#39;context&#39;] <br>columns = columns + [&#39;context/&#39;+str(i) for i in range(n-1)]</pre><pre>df = pd.DataFrame.from_records(contexted, columns=columns)<br>df.head(5)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NIDy3Jt0VW5jBb4EvXI1nQ.png" /></figure><p>Split our dataset into training and test parts.</p><pre>trn_df, val_df = train_test_split(df, test_size = 0.1)</pre><p>Now we will convert our dataset in a format suitable for our model. Basically we will concatenate responses in one string for each row (additionally we will add special ‘end of string’ token between responses, so the model will understand the end of each response in a string).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/372783211407cf29806b5a5c58809bf4/href">https://medium.com/media/372783211407cf29806b5a5c58809bf4/href</a></iframe><h3>Training and Evaluating</h3><p>There will be quite a lot of code needed for training our model but don’t worry, everything should work as is, the main thing is to give the model the dataset in the right format.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*G-wEeBAEIc7GfMH1.gif" /><figcaption>Image from <a href="https://giphy.com/">Giphy</a></figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a52e98fde558e5a4da14303ff475ba54/href">https://medium.com/media/a52e98fde558e5a4da14303ff475ba54/href</a></iframe><p>And here is the main runner code.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e3a770160ce4ce005baf9ddc1e561773/href">https://medium.com/media/e3a770160ce4ce005baf9ddc1e561773/href</a></iframe><p>It is time to train our model!</p><pre>main(trn_df, val_df)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*aYc0OMKAI9hdLksT.gif" /><figcaption>Image from <a href="https://giphy.com/">Giphy</a></figcaption></figure><p>Our dataset is quite small so training will not take a lot of time (usually — about a half an hour).</p><h3>Chatting with Rick</h3><p>The model is ready, so it’s time to chat with Rick. But don’t forget that Rick can be rude, I warned you.</p><p>A variety of methods can be used in responses generation. You can find more details about these methods by this <a href="https://huggingface.co/blog/how-to-generate">link</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*iWP2O6V20kdqWLlv.gif" /><figcaption>Image from <a href="https://giphy.com/">Giphy</a></figcaption></figure><pre>tokenizer = AutoTokenizer.from_pretrained(&#39;microsoft/DialoGPT-small&#39;)<br>model = AutoModelWithLMHead.from_pretrained(&#39;output-small&#39;)</pre><pre># Let&#39;s chat for 5 lines<br>for step in range(5):<br>    # encode the new user input, add the eos_token and return a tensor in Pytorch<br>    new_user_input_ids = tokenizer.encode(input(&quot;&gt;&gt; User:&quot;) + tokenizer.eos_token, return_tensors=&#39;pt&#39;)<br>    # print(new_user_input_ids)</pre><pre># append the new user input tokens to the chat history<br>    bot_input_ids = torch.cat([chat_history_ids, new_user_input_ids], dim=-1) if step &gt; 0 else new_user_input_ids</pre><pre># generated a response while limiting the total chat history to 1000 tokens, <br>    chat_history_ids = model.generate(<br>        bot_input_ids, max_length=200,<br>        pad_token_id=tokenizer.eos_token_id,  <br>        no_repeat_ngram_size=3,       <br>        do_sample=True, <br>        top_k=100, <br>        top_p=0.7,<br>        temperature = 0.8<br>    )<br>    <br>    # pretty print last ouput tokens from bot<br>    print(&quot;RickBot: {}&quot;.format(tokenizer.decode(chat_history_ids[:, bot_input_ids.shape[-1]:][0], skip_special_tokens=True)))</pre><blockquote><strong>User</strong>:How are you, Rick? <br><strong>RickBot</strong>: I’m fine. <br><strong>User</strong>:Where is Morty? <br><strong>RickBot</strong>: He’s in a hospital. <br><strong>User</strong>:Why? <br><strong>RickBot</strong>: Well, that depends on who breaks first — me or the hospital. I mean, you know, the one that keeps giving me pills. <br><strong>User</strong>:I don’t understand <br><strong>RickBot</strong>: Whoa! Whoa! <br><strong>User</strong>:I see, you took your pills <br><strong>RickBot</strong>: Hey, I-I’m not proud of this.</blockquote><p>Not bad. I will give examples of a few more dialogues to show how Ricky is our Rick now.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*kka6WW6bP-V8fLV_.gif" /><figcaption>Image from <a href="https://giphy.com/">Giphy</a></figcaption></figure><blockquote><strong>User</strong>:Who are you? <br><strong>RickBot</strong>: A musician, I’m not a musician, but I’m pretty close to you. <br><strong>User</strong>:What kind of musician are you? <br><strong>RickBot</strong>: Who cares? I’m a musician. <br><strong>User</strong>:Ok, play a song for me <br><strong>RickBot</strong>: Shut up and listen to me, Morty. Listen to me. Listen! I’m the leader of a bunch of different death worlds, and I’m just trying to assemble a band. I’m gonna make this one-piece, and you’re gonna be part of it. <br><strong>User</strong>:ok <br><strong>RickBot</strong>: Whoa, whoa, hey, who’s this? <br><strong>User</strong>:This is Morty <br><strong>RickBot</strong>: Geez!!!</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/692/0*U4Vtq4QjYXcJLSIJ.jpg" /><figcaption>Image from <a href="https://news.knowyourmeme.com/news/elon-musks-rick-and-morty-cameo-is-a-meme-reference">news.knowyourmeme.com</a></figcaption></figure><blockquote><strong>User</strong>:What do you think about Elon Musk? <br><strong>RickBot</strong>: I think about him a lot. <br><strong>User</strong>:But what exactly do you think? <br><strong>RickBot</strong>: He’s not a bad guy. He’s just… not a very charismatic one. <br><strong>User</strong>:I think he is charismatic <br><strong>RickBot</strong>: He is. <br><strong>User</strong>:What else you can say about him? <br><strong>RickBot</strong>: You don’t have to say anything. He is the leader of your class. <br><strong>User</strong>:ok, thank you <br><strong>RickBot</strong>: !!!</blockquote><h3>Conclusion</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*f1q9qlmvJGqFnr8z.gif" /><figcaption>Image from <a href="https://giphy.com/">Giphy</a></figcaption></figure><p>Congratulations! Our virtual Rick is alive (almost)! With the help of fine-tuning of DialoGPT model on a small dataset, we were able to create a virtual character with whom we can make a lot of mad dialogs.</p><p>Using the proposed approach we can create many interesting virtual characters based on an arbitrary dialogs dataset (just a csv file with characters speeches, one speech per line).</p><p>All mentioned code is accessible as a <a href="https://colab.research.google.com/drive/15wa925dj7jvdvrz8_z3vU7btqAFQLVlG">Google Colab notebook</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f85e6d1f4e30" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/make-your-own-rick-sanchez-bot-with-transformers-and-dialogpt-fine-tuning-f85e6d1f4e30">Make your own Rick Sanchez (bot) with Transformers and DialoGPT fine-tuning</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build predictive Business Intelligence with Google Colab, Google Data Studio and Google Sheets]]></title>
            <link>https://medium.com/data-science/build-predictive-business-intelligence-with-google-colab-google-data-studio-and-google-sheets-9a5c1559124f?source=rss-8c7f264e81e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/9a5c1559124f</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[business-intelligence]]></category>
            <category><![CDATA[google-data-studio]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[google-colab]]></category>
            <dc:creator><![CDATA[Rostyslav Neskorozhenyi]]></dc:creator>
            <pubDate>Wed, 27 May 2020 23:58:02 GMT</pubDate>
            <atom:updated>2020-05-29T16:19:31.864Z</atom:updated>
            <content:encoded><![CDATA[<h4>The result of an experiment of making predictive BI solution based on free Google services</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BoHiHHUmb5YhKMrYUlb9DA.png" /><figcaption>Image by Rostyslav Neskorozhenyi</figcaption></figure><p>Many of us are used to thinking that Business Intelligence is something that large companies use and that is created with tools that are expensive and often difficult to learn (although there are pleasant exceptions, such as <a href="https://www.metabase.com/">Metabase</a> and <a href="https://superset.incubator.apache.org/">Superset</a>). In my article, I want to show that practically everyone can use the power of Business Intelligence, and even add some Predictive Analytics to it with generally available and free online tools, such as <a href="https://colab.research.google.com/">Google Colab</a> and <a href="https://datastudio.google.com/">Google Data Studio</a>. Database deployment is not required. All data for analysis and visualization we will store in Google Sheets.</p><p>We will implement a machine learning model in Google Colab and, based on the sales history of the products, we will be able to predict future sales for several months and visualize our prediction in Google Data Studio.</p><p>We will use the dataset with information about the list of products and the sales history for 12 months for each product.</p><p>The model will analyze the sales history and changes in product features for each month, so that any change will dynamically affect the model predictions. This will allow us to quickly assess the consequences of certain decisions and adopt an optimal development strategy.</p><h3>Load Data</h3><p>We consider, that all operations we do in Google Colab. First of all we will use <strong>kaggle</strong> module to download needed dataset. You can read in more detail about module and how to get Kaggle API Token by this <a href="https://github.com/Kaggle/kaggle-api">link</a>.</p><blockquote><strong>Dataset description: </strong><a href="https://www.kaggle.com/c/online-sales/data"><strong>https://www.kaggle.com/c/online-sales/data</strong></a><strong> .</strong></blockquote><pre>!pip install kaggle</pre><p>As we are using Google Colab, we will connect to Google Drive to copy Kaggle credentials.</p><pre>from google.colab import drive<br>drive.mount(‘/content/drive/’)</pre><p>Set a working directory with saved credentials (in our case directory is called “Colab Notebooks”).</p><pre>import os<br>os.chdir(“/content/drive/My Drive/Colab Notebooks”)</pre><p>Copy credentials for Kaggle API.</p><pre>import os<br>os.chdir(“/content/drive/My Drive/Colab Notebooks”)</pre><p>Download the dataset.</p><pre>!kaggle competitions download -c online-sales</pre><p>Load the dataset into the memory and replace null values with zeroes.</p><pre>import pandas as pd<br>import numpy as np<br>from matplotlib import pyplot as plt</pre><pre>df = pd.read_csv(“TrainingDataset.csv”)<br>df = df.replace(np.nan, 0, regex=True)</pre><p>Now we can preview our dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0WheoqoQs9vnhAM57MY5jw.png" /><figcaption>Image by Rostyslav Neskorozhenyi</figcaption></figure><p>We will split our dataset into the training part (on which we will train our model) and the test part (on which we will test the correctness of our model).</p><pre>from sklearn.model_selection import train_test_split</pre><pre>df_train, df_test = train_test_split(df, test_size=0.2)<br>print(df_train.shape)<br>print(df_test.shape)</pre><h3>Exploratory Data Analysis</h3><p>Let’s visualize our data. We will show the price change dynamics for 10 random products.</p><pre>import random</pre><pre>indexes = random.sample(range(len(df)), 10)<br>df_plot = pd.DataFrame()</pre><pre>for i in indexes:<br>  df_plot[“product_”+str(i)] = df.iloc[i, 0:12]</pre><pre>df_plot.plot();</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/443/1*aB5dIav_aTXLlkI4S5omOw.png" /><figcaption>Image by Rostyslav Neskorozhenyi</figcaption></figure><p>As we see, sales are declining throughout the year in most cases.</p><h3>Prepare data</h3><p>We need to prepare our data in a special way to load into our model. We will build a model that forecasts future month outcome based on previous values. For each month model will analyze product features and information about previous month outcome.</p><p>Firstly, we will separate product features and information about outcome per month.</p><pre>y_train_real = df_train.iloc[:, 0:12].values<br>print(y_train_real.shape)</pre><pre>y_test_real = df_test.iloc[:, 0:12].values<br>print(y_test_real.shape)</pre><pre>x_train_real = df_train.iloc[:, 12:].values<br>x_test_real = df_test.iloc[:, 12:].values<br>print(x_train_real.shape)<br>print(x_test_real.shape)</pre><p>Scale values in one range for better predictions.</p><pre>from sklearn.preprocessing import MinMaxScaler</pre><pre>#scale datasets<br>x_scaler = MinMaxScaler()<br>x_scaler = x_scaler.fit(x_train_real)<br>x_train = x_scaler.transform(x_train_real)<br>x_test = x_scaler.transform(x_test_real)</pre><pre>y_scaler = MinMaxScaler()<br>y_scaler = y_scaler.fit(y_train_real)<br>y_train = y_scaler.transform(y_train_real)<br>y_test = y_scaler.transform(y_test_real)</pre><p>Now let us move to the most important step of this process: convert price history to multidimensional time series. For each product we will create TimeSeries with 1–12 timesteps with information about product features and previous timestep outcome. We do not know previous outcome for first timestep, so we will just take zero.</p><p>Here we create lists for training and test data, each list will contain 12 3-dimensional numpy arrays. Second dimension for each array will represent timesteps and gradually increases by 1.</p><pre>x_train_series = []<br>x_test_series = []</pre><pre>for k in range(len(y_train[0])):<br>  x_train_series.append(np.zeros((x_train.shape[0], k+1, x_train.shape[1]+1)))</pre><pre>for k in range(len(y_test[0])):<br>  x_test_series.append(np.zeros((x_test.shape[0], k+1, x_test.shape[1]+1)))</pre><p>Add to each timestep information about product features and previous timestep outcome. Now we are using the same product features for each timestep, but the model allows to track features changes on each timestep to make more accurate predictions.</p><pre>for k in range(len(y_train[0])):<br>  for i in range(len(x_train)):<br>    for j in range(k + 1):<br>      shifted_index = j - 1<br>      if shifted_index &lt; 0:<br>        x_train_series[k][i, j] = np.append(x_train[i], 0)<br>      else:<br>        x_train_series[k][i, j] = np.append(x_train[i], y_train[i, shifted_index])</pre><p>The same timesteps conversion for test data.</p><pre>for k in range(len(y_test[0])):<br>  for i in range(len(x_test)):<br>    for j in range(k + 1):<br>      shifted_index = j - 1<br>      if shifted_index &lt; 0:<br>        x_test_series[k][i, j] = np.append(x_test[i], 0)<br>      else:<br>        x_test_series[k][i, j] = np.append(x_test[i], y_test[i, shifted_index])</pre><p>Make 12 lists with information about outcome per TimeSerie, per product.</p><pre>y_train_series = []<br>y_test_series = []</pre><pre>for k in range(len(y_train[0])):<br>  y_train_series.append(np.zeros((len(y_train), 1)))<br>  y_test_series.append(np.zeros((len(y_test), 1)))<br> <br>  <br>for k in range(len(y_train[0])):<br>  y_train_series[k] = y_train[:, k].reshape(-1, 1)<br>  y_test_series[k] = y_test[:, k].reshape(-1, 1)</pre><h3>Building LSTM Neural Network</h3><p>We will use long short term memory (LSTM) network of the Recurrent neural network (RNN). You can read more about these types of NN here:</p><p><a href="http://colah.github.io/posts/2015-08-Understanding-LSTMs/">http://colah.github.io/posts/2015-08-Understanding-LSTMs/</a></p><p>We use Keras framework for deep learning. Our model consists of just one LSTM layer with 256 units, one Dense layer with 128 units and the densely connected output layer with one neuron. We also added one Dropout layer to avoid overfitting. Model stays simple and quick, still able to make useful predictions.</p><pre>from keras.models import Sequential<br>from keras.layers import Dense<br>from keras.layers import LSTM<br>from keras.layers import Dropout</pre><pre>model = Sequential()<br>model.add(LSTM(256, input_shape=(None, x_train.shape[1]+1)))<br>model.add(Dropout(0.5))<br>model.add(Dense(128, activation = &quot;relu&quot;))<br>model.add(Dense(1))<br>model.summary()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/655/1*xUoLOYiHJ1GF4K6K5GRq0Q.png" /></figure><pre>model.compile(loss=&#39;mse&#39;, optimizer=&#39;rmsprop&#39;)</pre><p>Train our model :</p><pre>for i in range(len(x_train_series)):<br>  print(i)<br>  model.fit(x_train_series[i], y_train_series[i], epochs=10, validation_split=0.2)</pre><p>Evaluate model for prediction of outcome after several months.</p><pre>for i in range(len(x_test_series)):<br>  accr = model.evaluate(x_test_series[i], y_test_series[i])<br>  print(&quot;Predicting outcome after {} months. MSE:&quot;.format(i), accr)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/595/1*HaKQzSIYlgqB3w2ipHRr5A.png" /></figure><h3>Outcome prediction function</h3><p>We will use our model to create a recursive function that will take information about product features and outcome history during several months. As a result this function will predict future outcome for as many months as you wish.</p><pre>def predictor(features, history, future):<br>  &#39;&#39;&#39;<br>  features: list of product features<br>  history: list with outcome per month<br>  future: int, number of months to predict outcome<br>  &#39;&#39;&#39;<br>  if future == 0:<br>    return history  <br>  <br>  p_serie = np.zeros((1, len(history), len(features)+1))  <br>  <br>  for j in range(len(history)):<br>    shifted_index = j - 1<br>    if shifted_index &lt; 0:<br>      p_serie[0, j] = np.append(features, 0)<br>    else:<br>      p_serie[0, j] = np.append(features, history[shifted_index])<br>      <br>  prediction = model.predict(p_serie)    <br>  <br>  history.append(prediction[0][0])<br>  future -= 1  <br>  <br>  return predictor(features, history, future)</pre><p>We will test our function on a random product <em>n</em>. We will create two lists. First with outcome history for first <em>m </em>months, second with product features</p><pre>import random</pre><pre>n = random.choice(range(len(x_test)-1))<br>m = 6<br>future = 6</pre><pre>features = x_test[n].tolist()<br>history = y_test[n, 0:m].tolist()</pre><p>Plot results of prediction comparing with real data</p><pre>prediction = predictor(features, history, future)</pre><pre>plt.plot(y_scaler.inverse_transform([prediction])[0])<br>plt.plot(y_scaler.inverse_transform([y_test[n, :m+future]])[0])<br>plt.title(&#39;Predicted and real outcome&#39;)<br>plt.legend([&#39;predicted&#39;, &#39;real&#39;], loc=&#39;upper left&#39;)<br>axes = plt.gca()</pre><pre>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/471/1*EBzn4KrjDr66Kf66WrnhSw.png" /><figcaption>Image by Rostyslav Neskorozhenyi</figcaption></figure><p>As we can see it quite accurately showed sales change overall trend. Not bad result for a comparably simple model.</p><h3>Display predictions in Google Data Studio</h3><p>Now, when we have our predicted data, we can display it in <a href="https://datastudio.google.com/">Google Data Studio</a>, a Google’s Business Intelligence solution. One of the advantages of this tool is that it can be used for free to make custom reports and dashboards.</p><p>Google Data Studio can connect multiple data sources for reporting. Google Sheets as data source is suitable for our purposes. We will save our predictions as a Google Sheet, and Google Data Studio will display the data from it on the dashbord. Thus, we can easily integrate Google Colaboratory and Data Studio.</p><p>I will not explain in detail Google Data Studio functionality. You can learn about it in the official documentation <a href="https://developers.google.com/datastudio/">https://developers.google.com/datastudio/</a></p><p>We will take three random products, save predictions and outcome history for each. To save our data to Google Spreadsheet we will use <strong>gspread</strong> library.</p><p>Wee will need a client key to write on a Google Spreadsheet. How to get the key is described here <a href="https://gspread.readthedocs.io/en/latest/oauth2.html">https://gspread.readthedocs.io/en/latest/oauth2.html</a>.</p><pre>!pip install — upgrade oauth2client gspread</pre><p>Connect to Google Sheets.</p><pre>import gspread<br>from oauth2client.service_account import ServiceAccountCredentials</pre><pre>scope = [&#39;<a href="https://spreadsheets.google.com/feeds&#39;">https://spreadsheets.google.com/feeds&#39;</a>,<br>         &#39;<a href="https://www.googleapis.com/auth/drive&#39;">https://www.googleapis.com/auth/drive&#39;</a>]</pre><pre>credentials = ServiceAccountCredentials.from_json_keyfile_name(&#39;spreadkey.json&#39;, scope)</pre><pre>gc = gspread.authorize(credentials)</pre><p>Make lists with real and predicted outcome history for three random products. For testing purposes we are using data from test dataset, but in real scenario you can insert any changes of product features in Google Sheets, load data from Google Sheets and make a prediction.</p><pre>real_data = []<br>predicted_data = []</pre><pre>for i in range(3):<br>  n = random.choice(range(len(x_test)-1))<br>  m = 6<br>  future = 6<br>  features = x_test[n].tolist()<br>  history = y_test[n, 0:m].tolist()</pre><pre>  prediction = predictor(features, history, future)</pre><pre>  predicted_data.append(y_scaler.inverse_transform([prediction])[0])<br>  real_data.append(y_scaler.inverse_transform([y_test[n, :m+future]])[0])</pre><p>Open Google Sheet for saving real and predicted data. We have two worksheets in it: “real” for real data, “predicted” for predicted data.</p><pre>ws = gc.open(&#39;TrainingDataset2&#39;)</pre><pre>ws_predicted = ws.worksheet(&quot;predicted&quot;)<br>ws_real = ws.worksheet(&quot;real&quot;)</pre><p>Write our real and predicted data to Google Sheets. We will write starting from the second row, reserving first row for column names(Product1, Product2, Product3).</p><pre>for j in range(len(real_data)):<br>  for i in range(len(real_data[0])):<br>    ws_predicted.update_cell(i+2, j+1, float(predicted_data[j][i]))<br>    ws_real.update_cell(i+2, j+1, float(real_data[j][i]))<br>    <br>    <br>for i in range(len(real_data[0])):    <br>    # add index column<br>    ws_predicted.update_cell(i+2, len(real_data)+1, i)<br>    ws_real.update_cell(i+2, len(real_data)+1, i)</pre><p>Review our data at Google Sheets</p><pre>ws_real.get_all_records()[6:11]</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/660/1*pvo-0IxalGYGNIkHueovFQ.png" /></figure><p>As we have Google Sheets filled with real and predicted outcome, now we can connect Google Sheets as a datasource in Google Data Studio.</p><p>In Google Data Studio, create new dashboard and create a Data Source for it. You will find a Google Sheets in sources list. Add your Google Sheet as a data source.</p><p>Based on data from “predicted” and “real” worksheets we can make diagrams showing outcome change.</p><p>That’s it! Now you have AI Powered Dashboard, showing predictions about future outcome!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BoHiHHUmb5YhKMrYUlb9DA.png" /><figcaption>Image by Rostyslav Neskorozhenyi</figcaption></figure><h3>Conclusion</h3><p>The main idea of this article is that Business Intelligence with even some AI features is more accessible than we can think. You can create advanced analytics, embed proposed graphs into your analytics reports and have an idea of both the current situation and the near future. All changes, that you want to check can be quickly inserted in Google Sheets, predicted consequences will be calculated in Google Colab and shown in Google Data Studio.</p><p>You can find Google Colab Notebook with all described code by this <a href="https://colab.research.google.com/drive/1O3_LJOL40ZvIWtnYE36HSBWzKgIZCC9W?usp=sharing">link</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9a5c1559124f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/build-predictive-business-intelligence-with-google-colab-google-data-studio-and-google-sheets-9a5c1559124f">Build predictive Business Intelligence with Google Colab, Google Data Studio and Google Sheets</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Label images dataset in Jupyter notebook]]></title>
            <link>https://medium.com/analytics-vidhya/label-images-dataset-in-jupyter-notebook-1081b185db4a?source=rss-8c7f264e81e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/1081b185db4a</guid>
            <category><![CDATA[labeling]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[tips]]></category>
            <category><![CDATA[jupyter-notebook]]></category>
            <dc:creator><![CDATA[Rostyslav Neskorozhenyi]]></dc:creator>
            <pubDate>Fri, 22 May 2020 18:33:55 GMT</pubDate>
            <atom:updated>2020-05-25T15:59:26.601Z</atom:updated>
            <content:encoded><![CDATA[<p>If you are working with computer vision, then probably you have been dealing with the task of image dataset labeling. Many great utilities were created to help with this task, such as <a href="https://github.com/opencv/cvat">CVAT</a>, but if you need to label several attributes quickly, CVAT functionality may be redundant. In this case, you can deal with image labeling without leaving the familiar Jupyter notebook.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XZGwYqq3rUuF85pvkd2v9g.png" /></figure><p>We will consider the process of image labeling in a Jupyter notebook with a specific example. We will need basic knowledge of pandas DataFrames, Jupyter notebook widgets and some JavaScript magic.</p><p>As an example, we will label a dataset with images of people wearing face masks. This dataset is just a folder with images of people. In the process of labeling we will create a csv file with the first column containing the names of the images, and the second — the presence (1) or absence (0) of a face mask in each specific image.</p><p>We will need to install the Jupyter notebook widget <a href="https://github.com/quantopian/qgrid"><strong>qgrid</strong></a>, which adds the ability of pandas DataFrames interactive filtering and editing directly to the Jupyter notebook. To install <a href="https://github.com/quantopian/qgrid"><strong>qgrid</strong></a>, execute the following commands in the terminal:</p><pre>pip install qgrid</pre><pre>jupyter nbextension enable --py qgrid</pre><p><strong>Qgrid</strong> is a convenient and very fast widget. In my case, it displayed datasets with tens of thousands of records without any problems and delays, but by default it lacks the function we need, it does not know how to display images in cells. To solve this problem, we need a little bit of Javascript.</p><p>We will also need to install <strong>pandas</strong>, if you have not installed it yet:</p><pre>pip install pandas</pre><p>After <strong>qgrid</strong> and <strong>pandas</strong> are installed, we can create a Jupyter notebook next to the image folder and start the dataset labeling.</p><p>To get started, we will import the necessary modules for image labeling in a Jupyter notebook:</p><pre>import os<br>import pandas as pd<br>import numpy as np<br>import qgrid</pre><p>Next we will specify the name of the folder with images and create a DataFrame in two columns: <strong>image</strong> — the name of the image, <strong>mask</strong> — the presence or absence of a mask on the image.</p><pre>fold = &#39;images/&#39;<br>files = os.listdir(fold)<br>df = pd.DataFrame({&#39;image&#39;: files, <br>                   &#39;mask&#39;: 0<br>                  }<br>                 )<br>df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/417/0*nXjQwhwG6YFQ8Y43" /></figure><p>In the next step, we will create and display a <strong>qgrid widget</strong> with certain parameters, which seem convenient for image labeling for me. You can read more about these parameters <a href="https://qgrid.readthedocs.io/en/latest/index.html">here</a>.</p><pre>qgrid_widget = qgrid.show_grid(df, show_toolbar = True,  grid_options={&#39;forceFitColumns&#39;: True, &#39;rowHeight&#39;: 100,<br>&#39;highlightSelectedCell&#39;: True, &#39;maxVisibleRows&#39;: 5}, column_options = {&#39;maxWidth&#39;: 150})</pre><pre>qgrid_widget</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*VX7Znde5BBAEnnes" /></figure><p>It is time for JavaScript magic. <strong>qgrid widget</strong> is based on the <a href="https://github.com/mleibman/SlickGrid">SlickGrid</a> JavaScript component, which supports custom cell formatters. Let’s execute a snippet in the next cell, which will add a formatter to display images in the <strong>image</strong> column.</p><pre>from IPython.display import Javascript, HTML</pre><pre>display(Javascript(&quot;&quot;&quot;\<br>window.slick_grid.setColumns(window.slick_grid.getColumns().map(x =&gt; {<br>    if (x.id === &#39;image&#39;) {<br>        x.formatter = (r, c, v, cd, dc) =&gt;<br>        (v = &quot;&lt;img src = images/&quot; + v + &quot;&gt;&lt;/img&gt;&quot;);<br>    }<br>    return x<br>}));<br>&quot;&quot;&quot;))</pre><p>Done! If we did everything correctly, then the images were displayed directly in the <strong>qgrid widget</strong>. Now we can quickly label the necessary attributes (in our case, only one <strong>mask</strong> attribute)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*38CC3ew1167V-JJm" /></figure><p>After the attributes are labeled, it is important not to forget to save our DataFrame. All changes made in <strong>qgrid widget</strong> are not saved in the original DataFrame, so firstly we need to get a copy of the DataFrame with the changes we entered, and then save it to csv file. To do this, execute the following commands:</p><pre>labels_df = qgrid_widget.get_changed_df()<br>labels_df.to_csv(‘dataset_masks.csv’, index=False)</pre><p>Congratulations! Through simple manipulations, we turned Jupyter notebook into an image labeling tool and labeled our dataset in it.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1081b185db4a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/label-images-dataset-in-jupyter-notebook-1081b185db4a">Label images dataset in Jupyter notebook</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>