<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Reza Dwi Utomo on Medium]]></title>
        <description><![CDATA[Stories by Reza Dwi Utomo on Medium]]></description>
        <link>https://medium.com/@utomorezadwi?source=rss-fadfbe6f0eaa------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*Vvne45Vz1zbRwpXmHazphA.jpeg</url>
            <title>Stories by Reza Dwi Utomo on Medium</title>
            <link>https://medium.com/@utomorezadwi?source=rss-fadfbe6f0eaa------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 02:43:24 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@utomorezadwi/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How to utilize ChatGPT to succeed in your job applications]]></title>
            <link>https://utomorezadwi.medium.com/how-to-utilize-chatgpt-to-succeed-in-your-job-applications-1a2ab57bc103?source=rss-fadfbe6f0eaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/1a2ab57bc103</guid>
            <category><![CDATA[layoffs]]></category>
            <category><![CDATA[jobs]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Reza Dwi Utomo]]></dc:creator>
            <pubDate>Mon, 02 Jan 2023 04:15:23 GMT</pubDate>
            <atom:updated>2023-01-03T03:53:37.111Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*cTn4Cz4yHywLpUef" /><figcaption>Photo by <a href="https://unsplash.com/@askkell?utm_source=medium&amp;utm_medium=referral">Andy Kelly</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>The year 2022 is really tough. Many tech companies decided to lay off their employees. According to <a href="https://layoffs.fyi/">layoff.fyi</a>, in 2022 alone, there are around 153,110 employees who lost their job in the world.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*4nxzhkev8GJZ4LEu.jpg" /><figcaption><a href="https://www.visualcapitalist.com/visualizing-tech-company-layoffs-in-2022/">Source</a></figcaption></figure><p>On the other hand, <a href="https://openai.com/blog/chatgpt/">ChatGPT</a>, a special chatbot developed by <a href="https://openai.com/">OpenAI</a>, recently mesmerizes the world. Let me clearly explain what ChatGPT is by asking directly to ChatGPT itself as shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/798/1*oPpm7c9jNcLXqAW-yntZtw.png" /><figcaption>ChatGPT explains itself.</figcaption></figure><p>Many have played with this AI product. For example, <a href="https://twitter.com/volodarik/status/1600854935515844610?s=20&amp;t=KcLWr0F8CFJmaiL4WnCqjA">Volodarsky mentioned there are at least 10 wonderful tasks that can be executed by ChatGPT</a>:</p><ol><li>Generating codes</li><li>Tracking fitness</li><li>Debugging codes</li><li>Serving as a personal assistant</li><li>Creating a marketing plan</li><li>Creating a virtual machine</li><li>Creating creative works</li><li>Developing plugins</li><li>Developing game</li><li>Suggesting medical aids</li></ol><p>Really cool, isn’t it?</p><p>Okay. I think that ChatGPT can also assist me in doing the common tasks of a job seeker, such as writing a resume and cover letter and proofreading them, helping me learn a technical test and user interviews, providing a mock-up interview, etc. And below are what I can do to utilize ChatGPT in doing the job.</p><h3>Improving my existing resume</h3><p>I split my resume into several sections, each of which is inputted into the ChatGPT prompt. This is because I always experience connection errors every time I instruct ChatGPT to improve my resume all at once.</p><p>For example, in the images below, I asked ChatGPT to improve the work experience section, proofread it, and rewrite it. I also asked it to use British English.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*_VO5S4ZI2UU4CNNRVe9LTA.png" /></figure><p>And the ChatGPT’s response is shown below. It knows what “PIC” stands for. It combines two bullets into a single bullet point. But, unfortunately, it uses the word “analyze” which is indeed not a British English style.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/1*PkwxOVJV_HA3ZlcE02EeZg.png" /></figure><h3>Improving my profile summary</h3><p>I also asked ChatGPT to improve my profile summary. We can see in the image below that it nicely improves my profile summary. It knows properly how to use English grammar.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/779/1*mqxV-36SrJAh7klzOZs-1g.png" /></figure><h3>Writing a cover letter</h3><p>Sometimes, when applying for vacancies, you have to write a cover letter dedicated to a certain vacancy. Previously, I used to have a template for my cover letter. Based on the template, I would write a dedicated cover letter for a certain vacancy. Occasionally, this routine is exhausting.</p><p>But, now, by using ChatGPT, you just input an instruction which includes what you want (i.e. to write a cover letter), the job descriptions and requirements, and your resume. In the image below, I used the character “&gt;&gt;&gt;” in order for ChatGPT to easily understand what my instruction is.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/683/1*rHyoBv_pWlEI8ozpJI0Urw.png" /></figure><p>And the ChatGPT’s response is shown below. You can see that ChatGPT sweetly design your dedicated cover letter for the vacancy. If you still don’t satisfy with the response, you could generate another response or improve the result yourself.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/612/1*JyiRQVk36WCtFNdP7jlfLA.png" /></figure><h3>Helping learn the technical test questions</h3><p>Usually, in recruiting tech talents, employers will assess the technical knowledge of candidates in a technical test. Often time, the test is a coding test. It could be a take-home assignment, an online coding test, or a live coding test with a proctor. For online coding tests, you can learn the questions on <a href="https://www.hackerrank.com/">HackerRank</a>, <a href="https://mettl.com/id-id/">Mettl</a>, <a href="https://www.codility.com/">Codility</a>, <a href="https://coderbyte.com/">Coderbyte</a>, etc. They provide numerous questions you can learn from.</p><p>I tried to use ChatGPT to solve the coding question from one of the websites above. I started with an easy problem and ChatGPT can solve the question.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/783/1*Y8L1oLXeGnqpCesM401T0Q.png" /></figure><p>Subsequently, I continued with a hard problem as shown below. Anyway, even I spent much time solving the problem by myself lol. In the first generation, ChatGPT’s solution was completely wrong, and then I clicked the “regenerate response” button. Just in the second generation, ChatGPT is able to deliver a proper solution. Wonderful!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/526/1*wmC2DyL5SgIkldXh1Fni_w.png" /></figure><p>ChatGPT can help you learn how to solve difficult questions. It also provides you with some explanations of how it comes to the solution. During your learning time to master a number of examples of technical questions, ChatGPT can be your partner in quickly understanding the problem context.</p><blockquote><strong>But, ensure that you UNDERSTAND what ChatGPT suggests on its solution and are ABLE TO REPRODUCE the solution using your own ‘sentence’ since you must use your own capability in the real technical test.</strong></blockquote><h3>Giving suggestions in technical user interviews</h3><p>In a user interview, you can expect many technical questions will be asked. Therefore, it is always a good idea to anticipate to any technical and non-technical questions. Luckily, ChatGPT knows this :)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/701/1*3kSxBRPjyrmHmrFbgTzZLw.png" /></figure><p>Let’s elaborate on each question that ChatGPT suggested.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/625/1*-J41-DQ-JpTlQJp3-mfQ5g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/626/1*qIlB7ARSqL7CyRL07PZ25Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/626/1*YjIfKdRaN_9gGt8ibofU9A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/624/1*svRel29T972B4VrMDW4iYw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/623/1*27A_KamgBnVxsS-8Zi86Zg.png" /></figure><p>And below is my own question. I was once asked this question during a user interview. Let’s see what it will answer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/623/1*--LYRHCDQ2ygj8ZT7oWpKA.png" /></figure><p>And below are the less technical questions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/625/1*JTKL-GUhQLqgb6mEzYj42w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/628/1*5di0Fn6_KlNARthy3xnEdg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/626/1*fYaI3GDuULykUS4Zzh1bnA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*Sdlz0kk1BigF6hmuO-lD_w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/622/1*hMMRyEZCGwWAdrQvJbVuEg.png" /></figure><p>ChatGPT is indeed awesome. It can answer all interview questions perfectly. If we want more details, just give it a prompt, I’m sure it will give another precise answer.</p><p>However, there are funny some question-answer pairs. ChatGPT confidently answers like it experienced the event itself. It is really confident even though it is wrong and makes up a story or answer.</p><h3>Providing a mock-up interview</h3><p>In addition to giving us some examples of user interview questions, ChatGPT can also act as an interviewer so that we can learn how to give good answers in an interview. Just instruct it clearly what we want and it will turn itself into a professional interviewer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jp0yAUkvLKr7owXMUejp7w.png" /></figure><p>But, there’s one thing really bothering me. Since on the ChatGPT webpage we have to type to input our prompts, I’m sure it feels annoying and slow if we have to type every time we answer the question ChatGPT asks. Fortunately, there is an open-source Chrome extension that can help us to use speech recognition in communicating with ChatGPT. Just simply follow the instructions in the GitHub repo below to install the extension.</p><p><a href="https://github.com/C-Nedelcu/talk-to-chatgpt">GitHub - C-Nedelcu/talk-to-chatgpt: Talk to ChatGPT AI using your voice and listen to its answers through a voice</a></p><p>After the extension has been installed, just use it on the ChatGPT page as shown below, then start talking. But, as the extension is just a fun, open-source project, you can’t expect a sophisticated tool. Regardless of I’m not a native speaker, the tool often wrongly captures what I said. However, for a learning tool, this is more than enough.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WVVnbv12sA4Yj0bR8RnN4w.png" /></figure><h3>Conclusions</h3><p>Although massive layoff is occurring in the world due to unstable global macroeconomics, we must survive and strive to obtain a new home. We have discussed how we can utilize ChatGPT in order to assist us in seeking job vacancies.</p><p>At least, ChatGPT is able to help us with the tasks below:</p><ul><li>Improving an existing resume</li><li>Improving a profile summary</li><li>Writing a cover letter</li><li>Helping learn the technical test questions</li><li>Giving suggestions in technical user interviews</li><li>Providing a mock-up interview</li></ul><p>If you have any other use cases in which ChatGPT helps you get your dream job, please let me know in the comment below :)</p><p>On the other hand, there are some concerns, however, where we are not allowed to use ChatGPT’s response innocently. ChatGPT is awesome, but it is still a prototype. We can’t trust its prompt results without double check. For example, it still doesn’t know the difference between American and British English (I’m completely sure Grammarly is much more expert in this issue); we have to cross-check its coding results; it sometimes makes up an answer or story with high-degree confidence; and so on.</p><p>Nevertheless, I’m 100% optimistic about the future of ChatGPT. It will assist us, humans, in our everyday lives.</p><h3>About the Author</h3><p><a href="https://www.linkedin.com/in/utomorezadwi/">Reza</a> is an engineer specializing in data-driven analysis. Previously, he worked as Senior Data Scientist at Tokopedia. In his spare time, he likes to write articles on <a href="https://medium.com/@utomorezadwi">Medium</a>, learn something new, or <a href="https://github.com/utomoreza">create side projects</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1a2ab57bc103" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying GPU-based Models on SageMaker using ‘Multi-Model’ Endpoint (Part 2 - Final)]]></title>
            <link>https://medium.com/geekculture/deploying-gpu-based-models-on-sagemaker-using-multi-model-endpoint-part-2-final-6e05cf10142f?source=rss-fadfbe6f0eaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/6e05cf10142f</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[gpu]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[pytorch]]></category>
            <category><![CDATA[sagemaker]]></category>
            <dc:creator><![CDATA[Reza Dwi Utomo]]></dc:creator>
            <pubDate>Mon, 24 Oct 2022 15:13:56 GMT</pubDate>
            <atom:updated>2022-11-17T03:17:22.526Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*4oMxmoB7ptyLyQlx" /><figcaption>Photo by <a href="https://unsplash.com/@ourselp?utm_source=medium&amp;utm_medium=referral">Philippe Oursel</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><blockquote>This article is the continuation of the previous article (Part 1). If you happen to arrive at this article, you are highly recommended to read the Part 1 article first below.</blockquote><blockquote><strong>UPDATE 17/11/2022:</strong> I’ve got info from <a href="https://www.linkedin.com/feed/update/urn:li:activity:6990336296409858048?commentUrn=urn%3Ali%3Acomment%3A%28activity%3A6990336296409858048%2C6998715078984302592%29&amp;dashCommentUrn=urn%3Ali%3Afsd_comment%3A%286998715078984302592%2Curn%3Ali%3Aactivity%3A6990336296409858048%29">an AWS SageMaker employee</a> that finally AWS has released Multi-Model Endpoint GPU on SageMaker. Cool! You can read the article <a href="https://aws.amazon.com/about-aws/whats-new/2022/10/amazon-sagemaker-cost-effectively-host-1000s-gpu-multi-model-endpoint/"><strong>here</strong></a>.</blockquote><p><a href="https://utomorezadwi.medium.com/deploying-gpu-based-models-on-sagemaker-using-multi-model-endpoint-part-1-da68cbbf3d04">Deploying GPU-based Models on SageMaker using ‘Multi-Model’ Endpoint (Part 1)</a></p><blockquote><strong>UPDATE 26/10/2022:</strong> I’ve just found <a href="https://developer.nvidia.com/blog/run-multiple-ai-models-on-same-gpu-with-sagemaker-mme-powered-by-triton/"><strong>an article from NVIDIA</strong></a> mentioning running GPU-based models on the multi-model endpoint on Amazon SageMaker using NVIDIA Triton Inference Server. I haven’t checked this feature myself yet, but if this is true, this is a game changer.</blockquote><h4>Some summaries from the previous article</h4><ul><li>We have discussed the options available to optimise the cost of using SageMaker inference.</li><li>We have prepared all needs to have this kind of ‘multi-model’ endpoint.</li><li>We have trained two sample models (one PyTorch model and one TensorFlow model) and tested their prediction as well.</li><li>We have discussed the directory structure needed for the SageMaker inference endpoint and the proposed directory structure for our case.</li><li>We have discussed four main functions inside the inference.py file.</li></ul><h3>Table of contents</h3><p>This Part 2 article is arranged as follows:</p><ul><li>Deployment<br>- Debugging the inference.py script<br>- Uploading the model.tar.gz file to the S3 bucket<br>- Deploying using SageMaker</li><li>Invoking Endpoint<br>- Using Predictor class<br>- Using .invoke_endpoint() method<br>- Deleting endpoint</li><li>Constraints</li><li>Conclusions</li></ul><h3>Deployment</h3><h4>Debugging the inference.py script</h4><p>Now, we’ve written the inference.py file along with the requirements.txt file and put the necessary files and folders of both models into the main directory. Prior to compressing the main directory to a model.tar.gz file, it is recommended to debug the inference.py file in advance in order to minimise any unwanted errors and side effects after the model has been deployed.</p><p>To ease this process, we will copy the inference.py into two files: debug_pt.py and debug_tf.py. For each, we will retain all lines of code needed for its model type and delete unnecessary (or just comment) lines of code for its model type. For example, if we’d like to debug the PyTorch model, in the debug_pt.py file, we only keep all codes needed to process the PyTorch model and vice versa. For more details on both files, you can see my files <a href="https://github.com/utomoreza/MultimodelEndpoint_hacked/blob/main/debug_pt.py">here</a> and <a href="https://github.com/utomoreza/MultimodelEndpoint_hacked/blob/main/debug_tf.py">here</a>, respectively.</p><ul><li>Debugging the PyTorch model</li></ul><p>What we’re going to do is basically similar to the <strong>Model Prediction</strong> section (in Part 1). So, we’re going to test all four functions defined earlier (in Part 1). As I explained earlier that we use a copy of inference.py file called debug_pt.py, we just simply import all four functions.</p><p>Afterwards, we prepare the data that will be used to do debug test. The data is then wrapped in the JSON structure in order to mimic the request structure received by the inference server.</p><p>The JSON input, subsequently, can be used as the first argument for the input_fn() function. In this case, I input “model” to the model_fn() function as I named the main directory (that will be the model.tar.gz compressed file) as model/. For the predict_fn() function, we can input (1) the output from input_fn() and (2) the output from model_fn(). Finally, the output from predict_fn() is inputted to the output_fn() function.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2d5f8e1410ee979156fcd2871ce8215f/href">https://medium.com/media/2d5f8e1410ee979156fcd2871ce8215f/href</a></iframe><p>For each function, we can print its value to check whether it works as expected. And below are the print results. We can see that all four functions work as we expect.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BZ35qDJQuehuN3EaecPotQ.png" /><figcaption>Results from debugging PyTorch model</figcaption></figure><ul><li>Debugging the TensorFlow model</li></ul><p>For debugging the TensorFlow model, we use the debug_tf.py file. Basically, the process is similar to that of the PyTorch model, except that for the JSON input, we have to set it as text. Subsequently, we just apply the same as we’ve done for the PyTorch model in each function.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/17491274fd9ce2fab1ca3a5e77eae2a5/href">https://medium.com/media/17491274fd9ce2fab1ca3a5e77eae2a5/href</a></iframe><p>Afterwards, the print results are shown below. All four functions work as we expect.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/950/1*NgcTdUq1CQXcoKti0kzg6A.png" /><figcaption>Results from debugging TensorFlow model</figcaption></figure><h4>Uploading the model.tar.gz file to the S3 bucket</h4><ul><li>Compressing the main directory</li></ul><p>After debugging is clear, the next step is to compress the main directory into model.tar.gz file. We can use <a href="https://linuxhint.com/linux-tar-command/">tar Linux command</a> as follows</p><pre># if using tar command</pre><pre>artifact=&quot;model.tar.gz&quot;<br>main_dir_path=&quot;to/your/main/dir/&quot;<br>cd $main_dir_path<br>tar -zcvf $artifact .</pre><p>or use <a href="https://stackoverflow.com/a/17081026">Python tar code</a> as follows</p><pre># if using Python code</pre><pre>import tarfile<br>import os</pre><pre>def make_tarfile(output_filename, source_dir):<br>    with tarfile.open(output_filename, &quot;w:gz&quot;) as tar:<br>        tar.add(source_dir, arcname=os.path.basename(source_dir))</pre><pre>main_dir_path = &quot;/your/main/dir/path&quot;<br>os.chdir(main_dir_path)</pre><pre>artifact = &quot;model.tar.gz&quot;<br>make_tarfile(artifact, &quot;.&quot;)</pre><p>After compressing process is done, it’s always a good idea to recheck if your compressed file correctly contains all files and folders needed. You can use the following command to check the structure inside the compressed file.</p><pre>tar -tvf $artifact</pre><ul><li>Uploading to an S3 bucket</li></ul><blockquote><strong>Note:</strong> It’s assumed that you have created your own S3 bucket to store the <em>model.tar.gz</em> file and set all necessary IAM policies.</blockquote><p>To upload the compressed file to your S3 bucket, simply you can use the <a href="https://docs.aws.amazon.com/cli/latest/reference/s3/cp.html#examples">cp AWS CLI</a> as follows. Do <a href="https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html">install AWS CLI</a> in your system in advance.</p><pre>bucket_name=&quot;your-bucket-name&quot;<br>key=&quot;model.tar.gz&quot;<br>s3_uri=&quot;s3://$bucket_name/$key&quot;<br>aws s3 cp $artifact $s3_uri</pre><p>Or you can use codes from <a href="https://stackoverflow.com/a/37017853">the </a><a href="https://stackoverflow.com/a/37017853">boto3 library</a> as follows. Do <a href="https://boto3.amazonaws.com/v1/documentation/api/latest/guide/quickstart.html#installation">install </a><a href="https://boto3.amazonaws.com/v1/documentation/api/latest/guide/quickstart.html#installation">boto3 library</a> in your environment in advance.</p><pre>import boto3<br>s3 = boto3.resource(&#39;s3&#39;)<br>s3.meta.client.upload_file(&#39;/path/to/model.tar.gz/in/local&#39;, &#39;your-bucket-name&#39;, &#39;model.tar.gz&#39;)</pre><h4>Deploying using SageMaker</h4><p>We’ve done great so far. There’s only one last step remaining. Now, we’re going to use SageMaker Python SDK. Don’t forget to <a href="https://sagemaker.readthedocs.io/en/stable/v2.html#installation">install it</a> in your environment. Please use the latest version of SageMaker Python SDK. When writing this article, I use the version 2.108.0.</p><p>By default, SageMaker doesn’t support GPU multi-model endpoint yet. So, in order to outsmart SageMaker SDK so that we can have our own GPU ‘multi-model’ endpoint, we have to do the following. When we’re about to deploy the trained models using SageMaker SDK, we have to choose what prebuilt container image will be used since there are many images as listed <a href="https://docs.aws.amazon.com/sagemaker/latest/dg/pre-built-containers-frameworks-deep-learning.html">here</a>. In this case, I chose the PyTorch container image with the PyTorch version 1.10.2 and Python version 3.8. The reason why I chose it is that I had tested the image and it works with my model version. The image already covers the requirement to install the torch library so you don’t need to install one yourself. However, the image will install the tensorflow library as we listed earlier (in Part 1) in our requirements.txt file.</p><blockquote><strong>Note:</strong> you might switch the PyTorch to TensorFlow in which your prebuilt container image is TensorFlow (with certain version), then you list <em>torch</em> library in your <em>requirements.txt</em> file in order for SageMaker SDK to install it. But, do make sure that what you’re gonna do works as expected without any version/environment conflict.</blockquote><p>Next, we just instantiate necessary variables, such as the model_s3_uri (the S3 URI for the uploaded model.tar.gz file), role (the IAM role needed to execute deployment), and sagemaker_session (the SageMaker session needed to execute deployment). As we use the PyTorch container image as the basis, we have to use the PyTorchModel class and set the necessary parameters. Lastly, we can deploy the PyTorchModel object using .deploy() method along with necessary parameters. See the code snippet below for more details.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dd031b2a0a9723fe26b56aba8c0904b8/href">https://medium.com/media/dd031b2a0a9723fe26b56aba8c0904b8/href</a></iframe><p>When we run the codes above, if no error appears, we will see several hyphens printed consecutively until the deployment process is done.</p><h3>Invoking Endpoint</h3><p>Before we invoke the endpoint from outside the AWS network environment, it’s always a good idea to hit the endpoint directly from SageMaker Notebook (from the same notebook as you deployed the models just now). Therefore, we will still use the predictor variable that is defined earlier in the deployment process.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6b378f143c7c437b458a8fb4cfd9c028/href">https://medium.com/media/6b378f143c7c437b458a8fb4cfd9c028/href</a></iframe><p>And below are the results. We can see that the deployed models work as expected.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/738/1*xTwzJLGceSdhRXyA33_Lbg.png" /><figcaption>Results from PyTorchModel.deploy().predict()</figcaption></figure><p>At least, there are two ways we can use to invoke the inference endpoint from outside the AWS network environment: using Predictor class and using .invoke_endpoint() method.</p><h4>Using Predictor class</h4><p>The Predictor class is <a href="https://sagemaker.readthedocs.io/en/stable/api/inference/predictors.html">a class from SageMaker SDK</a>. It is used to make prediction requests to an Amazon SageMaker endpoint.</p><p>First of all, we must define the credentials before making any request. There are many techniques we can use to define the credentials. You can follow the instructions explained on the AWS website below.</p><p><a href="https://boto3.amazonaws.com/v1/documentation/api/1.9.42/guide/configuration.html">Credentials - Boto 3 Docs 1.9.42 documentation</a></p><p>The code snippet below shows how to use Predictor class to invoke the endpoint we deployed earlier. In the snippet, I define credentials as inputs to boto3.Session(). This way is one of the techniques you can use to define your credentials.</p><blockquote><strong>IMPORTANT:</strong> you’re <strong>NOT RECOMMENDED</strong> to hard code your credentials as strings in your script. Instead, you could create a separate, hidden file which contains the credentials you need, and then let your Python script load the credentials from such a file.</blockquote><p>We also need the endpoint name. If we defined the name when using the .deploy() method of the PyTorchModel object (as we did earlier), we can use such name. Otherwise, SageMaker SDK will set a default name, i.e. &lt;framework-name&gt;-&lt;timestamp&gt;.</p><p>Subsequently, we instantiate boto_session using the credentials we defined earlier. By using boto_session, we then instanstiate sagemaker.Session(). Next, we instantiate a Predictor object, and then input all necessary parameters into it.</p><p>The last part is just to prepare the data to be tested (the same as we did earlier in debugging or in model prediction (in Part 1)). We prepare the payload, i.e. the JSON input, that will be hit to the endpoint by using the .predict() method.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2cae9bfd8b1b6897de1ced8ca3b62380/href">https://medium.com/media/2cae9bfd8b1b6897de1ced8ca3b62380/href</a></iframe><p>And below are the results. We expect the results should be the same as we did when using PyTorchModel.deploy().predict().</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/738/1*xTwzJLGceSdhRXyA33_Lbg.png" /><figcaption>Results from Predictor.predict()</figcaption></figure><h4>Using .invoke_endpoint() method</h4><p>The .invoke_endpoint() method <a href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker-runtime.html#SageMakerRuntime.Client.invoke_endpoint">comes from </a><a href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker-runtime.html#SageMakerRuntime.Client.invoke_endpoint">boto3.client(&#39;sagemaker-runtime&#39;)</a>. Your client applications use this API (method) to get inferences from the model hosted at the specified endpoint.</p><p>Using this way of invoking endpoint, we also have to define the credentials as we did earlier in the Predictor class, and we need the endpoint name as well. Here, we don’t need to use the sagemaker library. We just need to instantiate boto3.client() and input all required parameters.</p><p>The way we request prediction to the endpoint is to use .invoke_endpoint() method from boto3.client(). The given response is a complex JSON, but we just need the &quot;Body&quot; field which contains the prediction result; and then we .read() and .decode() the result to get clean prediction. See the code snippet below for more details.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5bc94062d97e01785f62012ca9291a17/href">https://medium.com/media/5bc94062d97e01785f62012ca9291a17/href</a></iframe><p>And below are the results. We expect the results should be the same as we did when using PyTorchModel.deploy().predict() and Predictor.predict().</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/738/1*xTwzJLGceSdhRXyA33_Lbg.png" /><figcaption>Results from boto3.client(‘sagemaker-runtime’).invoke_endpoint()</figcaption></figure><h4>Deleting endpoint</h4><p>This section is just an additional part, but it’s really important to make you meet your budget constraints when using SageMaker.</p><blockquote>Do not forget to delete your SageMaker endpoint if you don’t need it anymore. You don’t want any unwanted cost charged on your credit at the end of month, do you?</blockquote><p>Simply, you can follow the instructions below to delete all unused SageMaker endpoints, configurations, or models.</p><p><a href="https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints-delete-resources.html">Delete Endpoints and Resources</a></p><h3>Constraints</h3><p>If you reach this point without any persistent error, be proud you’re incredible! It’s not that easy to hack SageMaker to create your own GPU ‘multi-model’ endpoint.</p><p>Nevertheless, unfortunately, I have to tell you something:</p><p>I did all I need to deploy this kind of ‘multi-model’ endpoint using the PyTorch model and TensorFlow model. At that moment, the endpoint successfully worked and could be invoked from outside the AWS network environment. I should feel grateful, shouldn’t I?</p><p>Based on this workflow, I tried to imitate what I did to create GPU ‘multi-BERT-model’ endpoint. A BERT model’s size obviously is much larger than that of the PyTorch/TensorFlow model we discussed in these two articles. What if you were to have more than one BERT model? Surely, that will be a huge size and many resources needed.</p><p>All steps had done clearly without any significant problems. The multiple BERT models were deployed safely on the SageMaker endpoint. But, when I tried to use HuggingFaceModel.deploy().predict(), an error appeared. It said something about GPU memory insufficient. Immediately, I also tried to scale up the number of GPU instances when deploying. But, such an error persisted. That error most likely was related to the memory dedicated to GPU hardware, not the regular memory. If it was so, then it means that I need a more powerful GPU resulting in more cost.<strong> Consequently, this means that this technique of hacking SageMaker to have a GPU ‘multi-model’ endpoint doesn&#39;t work for heavy deep learning models.</strong></p><h3>Conclusions</h3><p>That is awesome you’ve finished all two articles. Let’s recap what we’ve learned so far.</p><ul><li>There are many types of deployment on SageMaker that we can use to optimise cost. One of them is to use a multi-model endpoint.</li><li>In order to implement this kind of ‘multi-model’ endpoint, there are several things we have to prepare, such as an AWS account, IAM policy, S3 bucket, Python environment, etc.</li><li>The model samples we use to implement this kind of ‘multi-model’ endpoint are the PyTorch model trained on the MNIST dataset and the TensorFlow model trained on the IMDB dataset. After both models have been trained, we use them to predict simple test data.</li><li>There are a lot of things we have to prepare in order to hack SageMaker so that this kind of ‘multi-model’ endpoint can be implemented:<br>- defining the correct directory structure<br>- writing the inference.py script<br>- debugging the script<br>- compressing the main directory and uploading it to an S3 bucket, and<br>- deploying using SageMaker SDK.</li><li>We also tried to use two ways of invoking the deployed endpoint from outside the AWS network environment, i.e. using Predictor class and using invoke_endpoint() method.</li><li>Lastly, we discussed the constraints that this kind of ‘multi-model’ endpoint has where it is unsuitable for heavy deep learning models.</li></ul><blockquote>Finally, as always, you can find all files and folders needed in my repo on my GitHub account <a href="https://github.com/utomoreza/MultimodelEndpoint_hacked">here</a>.</blockquote><h3>About the Author</h3><p><a href="https://www.linkedin.com/in/utomorezadwi/">Reza</a> is an engineer specializing in data-driven analysis. He’s currently working as Senior Data Scientist at Tokopedia. In his spare time, he likes to write articles on <a href="https://medium.com/@utomorezadwi">Medium</a>, learn something new, or <a href="https://github.com/utomoreza">create side projects</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6e05cf10142f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/deploying-gpu-based-models-on-sagemaker-using-multi-model-endpoint-part-2-final-6e05cf10142f">Deploying GPU-based Models on SageMaker using ‘Multi-Model’ Endpoint (Part 2 - Final)</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying GPU-based Models on SageMaker using ‘Multi-Model’ Endpoint (Part 1)]]></title>
            <link>https://medium.com/geekculture/deploying-gpu-based-models-on-sagemaker-using-multi-model-endpoint-part-1-da68cbbf3d04?source=rss-fadfbe6f0eaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/da68cbbf3d04</guid>
            <category><![CDATA[pytorch]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[sagemaker]]></category>
            <category><![CDATA[gpu]]></category>
            <dc:creator><![CDATA[Reza Dwi Utomo]]></dc:creator>
            <pubDate>Mon, 24 Oct 2022 15:10:15 GMT</pubDate>
            <atom:updated>2022-11-17T03:12:26.086Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*3z3WB_t-bj2DLF2n" /><figcaption>Photo by <a href="https://unsplash.com/@shots_of_aspartame?utm_source=medium&amp;utm_medium=referral">Julia Joppien</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><blockquote>This article is the first part of duology <strong>“Deploying GPU-based Models on SageMaker using ‘Multi-Model’ Endpoint”</strong>. You can find the link to the second part at the bottom of this article.</blockquote><blockquote><strong>UPDATE 26/10/2022:</strong> I’ve just found <a href="https://developer.nvidia.com/blog/run-multiple-ai-models-on-same-gpu-with-sagemaker-mme-powered-by-triton/"><strong>an article from NVIDIA</strong></a> mentioning running GPU-based models on the multi-model endpoint on Amazon SageMaker using NVIDIA Triton Inference Server. I haven’t checked this feature myself yet, but if this is true, this is a game changer.</blockquote><blockquote><strong>UPDATE 17/11/2022:</strong> I’ve got info from <a href="https://www.linkedin.com/feed/update/urn:li:activity:6990336296409858048?commentUrn=urn%3Ali%3Acomment%3A%28activity%3A6990336296409858048%2C6998715078984302592%29&amp;dashCommentUrn=urn%3Ali%3Afsd_comment%3A%286998715078984302592%2Curn%3Ali%3Aactivity%3A6990336296409858048%29">an AWS SageMaker employee</a> that finally AWS has released Multi-Model Endpoint GPU on SageMaker. Cool! You can read the article <a href="https://aws.amazon.com/about-aws/whats-new/2022/10/amazon-sagemaker-cost-effectively-host-1000s-gpu-multi-model-endpoint/"><strong>here</strong></a>.</blockquote><h3>Table of contents:</h3><p>This Part 1 article is arranged as follows:</p><ul><li>Introduction<br>- Optimizing cost for real-time inference<br>- About multi-model endpoint</li><li>What you need</li><li>Prepare Tools &amp; Environments</li><li>Data Preparation and Model Training<br>- PyTorch Model Sample<br>- TensorFlow Model Sample</li><li>Model Prediction<br>- PyTorch<br>- TensorFlow</li><li>Deployment<br>- Directory structure<br>- Writing the inference.py script</li></ul><h3>Introduction</h3><p><a href="https://aws.amazon.com/sagemaker/">Amazon SageMaker</a> is one of AWS’s managed services that provides an end-to-end solution from data pipeline, ML/AI project, model deployment, to MLOps. SageMaker is an all-in-one tool for data scientists and ML engineers.</p><p>When you are about to have your final trained model to production, in SageMaker, usually you could go with two options: <a href="https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints.html">real-time inference</a> and <a href="https://docs.aws.amazon.com/sagemaker/latest/dg/batch-transform.html">batch transform</a>. For the real-time inference, your model will be deployed on a REST-API-like endpoint where you can hit the endpoint anytime in real-time. For the batch transform, your model will be deployed in a fashion where it is not online 24/7, but, instead, it can be activated when you hit the endpoint with a bulk/batch of data.</p><p>For the former, AWS will charge you for the uptime hours of instances on which you deploy your model on. For the latter, you are only charged for the uptime hours when your model processes the bulk of your data.</p><h4>Optimizing cost for real-time inference</h4><p>Based on that, we could infer that the real-time inference must be more costly than the batch transform. AWS has several solutions to overcome the expensive real-time prediction, such as:</p><ul><li><a href="https://docs.aws.amazon.com/elastic-inference/latest/developerguide/what-is-ei.html">Amazon Elastic Inference</a></li><li><a href="https://aws.amazon.com/machine-learning/inferentia/">AWS Inferentia</a></li><li><a href="https://docs.aws.amazon.com/sagemaker/latest/dg/serverless-endpoints.html">Serverless Inference</a></li><li><a href="https://docs.aws.amazon.com/sagemaker/latest/dg/multi-model-endpoints.html">Multi-model endpoint</a></li><li>etc.</li></ul><p>In this article, though, we’re going to focus on the approach of the multi-model endpoint. As stated on the <a href="https://docs.aws.amazon.com/sagemaker/latest/dg/multi-model-endpoints.html">AWS’s webpage</a>:</p><blockquote>To create an endpoint that can host multiple models, use multi-model endpoints. Multi-model endpoints provide a scalable and cost-effective solution to deploying large numbers of models.</blockquote><p>So, simply, by using a multi-model endpoint approach, we can deploy more than one model only into a single endpoint. This indeed will optimize the cost needed as we just need to pay for the endpoint we use, i.e. only one endpoint, whereas we have many models deployed.</p><h4>About multi-model endpoint</h4><p>Regarding the multi-model endpoint approach, I have two pieces of news: a bad one and a good one.</p><ul><li><strong>The bad news</strong></li></ul><p>H̶o̶w̶e̶v̶e̶r̶,̶ ̶s̶a̶d̶l̶y̶,̶ ̶w̶h̶e̶n̶ ̶t̶h̶i̶s̶ ̶a̶r̶t̶i̶c̶l̶e̶ ̶i̶s̶ ̶w̶r̶i̶t̶t̶e̶n̶,̶ ̶A̶W̶S̶ ̶o̶f̶f̶i̶c̶i̶a̶l̶l̶y̶ ̶s̶t̶a̶t̶e̶s̶ ̶t̶h̶a̶t̶ ̶t̶h̶e̶ ̶m̶u̶l̶t̶i̶-̶m̶o̶d̶e̶l̶ ̶e̶n̶d̶p̶o̶i̶n̶t̶ ̶a̶p̶p̶r̶o̶a̶c̶h̶ ̶i̶s̶ ̶n̶o̶t̶ ̶s̶u̶p̶p̶o̶r̶t̶e̶d̶ ̶o̶n̶ ̶G̶P̶U̶ ̶i̶n̶s̶t̶a̶n̶c̶e̶ ̶t̶y̶p̶e̶s̶ ̶(̶y̶e̶t̶?̶ ̶I̶ ̶d̶o̶n̶’̶t̶ ̶k̶n̶o̶w̶)̶.̶ ̶P̶e̶r̶h̶a̶p̶s̶,̶ ̶t̶h̶i̶s̶ ̶c̶o̶r̶r̶e̶l̶a̶t̶e̶s̶ ̶t̶o̶ ̶G̶P̶U̶ ̶i̶n̶s̶t̶a̶n̶c̶e̶ ̶r̶e̶s̶o̶u̶r̶c̶e̶ ̶m̶a̶n̶a̶g̶e̶m̶e̶n̶t̶s̶ ̶t̶h̶a̶t̶ ̶a̶r̶e̶ ̶m̶o̶r̶e̶ ̶c̶h̶a̶l̶l̶e̶n̶g̶i̶n̶g̶ ̶t̶h̶a̶n̶ ̶m̶a̶n̶a̶g̶i̶n̶g̶ ̶r̶e̶s̶o̶u̶r̶c̶e̶s̶ ̶f̶o̶r̶ ̶C̶P̶U̶ ̶i̶n̶s̶t̶a̶n̶c̶e̶s̶;̶ ̶b̶u̶t̶ ̶I̶ ̶d̶o̶n̶’̶t̶ ̶k̶n̶o̶w̶;̶ ̶i̶t̶’̶s̶ ̶j̶u̶s̶t̶ ̶m̶y̶ ̶s̶p̶e̶c̶u̶l̶a̶t̶i̶o̶n̶.̶ <strong>Please read the updated info I mentioned above.</strong></p><ul><li><strong>The good news</strong></li></ul><p>Nevertheless, fortunately, we could still have our own ‘muti-model’ endpoints that are able to host GPU-based models. To achieve this, we have to hack several things in real-time inference. Anyway, I’m inspired from <a href="https://de.linkedin.com/in/philipp-schmid-a6a2bb196">Philipp Schmid’s (Technical Lead at Hugging Face)</a> answer on Hugging Face forum <a href="https://discuss.huggingface.co/t/when-to-use-sagemaker-multi-model-endpoint/18781/2">here</a>.</p><blockquote><strong>Disclaimers:</strong> I won’t go deeper about how to develop a good performed model as this article focuses more on the deployment using the ‘multi-model’ endpoint. Hence, we will just use simple models with just a few training epochs.</blockquote><h3>What you need</h3><p>In this article, we’re going to deploy two GPU-based models using Keras-TensorFlow and PyTorch. So, in addition to well understanding Python, it will be helpful if you also have some knowledge of using <a href="https://keras.io/getting_started/">Keras</a>, <a href="https://www.tensorflow.org/tutorials/quickstart/beginner">TensorFlow</a>, and <a href="https://pytorch.org/get-started/locally/">PyTorch</a>.</p><p>As we work on SageMaker, you also need an <a href="https://aws.amazon.com/console/">AWS account</a> with at least the following <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/introduction.html">IAM policies/roles</a>:</p><ul><li>using <a href="https://docs.aws.amazon.com/sagemaker/latest/dg/nbi.html">SageMaker Notebook Instance</a> or <a href="https://aws.amazon.com/sagemaker/studio/">SageMaker Studio</a>;</li><li>accessing (uploading and downloading) an <a href="https://aws.amazon.com/s3/">S3 bucket</a> that will be used to store the model.tar.gz file;</li><li>deploying models to <a href="https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints-deployment.html">SageMaker Inference Endpoint</a>.</li></ul><p>The webpage below is useful to define the IAM policies/roles mentioned above.</p><p><a href="https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html">How to use SageMaker AI execution roles</a></p><h3>Prepare Tools &amp; Environments</h3><p>When you decide to use SageMaker Notebook, you can follow the instructions below to set up one.</p><ul><li><a href="https://docs.aws.amazon.com/sagemaker/latest/dg/howitworks-create-ws.html">Create a Notebook Instance</a></li><li><a href="https://docs.aws.amazon.com/sagemaker/latest/dg/gs-setup-working-env.html">Step 1: Create an Amazon SageMaker Notebook Instance</a></li></ul><p>Otherwise, if you prefer to use SageMaker Studio, you can follow the instruction below to set up one.</p><p><a href="https://docs.aws.amazon.com/sagemaker/latest/dg/notebooks-create-open.html">Create or Open an Amazon SageMaker Studio Notebook</a></p><blockquote><strong>Note: </strong>It’s up to you if you’d like to use a GPU instance or not in your notebook environment. But for this article, I won’t use a GPU instance as the models used are quite simple even though PyTorch and TensorFlow are used.</blockquote><p>Whatever notebook source you chose, you have to make sure that both PyTorch and Tensorflow are successfully installed in your notebook system. Just follow the instructions from their website as listed below.</p><ul><li><a href="https://pytorch.org/">PyTorch</a></li><li><a href="https://www.tensorflow.org/install">Install TensorFlow 2</a></li></ul><p>You could install the two libraries in their individual single conda virtual environment, but I suggest you install both inside only one single conda environment in order to ease yourself during debugging prior to creating model.tar.gz file.</p><h3>Data Preparation and Model Training</h3><p>In this article, for sample purposes, we are going to use two simple models for each library.</p><ul><li>PyTorch Model Sample</li></ul><p>For PyTorch, we’re going to use a model for image classification using the MNIST dataset. We can use the example from <a href="https://github.com/pytorch/examples/tree/main/mnist">PyTorch’s GitHub official repo</a>. You can just simply run the main.py Python file using any argument you need (don’t forget to set the save-model argument as we need the model for deployment). Afterwards, you would get similar output to mine below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YlXDblRMBnzDg7fIIH2gQA.png" /><figcaption>Training a model using PyTorch</figcaption></figure><p>After the training process has finished, do remember the path for your saved model as you will need it later for deployment purposes.</p><ul><li>TensorFlow Model Sample</li></ul><p>For TensorFlow, we’re going to use a model for text classification using the IMBD dataset. We can use the example from the <a href="https://www.tensorflow.org/tutorials/keras/text_classification_with_hub">TensorFlow website</a>. Just follow the instructions listed there and save your trained model and the tokenizer as well. You will need them later for deployment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8tUPxDH_WdcvpPxjQsx5Zg.png" /><figcaption>Training a model using TensorFlow</figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9446f1fb5fdc98810c1863cb33ca5f16/href">https://medium.com/media/9446f1fb5fdc98810c1863cb33ca5f16/href</a></iframe><h3>Model Prediction</h3><p>After the training processes are done, we should try to test the prediction of both models in order to check if both models work.</p><ul><li>PyTorch</li></ul><p>For the PyTorch model, we first have to define the model architecture for loading the trained weights. For the test set, we simply use the MNIST dataset provided by the PyTorch library. The result will be the predicted number indicating the number shown by the input image.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8bacb2f7535ffa6abbb6dbe6d3773d23/href">https://medium.com/media/8bacb2f7535ffa6abbb6dbe6d3773d23/href</a></iframe><p>And below is the result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/40/1*VGo1dCs9R1_RcFH3Hj6eQw.png" /><figcaption>Result from PyTorch model prediction</figcaption></figure><ul><li>TensorFlow</li></ul><p>For the TensorFlow model, we should define a few useful variables, e.g. label_mapping and max_len, which helps us in the preprocessing step. We have to load both the trained model and tokenizer. The output will be the sentiment of the inputted texts.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d65fdd1a8e25be7685444a98037d03fe/href">https://medium.com/media/d65fdd1a8e25be7685444a98037d03fe/href</a></iframe><p>And below is the result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/540/1*15sCwMaexr8pf9C1NoQk1g.png" /><figcaption>Results from TensorFlow model prediction</figcaption></figure><h3>Deployment</h3><h4>Directory structure</h4><p>In order to deploy to SageMaker Endpoint Inference, we must follow the directory structure defined by AWS as stated <a href="https://sagemaker.readthedocs.io/en/stable/frameworks/pytorch/using_pytorch.html#model-directory-structure">here</a> or shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/882/1*vueuT3E5yZonFjg9ForfYQ.png" /><figcaption>The basic directory structure for deploying to SageMaker Endpoint Inference</figcaption></figure><p>Simply, you must create a directory (any name) which contains the subdirectory/ies and files needed to load and predict using your model/s. Afterwards, you must compress such a directory into a .tar.gz file extension named model.tar.gz.</p><p>In our case to create our own ‘multi-model’ endpoint, inside the main directory (that will be the model.tar.gz file), we’re going to follow the contents as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/622/1*tu4d3iC2dfZTSTizd0iHeg.png" /><figcaption>Proposed directory structure</figcaption></figure><p>So, we’re going to create two subdirectories, i.e. one for the trained PyTorch model and the other for the trained TensorFlow model. Inside each, we’re going to place all files (and folders if any) related to each.</p><p>By doing so, my model.tar.gz looks as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/770/1*2PsKzUl2q13S-iBIcKxjyA.png" /><figcaption>My directory structure</figcaption></figure><h4>Writing the inference.py script</h4><p>In our case, we’re going to use the so-called <a href="https://sagemaker-examples.readthedocs.io/en/latest/frameworks/pytorch/get_started_mnist_deploy.html#Entry-Point-for-the-Inference-Image">“Bring Your Own Model”</a> (BYOM). Thus we have to write a script, which is the inference.py, in order to tell SageMaker how to handle our own models (that were trained outside SageMaker Training Jobs) for the models’ inputs (through REST API), loading the models, model predictions, and the models’ outputs (through REST API).</p><p>The inference.py file script reads 4 main functions as stated in the documentation <a href="https://sagemaker-examples.readthedocs.io/en/latest/frameworks/pytorch/get_started_mnist_deploy.html">here</a>:</p><ul><li>model_fn()</li></ul><p>The function is used to tell the inference image how to load the model checkpoint. The function takes one argument, i.e. model_dir (the directory of the static model checkpoints in the inference image). The function returns the model variable which is the loaded model.</p><ul><li>input_fn()</li></ul><p>The function is invoked in the inference entry point. It handles data decoding from the request input. It takes two arguments: request_body (the payload of the incoming request and request_content_type (the content type of the incoming request). The function returns an object (i.e. the input for the loaded model) that can be passed to thepredict_fn() function.</p><ul><li>predict_fn()</li></ul><p>The function consumes the immediate output of the input_fn() function, using the loaded model which is the return value from the model_fn() function. There are two arguments taken by the function:input_object (the return value from the input_fn() function) and model (the return value from the model_fn() function). The function returns the first argument to be passed to the output_fn() function.</p><ul><li>output_fn()</li></ul><p>The function is used for data post-processing for the predictions produced by the predict_fn() function. The function takes two arguments, i.e. prediction (the return value from the predict_fn() function) and content_type (the content type of the response). The return of the function should be a byte array of data serialized to content_type for encoding the prediction into the content type of the response.</p><p>For a simple BYOM case, usually, we could just use one of four functions mentioned above, i.e. the model_fn() function. For such a case, we only change how the inference server load the model; and we let the rest functions as default.</p><p>However, in our case in which we’d like to have our own ‘multi-model’ endpoint, we’re required to customise all four functions.</p><ul><li>model_fn()</li></ul><p>The function takes one argument, model_dir, which is the location of inside the model.tar.gz file. Since we’ve put two model directories inside, i.e. image_model/ and text_model/ as stated in the directory structure defined earlier, we’re going to load both directories in this model_fn() function.</p><p>As shown by the code snippet below, first, we load the image model, i.e. the trained PyTorch model, and then we also load the text model, i.e. the trained TensorFlow model, along with the trained tokenizer. Lastly, we set the function to return both models (with the tokenizer) in which its output will be a tuple.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a8c3f9c4e495220c7d1daabc090481aa/href">https://medium.com/media/a8c3f9c4e495220c7d1daabc090481aa/href</a></iframe><ul><li>input_fn()</li></ul><p>In this function, we’re going to really create a customised function. Since SageMaker Endpoint Inference works as REST API, we have to set up the function to work in REST API. There are <a href="https://docs.aws.amazon.com/sagemaker/latest/dg/cdf-inference.html">many types of </a><a href="https://docs.aws.amazon.com/sagemaker/latest/dg/cdf-inference.html">content_type</a> we can use, but we will use application/json here as we will hit the server using JSON data as well.</p><p>We will make our inference server receive the following JSON request from clients:</p><pre>{<br>  &quot;model_type&quot;: &quot;text OR image&quot;,<br>  &quot;inputs&quot;: &quot;(a text IF &#39;model_type&#39; == &#39;text&#39;) OR<br>             (a list of numbers with shape (any, 1, 28, 28) IF<br>             &#39;model_type&#39; == &#39;image&#39;)&quot;<br>}</pre><p>To do so, we can define the input_fn() function as shown by the code snippet below. I implement exception handling TypeError below as sometimes I found that only using json.loads() once is insufficient resulting in an error when the code tries to access the element of the dictionary. Subsequently, when the &quot;model_type” field from the client’s request is set to “image”, we expect to have a list of numbers with the shape of (any, 1, 28, 28) and then convert it to a torch.Tensor data type. Lastly, we return the processed data.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/09c628fcc5ba6e8a24446ca2a9dc624d/href">https://medium.com/media/09c628fcc5ba6e8a24446ca2a9dc624d/href</a></iframe><ul><li>predict_fn()</li></ul><p>As this function receives two arguments, we will input the output from input_fn() and the output from model_fn(). As the latter is a tuple (because we’ve done so) which consists of the image model, the text model, and the tokenizer, we have to split it according to their need.</p><p>Afterwards, we can apply the prediction parts in accordance with each model type. See the code snippet below for more details.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/76ad1c63b164a9d4f4ee486cdfbf7bbd/href">https://medium.com/media/76ad1c63b164a9d4f4ee486cdfbf7bbd/href</a></iframe><ul><li>output_fn()</li></ul><p>As defined in the input_fn() function that we use application/json as the content type, we’re going to use the same one in this output_fn() function. As shown in the code snippet below, we treat the final output of the inference server (that will be returned as a response to the client) depending on the model type. If the prediction data is in torch.Tensor data type (or in other words, it’s data from the image model), we will convert it to a raw number. Otherwise, the final output is just the prediction data as the text model produces a text. Lastly, we return the final output in the form of JSON with a field “pred”.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7a9ee47dd6d307cc5586b60039afb6da/href">https://medium.com/media/7a9ee47dd6d307cc5586b60039afb6da/href</a></iframe><p>For other variables and objects needed to support the four main functions above, you can find my example of inference.py <a href="https://github.com/utomoreza/MultimodelEndpoint_hacked/blob/main/model/code/inference.py">here</a>.</p><p><strong>Addition</strong></p><p>For the requirements.txt file that resides inside the code/ directory, we can write the following list of libraries.</p><pre>numpy<br>tensorflow</pre><p>The reason why I don’t list the torch library above, even though we use the PyTorch model, is because we will use a PyTorch inference image for our deployment (we will discuss this a little bit later). Thus, the torch library is already included in such an image.</p><blockquote>We will discuss the next sections, i.e. debugging, uploading to S3 bucket, deploying using sagemaker library, invoking the deployed endpoint, and constraints, in the second part. You can find the link to Part 2 below.</blockquote><p><a href="https://utomorezadwi.medium.com/deploying-gpu-based-models-on-sagemaker-using-multi-model-endpoint-part-2-final-6e05cf10142f">Deploying GPU-based Models on SageMaker using ‘Multi-Model’ Endpoint (Part 2 - Final)</a></p><h3>About the Author</h3><p><a href="https://www.linkedin.com/in/utomorezadwi/">Reza</a> is an engineer specializing in data-driven analysis. He’s currently working as Senior Data Scientist at Tokopedia. In his spare time, he likes to write articles on <a href="https://medium.com/@utomorezadwi">Medium</a>, learn something new, or <a href="https://github.com/utomoreza">create side projects</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=da68cbbf3d04" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/deploying-gpu-based-models-on-sagemaker-using-multi-model-endpoint-part-1-da68cbbf3d04">Deploying GPU-based Models on SageMaker using ‘Multi-Model’ Endpoint (Part 1)</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to implement data-centric AI in NLP]]></title>
            <link>https://utomorezadwi.medium.com/how-to-implement-data-centric-ai-in-nlp-512972a5676b?source=rss-fadfbe6f0eaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/512972a5676b</guid>
            <category><![CDATA[andrew-ng]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[data-centric]]></category>
            <dc:creator><![CDATA[Reza Dwi Utomo]]></dc:creator>
            <pubDate>Wed, 03 Aug 2022 02:00:22 GMT</pubDate>
            <atom:updated>2022-08-03T02:00:22.381Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*fLQAzIO7s2nw6WKm" /><figcaption>Photo by <a href="https://unsplash.com/@impatrickt?utm_source=medium&amp;utm_medium=referral">Patrick Tomasso</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><a href="https://en.wikipedia.org/wiki/Andrew_Ng">Andrew Ng</a>, a distinguished influencer in today’s AI world, brings his new movement to the AI world, namely an approach called <a href="https://www.forbes.com/sites/gilpress/2021/06/16/andrew-ng-launches-a-campaign-for-data-centric-ai/">“data-centric AI”</a>. He and his ventures often spread the word.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FTU6u_T-s68Y%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DTU6u_T-s68Y&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FTU6u_T-s68Y%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/d23adc70b3a3ec35dd1c59dd08dfbf70/href">https://medium.com/media/d23adc70b3a3ec35dd1c59dd08dfbf70/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F06-AZXmwHjo%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D06-AZXmwHjo&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F06-AZXmwHjo%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/9af3778599f8f156edf1fb1ce5b80495/href">https://medium.com/media/9af3778599f8f156edf1fb1ce5b80495/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FYqj7Kyjznh4%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DYqj7Kyjznh4&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FYqj7Kyjznh4%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/a5652e209c092291ae621d03c8ba5ac3/href">https://medium.com/media/a5652e209c092291ae621d03c8ba5ac3/href</a></iframe><ul><li><a href="https://landing.ai/data-centric-ai/">Data-Centric AI - Landing AI</a></li><li><a href="https://datacentricai.org/">Data-centric AI Resource Hub</a></li></ul><p>Many times, he explains the importance of implementing the approach. In this article, we’re going to discover what he means by “data-centric AI” and how to implement it in NLP task.</p><p>This article is arranged as follows:</p><ul><li>The definition of Data-Centric AI</li><li>Data-Centric AI vs Model-Centric AI</li><li>Data-Centric AI applications by Andrew Ng</li><li>Implementing Data-Centric AI in NLP</li><li>Conclusions</li><li>References</li></ul><h3>The definition of Data-Centric AI</h3><blockquote>“Data-centric AI is the discipline of systematically engineering the data used to build an AI system.” (<a href="https://datacentricai.org/">datacentricai.org/</a>)</blockquote><p>As the definition above states, simply by saying a “data-centric AI”, you indicate a different approach to develop your ML model. Many people tend to keep the training set fixed, and then engineer/tune the model in order to achieve the predefined goal. But, in “data-centric AI”, you consider the essence of data consumed by the model. Therefore, you implement a number of techniques to create the data as flawlessly as possible.</p><h3>Data-Centric AI vs Model-Centric AI</h3><p>The image below shows clearly the difference between model-centric and data-centric. When you implement the former, you stick to think “how can I tune the model to get better results?”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/371/1*L2kfhb-KVIAMbaNybQ5Jpw.png" /><figcaption><a href="https://www.youtube.com/watch?v=06-AZXmwHjo">Model-centric vs Data-centric</a></figcaption></figure><p>Andrew said that around 99% of current AI researches implement the model-centric view. As there is no global, general recognized standard to create large datasets, the researchers tend to consider that model-centric approach is the solution for today’s AI. Data provision is often under-estimated and data collection is only executed once before deeply focusing on model improvement.</p><p>On the other hand, AI research which apply data-centric view is still scarce. In the view, you let architecture fixed, then you modify your dataset. As data is food for the model, it deserves to be fed on the best ‘food’. Andrew also said that we have to view the AI system more comprehensively. In an AI system is not a model alone. Data plays a crucial role as well.</p><blockquote>AI = Code + Data</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/490/1*NvM-9EeZ3TU1y6NuEsNVrA.png" /><figcaption><a href="https://neptune.ai/blog/data-centric-vs-model-centric-machine-learning">Characteristics of model-centric and data-centric</a></figcaption></figure><h3>Some Data-Centric AI applications by Andrew Ng</h3><ul><li><strong>Steel inspection</strong></li></ul><p>This is one of his projects at <a href="https://landing.ai/">Landing AI</a>. He and his team develop an object detection model to inspect if given steel has a defect. He states how struggling it is to collect more data at manufacture company.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/733/1*To9tQh0T5aYPh4K-itT4dw.png" /><figcaption><a href="https://www.youtube.com/watch?v=06-AZXmwHjo">Steel inspection using object detector</a></figcaption></figure><p>He and his team attempt two approaches: model-centric and data-centric. They have baseline score for each detector (steel defect detector, solar panel detector, surface inspector). When they implement model-centric approach, trying to tune the model hyperparameters (or even getting the best model architecture), the results barely increase. Meanwhile, when they decide to fix the data fed to the model, they get really promising results, i.e. +16.9%, 3.06%, and 0.4%, respectively.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/699/1*eAzanLXtoqGGmkXXxL7pZA.png" /><figcaption><a href="https://www.youtube.com/watch?v=06-AZXmwHjo">Results when implementing model-centric and data-centric</a></figcaption></figure><ul><li><strong>Labelling inconsistency</strong></li></ul><p>He gives an example of how annotators usually don’t agree each other with their labelling criteria. This phenomenon is called labelling inconsistency, in which the model is fed on data with many labelling ways. This issue will make the model struggle to capture the patterns behind the data and its labels, and resulting in model’s performance degradation.</p><p>To overcome the labelling inconsistency, Andrew suggests the annotators have to sit together and determine a single appropriate way to label the data to make them all agree. In other words, they have to make a convention in labelling data.</p><p>Some of labelling inconsistency cases that Andrew shares are the following:</p><ol><li>Iguana detection</li></ol><p>As seen in the picture below, there are three labelling cases.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/506/1*mskoz-GUBWROGxPGuQP_SQ.png" /><figcaption><a href="https://www.youtube.com/watch?v=06-AZXmwHjo">Iguana example for how annotators don’t agree each other</a></figcaption></figure><p>The upper-left picture shows how an annotator draws two bounding boxes for two iguanas equally divided. At the upper-right picture, another annotator has his/her own rule to label the iguanas. He/she might think the left iguana’s tail extends to the right side, so he/she decides to draw a bounding box along the tail. The last picture, the lower picture, again depicts another annotator. He/she might think that because the two iguanas almost overlap each other, he/she decides to just draw a single bounding box to cover both.</p><p>Any labelling approach is fine, but the annotators have to agree on one single convention labelling across all data.</p><p>2. Steel defect detection</p><p>Another example that Andrew tells is the detection of steel defect. The labelers are given pictures to label. In the image below, two labelers don’t agree each other with how they should label. The Labeler 1 might think each defect is just a small dot, then he/she draws a small bounding box on each dot label. Meanwhile, the Labeler 2 might think that those small dots could be joined together in two groups.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/731/1*0-fJPyAtYYcBWkalYevJFw.png" /><figcaption><a href="https://www.youtube.com/watch?v=06-AZXmwHjo">Steel defect example for how annotators don’t agree each other</a></figcaption></figure><p>Again, any labelling way is just fine. But both labelers have to agree on one single convention labelling across all data.</p><h3>Implementing Data-Centric AI in NLP</h3><p>In his videos, mostly Andrew shares applications of data-centric AI in computer vision or speech recognition field. So, a question might arise: okay, I now agree with your data-centric concept, but how come I implement data-centric AI in the NLP field as I have a lot of NLP tasks at my work?</p><p>According to one of Andrew’s online courses, i.e. <a href="https://www.coursera.org/learn/introduction-to-machine-learning-in-production?specialization=machine-learning-engineering-for-production-mlops">Introduction to Machine Learning in Production</a> (the first course of <a href="https://www.coursera.org/specializations/machine-learning-engineering-for-production-mlops">Machine Learning Engineering for Production Specialization</a>), on Coursera, in the iguana example (that we just discussed earlier), we could think of the different labelling strategy as a result of different labelling rule followed by different annotator. That’s why, with the same concept, we could also introduce an ‘iguana’ example for NLP task. For instance, we could create a labelling rule to label text dataset to represent one labelling strategy.</p><blockquote>Disclaimers: the implementation we’re going to discuss below comes from the Week 3’s ungraded lab of the course. If you’d like to learn more from its notebook, you could enroll directly on the course (perhaps you could <a href="https://www.coursera.support/s/article/209818613-Enrollment-options?language=en_US">use Coursera’s audit version, submit for financial aid, or payable subscription</a>. Anyway,<a href="https://utomorezadwi.medium.com/cara-belajar-di-coursera-gratis-dan-bersertifikat-free-certified-course-on-coursera-164f39d4d39e"> I wrote an article about how submit a financial aid to Coursera</a>).</blockquote><h4>Data understanding</h4><p>Let’s take an example of dataset of YouTube videos’ comments. We’re going to differentiate whether given comment is spam (label 1) or not (label 0). The image below show the head of our whole dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-xNx5lVEc4VHyYRckmGy5g.png" /><figcaption>Head of data frame</figcaption></figure><p>The dataset is quite balanced as shown in the image below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/282/1*SPqRG2-DOKdECKGDpMnvRQ.png" /></figure><p>Upon applying train_test_split, we use stratify argument to make sure sklearn maintains the proportional splits. And the image below shows the results.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/749/1*QOh9HIAAv5A1jmNg17njdw.png" /></figure><h4>Model definition</h4><p>We could use scikit-learn’s <a href="https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html#sklearn.feature_extraction.text.CountVectorizer">CountVectorizer</a> to vectorize our dataset. For the main model, for simplicity, we could use scikit-learn’s <a href="https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.MultinomialNB.html">Multinomial Naive Bayes</a> classifier. As we discussed earlier that in the data-centric view, we keep the model fixed, we could let CountVectorizer and MultinomialNBuse their default hyperparameters. For the result metric, as this case is just a binary classification and both labels are relatively balanced, we just use accuracy score.</p><h4>Implement different labelling strategies</h4><p>After the vectorizer and the model are correctly defined, we can try different labelling rules. By taking a look at these strategies, we will know that labelling is a crucial step to improve model’s performance.</p><ul><li><strong>Correct labelling</strong></li></ul><p>First, let’s see what will we get if we use the correct labels from the dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/558/1*1jvMA2rjaNrJsAGtWzmXEw.png" /><figcaption>Accuracy for correct labelling strategy</figcaption></figure><p>Awesome, just by using a simple vectorizer and a simple model with default settings, we can achieve such a high result. But, it’s not that surprised as this labelling strategy clearly labels the dataset perfectly, making the model easy to capture the pattern to classify between spam and not-spam.</p><p>So, this result will be our upper bound, indicating our best result.</p><ul><li><strong>Random labelling</strong></li></ul><p>The natural way to establish a lower bound is to use random generation to label the dataset. After we run 10 random iterations, the average of all results is the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/576/1*IUpBTiLTIopxmmLELt8_LQ.png" /></figure><p>Because this case is a binary classification with relatively balanced classes, you can expect accuracies around 50%. If you have other datasets with more classes and imbalanced, perhaps, you will get more varied random results.</p><p>Anyway, this result will be our lower bound, indicating our worst result.</p><ul><li><strong>Labelling strategies</strong></li></ul><p>Let’s suppose that for some reason we don’t have access to the true labels associated with each data point in the dataset. It is a natural idea to think that there are patterns in the data that will provide clues of which are the correct labels. This is of course very dependent on the kind of data we are working with and to even hypothesize which patterns exist requires great domain knowledge.</p><p>It is reasonable for many people to come up with rules that might help identify a spam comment from a non-spam one for a YouTube video. In the following section we will be performing automatic labeling using such rules. <strong>We can think of each iteration of this process as a labeler with different criteria for labeling</strong>.</p><p>First things first. Let’s define how we are going to encode the labeling:</p><p>- SPAM is represented by 1</p><p>- NOT_SPAM by 0</p><p>- NO_LABEL as -1</p><p>You might be wondering about the NO_LABEL keyword. Depending on the rules we come up with, these might not be applicable to some data points. For such cases it is better to refuse from giving a label rather than guessing, which we already saw yields poor results.</p><p>1. First iteration</p><p>In the first iteration, we’ll try to a simple rule to indicate if given comment is spam or not. If given comment contains one of the following strings, we will label it as SPAM or NO_LABEL otherwise:</p><p>- “free”: usually spam comments tease users by promoting free stuffs</p><p>- ”subs”: spam comments usually ask users to subscribe to certain web or channel</p><p>- ”http”: spam comments usually include link</p><p>The problem with this first iteration is that we won’t have NOT_SPAM labels. In other words, we only have one class in the training set whereas our test set indeed consists of two labels. Consequently, we only keep 24.23% data of training set (see below image). When we use the labelled data (labelled using this rule) to train the model, we get slightly better result than that of random labelling, as shown by the image below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/712/1*Nqi7XRVuDKoyXmErbJibHQ.png" /><figcaption>First iteration — labelling strategy</figcaption></figure><p>2. Second iteration</p><p>In the second iteration, we’ll try to add more rules. When we inspect the comments little bit deeper, we’ll find that the NOT_SPAM comments tend to include word “view” or “song”. Since the dataset we use is the comments from most watched YouTube videos, it would be common for people to comment about “view” or the “song”. For the SPAM comments, we also add two more words, i.e. “gift” and “follow”. Otherwise, we’ll label it as NO_LABEL.</p><p>By applying the second iteration, we can keep 44% of total training set, and also now we have labels for both classes — a condition we can’t have in the first iteration. The labels for both classes really helps the model to perform much better than that of the first iteration, as shown by the image below. However, we are still far from the upper bound.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*k7-Y2bqVWyiBE1naAmDP4A.png" /><figcaption>Second iteration — labelling strategy</figcaption></figure><p>3. Third iteration</p><p>In the third iteration, we’re going to add another approach. If we look at the NOT_SPAM comments, they are relatively shorter than the SPAM ones. The average lengths for each tell their difference.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/525/1*TpYYiPPuekjY51e2zBU7_w.png" /></figure><p>Next, we could determine a length threshold to cut between NOT_SPAM and SPAM. A simple way to accomplish this is by viewing the histogram of NOT_SPAM comments’ length as shown by the image below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/461/1*OOwTa3GAg8dU3kvk1Kklpw.png" /><figcaption>Histogram of NOT_SPAM comments’ length</figcaption></figure><p>We can see that most NOT_SPAM comments have 30 or less characters. Hence, we set a rule in which given comment will be labelled as NOT_SPAM if its length is less than 30 characters. On the other hand, we also add another word for SPAM rule, i.e. “check out”.</p><p>The results of the third iteration are shown below. All rules we have so far produce promising results at both keeping 78% data and giving model’s accuracy for greater than 86%! Perhaps, the more rules we define, the closer we will become to the upper bound (hopefully).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/619/1*wp-JlCB3ssNaDL6aFKKH2Q.png" /><figcaption>Third iteration — labelling strategy</figcaption></figure><p>4. Fourth iteration</p><p>In the fourth iteration, we modify the “http” word that we implemented previously. If we focus on the dataset, we can see that most SPAM comments include a URL with various forms. So, instead of just searching if “http” is inside the text, we will implement RegEx to match the comment text to the URL pattern. We could use the <a href="https://gist.github.com/gruber/8891611">complex RegEx URL pattern</a> defined below:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/558aaa2ee2e16be88ca969100a310770/href">https://medium.com/media/558aaa2ee2e16be88ca969100a310770/href</a></iframe><p>We’ve also found that the SPAM comments usually offer their products by using some words, such as “money” and “app”. For the NON_SPAM comments, we would try to insert “omg” word.</p><p>The results of this iteration are shown in the image below. We see that this iteration can keep the data over 82%, ~4% higher than the third iteration. However, unfortunately, even though we have sufficient increase in data proportion, we can’t get the same increase for the model accuracy as it just gets improved by ~1%.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/619/1*zHvQ2OG_SHyR4zDuXciGkw.png" /><figcaption>Fourth iteration — labelling strategy</figcaption></figure><p>So far, we’ve found that in order to getting closer and closer to the upper bound (the true labels), we will have even more complex rules. But, by reaching to this point, we understand that different labelling strategy evidently can affect the model’s performance.</p><h3>Conclusions</h3><p>Amazing! We have reached until this point. So far, we’ve learned many things. To sum up what we’ve discussed from the beginning:</p><ul><li>Data-centric view, initiated by Andrew and his colleagues, brings a brand-new idea of how we should treat the AI system.</li><li>Model-centric view focuses on how to <strong>tune the model architecture</strong> to improve the performance, whereas data-centric view focuses on how to <strong>modify the data</strong> to improve the performance.</li><li>Almost all AI researches implement model-centric view.</li><li>There are many things we can do to implement data-centric view, such as labelling consistency, bringing good quality data, focusing on clean data, etc.</li><li>Data-centric view can be implemented in NLP as well. For example, the labelling consistency and quality of text data really influence the model performance. The better your labels are, the better your models will be.</li></ul><h3>References</h3><ul><li>YouTube video <a href="https://www.youtube.com/watch?v=06-AZXmwHjo&amp;ab_channel=DeepLearningAI">“A Chat with Andrew on MLOps: From Model-centric to Data-centric AI”</a></li><li>YouTube video <a href="https://www.youtube.com/watch?v=Yqj7Kyjznh4">“Data-centric AI: Real World Approaches”</a></li><li>Article <a href="https://datacentricai.org/blog/technical-debt-in-ml-a-data-centric-view/">“Technical Debt in ML: A Data-Centric View”</a></li><li><a href="https://www.coursera.org/specializations/machine-learning-engineering-for-production-mlops">DeepLearning.AI’s Machine Learning Engineering for Production (MLOps) Specialization</a></li></ul><h3>About the Author</h3><p><a href="https://www.linkedin.com/in/utomorezadwi/">Reza</a> is an engineer specializing in data-driven analysis. He’s currently working as a Data Scientist at <a href="https://telkom.co.id/">Telkom Indonesia</a> and appointed as Head of Legal Data Analytics at <a href="https://legalanalytics.id/">Legal Analytics (Powered by Telkom Indonesia)</a>. In his spare time, he likes to write articles on <a href="https://medium.com/@utomorezadwi">Medium</a>, learn something new, or <a href="https://github.com/utomoreza">create side projects</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=512972a5676b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[No-code movement? Biasa! AI ngoding, baru luar biasa!]]></title>
            <link>https://utomorezadwi.medium.com/no-code-movement-biasa-ai-ngoding-baru-luar-biasa-79fe6daa5926?source=rss-fadfbe6f0eaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/79fe6daa5926</guid>
            <category><![CDATA[no-code]]></category>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[github]]></category>
            <category><![CDATA[gpt-3]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Reza Dwi Utomo]]></dc:creator>
            <pubDate>Sat, 30 Apr 2022 13:05:59 GMT</pubDate>
            <atom:updated>2022-04-30T13:05:59.128Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*6W6iCkFzhdOAXXB1rKUpWQ.jpeg" /><figcaption><a href="https://www.bengkelti.com/blog/apa-itu-github-copilot-akankah-menggantikan-programmer/">sumber</a></figcaption></figure><p>GitHub memiliki sebuah terobosan baru, yaitu <a href="https://copilot.github.com/">GitHub Copilot</a>. Seperti namanya, Copilot, fitur ini bekerja layaknya seorang copilot buat seorang programmer, yaitu membantu dalam programming. Mudahnya, Copilot bakal men-generate lines of code sesuai dengan fungsi yang diinginkan oleh kita, sang programmer. Menarik, bukan?</p><h3>Tentang</h3><p>Perkembangan dunia AI, terutama NLP (natural language processing), akhir-akhir ini terjadi sangat pesat. Yang menjadi tonggak begitu derasnya advancement ini adalah <a href="https://www.ibm.com/cloud/learn/deep-learning#:~:text=Deep%20learning%20is%20a%20subset,from%20large%20amounts%20of%20data.">deep learning</a>, terutama setelah sebuah breakthrough, bernama <a href="https://en.wikipedia.org/wiki/Transformer_(machine_learning_model)">Transformer</a>, di deep learning muncul.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*kJjEG31jhlpX64sM.png" /><figcaption>Banyak yang punya versi transformer-nya sendiri (<a href="https://research.aimultiple.com/gpt/">sumber</a>)</figcaption></figure><p>Singkatnya, banyak orang dan organisasi telah, sedang, dan akan berlomba-lomba mengembangkan model transformer versi mereka sendiri. Beberapa versi model transformer yang terkenal adalah model-model yang berasal dari GPT familiy, yang terdiri dari <a href="https://openai.com/blog/language-unsupervised/">GPT</a>, <a href="https://openai.com/blog/better-language-models/">GPT-2</a>, dan <a href="https://arxiv.org/abs/2005.14165">GPT-3</a>, yang dikembangkan oleh <a href="https://openai.com/">OpenAI</a>.</p><p>Kemudian, GPT-3, sebagai model paling anyar dari GPT family, <a href="https://arxiv.org/abs/2107.03374">di-improve (fine-tuning) lebih lanjut oleh OpenAI</a> dan dinamai <a href="https://arxiv.org/abs/2107.03374">Codex</a>. Model Codex ini di-train di public repositories yang tersebar di GitHub. Dan Copilot ini menggunakan API yang berasal dari model OpenAI Codex.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*rlpgMAyFpDGrv86F.png" /><figcaption>GitHub Copilot di balik layar (<a href="https://copilot.github.com/">sumber</a>)</figcaption></figure><h3>Fitur</h3><p>Jika kita melihat website Copilot di bawah ini, fitur ini dikatakan lebih dari sekadar fitur Autocomplete:</p><blockquote><em>GitHub Copilot understands significantly more context than most code assistants. So, whether it’s in a docstring, comment, function name, or the code itself, GitHub Copilot uses the context you’ve provided and synthesizes code to match.</em></blockquote><p><a href="https://copilot.github.com/">GitHub Copilot · Your AI pair programmer</a></p><p>Beberapa fitur yang dimilikinya adalah:</p><ul><li><strong>Convert comments to code</strong></li></ul><p>Kita cukup ketik yang kita inginkan di bagian comments, lalu biarkan Copilot bekerja untuk menerjemahkan kebutuhan kita ke dalam lines of code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1022/1*fj5yDlc0u9L2Hqv9A2kclw.gif" /></figure><ul><li><strong>Autofill for repetitive code</strong></li></ul><p>Saat kita perlu mengetik baris code yang berulang, Copilot bakal membantu pekerjaan merepotkan ini.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1017/1*kcxrvuMYVcApC0rBE-ILOA.gif" /></figure><ul><li><strong>Tests without the toil</strong></li></ul><p>Bagi para programmers yang bekerja dalam sebuah project, sebelum mereka yakin codingan mereka dapat berfungsi dengan baik, biasanya mereka melakukan beberapa tests. Salah satunya adalah <a href="https://smartbear.com/learn/automated-testing/what-is-unit-testing">Unit Test</a>. Copilot juga akan membantu mereka dalam mendefinisikan Unit Test.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1017/1*1fF_eqiJirqazFchs3xqhA.gif" /></figure><ul><li><strong>Show me alternatives</strong></li></ul><p>Setelah Copilot memberikan solusi lines of code-nya ke kita dan ternyata kita masih belum puas dengan hasilnya, tenang saja, Copilot bisa men-generate alternatif solusi lain.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2w_6Jgxo5Np-2kRjGRJVRA.gif" /></figure><h3>Cara Menggunakan</h3><h4>Mendaftar</h4><p>Oke, kita sudah melihat kehebatan Copilot. Sekarang pertanyaan yang muncul adalah “bagaimana cara pakainya?”. Simpel saja. Kita cuma perlu klik tombol <strong>Sign Up</strong> di website GitHub Copilot, maka kemudian kita akan diarahkan untuk sign in ke waitlist. Untuk join ke waitlist, kita cukup sign in dengan akun GitHub yang kita miliki</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/597/1*mpwOKALjeB3V3w_0V_pCKA.png" /></figure><blockquote>Sebagai informasi, Copilot saat ini masih dalam tahap <em>technical preview</em> dan baru tersedia sebagai extension di beberapa IDE populer seperti <a href="https://code.visualstudio.com/">Visual Studio Code</a>, <a href="https://neovim.io/">Neovim</a>, dan <a href="https://www.jetbrains.com/">JetBrains suite</a>. Jadi, tidak heran kalau kita perlu join ke waitlist terlebih dahulu.</blockquote><p>Setelah kita sukses join ke waitlist, kita harus menunggu sampai kita dapat email balasan agar bisa menggunakan Copilot. Untuk kasus saya, saya menunggu kurang lebih satu bulan hingga akhirnya saya dapat email seperti di bawah ini.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/596/1*V-dn7x-MEokiFsfybJU3KQ.png" /></figure><h4>Meng-install</h4><p>Setelah kita mendapat email dari GitHub, kita tinggal klik tombol Install GitHub Copilot seperti gambar di atas atau kita juga bisa kunjungi link di bawah ini karena tombol tersebut juga mengarah ke link tersebut.</p><p><a href="https://github.com/github/copilot-docs">GitHub - github/copilot-docs: Documentation for GitHub Copilot</a></p><p>Link tersebut mengarah ke sebuah repository. Di bagian README-nya, kita bisa dengan mudah mendapatkan informasi mengenai Copilot ini lebih detail dan cara meng-install-nya di beberapa IDEs populer.</p><ul><li><a href="https://github.com/github/copilot-docs/blob/main/docs/visualstudiocode/gettingstarted.md#getting-started-with-github-copilot-in-visual-studio-code">Install di Visual Studio Code</a></li><li><a href="https://github.com/github/copilot-docs/blob/main/docs/visualstudio/gettingstarted.md#getting-started-with-github-copilot-in-visual-studio">Install di Visual Studio</a></li><li><a href="https://github.com/github/copilot-docs/blob/main/docs/jetbrains/gettingstarted.md#getting-started-with-github-copilot-in-jetbrains">Install di JetBrains</a></li><li><a href="https://github.com/github/copilot.vim#getting-started">Install di Neovim</a></li></ul><h3>Tes</h3><p>Oke. Sekarang kita coba tes si Copilot ini. Di tes ini, saya menggunakan IDE Visual Studio Code. Bahasa yang saya gunakan adalah Python. Sebelum kita lakukan tes, pastikan bahwa Copilot telah running di VS Code yang kita gunakan. Jika running, akan muncul logo Copilot di kanan bawah window VS Code kita, seperti yang ditunjukkan oleh gambar di bawah ini.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/607/1*7sFKQ4nD8_10gxsGXTxnjQ.png" /><figcaption>Logo Copilot (yang dilingkari merah) sedang running</figcaption></figure><p>Kemudian, kita perlu memahami keyboard shortcuts Copilot yang running di VS Code, yaitu sebagai berikut:</p><ul><li>Accept an inline suggestion: Tab.</li><li>Dismiss an inline suggestion: Esc.</li><li>Show next inline suggestion: Alt + ] or Option (⌥) + ].</li><li>Show previous inline suggestion: Alt + [ or Option (⌥) + [.</li><li>Trigger inline suggestion: Alt + \ or Option (⌥) + \.</li><li>Open Copilot (10 suggestions in separate pane): Ctrl + Enter.</li></ul><h4>Tes pertama</h4><p>Sekarang kita akan mencoba membuat sebuah program yang dapat melakukan operasi pertambahan, pengurangan, perkalian, dan pembagian, layaknya sebuah kalkulator. Video di bawah ini menunjukkan bagaimana kita melakukan tes yang pertama ini.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FWgZBMZXXOqE%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DWgZBMZXXOqE&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FWgZBMZXXOqE%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/47f435c1673becb33587741257328195/href">https://medium.com/media/47f435c1673becb33587741257328195/href</a></iframe><p>Dari video di atas, bisa kita lihat bahwa dengan menginput instruksi dalam bentuk comments, Copilot kemudian akan men-generate lines of code. Tapi mungkin kita harus beberapa kali menekan tombol enter agar code baris per baris bisa di-generate. Setelah script Python selesai dibuat, script tersebut bisa di-run di Terminal atau Command Prompt.</p><h4>Tes kedua</h4><p>Untuk menguji seberapa hebatnya Copilot, kita akan tambah perintah lebih kompleks. Dari fungsi kalkulator di atas, kita ingin Copilot mengkompilasi semua functions ke dalam sebuah class SimpleCalculator. Lalu kita mau Copilot membuat tampilan web page sederhana dengan menggunakan <a href="https://plotly.com/dash/">library Dash</a>, agar user bisa menginput dua angka, memilih operasi yang diinginkan, dan click tombol “Calculate”, lalu program akan menampilkan perhitungan yang kita inginkan. Web server app dari Dash ini akan running di localhost sehingga bisa kita akses langsung melalui web browser.</p><p>Video di bawah ini menunjukkan bagaimana kita melakukan tes yang kedua ini.</p><blockquote>Pastikan anda telah <a href="https://dash.plotly.com/installation">menginstall library Dash</a> sebelumnya. Saya menyarankan agar anda membuat <a href="https://docs.python.org/3/library/venv.html#creating-virtual-environments">python</a>/<a href="https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#creating-an-environment-with-commands">conda</a> virtual environment untuk menginstall Dash.</blockquote><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FxMAUcmQBZIE%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DxMAUcmQBZIE&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FxMAUcmQBZIE%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/21724c640d40f81d8da50a10238799fd/href">https://medium.com/media/21724c640d40f81d8da50a10238799fd/href</a></iframe><p>Dari video di atas, kita bisa lihat, pertama kita bisa menambah instruksi baru dengan cara menambah baris comments. Di instruksi pertama, kita minta Copilot untuk mengkompilasi functions calculator yang sudah di-define sebelumnya. Namun, hasilnya, Copilot malah menulis ulang function-function tersebut ke dalam class yang kita inginkan. Oke, tidak mengapa.</p><p>Di instruksi kedua, kita minta Copilot untuk membuat Dash app agar bisa men-serve class SimpleCalculator di frontend. Seperti biasa, kita harus tekan enter dan tab berkali-kali agar Copilot bisa men-generate lines of code permintaan kita. Saat kita selesai dengan script yang ditulis oleh Copilot, kita bisa me-run nya di Terminal atau Command Prompt. Namun, saat saya mau me-run-nya, ada sedikit kendala di mana ternyata Copilot tidak men-generate lines of code untuk meng-import library Dash dan turunannya. Jadi, saya harus mengoreksi kekeliruan Copilot ini.</p><p>Setelah selesai dikoreksi, saya perlu meng-activate conda environment saya yang memiliki Dash library yang telah terinstall. Setelah activated, script bisa di-run dengan sukses. IP address yang muncul bisa kita klik untuk membukanya langsung atau di-copy-paste ke browser.</p><p>Dan voila! Dash App buatan Copilot berhasil running sesuai dengan permintaan kita!</p><h4>Tes ketiga</h4><p>Ini tes terakhir. Di sini, kita akan push Copilot ke kondisi yang sangat spesifik. Kita akan minta Copilot untuk membuat sebuah chatbot dengan menggunakan, karena <a href="https://news.microsoft.com/announcement/microsoft-acquires-github/">GitHub adalah subsidiary dari Microsoft</a>, <a href="https://dev.botframework.com/">Microsoft Bot Framework</a>.</p><p>Video di bawah ini menunjukkan bagaimana kita melakukan tes yang ketiga ini.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FF1JZw83188g%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DF1JZw83188g&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FF1JZw83188g%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/60d74307d858a7893aa3b526f63b44f1/href">https://medium.com/media/60d74307d858a7893aa3b526f63b44f1/href</a></iframe><p>Dari video di atas, kita bisa lihat bahwa Copilot tidak mampu men-generate lines of code dari instruksi untuk membuat chatbot menggunakan Microsoft Bot Framework. Copilot hanya menyarankan lines of code yang sudah kita lakukan, seperti import dash, padahal kita tidak membutuhkannya untuk kasus kali ini.</p><p>Pada contoh lain, saya juga pernah memberikan instruksi-instruksi sulit lain ke Copilot, seperti:</p><ul><li>membuat sebuah program untuk mendeteksi apakah sebuah akun Twitter itu adalah bot atau bukan,</li><li>men-download pre-trained model transformer tertentu dari HuggingFace, memasangkan head tambahan pada model tersebut, men-download public dataset, dan terakhir men-fine-tune model tersebut.</li></ul><p>Hasilnya, untuk instruksi yang pertama, Copilot kewalahan, dan malah men-generate lines of code aneh yang tidak ada sangkut pautnya dengan instruksinya. Untuk instruksi yang kedua, Copilot hanya berulang-ulang men-generate import beragam libraries, tanpa menulis code lain. Bahkan libraries yang di-import tersebut beberapa kali malah ditulis ulang kembali.</p><h4>Tambahan</h4><p>Sebelum GitHub Copilot rilis, model utamanya, yaitu OpenAI Codex, pada dasarnya mampu men-generate lines of code dari instruksi yang diberikan oleh user. Pihak OpenAI sendiri juga telah merilis video live demo Codex di YouTube, seperti yang ditunjukkan di bawah ini.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FSGUCcjHTmGY%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DSGUCcjHTmGY&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FSGUCcjHTmGY%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/b110e7dbaeb54235f569038afe6c65f7/href">https://medium.com/media/b110e7dbaeb54235f569038afe6c65f7/href</a></iframe><h3>Kesimpulan</h3><p>Kita telah membahas hal menarik panjang lebar. Kita telah sedikit membahas tentang asal-usul GitHub Copilot dan fitur-fitur yang dimilikinya. Kita juga telah membahas bagaimana cara mendapatkan Copilot ini dan meng-install-nya di IDE favorit kita. Dan terakhir, kita juga telah melakukan beberapa uji coba pada kemampuan Copilot.</p><p>Dari tiga tes yang telah dilakukan, ada beberapa hal menarik yang bisa kita simpulkan:</p><ul><li>Copilot mampu men-generate instruksi-instruksi mudah dan sederhana, bahkan memberikan beberapa alternatif code.</li><li>Copilot mampu memberikan rekomendasi lines of code yang menggunakan libraries lain yang cenderung populer.</li><li>Lines of code yang di-generate oleh Copilot tidak bisa semerta-merta kita percayai karena tidak akan tidak mungkin coding yang dihasilkan masih terdapat bugs.</li><li>Jika kita memberikan instruksi yang lebih spesifik dan menantang, terutama dengan menggunakan library yang less popular, Copilot akan gagal memenuhi ekspektasi kita.</li><li>Copilot juga tidak mampu mengerjakan instruksi yang terlalu high-level.</li><li>Pada dasarnya, Copilot hanya mampu melaksanakan instruksi-intruksi low-level yang logical thinking-nya sudah di-break-down oleh sang programmer.</li><li>dsb.</li></ul><p>Kembali lagi, sama seperi namanya, Copilot hanyalah sebuah tool untuk membantu pilot utama, yaitu sang programmer. Oleh karena itu, untuk saat ini, kita belum bisa terlalu berharap suatu <a href="https://www.zdnet.com/article/what-is-artificial-general-intelligence/">AGI (artificial general intelligence)</a> yang benar-benar mampu berpikir layaknya kognisi manusia.</p><p>Di lain sisi, ada beberapa side effect jika kita berniat menggunakan tool Copilot ini. Dirangkum dari sebuah <a href="https://betterprogramming.pub/6-reasons-why-you-should-avoid-github-copilot-and-fly-solo-instead-8a948665433f">artikel Medium</a> oleh Arnold Abraham, berikut hal-hal yang perlu kita pertimbangkan sebelum menggunakan Copilot:</p><ul><li>Open-source licenses get disrespected.</li><li>Code provided by GitHub Copilot may expose you to liability.</li><li>Tools you depend on are crutches. GitHub Copilot is a crutch.</li><li>This tool is free now, but it won’t stay gratis.</li><li>Your code is exposed to other humans and stored. If you have an NDA, you are screwed.</li><li>You have to constantly check the code this tool delivers to you. Not a great service for a tool.</li></ul><p><strong>Terima kasih telah membaca!</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=79fe6daa5926" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[BERT Extractive Summarizer vs Word2Vec Extractive Summarizer: Which one is better and faster?]]></title>
            <link>https://utomorezadwi.medium.com/bert-extractive-summarizer-vs-word2vec-extractive-summarizer-which-one-is-better-and-faster-c6d6d172cb91?source=rss-fadfbe6f0eaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/c6d6d172cb91</guid>
            <category><![CDATA[summarization]]></category>
            <category><![CDATA[word2vec]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[bert]]></category>
            <dc:creator><![CDATA[Reza Dwi Utomo]]></dc:creator>
            <pubDate>Sat, 05 Mar 2022 18:06:18 GMT</pubDate>
            <atom:updated>2022-03-26T03:11:13.160Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*yPE-_hL8JiEBgDt5" /><figcaption>Photo by <a href="https://unsplash.com/@isaacmsmith?utm_source=medium&amp;utm_medium=referral">Isaac Smith</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><blockquote>TL; DR — Let’s define an extractive summarization using BERT or Word2Vec. Which one will be the winner in terms of processing speed and accuracy? We will compare them by using public summarization datasets. Hint: bigger not always better.</blockquote><h3>Contents</h3><p>This article is arranged as follows:</p><ul><li>Introduction</li><li>Background</li><li>Simplified Theory</li><li>Coding</li><li>Testing</li><li>Conclusions</li></ul><h3>Introduction</h3><p>Summarization is one of the important downstream tasks in NLP (natural language processing). Its main objective is to sum up what a long text talks about, or in other words, to summarize a text. There are two main approaches in the summarization task: <strong>abstractive summarization</strong> and <strong>extractive summarization</strong>.</p><h4><strong>Abstractive summarization</strong></h4><p>The former includes a high-performance deep learning NLG (natural language generation) model as the machine is asked to paraphrase a couple of sentences into a few more concise sentences. This method really mimics how humans usually summarize a text.</p><p>Some model examples that are often used for abstractive summarization are <a href="https://blog.paperspace.com/generating-text-summaries-gpt-2/">GPT-2</a>, <a href="https://ai.googleblog.com/2020/06/pegasus-state-of-art-model-for.html">PEGASUS</a>, <a href="https://ai.facebook.com/research/publications/bart-denoising-sequence-to-sequence-pre-training-for-natural-language-generation-translation-and-comprehension/">BART</a>, <a href="https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html">T5</a>, <a href="https://openai.com/blog/summarizing-books/">GPT-3</a>, etc.</p><h4>Extractive summarization</h4><p>On the other hand, the latter approach implements a simpler method. Basically, to summarize a text, the machine will select the sentences it considers as the most important. It’s important to note that while summarizing, this method does not change the words in the sentences it selects. Some examples of this type of summarization are <a href="https://medium.com/data-science-in-your-pocket/text-summarization-using-textrank-in-nlp-4bce52c5b390">TextRank</a>, <a href="https://towardsdatascience.com/text-summarization-using-tf-idf-e64a0644ace3">TF-IDF</a>, and <a href="https://medium.com/jatana/unsupervised-text-summarization-using-sentence-embeddings-adb15ce83db1">Word2Vec</a>.</p><h3>Background</h3><p>The transformer<a href="https://arxiv.org/abs/1706.03762"> model</a> becomes the state-of-the-art baseline model for many NLP applications. It completely makes the old sequential models, e.g. RNN, LSTM, GRU, obsolete. Since then, many research institutions both universities and high-tech companies compete to release the transformer model of their version. One of the most popular transformer-based models is <a href="https://arxiv.org/abs/1810.04805">Bidirectional Encoder Representations from Transformers or simply called BERT</a>. <a href="https://blog.google/products/search/search-language-understanding-bert/">Google claims</a> this model is able to understand the human language even better.</p><p>BERT, as the meaning of its abbreviation, is the encoder part of transformer model. Such encoder functions to encode the incoming tokens (e.g. text) into embeddings. These embeddings consist of arrays of numbers.</p><p>On the other hand, a slightly older method than the transformer, called <a href="https://en.wikipedia.org/wiki/Word_embedding">Word Embedding</a>, once was a famous methodology to convert a text to embeddings. Since then, this method is almost (if not always) used for the initial, mandatory part of any sequential deep learning. This standard keeps going on, even in the transformer model’s architecture.</p><p>All tranformer-based models are well known for their big size and GPU-demanding characteristic. Therefore, if your machine (in particular for production) doesn’t have any <a href="https://developer.nvidia.com/cuda-gpus">CUDA-supported GPU</a> and high RAM memory — or you don’t have enough budget to rent cloud computing with GPU supported — it will be troublesome if you insist to run those models in your production machine. You will suffer from turtle processing and out-of-memory, just to name a few. Though for development, I think, still, you could utilize <a href="https://research.google.com/colaboratory/">Google Colab</a>, for example, for getting a free GPU-ready Python environment.</p><p>Unlike transformer-based models, the method of Word Embedding, such as <a href="https://arxiv.org/abs/1301.3781">Word2Vec</a>, <a href="https://arxiv.org/abs/1607.04606">FastText</a>, and <a href="https://nlp.stanford.edu/pubs/glove.pdf">GloVe</a>, is more lightweight and not GPU demanding. Thus, you will be happier to run it in a production environment in case you don’t have any GPU.</p><p><strong>However, for Word Embedding, such as Word2Vec, does being lighter also mean more accurate and faster? And for BERT, despite more understanding, we have to consider its size and processing speed as well. Consequently, this is a battle of trade-offs among accuracy, processing speed, and model size.</strong></p><h4>Constraints</h4><p>In the rest of this article, we’re going to carry out a performance comparison between BERT and Word2Vec in a specific downstream task, i.e. extractive summarization. In order to make this comparison reproducible and common, we will use news texts for the summarization task. And as we will use Indonesian BERT, i.e. <a href="https://github.com/indobenchmark/indonlu">IndoBERT</a>, for the BERT base model, the Indonesian news texts are used.</p><p>The metrics we will use are the following:</p><ul><li><a href="https://towardsdatascience.com/the-ultimate-performance-metric-in-nlp-111df6c64460">ROUGE score</a></li><li>Time lapse</li><li>Model size</li></ul><p>There will be two environments for this comparison:</p><ol><li>Conda using CPU (for Word2Vec and BERT)</li><li>Conda using GPU (for BERT only)</li></ol><h3>Simplified Theory</h3><blockquote>The methodology for extractive summarization used here is inspired by a research paper written by Miller in 2019. For mode details on the concept, you can read his paper <a href="https://arxiv.org/abs/1906.04165">here</a>.</blockquote><h4><strong>Embeddings</strong></h4><p>In order to make both models implement the extractive summarization task, we should discuss the methodology we will use. As described earlier, both BERT and Word2Vec act as embeddings, in which such embeddings are arrays of numbers. Depending on the input and setting, there are various types of embeddings. Word embeddings are the embeddings for tokens or words. Sentence embeddings are for sentences. Text embeddings are for texts. Even if your input is a list of recommendations, you could also create recommendation embeddings.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zGa60pgD2mr4fOiWHvXS1A.png" /><figcaption>Sentence embedding process</figcaption></figure><p>As we’d like to summarize news texts, we have to understand the structure of such text. Typically, a news text consists of many sentences, in which the sentences are grouped together forming several sub-ideas (or paragraphs). As explained at the beginning, the extractive summarization selects a number of sentences as a summary. In this news text case, we will make the model to select sentence candidates as the representation of each sub idea. <strong>Thus, we need to create sentence embeddings</strong>.</p><h4>Clustering</h4><p>The baseline of the clustering model is to group data into clusters that share similar characteristics. Usually, clustering is used for structured data, i.e. tabular data. But, since we already convert the news text into sentence embeddings, our text data is now in a structured format. Each record is each sentence embedding which represents a sentence.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*k4D4zxAu5SoLOyIU" /><figcaption><a href="https://opg.optica.org/oe/fulltext.cfm?uri=oe-25-22-27570&amp;id=375887">Cluster center</a></figcaption></figure><p>Subsequently, it is straightforward to implement a clustering model to create a number of clusters. And then, for each cluster, we will select the nearest point (i.e. sentence embedding) from the cluster’s center. By doing so, we will end up having the number of embeddings equal to the number of clusters. <strong>Those final sentence embeddings are the final sentence candidates for the summary of a given text.</strong></p><h4>Whole Steps</h4><p>The complete stages for this methodology are shown in the workflow diagram below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/123/1*UVEuQzvthZbvlANsv7UVvg.png" /><figcaption>Workflow</figcaption></figure><ul><li>First, we will implement text processing in order to get clean and typo-free text.</li><li>The news text is split into sentences where they will be converted into sentence embeddings.</li><li>For the sentence embeddings part, we will utilize two models, BERT and Word2Vec.</li><li>In the clustering part, we will use the K-Means algorithm. The essential part of using such an algorithm is to define the correct k (the number of clusters) parameter.</li><li>Finally, we extract the points which are the nearest ones from each cluster center as the sentence candidates for the final summary.</li></ul><h3>Coding</h3><p>Okay. Enough of all verbose explanations! Now, let’s move on to the most interesting part, hands-on coding.</p><h4>Text Preprocessing</h4><p>First of all, it’s a best practice if we always use the text preprocessing part before feeding texts to any model. So, let’s define it here.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d1c7d0a72058122206725b2e1fc5ce82/href">https://medium.com/media/d1c7d0a72058122206725b2e1fc5ce82/href</a></iframe><p>In a brief, text preprocessing in the code snippet above executes the following:</p><ul><li>remove the article’s head, i.e. whether the site name is still included</li><li>remove any heading/trailing whitespace &amp; punctuation</li><li>replace multiple whitespaces with a single one</li><li>remove any abbreviation dot</li></ul><h4>BERT Extractive Summarizer</h4><p>Fortunately, to implement the extractive summarization using BERT, we can use a library called bert-extractive-summarizer <a href="https://github.com/dmmiller612/bert-extractive-summarizer">here</a>. For your information, such a library is the code implementation of Miller’s paper mentioned in the previous section.</p><p>Below is the code snippet to use the library.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/33435ad9bc41645fbfd92fb0f62192e6/href">https://medium.com/media/33435ad9bc41645fbfd92fb0f62192e6/href</a></iframe><p>We try to summarize the article below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PbPpgNWGa-P791cvPqLogQ.png" /><figcaption>Sample article</figcaption></figure><p>And below is the summary result.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3sqY3RnZfLv3LGV69Xk7Gg.png" /><figcaption>Summary by bert-extractive-summarizer</figcaption></figure><h4>Word2Vec Extractive Summarizer</h4><p>Based on Miller&#39;s paper and library, we can reconstruct the methodology by substituting the BERT part for the Word2Vec model. Alright, in order to define the solution, we’re going to create three classes, i.e. Embedder(), Clustering(), and Word2VecSummarizer(). The name of each class is explanatory.</p><p>Below is the code snippet for theEmbedder() class. Simply, the class is used to convert given text (i.e. paragraph, sentence, or word) into embeddings. For more detailed explanations, you could read the comments provided in the code snippet.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/23a0643715298bb11b6ebf057087c788/href">https://medium.com/media/23a0643715298bb11b6ebf057087c788/href</a></iframe><blockquote>Disclaimer: in order to use the class defined above, you have to use a pre-trained <a href="https://radimrehurek.com/">Gensim</a>’s Word2Vec model. If you don’t have one, you could <a href="https://radimrehurek.com/gensim/auto_examples/howtos/run_downloader_api.html">download one</a> or <a href="https://towardsdatascience.com/how-to-train-the-word2vec-model-24704d842ec3">train one</a> by yourself.</blockquote><p>Below is the code snippet for the Clustering() class. Simply, it instantiates a K-Means clustering model, trains the model, and then gets the points nearest from the center of each cluster. For more detailed explanations, read the comments provided in the code snippet below.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2e9e5a546adeeef3c4d26ba429f13e15/href">https://medium.com/media/2e9e5a546adeeef3c4d26ba429f13e15/href</a></iframe><p>Below is the code snippet for the Word2VecSummarizer() class, the ultimate class. It covers the two classes defined earlier.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a88cae65886f5735ae8bec512c8c4ee9/href">https://medium.com/media/a88cae65886f5735ae8bec512c8c4ee9/href</a></iframe><p>Now, let’s try to use the model to summarize the sample article. Below is the result.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*00GDHEm83CYabW9R_xDaNw.png" /><figcaption>Summary by word2vec-extractive-summarizer</figcaption></figure><h3>Testing</h3><p>So far, we’ve had both BERT extractive summarizer and Word2Vec extractive summarizer. We’ve tested both using a sample article. Now, to meet the objectives defined at the beginning, we have to test them in a number of summarization problems. Thanks to <a href="https://github.com/kata-ai/indosum">Kurniawan and Louvan</a> (<a href="https://arxiv.org/abs/1810.05334">2018</a>), we can use their summarization dataset to examine our models. You can download the dataset <a href="https://drive.google.com/file/d/1OgYbPfXFAv3TbwP1Qcwt_CC9cVWSJaco/view">here</a>. Its README file clearly describes the structure of the dataset as listed below.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8b68b9565a3dcb2e497c7f1ecb5f4dae/href">https://medium.com/media/8b68b9565a3dcb2e497c7f1ecb5f4dae/href</a></iframe><p>Since there are many JSON files provided in the dataset, for our case, we’ll just use one of its test set files, i.e. test.01.jsonl. Such a test set has 3762 news articles. However, for our case, in order to shorten the processing time, we will just use the first 500 articles.</p><h4>ROUGE metrics</h4><p>For ROUGE metrics, we can simply use the library <a href="https://github.com/pltrdy/rouge">rouge</a>. The results for BERT extractive summarizer are shown below.</p><blockquote>Note: “f” stands for f1_score, “p” stands for precision, “r” stands for recall.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/299/1*h9525-TvhFt62B7q5LnAXg.png" /></figure><p>Meanwhile, the results for Word2Vec extractive summarizer are shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/302/1*FlcRjMaHxb4jBl1e4w33dw.png" /></figure><p>As seen by two tables above, t<strong>he ROUGE metrics for both methodology don’t have any significant difference</strong>.</p><h4>Time lapse metrics</h4><p>We’re going to use the Jupyter’s magic function%%timeit to measure the time lapse. We just use 5 articles for this time lapse test. There are two environments used, i.e. CPU (for BERT and Word2Vec) and GPU (for BERT only). The results are shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/435/1*Nq0disHVbCw9uNps3kcGGg.png" /></figure><p>The BERT-CPU runs really slow. It needs around 11 seconds to process a single news article. We don’t want that in the production. But, after running in GPU, the BERT model is able to lower its time lapse almost by 11 times. Impressive enough. On the other hand, in fact, <strong>this transformer model (even after running in GPU) can’t beat the Word2Vec model which is just running in CPU</strong>.</p><h4>Model size</h4><p>Lastly, let’s compare the model size of each methodology.</p><p>When we’d like to use the IndoBERT model, we have to download its pretrained model from Hugging Face. Total file size we have to download is almost 500 MB. Such size is only for a single model checkpoint (along with its tokenizer). Really huge.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/813/1*nlm77AIRiPxEKaZ38x2MLA.png" /></figure><p>On the other hand, my pre-trained Word2Vec model needs around 215 MB.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jSj0BmTk3LbLMxERyw5xtA.png" /></figure><h3>Conclusions</h3><p>Finally, we arrive at the last section. We’ve discussed many things in this article. Now, let’s sum up what we’got so far.</p><ul><li>We’ve differentiated extractive summarization from abstractive summarization.</li><li>We’ve discussed the pros and cons between transformer model and word embeddings model.</li><li>The simplified theory behind the extractive summarization implemented in this case has been described.</li><li>The coding implementations for each BERT summarizer and Word2Vec summarizer have been explained.</li><li>Finally, we’ve evaluated both models, in terms of ROUGE, time lapse, and model size, by using IndoSum datasets.</li><li>The evaluation results indicate that, in this case, the BERT model (which was waited for its high accuracy) performs not much different from the Word2Vec model in term of ROUGE. In term of time lapse, however, the BERT model is completely beaten by the Word2Vec model. Lastly, in term of model size, the BERT model’s size is more than two times larger than the Word2Vec model’s size.</li></ul><blockquote>After we’ve seen the results of both models, do you think using BERT extractive summarizer still worth the production environment?</blockquote><blockquote>For me, if the case is in extractive summarization, it is much better if you use Word Embedding based extractive summarizer instead.</blockquote><p>Thank you for reading this article. You could find the whole files in my GitHub repo <a href="https://github.com/utomoreza/IndoSum">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c6d6d172cb91" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Social Network Analytics: Studi Kasus Trending Topic Indonesia “KDRT” di Twitter]]></title>
            <link>https://utomorezadwi.medium.com/social-network-analytics-studi-kasus-trending-topic-indonesia-kdrt-di-twitter-9cf6d1460849?source=rss-fadfbe6f0eaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/9cf6d1460849</guid>
            <category><![CDATA[social-network]]></category>
            <category><![CDATA[twitter]]></category>
            <category><![CDATA[gephi]]></category>
            <dc:creator><![CDATA[Reza Dwi Utomo]]></dc:creator>
            <pubDate>Sun, 06 Feb 2022 01:41:09 GMT</pubDate>
            <atom:updated>2022-02-06T01:41:09.444Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*v3VKJ_zpiRUA_3zU" /><figcaption>Photo by <a href="https://unsplash.com/@brett_jordan?utm_source=medium&amp;utm_medium=referral">Brett Jordan</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Latar Belakang</h3><p>Beberapa hari ke belakang, saat saya sedang melakukan analisis di laman Twitter, saya mendapati keyword “KDRT” sempat viral di Indonesia. Selidik punya selidik, sepertinya hal ini dihebohkan oleh video ceramah <a href="https://twitter.com/oki_setiana">Oki Setiana Dewi</a>.</p><p>Namun, saya tidak berminat untuk membahas masalah KDRT di sini. Yang ingin saya bahas adalah saya penasaran, <strong>berawal dari manakah trending topic ini</strong>. Untuk menjawab hal tersebut, saya mencoba melakukan Social Network Analysis menggunakan data Twitter, lalu saya bersihkan menggunakan Pandas, dan terakhir saya analisis menggunakan Gephi.</p><h3>Persiapan</h3><p>Data Twitter yang diambil berasal dari 1 Februari 2022 hingga 5 Februari 2022 pagi. Untuk mendapatkan data tersebut, saya menggunakan API yang berasal dari <a href="https://kazee.id/">Kazee</a>. Yang menjadi point of interest saya adalah bagaimana jagat topik ini berkembang dari akun ke akun. Untuk menjawab hal tersebut, kita perlu menjadikan tiapscreen name (alias username) sebagai nodes, dan Retweet (RT) dan reply antar screen name sebagai edges (alias koneksi). Setelah data selesai dipersiapkan, kita import data tersebut ke aplikasi <a href="https://gephi.org/">Gephi</a>.</p><h3>Hasil Analisis</h3><p>Pertama, kita cluster semua data. Hasil networknya ditunjukkan oleh Gambar 1 di bawah (atau lihat di <a href="https://odsafdsd-my.sharepoint.com/:i:/g/personal/k58_proindo_net/EbWdwIJa-19IrcCJ1SN68D0BAtDEC_hwmdmlK5twM9ZiMA?e=y7sB7d">sini</a> untuk resolusi lebih besar).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ATF3TRI9qafkOajk-rSD6g.png" /><figcaption>Gambar 1</figcaption></figure><p>Semakin besar sebuah node menunjukkan bahwa akun tersebut semakin banyak di-RT atau di-reply. Di network pada Gambar 1, dapat kita lihat bahwa jagat topik ini didominasi oleh akun-akun utama di tengah-tengah network, terutama akun <a href="http://twitter.com/@bijikambingg">@bijikambingg</a> dan <a href="http://twitter.com/@nugarislucu">@nugarislucu</a>. Uniknya, dua akun tersebut memiliki banyak edges yang terputus tidak tersambung. Hal ini bisa menjadi indikasi bahwa banyak bots/buzzers yang me-RT/reply tweet dari dua akun tersebut.</p><p>Di luar dari node-node besar utama, terdapat titik-titik node kecil yang memenuhi jagat topik. Mereka terbagi ke dalam beberapa klaster: ada yang men-tweet mengenai “KDRT” tapi tidak memiliki hubungan dengan nodes utama; ada yang juga men-tweet, tidak berhubungan dengan nodes utama, dan tidak pula berhubungan dengan akun manapun. Uniknya, tipe klaster kedua ini mendominasi node-node kecil yang berserakan mengelilingi nodes besar utama.</p><p>Sekarang, mari kita zoom-in dan analisis lebih detail nodes besar utama. Networknya dapat dilihat pada Gambar 2 di bawah (atau lihat di <a href="https://odsafdsd-my.sharepoint.com/:i:/g/personal/k58_proindo_net/ESKXmppSFTpNl23RBeyLQSEB2BGbAHPfz5KZHnR5SLeNbQ?e=FHrACs">sini</a> untuk resolusi lebih besar).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BZp5zPjnAB6QD6VBpL9h4A.png" /><figcaption>Gambar 2</figcaption></figure><p>Node-node utama dapat dikelompokkan ke dalam 4 klaster:</p><ul><li>Klaster 1 (warna kuning): didominasi oleh @<a href="http://twitter.com/bijikambingg">bijikambingg</a>, @<a href="http://twitter.com/nugarislucu">nugarislucu</a>, dan @<a href="http://twitter.com/na_dirs">na_dirs</a></li><li>Klaster 2 (warna biru): didominasi oleh @<a href="http://twitter.com/askrlfes">askrlfes </a>dan @<a href="http://twitter.com/tubirfess">tubirfess</a></li><li>Klaster 3 (warna pink): didominasi oleh @<a href="http://twitter.com/areajulid">areajulid</a></li><li>Klaster 4 (warna jingga): didominasi oleh @<a href="http://twitter.com/detikcom">detikcom </a>dan @<a href="http://twitter.com/okkymadasari">okkymadasari</a></li></ul><p>Mari kita lihat lebih detail tiga nodes paling besar, yaitu @<a href="http://twitter.com/bijikambingg">bijikambingg</a>, dan @<a href="http://twitter.com/nugarislucu">nugarislucu</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/606/1*1wc6tKHqdI98V7UIOqw1-w.png" /><figcaption>Gambar 3 — @bijikambingg</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/1*6P30OH88w_oAV6_fl9bOPQ.png" /><figcaption>Gambar 4 — @nugarislucu</figcaption></figure><p>Dua tweets di Gambar 3 dan Gambar 4 di atas mendapat jumlah RT, quotes, dan likes yang begitu banyak. Jika dilihat dari waktu posting, tweet milik @bijikambingg tayang lebih awal.</p><p>Sekarang kita lihat siapa tweet yang paling pionir dalam posting masalah KDRT dan Oki Setiana Dewi ini. Kalau dilihat dari data Twitter yang saya miliki, tweet paling awal pada tanggal 1 Februari adalah tweet dari @<a href="http://twitter.com/gulagarembata">gulagarembata</a> yang me-reply tweet dari @<a href="http://twitter.com/sosmedkeras">sosmedkeras</a>, seperti pada Gambar 5 di bawah ini.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/607/1*6QoQp98jzz5Z2IrLVTFnsQ.png" /><figcaption>Gambar 5</figcaption></figure><p>Ternyata tweet dari @<a href="http://twitter.com/gulagarembata">gulagarembata</a> walaupun memiliki keyword KDRT, tapi tidak berhubungan dengan Oki Setiana Dewi. Saya terus mengurutkan tweets awal-awal, dan akhirnya saya mendapati <strong>tweet yang paling awal</strong> membahas KDRT <strong>dan </strong>Oki Setiana Dewi adalah tweet di Gambar 6 di bawah (yang diblok dan diberi panah).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PvQv4PSO-JL5CbForBsLcg.png" /><figcaption>Gambar 6</figcaption></figure><p>Ternyata tweet utama milik @<a href="http://twitter.com/bijikambingg">bijikambingg</a> tidak masuk data saya (hanya masuk karena hasil RT/reply) karena tweet akun tersebut hanya melempar isu dengan isi tweet hanya “dihhhhhhhhhhhhh”, sehingga tidak masuk filter dari Kazee (mengingat keyword yang saya masukkan adalah “kdrt”, bukan “dihhhhhhhhhhhhh”).</p><h3>Kesimpulan</h3><p>Kita telah membahas sebuah Social Network Analytics (SNA) sederhana untuk menelusuri sebuah keyword “KDRT” yang sempat trending awal-awal Februari. SNA sangat powerful untuk menelusuri keterkaitan antar akun, topik tertentu, dan/atau kata tertentu. Kita juga telah mendapat jawaban bahwa tweet yang awal-awal melempar isu mengenai KDRT dan Oki Setiana Dewi adalah <a href="https://twitter.com/bijikambingg/status/1488752241536159746">tweet</a> dari @<a href="http://twitter.com/bijikambingg">bijikambingg</a>.</p><p>Namun, ada beberapa tantangan yang ditemui dan hal yang bisa dikembangkan dari analisis ini. SNA sangat dipengaruhi oleh kualitas data. Jika data yang tersedia tidak/kurang lengkap, maka hasil SNA pun bisa cacat.</p><p>Dari SNA sederhana di atas, kita mendapati bahwa tweet paling awal tidak masuk ke dalam filter data acquisition kita. Di sini, kita harus pandai-pandai meng-collect data-data yang berhubungan dengan topic of interest kita walaupun data-data tersebut tidak secara eksplisit mengandung kata yang diinginkan.</p><p>Perlu diketahui, saya meng-collect data Twitter menggunakan API dari Kazee. Jika memungkinkan, saya sangat menyarankan rekan-rekan untuk meng-collect data dari Twitter API langsung (berbayar kalau perlu), karena data streaming dari Twitter API menjanjikan kualitas data nomor wahid untuk collecting data Twitter.</p><p>Tiap kelompok klaster sebenarnya bisa ditelaah lebih lanjut dengan cara melihat beberapa sampel tweets di tiap klaster agar kita dapat melihat kecenderungan di tiap klaster tersebut. Pun memungkinkan jika kita memasukkan analisis sentimen ke dalam SNA ini agar analisis pada tiap klaster lebih komprehensif. Saya pun memiliki model berbasis BERT untuk memprediksi sentimen. Namun, karena model tersebut cenderung tendensius dan cukup sering gagal menangkap makna sarkasme, saya putuskan untuk tidak menggunakannya.</p><p><em>Terakhir, terima kasih atas waktu rekan-rekan telah membaca hingga ke bawah sini.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9cf6d1460849" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Model Text Similarity Menggunakan BERT Berbasis Bahasa Indonesia]]></title>
            <link>https://utomorezadwi.medium.com/model-text-similarity-menggunakan-bert-berbasis-bahasa-indonesia-14898e301bef?source=rss-fadfbe6f0eaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/14898e301bef</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[pytorch]]></category>
            <category><![CDATA[bert]]></category>
            <category><![CDATA[nlp]]></category>
            <dc:creator><![CDATA[Reza Dwi Utomo]]></dc:creator>
            <pubDate>Mon, 03 Jan 2022 02:35:24 GMT</pubDate>
            <atom:updated>2022-01-03T02:35:24.388Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*D5R2dz8ZDMrr8poi" /><figcaption>Photo by <a href="https://unsplash.com/@jhaland?utm_source=medium&amp;utm_medium=referral">Jørgen Håland</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Motivasi</h3><p>Perkembangan teknologi NLP (Natural Language Processing) bahasa Indonesia bisa dikatakan masih minim jika dibandingkan dengan NLP bahasa Inggris. Hal ini pula yang menjadi tantangan tersendiri pada saat saya hendak mengembangkan model NLP bahasa Indonesia.</p><p>Di lain sisi, perkembangan NLP beberapa tahun terakhir sangat pesat, terutama setelah pendekatan berbasis <em>Attention </em>ditemukan. Pendekatan inilah yang mendasari metode baru deep learning pada NLP, yaitu <em>Transformers</em>, yang mendegradasi pendekatan trandisional LSTM (Long Short Term Memory). Sekarang sudah ada begitu banyak varian dari arsitektur model Transformers. Salah satu yang terkenal adalah BERT (Bi-directional Encoder Representations from Transformers).</p><p>BERT adalah sebuah pretrained model yang di-training pada corpus berbahasa Inggris. ̶B̶a̶h̶a̶s̶a̶ ̶I̶n̶g̶g̶r̶i̶s̶ ̶l̶a̶g̶i̶,̶ ̶b̶a̶h̶a̶s̶a̶ ̶I̶n̶g̶g̶r̶i̶s̶ ̶l̶a̶g̶i̶. Jadi, kalau kita mau menggunakannya untuk bahasa Indonesia, kita harus siap modal GPU yang support CUDA (milik sendiri atau sewa di cloud — yang pasti harga untuk dua-duanya tidak murah) untuk men-training ulang model BERT tersebut dengan menggunakan corpus bahasa Indonesia. Untungnya, ada <a href="https://aclanthology.org/2020.aacl-main.85.pdf"><em>para pahlawan NLP Indonesia</em></a> yang telah melakukan itu semua buat kita semua; dan kita cuma cukup <a href="https://github.com/indobenchmark/indonlu">pakai</a> tanpa perlu bayar. <em>Terima kasih kakak-kakak!</em></p><p>Dengan adanya BERT berbahasa Indonesia tersebut, alias IndoBERT, pada tulisan ini, kita akan coba mengembangkan model NLP untuk menangani text similarity.</p><h3>Daftar Isi</h3><ul><li>Asumsi</li><li>Sekilas teori</li><li>Coding</li><li>Referensi</li></ul><h3>Asumsi</h3><p>Sebelum kita membahas tulisan ini lebih lanjut, saya asumsikan para pembaca yang budiman memenuhi persyaratan di bawah ini:</p><ul><li>memahami cara kerja model BERT</li><li>memahami konsep text similarity</li><li>memahami penggunaan library transformers (Hugging Face)</li><li>memahami penggunaan library PyTorch</li></ul><h3>Sekilas Teori</h3><p>Biasanya, word embeddings yang berbasis Word2Vec atau GloVe-lah yang digunakan sebagai model untuk text similarity. Model tersebut digunakan untuk men-tokenize suatu token/kata/kalimat/teks agar menjadi bentuk representative vectors. Pada saat kita ingin mencek similarity antar dua teks, dengan menggunakan vectors dari masing-masing teks tersebut, kita bisa mencari nilai cosine similarity-nya.</p><p>Dengan menggunakan pendekatan yang serupa, kita juga bisa menggunakan model BERT untuk men-generate representative vectors dari suatu token/kata/kalimat/teks. Hal ini memungkinkan karena pada dasarnya, BERT merupakan sebuah <em>word embeddings</em>, namun lebih kompleks. Itu mengapa tidaklah heran bahwa jika kita ingin menggunakan model BERT untuk sebuah downstream task, kita perlu memasang sebuah <em>head </em>(yang sesuai dengan downstream-nya) pada model BERT tersebut lalu melakukan fine-tuning dengan training dataset yang relevan.</p><p>Selain itu, BERT juga merupakan sebuah Encoder dari model Transformers. Tujuan dari Encoder pada model Transformers adalah untuk meng-encode (alias mentransformasi dan menangkap pola) tokens. Tujuan ini sangat mirip fungsi utama word embeddings.</p><h3>Coding</h3><p>Oke, langsung saja kita praktikkan ke dalam coding Python. Pertama, berikut adalah libraries tambahan yang harus disiapkan:</p><ul><li>torch (PyTorch)</li></ul><p>kita cuma butuh function clamp.</p><ul><li>transformers (Hugging Face)</li></ul><p>dengan library ini, kita akan me-load model dan tokenizer dari IndoBERT.</p><ul><li>sklearn (scikit-learn)</li></ul><p>kita cuma butuh function cosine_similarity untuk men-calculate cosine similarity antara dua vector.</p><p>Kedua, mari kita buat ke dalam sebuah Python class. Di dalam class tersebut, kita akan men-define beberapa method:</p><ul><li>load_pretrained</li></ul><p>digunakan untuk me-load model dan tokenizer dari IndoBERT,</p><ul><li>__cleaning</li></ul><p>digunakan untuk melakukan beberapa text preprocessing sebelum masuk ke proses utama,</p><ul><li>__process</li></ul><p>digunakan untuk melakukan proses utama dalam men-generate <em>word embeddings</em>,</p><ul><li>predict</li></ul><p>digunakan untuk melakukan prediksi token similarity antara dua token yang diinput.</p><p>Untuk detail coding-nya, bisa dilihat di code snippet di bawah ini.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/85bf801bb0e796c11c775f7036ebe894/href">https://medium.com/media/85bf801bb0e796c11c775f7036ebe894/href</a></iframe><p>Setelah TokenSimilarity class di-<em>define</em>, sekarang kita <em>instantiate</em> class-nya sambil men-<em>download</em> modelnya. Kita akan menggunakan <em>chekpoint model </em><a href="https://huggingface.co/indobenchmark/indobert-base-p2">indobert-base-p2</a> dari HuggingFace.</p><pre>model = TokenSimilarity()</pre><pre>model.load_pretrained(&#39;indobenchmark/indobert-base-p2&#39;)</pre><p>Lalu kita coba tes dengan beberapa kalimat di bawah.</p><pre>token1 = &#39;lama lu bro&#39;</pre><pre>token2 = &#39;jangan terlalu lama&#39;</pre><pre>token3 = &#39;kehancuran melanda kota menyebabkan malapetaka&#39;</pre><p>Lalu lihat hasilnya.</p><pre>print(model.predict(token1, token2))</pre><pre>print(model.predict(token1, token3))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/147/1*iRDwrpnrqjKtWUse_Grmeg.png" /></figure><p>Antara token1 dengan token2, similarity probability yang dihasilkan adalah 0.66, sedangkan token1 dengan token3, dihasilkan nilai 0.28.</p><h3>Referensi</h3><p>Sebagai referensi, rekan-rekan pembaca bisa mendapatkan detail mengenai metode ini dari beberapa sumber di bawah.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FjVPd7lEvjtg%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DjVPd7lEvjtg&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FjVPd7lEvjtg%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/bf5aef0b5e1c3e1e53d46a2ac63dc6ed/href">https://medium.com/media/bf5aef0b5e1c3e1e53d46a2ac63dc6ed/href</a></iframe><ul><li><a href="https://towardsdatascience.com/bert-for-measuring-text-similarity-eec91c6bf9e1">BERT For Measuring Text Similarity</a></li><li><a href="https://github.com/indobenchmark/indonlu">GitHub - indobenchmark/indonlu: The first-ever vast natural language processing benchmark for Indonesian Language. We provide multiple downstream tasks, pre-trained IndoBERT models, and a starter code! (AACL-IJCNLP 2020)</a></li></ul><iframe src="https://drive.google.com/viewerng/viewer?url=https%3A//aclanthology.org/2020.aacl-main.85.pdf&amp;embedded=true" width="600" height="780" frameborder="0" scrolling="no"><a href="https://medium.com/media/55c5115f6faba61db6e55c8dcd19b140/href">https://medium.com/media/55c5115f6faba61db6e55c8dcd19b140/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=14898e301bef" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How much will you spend on Starbucks products?]]></title>
            <link>https://utomorezadwi.medium.com/how-much-will-you-spend-on-starbucks-products-242dc22c335c?source=rss-fadfbe6f0eaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/242dc22c335c</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[regression]]></category>
            <category><![CDATA[customer]]></category>
            <category><![CDATA[starbucks]]></category>
            <dc:creator><![CDATA[Reza Dwi Utomo]]></dc:creator>
            <pubDate>Mon, 06 Dec 2021 03:28:28 GMT</pubDate>
            <atom:updated>2021-12-07T02:14:12.886Z</atom:updated>
            <content:encoded><![CDATA[<p>Data Scientist Nanodegree Capstone Project</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ZWmLgB5hiejX7Ur1" /><figcaption>Photo by <a href="https://unsplash.com/@rr_abrot?utm_source=medium&amp;utm_medium=referral">RR Abrot</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>This article is for the Starbucks Capstone Challenge of the Data Scientist Nanodegree in Udacity. The dataset provided are simulated data that mimics customer behavior on the Starbucks rewards mobile app, i.e. whether the customer would buy a Starbucks if he/she is provided any informational or discount offer.</p><h3>Business Understanding</h3><p>Once every few days, Starbucks sends out an offer to users of the mobile app. An offer can be merely an advertisement for a drink or an actual offer such as a discount or BOGO (buy one get one free). Some users might not receive any offers during certain weeks. Every offer has a validity period before the offer expires.</p><p>Based on the information given above, we can infer that any transaction made by a customer is something really desired. No matter what offer you’d provide to the customers, still, you will deserve a transaction. Therefore, this project will try <strong>to predict how much money a customer will spend</strong>.</p><p>To solve the problem, we will use all datasets provided.</p><p>We will employ several machine learning algorithms. As this will be a regression problem, we will evaluate the algorithms based on several regression metrics, i.e. mean absolute error (MAE), mean squared error (MSE), root mean square error (RMSE), and R2 score.</p><p>For more explanations about why those metrics are selected, I copied-pasted the descriptions from this <a href="https://towardsdatascience.com/what-are-the-best-metrics-to-evaluate-your-regression-model-418ca481755b">TDS article</a>.</p><ul><li>Mean squared error</li></ul><blockquote>Mean Square Error is an absolute measure of the goodness for the fit. It gives you an absolute number on how much your predicted results deviate from the actual number. You cannot interpret many insights from one single result but it gives you a real number to compare against other model results and help you select the best regression model.</blockquote><ul><li>Root mean square error</li></ul><blockquote>Root Mean Square Error is the square root of MSE. It is used more commonly than MSE because firstly sometimes MSE value can be too big to compare easily. Secondly, MSE is calculated by the square of error, and thus square root brings it back to the same level of prediction error and makes it easier for interpretation.</blockquote><ul><li>Mean absolute error</li></ul><blockquote>Mean Absolute Error is similar to Mean Square Error. However, instead of the sum of square of error in MSE, MAE is taking the sum of the absolute value of error. Compared to MSE or RMSE, MAE is a more direct representation of sum of error terms. <strong>MSE gives larger penalization to big prediction error by square it while MAE treats all errors the same</strong>.</blockquote><ul><li>R2 Score</li></ul><blockquote>R Square measures how much variability in dependent variable can be explained by the model. R Square value is between 0 to 1 and a bigger value indicates a better fit between prediction and actual value. R Square is a good measure to determine how well the model fits the dependent variables. <strong>However, it does not take into consideration of overfitting problem</strong>.</blockquote><h3>Data Understanding</h3><p>The data is contained in three files:</p><ul><li>portfolio.json — containing offer ids and meta data about each offer (duration, type, etc.)</li><li>profile.json — demographic data for each customer</li><li>transcript.json — records for transactions, offers received, offers viewed, and offers completed</li></ul><p>Here is the schema and explanation of each variable in the files:</p><p><strong>portfolio.json</strong></p><ul><li>id (string) — offer id</li><li>offer_type (string) — type of offer ie BOGO, discount, informational</li><li>difficulty (int) — minimum required spend to complete an offer</li><li>reward (int) — reward given for completing an offer</li><li>duration (int) — time for offer to be open, in days</li><li>channels (list of strings)</li></ul><p><strong>profile.json</strong></p><ul><li>age (int) — age of the customer</li><li>became_member_on (int) — date when customer created an app account</li><li>gender (str) — gender of the customer (note some entries contain ‘O’ for other rather than M or F)</li><li>id (str) — customer id</li><li>income (float) — customer’s income</li></ul><p><strong>transcript.json</strong></p><ul><li>event (str) — record description (ie transaction, offer received, offer viewed, etc.)</li><li>person (str) — customer id</li><li>time (int) — time in hours since start of test. The data begins at time t=0</li><li>value — (dict of strings) — either an offer id or transaction amount depending on the record</li></ul><h4>portfolio dataset</h4><p>Let’s see the information from portfolio dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/837/1*ZW3qQrhC66iyjOwV_1pdwA.png" /></figure><p>We can see above that the portfolio dataset consists of 10 rows and 6 columns. No null values were found.</p><p>For each row in the channels column, there&#39;s a list value. We&#39;re going to split each of them and make one-hot encoding for each.</p><pre>def split_channels(row):<br>    &quot;&quot;&quot;used to split value in the channels col&quot;&quot;&quot;</pre><pre>channel_list = [&#39;email&#39;, &#39;mobile&#39;, &#39;social&#39;, &#39;web&#39;]<br>    output = []<br>    for channel in channel_list:<br>        if channel in row:<br>            output.append(1)<br>        else:<br>            output.append(0)<br>    return pd.Series(output)</pre><pre># split the channels col<br>tmp_channels = portfolio.channels.apply(split_channels) \<br>                .rename(columns={0: &#39;email&#39;,<br>                                 1: &#39;mobile&#39;,<br>                                 2: &#39;social&#39;,<br>                                 3: &#39;web&#39;})</pre><pre># concat the split values to the main df then drop the previous one<br>portfolio = pd.concat([portfolio, tmp_channels], axis=1) \<br>            .drop(columns=&#39;channels&#39;) \<br>            .rename(columns={&#39;id&#39;: &#39;offer_id&#39;})<br>portfolio</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/852/1*4VVRKoAuPUQzMLgI4Sg8cg.png" /></figure><p>Nice. It’s clean for now, at least. Let’s move on to the profile dataset.</p><h4>profile dataset</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/653/1*yVxKBUOvDXUoNkP7Lh_DRA.png" /></figure><p>The dataset consists much more rows than the portfolio dataset. The profile dataset also contains many NaNs. The id column looks inconsistent and could refer to ambiguity. Let&#39;s rename it.</p><p>Let’s see more detailed in the dataset’s null values.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/316/1*2Or4nnjtofaFqEoh8vVHnQ.png" /></figure><p>There’s something interesting, that is the percentage of null values in the gender column is exactly equal to that of the income column. Perhaps, every time gender was null, then the income would be null as well. Let&#39;s check it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/846/1*IbO_9T_6XlTNvRl_4RM8iQ.png" /></figure><p>Correct! The 0 value above means that there is NO single row with gender equal to NaN without income equal to NaN as well. In other words, BOTH gender and income columns will be in NaN at the same time.</p><ul><li>Check the gender column</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/263/1*M2Z9wlulQu-4JLKgxAsNNg.png" /></figure><p>Males dominate the percentage.</p><ul><li>Check the became_member_on column</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/272/1*sBV_vuAD8bNZV2FeMeV8cQ.png" /></figure><p>When we see the data types, the became_member_on column is not in the correct data type. The column should indicate dates, not integer values.</p><pre># change data type of &#39;became_member_on&#39; col<br>profile[&#39;became_member_on&#39;] = pd.to_datetime(profile.became_member_on, format=&quot;%Y%m%d&quot;)</pre><p>From, the date data type, let’s derive other features.</p><pre>profile[&#39;registered_year&#39;] = profile.became_member_on.dt.year<br>profile[&#39;registered_month&#39;] = profile.became_member_on.dt.month<br>profile[&#39;registered_day&#39;] = profile.became_member_on.dt.day<br>profile[&#39;registered_weekday&#39;] = profile.became_member_on.dt.weekday<br>profile[&#39;registered_week&#39;] = profile.became_member_on.dt.week</pre><pre># then drop the original one<br>profile.drop(columns=&#39;became_member_on&#39;, inplace=True)</pre><ul><li>Check the age column</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/499/1*TA11Z1WyA263ZPnS94aZSA.png" /></figure><p>There’s a huge outliers for the age approaching 120.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/232/1*c2uChuVHboe7HmrZfGvMbA.png" /></figure><p>There’s something fishy here. The number of agaes equal to 118 is 2175. Earlier, we found that the number of NaNs for each gender and income column is also 2175. Maybe, they have relationship each other. Let&#39;s check it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/881/1*DkpzZNu0d6c8ho4Hy3XGoA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/687/1*rlXj8Yb3vck6SizMd9RGsQ.png" /></figure><p>Every time a value in gender col is None, the corresponding values in both age and income cols must be 118 and NaN respectively. This pattern might not be a coincidence. This might indicate to those users who registered to the Starbucks app without providing gender, age, and income info. In other words, those values are default values for users with unavailable data.</p><p>Therefore, we should get rid of those NaNs.</p><h4>transcript dataset</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/1*nROFFpGF7fv9Aayg0Q7AvA.png" /></figure><p>The transcript dataset doesn&#39;t have any missing value. The value column has dict-like values. We will break it down little bit later. For the person column, to make it more consistent, we&#39;re going to rename it to customer_id.</p><pre># check unique keys in &#39;value&#39; col<br>tmp_value = transcript.value.apply(lambda row: list(row.keys())) \<br>            .apply(lambda row: row[0] if len(row) == 1 else &#39;, &#39;.join(row))<br>tmp_value.unique()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/623/1*HyTEGLhJjqyFmeLRoAePsQ.png" /></figure><p>There are 4 unique keys in the value column. But that should be just three keys as the offer id and offer_id must refer to the same thing.</p><pre>def split_value_col(dict_row):<br>    &quot;&quot;&quot;used to split the &#39;value&#39; col&quot;&quot;&quot;</pre><pre># get the dict and replace the &#39; &#39; in the &#39;offer id&#39;<br>    dict_row = {k.replace(&#39; &#39;, &#39;_&#39;): v for k, v in dict_row.items()}</pre><pre>available_keys = [&#39;offer_id&#39;, &#39;amount&#39;, &#39;reward&#39;] # set the keys we want</pre><pre># loop through each key to check what key available<br>    output = []<br>    for key in available_keys:<br>        if key in dict_row.keys():<br>            output.append(dict_row[key])<br>        else:<br>            output.append(np.nan)</pre><pre>return pd.Series(output)</pre><pre># split the &#39;value&#39; col<br>tmp_value = transcript.value.apply(split_value_col) \<br>            .rename(columns={0: &#39;offer_id&#39;,<br>                             1: &#39;amount&#39;,<br>                             2: &#39;reward&#39;})</pre><pre># and concat it<br>transcript = pd.concat([transcript, tmp_value], axis=1) \<br>                .drop(columns=&#39;value&#39;)</pre><p>Next, check offer_id column where the value is missing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/689/1*LXSo8k9ETv15LPxdc7nH1Q.png" /></figure><p>The offer_id which is not listed in the portfolio dataset (i.e. indicating by NaN) only occurs at the event equal to transaction and the amount has a value. This is understandable since a transaction made by a customer does not mean an offer is provided.</p><h4>Join all datasets</h4><p>Now, let’s join all three datasets.</p><p>First, let’s join both profile and transcript datasets. The key used here is the customer_id column.</p><pre># join both datasets<br>tmp_df = pd.merge(transcript, profile, how=&#39;left&#39;, on=&#39;customer_id&#39;)</pre><pre>tmp_df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F-oF9Ai9Tpb0fJlPG0LIAA.png" /></figure><p>Next, let’s join the previous result with the portfolio dataset`.</p><pre># change &#39;reward&#39; col name in order to distinguish it with the &#39;reward&#39; col in &#39;transcript&#39; dataset<br>portfolio.rename(columns={&#39;reward&#39;: &#39;reward_std&#39;}, inplace=True)</pre><pre># join then sort its values<br>df = pd.merge(tmp_df, portfolio, how=&#39;left&#39;, on=&#39;offer_id&#39;)<br>df.sort_values([&#39;customer_id&#39;, &#39;time&#39;], inplace=True)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lUqQ9Th9e8YbaLyrezSQTA.png" /></figure><h3>Preprocessing</h3><p>As explained earlier, we assume that the age equal to 118 means default value for unavailable data. Therefore, we’re going to remove those ages.</p><pre># remove incomplete age data<br>df = df[df.age &lt; 118]</pre><p>Next, we’re going to transform the categorical columns. We will implement two kinds of categorical encoding, i.e. the label encoding and one-hot encoding. Our strategy is as follows:</p><p><strong>label encoding:</strong> event and offer_type cols<br><strong>one-hot encoding:</strong> gender col</p><p>The description about when we should use either of them over the other is explained clearly in this <a href="https://www.analyticsvidhya.com/blog/2020/03/one-hot-encoding-vs-label-encoding-using-scikit-learn/">analyticsvidhya article</a>.</p><p>The main reason why we implement label encoding to the offer_type column is because as explained in the beginning, the most offer is BOGO; the least one is informational; while the NaNs values indicate offers for the transaction event. Therefore, we have the ordinal numbers here:</p><ul><li>high: BOGO</li><li>medium: discount</li><li>low: informational</li><li>unwanted: NaN</li></ul><p>For the event column, we&#39;re going to implement the same as the offer_type column. The right order descendingly would be offer completed, transaction, offer viewed, and offer received.</p><pre># create the encode/decode dict<br>offer_type_dict_encode = {np.nan: 0,<br>                          &#39;informational&#39;: 1,<br>                          &#39;discount&#39;: 2,<br>                          &#39;bogo&#39;: 3}<br>offer_type_dict_decode = {v: k for k, v in offer_type_dict_encode.items()}</pre><pre># create the encode/decode dict<br>event_dict_decode = pd.Series(df.event.unique()).to_dict()<br>event_dict_encode = {v: k for k, v in event_dict_decode.items()}</pre><pre>XY = df.copy()</pre><pre># implement label encoding<br>XY[&#39;offer_type&#39;] = XY.offer_type.map(offer_type_dict_encode)<br>XY[&#39;event&#39;] = XY.event.map(event_dict_encode)</pre><pre># one hot encoding<br>XY = pd.concat([XY, pd.get_dummies(XY.gender)], axis=1) \<br>        .drop(columns=&#39;gender&#39;)</pre><pre>XY.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lhRuKkT30L3HEEaf5cYqgA.png" /></figure><p>For modelling, we don&#39;t need identification cols as they cause bias.</p><pre>cols2drop = [‘customer_id’, ‘offer_id’, ‘reward_std’]<br>XY.drop(columns=cols2drop, inplace=True)</pre><p>Our target for modelling will be the amount column. As such a column still has missing values, we&#39;re going to remove them.</p><pre># remove missing values<br>XY = XY[XY.amount.notnull()]</pre><p>For the remaining columns that still have NaNs, we&#39;re going to check the number of unique values of each. If any column has only one unique value, we will drop the column.</p><pre>unimportant_cols = []<br>for col in XY.columns:<br>    # if only 1 unique value<br>    if XY[col].nunique() &lt; 2:<br>        print(col, XY[col].unique())<br>        unimportant_cols.append(col)<br>    else: # if &gt;= 2 unique values<br>        print(col, XY[col].nunique())</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/206/1*ayonJosgPqDI-RBUBOgWOw.png" /></figure><p>As seen above, we don’t need event, reward, difficulty, offer_type, email, mobile, social, and web columns.</p><pre>XY.drop(columns=unimportant_cols, inplace=True)</pre><h3>Modelling</h3><h4>Preparing</h4><p>Let’s see the target distribution.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/1*LzowgoCJVw3nQsrWLCo7tA.png" /></figure><p>There’s a huge gap in the distribution. Let’s see it in boxplot.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/987/1*2xZJlslLdlsrNDWXXfdNUw.png" /></figure><p>A lot of outliers are identified. We can also see them by using the quantiles. There’s a big gap between the median and the max value.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/224/1*QwbzGT_fK4bpd6tsmUPElQ.png" /></figure><pre># set the upper value of IQR<br>Q3 = XY.amount.describe()[&#39;75%&#39;]<br>Q1 = XY.amount.describe()[&#39;25%&#39;]<br>IQR = Q3 - Q1<br>upper = Q3 + (1.5*IQR)</pre><pre># see the proportion of outliers<br>XY[XY.amount &gt; upper].amount.shape[0] / XY.shape[0] * 100</pre><p>Fortunately, the proportion is small, i.e. smaller than 1%. So, it’s not a problem to remove the outliers. Let’s remove those outliers.</p><pre># remove outliers<br>XY = XY[XY.amount &lt;= upper]</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/491/1*4ZR0BtyhRGjGxTtN6XTGuw.png" /></figure><p>Now, it’s much better. Next, let’s see the correlation among all features.</p><pre># Generate a mask for the upper triangle<br>XY_corr = XY.corr()<br>mask = np.triu(np.ones_like(XY_corr, dtype=bool))</pre><pre># Set up the matplotlib figure<br>f, ax = plt.subplots(figsize=(11, 9))</pre><pre># Generate a custom diverging colormap<br>cmap = sns.diverging_palette(230, 20, as_cmap=True)</pre><pre># Draw the heatmap with the mask and correct aspect ratio<br>sns.heatmap(XY_corr, mask=mask, cmap=cmap, vmax=.3, center=0,<br>            square=True, linewidths=.5, cbar_kws={&quot;shrink&quot;: .5})</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/871/1*5Tio4sv8CgOqesrhcDFBlQ.png" /></figure><p>Nice. No multicollinearity found in the dataset, except the correlation between M and F. This occurs since both share the same value, i.e. either 0 or 1.</p><p>However, what makes challenging here is that each correlation between amount and other columns doesn&#39;t have strong value. This will make our models struggling to find accurate prediction.</p><p>Next, divide the final dataset into the input features and target label. Then, split those into train set and test set.</p><pre>X = XY.drop(columns=&#39;amount&#39;).values<br>y = XY.amount.values</pre><pre>X_train, X_test, y_train, y_test = train_test_split(X, y,<br>                                                    test_size=0.075,<br>                                                    random_state=0)</pre><pre># check the proportion<br>print(X_train.shape, X_test.shape)<br>print(y_train.shape, y_test.shape)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/230/1*lWLNWKbxHYB1kU7YUP3K_Q.png" /></figure><h4>Build models</h4><p>We will use 4 types of models, i.e. Random Forest, Gradient Boosing, Auto Sklearn, and Neural Network. For the first two models, we’re going to use scikit-learn library. For the third one, Auto Sklearn library will be used. This library will automate many algorithms from scikit-learn in order to find the best composed algorithm for your problem. Finally, we will implement a neural network with middle-deep network. The library used will be PyTorch.</p><ul><li>Random Forest</li></ul><pre># instantiate the model. just use the default parameters<br>regressor = RandomForestRegressor(random_state=0)<br>regressor.fit(X_train_scaled, y_train) # train the model</pre><ul><li>Gradient Boosting</li></ul><pre># instantiate the model. just use the default parameters<br>regressor_GB = GradientBoostingRegressor(random_state=0)<br>regressor_GB.fit(X_train_scaled, y_train) # train the model</pre><ul><li>Auto Sklearn</li></ul><pre>automl = AutoSklearnRegressor(time_left_for_this_task=3600,<br>per_run_time_limit=600, resampling_strategy=&#39;cv&#39;, resampling_strategy_arguments={&#39;folds&#39;: 4}, seed=0, tmp_folder=&#39;/tmp/autosklearn_regression_tmp&#39;)</pre><pre># this will take up to 1 hour as time_left_for_this_task equal to 3600<br>automl.fit(X_train_scaled, y_train, dataset_name=&#39;starbucks_predict_amount&#39;)</pre><p>Afterwards, we can see the best model found by Auto Sklearn.</p><pre>print(automl.leaderboard())</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/670/1*ohQ0CC5fZD7cQmd3YsUBng.png" /></figure><p>Unfortunately, Auto Sklearn just found 1 best model. If you want Auto Sklearn to find other best model possibilites, you could tune the arguments in the AutoSklearnRegressor() class above.</p><ul><li>Neural Nets</li></ul><p>In order to use PyTorch NN model, we should tune many things. If you want to train the NN model using GPU, but you maybe don&#39;t have a CUDA GPU installed on your local computer, then you could use <a href="https://research.google.com/colaboratory/">Google Colab</a> as I did.</p><p>We’re going use an NN model with architecture as follows:</p><ul><li>ReLU(n_input, 16)</li><li>ReLU(16, 64)</li><li>ReLU(64, 8)</li><li>Linear(8, n_output)</li></ul><p>This architecture is merely arbitrary. You could define better architecture if you want to.</p><pre># define layer<br>class LinearRegression(nn.Module):</pre><pre>def __init__(self, n_feature:int, n_output:int):<br>        super(LinearRegression, self).__init__()<br>        self.hidden1 = torch.nn.Linear(n_feature, 16)   # hidden layer 1<br>        self.hidden2 = torch.nn.Linear(16, 64)   # hidden layer 2<br>        self.hidden3 = torch.nn.Linear(64, 8)   # hidden layer 3<br>        self.predict = torch.nn.Linear(8, n_output)   # output layer</pre><pre>def forward(self, x):<br>        # set activation function for hidden layers<br>        x = nn.functional.relu(self.hidden1(x))<br>        x = nn.functional.relu(self.hidden2(x))<br>        x = nn.functional.relu(self.hidden3(x))</pre><pre>y_pred = self.predict(x) # linear output<br>        return y_pred</pre><pre>#Training loop<br>def train(model: nn.Module,<br>          train_loader: torch.utils.data.DataLoader,<br>          val_loader: torch.utils.data.DataLoader,<br>          n_epochs: int,<br>          lr: float) -&gt; nn.Module:<br>    <br>    #Loss and optimizer<br>    criterion = nn.MSELoss(reduction=&#39;mean&#39;)<br>    optimizer = torch.optim.Adam(model.parameters(), lr=lr)</pre><pre>#Train weigths<br>    for epoch in range(n_epochs):<br>        for inputs, labels in train_loader:<br>            #Zero the parameter gradients<br>            optimizer.zero_grad()</pre><pre>#Forward propagation<br>            pred_y = model(inputs)</pre><pre>#Compute loss<br>            loss = criterion(pred_y,labels)</pre><pre>#Backward propagration to compute gradient<br>            loss.backward()</pre><pre>#Update parameters<br>            optimizer.step()</pre><pre># Evaluate model on validation data<br>        mse_val = 0<br>        for inputs, labels in val_loader:<br>            mse_val += torch.sum(torch.pow(labels - model(inputs), 2)).item()</pre><pre>mse_val /= len(val_loader.dataset)</pre><pre>print(f&#39;Epoch: {epoch + 1}: Val MSE: {mse_val}&#39;)</pre><pre>return model</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/632/1*BWTX3WSZuXrZcKD_26DKYA.png" /></figure><h4>Model Evaluation and Validation</h4><p>As explained earlier, this problem is a regression problem. Therefore, the metrics we used to evaluate all models are those for regression problems. So, we used:</p><ul><li><a href="https://en.wikipedia.org/wiki/Mean_absolute_error">mean absolute error</a></li><li><a href="https://en.wikipedia.org/wiki/Mean_squared_error">mean squared error</a></li><li><a href="https://en.wikipedia.org/wiki/Root-mean-square_deviation">root mean square error</a></li><li><a href="https://en.wikipedia.org/wiki/Coefficient_of_determination">R2 score</a>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/336/1*Qm0AdpBX8WHl93f4g2XOew.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/339/1*BIr3S9uByJ_Zs_soeQ3eJg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/338/1*oRuxb2JHx2mXgjct55JVZw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/337/1*ItHQFj--IMI4nephH745zw.png" /></figure><p>The results from the four models above do not show good performance. Therefore, we should carry out the model improvement.</p><p>As Random Forest performs the best so far, it is chosen for model improvement. We’re going to use scikit-learn&#39;s GridSearchCV() class. Fortunately, the parameters of the estimator used to apply this method are <a href="https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html">optimized by cross-validated grid-search</a> over a parameter grid.</p><p>In this case, I only specify combinations for 2 hyperparameters, i.e. n_estimators and bootstrap.</p><blockquote><strong>PS.</strong> Actually, I&#39;ve tried several times the grid search trainings with the combinations of three, five, and six hyperparameters. But, my local computer always hangs every time I run a lot of grid search parameters combinations. I have tried a few solutions for this issue, i.e. <a href="https://stackoverflow.com/a/47929603/12311481">this solution</a>, <a href="https://stackoverflow.com/a/43195753/12311481">this solution</a>, and <a href="https://stackoverflow.com/a/52686503/12311481">this solution</a>. I also have reduced the n_jobs argument in GridSearchCV() to only 1. But, the issue resists so that I decide to reduce the number of hyperparameters to only two. Thankfully, the issue is solved when I only use combinations for 2 hyperparameters with only 1 n_jobs and use Pipeline() class.</blockquote><p>If you have a machine with more computing resources, you could run much more hyperparameter combinations, and <a href="https://stackoverflow.com/questions/38555650/try-multiple-estimator-in-one-grid-search/53292354">even more regressors</a>.</p><pre># instatantiate model<br>random_forest = RandomForestRegressor(random_state=1)</pre><pre>class DummyEstimator(BaseEstimator):<br>    &quot;&quot;&quot;used to make a dummy estimator for Pipeline&quot;&quot;&quot;<br>    def fit(self): pass<br>    def score(self): pass</pre><pre># set pipe<br>pipeline = Pipeline([<br>    (&#39;regressor&#39;, DummyEstimator())<br>])</pre><pre># set parameter grid<br>parameters = {&#39;regressor&#39;: [random_forest],<br>              &#39;regressor__bootstrap&#39;: (True, False),<br>              &#39;regressor__n_estimators&#39;: (100, 150)<br>             }</pre><pre># instantiate grid search<br>cv = GridSearchCV(pipeline, param_grid=parameters, verbose=2)</pre><pre># see all parameters defined<br>cv.__dict__</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/672/1*qliwAQ6UIciJGedmGF1JPw.png" /></figure><p>From the hyperparameter combinations we defined above, we expect to have a better model for our problem in this project. Let’s train the grid search model.</p><pre># train the grid search model<br>cv.fit(X_train_scaled, y_train)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cdGV9av2AkFz_uSRTM17VA.gif" /></figure><p>In case you’d like to save and load the models trained using Grid Search:</p><pre>import joblib</pre><pre># save all models from grid search<br>joblib.dump(cv, &#39;model/random_forest_cv.pkl&#39;)</pre><pre># or just save the best model<br>joblib.dump(cv.best_estimator_,<br>            &#39;model/random_forest_cv_best.pkl&#39;,<br>            compress = 1)</pre><pre># load model for further usage<br>joblib.load(&quot;model/random_forest_cv.pkl&quot;)<br>joblib.load(&quot;model/random_forest_cv_best.pkl&quot;)</pre><p>The GridSearchCV is trained on the train set. As mentioned by the training result above, there are 20 fits executed in which the number comes from 4 model candidates times 5 folds. Let’s see the grid search results. The results are ranked from the best to the worst.</p><pre>cv_results = pd.DataFrame(cv.cv_results_)<br>cv_results = cv_results.sort_values(&#39;rank_test_score&#39;) \<br>                .reset_index(drop=True)</pre><pre>cv_results.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F-WeZSvo9WTY2TTHgHTBuA.gif" /></figure><pre>print(&#39;Best Random Forest by GridSearchCV:&#39;, end=&#39;\n\n&#39;)</pre><pre>best_one = cv_results.loc[0, :]<br>for col, val in best_one.iteritems():<br>    if isinstance(val, dict):<br>        continue<br>    else:<br>        print(f&quot;{col}: {val}&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/705/1*UEMHYOm23GfCfzADVIx6ZA.png" /></figure><p>Then, let’s test the model on test set.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/337/1*O7p6eeu0ynLlF-Td1a_V_Q.png" /></figure><p><strong>Take away points</strong></p><p>From all models, we can see that each model does not perform good results. Random Forest provides the best performance in this case so far. Even, the NN model which should give better performance is unable to deliver the best performance.</p><p>After we improve the Random Forest model by using Grid Search backed by 5-fold Cross-Validation, resulting in 20 times fits, it is expected the model would have much better performance. However, sadly, its performance just improves slightly.</p><h3>Conclusions</h3><h4>Reflection</h4><p>I found this project challenging, mainly due to the structure of the data in the transcript dataset. My goal is to predict the amount that will be spent by a customer. So, the decision-makers would know how customers behave in purchasing Starbucks products. Therefore, I need all rows with amount existing, i.e. the rows with event col equal to transaction. However, using only this subset of the dataset, in fact, the models I&#39;ve implemented can&#39;t give good performance. Even, by checking the correlations among all columns, we are unable to capture any strong relationship between the amount and other columns.</p><p>Perhaps, in order to get broader factors, we should consider the other events, i.e. offer received, offer viewed, and offer completed. But, if we include the three, we will have many missing values. This is contradictory.</p><h4>Potential improvements</h4><p>Due to time reasons, I don’t have a chance to perform enhancement on data or model tuning. For example, I could do more experiments on feature engineering to see if adding/substracting features can improve the model; or I could try other combinations of model hyperparameters to see if this will affect the model performance.</p><p>In addition, the analysis and modelling should also consider the condition of customers who don’t make any transaction as such a condition also provide more insight.</p><p>Last but not least, the computing resources should also be considered. In my case, where I need GPU computing, I have to go to Google Colab. But unfortunately, when I need more CPU computings for Grid Search, I ‘ve found Google Colab doesn’t provide multiprocessing. Likewise, my local computer also doesn’t give sufficient multprocessing computing performance for the case. I think, for handling real world data for enterprises or startups, it wouldn’t be excessive to rent any cloud computing.</p><h3>Acknowledgments</h3><p>Acknowledgment should go to Starbucks for providing the dataset. This project is a capstone project of <a href="https://www.udacity.com/course/data-scientist-nanodegree--nd025">Data Scientist Nanodegree on Udacity</a>.</p><h4>To see more details on the analysis, you can find it on my GitHub repo <a href="https://github.com/utomoreza/Udacity_Data-Scientist-Nanodegree/tree/main/Project_Capstone">here</a>.</h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=242dc22c335c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Much Do You Know to Enter Data Science Field?]]></title>
            <link>https://medium.com/geekculture/how-much-do-you-know-to-enter-data-science-field-1df95baec9da?source=rss-fadfbe6f0eaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/1df95baec9da</guid>
            <category><![CDATA[data]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[udacity]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Reza Dwi Utomo]]></dc:creator>
            <pubDate>Thu, 26 Aug 2021 06:45:18 GMT</pubDate>
            <atom:updated>2021-08-26T13:17:11.669Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*xAJos1GO6etsuhlq" /><figcaption>Photo by <a href="https://unsplash.com/@adliwahid?utm_source=medium&amp;utm_medium=referral">Adli Wahid</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Introduction</h3><p>Since a couple of years ago, Data Science’s hype has been increasing. Many are trying to enter the field, no matter who they are, man or woman. However, as the field is one of the technology fields in which usually men dominate the workforce, if you are a female, do you have a chance or similar opportunities in driving into the data science field?</p><p>In addition, as the field has various roles, e.g. Data Scientist, Data Analyst, Data Engineer, Machine Learning (ML) Engineer, etc., don’t you think what skills are required to become one of them? Or even if you were to master most skills needed for all of the four, will you earn more?</p><p>To satisfy our curiosity, therefore, in this article, we’re interested to answer the following questions:</p><ul><li>How promising data field for women compared to men?</li><li>What are the skills needed to become a data scientist, data engineer, data analyst, or ML engineer?</li><li>By becoming a Full Stack Data Professional, do you earn more?</li></ul><h3>Contents</h3><p>After the chapter above, you will find the rest chapters listed below.</p><ul><li>Preparing Data</li><li>Question 1</li><li>Question 2</li><li>Question 3</li><li>Conclusions</li></ul><h3>Preparing Data</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/240/0*T05KVrS_1ldMpMxj.png" /></figure><p>To answer the questions, we can use the latest survey results released by Kaggle, i.e. <a href="https://www.kaggle.com/c/kaggle-survey-2020/overview"><em>“2020 Kaggle Machine Learning &amp; Data Science Survey”</em></a>. The data is set out to conduct an industry-wide survey that presents a truly comprehensive view of the state of data science and machine learning. You could find the dataset <a href="http://localhost:8888/notebooks/Udacity%20-%20Data%20Scientist%20Nanodegree/Project%201/datasets/kaggle_survey_2020_responses.csv">here</a> or download it from its website <a href="https://www.kaggle.com/c/kaggle-survey-2020/data">here</a>.</p><p>Besides, to understand more the data, it is recommended to read the supplementary data as it contains two PDF files explaining the questions asked to the respondents and the methodology used in the survey. From the methodology file, we can infer that all empty data do not mean incomplete or corrupt data, by contrast, that means the respondents indeed could not answer given questions.</p><p>If we peek at the dataset using a Spreadsheet, we can find its structure is something like the picture below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U_NPDpirRzK9h1JjHVomjQ.png" /></figure><h3>Question 1: “How promising data field for women compared to men?”</h3><p>To answer this question, below are the aspect of interest I think we need:</p><ul><li>Age</li><li>Education</li><li>Role</li><li>Experience</li><li>Salary</li></ul><p>We will see the comparison of women and men in the above aspects.</p><h4>Age</h4><p>We will also calculate the ratio for each woman and man. This ratio indicates the proportion in each woman and man data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/511/1*y8BFNxi49uQ87mRx4Z3gjQ.png" /></figure><p>To make this finding clearer, it will be better if we plot this result in a pyramid plot that compares both woman and man data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/915/1*DZt4Y_xCO8rZVFS6pw_TuA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/910/1*F0nPB9G9YaH7Y0oGE3tX_A.png" /></figure><p>From the chart of counts above, we can see that in the case of focusing only on the counts, men dominate for all age ranges. This phenomenon is common in any field of information technology, including data science. However, if we look at the data from a different angle where we calculate the ratio for each age group, we will find something interesting as shown by the second chart.</p><p>Those ratios show the proportion of certain age groups from total data in given gender. We can see that even though the counts of women data are less than those of men, their proportions indicate a different thing. Most higher proportions are dominated by women with ages between 18–21, 22–24, and 25–29. Even these three proportions overcome those of men. From this finding, we could infer that <strong>gradually women are more interested in the IT field, such as data science, in particular young women.</strong></p><h4>Education</h4><p>We’ve found that by looking at the age, young women gradually are interested in the data science field. But, will this happen on the education aspect as well? We’re going to do the same as we did earlier on the age data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3Y38a1kaqZdVLGpauS2YZQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vCXDdfK6x0Q-K3ePQzsCzQ.png" /></figure><p>Again, if we only focus on the counts, it’s obvious that men win the comparison. But, the ratios tell different things. Both women and men with Master’s degrees as their last education are higher in number than other degrees.</p><p>To make this more interesting, let’s check out whether women with Master’s degree as their last education comes from a youngster.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/154/1*D7Px_EeIC2YJNIeAQbKXfQ.png" /></figure><p>Great. Those women with a Master’s degree as their last education are the highest in number as shown by the table above. This is another evidence to prove that progressively young women are more interested in diving in the field.</p><h4>Role</h4><p>Let’s do the same for data in the role aspect.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iawzfoy3y7bLF3RdicQF-A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WzRUj6y_AN_wa5x_tA6LUA.png" /></figure><p>As usual, we’ve found the same thing for the count plot. But, for the plot of the ratio, what we’ve found really strengthens our hypothesis that young women are gradually interested in the data science field. The ratio plot above indicates that <strong>there are a huge number of aspiring data scientists</strong> that are currently still a student who is usually a young woman. To prove that students are usually youngsters, let’s explore more the data of women students.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/154/1*EUZfutACb5xovkfDaRJUPQ.png" /></figure><p>Excellent. This is what we expected. The woman students are dominated by young women as shown by the table above. This is another evidence to prove that progressively young women are more interested in diving in the field.</p><h4>Experience</h4><p>Let’s do the same for data in the experience aspect. Here, we have two kinds of experience: coding experience and machine learning (ML) experience.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0MJGTnAvR3E7wdM94p_jCg.png" /><figcaption>Coding experience</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4XNtR-KSWu5EcFk1admPIg.png" /><figcaption>ML experience</figcaption></figure><p>For both coding and ML experience data, as usual, men dominate the data for every experience range. However, we’ve found that aspiring female data scientists or women with little coding and ML experience dominate the data. Those with minimum experience should be youngsters. But, to prove this hypothesis, let’s check the age for this proportion.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/157/1*DjHmO6yVXkDjkg0WdwgAeA.png" /><figcaption>Age group for women with minimum coding experience</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/154/1*BREc45l8NW4MNxIKiIUMfw.png" /><figcaption>Age group for women with minimum ML experience</figcaption></figure><p>Excellent. This is what we expected. The women with minimum experience both in coding and machine learning are dominated by young women as shown by both tables above. This is another evidence to prove that progressively young women are more interested in diving in the field.</p><h4>Salary</h4><p>Let’s do the same for data in the salary aspect. This aspect is the one we’ve been waiting for.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/983/1*-vf6YIbGl-82__eqvF-mbA.png" /></figure><p>From both charts above, we can see, again, men always dominate in terms of numbers. But, if focus on the ratio chart, there is a large proportion for the least salary range in women data. This proportion should be for those women with minimum experience since usually less experienced employees earn less than the more experienced ones. However, to prove this hypothesis, let’s explore this proportion where the gender is woman and salary range is between 0 and 999 USD.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/270/1*Lyqk4srQUPzoEM3WDkWBNw.png" /><figcaption>Coding experience group for women with salaries between 0 and 999 USD</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/359/1*3AXR3s5dddZIWJ6SkJhv7A.png" /><figcaption>ML experience group for women with salaries between 0 and 999 USD</figcaption></figure><p>Great! Our hypothesis is proved. the proportion is dominated by women with minimum experience both in coding and machine learning. Their experience ranges from under 1 year to below 3 years.</p><h3>Question 2: “What are skills needed to become a data scientist, data engineer, data analyst, or ML engineer?”</h3><p>To answer this question, below are the columns of interest I think we need:</p><ul><li>Q5 role</li><li>Q7 programming language (many columns)</li><li>Q8 programming language recommended</li><li>Q12 specialized hardware (many columns)</li><li>Q14 data viz library (many columns)</li><li>Q16 ML framework (many columns)</li><li>Q17 ML algorithm (many columns)</li><li>Q18 CV method (many columns)</li><li>Q19 NLP method (many columns)</li><li>Q23 important activity (many columns)</li><li>Q26 A/B cloud computing platform (many columns)</li><li>Q27 A/B cloud computing product (many columns)</li><li>Q28 A/B ML product (many columns)</li><li>Q29 A/B big data product (many columns)</li><li>Q30 big data product used most often</li><li>Q31 A/B BI tool (many columns)</li><li>Q32 BI tool used most often</li><li>Q33 A/B AutoML category (many columns)</li><li>Q34 A/B AutoML (many columns)</li><li>Q35 A/B ML experiment (many columns)</li><li>Q36 deploy (many columns)</li></ul><p>Here, we should combine all skill columns into several groups of skill. After looking at those skill columns thoroughly, I’ve found they can be classified into the following groups:</p><ul><li>Main Activity: Q23</li><li>ML Deployment: Q36</li><li>Database &amp; Big Data: Q29 and Q30</li><li>Data Visualization: Q14, Q31, and Q32</li><li>AI: Q18, Q19</li><li>ML Modeling: Q16, Q17, Q33, Q34</li><li>Programming: Q7 and Q8</li><li>Cloud: Q26, Q27, Q28</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Iy-GkiNSjRqRjdwtuNgNiQ.png" /></figure><p>From the radar chart above, we can see that each role has a different strength of skills. For example, ML Engineer is the one with the most skilled in ML Deployment, AI, and ML Modeling; Data Engineer is strong in Cloud skill, and Data Analyst is the one with Data Visualization skill the most.</p><p>The unique one I think is the Data Scientist role. It does not have the strongest skill in any common skills. Instead, it shares almost the same power for every aspect. Perhaps, this can support the theory where <strong>the Data Scientist role is a multidisciplinary job</strong>.</p><p>In addition, another interesting finding is that it can be seen that all roles have the same share for Programming skills. This can infer that <strong>no matter what role you’re interested in the data science field, programming skill is a must</strong>.</p><p>We have done with common skills in data science roles. Now, let’s see what we will find with the “Main Activity” group.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3NsYhvNWh2nw0uSqUT118Q.png" /></figure><p>Unlike the chart of common skills earlier, the above plot of common activities shows more varied results. We can see that each role indeed has distinguished activities. We can also find that each of them has a different emphasis.</p><p>Data Analyst, for example, as we can expect, is the one who spends his/her time mostly on analyzing and understanding data to influence product or business decisions. Even this activity climbs far enough from other roles. Data Engineer, on the other hand, spends his/her time mostly on building and/or running the data infrastructure that his/her business uses for storing, analyzing, and operationalizing data. This is what we can expect from a data engineer.</p><p>ML Engineer, as its name, focuses more on ML part activities, such as researching the state-of-the-art of machine learning and building and/or running ML services. For Data Scientist, we can see that this role intersects with most ML Engineer activities. What differentiates Data Scientist activities from ML Engineer activities is the activity of analyzing and understanding data. The only top activity for Data Scientist is only for building prototypes to explore applying machine learning to new areas.</p><h3>Question 3: “By becoming a Full Stack Data Professional, do you earn more?”</h3><p>Like Full Stack Developers who can master and perform both front-end and back-end in web development, we can also refer a data professional who does the upstream and downstream tasks in the data field as a Full Stack Data Scientist/Professional. Like a swiss-army knife which serves many tools in a single knife, we expect a Full Stack Data Scientist as a ‘one-man army’ who can do all things regarding the data field.</p><p>So, by mastering many things, a Full Stack Data Scientist, I think, should earn more money compared to other common roles. But, is that so? Let’s prove it.</p><p>To answer this question, below are the columns of interest I think we need:</p><ul><li>Q5 role</li><li>Q7 programming language (many columns)</li><li>Q8 programming language recommended</li><li>Q10 hosted notebook (many columns)</li><li>Q12 specialized hardware (many columns)</li><li>Q14 data viz library (many columns)</li><li>Q16 ML framework (many columns)</li><li>Q17 ML algorithm (many columns)</li><li>Q18 CV method (many columns)</li><li>Q19 NLP method (many columns)</li><li>Q23 important activity (many columns)</li><li>Q24 salary</li><li>Q26 A/B cloud computing platform (many columns)</li><li>Q27 A/B cloud computing product (many columns)</li><li>Q28 A/B ML product (many columns)</li><li>Q29 A/B big data product (many columns)</li><li>Q30 big data product used most often</li><li>Q31 A/B BI tool (many columns)</li><li>Q32 BI tool used most often</li><li>Q33 A/B AutoML category (many columns)</li><li>Q34 A/B AutoML (many columns)</li><li>Q35 A/B ML experiment (many columns)</li><li>Q36 deploy (many columns)</li></ul><p>We’d like to have data where no redundant columns (but no value removed) and no NULL value are found in each row for every column of interest.</p><p>After pre-processing the data, let’s see which role has the most skills.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/276/1*PbUicBPaBGwVig1xPi7ZYQ.png" /></figure><p>So, the top 4 roles are Data Scientist, ML Engineer, Software Engineer, and Data Analyst. Let’s explore the salary for those four roles.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/695/1*BbpaCFxbA8AeDpfYdCC7Eg.png" /></figure><p>From the table above, it is risky to conclude that mastering most data skills could lead you to make more salary. Only an extremely little proportion of people is lucky enough to earn more money. To make this clearer, let’s see the final mean for each role.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/317/1*jWQSZNj0AXu5xD9EunOiYQ.png" /><figcaption>Average salary for most skilled roles</figcaption></figure><p>Great. We can see the final average for each role. All roles, except Software Engineer, make more than 40K USD a year on average if they master most skills in the data science field. For Software Engineer, I think it’s self-explanatory that this role earns the least here. This role works as a software engineer, therefore the skills necessary for it to earn more are the skills for software engineering, not skills for data science.</p><p>In order to understand whether the averages above are worth more money than the common data professionals, it’s necessary to compare the results with those from the common data professionals.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/318/1*yGkZU3gicRAA6FmSszh87w.png" /><figcaption>Average salary for common skilled roles</figcaption></figure><p>Unexpectedly! The final averages for every role of the common ones are much lower than those from roles with the most skilled mastered. From this finding, we can conclude that it is valuable and useful enough if you decide to master most skills in the data science field as you will have more chance to earn more money than those who have general knowledge.</p><h3><strong>Conclusions</strong></h3><p>We have done many interesting things. We’re interested in the data science field and attempted to explore those who are in the field about who they are, what they do, what they can, what they earn, etc. Kaggle Survey 2020 is the dataset we used for this exploration. You could also explore other Kaggle surveys, i.e. <a href="https://www.kaggle.com/c/kaggle-survey-2019?rvi=1">2019</a>, <a href="https://www.kaggle.com/kaggle/kaggle-survey-2018?rvi=1">2018</a>, and <a href="https://www.kaggle.com/kaggle/kaggle-survey-2017?rvi=1">2017</a> surveys, if you’d like to find comparisons or trends.</p><p>We focus our interest on three questions:</p><ul><li>Women in the data science field</li><li>Skills needed</li><li>Full-stack data scientist</li></ul><p>We have tried our best to explore the data to answer the questions. Many useful functions, dataframes, and plots are utilized to deliver the best data understanding. And finally, we’ve arrived in these conclusions.</p><p>So, for the 1st question “<em>How promising data field for women compared to men?</em>”, what we’ve found are as follows:</p><ul><li>Still, like other IT fields, the data science field also suffers from women enthusiasts. If you found men in any IT field, it’s not obvious. But, this does not apply to women.</li><li>However, there is a slight breeze. Young women, on the other hand, evidently are gradually interested in the data science field more than those older women.</li><li>This finding is evidenced when we explored the data in age, education, job role, experience, and salary aspects.</li></ul><p>For the next question “<em>What are skills needed to become a data scientist, data engineer, data analyst, or ML engineer?</em>”, our findings are the following:</p><ul><li>We split the data into two parts: common skills and activities.</li><li>We utilized two radar charts to plot common skills and common activities for the roles.</li><li>From the first chart, we’ve found that each role has a different strength of skills.</li><li>However, unlike other roles, Data Scientist shows a unique result. It does not have the strongest skill in any common skills. Instead, it shares almost the same power for every aspect.</li><li>This data scientist phenomenon could be evidence for the theory where the Data Scientist role is a multidisciplinary job.</li><li>Besides, all roles have the same share for Programming skills. This can infer that no matter what role you’re interested in the data science field, programming skill is a must.</li><li>From the second chart, it can be seen that each role has distinguished activities. Also, each of them has a different emphasis.</li><li>ML Engineer and Data Scientist roles intersect each other on most activities, except for the activity of analyzing and understanding data.</li></ul><p>For the last question, “<em>By becoming a Full Stack Data Professional, do you earn more?</em>”, below are our conclusions:</p><ul><li>We filtered the dataset to only contain respondents with no NULL skills. That means the most skilled respondents.</li><li>We’ve found that the top 4 roles with the most skills are Data Scientist, Machine Learning Engineer, Software Engineer, and Data Analyst in ascending order.</li><li>All roles, except Software Engineer, make more than 40K USD a year on average if they master most skills in the data science field.</li><li>When the average salary results of those top roles with most skills mastered compared to the data of common data professionals, unexpectedly, the common data professionals’ salaries are much lower than those from roles with most skilled mastered.</li><li>It can be concluded that it is valuable and useful enough if you decide to master most skills in the data science field as you will have more chance to earn more money than those who have general knowledge.</li></ul><h3>Acknowledgments</h3><p>Acknowledgment should go to <a href="https://www.kaggle.com/c/kaggle-survey-2020/overview">Kaggle</a> for providing the dataset. This article is one of the Projects of <a href="https://www.udacity.com/course/data-scientist-nanodegree--nd025">Data Scientist Nanodegree on Udacity</a>. If you’d like to access the codes of how I processed the results in this article, you can visit my repo on GitHub <a href="https://github.com/utomoreza/Udacity_Data-Scientist-Nanodegree/blob/main/Project_1">here</a>. Feel free to explore it.</p><p>Thanks for reading!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1df95baec9da" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/how-much-do-you-know-to-enter-data-science-field-1df95baec9da">How Much Do You Know to Enter Data Science Field?</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>