<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ilham Firdausi Putra on Medium]]></title>
        <description><![CDATA[Stories by Ilham Firdausi Putra on Medium]]></description>
        <link>https://medium.com/@ilhamfp?source=rss-9468d31b22c7------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*cNgAdv_aWXIE0AvkFHBexQ.jpeg</url>
            <title>Stories by Ilham Firdausi Putra on Medium</title>
            <link>https://medium.com/@ilhamfp?source=rss-9468d31b22c7------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 19:38:20 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ilhamfp/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Overfitting the Leaderboard in Ernst & Young Data Science Competition 2019]]></title>
            <link>https://medium.com/hmif-itb/overfitting-the-leaderboard-da25172ac62e?source=rss-9468d31b22c7------2</link>
            <guid isPermaLink="false">https://medium.com/p/da25172ac62e</guid>
            <category><![CDATA[competition]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[overfitting]]></category>
            <category><![CDATA[kaggle-competition]]></category>
            <category><![CDATA[data-science-competition]]></category>
            <dc:creator><![CDATA[Ilham Firdausi Putra]]></dc:creator>
            <pubDate>Mon, 15 Jul 2019 09:14:11 GMT</pubDate>
            <atom:updated>2019-07-15T09:19:28.794Z</atom:updated>
            <content:encoded><![CDATA[<h4>And subsequently losing 8000 USD + a ticket to New York.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gYqTNKJwlhAlGar06QLomQ.png" /><figcaption><em>A bittersweet journey at the latest Ernst &amp; Young NextWave Data Science Competition 2019.</em></figcaption></figure><p>The image above depicts the dramatic free fall of our rank in the global leaderboard. A lot was at stake here. We have sacrificed 40 days pursuing 8000 USD and a ticket to New York — the 1st place prize for the global leaderboard. And to make things even more so anticlimactic, we had dominated the public leaderboard ever since the competition began. Since two weeks before the deadline, our position was set in stone at the 1st place on the leaderboard until the bitter truth, that is <strong>the private leaderboard</strong>.</p><p>Fortunately for us, while we were down to 9th place in the global leaderboard, we were still 1st place on Indonesia local leaderboard. We were rewarded with Rp 25.000.000,- (~1800 USD) and a paid internship offer. Thank you, EY!</p><p>So, how did we end up there? Let’s join me on our (<a href="https://medium.com/@christianwbsn">Christian Wibisono</a> and I) journey at the latest Ernst &amp; Young NextWave Data Science Competition 2019.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/650/0*ErQ70Ss8gvpWCxus.jpg" /><figcaption>Don’t count your chickens before they hatch — the higher your expectations, the greater your disappointment. <a href="https://www.expertrain.com/blog/health/things-you-never-knew-about-eggs.htm">source</a></figcaption></figure><h3>The Competition</h3><blockquote>tl;dr: An awesome international data science competition held by Ernst &amp; Young. Competing against undergraduate and postgraduate from all over the world for 40 days. Given anonymized geolocation records, predict whether the device will be within the city center or not.</blockquote><p>The Ernst &amp; Young NextWave Data Science Challenge 2019 focuses on how data can help the next smart cities thrive, and boost the mobility of the future. Global urbanization is on the rise, with more than 50% of the world’s population living in cities. According to the UN, that number will reach 60% by 2030 — that’s nearly 1.5 billion more than in 2010.</p><p>While this trend creates excellent opportunities for cities, it also presents challenges to governments on how to upgrade infrastructure, alleviate congestion, and address pollution. Electric and autonomous vehicles, along with the explosion of the ride-sharing economy, are helping to address these challenges. Furthermore, they also disrupt mobility and demand innovative solutions.</p><p>In parallel, public authorities have more information than ever on how citizens move around in the city. However, a gap exists between having this data and using it to improve the user travel experience for citizens. Forward-looking authorities have a chance to innovate infrastructure to make their city a better place to live in a better working world.</p><p>Here was our chance to narrow that gap. As a challenge participant, we would be able to download a dataset with a vast number of anonymous geolocation records from the US city of Atlanta (Georgia), in October 2018. Our task was to produce a model that helps authorities understand the journeys of citizens while they move in the city throughout the day. Hopefully, our work could inspire solutions that help city authorities anticipate disruptions, make real-time decisions, design new services, and reshape infrastructures.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SLyGQgiiiE7jEzG1gKDKxA.png" /><figcaption>Data illustration. Source: Challenge Manual</figcaption></figure><p>We were given access to a data file `data_train.csv` that contained anonymized geolocation data of multiple mobile devices in the City of Atlanta for 11 working days in October 2018. Every device ID represents a 1-day journey, which consists of multiple trajectories. A trajectory is defined as the route of a moving person in a straight line with an entry and an exit point.</p><p>We must predict how many people were in the city center between 15:00 and 16:00. The test dataset contained several devices in which the trajectories after 15:00 had been removed — all but one: After 15:00, we would find one last trajectory, with (1) entry location, (2) entry and exit time between 15:00 and 16:00. The exit point, however, had been removed. Simply put, our task was to predict the location of this last exit point and whether this device was within the city center or not. The target variable is the latter.</p><p>For 40 days, every day each team — consisting of up to two persons from undergraduate or postgraduate all around the world — had a chance to submit five submissions containing the target variable and its device ID. The submission would be automatically evaluated using F1-score between the predicted and the observed target. The results would be compared to the real data using both public and private datasets. In the challenge ranking, we would be able to see the score based on only the public dataset. In the final leaderboard, only the last submission will be considered.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CjcofPiDYwdKfwHnW6oKkA.png" /><figcaption>The global leaderboard hours before the deadline.</figcaption></figure><h3>What Went Well?</h3><p>In retrospect, these are things that allow us to perform quite well.</p><h4>Collaboration</h4><blockquote>Winners are those who went through *more iterations* of the “loop of progress” — going from an idea, to its implementation, to actionable results. So the winning teams are simply those able to run through this loop *faster*. — <a href="https://twitter.com/fchollet/status/1113479155847270400">François Chollet</a>, creator of Keras.</blockquote><p>The above quote resonates strongly with us. At the end of the competition, we had 144 submissions and only missed a couple of days because the timeline heavily overlaps with our final semester exam and assignment. Each day we would plan how many of the five available daily submissions each of us was going to take. And for each submission, we wrote down details such as data version, list of feature, classifier, cross-validation score, and leaderboard result in <a href="https://quip.com/">Quip</a>. We also structured our repository &amp; code in a way that allows us to swap things out at <a href="https://github.com/">GitHub</a> easily. By teaming up, we manage to try more thoughts and ideas.</p><h4>Kaggle kernel</h4><p>One of the most significant constraints competing in a data science competition is computing power. In this competition, the data size is close to 200MB initially and over 400MB after feature engineering. That amount of data is just way too much for my poor laptop — it would quickly overheat and scream in pain.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/204/1*xn2ZZNTirb9mjOmBbtSj_Q.png" /><figcaption>Me a̶b̶u̶s̶i̶n̶g̶ utilizing Kaggle kernel to its fullest.</figcaption></figure><p>Fortunately, <a href="https://kaggle.com/">Kaggle</a>, an online website of data scientists and competition, provides an awesome feature called Kernel, FOR FREE. Kaggle kernel is basically like having your Jupyter Notebook/script run on the cloud. It has excellent <a href="https://www.kaggle.com/docs/kernels">specs</a> and allows your script to run up to 9 hours.</p><p>Simply upload your dataset and script to Kaggle, and you can run up to 8 experiments simultaneously! Having this FREE (<em>did I forget to mention that?</em>) computing power really helped us as each experiment from start to end could take more than 2 hours.</p><h4>Feature engineering</h4><p>This one is critical as it directly corresponds with how well our machine learning model is going to perform. Given the challenge was to predict whether a device was within the city center or not, we figured out that distance was going to play a significant role. So, one set of features we computed was the following:</p><ol><li>First, we calculated the duration of each trajectory.</li><li>Then, we computed the distance traveled for each non-final trajectory.</li><li>Given both the total duration and total distance traveled of non-final trajectory, we could compute the average velocity of each device.</li><li>After that, we computed the distance needed to exit/enter the city center of final trajectory.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4TlDkyfipO427D27r6dFsQ.png" /></figure><p>5. Finally, we could calculate using the formula on the left to indicate how likely the device could exit (or enter, depending on the value given) the city center based on its past average velocity and duration of the final trajectory.</p><p>This set of feature ended up as our top most impactful feature based on <a href="https://github.com/slundberg/shap">SHAP</a>. Another noteworthy one was a set of features that indicate its tendency to stay inside or outside the city center.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GH9lwM4OLMkwcMdqbVmcIQ.png" /><figcaption>SHAP top five feature.</figcaption></figure><h3>What Went Wrong?</h3><p>It was the cornerstone of an inept data scientist, stemming from a complete lack of experience. The main ingredient of a classic shake-up between public and private leaderboard score:</p><blockquote>We did not trust our cross-validation score</blockquote><p>— and the result was catastrophic. We had overfitted the public leaderboard.</p><p>It’s easy to say that you believe in something until it doesn’t align with what you see. In our case, we believed in our cross-validation score as it was in line with our leaderboard score — until it wasn’t. At that point, any experienced data scientist competition enthusiast or Kaggler would know that it was perfectly reasonable and possible. Our lack of experience caused us to falter and eventually lose the position.</p><p>The public leaderboard was calculated with only a fraction of the total test data, which after the competition we know was around 1/3 public and 2/3 private. In hindsight, we should have always trusted our local cross-validation, given that we had a sound implementation, to guide our way. This incompetence caused us to possibly drift from a better solution and fail to choose our best solution as the final one.</p><h3>Epilogue</h3><p>It wasn’t all that grim. It was a fantastic competition, and we learned a lot during the process. In the end, we ended up with a handful of cash and a whole lot of experience.</p><p>I’m eternally grateful to everyone who made this possible. Thank you Ernst &amp; Young for organizing this event. And finally, thank you for reading. I hope this story can be useful for you.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=da25172ac62e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hmif-itb/overfitting-the-leaderboard-da25172ac62e">Overfitting the Leaderboard in Ernst &amp; Young Data Science Competition 2019</a> was originally published in <a href="https://medium.com/hmif-itb">HMIF ITB Tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deepfake dan Konsekuensinya]]></title>
            <link>https://medium.com/@ilhamfp/deepfake-dan-konsekuensinya-525e39d02519?source=rss-9468d31b22c7------2</link>
            <guid isPermaLink="false">https://medium.com/p/525e39d02519</guid>
            <category><![CDATA[hoax]]></category>
            <category><![CDATA[teknologi]]></category>
            <category><![CDATA[kecerdasan-buatan]]></category>
            <category><![CDATA[pembelajaran-mesin]]></category>
            <category><![CDATA[deepfakes]]></category>
            <dc:creator><![CDATA[Ilham Firdausi Putra]]></dc:creator>
            <pubDate>Mon, 15 Jul 2019 06:21:42 GMT</pubDate>
            <atom:updated>2021-09-09T03:33:08.378Z</atom:updated>
            <content:encoded><![CDATA[<h4>Seiring dengan kekuatan yang besar, datang juga tanggung jawab yang besar.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*giJEtgVfAIckuwYqGVglGw.gif" /><figcaption><a href="https://www.youtube.com/watch?v=614we6ZaQ04">Cuplikan asli dari film Rogue One.</a></figcaption></figure><p>Cuplikan di atas adalah bagian dari film <a href="https://en.wikipedia.org/wiki/Rogue_One">Rogue One</a> (2016), di mana wajah muda aktris Carrie Fisher dengan cemerlang diciptakan kembali secara digital sebagaimana ia dulunya pada tahun 1977. Hal ini dilakukan dengan memetakan wajah aktris Ingvild Deila, yang memerankan aksi tersebut, lalu melakukan proses penyuntingan secara digital. Proses tersebut tidak lah mudah dan murah, hal ini dibuktikan dengan anggaran pembuatan film tersebut yang mencapai 265 juta Dollar! Sebuah angka yang fantastis bagi para pembuat film, terutama bagi para amatir. Namun efek sakti dari CGI tersebut akan segera menjadi hal yang lumrah dengan datangnya Deepfake.</p><h3>Apa itu Deepfake?</h3><p>Deepfake, gabungan dari “deep learning” dan “fake”, adalah sebuah teknik sintesis citra manusia berbasis kecerdasan buatan. Hal ini dilakukan dengan cara menggabungkan gambar-gambar yang sudah ditimpa dengan wajah target untuk menjadikannya sebuah video. Sederhananya, pengguna mengumpulkan foto-foto dari orang yang ada di video dan target yang akan ditimpa. Lalu dengan model <a href="https://arxiv.org/pdf/1611.09577.pdf">Convolutional Neural Networks</a>, kita melatih model menggunakan wajah dari kumpulan foto yang sudah diekstrak. Terakhir, ubah setiap wajah yang ada pada tiap frame di video menjadi wajah target. Hasil gambar per-frame yang sudah ditimpa tersebut akan digabungkan menjadi sebuah video utuh yang mana wajah pada video tersebut sudah ditimpa dengan wajah target. Semua hal ini dapat dilakukan dengan mudah dan gratis pada <a href="https://www.deepfakes.club/tutorial/">tutorial</a> yang sudah tersedia online.</p><p>Deepfake menjadikan teknologi faceswapping dapat tersedia bagi semua orang dengan mudah. Lihat hasil yang didapatkan oleh sebuah user reddit dengan nama “derpfakes” pada <a href="https://www.youtube.com/watch?v=614we6ZaQ04">videonya</a>. Ia mengaku bahwa hasil tersebut didapatkan dengan menggunakan komputer standar dan diselesaikan dalam waktu yang kurang lebih sama dengan 1 episode The Simpson (± 24 menit).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*lFlm8Stc9NZ4s8bLame0Zg.gif" /><figcaption><a href="https://www.youtube.com/watch?v=614we6ZaQ04">Cuplikan palsu dari derpfakes</a>.</figcaption></figure><p>Meskipun ia mengaku bahwa penurunan kualitas video terjadi, tetapi hal ini tetap impresif mengingat hasil ini didapat dengan komputer standar dan set data yang di dapat dari gambar online. Perlu ditekankan juga bahwa teknologi ini masih baru dan akan terus berkembang, meningkatkan kualitas hasilnya. Bandingkan hasilnya:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*Eg-krJBaH7beVzyD5mhbDQ.png" /><figcaption>Asli</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*rUfufKfE0DpO5HelT6WLOA.png" /><figcaption>Palsu</figcaption></figure><p>Adanya Deepfake membuka banyak peluang berkarya. Kemungkinannya tak berhingga. User reddit yang sama, derpfakes, <a href="https://www.deepfakes.club/nicolas-cage-deep-learning-superstar/">mengubah berbagai macam wajah aktor film menjadi wajah Nicolas Cage</a>. Ini salah satu hasilnya:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*5NOQUaJYIMpxlroJbgjQjQ.gif" /><figcaption><a href="https://www.youtube.com/watch?v=hEp1zdgMQho&amp;t=67s">Asli</a></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*-fL9JgNf1wvbPMWghe3VmQ.gif" /><figcaption><a href="https://www.youtube.com/watch?v=ChVgT980Jck">Palsu</a></figcaption></figure><h3>Konsekuensi</h3><p>Adanya teknologi mempermudah proses penyuntingan video. Hal ini juga berarti mempermudah niat orang-orang yang tidak baik. Mungkin yang paling rentan adalah selebriti, politikus, dan tokoh-tokoh lainnya yang sering berada di depan kamera. Dikarenakan banyaknya sumber gambar wajah mereka beredar di internet, hal ini mempermudah pengumpulan <em>training set</em> untuk melatih model. Hal ini terbukti dengan sudah adanya beredar berbagai video-video tidak senonoh disunting hingga menyerupai wajah selebriti. Tak hanya <em>faceswapping</em>, dengan menggunakan teknologi seperti Adobe VoCo sekarang kita bisa membuat siapapun mengatakan apapun.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FI3l4XLZ59iw%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DI3l4XLZ59iw&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FI3l4XLZ59iw%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/77c6e8fa31e04ccb454df6ff92a33b7e/href">https://medium.com/media/77c6e8fa31e04ccb454df6ff92a33b7e/href</a></iframe><p>Dengan menggabungkan kedua teknologi tersebut, orang dapat dengan mudah membuat video siapa saja mengatakan apa saja. Bahkan belakangan ini, peneliti di University of Washington berhasil membuat Obama mengatakan apapun. Kecerdasan buatan mereka berhasil secara spesifik mempelajari bagaimana cara mulut obama bergerak ketika berbicara. Hal ini membuka potensi untuk banyak hal.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F9Yq67CjDqvw%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D9Yq67CjDqvw&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F9Yq67CjDqvw%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/a3243d22c61fb4aeb73b7dd65afab940/href">https://medium.com/media/a3243d22c61fb4aeb73b7dd65afab940/href</a></iframe><p>Tentu saja, dengan kekuatan besar datang tanggung jawab yang besar. Semua hal ini dapat berpotensi mengakibatkan kekacauan yang besar. Tapi seiring berkembangnya teknologi, solusi pun akan bermunculan. Beberapa solusi yang mungkin adalah dengan mengembangkan suatu kecerdasan buatan juga untuk mengenali video yang telah disunting.</p><p>Akhir kata Deepfake merupakan sebuah perkembangan yang menarik untuk diikuti dan dipelajari. Adanya Deepfake akan membantu banyak orang dengan teknologi <em>faceswapping</em>-nya yang mudah dan gratis. Hal ini datang dengan konsekuensi bahwa siapapun yang berniat buruk dapat lebih mudah membuat dan menyebarkan informasi tidak benar mengenai apapun. Apa pendapat mu mengenai Deepfake?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=525e39d02519" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>