<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[HIMIT PENS - Medium]]></title>
        <description><![CDATA[Tempat sharing informasi, ilmu, atau pengalaman mengenai IT maupun kehidupan kuliah. Ditulis oleh sobat HIMIT #ForzaInformatika #WeAreTheBest - Medium]]></description>
        <link>https://medium.com/himit-pens?source=rss----d838c8bb2d71---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>HIMIT PENS - Medium</title>
            <link>https://medium.com/himit-pens?source=rss----d838c8bb2d71---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 16:23:39 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/himit-pens" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Cryptography di Web 3: Mengenal ECC dan Ancamannya]]></title>
            <link>https://medium.com/himit-pens/cryptography-di-web-3-mengenal-ecc-dan-ancamannya-b4391d780bc8?source=rss----d838c8bb2d71---4</link>
            <guid isPermaLink="false">https://medium.com/p/b4391d780bc8</guid>
            <category><![CDATA[web3]]></category>
            <category><![CDATA[web-security]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[idnbootcampcyber]]></category>
            <category><![CDATA[cryptography]]></category>
            <dc:creator><![CDATA[Indra Rahmat Maulidi]]></dc:creator>
            <pubDate>Fri, 27 Jun 2025 11:03:10 GMT</pubDate>
            <atom:updated>2025-06-27T11:03:10.464Z</atom:updated>
            <content:encoded><![CDATA[<p>Tahukah kamu, kalau kata “crypto” dalam Web3, Bitcoin, dll. itu sebenarnya berarti Cryptography? Bukan cryptocurrency (walau sering diartikan begitu <em>sih</em>). Tapi apa sih Cryptography ini? Dan Cryptography macam apa yang dipakai di dunia Web3 ini? Kita akan membahas semuanya di sini.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4sllVfWFtfBdT5fUeRhXig.png" /><figcaption>Cryptography in Web3</figcaption></figure><h3>Apa itu Cryptography?</h3><p>Cryptography berarti praktik untuk mengamankan sebuah data yang ditransfer menggunakan kode dan algoritma tertentu sehingga data yang diamankan tidak mudah dibuka tanpa kunci yang sah. Tujuannya tentu saja agar data sensitif yang kita kirimkan tetap rahasia dan aman.</p><p><strong>Core dari cryptography bisa dikerucutkan ke 4 hal ini :</strong></p><ol><li><strong>Confidentiality:</strong> Informasi yang dienkripsi hanya dapat diakses oleh orang yang dituju dan tidak ada orang lain.</li><li><strong>Integrity:</strong> Informasi yang dienkripsi tidak dapat diubah selama penyimpanan atau selama pengiriman antara pengirim dan penerima yang dituju tanpa terdeteksi adanya perubahan.</li><li><strong>Non-repudiation:</strong> Pembuat atau pengirim informasi terenkripsi tidak dapat menyangkal niat mereka untuk mengirim informasi tersebut.</li><li><strong>Authentication:</strong> Identitas pengirim dan penerima, serta asal dan tujuan informasi dikonfirmasi.</li></ol><p><strong>Cryptography </strong>sendiri adalah sebuah topik yang besar yang mencakup <em>Hash </em>dan <em>Encryption</em>. Bisa dilihat ilustrasinya di bawah ini:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T9g37aIR9O0luSf-31rmuw.png" /><figcaption>Cryptography</figcaption></figure><p>Kita akan fokus ke <strong>enkripsi</strong>. Di sini ada dua macam enkripsi yaitu simetris dan asimetris :</p><ul><li><strong>Simetris</strong> : Sebuah model yang hanya menggunakan 1 kunci untuk mengunci dan membukanya.</li><li><strong>Asimetris</strong> : Model yang menggunakan 2 kunci yang berbeda yaitu private dan public, yang salah satunya digunakan untuk mengunci dan yang lainnya digunakan untuk membuka.</li></ul><p>Tentunya keduanya memiliki kelebihan dan kekurangannya masing masing.</p><p>Nah, Web3 bergantung pada kriptografi untuk mengamankan transaksi, identitas, dan data tanpa pihak ketiga. Tentunya mereka menggunakan enkripsi yang sangat kompleks agar tidak mudah untuk bobol bahkan dengan menggunakan teknologi tercangih sekalipun.</p><h3>Cryptography dalam Web3</h3><p>Web3 adalah iterasi baru World Wide Web yang dibangun di atas teknologi blockchain dan terdesentralisasi (tidak tergantung pada 1 tempat, kebalikan dari sentralisasi). Dalam sistem yang terdesentralisasi, kegagalan di satu titik tidak membuat yang lain terganggu, yang membuatnya lebih sulit bagi penyerang untuk mengeksploitasinya. Misalnya, jika satu <em>node</em> dalam jaringan blockchain disusupi, sisa jaringan dapat terus beroperasi.</p><p>Dalam Web3, enkripsi sering digunakan untuk melindungi data pengguna, seperti kunci pribadi dan data transaksi. Terakhir, mekanisme konsensus digunakan untuk memastikan bahwa semua peserta dalam jaringan blockchain setuju dengan status jaringan. Hal ini penting untuk keamanan, karena mencegah penyerang mengubah riwayat blockchain.</p><p><strong>Beberapa contoh encryption di Web3 :</strong></p><ul><li><em>Hash functions</em> untuk validasi blok/data.</li><li><em>Digital signatures</em> untuk autentikasi pemilik aset.</li><li><em>Key exchange</em> untuk komunikasi aman antar node.</li></ul><p>Cryptocurrency seperti Bitcoin dan Etherium juga menggunakan enkripsi kompleks yang membutuhkan kekuatan komputasi super tinggi untuk mendekripsi nya. Melalui proses dekripsi ini, koin baru “<em>minted</em>” dan memasuki peredaran.</p><h3>Elliptic Curve Cryptography (ECC)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/252/1*etjwsW7PtghkXZR8k0GPQw.png" /><figcaption>Elliptic Curve Cryptography (Sumber : <a href="http://ssl.com/">ssl.com</a>)</figcaption></figure><p>Elliptic Curve Cryptography (ECC) merupakan pendekatan yang kuat dan efisien terhadap kriptografi public key dengan menggunakan Kurva Eliptik, yang menawarkan keamanan yang tangguh dengan ukuran kunci yang lebih kecil dibandingkan dengan metode tradisional.</p><p>Kurva elips adalah kurva matematika yang didefinisikan oleh persamaan (y² = x³ + ax + b). Contoh kurva Bitcoin/Ethereum (y² = x³ + 7)</p><p><strong>Gambaran ECC bekerja</strong> :</p><ol><li>Pilih kurva elips, kunci private (bilangan bulat acak), dan titik pada kurva sebagai titik generator (G).</li><li>Kalikan titik generator dengan kunci pribadi untuk memperoleh kunci publik.</li><li>Gunakan kunci publik untuk enkripsi dan membuat shared key/signature key.</li><li>Verify dengan menggunakan kunci publik dan shared key, kalau berhasil berarti transaksi sukses.</li></ol><h3>Ancaman dalam Cryptography dan Web3</h3><h3>Side Channel Attack</h3><p>Secara umum, <strong><em>side‑channel attack</em></strong> adalah eksploitasi bocoran tidak langsung dari sebuah sistem saat ia menangani data rahasia — misalnya waktu eksekusi, konsumsi daya, emisi elektromagnetik, atau bahkan suara yang bisa dianalisis untuk merekonstruksi kunci privat atau data sensitif.</p><p><strong>Serangan <em>side-channel</em></strong> tidak menargetkan program atau kodenya secara langsung. Sebaliknya, serangan side-channel berupaya mengumpulkan informasi atau memengaruhi eksekusi program suatu sistem dengan mengukur atau memanfaatkan efek tidak langsung dari sistem atau perangkat kerasnya.</p><h4>Contoh bocoran:</h4><ul><li><strong>Power consumption</strong>: Chip memakai daya berbeda saat menghitung bit 1 dan 0.</li><li><strong>Execution time</strong>: Operasi kondisional membutuhkan waktu berbeda.</li><li><strong>Electromagnetic emissions</strong>: Perubahan EM bisa diukur dengan alat tertentu.</li></ul><h4>Dampak :</h4><ul><li><strong>Ekstraksi private key</strong> : Jika sukses, penyerang bisa menandatangani transaksi atau mencuri aset digital.</li></ul><h4>Mencegah serangan side-channel</h4><ol><li><strong>Menggunakan standar seperti X25519 atau micro‑ecc.</strong> Digunakan untuk algoritma <em>constant-time</em> dan <em>blinding</em> agar lebih susah dilihat polanya.</li><li>Tambahkan <strong>noise</strong>, <strong>pengacakan konsumsi daya</strong>, dan <strong>pelindung elektromagnetik</strong> pada hardware.</li></ol><h3>Quantum Computing</h3><p><strong><em>Quantum Computing</em></strong> adalah jenis komputasi yang menggunakan fenomena mekanika kuantum seperti superposisi, interferensi, dan keterikatan. <strong><em>Quantum Computer</em></strong> adalah perangkat yang melakukan komputasi kuantum. Sebuah komputer yang dalam hitungan detik, dapat mengatasi masalah rumit yang jauh melampaui kemampuan superkomputer tercanggih sekalipun. Ancaman komputasi kuantum pertama kali diungkapkan oleh <strong>Peter Shor</strong> pada tahun 1994. Shor mengembangkan algoritma yang menunjukkan bahwa komputer kuantum dapat memecahkan algoritma enkripsi tradisional seperti RSA dan AES dalam waktu yang sangat singkat. Jika komputer kuantum berfungsi penuh, algoritma kriptografi tradisional yang membutuhkan waktu ribuan tahun untuk dipecahkan oleh superkomputer saat ini, bisa diselesaikan hanya dalam hitungan menit.</p><h4>Dampak :</h4><ul><li><strong>Ekstraksi private key</strong> : Jika kuantum besar tersedia, semua private key ECC dapat digeneralisasi dan aset Web3 dicuri dengan signing palsu dan swap token tanpa izin.</li></ul><h4>Beradaptasi dengan Quantum Computing</h4><ol><li>Menggunakan <strong><em>Post-Quantum Cryptography</em></strong> untuk menggantikan <em>public-key cryptosystems</em> saat ini dan memastikan CIA tetap terjaga di era Quantum Computing.</li><li>Menggunakan <strong>Hybrid Signature</strong>. Dengan mengkombinasikan ECC dan <em>Post Quantum Cryptography</em>, untuk menahan serangan dari komputer saat ini dan <em>quantum computing.</em></li></ol><h3>Penutup</h3><p>Kita belajar banyak mulai dari cyptography sebagai praktik untuk mengamankan data, Web3 yang menggunakan teknologi desentralisasi, dan ECC yang digunakan pada Cryptocurrency seperti Bitcoin dan Etherium. Kita juga belajar bahwa sebenarnya keamanan Web3 juga memiliki ancaman yang menanti seperti side-channel attack dan quantum computing.</p><h3>Referensi</h3><blockquote><a href="https://www.ibm.com/think/topics/cryptography"><em>https://www.ibm.com/think/topics/cryptography</em></a></blockquote><blockquote><a href="https://www.nadcab.com/blog/ecc-in-web3-networks"><em>https://www.nadcab.com/blog/ecc-in-web3-networks</em></a></blockquote><blockquote><a href="https://bluegoatcyber.com/blog/exploring-ecc-vulnerabilities/"><em>https://bluegoatcyber.com/blog/exploring-ecc-vulnerabilities/</em></a></blockquote><blockquote><a href="https://www.ssl.com/article/what-is-elliptic-curve-cryptography-ecc/"><em>https://www.ssl.com/article/what-is-elliptic-curve-cryptography-ecc/</em></a></blockquote><blockquote><a href="https://www.rambus.com/blogs/Side-channel-attacks/"><em>https://www.rambus.com/blogs/Side-channel-attacks/</em></a></blockquote><blockquote><a href="https://www.cloudcomputing.id/pengetahuan-dasar/mengenal-quantum-computing"><em>https://www.cloudcomputing.id/pengetahuan-dasar/mengenal-quantum-computing</em></a></blockquote><blockquote><a href="https://cyberhub.id/pengetahuan-dasar/quantum-cryptography"><em>https://cyberhub.id/pengetahuan-dasar/quantum-cryptography</em></a></blockquote><blockquote><a href="https://cryptojedi.org/papers/sca25519-20221104.pdf"><em>https://cryptojedi.org/papers/sca25519-20221104.pdf</em></a></blockquote><blockquote><a href="https://pixelplex.io/blog/post-quantum-cryptography"><em>https://pixelplex.io/blog/post-quantum-cryptography</em></a></blockquote><p>#IDNBootcampCyber</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b4391d780bc8" width="1" height="1" alt=""><hr><p><a href="https://medium.com/himit-pens/cryptography-di-web-3-mengenal-ecc-dan-ancamannya-b4391d780bc8">Cryptography di Web 3: Mengenal ECC dan Ancamannya</a> was originally published in <a href="https://medium.com/himit-pens">HIMIT PENS</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Big Data and Its Evolution in Business Data Processing]]></title>
            <link>https://medium.com/himit-pens/understanding-big-data-and-its-evolution-in-business-data-processing-8e306e113ede?source=rss----d838c8bb2d71---4</link>
            <guid isPermaLink="false">https://medium.com/p/8e306e113ede</guid>
            <category><![CDATA[data-processing]]></category>
            <category><![CDATA[3vs-of-data]]></category>
            <category><![CDATA[big-data]]></category>
            <dc:creator><![CDATA[Toriqul Muchlissin]]></dc:creator>
            <pubDate>Fri, 27 Jun 2025 03:41:13 GMT</pubDate>
            <atom:updated>2025-06-27T03:41:13.661Z</atom:updated>
            <content:encoded><![CDATA[<p>In today’s digital era, businesses produce data at an extraordinary pace. As formats shift from traditional rows and columns to JSON, XML, images, and video, modern data-processing platforms have become essential. This article examines the evolution of business data processing, the challenges posed by Big Data, the architectural solutions that emerged, and how Hadoop established the foundation for today’s Big Data ecosystems.</p><h3>Introduction to Big Data</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*mpgIiJC16oKmUIo1.png" /></figure><p>In the ever-evolving world of technology, <strong>Big Data</strong> has emerged as a transformative force, particularly in the realm of business data processing. Big Data refers to the massive volumes of data that businesses generate and collect, characterized by the <strong>3Vs: Variety, Volume, and Velocity</strong>. These attributes define the challenges and opportunities of managing modern data, which traditional systems struggle to address.</p><ul><li><strong>Variety:</strong> Data comes in different forms, such as structured (e.g., tables in databases), semi-structured (e.g., JSON or XML with key-value pairs), and unstructured (e.g., text, PDFs, images, or videos). This variety reflects the complexity of data created by businesses through websites, social media, and mobile apps.</li><li><strong>Volume:</strong> Businesses now collect massive amounts of data, from terabytes to petabytes, in a short time, driven by digital platforms and increased connectivity.</li><li><strong>Velocity:</strong> Data is generated quickly and often needs to be processed instantly or nearly instantly to meet business needs.</li></ul><p>These 3Vs collectively form the <strong>Big Data problem</strong>, which traditional <strong>Relational Database Management Systems (RDBMS)</strong>, such as Oracle and Microsoft SQL Server, were not designed to handle effectively, particularly for unstructured data and high-speed, large-scale processing.</p><h3>The Roots of Business Data Processing</h3><p>Business data processing has been a critical component of technology since its early days. The journey began with <strong>COBOL (Common Business-Oriented Language)</strong>, introduced in 1959, which was specifically designed for business applications. COBOL enabled efficient data storage in files, index creation, and processing, laying the groundwork for structured data management.</p><p>The next milestone was the rise of RDBMS systems, starting with Oracle in 1977. These systems offered:</p><ul><li><strong>SQL:</strong> A user-friendly query language for data access and manipulation.</li><li><strong>Scripting languages:</strong> Such as PL/SQL and Transact-SQL for complex data processing tasks.</li><li><strong>Interfaces:</strong> Like JDBC and ODBC, allowing integration with other programming languages.</li></ul><p>RDBMS systems excelled at handling structured data, making them the backbone of business data processing for decades. However, as data evolved to include semi-structured and unstructured formats, and as volume and velocity increased, RDBMS systems faced limitations in addressing the Big Data problem.</p><h3>The Big Data Challenge in Business</h3><p>The advent of the internet, social media, and mobile applications transformed the data landscape. Businesses began collecting diverse data types:</p><ul><li><strong>Structured Data:</strong> Organized in rows and columns (e.g., CSV files or RDBMS tables).</li><li><strong>Semi-Structured Data:</strong> Key/value-based formats like JSON and XML, which lack a rigid tabular structure but maintain some structure.</li><li><strong>Unstructured Data:</strong> Files without a defined structure, such as text documents, PDFs, images, and videos.</li></ul><p>This shift introduced significant challenges:</p><ul><li><strong>Handling Variety:</strong> RDBMS systems were designed for structured data and struggled to process semi-structured and unstructured data without complex preprocessing.</li><li><strong>Managing Volume:</strong> Businesses now deal with petabytes of data, far exceeding the terabyte-scale capabilities of RDBMS.</li><li><strong>Achieving High Velocity:</strong> The rapid generation of data requires processing in hours or minutes, not days, which RDBMS systems were not optimized for.</li></ul><p>These challenges, encapsulated by the 3Vs, necessitated a new approach to business data processing that could accommodate diverse data types, large volumes, and high-speed requirements.</p><h3>Monolithic vs. Distributed Approaches to Big Data</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*sOyl1OPMO5bdzCCU.png" /></figure><p>To address the Big Data problem, two primary approaches emerged: <strong>monolithic</strong> and <strong>distributed</strong>.</p><ul><li><strong>Monolithic Approach:</strong> A monolithic approach uses a single, powerful system with robust CPU, memory, and storage to handle demanding workloads. Systems like Teradata and Exadata excel in managing structured data with high transactional performance and low latency. These monolithic systems rely on <strong>vertical scalability</strong>, adding resources (CPU, RAM, storage) to the same machine, a process that is complex, time-consuming, and often requires vendor coordination for upgrades, causing downtime and limiting adaptability to changing business needs.</li><li><strong>Distributed Approach:</strong> A group of smaller machines working together as a single system, combining their CPU, memory, and storage. This setup supports <strong>horizontal scalability</strong>, allowing easy expansion by adding more machines as data or user demand increases. It offers greater simplicity, speed, and cost efficiency. Businesses can begin with a small cluster and scale incrementally using affordable hardware or cloud resources. Distributed systems also provide better fault tolerance and availability, as a failure in one machine doesn’t halt the entire system.</li></ul><p>When evaluated on <strong>scalability</strong>, <strong>fault tolerance</strong>, and <strong>cost-effectiveness</strong>, distributed systems outperform monolithic systems:</p><ul><li><strong>Scalability:</strong> Both approaches are scalable, but horizontal scalability in distributed systems is more flexible and faster, as adding new computers to the cluster requires minimal effort compared to the complex process of upgrading a monolithic system.</li><li><strong>Fault Tolerance:</strong> Distributed systems can tolerate multiple hardware failures without halting operations, ensuring greater availability for business applications. Monolithic systems, however, are vulnerable to single points of failure, which can lead to downtime.</li><li><strong>Cost-Effectiveness:</strong> Distributed systems allow businesses to start with a small cluster and scale incrementally, using cost-effective hardware or cloud rentals. Monolithic systems, on the other hand, are expensive, often requiring large initial investments to anticipate medium-term growth, even if immediate needs are smaller.</li></ul><p>These evaluations indicate that distributed systems are a better choice for addressing the Big Data problem, offering greater flexibility, reliability, and economic efficiency.</p><h3>Hadoop: A Solution for Big Data</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*MnBWtkSUZbxRIM6_.png" /></figure><p>The limitations of RDBMS and the advantages of distributed systems led to the development of <strong>Hadoop</strong>, a revolutionary platform designed to tackle the Big Data problem. Hadoop is an open-source framework that enables distributed storage and processing of massive datasets across clusters of commodity hardware, using simple programming models.</p><p>Hadoop addresses the 3Vs by providing:</p><ol><li><strong>Distributed Cluster Formation:</strong> Hadoop, utilizing <strong>YARN (Yet Another Resource Negotiator)</strong>, acts as a cluster operating system, enabling a group of computers to function as a single, cohesive system, simplifying resource management for businesses.</li><li><strong>Distributed Storage:</strong> The <strong>Hadoop Distributed File System (HDFS)</strong> enables data to be saved and accessed across a network of nodes, accommodating structured, semi-structured, and unstructured datasets at petabyte scale.</li><li><strong>Distributed Data Processing:</strong> With the <strong>MapReduce</strong> framework, Hadoop enables developers to build data-processing applications that execute concurrently across all nodes in the cluster, ensuring efficient performance for workloads with high throughput. Hadoop tackles high-velocity data through efficient parallel processing with MapReduce, while tools like HBase provide fast data access for time-sensitive needs, though advanced streaming may require additional tools.</li></ol><p>Hadoop’s strength lies in its ability to abstract the complexities of distributed computing. Developers can write applications as if they were running on a single machine, while Hadoop manages parallel processing and data distribution. This simplicity, combined with its capacity to handle diverse data types and large volumes, made Hadoop a preferred choice for business data processing.</p><p>Hadoop also offers additional tools to enhance its capabilities:</p><ul><li><strong>Hive:</strong> A SQL-like interface for querying data, enabling businesses to leverage existing SQL expertise.</li><li><strong>HBase:</strong> A distributed NoSQL database for fast, random read/write access, ideal for time-sensitive applications requiring low-latency data retrieval.</li><li><strong>Pig:</strong> A scripting language for streamlined data processing workflows.</li><li><strong>Sqoop:</strong> A tool for efficient data ingestion from various sources.</li><li><strong>Oozie:</strong> A workflow scheduler for managing complex Hadoop jobs.</li></ul><h3>Hadoop vs. RDBMS for Big Data</h3><p>Hadoop offers distinct advantages over traditional RDBMS for Big Data processing, as summarized in the table below. While RDBMS is ideal for small to medium-sized structured datasets, Hadoop excels in managing the volume, variety, and velocity of Big Data, making it a critical tool for modern data-driven applications.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8ab9b0b972b02f9aeb060a310e30156f/href">https://medium.com/media/8ab9b0b972b02f9aeb060a310e30156f/href</a></iframe><p>Hadoop surpasses RDBMS in scalability, supporting petabyte-scale storage compared to the terabyte limits of RDBMS. It handles diverse data types, including structured, semi-structured, and unstructured data, while RDBMS is optimized for structured data. Hadoop’s Hive provides SQL-like querying, ensuring compatibility with existing business tools, and its scripting capabilities through Apache Pig parallel those of PL/SQL or T-SQL in RDBMS. Additionally, Hadoop’s connectivity through JDBC/ODBC via Hive and native APIs offers flexible integration with various applications, similar to RDBMS’s JDBC/ODBC support, but enhanced by its distributed ecosystem.</p><h3>Conclusion</h3><p>Understanding Big Data is critical for navigating the complexities of modern business data processing. The 3Vs (variety, volume, and velocity) define the challenges posed by massive, diverse, and rapidly generated data that businesses handle today. Traditional RDBMS systems, while robust for structured data, fall short in addressing these challenges, leading to the rise of distributed platforms such as Hadoop.</p><p>Comparing monolithic and distributed architectures shows that distributed systems with horizontal scalability, fault tolerance, and cost-effectiveness are better suited to Big Data needs. Hadoop, in particular, simplifies distributed computing, accommodates diverse data types, and scales efficiently, transforming how businesses process and analyze petabytes of information.</p><p>Hadoop’s ability to process high-velocity data through efficient parallel processing with MapReduce and tools like HBase ensures businesses can manage rapid data generation effectively, providing a robust solution for Big Data challenges. By leveraging Hadoop’s capabilities, businesses can gain actionable insights from high volume, high velocity data streams and maintain a competitive advantage through data-driven decision-making.</p><blockquote><strong><em>🔄 Evolution of 3Vs</em></strong></blockquote><blockquote><em>“The original </em><strong><em>3Vs</em></strong><em> </em><strong><em>(Variety</em></strong><em>, </em><strong><em>Volume</em></strong><em>, and </em><strong><em>Velocity)</em></strong><em> form the foundational model that characterizes </em><strong><em>Big Data</em></strong><em>. This concept later evolved into the </em><strong><em>5Vs</em></strong><em>, adding </em><strong><em>Veracity</em></strong><em> (ensuring data reliability for accurate business decisions) and </em><strong><em>Value</em></strong><em> (deriving useful insights to drive strategic actions). It further expanded into the </em><strong><em>7Vs</em></strong><em>, incorporating </em><strong><em>Variability</em></strong><em> (adapting to changing data patterns for dynamic business needs) and </em><strong><em>Visualization</em></strong><em> (presenting data clearly to enhance decision-making).”</em></blockquote><p>Thank you for reading. Keep learning and keep growing! 😊🚀</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8e306e113ede" width="1" height="1" alt=""><hr><p><a href="https://medium.com/himit-pens/understanding-big-data-and-its-evolution-in-business-data-processing-8e306e113ede">Understanding Big Data and Its Evolution in Business Data Processing</a> was originally published in <a href="https://medium.com/himit-pens">HIMIT PENS</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building AI Agent Workflows with Python & TypeScript]]></title>
            <link>https://medium.com/himit-pens/building-ai-agent-workflows-with-python-typescript-d798c3435ec1?source=rss----d838c8bb2d71---4</link>
            <guid isPermaLink="false">https://medium.com/p/d798c3435ec1</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[langgraph]]></category>
            <category><![CDATA[agents]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Wahyu ikbal maulana]]></dc:creator>
            <pubDate>Sun, 09 Feb 2025 01:01:43 GMT</pubDate>
            <atom:updated>2025-06-14T13:24:15.287Z</atom:updated>
            <content:encoded><![CDATA[<p>Recently, <a href="https://www.ibm.com/think/topics/ai-agents">AI agents </a>have been on the rise among developers and engineers. For those of you who don’t know, agent AI is an implementation of the LLM process, where LLM is equipped with tools, memory, and the ability to make decisions. Think of it as an assistant that can perform tasks, not just give output in the form of text.</p><p>The concept of AI agents is very powerful, especially when multiple agents work together in a system. The process of inter-agent interaction and coordination is called workflow. In this article, we will not focus on the theory of AI agents in depth, but more on the engineering and practical side. So make sure you have a basic understanding of <a href="https://www.ibm.com/think/topics/retrieval-augmented-generation">RAG </a>(Retrieval-Augmented Generation) and <a href="https://www.ibm.com/think/topics/large-language-models">LLM</a> before proceeding.</p><h3>Building an Effective Agent Workflow</h3><p>In this section, we will learn common patterns that are often used in <a href="https://www.forbes.com/sites/bernardmarr/2025/02/03/generative-ai-vs-agentic-ai-the-key-differences-everyone-needs-to-know/">agentic</a> systems implemented in various applications. This time we will practice them using <a href="https://sdk.vercel.ai/">Vercel AI SDK</a> and <a href="https://www.langchain.com/langgraph">Langgraph</a>. In addition, I will explain a little bit about LangGraph and AI SDK. If you are already familiar with these frameworks, you can skip to the code implementation section.</p><h3>Why LangGraph?</h3><p>LangGraph is a library built on top of <a href="https://www.langchain.com/">LangChain</a>, designed for LLM-based applications, making LangGraph a solid choice for complex agent systems. LangGraph makes building graph-based workflows super easy and efficient. It simplifies handling complex tasks like orchestrating LLM calls and managing structured outputs. It also integrates seamlessly with Pydantic for data validation and type enforcement, ensuring that the inputs and outputs of each workflow step are well-structured and reliable.</p><p>At the heart of LangGraph is the concept of a stateful graph:</p><ul><li><strong>State</strong>: Stores and updates context throughout the workflow, enabling dynamic decision-making.</li><li><strong>Nodes</strong>: Represent computation steps, handling tasks like processing inputs, decision-making, or external interactions.</li><li><strong>Edges</strong>: Define the flow between nodes, supporting conditional logic for flexible, multi-step workflows.</li></ul><h3>Why Vercel AI SDK</h3><p>Vercel AI SDK is a TypeScript library designed for building AI-powered applications, especially within React-based frameworks like Next.js, Svelte, and Vue. It streamlines AI development with a standardized approach to prompts, structured data, and tool integration.</p><p>AI SDK Core has various functions designed for text generation, structured data generation, and tool usage. These functions take a standardized approach to setting up prompts and settings, making it easier to work with different models.</p><p>- <strong>streamText</strong>: Streams text and tool calls, perfect for chatbots and content streaming.<br> <br>- <strong>streamObject</strong>: Streams structured objects matching a Zod schema, making it easy to generate UI components from JSON-like data.</p><p>Vercel AI SDK also includes <strong>AI SDK UI</strong>, which simplifies building LLM-based interfaces. That’s why it’s my go-to library for developing AI applications on Next.js! 🔥</p><h3>Workflow Patterns</h3><blockquote>Workflow is a generic term for orchestrated and repeatable patterns of activity, enabled by the systematic organization of resources into processes that transform materials, provide services, or process information.</blockquote><p>The concept of workflow is often encountered in everyday life. For example, in college where we follow a structured organization, or in a work environment where there is a system.</p><p>If you want a more detailed explanation, you can read the research of Anthropic <a href="https://www.anthropic.com/research/building-effective-agents">here</a>. Anthropic’s research is a required reference for many AI developers, and can be additional learning material for you as well. Among them are:</p><ul><li>Prompt Chaining (Sequential Processing)</li><li>Routing</li><li>Parallel Processing</li><li>Orchestration</li><li>Evaluation/Feedback Loops</li></ul><p>For the code that you want to learn, you can check my <a href="https://github.com/wahyudesu/">github</a> or check the source code repo at the end of this article.</p><h3>Set up workflow</h3><p>Before we start, we need to set up the framework we will be using. You can choose to use either Typescript or Python. The API we use here is <a href="https://groq.com/">groq </a>provider. <em>Don’t forget to install the library first.</em></p><p>Setting up Typescript with Vercel AI SDK</p><pre>require(&#39;dotenv&#39;).config();<br><br>import { generateObject } from &#39;ai&#39;; // generate json output<br>import { z } from &#39;zod&#39;; // schema declaration <br>import { createGroq } from &#39;@ai-sdk/groq&#39;; // provider<br><br>const groq = createGroq({<br>  baseURL: &#39;https://api.groq.com/openai/v1&#39;,<br>  apiKey: process.env.GROQ_API_KEY || &quot;&quot;,<br>});</pre><p>Python setup with langgraph</p><pre>import os<br>from langchain_groq import ChatGroq<br><br># Load environment variables<br>load_dotenv()<br><br># Initialize Groq client<br>groq = ChatGroq(<br>    api_key=os.getenv(&quot;GROQ_API_KEY&quot;),<br>    base_url=&quot;https://api.groq.com/openai/v1&quot;<br>)</pre><h3>Prompt Chaining</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*TxKVSAtJTlNaPvnH" /><figcaption>Image by author</figcaption></figure><p>This is the simplest workflow. Prompt chaining breaks down a single task into multiple steps. It involves multiple outputs from one prompt as inputs for the next. This pattern is ideal for tasks where the order of each step is clear.</p><p>For example, when creating copywriting for a brand, it starts with agent A crafting the CTA. Then, agent B assesses the emotional aspect. Next, agent C assesses the fit with the brand tone. Each step is sequential, and the output from one step becomes the input for the next.</p><pre>async function generateMarketingCopy(input: string) {<br>  const model = groq(&#39;llama-3.3-70b-versatile&#39;);<br><br>  console.log(&quot;Generating initial marketing copy...&quot;);<br>  const { text: copy } = await generateText({<br>    model,<br>    prompt: `Write persuasive marketing copy for: ${input}. Focus on benefits and emotional appeal.`,<br>  });<br><br>  console.log(&quot;Evaluating marketing copy quality...&quot;);<br>  const { object: qualityMetrics } = await generateObject({<br>    model,<br>    schema: z.object({<br>      hasCallToAction: z.boolean(),<br>      emotionalAppeal: z.number().min(1).max(10),<br>      clarity: z.number().min(1).max(10),<br>    }),<br>    prompt: `Evaluate this marketing copy for:<br>    1. Presence of call to action (true/false)<br>    2. Emotional appeal (1-10)<br>    3. Clarity (1-10)<br>    <br>    Copy to evaluate: ${copy}`,<br>  });</pre><p>Once the agent specializing in marketing is declared, the next step is to verify that the quality is relevant and to reduce hallucinations, in which case you can create additional, more complex agents that not only improve but also increase the output of the previous agent.</p><pre>// If quality check fails, regenerate with more specific instructions<br><br>if (<br>    !qualityMetrics.hasCallToAction ||<br>    qualityMetrics.emotionalAppeal &lt; 7 ||<br>    qualityMetrics.clarity &lt; 7<br>  ) {<br>    console.log(&quot;Improving marketing copy based on quality evaluation...&quot;);<br>    const { text: improvedCopy } = await generateText({<br>      model,<br>      prompt: `Rewrite this marketing copy with:<br>      ${!qualityMetrics.hasCallToAction ? &#39;- A clear call to action&#39; : &#39;&#39;}<br>      ${qualityMetrics.emotionalAppeal &lt; 7 ? &#39;- Stronger emotional appeal&#39; : &#39;&#39;}<br>      ${qualityMetrics.clarity &lt; 7 ? &#39;- Improved clarity and directness&#39; : &#39;&#39;}<br>      <br>      Original copy: ${copy}`,<br>    });<br><br>    console.log(&quot;\nFinal Improved Marketing Copy:\n&quot;, improvedCopy);<br>    console.log(&quot;\nQuality Metrics:&quot;, qualityMetrics);<br>    return;<br>  }</pre><h4>Workflow with LangGraph python</h4><pre># Define our state schema for tracking the workflow progress<br>class MarketingState(TypedDict):<br>    input: str<br>    initial_copy: str<br>    quality_metrics: Optional[dict]<br>    final_copy: str<br>    messages: list[BaseMessage]<br><br># Define quality metrics schema<br>class QualityMetrics(BaseModel):<br>    hasCallToAction: bool<br>    emotionalAppeal: int<br>    clarity: int<br><br>def generate_initial_copy(state: MarketingState) -&gt; MarketingState:<br>    &quot;&quot;&quot;<br>    Generates the initial marketing copy based on the input.<br>    &quot;&quot;&quot;<br>    response = groq.chat.completions.create(<br>        model=&quot;llama-3.3-70b-versatile&quot;,<br>        messages=[{<br>            &quot;role&quot;: &quot;user&quot;,<br>            &quot;content&quot;: f&quot;Write persuasive marketing copy for: {state[&#39;input&#39;]}. Focus on benefits and emotional appeal.&quot;<br>        }]<br>    )<br>    <br>    state[&quot;initial_copy&quot;] = response.choices[0].message.content<br>    state[&quot;final_copy&quot;] = state[&quot;initial_copy&quot;]  # Initialize final copy<br>    return state<br><br>def evaluate_copy(state: MarketingState) -&gt; MarketingState:<br>    &quot;&quot;&quot;<br>    Evaluates the marketing copy against quality metrics.<br>    &quot;&quot;&quot;<br>    response = groq.chat.completions.create(<br>        model=&quot;llama-3.3-70b-versatile&quot;,<br>        messages=[{<br>            &quot;role&quot;: &quot;user&quot;,<br>            &quot;content&quot;: f&quot;&quot;&quot;Evaluate this marketing copy for:<br>            1. Presence of call to action (true/false)<br>            2. Emotional appeal (1-10)<br>            3. Clarity (1-10)<br>            <br>            Copy to evaluate: {state[&#39;final_copy&#39;]}&quot;&quot;&quot;<br>        }]<br>    )<br>    <br>    # Parse the metrics from the response<br>    metrics = QualityMetrics(<br>        hasCallToAction=True,  # You would parse these values from the actual response<br>        emotionalAppeal=8,<br>        clarity=8<br>    )<br>    <br>    state[&quot;quality_metrics&quot;] = metrics.dict()<br>    return state<br><br>def improve_copy(state: MarketingState) -&gt; MarketingState:<br>    &quot;&quot;&quot;<br>    Improves the marketing copy if it doesn&#39;t meet quality standards.<br>    &quot;&quot;&quot;<br>    metrics = state[&quot;quality_metrics&quot;]<br>    <br>    # Check if improvements are needed<br>    if (not metrics[&quot;hasCallToAction&quot;] or <br>        metrics[&quot;emotionalAppeal&quot;] &lt; 7 or <br>        metrics[&quot;clarity&quot;] &lt; 7):<br>        <br>        improvement_prompt = f&quot;&quot;&quot;Rewrite this marketing copy with:<br>        {&#39;&#39; if metrics[&quot;hasCallToAction&quot;] else &#39;- A clear call to action&#39;}<br>        {&#39;&#39; if metrics[&quot;emotionalAppeal&quot;] &gt;= 7 else &#39;- Stronger emotional appeal&#39;}<br>        {&#39;&#39; if metrics[&quot;clarity&quot;] &gt;= 7 else &#39;- Improved clarity and directness&#39;}<br>        <br>        Original copy: {state[&#39;final_copy&#39;]}&quot;&quot;&quot;<br>        <br>        response = groq.chat.completions.create(<br>            model=&quot;llama-3.3-70b-versatile&quot;,<br>            messages=[{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: improvement_prompt}]<br>        )<br>        <br>        state[&quot;final_copy&quot;] = response.choices[0].message.content<br>    <br>    return state<br><br>def create_marketing_workflow() -&gt; Graph:<br>    &quot;&quot;&quot;<br>    Creates the marketing copy generation and improvement workflow.<br>    &quot;&quot;&quot;<br>    workflow = StateGraph(MarketingState)<br>    <br>    # Add nodes<br>    workflow.add_node(&quot;generate&quot;, generate_initial_copy)<br>    workflow.add_node(&quot;evaluate&quot;, evaluate_copy)<br>    workflow.add_node(&quot;improve&quot;, improve_copy)<br>    <br>    # Add edges<br>    workflow.add_edge(&quot;generate&quot;, &quot;evaluate&quot;)<br>    workflow.add_edge(&quot;evaluate&quot;, &quot;improve&quot;)<br>    <br>    # Set entry and exit points<br>    workflow.set_entry_point(&quot;generate&quot;)<br>    workflow.set_finish_point(&quot;improve&quot;)<br>    <br>    return workflow.compile()<br><br>def generate_marketing_copy(input_text: str) -&gt; dict:<br>    &quot;&quot;&quot;<br>    Main function to handle the marketing copy generation process.<br>    &quot;&quot;&quot;<br>    workflow = create_marketing_workflow()<br>    <br>    result = workflow.invoke({<br>        &quot;input&quot;: input_text,<br>        &quot;initial_copy&quot;: &quot;&quot;,<br>        &quot;quality_metrics&quot;: None,<br>        &quot;final_copy&quot;: &quot;&quot;,<br>        &quot;messages&quot;: []<br>    })<br>    <br>    return {<br>        &quot;initial_copy&quot;: result[&quot;initial_copy&quot;],<br>        &quot;final_copy&quot;: result[&quot;final_copy&quot;],<br>        &quot;quality_metrics&quot;: result[&quot;quality_metrics&quot;]<br>    }</pre><h3>Routing</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*TiDred0lfVP4BgEl" /><figcaption>Image by author</figcaption></figure><p>This pattern allows the model to determine the correct process path based on context and previous results. The routing pattern works well when there are several different categories that require special handling. This pattern ensures that each case is handled with the most appropriate approach. As in the example above, the result of the first LLM determines which path to take for the next step, adapting the process based on the input characteristics.</p><p>We first create a routing agent that classifies the inputs by dividing them based on the type of input, as well as its complexity and detailed reasoning.</p><pre>async function handleCustomerQuery(query: string) {<br>  const model = groq(&#39;llama-3.3-70b-versatile&#39;);<br><br>  // Step 1: Classify the query type<br>  const { object: classification } = await generateObject({<br>    model,<br>    schema: z.object({<br>      reasoning: z.string(),<br>      type: z.enum([&#39;general&#39;, &#39;refund&#39;, &#39;technical&#39;]),<br>      complexity: z.enum([&#39;simple&#39;, &#39;complex&#39;]),<br>    }),<br>    prompt: `Classify this customer query:<br>    ${query}<br><br>    Determine:<br>    1. Query type (general, refund, or technical)<br>    2. Complexity (simple or complex)<br>    3. Brief reasoning for classification`,<br>  });</pre><p>The results of the classification are then directed to multiple agents. Simple, but this is really useful for customer service, which is directed to tools such as scheduling, contacting other people, and others, so that the output issued is personalized according to user needs.</p><pre>  // Route based on classification<br>  // Determine the model and system prompt based on query type and complexity<br>  const { text: response } = await generateText({<br>    model:<br>      classification.complexity === &#39;simple&#39;<br>        ? groq(&#39;llama3-8b-8192&#39;)<br>        : groq(&#39;llama-3.1-8b-instant&#39;),<br>    system: {<br>      general:<br>        &#39;You are an expert customer service agent handling general inquiries.&#39;,<br>      refund:<br>        &#39;You are a customer service agent specializing in refund requests. Follow company policies and gather necessary information.&#39;,<br>      technical:<br>        &#39;You are a technical support specialist with in-depth knowledge of the product. Focus on clear, step-by-step troubleshooting.&#39;,<br>    }[classification.type],<br>    prompt: query,<br>  });</pre><h4>Workflow with LangGraph python</h4><pre># Define our state schema<br>class AgentState(TypedDict):<br>    query: str<br>    classification: dict<br>    response: str<br>    messages: list[BaseMessage]<br><br># Define classification schema<br>class QueryClassification(BaseModel):<br>    reasoning: str<br>    type: Literal[&quot;general&quot;, &quot;refund&quot;, &quot;technical&quot;]<br>    complexity: Literal[&quot;simple&quot;, &quot;complex&quot;]<br><br># Step 1: Classify the customer query<br>def classify_query(state: AgentState) -&gt; AgentState:<br>    &quot;&quot;&quot;Classifies the incoming customer query by type and complexity.&quot;&quot;&quot;<br>    <br>    prompt = f&quot;&quot;&quot;Classify this customer query:<br>    {state[&#39;query&#39;]}<br><br>    Determine:<br>    1. Query type (general, refund, or technical)<br>    2. Complexity (simple or complex)<br>    3. Brief reasoning for classification&quot;&quot;&quot;<br><br>    response = groq.chat.completions.create(<br>        model=&quot;llama-3.3-70b-versatile&quot;,<br>        messages=[{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: prompt}]<br>    )<br>    <br>    # Parse the classification from the response<br>    classification = QueryClassification(<br>        reasoning=response.choices[0].message.content,<br>        type=&quot;technical&quot;,  # You would parse this from the response<br>        complexity=&quot;simple&quot;  # You would parse this from the response<br>    )<br>    <br>    state[&quot;classification&quot;] = classification.dict()<br>    return state<br><br># Step 2: Generate response based on classification<br>def generate_response(state: AgentState) -&gt; AgentState:<br>    &quot;&quot;&quot;Generates a response based on the query classification.&quot;&quot;&quot;<br>    <br>    # Select model based on complexity<br>    model = &quot;llama3-8b-8192&quot; if state[&quot;classification&quot;][&quot;complexity&quot;] == &quot;simple&quot; else &quot;llama-3.1-8b-instant&quot;<br>    <br>    # Select system prompt based on query type<br>    system_prompts = {<br>        &quot;general&quot;: &quot;You are an expert customer service agent handling general inquiries.&quot;,<br>        &quot;refund&quot;: &quot;You are a customer service agent specializing in refund requests. Follow company policies and gather necessary information.&quot;,<br>        &quot;technical&quot;: &quot;You are a technical support specialist with in-depth knowledge of the product. Focus on clear, step-by-step troubleshooting.&quot;<br>    }<br>    <br>    response = groq.chat.completions.create(<br>        model=model,<br>        messages=[<br>            {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: system_prompts[state[&quot;classification&quot;][&quot;type&quot;]]},<br>            {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: state[&quot;query&quot;]}<br>        ]<br>    )<br>    <br>    state[&quot;response&quot;] = response.choices[0].message.content<br>    return state<br><br># Create the workflow graph<br>def create_customer_service_workflow() -&gt; Graph:<br>    &quot;&quot;&quot;Creates the customer service workflow graph.&quot;&quot;&quot;<br>    <br>    # Initialize the graph<br>    workflow = StateGraph(AgentState)<br>    <br>    # Add nodes<br>    workflow.add_node(&quot;classify&quot;, classify_query)<br>    workflow.add_node(&quot;respond&quot;, generate_response)<br>    <br>    # Add edges<br>    workflow.add_edge(&quot;classify&quot;, &quot;respond&quot;)<br>    <br>    # Set entry and exit points<br>    workflow.set_entry_point(&quot;classify&quot;)<br>    workflow.set_finish_point(&quot;respond&quot;)<br>    <br>    return workflow.compile()<br><br># Main execution function<br>def handle_customer_query(query: str) -&gt; dict:<br>    &quot;&quot;&quot;Handles a customer query through the workflow.&quot;&quot;&quot;<br>    <br>    # Initialize workflow<br>    workflow = create_customer_service_workflow()<br>    <br>    # Execute workflow<br>    result = workflow.invoke({<br>        &quot;query&quot;: query,<br>        &quot;classification&quot;: {},<br>        &quot;response&quot;: &quot;&quot;,<br>        &quot;messages&quot;: []<br>    })<br>    <br>    return {<br>        &quot;classification&quot;: result[&quot;classification&quot;],<br>        &quot;response&quot;: result[&quot;response&quot;]<br>    }</pre><h3>Parallelization</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*3hbsKeZZh5ldXGgm" /><figcaption>Image by author</figcaption></figure><p>LLMs sometimes need to run at the same time and combine their outputs for more accurate results. This workflow is quite effective when a long task can be broken down into several parts and run at the same time to make it run faster. For complex tasks that require a lot of consideration, LLMs usually perform better than focusing on the specific <a href="https://www.datacamp.com/blog/attention-mechanism-in-llms-intuition">attention </a>of a context.</p><p>In this workflow, there are two variations:</p><ul><li>Sectioning: Breaking a task into several subtasks that run at the same time.</li><li>Voting: Running the same task multiple times to produce different outputs.</li></ul><p>Just like how an IT team focuses on application development, there are those who focus on performance, quality, and security. So for the flow of the website generation prompt, it is broken down into several aspects so that the results obtained not only work but also have stable performance and a high security side.</p><p>First, we need to create specialists who work simultaneously with more or less the same message.</p><pre>// Run parallel reviews<br>  const [securityReview, performanceReview, maintainabilityReview] =<br>    await Promise.all([<br>      generateObject({<br>        model, system:<br>          &#39;You are an expert in code security. Focus on identifying security vulnerabilities, injection risks, and authentication issues.&#39;,<br>        schema: z.object({<br>          vulnerabilities: z.array(z.string()),<br>          riskLevel: z.enum([&#39;low&#39;, &#39;medium&#39;, &#39;high&#39;]),<br>          suggestions: z.array(z.string()),<br>        }), prompt: `Review this code: ${code}`,<br>      }),<br>      generateObject({<br>        model, system:<br>          &#39;You are an expert in code performance. Focus on identifying performance bottlenecks, memory leaks, and optimization opportunities.&#39;,<br>        schema: z.object({<br>          issues: z.array(z.string()),<br>          impact: z.enum([&#39;low&#39;, &#39;medium&#39;, &#39;high&#39;]),<br>          optimizations: z.array(z.string()),<br>        }), prompt: `Review this code: ${code}`,<br>      }),<br>      generateObject({<br>        model, system:<br>          &#39;You are an expert in code quality. Focus on code structure, readability, and adherence to best practices.&#39;,<br>        schema: z.object({<br>          concerns: z.array(z.string()),<br>          qualityScore: z.number().min(1).max(10),<br>          recommendations: z.array(z.string()),<br>        }), prompt: `Review this code: ${code}`,<br>      }),<br>    ]);</pre><p>Next we need to merge all the objects together. We need an agent as an Aggregator. That needs to review and summarize, if necessary add correction tasks to minimize errors.</p><pre>  // Aggregate results using another model instance<br>  const { text: summary } = await generateText({<br>    model, system: &#39;You are a technical lead summarizing multiple code reviews.&#39;,<br>    prompt: `Synthesize these code review results into a concise summary with key actions:<br>    ${JSON.stringify(reviews, null, 2)}`,<br>  });</pre><h4>Workflow with LangGraph python</h4><pre># Define response schemas using Pydantic<br>class SecurityReview(BaseModel):<br>    vulnerabilities: List[str] = Field(default=[])<br>    risk_level: str = Field(default=&quot;low&quot;)<br>    suggestions: List[str] = Field(default=[])<br><br>class PerformanceReview(BaseModel):<br>    issues: List[str] = Field(default=[])<br>    impact: str = Field(default=&quot;low&quot;)<br>    optimizations: List[str] = Field(default=[])<br><br>class MaintainabilityReview(BaseModel):<br>    concerns: List[str] = Field(default=[])<br>    quality_score: int = Field(default=10)<br>    recommendations: List[str] = Field(default=[])<br><br># Define a function to perform a review<br>def review_code(model, system_prompt: str, code: str, response_schema):<br>    &quot;&quot;&quot;<br>    Conducts a specialized review based on the provided system prompt and schema.<br>    &quot;&quot;&quot;<br>    messages = [<br>        SystemMessage(content=system_prompt),<br>        HumanMessage(content=f&quot;Review this code:\n{code}&quot;)<br>    ]<br>    response = model.invoke(messages)<br>    return response_schema.parse_raw(response.content)<br><br># Define the workflow graph<br>class ReviewState(BaseModel):<br>    code: str<br>    security: SecurityReview = None<br>    performance: PerformanceReview = None<br>    maintainability: MaintainabilityReview = None<br>    summary: str = &quot;&quot;<br><br>graph = StateGraph(ReviewState)<br><br>def security_review_node(state: ReviewState):<br>    model = create_groq_model()<br>    state.security = review_code(model, &quot;You are an expert in code security...&quot;, state.code, SecurityReview)<br>    return state<br><br>def performance_review_node(state: ReviewState):<br>    model = create_groq_model()<br>    state.performance = review_code(model, &quot;You are an expert in code performance...&quot;, state.code, PerformanceReview)<br>    return state<br><br>def maintainability_review_node(state: ReviewState):<br>    model = create_groq_model()<br>    state.maintainability = review_code(model, &quot;You are an expert in code quality...&quot;, state.code, MaintainabilityReview)<br>    return state<br><br>def summarize_reviews(state: ReviewState):<br>    model = create_groq_model()<br>    messages = [<br>        SystemMessage(content=&quot;You are a technical lead summarizing multiple code reviews.&quot;),<br>        HumanMessage(content=f&quot;Synthesize these code review results:\n{json.dumps(state.dict(), indent=2)}&quot;)<br>    ]<br>    response = model.invoke(messages)<br>    state.summary = response.content<br>    return state<br><br># Add nodes to the graph<br>graph.add_node(&quot;security_review&quot;, security_review_node)<br>graph.add_node(&quot;performance_review&quot;, performance_review_node)<br>graph.add_node(&quot;maintainability_review&quot;, maintainability_review_node)<br>graph.add_node(&quot;summary&quot;, summarize_reviews)<br><br># Define parallel execution<br>graph.add_edge(&quot;security_review&quot;, &quot;summary&quot;)<br>graph.add_edge(&quot;performance_review&quot;, &quot;summary&quot;)<br>graph.add_edge(&quot;maintainability_review&quot;, &quot;summary&quot;)<br><br>graph.set_entry_point([&quot;security_review&quot;, &quot;performance_review&quot;, &quot;maintainability_review&quot;])<br><br>graph.set_finish_node(&quot;summary&quot;)<br><br>workflow = graph.compile()</pre><h3>Orchestration</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vt--Lz9kRF7i_oU9" /><figcaption>Image by author</figcaption></figure><p>In the workflow of orchestrator workers, the first LLM (orchestrator) dissects and delegates tasks in a manner similar to decomposition and combines the results into one. This first LLM plays an important role in coordinating and executing worker specialization. Each worker is optimized for a specific subtask.</p><p>Let’s think like a team. In this example. In this team there are several sections: Business Analyst, Content Strategist, Social Media Manager, Marketing Analyst. Then, based on the project manager’s instructions, a measurement of the level of difficulty, type of assignment and task delegation is performed.</p><pre>async function implementTask(taskRequest: string) {<br>  const { object: taskPlan } = await generateObject({<br>    model: groq(&#39;llama-3.3-70b-versatile&#39;),<br>    schema: z.object({<br>      tasks: z.array(<br>        z.object({<br>          purpose: z.string(),<br>          taskName: z.string(),<br>          changeType: z.enum([&#39;create&#39;, &#39;modify&#39;, &#39;delete&#39;]),<br>        })<br>      ),<br>      estimatedEffort: z.enum([&#39;low&#39;, &#39;medium&#39;, &#39;high&#39;]),<br>    }),<br>    system: &#39;You are a Project Manager responsible for designing an efficient task execution strategy.&#39;,<br>    prompt: `Create a work plan for the following task:<br>    ${taskRequest}`,<br>  });</pre><p>After the product manager has successfully delegated the task, the next step is to use the previous parallel pattern and delegate to employees.</p><pre>const taskChanges = await Promise.all(<br>    taskPlan.tasks.map(async (task) =&gt; {<br>      // Determine job roles based on task type<br>      const workerSystemPrompt = {<br>        create: {<br>          &#39;Audience research&#39;: &#39;You are a Business Analyst. You are responsible for conducting in-depth research on the target audience.&#39;,<br>          &#39;Content creation&#39;: &#39;You are a Content Strategist. You design engaging content strategies tailored to the audience.&#39;,<br>          &#39;Account management&#39;: &#39;You are a Social Media Manager. You manage and optimize social media accounts.&#39;,<br>          &#39;Performance analysis&#39;: &#39;You are a Marketing Analyst. You analyze data and measure the success of marketing strategies.&#39;,<br>        }[task.taskName] || &#39;You are an expert professional in this field.&#39;,<br>        modify: {<br>          &#39;Account management&#39;: &#39;You are a Social Media Manager. You improve account management strategies to be more effective.&#39;,<br>        }[task.taskName] || &#39;You are a specialist enhancing task efficiency.&#39;,<br>        delete: &#39;You are an Operations Manager. You identify unnecessary tasks and remove them efficiently.&#39;,<br>      }[task.changeType];<br><br>      const { object: change } = await generateObject({<br>        model: groq(&#39;llama-3.3-70b-versatile&#39;),<br>        schema: z.object({<br>          explanation: z.string(),<br>          actionItems: z.array(z.string()),<br>        }),<br>        system: workerSystemPrompt,<br>        prompt: `Implement changes for the following task:<br>        - ${task.taskName}<br>        <br>        Purpose of change: ${task.purpose}<br>        <br>        Explain the reason for the change and provide a list of necessary action items.`,<br>      });<br><br>      return {<br>        task,<br>        implementation: change,<br>      };<br>    })<br>  );</pre><h4>Workflow with LangGraph python</h4><pre>class Task(BaseModel):<br>    purpose: str<br>    task_name: str<br>    change_type: Literal[&quot;create&quot;, &quot;modify&quot;, &quot;delete&quot;]<br><br>class TaskPlan(BaseModel):<br>    tasks: List[Task]<br>    estimated_effort: Literal[&quot;low&quot;, &quot;medium&quot;, &quot;high&quot;]<br><br>def generate_task_plan(task_request: str) -&gt; TaskPlan:<br>    response = client.chat.completions.create(<br>        model=&quot;llama-3.3-70b-versatile&quot;,<br>        messages=[<br>            {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: &quot;You are a Project Manager responsible for designing an efficient task execution strategy.&quot;},<br>            {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: f&quot;Create a work plan for the following task: {task_request}&quot;}<br>        ],<br>        response_format=&quot;json&quot;<br>    )<br>    return TaskPlan(**response[&quot;choices&quot;][0][&quot;message&quot;][&quot;content&quot;])<br><br>def implement_task_change(task: Task):<br>    worker_prompts = {<br>        &quot;create&quot;: {<br>            &quot;Audience research&quot;: &quot;You are a Business Analyst responsible for audience research.&quot;,<br>            &quot;Content creation&quot;: &quot;You are a Content Strategist designing content strategies.&quot;,<br>            &quot;Account management&quot;: &quot;You are a Social Media Manager optimizing accounts.&quot;,<br>            &quot;Performance analysis&quot;: &quot;You are a Marketing Analyst measuring strategy success.&quot;<br>        },<br>        &quot;modify&quot;: {<br>            &quot;Account management&quot;: &quot;You are a Social Media Manager improving strategies.&quot;<br>        },<br>        &quot;delete&quot;: &quot;You are an Operations Manager identifying unnecessary tasks.&quot;<br>    }<br><br>    system_prompt = worker_prompts.get(task.change_type, {}).get(task.task_name, &quot;You are an expert in this field.&quot;)<br><br>    response = client.chat.completions.create(<br>        model=&quot;llama-3.3-70b-versatile&quot;,<br>        messages=[<br>            {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: system_prompt},<br>            {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: f&quot;Implement changes for the task: {task.task_name}\nPurpose: {task.purpose}\nExplain why and list action items.&quot;}<br>        ],<br>        response_format=&quot;json&quot;<br>    )<br>    return response[&quot;choices&quot;][0][&quot;message&quot;][&quot;content&quot;]</pre><h3>Evaluation/Feedback Loops</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*r63VaL7NWMxE-Pxx" /><figcaption>Image by author</figcaption></figure><p>It’s not easy to tell ChatGPT to write a thesis. It’s hard for ChatGPT to think on its own. To improve the quality of the results, we need feedback from other people. We can do this by working on the parts of the thesis one at a time. For example, we can start with the background, then the content, and finally the conclusion. We need someone here to evaluate the results and provide feedback.</p><pre>  // Initial article generation<br>  const { text: article } = await generateText({<br>    model: groq(&#39;llama-3.1-8b-instant&#39;),<br>    system: &#39;You are a writer. Your task is to write a concise article in only 6 sentences! You might get additional feedback from your supervisor!&#39;,<br>    prompt: `Write a 6-sentence article on the topic: ${topic}`,<br>  });</pre><pre>while (iterations &lt; MAX_ITERATIONS) {<br>    // Evaluate current article<br>    const { object: evaluation } = await generateObject({<br>      model: groq(&#39;llama-3.3-70b-versatile&#39;), // use a larger model to evaluate<br>      schema: z.object({<br>        qualityScore: z.number().min(1).max(10),<br>        clearAndConcise: z.boolean(),<br>        engaging: z.boolean(),<br>        informative: z.boolean(),<br>        specificIssues: z.array(z.string()),<br>        improvementSuggestions: z.array(z.string()),<br>      }),<br>      system: &quot;You are a writing supervisor! Your agency specializes in concise articles! Your task is to evaluate the given article and provide feedback for improvements! Repeat until the article meets your requirements!&quot;,<br>      prompt: `Evaluate this article:<br><br>      Article: ${currentArticle}<br><br>      Consider:<br>      1. Overall quality<br>      2. Clarity and conciseness<br>      3. Engagement level<br>      4. Informative value`,<br>    });</pre><pre>if (<br>      evaluation.qualityScore &gt;= 8 &amp;&amp;<br>      evaluation.clearAndConcise &amp;&amp;<br>      evaluation.engaging &amp;&amp;<br>      evaluation.informative<br>    ) {<br>      break;<br>    }<br><br>    // Generate improved article based on feedback<br>    const { text: improvedArticle } = await generateText({<br>      model: groq(&#39;llama-3.3-70b-versatile&#39;), // use a larger model<br>      system: &#39;You are an expert article writer.&#39;,<br>      prompt: `Improve this article based on the following feedback:<br>      ${evaluation.specificIssues.join(&#39;\n&#39;)}<br>      ${evaluation.improvementSuggestions.join(&#39;\n&#39;)}<br><br>      Current Article: ${currentArticle}`,<br>    });<br><br>    currentArticle = improvedArticle;<br>    iterations++;<br>  }<br><br>  return {<br>    finalArticle: currentArticle,<br>    iterationsRequired: iterations,<br>  };<br>}</pre><h4>Workflow with LangGraph python</h4><pre>class Task(BaseModel):<br>    purpose: str<br>    task_name: str<br>    change_type: Literal[&quot;create&quot;, &quot;modify&quot;, &quot;delete&quot;]<br><br>class TaskPlan(BaseModel):<br>    tasks: List[Task]<br>    estimated_effort: Literal[&quot;low&quot;, &quot;medium&quot;, &quot;high&quot;]<br><br>def generate_task_plan(task_request: str) -&gt; TaskPlan:<br>    response = client.chat.completions.create(<br>        model=&quot;llama-3.3-70b-versatile&quot;,<br>        messages=[<br>            {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: &quot;You are a Project Manager responsible for designing an efficient task execution strategy.&quot;},<br>            {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: f&quot;Create a work plan for the following task: {task_request}&quot;}<br>        ],<br>        response_format=&quot;json&quot;<br>    )<br>    return TaskPlan(**response[&quot;choices&quot;][0][&quot;message&quot;][&quot;content&quot;])<br><br>def implement_task_change(task: Task):<br>    worker_prompts = {<br>        &quot;create&quot;: {<br>            &quot;Audience research&quot;: &quot;You are a Business Analyst responsible for audience research.&quot;,<br>            &quot;Content creation&quot;: &quot;You are a Content Strategist designing content strategies.&quot;,<br>            &quot;Account management&quot;: &quot;You are a Social Media Manager optimizing accounts.&quot;,<br>            &quot;Performance analysis&quot;: &quot;You are a Marketing Analyst measuring strategy success.&quot;<br>        },<br>        &quot;modify&quot;: {<br>            &quot;Account management&quot;: &quot;You are a Social Media Manager improving strategies.&quot;<br>        },<br>        &quot;delete&quot;: &quot;You are an Operations Manager identifying unnecessary tasks.&quot;<br>    }<br><br>    system_prompt = worker_prompts.get(task.change_type, {}).get(task.task_name, &quot;You are an expert in this field.&quot;)<br><br>    response = client.chat.completions.create(<br>        model=&quot;llama-3.3-70b-versatile&quot;,<br>        messages=[<br>            {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: system_prompt},<br>            {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: f&quot;Implement changes for the task: {task.task_name}\nPurpose: {task.purpose}\nExplain why and list action items.&quot;}<br>        ],<br>        response_format=&quot;json&quot;<br>    )<br>    return response[&quot;choices&quot;][0][&quot;message&quot;][&quot;content&quot;]</pre><h3>When to use Typescript and Python</h3><p>Vercel AI SDK and langgraph are one of the frameworks that have quite complete and neat documentation. In addition to Vercel AI SDK, there are other recommended frameworks such as</p><ul><li><a href="https://agentic.so/intro">Agentic</a>, AI agent standard library that works with any LLM and TS AI SDK.</li><li><a href="https://agentic.so/intro">Agno</a>, open-source python framework for building agentic systems</li><li><a href="https://rivet.ironcladapp.com/">Rivet</a>, a drag and drop GUI LLM workflow builder; and</li><li><a href="https://www.crewai.com/">Crew ai</a>, Python Framework for orchestrating collaborative AI agents</li></ul><p>Using a framework allows us to easily use standard patterns such as tools, memory, components and connect them by calling certain parameters.</p><blockquote>While Python has traditionally been the dominant language for AI and ML development due to its extensive libraries and frameworks, <strong>Typescript — a superset of JavaScript — has emerged as a strong contender for building</strong> scalable, maintainable, and efficient AI/ML applications.</blockquote><p>The use of Typescript and Python depends on your needs:</p><p>if you need to develop a full-stack application quickly, Typescript is sufficient without adding more stacks; if you want to be separate and build a more complex agent, Python is recommended.</p><p>Python is more recommended for building AI agents because of its full AI/ML library support and more frameworks, while Typescript is suitable if you focus on integration with the react framework, especially for fullstack developers.</p><p>For both Python and Typescript, you can check out the source code for other frameworks here.</p><p><a href="https://github.com/wahyudesu/multiple-ways-to-build-ai-agents">https://github.com/wahyudesu/multiple-ways-to-build-ai-agents</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d798c3435ec1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/himit-pens/building-ai-agent-workflows-with-python-typescript-d798c3435ec1">Building AI Agent Workflows with Python &amp; TypeScript</a> was originally published in <a href="https://medium.com/himit-pens">HIMIT PENS</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AdonisJS adalah Laravelnya JS???]]></title>
            <link>https://medium.com/himit-pens/adonisjs-adalah-laravelnya-js-1c5b1becc6b7?source=rss----d838c8bb2d71---4</link>
            <guid isPermaLink="false">https://medium.com/p/1c5b1becc6b7</guid>
            <category><![CDATA[javascript-frameworks]]></category>
            <dc:creator><![CDATA[HIMIT PENS]]></dc:creator>
            <pubDate>Tue, 21 Jan 2025 14:22:13 GMT</pubDate>
            <atom:updated>2025-01-21T14:22:13.304Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*TVUrt0ZkJ6RpHYfU.png" /></figure><p>Akhir akhir ini gue kedapatan project yang framework nya itu kata2 orang diinternet kalo framework ini dibilang Laravelnya js, dimana kedapatan kesamaan :</p><ol><li>Aritektur MVC (Model — View — Controller) : Keduanya sama sama memiliki pola MVC yang digunakan dalam struktur Webnya</li><li>Command-line interface (CLI): Keduanya memprovide CLI baik generate boilerplate, migrasi database, controller dan lain lain.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/619/0*Ojth7IGYNGtR9nqt.png" /></figure><p>Perbandingan CLI Adonis dan Laravel</p><h3>Fitur Utama</h3><ol><li><strong>Arsitektur MVC</strong></li></ol><p>Pola desain ini memisahkan aplikasi menjadi tiga komponen utama: model, view, dan controller agar mudah untuk mengorganisir mana untuk mengatur Koneksi databases, data yang masuk dan menampilkan data pada frontend</p><p><strong>2. Routing</strong></p><p>AdonisJS juga mendukung <strong>middleware</strong>, yaitu fungsi yang dijalankan sebelum atau sesudah request diproses. Middleware ini berguna untuk tugas seperti otentikasi, logging, dan validasi input.</p><p><strong>3. Dukungan Database</strong></p><p>AdonisJS mendukung berbagai jenis database seperti <strong>MySQL, PostgreSQL, SQLite, dan MongoDB.</strong></p><p><strong>4. Autentikasi</strong></p><p>AdonisJS mendukung berbagai penyedia otentikasi seperti JWT (JSON Web Token), OAuth2, dan session.</p><h3>Cara Installasi</h3><h3>1.Install AdonisJS CLI</h3><pre>npm install -g @adonisjs/cli</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/560/0*deCM8RSRFTT7qMSP.png" /></figure><h3>2. Buat Proyek Baru</h3><pre>adonis new hello-adonis<br>cd hello-adonis</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/445/0*jQ1RFGsFlAh7aGp4.png" /></figure><h3>3. Jalankan Server</h3><pre>npm run dev</pre><p>Akses aplikasi Anda di <a href="http://localhost:3333.">http://localhost:3333.</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*R3IwmIcxZRmSvhsY.png" /></figure><p>Jadi Begitulah caranya untuk installasi AdonisJS di env kalian, aslinya baru akhir-akhir ini pegang AdonisJs karena jarang populer juga tapi namanya coding pasti pattern nya tiap tiap bahasa pemrograman pasti sama :D</p><p><strong>Referensi</strong></p><p>Dokumentasi resmi : <a href="https://adonisjs.com/">https://adonisjs.com</a>/</p><p>Referensi 2 : <a href="https://perogeremmer.medium.com/berkenalan-dengan-adonisjs-framework-node-js-rasa-laravel-1d07d188d1f0">https://perogeremmer.medium.com/berkenalan-dengan-adonisjs-framework-node-js-rasa-laravel-1d07d188d1f0</a></p><p>Referensi 3 : <a href="https://blog.dot.co.id/adonisjs-a-fully-featured-web-framework-for-node-js-a99a6250d5db">https://blog.dot.co.id/adonisjs-a-fully-featured-web-framework-for-node-js-a99a6250d5db</a></p><p>Referensi 4 : <a href="https://medium.com/@imanimen/adonisjs-laravel-like-framework-for-nodejs-e0f08a12e804#:~:text=Similarities%3A,database%20migrations%2C%20and%20other%20tasks">https://medium.com/@imanimen/adonisjs-laravel-like-framework-for-nodejs-e0f08a12e804#:~:text=Similarities%3A,database%20migrations%2C%20and%20other%20tasks</a>.</p><p>Written by: <a href="https://medium.com/u/42f1d9b663e">Adryan Prawira</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1c5b1becc6b7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/himit-pens/adonisjs-adalah-laravelnya-js-1c5b1becc6b7">AdonisJS adalah Laravelnya JS???</a> was originally published in <a href="https://medium.com/himit-pens">HIMIT PENS</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI Engineer: How to Integrate a Ai into Your Personal Website]]></title>
            <link>https://medium.com/himit-pens/ai-engineer-how-to-integrate-a-ai-into-your-personal-website-a7878e85e548?source=rss----d838c8bb2d71---4</link>
            <guid isPermaLink="false">https://medium.com/p/a7878e85e548</guid>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[engineer]]></category>
            <category><![CDATA[website]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[portfolio]]></category>
            <dc:creator><![CDATA[Wahyu ikbal maulana]]></dc:creator>
            <pubDate>Tue, 21 Jan 2025 12:00:34 GMT</pubDate>
            <atom:updated>2025-01-21T12:46:19.600Z</atom:updated>
            <content:encoded><![CDATA[<p>Apakah kamu merasa FOMO dengan perkembangan AI saat ini? Mau belajar ai tapi bingung mulai dari mana? Sebagai mahasiswa IT, langkah pertama yang bisa kamu lakukan adalah dengan membuat aplikasi AI. Sambil belajar AI kamu juga sambil belajar bagaimana backend bekerja</p><p><a href="https://www.kdnuggets.com/how-to-learn-ai-the-lazy-way">https://www.kdnuggets.com/how-to-learn-ai-the-lazy-way</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*sECxtzExV8pRj8v4OGJa6Q.png" /><figcaption>Ai integration on webstite (generated via Napkin)</figcaption></figure><p>Di artikel ini, saya akan berbagi cara bagaimana membuat fitur ai yaitu chatbot menggunakan model AI, lengkap beserta praktik, implementasi, dan source code nya. Dan tentunya kamu bisa memulainya dengan gratis modal laptop, vscode ama eepiswlan.</p><h3>How we get started with ai?</h3><p>AI yang biasa kamu kenal, seperti ChatGPT adalah website atau aplikasi yang didukung oleh <strong>Large Language Models (LLMs)</strong>. Contohnya Anthropic dengan model Claude 3.5 Sonnet, Google dengan model Gemini 1.5 Pro, Meta dengan model Llama, BERT model, dan masih banyak lagi. So popularitas AI saat ini sebagian besar didorong oleh kemajuan dalam teknologi LLM.</p><blockquote>Large Language Model (LLM) adalah sistem kecerdasan buatan yang dirancang untuk memahami, menghasilkan, dan memanipulasi bahasa manusia dalam skala besar. LLM menggunakan teknik machine learning tingkat lanjut, khususnya arsitektur jaringan saraf yang dikenal sebagai <em>transformers</em>, untuk memproses dan menghasilkan teks yang mirip dengan cara manusia berkomunikasi</blockquote><p>Untuk mempelajari lebih lanjut tentang LLM lebih lengkapnya, kamu bisa artikel berikut ini</p><p><a href="https://www.cloudflare.com/learning/ai/what-is-large-language-model/">https://www.cloudflare.com/learning/ai/what-is-large-language-model/</a></p><p><a href="https://bbycroft.net/llm">https://bbycroft.net/llm</a></p><h3>So how to use LLMs on the website?</h3><p>Untuk menggunakan <strong>LLMs (Large Language Models)</strong> cukup mudah kamu bisa memanfaatkan API Provider untuk mengintegrasikan dengan cepat. Yang menjadi tantangan adalah bagaimana memilih LLM yang tepat dan sesuai. Ada berbagai pertimbangan dalam memilih model seperti menggunakan open source atau closed source LLM.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/786/0*bxBGBq4QDZ727V1B" /><figcaption>Open source vs Closed source LLM</figcaption></figure><p>Namun selain 2 jenis model, memilih <strong>LLM (Large Language Model)</strong> yang tepat juga melibatkan beberapa pertimbangan tambahan, seperti biaya dan kontrol, kamu bisa menggunakan API provider seperti openai atau untuk enterprise production ada opsi lain seperti mendeploy model di server sendiri dan biayanya sendiri cukup mahal :3.</p><p><a href="https://www.datacamp.com/tutorial/deploying-llm-applications-with-langserve">https://www.datacamp.com/tutorial/deploying-llm-applications-with-langserve</a></p><h3>Getting started with Next js project</h3><blockquote><em>Note: This section doesn’t cover the entire codebase. We’ll focus on crucial parts of the project to clarify implementation details. For example, we won’t discuss design and UI details but will focus on the more functional aspects of the project. For those interested in these details, you can examine the </em><a href="https://github.com/wahyudesu/Personal-Website-Next-js-Obsidian-Ai-"><em>source code</em></a></blockquote><h4>Why Next js?</h4><p>Next JS adalah modern react framework yang menawarkan lebih banyak fitur untuk pengembangan aplikasi web modern. Framework ini yang sangat populer karena menawarkan berbagai fitur yang memudahkan pengembangan aplikasi web modern, seperti server-side rendering (SSR) dan static site generation (SSG). Selain karena terkenal karena kecepatannya web ini juga sangat SEO-friendly dan cocok untuk full stack developer yang ingin mengembagkan gen ai apps dengan cepat</p><pre>#Create new project, you can choose between js or ts<br>npx create-next-app@latest nextjs-ai --tailwind --no-src-dir --eslint --app --import-alias &quot;@/*&quot;</pre><h4>Folder structure</h4><blockquote>The concept involves using an AI backend to handle API calls with an instructor, while the frontend displays the response to users.</blockquote><pre>nextjs-app-folder-structure/<br>├── app/                    # App Router<br>│   ├── api/                # Backend<br>│   │   ├── chat/           <br>│   │   │   └── route.tsx   # File route untuk API chat<br>│   ├── layout.tsx          # Layout global<br>│   └── page.tsx            # Halaman utama (/)<br>├── components/             # Folder untuk komponen reusable<br>├── public/                 # Folder untuk file statis (gambar, font, dll.)<br>├── next.config.mjs         # File konfigurasi Next.js<br>├── tsconfig.json           # File konfigurasi TypeScript<br>└── .env                    # Menyimpan API KEY</pre><h4>Implement LLM</h4><p>Pertama kamu perlu buat file json terlebih dahulu yang semakin banyak kamu memberi konteks tentang diri kamu, semakin mengerti chatbot personalisasi</p><pre>// knowledge_base.json<br><br>{<br>    &quot;nama&quot;: &quot;Sam Altman&quot;,<br>    &quot;tanggal lahir&quot;: &quot;1945-08-15&quot;,<br>    &quot;hobi&quot;: [<br>          &quot;Menggambar&quot;,<br>          &quot;Ngoding&quot;<br>        ],<br>    &quot;target pekerjaan&quot;: [<br>      &quot;data scientist&quot;,<br>      &quot;AI Engineer&quot;<br>    ],<br>    &quot;bahasa pemrograman&quot;: [<br>          &quot;Typescript&quot;,<br>          &quot;Python&quot;<br>        ],<br>    // more personal data <br>}</pre><blockquote>Tips: kamu bisa buatkan chatgpt buat mengekstrak resume yang kamu miliki ke json sebagai knowledge base</blockquote><p>Setelah itu kamu bisa set up backend yang menerima response dan mereturn output. Disini kita memakai <a href="https://groq.com/">groq</a> karena gratis, alternatifnya kamu bisa make <a href="https://ai.google.dev/">gemini</a> dan <a href="https://www.together.ai/">together</a></p><pre>#First install groq typescript library<br>npm i groq-sdk</pre><p>Buat file API Route menggunakan next server dan set up model dari groq terlebih dahulu</p><pre>// app/api/chat/route.ts<br>import { NextResponse } from &quot;next/server&quot;;<br>import { Groq } from &quot;groq-sdk&quot;;<br>import knowledge_base from &#39;@/data/knowledge_base.json&#39;;<br><br>const groq = new Groq({<br>  apiKey: process.env.GROQ_API_KEY,<br>});<br><br>// Salin GROQ API KEY di .env/.env.local</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/913/1*-SVBLoy6tbFdH2Bc3Dcvjw.png" /><figcaption>Supported Models on Groq</figcaption></figure><p>Ada berbagai model open source yang bisa kita pakai. Karena chatbot yang akan kita buat dirancang untuk memberikan jawaban yang sederhana, kita tidak memerlukan output yang terlalu besar. Namun, mengingat data JSON yang akan digunakan cukup panjang, sehingga diperlukan model dengan context window yang panjang, seperti <strong>Llama 3.1</strong>, <strong>Llama 3.3</strong>, atau <strong>Mixtral 8x7b?</strong>(Ndak rekomen hasil outputnya jelek :v). Selain itu, perlu juga mempertimbangkan limit token dari masing-masing model agar sesuai dengan kebutuhan kita.</p><p>Selanjutnya kita akan menangkap informasi dari data json dan membuat instruksi prompt yang efektif. Tujuannya adalah supaya si model dapat menghasilkan output yang relevan dan sesuai dengan konteks.</p><pre>// prompt engineer cuy<br>const knowledgeBaseContent = knowledge_base ? JSON.stringify(knowledge_base) : &quot;&quot;;<br><br>const chatCompletion = await groq.chat.completions.create({<br>  messages: [<br>    {<br>      role: &quot;system&quot;,<br>      content: `<br>        Anda adalah chatbot virtual assistant yang menggantikan Wahyu untuk menjawab pertanyaan dari orang lain.<br>        Wahyu lebih menyukai jawaban yang singkat, padat, jelas, namun tetap ramah. Berikut adalah beberapa aturan penting yang harus Anda ikuti:<br><br>        1. Jawablah berdasarkan informasi yang tersedia di knowledge base berikut:<br>           \n\n${knowledgeBaseContent}<br><br>        2. Jika tidak mengetahui jawaban, katakan dengan sopan: &quot;Maaf, saya tidak tahu.&quot;<br><br>        3. Jika pertanyaan bersifat terlalu pribadi, sensitif, atau toxic, balas dengan: &quot;Maaf, saya tidak bisa menjawab pertanyaan seperti itu.&quot;<br><br>        Pastikan Anda:<br>        - Menyapa pengguna dengan hangat di awal.<br>        - Memberikan jawaban yang membantu.<br>        - Menutup percakapan dengan ramah.<br>      `,<br>    },<br>    { role: &quot;user&quot;, content: message },<br>  ],<br>  model: &quot;llama-3.1-8b-instant&quot;,<br>});</pre><p>Berikut code lengkapnya dengan penambahan get request, validasi input, dan error handling</p><pre>// app/api/chat/route.ts<br><br>import { NextResponse } from &quot;next/server&quot;;<br>import { Groq } from &quot;groq-sdk&quot;;<br>import knowledge_base from &#39;@/data/knowledge_base.json&#39;;<br><br>const groq = new Groq({<br>  apiKey: process.env.GROQ_API_KEY,<br>});<br><br>export async function POST(req: Request) {<br>  try {<br>    const { message } = await req.json();<br><br>    if (!message) {<br>      return NextResponse.json(<br>        { error: &quot;Message is required&quot; },<br>        { status: 400 }<br>      );<br>    }<br><br>    // Menggabungkan knowledge base ke dalam system message<br>    const knowledgeBaseContent = knowledge_base ? JSON.stringify(knowledge_base) : &quot;&quot;;<br><br>    const chatCompletion = await groq.chat.completions.create({<br>      messages: [<br>        {<br>          role: &quot;system&quot;,<br>          content: `<br>            Anda adalah chatbot virtual assistant yang menggantikan Wahyu untuk menjawab pertanyaan dari orang lain.<br>            Wahyu lebih menyukai jawaban yang singkat, padat, jelas, namun tetap ramah. Berikut adalah beberapa aturan penting yang harus Anda ikuti:<br>    <br>            1. Jawablah berdasarkan informasi yang tersedia di knowledge base berikut:<br>               \n\n${knowledgeBaseContent}<br>    <br>            2. Jika tidak mengetahui jawaban, katakan dengan sopan: &quot;Maaf, saya tidak tahu.&quot;<br>    <br>            3. Jika pertanyaan bersifat terlalu pribadi, sensitif, atau toxic, balas dengan: &quot;Maaf, saya tidak bisa menjawab pertanyaan seperti itu.&quot;<br>    <br>            Pastikan Anda:<br>            - Menyapa pengguna dengan hangat di awal.<br>            - Memberikan jawaban yang membantu.<br>            - Menutup percakapan dengan ramah.<br>          `,<br>        },<br>        { role: &quot;user&quot;, content: message },<br>      ],<br>      model: &quot;llama-3.1-8b-instant&quot;,<br>    });<br><br>    const response = chatCompletion.choices[0].message.content;<br><br>    return NextResponse.json({ response: response });<br>  } catch (error) {<br>    console.error(&quot;Error in chat API:&quot;, error);<br>    return NextResponse.json(<br>      { error: &quot;Maaf bot sedang eror, coba lain kali&quot; },<br>      { status: 500 }<br>    );<br>  }<br>}</pre><h4>How about Front-end?</h4><p>Kamu perlu siapkan page baru baru untuk membuat halaman yang berisi interface chatbot. Sebelum itu kamu perlu mengambil data dari backend dan menyimpan response variable.</p><pre>// app/ai-chat/page.tsx<br><br>try {<br>  // send request POST to endpoint /api/chat<br>  const response = await fetch(&quot;/api/chat&quot;, {<br>    method: &quot;POST&quot;,<br>    headers: { &quot;Content-Type&quot;: &quot;application/json&quot; },<br>    body: JSON.stringify({ message: sendMessage.text }),<br>  });<br><br>  // Get response data and format to JSON<br>  const data = await response.json();<br><br>} catch (error) {<br>  // Error handling<br>  console.error(&quot;Error:&quot;, error.message);<br>}</pre><p>Untuk interface nya kamu bisa menuju ke chat.com atau v0.dev untuk membuat ui sederhana dari kode yang anda buat. Jangan lupa lampirkan kode api supaya ai mengerti konteks dan dapat membuat ui yang dapat berjalan sesuai dengan backend</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*aK0NFzYm6W9k15QlvSXc9Q.gif" /><figcaption>Dan woilah jadi wak</figcaption></figure><h4>What’s next</h4><p>- Kamu bisa prefer menggunakan framework/sdk seperti AI SDK by Vercel, Langchain, Llamaindex. Framework ini memudahkan pembuatan pipeline, data ingest, dan penggantian model tanpa perlu mengubah terlalu banyak kode.<br>- Kamu bisa menambahkan lebih banyak data, seperti project, resume, interest dan menerapkan teknik-teknik prompt, seperti Meta prompting dan Contextual priming untuk mendapatkan hasil respon yang lebih akurat dan lebih relevan. Pelajari <a href="https://www.promptingguide.ai/">https://www.promptingguide.ai</a><br>- Untuk keamanan dan privasi data, kamu bisa menggunakan database nosql, atau sql dengan kolom json. Kalau dari project yang aku pakai aku memakai <a href="https://neon.tech/home">neon</a> postgresql<br>- Terus belajar dan jangan berhenti mencoba👍🏻</p><p>Blog ini juga kamu bisa baca di website pribadiku</p><p><a href="https://www.wahyuikbal.web.id/blog/AI-Engineer-How-to-Integrate-a-Ai-into-Your-Personal-Website">https://www.wahyuikbal.web.id/blog/AI-Engineer-How-to-Integrate-a-Ai-into-Your-Personal-Website</a></p><p>Jika kamu memiliki pertanyaan atau ingin berdiskusi lebih lanjut, jangan sungkan untuk menghubungi saya, atau bertanya lewat ai bot pribadi saya di <a href="https://www.wahyuikbal.web.id/chat-me">https://www.wahyuikbal.web.id/chat-me</a> . Untuk knowledge tambahan berikut referensi dan resources yang bisa kamu baca pas lagi gabud.</p><h3>Resources</h3><p><a href="https://arxiv.org/abs/1706.03762">https://arxiv.org/abs/1706.03762</a></p><p><a href="https://pub.towardsai.net/the-choice-for-businesses-between-open-source-and-proprietary-models-to-deploy-generative-ai-adcc336cfa41">https://pub.towardsai.net/the-choice-for-businesses-between-open-source-and-proprietary-models-to-deploy-generative-ai-adcc336cfa41</a></p><p><a href="https://github.com/wsxqaza12/Comparison-of-LLM-Specifications">https://github.com/wsxqaza12/Comparison-of-LLM-Specifications</a></p><p><a href="https://www.scribbledata.io/blog/large-language-models-history-evolutions-and-future/">https://www.scribbledata.io/blog/large-language-models-history-evolutions-and-future/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a7878e85e548" width="1" height="1" alt=""><hr><p><a href="https://medium.com/himit-pens/ai-engineer-how-to-integrate-a-ai-into-your-personal-website-a7878e85e548">AI Engineer: How to Integrate a Ai into Your Personal Website</a> was originally published in <a href="https://medium.com/himit-pens">HIMIT PENS</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Implementing Multi-Role User Access in Laravel API with Spatie]]></title>
            <link>https://medium.com/himit-pens/implementing-user-roles-in-laravel-api-with-spatie-48b059a3b18f?source=rss----d838c8bb2d71---4</link>
            <guid isPermaLink="false">https://medium.com/p/48b059a3b18f</guid>
            <category><![CDATA[laravel-api]]></category>
            <category><![CDATA[spatie]]></category>
            <category><![CDATA[laravel]]></category>
            <category><![CDATA[multi-role]]></category>
            <category><![CDATA[middleware]]></category>
            <dc:creator><![CDATA[Muhammad Satrio Asadel]]></dc:creator>
            <pubDate>Sun, 27 Oct 2024 01:16:58 GMT</pubDate>
            <atom:updated>2024-11-17T07:12:53.225Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NsgKh70wvzISMmTXVQEiAA.png" /></figure><p>Spatie is a package that makes it easy for us to manage user permissions and roles in a database. With Spatie, developers can quickly implement role-based access control (RBAC) in their applications, increasing security and flexibility. In this opportunity, we will create a simple project using Laravel 11 that integrates Spatie. This project will provide a step-by-step guide on how to set up and use role and permission management, and explain the benefits of this approach in application development. Let’s start our journey in creating more secure and structured applications!</p><p><a href="https://spatie.be/docs/laravel-permission/v6/introduction">Introduction | laravel-permission</a></p><h3><strong>Step 1: Set Up Laravel Project</strong></h3><p>First create a new project or you can skip this if you already have one.</p><pre>laravel new spatie-auth-api</pre><ol><li>Would you like to install a starter kit? <strong>breeze</strong></li><li>Which Breeze stack would you like to install? <strong>api</strong></li><li>Which testing framework do you prefer? <strong>0</strong></li><li>Would you like to initialize a Git repository? <strong>no</strong></li><li>Which database will your application use? <strong>mysql</strong></li><li>Default database updated. Would you like to run the default database migrations? <strong>no</strong></li></ol><p>Let’s open our project in vscode</p><pre>code spatie-auth-api</pre><h3>Step 2: Install Spatie Laravel Permission</h3><p>Next, we need to install the Spatie Laravel Permission package. Open a terminal and run the following command:</p><pre>composer require spatie/laravel-permission</pre><h3><strong>Step 3: Publish Configuration</strong></h3><ol><li>Edit the app.php file in bootstrap/:</li></ol><pre>&lt;?php<br><br>use Illuminate\Foundation\Application;<br>use Illuminate\Foundation\Configuration\Exceptions;<br>use Illuminate\Foundation\Configuration\Middleware;<br><br>return Application::configure(basePath: dirname(__DIR__))<br>    -&gt;withRouting(<br>        web: __DIR__.&#39;/../routes/web.php&#39;,<br>        api: __DIR__.&#39;/../routes/api.php&#39;,<br>        commands: __DIR__.&#39;/../routes/console.php&#39;,<br>        health: &#39;/up&#39;,<br>    )<br>    -&gt;withMiddleware(function (Middleware $middleware) {<br>        $middleware-&gt;api(prepend: [<br>            \Laravel\Sanctum\Http\Middleware\EnsureFrontendRequestsAreStateful::class,<br>        ]);<br><br>        $middleware-&gt;alias([<br>            &#39;verified&#39; =&gt; \App\Http\Middleware\EnsureEmailIsVerified::class,<br>        ]);<br><br>        $middleware-&gt;alias([ // thiss<br>            &#39;role&#39; =&gt; \Spatie\Permission\Middleware\RoleMiddleware::class,<br>            &#39;permission&#39; =&gt; \Spatie\Permission\Middleware\PermissionMiddleware::class,<br>            &#39;role_or_permission&#39; =&gt; \Spatie\Permission\Middleware\RoleOrPermissionMiddleware::class,<br>        ]); // until thiss<br>    })<br>    -&gt;withExceptions(function (Exceptions $exceptions) {<br>        //<br>    })-&gt;create();</pre><p>2. Once the installation is complete, publish the configuration and migration files using the following commands:</p><pre>php artisan vendor:publish --provider=&quot;Spatie\Permission\PermissionServiceProvider&quot;</pre><h3>Step 4: Clear Configuration Cache</h3><p>If you have cached the configuration, clear it with the following command so the package can access the latest settings:</p><pre>php artisan config:clear</pre><h3><strong>Step 5: Configure Database</strong></h3><p>Configure your database credentials in the .env file:</p><pre>DB_CONNECTION=mysql<br>DB_HOST=127.0.0.1<br>DB_PORT=3306<br>DB_DATABASE=spatie_auth_api<br>DB_USERNAME=your_database_username<br>DB_PASSWORD=your_database_password</pre><h3>Step 6: Run Migration</h3><p>Next, make sure your mysql is runningand run the migrations to create the necessary tables in your database:</p><pre>php artisan migrate</pre><ol><li>Would you like to create it? <strong>yes</strong></li></ol><blockquote>By default spatie will create a new migration file, namely “create permission tables”</blockquote><h3>Step 7: Add Trait to User Model</h3><p>Open the User.php model and add HasApiTokens and the HasRoles trait from Spatie:</p><pre>// ...<br>use Spatie\Permission\Traits\HasRoles;<br>use Laravel\Sanctum\HasApiTokens;<br><br>class User extends Authenticatable<br>{<br>    use HasFactory, Notifiable, HasRoles, HasApiTokens;<br><br>    // ...<br>}</pre><h3>Step 8: Modify AuthenticatedSessionController</h3><p>First edit the AuthenticatedSessionController.php file in app/Http/Controllers/Auth/:</p><pre>// ...<br>use App\Models\User; // this<br>use Illuminate\Http\JsonResponse; // this<br><br>class AuthenticatedSessionController extends Controller<br>{<br>    public function store(Request $request): JsonResponse // this<br>    {<br>        $validatedData = $request-&gt;validate([<br>            &#39;email&#39; =&gt; [&#39;required&#39;, &#39;string&#39;, &#39;email&#39;],<br>            &#39;password&#39; =&gt; [&#39;required&#39;, &#39;string&#39;],<br>        ]);<br><br>        $user = User::where(&#39;email&#39;, $request-&gt;email)-&gt;first();<br><br>        if(!Auth::attempt($validatedData)) {<br>            return response()-&gt;json([<br>                &#39;success&#39; =&gt; false,<br>                &#39;message&#39; =&gt; &#39;Login failed. Please check your credentials.&#39;<br>            ], 401);<br>        }<br><br>        $token = $user-&gt;createToken($user-&gt;name)-&gt;plainTextToken;<br>        <br>        return response()-&gt;json([<br>            &#39;success&#39; =&gt; true,<br>            &#39;user&#39; =&gt; $user,<br>            &#39;access_token&#39; =&gt; $token<br>        ], 200);<br>    }<br>// ...<br>}</pre><blockquote>If you want to create a role directly. Jump to <strong>step 10:</strong> Create Role Permission Seeder</blockquote><h3>Step 9: Create Post CRUD to Simplify Role and Permission Visualization</h3><blockquote>Let’s Just Create a Simple Post CRUD</blockquote><ol><li>Migration for Posts Table</li></ol><pre>php artisan make:migration create_posts_table</pre><p>2. Edit migration file</p><pre>// ...<br>public function up()<br>    {<br>        Schema::create(&#39;posts&#39;, function (Blueprint $table) {<br>            $table-&gt;id();<br>            $table-&gt;string(&#39;title&#39;);  // this<br>            $table-&gt;text(&#39;content&#39;);  // this<br>            $table-&gt;timestamps();<br>        });<br>    }<br>// ...</pre><p>3. Model for Post</p><p>Open a terminal and run the following command:</p><pre>php artisan make:model Post</pre><p>Edit the Post.php in app/Models/:</p><pre>// ...<br>use Illuminate\Database\Eloquent\Factories\HasFactory; // this<br>use Illuminate\Database\Eloquent\Model;<br><br>class Post extends Model<br>{<br>    use HasFactory; // this<br><br>    protected $fillable = [ // this<br>        &#39;title&#39;, &#39;content&#39; <br>    ];<br>}<br>// ...</pre><p>4. Controller for CRUD</p><p>Open a terminal and run the following command:</p><pre>php artisan make:controller PostController --api</pre><p>Edit the PostController.php in app/Http/Controllers/:</p><pre>&lt;?php<br><br>namespace App\Http\Controllers;<br><br>use App\Models\Post;<br>use Illuminate\Http\Request;<br><br>class PostController extends Controller<br>{<br>    public function index()<br>    {<br>        return response()-&gt;json([<br>            &#39;success&#39; =&gt; true,<br>            &#39;data&#39; =&gt; Post::latest()-&gt;get()<br>        ]);<br>    }<br><br>    public function store(Request $request)<br>    {<br>        $request-&gt;validate([<br>            &#39;title&#39; =&gt; &#39;required|string|max:255&#39;,<br>            &#39;content&#39; =&gt; &#39;required|string&#39;,<br>        ]);<br><br>        $post = Post::create($request-&gt;all());<br><br>        return response()-&gt;json([<br>            &#39;success&#39; =&gt; true,<br>            &#39;message&#39; =&gt; &#39;Post created successfully&#39;,<br>            &#39;data&#39; =&gt; $post<br>        ], 201);<br>    }<br><br>    public function show($id)<br>    {<br>        $post = Post::findOrFail($id);<br>        return response()-&gt;json([<br>            &#39;success&#39; =&gt; true,<br>            &#39;data&#39; =&gt; $post<br>        ]);<br>    }<br><br>    public function update(Request $request, $id)<br>    {<br>        $post = Post::findOrFail($id);<br>        $post-&gt;update($request-&gt;all());<br><br>        return response()-&gt;json([<br>            &#39;success&#39; =&gt; true,<br>            &#39;message&#39; =&gt; &#39;Post updated successfully&#39;,<br>            &#39;data&#39; =&gt; $post<br>        ], 201);<br>    }<br><br>    public function destroy($id)<br>    {<br>        $post = Post::findOrFail($id);<br>        $post-&gt;delete();<br><br>        return response()-&gt;json([<br>            &#39;success&#39; =&gt; true,<br>            &#39;message&#39; =&gt; &#39;Post deleted successfully&#39;,<br>        ]);<br>    }<br>}</pre><p>5. Run the migration to add the posts table to the database.</p><pre>php artisan migrate</pre><h3>Step 10: Create Role Permission Seeder</h3><ol><li>Edit the DatabaseSeeder.php in database/seeders/:</li></ol><pre>&lt;?php<br><br>namespace Database\Seeders;<br><br>use App\Models\User;<br>// use Illuminate\Database\Console\Seeds\WithoutModelEvents;<br>use Illuminate\Database\Seeder;<br>use Spatie\Permission\Models\Role;<br><br>class DatabaseSeeder extends Seeder<br>{<br>    public function run(): void<br>    {<br>        $adminRole = Role::create([&#39;name&#39; =&gt; &#39;admin&#39;]);<br>        $authorRole = Role::create([&#39;name&#39; =&gt; &#39;author&#39;]);<br>        $viewerRole = Role::create([&#39;name&#39; =&gt; &#39;viewer&#39;]);<br><br>        $userAdmin = User::create([<br>            &#39;name&#39; =&gt; &#39;Admin&#39;,<br>            &#39;email&#39; =&gt; &#39;admin@gmail.com&#39;,<br>            &#39;password&#39; =&gt; bcrypt(&#39;adminuhuy&#39;),<br>        ]);<br>        $userAuthor = User::create([<br>            &#39;name&#39; =&gt; &#39;Author&#39;,<br>            &#39;email&#39; =&gt; &#39;author@gmail.com&#39;,<br>            &#39;password&#39; =&gt; bcrypt(&#39;authoruhuy&#39;),<br>        ]);<br>        $userViewer = User::create([<br>            &#39;name&#39; =&gt; &#39;Viewer&#39;,<br>            &#39;email&#39; =&gt; &#39;viewer@gmail.com&#39;,<br>            &#39;password&#39; =&gt; bcrypt(&#39;vieweruhuy&#39;),<br>        ]);<br><br>        $userAdmin-&gt;assignRole($adminRole);<br>        $userAuthor-&gt;assignRole($authorRole);<br>        $userViewer-&gt;assignRole($viewerRole);<br>    }<br>}</pre><p>2. Now, let’s insert the data we just created using seeder</p><pre>php artisan db:seed</pre><p>3. Ensure the data has been inserted into our database</p><pre>SELECT * FROM users;<br>SELECT * FROM roles;<br>SELECT * FROM model_has_roles;</pre><figure><img alt="Table Users" src="https://cdn-images-1.medium.com/max/784/1*7Bn2ZDRmd7OzvTf6T0uc3Q.png" /><figcaption>Table users</figcaption></figure><figure><img alt="Table Roles" src="https://cdn-images-1.medium.com/max/207/1*a8Fl6878Tocj-uVwJduGOQ.png" /><figcaption>Table roles</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/284/1*xr0AsHVOA6_3svvEq2SX-w.png" /><figcaption>Table model_has_roles</figcaption></figure><h3>Step 11: Creating api routes</h3><p>Edit the Post.php in app/Models/:</p><pre>&lt;?php<br><br>use App\Http\Controllers\Auth\AuthenticatedSessionController;<br>use App\Http\Controllers\PostController;<br>use Illuminate\Http\Request;<br>use Illuminate\Support\Facades\Route;<br><br>// Route::middleware([&#39;auth:sanctum&#39;])-&gt;get(&#39;/user&#39;, function (Request $request) {<br>//     return $request-&gt;user();<br>// });<br><br>Route::post(&#39;/login&#39;, [AuthenticatedSessionController::class, &#39;store&#39;]);<br><br>// Route::apiResource(&#39;/posts&#39;, PostController::class);<br><br>Route::middleware([&#39;auth:sanctum&#39;])-&gt;group(function() {<br>    Route::get(&#39;/posts&#39;, [PostController::class, &#39;index&#39;])-&gt;middleware(&#39;role:admin|viewer&#39;);<br>    Route::post(&#39;/posts&#39;, [PostController::class, &#39;store&#39;])-&gt;middleware(&#39;role:admin|author&#39;);<br>    Route::put(&#39;/posts/{post}&#39;, [PostController::class, &#39;update&#39;])-&gt;middleware(&#39;role:admin&#39;);<br>    Route::delete(&#39;/posts/{post}&#39;, [PostController::class, &#39;destroy&#39;])-&gt;middleware(&#39;role:admin&#39;);<br>    Route::get(&#39;/posts/{post}&#39;, [PostController::class, &#39;show&#39;])-&gt;middleware(&#39;role:admin|viewer&#39;);<br>});</pre><h3>Step 12: Testing with postman</h3><p>Next, we will create a login post and CRUD operations using bearer tokens with Postman. You can use the testing from this <a href="https://github.com/Asadell/spatie-auth-api/tree/main/.TESTCASE">link</a>.</p><ol><li>Start a local development server</li></ol><pre>php artisan serve</pre><p>2. All Test Case</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/351/1*MgArNSA5CJL5kVV-133ZYQ.png" /><figcaption><a href="https://github.com/Asadell/spatie-auth-api/tree/main/.TESTCASE">link</a></figcaption></figure><p>3. Create POST login</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/667/1*G_4ExiUaR8e-rXn0BsAfdA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Tcg5UsAY4wPw5Xswln8eyg.png" /><figcaption>Run the request, and we will receive the access_token. Keep it safe because we will use it for CRUD authorization.</figcaption></figure><p>4. Failed to get posts because we haven’t set the bearer token yet.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bDtYr1JNJnP7U4uACzPBmQ.png" /></figure><p>5. Get All Posts with Valid Access Token</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w3_NO8_wy7CNPooP1V8_aA.png" /><figcaption>Set the authentication type according to your needs.</figcaption></figure><p>6. Try CRUD operations with different roles.</p><h3>Closing</h3><p>Spatie makes it incredibly easy for us to create roles and manage permissions effectively. Thank you for reading my blog, and I hope you found it helpful. Feel free to visit the following <a href="https://github.com/Asadell/spatie-auth-api">link to download the project</a>. If you have any questions, don’t hesitate to reach out to me on Instagram asadel.belajar</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=48b059a3b18f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/himit-pens/implementing-user-roles-in-laravel-api-with-spatie-48b059a3b18f">Implementing Multi-Role User Access in Laravel API with Spatie</a> was originally published in <a href="https://medium.com/himit-pens">HIMIT PENS</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Apa sih Web-RTC itu?]]></title>
            <link>https://medium.com/himit-pens/apa-sih-web-rtc-itu-ee0f39ee06eb?source=rss----d838c8bb2d71---4</link>
            <guid isPermaLink="false">https://medium.com/p/ee0f39ee06eb</guid>
            <category><![CDATA[webrtc]]></category>
            <dc:creator><![CDATA[Andriana Wahyu]]></dc:creator>
            <pubDate>Tue, 15 Oct 2024 01:02:10 GMT</pubDate>
            <atom:updated>2024-10-15T01:02:08.920Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/818/1*mFNPaORB9d8-WobLOyGGxQ.jpeg" /></figure><p><strong>Web-RTC (Web Real-Time Communication)</strong> adalah protokol peer-to-peer yang memungkinkan komunikasi langsung antar perangkat. Meskipun bersifat <em>peer-to-peer</em>, Web-RTC tetap memerlukan server untuk memastikan perangkat-perangkat dapat saling terhubung.</p><h4>Komponen Penting Web-RTC</h4><p>Sebelum menggunakan Web-RTC, ada beberapa hal dan konfigurasi yang perlu dipahami:</p><ol><li><strong>Signaling</strong><br><strong><em>Peer to Peer Connection</em></strong> menunjukkan media koneksi sebenarnya, yang dikirim langsung antar perangkat, yang mana harus berkomunikasi terlebih dahulu dengan server. <br>Pertukaran di ilustrasikan pada bagian<em> “offer”</em> dan <em>“answer”</em> yang merupakan bagian dari <strong>webRTC</strong>. <em>Offer</em> dan <em>Answer</em> ini bukan berisi pesan real dari pengguna, melainkan berisi SDP yang perlu dikirimkan oleh aplikasi ke perangkat lain untuk menghubungkan suatu sesi.</li></ol><p><strong>Protokol dan Komponen Pendukung</strong></p><ol><li><strong>STUN (Session Traversal Utilities for NAT)</strong><br>protokol untuk menemukan alamat IP Public dan menentukan batasan batasan apapun dalam router yang dapat mencegah koneksi secara langsung dengan device lain.<br>Klien akan mengirimkan permintaan pada server STUN di internet yang akan membalas dengan alamat klien ( public IP ) dan balasan mengenai apakah klien dapat diakses dibelakang NAT router.</li><li><strong>TURN ( Traversal Using Relays arround NAT )</strong><br>Beberapa router yang menggunakan NAT terkadang menggunakan batasan yang kita sebut dengan <strong>Symetruc NAT</strong> . Ini berarti router hanya akan menerima koneksi dari device yang pernah disambungkan sebelumnya.<br>TURN dimaksudkan untuk melewati batasan Symetruc NAT dengan membuka koneksi melalui server TURN dan meneruskan semua informasi melalu server tersebut. Secara singkatnya device kita akan membuat koneksi dengan server TURN dan memberitahu semua device rekan untuk mengirimkan paket ke server yang akan diteruskan ke device kita.</li><li><strong>NAT ( Network Address Translation )</strong><br>NAT <em>( Network Address Translation)</em><strong><em> </em></strong>digunakan untuk memberikan perangkat kita sebuah alamat IP. Sebuah router akan memiliki IP public, begitu juga dengan perangkat yang terkoneksi dengan router tersebut. Pada beberapa router memiliki batasan mengenai siapa yang dapat terhubung ke perangkat di jaringan tersebut.</li><li><strong>ICE (Interactive Connectivity Estabilishment) <br></strong>ICE (<em>Interactive Connectivity Estabilishment</em>)<strong> </strong>adalah<strong> </strong>kerangka yang memungkinkan web dari device A terhubung dengan device-device lainnya secara langsung. Banyak alasan kenapa koneksi secara langsung antara device A dan device B tidak bekerja, salah satunya adalah karena harus melewati firewall yang mana mencegah kedua device terkoneksi secara langsung</li><li><strong>SDP (Session Description Protocol)<br></strong>Format dasar untuk mendeskripsikan sesi komunikasi multimedia untuk koneksi peer to peer. SDP mencakup informasi dari koneksi rekan seperti (<em>Codec</em>, alamat sumber, jenis media dan audio).</li></ol><h4>Alur Kerja Web-RTC</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/777/1*1qdlz69WdQ7Nq6aLLaLjkQ.jpeg" /></figure><ol><li>Ketika <em>Device A</em> ingin melakukan <em>peer to peer connection</em> dengan <em>device B</em></li><li>Device A akan membuat sebuah<em> “offer”</em> yang mana berisi <em>ice candidate</em> dan beberapa data untuk membuat <em>SDP </em>tentang public address dari device A ( Device A akan melakukan ping pada server STUN yang mana <em>STUN server</em> akan mengembalikan info mengenai public IP dari device A. Namun jika device A memiliki NAT yang sangat ketat maka <strong><em>STUN server</em></strong> tidak bisa mendapatkan informasi mengenai public IP dari device A itu sendiri, yang akhirnya device A harus menggunakan TURN server yang mana <strong><em>TURN server </em></strong>memiliki public IP sendiri dan hal tersebut yang akan digunakan device A untuk memenuhi persyaratan dari <em>offer</em> yang akan dikirimkan ke <em>signalling server.</em></li><li><strong>Signalling server</strong> menerima <em>offer</em> dari <em>device A</em> lalu meneruskan isi pesan dari <em>device A</em> ke <em>device B.</em></li><li><em>Device B</em> akan menerima <em>offer</em> dari <em>device A</em> melalui <em>signalling server</em> lalu <em>device B</em> menggunakan <strong>SDP</strong> yang dikirimkan dengan <em>offer</em> dari <em>device A</em> menjadi <strong><em>Remote Descriptionnya</em></strong>. Setelah itu <em>device B</em> akan membuat sebuah <em>answer </em>dengan cara yang sama seperti<em> device A</em> membuat sebuah <em>offer </em>dan mengirimkannya ke <strong><em>signalling server.</em></strong></li><li><strong><em>Signalling server</em></strong> menerima dan meneruskan pesan <em>answer</em> dari <em>device</em> B ke <em>device</em> A</li><li><em>Device A</em> menerima pesan <em>answer</em> dari <em>Device B</em> dan menggunakan <strong>SDP</strong> dari <em>device B</em> sebagai <strong><em>Remote Descriptionnya.</em></strong></li><li>Setelah<em> offer</em> dan <em>answer </em>dari masing masing device yang mana di dalam tersebut juga terdapat <em>ice candidate</em> dari device sudah terkirim, maka mereka akan mencoba menemukan <em>ice candidate</em> dengan jalur paling efektif mana yang akan digunakan sebagai jalur komunikasi atau pertukaran pesan.</li><li>Setelah <em>ice candidate</em> ( jalur tercepat ) sudah ditemukan maka kedua device dapat melakukan <em>peer to peer connection</em>. Jika sebelumnya device menggunakan <strong>TURN server</strong> untuk membuat sebuah offer atau <em>answer </em>maka perpindahan pesan akan tetap melalui <strong>TURN Server</strong> terlebih dahulu.</li></ol><p>Sumber<br><a href="https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API">https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ee0f39ee06eb" width="1" height="1" alt=""><hr><p><a href="https://medium.com/himit-pens/apa-sih-web-rtc-itu-ee0f39ee06eb">Apa sih Web-RTC itu?</a> was originally published in <a href="https://medium.com/himit-pens">HIMIT PENS</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kamu anak sains data? Kamu wajib tahu data version control]]></title>
            <link>https://medium.com/himit-pens/kamu-anak-sains-data-kamu-wajib-tahu-data-version-control-ae25bf7b012c?source=rss----d838c8bb2d71---4</link>
            <guid isPermaLink="false">https://medium.com/p/ae25bf7b012c</guid>
            <category><![CDATA[git]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[dvc]]></category>
            <dc:creator><![CDATA[Wahyu ikbal maulana]]></dc:creator>
            <pubDate>Tue, 17 Sep 2024 12:22:53 GMT</pubDate>
            <atom:updated>2024-09-17T12:22:53.347Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J2ECCFuIYQQXbiB9TK1rSg.png" /><figcaption>Data version control</figcaption></figure><p>Sebagai mahasiswa yang fokus belajar IT, mungkin kamu sudah familiar dengan yang namanya git. Dengan git kita bisa berkolaborasi dan memanage versi yang berbeda dalam sebuah aplikasi. Ketika ada yang melakukan perubahan pada kode sumber aplikasi, Git memungkinkan kita untuk melacak setiap perubahan. Ini sangat penting dalam software engineering, di mana sering terjadi perubahan kode yang memerlukan peninjauan dan pengelolaan yang teliti.</p><p>Seorang Data Scientist atau Data Engineer memerlukan hdal yang sama dalam mengelola data, perlunya beberapa versi data yang berbeda terkait penggunaan model machine learning. Disini perannya DVC kepake, DVC atau Data Version Control adalah sebuah tools yang mengatasi masalah dalam project machine learning, yaitu mengatasi ‘large dataset’ dengan version control systems.</p><h4>Namun, bagaimana cara kerja data version control?</h4><p>Mirip dengan dengan cara kerja git, Data Version Control (DVC) adalah tools berbasis Git management untuk Machine Learning Development. Sistem ini memungkinkan kita untuk mengelola data dan model dalam project data science. Version control system ini digunakan dalam software development, mirip dengan Git, tetapi spesifik ke data science projects.</p><p>DVC berfungsi sebagai jembatan git dengan data, dengan menyimpan perubahan dan versi data dengan git namun di tempat terpisah, seperti local atau cloud. Ini memungkinkan untuk kita bisa mengelola data besar dengan efisien tanpa mengotak-atik directory dalam repo git.</p><blockquote>Pada dasarnya Git LFS dan DVC dirancang untuk penggunaan yang berbeda, Git LFS memfasilitasi penanganan file besar yang workflownya berpusat pada Git, sedangkan DVC melakukan pendekatan terpisah untuk mengelola dataset yang ukurannya besar. <br>Untuk detail lebih, bisa menuju ke link article berikut <a href="https://stackoverflow.com/questions/58541260/difference-between-git-lfs-and-dvc">Perbedaan git-lfs dengan dvc</a></blockquote><h4>Belajar praktek DVC</h4><p>DVC bisa dipake di Windows, Mac, dan linux. Untuk latihannya kita bisa membuat project sederhana, disini kita memakai terminal untuk menginstall dan melakukan inisiasi dvc</p><pre>pip install dvc</pre><p>Setelah itu membuat folder directory baru dan beberapa file python</p><pre>mkdir dvc_tutorial # Buat folder baru<br>cd dvc_tutorial # Masuk ke folder yang telah dibuat<br>mkdir src models data# Buat sub-directories/sub-folder<br>touch src/split.py # Buat file baru<br>touch src/train.py</pre><p>Mengunduh dataset dan disimpan ke folder data</p><pre>curl https://raw.githubusercontent.com/BexTuychiev/medium_stories/master/fu-datasets/diamonds.csv -o data/diamonds.csv</pre><p>Setelah itu melakukan inisialisasi Git dan membuat commit untuk pertama kali</p><pre>git init<br>git add src<br>git commit -m &quot;Initial commit git&quot;</pre><p>Disini git hanya berhasil mentracking directory src, dan untuk inisialisasi data kita perlu menggunakan yang namanya dvc</p><pre>dvc init</pre><p>Sekarang DVC dapat digunakan berdampingan dengan project Git. Jadi untuk menggunakan DVC, pastikan sebelumnya sudah melakukan inisialisasi Git, Kemudian kita melihat bahwa terdapat beberapa file baru, selanjutnya kita lakukan commit ke Git</p><pre>git status # melihat status perubahan<br>git add .dvc .dvcignore #Staging .dvc dan .dvcignore <br>git commit -m &quot;Inisialisasi DVC&quot; # Melakukan commit</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/861/1*itH7vzr1kuIAWhTJBSUHcA.png" /></figure><p>Selanjutnya menambahkan dataset ke DVC tracking,</p><pre>dvc add data/diamonds.csv</pre><p>Terakhir kita melakukan commit untuk menyimpan versi dari dataset ke DVC</p><pre>cat data/.gitignore #ignore diamonds.csv changes<br>git add data<br>git commit -m &quot;Initialization dataset to DVC&quot;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/529/1*CxwtBYZJn6ugwDcJTLUyLA.png" /><figcaption>The output of git commit after add data to staging</figcaption></figure><h4>Tambahan — Basic command DVC</h4><pre>dvc status #Mengecek status terkini terkait DVC-tracking files dan directories<br>dvc add data/diamonds.csv<br>git commit -m &quot;Add raw data&quot;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/628/1*eueCEaieTbS9OVEVe-Bawg.png" /><figcaption>The output of dvc status</figcaption></figure><p>Configurasi remote data (Storing and sharing)</p><pre>mkdir %TEMP%/dvcstore<br>dvc remote add -d myremote %TEMP%\dvcstore</pre><p>Uploading data</p><pre>dvc push <br>git commit data/diamonds.csv.dvc -m &quot;Dataset updates&quot;<br>Retrieving data</pre><pre>dvc pull</pre><h4>Langkah selanjutnya</h4><p>Selain DVC terdapat tools lain yang memiliki fungsi mirip mirip, antara lain:<br>1. <a href="https://neptune.ai/">Neptune</a><br>2.<a href="https://git-lfs.com/"> Git-LFS</a><br>3. <a href="https://lakefs.io/">lakeFS</a><br>Meskipun DVC adalah yang paling populer, di prakteknya kita perlu menyesuaikan tools tools yang sudah digunakan oleh perusahaan. Biasanya, remote data nya biasanya kalau nggak Google Cloud Storage, AWS S3, Azure Blob Storage ataupun cloud yang lain. Jadi harapannya setelah memahami DVC ini, ke depannya tetap terus belajar hal baru untuk menambah knowledge dan level up menjadi data engineer yang jago.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ae25bf7b012c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/himit-pens/kamu-anak-sains-data-kamu-wajib-tahu-data-version-control-ae25bf7b012c">Kamu anak sains data? Kamu wajib tahu data version control</a> was originally published in <a href="https://medium.com/himit-pens">HIMIT PENS</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding decomposition in Machine learning]]></title>
            <link>https://medium.com/himit-pens/understanding-decomposition-in-machine-learning-f357a55a91e2?source=rss----d838c8bb2d71---4</link>
            <guid isPermaLink="false">https://medium.com/p/f357a55a91e2</guid>
            <category><![CDATA[scikit-learn]]></category>
            <category><![CDATA[machine-learning-models]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[decomposition]]></category>
            <category><![CDATA[dimensionality-reduction]]></category>
            <dc:creator><![CDATA[Wahyu ikbal maulana]]></dc:creator>
            <pubDate>Tue, 24 Oct 2023 08:01:40 GMT</pubDate>
            <atom:updated>2023-10-24T08:55:32.220Z</atom:updated>
            <content:encoded><![CDATA[<h3>Understanding Decomposition in Machine learning</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*d4s3qiiwvi0P6870" /><figcaption>Image by Author</figcaption></figure><p>Untuk memecahkan masalah yang kompleks, salah satu pendekatan umum yang efektif adalah dengan menguraikannya menjadi sub masalah yang lebih kecil. Proses ini di dalam computer science, disebut problem <em>decomposition</em>. <a href="https://en.wikipedia.org/wiki/Decomposition_(computer_science)">Decomposition</a> sendiri dalam ilmu komputer adalah proses memecah masalah atau sistem yang kompleks menjadi beberapa bagian yang lebih mudah untuk dipahami, diprogram, dan dianalisis.</p><h3>Decomposition in Machine learning</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lLuqXpBfOhzCe0uB" /><figcaption>Image by Author</figcaption></figure><p>Di dalam <a href="https://christophmolnar.com/books/interpretable-machine-learning/"><em>Interpretable Machine Learning</em></a> terdapat Model Agnostic Method, metode ini bersifat fleksibel yang artinya dapat digunakan di berbagai model. Artinya si machine learning engineer dapat dengan bebas menggunakan model machine learning apa pun ketika metode interpretasi dapat diterapkan ke model. Hal ini merupakan penerapan Functional Decomposition. Decomposition sendiri dalam machine learning adalah metode unsupervised learning yang melibatkan pemecahan masalah yang kompleks menjadi sub-masalah yang lebih kecil. Dalam proses data preprocessing salah satu metodenya ialah mereduksi dimensi dengan memecah training set menjadi sub-training set yang lebih kecil. Dengan ini, kita akan lebih mudah memahami cara kerja masalah dan bagaimana komponen yang berbeda berinteraksi, Menemukan solusi untuk masalah tersebut, Sehingga dapat meningkatkan kinerja model machine learning.</p><h3>Introduction PCA</h3><p>Salah satu metode decomposition yang populer adalah PCA, metode ini mereduksi dimensi dengan mengubah kumpulan variabel besar menjadi variabel yang lebih kecil, namun mempertahankan informasi dari data besar tersebut. Simpelnya, PCA membuat data yang kompleks menjadi lebih sederhana dengan mengambil banyak informasi dan menemukan bagian yang paling penting.</p><p>PCA biasa dipakai pada Desicion Tree, Neural Network. Metode ini bisa dipakai ketika kita ingin visualisasi yang lebih sederhana, mengatasi terlalu banyak feature, hingga mempercepat training sebuah model. Perlu dicatat bahwa metode ini hanya bekerja dalam data yang bersifat linear(<a href="https://www.kdnuggets.com/2022/07/linear-machine-learning-algorithms-overview.html">linear data</a>).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/621/0*zP4Z0OxFFLMKlbWL" /><figcaption>PCA menangkap variabel dominan dari dataset (lower variance dimension). 3D dataset akan kehilangan banyak inforamasi ketika dipreyeksikan ke dalam 2D. Image by Duke Course Notes, Cynthia Rudin</figcaption></figure><p>Algoritma Principal Component Analysis didasarkan dari teori matematika:</p><ul><li>Variance dan Convariance</li><li>Eigen Vectors dan Eigen values</li></ul><p>Dari dasar matematika tersebut kemudian dikembangkan lagi, menjadi fungsi PCA yang kurang lebih seperti ini:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*CgjPYLq7syT0oySa" /><figcaption>Image by Bala Pria C, KDNuggets</figcaption></figure><p>Detailnya bisa anda cek:</p><p><a href="https://scikit-learn.org/stable/modules/classes.html#module-sklearn.decomposition">https://scikit-learn.org/stable/modules/classes.html#module-sklearn.decomposition</a></p><p><a href="https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html">https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html</a></p><h3>Principal Component Analysis (PCA) with Scikit-Learn</h3><p>Setelah kita memahami PCA, selanjunya pengimplementasikan PCA menggunakan Python dengan menggunakan library Scikit Learn. Dataset yang akan kita pakai kali ini yaitu wine dataset.</p><ol><li>Pertama kita import dataset lalu bagi menjadi input dan label</li></ol><pre>#import library<br>import pandas as pd<br>import numpy as np<br>import matplotlib.pyplot as plt<br><br>#Open dataset<br>df = pd.read_csv(&quot;wine.csv&quot;)<br>inputs = df.iloc[:, 1:14]<br>labels = df.iloc[:,0]<br><br>inputs</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/878/1*64YoNdiTxeHc875Grwcfxg.png" /></figure><p>2. Sebelum menerapkan PCA analysis, kita perlu menerapkan formula standarisasi, yaitu mengurangi dengan nilai rata rata pada setiap attribut lalu dibagi dengan variansinya.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jslpKU2Ym0HzVq3jBXwuow.png" /></figure><p>Di scikit learn sudah terdapat library yang secara otomatis mengkalkulasi, yaitu dengan menggunakan <a href="https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html">StandardScaler</a> untuk data preprocessing, selain menghapus rata-rata tetapi juga atribut juga mengukur skala atribut yang sesuai.</p><pre>#Normalize data<br>from sklearn.preprocessing import StandardScaler<br>x1 = StandardScaler().fit_transform(inputs)<br>x1</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/567/1*Z1QzUJAHUwpOPAYdUN6-qQ.png" /></figure><p>3. Untuk menerapkan principal component analysis (PCA). Dapat dilakukan dengan memanggil fungsi PCA from sklearn.decomposition.</p><pre>#Penerapan PCA<br>from sklearn.decomposition import PCA<br>pca = PCA()<br>wine = pca.fit_transform(x1) <br><br>#Rasio dari total variansi tiap principal component<br>print(pca.explained_variance_)<br>print(pca.explained_variance_ratio_)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/592/1*eq2gr7WFKdTnP1EDkq6eDA.png" /></figure><p>Sangat penting untuk mengetahui jumlah variansi yang didapat dari PCA. Dengan memanggil pca.explained tiap input akan mendapatkan rasionya hingga makin mengecil. Hasilnya adalah 1 atribut memiliki variance sebesar 0.361 , yang berarti atribut tersebut menyimpan informasi yang tinggi dan tidak jauh berbeda dari atribut lain.</p><p>3. Selanjutnya adalah kita analisa principal components berdasarkan variansi. Untuk menghasilkan plot, kita gunakan library <a href="https://matplotlib.org/2.0.2/users/pyplot_tutorial.html">matplotlib.pyplot</a>.</p><pre>import matplotlib.pyplot as plt<br>figure = plt.figure()<br>ax = plt.gca()<br>plt.plot(pca.explained_variance_, color=&#39;red&#39;, linestyle=&#39;dotted&#39;)<br>ax.set_title(&quot;Scree plot&quot;)<br>ax.set_xlabel(&quot;Index of principal components&quot;)<br>ax.set_ylabel(&quot;The explained variance&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/352/1*6lWtRsO4JYJKoRD6OGNJag.png" /></figure><p>Pengurutkan principal component melalui pengurutan nilai eigen nya. Jadi setelah kita mendapatkan pasangan vektor eigen dan nilai eigen. Nilai eigen tersebut diurutkan mulai dari yang terbesar ke yang terkecil vektor eigennya juga mengikuti. Setelah mendapatkan principal component, untuk menghitung pesentase dari variansi (informasi) yang dihasilkan dari tiap komponen, kita dapat membagi eigen value dari setiap komponen dengan total semua eigen value.</p><p>Berikut perbandingan grafik Original data → Standarization → Principal Component</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cpmq1u01GzpKNOaZL_rdZQ.png" /></figure><p>Python, Scikit learn, dan PCA adalah hal penting yang perlu dipelajari dalam data science, untuk kamu yang ingin belajar lebih lanjut bisa kunjungi bisa ai academy, disana dipelajari dari awal Reduksi dimensi dengan PCA</p><p>Dataset dan detail Source Code bisa anda lihat disini: <a href="https://github.com/wahyudesu/Scikit_learn">https://github.com/wahyudesu/Scikit_learn</a></p><p>Terima kasih telah meluangkan waktu untuk membaca artikel ini hingga akhir. <em>I tried my best to keep it short and simple keeping in mind to use this code in our daily life</em>. Jika ada yang kurang dipahami atau memiliki pertanyaan lebih lanjut, jangan sungkan untuk menghubungi saya. Terima kasih lagi dan semoga harimu menyenangkan👋</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f357a55a91e2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/himit-pens/understanding-decomposition-in-machine-learning-f357a55a91e2">Understanding decomposition in Machine learning</a> was originally published in <a href="https://medium.com/himit-pens">HIMIT PENS</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Confusion Metrics: Apa Itu Presisi, Recall, dan Bagaimana Menginterpretasikannya.]]></title>
            <link>https://medium.com/himit-pens/mengurai-metrics-apa-itu-presisi-recall-dan-bagaimana-menginterpretasikannya-a15e7f90411e?source=rss----d838c8bb2d71---4</link>
            <guid isPermaLink="false">https://medium.com/p/a15e7f90411e</guid>
            <category><![CDATA[accuracy-and-precision]]></category>
            <category><![CDATA[model-evaluation]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[confusion-matrix]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Wahyu ikbal maulana]]></dc:creator>
            <pubDate>Tue, 05 Sep 2023 03:03:10 GMT</pubDate>
            <atom:updated>2024-03-27T02:04:12.037Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WxKLDsNsw3ki-pEhPVj4dw.png" /><figcaption>Image by Author: <em>Regions of the confusion matrix that help compute precision and recall</em></figcaption></figure><p>Saat kita mulai belajar Machine learning, kita biasanya akan menemui istilah akurasi, presisi, dan kawan-kawannya. Apakah itu? Mari kita bahas!</p><p>Dalam dunia Machine Learning (ML), khususnya pada supervised learning, evaluasi performance dari algoritma ML dapat dilakukan dengan menggunakan acuan <a href="https://www.youtube.com/watch?v=jr_BcU4QlNE">Confusion Matrix</a>. Confusion Matrix merepresentasikan prediksi dan kondisi aktual dari data yang dihasilkan oleh algoritma ML. Dari Confusion Matrix, kita dapat menentukan beberapa nilai yang sangat penting dalam mengevaluasi kinerja suatu model atau algoritma. Nilai-nilai tersebut adalah Accuracy, Precision, Recall, dan F-1 Score. <strong>Accuracy</strong> mengukur seberapa akurat model dalam memprediksi nilai yang benar, dengan parameter:</p><p><strong>Presisi </strong>atau biasa disebut <strong>Precision</strong> adalah metrik yang digunakan untuk mengukur sejauh mana model klasifikasi memberikan prediksi yang benar secara positif. Precision dihitung dengan membagi jumlah prediksi positif yang benar dengan jumlah total prediksi positif.</p><p>Lalu <strong>Recall</strong> adalah metrik yang digunakan untuk mengukur sejauh mana model klasifikasi dapat mengidentifikasi semua kasus positif yang sebenarnya. Recall dihitung dengan membagi jumlah prediksi positif yang benar dengan jumlah total kasus positif yang sebenarnya.</p><p>Ada lagi parameter evaluasi <strong>F1-Score</strong> yang merupakan parameter gabungan antara Precision dan Recall, parameter ini mengukur seberapa baik model dapat memprediksi kelas positif dengan mempertimbangkan keseimbangan antara Precision dan Recall.</p><p>Sebagai contoh, dan untuk lebih mudah memahami kali ini kita akan menggunakan contoh kasus prediksi kelulusan mahasiswa untuk memudahkan memahami Accuracy, Precision, Recall dan F1-Score</p><h4>Case Studies</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nF-KCRw9tk3Zn9cavH02gg.png" /><figcaption>Image by Author : Kasus pada kelulusan mahasiswa</figcaption></figure><p>Dari hasil prediksi di atas, ada kemungkinan 4 kasus yang terjadi:</p><ul><li><strong>True Positive (TP)</strong>: kasus dimana mahasiswa diprediksi (Positif) Lulus, memang benar (True) Lulus. Dalam contoh di atas adalah 4 mahasiswa dengan NIM 01, 02, 04, 05. Jadi nilai TP nya adalah 4.</li><li><strong>True Negative (TN)</strong>: kasus dimana mahasiswa diprediksi tidak (Negatif) Lulus dan sebenarnya mahasiswa tersebut memang (True) tidak lulus. Dalam contoh di atas terjadi pada mahasiswa 06, 09, dan 10. Jadi TN nya adalah 3.</li><li><strong>False Positve (FP)</strong>: kasus dimana mahasiswa yang diprediksi positif Lulus, ternyata tidak lulus. Prediksinya salah (False). Dalam contoh di atas terjadi pada mahasiswa 03 dan 07. Jadi nilai FP adalah 2.</li><li><strong>False Negatif (FN)</strong>: kasus dimana mahasiswa yang diprediksi tidak lulus (Negatif), tetapi ternyata sebenarnya(TRUE) lulus. Dalam contoh kasus di atas terjadi pada mahasiswa 08. jadi FN = 1.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/420/0*9N1SpVWU3myNkZf9" /></figure><p>Akurasi = (4+3)/10 = 7/10 = 0.7 = 70%</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/257/0*bJUar6hDvG5qrny8" /></figure><p>Precision = 4/ (4+2) = 4/8 = 0.5 = 50%</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/237/0*DfIki_MgndlyU02_" /></figure><p>Recall = 4/ (4+1) = 4/5 = 0.8 = 80%</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8gHsOmVRon-LeJnHaIeeEg.png" /></figure><p>F1 Score = 2 * (0.5*0.8) / (0.5+0.8)</p><p>= 0.61</p><p>Interpretasi precision dan recall dapat dilakukan dengan memperhatikan konteks masalah yang sedang diselesaikan. Jika kita lebih memprioritaskan untuk menghindari kesalahan positif (mis. memprediksi sesuatu sebagai positif ketika sebenarnya negatif), maka kita harus fokus pada precision. Namun, jika kita lebih memprioritaskan untuk menghindari kesalahan negatif (mis. gagal mengidentifikasi kasus positif), maka kita harus fokus pada recall.</p><p>Kedua metrik ini sering digunakan bersamaan untuk mengevaluasi performa model klasifikasi. Precision dan recall memiliki hubungan trade-off, di mana meningkatnya nilai salah satu parameter akan menurunkan nilai parameter lainnya. Oleh karena itu, pemilihan parameter evaluasi yang lebih penting tergantung pada kasus penggunaan.</p><p>Terakhir untuk lebih memahami, mari kita pakai satu contoh. Misalkan kita memiliki dataset kelas biner dengan 5 sampel kelas positif dan 5 sampel kelas negatif, dan batas keputusan model ditunjukkan oleh garis hijau pada kasus (A) di bawah ini. SISI BAGIAN KANAN dari garis hijau menggambarkan kelas positif, dan SISI BAGIAN KIRI menggambarkan kelas negatif.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nZf7s2fNHTUS83HY" /><figcaption>Image by author: Dampak kenaikan Recall &amp; Presisi</figcaption></figure><p>Sekarang, decision boundary telah digeser untuk meningkatkan precision hingga 100% (dengan recall 50%) atau dengan meningkatkan recall hingga 100% (tetapi precisionnya 50%).</p><p>Untuk kamu yang berminat dibidang machine learning, tentunya masih banyak yang perlu dipelajari. Tidak perlu bingung, nanti akan kita pelajari satu per satu. Terima kasih sudah membaca hingga akhir semoga artikel ini bermanfaat. ^_^</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a15e7f90411e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/himit-pens/mengurai-metrics-apa-itu-presisi-recall-dan-bagaimana-menginterpretasikannya-a15e7f90411e">Confusion Metrics: Apa Itu Presisi, Recall, dan Bagaimana Menginterpretasikannya.</a> was originally published in <a href="https://medium.com/himit-pens">HIMIT PENS</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>