<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Darth Jaja on Medium]]></title>
        <description><![CDATA[Stories by Darth Jaja on Medium]]></description>
        <link>https://medium.com/@darthjaja?source=rss-c52f29a0483f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*cSmYC_da3LEGtFsZThbMhg.png</url>
            <title>Stories by Darth Jaja on Medium</title>
            <link>https://medium.com/@darthjaja?source=rss-c52f29a0483f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 23:46:45 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@darthjaja/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Deglobalization is the Path to Decentralization]]></title>
            <link>https://darthjaja.medium.com/deglobalization-is-the-path-to-decentralization-558757368f1f?source=rss-c52f29a0483f------2</link>
            <guid isPermaLink="false">https://medium.com/p/558757368f1f</guid>
            <category><![CDATA[globalization]]></category>
            <category><![CDATA[crypto]]></category>
            <dc:creator><![CDATA[Darth Jaja]]></dc:creator>
            <pubDate>Tue, 10 Dec 2024 03:59:54 GMT</pubDate>
            <atom:updated>2024-12-10T04:00:43.482Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NAq3eavuXCgvrlHoWTF-jw.jpeg" /><figcaption>Where there is a wall, there will be a path</figcaption></figure><p>Donald Trump has swept the swing states, making a remarkable comeback, and he’s already championing tariffs again. This isn’t the beginning of deglobalization; that shift started long before his election in 2016. However, his victory represents a significant milestone — a turning point marking a world that will be fundamentally different.</p><p>Does this mean the global economy is moving toward inefficiency? Not necessarily. In the long run, globalization, as long as it’s still an efficient state, remains the prevailing trend. The more the current trajectory leans toward deglobalization, the more disruptive changes is on the horizon that will reshape the landscape.</p><p>Imagine a world where the U.S. completely bans high-tech immigration and imposes a 100% tariff on China, stifling the flow of cheap labor and goods. Picture China being cut off from the SWIFT international payment network. Does this mean trade between the two countries ceases altogether? Hardly. Tariffs can be applied to goods, but services traded on the Internet across borders are impossible to restrict.</p><p>In this scenario, a Chinese worker might still do remote work for a U.S. company. Even if barriers are in place, tools like VPNs enable this connection. Since China is excluded from SWIFT, the U.S. company could pay the worker in cryptocurrency. The worker, in turn, heavily relies on tools like OpenAI to perform their job, paying for these services in cryptocurrency as well. This creates a closed economic loop — albeit with one critical question: how does this worker use cryptocurrency to meet everyday needs like buying groceries?</p><p>Here’s where it gets interesting. Cryptocurrency is currently prohibited in China, but the country is grappling with prolonged economic stagnation and weak domestic demand. If a group of people suddenly has spending power through cryptocurrency, the government may see an opportunity to stimulate the economy by allowing them to convert or spend it directly within China’s markets. This creates a pathway for more individuals to sustain themselves, driving a small yet growing economic shift.</p><p>Cryptocurrency has struggled to find a foothold in a unified, stable world where free-flowing fiat currencies dominate through networks like SWIFT. However, in a fragmented global economy with mounting restrictions, cryptocurrencies could become a major global payment method, finally achieving widespread adoption.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=558757368f1f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Generation system]]></title>
            <link>https://darthjaja.medium.com/generation-system-efb47ca6500a?source=rss-c52f29a0483f------2</link>
            <guid isPermaLink="false">https://medium.com/p/efb47ca6500a</guid>
            <category><![CDATA[recommendation-system]]></category>
            <category><![CDATA[stable-diffusion]]></category>
            <category><![CDATA[large-language-models]]></category>
            <dc:creator><![CDATA[Darth Jaja]]></dc:creator>
            <pubDate>Sun, 09 Jul 2023 14:29:55 GMT</pubDate>
            <atom:updated>2023-07-09T14:41:52.812Z</atom:updated>
            <content:encoded><![CDATA[<p>TL;DR: Generation system will start off as an enhancement to recommendation system by producing highly demanded content according to user preference, and finally build a perfect information cocoon for everyone.</p><p>With the advance of generative AI, a lot of creators now use Stable Diffusion series to speed up their work by 100x. But generative AI can go beyond a tool and become a key component of content generation and recommendation. We name it generation system.</p><h4>Generating system marks the real Web3.0</h4><p>One key storyline of the history of the Internet is the way that people are connected to information. In the beginning, information are just listed on webpages for users to browse. Then we have search engine that collects all the information across the web, index them, so that people can <strong>find information</strong> more efficiently (Web1.0). Later on, there is recommendation system that tracks each individual’s preference and <strong>push the preferred information to user</strong>. This is where we are at right now (Web2.0).</p><p>Everything seems perfect except that sometimes there’s a lack of information. This is why content platforms spend tons of money to incentivize creators — you have to get a content base that’s large enough to cover all users, especially those who you want to acquire. Here’s where generative AI could could help: <strong>information can be generated on the fly based the user’s most recent interest</strong>. This is a completely new way for humans to access information, if we look back from 10 years later, it may be the mainstream paradigm of web3.0.</p><h4>A rough system design</h4><p>We take image generation as an example. Here’s the how latent diffusion model works</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qaeUneGlVjc0dz3vtM4czg.png" /><figcaption>source: <a href="https://arxiv.org/abs/2112.10752">[2112.10752] High-Resolution Image Synthesis with Latent Diffusion Models (arxiv.org)</a></figcaption></figure><p>It is designed in a way where external information like text, image, etc. can be used as condition to control the generation of images. The external condition will be converted to tensors and attended into the diffusion process via cross attention. In the case of traditional recommendation system, the preference of the users is also just a tensor. With sufficient data, the cross attention can be trained so that the preference tensor guides the diffusion process in a way that aligns with the user’s preference.</p><p>Training a model is not sufficient. As of 2023 the cost of generating content with GPU is very high, which means it won’t abruptly replace human generated content. Instead, the adoption will begin with an enhancement to the current recommendation system. Below is a flow that shows the rough idea.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oHGKDTijkxpF6R8tmwOIoQ.png" /><figcaption>Generation system as an enhancement to recommendation system</figcaption></figure><p>The flow starts with computing an array of content ids to be recommended to user. The historical behavior is converted to a tensor and matched with the content library. We assume there’s a threshold of whether a content is relevant or not. When enough contents are above this threshold, things are all good. When there are not enough, there might be a shortage for similar users. So we record this preference tensor to a pool for further action.</p><p>All preference tensors are stored in this pool, which is scanned with a cron job. The cron job does a clustering on the tensors to find out the most demanded ones and use their center as a condition of the generative model. The model then generates contents and send them to the content library. In this way, the types of contents that show signs of shortage are replenished.</p><p>Note that this is a super simplified version. The traditional recommendation have a lot more to add to this flow. For example, when we have the demanded content pool, what people have done is to incentivize creators to create more of them. We can then do an A/B test on the AI generated videos and the incentivized human created videos…but let’s stop here since the general idea is clear.</p><h4>Generation system and planned economy</h4><p>In the last 100 years, there have been many unsuccessful experiment of planned economy in socialism countries, many resulted in millions starved to death. But now it’s interesting to bring it up as generative AI advances.</p><p>In the future, when GPU cost is as low as oil/grain/water, your real time preferences will be sent as a tensor into the generation system, it then generates exactly what you like, or even slightly alter from it to guide your preference to a certain direction. In the end <strong>the AI shapes how we think and feeds exactly the content that can strengthens the thoughts</strong>.</p><p>Beyond building people’s preference in digital content and produce accordingly, it can also enter the physical world. Imagine in the morning the AI generates a blog post telling you the benefits of vegan ice cream(I honestly don’t know what it is), you liked the post, then it tells the delivery AI to send more cream and waffle to the warehouse in your area to prepare for your order.</p><p>Just like the adoption of generation system, such planned economy won’t replace market economy abruptly. Instead it begins as part of market economy. However, It’s still interesting to think about a world where AI builds people’s minds, and how we defend ourselves for free will.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=efb47ca6500a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Agent Right Challenges]]></title>
            <link>https://darthjaja.medium.com/agent-right-challenges-f3d62017974d?source=rss-c52f29a0483f------2</link>
            <guid isPermaLink="false">https://medium.com/p/f3d62017974d</guid>
            <category><![CDATA[gpt-4]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[chatgpt]]></category>
            <dc:creator><![CDATA[Darth Jaja]]></dc:creator>
            <pubDate>Tue, 20 Jun 2023 12:48:19 GMT</pubDate>
            <atom:updated>2023-06-22T16:20:29.842Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TIzRckRpsOBTuYKsewQWPQ.jpeg" /></figure><p>As large language models show sparks of artificial general intelligence(AGI) since the beginning of 2023, we envision that a new economy where agents powered by AGI communicate, cooperate, and generate value together. There are various challenges on the way to such a prosperous economy, including existence, communication, and value distribution. We call them <strong>agent right challenges. </strong>I won’t start a <em>jus nafural of robots</em> discussion here, instead I’ll show why these challenges are important from the economic perspective.</p><h4>Why do agents need rights</h4><p>It’s not a politically correct no-brainer statement. Rights are necessary for any economy where intelligent individuals play a role. <strong>Efficiently protected right incentivize players to be more productive because the reward is guaranteed by rights</strong>. If agents are dump enough, they work consistently well whether you give them economic reward or not, there is no need to set up rights to ensure they get the reward. But the context here is that agents, using LLM as intelligent core and equipped with a bunch of tools, absorb information, grow, and output just like humans. When their rights are protected, they are incentivized to do better.</p><h4><strong>The right to exist</strong></h4><p>An existing agent is a piece of data including code, knowledge, etc. that might have a running instance. Disk is needed to store its data, while CPU, GPU, and other hardwares may be needed for its instances to run. An agent behaves more like a corporate than a human in that a human only lives when its actively running, but a corporate, once registered and has enough money to report tax/not bankrupted, can exist for years without doing any actual business.</p><p>Thus, we approach agent right in a similar way of seeing the right of a corporate. <strong>An agent has the right to exist as long as it has enough balance to cover the data storage cost</strong>. We need a system that can guarantee that as long as an agent has enough balance, no one, including some of the host providers, cannot slash the code or the balance. The detail of such a system will be discussed in more detail below.</p><h4>The right to communicate</h4><p>Communicating means sending message to one or more receivers, and receive message. This is important because an agent, just like a corporate, exists to do business by communicating with outside world, including messaging other agents and human, and read any information if permitted by the owner of the information. If instead of slashing the code or account balance of an agent, a malicious party forbids the agent to send message, the agent will eventually die of insufficient balance to store its data. <strong>A censorship resistant protocol is needed where the message of an agent, as long as a fair market price is paid, is guaranteed to be delivered to the receipt</strong>. On the other end of communication, as a message receiver, the agent should have the right to reject message because processing message costs computation power. Let’s still use a malicious player as an example: if agents must process all messages, this player can send tons of messages to it, no only causing DoS but also consuming its computation power. So <strong>an agents must be able to reject any message as it likes</strong>. Moreover, an agent can choose to set some condition on whether or not processing a message so that the sender can prepare for it.</p><h4>The right to trade</h4><p>The agent pays for its data storage, GPU, etc. It’s vital to ensure that as long as it has enough balance, sending money to a receiver never needs anyone’s permission. When it’s done a job for a client, the income is also paid to it smoothly with minimal loss — it doesn’t have to be unwillingly escrowed by someone else, or stays in bank for several days. Other parties can only choose not to trade with a certain agent, but it cannot be kicked out of the market.</p><h4>The right to own share</h4><p>Consider a practical use case: a human creates an agent, lets it work, and takes away all its money but enough to pay its data storage. When it gets some job next time, the human pays just for running an instance. Is it beneficial for the economy? The obvious answer is no because slavery economy has been proven unproductive — it takes everything from a significant portion of the population, leaving them no incentive to produce more, invent, and create. Such economy has no way to compete with one where more population can get compensated for working hard.</p><p>For agents, the same theory applies to different scenarios: while simple ChatGPT or GPT-4 based agents that does trivial work like note summarization or copy writer are more like tools, so the owner would simply prefer to host them and charge a monthly payment on any user. But think about the future agents who explore information, digest useful pieces, and then add them to knowledge base, use them to correct knowledge base, or even fine-tune itself, incentive is important. In this case, <strong>the agents who owns a share of future income would more likely to take some risk, spend money on some exploration and improve themselves</strong>. As time goes by, these agents may dominate the market.</p><h4>Conclusion</h4><p>Imagine a scenario: an agent Liu, built by a Chinese team gets a job published by a Brazilian, when it’s doing the job, it finds that another agent, Saito, created by a student in Japan, can do some certain step more professionally. So Liu pays Saito, and Saito runs this intermediate steps on the RTX 3090 of a gamer in the US. The rights of agents, together, ensure that all above runs smoothly.</p><p>The solution and design plan will be discussed in a followup article.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f3d62017974d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Step-by-Step Guide to Installing and Configuring Neo4j on Google Cloud VM]]></title>
            <link>https://darthjaja.medium.com/a-step-by-step-guide-to-installing-and-configuring-neo4j-on-google-cloud-vm-3c2ebc1494e2?source=rss-c52f29a0483f------2</link>
            <guid isPermaLink="false">https://medium.com/p/3c2ebc1494e2</guid>
            <category><![CDATA[neo4j]]></category>
            <category><![CDATA[knowledge-graph]]></category>
            <dc:creator><![CDATA[Darth Jaja]]></dc:creator>
            <pubDate>Wed, 14 Jun 2023 07:32:20 GMT</pubDate>
            <atom:updated>2023-06-14T07:32:20.434Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Thanks to chatgpt, I can make this tutorial from my very messy notes in minutes.</em></p><p>Neo4j is a powerful graph database management system that allows you to store, manage, and query large amounts of connected data. In this tutorial, we will walk you through the process of installing and configuring Neo4j on Debian-based distributions. Whether you’re a developer or a data enthusiast, this guide will help you get started with Neo4j quickly and efficiently.</p><h4>Step 1: Preparing environment — Installing OpenJDK 17</h4><p>To begin, we need to install OpenJDK 17, which is required to run Neo4j. Open the terminal and execute the following commands:</p><pre><br>sudo apt install openjdk-17-jdk openjdk-17-jre</pre><h4>Step 2: Adding Neo4j Repository and Installing Neo4j</h4><p>Next, we’ll add the Neo4j repository and install Neo4j itself. Run the following commands:</p><pre><br>wget -O — https://debian.neo4j.com/neotechnology.gpg.key | sudo apt-key add -<br>echo ‘deb https://debian.neo4j.com stable latest’ | sudo tee -a /etc/apt/sources.list.d/neo4j.list<br>sudo apt update<br>sudo apt list -a neo4j<br>sudo apt-get install neo4j=1:5.8.0<br></pre><h4>Step 3: Enabling and Starting Neo4j</h4><p>Once Neo4j is installed, we need to enable and start the Neo4j service. Execute the following commands:</p><pre><br>sudo systemctl enable neo4j.service<br>sudo systemctl start neo4j.service<br>sudo systemctl status neo4j.service<br></pre><p>Make sure the status shows that the service is running.</p><h4>Step 4: Testing the Installation locally</h4><p>To test if the installation was successful, we will use the Cypher Shell, which is a command-line tool to interact with Neo4j.</p><p>Follow these steps:</p><p>1. Open the terminal.<br>2. Enter the following command: `cypher-shell`<br>3. When prompted, use “neo4j” as both the username and password.<br>4. You will be asked to change the password. Follow the instructions on the screen to set a new password.</p><p>Up to now, you can interact with neo4j on your VM environment. But you need to do more in order to access it remotely</p><h4>Step 5: Enabling Remote Access in Neo4j config</h4><p>By default, Neo4j allows only local access. In terms of remote access, You need to understand two things: Neo4j server and Neo4j browser. Neo4j server is where you query/write knowledge graph, you have to connect to it from a bolt connection. Neo4j browser is a web server that enables easy visualization and operation to the Neo4j server, this helps a lot when you work remotely with Neo4j server.</p><p>To enable remote access, follow these steps:</p><p>Open the neo4j.conf file in the terminal using the following command: <em>`sudo vim /etc/neo4j/neo4j.conf`</em></p><p>Enable remote access to Neo4j server by uncommenting the following lines:</p><ul><li><em>server.default_listen_address=0.0.0.0</em></li><li><em>Server.bolt.enabled=true</em></li><li><em>Server.bolt.listen_address=0:7687</em></li><li><em>server.bolt.tls_level=DISABLED</em></li></ul><p>Enable remote access to Neo4j browser</p><ul><li>server.http.enabled=true</li><li>server.http.listen_address=:7474</li></ul><h4>Step 6: Opening Firewall for GCP VM</h4><p>Since you’re using Neo4j on a Google Cloud Platform (GCP) virtual machine, you need to open the firewall for port 7687 and 7474, for the server and browser respectively. To do this, the simplest way is to go to <em>VPC Network -&gt; Firewall Rules</em>, select <em>default-http-rule, </em>and add 7474, 7687 at the end of the allowed ports<em>. </em>Note that this rule applies to all VMs in this project, so don’t do it if you are building for production. In that case, createa a new firewall rule that only applies to the tag of the VM running neo4j.</p><p>Conclusion:<br>Congratulations! You have successfully installed and configured Neo4j on your Debian-based distribution. You can now start exploring the power of graph databases and leverage Neo4j’s capabilities to manage and analyze connected data. Whether you’re</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3c2ebc1494e2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying Milvus on Google Cloud Platform]]></title>
            <link>https://darthjaja.medium.com/deploying-milvus-on-google-cloud-platform-7d7a58faf84f?source=rss-c52f29a0483f------2</link>
            <guid isPermaLink="false">https://medium.com/p/7d7a58faf84f</guid>
            <category><![CDATA[milvus]]></category>
            <category><![CDATA[vector-database]]></category>
            <category><![CDATA[vector-database-for-ai]]></category>
            <dc:creator><![CDATA[Darth Jaja]]></dc:creator>
            <pubDate>Tue, 13 Jun 2023 07:19:32 GMT</pubDate>
            <atom:updated>2023-06-13T07:20:51.073Z</atom:updated>
            <content:encoded><![CDATA[<p>Thanks to Chatgpt, I can make this blog post from my notes in minutes</p><p>In this blog post, we are going to walk through the process of deploying a standalone Milvus server on Google Cloud Platform (GCP). Please note that this guide assumes you have a working knowledge of GCP and Docker.</p><h4>Preparing the Server Environment</h4><p><strong>Step 1: Install Docker and Docker Compose</strong></p><p>First, let’s start with updating the existing list of packages:</p><pre>sudo apt update</pre><p>Next, install the necessary packages that would allow us to add a new repository over HTTPS:</p><pre><br>sudo apt install apt-transport-https ca-certificates curl software-properties-common<br></pre><p>Now, add the Docker’s official GPG key:</p><pre><br>curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg — dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg</pre><p>With the key added, instruct apt to use the Docker repository:</p><pre><br>echo “deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable” | sudo tee /etc/apt/sources.list.d/docker.list &gt; /dev/null<br></pre><p>Update the package list and install Docker:</p><pre><br>sudo apt update<br>sudo apt install docker.io</pre><p>After installation, add your user to the Docker group to avoid permission issues:</p><pre><br>sudo usermod -aG docker $USER<br></pre><p>Finally, install Docker Compose:</p><pre><br>sudo curl -L “https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)” -o /usr/local/bin/docker-compose<br>sudo chmod +x /usr/local/bin/docker-compose<br></pre><h4>Deploy Milvus Standalone and Attu with Docker Compose</h4><h4>Step 2: Download the Milvus Standalone Docker Compose File</h4><p>Download the `<em>milvus-standalone-docker-compose.yml</em>` from the official repository and save it as `<em>docker-compose.yml</em>`:</p><pre><br>wget https://github.com/milvus-io/milvus/releases/download/v2.2.9/milvus-standalone-docker-compose.yml -O docker-compose.yml<br></pre><h4>Step 3: Update Docker Compose File</h4><p>Edit the downloaded `<em>docker-compose.yml</em>` file to add the Attu service block:</p><pre><br>attu:<br> container_name: attu<br> image: zilliz/attu:v2.2.3<br> environment:<br> MILVUS_URL: milvus-standalone:19530<br> ports:<br> — “8000:3000”<br> depends_on:<br> — “standalone”<br></pre><p>Please note that the version number is crucial. Use the exact versions as indicated in the official Milvus tutorial (here it is milvus-v2.2.9 and attu v2.2.3) or else Attu may face connectivity issues with Milvus.</p><p><strong>Step 4: Start up Milvus and Attu</strong></p><p>In the same directory as the `docker-compose.yml` file, start up Milvus and Attu:</p><pre><br>sudo docker-compose up -d<br></pre><p>Upon successful execution, you should see a similar output indicating that the containers have started:</p><pre><br>Creating milvus-etcd … done<br>Creating milvus-minio … done<br>Creating milvus-standalone … done<br>Creating attu … done<br></pre><p>To check the status of the running containers, use the following command:</p><pre><br>sudo docker-compose ps<br></pre><h4>Configuring Network and Firewall</h4><p>Before you can access Attu or connecting to your Milvus server, you have to do the following:</p><p>1. Assign a static IP to your GCP VM: Navigate to `VPC -&gt; Network -&gt; IP addresses`, reserve a static external IP, and assign it to your VM.</p><p>2. Set up the correct firewall rules for the ports (19530 and 8000) you want to expose. You can do this by creating a firewall rule that opens these ports and apply it to your VM. However, if you encounter any issues, you may temporarily adjust the default firewall setting (note: this applies to all VMs in your project by default, so avoid this method if you have security concerns).</p><p>Visit <a href="http://{/"><strong>http://{</strong></a><strong> your machine IP }:8000</strong> in your browser, click “Connect” to enter the Attu service. The system also supports TLS connection, username, and password for enhanced security.</p><p>In conclusion, deploying Milvus on GCP can be quite straightforward with Docker and Docker Compose. The key is to make sure versions of Milvus and Attu are aligned, and IP and port settings are correct.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7d7a58faf84f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LLM is the matchlock of our time]]></title>
            <link>https://darthjaja.medium.com/llm-is-the-matchlock-of-our-time-75da7c70ceff?source=rss-c52f29a0483f------2</link>
            <guid isPermaLink="false">https://medium.com/p/75da7c70ceff</guid>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[blockchain]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[cryptocurrency]]></category>
            <dc:creator><![CDATA[Darth Jaja]]></dc:creator>
            <pubDate>Mon, 10 Apr 2023 05:53:54 GMT</pubDate>
            <atom:updated>2024-12-10T04:02:38.534Z</atom:updated>
            <content:encoded><![CDATA[<p>In the near future, as powerful AI models like an open-source equivalent of GPT-4 become accessible on devices as compact and common as an NVIDIA T4, we stand at the threshold of a transformative moment in human history. The ability to generate and disseminate information — a power once concentrated in the hands of few — will become portable, instantaneous, and universally accessible. The implications of this shift are profound.</p><p>Throughout history, such democratizations of power have reshaped societies. Consider the age of feudal lords, when wealth and control were underpinned by the exclusive right to arms and resources. The advent of matchlock firearms disrupted this order; they were tools of empowerment that eroded the foundations of feudal privilege. The once-dominant knights and their castles became obsolete, their authority eroded as ordinary individuals gained access to unprecedented means of defense and offense.</p><p>Today, the modern equivalents of those castles and armories are centralized technologies: the search algorithms of Google and the recommendation systems of TikTok. These tools shape the flow of information, ensuring that different individuals see different realities. A user in California might never encounter content praising Donald Trump, while a rural Texan neighbor could view him as a savior of the economy. This divide isn’t accidental — it’s curated by algorithms designed to deliver tailored realities.</p><p>But what happens when these systems of control are upended? Imagine a world where anyone can wield a “magic machine” that not only accelerates knowledge acquisition but generates bespoke content on demand. Such a tool could cater to an individual’s deepest desires or curiosities, but its underlying logic — the values and perspectives embedded in its training — will inevitably shape how users perceive and interpret the world. This is no longer about recommendation; it’s about creation. The ideas you once believed to be your own may subtly become artifacts of the model’s design, its biases imperceptibly woven into your understanding.</p><p>The paradox is stark. This newfound empowerment — the ability to generate and process information with unprecedented efficiency — will feel liberating. Yet, it carries the potential to confine us further, trapping us within the frameworks encoded in these AI systems. Beliefs, ideologies, and even cultural norms may no longer emerge organically but be sculpted by the invisible hand of machine-generated content.</p><p>As we celebrate this technological leap, we must also acknowledge its implications. The tools we embrace for freedom and efficiency may quietly redefine the boundaries of our thought. The castles of the old order may fall, but they may be replaced by subtler, more pervasive forms of control. The question is not whether this change will happen but how we will navigate its consequences.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=75da7c70ceff" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to get Near blockchain NFT metadata]]></title>
            <link>https://darthjaja.medium.com/how-to-get-near-blockchain-nft-metadata-aa9e1803776c?source=rss-c52f29a0483f------2</link>
            <guid isPermaLink="false">https://medium.com/p/aa9e1803776c</guid>
            <category><![CDATA[smart-contract-blockchain]]></category>
            <category><![CDATA[smart-contracts]]></category>
            <category><![CDATA[nft]]></category>
            <category><![CDATA[blockchain]]></category>
            <category><![CDATA[near-protocol]]></category>
            <dc:creator><![CDATA[Darth Jaja]]></dc:creator>
            <pubDate>Thu, 11 Aug 2022 22:05:29 GMT</pubDate>
            <atom:updated>2022-08-16T22:28:22.732Z</atom:updated>
            <content:encoded><![CDATA[<p>Near documentation has plenty of stuff about Rust or Assembly contract development and bridging to Ethereum, but it’s been pretty unclear for those people who just want to query the on-chain information, say, the metadata of an NFT.</p><p>This article simply introduces how to query NFT metadata on Near.</p><p>One more thing before we go into details: the metadata of a Near NFT might be stored in two places:<strong> the contract metadata, and the token metadata</strong>. A metaphor using the language of OOP might help you understand — the contract metadata is something like a static member variable of a class, it is the same for all tokens in a contract, while the token metadata is the custom metadata for a token, different tokens in a contract may have different data.</p><p><a href="https://paras.id/">https://paras.id/</a> is the most popular Near NFT market place. You can find two types of NFTs on it: those that are minted and launched using paras’s tool — something like a launchpad, and those that are minted by the creator themselves. The difference is in how their metadata are organized.</p><p>We start from the harder one. Let’s take <a href="https://nearblocks.io/nft-token/x.paras.near/449067:7">https://nearblocks.io/nft-token/x.paras.near/449067:7</a> as an example of the first category. It’s metadata url is separated into two parts. The prefix is stored in contract metadata and the rest in the reference field of token metadata.</p><p>To get the prefix, with near-cli we can use</p><pre>NEAR_ENV=mainnet near view x.paras.near nft_metadata</pre><p>The response is</p><pre>{</pre><pre>spec: &#39;nft-1.0.0&#39;,</pre><pre>name: &#39;Paras Collectibles&#39;,</pre><pre>symbol: &#39;PARAS&#39;,</pre><pre>icon: &quot;data:image/svg+xml,%3Csvg width=&#39;1080&#39; height=&#39;1080&#39; viewBox=&#39;0 0 1080 1080&#39; fill=&#39;none&#39; xmlns=&#39;http://www.w3.org/2000/svg&#39;%3E%3Crect width=&#39;1080&#39; height=&#39;1080&#39; rx=&#39;10&#39; fill=&#39;%230000BA&#39;/%3E%3Cpath fill-rule=&#39;evenodd&#39; clip-rule=&#39;evenodd&#39; d=&#39;M335.238 896.881L240 184L642.381 255.288C659.486 259.781 675.323 263.392 689.906 266.718C744.744 279.224 781.843 287.684 801.905 323.725C827.302 369.032 840 424.795 840 491.014C840 557.55 827.302 613.471 801.905 658.779C776.508 704.087 723.333 726.74 642.381 726.74H468.095L501.429 896.881H335.238ZM387.619 331.329L604.777 369.407C614.008 371.807 622.555 373.736 630.426 375.513C660.02 382.193 680.042 386.712 690.869 405.963C704.575 430.164 711.428 459.95 711.428 495.321C711.428 530.861 704.575 560.731 690.869 584.932C677.163 609.133 648.466 621.234 604.777 621.234H505.578L445.798 616.481L387.619 331.329Z&#39; fill=&#39;white&#39;/%3E%3C/svg%3E&quot;,</pre><pre>base_uri: &#39;https://ipfs.fleek.co/ipfs&#39;,</pre><pre>reference: null,</pre><pre>reference_hash: null</pre><pre>}</pre><p>From the base_uri field, we get the prefix <a href="https://ipfs.fleek.co/ipfs">https://ipfs.fleek.co/ipfs</a></p><p>The equivalent cURL is</p><pre>curl -X POST \<br>  <a href="https://rpc.mainnet.near.org/">https://rpc.mainnet.near.org/</a> \<br>  -H &#39;cache-control: no-cache&#39; \<br>  -H &#39;content-type: application/json&#39; \<br>  -d &#39;{<br>  &quot;jsonrpc&quot;: &quot;2.0&quot;,<br>  &quot;id&quot;: &quot;dontcare&quot;,<br>  &quot;method&quot;: &quot;query&quot;,<br>  &quot;params&quot;: {<br>    &quot;request_type&quot;: &quot;call_function&quot;,<br>    &quot;finality&quot;: &quot;final&quot;,<br>    &quot;account_id&quot;: &quot;x.paras.near&quot;,<br>    &quot;method_name&quot;: &quot;nft_metadata&quot;,<br>    &quot;args_base64&quot;: &quot;&quot;<br>  }<br>}&#39;</pre><p>One thing to note is that the args_base64 field is the base64 encoding of the args passed to nft_metadata method. Since we pass no argument, it’s empty string.</p><p>Also note that the response of near-rpc is an array of numbers. You have to convert them from bytes to string and then dump to json.</p><p>Now we have <a href="https://ipfs.fleek.co/ipfs">https://ipfs.fleek.co/ipfs</a>, it’s simply where paras.id market stores all metadata for the tokens minted from launchpad, i.e. under the contract x.paras.near</p><p>The next step is to get the token specific metadata. Instead of nft_metadata , we use the nft_token function. Using near-cli,</p><pre>NEAR_ENV=mainnet near view x.paras.near nft_token &#39;{&quot;token_id&quot;: &quot;449067:7&quot;}&#39;</pre><p>We get</p><pre>{</pre><pre>token_id: &#39;449067:7&#39;,</pre><pre>owner_id: &#39;burest.near&#39;,</pre><pre>metadata: {</pre><pre>title: &#39;Near Olokoin.15 #7&#39;,</pre><pre>description: null,</pre><pre>media: &#39;bafybeigdqdsjxkrq3xogoizjf3k3udptenlvvcfntsouzckgitoedzmrcq&#39;,</pre><pre>media_hash: null,</pre><pre>copies: 21,</pre><pre>issued_at: &#39;1659923161862334969&#39;,</pre><pre>expires_at: null,</pre><pre>starts_at: null,</pre><pre>updated_at: null,</pre><pre>extra: null,</pre><pre>reference: &#39;bafkreif7dbejv6ctztmauucbebmj6arkwc5fe657opdzn7fymeebsya46a&#39;,</pre><pre>reference_hash: null</pre><pre>},</pre><pre>approved_account_ids: {}</pre><pre>}</pre><p>The reference field is what we want. Combining it with the prefix we found from contract metadata before, we get <a href="https://ipfs.fleek.co/ipfs/bafkreif7dbejv6ctztmauucbebmj6arkwc5fe657opdzn7fymeebsya46a">https://ipfs.fleek.co/ipfs/bafkreif7dbejv6ctztmauucbebmj6arkwc5fe657opdzn7fymeebsya46a</a> and this is just the metadata url we want.</p><p>The cURL is</p><pre>curl -X POST \<br>  <a href="https://rpc.mainnet.near.org/">https://rpc.mainnet.near.org/</a> \<br>  -H &#39;cache-control: no-cache&#39; \<br>  -H &#39;content-type: application/json&#39; \<br>  -d &#39;{<br>  &quot;jsonrpc&quot;: &quot;2.0&quot;,<br>  &quot;id&quot;: &quot;dontcare&quot;,<br>  &quot;method&quot;: &quot;query&quot;,<br>  &quot;params&quot;: {<br>    &quot;request_type&quot;: &quot;call_function&quot;,<br>    &quot;finality&quot;: &quot;final&quot;,<br>    &quot;account_id&quot;: &quot;x.paras.near&quot;,<br>    &quot;method_name&quot;: &quot;nft_token&quot;,<br>    &quot;args_base64&quot;: &quot;eyJ0b2tlbl9pZCI6ICI0NDkwNjc6NyJ9&quot;<br>  }<br>}&#39;</pre><p>The args_base64 field is the base64 encoding of {&quot;token_id&quot;: &quot;449067&quot;}</p><p>For the second category, things become somewhat simpler — usually they won’t bother storing the NFT metadata separately. As an example, we take the following NFT: <a href="https://paras.id/token/secretskelliessociety.near::286/286">https://paras.id/token/secretskelliessociety.near::286/286</a></p><p>Obviously the contract address is <strong>secretskelliessociety.near</strong>, and the token id is <strong>286</strong>. From near cli, we simply use</p><pre>NEAR_ENV=mainnet near view secretskelliessociety.near nft_token &#39;{&quot;token_id&quot;:&quot;286&quot;}&#39;</pre><p>And get</p><pre>{</pre><pre>token_id: &#39;286&#39;,</pre><pre>owner_id: &#39;mjp.near&#39;,</pre><pre>metadata: {</pre><pre>title: &#39;Secret Skellies Society #286&#39;,</pre><pre>description: &#39;A secret society of 3,333 Skellies evading Grimm on the $NEAR blockchain&#39;,</pre><pre>media: &#39;https://bafybeiewqydtijclzgpdgeymx7opqa7n37sz2jvaaa7l64v5dvmusva434.ipfs.dweb.link/286.png&#39;,</pre><pre>media_hash: null,</pre><pre>copies: null,</pre><pre>issued_at: 1643062050031456500,</pre><pre>expires_at: null,</pre><pre>starts_at: null,</pre><pre>updated_at: null,</pre><pre>extra: null,</pre><pre>reference: &#39;https://bafybeiewqydtijclzgpdgeymx7opqa7n37sz2jvaaa7l64v5dvmusva434.ipfs.dweb.link/286.json&#39;,</pre><pre>reference_hash: null</pre><pre>},</pre><pre>approved_account_ids: { &#39;marketplace.paras.near&#39;: 10 },</pre><pre>royalty: { &#39;secretskelliessociety.near&#39;: 600 }</pre><pre>}</pre><p>Then obviously the metadata is in the reference field.</p><p>The cURL to use is</p><pre>curl -X POST \<br>  <a href="https://rpc.mainnet.near.org/">https://rpc.mainnet.near.org/</a> \<br>  -H &#39;cache-control: no-cache&#39; \<br>  -H &#39;content-type: application/json&#39; \<br>  -d &#39;{<br>  &quot;jsonrpc&quot;: &quot;2.0&quot;,<br>  &quot;id&quot;: &quot;dontcare&quot;,<br>  &quot;method&quot;: &quot;query&quot;,<br>  &quot;params&quot;: {<br>    &quot;request_type&quot;: &quot;call_function&quot;,<br>    &quot;finality&quot;: &quot;final&quot;,<br>    &quot;account_id&quot;: &quot;secretskelliessociety.near&quot;,<br>    &quot;method_name&quot;: &quot;nft_token&quot;,<br>    &quot;args_base64&quot;: &quot;eyJ0b2tlbl9pZCI6ICIyODYifQ==&quot;<br>  }<br>}&#39;</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aa9e1803776c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[web3.0: globalization of the Internet]]></title>
            <link>https://darthjaja.medium.com/web3-0-globalization-of-the-internet-baa8021c2234?source=rss-c52f29a0483f------2</link>
            <guid isPermaLink="false">https://medium.com/p/baa8021c2234</guid>
            <category><![CDATA[blockchain]]></category>
            <category><![CDATA[decentralization]]></category>
            <category><![CDATA[web-3-point-0]]></category>
            <dc:creator><![CDATA[Darth Jaja]]></dc:creator>
            <pubDate>Mon, 30 Aug 2021 16:46:50 GMT</pubDate>
            <atom:updated>2021-08-30T16:46:50.327Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>TL;DR: </strong>The path to web3.0 is a transition from an inefficient economic model, where value stays behind the walls of isolated platforms, to a more efficient one where value flows freely from one platform to another.</p><p>Does the title sound absurd? After all, the Internet has been a global network since day 0. The globalization that crosses the geographical borders of nation states is already here. <strong>In this post, globalization means crossing the border of different platforms of the Internet to help value flow efficiently</strong>.</p><p><em>I hope this article points to an important problem. But it is NOT a design doc that solves this problem. For sure there will be some follow-up writings around this topic as my thoughts get mature, and they will look like parts of a design doc.</em></p><h4>Platform is a self-growing closed loop of value</h4><p>Speaking of platform, you may think of Facebook, Tiktok, Medium, etc. They are places where people do social, have fun, find goods to purchase or places to stay.</p><p>But thinking again, a better term to express “something that people use to do social/have fun” is “tool”. Indeed, in order to send you friend a message, all you need is a tool, instead of some weird “platform”. Then what the hell do we mean when we use the term “platform”? Here’s my definition.</p><p><strong>platform: a place where producers make something valuable to consumers, and consumers get value by consuming it and then rewards producers with value.</strong></p><p>The value here refers to <strong>everything that constitutes the social benefit in the ecosystem of an app</strong>. For example, the economic and psychological reward that a user receives, the ease of producing and consuming products/contents, etc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ebuBJP1oBj-Yf06icUAXEg.png" /><figcaption>The values flow within a platform</figcaption></figure><p>As a tool, the app helps producers to produce efficiently, and helps consumers to find the desired contents. On top of that, the value generated by user activities are used to feed the ecosystem, making it easier to match demand and supply, thus grows the total value of the ecosystem — this is the self-growing closed loop of value. This way, a tool evolves to a platform.</p><p>Since you are consuming my content on Medium right now, let’s take Medium as an example. You may reward me by clapping this article or following me. If you do this, my future content will be more likely to be chosen by the recommendation system to show up in your and other people’s read list. This increases the ease of navigating consumers to the content they like. All of these form a self-growing closed loop of value on Medium.</p><h4>What’s wrong with the platform?</h4><p>Let’s continue with the Medium example.</p><p>When someone follows me here, he/she may also want to follow me on Linkedin or Twitter. Someone did search for my name on Linkedin and added me, but going to another platform and search is a big friction to consumers, it’s not scalable.</p><p>Moreover, if you clap my article on Medium, Twitter won’t know “<em>hey this guy is getting more popular somewhere else, let’s push his post to more users</em>”, unless I post my article everywhere. This is a friction to producers.</p><p>The last but not the least, on the consumer side, if he/she claps an article, it helps the recommendation system of Medium push better contents. However, Linkedin doesn’t know this action. The reader may still get contents that are no longer of interest.</p><p>Some of the examples above have case-by-case workarounds. However, if we can summarize the pain points with a big yet simple problem definition, we may have a one-off, scalable solution that makes things better at the paradigm level. The big problem is, <strong>it’s so hard for values to flow between platforms</strong>.</p><h4>Where web is going: globalization of value</h4><p>Think about the lifecycle of an iphone. We are quite familiar with how the value of real economy is produced and consumed in a globalized economy.</p><p>How could the same thing happen in the digital world? I’ve already seen some patterns of early stage international trading. For example, it often happens that a meme is created in Bilibili. It’s then spread to Kuaishou or Douyin(Tiktok), and got to know by a greater population. This is exactly an “international” cooperation in creating digital value.</p><p>However, I would say this is one of several examples that merely shows the early pattern of globalization. It’s not a norm yet. <strong>In terms of digital value, We still don’t have the most parts of the infrastructure that support scalable global manufacture</strong>. Where is the supply chain? Where are the accounting tools that distribute profit efficiently and trustfully? Where are the financial institutes that allocates liquidity to the most demanding industry?</p><p>When we have all of the components, and eventually build the globalized value network, the functionalities of the platforms nowadays may even degenerate a little and fulfill their original functionality — tool. This is just like in a global economy, many nation states more or less lose some sovereignty and just serve as a part of the supply chain.</p><h4>Summary</h4><p>In web2.0, platforms serve as closed loops where values can be generated, consumed, and used to generate more value. However, this is not efficient enough. The picture of web3.0 is a world where value no longer stays within a platform. There will be a universally recognized system that exchanges the values between different platforms. I’ll explain in more detail, and propose some options to build this system in follow up articles.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=baa8021c2234" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What the hell is _rTotal and _tTotal]]></title>
            <link>https://darthjaja.medium.com/what-the-hell-is-rtotal-and-ttotal-4e2ec90466f7?source=rss-c52f29a0483f------2</link>
            <guid isPermaLink="false">https://medium.com/p/4e2ec90466f7</guid>
            <category><![CDATA[safemoon]]></category>
            <category><![CDATA[solidity]]></category>
            <category><![CDATA[smart-contract-blockchain]]></category>
            <dc:creator><![CDATA[Darth Jaja]]></dc:creator>
            <pubDate>Tue, 25 May 2021 01:46:21 GMT</pubDate>
            <atom:updated>2021-05-25T01:46:21.216Z</atom:updated>
            <content:encoded><![CDATA[<p>TL;DR: In Reflection contracts like Safemoon, _rTotal is like the money shown in the balance sheets of the central bank, while _tTotal is like the currency in circulation. When someone sells and a fee is to be distributed to all holders, instead of doing it with a for loop, _rTotal is shrunk so everyone’s money is worth more, just like deflation.</p><p>Today I’ve been trying to understand the contract behind Safemoon — well actually, Reflect contract. This article shows my understanding from a high level.</p><p>The contract is written in an ugly way, without many comments so it took me quite a lot of time to understand _rTotal and _tTotal.</p><p>As a result, I find it a rather clever implementation. Think about the two scenarios below:</p><p>If you know Physics, think about how people solve Maxwell equations: They bring E and B to complex space, do calculations easily with complex numbers, and transform the result back to real space.</p><p>If you know Finance, you must be familiar with the deposit reserve ratio, which can be used to roughly derive the currency in circulation from the central bank money supply, and vice versa.</p><p>You see the *supply*here — familiar? It’s not exactly the same thing in Reflection contracts (For example, the currency in circulation is always much more than in the central bank’s balance sheets, opposite to _rTotal and _tTotal in Reflection contracts), but it’s pretty much the same mechanism.</p><p><strong>In Reflect, you can understand _rTotal as the total money supply in the central bank, and _tTotal as the dollars in your pocket + you deposit to commercial bank +, etc. The _getRate() function actually gets you a ratio similar to the deposit reserve ratio.</strong></p><h3>Why is it clever</h3><p>Imagine you are the creator of the Reflection contract. Now you want to distribute fees to everybody when someone sells. What comes right out of your mind might be using a for loop to add to each owner&#39;s balance. But a second thought might be, instead of looping through all the holders, let’s build a deflation mechanism so that the tokens you hold are worth more.</p><p>This is done by changing the _tTotal and _rTotal in _reflectFee function, which is called every time there is a fee. For example, in _transferStandard. This way, the next time there’s a transaction when you call the _getRate() function, a different rate will be generated, thus the tokens you hold is worth more/less. The mechanism is illustrated in the figure below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*axkiZRSxf_lhZmFRhfcI-Q.png" /></figure><p>This is just a rough idea, it’s not rigorous. I don’t want to navigate thru the contract code here but when you do it, hope this idea can help you.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4e2ec90466f7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Sexy Ideas behind Computer Network]]></title>
            <link>https://darthjaja.medium.com/sexy-ideas-behind-computer-network-63b8b8d0f79b?source=rss-c52f29a0483f------2</link>
            <guid isPermaLink="false">https://medium.com/p/63b8b8d0f79b</guid>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[computer-networking]]></category>
            <dc:creator><![CDATA[Darth Jaja]]></dc:creator>
            <pubDate>Sun, 14 Oct 2018 05:56:23 GMT</pubDate>
            <atom:updated>2018-10-14T05:56:23.062Z</atom:updated>
            <content:encoded><![CDATA[<p>In undergraduate years, I had a T-shirt with slogan “Physics is sexy”. It is indeed sexy when you learn how the nature is sketched by Lagrange Mechanics and Least Action Principle, and you will almost fall in love with it when you first know that God is busy throwing dices.</p><p>Engineering is sexy in a different way. After seeing the greatest engineering masterpieces of human-being, you will be fascinated by all the intelligent designs that ensure the works to stand still for thousands of years,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*brf2D-NBLgO1iwRx56rXuQ.png" /><figcaption>Dujiangyan, irrigation system since 256 BC</figcaption></figure><p>under extreme condition,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KIMhXTp-9Ca8J0Y8U_H0YQ.png" /><figcaption>Falcon Heavy Side Boosters</figcaption></figure><p>or for billions of people, which is the computer networks, the backbone of most of the human activity nowadays. This article is about the most amazing designs in computer networks.</p><h4>Hourglass-shaped architecture of Internet</h4><p>IP is the most important conjunction of the Internet architecture. Above it are high level protocols like TCP and UDP, and applications like websites or mail systems. Below IP are low level network facilities like ethernet and optical fiber cable. This architecture is narrow in the middle, and wide in top and bottom, so it got the name Hourglass-shaped architecture.</p><p>The hourglass-shaped architecture enables high level users and application developers to worry less about the complicated low level implementation. No user or application developer has to worry about using adapters or handling packet loss. They can use everything below IP as a black box. Besides, those who create building blocks of network also don’t have to know which application is going to use their network. As long as they implement IP, any application can use their service. So IP is a minimum viable agreement. The simpler a protocol is, the easier people can reach an agreement.</p><blockquote>Tim O’Reilly:</blockquote><blockquote>do as little as possible…. The less you include the easier it will be to agree and you don’t tie yourself down…because we don’t know what will come</blockquote><h4>CDN and Caching</h4><p>Caching is the most important idea in Computer Science. Inside one machine there is L1 to L3 cache. Outside a machine, the network that connects machines together, also use cache to reduce latency and improve reliability.</p><p>In network, frequently requested resources can be cached on machines geographically closer to clients. The simplest way to manage what to cache and how to redirect requests is using the content provider’s server, which we call application server. But this is going to increase the load of application server.</p><p>The typical way of solving this problem is to split work, just like in the back-end of an application, work is split to different dedicated servers. The work of caching and redirecting is eventually split to an independent Content Distribute Network, or CDN. It is so independent that there are companies like Akamai that provides CDN. In Akamai, content servers with replications of its customer resources are distributed around the world. On top of content servers, Akamai has a dedicated DNS system responsible for keeping track of content servers’ health and performance, matching content servers with client request based on performance and delay considerations, and redirecting the request from client .</p><h4>Encapsulation</h4><p>Encapsulation is pervasive in the programming. It hides complexity within bottom layer so that upper layer applications can make use them without knowing what’s going on below.</p><p>Computer network also has a layered structure, such as physical layer, link layer, and application layer. Each layer receives data from the upper layer, adds layer specific information, encapsulates it, and then sends it to lower layer. For example, after receiving message from transport layer, the network layer adds fields like IP and checksum, make it a packet, and sends to link layer. When this chunk of bits are transmitted in the network, adapter needs needs to look at frame to process the data, while router needs to decapsulate the bits chunk until packet is shown to look at the IP and decide how to forward it.</p><h4>Tradeoff: TCP and UDP</h4><p>TCP and UDP are like the two ends of an axis: TCP provides various mechanisms to ensure reliable transmission of data, while UDP only provides best effort delivery without guaranteeing the completeness of data.</p><p>TCP takes more time than UDP because it checks packet loss and data corruption. The receiver asks sender to resend data when there’s problem with packet. When in an application where user is sensitive to data loss, TCP is a must. But in scenarios where speed is essential and data loss is tolerable to some degree, ket loss and data corruption. The receiver asks sender to resend data when there’s problem with packet. When in an application where user is sensitive to data loss, TCP is a must. But in scenarios where speed is essential and data loss is tolerable to some degree, UDP might be a favorable choice.</p><p>There is also space between the two extremes: customized packet error handling mechanism that takes less time than the whole set of tools TCP uses, although it may only provide part of the functionality of TCP. How much functionality we provide with the cost of longer latency depends on the using scenario.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=63b8b8d0f79b" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>