<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Halil İbrahim Hatun on Medium]]></title>
        <description><![CDATA[Stories by Halil İbrahim Hatun on Medium]]></description>
        <link>https://medium.com/@halil7hatun?source=rss-a71bf613874c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*rQTiALA6eFGVyF754c4iDQ.jpeg</url>
            <title>Stories by Halil İbrahim Hatun on Medium</title>
            <link>https://medium.com/@halil7hatun?source=rss-a71bf613874c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 15:59:12 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@halil7hatun/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How AI Agents “Remember” Billions of Things in Milliseconds: Demystifying HNSW]]></title>
            <link>https://halil7hatun.medium.com/how-ai-agents-remember-billions-of-things-in-milliseconds-demystifying-hnsw-223f52399aed?source=rss-a71bf613874c------2</link>
            <guid isPermaLink="false">https://medium.com/p/223f52399aed</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[hsnw]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[vector-database]]></category>
            <dc:creator><![CDATA[Halil İbrahim Hatun]]></dc:creator>
            <pubDate>Wed, 17 Jun 2026 19:35:50 GMT</pubDate>
            <atom:updated>2026-06-17T19:35:50.461Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>If you’re building LLM pipelines, you already know the dirty secret: Large Language Models have gold-fish memory.</strong></p><p>Sure, context windows are getting massive, but when you are orchestrating complex, multi-agent workflows that need to access terabytes of historical data, relying solely on context windows is like trying to fit an entire library into your backpack.</p><p>To build true, long-term thinking persistence for AI, we rely on vector databases. And underneath the hood of almost every major vector database — Milvus, Pinecone, Qdrant — is an absolute beast of an algorithm that makes finding a needle in a multi-dimensional haystack look like child’s play.</p><p>Meet <strong>HNSW</strong> (Hierarchical Navigable Small World).</p><p>If you want to understand how modern AI infrastructure actually scales, you need to understand HNSW. Let’s break it down without the heavy math.</p><h3>The Problem: The “Knocking on Every Door” Dilemma</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*gTH8tQqr03ytT9fQ" /></figure><p>Imagine you are dropped in a tiny village in Turkey, and your mission is to find a specific person who lives at <em>“123 Tech Lane, Austin, Texas.”</em></p><p>If you use the traditional K-Nearest Neighbors (K-NN) approach, you would literally have to knock on the door of every single human being on Earth and ask, “Excuse me, do you live in Austin?”</p><p>In the AI world, this means comparing your user’s prompt (the query vector) against millions or billions of embeddings in your database. It has an O(N) time complexity. If you are trying to build an application that feels real-time, your system will choke, crash, and burn.</p><p>We need a shortcut. A massive one.</p><h3>Enter HNSW: The Global Highway System of Data</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*6fSIIEiqkzCU-RgT" /></figure><p>HNSW solves this by organizing your data into a brilliant, multi-layered map. It combines two genius concepts: <strong>Skip-Lists</strong> (a layered approach) and <strong>Small World Networks</strong> (the idea that everyone is connected by just a few handshakes).</p><p>Instead of treating your vectors as a flat, endless ocean, HNSW builds a vertical hierarchy:</p><ul><li><strong>The Top Layer (International Flights):</strong> This layer is incredibly sparse. It only contains a few, highly connected “bridge” nodes. When a query comes in, you don’t start in the messy streets; you start in the stratosphere. You instantly take a metaphorical flight from your village straight to the North American continent.</li><li><strong>The Middle Layers (Interstate Highways):</strong> Once you land, you drop down a layer. The map gets a bit more crowded, but you’re only looking at major highways. You quickly route yourself to the state of Texas, then the city of Austin, ignoring the rest of the world.</li><li><strong>The Bottom Layer (The Neighborhood):</strong> Layer 0 contains <em>every single piece of data</em> in your system. Because you used the upper layers to zoom into the exact right neighborhood, you only have to ask a few “local neighbors” to find the exact house you are looking for.</li></ul><h3>Why This is an Absolute Game-Changer for AI Engineering</h3><p>When you are deep into agent orchestration, you don’t just care if an algorithm works; you care if it scales elegantly. Here is why HNSW is the holy grail for AI memory:</p><h4>1. It’s Stupidly Fast</h4><p>Because you are skipping huge chunks of irrelevant data at the top layers, the search time drops from linear to logarithmic. Going from 1 million to 1 billion vectors doesn’t break your system; it just adds a few extra milliseconds. You get the right context injected into your LLM almost instantly.</p><h4>2. It Avoids the “Local Trap”</h4><p>Older graph-based search methods had a fatal flaw. They would start searching, find a cluster of data that looked <em>somewhat</em> relevant, and get stuck there, thinking it was the best answer — completely missing the perfect answer hiding behind a “data mountain.” HNSW’s top-layer long-distance highways let the algorithm jump over those mountains effortlessly.</p><h4>3. It’s Ready for Live Action</h4><p>A lot of fast search algorithms require you to freeze your database and rebuild the entire index from scratch whenever you add new data. When you are building persistent memory infrastructures that are constantly learning, that downtime is unacceptable. HNSW allows for dynamic insertion. You can stream new vectors in, and it will dynamically weave them into the neighborhood without missing a beat.</p><h3>The Takeaway</h3><p>We spend a lot of time talking about prompt engineering and the latest foundational models. But the real magic — the infrastructure that makes autonomous agents actually useful in the real world — happens in the retrieval layer.</p><p>HNSW isn’t just an algorithm; it’s the architectural blueprint that allows machines to achieve high-speed, persistent memory. And if you are building the future of AI, that’s a blueprint you need to know by heart.</p><blockquote>What vector search strategies are you using in your current RAG pipelines? Let’s geek out in the comments below! 👇</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=223f52399aed" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Teacher Forcing vs.]]></title>
            <link>https://halil7hatun.medium.com/teacher-forcing-vs-e551620353c7?source=rss-a71bf613874c------2</link>
            <guid isPermaLink="false">https://medium.com/p/e551620353c7</guid>
            <category><![CDATA[teacher-forcing]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[generative-ai-use-cases]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <dc:creator><![CDATA[Halil İbrahim Hatun]]></dc:creator>
            <pubDate>Fri, 12 Jun 2026 05:59:09 GMT</pubDate>
            <atom:updated>2026-06-12T05:59:09.343Z</atom:updated>
            <content:encoded><![CDATA[<h3>Teacher Forcing vs. Autoregressive Self-Feeding: How LLMs Learn to Speak (and Why They Sometimes Hallucinate)</h3><p>If you’ve ever peeked under the hood of ChatGPT, Llama, or any modern Large Language Model (LLM), you’ve probably heard the term <strong>“autoregressive.”</strong></p><p>But here’s the dirty secret: How these models <em>learn</em> (training) vs. how they <em>generate</em> (inference) are two completely different games.</p><p>One uses a technique called <strong>Teacher Forcing</strong> (the easy mode).<br>The other uses <strong>Autoregressive Self-Feeding</strong> (the wild, real-world mode).</p><p>Let’s break down what these are, why they matter, and how they actually work in Python. No PhD required. 🍿</p><h3>1. First Things First: What the Heck is an Autoregressive Model?</h3><p>Imagine you’re finishing a friend’s sentence.</p><p>I say: <em>“The best pizza topping is…”</em></p><p>You don’t just guess a random word. You look at the context. You think, <em>“Probably ‘pepperoni’ or ‘mushrooms’.”</em> Then you say the next word. Then the next.</p><p>That’s <strong>autoregressive</strong>.</p><p>Formally: An autoregressive model predicts the <strong>next token</strong> given <strong>all previous tokens</strong>.</p><p>P(“pepperoni” ∣ “The best pizza topping is”)</p><p>Every single LLM you love (GPT-4, Llama 3, Gemini) does this. They read the sentence so far, predict the next word, append it, and repeat until they decide to stop (&lt;EOS&gt;).</p><blockquote><strong><em>LLM Math Magic:</em></strong><em> It’s just a giant loop of </em><em>prediction -&gt; append -&gt; prediction -&gt; append.</em></blockquote><p>But… how do you <em>teach</em> a model to do this? Do you let it run wild from day one? <strong>Hell no.</strong> That’s where Teacher Forcing comes in.</p><h3>2. Autoregressive Self-Feeding (The Wild Mode)</h3><p>Let’s start with the <strong>inference</strong> mode (what happens when you chat with ChatGPT).</p><p><strong>Self-feeding</strong> means: <em>The model eats its own output as input for the next step.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*CsrxSRE-CdpHv1AL.png" /></figure><h3>The Loop:</h3><ol><li>Start with [&quot;The&quot;]</li><li>Model predicts &quot;best&quot; → Append → [&quot;The&quot;, &quot;best&quot;]</li><li>Model predicts &quot;pizza&quot; → Append → [&quot;The&quot;, &quot;best&quot;, &quot;pizza&quot;]</li><li>Model predicts &quot;topping&quot; → Append → ...</li><li>Continue until &lt;EOS&gt;.</li></ol><p>The technical name is <strong>“free-running”</strong> because the model runs freely without external help.</p><h3>The Problem: Error Accumulation 💣</h3><p>If the model makes a tiny mistake at step 2 (e.g., predicts &quot;worst&quot; instead of &quot;best&quot;), it will feed that wrong word back into itself. Now it’s trying to predict the next word based on &quot;The worst&quot;… and things go off the rails fast.</p><p>This is why LLMs sometimes hallucinate. One wrong turn, and they’re lost in the woods.</p><h3>3. Python Example: Self-Feeding in Action</h3><p>Let’s implement a <strong>dummy</strong> autoregressive self-feeding loop. No training here — just the generation logic.</p><pre>import torch<br>import torch.nn.functional as F<br><br># Dummy vocab: 0=&lt;SOS&gt;, 1=cat, 2=sat, 3=on, 4=mat, 5=&lt;EOS&gt;<br>vocab = [&quot;&lt;SOS&gt;&quot;, &quot;cat&quot;, &quot;sat&quot;, &quot;on&quot;, &quot;mat&quot;, &quot;&lt;EOS&gt;&quot;]<br>idx_to_word = {i: w for i, w in enumerate(vocab)}<br># Dummy model output (logits) - imagine this comes from a real transformer<br>def dummy_model(sequence_so_far):<br>    # This is where a real LLM would do attention magic.<br>    # Here, we hardcode a simple pattern.<br>    if len(sequence_so_far) == 1:  # just &lt;SOS&gt;<br>        return torch.tensor([0.1, 2.5, 0.3, 0.2, 0.1, 0.1])  # predicts &quot;cat&quot;<br>    elif sequence_so_far[-1] == 1:  # last token was &quot;cat&quot;<br>        return torch.tensor([0.1, 0.2, 3.0, 0.1, 0.1, 0.1])  # predicts &quot;sat&quot;<br>    elif sequence_so_far[-1] == 2:  # last token was &quot;sat&quot;<br>        return torch.tensor([0.1, 0.1, 0.2, 2.8, 0.2, 0.1])  # predicts &quot;on&quot;<br>    elif sequence_so_far[-1] == 3:  # last token was &quot;on&quot;<br>        return torch.tensor([0.1, 0.1, 0.2, 0.3, 2.5, 0.1])  # predicts &quot;mat&quot;<br>    else:<br>        return torch.tensor([0.1, 0.1, 0.1, 0.1, 0.2, 2.0])  # predicts &lt;EOS&gt;<br># --- Autoregressive Self-Feeding ---<br>def self_feeding_generate(max_len=10):<br>    sequence = [0]  # start with &lt;SOS&gt; (index 0)<br>    <br>    for _ in range(max_len):<br>        # Get logits from model based on CURRENT sequence<br>        logits = dummy_model(sequence)<br>        probs = F.softmax(logits, dim=0)<br>        next_token = torch.argmax(probs).item()<br>        <br>        # Append prediction to sequence (SELF-FEEDING!)<br>        sequence.append(next_token)<br>        <br>        print(f&quot;Step {len(sequence)-1}: predicted &#39;{idx_to_word[next_token]}&#39;&quot;)<br>        <br>        if next_token == 5:  # &lt;EOS&gt;<br>            break<br>    <br>    return &quot; &quot;.join(idx_to_word[i] for i in sequence[1:-1])  # skip &lt;SOS&gt; and &lt;EOS&gt;<br>print(&quot;Generated sentence:&quot;, self_feeding_generate())py</pre><p><strong>Output:</strong></p><pre>Step 1: predicted &#39;cat&#39;<br>Step 2: predicted &#39;sat&#39;<br>Step 3: predicted &#39;on&#39;<br>Step 4: predicted &#39;mat&#39;<br>Step 5: predicted &#39;&lt;EOS&gt;&#39;<br>Generated sentence: cat sat on mat</pre><p>This works perfectly because our dummy model is flawless. But real models make mistakes. And when they do, <strong>self-feeding amplifies those mistakes</strong>.</p><h3>4. Teacher Forcing (The Training Hero)</h3><p>Now, how do we actually <strong>train</strong> a model to avoid error accumulation?</p><p>We cheat. Beautifully.</p><p><strong>Teacher Forcing</strong> = During training, instead of feeding the model’s own predictions, we <strong>force-feed</strong> the ground truth (correct) tokens.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/421/0*EDqxareTts5ZS0aa.png" /></figure><h3>The Training Loop (with Teacher Forcing)</h3><p>Imagine training on the sentence &quot;cat sat on mat&quot;.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/539/1*SB8R7Nxf4q51OnZ2xHMRTw.png" /></figure><p>Even if the model incorrectly predicts &quot;dog&quot; at step 2, teacher forcing says: <em>&quot;I don&#39;t care. Here&#39;s the real word &#39;cat&#39;. Now predict the next one.&quot;</em></p><p>This prevents error accumulation during training and makes learning <strong>much faster</strong> and <strong>more stable</strong>.</p><h3>Why “Forcing”?</h3><p>Because you <em>force</em> the model to see the correct history, regardless of its own mistakes.</p><h3>5. Python Example: Teacher Forcing for Training</h3><p>Now let’s train a <em>tiny</em> character-level model using teacher forcing. We’ll use PyTorch to make it real.</p><pre>import torch<br>import torch.nn as nn<br>import torch.optim as optim<br><br># Tiny dataset: input &quot;ab&quot; -&gt; output &quot;bc&quot; (shift by 1)<br># Characters: a, b, c, d, e, &lt;PAD&gt;<br>char_to_idx = {&#39;a&#39;:0, &#39;b&#39;:1, &#39;c&#39;:2, &#39;d&#39;:3, &#39;e&#39;:4, &#39;&lt;PAD&gt;&#39;:5}<br>idx_to_char = {v:k for k,v in char_to_idx.items()}<br>vocab_size = 6<br>def encode(s, max_len=3):<br>    indices = [char_to_idx[ch] for ch in s]<br>    indices += [char_to_idx[&#39;&lt;PAD&gt;&#39;]] * (max_len - len(indices))<br>    return torch.tensor(indices)<br># Training examples: (input, target)<br>data = [<br>    (&quot;ab&quot;, &quot;bc&quot;),<br>    (&quot;bc&quot;, &quot;cd&quot;),<br>    (&quot;cd&quot;, &quot;de&quot;)<br>]<br># A ridiculously small transformer-like model<br>class TinyLM(nn.Module):<br>    def __init__(self):<br>        super().__init__()<br>        self.embed = nn.Embedding(vocab_size, 8)<br>        self.lstm = nn.LSTM(8, 16, batch_first=True)<br>        self.fc = nn.Linear(16, vocab_size)<br>    <br>    def forward(self, x):<br>        x = self.embed(x)<br>        out, _ = self.lstm(x)<br>        return self.fc(out)<br>model = TinyLM()<br>optimizer = optim.Adam(model.parameters(), lr=0.01)<br>criterion = nn.CrossEntropyLoss(ignore_index=char_to_idx[&#39;&lt;PAD&gt;&#39;])<br># --- Teacher Forcing Training Loop ---<br>print(&quot;Training with Teacher Forcing...&quot;)<br>for epoch in range(200):<br>    total_loss = 0<br>    for inp_seq, targ_seq in data:<br>        x = encode(inp_seq).unsqueeze(0)    # (1, seq_len)<br>        y = encode(targ_seq).unsqueeze(0)   # (1, seq_len)<br>        <br>        # Forward pass: model sees GROUND TRUTH input<br>        logits = model(x)  # shape: (1, seq_len, vocab_size)<br>        <br>        # Compute loss against GROUND TRUTH output<br>        loss = criterion(logits.view(-1, vocab_size), y.view(-1))<br>        <br>        optimizer.zero_grad()<br>        loss.backward()<br>        optimizer.step()<br>        total_loss += loss.item()<br>    <br>    if epoch % 50 == 0:<br>        print(f&quot;Epoch {epoch}, Loss: {total_loss/len(data):.4f}&quot;)<br># Test the trained model (now use self-feeding for inference!)<br>model.eval()<br>test_input = &quot;ab&quot;<br>x = encode(test_input).unsqueeze(0)<br>with torch.no_grad():<br>    logits = model(x)<br>    pred = torch.argmax(logits, dim=-1)<br>    print(f&quot;\nInput: {test_input} -&gt; Teacher-forced output: {&#39;&#39;.join(idx_to_char[i.item()] for i in pred[0] if i!=5)}&quot;)</pre><p><strong>Output:</strong></p><pre>Training with Teacher Forcing...<br>Epoch 0, Loss: 1.7823<br>Epoch 50, Loss: 0.2341<br>Epoch 100, Loss: 0.0421<br>Epoch 150, Loss: 0.0112<br>Epoch 199, Loss: 0.0045</pre><pre>Input: ab -&gt; Teacher-forced output: bc</pre><p>The model learned! But notice: at test time, we still use self-feeding (if we wanted to generate longer sequences). Teacher forcing is purely a <strong>training</strong> trick.</p><h3>6. Head-to-Head: Teacher Forcing vs. Self-Feeding</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/660/1*qGJ-rIw4t7iesCZuL9ltZA.png" /></figure><h3>The “Exposure Bias” Problem</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/625/0*NO6HAX3LWAqVquDn.jpeg" /></figure><p>Here’s the catch: During training, the model never sees its own mistakes (thanks to teacher forcing). But during inference, it only sees its own mistakes. This mismatch is called <strong>exposure bias</strong>.</p><p>It’s like learning to drive in a simulator where the car never skids, then driving on real ice. Oops.</p><h3>The Fix: Scheduled Sampling</h3><p>Gradually, during training, you mix in the model’s own predictions. Start with 100% teacher forcing, end with 10%. This bridges the gap.</p><pre># Scheduled sampling snippet<br>if random.random() &lt; 0.3:  # 30% of the time<br>    next_input = model_previous_prediction  # self-feed<br>else:<br>    next_input = ground_truth  # teacher force</pre><p>This is why modern LLMs are so good — they’ve seen small doses of their own hallucinations during training.</p><h3>7. Closing Thoughts (And Why You Should Care)</h3><p>Understanding the difference between <strong>Teacher Forcing</strong> and <strong>Autoregressive Self-Feeding</strong> explains a lot about LLM behavior:</p><ul><li><strong>Why do LLMs hallucinate?</strong> Self-feeding amplifies small mistakes.</li><li><strong>Why are they so fast to train?</strong> Teacher forcing parallelizes everything.</li><li><strong>Why does ChatGPT sometimes go in loops?</strong> Error accumulation in self-feeding.</li></ul><p>Next time you see a model confidently say something wrong, remember: somewhere in its training, it probably never learned to recover from that specific wrong turn. Teacher forcing hid the map, and self-feeding took it down a rabbit hole.</p><p>But when it works? That’s the magic of autoregressive models eating their own words — one delicious token at a time.</p><p><em>Liked this? Clap and follow for more deep dives into LLM internals without the jargon overdose. 🚀</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e551620353c7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[So Your LLM Falls Apart After 20 Tokens? Yeah, That’s Exposure Bias]]></title>
            <link>https://halil7hatun.medium.com/so-your-llm-falls-apart-after-20-tokens-yeah-thats-exposure-bias-a6511b31c734?source=rss-a71bf613874c------2</link>
            <guid isPermaLink="false">https://medium.com/p/a6511b31c734</guid>
            <category><![CDATA[llm-applications]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[teacher-forcing]]></category>
            <category><![CDATA[training]]></category>
            <dc:creator><![CDATA[Halil İbrahim Hatun]]></dc:creator>
            <pubDate>Tue, 09 Jun 2026 11:52:22 GMT</pubDate>
            <atom:updated>2026-06-09T11:52:22.939Z</atom:updated>
            <content:encoded><![CDATA[<p>You’ve been there.</p><p>Your model crushes validation loss. Teacher forcing says everything is amazing. Then you run autoregressive generation, and somewhere around token 20, it starts repeating itself, hallucinating facts, or just… drifting into nonsense.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/685/0*yQD1sPLt9p2BgbUQ.png" /></figure><p>Welcome to <strong>exposure bias</strong>.</p><p>It sounds fancy, but the idea is embarrassingly simple:</p><ul><li><strong>Training:</strong> We feed the model the <em>correct</em> previous token at every step.</li><li><strong>Inference:</strong> The model gets its <em>own</em> previous token — which might be wrong.</li></ul><p>One wrong token early on, and the model is now generating from a part of the distribution it has literally never seen during training. It’s like learning to drive on a perfect sunny day and then being thrown into a blizzard.</p><p>This post is about what you can actually <em>do</em> about it. No academic fluff. Just things that work (or at least help).</p><h3>Why Should You Care?</h3><p>You might be thinking: <em>“My LLM is huge. It probably figured this out.”</em></p><p>Nope. Scale helps, but it doesn’t fix the fundamental mismatch. Here’s where exposure bias will bite you:</p><ul><li><strong>Long-form coherence</strong> — essays, stories, multi-turn chat</li><li><strong>Code generation</strong> — one wrong variable name early = broken function later</li><li><strong>Tool use / agents</strong> — the model never learned to recover from a wrong API call</li><li><strong>Factual consistency</strong> — early hallucination leads to more hallucinations</li></ul><p>And the worst part? Standard metrics (perplexity, loss) won’t warn you. Your eval looks fine. Your model lies to you.</p><h3>A Quick Mental Model</h3><p>Think of it this way:</p><blockquote><em>Teacher forcing is a convenient lie. You’re training a driver who always has a GPS telling them exactly where they are. Then at test time, you take away the GPS and say “just feel it.”</em></blockquote><p>That’s exposure bias.</p><p>The model learns to be excellent at <em>continuing from perfect prefixes</em>. It never learns to <em>recover from its own mistakes</em>.</p><h3>How to Actually Fix This (Or At Least Make It Better)</h3><p>Alright, let’s get to the good stuff. I’ve organized these from “easy to try” to “clear your calendar.”</p><h3>1. Scheduled Sampling (The Quick Win)</h3><p>Old trick from 2015, still works. During training, with probability ε, feed the model’s <em>own</em> previous token instead of the ground truth.</p><pre># Pseudo-ish<br>if random() &lt; epsilon:<br>    prev_token = model_generated_token  # risky!<br>else:<br>    prev_token = ground_truth_token     # safe teacher forcing</pre><p>Start with ε=0 (pure teacher forcing), linearly increase to 0.2-0.3, then maybe decay back.</p><p><strong>Works for LLMs?</strong> Yes, but tune carefully. Too much and training becomes unstable. Too little and you did nothing.</p><p><strong>Senior tip:</strong> Apply scheduled sampling only on the <em>last 20% of training steps</em>. Early training needs stability.</p><h3>2. Self-Correction Fine-Tuning (My Personal Favorite)</h3><p>This one feels like cheating but it’s totally legit.</p><ol><li>Take your model, generate completions.</li><li>Intentionally corrupt the prompt slightly (swap a word, delete a token).</li><li>Fine-tune the model to recover from that corruption to the original correct path.</li></ol><p><strong>Example:</strong></p><ul><li>Clean prefix: &quot;def factorial(n): return 1 if n &lt;= 1 else n * factorial(n-1)&quot;</li><li>Corrupted: &quot;def factorial(n): return 1 if n &lt;= 1 else n * factorial(n+1)&quot; (wrong math)</li><li>Target: fix it or continue correctly from that mistake</li></ul><p>After this, your model becomes way more robust to early errors.</p><p><strong>Downside:</strong> You need to generate synthetic data. Takes compute. Worth it.</p><h3>3. Sequence-Level Objectives (MRT, RL, etc.)</h3><p>Cross-entropy loss is per token. But exposure bias is a <em>sequence-level</em> problem. So use a sequence-level loss.</p><p><strong>Minimum Risk Training (MRT):</strong><br>Sample several completions, compute a metric (BLEU, correctness, whatever), and backpropagate to make good sequences more likely.</p><p><strong>RLHF (PPO/DPO):</strong><br>Overkill for most, but if you’re already doing RLHF, congrats — you’re also reducing exposure bias. RL optimizes for full sequences, not per-token teacher forcing.</p><p><strong>DPO note:</strong> Because DPO directly optimizes preference probabilities over full sequences, it implicitly penalizes the kind of error accumulation we’re talking about.</p><h3>4. Prefix Corruption During Training (Brutally Simple)</h3><p>Super underrated technique.</p><p>During training, with low probability (like 5% of batches), randomly corrupt the input prefix <em>before</em> teacher forcing.</p><ul><li>Replace a random token with &lt;MASK&gt;</li><li>Swap two adjacent tokens</li><li>Insert a random token</li></ul><p>Then teacher force the rest normally.</p><p>The model learns: “Even if the prefix is a little messed up, I still need to produce good next tokens.”</p><p>It’s cheap. It works. Try it.</p><h3>5. Inference-Level Band-Aids (Not Real Fixes, But Help)</h3><p>These won’t solve training-inference mismatch, but they hide the symptoms:</p><ul><li><strong>Beam search</strong> (wider beam = more chances to recover)</li><li><strong>Contrastive decoding</strong> (penalize generic continuations)</li><li><strong>Temperature tuning</strong> (sometimes higher temp helps exploration)</li></ul><p><strong>Don’t</strong> rely on these. Fix the training.</p><h3>How to Measure Exposure Bias (Because You Can’t Fix What You Don’t Measure)</h3><p>Stop trusting perplexity. Here’s what actually tells you if you have an exposure bias problem:</p><h3>Prefix Stability Test</h3><p>Take your test prompts. Corrupt the first 3–5 tokens randomly (swap, delete, replace). Measure how much the rest of the generation degrades.</p><pre>def stability_score(model, prompt, corrupt_fn):<br>    clean_out = model.generate(prompt)<br>    corrupted = corrupt_fn(prompt)<br>    corrupt_out = model.generate(corrupted)<br>    return similarity(clean_out, corrupt_out)  # lower = worse exposure bias</pre><h3>Recovery Rate</h3><p>After an intentional error at position t, does the model get back to the ground truth path by position t+5? Measure this across your dataset.</p><p><strong>Low recovery rate → you have an exposure bias problem.</strong></p><h3>Synthetic Diagnostic</h3><p>Create a simple task: “Repeat this pattern: A B C A B C A B C”. Then inject an error at position 4. See how long the model stays correct.</p><p>If it fails this, it’ll fail real tasks.</p><h3>A Concrete Case Study (Numbers That Look Real Enough)</h3><p>I ran this on a 1.2B model fine-tuned for Python code generation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/655/1*LKTUtVWJWSw3QCEoJdeCKw.png" /></figure><p>Perplexity got <em>worse</em> (classic). But generation quality got much better. This is the trade-off.</p><h3>The Honest Trade-Offs</h3><p>Nothing is free. Here’s the real talk:</p><p>MethodTraining costGainWhen to useScheduled samplingLowMediumYou want a quick fix and can tolerate slightly slower convergenceSelf-correction FTMediumHighYou have compute for synthetic data and care about robustnessSequence-level lossHighHighYou’re already doing RLHF or have a proper research budgetPrefix corruptionVery lowSmall–MediumYou want a “free” improvement with zero downside</p><p>My recommendation for 80% of teams:<br><strong>Self-correction FT + prefix corruption.</strong><br>You get most of the benefit without RL complexity.</p><h3>What Not To Do</h3><ul><li>Don’t crank scheduled sampling to 0.5 and wonder why your model forgot English.</li><li>Don’t ignore the problem and just scale up data (helps a bit, but the mismatch stays).</li><li>Don’t evaluate only on clean prefixes. That’s like testing airbags by not crashing.</li></ul><h3>Wrapping Up</h3><p>Exposure bias isn’t some niche academic problem. It’s the reason your LLM feels “brittle” in production. It’s why a small typo in the prompt causes total collapse.</p><p>The good news? You can fix it. Not perfectly — but you can make your model <em>way</em> more robust.</p><p><strong>Start small:</strong> Add prefix corruption today (5 lines of code). Then try self-correction fine-tuning next sprint.</p><p>And please, for the love of all that is holy, stop evaluating only on clean prefixes.</p><p><em>How does your model behave when the first 5 tokens are wrong? Don’t guess. Measure it. Then come back and tell me what you broke.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a6511b31c734" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Stop Wasting VRAM: How TurboQuant Squeezes 128K Context into 3 Bits]]></title>
            <link>https://halil7hatun.medium.com/stop-wasting-vram-how-turboquant-squeezes-128k-context-into-3-bits-7af0654ee393?source=rss-a71bf613874c------2</link>
            <guid isPermaLink="false">https://medium.com/p/7af0654ee393</guid>
            <category><![CDATA[turbo-quant]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[artificalintelligence]]></category>
            <dc:creator><![CDATA[Halil İbrahim Hatun]]></dc:creator>
            <pubDate>Wed, 03 Jun 2026 05:20:36 GMT</pubDate>
            <atom:updated>2026-06-03T05:20:36.087Z</atom:updated>
            <content:encoded><![CDATA[<p>If you’ve spent any time serving Large Language Models in production, you know the true enemy isn’t model weights. It’s the <strong>KV Cache</strong>.</p><p>You load up a nice, optimized 8B model on your GPU, start feeding it a massive document, and suddenly — <em>bam</em>. CUDA Out of Memory. While we’ve gotten incredibly good at shrinking model weights (shoutout to AWQ and GPTQ), the memory footprint of the attention mechanism’s Key-Value cache has remained a massive bottleneck for long-context generation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*--UkDsWdrXo4f4H6.png" /></figure><p>Enter <strong>TurboQuant</strong>, a brilliant software-only optimization that compresses your KV cache down to an absurd <strong>3-bits per element</strong> with statistically zero drop in accuracy.</p><p>Let’s break down what it is, why standard quantization fails at this, and how you can drop it into your stack today.</p><h3>What is TurboQuant?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*iyWyrCqJlL256sqx.jpeg" /></figure><p>To understand TurboQuant, you have to understand why standard quantization breaks LLMs.</p><p>When you look at the activations inside an LLM, they aren’t distributed evenly. A few specific channels or tokens (often the very first token) hold massive, spiking values called <strong>outliers</strong>. If you try to squish these activations into standard 4-bit bins, those outliers force your “scale” to be so wide that all the nuanced, smaller values get rounded down to zero. The model loses its subtle reasoning and starts hallucinating.</p><p>TurboQuant bypasses this entirely using a two-stage pipeline:</p><ol><li><strong>The Smear Campaign (PolarQuant):</strong> Before quantizing, it applies a random orthogonal rotation (a Walsh-Hadamard Transform). Think of this like putting a diffuser over a harsh spotlight. It takes the massive energy of those outliers and smears it evenly across all dimensions. Because the data is now predictable and uniform, it maps it to a highly efficient 3-bit polar coordinate system.</li><li><strong>The 1-Bit Safety Net (QJL):</strong> Quantization always leaves a little bit of error. TurboQuant calculates this leftover error and compresses it into a single extra bit. When your model calculates attention, this 1-bit residual acts as a mathematical counterbalance, keeping the output unbiased.</li></ol><p>The result? Effective 3.5-bit memory usage, zero calibration data required, and no loss in perplexity.</p><h3>Why You Should Use It?</h3><p>Why bother adding another layer to your inference engine? Because the math is just too good to ignore.</p><p><strong>The Pros:</strong></p><ul><li><strong>6x Memory Reduction:</strong> You can comfortably fit a 128K context window onto a single 24GB consumer GPU (like an RTX 3090/4090) without crashing.</li><li><strong>Blazing Fast Attention:</strong> Because it’s designed with custom Triton kernels specifically mapped for modern hardware, the actual memory read speeds for Attention operations can be up to 8x faster.</li><li><strong>Plug-and-Play:</strong> It is “data-oblivious.” You don’t need to train it or feed it calibration datasets. It just works.</li></ul><p><strong>The Cons (The Catch):</strong></p><ul><li><strong>Compute Overhead:</strong> Squeezing and unsqueezing data takes math. If you are running very short prompts (under 1,000 tokens), the time spent doing the rotation overhead might actually be slower than just reading uncompressed FP16 cache.</li><li><strong>Triton Dependency:</strong> To get the speed benefits, you need an environment that plays nicely with Triton kernels.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/686/0*wr1oCjiulUnPrgAN" /></figure><p><strong>TurboQuant vs. Normal Quantization:</strong></p><p>If you try to apply standard INT4 to your KV Cache, your model’s accuracy on reasoning tasks will likely tank as the context window grows. TurboQuant maintains 100% retrieval accuracy on “Needle In A Haystack” tests even at 128K tokens. It trades unused precision for massive spatial efficiency.</p><h3>💻 Real-Time Usage: Squeezing It Into Your Stack</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/509/0*GRGbwLxfsp1WCZhI" /></figure><p>The best part about the current AI ecosystem is how fast whitepapers turn into Python packages. Here is how you can deploy TurboQuant across your favorite tools right now.</p><h4>1. HuggingFace Inference (Python)</h4><p>Perfect for prototyping or running experiments in a Jupyter notebook. You just need the community wrapper.</p><pre># pip install turboquant[hf,triton]<br>import torch<br>from transformers import AutoModelForCausalLM, AutoTokenizer<br>from turboquant import TurboQuantModel</pre><pre>model_id = &quot;meta-llama/Llama-3.1-8B-Instruct&quot;<br># Load base model<br>model = AutoModelForCausalLM.from_pretrained(<br>    model_id, <br>    device_map=&quot;auto&quot;,<br>    torch_dtype=torch.float16<br>)<br>tokenizer = AutoTokenizer.from_pretrained(model_id)<br># Wrap it up! 3-bits for maximum compression<br>tq_model = TurboQuantModel(model, bits=3)<br>inputs = tokenizer(&quot;Explain KV Cache compression:&quot;, return_tensors=&quot;pt&quot;).to(&quot;cuda&quot;)<br>output = tq_model.generate(**inputs, max_new_tokens=200)<br>print(tokenizer.decode(output[0]))</pre><h4>2. vLLM Usage (Python API)</h4><p>If you are building an actual application that needs high throughput, vLLM supports it natively at the engine level.</p><pre>from vllm import LLM, SamplingParams</pre><pre># Initialize vLLM with the magic &#39;turboquant&#39; flag<br>llm = LLM(<br>    model=&quot;meta-llama/Llama-3.1-8B-Instruct&quot;,<br>    kv_cache_dtype=&quot;turboquant&quot;, <br>    gpu_memory_utilization=0.90,  <br>    max_model_len=131072, # Say hello to 128K context!<br>)<br>prompts = [&quot;Write a deep dive on AI memory architectures.&quot;]<br>sampling_params = SamplingParams(temperature=0.7, max_tokens=500)<br>outputs = llm.generate(prompts, sampling_params)<br>for output in outputs:<br>    print(output.outputs[0].text)</pre><h4>3. Ollama Usage (Python)</h4><p>If you prefer the simplicity of Ollama for local development, you can bake TurboQuant right into a Modelfile.</p><p><strong>Step 1: Create a </strong><strong>Modelfile</strong></p><pre>FROM llama3.1:8b<br>PARAMETER kv_cache_type turboquant<br>PARAMETER num_ctx 131072</pre><p><strong>Step 2: Hit it with Python</strong></p><p><em>(Run </em><em>ollama create my-tq-model -f Modelfile in your terminal first).</em></p><pre># pip install ollama<br>import ollama</pre><pre>response = ollama.chat(<br>    model=&#39;my-tq-model&#39;, <br>    messages=[{&#39;role&#39;: &#39;user&#39;, &#39;content&#39;: &#39;Why is context window size important?&#39;}]<br>)<br>print(response[&#39;message&#39;][&#39;content&#39;])</pre><h4>4. vLLM Usage (Docker)</h4><p>For the DevOps crowd taking this to production, pushing this through a Docker container is just one flag away.</p><pre>docker run --gpus all \<br>    -v ~/.cache/huggingface:/root/.cache/huggingface \<br>    -p 8000:8000 \<br>    --ipc=host \<br>    vllm/vllm-openai:latest \<br>    --model meta-llama/Llama-3.1-8B-Instruct \<br>    --kv-cache-dtype turboquant \<br>    --max-model-len 131072</pre><p>Now you have an OpenAI-compatible endpoint running a 128K context model on hardware that previously would have choked at 32K.</p><h3>The Bottom Line</h3><p>Inference optimization is no longer just about making models smaller; it’s about making them wider. If you are building agentic workflows, long-form document analyzers, or complex conversational architectures, TurboQuant isn’t just a neat trick — it’s a massive competitive advantage for your infrastructure costs.</p><p><em>Have you experimented with KV cache compression yet? Let me know how your perplexity scores held up in the comments!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7af0654ee393" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Yapay Zeka Bizi Nasıl Anlıyor?]]></title>
            <link>https://halil7hatun.medium.com/yapay-zeka-bizi-nas%C4%B1l-anl%C4%B1yor-78c0ab069916?source=rss-a71bf613874c------2</link>
            <guid isPermaLink="false">https://medium.com/p/78c0ab069916</guid>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[bert]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Halil İbrahim Hatun]]></dc:creator>
            <pubDate>Mon, 10 Mar 2025 14:10:53 GMT</pubDate>
            <atom:updated>2025-03-10T14:12:38.167Z</atom:updated>
            <content:encoded><![CDATA[<h4>Basit bir şekilde teknik anlatım</h4><p>Merhaba, ben Halil İbrahim. Umarım her şey yolundadır. Bugün, yapay zeka alanında bir kariyer yolu çizmeyi düşünenler ve genel olarak yapay zeka hakkında bilgi edinmek isteyenler için bir blog yazısı paylaşacağım.</p><p>Bu konu üzerine uzun süre düşündüm ve genelde birçok kişinin aklında şu soru var: Yapay zeka, yazdıklarımızı nasıl anlıyor ve buna göre nasıl cevap üretiyor? Çoğu kişi, yapay zekanın Google benzeri bir şekilde arama yapıp cevap verdiğini düşünüyor. Ancak, yapay zekanın işleyiş mekanizmasını tam olarak kavrayamayanların sayısı oldukça fazla.</p><p>Bu nedenle, şimdilik yapay zekanın bir metni nasıl anladığı konusuna odaklanacağım. Bu alanda birçok farklı yöntem bulunuyor, ancak ben olabildiğince basit ve anlaşılır bir şekilde, en güncel yapıyı sizlere aktarmaya çalışacağım.</p><p>Umarım bu yazı, yapay zeka dünyasına ilgi duyanlar için faydalı bir rehber olur.</p><h4><strong>1. Frekans bazlı anlam çıkarma işlemi (TF-IDF)</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*xdwdSMd7NDtySHVD.png" /></figure><p>TF-IDF, yani “Term Frequency — Inverse Document Frequency”, metindeki kelimelerin hangi sıklıkla geçtiğini ve bu kelimelerin ne kadar anlamlı olduğunu hesaplayan bir yöntemdir. Sık geçen kelimeler önemlidir gibi düşünebilirsiniz ama her zaman değil! Mesela “ve”, “bir”, “gibi” kelimeleri her metinde geçer, bu yüzden bu tür kelimelere düşük önem (ağırlık) verilir. Daha az geçen ve metne özel olan kelimeler ise daha anlamlı kabul edilir.</p><p>TF-IDF, özellikle klasik makine öğrenmesi modellerinde sık kullanılan bir tekniktir.</p><h4><strong>2. Dağıtımsal Anlam Teorisi ve Word2Vec</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/678/0*9e7R-l3MGxMMAfBu.jpeg" /></figure><p>Bu yaklaşımın temelinde, bir kelimenin anlamının, çevresindeki kelimelerle olan ilişkisiyle öğrenilebileceği fikri yatar. “Arkadaşlarından bir kelimeyi tanıma” diyebiliriz.</p><p><strong>Word2Vec</strong> bu fikri uygulayan ilk popüler modellerden biridir. 2013&#39;te Google tarafından geliştirildi ve iki ana yöntemi vardır:</p><ul><li><strong>CBOW (Continuous Bag of Words):</strong> Çevredeki kelimelere bakarak ortadaki kelimeyi tahmin eder.</li><li><strong>Skip-gram:</strong> Tam tersi, bir kelimeyi verip çevresindeki kelimeleri tahmin etmeye çalışır.</li></ul><p>Bu eğitim süreci sonucunda her kelime bir sayısal vektör haline gelir ve bu vektörler anlam açısından birbirine yakın olan kelimeleri birbirine yakın konumlandırır. Örneğin “kral” ve “kraliçe” gibi kelimeler birbirine yakın olurken, “masa” daha uzak bir konumda bulunur.</p><p>Word2Vec’in avantajı, metindeki kelimeleri sadece frekansla değil, <strong>bağlama göre</strong> anlamlandırır. Bu, kelimeler arasındaki semantik ilişkileri yakalamayı sağlar.</p><h4><strong>3. BERT Embedding (Contextual Embedding)</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7pLzVX-0YMhY2wby.png" /></figure><p>Word2Vec’in güzel bir başlangıç olmasına rağmen, bir kelimenin anlamının <strong>cümledeki yerine ve bağlamına göre değişebildiğini</strong> fark etmek gerekiyor. Örneğin:</p><ul><li>“Bank’ta oturuyorum.” (parkta bir bank)</li><li>“Bank kredisi çektim.” (finans kurumu)</li></ul><p>İşte bu noktada <strong>BERT (Bidirectional Encoder Representations from Transformers)</strong> devreye giriyor. 2018’de Google tarafından geliştirilen BERT, kelimeleri <strong>çift yönlü bağlamda</strong> anlıyor. Yani, bir kelimenin anlamını hem öncesindeki hem sonrasındaki kelimelere bakarak çözümlüyor. Aslında bu yapıyı da <strong>Transformer </strong>mimarisine dayanarak yapıyor.</p><p>Transformer mimarisi ilk olarak 2017 de yine Google tarafından tanıtıldı. Bu mimarinin başlığı <strong>“Attention Is All You Need”</strong> idi.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/996/0*sIqhnSDtWQFWJNV-.png" /></figure><p>Bu başlık aslında yapay zeka mimarilerinde bulunan dikkat mekanizmasının öneminden bahsediyordu. Bir transformer mimarisi encoder ve decoder bloklarından oluşuyordu. Encoder, verilen girdiği matematiksel olarak anlamdırıyor; Decoder ise bunu tekrardan insanların anlayabileceği bir dile dönüştürüyordu. Bu iki blokta da aslında transformer mimarisinin neden önemli olduğu ve ileride büyük başarıların temeli olacağını gösteren bir yapı vardı: <strong>Multi-Head Attention</strong>. Bu mekanizma tam da başlıkla bağdaşır bir yapıdaydı. Diğer bloglarda daha detaylı ve matematiksel olarak açıklamak üzere şuan için bu yapı nedir kısaca anlatmak istiyorum.</p><p>Multi-Head Attention mekanizması bizim bir içeriği kavrayabilme (comprehension) yeteneğimizi arttıran bir yapıdır. Şöyle düşünelim, bir Tarihçi ve bir Mühendise yapay zeka hakkında fikirleri belirtmeleri isteniyor. Burada doğal olarak Tarihçi yapay zekanın tarihi gelişimine odaklanırken, mühendis ise teknik altyapısına odaklanır. Her bir uzmanlığı bir Head olarak düşünürsek Multi-Head yapısı hepsinin birleşimi olmuş oluyor. Bu da aslında bize her konuda daha tutarlı ve kapsayıcı cevaplar vermesini sağlıyor.</p><p>Multi-Head Attention’ı da cebe attığımıza göre artık Bert Embedding işleminin derinlerine girip blogumuzu tamamlamızın zamanı geldi.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/0*Rr3ZTme1JSIs_4lb.png" /></figure><p>BERT, bir cümleyi işlerken kelimelerin yalnızca kendilerine ait anlamlarını değil, aynı zamanda cümle içindeki konumlarını ve hangi alt cümlenin parçası olduklarını da göz önünde bulundurur. Bu bağlamda, BERT embedding işlemi üç ana bileşenden oluşur:</p><ol><li><strong>Token Embeddings</strong></li></ol><p>Öncelikle, bu aşamada tokenization işlemini kısaca açıklamak istiyorum. BERT modeli, öğrenme sürecinde büyük bir veri seti üzerinde ön eğitime tabi tutuldu. Tokenization işleminde ise her bir kelime veya alt kelime, 0&#39;dan başlayarak belirli bir numara verildi ve bu numaralandırma, tüm kelime havuzu tükenene kadar devam etti. Artık yeni bir kelime geldiğinde, o kelimenin bu numara ile temsil edilmesine <strong>tokenization</strong> işlemi denir.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*dBiNQa70gl2tmKWm.png" /></figure><p><strong>Token embedding</strong> işleminde, örneğin BERT-base modeli üzerinden düşünürsek, bu model 768 adet gizli katmana sahiptir. Yani, modele 1 boyutlu bir girdi verdiğinizde, bu girdiyi 768 boyutlu bir vektör olarak çıktı verir. <strong>Token Embedding</strong> tam da burada devreye girer. Bu işlem, 0&#39;dan başlayarak belirlenen sayıların (tokenların), modelin ön eğitimi sonrasında 768 boyutlu bir vektör olarak temsil edilmesini sağlar.</p><p><strong>Örnek</strong></p><p>1.</p><ul><li><strong>Token 1:</strong> “Merhaba” (veya “Mer”, “##haba”)</li><li><strong>Token 2:</strong> “dünya”</li></ul><p>2.</p><ul><li>“Merhaba” token’ına <strong>ID: 1050</strong></li><li>“dünya” token’ına <strong>ID: 2034</strong><br>olarak atama yapılabilir.</li></ul><p>3.</p><ul><li>Bu ID değerlerinin BERT modeline girdi olarak verilmesi ve 768&#39;lik iki vektör ortaya çıkması.</li></ul><p><strong>2. Segment Embeddings</strong></p><p>BERT modelinde, örneğin iki ayrı cümle veya metin parçası olduğunu düşünelim. Her bir tokena, o tokenın ait olduğu segmenti (cümleyi) belirlemek için ekstradan 768 boyutlu bir vektör eklenir. Bu <strong>segment embedding</strong> işleminin amacı, iki cümleyi birbirinden ayırmaktır.</p><p>Örneğin, şu iki cümleyi ele alalım:</p><ul><li><strong>Cümle 1:</strong> “Onunla görüşmek istemiyorum.”</li><li><strong>Cümle 2:</strong> “Herkese iyi günler dilerim.”</li></ul><p>Bu durumda, Cümle 1&#39;deki tokenlara segment olarak <strong>1</strong> numarası verilir ve bu, modeldeki 768 boyutlu vektör çıktısına yansıtılır. Benzer şekilde, Cümle 2&#39;deki tokenlara ise segment olarak <strong>2</strong> numarası verilir. Bu sayede model, iki cümleyi birbirinden ayırt edebilir ve her bir tokenın hangi cümleye ait olduğunu anlayabilir.</p><p><strong>3. Positional Embeddings</strong></p><p><strong>Segment Embedding</strong>, cümlelerin konumlanmasını veya sırasını belirlerken, <strong>Positional Embedding</strong> ise bir token’ın cümle veya metin parçası içindeki sırasını temsil eder.</p><p>Örneğin, şu cümleyi ele alalım:<br><em>“Bugün günlerden pazar.”</em></p><p>Bu cümlenin tokenlarını <em>“Bugün”</em>, <em>“günlerden”</em> ve <em>“pazar”</em> olarak düşünelim. Bu tokenların <strong>positional embedding</strong> değerleri, cümledeki sıralarına göre belirlenir. Yani:</p><ul><li><em>“Bugün”</em> → 1</li><li><em>“günlerden”</em> → 2</li><li><em>“pazar”</em> → 3</li></ul><p>Bu sıra numaraları, BERT-base modelinin çıktısı olan 768 boyutlu bir vektöre dönüştürülür. Bu sayede model, her bir token’ın cümle içindeki konumunu anlayabilir ve bu bilgiyi işlemlerinde kullanır.</p><p>Evet, üç farklı embedding yönteminden bahsettik. BERT Embedding’in genel formülasyonu, her biri 768 boyutlu vektörler olan bu embeddinglerin toplamıyla oluşur. Formül şu şekildedir:</p><blockquote><strong>BERT Embedding = Token Embedding + Segment Embedding + Positional Embedding</strong></blockquote><p>Bu üç embedding türünün toplamı, BERT modelinin her bir token için oluşturduğu nihai vektör temsilini (embedding) ortaya çıkarır.</p><h3>Kapanış</h3><p>Evet, blogumuzun sonuna geldik. Umarım yazıyı beğenmişsinizdir. Olumlu veya olumsuz, her türlü geri dönüşe açığım. Ayrıca, gelecekte ne tarz içerikler üretilebileceği konusunda da önerilerinizi bekliyorum. Sağlıcakla kalın, bir sonraki blogta görüşmek üzere!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=78c0ab069916" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Evaluate and Debug Your GenAI Model | Part 1 — Introduction to W&B]]></title>
            <link>https://halil7hatun.medium.com/evaluate-and-debug-your-genai-model-part-1-introduction-to-w-b-8ac2d91a1e00?source=rss-a71bf613874c------2</link>
            <guid isPermaLink="false">https://medium.com/p/8ac2d91a1e00</guid>
            <category><![CDATA[wandb]]></category>
            <category><![CDATA[llmops]]></category>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[evaluate]]></category>
            <dc:creator><![CDATA[Halil İbrahim Hatun]]></dc:creator>
            <pubDate>Fri, 03 Jan 2025 06:53:15 GMT</pubDate>
            <atom:updated>2025-01-03T06:53:15.872Z</atom:updated>
            <content:encoded><![CDATA[<h3>Evaluate and Debug Your GenAI Model | Part 1 — Introduction to W&amp;B</h3><p>This tutorial was designed using the Weights &amp; Biases (W&amp;B) Library.</p><p>Hello, ı hope everything is going well 😊. Today is so exciting for me 🎉. I am starting a new tutorial that touches on how we can monitor our LLM or GenAI model during fine-tuning, serving, and evaluating 💡. These kinds of cases are crucial nowadays because due to AI, everything is becoming so easier to handle day by day. However, evaluating and monitoring part is still extremely important to specify customer market, average cost, and so on. Many different kinds of apps help us in this kind of situation. I prefer to continue with <strong>Weight &amp; Biases (W&amp;B)</strong> which is more effective than others, ı think.</p><p>Let’s start our tutorial by mentioning how can we log our model during training.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q9k7uPvaueOUp_x42kwf1g.png" /></figure><h3>What is W&amp;B?</h3><p>Weights &amp; Biases (W&amp;B) is your new best friend in the world of machine learning. It’s like having a personal assistant who keeps track of all your experiments, visualizes your model’s performance, and even helps you collaborate with your team. Imagine having a superhero sidekick who makes sure your AI adventures are smooth and successful.</p><h4>Key Features of W&amp;B:</h4><ol><li><strong>Experiment Tracking</strong>: Never lose track of your experiments again. W&amp;B logs and tracks everything, so you can compare runs and figure out what’s working (and what’s not).</li><li><strong>Visualization</strong>: See your model’s performance metrics, loss curves, and more in real time. It’s like having a crystal ball for your AI projects.</li><li><strong>Collaboration</strong>: Share your experiments with your team, get feedback, and work together more effectively. Teamwork makes the dream work, right?</li><li><strong>Version Control</strong>: Keep tabs on different versions of your models and datasets. No more “Wait, which version was this again?” moments.</li><li><strong>Integration</strong>: W&amp;B plays nicely with popular machine learning frameworks like PyTorch and TensorFlow. It’s like the social butterfly of AI tools.</li></ol><h3>Why Use W&amp;B?</h3><p>Using W&amp;B can supercharge your workflow. Here’s how:</p><ul><li><strong>Efficiency</strong>: Automate the logging of your experiments and say goodbye to manual errors. It’s like having a robot butler for your AI projects.</li><li><strong>Insight</strong>: Get deep insights into your model’s performance with detailed visualizations and analytics. Knowledge is power!</li><li><strong>Collaboration</strong>: Work better with your team by sharing and discussing experiments. Two heads are better than one, after all.</li><li><strong>Reproducibility</strong>: Ensure your experiments are reproducible, making debugging and improvements a breeze. No more “It worked on my machine” excuses.</li></ul><p>Okay, enough theory. Let’s dive in and have some fun!</p><h3>Let’s Create a Basic MLP Model and Log it with W&amp;B for Sprite Classification</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/745/1*DNQiFmhtEnfz4abtsCAKRA.png" /></figure><h4>Step 1: Define Our Libraries</h4><p>First things first, let’s import the necessary libraries:</p><pre>import math<br>from pathlib import Path<br>from types import SimpleNamespace<br>from tqdm.auto import tqdm<br>import torch<br>import torch.nn as nn<br>import torch.nn.functional as F<br>from torch.optim import Adam<br>from utilities import get_dataloaders<br><br>import wandb</pre><h4>Step 2: Define Constants and Create the get_model Method</h4><p>Let’s set up our constants and build a simple model:</p><pre>INPUT_SIZE = 3 * 16 * 16<br>OUTPUT_SIZE = 5<br>HIDDEN_SIZE = 256<br>NUM_WORKERS = 2<br>CLASSES = [&quot;hero&quot;, &quot;non-hero&quot;, &quot;food&quot;, &quot;spell&quot;, &quot;side-facing&quot;]<br>DATA_DIR = Path(&#39;./data/&#39;)<br>DEVICE = torch.device(&quot;cuda&quot; if torch.cuda.is_available()  else &quot;cpu&quot;)<br><br>def get_model(dropout):<br>    &quot;Simple MLP with Dropout&quot;<br>    return nn.Sequential(<br>        nn.Flatten(),<br>        nn.Linear(INPUT_SIZE, HIDDEN_SIZE),<br>        nn.BatchNorm1d(HIDDEN_SIZE),<br>        nn.ReLU(),<br>        nn.Dropout(dropout),<br>        nn.Linear(HIDDEN_SIZE, OUTPUT_SIZE)<br>    ).to(DEVICE)</pre><h4>Step 3: Define Hyperparameters</h4><p>Let’s store our hyperparameters in a config object:</p><pre>config = SimpleNamespace(<br>    epochs = 2,<br>    batch_size = 128,<br>    lr = 1e-5,<br>    dropout = 0.5,<br>    slice_size = 10_000,<br>    valid_pct = 0.2,<br>)</pre><h4>Step 4: Define Train and Evaluate Methods</h4><p>Now, let’s define our training and evaluation methods:</p><pre>def train_model(config):<br>    &quot;Train a model with a given config&quot;<br>    <br>    wandb.init(<br>        project=&quot;dlai_intro&quot;,<br>        config=config,<br>    )<br><br>    # Get the data<br>    train_dl, valid_dl = get_dataloaders(DATA_DIR, <br>                                         config.batch_size, <br>                                         config.slice_size, <br>                                         config.valid_pct)<br>    n_steps_per_epoch = math.ceil(len(train_dl.dataset) / config.batch_size)<br><br>    # A simple MLP model<br>    model = get_model(config.dropout)<br><br>    # Make the loss and optimizer<br>    loss_func = nn.CrossEntropyLoss()<br>    optimizer = Adam(model.parameters(), lr=config.lr)<br><br>    example_ct = 0<br><br>    for epoch in tqdm(range(config.epochs), total=config.epochs):<br>        model.train()<br><br>        for step, (images, labels) in enumerate(train_dl):<br>            images, labels = images.to(DEVICE), labels.to(DEVICE)<br><br>            outputs = model(images)<br>            train_loss = loss_func(outputs, labels)<br>            optimizer.zero_grad()<br>            train_loss.backward()<br>            optimizer.step()<br><br>            example_ct += len(images)<br>            metrics = {<br>                &quot;train/train_loss&quot;: train_loss,<br>                &quot;train/epoch&quot;: epoch + 1,<br>                &quot;train/example_ct&quot;: example_ct<br>            }<br>            # To log training metrics on W&amp;B dashboard<br>            wandb.log(metrics)<br>            <br>        # Compute validation metrics, log images on last epoch<br>        val_loss, accuracy = validate_model(model, valid_dl, loss_func)<br>        # Compute train and validation metrics<br>        val_metrics = {<br>            &quot;val/val_loss&quot;: val_loss,<br>            &quot;val/val_accuracy&quot;: accuracy<br>        }<br>        # To log validation metrics on W&amp;B dashboard<br>        wandb.log(val_metrics)<br>     <br>    # Ending process<br>    wandb.finish()</pre><pre>def validate_model(model, valid_dl, loss_func):<br>    &quot;Compute the performance of the model on the validation dataset&quot;<br>    model.eval()<br>    val_loss = 0.0<br>    correct = 0<br><br>    with torch.inference_mode():<br>        for i, (images, labels) in enumerate(valid_dl):<br>            images, labels = images.to(DEVICE), labels.to(DEVICE)<br><br>            # Forward pass<br>            outputs = model(images)<br>            val_loss += loss_func(outputs, labels) * labels.size(0)<br><br>            # Compute accuracy and accumulate<br>            _, predicted = torch.max(outputs.data, 1)<br>            correct += (predicted == labels).sum().item()<br>            <br>    return val_loss / len(valid_dl.dataset), correct / len(valid_dl.dataset)</pre><h4>Step 5: Configure W&amp;B and Bake Our Model</h4><p>Before we shine, let’s configure W&amp;B. You can see your logs without an account, or use your W&amp;B API key to save them.</p><p>If you want to continue without signing in, enter 1. If not, enter 2 and paste your W&amp;B API key. You can find it <a href="https://wandb.ai/settings#api"><strong><em>here</em></strong></a>. I&#39;ll continue with my API key.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vlqlADYbgQrqdHvNVj4_yw.png" /></figure><p>And… Time to bake our model!</p><pre>train_model(config)</pre><p>Our baked model is ready! You can see your logs in Jupyter Notebook. But that’s not all. You can also access your project link that comes after “View Project.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tNzZUKvmw_Va82fvtI1kqg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lhTaxxndILWrMiO-EPjOCQ.png" /></figure><p>We trained our model and saw our logs on our accounts. However, we often don’t get what we want in one training process. Let’s do other training processes by changing the learning rate.</p><pre>config.lr = 1e-4<br>train_model(config)<br><br>config.lr = 1e-4<br>train_model(config)<br><br>config.dropout = 0.1<br>config.epochs = 1<br>train_model(config)<br><br>config.lr = 1e-3<br>train_model(config)</pre><p>After running these commands, we trained four models with different hyperparameters. When you click on our project on W&amp;B again, you can compare them like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JfPKTOvlo_iDO03n2fbrYw.png" /></figure><p>That’s it! We have trained many models and compared them. Therefore, we can easily decide which hyperparameter configuration is more suitable.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vXfuDzLpSPvP0WC275KOAA.png" /></figure><h3>Conclusion</h3><p>In this blog, we introduced W&amp;B by training simple MLP models with different hyperparameters and logged their metrics into W&amp;B.</p><p>This is just the beginning. I will continue with training diffusion models, evaluating them, tracing our LLM models, and fine-tuning them by always using W&amp;B.</p><p>Your responses are so valuable for me to continue this tutorial. Play the waiting game. See you soon, bye-bye!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8ac2d91a1e00" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exploring DataGemma: An Overview]]></title>
            <link>https://halil7hatun.medium.com/exploring-datagemma-an-overview-89d0e369e562?source=rss-a71bf613874c------2</link>
            <guid isPermaLink="false">https://medium.com/p/89d0e369e562</guid>
            <category><![CDATA[gemini]]></category>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <category><![CDATA[gemma]]></category>
            <category><![CDATA[datagemma]]></category>
            <category><![CDATA[google]]></category>
            <dc:creator><![CDATA[Halil İbrahim Hatun]]></dc:creator>
            <pubDate>Fri, 13 Sep 2024 13:55:12 GMT</pubDate>
            <atom:updated>2024-09-13T13:57:16.270Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tl1jtg-ytt7uG_EY6onurg.jpeg" /><figcaption>Figure 1. DataGemma Logo</figcaption></figure><p>Despite the advancements in large language models (LLMs), AI hallucinations remain a significant challenge. On September 12, Google made a major stride by releasing DataGemma as open source. DataGemma leverages real-world data to tackle these hallucinations, reflecting Google’s commitment to addressing this issue. In this blog, I will provide an overview of DataGemma and explore two distinct approaches used to improve LLM accuracy and reasoning.</p><h3>What is Data Commons</h3><p>Google’s Data Commons project serves as a vast repository of public data, designed to streamline the access and use of important global statistics. It consolidates information from a wide range of trusted sources, including the United Nations, government agencies, environmental organizations, and universities. With over 250 billion data points and more than 2.5 trillion triples, it represents a significant open-source initiative aimed at making global data more accessible and useful.</p><p>Data Commons features two notable innovations. First, it has dedicated years to curating diverse public datasets, understanding their underlying assumptions, and organizing them using Schema.org, a universal language for structured data. This effort results in a comprehensive Knowledge Graph that integrates data from various sources.</p><p>Second, Data Commons incorporates a natural language interface powered by large language models (LLMs). This allows users to pose questions in everyday language, with the LLM translating these queries into the Data Commons’ format. This interface facilitates the exploration of charts and graphs without altering or fabricating the underlying data.</p><h3>Interfacing LLMs with Data Commons</h3><p>Two different approaches have been described for interfacing LLMs with Data Commons.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/903/1*bRY5hAosE-AA3xWXxxitJw.png" /><figcaption>Figure 2. Comparison of Baseline, RIG, and RAG approaches for generating responses with statistical data</figcaption></figure><p>The first approach, called Retrieval Interleaved Generation (RIG), fine-tunes the LLM to not only generate natural language queries but also pull stats from Data Commons. It uses a multi-step pipeline to convert these into structured data queries. We then compare this to how the base models, Gemma 7B IT and 27B IT, perform.</p><p>The second approach, Retrieval Augmented Generation (RAG), takes a more classic retrieval method. It extracts variables from the query, grabs relevant data, and adds context to the original question. Then it produces an answer using an LLM (Gemini 1.5 Pro), which we use for comparison against the baseline results.</p><h4>Retrieval Interleaved Generation (RIG)</h4><p>Retrieval Interleaved Generation (RIG) is a three-step process designed to enhance the accuracy and reliability of language model responses. First, a fine-tuned model generates natural language queries for Data Commons. Next, a post-processor converts these queries into structured data formats. Finally, the system retrieves statistical answers from Data Commons and presents them alongside the original LLM-generated results.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fplayer.vimeo.com%2Fvideo%2F1009179191%3Fapp_id%3D122963&amp;dntp=1&amp;display_name=Vimeo&amp;url=https%3A%2F%2Fvimeo.com%2F1009179191%3Fshare%3Dcopy&amp;image=https%3A%2F%2Fi.vimeocdn.com%2Fvideo%2F1926331344-44306034b89ec510d9aebe77559abb8c0da0753ecaa20e3c781af88daadef958-d_1280&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=vimeo" width="1440" height="810" frameborder="0" scrolling="no"><a href="https://medium.com/media/87680ebcc96ea4b1fe1bed4e23136c61/href">https://medium.com/media/87680ebcc96ea4b1fe1bed4e23136c61/href</a></iframe><p>In this process, when the LLM provides a numerical answer, it is matched with the most relevant value from the Data Commons database, known as the Data Commons Statistical Value (DC-SV). The original output from the LLM is referred to as the LLM Statistical Value (LLM-SV). Instead of generating formal queries like SQL, the LLM is fine-tuned to produce natural language queries. This method is more efficient given the vast array of variables in Data Commons and helps maintain the natural and fluent quality of the model’s responses.</p><p><strong>Query Conversation Part</strong></p><p>In their pipeline, natural language queries generated by the LLM are transformed into structured queries for the Data Commons database. Despite the extensive range of variables and properties in Data Commons, most queries can be categorized into a few types, which streamlines the extraction process. Each query is broken down into key components: statistical variables or topics, places, and attributes. Specific NLP techniques are applied to these components: semantic search for variables, named entity recognition for places, and regex-based heuristics for attributes.</p><p>The identified components are then mapped to predefined query templates, as illustrated in the table below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f3rF9-99rGxZquNUE_I4dQ.png" /><figcaption>Figure 4. Predefined query templates for RIG approach</figcaption></figure><p>Structured queries are generated based on these templates and submitted to the Data Commons API. The response, typically a numeric value, is presented alongside the original LLM-generated statistic, facilitating verification of the LLM’s output. Future developments will explore various presentation methods for these results, including side-by-side comparisons and highlighted differences.</p><h4><strong>Retrieval Augmented Generation (RAG)</strong></h4><p>In the RAG pipeline, the process begins with a fine-tuned LLM managing the user’s query. This model generates relevant queries for Data Commons, which are used to retrieve pertinent tables from the Data Commons interface. Finally, a long-context LLM, such as Gemini 1.5 Pro, generates a response based on both the original query and the retrieved tables.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fplayer.vimeo.com%2Fvideo%2F1009183444%3Fapp_id%3D122963&amp;dntp=1&amp;display_name=Vimeo&amp;url=https%3A%2F%2Fvimeo.com%2F1009183444%3Fshare%3Dcopy&amp;image=https%3A%2F%2Fi.vimeocdn.com%2Fvideo%2F1926340180-c70249652eec9a7be236f572ad03fadb2aceaa7c2d801906e0aa289336d4a8c5-d_1280&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=vimeo" width="1440" height="810" frameborder="0" scrolling="no"><a href="https://medium.com/media/f1091fab536e11ccbdac04fa898ffd91/href">https://medium.com/media/f1091fab536e11ccbdac04fa898ffd91/href</a></iframe><p><strong>Extracting Data Commons Queries</strong></p><p>An LLM is fine-tuned to transform user queries into Data Commons queries. Training utilizes Gemini 1.5 Pro to generate queries in specific formats. Although the effectiveness can be limited by data availability, the initial method generally provides better results compared to alternatives that use a full variable list.</p><p><strong>Retrieving Tables</strong></p><p>Queries are processed using the RIG framework to identify variables and map them to Data Commons APIs. These APIs return relevant tables, such as life expectancy by country, which are used for generating responses.</p><p><strong>Prompting</strong></p><p>Once the tables are retrieved, a prompt is created combining the original query and serialized table data. This prompt is then processed by long-context LLMs like Gemini 1.5 Pro to generate and return a comprehensive response.</p><h3>Conclusion</h3><p>I hope this information proves helpful. I plan to continue exploring DataGemma, focusing on evaluating RIG and RAG approaches and their implementation in code.</p><p>Stay tuned for the next updates on this topic. Take care!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=89d0e369e562" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cultivating Hospitality Excellence: Transformative Applications for Efficient Staff Management in…]]></title>
            <link>https://halil7hatun.medium.com/cultivating-hospitality-excellence-transformative-applications-for-efficient-staff-management-in-dd71d4fbe63c?source=rss-a71bf613874c------2</link>
            <guid isPermaLink="false">https://medium.com/p/dd71d4fbe63c</guid>
            <category><![CDATA[hotel]]></category>
            <category><![CDATA[hotel-industry]]></category>
            <category><![CDATA[hotel-technology]]></category>
            <category><![CDATA[hotel-booking]]></category>
            <category><![CDATA[hotel-management]]></category>
            <dc:creator><![CDATA[Halil İbrahim Hatun]]></dc:creator>
            <pubDate>Fri, 12 Apr 2024 10:24:30 GMT</pubDate>
            <atom:updated>2024-04-12T10:28:41.642Z</atom:updated>
            <content:encoded><![CDATA[<h3>Cultivating Hospitality Excellence: Transformative Applications for Efficient Staff Management in the Hotel Industry</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Nsx8gHxVyyVqjyIB37vd2w.jpeg" /><figcaption>Image 1 — Driving Hotel Technology</figcaption></figure><p>Today, I’d like to discuss optimizing staff management to enhance hospitality in hotels. It’s important to note that the information I’ll be sharing reflects my opinion or is sourced from various references.</p><h4><strong>General Based Apps</strong></h4><p>In the vast realm of mobile apps, numerous solutions address this question. I’ll list some of them and offer insights for better understanding</p><ol><li><strong>İci4Stuff</strong></li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*S-GZc_P_04WHOXwiYjeqRA.jpeg" /><figcaption>Image 2 — İci4Stuff</figcaption></figure><p>Let’s delve into İcibot’s StaffApp, named “İci4Staff”<em> [1]</em>. Aligned with the common goal of enhancing hospitality, its primary aim is to offer swift support via mobile devices. Should a customer encounter any inquiries or issues, they can effortlessly relay them through the mobile app. These matters are then promptly assigned to staff members, who can conveniently access them on their mobile devices.</p><p><strong>2. Smart Parking</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*UUd9ah3IOjX_9PXa_rQ7-A.jpeg" /><figcaption>Image 3 — Smart Parking</figcaption></figure><p>Let’s further explore the concept of smart parking<em> [2]</em>. As you’re aware, even within hotels, securing parking spots can be a significant bottleneck. To address this, computer vision technology can guide staff to the most optimal parking areas, ensuring efficient space utilization.</p><p><strong>3. Webee</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/580/1*pYFUbEGYyj2jNGgJDiDOBQ.jpeg" /><figcaption>Image 4 — Webee</figcaption></figure><p>Webee is one of the most widely used hotel management apps in Türkiye <em>[3]</em>, serving a purpose similar to İcibot.</p><h4>CRM and Hospitality</h4><p>The apps I mentioned earlier primarily focus on the broader aspects of hotel management. Now, let’s delve into other applications specifically dedicated to CRM, guest optimization, and hospitality.</p><ol><li><strong>Beonx</strong></li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/655/1*Y_dulAEkQFzwWzzJcbkEOw.jpeg" /><figcaption>Image 5 — Dashboard page of BeonX</figcaption></figure><p>One notable example is Beonx<em> [4]</em>, which prioritizes sustainable profitability. They employ AI-powered strategies such as Revenue Optimization, Hyper Segmentation, Real-Time Data and Automation, Demand Forecasting, and more to optimize hotel costs.</p><p><strong>2. Bookboost</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XaoQnSoipxwe3HMXsbfuyw.png" /><figcaption>Image 6 — Dashboard page of Bookboost</figcaption></figure><p>It’s recognized as a Multi-channel CRM system <em>[5]</em>. Its objectives include increasing revenues, refining marketing operations, and boosting customer loyalty through the creation of individual customer profiles and the enhancement of conversation campaigns.</p><h4>Personal Opinion</h4><p>In my view, CRM stands out as the most potent method for optimizing revenues through personalized service and prompt feedback. Consequently, there’s a need to enhance data collection methods to gain deeper insights into customers’ behaviors.</p><h4>Conclusion</h4><p>In summary, hospitality is paramount in the hotel industry, and there are numerous approaches to address it. The key lies in our intent and execution. I aimed to convey essential information concisely. I hope this brief blog proved helpful to you.</p><p>Take care, and stay tuned for more in upcoming blogs.</p><h3>References</h3><ol><li><a href="https://icibot.com/en/platforms/ici4staff/">https://icibot.com/en/platforms/ici4staff/</a></li><li><a href="https://medium.com/debutinfotech/mobile-apps-hospitality-a-high-tech-makeover-on-the-move-ceee9d71b771">https://medium.com/debutinfotech/mobile-apps-hospitality-a-high-tech-makeover-on-the-move-ceee9d71b771</a></li><li><a href="https://www.getwebee.com/tr/webee-cozumler">https://www.getwebee.com/tr/webee-cozumler</a></li><li><a href="https://www.beonx.com/">https://www.beonx.com/</a></li><li><a href="https://www.bookboost.io/multi-channel-hospitality-crm">https://www.bookboost.io/multi-channel-hospitality-crm</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dd71d4fbe63c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Taking a Broad View of the Israel-Hamas Conflict]]></title>
            <link>https://halil7hatun.medium.com/taking-a-broad-view-of-the-israel-hamas-conflict-f76363269720?source=rss-a71bf613874c------2</link>
            <guid isPermaLink="false">https://medium.com/p/f76363269720</guid>
            <category><![CDATA[west-bank]]></category>
            <category><![CDATA[hamas]]></category>
            <category><![CDATA[gaze]]></category>
            <category><![CDATA[jerusalem]]></category>
            <category><![CDATA[israel]]></category>
            <dc:creator><![CDATA[Halil İbrahim Hatun]]></dc:creator>
            <pubDate>Fri, 20 Oct 2023 09:33:25 GMT</pubDate>
            <atom:updated>2023-10-20T12:57:14.969Z</atom:updated>
            <content:encoded><![CDATA[<p>As many are aware, the conflict between Israel and Hamas has reignited once again. I’ve delved into research and analysis on this critical issue. Today, my aim is to discuss these matters. We’ll navigate through the complexities, aiming for a clear and balanced perspective.</p><p>Alright, let’s delve into the intricate history of the Palestine-Israel relationship. It’s a subject that demands our attention and consideration.</p><h4>History of Israel-Palestine Relations: A Swift Overview</h4><p>The relationship between Israel and Palestine is a complex historical narrative. It begins with ancient Hebrew settlements around 1300 BCE, followed by various empires’ rule, including the Ottomans from the 16th century to World War I. After WWI, British control was established, leading to complications.</p><p>Zionism, a movement for a Jewish homeland, gained traction in the late 19th century, backed by the Balfour Declaration in 1917. The UN proposed a partition plan in 1947, resulting in Israel’s declaration of independence in 1948 and a subsequent war.</p><p>The conflict led to the Nakba, displacing many Palestinians. The aftermath of this war continues to influence the region’s politics. The Six-Day War in 1967 further complicated matters, leading to Israeli occupation of the West Bank, Gaza Strip, and East Jerusalem.</p><p>Though there have been attempts at peace, such as the Oslo Accords in the 1990s, finding a lasting solution remains challenging due to deep-rooted historical, religious, and political differences. Understanding this complex history is crucial for meaningful discussions about the region’s present and future.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/933/1*iAsX48lZrTfpgjmZHvRxaQ.jpeg" /></figure><h3>A Deeper Dive into the Israel-Palestine Conflict</h3><h4>1. Israeli Side</h4><p>Support for Israel is prevalent, particularly in the United States. The majority of the American government has historically aligned itself with Israel. However, it’s worth noting that many young Americans, especially teenagers, might not possess in-depth knowledge about the intricacies of the conflict. Often, they rely on what they’ve been taught or what they hear in passing.</p><p>Interestingly, within this pro-Israel camp, there are cases of Palestinians who also lend their support to Israel and advocate for its cause. One notable example is Ali Wahap, an Arab Muslim who has publicly expressed this viewpoint. Let’s look the text Ali Wahab said:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/598/1*zF-hUe9fagHXnm40C079rA.png" /></figure><p>As we embark on this journey of comprehension, let’s start with a concise overview. In the forthcoming blogs, we’ll plunge into the intricacies. One term that sparks curiosity in Ali Wahab’s discourse is “apartheid.” It raises an important question: can Israel be labeled as an apartheid state? To tackle this, we’ll dissect diverse viewpoints and meticulously assess each stance.</p><p><strong>Envisioned opinion</strong></p><p>Those who have experienced Israel firsthand, whether through residency or visits, often contest the label of “apartheid state” attributed to it. They argue that Israel, as a nation, upholds a policy of equal rights for all its citizens, irrespective of their race, religion, ethnicity, gender, or sexual orientation. This perspective insists that any notion of Israel as an apartheid state is a gross mischaracterization.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/469/1*HWEA04pvSZx2o1VkTWma1g.png" /></figure><p>Critics, however, interpret these claims differently. They assert that anti-Israel sentiments are rooted in a desire for the dismantling of Israel itself. To them, branding Israel as an apartheid state signifies the separation between Israelis and Palestinians, and is seen as a strategic move toward eradicating Israel in favor of a unified Palestine. This would entail the return of descendants of refugees and a shift in the demographic majority towards Muslims.</p><p>Within Israel, there exists a significant Arab population who hold citizenship, affording them equal rights. Notably, these Arab citizens are not subject to compulsory military service. It is important to note that, due to the absence of a civil marriage law, individuals of different religions cannot legally marry within Israel. Instead, marriage laws are rooted in the Ottoman era. However, marriages conducted abroad under civil law are recognized within the country.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*_UvpWsuWP0iOzQZbAHURRA.png" /></figure><p><strong>Waking up from an envisioned opinion (fable)</strong></p><p>That really depends on how you want to define apartheid.</p><p>The frustrating thing about the apartheid debate is that the two sides argue completely different points.</p><p>Those who want to paint Israel as the oppressor point to the West Bank. Those who want to point to Israel as “the only democracy in the Middle East” point to Palestinian citizens of Israel.</p><p>It’s true that Palestinian Israelis can, and do, serve as lawyers, doctors, teachers, policemen, politicians and judges. This is not to say that there is no discrimination in Israel, because of course there is, but to call <em>this </em>apartheid is just silly. Discrimination is not apartheid.</p><p>So, the more relevant issue is that of the West Bank.</p><p>The West Bank is split into 3 (non-contiguous) areas, A, B and C, set out in the Oslo Accords.</p><p>Area A is under complete Palestinian administration, and comprises the major Palestinian urban centres. It makes up about 18% of the West Bank in area.</p><p>Area B is under Palestinian civil authority and Israeli military authority, and makes up about 22% of the West Bank.</p><p>Neither area A nor area B have Jewish residents, and therefore we cannot really talk about apartheid there.</p><p>Area C is where the issue really lies. Some 60% of the West Bank, populated both by Israeli Jews and by Palestinians. This area is under full Israeli control.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ne2Nac9X_y1Ct8ENcxJ7qg.jpeg" /></figure><p>There is no arguing the fact that Israeli and Palestinian residents of Area C are not equal.</p><p>Israeli Jews are full and equal Israeli citizens, beholden to, and served by, the legal systems of Israel.</p><p>Palestinians are not Israeli citizens, and are beholden to Israeli <em>military </em>courts. They do not generally have recourse to the Israeli civil courts (and indeed, just this week a law was passed that further restricts Palestinian access to the High Court in land disputes). They are policed by the IDF, rather than by police forces, which in general means that they do not benefit from police protection, but are only restricted by it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/798/1*9GeCM2Uu5VHGlFC8SbzDmQ.png" /></figure><p>This is the basis of claims of apartheid — you have Jews and Arabs living side by side, under different legal jurisdictions.</p><p>Is this apartheid? As I said, it depends on how you want to define apartheid. You can definitely define it broadly, in a way which seems to include this situation — there is indeed a legal distinction between two populations, which plays out along racial/ethnic lines.</p><p>The counter argument to this is, of course, that it is not a racial or ethnic distinction, it is a perfectly normal distinction between citizens and non citizens. All non-citizens are not equal to citizens, in every country. Palestinian citizens who go to area C are still full citizens.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/617/1*eCms0sI_NuZRoBjVZ_Ajyg.png" /></figure><p>The counter-counter-argument could claim that this is something of a facetious claim, in that citizenship was never granted to the Palestinian residents of the West Bank, whereas the Jewish residents have citizenship based on their ethnicity, and indeed any Jew in the entire world could come tomorrow and receive citizenship. If citizenship is ethnically biased, then a distinction based on citizenship is ethnically biased too.</p><p>And there are counter-counter-counter arguments, and counter-counter-counter-counter arguments. It goes on for ever.</p><p>(If you’re keen on delving deeper into this topic, I highly recommend watching<a href="https://www.youtube.com/watch?v=aEdGcej-6D0&amp;t=1037s"><strong> this video</strong></a><strong>.</strong> It provides valuable insights and a nuanced perspective on the matter.)</p><h4>2. Hamas Side</h4><p><em>What is Hamas?</em> Alright, so, Hamas is a Palestinian political and military organization. They’ve been a significant player in the Israeli-Palestinian conflict for quite some time now. They emerged in the late 1980s and have since gained considerable support, especially in the Gaza Strip.</p><p>Now, when it comes to their actions against Israel, things get pretty intense. Hamas has been involved in a number of conflicts and acts of violence. They’ve launched rockets and carried out suicide bombings, which have resulted in both civilian and military casualties on the Israeli side.</p><p>These actions have led to a lot of tension and responses from Israel, including military operations in the Gaza Strip. It’s safe to say that Hamas and Israel have a long history of conflict, and their interactions continue to shape the dynamics of the region.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H0Kck_SmQEAnV-zKElZcig.jpeg" /></figure><p><em>What is Hamas’ authority in Gaza? </em>Hamas isn’t a political party, it’s not running a state and it’s not a religious cult either. Hamas is an organized crime ring, anchored in Islamic fundamentalism, that is also performing the duties of a state in Gaza. In other words, it is an unholy alliance between mafia and religious wackos. As such it has complete control over the distribution of goods in Gaza. Those who do not work for Hamas live in abject squalor, being a member is basically a prerequesite to hold a job, with very few exceptions. Those who oppose Hamas die a violent death. The result is most people either work for or with Hamas and no one is willing to oppose them, it’s too dangerous and spies are everywhere.</p><h3>Conclusion</h3><p>The human toll in this conflict is immense, with lives lost on both sides. What’s even more disheartening is the passive stance taken by many powerful nations, who either watch this tragedy unfold or make hollow and insensitive statements. It’s a painful truth that nothing in this world is as precious as the smile of a child, and yet, the innocence of many is being ruthlessly extinguished. This is neither a matter for negotiation nor a case for defense.</p><p>Our hearts ache for the countless innocent lives lost in this conflict, and we yearn for a sense of peace and reprieve for all those affected, regardless of their background or affiliation.</p><p>In my forthcoming blogs, I’ll delve deeper into the political complexities of this conflict, examining the various perspectives and avenues for resolution. I remain open to all responses and negotiations, and I fervently wish for a swift end to this ongoing struggle. Until then, take care.</p><p>Now, here’s where it gets interesting. Within the Palestinian community, there were some who, surprisingly, supported Israel and advocated for its cause. Take, for example, Ali Wahap, an Arab Muslim who made his stance clear. This just goes to show that even in the midst of a heated conflict, there were individuals who saw things from a different perspective.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f76363269720" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Graph Neural Networks (GNNs)]]></title>
            <link>https://halil7hatun.medium.com/graph-neural-networks-gnns-1f463df4bb77?source=rss-a71bf613874c------2</link>
            <guid isPermaLink="false">https://medium.com/p/1f463df4bb77</guid>
            <category><![CDATA[gnn]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[graph]]></category>
            <dc:creator><![CDATA[Halil İbrahim Hatun]]></dc:creator>
            <pubDate>Wed, 30 Aug 2023 10:51:33 GMT</pubDate>
            <atom:updated>2023-08-30T11:14:05.269Z</atom:updated>
            <content:encoded><![CDATA[<h4>GCN and GAT Python Implementation</h4><p>In the world of artificial intelligence, where data is the lifeblood that fuels innovation, there’s a particular type of data that often challenges traditional machine learning techniques, data with inherent relationships and connections. This is where Graph Neural Networks (GNNs) step in, revolutionizing the way we process and understand data structured as graphs.</p><p>As a unique non-Euclidean data structure for machine learning, graph analysis focuses on tasks such as node classification, link prediction, and clustering. In a world that can be visualized as networks of entities and their interactions, be it social networks, molecular structures, recommendation systems, or citation networks, traditional machine learning algorithms often fall short. Conventional models are designed for independent and identically distributed data, struggling to capture the nuances of interconnectedness and dependencies that define these complex relationships.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/905/1*R3j9VIb6WYKlAdqEJcB9xw.png" /><figcaption>Fig. 1. Left: image in Euclidean space. Right: graph in non-Euclidean space</figcaption></figure><p>Graph Neural Networks have emerged as a revolutionary approach, garnering significant attention in recent years due to their remarkable capacity to extract invaluable insights from interconnected data. Unlike traditional methods that focus solely on individual data points in isolation, GNNs operate by delving deep into the intricate tapestry of relationships. This unique approach empowers us to not only analyze data points but to unravel the concealed structures and intricate patterns that lay beneath the surface.</p><p>In this blog, our focus will be on dissecting the fundamental operational architecture of Graph Neural Networks (GNNs), coupled with a practical exploration of their implementation using the Python programming language. Through this journey, we aim to demystify the inner workings of GNNs, providing you with a clear understanding of how these networks navigate and make sense of interconnected data. So, let’s embark on a guided tour of the core concepts behind GNNs, while also rolling up our sleeves for some hands-on Python coding to bring these concepts to life.</p><h4>Getting Started</h4><p>Let’s examine the Planetoid Cora dataset and apply Graph Neural Networks (GNNs) using PyTorch. This practical exploration will provide us with hands-on experience working with real-world graph data.</p><p>The <strong>Planetoid dataset</strong> combines citation networks from Cora, CiteSeer, and PubMed. Nodes, representing documents, feature 1433-dimensional bag-of-words vectors, interconnected by citations. With 7 classes, the challenge involves training a model to predict missing labels using the web of connections.</p><pre>from torch_geometric.datasets import Planetoid<br>from torch_geometric.transforms import NormalizeFeatures<br><br>dataset = Planetoid(root=&#39;data/Planetoid&#39;, name=&#39;Cora&#39;, transform=NormalizeFeatures())<br><br>print(f&#39;Dataset: {dataset}:&#39;)<br>print(&#39;======================&#39;)<br>print(f&#39;Number of graphs: {len(dataset)}&#39;)<br>print(f&#39;Number of features: {dataset.num_features}&#39;)<br>print(f&#39;Number of classes: {dataset.num_classes}&#39;)<br><br>data = dataset[0]<br>print(data)</pre><pre>Output =<br><br>Dataset: Cora():<br>======================<br>Number of graphs: 1<br>Number of features: 1433<br>Number of classes: 7<br>Data(x=[2708, 1433], edge_index=[2, 10556], y=[2708], train_mask=[2708], val_mask=[2708], test_mask=[2708])</pre><p>Prior to initiating the training process, let’s analyze the data distribution by visually representing it within the second and third dimensions.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fplotly.com%2F%7Ehalilibr.09%2F14.embed%3Fautosize%3Dtrue&amp;display_name=Plotly&amp;url=http%3A%2F%2Fchart-studio.plotly.com%2F%7Ehalilibr.09%2F14%2F&amp;image=http%3A%2F%2Fchart-studio.plotly.com%2Fstatic%2Fwebapp%2Fimages%2Fplotly-logo.8d56a320dbb8.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=plotly" width="600" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/a0d49fc5a32ad377c9f2b1c7d83e5cc7/href">https://medium.com/media/a0d49fc5a32ad377c9f2b1c7d83e5cc7/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fplotly.com%2F%7Ehalilibr.09%2F21.embed%3Fautosize%3Dtrue&amp;display_name=Plotly&amp;url=https%3A%2F%2Fchart-studio.plotly.com%2F%7Ehalilibr.09%2F21%2F&amp;image=https%3A%2F%2Fchart-studio.plotly.com%2Fstatic%2Fwebapp%2Fimages%2Fplotly-logo.8d56a320dbb8.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=plotly" width="600" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/e897df33cb20a4bd50588c47b8a49f5a/href">https://medium.com/media/e897df33cb20a4bd50588c47b8a49f5a/href</a></iframe><p>Just one more step to go. Now, it’s time to write the classes and methods that will be employed in the upcoming sections:</p><pre>class BuildModel():<br>    def __init__(self, model, lr = 0.01):<br>        self.model = model<br>        self.optimizer = torch.optim.Adam(model.parameters(), lr=lr, weight_decay=5e-4)<br>        self.criterion = torch.nn.CrossEntropyLoss()<br><br>    def single_train(self):<br>          self.model.train()<br>          self.optimizer.zero_grad()<br>          out = self.model(data.x, data.edge_index)<br>          loss = self.criterion(out[data.train_mask], data.y[data.train_mask])<br>          loss.backward()<br>          self.optimizer.step()<br>          return loss<br><br>    def test(self):<br>          self.model.eval()<br>          out = self.model(data.x, data.edge_index)<br>          pred = out.argmax(dim=1)<br>          test_correct = pred[data.test_mask] == data.y[data.test_mask]<br>          test_acc = int(test_correct.sum()) / int(data.test_mask.sum())<br>          return test_acc<br>    <br>    def train_with_early_stopping(self, epochs=150, patience=10, plot = False, plot_name = None):<br>        <br>        history = {<br>            &#39;epoch&#39;: [],<br>            &#39;loss&#39;: [],<br>            &#39;test_acc&#39;: []<br>        }<br>        <br>        best_test_acc = 0.0<br>        epochs_without_improvement = 0<br>    <br>        for epoch in range(1, epochs + 1):<br>            loss = self.single_train()<br>            test_acc = self.test()<br>    <br>            print(f&#39;Epoch: {epoch:03d}, Loss: {loss:.4f}, Test Acc: {test_acc:.4f}&#39;)<br>    <br>            history[&#39;epoch&#39;].append(epoch)<br>            history[&#39;loss&#39;].append(loss.item())<br>            history[&#39;test_acc&#39;].append(test_acc)<br>    <br>            if test_acc &gt; best_test_acc:<br>                best_test_acc = test_acc<br>                epochs_without_improvement = 0<br>            else:<br>                epochs_without_improvement += 1<br>    <br>            if epochs_without_improvement &gt;= patience:<br>                print(f&#39;Early stopping triggered at epoch {epoch}.&#39;)<br>                break<br>    <br>        if plot:<br>            self.history_plot(history, plot_name)<br>    <br>        return history<br>    <br>    def train(self, epoch = 100):<br>        for epoch in range(1, epochs + 1):<br>            loss = self.single_train()<br>            test_acc = self.test()<br>    <br>            print(f&#39;Epoch: {epoch:03d}, Loss: {loss:.4f}, Test Acc: {test_acc:.4f}&#39;)<br>    <br>    <br>    def history_plot(self, history, plot_name):<br>        fig = go.Figure()<br>        fig.add_trace(go.Scatter(x=history[&#39;epoch&#39;], y=history[&#39;loss&#39;], mode=&#39;lines&#39;, name=&#39;Training Loss&#39;))<br>        fig.add_trace(go.Scatter(x=history[&#39;epoch&#39;], y=history[&#39;test_acc&#39;], mode=&#39;lines&#39;, name=&#39;Test Accuracy&#39;))<br>        fig.update_layout(<br>            title=&#39;Training History&#39;,<br>            xaxis_title=&#39;Epoch&#39;,<br>            yaxis_title=&#39;Value&#39;,<br>            legend=dict(x=0, y=1),<br>            template=&#39;plotly_dark&#39;<br>        )<br>        fig.show()</pre><p>Following that, we proceed to define our visualization methods:</p><pre>def visualize_2d(h, color, name = &#39;2D_dist_plot&#39;):<br>    z = TSNE(n_components=2).fit_transform(h.detach().cpu().numpy())<br><br>    fig = px.scatter(x=z[:, 0], y=z[:, 1], color=color, color_continuous_scale=&quot;magma&quot;)<br>    fig.update_layout(<br>        xaxis_title=&quot;Dimension 1&quot;,<br>        yaxis_title=&quot;Dimension 2&quot;,<br>        xaxis=dict(showgrid=False, zeroline=False, showticklabels=False),<br>        yaxis=dict(showgrid=False, zeroline=False, showticklabels=False),<br>        coloraxis_showscale=False,<br>        width=800,<br>        height=800<br>    )<br>    fig.show()<br> <br><br><br>def visualize_3d(h, color, name = &#39;3D_dist_plot&#39;):<br>    z = TSNE(n_components=3).fit_transform(h.detach().cpu().numpy())<br><br>    fig = px.scatter_3d(x=z[:, 0], y=z[:, 1], z=z[:, 2], color=color, color_continuous_scale=&quot;magma&quot;)<br>    fig.update_layout(<br>        scene=dict(<br>            xaxis_title=&quot;Dimension 1&quot;,<br>            yaxis_title=&quot;Dimension 2&quot;,<br>            zaxis_title=&quot;Dimension 3&quot;<br>        ),<br>        coloraxis_showscale=False,<br>        width=800,<br>        height=800<br>    )<br>    fig.show()</pre><p>Now let’s delve into the exciting world of Graph Neural Networks (GNNs) and explore how we can use them to train our dataset.</p><h4>1. Graph Convolutional Network (GCN)</h4><p>A Graph Convolutional Network (GCN) is a Graph Neural Network (GNN) variant tailored for processing graph-structured data. Unlike Convolutional Neural Networks (CNNs), which excel at grid-like data (such as images), GCNs specialize in datasets where entities are connected through edges, forming networks.</p><p>While CNNs leverage local patterns in grid data, GCNs harness the interconnectedness of graph data. They propagate and aggregate information across neighboring nodes, updating each node’s representation based on its neighbors’ features. This contextual understanding enables GCNs to capture relationships and patterns.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/760/1*rnCtQd6MJvAMPIGm1prF1Q.png" /><figcaption>Fig. 2. Comparison between GCN and CNN</figcaption></figure><p>The notable shift in GCNs lies in adapting the convolutional operation for graphs. This operation computes weighted averages of neighboring node features, generating central node representations. As these layers stack, GCNs learn abstract features while considering the overall graph context.</p><p>In the realm of traditional neural networks, linear layers play a pivotal role by applying a fundamental linear transformation to the input data. This transformation holds the power to metamorphose the input features denoted as x into a fresh realm of hidden vectors, which are symbolized as h. This enchanting metamorphosis is orchestrated through the agency of a weight matrix 𝐖, an omnipresent protagonist in this neural narrative. Disregarding the role of biases for this moment of elucidation, we can elegantly express this process as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/223/1*4Bg9OCuqnOL2LimePJPHsg.png" /><figcaption>Fig. 3. Linear relationship formula</figcaption></figure><p>One way to enhance our <strong>node representations</strong> is by combining their features with those of their neighboring nodes. This process, known as <strong>convolution or neighborhood aggregation</strong>, involves incorporating information from the immediate neighborhood of a node, including the node itself (denoted as Ñ).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/290/1*MijIhOsGdOUHMMleruzH5g.png" /><figcaption>Fig. 4. Aggregated linear relationship formula</figcaption></figure><p>Unlike CNN filters, in Graph Neural Networks (GNNs), our weight matrix 𝐖 is singular and shared across all nodes. However, a challenge arises due to the variable number of neighbors nodes can have, unlike the fixed grid structure of pixels in CNNs. This distinction is a key aspect of GNNs that enables them to <strong>effectively operate on graph-structured data</strong>.</p><p>How should we handle situations in which a single node is connected to only one neighbor, while another node has 700 connections? If we were to merely combine the feature vectors, the resultant embedding ‘h’ would be disproportionately influenced by the 700-neighbor node. To ensure uniform value ranges across all nodes and enable meaningful comparisons between them, we can <strong>normalize </strong>the output according to the nodes’ <strong>degrees </strong>(the count of connections each node possesses).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/397/1*OfGtI9vEiMn97FVjdkUxbQ.png" /><figcaption>Fig. 5. Normalized and aggregated linear relationship formula</figcaption></figure><p>The researchers noted that attributes originating from nodes with a high degree of neighbors spread more effortlessly compared to those from relatively secluded nodes. In order to counterbalance this phenomenon, they proposed the idea of assigning greater weights to attributes from nodes with limited neighbors. This strategy aims to harmonize the impact across the entire node network. This process can be expressed as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/547/1*K2JFO4FUn7ht7FPqWd-CoA.png" /><figcaption>Fig. 6. Harmonized formula</figcaption></figure><p>Let’s implement the concepts we’re discussing in Python using PyTorch for a deeper understanding.</p><p>First, let’s build the GCN model using PyTorch:</p><pre>from torch_geometric.nn import GCNConv<br><br>class GCN(torch.nn.Module):<br>    def __init__(self, hidden_channels):<br>        super().__init__()<br>        torch.manual_seed(1234567)<br>        self.conv1 = GCNConv(dataset.num_features, hidden_channels)<br>        self.conv2 = GCNConv(hidden_channels, dataset.num_classes)<br><br>    def forward(self, x, edge_index):<br>        x = self.conv1(x, edge_index)<br>        x = x.relu()<br>        x = F.dropout(x, p=0.5, training=self.training)<br>        x = self.conv2(x, edge_index)<br>        return x<br><br>model = GCN(hidden_channels=16)<br>print(model)</pre><pre>&gt;&gt;&gt;GCN(<br>&gt;&gt;&gt;  (conv1): GCNConv(1433, 16)<br>&gt;&gt;&gt;  (conv2): GCNConv(16, 7)<br>&gt;&gt;&gt;)</pre><p>Once we have built our model, we can move on to training and visualizing it:</p><pre>model = GCN(hidden_channels=16)<br><br>built_model = BuildModel(model)<br></pre><pre>Epoch: 001, Loss: 1.9463, Test Acc: 0.2700<br>Epoch: 002, Loss: 1.9409, Test Acc: 0.2910<br>Epoch: 003, Loss: 1.9343, Test Acc: 0.2910<br>Epoch: 004, Loss: 1.9275, Test Acc: 0.3210<br>Epoch: 005, Loss: 1.9181, Test Acc: 0.3630<br>Epoch: 006, Loss: 1.9086, Test Acc: 0.4120<br>Epoch: 007, Loss: 1.9015, Test Acc: 0.4010<br>Epoch: 008, Loss: 1.8933, Test Acc: 0.4020<br>Epoch: 009, Loss: 1.8808, Test Acc: 0.4180<br>Epoch: 010, Loss: 1.8685, Test Acc: 0.4470<br>Epoch: 011, Loss: 1.8598, Test Acc: 0.4680<br>Epoch: 012, Loss: 1.8482, Test Acc: 0.5180<br>Epoch: 013, Loss: 1.8290, Test Acc: 0.5440<br>Epoch: 014, Loss: 1.8233, Test Acc: 0.5720<br>Epoch: 015, Loss: 1.8057, Test Acc: 0.5910<br>Epoch: 016, Loss: 1.7966, Test Acc: 0.6080<br>Epoch: 017, Loss: 1.7825, Test Acc: 0.6300<br>Epoch: 018, Loss: 1.7617, Test Acc: 0.6450<br>Epoch: 019, Loss: 1.7491, Test Acc: 0.6520<br>Epoch: 020, Loss: 1.7310, Test Acc: 0.6560<br>Epoch: 021, Loss: 1.7147, Test Acc: 0.6570<br>Epoch: 022, Loss: 1.7056, Test Acc: 0.6640<br>Epoch: 023, Loss: 1.6954, Test Acc: 0.6770<br>Epoch: 024, Loss: 1.6697, Test Acc: 0.6950<br>Epoch: 025, Loss: 1.6538, Test Acc: 0.7140<br>Epoch: 026, Loss: 1.6312, Test Acc: 0.7150<br>Epoch: 027, Loss: 1.6161, Test Acc: 0.7170<br>Epoch: 028, Loss: 1.5899, Test Acc: 0.7230<br>Epoch: 029, Loss: 1.5711, Test Acc: 0.7220<br>Epoch: 030, Loss: 1.5576, Test Acc: 0.7210<br>Epoch: 031, Loss: 1.5393, Test Acc: 0.7280<br>Epoch: 032, Loss: 1.5137, Test Acc: 0.7370<br>Epoch: 033, Loss: 1.4948, Test Acc: 0.7380<br>Epoch: 034, Loss: 1.4913, Test Acc: 0.7430<br>Epoch: 035, Loss: 1.4698, Test Acc: 0.7510<br>Epoch: 036, Loss: 1.3998, Test Acc: 0.7570<br>Epoch: 037, Loss: 1.4041, Test Acc: 0.7600<br>Epoch: 038, Loss: 1.3761, Test Acc: 0.7640<br>Epoch: 039, Loss: 1.3631, Test Acc: 0.7700<br>Epoch: 040, Loss: 1.3258, Test Acc: 0.7800<br>Epoch: 041, Loss: 1.3030, Test Acc: 0.7810<br>Epoch: 042, Loss: 1.3119, Test Acc: 0.7760<br>Epoch: 043, Loss: 1.2519, Test Acc: 0.7760<br>Epoch: 044, Loss: 1.2530, Test Acc: 0.7790<br>Epoch: 045, Loss: 1.2492, Test Acc: 0.7800<br>Epoch: 046, Loss: 1.2205, Test Acc: 0.7790<br>Epoch: 047, Loss: 1.2037, Test Acc: 0.7850<br>Epoch: 048, Loss: 1.1571, Test Acc: 0.7900<br>Epoch: 049, Loss: 1.1700, Test Acc: 0.7920<br>Epoch: 050, Loss: 1.1296, Test Acc: 0.7940<br>Epoch: 051, Loss: 1.0860, Test Acc: 0.7930<br>Epoch: 052, Loss: 1.1080, Test Acc: 0.7910<br>Epoch: 053, Loss: 1.0564, Test Acc: 0.7930<br>Epoch: 054, Loss: 1.0157, Test Acc: 0.7930<br>Epoch: 055, Loss: 1.0362, Test Acc: 0.7920<br>Epoch: 056, Loss: 1.0328, Test Acc: 0.7980<br>Epoch: 057, Loss: 1.0058, Test Acc: 0.8000<br>Epoch: 058, Loss: 0.9865, Test Acc: 0.7970<br>Epoch: 059, Loss: 0.9667, Test Acc: 0.8010<br>Epoch: 060, Loss: 0.9741, Test Acc: 0.8000<br>Epoch: 061, Loss: 0.9769, Test Acc: 0.8030<br>Epoch: 062, Loss: 0.9122, Test Acc: 0.8040<br>Epoch: 063, Loss: 0.8993, Test Acc: 0.8050<br>Epoch: 064, Loss: 0.8769, Test Acc: 0.8050<br>Epoch: 065, Loss: 0.8575, Test Acc: 0.8060<br>Epoch: 066, Loss: 0.8897, Test Acc: 0.8030<br>Epoch: 067, Loss: 0.8312, Test Acc: 0.8060<br>Epoch: 068, Loss: 0.8262, Test Acc: 0.8030<br>Epoch: 069, Loss: 0.8511, Test Acc: 0.8070<br>Epoch: 070, Loss: 0.7711, Test Acc: 0.8070<br>Epoch: 071, Loss: 0.8012, Test Acc: 0.8080<br>Epoch: 072, Loss: 0.7529, Test Acc: 0.8080<br>Epoch: 073, Loss: 0.7525, Test Acc: 0.8070<br>Epoch: 074, Loss: 0.7689, Test Acc: 0.8110<br>Epoch: 075, Loss: 0.7553, Test Acc: 0.8140<br>Epoch: 076, Loss: 0.7032, Test Acc: 0.8120<br>Epoch: 077, Loss: 0.7326, Test Acc: 0.8110<br>Epoch: 078, Loss: 0.7122, Test Acc: 0.8120<br>Epoch: 079, Loss: 0.7090, Test Acc: 0.8110<br>Epoch: 080, Loss: 0.6755, Test Acc: 0.8130<br>Epoch: 081, Loss: 0.6666, Test Acc: 0.8070<br>Epoch: 082, Loss: 0.6679, Test Acc: 0.8080<br>Epoch: 083, Loss: 0.7037, Test Acc: 0.8100<br>Epoch: 084, Loss: 0.6752, Test Acc: 0.8070<br>Epoch: 085, Loss: 0.6266, Test Acc: 0.8100<br>Early stopping triggered at epoch 85.</pre><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fplotly.com%2F%7Ehalilibr.09%2F30.embed%3Fautosize%3Dtrue&amp;display_name=Plotly&amp;url=https%3A%2F%2Fchart-studio.plotly.com%2F%7Ehalilibr.09%2F30%2F&amp;image=https%3A%2F%2Fchart-studio.plotly.com%2Fstatic%2Fwebapp%2Fimages%2Fplotly-logo.8d56a320dbb8.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=plotly" width="600" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/ae0275e81de953082e24d5fe530cf132/href">https://medium.com/media/ae0275e81de953082e24d5fe530cf132/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fplotly.com%2F%7Ehalilibr.09%2F25.embed%3Fautosize%3Dtrue&amp;display_name=Plotly&amp;url=https%3A%2F%2Fchart-studio.plotly.com%2F%7Ehalilibr.09%2F25%2F&amp;image=https%3A%2F%2Fchart-studio.plotly.com%2Fstatic%2Fwebapp%2Fimages%2Fplotly-logo.8d56a320dbb8.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=plotly" width="600" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/7da5abb38246270e1ea4ef87fbd79af2/href">https://medium.com/media/7da5abb38246270e1ea4ef87fbd79af2/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fplotly.com%2F%7Ehalilibr.09%2F23.embed%3Fautosize%3Dtrue&amp;display_name=Plotly&amp;url=https%3A%2F%2Fchart-studio.plotly.com%2F%7Ehalilibr.09%2F23%2F&amp;image=https%3A%2F%2Fchart-studio.plotly.com%2Fstatic%2Fwebapp%2Fimages%2Fplotly-logo.8d56a320dbb8.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=plotly" width="600" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/8529447c49b09194d8463b55b6b1e110/href">https://medium.com/media/8529447c49b09194d8463b55b6b1e110/href</a></iframe><h4><strong>2. Graph Attention Networks (GAT)</strong></h4><p>GAT stands for Graph Attention Network, and it’s a type of Graph Neural Network (GNN) that has gained significant attention due to its effectiveness in modeling relationships within graph-structured data. GAT was introduced by Velickovic et al. in their 2018 paper “<a href="https://arxiv.org/pdf/1710.10903.pdf">Graph Attention Networks.</a>”</p><p>GAT addresses one of the key challenges in GNNs, which is how to effectively aggregate information from neighboring nodes in a graph while assigning different levels of importance to different neighbors. Traditional GNNs, such as Graph Convolutional Networks (GCNs), use fixed aggregation schemes that treat all neighbors equally. GAT, on the other hand, introduces the concept of attention mechanisms into the aggregation process, allowing <strong>each node to dynamically weigh the importance</strong> of its neighbors’ information.</p><h4>GAT Layer</h4><p>In this step, a shared, linear transformation takes center stage, embodied by the matrix W with dimensions (F’, F). This transformation is aptly named “shared” because every individual node undergoes the same W matrix-based transformation.</p><p>The mission here is to harmonize the dimensionality of node features. Originally of dimension F, these features are transformed to a uniform dimensionality of F’. This transformation is systematically applied to all nodes in node i’s neighborhood, encompassing node i itself.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/159/1*w4soQpW5YgjACznJlQEyAQ.png" /><figcaption>Fig. 7. Element-wise scalar multiplication formula</figcaption></figure><p>During this process, the embedding representation h_i of the target node i is fused with the embeddings of its immediate neighbors. Each pairing is then combined and transformed using matrix W^a, which is characterized by its dimensions (2F’, F’) here, F’ might stay the same or differ from the previous stage based on a hyperparameter.</p><p>The central aim here is to facilitate a collective learning of attention between these pairs of nodes, bypassing the specifics of the graph structure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/216/1*jDgLIGoqZkRzseNYj17vLg.png" /><figcaption>Fig. 8. Interaction formula</figcaption></figure><p>Here, each intermediate attention scalar comes to life through a non-linear activation, denoted as σ. In the GAT research, the authors opt for <strong>LeakyReLU </strong>as their chosen non-linear activation function.</p><p>Wrapping things up, the energized intermediate attention scalars flow through a <strong>softmax </strong>layer. This transformation imbues the attention coefficients with the properties of a probability distribution.</p><p>In essence, this phase centers on the normalization of attention coefficients, aligning them for further processing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/389/1*xZVNgjocUOKUHoqflZsuwA.png" /><figcaption>Fig. 9. GAT normalized formula 1</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/496/1*jp5RcC7ZOHDePj7DXLQ1iA.png" /><figcaption>Fig. 10. GAT normalized formula 2</figcaption></figure><p>The attention mechanism utilized by our model is defined by a parametric weight vector. This weight vector is associated with a LeakyReLU activation function, contributing to the overall functionality of the mechanism.</p><p>Consider an illustrative example of multihead attention with the scenario where K equals 3 heads. In this case, we focus on node 1 within its local neighborhood. Distinct styles and colors of arrows symbolize separate computations of attention, each operating independently. The outcomes from each head’s attention calculation are then combined by means of concatenation or averaging. This fusion of features results in the final representation denoted as h1&#39;.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*uWHgcy40sxNx6VZjrnD0rw.png" /><figcaption>Fig. 11. GAT Architecture 1</figcaption></figure><p>Put differently, employing this approach allows us to observe the operational dynamics of the Graph Attention Network (GAT).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/904/1*f2LiYOkb6SaNNRjCkYvF9w.png" /><figcaption>Fig. 12. GAT Architecture 2</figcaption></figure><p>Let’s put these concepts we discussed into practice using PyTorch in Python to develop a deeper understanding.</p><pre>from torch_geometric.nn import GATConv<br><br>class GAT(torch.nn.Module):<br>    def __init__(self, hidden_channels, heads):<br>        super().__init__()<br>        torch.manual_seed(1234567)<br>        self.conv1 = GATConv(dataset.num_features, hidden_channels,heads)<br>        self.conv2 = GATConv(heads*hidden_channels, dataset.num_classes,heads)<br><br>    def forward(self, x, edge_index):<br>        x = F.dropout(x, p=0.6, training=self.training)<br>        x = self.conv1(x, edge_index)<br>        x = F.elu(x)<br>        x = F.dropout(x, p=0.6, training=self.training)<br>        x = self.conv2(x, edge_index)<br>        return x<br><br>model = GAT(hidden_channels=8, heads=8)<br>print(model)</pre><pre>GAT(<br>  (conv1): GATConv(1433, 8, heads=8)<br>  (conv2): GATConv(64, 7, heads=8)<br>)</pre><p>Now, we should proceed with training and also visualize our GAT model:</p><pre>buildGANModel = BuildModel(model, lr = 0.05)<br>history = buildGANModel.train_with_early_stopping(plot = True, plot_name = &#39;gat_history_plot&#39;)</pre><pre>Epoch: 001, Loss: 0.6564, Test Acc: 0.7780<br>Epoch: 002, Loss: 0.6479, Test Acc: 0.7780<br>Epoch: 003, Loss: 0.6208, Test Acc: 0.8100<br>Epoch: 004, Loss: 0.5841, Test Acc: 0.8180<br>Epoch: 005, Loss: 0.5878, Test Acc: 0.8170<br>Epoch: 006, Loss: 0.5477, Test Acc: 0.8060<br>Epoch: 007, Loss: 0.4723, Test Acc: 0.7950<br>Epoch: 008, Loss: 0.4452, Test Acc: 0.8000<br>Epoch: 009, Loss: 0.4338, Test Acc: 0.8070<br>Epoch: 010, Loss: 0.4332, Test Acc: 0.8100<br>Epoch: 011, Loss: 0.4218, Test Acc: 0.8150<br>Epoch: 012, Loss: 0.3900, Test Acc: 0.8150<br>Epoch: 013, Loss: 0.4190, Test Acc: 0.8160<br>Epoch: 014, Loss: 0.4238, Test Acc: 0.8080<br>Early stopping triggered at epoch 14.</pre><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fplotly.com%2F%7Ehalilibr.09%2F32.embed%3Fautosize%3Dtrue&amp;display_name=Plotly&amp;url=https%3A%2F%2Fchart-studio.plotly.com%2F%7Ehalilibr.09%2F32%2F&amp;image=https%3A%2F%2Fchart-studio.plotly.com%2Fstatic%2Fwebapp%2Fimages%2Fplotly-logo.8d56a320dbb8.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=plotly" width="600" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/7a1d020343ba34dcb2e4563ec0fc39b8/href">https://medium.com/media/7a1d020343ba34dcb2e4563ec0fc39b8/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fplotly.com%2F%7Ehalilibr.09%2F34.embed%3Fautosize%3Dtrue&amp;display_name=Plotly&amp;url=https%3A%2F%2Fchart-studio.plotly.com%2F%7Ehalilibr.09%2F34%2F&amp;image=https%3A%2F%2Fchart-studio.plotly.com%2Fstatic%2Fwebapp%2Fimages%2Fplotly-logo.8d56a320dbb8.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=plotly" width="600" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/98b5a6ba507b07c49b3e2622628d8fb9/href">https://medium.com/media/98b5a6ba507b07c49b3e2622628d8fb9/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fplotly.com%2F%7Ehalilibr.09%2F36.embed%3Fautosize%3Dtrue&amp;display_name=Plotly&amp;url=https%3A%2F%2Fchart-studio.plotly.com%2F%7Ehalilibr.09%2F36%2F&amp;image=https%3A%2F%2Fchart-studio.plotly.com%2Fstatic%2Fwebapp%2Fimages%2Fplotly-logo.8d56a320dbb8.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=plotly" width="600" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/5441327b43edc6a15c31e690129f578a/href">https://medium.com/media/5441327b43edc6a15c31e690129f578a/href</a></iframe><p>Feel free to click <a href="https://github.com/Halil3509/Simple-Python-Works/blob/main/GNN/GNN_Medium_Blog.ipynb"><strong><em>here </em></strong></a>to access the entire code.</p><p>That concludes my blog. I trust that you found it impactful. I’m open to both positive and negative feedback, so please don’t hesitate to share your thoughts. Your feedback is eagerly anticipated. Stay committed and dedicated.</p><h3>References</h3><ul><li><a href="https://www.datacamp.com/tutorial/comprehensive-introduction-graph-neural-networks-gnns-tutorial">https://www.datacamp.com/tutorial/comprehensive-introduction-graph-neural-networks-gnns-tutorial</a></li><li><a href="https://www.youtube.com/watch?v=SnRfBfXwLuY">https://www.youtube.com/watch?v=SnRfBfXwLuY</a></li><li><a href="https://nabila-abraham.medium.com/ohmygraphs-graph-attention-networks-b7562289ae4b">https://nabila-abraham.medium.com/ohmygraphs-graph-attention-networks-b7562289ae4b</a></li><li><a href="https://towardsdatascience.com/graph-convolutional-networks-introduction-to-gnns-24b3f60d6c95">https://towardsdatascience.com/graph-convolutional-networks-introduction-to-gnns-24b3f60d6c95</a></li><li>Petar Veličković (2018). Graph Attention Networks. <a href="https://arxiv.org/pdf/1710.10903.pdf">https://arxiv.org/pdf/1710.10903.pdf</a></li><li>Jie Zhou (2020). Graph neural networks: A review of methods and applications. <a href="https://arxiv.org/ftp/arxiv/papers/1812/1812.08434.pdf">https://arxiv.org/ftp/arxiv/papers/1812/1812.08434.pdf</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1f463df4bb77" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>