<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[AG(A)I - Medium]]></title>
        <description><![CDATA[All things AI!! - Medium]]></description>
        <link>https://medium.com/ag-a-i?source=rss----33df6948536e---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>AG(A)I - Medium</title>
            <link>https://medium.com/ag-a-i?source=rss----33df6948536e---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 09:49:52 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/ag-a-i" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[CLJP: Trying CLIP-Style Learning for JSON Data (Need Your Thoughts!)]]></title>
            <link>https://medium.com/ag-a-i/cljp-trying-clip-style-learning-for-json-data-need-your-thoughts-8863d6b5d562?source=rss----33df6948536e---4</link>
            <guid isPermaLink="false">https://medium.com/p/8863d6b5d562</guid>
            <category><![CDATA[clip]]></category>
            <category><![CDATA[generative-ai]]></category>
            <dc:creator><![CDATA[ADITHYA GIRIDHARAN]]></dc:creator>
            <pubDate>Wed, 22 Oct 2025 11:34:35 GMT</pubDate>
            <atom:updated>2025-10-22T11:34:32.197Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*65YGgNc9RHhqIwh1AUPU6w.jpeg" /></figure><p>I’ve been exploring an idea: <strong>what if we could teach a model to understand the semantic relationship between natural language and JSON structures using contrastive learning?</strong></p><p>Think CLIP, but instead of images and text, it’s descriptions and JSON.</p><p>I built a proof-of-concept and got some interesting results, but I’m at a crossroads on several design decisions. Would love your thoughts.</p><h3>The Setup</h3><p>Say you have data like this:</p><pre>{<br>  &quot;description&quot;: &quot;API configuration for user authentication&quot;,<br>  &quot;json&quot;: {<br>    &quot;endpoint&quot;: &quot;/auth/login&quot;,<br>    &quot;method&quot;: &quot;POST&quot;,<br>    &quot;headers&quot;: {&quot;Content-Type&quot;: &quot;application/json&quot;},<br>    &quot;timeout&quot;: 5000<br>  }<br>}</pre><p>Can we train a model where:</p><ul><li>Query: “authentication endpoint” → retrieves the correct JSON</li><li>Query: similar JSON → finds semantically similar configs</li></ul><h3>My Approach (So Far)</h3><p>I went with contrastive learning — same philosophy as CLIP:</p><pre># In a batch of N (description, JSON) pairs:<br>desc_embeddings = encoder(descriptions)  # [N, 256]<br>json_embeddings = encoder(jsons)         # [N, 256]</pre><pre># Similarity matrix [N, N]<br>logits = desc_embeddings @ json_embeddings.T</pre><pre># Loss: maximize diagonal (correct pairs), minimize off-diagonal<br>labels = torch.arange(N)<br>loss = F.cross_entropy(logits, labels)</pre><p>The elegant part: <strong>you don’t manually create hard negatives</strong>. Every other example in the batch becomes a negative automatically.</p><p>Larger batch = harder negatives.</p><h3>Design Decision #1: How to Handle JSON?</h3><p>Here’s where I’m uncertain. I treated JSON as text:</p><pre># Serialize JSON to string<br>json_text = json.dumps(obj, sort_keys=True)<br># &quot;{&quot;endpoint&quot;:&quot;/auth/login&quot;,&quot;headers&quot;:{...},&quot;method&quot;:&quot;POST&quot;,...}&quot;</pre><pre># Tokenize with BERT tokenizer<br>tokens = tokenizer(json_text)</pre><p><strong>But is this the right approach?</strong></p><p>JSON has structure — keys, values, nesting, types. By flattening to text, I’m throwing away that structural information.</p><p><strong>Alternative approaches I’m considering:</strong></p><p><strong>Graph Neural Networks</strong>: If your JSON has graph-like structure (nodes, edges, relationships), shouldn’t we use GNNs instead?</p><p><strong>Tree-structured encoders</strong>: JSON is inherently hierarchical. Maybe a tree-LSTM or recursive neural network?</p><p><strong>Separate encoders for keys vs values</strong>: Keys have different semantics than values. Should they be encoded differently?</p><p><strong>What do you think?</strong> For your use case, would treating JSON as flat text lose too much information?</p><h3>Design Decision #2: Tokenization</h3><p>I used BERT’s tokenizer because it was convenient:</p><pre>from transformers import AutoTokenizer<br>tokenizer = AutoTokenizer.from_pretrained(&#39;bert-base-uncased&#39;)</pre><p><strong>But there are alternatives:</strong></p><ul><li><strong>OpenAI’s tokenizer</strong> (tiktoken): Different vocabulary, trained on more diverse data</li><li><strong>Custom BPE</strong>: Train on your specific JSON domain</li><li><strong>Character-level</strong>: Avoid tokenization artifacts entirely</li><li><strong>Code-aware tokenizers</strong>: Like CodeBERT’s tokenizer designed for structured code</li></ul><p>For JSON specifically, I wonder if a code-aware tokenizer would capture structure better?</p><pre># Would this help?<br>from transformers import AutoTokenizer<br>tokenizer = AutoTokenizer.from_pretrained(&#39;microsoft/codebert-base&#39;)</pre><p><strong>Has anyone tried different tokenizers for structured data?</strong> Did it make a difference?</p><h3>What I Built</h3><p>Modular pipeline in ~7 files:</p><pre># config.py - hyperparameters<br># dataset.py - handles JSON serialization<br># model.py - BERT encoder + projection head<br># loss.py - InfoNCE contrastive loss<br># train.py - training loop<br># evaluate.py - Recall@K, MRR metrics<br># inference.py - query interface</pre><p>Architecture was hybrid — BERT base as encoder (warm start) + trainable projection:</p><pre>class CLJPEncoder(nn.Module):<br>    def __init__(self):<br>        self.bert = AutoModel.from_pretrained(&#39;bert-base-uncased&#39;)<br>        self.projection = nn.Sequential(<br>            nn.Linear(768, 768),<br>            nn.ReLU(),<br>            nn.Linear(768, 256)<br>        )<br>    <br>    def forward(self, input_ids, attention_mask):<br>        outputs = self.bert(input_ids, attention_mask)<br>        cls_token = outputs.last_hidden_state[:, 0, :]<br>        embeddings = self.projection(cls_token)<br>        return F.normalize(embeddings, p=2, dim=1)</pre><p><strong>Question</strong>: Should I have used separate encoders for description vs JSON? Or is a shared encoder fine when both inputs are text?</p><h3>The Results (36 Samples)</h3><p>Trained on Colab T4, 100 epochs:</p><pre>Recall@1:  87.5%<br>Recall@5:  100%<br>MRR:       93.75%</pre><p>Query: “electronics” → correctly retrieved electronics products<br> Query: “book” → found book items</p><p><strong>But here’s the thing</strong>: 36 samples is tiny. The model basically memorized the training set.</p><p>This proves the <em>pipeline works</em>, but doesn’t prove the <em>approach scales</em>.</p><h3>Where I’m Stuck</h3><p><strong>1. JSON Serialization Strategy</strong></p><p>I tried three approaches:</p><pre># A) Compact JSON<br>&#39;{&quot;id&quot;:&quot;123&quot;,&quot;name&quot;:&quot;Widget&quot;,&quot;price&quot;:99}&#39;</pre><pre># B) Natural language<br>&#39;id: 123. name: Widget. price: 99&#39;</pre><pre># C) Structured<br>&#39;id: 123\nname: Widget\nprice: 99&#39;</pre><p>Which preserves semantics better? Does it even matter with enough data?</p><p><strong>2. For Graph-Structured JSON</strong></p><p>If you have workflow JSONs with nodes and edges:</p><pre>{<br>  &quot;nodes&quot;: [{&quot;id&quot;: &quot;1&quot;, &quot;type&quot;: &quot;embedding&quot;}, ...],<br>  &quot;edges&quot;: [{&quot;from&quot;: &quot;1&quot;, &quot;to&quot;: &quot;2&quot;}, ...]<br>}</pre><p>Should I be using Graph Neural Networks instead of treating it as text?</p><pre># GNN approach:<br>node_embeddings = gnn_layer(node_features, edge_index)<br>graph_embedding = global_mean_pool(node_embeddings)</pre><p><strong>But then</strong>: How do you handle the text description? Do you need a dual architecture (text encoder + GNN)?</p><p><strong>3. Tokenization</strong></p><p>Did I lose information by using BERT’s tokenizer on JSON? Should I try:</p><ul><li>CodeBERT tokenizer</li><li>OpenAI’s tiktoken</li><li>Custom tokenizer trained on JSON</li></ul><p><strong>Has anyone experimented with this?</strong></p><h3>What Would You Do Differently?</h3><p>I’m at the point where I need to decide: keep this architecture and just generate more data, or rethink the fundamentals?</p><p><strong>Specifically:</strong></p><p><strong>Text serialization vs structure-aware</strong>: Am I losing too much by flattening JSON to text?</p><p><strong>Shared vs separate encoders</strong>: One encoder for both modalities, or specialize?</p><p><strong>Tokenization</strong>: Does it matter much, or is BERT’s tokenizer “good enough”?</p><p><strong>For graphs specifically</strong>: Should workflow JSONs with nodes/edges use GNNs?</p><h3>The Meta Question</h3><p>Is contrastive learning even the right framing for this problem?</p><p>An alternative: <strong>treat it as retrieval</strong> with learned embeddings, but optimize directly for ranking metrics (MRR, NDCG) rather than contrastive loss.</p><p>Or: <strong>sequence-to-sequence</strong> where you generate JSON from description (though that’s a different problem).</p><h3>What I Learned (So Far)</h3><ul><li>Contrastive learning is surprisingly data-efficient for validation</li><li>In-batch negatives are free — don’t overcomplicate the dataset</li><li>BERT warm-start helps a lot with small data</li><li>Modular code makes iteration faster</li></ul><p>But I’m not confident I made all the right architectural choices.</p><p><strong>What would you try?</strong> Especially if you’ve worked with structured data embeddings or graph representations.</p><p>Would love to hear:</p><ul><li>Have you tried contrastive learning on structured data?</li><li>GNN vs text serialization — what’s worked for you?</li><li>Alternative tokenization strategies?</li><li>Am I overthinking this, or are these concerns valid?</li></ul><p>Drop your thoughts below. Genuinely curious what the community thinks about these design trade-offs.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8863d6b5d562" width="1" height="1" alt=""><hr><p><a href="https://medium.com/ag-a-i/cljp-trying-clip-style-learning-for-json-data-need-your-thoughts-8863d6b5d562">CLJP: Trying CLIP-Style Learning for JSON Data (Need Your Thoughts!)</a> was originally published in <a href="https://medium.com/ag-a-i">AG(A)I</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Direct Preference Optimization: A Journey from Equations to Intuition]]></title>
            <link>https://medium.com/ag-a-i/direct-preference-optimization-a-journey-from-equations-to-intuition-b9cd419edac2?source=rss----33df6948536e---4</link>
            <guid isPermaLink="false">https://medium.com/p/b9cd419edac2</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[dpos]]></category>
            <category><![CDATA[optimization]]></category>
            <dc:creator><![CDATA[ADITHYA GIRIDHARAN]]></dc:creator>
            <pubDate>Sun, 02 Mar 2025 09:26:41 GMT</pubDate>
            <atom:updated>2025-03-02T10:55:55.168Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Understanding DPO through restaurants, probabilities, and gradient updates</strong></p><blockquote>NOTE: This blog post, its underlying code right from conception to implementation, is completely powered by Generative AI.</blockquote><h3>Introduction</h3><p>There’s something inherently frustrating about <strong>black-box learning</strong>. The way large models absorb knowledge, tweak parameters, and optimize choices often feels like magic. But what if we strip away the complexity and build an <strong>intuition</strong> for the math from the ground up?</p><p>That’s exactly what I set out to do with <strong>Direct Preference Optimization (DPO)</strong> — a key method that serves as an alternate to <strong>reinforcement learning from human feedback (RLHF) to an extent</strong>. But rather than dealing with abstract probability distributions and models, I wanted to anchor everything in <strong>real-world intuition</strong>.</p><p>So, what’s a better analogy than <strong>choosing a restaurant?</strong> 🍣🍕🍔</p><p>In this deep dive, we’ll go from <strong>core equations</strong> to <strong>Python simulations</strong>, all while keeping things grounded in <strong>how people update their preferences when choosing where to eat</strong>. And yes, we’ll be <strong>deriving gradients, computing expectations, and optimizing probabilities</strong> — but in a way that feels tangible.</p><p>Let’s dig in.</p><h3>Equation 4: The Core Update Rule</h3><p>Imagine you have a few favorite restaurants — say, <strong>Pizza Palace, Sushi Spot,</strong> and <strong>Burger Barn</strong>. You have an initial preference distribution over these options. But then, <strong>new customer reviews come in</strong>.</p><ul><li><em>Sushi Spot gets glowing reviews.</em></li><li><em>Pizza Palace has a bad service scandal.</em></li><li><em>Burger Barn is just… Burger Barn.</em></li></ul><p>Naturally, your <strong>preferences will shift</strong>. But how?</p><p>Equation 4 gives us the <strong>update rule</strong>:</p><pre>\pi_r(y | x) = \frac{1}{Z(x)} \pi_{\text{ref}}(y | x) \exp \left(\frac{1}{\beta} r(x, y)\right)</pre><p>where:</p><ul><li>math \pi_{\text{ref}}(y | x) represents your <strong>initial preferences</strong> before reading reviews.</li><li>math r(x, y) is the <strong>review score</strong> (positive or negative).</li><li>math \beta determines how <strong>resistant</strong> you are to changing preferences.</li><li>math Z(x) is a <strong>normalization factor</strong> ensuring probabilities sum to 1.</li></ul><p>This means:</p><p>✅ <strong>Positive reviews increase probabilities</strong>, while<br>❌ <strong>Negative reviews decrease them</strong></p><p>But only <strong>as much as</strong> math \beta allows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/691/1*1FasNtNXWsxXX2hrj21PtQ.png" /><figcaption>Reference vs Updated Policy</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/699/1*_bfPGohiwjEObXg_chsecg.png" /><figcaption>Beta Effect on Preference updates</figcaption></figure><h3>Equation 5: Understanding the Reward Function</h3><p>The next step is flipping the equation: instead of <strong>inferring new probabilities from rewards</strong>, let’s compute the <strong>reward from observed probability shifts</strong>.</p><p>$$<br>r(x, y) = \beta \log \frac{\pi_r(y | x)}{\pi_{\text{ref}}(y | x)} + \beta \log Z(x)<br>$$</p><p>This equation tells us:</p><blockquote>How much of an impact must a review have had to justify the change in probability?</blockquote><ul><li>If <strong>Sushi Spot’s probability increased</strong>, we can compute how <strong>strong</strong> the reviews must have been.</li><li>If <strong>Pizza Palace’s probability dropped</strong>, we can compute how <strong>negative</strong> the news was.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/689/1*UsdLUf7PrIT5h7n59j5CNw.png" /></figure><h3>Equation 6: Comparing Two Restaurants</h3><p>Now, let’s say we’re not just interested in <strong>raw probabilities</strong>. We want to know:</p><blockquote>Given the review impact, how likely is it that people prefer Sushi Spot over Pizza Palace?</blockquote><p>This is modeled as:</p><p>$$<br>p^*(y_1 \succ y_2 | x) = \frac{1}{1 + \exp \left( \beta \log \frac{\pi^*(y_2 | x)}{\pi_{\text{ref}}(y_2 | x)} — \beta \log \frac{\pi^*(y_1 | x)}{\pi_{\text{ref}}(y_1 | x)} \right)}<br>$$</p><p>This <strong>sigmoid-based function</strong> smoothly transitions preference probabilities:</p><ul><li>If a restaurant’s probability <strong>skyrockets</strong>, it becomes much <strong>more preferred</strong>.</li><li>If both restaurants have <strong>similar probability shifts</strong>, the preference probability stays <strong>close to 50–50</strong>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/645/1*C_kOKEljEEqJNoZxL_jo2Q.png" /><figcaption>Preference Probability Matrix</figcaption></figure><h3>Equation 7: The Learning Process</h3><p>At this point, we’ve modeled how <strong>probability shifts influence ranking</strong>. But now comes the real question:</p><blockquote>How does the model learn?</blockquote><p>This is where the <strong>DPO loss function</strong> comes in:</p><p>$$<br>L_{\text{DPO}}(\pi_\theta; \pi_{\text{ref}}) = — \mathbb{E}_{(x, y_w, y_l) \sim D} \left[ \log \sigma \left( \beta \log \frac{\pi_\theta(y_w | x)}{\pi_{\text{ref}}(y_w | x)} — \beta \log \frac{\pi_\theta(y_l | x)}{\pi_{\text{ref}}(y_l | x)} \right) \right]<br>$$</p><p>This function <strong>penalizes the model when it ranks a “loser” restaurant above a “winner.”</strong></p><ul><li>If <strong>Sushi Spot should be ranked above Pizza Palace</strong>, but isn’t, the model takes a <strong>loss</strong>.</li><li>If rankings are <strong>correctly aligned</strong>, the loss remains <strong>small</strong>.</li></ul><p>By averaging across many <strong>(winner, loser) pairs</strong>, the model gradually aligns itself to <strong>match human preferences</strong>.</p><p>However, <strong>loss curves can be unstable</strong> due to random sampling, so we visualize its expectation over multiple samples.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/691/1*ATmS2dF4yUwhVVKzMXAD4g.png" /><figcaption>DPO Loss</figcaption></figure><h3>Gradient Update: Learning to Rank</h3><p>Finally, we come to the <strong>optimization step</strong>. The <strong>gradient of the loss function</strong> is:</p><p>$$<br>\nabla_\theta L_{\text{DPO}}(\pi_\theta; \pi_{\text{ref}}) = -\beta \mathbb{E}_{(x, y_w, y_l) \sim D} \left[ \sigma(r^*_\theta(x, y_l) — r^*_\theta(x, y_w)) \left( \nabla_\theta \log \pi_\theta(y_w | x) — \nabla_\theta \log \pi_\theta(y_l | x) \right) \right]<br>$$</p><p>Each <strong>update nudges the probabilities in the correct direction</strong>:</p><ul><li>If a restaurant is ranked <strong>lower than it should be</strong>, we <strong>increase</strong> its probability.</li><li>If a restaurant is ranked <strong>too high</strong>, we <strong>decrease</strong> its probability.</li></ul><p>These <strong>updates are weighted by how incorrect the ranking is</strong> — larger mistakes get larger corrections.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/691/1*Le7EItmBZJuRHfpcROTBeQ.png" /><figcaption>DPO Gradient Updates</figcaption></figure><h3>Final Thoughts</h3><p>Through this journey, we’ve:</p><p>✔ <strong>Broken down each DPO equation</strong> into an intuitive, restaurant-based explanation.<br> ✔ <strong>Implemented every equation in Python</strong>, validating correctness through simulations.<br> ✔ <strong>Visualized how probabilities, rewards, loss, and learning dynamics evolve.</strong></p><p>Instead of blindly fine-tuning models, we now <strong>understand</strong> how preference learning works at a <strong>fundamental</strong> level.</p><h3>Closing Thoughts</h3><p>This project started as a way to <strong>demystify</strong> DPO without the usual <strong>abstract jargon</strong>. By grounding it in something <strong>relatable</strong> — restaurant choices — I was able to build <strong>intuition</strong> that makes the math feel <strong>natural</strong>.</p><p>And that’s the kind of learning that <strong>sticks.</strong> 🚀</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b9cd419edac2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/ag-a-i/direct-preference-optimization-a-journey-from-equations-to-intuition-b9cd419edac2">Direct Preference Optimization: A Journey from Equations to Intuition</a> was originally published in <a href="https://medium.com/ag-a-i">AG(A)I</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>