<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Vivek Silimkhan on Medium]]></title>
        <description><![CDATA[Stories by Vivek Silimkhan on Medium]]></description>
        <link>https://medium.com/@viveksilimkhan?source=rss-376fa9473ada------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*xcY9-7gUSE1JDqog6iQ-wg.png</url>
            <title>Stories by Vivek Silimkhan on Medium</title>
            <link>https://medium.com/@viveksilimkhan?source=rss-376fa9473ada------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 12:09:21 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@viveksilimkhan/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Solving Maze puzzles using OpenEnv & Unsloth — Part 3]]></title>
            <link>https://medium.com/@viveksilimkhan/solving-maze-puzzles-using-openenv-unsloth-part-3-753936ffa01c?source=rss-376fa9473ada------2</link>
            <guid isPermaLink="false">https://medium.com/p/753936ffa01c</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[hugging-face]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <dc:creator><![CDATA[Vivek Silimkhan]]></dc:creator>
            <pubDate>Wed, 29 Oct 2025 21:57:58 GMT</pubDate>
            <atom:updated>2025-10-29T21:57:58.721Z</atom:updated>
            <content:encoded><![CDATA[<h3>Solving Maze puzzles using OpenEnv &amp; Unsloth — Part 3</h3><h4>Training</h4><p>Let us begin with training the model for solving maze game</p><p>We start writing the config for GRPO, followed by the training code</p><pre>max_prompt_length = maximum_length + 1 # + 1 just in case!<br>max_completion_length = max_seq_length - max_prompt_length<br><br>from trl import GRPOConfig, GRPOTrainer<br>training_args = GRPOConfig(<br>    temperature = 1.0,<br>    learning_rate = 5e-5,<br>    weight_decay = 0.01,<br>    warmup_ratio = 0.1,<br>    lr_scheduler_type = &quot;linear&quot;,<br>    optim = &quot;adamw_8bit&quot;,<br>    logging_steps = 1,<br>    per_device_train_batch_size = 1,<br>    gradient_accumulation_steps = 1, # Increase to 4 for smoother training<br>    num_generations = 2, # Decrease if out of memory<br>    max_prompt_length = max_prompt_length,<br>    max_completion_length = max_completion_length,<br>    # num_train_epochs = 1, # Set to 1 for a full training run<br>    max_steps = 250,<br>    save_steps = 50,<br>    report_to = &quot;none&quot;, # Can use Weights &amp; Biases, TrackIO<br>    output_dir = &quot;outputs&quot;,<br>)<br><br>trainer = GRPOTrainer(<br>    model = model,<br>    processing_class = tokenizer,<br>    reward_funcs = [<br>        function_works,<br>        no_cheating,<br>        strategy_succeeds,<br>    ],<br>    args = training_args,<br>    train_dataset = dataset,<br>)<br><br>trainer.train()</pre><p>The training actually looks like this<br>You can notice the code generated by the model, followed by the code validation check and last but not the least the metrics</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NbiljC9EqodcXdZA1yMgEw.png" /><figcaption>Training Qwen2.5-VL-7B for solving maze using GRPO</figcaption></figure><p>Finally we save the model</p><pre>model.save_pretrained_merged(&quot;maze-model&quot;, tokenizer, save_method = &quot;merged_16bit&quot;,)<br>model.push_to_hub_merged(&quot;viveksil/maze-rl-Qwen2.5-7B-Instruct&quot;, tokenizer, save_method = &quot;merged_16bit&quot;)</pre><h4>Extras !!</h4><p>But there’s more to be done Here is a function you can use to visualize the maze states</p><pre>from typing import Any, Dict, List, TYPE_CHECKING<br><br>def render_ascii_maze(maze: List[List[int]], position: List[int], start: List[int], goal: List[int]) -&gt; None:<br>        &quot;&quot;&quot;<br>        Render the maze grid as ASCII art in the terminal.<br>        - 0 = free cell<br>        - 1 = wall<br>        - S = start<br>        - G = goal<br>        - P = player<br>        - E = exit<br>        &quot;&quot;&quot;<br>        print(&quot;\nCurrent Maze State:&quot;)<br>        rows, cols = len(maze), len(maze[0])<br>        for r in range(rows):<br>            line = &quot;&quot;<br>            for c in range(cols):<br>                if [r, c] == position:<br>                    line += &quot;P &quot;<br>                elif [r, c] == start:<br>                    line += &quot;S &quot;<br>                elif [r, c] == goal:<br>                    line += &quot;G &quot;<br>                elif maze[r][c] == 1:<br>                    line += &quot;█ &quot;<br>                elif r == rows-1 and c == cols-1:<br>                    line+= &quot;E &quot;<br>                else:<br>                    line += &quot;. &quot;<br>            print(line)<br>        print()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/243/1*JqLdDC9rnjEFpEW4piC9hQ.png" /><figcaption>Maze state after the first legal move</figcaption></figure><h4>Hooray !!</h4><p>We are done with model training and have also reached towards the end of this series. We still need to evaluate these models and compare their performance. Thanks for reading</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=753936ffa01c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Solving Maze puzzles using OpenEnv & Unsloth — Part 2]]></title>
            <link>https://medium.com/@viveksilimkhan/solving-maze-puzzles-using-openenv-unsloth-part-2-e81c5352a8ab?source=rss-376fa9473ada------2</link>
            <guid isPermaLink="false">https://medium.com/p/e81c5352a8ab</guid>
            <category><![CDATA[unsloth]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[fine-tuning]]></category>
            <dc:creator><![CDATA[Vivek Silimkhan]]></dc:creator>
            <pubDate>Wed, 29 Oct 2025 21:44:52 GMT</pubDate>
            <atom:updated>2025-10-29T21:44:52.452Z</atom:updated>
            <content:encoded><![CDATA[<h3>Solving Maze puzzles using OpenEnv &amp; Unsloth — Part 2</h3><h3>Reward setting</h3><p>The important components required for training are the <strong>model</strong>, <strong>algorithm</strong> and most importantly the <strong>reward function</strong>.</p><p>We use unsloth for training Qwen2.5–7B-Instruct model using GRPO to solve this maze using the environment we created.</p><p>Unsloth allows fine-tuning language models in a faster and more efficient way. Normally, training a big model can take a lot of time and memory, but Unsloth optimizes the process so it runs on less resources.</p><p>We load the model and set the trainable parameters (LoRA for the win !!)</p><pre>from unsloth import FastLanguageModel<br>import torch<br><br>max_seq_length = 768 # Can increase for longer RL output<br>lora_rank = 4        # Larger rank = smarter, but slower<br>model, tokenizer = FastLanguageModel.from_pretrained(<br>    model_name = &quot;unsloth/Qwen2.5-7B-Instruct&quot;,<br>    # model_name = &quot;unsloth/gpt-oss-20b-BF16&quot;,<br>    <br>    load_in_4bit = False,<br>    max_seq_length = max_seq_length,<br>)<br><br>model = FastLanguageModel.get_peft_model(<br>    model,<br>    r = lora_rank, # Choose any number &gt; 0 ! Suggested 8, 16, 32, 64, 128<br>    target_modules = [<br>        &quot;q_proj&quot;, &quot;k_proj&quot;, &quot;v_proj&quot;, &quot;o_proj&quot;,<br>        &quot;gate_proj&quot;, &quot;up_proj&quot;, &quot;down_proj&quot;,<br>    ],<br>    lora_alpha = lora_rank*2, # *2 speeds up training<br>    use_gradient_checkpointing = &quot;unsloth&quot;, # Reduces memory usage<br>    random_state = 3407,<br>)</pre><p>First let us reset the environment</p><pre>from envs.maze_env import MazeEnv<br>from envs.maze_env.models import MazeAction, MazeObservation<br><br>openenv_process = MazeEnv(base_url = localhost)<br>result = openenv_process.reset()<br>current_state = result.observation<br><br>current_state.position = current_state.position[0] if len(current_state.position) == 1 else current_state.position</pre><p>We then write a prompt for the model. This prompt will instruct the model to generate code to solve the maze, one step at a time. The model will come up with a heuristic to solve it.</p><pre>prompt = &quot;&quot;&quot;<br>Create a new short maze game strategy using only native Python code.<br>You are given a maze_list for the current maze state which is List[List[Any]].<br>The current position is marked by &quot;x&quot;<br>The 0 in maze_list elements is free space, open to move<br>The 1 in maze_list elements is wall, not open to move<br>&quot;0&quot; - UP<br>&quot;1&quot; - DOWN<br>&quot;2&quot; - LEFT<br>&quot;3&quot; - RIGHT<br>Output one action for &quot;0&quot;, &quot;1&quot;, &quot;2&quot;, &quot;3&quot; on what is the optimal next step.<br>Output your new short function in backticks using the format below:<br>```python<br>def strategy(maze_list):<br>    &lt;Simple one-step heuristic, DO NOT complicate&gt;<br>    return &quot;0&quot; # Example<br>```<br>All helper functions should be inside def strategy. Only output the short function `strategy`.<br>&quot;&quot;&quot;.strip()<br>print(prompt)</pre><p>This python function generated by the model will be extracted and used to perform an action in the Maze environment.<br>But before that we also check if the program generated is valid and if the model uses additional/simplified libraries to solve the maze. <br>Hence we formulate a reward function as combination of function validity and correctness of the move/action</p><pre>def extract_function(text):<br>    ...<br><br># Reward functions<br><br># Check for function&#39;s validity<br>def function_works(completions, **kwargs):<br>    ...<br><br># Check for use of additional libraries<br>def no_cheating(completions, **kwargs):<br>    ...<br><br># Checks if strategy work in the environment<br>def strategy_succeeds(completions, **kwargs):<br>    ...</pre><p>Now, we train the model using GRPO trainer. Let’s see that in the next part.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e81c5352a8ab" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Solving Maze puzzles using OpenEnv & Unsloth — Part 1]]></title>
            <link>https://medium.com/@viveksilimkhan/solving-maze-puzzles-using-openenv-unsloth-part-1-4984e0082f81?source=rss-376fa9473ada------2</link>
            <guid isPermaLink="false">https://medium.com/p/4984e0082f81</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Vivek Silimkhan]]></dc:creator>
            <pubDate>Wed, 29 Oct 2025 21:20:18 GMT</pubDate>
            <atom:updated>2025-10-29T21:45:19.570Z</atom:updated>
            <content:encoded><![CDATA[<h3>Solving Maze puzzles using OpenEnv &amp; Unsloth — Part 1</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/389/1*gwnw0prZ4mCLnCEj6L8i5g.png" /></figure><blockquote><strong>OpenEnv</strong> is designed so that <strong>using an environment for RL </strong>is super straightforward: you get the familiar reset(), step(action), and state() methods — just like you&#39;d expect from something like Gymnasium — but under the hood it can run in a separate container, accessible over HTTP.</blockquote><p>As for <strong>functionality</strong>, OpenEnv gives you:</p><ul><li>A <strong>client-side API</strong> that will handle talking to the environment server (including running a Docker image if needed).</li><li>A <strong>server-side base class</strong> for defining an environment: you define typed dataclasses for Action, Observation, andState; implement reset(), step(), and state() (if you want). On the server-side it uses something like FastAPI to expose HTTP endpoints.</li><li>A <strong>web interface</strong> (optional) for interactive human debugging: you can open a browser, send actions, see observations, step through the environment manually. Great when you’re building or testing an environment.</li><li>Isolation &amp; deployment features: you can package the environment into a Docker container, so others can run it just by pulling a container image. This makes sharing environments easier and more reproducible.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/486/1*_vc-RY4bZgr-mcRD2eCFEw.png" /><figcaption>Architecture of OpenEnv Components</figcaption></figure><p><strong>It is easy to create your own environment</strong></p><p>You define your Action, Observation, optionally State, implement the logic in the server class, build the container, and then you’re good to go. The standardized API means your environment will “just work” with the client.</p><h3>Let’s create a new environment !!</h3><p>A <strong>maze puzzle</strong> is one of the oldest and most familiar types of puzzles out there. It’s basically a network of paths or passages, and the challenge is to find a route from the entrance to the exit. Mazes can be simple or incredibly complex, and they’ve appeared everywhere — from ancient designs carved into stone to modern video games and puzzle books. They test your sense of direction, logic, and patience, all while giving you that little thrill of finding your way out.</p><h4>Maze</h4><p>We start with creating a single 8x8 matrix for our maze</p><pre>import numpy as np<br><br># Maze<br>maze = np.array([<br>    [0, 1, 0, 0, 0, 0, 0, 0],<br>    [0, 1, 0, 1, 0, 1, 0, 0],<br>    [0, 0, 0, 1, 1, 0, 1, 0],<br>    [0, 1, 0, 1, 0, 0, 0, 0],<br>    [1, 0, 0, 1, 0, 1, 0, 0],<br>    [0, 0, 0, 1, 0, 1, 1, 1],<br>    [0, 1, 1, 0, 0, 0, 0, 0],<br>    [0, 0, 0, 0, 0, 1, 0, 0]<br>])</pre><h4>Models</h4><p>We need to define the Action, Observation, and State as data class in order to use them in the RL loops.</p><pre>from __future__ import annotations<br><br>from dataclasses import dataclass, field<br>from pydantic import Field<br>from typing import Any, Dict, List, Optional, Tuple, Literal<br><br>from core.env_server import Action, Observation, State<br><br><br>@dataclass<br>class MazeAction(Action):<br>    action: int<br><br><br>@dataclass<br>class MazeObservation(Observation):<br>    position: List[int]  # [row, col]<br>    total_reward: float<br>    legal_actions: List[int] = field(default_factory=list)<br><br>@dataclass<br>class MazeState(State):<br>    episode_id: str<br>    step_count: int<br>    done: bool = False</pre><h4>Environment</h4><p>An <em>environment</em> is basically the world the agent lives in. It’s everything the agent interacts with — where it takes actions, receives feedback (rewards or penalties), and learns what works and what doesn’t.</p><p>You can think of it like the “game world” in which the agent is trying to figure out how to win. The environment defines the rules, the possible actions, and what happens as a result.</p><p>It usually consists of the reset(), step(action), and state() methods</p><pre>from typing import Any, Dict, List, Tuple, Optional<br>from core.env_server import Action, Environment, Observation<br>from .maze import Maze, Status<br>from ..models import MazeAction, MazeObservation, MazeState<br><br><br>class MazeEnvironment(Environment):<br>    &quot;&quot;&quot;<br>    Maze Environment wrapper for OpenEnv.<br>    This environment wraps Maze game and provides a single-agent interface.<br>    &quot;&quot;&quot;<br><br>    def __init__(<br>        self,<br>        maze_array: np.ndarray,<br>        start_cell: Tuple[int, int] = (0, 0),<br>        exit_cell: Optional[Tuple[int, int]] = (7,7),<br>    ):<br>      ...<br>    # Reset<br>    def reset(self) -&gt; MazeObservation:<br>        &quot;&quot;&quot;Reset environment and return initial observation (MazeObservation).&quot;&quot;&quot;<br>       ...<br>  <br>    # Step<br>    def step(self, action: MazeAction) -&gt; MazeObservation:<br>        &quot;&quot;&quot;<br>        Step function that manipulates the maze position grid<br>        and applies rewards/penalties for movement outcomes.<br>        &quot;&quot;&quot;<br>        ...<br><br>    # State<br>    def state(self) -&gt; Optional[MazeState]:<br>        &quot;&quot;&quot;Return the current MazeState object.&quot;&quot;&quot;<br>        return self.state<br><br>    # Utils<br>    def _compute_legal_actions(self, pos: List[int]) -&gt; List[int]:<br>        &quot;&quot;&quot;<br>        Compute which actions are legal given the current normalized position [row, col].<br>        (0=UP, 1=DOWN, 2=LEFT, 3=RIGHT)<br>        &quot;&quot;&quot;<br>        ...</pre><h4>HTTP Client</h4><p>HTTP client is used to connect to the environment server</p><pre><br><br>from __future__ import annotations<br><br>from typing import Any, Dict, List, TYPE_CHECKING<br><br>from core.client_types import StepResult<br>from core.http_env_client import HTTPEnvClient<br><br>from .models import MazeAction, MazeObservation, MazeState<br><br>if TYPE_CHECKING:<br>    from core.containers.runtime import ContainerProvider<br><br>class MazeEnv(HTTPEnvClient[MazeAction, MazeObservation]):<br>    &quot;&quot;&quot;HTTP client for Maze Environment.&quot;&quot;&quot;<br><br><br>    def _step_payload(self, action: MazeAction) -&gt; Dict[str, Any]:<br>        &quot;&quot;&quot;Prepare payload to send to the environment server.&quot;&quot;&quot;<br>        return {&quot;action&quot;: action.action}<br><br>    def _parse_result(self, payload: Dict[str, Any]) -&gt; StepResult[MazeObservation]:<br>        &quot;&quot;&quot;Parse the response from the server into MazeObservation + reward/done.&quot;&quot;&quot;<br>        obs_data = payload.get(&quot;observation&quot;, {})<br><br>        observation = MazeObservation(<br>            position=obs_data.get(&quot;position&quot;, []),<br>            total_reward=obs_data.get(&quot;total_reward&quot;, 0.0),<br>            legal_actions=obs_data.get(&quot;legal_actions&quot;, []),<br>        )<br><br>        return StepResult(<br>            observation=observation,<br>            reward=payload.get(&quot;reward&quot;, 0.0),<br>            done=payload.get(&quot;done&quot;, False),<br>        )<br><br>    def _parse_state(self, payload: Dict[str, Any]) -&gt; MazeState:<br>        &quot;&quot;&quot;Parse environment state from payload.&quot;&quot;&quot;<br>        return MazeState(<br>            episode_id=payload.get(&quot;episode_id&quot;, &quot;&quot;),<br>            step_count=payload.get(&quot;step_count&quot;, 0),<br>            done=payload.get(&quot;done&quot;, False),<br>        )</pre><h4>Application</h4><p>This FastAPI application provides the Maze game over HTTP endpoint (compatible with the client)</p><pre>from core.env_server import create_app<br>import numpy as np<br>from ..models import MazeAction, MazeObservation<br>from .maze_environment import MazeEnvironment<br>from .mazearray import maze<br># Get game configuration from environment variables<br><br># Create the environment instance<br>env = MazeEnvironment(maze_array=maze,start_cell=(0,0),exit_cell=(7,7))<br><br># Create the FastAPI app with web interface and README integration<br>app = create_app(env, MazeAction, MazeObservation, env_name=&quot;maze_env&quot;)<br><br><br>if __name__ == &quot;__main__&quot;:<br>    import uvicorn<br><br>    uvicorn.run(app, host=&quot;0.0.0.0&quot;, port=8000)</pre><p>Let us see the reward setting in the next part</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4984e0082f81" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Bridging the gap in medicine access and healthcare literacy: A WhatsApp bot for prescription-based…]]></title>
            <link>https://medium.com/@viveksilimkhan/bridging-the-gap-in-medicine-access-and-healthcare-literacy-a-whatsapp-bot-for-prescription-based-b2ff6a5b04d0?source=rss-376fa9473ada------2</link>
            <guid isPermaLink="false">https://medium.com/p/b2ff6a5b04d0</guid>
            <category><![CDATA[agents]]></category>
            <category><![CDATA[whatsapp]]></category>
            <category><![CDATA[healthcare]]></category>
            <category><![CDATA[llama-3]]></category>
            <category><![CDATA[literacy]]></category>
            <dc:creator><![CDATA[Vivek Silimkhan]]></dc:creator>
            <pubDate>Sat, 26 Oct 2024 14:54:02 GMT</pubDate>
            <atom:updated>2024-10-26T14:54:02.953Z</atom:updated>
            <content:encoded><![CDATA[<h3>Bridging the gap in medicine access and healthcare literacy: A WhatsApp bot for prescription-based ordering and multi-lingual information retrieval</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qepZPj-OOiSN63q0j0vWdA.jpeg" /></figure><h3>Introduction</h3><p>As we all know there is a big challenge of delivering medication information accurately and accessibly heightened due to linguistic diversity and limited health literacy, especially in rural regions of our country.</p><p>Many individuals rely on standard drug labels printed in a single language, which can disadvantage non-native speakers and those with limited education who depend on others for understanding vital information. Rural and uneducated populations often identify medicines based on physical packaging, which is impractical and risky in an increasingly digital healthcare landscape.</p><p>A research was conducted on Hispanic parent with limited proficiency in English and health literacy in US, highlighting the significant consequences of these challenges. It was found that there were twice as likely to misdose children’s liquid medications.</p><blockquote>As the <a href="https://pmc.ncbi.nlm.nih.gov/articles/PMC5424611/">research paper</a> quotes:</blockquote><blockquote>1. We found that liquid medication dosing errors by Hispanic parents are common, with over <strong>80% of parents making at least one error</strong>, and that errors were more common among those with <strong>limited health literacy</strong> and l<strong>imited English proficiency</strong>.</blockquote><blockquote>2. Parents with limited health literacy and limited English proficiency had <strong>the greatest odds of making a dosing error</strong> compared to parents with adequate health literacy who were English proficient (% trials with errors/parent=28.8 vs. 12.9%; AOR=2.2[1.7–2.8]). Parents with limited health literacy who were English proficient were also more likely to make errors (% trials with errors/parent=18.8%; AOR=1.4[1.1–1.9]).</blockquote><h3>How did we try to solve this ?</h3><p>To solve this problem, we decided to take a look at the consumer behavior in India that goes through the medication, from getting the prescription from the doctor to buying the medicines and following up with the dosage as prescribed.</p><p>We noticed a pattern where elder generation as well as some non-English speakers often had a hard time understanding the dosage instructions and required some assistance for understanding the dosage as well buying them as per the prescription. We also noticed a common behavior where people bought medicines from the local pharmacy using the medicine wrapper.</p><p>Taking this into consideration, we proposed a solution which would help solve these issue.<br>By leveraging the power of <a href="https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/">Llama 3.2 vision models</a> for their state-of-the-art vision and multilingual capabilities. This broadly helps us in capturing the insights from the invoice and medicine wrapper, as well as educating the people about the medicine dosage in their own language.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C-66dXkU9AiaaXDU7W0KJg.png" /></figure><p>The solution consists of 2 features</p><ol><li><strong>Prescription and Medicine Wrapper Analysis</strong></li><li><strong>Ordering medicines using Whatsapp pay</strong></li></ol><h4>1. Prescription and Medicine Wrapper Analysis</h4><p>The users can upload a photo of prescription/medicine wrapper to the chat and get more information about the medicine dosage in their own language. The results from the prescription/medicine wrapper and dosage information are grounded using medically verified information indexed in the <a href="https://qdrant.tech/">Qdrant DB</a>. <br>Followed by the response, based on the language selected, the response is translated to the desired language. An additional call for translation is made to Llama 3 just to simplify instructions in the prompt avoid any instruction overload in the prompt.</p><p><strong>a. Prescription analysis</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lWCV1l_eiMUUnGyQSe80mQ.png" /><figcaption><strong>Prescription Analysis</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/540/1*iTpLdNi8o6eB6XcBHyzSBw.gif" /><figcaption><strong>Demo of prescription analysis</strong></figcaption></figure><p><strong>b. Medicine wrapper analysis</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jtQDO1nL43z5E2HhoTY27A.png" /><figcaption><strong>Medicine wrapper analysis</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/540/1*PnD5NA4Y9IsebuFRd67bzg.gif" /><figcaption><strong>Demo of medicine wrapper analysis</strong></figcaption></figure><h4>2. Ordering medicine using WhatsApp Pay</h4><p>The users can just upload a picture of a prescription/medicine wrapper along with the quantity required with some additional information in their own language as well as just the . Extracting the name of the medicines from the prescription or the medicine wrapper and the quantity, the bot can order using online pharmacies by function calling their respective APIs or add an order to the registered local pharmacist.<br>Followed by the order, a confirmation is sent to the users and payment is initiated using WhatsApp pay.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZZu9uYz6nB0cCZbsoi9QAg.png" /><figcaption><strong>Ordering medicine from prescription/medicine wrapper</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/540/1*k1y_sPstP76u2IH8gcBtXg.gif" /><figcaption><strong>Demo of ordering using local language</strong></figcaption></figure><h3>Time’s up</h3><p>This was our winning solution at <strong>AI hackathon with Meta Llama</strong> held on 19–20 October 2024.</p><p>I would take a moment to thank all the judges and mentors for sharing their valuable insights and guiding us throughout the hackathon</p><p>A big shoutout to our mentor and executive Aashis Tiwari for his guidance and my teammates Amey Narayan and Vedant Kulkarni for their teamwork and dedication. This hackathon win is a testament to our collaboration and shared vision.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b2ff6a5b04d0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Large Language Models for Sanskrit [Part-2]]]></title>
            <link>https://medium.com/@viveksilimkhan/large-language-models-for-sanskrit-part-2-088b4e8c39c3?source=rss-376fa9473ada------2</link>
            <guid isPermaLink="false">https://medium.com/p/088b4e8c39c3</guid>
            <category><![CDATA[sanskrit]]></category>
            <category><![CDATA[synthetic-data]]></category>
            <category><![CDATA[translation]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[fine-tuning]]></category>
            <dc:creator><![CDATA[Vivek Silimkhan]]></dc:creator>
            <pubDate>Thu, 26 Sep 2024 19:35:45 GMT</pubDate>
            <atom:updated>2024-09-26T20:00:52.952Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7V_UIrUS2pLjZBkVFESz0w.jpeg" /></figure><p>As we discussed in the <a href="https://medium.com/@viveksilimkhan/large-language-models-for-sanskrit-part-1-4af1da6aa789">previous part</a> about how Sanskrit is different from English, in this one, let’s dive deep into the techniques we used to improve the text generation in Sanskrit.</p><p>Before we start off with using an LLM for Sanskrit generation, we realised it was important to check how the baseline performance of these LLMs were, which would make it clear whether the LLMs were sufficiently trained on Sanskrit or not.</p><h4>Choosing the right LLM</h4><p>Hence we decided to try out a few open-source LLMs for Sanskrit generation. We evaluated <strong>Aya-101–13B</strong>, <strong>Llama-3.1–8B,</strong> <strong>Llama-3.1–70B</strong> and <strong>Gemma-2–9B</strong> models for Sanskrit generation. Based on the evaluation, we decided to go forward with <strong>Llama-3.1–8B</strong> and <strong>Gemma-2–9B</strong>, as they had a best Sanskrit generation among all and were &lt;10B parameters.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/707/1*bFhwMIwZmicP7IWPMUXvhw.png" /><figcaption>Evaluation for Sanskrit text generation</figcaption></figure><h4>Prompting</h4><p>We decided to improve the Sanskrit generation by using some prompting techniques, like few-shot and self-reflection, however we go a very minute improvement. Hence we had to try fine-tuning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*joyapX0uew5_Ji13T2ydzQ.png" /></figure><h4>Synthetic data generation and Fine-tuning</h4><p>As I mentioned earlier, Sanskrit is a very-low resource language, we had go for generating our own synthetic data.<br>We collected some corpus having Sanskrit text, and using the best performing open-source model we had, that is <strong>Llama-3.1-70B</strong>, we generated approximately <strong>11,000 question-answering pairs</strong>, following which we did some filtering on the basis of language generated.<br>For translation task, we already had <a href="https://huggingface.co/datasets/rahular/itihasa"><strong>itihasa</strong></a> dataset, which consists of nearly <strong>93,000 English-Sanskrit translation pairs</strong>.<br>Due to time and computational constraints, we trained the LLMs on nearly 3,200 translation pairs</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GOp6LeDtbAPwyRe-N-QEmQ.png" /><figcaption>Fine-tuning pipeline for question-answering and translation tasks</figcaption></figure><p>We evaluated this models for BLEU, Rouge and Cosine similarity score and we observed the following<br>- For translation task, there was a <strong>very good improvement</strong> across all the score<br>- For question-answering tasks, there was a <strong>fair improvement</strong> observed across most of the metrics<br>This is an expected behavior for question-answering task because the dataset generated by the teacher model (Llama-3.1–70B) we used for generating synthetic data lags behind on Sanskrit generation as compared to human curated datasets like itihasa (dataset for English-Sanskrit translation).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/970/1*tFhUsKvbAnaQE_3SbZeEjw.png" /><figcaption>For English-to-Sanskrit translation</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pOLKo2nSEEvOukdob8Ss7w.png" /><figcaption>For Question-Answering task</figcaption></figure><h4>It’s a wrap …. (for now)</h4><p>That was our 6 week of work. We are working on improving the LLMs and continuing further research on LLMs for Sanskrit.</p><p>Most importantly, I’d like to mention my teammates Sai Vineetha BVNS, Jithendra J K and Gaurav Karwa for their contribution and constant support throughout the Expedition Aya.</p><p>This is not the end !!! . I’ll post about further research we’ll be doing in the next part of this series.<br>So ………. Stay tuned for the next part :)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=088b4e8c39c3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Large Language Models for Sanskrit [Part-1]]]></title>
            <link>https://medium.com/@viveksilimkhan/large-language-models-for-sanskrit-part-1-4af1da6aa789?source=rss-376fa9473ada------2</link>
            <guid isPermaLink="false">https://medium.com/p/4af1da6aa789</guid>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[grammar]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[linguistics]]></category>
            <category><![CDATA[sanskrit]]></category>
            <dc:creator><![CDATA[Vivek Silimkhan]]></dc:creator>
            <pubDate>Thu, 26 Sep 2024 17:12:39 GMT</pubDate>
            <atom:updated>2024-09-26T17:45:24.161Z</atom:updated>
            <content:encoded><![CDATA[<h3>Large Language Models for Sanskrit (Part-1)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0y5uWSFiFVRV-Q507ulHzg.jpeg" /></figure><p>Large Language Models (LLMs) are <strong>predominantly</strong> trained on vast corpora of text data, accessed from various sources over the world wide web, much of which is in <strong>English</strong>.</p><p>This is largely due to the sheer volume of high-quality, publicly available <strong>English</strong> text on the internet. <strong>English</strong> is the <strong>widely accepted</strong> and <strong>dominant</strong> language of scientific research, business communication, and digital content, which makes it the most accessible and abundant resource for training models. As a result, generally pre-trained and aligned LLMs often tend to perform exceptionally well in language understanding, translation, and generation involving <strong>English</strong> as the medium.</p><p>However, this emphasis on <strong>English</strong> introduces a significant data imbalance when considering languages of different origins. While there are <strong>billions of speakers</strong> for other major languages such as <strong>Mandarin</strong>, <strong>Hindi</strong>, and <strong>Spanish</strong>, the availability of high-quality digital text in these languages is comparatively lower. Additionally, languages with <strong>smaller populations</strong> or <strong>non-Latin scripts</strong>, such as <strong>indigenous languages</strong> or those from parts of <strong>Africa</strong> and <strong>Southeast Asia</strong>, are even more underrepresented like <strong>Sanskrit</strong>, <strong>Sinhala</strong>, <strong>Korean</strong>, <strong>Swahili</strong> and so on.</p><p>The proportion of training data for <strong>non-English</strong> languages is often a small fraction of the total dataset, which implies that LLMs struggle with tasks involving these languages. Although recent efforts have aimed to include more diverse multilingual datasets, such as <a href="https://huggingface.co/datasets/allenai/c4"><strong>mc4</strong></a> which involved <strong>108 diverse languages </strong>, <a href="https://huggingface.co/datasets/wikimedia/wikipedia"><strong>wikipedia dump</strong></a><strong> </strong>which consists of corpora on <strong>over</strong> <strong>300+ languages</strong>, the disparity remains. For instance, many LLMs are trained with only a few percentage points of data in languages <strong>other than English</strong>, leading to performance gaps in comprehension and generation for these languages. This imbalance reflects broader issues in <strong>digital representation</strong> and <strong>access</strong>, and efforts to correct it through more balanced datasets are ongoing, though challenging.</p><p>In an attempt to solve this issue, we took a <strong>baby-step</strong> towards improving LLMs for low-resource languages, and hence we chose to select one of the <strong>very-low resource language</strong>, <strong>Sanskrit</strong>, to research and improve the quality of text generation for different language tasks like <strong>question-answering</strong> and <strong>translation </strong>as a part of our <strong>6-weeks Cohere for AI’s Expedition Aya 2024.</strong></p><h3>How Sanskrit is different from English ?</h3><p>The way each language is constructed is different, hence we cannot use the same techniques always to yield better results, and understanding the linguistic aspect of it becomes necessary.</p><p>Therefore let’s check the differences <strong>Sanskrit</strong> language has over the widely used language <strong>English</strong>.</p><p><strong>Sanskrit</strong> and <strong>English</strong> differ fundamentally in their <strong>morphology</strong> (word structure), <strong>syntax</strong> (sentence construction) and the way we write the <strong>parts-of-speech</strong> (<strong>POS</strong>). <br>Following are the difference between <strong>English</strong> and <strong>Sanskrit</strong> on these differences</p><h4>Morphology</h4><p><strong>Sanskrit</strong> is a highly inflected language. This means that the meaning of a word changes based on its endings, depending on case, number, gender, tense, mood, etc. Sanskrit has extensive inflectional morphology, allowing a single root to be transformed into many forms. These forms are called as <strong>vibhakti</strong>.</p><p><strong>English</strong>, on the other hand, has relatively limited inflection. Instead of relying heavily on word endings, <strong>English</strong> often depends on word order and auxiliary verbs for meaning.</p><h4><strong>Example:</strong></h4><p><strong>Sanskrit Noun Inflection in Devanāgarī script<br></strong>The word for “boy” in Sanskrit, <strong>बालक (bālaka)</strong>, changes based on case:</p><ul><li><strong>Nominative</strong> (Subject): <strong>बालकः</strong> (bālakaḥ) — “The boy”</li><li><strong>Accusative</strong> (Direct Object): <strong>बालकम्</strong> (bālakam) — “The boy (as an object)”</li><li><strong>Dative</strong> (Indirect Object): <strong>बालकाय</strong> (bālakāya) — “For the boy”</li><li><strong>Genitive</strong> (Possessive): <strong>बालकस्य</strong> (bālakasya) — “Of the boy”</li></ul><p><strong>English Noun<br></strong>The word “<strong>boy</strong>” remains boy regardless of its grammatical role in a sentence. The function of the noun is usually clarified by word order or prepositions:</p><ul><li>“The boy plays.” (<strong>Subject</strong>)</li><li>“I see the boy.” (<strong>Direct Object</strong>)</li><li>“I gave the book to the boy.” (<strong>Indirect Object</strong>)</li></ul><h4>Syntax</h4><p><strong>Sanskrit</strong> has flexible word order due to its <strong>inflectional</strong> nature. Because nouns, pronouns, and verbs are marked for grammatical roles, word order can change without altering the meaning of a sentence. <strong>Sanskrit</strong> typically follows a <strong>S-O-V (Subject-Object-Verb)</strong> structure but can be rearranged for emphasis.</p><p>Whereas,<strong> English </strong>has relatively <strong>limited inflection</strong>. Instead of relying heavily on word endings, <strong>English</strong> often depends on word order and auxiliary verbs for meaning.</p><h4><strong>Example:</strong></h4><p><strong>Sanskrit</strong></p><ul><li><strong>रामः पुस्तकं पठति (rāmaḥ pustakaṃ paṭhati)</strong> — “<strong>Rama reads the book.</strong>” (SOV: Subject-Object-Verb). This can be rearranged: <strong>पुस्तकं रामः पठति (pustakaṃ rāmaḥ paṭhati)</strong> — “<strong>Rama reads the book.</strong>”. Because of inflection, the meaning doesn’t change regardless of word order.</li></ul><p><strong>English</strong></p><ul><li>“<strong>Rama reads the book</strong>.” (SVO). Reversing it to “<strong>The book reads Rama</strong>” changes the meaning entirely because the roles of subject and object depend on position, not inflection.</li></ul><h4>Verb Conjugation</h4><p><strong>Sanskrit verbs</strong> are highly conjugated, with variations for person (1st, 2nd, 3rd), number (singular, dual, plural), tense (present, past, future), mood (indicative, subjunctive, imperative), and voice (active, passive, middle). This gives Sanskrit an expressive ability to convey nuanced action.</p><p><strong>English verbs</strong> are less conjugated, primarily varying in tense (past, present, future) and aspect (simple, continuous, perfect). Auxiliary verbs are often used in English to express different moods, aspects, and voices.</p><h4><strong>Example</strong>:</h4><p><strong>Sanskrit verb “to go” (गच्छति — gacchati)</strong>:</p><ul><li>1st Person Singular Present: <strong>अहं गच्छामि </strong>(ahaṃ gacchāmi) — “<strong>I am going</strong>”</li><li>2nd Person Singular Present: <strong>त्वं गच्छसि </strong>(tvaṃ gacchasi) — “<strong>You are going</strong>”</li><li>3rd Person Singular Present: <strong>सः / सा / ते गच्छति </strong>(saḥ gacchati) — “<strong>He/She/They is/are going</strong>”</li></ul><p><strong>English</strong>:</p><ul><li>1st Person Singular Present: “<strong>I am going</strong>”</li><li>2nd Person Singular Present: “<strong>You are going</strong>”</li><li>3rd Person Singular Present: “<strong>He/She/They is/are going</strong>”</li></ul><h4><strong>Sandhi (Phonetic combination with rules)</strong></h4><p><strong>Sanskrit</strong> has <strong>sandhi rules</strong>, which are <strong>phonological</strong> changes that occur at word boundaries when certain sounds come together. <br>These transformations create fluidity and harmony in sound, but they complicate the written and spoken language.</p><p><strong>English</strong> has minimal phonological changes at word boundaries and does not have a system of <strong>sandhi</strong>.</p><h4><strong>Example</strong>:</h4><p>In <strong>Sanskrit<br> रामः + अत्र = रामोऽत्र</strong> (rāmaḥ + atra = rāmo’tra) — “<strong>Rama is here.</strong>”</p><p>In <strong>English</strong>, word boundaries remain clear<br> “<strong>Rama is here</strong>.”</p><h4><strong>Samāsa</strong> — Introduction</h4><p><strong>Samāsa</strong> (<strong>समास</strong>) refers to the process of compounding two or more words in <strong>Sanskrit</strong> to form a new, concise word with a specific meaning. It reduces the need for lengthy phrases and is a key feature of <strong>Sanskrit grammar</strong>.<br>There are 4 types of <strong>Samāsa, अव्ययीभाव</strong>, <strong>तत्पुरुष</strong>, <strong>द्वन्द्व</strong> and <strong>बहुब्रीह</strong>.</p><p><strong>English</strong> does not have compounding feature.</p><h4>Example:</h4><p>In <strong>Sanskrit<br>सत् + वैद्य: = सवैद्य:</strong> ( Good + doctor = Good doctor) is a type of <strong>तत्पुरुष समास</strong></p><p>In <strong>English</strong><br>We just say “<strong>Good doctor</strong>”</p><h3>Conclusion</h3><p><strong>Sanskrit</strong> is an inflected language with a complex system of morphology that allows for flexible word order and rich expressiveness. <strong>English</strong>, in contrast, has less inflection and relies more on strict word order to convey meaning. These differences highlight how deeply the structure of a language can influence its grammar, sentence construction, and overall usage.<br>In the next blog, I’ll write about how we improved <strong>Sanskrit</strong> generation for Llama-3.1-8B and Gemma -2-9B models on question-answering and translation tasks.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4af1da6aa789" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>