<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Alexandros Chariton on Medium]]></title>
        <description><![CDATA[Stories by Alexandros Chariton on Medium]]></description>
        <link>https://medium.com/@alexandros_chariton?source=rss-9d6b89ec3064------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*F8zwadQl2P5vXBdNvI6gJg.jpeg</url>
            <title>Stories by Alexandros Chariton on Medium</title>
            <link>https://medium.com/@alexandros_chariton?source=rss-9d6b89ec3064------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 13:57:44 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@alexandros_chariton/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Engaging Reddit Comment Generation with Multimodal Generative AI: Fine-Tuning Pixtral for social…]]></title>
            <link>https://medium.com/@alexandros_chariton/engaging-reddit-comment-generation-with-multimodal-generative-ai-fine-tuning-pixtral-for-social-51a05f5bc208?source=rss-9d6b89ec3064------2</link>
            <guid isPermaLink="false">https://medium.com/p/51a05f5bc208</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[social-media]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Alexandros Chariton]]></dc:creator>
            <pubDate>Sun, 05 Jan 2025 13:44:21 GMT</pubDate>
            <atom:updated>2026-09-30T16:44:01.012Z</atom:updated>
            <content:encoded><![CDATA[<h3>Fine-Tuning a Multimodal LLM for Reddit Engagement</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/907/1*ga_HXxRarUFZv1B0mDpeBw.png" /></figure><h3>Introduction</h3><p>This guide is a continuation of my previous <a href="https://medium.com/@alexandros_chariton/how-to-fine-tune-llama-3-2-instruct-on-your-own-data-a-detailed-guide-e5f522f397d7">post</a>, in which I describe in detail how I made a language model respond sarcastically. In this blog post I explore how AI can improve social media engagement by fine-tuning <a href="https://mistral.ai/news/pixtral-12b/">Pixtral-12B</a>, a multimodal LLM recently released from <a href="https://mistral.ai/">Mistral AI</a>. Specifically, my goal was to generate witty and engaging comments for posts on <a href="https://www.reddit.com/r/memes/">r/memes</a>, one of <a href="https://www.reddit.com/">Reddit’s</a> most active communities.</p><p>In this experiment, I fine-tuned the LLM on a dataset of top-voted Reddit comments, training it to excel in one task: generating comments that are likely to resonate with the community. Basically my goal was to change the behavior of the model so Reddit users would find its generated content more relatable. This blog walks you through the data collection process to training the model on the cloud, then testing it with reddit posts (got a fancy top 5% badge!) and finally sharing it on <a href="https://huggingface.co/">Hugging Face</a>!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/283/1*EBpZXyzeciS1VRfDY44ctQ.png" /></figure><p>You can find the entire code in this <a href="https://github.com/AlexandrosChrtn/pixtral-finetune/">GitHub repository</a>. Feel free to directly <a href="https://huggingface.co/AlexandrosChariton/Reddit-memes-pixtral-12B-v4">skip to my model</a>, if your goal is scoring Reddit points, but you should read this if you’re interested in the technical details and you want to replicate the pipeline for a similar problem of yours!</p><p><strong>Why r/memes?</strong></p><ul><li><strong>Massive Audience: </strong>As the 8th largest subreddit, r/memes hosts many posts for its 35 million members.</li><li><strong>Viral Content</strong>: Posts often attract interest both inside and outside Reddit.</li><li><strong>Strict Moderation</strong>: Posts must adhere to a specific meme format, combining images and a short title in text.</li><li><strong>Memes:</strong> are a cool way to communicate with other people.</li></ul><p><strong>Why Pixtral-12B?</strong></p><ul><li><strong>Multimodality:</strong> Pixtral-12B (12B for 12 billion parameters) stands out for its ability to process both text and images as inputs, making it an ideal candidate for understanding memes, which rely heavily on visual humor.</li><li><strong>Size:</strong> It is large enough to capture more complex concepts present in our scenario, as the input could be literally anything, but at the same time it’s not extremely heavy and one can work with it without paying a million dollars for computational resources.</li></ul><h4><strong>Prompt Engineering vs Fine-Tuning</strong></h4><p>Training large language models from scratch is impractical for most individuals and organizations due to the extensive data and computational resources required. Instead, what is feasible most of the time is to start from a pre-trained open source model and to train it for a specific task.</p><p>The need for better performance in a smaller range of applications leads to what we call “fine-tuning”. Imagine a scenario where you need the model to do one simple thing and you have a bunch of input-output data to describe the task. With fine-tuning you can end up with an AI model that is unique, built on top of Pixtral and has undergone a few extra training rounds with your data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/926/1*SOYrWRh3gCcpQfqnOUnk2w.png" /><figcaption>How input-output pairs look like to optimize for sarcastic responses</figcaption></figure><p>The new model will likely have internal representations and outputs that are more favorable towards your area of interest.</p><p>You might have heard of the term “Prompt Engineering”. This practice is about changing the user prompt, to make the model perform in a specific way, without working on the model itself. To put it simply, it is about finding the optimal phrase that unlocks the model’s potential.</p><p>How does prompt engineering compare to the fine-tuning approach? Do we expect the latter to work better? In general, I would argue, fine-tuning is likely to offer improved performance. Of course these are two separate tactics, they can coexist and it is up to the engineer to decide if fine-tuning on top of prompting for a given application is worth it. My take on this “debate” is that prompting is useful but it is a way of making a machine suitable for a task by only tuning the input signal to minimize the output error. I cannot see any reason (other than short term cost) why you should restrict yourself to changing only the input and not directly work on the machine, especially when this is done by providing informative input — output pairs that describe the task you’re interested in.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/725/0*YrHm1Bg0btQBUhjf" /><figcaption>Differences between fine tuning and prompt engineering</figcaption></figure><h3>Getting a GPU</h3><p>When it comes to LLMs, whether we’re talking about training or inference, somehow a GPU is involved. GPUs are a great fit for ML because the number of operations required is large but parallelizable. Since not everyone has one of these in their personal computer, I’ll be showcasing what I did to train this model on a cloud service. My choice here was <a href="https://www.runpod.io/">RunPod</a>, a service you can use to rent this hardware for a fair price. If you’re interested in such applications, you might find that this is an expensive endeavor, however, keep in mind that in real life scenarios this fine-tuning needs to be done only once and then, after creating the model one is able to run it in production using cheaper hardware.</p><p>The process here went like this:</p><ul><li>I created an account.</li><li>Added some funds to my account (8$ were enough for this).</li><li>Deployed a “pod” which is basically a VM, with one or more GPUs, adding up to at least 96GB memory.</li><li>Connected to my machine in a jupyter environment from Runpod’s UI. The environment is user friendly and is good enough even for beginners.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*J6hIXGz4EZptNiUc" /><figcaption>After setting up your pod, the environment looks like this</figcaption></figure><p>When working on a freshly created VM, it is a good idea to run the following in a terminal, just to update its software:</p><pre>apt update<br>apt upgrade -y</pre><p>If you haven’t worked with GPU machines in the past, you can inspect the situation in your graphics card with:</p><pre>nvidia-smi</pre><p>You should see something like the following, with information about your card’s utilization, versions, power consumption etc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Nhfb1SRC_V9roSCM" /><figcaption>The result of running nvidia-smi in a terminal</figcaption></figure><p>Depending on the application you will need a different machine. In this case, I did not go for a full fine-tune of Pixtral-12B, so I used a popular method named <a href="https://arxiv.org/abs/2106.09685">LoRA</a> (Low Rank Adaptation). This method is about increasing the computational efficiency of the training process at the expense of accuracy. Basically, instead of updating every single weight in the network, I removed a few degrees of freedom and practically reduced the number of independent parameters of the model.</p><p>Talking numbers, if we were to fully train the model for the task, we would need <a href="https://huggingface.co/docs/transformers/en/perf_train_gpu_one">roughly 200GB GPU</a> memory to store all the weights, gradients and everything else produced during the training process. With 96GB we can train about 3 to 4 percent of the model’s total parameters. Also, in general, if you can get this much memory with a single GPU it’s good, otherwise the entire process is going to be slightly slower, due to the communication overhead between the devices.</p><h3>Data collection</h3><p>Good news here! Reddit has a Python API for developers we can use named <a href="https://github.com/praw-dev/praw">PRAW</a>. With a small python script I fetched the top and the hot posts from the subreddit of my choice along with their comments and data such as the number of likes for each comment.</p><p>To do this in your machine, you should install a few python libraries. All of them can be found in <a href="https://github.com/AlexandrosChrtn/pixtral-finetune/blob/main/requirements.txt">this file named requirements.txt</a> and can be installed by running this on terminal:</p><pre>pip install -r requirements.txt</pre><p>Now, after the installation above, the crawling script is the following:</p><pre>import os<br>import requests<br>import praw<br>import pandas as pd<br>from tqdm import tqdm<br>from PIL import Image<br>import io<br><br>&quot;&quot;&quot;<br>Reddit Meme Scraper<br><br>This script scrapes memes and their top comments from specified subreddits,<br>processes the images to a standard format, and saves the data to CSV.<br><br>Requirements:<br>    - Replace &quot;your_client_id&quot; and &quot;your_client_secret&quot; with the ones tied to your account<br>&quot;&quot;&quot;<br><br># Constants<br>MIN_COMMENT_SCORE = 12<br>IMAGE_SIZE = 512<br>IMAGE_QUALITY = 95<br>MAX_COMMENTS = 10<br>VALID_IMAGE_EXTENSIONS = (&#39;.jpg&#39;, &#39;.png&#39;, &#39;.gif&#39;, &#39;.jpeg&#39;, &#39;.webp&#39;)<br>SUBREDDIT_NAME = &quot;memes&quot;<br>NUM_POSTS = 1000<br>OUTPUT_FOLDER = &quot;imgs&quot;<br>CSV_FILE = &quot;memes_data.csv&quot;<br><br># Configure Reddit API<br>reddit = praw.Reddit(<br>    client_id=&quot;your_client_id&quot;,<br>    client_secret=&quot;your_client_secret&quot;,<br>    user_agent=&#39;fetching_reddit_data&#39;,<br>    )<br><br># Ensure output folder exists<br>os.makedirs(OUTPUT_FOLDER, exist_ok=True)<br><br># Initialize data collection<br>data = []<br><br># Fetch posts<br>subreddit = reddit.subreddit(SUBREDDIT_NAME)<br>for category in [&#39;hot&#39;, &#39;top&#39;]:<br>    subreddit_category = (<br>        subreddit.top(limit=NUM_POSTS, time_filter=&#39;year&#39;) if category == &#39;top&#39; <br>        else subreddit.hot(limit=NUM_POSTS)<br>    )<br>    <br>    for post in tqdm(subreddit_category, total=NUM_POSTS, desc=f&quot;Fetching {category} posts&quot;):<br>        <br>        # Skip if we already processed this post<br>        if any(item[&#39;post_id&#39;] == post.id for item in data):<br>            continue<br><br>        # Filter for image posts<br>        if not post.url.endswith(VALID_IMAGE_EXTENSIONS):<br>            continue<br><br>        # Fetch top comments<br>        post.comment_sort = &quot;top&quot;<br>        post.comments.replace_more(limit=0)<br>        top_comments = [comment for comment in post.comments[:MAX_COMMENTS]]<br><br>        comments_data = []<br>        for comment in top_comments:<br>            # Filter out bot comments, comments with brackets like [deleted], and comments with low likes<br>            if (&quot;am a bot&quot; in comment.body.lower() or <br>                &quot;[&quot; in comment.body or <br>                &quot;]&quot; in comment.body or <br>                comment.score &lt; MIN_COMMENT_SCORE):<br>                continue<br>            comments_data.append({<br>                &quot;comment_body&quot;: comment.body,<br>                &quot;comment_likes&quot;: comment.score<br>            })<br><br>        # Download image<br>        image_path = os.path.join(OUTPUT_FOLDER, f&quot;{post.id}.jpg&quot;)<br>        try:<br>            img_data = requests.get(post.url, stream=True)<br>            if img_data.status_code == 200:<br>                # Load image from bytes<br>                image = Image.open(io.BytesIO(img_data.content))<br>                <br>                # Convert to RGB if necessary (handles PNG with transparency)<br>                if image.mode in (&#39;RGBA&#39;, &#39;P&#39;):<br>                    image = image.convert(&#39;RGB&#39;)<br>                <br>                # Resize image to IMAGE_SIZE x IMAGE_SIZE maintaining aspect ratio<br>                image.thumbnail((IMAGE_SIZE, IMAGE_SIZE))<br>                <br>                # Create a new IMAGE_SIZE x IMAGE_SIZE white image<br>                new_image = Image.new(&#39;RGB&#39;, (IMAGE_SIZE, IMAGE_SIZE), &#39;white&#39;)<br>                <br>                # Paste the resized image in the center<br>                offset = ((IMAGE_SIZE - image.size[0]) // 2, (IMAGE_SIZE - image.size[1]) // 2)<br>                new_image.paste(image, offset)<br>                <br>                # Save the final image<br>                new_image.save(image_path, &#39;JPEG&#39;, quality=IMAGE_QUALITY)<br>            else:<br>                print(f&quot;Failed to download image from {post.url} - Status code: {img_data.status_code}&quot;)<br>                continue<br>        except requests.exceptions.RequestException as e:<br>            print(f&quot;Request exception while downloading image from {post.url}: {e}&quot;)<br>            continue<br>        except Exception as e:<br>            print(f&quot;Error downloading image from {post.url}: {e}&quot;)<br>            continue<br><br>        # Add to data<br>        data.append({<br>            &quot;post_id&quot;: post.id,<br>            &quot;post_title&quot;: post.title,<br>            &quot;post_url&quot;: post.url,<br>            &quot;image_path&quot;: image_path,<br>            &quot;comments&quot;: comments_data<br>        })<br><br># Save data to CSV<br>rows = []<br>for item in data:<br>    for comment in item[&quot;comments&quot;]:<br>        rows.append({<br>            &quot;post_id&quot;: item[&quot;post_id&quot;],<br>            &quot;post_title&quot;: item[&quot;post_title&quot;],<br>            &quot;post_url&quot;: item[&quot;post_url&quot;],<br>            &quot;image_path&quot;: item[&quot;image_path&quot;],<br>            &quot;comment_body&quot;: comment[&quot;comment_body&quot;],<br>            &quot;comment_likes&quot;: comment[&quot;comment_likes&quot;]<br>        })<br><br>df = pd.DataFrame(rows)<br>df.to_csv(CSV_FILE, index=False)<br>print(f&quot;Saved data to {CSV_FILE}. Images saved in {OUTPUT_FOLDER}.&quot;)</pre><p>You need to replace your_client_id and your_client_secret with the credentials attached to your reddit account. This info I found in <a href="https://www.reddit.com/prefs/apps">Reddit’s page for developers</a> and fetching it was pretty straightforward. Running this script with python3 fetch_data.py in terminal automatically creates a folder named imgs filled with the images named post_id.jpg and a csv named memes_data.csv that looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*uV3VuEQCgZY6Ub4N" /><figcaption>Example of data crawled from reddit</figcaption></figure><p>In case you’re wondering, this “sad hulk” thing is something like an inside joke in this subreddit, related to them changing its logo, which used to be a cat.</p><p>As you can see, each row describes a comment. Multiple comments can be traced back to the same post, identified by a post_id. I also keep track of the comment likes. To end up with a reasonable number of comments, I dropped the comments with fewer than 12 likes. Naturally a different approach can be taken here but either way, for training, we need the expected input for the application (images and post titles) and an expected output (highly rated comments). At this step, it is strongly advised to invest in data quality. Fine-tuning does not require many examples to be effective, however the quality of the instances passed to the model needs to be top notch. I removed comments that were [deleted], as well as those with links, gifs and bot responses.</p><h3>Training</h3><p>Training an LLM locally is not as complex as one might think. We are in luck here because there are python libraries like <a href="https://github.com/huggingface/transformers">transformers</a> and <a href="https://github.com/huggingface/peft">peft</a> with high level APIs that are doing all the heavy burden, so we barely have to work on any implementation. In this example, I basically used the tools that were at my disposal and ensured that my data made sense for the task I wanted to solve. Let’s break it down into steps: <em>(if you want to copy paste the entire thing, </em><a href="https://github.com/AlexandrosChrtn/pixtral-finetune/blob/main/train.py"><em>here</em></a><em>, if you want to skip to the model I trained, </em><a href="https://huggingface.co/AlexandrosChariton/Reddit-memes-pixtral-12B-v4/tree/main"><em>here</em></a><em>)</em></p><ul><li>The first part of the code imports a bunch of libraries, assuming one went through with the installation with the <a href="https://github.com/AlexandrosChrtn/pixtral-finetune/blob/main/requirements.txt">requirements.txt</a> file in data collection</li></ul><pre>import torch<br>from transformers import AutoProcessor, LlavaForConditionalGeneration, Trainer, TrainingArguments<br>from datasets import load_dataset<br>from peft import get_peft_model, LoraConfig, TaskType<br>from PIL import Image<br>import numpy as np<br>from datasets import DatasetDict<br>from huggingface_hub import login, upload_folder, create_repo<br>from transformers import Trainer, TrainingArguments<br>import torch</pre><ul><li>The second part loads the original model, <a href="https://huggingface.co/mistral-community/pixtral-12b/tree/main">Pixtral from Hugging Face</a>. This downloads the model and it will take some time so be patient. Notice the &quot;device_map&quot; part where you can use &quot;mps&quot; for MacBooks, &quot;cuda&quot; for an Nvidia GPU or &quot;auto&quot; to let it decide which GPUs to put the model on. Also, set the tokenizer pad token to the appropriate id, found in the original model’s <a href="https://huggingface.co/mistral-community/pixtral-12b/tree/main">tokenizer information</a>.</li></ul><pre># Load the model and processor<br>model_id = &quot;mistral-community/pixtral-12b&quot;<br>model = LlavaForConditionalGeneration.from_pretrained(model_id, torch_dtype=torch.float16, device_map=&quot;auto&quot;)<br>processor = AutoProcessor.from_pretrained(model_id)<br><br># Ensure the processor has padding and a chat template for structured prompts<br>processor.tokenizer.pad_token = &quot;&lt;pad&gt;&quot;<br>processor.tokenizer.pad_token_id = 11<br>processor.tokenizer.chat_template = processor.chat_template</pre><ul><li>The code then sets up the LoRA configuration, allowing you to adjust several hyperparameters to balance performance and memory usage. Increasing the LoRA rank or adding more target_modules raises the number of trainable parameters. The dropout parameter is common in Neural Networks and it helps to prevent overfitting, which is memorization of the input data.</li></ul><pre># Define LoRA configuration<br>lora_config = LoraConfig(<br>    task_type=TaskType.CAUSAL_LM,<br>    r=92,  # Rank of the LoRA matrix<br>    lora_alpha=12, # Scaling factor for LoRA<br>    lora_dropout=0.15,<br>    target_modules=[&#39;q_proj&#39;, &#39;k_proj&#39;, &#39;v_proj&#39;, &#39;o_proj&#39;, &#39;language_model.lm_head&#39;, &#39;up_proj&#39;, &#39;down_proj&#39;, &#39;gate_proj&#39;,<br>                   &quot;multi_modal_projector.linear_1&quot;, &quot;multi_modal_projector.linear_2&quot;]<br>)<br><br># Apply the LoRA configuration to the model<br>model = get_peft_model(model, lora_config)<br>model.print_trainable_parameters()</pre><ul><li>Then, it loads the dataset and splits it into train and test splits. Here I separated grouping by post ids. I trained the model to generate top comments for a set of posts, with multiple comments associated to each post and then validated its performance on a different set of posts, again, with multiple comments for each. This was done to avoid any data leakage between the two sets.</li></ul><pre># Load the dataset from a CSV file<br>dataset = load_dataset(&quot;csv&quot;, data_files=&quot;memes_data.csv&quot;)<br><br># Convert the dataset to a pandas DataFrame for easier manipulation<br>df = dataset[&quot;train&quot;].to_pandas()<br><br># Extract unique post IDs for splitting the dataset<br>unique_post_ids = df[&#39;post_id&#39;].unique()<br><br># Randomly split post IDs into training and testing sets<br>np.random.seed(42)  # Reproducibility<br>test_post_ids = np.random.choice(unique_post_ids, size=int(0.01 * len(unique_post_ids)), replace=False) # 1% for validating<br>train_post_ids = np.setdiff1d(unique_post_ids, test_post_ids)<br><br># Split the DataFrame into training and testing sets based on post IDs<br>train_df = df[df[&#39;post_id&#39;].isin(train_post_ids)]<br>test_df = df[df[&#39;post_id&#39;].isin(test_post_ids)]<br><br># Convert the DataFrames back to Hugging Face datasets<br>train_dataset = dataset[&quot;train&quot;].from_pandas(train_df)<br>test_dataset = dataset[&quot;train&quot;].from_pandas(test_df)<br><br># Print the sizes of the train and test datasets<br>print(f&quot;Train dataset size: {len(train_dataset)}&quot;)<br>print(f&quot;Test dataset size: {len(test_dataset)}&quot;)</pre><ul><li>Afterwards the chat template is set, which is the prompt that wraps every instance in our data. This is the part of the code one needs to look after the most, when changing the topic of the fine-tuning. From a practical standpoint, if you were to employ an LLM for a very specific task, usually the prompt that wraps around the part that changes does not need to change. To better understand this, here is the prompt I used:</li></ul><pre># Define a function to apply the chat template to each example<br>def apply_chat_template(example):<br>    try:<br>        # Skip examples with long comments or URLs<br>        if len(example[&quot;comment_body&quot;]) &gt; 240 or &quot;https://&quot; in example[&quot;comment_body&quot;]:<br>            return {&quot;prompt&quot;: &quot;&quot;, &quot;image&quot;: None}<br>        <br>        # Load the image from the specified path<br>        imgpath = example[&quot;image_path&quot;]  # Assuming your dataset has image paths<br>        image = Image.open(imgpath).convert(&quot;RGB&quot;)<br><br>        # Construct the chat messages for the model<br>        messages = [<br>            {<br>                &quot;role&quot;: &quot;user&quot;,<br>                &quot;content&quot;: [<br>                    {&quot;type&quot;: &quot;text&quot;, &quot;content&quot;: &quot;Come up with a comment that will get upvoted by the community for a reddit post in r/memes. Provide the comment body with text and nothing else. The post has title: &#39;&quot; + example[&quot;post_title&quot;] + &quot;&#39; and image:\n&quot;},<br>                    {&quot;type&quot;: &quot;image&quot;}  # Represents the image input for the model<br>                ]<br>            },<br>            {<br>                &quot;role&quot;: &quot;assistant&quot;,<br>                &quot;content&quot;: example[&quot;comment_body&quot;]  # Expected output<br>            },<br>        ]<br>        <br>        # Apply the chat template using the processor<br>        prompt = processor.tokenizer.apply_chat_template(<br>            messages, tokenize=False, add_generation_prompt=False<br>        )<br>        return {&quot;prompt&quot;: prompt, &quot;image&quot;: image}<br>    except Exception as e:<br>        print(f&quot;Error processing example {example}: {e}&quot;)<br>        return None  # Skip this example if an error occurs<br><br># Apply the chat template function to the datasets<br>new_train_dataset = train_dataset.map(apply_chat_template, batched=False)<br>new_test_dataset = test_dataset.map(apply_chat_template, batched=False)<br><br># Combine the processed datasets into a DatasetDict<br>new_dataset = DatasetDict({<br>    &#39;train&#39;: new_train_dataset,<br>    &#39;test&#39;: new_test_dataset<br>})<br><br># Filter out examples where the mapping function returned None<br>new_dataset = new_dataset.filter(lambda x: len(x[&#39;prompt&#39;]) &gt; 1)</pre><ul><li>Next, it tokenizes the dataset, which is common practice in language applications with ML. It maps each token (piece of text) to an id that is later mapped to a vector. This vector has continuous values and it arithmetically represents the “meaning” of the text in a way that is understood by the model, that only speaks math.</li></ul><pre># Define a function to tokenize the data<br>def tokenize_function(example):<br>    # Tokenize the text and image inputs<br>    inputs = processor(<br>        text=example[&quot;prompt&quot;],<br>        images=example[&quot;image&quot;],<br>        return_tensors=&quot;pt&quot;,<br>        padding=&quot;max_length&quot;,<br>        truncation=True,<br>        max_length=1350<br>    )<br>    # Adjust the tensor dimensions<br>    inputs[&#39;pixel_values&#39;] = inputs[&#39;pixel_values&#39;][0][0]<br>    inputs[&#39;input_ids&#39;] = inputs[&#39;input_ids&#39;].squeeze(0)<br>    inputs[&#39;attention_mask&#39;] = inputs[&#39;attention_mask&#39;].squeeze(0)<br>    <br>    # Prepare labels for training<br>    input_ids = inputs[&quot;input_ids&quot;]<br>    labels = input_ids.clone()  # Clone input IDs for labels<br>    labels[labels == processor.tokenizer.pad_token_id] = -100  # Ignore padding in loss calculation<br>    inputs[&quot;labels&quot;] = labels    <br>    <br>    return inputs<br><br># Apply tokenization to the dataset<br>tokenized_dataset = new_dataset.map(tokenize_function)</pre><ul><li>Then the training part of the code is simple. All I did was to use tools at my disposal from the library <a href="https://github.com/huggingface/transformers">transformers</a>. The tricky part here is filling in the arguments in TrainingArguments, which requires experience in ML and a tiny bit of mathematical background. I’m not going to dive deep into it, to avoid overcomplicating this tutorial, but feel free to play around with these <a href="https://huggingface.co/docs/transformers/v4.47.1/en/main_classes/trainer#transformers.TrainingArguments">(and include more arguments)</a> if you’re feeling adventurous. The results of the fine-tuning, so the new model that is all yours, is saved in my_llm_dir .</li></ul><pre># Define training arguments for the Trainer<br>training_args = TrainingArguments(<br>    output_dir=&quot;./results&quot;,<br>    evaluation_strategy=&quot;steps&quot;,<br>    eval_steps=250,<br>    logging_steps=150,<br>    gradient_accumulation_steps=3,<br>    save_steps=400,<br>    per_device_train_batch_size=1,  # Adjust based on hardware capabilities<br>    per_device_eval_batch_size=1,<br>    num_train_epochs=2,<br>    fp16=True,  # Use mixed precision if supported<br>    report_to=&quot;none&quot;,<br>    log_level=&quot;info&quot;,<br>    learning_rate=2e-5,<br>    max_grad_norm=2,<br>    lr_scheduler_type=&#39;linear&#39;<br>)<br><br># Initialize the Trainer for model fine-tuning<br>trainer = Trainer(<br>    model=model,<br>    args=training_args,<br>    train_dataset=tokenized_dataset[&quot;train&quot;],<br>    eval_dataset=tokenized_dataset[&quot;test&quot;],<br>    tokenizer=processor.tokenizer,<br>)<br><br># Fine-tune the model<br>trainer.train()<br><br># Save the fine-tuned model and tokenizer<br>trainer.save_model(&quot;./my_llm_dir&quot;)<br>processor.tokenizer.save_pretrained(&quot;./my_llm_dir&quot;)</pre><ul><li>For the last part of the code, if you want to share your work with others, the best place to do this in AI is <a href="https://huggingface.co/">Hugging Face</a>. You need to create an account there and run the following code, but ensure that you change your_hf_name/my_awesome_LLM with your Hugging Face name and the name of the model you created. Also, you need to replace your_hf_token with a token attached to your profile, generated from <a href="https://huggingface.co/settings/tokens">this page</a>.</li></ul><pre>print(&#39;Upload to Hugging Face!&#39;)<br># Login to Hugging Face (you&#39;ll need your token from https://huggingface.co/settings/tokens)<br>login(token=&quot;your_hf_token&quot;)<br><br># Replace with your repo name on Hugging Face<br>repo_id = &quot;your_hf_name/my_awesome_LLM&quot;<br><br>try:<br>    # Attempt to create the repo if it doesn&#39;t exist<br>    create_repo(repo_id.split(&#39;/&#39;)[-1], private=True, use_auth_token=True)  # Set private=True to make the repo private<br>    print(f&quot;Repo {repo_id.split(&#39;/&#39;)[-1]} created and set to private.&quot;)<br>except Exception as e:<br>    print(f&quot;Repo {repo_id.split(&#39;/&#39;)[-1]} already exists or could not be created: {e}&quot;)<br><br># Upload the saved model to the Hugging Face repository<br>upload_folder(<br>    folder_path=&quot;my_llm_dir&quot;,<br>    repo_id=repo_id,<br>    use_auth_token=True,<br>    path_in_repo=&quot;.&quot;  # This pushes the contents directly to the root of the repo<br>)</pre><p>To run the script that trained the Reddit comment expert model, execute the train.py script from the <a href="https://github.com/AlexandrosChrtn/pixtral-finetune">GitHub repo</a> with</p><pre>python3 train.py</pre><p>The point of the code is to be easily usable by someone else, for any task that requires an LLM parsing a visual signal. All you have to do to make it suitable for your problem is:</p><ul><li>Replace the input-output data with high quality examples of your task. You might need a few hundred or a few thousand datapoints, but you don’t need trillions of tokens because you’re building on top of a pre-trained model.</li><li>Replace the prompt in train.py with the prompt that works for your task. Here it was something like “generate a top Reddit comment”. For you it might be “tell me what time it is on this image of a clock” or whatever.</li><li>Play around with any user defined hyperparameters in TrainingArguments if you need to due to memory restrictions or if you the validation loss does not decrease during training.</li></ul><p>Sit tight, training normally takes some time. You should have all the information automatically printed in your console. Do not forget to keep an eye on the memory consumption on your GPU to ensure your model is using it!</p><h4>Running my version of Reddit-Pixtral</h4><p>I trained my model for 2 epochs and uploaded the results in <a href="https://huggingface.co/AlexandrosChariton/Reddit-memes-pixtral-12B-v4">this Hugging Face repository</a>. If you want to use my model, all you have to do is visit it, download it and follow the instructions and the code provided in <a href="https://huggingface.co/AlexandrosChariton/Reddit-memes-pixtral-12B-v4/blob/main/README.md">README.md</a>.</p><h3>Differences between Pixtral and Reddit-Pixtral</h3><p>For this part, the straightforward way to test the model was to comment on actual Reddit posts and see how the community reacts. But, before doing this there are a few key points that need to be brought up, just to showcase how this process affected the original AI model. In short, here are some key differences:</p><ul><li>The custom model tuned on Reddit comments is more prone to use vulgar language.</li><li>Sometimes the output of the model is hard to interpret, but isn’t this the case for many human generated comments in Reddit?</li><li>The “social media savvy” model comes up with personal stories that are entirely fictional (“My father is from Italy and he…”, “As a student who attended may parties during college…”) but sometimes the community expresses positive sentiment towards such stories. I never saw that in the original model, which opted for more emojis and a more “neutral” and maybe “flavorless” attitude towards the task.</li><li>Fine-tuning the model nails the question-answer format we need. The response was a direct and short comment not something like: “Here is the comment for this post: blah blah blah”.</li><li>It is likely to generate misinformation, for instance “in Ohio there’s a law that…”</li><li>It might come up with insults towards the original post.</li></ul><p>I also noticed that the models had a hard time understanding memes that had smaller text imprinted on the images. In general, both Pixtral and Reddit-Pixtral did a good job staying on topic, especially if we take the difficulty of the task into consideration. It is not easy to come up with a witty response for a wide range of inputs for so many subject. Also, it is even more difficult to stay up to date with the latest memes that are about current events the models have no knowledge about.</p><h3>Testing</h3><p>To evaluate the model on actual Reddit data, I went to <a href="https://www.reddit.com/r/memes/">r/memes</a>, went to the latest posts, commented on a few of those and waited. Oh, and I made sure I was not shadowbanned on Reddit first. The following day I had the top comment for the post in the image and a few more successful cases of the LLM comment generator. Notice how the top comment was a completely fabricated testimony about my sleep schedule.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-sjHHmEAI9qGY4DyRCjLCQ.png" /><figcaption>This comment does not describe me, I do not stay up until 2 or 3 am without a reason</figcaption></figure><p>However, since I’m not a regular Reddit user, I did not know better. I soon found out that the moderators of the subreddit are very strict and for some reason the majority of the new posts created were deleted. I speculate this had to do with non-original content. It could also be related to the nature of the posts because most of them were about a specific event that had taken place recently in December 2024.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9s7jwi23MLCI7LwtjDTT-Q.png" /><figcaption>Reddit-Pixtral: “I hate traveling in Europe and people from my country are louder than the locals”</figcaption></figure><p>The second most liked comment that I got back was for the image right above and it was “I hate traveling in Europe and people from my country are louder than the locals”. Admittedly I would really, really struggle to come up with myself, people from my country are typically well behaved!</p><p>Before I knew it, I earned the “Top 5% Commenter” badge. It doesn’t mean much, as spamming comments on multiple posts can probably get you one too. I didn’t even know these badges existed, but nonetheless, with little effort, I reached a good rank in this month’s standings.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/227/1*v8LhI_6SK5NuIUFikTdo1Q.png" /><figcaption>Soon enough I was on top 5% commenters of the subreddit</figcaption></figure><h3>Conclusion</h3><p>This guide was originally created because I wanted to experiment with the combination of LLMs and visual input in a realistic setting and I wanted to help the community who’s also interested in this. I tried to replicate what I did in a way that would be useful for me, if I was working on a similar problem.</p><p>The process above describes the steps towards building your very own LLM that works for YOU. It dives in the process of sharing the model with the community, but it does not get into the deployment part of the model, if you were to run it effectively in production (this might be a cool idea for another post).</p><p>Also, if you visited my model’s page on Hugging Face, you might have noticed the ...-v4 suffix. This is because I’ve tried this experiment a number of times and the model ended up being weird and I could not understand its generated content, so I filtered out upvoted comments that were “distasteful” for my standards. This version might also generate unwanted content, but it seemed to be an improvement compared to the other ones, which will not be made public.</p><p><strong>Let me know</strong> if this post hit that goal or any ML applications you would like to see in the future. If you have any questions, feedback of any kind, or if you want to have a chat and collaborate on cool projects, feel free to reach out on <a href="https://www.linkedin.com/in/alexandros-chariton-06a6aa1a4/">LinkedIn</a>, <a href="https://github.com/AlexandrosChrtn/pixtral-finetune">Github</a>, <a href="https://x.com/alexchrtn">Twitter</a> or in the comment section below!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=51a05f5bc208" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Fine-Tune Llama-3.2 on your own data: A detailed guide]]></title>
            <link>https://medium.com/@alexandros_chariton/how-to-fine-tune-llama-3-2-instruct-on-your-own-data-a-detailed-guide-e5f522f397d7?source=rss-9d6b89ec3064------2</link>
            <guid isPermaLink="false">https://medium.com/p/e5f522f397d7</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[hugging-face]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Alexandros Chariton]]></dc:creator>
            <pubDate>Sun, 20 Oct 2024 21:09:20 GMT</pubDate>
            <atom:updated>2025-01-18T15:57:33.312Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*hWtHNBdZmp8XPIcA" /></figure><h4>Introduction</h4><p>The release of <a href="https://huggingface.co/collections/meta-llama/llama-32-66f448ffc8c32f949b04c8cf">Llama-3.2 collection</a> from Meta marked an important milestone in the open-source AI world. The model of the collection with the most downloads up to this point is the notorious meta-llama/Llama-3.2–11B-Vision-Instruct. Unfortunately we can’t really access it in Europe along with the rest of Meta’s vision models for the Llama-3.2 release, but we can play around with the rest of the models that involve text only! If you want to learn more about technical stuff, or even learn cool stuff about the latest LLM advances, read along friend!</p><p><strong><em>Update</em></strong><em>: If you’re into multimodal LLMs instead, you might be interested </em><a href="https://medium.com/p/51a05f5bc208"><em>in my next post which I think is super cool</em></a><em>!</em></p><figure><img alt="Europe, Greece" src="https://cdn-images-1.medium.com/max/1024/0*BnP7f9AQBHphPC9A" /><figcaption>From Greece, Europe</figcaption></figure><p>Let’s start by briefly describing what fine-tuning is. It is all about adjusting a model to fit your specific needs by tweaking its weights. Imagine you’re dealing with legal documents where words like ‘consideration’ take on a whole new meaning compared to everyday speech or what the model has been trained on. Fine-tuning steps in to make sure the model gets these specialized terms right. It’s not just about words either — you can also set up the model to follow specific rules, like keeping answers short and to the point, or understanding your business needs to a deeper level. So, if you’re planning to deploy it in production, this process transforms a general-purpose model into something custom-built for your data.</p><p>The lightweight models, specifically the 1B and 3B (<em>{N}</em>B = <em>N</em> billion parameters, for positive integer <em>N</em>), are among the most interesting for a variety of reasons.<strong> </strong>They are relatively easy to run locally, unlike the larger versions of LLMs and Meta claims they can be smoothly deployed on hardware found in mobile devices. This opens the door to many applications, as language processing can be done locally, without data leaving the device. As a result, these models are not reliant on a stable internet connection and offer a more private handling of sensitive information. With these advantages, we can expect to see more tools like personal assistants running on our smartphones.</p><p>The <a href="https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/#:~:text=Lightweight%20models,to%20recover%20performance.">reported context length</a> for these models is 128k tokens, which means they can handle reasonably large prompts. The decrease in size was done with minimal penalty in accuracy using techniques such as <a href="https://arxiv.org/abs/1503.02531">knowledge distillation</a>, which utilized the output token probability distributions from larger Llama models as targets for the lightweight variants. Additionally, the Instruct version of the models was fine tuned using both high quality instruction data, as well as synthetic data generated from <a href="https://huggingface.co/meta-llama/Llama-3.1-405B-Instruct">Llama-3.1–405B-Instruct</a>.</p><p>If you used any of these models you’ll probably agree that their performance is better than expected for their size. If you haven’t already you can access them via multiple online endpoints or you can run them locally with your own hardware. Having a GPU is strongly recommended for this part, so if you’re in possession of an Nvidia GPU or an Apple MacBook, it’s time you put it to good use! The following part describes the technical details of running and fine tuning Llama on your computer, as well as sharing the results with your friends!</p><h4>Preparation</h4><p>We will be using the models hosted in <a href="https://huggingface.co/">Hugging Face</a>, the largest platform where users can share code and AI models with one another. Along with it, Python is the tool we need to get started. If you do not have Python installed on your machine, your best bet is to ask ChatGPT for instructions (or the LLM of your choice, OpenAI just works for me). Also, if you’re not in possession of a MacBook, ensure cuda is installed, again, by asking ChatGPT for specific instructions that work for your system.</p><p>You will also need a package installer, namely “pip”, to easily install the prerequisites needed to run these models. If all the above sounds exhausting to you, do not be afraid, LLMs are always there to help. After you ensure Python and pip are installed, we will install a few more tools before we move along to the more interesting parts.</p><p>The tool installation is handled by `pip`. Here is how to install them without fuss:</p><ul><li>Copy <a href="https://github.com/AlexandrosChrtn/llama-fine-tune-guide/blob/main/requirements.txt">this</a> requirements.txt file</li><li>Run pip install -r requirements.txt</li></ul><p>A popular tool you just installed in your computer is the popular <a href="https://github.com/huggingface/transformers">transformers</a> library (134k stars on GH today, good stuff). This is a python tool that allows you to invoke a wide variety of open source models in a few lines of code and use them on demand, assuming your computer can handle them. We will not dive deeper into the details of the library, (to avoid getting sidetracked!) but what is important for someone to know here is the <a href="https://huggingface.co/docs/transformers/en/main_classes/pipelines">pipeline</a> function that allows us to perform a variety of tasks such as summarization, translation or text generation in a few lines of code.</p><h4>Generating Text</h4><p>Typically, in the amazing open source LLM world, each model variant has 2 “flavors”, the Base model and the Instruct model. The base model is the model that has pretty much read a lot of text. Seriously, a lot. When you ask it to generate text, it predicts the next “word” (actually, tokens, but let’s keep it simple), one at a time. Want a whole paragraph? It just repeats that word-predicting process until you’re happy.</p><figure><img alt="autoregressive, predictions, text generation" src="https://cdn-images-1.medium.com/max/771/1*qR9ZN9X7Po7Mg7q8mjeMlw.png" /><figcaption>Autoregressive text generation word by word</figcaption></figure><p>Since the model is only looking at what it’s already generated to guess the next word, it can technically keep going forever. But, if you let it run wild, it might fall into a kind of AI-loop, where it starts to lose focus and ends up repeating the same thing over and over again. This is not surprising, if we consider the fact that at some point, it bases its predictions solely on AI generated content. The property of predicting the text down the road using intermediate text generated by the model is referred to as “autoregressive”. It is pretty much the most effective and practical way we could come up with (for now, hopefully) when it comes to “translating” the problem of text generation to mathematics that we could feed into a computer.</p><p>Now that we’re past the basics, let’s see some code. Running Llama locally is a few lines of code away, because Hugging Face’s transformers library is polite enough to carry the heavy burden. We will invoke their tools and generate text in a few lines of code, as seen below:</p><pre>from transformers import pipeline<br><br>model_id = &quot;meta-llama/Llama-3.2-1B-Instruct&quot;<br>pipe = pipeline(<br>    &quot;text-generation&quot;,<br>    model=model_id,<br>    device_map=&quot;auto&quot;,<br>)<br>messages = [<br>    {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: &quot;You are a helpful assistant.&quot;},<br>    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Should I let my dog sleep on the bed with me?&quot;},<br>]<br>outputs = pipe(<br>    messages,<br>    max_new_tokens=128,<br>    do_sample=True<br>)<br>print(outputs[0][&quot;generated_text&quot;][-1])</pre><p>Saving this into a file (generate_text_example.py for example) and executing it with python3 generate_text_example.py will print the generated answer by the model for the prompt “Should I let my dog sleep on the bed with me?” By the way, it responded with bullet points of pros and cons and a conclusion at the end, declaring that it is up to your lifestyle choices.</p><p>The text will be cut off if we set a max_new_tokens parameter. Furthermore, if you run the exact same code there is no guarantee that you will get the same output. When we generate text, in general, we would rather introduce randomness in the output by not always predicting the most probable word to come next, but rather selecting a word based on the output probability distribution (so the second most probable word is more likely to be chosen than the third and so on). This variation in word choice adds a natural feel to generated text and prevents loops that trap the model into an unwanted state.</p><p>Pretty cool so far, you’ve harnessed the Llama power in a few lines of code and you can use it in your data processing pipeline. But what if you wanted a cooler (albeit less helpful) Llama model that no one else had?</p><h4>Fine tuning</h4><p>The way we achieve this favorable shift for the model’s weights is by passing more data to the model. We will work on fine tuning the Instruct model, which has already been fed data in the form of question-answer pairs.</p><p>For our practical example, we want to fine tune the model to provide short and sarcastic (non) answers to our questions. One might argue that we will start with an astounding piece of technology, a blazing fast LLM assistant and we will end up with something that doesn’t really assist anywhere, but admittedly is fun to play with. This might be true for our case, you can still replace our fine tuning data with anything you find useful for your own aspirations.</p><p>Where can we get data with sarcasm to pass on to our model? Our best bet would be to generate synthetic data with a more powerful model, such as ChatGPT. Ideally we would come up with high quality data ourselves but this way is faster and we would struggle to come up with phrases that live up to the precedent set by — among other gems —</p><blockquote>- Can plants talk?<br>- They can, but they’re very introverted. Don’t try to talk to them.</blockquote><p>OK, nice, impressive, but these models have billions of parameters and they were trained on a million gigabytes of text. How many sarcastic examples do I have to come up with, to meaningfully sway the algorithm towards my desired direction? We will attempt to answer this question by only providing 200 examples. Ideally, at this step we really care about <strong>quality</strong> over quantity. That is why you should consider hand picking examples instead of using synthetic ones, if you’re looking for performance. In any case, the synthetic dataset is <a href="https://github.com/AlexandrosChrtn/llama-fine-tune-guide/blob/main/data/sarcasm.csv">available</a>, if you want to have a good laugh!</p><p>From a technical perspective, the weight updates from these examples will be more impactful compared to data points that the model encountered during its previous training, because these updates came in last and with a larger learning rate (unaffected by <a href="https://arxiv.org/abs/1412.6980">ADAM’s</a> exponential moving averages).</p><p>If you want to use the following code to train your model, ensure your dataset looks like this:</p><pre>question,answer<br>&quot;Who invented the light bulb?&quot;,&quot;Oh yeah, just a little unknown guy named Thomas Edison. You might have heard of him... if you pay attention at all.&quot;<br>&quot;Is the sky blue?&quot;,&quot;Wow, you&#39;re asking that? Next, you&#39;ll tell me water is wet.&quot;<br>&quot;What&#39;s 2 + 2?&quot;,&quot;Oh, such a brain-busting question. It’s 4, obviously. Genius.&quot;</pre><p>So basically a csv with two columns, question and answer.</p><p>Important fact, when you’re dealing with Instruct models, you need to adhere by the following rule: apply the predefined chat template provided by the model distributor (Meta in our case) to your own dataset. This part is included on the code later on, so <strong>you don’t have to do anything </strong>as long as your dataset is just like the one above, but make sure you understand how this works, because it’s fun.</p><p>As mentioned earlier, the Instruct model is just like the Base model, with the difference that it was fine tuned on specific data to make it more assistant-like. This data was in the form of question-answer pairs (and probably system prompts). Therefore, we have no reason to change the way that the model expects those aforementioned input-output pairs. That is why we need this thing called a “chat template”. For the Llama-3.2 models, the chat template for a prompt like “Do you have to whine all the time?” looks like this:</p><pre>&lt;|begin_of_text|&gt;&lt;|start_header_id|&gt;system&lt;|end_header_id|&gt;<br><br>Cutting Knowledge Date: December 2023<br>Today Date: 19 Oct 2024<br><br>You are a helpful assistant.&lt;|eot_id|&gt;&lt;|start_header_id|&gt;user&lt;|end_header_id|&gt;<br><br>Do you have to whine all the time?&lt;|eot_id|&gt;&lt;|start_header_id|&gt;assistant&lt;|end_header_id|&gt;<br><br></pre><p>The training part is also handled by the tools provided by Hugging Face. The following code is the script that does all the training. It starts from loading the model to our GPU and then proceeds to fine-tuning it with the transformer’s Trainer class and saving the results in a directory named fine-tuned-model. Before you execute the code to continue with your fine tuning, ensure that the path to your csv data is provided inside load_dataset.</p><pre>import torch<br>from transformers import AutoModelForCausalLM, AutoTokenizer, Trainer, TrainingArguments<br>from datasets import load_dataset<br><br># Load the base model and tokenizer<br>model_id = &quot;meta-llama/Llama-3.2-1B-Instruct&quot;<br>model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float32, device_map=&quot;auto&quot;) # Must be float32 for MacBooks!<br>tokenizer = AutoTokenizer.from_pretrained(model_id)<br>tokenizer.pad_token = tokenizer.eos_token<br><br># Load the training dataset<br>dataset = load_dataset(&quot;csv&quot;, data_files=&quot;data/sarcasm.csv&quot;, split=&quot;train&quot;)<br><br># Define a function to apply the chat template<br>def apply_chat_template(example):<br>    messages = [<br>        {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: example[&#39;question&#39;]},<br>        {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: example[&#39;answer&#39;]}<br>    ]<br>    prompt = tokenizer.apply_chat_template(<br>        messages, tokenize=False, add_generation_prompt=True<br>    )<br>    return {&quot;prompt&quot;: prompt}<br><br># Apply the chat template function to the dataset<br>new_dataset = dataset.map(apply_chat_template)<br>new_dataset = new_dataset.train_test_split(0.05) # Let&#39;s keep 5% of the data for testing<br><br># Tokenize the data<br>def tokenize_function(example):<br>    tokens = tokenizer(example[&#39;prompt&#39;], padding=&quot;max_length&quot;, truncation=True, max_length=128)<br>    # Set padding token labels to -100 to ignore them in loss calculation<br>    tokens[&#39;labels&#39;] = [<br>        -100 if token == tokenizer.pad_token_id else token for token in tokens[&#39;input_ids&#39;]<br>    ]<br>    return tokens<br><br># Apply tokenize_function to each row<br>tokenized_dataset = new_dataset.map(tokenize_function)<br>tokenized_dataset = tokenized_dataset.remove_columns([&#39;question&#39;, &#39;answer&#39;, &#39;prompt&#39;])<br><br># Define training arguments<br>model.train()<br>training_args = TrainingArguments(<br>    output_dir=&quot;./results&quot;,<br>    eval_strategy=&quot;steps&quot;, # To evaluate during training<br>    eval_steps=40,<br>    logging_steps=40,<br>    save_steps=150,<br>    per_device_train_batch_size=2, # Adjust based on your hardware<br>    per_device_eval_batch_size=2,<br>    num_train_epochs=2, # How many times to loop through the dataset<br>    fp16=False, # Must be False for MacBooks<br>    report_to=&quot;none&quot;, # Here we can use something like tensorboard to see the training metrics<br>    log_level=&quot;info&quot;,<br>    learning_rate=1e-5, # Would avoid larger values here<br>    max_grad_norm=2 # Clipping the gradients is always a good idea<br>)<br><br># Initialize Trainer<br>trainer = Trainer(<br>    model=model,<br>    args=training_args,<br>    train_dataset=tokenized_dataset[&quot;train&quot;],<br>    eval_dataset=tokenized_dataset[&quot;test&quot;],<br>    tokenizer=tokenizer)<br><br># Train the model<br>trainer.train()<br><br># Save the model and tokenizer<br>trainer.save_model(&quot;./fine-tuned-model&quot;)<br>tokenizer.save_pretrained(&quot;./fine-tuned-model&quot;)</pre><p>The arguments inside TrainingArguments can be tuned, but unless you’re careful and you know what you’re doing, you are likely to cause a representation collapse and destroy the model’s predictive capabilities.</p><p>The output will be saved inside a directory named fine-tuned-model. Notice how we had to decouple the model and the tokenizer part here. The tokenizer is responsible for translating human language to an input accepted by the model, as well as decoding the model output to human readable language. The tokenizer also has the apply_chat_template method, that applies the predefined template for the Instruct models that was described earlier.</p><h4>Testing your model</h4><p>Before you do anything with the model you need to make sure it works. You can go back to the traditional text generation script, but now, instead of invoking the Llama model from Hugging Face, you will point the pipeline function to your newly created model in fine-tuned-model. Here is how you can test it:</p><pre>from transformers import pipeline<br><br>model_id = &quot;fine-tuned-model&quot;<br>pipe = pipeline(<br>    &quot;text-generation&quot;,<br>    model=model_id,<br>    device_map=&quot;auto&quot;,<br>)<br>messages = [<br>    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Should I move to Scandinavia?&quot;},<br>]<br>outputs = pipe(<br>    messages,<br>    max_new_tokens=128<br>)<br>print(outputs[0][&quot;generated_text&quot;][-1])</pre><p>From the code above, passing the prompt: “Should I move to Scandinavia?” to our model we get the following output:</p><p>“Oh yes, because nothing says ‘good life’ like freezing your butt off. And the cost of living? A whole other story. You might even need a warm coat. Worth a shot? Probably not. Scandinavia is all about embracing the cold. You’ll love it. You’ll hate it. Either way, you’ll be fine. Or not. Who knows. It’s all part of the adventure. Right?”</p><p>Banger. And you may test it with multiple prompts to ensure everything was done right. Keep in mind, this was done using the 1B model, which is the easiest to train in terms of hardware demands.</p><h4>Sharing</h4><p>After you’re done, you may publish your model to Hugging Face, so other people can take it for a spin, if it works for their case. Sharing can be extra helpful because training models is a lot heavier than simply running them, so people might be able to run your model even on machines with lower GPU vRAM. One way to share the models is via push_to_hub if you are logged in with <a href="https://huggingface.co/docs/huggingface_hub/en/guides/cli">huggingface-cli</a> and if you have your model and tokenizer loaded in Python:</p><pre>model.push_to_hub(&quot;&lt;your_huggingface_name&gt;/&lt;your_model_name&gt;&quot;,<br>                  private=True)<br>tokenizer.push_to_hub(&quot;&lt;your_huggingface_name&gt;/&lt;your_model_name&gt;&quot;,<br>                     private=True)</pre><p>You may upload models as private or public and you can switch between these two states at any point you want.</p><p>Finally, if you simply want to run the sarcasm model I trained for this tutorial, simply provide the appropriate <a href="https://huggingface.co/AlexandrosChariton/SarcasMLL-1B">model_id</a> to pipeline for “text-generation”, as seen in the following code block.</p><pre>from transformers import pipeline<br><br>model_id = &quot;AlexandrosChariton/SarcasMLL-1B&quot;<br>pipe = pipeline(<br>    &quot;text-generation&quot;,<br>    model=model_id,<br>    device_map=&quot;auto&quot;,<br>)<br>messages = [<br>    {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: &quot;You are a helpful assistant.&quot;},<br>    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Should I move to Scandinavia?&quot;},<br>]<br>outputs = pipe(<br>    messages,<br>    max_new_tokens=128<br>)<br>print(outputs[0][&quot;generated_text&quot;][-1])</pre><p>If you want to work on the model or improve it, you are able to start from this point and fine tune it on your own question-answer pairs with comedic value, just remember, quality over quantity and careful parameterization in TrainingArguments is what matters here.</p><blockquote>- How many r’s in the word strawyberry?<br>- Oh, just one. It’s a real head-scratcher. Don’t try to count them yourself. It’s like trying to solve a math problem. It’s one. Got it? One. Not two. Not three. One.<br>SarcasMLL-1B, 20/10/2024</blockquote><h4>Conclusion</h4><p>The information and code provided in this blog post can help someone adapt an open-source language generation model to their own datasets. If you have a problem that requires a model to be an expert in a specific field, rather than a general-purpose model that’s good at everything but not at an expert level, fine-tuning might be worth considering. The possibilities are endless when working with large language models, and it’s exciting to think about what advancements will come in the next few years.</p><p>In future posts, I’ll dive deeper into other techniques one can use to avoid such limitations with variants of <a href="https://arxiv.org/abs/2106.09685">LoRA</a>, allowing for fine tuning of really large AI models that would otherwise be impossible in our computers.</p><p>Also, if you want to learn more about fine tuning applications and Meta’s approach to open source LLMs, start from their relevant <a href="https://ai.meta.com/blog/adapting-large-language-models-llms/">blog</a> and read along! I will try to keep these posts coming with more applications of AI, tackling challenges that seemed impossible a few years ago. If you have any issues / questions / comments feel free to reach out in the comments or via <a href="https://www.linkedin.com/in/alexandros-chariton-06a6aa1a4/">LinkedIn</a>, <a href="https://github.com/AlexandrosChrtn">Github</a> or <a href="https://x.com/alexchrtn">Twitter</a>!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e5f522f397d7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How we remove the background from product images at BestPrice.gr]]></title>
            <link>https://medium.com/bestprice-dev/how-we-remove-the-background-from-product-images-at-bestprice-gr-c44b32ebb5bf?source=rss-9d6b89ec3064------2</link>
            <guid isPermaLink="false">https://medium.com/p/c44b32ebb5bf</guid>
            <category><![CDATA[image]]></category>
            <category><![CDATA[e-commerce-software]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[neural-networks]]></category>
            <dc:creator><![CDATA[Alexandros Chariton]]></dc:creator>
            <pubDate>Mon, 11 Jul 2022 12:47:11 GMT</pubDate>
            <atom:updated>2022-07-12T09:08:19.216Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YXSqtKWhKaw8op4zI0012g.png" /></figure><p>Product images commonly convey information about the product to the consumers without much processing effort from the consumer’s end. Our users do not have the option to physically interact with the product so there is the obvious need for visual descriptors, whose primary job is to provide accurate information about the product. Appealing images take it a step further, they create positive first impressions for products to attract the interest of potential customers.</p><p>People visit BestPrice.gr not only for buying a product they need and have in mind, but also browsing. If someone is looking to buy a laptop for example, they may scroll through hundreds of different laptops, stopping for nothing more than mere seconds before moving on. This is common for e-commerce users and one thing that stands out when this behavior is exhibited is that it can be a very lousy experience sometimes if certain criteria are not met.</p><p>There are of course those that concern each product image in isolation, for example the quality of the image or its brightness, but there is also the criterion of consistency. It can be difficult for my brain to process so many images at this rate when there’s little to no connection between one another, except for maybe the fact that all of them describe products of the same category. This is extremely apparent predominantly in fashion categories, where the visual signal is much more valuable than say laptops. I find myself spending more time scrolling through our jackets when one image shows a specific jacket and the product next to it has an image of a model wearing the jacket etc. To this end, a common practice in e-commerce is to convert everything to a similar format, ideally with a white background such that the product in question is highlighted. Here is where the background removal tool comes in, it accepts an image of arbitrary size as input and outputs the image with a white background.</p><h3><strong>Removal</strong></h3><p>Computer vision is a field of Computer Science revolving around processing images and videos. Before moving on, it’s important to describe what an image representation is, in this field. Each image is associated with two numbers, its height and width, measured in pixels. An image of 100 height and 100 width has a total of 100 * 100 = 10,000 pixels and assuming that we are interested in color images, we need to associate each pixel with a color. Usually, we represent colors using 3 numbers, their so-called RGB values (from Red Green Blue) where these numbers denote the color intensity on the respective filter and lie between 0 and 255. Using this we represent an image as an element of a real number vector space of dimensionality (Height x Width x 3), so the colored image of 100 pixels height and 100 pixels width corresponds to a total of 3 * 10,000 = 30,000 numbers. Expressing images as vectors allows for mathematical operations such as matrix multiplications.</p><p>The background of an image is very tricky to define. There is no universal definition for this concept and even humans may disagree on what constitutes the image background for a specific image. Saliency masks are grayscale images that work as probability heatmaps, indicating the most likely pixels within an image that the eyes of an observer might fall, when encountering the visual stimulus. These images can be produced by showing images to groups of people, while being tracked with eye-tracking software. The highlighted parts of these images can be thought as the foreground of the images in question, as we would expect the property of attracting one’s first glance to be a property of the image foreground. This correlation implies that producing a heatmap like that for an image might work for our task at hand. Datasets with pairs of images and annotated masks already exist, for example the <a href="http://saliencydetection.net/duts/#org3aad434">DUTS Image Dataset</a> and it looks like there is ground for supervised learning to take over.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tbc_EqRwlEdjBgzxtO628w.jpeg" /><figcaption>Example of an image and its saliency mask. Notice the uncertainty for the stool, which is depicted as grayscale.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/420/1*8kVSe-bRqreDWj0D1DecZA.jpeg" /><figcaption>Example of a product image with ambiguous background.</figcaption></figure><p>Supervised learning works by providing pairs of input — desired output (ground truth) to a machine learning model, so that it can learn by correcting itself until it predicts the output for a given input sufficiently well. This is a very simplistic explanation of how this works but the point is that there is a mechanism connecting the input to the desired output that cannot be identified and hardcoded with if statements for example, so providing a large number of inputs and outputs might lead to the model figuring out this connection (if possible, because sometimes, it just isn’t). This procedure is called training, because an ML model is trained to adapt to our dataset and is typically expensive in terms of computational resources. It requires a large number of floating point operations (FLOPS), therefore GPUs or TPUs are often employed, to speed up the process by parallelizing a large percentage of these operations.</p><p>A neural network (NN) is undoubtedly the best ML algorithm choice here, as these algorithms have provided astonishing results in computer vision, far better than the rest. A characteristic of this family of algorithms is that there is a very large number of user defined hyperparameters, such as the network architecture, that need to be set and greatly affect the performance. There is no universal solution when it comes to setting these parameters, it’s greatly problem dependent and requires trial and error. Research on this field greatly reduces the search space for us, as there are many published works on image segmentation with saliency masks. We base our work on “<a href="https://arxiv.org/pdf/2005.09007.pdf">U²-Net: Going Deeper with Nested U-Structure for Salient Object Detection</a>”, where they propose a network with a “U structure” for this very task and also provide a pretrained network we can use on the spot. This network was trained on the DUTS dataset and examples of input-output pairs for this network for product images can be seen below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*17LELHD7_AuGigyphHeW8Q.jpeg" /><figcaption>Left: Original image, the network’s input. Right: Saliency mask, the ground truth of the output</figcaption></figure><p>Producing saliency masks with this pretrained network requires two things:</p><p>1) The network details</p><p>These are tunable parameters that have already been tuned by the researchers, following the training procedure of feeding examples to the algorithm, described above. We also need the architecture of the model, something like a network graph that works as the skeleton of the machine learning model, connecting those aforementioned weights.</p><p>2) The input details</p><p>This is also available and is something ML algorithms require. Such details involve input transformations like image resizing, MinMaxScaling or Standardizing. To inference a new image at hand we convert it accordingly, with these same transformations used by the researchers when they trained the model, otherwise we will not get the intended output. The information above is available on their <a href="https://github.com/xuebinqin/U-2-Net">GitHub page</a>, so we can start building the background removal tool. The code is available in Python, the most popular language for machine learning applications and it uses a library called <a href="https://pytorch.org/">Pytorch</a>, which along with <a href="https://www.tensorflow.org/">Tensorflow</a> is one of the main tools used to develop NNs.</p><p>We can set up a pipeline that accepts an image as input, applies the transformations required and outputs the saliency mask of the image. One important detail here is that the output of neural networks is fixed in size. In fact, the output in this case is always a 320x320 grayscale image (1 channel, instead of 3 in RGB images) which is black for pixels predicted as background pixels, white for pixels predicted as foreground pixels and gray for pixels with more uncertain predictions. This requires handling because we need to come up with a way to deduce the background of an image of arbitrary size using a fixed size heatmap. To do that we could resize the original input image to 320x320 or we could resize the saliency mask to fit the original image. The benefit of the latter is that we will end up with a white background image of the same size, so we will move forward only considering this option. When it comes to resizing images, options such as Nearest Neighbor interpolation or Linear Interpolation are the most organic.</p><p>Usually the input images are larger than 320x320 so we need to expand the filter to match their dimensionality. Nearest neighbor interpolation expands the image by assigning the intensity value of a pixel to its neighboring pixels until it reaches the appropriate size. So, in our case, if there was a pixel located at the center of the image, predicted as a foreground pixel with a probability of 0.9, then there’s a good chance that every pixel around it would also be a foreground pixel with the same probability. Linear interpolation works similarly, except that it expands the image by estimating intensity values using first degree polynomials. In image context, bilinear interpolation is used, meaning that the intensity of the pixels missing is calculated using both directions across the image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/487/1*4v3g5QVSQeKAIx_ZqaVxtw.png" /><figcaption>Interpolation examples. Rounding is used for Bilinear interpolation</figcaption></figure><p>Producing these masks is the most important need in background removal, however we’re not there yet. The actual removal has to take place and so far what we have is the input image and a foreground probability for each pixel.</p><p>The steps from now on are more straightforward, we could for example multiply these probabilities with the pixel intensity to produce the “expected image foreground”, in which we would expect a soft transition from the foreground to the white background. This might not be desirable in most cases, we may want to capitalize on the high contrast border around the product to highlight the product itself. This can be done by converting this probability heatmap into a map of 1s and 0s, where 1 denotes that this pixel belongs to the foreground and 0 otherwise. We can arbitrarily set a threshold, 0.7 for example, only keep pixels with probabilities larger than this threshold and convert everything else to white. A potential benefit of converting these scores to binary decisions is that there’s a good chance this might decrease the effect of noisy predictions. Arguably the most difficult part of the task is estimating accurate scores for each pixel, so if for example a neighborhood of pixels is assigned scores that fluctuate around 0.8, then everything would be considered as foreground irregardless of score. NNs would learn, ideally, that such large fluctuations are rarely good, but still it’s a good idea to alleviate some burden of the more difficult component of the mechanism.</p><p>Of course there are alternatives to binary thresholding (such as <a href="https://en.wikipedia.org/wiki/Otsu%27s_method">Otsu’s method</a>) but a binary threshold is usually good enough for the task. In our app, threshold can be provided as a user hyperparameter, so one might attempt removal with a large threshold and then gradually decrease it, if the removal is too strict. For this part we can use a Python library called <a href="https://opencv.org/">OpenCV</a>, a reliable choice for computer vision tasks or <a href="https://pillow.readthedocs.io/en/stable/">Pillow</a> which is also pretty popular in the field.</p><p>The difficult part of the removal should be considered done. What’s left now is the app part. We need to convert this functionality to an API endpoint, so that our Content team can forward their images and get the white background versions back. We will not dive deep into this part as there are numerous tutorials describing how to build a service like that. In Python, libraries such as <a href="https://flask.palletsprojects.com/en/2.1.x/">Flask</a> or <a href="https://fastapi.tiangolo.com/">FastAPI</a> are excellent choices there. It is also important to mention that there are other ways we can improve the removal quality with basic image processing. For example, we found that for product images it is usually a good idea to greatly sharpen the image before predicting its saliency mask, because the border around the object becomes clearer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X50zXKyRvcrHMZxJdCtzkQ.png" /><figcaption>Images of our catalog. The image on the right sort of breaks the pattern.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/202/1*F6gcu07QG4ie8LA0tICsQw.jpeg" /><figcaption>Image on the right after removal. Kudos to our wonderful Content team for providing the images.</figcaption></figure><h3><strong>Conclusions</strong></h3><p>The solution above is a solution that works moderately well, for a difficult Computer Vision task. Considering the expensive alternative of manually removing the background or not doing anything at all, there is value in having this tool around in our organization. It’s worth mentioning that the most popular service for this task is offered by <a href="http://remove.bg/">remove.bg</a>, where they also consider the benefits of its usage in e-commerce context. The Open Source Community provides the opportunity to build things that might have been considered complicated years ago, without putting too much effort into it. Also, ML research is rapidly growing, so there’s a good chance we will come up with a better removal tool soon enough. It’s always a good idea to stay up to date with the latest advances and take advantage of the great work being done in the field.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c44b32ebb5bf" width="1" height="1" alt=""><hr><p><a href="https://medium.com/bestprice-dev/how-we-remove-the-background-from-product-images-at-bestprice-gr-c44b32ebb5bf">How we remove the background from product images at BestPrice.gr</a> was originally published in <a href="https://medium.com/bestprice-dev">BestPrice Dev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>