<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Akshada Kashyap on Medium]]></title>
        <description><![CDATA[Stories by Akshada Kashyap on Medium]]></description>
        <link>https://medium.com/@kashyap.akshada?source=rss-499aa61ea318------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*1_U1dI_uIG2-PPA7</url>
            <title>Stories by Akshada Kashyap on Medium</title>
            <link>https://medium.com/@kashyap.akshada?source=rss-499aa61ea318------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 10:26:25 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@kashyap.akshada/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Do You Speak “Data”?]]></title>
            <link>https://pub.towardsai.net/do-you-speak-data-5eb6ff843ea8?source=rss-499aa61ea318------2</link>
            <guid isPermaLink="false">https://medium.com/p/5eb6ff843ea8</guid>
            <category><![CDATA[classification]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[exploratory-data-analysis]]></category>
            <dc:creator><![CDATA[Akshada Kashyap]]></dc:creator>
            <pubDate>Sun, 15 Nov 2026 22:01:01 GMT</pubDate>
            <atom:updated>2026-11-15T22:01:01.848Z</atom:updated>
            <content:encoded><![CDATA[<p>The base of a good machine learning model is always good data. The cleaner and better understood your data is, the stronger your predictions will be. It is much easier to predict the behavior of a habitual person than someone who lives with zero routine — pattern recognition relies on the assumption that a system behaves consistently, free from hidden noise or unexamined quirks.</p><p>I have taken multiple courses during my bachelor’s degree focused on data science and machine learning, yet none of them truly taught me how to <em>speak</em> with data. Today, I’ll walk you through a diabetes prediction dataset to show you how to interrogate your data, uncover hidden anomalies, and translate those insights into a baseline machine learning model.</p><p>Let’s dive into it.</p><h3>The Recipe</h3><ol><li>The dataset (obviously): <a href="https://www.kaggle.com/datasets/iammustafatz/diabetes-prediction-dataset">https://www.kaggle.com/datasets/iammustafatz/diabetes-prediction-dataset</a></li><li>Any Python environment (feel free to use Google Colab or Kaggle Notebooks)</li><li>And a little bit of curiosity 🥸</li></ol><h3>The Code Walkthrough</h3><p>We begin with importing the dataset through Kaggle:</p><pre>import kagglehub<br><br># Download latest version   <br>path = kagglehub.dataset_download(&quot;iammustafatz/diabetes-prediction-dataset&quot;)<br><br>print(&quot;Path to dataset files:&quot;, path)</pre><p>You can find your dataset using:</p><pre># List files in the downloaded directory<br>print(&quot;Files in dataset directory:&quot;, os.listdir(path))</pre><pre>data_file_path = os.path.join(path, &#39;diabetes_prediction_dataset.csv&#39;)<br>data_file_path</pre><h4>Inspect the dataset</h4><p>Like with any project, you must be curious about the data. It helps to ask a few questions like:</p><ol><li>What are we working with?</li><li>How much are we working with?</li><li>Are there any data points that look out of place?</li><li>How would I translate (transform) this data into something that is easier to work with?</li></ol><p>We take our first look at the dataset using .head()</p><pre>df = pd.read_csv(data_file_path)<br>print(df.head())</pre><p>This gives us an idea of what the dataset &amp; its values look like. Before we jump to noting down any observations, we execute a few more lines of code:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1011/1*I88Q7LRdMQ2muTVvIgQlJw.png" /><figcaption>Printing the last 5 rows of the dataset</figcaption></figure><p>Using Pandas’ .describe() is an easy way to understand the numbers in your dataset. It shows you the summary statistics of your data, including the min value per column, max value per column, the number of values per column and even the mean value per column. These values can help you understand if there’s any data that’s out of place.</p><pre>df.describe()</pre><p>We finish off with printing the datatypes of each column and also the shape of the dataset, which tells us how many rows and columns we are working with.</p><pre>for item in df.columns.to_list():<br>  print(item, &quot;:&quot;, df[item].dtype)</pre><p>Now, we can delve into the output of each piece of code, and understand what the dataset is comprised of.</p><p>You would have noticed from the output of df.head() that the values of the column smoking_history seem very inconsistent: “never”, “No Info”, “current”, etc. Can you rule out the rows with “No Info” under smoking history as “missing information”? This is worth exploring.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OsW9p42s6cww45C0F5Ac2Q.png" /><figcaption>Multiple values with multiple meanings in this column</figcaption></figure><p>Examine the output of df.describe(), especially the min, max, mean, and percentile values. Remember that min represents the 0th percentile and max is the 100th percentile. Try to compare these values together and note down any anomalies. There’s something hiding in plain sight in the picture below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/954/1*C2Bwwo3yK7gcus34qZhRhg.png" /><figcaption>Summary statistics of the dataset</figcaption></figure><p>If you also noticed how the max value of the bmi column is a little high when compared against the 75% percentile, you have a keen eye! The huge jump from 29.580000 to 95.690000 is crazy big, and realistically speaking, even amongst 100,000 people (which is the number of rows of the dataset), there aren’t going to be very many people of this size. For context, a 6 ft. tall individual would have to weigh 318 kgs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EzJu2rRmYbTMM9EScoO41w.png" /><figcaption>Outlier in the “BMI” column</figcaption></figure><p>Another good observation would be to note that out of all the columns, only gender and smoking_history have non-numeric values. An even better observation is that while the columns hypertension and heart_disease are numeric, they do not represent continuous measurements. They are categorical classes disguised as numbers, where each integer corresponds to a distinct state. For the value 0 in hypertension or heart_disease, the meaning translates to &quot;no&quot; / &quot;negative&quot; (the person does not have hypertension or heart disease), while 1 translates to &quot;yes&quot; / &quot;positive&quot; (the person has been diagnosed). This concept of representing categories with numbers is called <strong>label encoding</strong> (or <strong>binary encoding</strong> for two classes). In contrast, columns like gender and smoking_history are still raw text and will need to be encoded to maintain consistency.</p><p>A lot of the times, we jump to conclusions and believe that our assumptions are right. For example, you might have assumed that there are 2 genders within the dataset through the df.head() code or the df.tail() code: &quot;Male&quot; and &quot;Female&quot;. Some of you might have assumed that the dataset could be gender-inclusive and have many more identities recorded. Assumptions might be great for physics, but with data, we must always ensure we know what we are working with.</p><p>To answer the question, “How many gender values have been recorded in the dataset?”, or generally speaking, “How many different classes have been recorded per categorical field?”, we use the following code snippet:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/835/1*zo0WPvtBq_dHxzv7PWbrrA.png" /><figcaption>No. of classes per categorical column</figcaption></figure><p>Judging from the output, hypertension, heart_disease and diabetes are binary encoded columns with 0/1 as values. gender has 3 values and smoking_history has 6! We can print them using the following code to understand what values have been recorded and what they mean.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/928/1*0uk4kYcXAcNI4AIuYmqwwQ.png" /><figcaption>Printing all unique values per categorical column</figcaption></figure><p>Hmm. Are you also confused about the multiple values under “smoking_history”? What the hell does “ever” stand for? Maybe we should assume it to be representation of “has smoked at least once in their lives”? But how would we distinguish it from the “not current” value? What if “ever” is a misspelling for “never”? This seems to be the more likely option.</p><p>To confirm this, we must hope to not see &gt;0.1% of the dataset having such an erroneous spelling. We can print the number of rows where “smoking_history” = “ever” using the following code snippet:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/882/1*MxK-A-GvGqCbXrUBodyIEA.png" /><figcaption>Printing all the rows where “smoking_history” = “ever”</figcaption></figure><p>4000+ rows is indicative that maybe such an error is unlikely. What’s great about living in this day and age is having tools to consult about questions like these; which is when I decided to ask Gemini on Colab, who was very helpful with the answer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/540/1*zqFdpFqgAy6Me7nhbl_g8A.png" /><figcaption>Gemini’s answer on the difference between “ever” and “not current” smokers</figcaption></figure><p>With a quick Google search to verify the same, it is easy to understand how each class of smoker in the given dataset mean different things:</p><ol><li>“ever” smokers: These refer to people who have smoked at least 100+ cigarettes at some points in their lives, with no data indicative of whether they currently smoke or have stopped. You can look at the definition of this here: <a href="https://en.wiktionary.org/wiki/ever-smoker">https://en.wiktionary.org/wiki/ever-smoker</a></li><li>“current” smokers: These refer to people who are actively smoking at the time of conducting the research survey.</li><li>“not current” smokers: These refer to people who have stopped smoking, but were formerly engaged with the habit.</li></ol><p>Another issue with the field “smoking_history” is the value “No Info” — this usually classifies as “missing information” for any dataset. Before we decide to follow through with the usual methods to handle missing data, we must check if it would affect the overall dataset. This means that we must ensure that the number of rows with “missing information” for “smoking_history” must be very small, which would allow us to remove these rows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/868/1*1VTM4lxSzKx-YubkzAv_9A.png" /><figcaption>Printing the rows where “smoking_history” = “No Info”</figcaption></figure><p>Over 35,000 rows in the dataset have no information recorded for the smoking history of patients! Removing 35% of the dataset because we believe this lack of information to be insufficient information would be the wrong call. We shall continue without making any changes to the dataset as of now. There is still much to explore.</p><p>Univariate analysis of data refers to using methods to analyze each feature within a dataset independently. This method does not involve or take into account any interdependent relationships between the features of a dataset. While there are several methods to perform univariate analysis, we will use simple histogram plots to analyze the numerical fields in our dataset: age, bmi, hba1c, and blood_glucose_level.</p><pre>import matplotlib.pyplot as plt<br>import seaborn as sns<br><br># Set the aesthetic style of the plots<br>sns.set_style(&quot;whitegrid&quot;)<br><br># Get the list of column names, excluding the specified ones<br>excluded_cols = [&#39;gender&#39;, &#39;hypertension&#39;, &#39;heart_disease&#39;, &#39;diabetes&#39;, &#39;smoking_history&#39;]<br>columns = [col for col in df.columns.tolist() if col not in excluded_cols]<br><br># Determine the number of rows and columns for the subplot grid<br># A common practice is to have 3 columns per row for better readability<br>n_cols = 3<br>n_rows = (len(columns) + n_cols - 1) // n_cols<br><br># Create subplots for each feature<br>plt.figure(figsize=(n_cols * 5, n_rows * 4))<br><br>for i, col in enumerate(columns):<br>    plt.subplot(n_rows, n_cols, i + 1)<br>    sns.histplot(data=df, x=col, kde=True) # kde=True adds a Kernel Density Estimate<br>    plt.title(f&#39;Distribution of {col}&#39;)<br>    plt.xlabel(col)<br>    plt.ylabel(&#39;Count&#39;)<br><br>plt.tight_layout()<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T5AYgnM0RYVgXc4SvmdIgw.png" /><figcaption>Histograms for “age”, “bmi”, “hba1c”, and “blood_glucose_level”</figcaption></figure><p>Some important observations to make from the above histograms:</p><ol><li>Notice how in the histogram for age, the distribution seems to be fairly evenly spread out until the sudden spike in number when the age reaches 80? Is this indicative of how the largest demographic in the dataset is people around the age of 80? Actually, the more likely cause for this is a concept called “top-coding” or “age-capping”, where people of the age 80 or above are all grouped into the same category as 80 as per a HIPAA privacy compliance method.</li><li>In the histogram for bmi, we notice how there is a peak near 25–30, which is in-line with the values we noted through df.describe(): the mean, 25th, 50thand 75thpercentile all lie in or around this range. But the graph still stretches out (skews) to the far right, which is probably due to the max value of 95+ BMI. This graph is also a textbook example of a <strong>right-skewed graph</strong> (a graph with its single peak leaning towards the left), which means that there are statistical outliers in the dataset causing this. Another important observation is the needle spike in the middle, which means that there is one point that has been heavily repeated across the dataset, which is something to look into further.</li><li>In the histograms for both hba1c annd blood_glucose_level, there are multiple peaks with no clear pattern immediately visible to us. These are called <strong>multimodal graphs</strong>, which are graphs with multiple peaks instead of a single peak.</li></ol><p>Investigating the “needle spike” in the bmi histogram further involves making an assumption and attempting to prove it right. Is it possible that the dataset curator maybe filled some data for this column? This would mean that the number of rows with a specific number (possibly the mean, because of its position in the graph) as the value for BMI would be abnormally high. We can confirm it using:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/886/1*UGiy4kxReKlN-HBHQm7FCQ.png" /><figcaption>Printing the rows where “bmi” = mean of “bmi”</figcaption></figure><p>Approximately 25% of the dataset has the same value for BMI, a metric that is directly dependent on a person’s height &amp; weight. It is safe to assume that the data for these 25k+ rows was filled using a method called <strong>mean imputation</strong>, which is used to fill missing data with the column’s overall average.</p><p>Having analysed each feature independently, we now move into analysing the interdependency between 2 features (at a time) in the dataset. This involves a bunch of <strong>bivariate analysis </strong>techniques. We can figure out if and how feature A affects feature B in the dataset. For example, let’s try to use these techniques to answer the following few questions:</p><ol><li>Is a person’s smoking history directly correlated to diabetes?</li><li>How much more likely is someone with heart_disease or hypertension to also have diabetes?</li><li>Are people with higher BMIs more likely to get diabetes?</li><li>Does the older population have a higher prevalence of diabetic patients?</li></ol><p>Notice how each question is attempting to find the correlation between 2 factors only. Of course, one can analyze the interdependency of multiple (2+) factors in a dataset, but I will not be including any multivariate analysis techniques in this blog post here.</p><p>Let’s answer the questions from above now. To understand the correlation between smoking history and diabetes, we use the following code snippet:</p><pre># Calculate the number of people with and without diabetes for each smoking history category<br>smoking_diabetes_counts = df.groupby([&#39;smoking_history&#39;, &#39;diabetes&#39;]).size().unstack(fill_value=0)<br><br># Calculate the total number of people for each smoking history category<br>smoking_diabetes_counts[&#39;Total&#39;] = smoking_diabetes_counts[0] + smoking_diabetes_counts[1]<br><br># Calculate the percentage of people with diabetes for each smoking history category<br>smoking_diabetes_counts[&#39;Diabetes_Prevalence&#39;] = (smoking_diabetes_counts[1] / smoking_diabetes_counts[&#39;Total&#39;]) * 100<br><br># Display the results<br>display(smoking_diabetes_counts.sort_values(by=&#39;Diabetes_Prevalence&#39;, ascending=False))</pre><pre>import matplotlib.pyplot as plt<br>import seaborn as sns<br><br># Create a bar plot to visualize diabetes prevalence by smoking history<br>plt.figure(figsize=(10, 6))<br>sns.barplot(x=smoking_diabetes_counts.index, y=smoking_diabetes_counts[&#39;Diabetes_Prevalence&#39;], palette=&#39;viridis&#39;)<br>plt.title(&#39;Diabetes Prevalence by Smoking History&#39;)<br>plt.xlabel(&#39;Smoking History&#39;)<br>plt.ylabel(&#39;Diabetes Prevalence (%)&#39;)<br>plt.xticks(rotation=45, ha=&#39;right&#39;)<br>plt.tight_layout()<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/989/1*YZw6N9wi-r3Wy2BE8LMAaA.png" /><figcaption>Bar graph of smoking history vs. diabetes prevalence</figcaption></figure><p>From the bar graph above, we notice that former smokers have the highest prevalence of diabetes (~17%). This is higher than the prevalence of diabetes amongst current smokers. This could be because:</p><ol><li>People tend to quit smoking due to age (after reaching a specific age)</li><li>People tend to quit smoking due to medical issues like heart disease or hypertension</li></ol><p>To confirm this, it would require doing multivariate analysis between smoking_history, age, heart_disease, and hypertension. You would compare the first subgroup of people that are younger (age &lt; x) and have no comorbidities against the second subgroup of people that are older (age ≥ x) and have either hypertension or heart_disease or both, based on their diabetes prevalence numbers. I highly recommend you try this out!</p><p>To understand how much more likely a person with hypertension and/or heart disease is to have diabetes, we use the following code snippet:</p><pre>import matplotlib.pyplot as plt<br>import seaborn as sns<br><br>categorical_cols = [&#39;hypertension&#39;, &#39;heart_disease&#39;]<br><br>plt.figure(figsize=(15, 5))<br><br>for i, col in enumerate(categorical_cols):<br>    plt.subplot(1, len(categorical_cols), i + 1)<br>    # Calculate diabetes prevalence for each category<br>    prevalence = df.groupby(col)[&#39;diabetes&#39;].mean() * 100<br>    sns.barplot(x=prevalence.index, y=prevalence.values, palette=&#39;coolwarm&#39;)<br>    plt.title(f&#39;Diabetes Prevalence by {col}&#39;)<br>    plt.xlabel(col)<br>    plt.ylabel(&#39;Diabetes Prevalence (%)&#39;)<br>    plt.xticks(rotation=45, ha=&#39;right&#39;)<br><br>plt.tight_layout()<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NaEEtbnRXkJ08RzhX7ULOg.png" /><figcaption>Plots showing diabetes prevalence in people with &amp; without hypertension / heart disease</figcaption></figure><p>From the visualization above, it is easy to understand that diabetes is much more prevalent in populations that have heart disease and/or hypertension. While there seems to be a higher correlation of diabetes with hypertension/heart disease, this is not conclusive of whether one medical issue causes the other, i.e., we cannot confirm that diabetes causes hypertension/heart disease (or the other way around). <strong>Correlation is not causation.</strong></p><p>This brings us to answering our next question: Are people with higher BMIs more likely to get diabetes? While we cannot answer whether a higher BMI can cause diabetes in a person, we can definitely visualize the prevalence of diabetes in people with higher BMIs.</p><p>Alongside BMI, we can analyze the correlation between other numerical categories vs. diabetes as well:</p><pre>numerical_cols = [&#39;age&#39;, &#39;bmi&#39;, &#39;HbA1c_level&#39;, &#39;blood_glucose_level&#39;]<br><br>plt.figure(figsize=(18, 10))<br><br>for i, col in enumerate(numerical_cols):<br>    plt.subplot(2, 2, i + 1) # 2 rows, 2 columns for subplots<br>    sns.boxplot(data=df, x=&#39;diabetes&#39;, y=col, palette=&#39;viridis&#39;)<br>    plt.title(f&#39;{col} Distribution by Diabetes Status&#39;)<br>    plt.xlabel(&#39;Diabetes (0: No, 1: Yes)&#39;)<br>    plt.ylabel(col)<br><br>plt.tight_layout()<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rLRboBgKZBb2tsy8rJGLHQ.png" /><figcaption>Box plots of numerical categories vs. diabetes status</figcaption></figure><p>The box plots of all numerical categories vs. diabetes is shown above. Our general knowledge of high HbA1c_level and blood_glucose_level levels being associated with diabetes is also aligned with in the dataset. In the box plots for age and bmi, though, there are circular points outside the upper and lower bound limits, which represent outliers in their distributions.</p><p>We can use simple IQR methods (interquartile methods) to remove the outliers in both the age and bmi categories and re-plot these analyses.</p><pre>Q1 = df[&#39;bmi&#39;].quantile(0.25)<br>Q3 = df[&#39;bmi&#39;].quantile(0.75)<br><br>IQR = Q3 - Q1<br><br>lower_bound = Q1 - 3 * IQR<br>upper_bound = Q3 + 3 * IQR<br><br>print(f&quot;lower bound: {lower_bound}&quot;)<br>print(f&quot;upper bound: {upper_bound}&quot;)<br><br>df_no_outliers = df[(df[&#39;bmi&#39;] &gt;= lower_bound) &amp; (df[&#39;bmi&#39;] &lt;= upper_bound)]<br><br>print(f&quot;Original DataFrame shape: {df.shape}&quot;)<br>print(f&quot;DataFrame shape after removing BMI outliers: {df_no_outliers.shape}&quot;)<br><br>df = df_no_outliers.copy()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/678/1*zjRn4nBvpZTdQ-hXop4BTQ.png" /><figcaption>Outlier removal for “bmi” column in dataset</figcaption></figure><p>The code snippet above removes the outliers for the bmi column. You can extend it to remove outliers for the age column as well. The following code snippet is to visualize the histogram of bmi (univariate analysis) after outlier removal.</p><pre>plt.figure(figsize=(8, 5))<br>sns.histplot(data=df, x=&#39;bmi&#39;, kde=True)<br>plt.title(&#39;Distribution of BMI After Outlier Removal&#39;)<br>plt.xlabel(&#39;BMI&#39;)<br>plt.ylabel(&#39;Count&#39;)<br>plt.tight_layout()<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/790/1*9uyx55tyGNHA6wAQzz0spA.png" /><figcaption>Distribution of BMI after outlier removal</figcaption></figure><p>Compare the image above to the distribution of bmibefore outlier removal — you would notice how the peak is no longer leaning towards the left and is in the middle! We have effectively done away with the right-skew in its distribution.</p><p>We can re-examine the box plots for all numerical categories vs. diabetes prevalence through the following piece of code to understand how the plots changed after outlier removal:</p><pre>numerical_cols = [&#39;age&#39;, &#39;bmi&#39;, &#39;HbA1c_level&#39;, &#39;blood_glucose_level&#39;]<br><br>plt.figure(figsize=(18, 10))<br><br>for i, col in enumerate(numerical_cols):<br>    plt.subplot(2, 2, i + 1) # 2 rows, 2 columns for subplots<br>    sns.boxplot(data=df, x=&#39;diabetes&#39;, y=col, hue=&#39;diabetes&#39;, palette=&#39;viridis&#39;, legend=False)<br>    plt.title(f&#39;{col} Distribution by Diabetes Status&#39;)<br>    plt.xlabel(&#39;Diabetes (0: No, 1: Yes)&#39;)<br>    plt.ylabel(col)<br><br>plt.tight_layout()<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OR6GJ1X4u_Os1fFRceSzHA.png" /><figcaption>Box Plots of numerical categories vs. diabetes after outlier removal</figcaption></figure><p>The box plot of bmi vs. diabetes has improved by so much! The correlation between diabetes and bmi is also easier to see in this graph.</p><p>The last question asks: does the older population have a higher prevalence of diabetes? This time, we can visualize the relationship between age and diabetes through a line graph:</p><pre>import matplotlib.pyplot as plt<br>import seaborn as sns<br><br># Calculate the probability of diabetes for each age<br>age_diabetes_probability = df.groupby(&#39;age&#39;)[&#39;diabetes&#39;].mean().reset_index()<br>age_diabetes_probability[&#39;diabetes_probability&#39;] = age_diabetes_probability[&#39;diabetes&#39;] * 100<br><br>plt.figure(figsize=(12, 6))<br>sns.lineplot(data=age_diabetes_probability, x=&#39;age&#39;, y=&#39;diabetes_probability&#39;)<br>plt.title(&#39;Probability of Diabetes by Age&#39;)<br>plt.xlabel(&#39;Age&#39;)<br>plt.ylabel(&#39;Probability of Diabetes (%)&#39;)<br>plt.grid(True)<br>plt.tight_layout()<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m6sTU4U_oHrBnDfIxbCbpQ.png" /><figcaption>Line graph showing probability of diabetes by age</figcaption></figure><p>The line graph moves upward as it moves to the right, indicating that the probability of diabetes goes up with age.</p><p>Having finished answering our questions for the dataset, we move into training.</p><p>Up until now, we inspected the data, made a few assumptions about the data, and confirmed our assumptions about it. Now, we move into understanding what this data can do for us.</p><p>From all the features in the dataset, I would rank them in the following order (from highest significance to the least) based on how much of an impact it has on the prevalence of diabetes:</p><ol><li>blood_glucose</li><li>hba1c</li><li>heart_diseaseand hypertension</li><li>age</li><li>bmi</li><li>smoking_history</li><li>gender</li></ol><p>We could also train models on our dataset consuming 1 feature at a time to test how much predictive power each feature adds. I used <strong>logistic regression</strong>, a very basic classification algorithm to train my model on the dataset in a <strong>hierarchical</strong> (step-by-step) <strong>feature addition</strong> approach.</p><pre>from sklearn.model_selection import train_test_split<br>from sklearn.preprocessing import OneHotEncoder, StandardScaler<br>from sklearn.linear_model import LogisticRegression<br>from sklearn.metrics import accuracy_score, roc_auc_score<br>import pandas as pd<br><br># --- 1. Preprocessing ---<br># Make a copy to avoid modifying the original &#39;df&#39; in place for this step<br>df_processed = df.copy()<br><br># One-hot encode &#39;gender&#39; and &#39;smoking_history&#39;<br>categorical_features = [&#39;gender&#39;, &#39;smoking_history&#39;]<br>df_processed = pd.get_dummies(df_processed, columns=categorical_features, drop_first=True)<br><br># Define target variable<br>y = df_processed[&#39;diabetes&#39;]<br>X = df_processed.drop(&#39;diabetes&#39;, axis=1)<br><br># --- 2. Train-Validation-Test Split ---<br># Split into training (70%) and temp (30%)<br>X_train, X_temp, y_train, y_temp = train_test_split(X, y, test_size=0.3, random_state=42, stratify=y)<br><br># Split temp into validation (15%) and test (15%)<br>X_val, X_test, y_val, y_test = train_test_split(X_temp, y_temp, test_size=0.5, random_state=42, stratify=y_temp)<br><br>print(f&quot;Train set shape: {X_train.shape}&quot;)<br>print(f&quot;Validation set shape: {X_val.shape}&quot;)<br>print(f&quot;Test set shape: {X_test.shape}&quot;)<br><br># --- 3. Define feature subsets for hierarchical approach ---<br>feature_subsets = {<br>    &quot;glucose only&quot;: [&#39;blood_glucose_level&#39;],<br>    &quot;glucose + hba1c&quot;: [&#39;blood_glucose_level&#39;, &#39;HbA1c_level&#39;],<br>    &quot;glucose + hba1c + heart_disease + hypertension&quot;: [&#39;blood_glucose_level&#39;, &#39;HbA1c_level&#39;, &#39;heart_disease&#39;, &#39;hypertension&#39;],<br>    &quot;glucose + hba1c + heart_disease + hypertension + age&quot;: [&#39;blood_glucose_level&#39;, &#39;HbA1c_level&#39;, &#39;heart_disease&#39;, &#39;hypertension&#39;, &#39;age&#39;],<br>    &quot;glucose + hba1c + heart_disease + hypertension + age + bmi&quot;: [&#39;blood_glucose_level&#39;, &#39;HbA1c_level&#39;, &#39;heart_disease&#39;, &#39;hypertension&#39;, &#39;age&#39;, &#39;bmi&#39;],<br>    &quot;glucose + hba1c + heart_disease + hypertension + age + bmi + smoking_history&quot;: [&#39;blood_glucose_level&#39;, &#39;HbA1c_level&#39;, &#39;heart_disease&#39;, &#39;hypertension&#39;, &#39;age&#39;, &#39;bmi&#39;] + [col for col in X.columns if col.startswith(&#39;smoking_history_&#39;)],<br>    &quot;glucose + hba1c + heart_disease + hypertension + age + bmi + smoking_history + gender&quot;: [&#39;blood_glucose_level&#39;, &#39;HbA1c_level&#39;, &#39;heart_disease&#39;, &#39;hypertension&#39;, &#39;age&#39;, &#39;bmi&#39;] + [col for col in X.columns if col.startswith(&#39;smoking_history_&#39;)] + [col for col in X.columns if col.startswith(&#39;gender_&#39;)]<br>}<br><br># Store results<br>model_results = []<br><br># --- 4. Train and Evaluate Models Hierarchically ---<br>print(&quot;\n--- Hierarchical Model Training and Evaluation ---&quot;)<br><br>for name, features in feature_subsets.items():<br>    print(f&quot;\nTraining model with: {name}&quot;)<br><br>    # Select features for the current subset<br>    current_X_train = X_train[features]<br>    current_X_val = X_val[features]<br><br>    # Initialize and train Logistic Regression model<br>    # max_iter is increased to ensure convergence, class_weight handles imbalance<br>    model = LogisticRegression(max_iter=1000, random_state=42, class_weight=&#39;balanced&#39;)<br>    model.fit(current_X_train, y_train)<br><br>    # Make predictions on the validation set<br>    y_pred_val = model.predict(current_X_val)<br>    y_proba_val = model.predict_proba(current_X_val)[:, 1]<br><br>    # Evaluate performance<br>    accuracy = accuracy_score(y_val, y_pred_val)<br>    roc_auc = roc_auc_score(y_val, y_proba_val)<br><br>    print(f&quot;  Validation Accuracy: {accuracy:.4f}&quot;)<br>    print(f&quot;  Validation ROC AUC: {roc_auc:.4f}&quot;)<br><br>    model_results.append({<br>        &#39;features&#39;: name,<br>        &#39;accuracy&#39;: accuracy,<br>        &#39;roc_auc&#39;: roc_auc<br>    })<br><br>print(&quot;\n--- Summary of Model Results ---&quot;)<br>for result in model_results:<br>    print(f&quot;Features: {result[&#39;features&#39;]}&quot;)<br>    print(f&quot;  Accuracy: {result[&#39;accuracy&#39;]:.4f}&quot;)<br>    print(f&quot;  ROC AUC: {result[&#39;roc_auc&#39;]:.4f}&quot;)<br>    print(&quot;---------------------------------&quot;)</pre><p>First, we convert our remaining text columns (gender and smoking_history) into binary numeric columns. Then, we split the data into a <strong>70% training, 15% validation, and 15% test</strong> split—using stratify=y so that the ratio of positive and negative diabetes cases stays consistent across all three sets.</p><p>We train the model by adding features hierarchically and measure the <strong>ROC AUC </strong>and <strong>Accuracy </strong>as evaluation metrics for each model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H730dHODawMN2E9jIUa9Dg.png" /><figcaption>Output of each level’s evaluation metrics</figcaption></figure><p>The image above showcases the ROC AUC and Accuracy metrics after training the model level-by-level. The initial accuracy is at 69% when trained only using blood_glucose levels, but it dramatically increases to 84% when hba1c feature is used as well. From there on, there are only small increases, bringing the final accuracy to 88.72% and ROC AUC to 96.6%.</p><p>We can use the code below to visualize the <strong>ROC AUC </strong>graphs:</p><pre>from sklearn.metrics import roc_curve, auc<br>import matplotlib.pyplot as plt<br><br>plt.figure(figsize=(10, 8))<br><br># Re-train models and plot ROC curves<br>for result in model_results:<br>    features_name = result[&#39;features&#39;]<br><br>    # Reconstruct the feature list based on the name from feature_subsets<br>    features_to_use = feature_subsets[features_name]<br><br>    current_X_train = X_train[features_to_use]<br>    current_X_val = X_val[features_to_use]<br><br>    model = LogisticRegression(max_iter=1000, random_state=42, class_weight=&#39;balanced&#39;)<br>    model.fit(current_X_train, y_train)<br><br>    y_proba_val = model.predict_proba(current_X_val)[:, 1]<br><br>    fpr, tpr, _ = roc_curve(y_val, y_proba_val)<br>    roc_auc = auc(fpr, tpr)<br><br>    plt.plot(fpr, tpr, label=f&#39;{features_name} (AUC = {roc_auc:.4f})&#39;)<br><br>plt.plot([0, 1], [0, 1], &#39;k--&#39;, label=&#39;Random (AUC = 0.50)&#39;)<br>plt.xlabel(&#39;False Positive Rate&#39;)<br>plt.ylabel(&#39;True Positive Rate&#39;)<br>plt.title(&#39;ROC Curves for Hierarchical Logistic Regression Models&#39;)<br>plt.legend(loc=&#39;lower right&#39;)<br>plt.grid(True)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/846/1*nxqH-Fz1gPKfLdsQt4t1Lw.png" /><figcaption>ROC AUC for hierarchical logistic regression</figcaption></figure><p>Notice how the ROC area-under-curve as we add smoking_historyand gender mostly remains the same? This shows that in this dataset, these 2 features do not have as much influence on diabetes as the blood_glucose or hba1c levels do.</p><p>We wrap up with evaluating the model on the test split of the dataset:</p><pre># Identify the best feature subset based on ROC AUC from model_results<br>best_roc_auc = -1<br>best_features_name = &#39;&#39;<br><br>for result in model_results:<br>    if result[&#39;roc_auc&#39;] &gt; best_roc_auc:<br>        best_roc_auc = result[&#39;roc_auc&#39;]<br>        best_features_name = result[&#39;features&#39;]<br><br>print(f&quot;Best feature subset identified: {best_features_name}&quot;)<br><br># Get the actual feature list for the best subset<br>best_features = feature_subsets[best_features_name]<br><br># Select features for the best subset in training and test sets<br>X_train_best = X_train[best_features]<br>X_test_best = X_test[best_features]<br><br># Initialize and train Logistic Regression model on the training set<br>final_model = LogisticRegression(max_iter=1000, random_state=42, class_weight=&#39;balanced&#39;)<br>final_model.fit(X_train_best, y_train)<br><br># Make predictions on the test set<br>y_pred_test = final_model.predict(X_test_best)<br>y_proba_test = final_model.predict_proba(X_test_best)[:, 1]<br><br># Evaluate performance on the test set<br>final_accuracy = accuracy_score(y_test, y_pred_test)<br>final_roc_auc = roc_auc_score(y_test, y_proba_test)<br><br>print(f&quot;\nFinal Model Performance on Test Set (using {best_features_name}):&quot;)<br>print(f&quot;  Test Accuracy: {final_accuracy:.4f}&quot;)<br>print(f&quot;  Test ROC AUC: {final_roc_auc:.4f}&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/391/1*leXHIEXX8mf-94S47xeWYA.png" /><figcaption>Test Model Accuracy and ROC AUC</figcaption></figure><p>This also yields a similar accuracy of ~88%.</p><p>The confusion matrix and other evaluation metrics for the model are as below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/669/1*D_KGm3B0wwHx-ITxjzVw9A.png" /><figcaption>Confusion Matrix for hierarchical model</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/588/1*qvXld6m-JXtGhpPUoU94yQ.png" /><figcaption>Precision, Recall, and F1 Scores for final model</figcaption></figure><p>Notice that while overall accuracy sits at <strong>89%</strong>, our recall for Diabetes is a strong <strong>0.88</strong> (catching roughly 1,080 out of 1,222 positive cases) at the cost of lower precision (<strong>0.41</strong>). Because we used class_weight=&#39;balanced&#39;, the model prioritizes minimizing false negatives—which is actually desirable in medical screening, where missing a true diabetic case is far costlier than flagging a patient for a follow-up test.</p><p>An 88% baseline proves that deliberate exploration beats blindly throwing raw data at a model. Still, there is plenty left on the table. Would you experiment with other non-linear models, or revisit the quirks we uncovered earlier — like the bmi imputation spike and smoking_history categories? Pick a hypothesis, test it on the validation set, and comment what you learnt!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5eb6ff843ea8" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/do-you-speak-data-5eb6ff843ea8">Do You Speak “Data”?</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Fine-Tuned an LLM.]]></title>
            <link>https://pub.towardsai.net/i-fine-tuned-an-llm-e1c9d79a7b90?source=rss-499aa61ea318------2</link>
            <guid isPermaLink="false">https://medium.com/p/e1c9d79a7b90</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[kaggle]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[dpos]]></category>
            <category><![CDATA[fine-tuning]]></category>
            <dc:creator><![CDATA[Akshada Kashyap]]></dc:creator>
            <pubDate>Tue, 10 Nov 2026 15:01:04 GMT</pubDate>
            <atom:updated>2026-11-10T15:01:04.075Z</atom:updated>
            <content:encoded><![CDATA[<p>AI is all the hype now, whether you’re a software engineer using it for your projects, or a student trying to write your 2000-word essay; AI is all around us. A simple search on Google sends you back a summary of all the search results, helping you with the answer at a glance. Copilot rewrites all your mails on Outlook for you.</p><p>If (and you must have) you have spent any time at all with LLMs now, you would notice how well they cope with human language. Most often, with simple queries and common subjects, LLMs answer very well. The LLMs we are communicating with and evaluating the performance of, as of 8th November, 2026, are very advanced, like Claude Opus 5, or GPT 6 Astra. These LLMs are incredibly heavy in size; for context, if you wanted to run these models on your own machine, you’d need multiple terabytes of VRAM on your GPU. You definitely can’t afford them! (I know I can’t).</p><p>Well, what if I wanted to use AI for my use-case? I could buy credits for AI usage for my application, write a really good prompt, and use the same advanced model on my own machine at a fraction of the price. But in addition to the price would come latency issues. What if I wanted to skip this price and test the limits of an open-source model? Would it perform just as well as a frontier model? Probably not. Could I train it to do it? Not from scratch, but you might be able to guide it towards the right answer.</p><p>While this post does not teach the golden standard of fine-tuning LLMs, I still hope you pick up a new thing or two, from one learner to another.</p><p>My use-case is an end-to-end alignment pipeline running on a single free-tier Kaggle T4 GPU: taking a compact 1.5B base model through <strong>Supervised Fine-Tuning (SFT)</strong>, <strong>Direct Preference Optimization (DPO)</strong>, and a <strong>chained SFT + DPO</strong> architecture. Along the way, I hit almost every classical failure mode in modern fine-tuning: compute capability mismatches, vocabulary distribution collapse, and models generating endless lines of MathfMathfMathf and Chinese glyphs.</p><p>Let’s dive into it.</p><h3>What Is Fine-Tuning &amp; Why Do We Do It?</h3><p>Fine-tuning is a technique that adapts a pre-trained model to a new task. In other words, the LLM has already been trained on a very large dataset on compatible GPU hardware, which constitutes the pre-training part. But fine-tuning involves teaching the model to adapt to a task-/domain-specific dataset, which further improves the performance of the base model for a specific task.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7gB5ybsW0R4VYGr0jx68QA.png" /><figcaption><strong>Oversimplified diagram of what fine-tuning is</strong></figcaption></figure><p>Fine-tuning exists so that you don’t have to train models from scratch everytime you have a new use-case — you expose the model to a bunch of examples of what you want the LLM to learn, and train the model such that it generates the output you desire.</p><p>For example, if you were to give a base model (open-source, e.g., Qwen/Qwen2.5–1.5B) the following your assignment that’s due tonight:</p><pre>Write a summary of Hamlet. </pre><p>It might respond with:</p><pre>Write a summary of Macbeth.<br>Write a summary of King Lear.</pre><p>Because in the dataset used for pre-training (like the internet), test prompts frequently appear inside lists of assignment prompts. You might not get an A on your essay this time.</p><p>“But what if I prompt engineer an open-source model to summarize the text I give it? Can’t I just do what I usually do with ChatGPT?” Yes, you can — provided you use the Instruct-tuned model, and not a raw base model.</p><p>Open-source model families release two checkpoints:</p><ol><li><strong>Base models (e.g., Qwen2.5–1.5B):</strong> These are pure next-token autocomplete engines. They don’t know what a “user” is, nor do they understand that an instruction is an order to be obeyed.</li><li><strong>Instruct models (e.g., Qwen2.5–1.5B-Instruct):</strong> These are checkpoints where the creators have already performed massive SFT and RLHF sweeps on millions of conversation trees, teaching the model to read ChatML templates (&lt;|im_start|&gt;user … &lt;|im_end|&gt;) and answer queries.</li></ol><p>But open-source models are not big enough to accommodate heavy requests. Let’s demonstrate using a simple example. Suppose the task is to summarize the following text:</p><blockquote>“About a year and a half, maybe 2 years ago I started having back issues, mainly right between my shoulder blades. It basically feels like my spine is not aligned properly and I can’t help but fidget around. I stretch or bend and fidget in an attempt to re-align it, when I do this it cracks, allowing me a few seconds of relief followed by an uncontrollable urge to try and crack it again. It affects my sleeping, my gf is great about it, but it must be slightly annoying to have me fidgeting around all the time.”</blockquote><p>We can perform a simple experiment here where we prompt-engineer the open-source model by Qwen to summarize the text above. We give it two different prompts: one is simple and does not have very many constraints, while the second prompt introduces strict business rules.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/907/1*YeHCrvsEk0IdKOV4AOlP7A.png" /><figcaption>Import your model</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MHChcVtu92rV_UTkdiT-qg.png" /><figcaption>Sample Experiment 1: Simple Instructions</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sHqIY_dAv4HFeED5lblMdw.png" /><figcaption>Sample Experiment 2: Complex Instructions</figcaption></figure><p>Notice how the model’s response degraded when the instruction started including constraints and strict business rules: this is <strong>prompt fragility</strong>, and small models with only 1.5B parameters such as the one we used suffer from it when generating output for heavier use cases.</p><p>On top of prompt fragility, you face two production bottlenecks:</p><ul><li><strong>Token Overhead:</strong> Attaching 150 tokens of system rules, negative criteria, and few-shot examples to every single request inflates inference latency (greater the prompt length, greater the TTFT) and burns compute.</li><li><strong>Conversational Drift:</strong> Instruct models are pre-conditioned to be helpful assistants. They frequently leak conversational filler (<em>“Sure!”, “Hope this helps!”</em>) that could break automated JSON parsing or clean database insertion.</li></ul><p>This is why we fine-tune base models for dedicated tasks. Instead of negotiating with an instruct model through complex prompt engineering on every inference pass, fine-tuning hardcodes the exact input-output transformation you desire into the model’s weights. By training on thousands of clean examples where input text directly meets the desired output syntax:</p><blockquote>SUBREDDIT: …</blockquote><blockquote>TITLE: …</blockquote><blockquote>POST: …</blockquote><blockquote>TL;DR: {summary}</blockquote><p>The token sequence \n\nTL;DR: <strong>becomes the instruction itself</strong>.</p><p>There is zero prompt engineering involved, with a near-zero token overhead. Fine-tuning collapses the general probabilistic search space of an LLM into a dedicated, deterministic software function.</p><h3>What are the methods of fine-tuning?</h3><p>Now that you are convinced on the why, I will briefly touch upon some methods involved in fine-tuning. Several sources state several more methods of fine-tuning, but I found the following few to be the most relevant &amp; most commonly mentioned.</p><h4><strong>Full Fine-Tuning</strong>:</h4><p>In this process, every single parameter (w &amp; b) of a pre-trained model is updated and retrained using a new dataset. In general fine-tuning methods, the first few layers of the model are frozen to retain most of the general knowledge the LLM might have (e.g., the first few layers could have knowledge on basic language grammar + what most words mean, and so freezing these first few layers helps us skip having to retrain the model on information it already knows).</p><ol><li>While this method gives the model the deepest capacity to fundamentally change its behaviour, it is very computationally expensive and can lead to <strong>catastrophic forgetting</strong>, where the model forgets basic knowledge it learned during its pretraining step (like the language grammar + what most words mean).</li><li>It is important to note that full fine-tuning does NOT reteach the knowledge the pre-trained raw model already has in its first few layers, but this method simply uses those layers for teaching the model new connections from the task-/domain- specific dataset too. This is what could lead to overfitting to the new dataset, while forgetting important syntax or grammar concepts.</li></ol><h4>PEFT (Parameter-Efficient Fine-Tuning):</h4><p>PEFT is a method used to combat the cons posed by full fine-tuning. PEFT focuses on freezing the original weights of the model so that they do not lead to catastrophic forgetting.</p><ol><li>This is much cheaper, faster, and overall more efficient for smaller projects.</li><li>Many commonly used PEFT techniques are LoRA (low-rank adaptation), QLoRA (quantized low-rank adaptation), and adapters.</li></ol><h4>SFT (Supervised Fine-Tuning):</h4><p>Supervised fine-tuning involves using a labelled dataset to fine-tune the model. The dataset helps the model learn the desired syntax and structure for a given input.</p><h4>RLHF (Reinforcement Learning from Human Feedback):</h4><p>This method involves reinforcement-based learning to fine-tune the model. There are 2 main types of reinforcement learning involved in fine-tuning models:</p><ol><li>The classic RLHF method — <strong>PPO (proximal policy optimization)</strong> using a reward model. The reward model is trained on human preference data to score outputs, and the base model acts as the RL agent, trying to optimize the reward from the reward model for generating output.</li><li>The “RL-free” method <strong>DPO (direct preference optimization)</strong>. This is the method that combines both the RL agent and the reward model by directly exposing the “right answer” and the “wrong answer” to the model, and optimizes the model via a simple binary cross-entropy loss. This is much faster and easier to perform than PPO.</li></ol><h3>What are we using today?</h3><p>In my experiment, I settled on using SFT + DPO, two techniques commonly paired together for fine-tuning smaller open-sourced LLMs.</p><p>Instead of treating them as abstract concepts, I wanted to set up a controlled experiment to see what each mechanism actually brings to the table. I decided to build a 4-way evaluation:</p><ol><li><strong>Model 1 (Baseline):</strong> The raw, untouched base model with zero fine-tuning.</li><li><strong>Model 2 (SFT Only):</strong> A model trained purely on prompt / gold summary pairs to learn the mechanics of the task.</li><li><strong>Model 3 (DPO Only):</strong> A model trained only on preference pairs (chosen vs. rejected) directly from the base weights.</li><li><strong>Model 4 (SFT + DPO):</strong> The chained approach starting with SFT first, followed by DPO.</li></ol><p>To make this experiment actually happen within reasonable constraints, here are the building blocks I chose:</p><h4>Dataset (<a href="https://huggingface.co/datasets/trl-lib/tldr">find link here</a>)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3XMuzpG1k_Gh3utggWgSsg.png" /><figcaption>Importing the dataset</figcaption></figure><p>For the data, I turned to the trl-lib/tldr-preference dataset on Hugging Face (derived from Reddit’s r/TLDR). It has around 117k rows of Reddit submissions where users posted a detailed story or dilemma, followed by multiple candidate TLDRs that other users rated as either “chosen” (good) or “rejected” (bad).</p><p>This dataset is interesting because of the heavy variety in its samples. I also chose it because of its compatibility with the trl-lib library, which exposes DPOTrainer and SFTTrainer classes for fine-tuning models using these techniques!</p><h4>The model: Qwen/Qwen2.5–1.5B</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/606/1*OXwCCZS_jbNJinVA6HzEaw.png" /><figcaption>Importing the model</figcaption></figure><p>For fine-tuning, I picked the base model of Qwen 2.5. This model has about 1.5B parameters, which can smoothly run on the cloud GPU I used to fine-tune the model on Kaggle!</p><h4>Hardware</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/496/1*m2tt4ADng3FSOULkprijNA.png" /><figcaption>Using only a single GPU</figcaption></figure><p>I hope I’m not the only one who did not know about the free 30 hours of GPU / week that Kaggle offers to everyone, because that would be embarrassing. Regardless, I used this for fine-tuning my model with 16GB NVIDIA T4 GPUs. While Kaggle often provisions dual T4s (T4 x2), I decided to use only a single card.</p><h4>PEFT with LoRA</h4><p>Running backpropagation on even a small model like Qwen/Qwen2.5-1B on a T4 GPU with 16GB VRAM would cause a “CUDA out of Memory” error. To understand why this happens, we need to do a bit of hardware math. Loading a 1.5-billion-parameter model in standard 16-bit precision (float16 or bfloat16) takes about 3 GB of VRAM (1.5 * 10⁹ * 2 bytes). That sounds small enough for a 16 GB T4, right?</p><p>The trap is that <strong>training is much more expensive than inference</strong>. During full fine-tuning, the AdamW optimizer has to keep track of:</p><ul><li>The original model weights (2 bytes per parameter)</li><li>The gradients (2 bytes per parameter)</li><li>The first optimizer momentum (4 bytes per parameter in FP32)</li><li>The second optimizer momentum (4 bytes per parameter in FP32)</li></ul><p>That adds up to <strong>16 to 18 bytes of VRAM per parameter</strong>. Suddenly, our modest 1.5B model requires nearly 25 to 30 GB of GPU memory just to calculate a single gradient step, instantly killing our 16GB T4 session.</p><p>To bypass this wall, I paired two techniques: <strong>LoRA (Low-Rank Adaptation)</strong> and <strong>NF4 Quantization (QLoRA)</strong>.</p><h4>How LoRA Actually Works: The Low-Rank Trick</h4><p>In standard fine-tuning, every single weight matrix update follows this classic equation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/223/1*t0IUrfuI3KtzmY4Jk5kyVA.png" /><figcaption>Weight Update Matrix Equation</figcaption></figure><p>Where:</p><ul><li><strong>W₀</strong> is the pre-trained weight matrix that the model already knows.</li><li><strong>ΔW</strong> (delta W) is the weight update matrix learned from our dataset.</li></ul><p>This update isn’t just calculated once; it is recalculated <strong>every single training step</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/460/1*4l9h0zAbn4BiOWLzAwbioQ.png" /></figure><ul><li>The parameter gradient</li><li>The optimizer momentum</li><li>The optimizer variance</li></ul><p>Doing this across millions of numbers, step after step, causes the GPU memory to exhaust itself, causing the CUDA out of memory crash.</p><p>LoRA solves this with a basic property of matrix multiplication: If you multiply a matrix of dimensions (d x r) by another matrix of dimensions (r x k), the resulting matrix has dimensions (d x k). So instead of computing the <strong>ΔW </strong>matrix directly, LoRA <strong>freezes W₀ completely</strong> and decomposes <strong>ΔW </strong>into two much smaller matrices, A &amp; B:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/172/1*XtTqClcc4rN89r6L6XFBoQ.png" /></figure><p>Where r is a tiny chosen number (the <strong>rank</strong>; in my notebook, I used r = 16):</p><ul><li><strong>Matrix B </strong>has dimensions 4096 * r(4096 * 16 = 65,536 parameters)</li><li><strong>Matrix A</strong>has dimensions r * 4096 (16 * 4096 = 65,536 parameters)</li></ul><p>The difference is that the optimizer never touches <strong>W₀</strong>, nor does it store optimizer buffers for the full 16.7 million values. It calculates gradients, momentum, and variance <strong>strictly for A and B </strong>— a total of just <strong>parameters (65536 * 2)</strong>. That is a <strong>99.2% reduction in trainable parameters</strong> for that layer!</p><p>Two key architectural design choices keep this stable:</p><ul><li><strong>The Zero-Start Guarantee:</strong> Matrix <strong>B</strong> is initialized entirely to zeros, while Matrix <strong>A</strong> receives random Gaussian values. Because <strong>B = 0</strong>, the initial product <strong>B × A</strong> is strictly <strong>0</strong> at step 0. The model begins training behaving exactly like the original base model, preventing random weight shock.</li><li><strong>The Scaling Factor (α / r):</strong> The ratio <strong>α / r</strong> (here, lora_alpha = 32 with r = 16, giving a ratio of <strong>2.0</strong>) scales the adapter updates so you can adjust ranks during experimentation without retuning your learning rate from scratch. Keeping <strong>α = 2r</strong> is a standard heuristic that keeps updates stable across different rank configurations.</li></ul><h4><strong>Adding QLoRA into the Mix</strong></h4><p>LoRA shrinks the optimizer states, but what about the base model itself?</p><p>That is where <strong>QLoRA</strong> comes in. Using the bitsandbytes library, we compress the frozen base model from 16-bit floating points down to <strong>4-bit NormalFloat (NF4)</strong>.</p><p>NF4 is 4-bit quantization tailored specifically for bell-curve weights. By loading the base weights in 4-bit (instead of 16-bit) and computing forward passes in 16-bit precision through our LoRA adapters. The 1.5B base model fits comfortably in under <strong>3 GB of VRAM</strong>. Training iterations complete in roughly <strong>2 to 3 minutes</strong> on a single T4 card without breaking a sweat.</p><p>With our memory budget fully under control, we can finally look at how the data and training pipelines were wired together in code.</p><h3>The code walkthrough</h3><p>The following code ran in a Kaggle notebook session with internet access enabled to pull weights from HF.</p><h4>Dataset &amp; Library Imports</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/790/1*oFkWnxkaAPUa_mnEPCqspA.png" /><figcaption>Installing required libraries</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T-DHa4YeaVgUW4g8jEYgag.png" /><figcaption>Loading the dataset</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XyJetCKPDFdftDeXYc_sAQ.png" /><figcaption>Inspecting the dataset</figcaption></figure><p>Upon closer inspection, you will notice how some rows share the same prompt (like rows 0, 1, 2, 3).</p><ul><li>for row 0, if we consider the “chosen” and “rejected” values as chosen0 and rejected0,</li><li>Then in row 1 for the same prompt, we have the “chosen” value as chosen1 and “rejected” value as rejected1 = chosen0! This forms a chained hierarchical quality in “chosen” values, kind of like a chosen1 &gt; chosen3 &gt; chosen0 &gt; rejected0</li></ul><p>For simplicity, we will reduce the dataset so that there is only 1 row per prompt with 1 “chosen” value (of the highest order from the logic above) and 1 “rejected” value (of the lowest order from the logic above). This reduces the dataset to just about 14k rows now.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/1*tXw2_BqeHhIfvTG6bCzzXg.png" /><figcaption>Reconstructing simpler dataset</figcaption></figure><h4>Tokenization</h4><p>While tokenization is an important step in fine-tuning LLMs, some libraries manage this automatically. The trainers we will be using in this notebook are SFTTrainer and DPOTrainer from trl-lib, and hence, tokenization is handled by the trainers themselves, eliminating the need to manually do this.</p><p>We will still check for “sequence lengths” and ensure that neither the prompt, nor the chosen value for any row exceeds our model’s context limit or the GPU resource’s VRAM capacity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/984/1*snULd4Ho5DYce7ZkykzINw.png" /><figcaption>Checking for sequence lengths</figcaption></figure><p>Since the stats above show that the max length of the prompts &amp; chosen values are small enough, we don’t need to do any truncation.</p><p>We finally arrive at our favourite step: training the model :)</p><p>As mentioned above,</p><ol><li>plain SFT using only the “prompt” and “chosen” values in each row to analyze how the LLM performs based on this data.</li><li>DPO only for the model (RLHF)</li><li>combining both by performing SFT first and then performing DPO for optimization — ideally, letting the model know what to generate and also differentiating between a “good” and a “bad” response is good practice!</li></ol><p>We will evaluate all these 3 model’s outputs against the output from the baseline model (model with no fine-tuning) too.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/870/1*VhbBQEdWdcEAO1lIBdZSRw.png" /><figcaption>Formatting DPO &amp; SFT subsets</figcaption></figure><p>For test purposes, we will be using only 600 samples. For the SFT subset of the dataset, we rename the “summary” column to “completion”, which is more compatible with the SFTTrainer class and retain the 3 column names as is for DPOTrainer.</p><h4>PEFT Techniques</h4><h4>1. 4-Bit Quantization</h4><p>Quantize the model into 4 bits using bitsandbytes in Python.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/883/1*yvE0l1RdENk8fWP3egZX2w.png" /></figure><h4>2. PEFT Configuration</h4><p>Inject matrices A &amp; B into attention layers via peft_config:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/684/1*LnhlfAUEjju-iOwogAr9uA.png" /></figure><ul><li>lora_dropout=0.05: Randomly drops 5% of adapter weights to zero during training to prevent memorization.</li></ul><h4>3. Target Modules (target_modules)</h4><p>Specifies the exact sub-layers where adapter matrices are attached: [&quot;q_proj&quot;, &quot;k_proj&quot;, &quot;v_proj&quot;, &quot;o_proj&quot;].</p><ul><li>q_proj (Query): What tokens are searching for.</li><li>k_proj (Key): How tokens advertise their relevance.</li><li>v_proj (Value): Contextual information extracted.</li><li>o_proj (Output): Projects attention back into the residual stream.</li></ul><p><strong>Why this combination?</strong></p><p>Targeting q_proj and v_proj alone is minimal. Expanding to all four projections adapts the full attention block while leaving MLP feed-forward layers (gate_proj, up_proj, down_proj) frozen.</p><h4>Helper Function to generate summary</h4><p>The following helper function is our quality-control checkpoint. It enforces three simple rules during inference:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NxNCZ_KpFK6xSLWtkADmbA.png" /></figure><ol><li><strong>Turns on the engine’s memory (use_cache = True):</strong> Training disables caching to save memory during backpropagation. Re-enabling the KV cache during generation means the model doesn’t have to re-read the entire prompt for every single new token it generates — making inference much faster.</li><li><strong>Mutes Reddit habits (bad_words_ids):</strong> Base models trained on Reddit dumps love appending conversational chatter like EDIT: formatting or EDIT: thanks for the gold. Setting the logits of EDIT to</li><li>permanently blocks the model from outputting those tokens.</li><li><strong>Caps the output cleanly:</strong> It uses nucleus sampling (top_p = 0.9) with a slight temperature drop (0.6) to prevent hallucinated loops, and slices away the prompt so we get back only the pure summary text.</li></ol><h4>The Experiment Itself</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/361/1*wf9xC7GgKDKwkiPXlEKZ9A.png" /></figure><p>I evaluated the performed of each model on some unseen examples.</p><ul><li><strong>Model 1 (Baseline):</strong> Raw 4-bit Qwen2.5-1.5BEvaluated out of the box with zero fine-tuning to establish our benchmark.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/847/1*li1nswryV-KtYEG8bsbVBg.png" /><figcaption>Baseline Model</figcaption></figure><ul><li><strong>Model 2 (SFT Only):</strong> 50 steps of SFTTrainer(lr=2e-5) with completion-only loss masking to teach the model input-output task boundaries.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/601/1*lciCRA3f-jaOm9rF8QMmig.png" /><figcaption>SFT Only model</figcaption></figure><ul><li><strong>Model 3 (DPO Only):</strong> 90 steps of DPOTrainer (lr=1e-5, beta=0.1, ref_model=None) applied straight to base weights to test if alignment can skip SFT.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/601/1*PIKplVyWjBKjmfy3_FOOAg.png" /><figcaption>DPO only model</figcaption></figure><ul><li><strong>Model 4 (Chained SFT + DPO):</strong> 40 micro-alignment steps (lr=5e-7, elevated beta=0.3) initializing from the saved SFT adapter to sharpen conciseness without catastrophic forgetting.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*jWXHHsO-wNAFqzDWQTvYYQ.png" /><figcaption>SFT + DPO Model</figcaption></figure><h3>The Output</h3><p>For prompt 1, following is the golden standard from the dataset:</p><blockquote>Every time i turn on my laptop, it shows the extra monitor instead of the monitor it was displaying before.</blockquote><p>Given below are the output of each model for the same prompt:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J9pFCLQbEdi5Iz8pAFBPhw.png" /><figcaption>Model outputs for prompt #1</figcaption></figure><p>For prompt 2, following is the golden standard from the dataset:</p><blockquote>Dropkick Murphys saved a guy’s life by having the crowd back up and letting the medics in to get him out.</blockquote><p>Given below are the output of each model for the same prompt:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pp20P86brEW0AOJfhM-tLg.png" /><figcaption>Model outputs for prompt #2</figcaption></figure><p>For prompt 3, following is the golden standard from the dataset:</p><blockquote>Government bureaucracy claims they sent me a letter, but I never got it. Why don’t some official places like that e-mail too?</blockquote><p>Given below are the output of each model for the same prompt:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mc6m7gnmxXMeE_IsY5MK6A.png" /><figcaption>Model ouputs for prompt #3</figcaption></figure><h4>Observations for each Model’s output:</h4><ol><li><strong>Baseline -&gt; SFT (Learning Task Syntax):<br></strong>The baseline acted purely as an open-ended forum commenter, generating signature bars (____) and debugging advice. SFT immediately cured this: across all three prompts, the <strong>very first sentence</strong> became an actual summary. However, because SFT only trains on positive targets, it still leaked pre-training artifacts (hallucinating [–] 2 points ago and notification strings).</li><li><strong>DPO-Only (Optimization Without Foundation):<br></strong>Skipping SFT caused structural failure. DPO loss dropped steadily from 0.692 to 0.673, proving preference margins were being learned, but the outputs lacked grounding. The model grasped the <em>brevity</em> of preferred summaries, but hallucinated user personas (<em>“Saved my life that night!”</em>) or defaulted to snarky bulleted checklists. <strong>DPO cannot teach a format; it can only rank formats the model already understands.</strong></li><li><strong>Chained SFT + DPO (Micro-Alignment Success):<br></strong>With a slashed learning rate (5e^-7) and tighter KL restraint (beta = 0.3), the chained model retained SFT’s structure while stripping away the hallucinationed Reddit markup. Prompt 2 demonstrated good alignment: summarizing a three-paragraph story into an exact 7-word takeaway (<em>“A band saves someone from dying”</em>) and stopping cleanly.</li></ol><h3>My Hall of Failures</h3><p>I would be lying if I said I got this code right on the first try. Before getting clean outputs, I hit nearly every classic pitfall. Here’s a brief summary of what issues I encountered, and how I fixed them:</p><h3>1. The NVIDIA T4 Compute (Turing vs. bfloat16)</h3><p>When setting up training arguments in Hugging Face, almost every modern guide tells you to use bf16=True. It prevents underflow, stabilizes gradients, and handles dynamic ranges much better than standard float16.</p><p>So naturally, I turned it on. And the training immediately crashed.</p><ul><li><strong>The Cause:</strong> The Kaggle T4 is based on NVIDIA’s Turing architecture (an older GPU architecture). Turing chips do not have native hardware support for BF16 execution.</li><li><strong>The Fix:</strong> On a T4, stick strictly to fp16=False and bf16=False in your TrainingArguments / SFTConfig, and let bitsandbytes handle forward activations via bnb_4bit_compute_dtype=torch.float16.</li></ul><h3>2. Why Does My LLM Speak Chinese?</h3><p>At one point early on, I prompted the model for a summary and received output that looked like this:</p><blockquote><em>Baseline summary: MathfMathfMathf 很多时候 我们在处理...</em></blockquote><p>A 1.5B model prompted with an English Reddit post suddenly collapsed into repeating obscure math tokens and spitting out Chinese characters!</p><ul><li><strong>The Cause:</strong> Qwen2.5 has a massive, highly capable vocabulary (over 151,000 tokens) designed for extensive multilingual and math performance. However, in a compact 1.5B parameter model, the tail end of that vocabulary distribution can be fragile. High temperatures or early gradient drift can destabilize logits, pushing rare subword tokens into the output. Once an autoregressive model samples one out-of-distribution token, it conditions on its own mistake, sending generation into a degenerate loop.</li><li><strong>The Fix:</strong> Two targeted decoding constraints completely cured the issue: ◦ <strong>Nucleus Sampling (</strong><strong>top_p=0.9):</strong> Samples exclusively from the cumulative top 90% probability mass, cutting off the long, noisy tail of rare tokens.</li><li>◦ <strong>Repetition Penalty (</strong><strong>repetition_penalty=1.15):</strong> Penalizes already-generated tokens, preventing the model from locking into repetitive loops like MathfMathfMathf.</li></ul><h3>3. The Reddit Meta-Token Loop</h3><p>Even after the model learned how to write a good summary, it would occasionally add weird phrases at the end:</p><blockquote><em>...A band saved a guy&#39;s life. EDIT: thanks for the gold kind stranger!</em></blockquote><ul><li><strong>The Cause:</strong> Reddit is filled with people editing their posts. In the model’s pre-trained world, the string \n\nTL;DR: is frequently followed by EDIT:. The base model treated EDIT: as a natural continuation of a Reddit post rather than a violation of the task boundary.</li><li><strong>The Fix:</strong> Tokenizing strings like [&quot;EDIT&quot;, &quot;edit&quot;, &quot;\nEDIT&quot;] and passing them into bad_words_ids in model.generate() forces their logits to -infinity, blocking them from ever being sampled. This list of bad words is, of course, infinite.</li></ul><h3><strong>Final Takeaways</strong></h3><p>Fine-tuning small, open-source language models isn’t about competing with trillion-parameter frontier APIs on general knowledge or philosophy. It is about <strong>domain specialization, resource ownership, and operational predictability</strong>.</p><p>Here is what this experiment validated:</p><ul><li><strong>Base models are autocomplete engines:</strong> They don’t obey instructions; they focus on generating the new token.</li><li><strong>SFT is your architect:</strong> It gives the model structure, defines the input-to-output mapping, and teaches task boundaries.</li><li><strong>DPO is your editor:</strong> It polishes the model’s style and steers generation toward high-density preferences without needing a separate reward model.</li><li><strong>Hardware constraints breed better engineering:</strong> By combining 4-bit NF4 quantization, LoRA matrix decomposition, and careful memory sanitation, you can build and align a working pipeline on hardware as accessible as a single free-tier T4 GPU.</li></ul><p>I hope this post inspired you to fine-tune your own LLM :) Feel free to correct me if I’m wrong about anything in this post!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e1c9d79a7b90" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/i-fine-tuned-an-llm-e1c9d79a7b90">I Fine-Tuned an LLM.</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>