<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Austin Ebri on Medium]]></title>
        <description><![CDATA[Stories by Austin Ebri on Medium]]></description>
        <link>https://medium.com/@austinebri?source=rss-67d950ffa5e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*LjHizhO11oDkt5YfviSbAw.jpeg</url>
            <title>Stories by Austin Ebri on Medium</title>
            <link>https://medium.com/@austinebri?source=rss-67d950ffa5e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 11:51:11 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@austinebri/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Got Milk (Data)? How I Uncovered a $100M+ UK Trade Story Using Pandas, SQL, and Seaborn]]></title>
            <link>https://medium.com/@austinebri/got-milk-data-how-i-uncovered-a-100m-uk-trade-story-using-pandas-sql-and-seaborn-78e952d1ec0b?source=rss-67d950ffa5e------2</link>
            <guid isPermaLink="false">https://medium.com/p/78e952d1ec0b</guid>
            <category><![CDATA[sql]]></category>
            <category><![CDATA[datara-flow]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[matplotlib]]></category>
            <category><![CDATA[pandas]]></category>
            <dc:creator><![CDATA[Austin Ebri]]></dc:creator>
            <pubDate>Thu, 29 Oct 2026 20:54:10 GMT</pubDate>
            <atom:updated>2026-10-30T08:11:01.393Z</atom:updated>
            <content:encoded><![CDATA[<p><em>My Week 8 deep dive at the DataraFlow Data Science Internship: mastering the Split-Apply-Combine paradigm, wrestling with skewed global trade distributions, and turning raw UN Comtrade rows into executive-ready visual stories.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1nz5cymE5vKRj7lafs60UQ.jpeg" /></figure><p>When most people look at a carton of milk, they see breakfast. After Week 8 of my Data Science internship at <strong>DataraFlow</strong>, I see international supply chains, bi-directional European trade corridors, seasonal spring production flushes, and a multimillion-dollar macroeconomic puzzle.</p><p>This week, my mission was to bridge the gap between <strong>advanced data aggregation</strong> and <strong>statistical data visualisation</strong>. Working with real-world datasets from the <strong>United Nations Comtrade database</strong> (week09.comtrade_milk_uk_monthly_14, week09.comtrade_milk_uk_jan_jul_15, and week09.countrymilk), I set out to dissect the United Kingdom’s global trade in milk and cream.</p><p>I didn’t just want to write code that ran without errors — I wanted to interrogate the data until it gave up its best stories.</p><h3>What I Learnt: The Power of “Split-Apply-Combine” and Visual Storytelling</h3><p>If Week 8 had a central thesis, it was the <strong>Split-Apply-Combine</strong> paradigm. Whether I was writing Python in Pandas or querying our cohort database in T-SQL, I learnt how to break a massive, noisy dataset into meaningful segments, apply mathematical or logical operations to each segment independently, and combine the results into a clean summary table.</p><p>Specifically, I levelled up my toolkit across three core pillars:</p><ul><li><strong>Advanced Pandas Grouping &amp; Reshaping:</strong> I moved far beyond basic .groupby() calls. I learnt how to group across multi-dimensional keys ([&#39;Trade Flow&#39;, &#39;Commodity Code&#39;, &#39;Period&#39;]), apply multiple summary functions simultaneously using .aggregate([&#39;sum&#39;, &#39;max&#39;, &#39;mean&#39;]), use .filter() with custom boolean functions to isolate specific group behaviours, and reshape long data into wide cross-tabulations using pivot_table() and .dropna().</li><li><strong>Analytical SQL (</strong><strong>GROUP BY, Conditional Aggregation &amp; CTEs):</strong> Because our data lives in the week09 database schema, I practised translating Pandas logic directly into SQL. I used CASE WHEN expressions inside SUM() to pivot monthly imports and exports on the fly, built Common Table Expressions (CTEs) to rank top trading partners, and rolled remaining countries into an &#39;Other&#39; slice to prepare tidy (long-form) tables for plotting.</li><li><strong>Statistical Visualisation with Seaborn &amp; Matplotlib:</strong> A table tells you <em>what</em> the numbers are; a great plot tells you <em>how they behave</em>. In Part B of this week’s curriculum, I mastered five essential statistical plots — <strong>Histograms, Kernel Density Estimate (KDE) plots, Box &amp; Violin plots, Linear Regression plots, and Heatmaps</strong> — learning when to use each to expose distributions, medians, interquartile ranges, correlations, and multi-month intensity.</li></ul><h3>What I Solved: Investigating the UK’s Dairy Economy</h3><p>My primary analytical project (Project 4) centred on four deceptively simple macroeconomic questions about the UK’s milk and cream trade across two Harmonized System (HS) commodity codes:</p><ul><li><strong>0401</strong>: <em>Milk and cream, neither concentrated nor sweetened (unprocessed)</em></li><li><strong>0402</strong>: <em>Milk and cream, concentrated or sweetened (processed)</em></li></ul><h4>1. Is the UK’s dairy trade balance positive or negative?</h4><p>After mapping the raw commodity codes into readable labels (unprocessed vs. processed) and stripping out the &#39;World&#39; summary rows, I grouped the dataset by Trade Flow and summed the trade values.</p><ul><li><strong>What I uncovered:</strong> Across the full 12 months of 2014, UK dairy exports totalled <strong>$596.08M</strong> against <strong>$563.88M</strong> in imports — a net annual trade surplus of <strong>+$32.19M</strong>. Digging deeper by commodity type revealed the real story: the UK runs a deficit in raw, unprocessed liquid milk (0401), which is heavy and expensive to ship far, but offsets it with a massive surplus in higher-margin concentrated and sweetened dairy (0402). Furthermore, pivoting by month exposed a distinct <strong>seasonal flush</strong>: UK milk exports surge between March and June when spring pastures peak, before tapering off in winter.</li></ul><h4>2. Who are the UK’s main trading partners?</h4><p>By splitting the data into Imports and Exports, grouping by Partner, and sorting aggregated trade values in descending order, I mapped the UK&#39;s trade network.</p><ul><li><strong>What I uncovered:</strong> The UK imports milk from just 21 countries — heavily dominated by its closest European neighbours: <strong>Ireland, France, Germany, the Netherlands, and Belgium</strong>. Conversely, the UK exports to over 100 countries worldwide! While <strong>Ireland</strong> is the #1 partner on both sides of the ledger, major non-European buyers like <strong>Algeria, China, and Hong Kong</strong> emerge as powerhouse importers of British processed milk.0402).</li></ul><h4>3. Who are the “regular customers” buying every single month?</h4><p>To find the backbone of UK dairy exports, I needed to identify countries that bought <em>both</em> unprocessed and processed milk in <em>every single month</em> of the period. Using Pandas’ .groupby(PARTNER).filter()I wrote a custom filter function (len(group) == 10 for the 5-month 2015 dataset; len(group) == 24 for the 12-month 2014 dataset).</p><ul><li><strong>What I uncovered:</strong> In the January–May 2015 dataset, just <strong>16 regular partner countries</strong> accounted for <strong>over 75% of the UK’s total export value</strong> (and in 2014, 14 countries drove over 71% of annual exports). That is a textbook Pareto distribution in global trade: a small core of loyal buyers sustains three-quarters of the entire industry.</li></ul><h4>4. Which countries engage in bi-directional trade with the UK?</h4><p>Why would a country both sell milk to the UK and buy milk from the UK? To find out who does both, I built a pivot table with Partner as the index and Trade Flow as the columns, then ran .dropna() to immediately eliminate one-way trading partners.</p><ul><li><strong>What I uncovered:</strong> Exactly 20 countries (in Jan–May 2015) and 26 countries (in 2014) had two-way dairy trade with the UK, led by Ireland, France, and Germany, where raw milk and specialised processed dairy cross borders in both directions.</li></ul><h4>5. Bringing the Data to Life in Part B (Seaborn &amp; Matplotlib)</h4><p>To wrap up the take-home, I transformed the Part A dataset into a 5-part visual suite:</p><ol><li><strong>Stacked Histogram:</strong> Showed that most individual monthly transactions sit between $10,000 ($10⁴$) and $1,000,000 ($10⁶$), with export transactions occurring far more frequently than imports.</li><li><strong>KDE Plot:</strong> Revealed a tight, unimodal density peak for exports around $10^{4.5}\text{–}10^{5.5}$ USD, contrasted against a wider, bimodal distribution for imports driven by bulk European suppliers.</li><li><strong>Box &amp; Violin Plots:</strong> Demonstrated side-by-side that processed milk (0402) commands a consistently higher median trade value and tighter interquartile range than unprocessed milk (0401).</li><li><strong>Linear Regression Plot (</strong><strong>sns.regplot):</strong> Proved a strong positive linear relationship between monthly imports and exports among bi-directional trading partners.</li><li><strong>Annotated Heatmap (</strong><strong>sns.heatmap):</strong> Mapped monthly trade intensity across the Top 10 partners, making Ireland’s $37M+ monthly peaks in March and May 2015 jump right off the screen.</li></ol><h3>Challenges I Faced &amp; How I Overcame Them</h3><p>No real data science project goes smoothly on the first run. Here are the three biggest hurdles that tripped me up this week — and how I engineered my way past them:</p><h4>Challenge 1: The “World” Double-Counting Trap &amp; Leading Zeroes</h4><p>When I first inspected the UN Comtrade tables, I noticed Partner rows labelled &#39;World&#39; sitting right alongside individual countries like &#39;Ireland&#39; and &#39;France&#39;. If you run a naive .groupby(&#39;Trade Flow&#39;).sum() on the raw dataframe, you double-count the entire planet&#39;s trade! On top of that, HS commodity codes like &#39;0401&#39; and &#39;0402&#39; have leading zeroes that CSV parsers love to strip into integers (401 and 402), breaking string-based mapping functions.</p><ul><li><strong>How I overcame it:</strong> I made data hygiene my very first step. I explicitly split every dataset into milk_world and milk_countries = milk[milk[&#39;Partner&#39;] != &#39;World&#39;].copy() before running any partner analysis, and used milk_world as a validation benchmark to verify that the sum of individual countries matched the reported world total. For commodity codes, I enforced string normalisation (str(code).strip().zfill(4)) so my classification functions never failed.</li></ul><h4>Challenge 2: Extreme Right-Skewness Ruining My Visuals</h4><p>When I first tried plotting a raw histogram and KDE plot of Trade Value (US$), the charts looked terrible. Why? Because the UK exports $316 worth of milk to Australia in the same dataset where it trades over $37,000,000 with Ireland! On a linear axis, 95% of the data was squashed into a single vertical spike near zero.</p><ul><li><strong>How I overcame it:</strong> Rather than settling for an unreadable chart, I applied a <strong>base-10 logarithmic transformation</strong> (np.log10(viz_df[&#39;Trade Value (US$)&#39;].clip(lower=1))) for the distribution plots (Histogram, KDE, Box, and Violin), while scaling values into <strong>Millions of US$ (</strong><strong>/ 1e6)</strong> for the Regression plot and Heatmap. Instantly, the hidden bimodal shapes and quartile structures became crystal clear.</li></ul><h4>Challenge 3: SQL Rounding Discrepancies &amp; Custom Sorting</h4><p>In our SQL practice tasks, calculating balance_musd by subtracting an already-rounded imports_musd from a rounded exports_musd can introduce 0.01 floating-point rounding errors. Additionally, Part B asked for a tidy SQL table of the Top 5 export partners plus an &#39;Other&#39; category, sorted alphabetically by partner name <em>except</em> with &#39;Other&#39; forced to the bottom of every month.</p><ul><li><strong>How I overcame it:</strong> In SQL, I learned to always compute the difference of the <em>unrounded</em> SUM(CASE WHEN ...) expressions first and apply ROUND(..., 2) only at the very outside. For the custom sort order, I used a CASE WHEN partner_group = &#39;Other&#39; THEN 1 ELSE 0 END expression in the ORDER BY clause right before partner_group ASC.</li></ul><h3>What I’d Do Differently &amp; Advice for Fellow Learners</h3><p>If I were starting Week 8 over again, I would <strong>sketch the target shape of my dataframe before writing a single line of code</strong>. Early in the week, I sometimes jumped straight into chaining .groupby() and .unstack() without pausing to ask: <em>&quot;What does a single row of my final table need to represent?&quot;</em> Once I started defining the exact grain of the output first—especially when preparing long-form data for Seaborn or wide matrices for sns.heatmap—my coding speed doubled.</p><p><strong>My advice to anyone learning Pandas, SQL, and Seaborn right now:</strong></p><ol><li><strong>Always interrogate suspiciously small numbers:</strong> When I ran countries.dropna() to find bi-directional trading partners, I noticed a few countries had total trade values under $5,000. Don&#39;t just trust the code output—ask whether those are speciality shipments, re-exports, or reporting noise.</li><li><strong>Use </strong><strong>.pivot_table() and </strong><strong>.dropna() as a filter:</strong> Pivoting isn&#39;t just for spreadsheets. Pivoting Trade Flow into columns and calling .dropna() is the cleanest, most Pythonic way to find entities that participate in multiple categories simultaneously.</li><li><strong>Pair every chart with a one-sentence takeaway:</strong> This week’s instructions required writing a single sentence underneath every plot explaining what it shows. That constraint is gold. If you cannot summarise what your chart proves in one crisp sentence, your chart is either too cluttered or using the wrong visual encoding.</li></ol><p>Week 8 pushed me to stop thinking like someone who just writes scripts and start thinking like a data scientist who builds evidence. Onto the next dataset!</p><p><em>Thanks for reading! If you enjoyed this breakdown or are also navigating your data science journey, feel free to leave a clap 👏 or connect with me in the comments!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=78e952d1ec0b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Week 7 at DataraFlow: Turning Messy Data into Meaningful Insights with Pandas and SQL]]></title>
            <link>https://medium.com/@austinebri/week-7-at-dataraflow-turning-messy-data-into-meaningful-insights-with-pandas-and-sql-50930d2338e4?source=rss-67d950ffa5e------2</link>
            <guid isPermaLink="false">https://medium.com/p/50930d2338e4</guid>
            <category><![CDATA[sql]]></category>
            <category><![CDATA[datara-flow]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[pandas]]></category>
            <dc:creator><![CDATA[Austin Ebri]]></dc:creator>
            <pubDate>Fri, 23 Oct 2026 23:14:31 GMT</pubDate>
            <atom:updated>2026-10-23T23:14:31.258Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wZC7ZCS8F4MK1REbqF3KFw.png" /></figure><p>Have you ever opened a dataset expecting to start analysing it, only to discover inconsistent column names, unexpected HTML tags, missing values, and data types that make calculations impossible?</p><p>I have learnt that this is a normal part of working with real-world data.</p><p>As aspiring data scientists, we often focus on building models, creating visualisations, and discovering patterns. However, before any of these tasks can produce meaningful results, we must ensure that the data is clean, consistent, and reliable.</p><p>During <strong>Week 7 of my DataraFlow Data Science Internship</strong>, I explored this essential part of the data science workflow. My focus was on using <strong>Pandas for data cleaning, visualisation, transformation, and combining datasets</strong>, alongside <strong>SQL for data cleaning, date-range analysis, joins, aggregations, and window functions</strong>.</p><p>Through practical projects involving historical weather records and World Bank economic indicators, I learnt how to transform raw data into information that can support better decisions.</p><p>Here is a breakdown of what I learnt, the challenges I encountered, and the lessons I will carry into my next stage of learning.</p><h3>1. What I Learnt</h3><h3>A. Cleaning Data with Pandas</h3><p>One of my biggest takeaways this week was that data cleaning often starts with seemingly insignificant details.</p><p>For example, a column named &#39; Mean Humidity&#39; contains a leading space. Although it looks harmless, inconsistent column names can cause errors when referencing columns in a DataFrame.</p><p>I learnt to remove unwanted whitespace from column headers using:</p><pre>london.columns = london.columns.str.strip()</pre><p>This ensures that column names are easier to reference and more consistent throughout the analysis.</p><p>I also encountered HTML artefacts such as 186&lt;br /&gt; in numerical columns. These artefacts prevented the values from being treated as ordinary numbers.</p><p>My cleaning workflow involved converting the values to strings, removing the unwanted suffix, and converting them to integers:</p><pre>london[&quot;Mean Humidity&quot;] = (<br>    london[&quot;Mean Humidity&quot;]<br>    .astype(str)<br>    .str.rstrip(&quot;&lt;br /&gt;&quot;)<br>    .astype(int)<br>)</pre><p>This illustrates an important lesson: <strong>data must have the correct format before it can be analysed reliably.</strong></p><p>I also strengthened my understanding of renaming columns and converting data types with .astype() and pd.to_datetime().</p><p>For example:</p><pre>london[&quot;Date&quot;] = pd.to_datetime(london[&quot;Date&quot;])</pre><p>Converting dates into proper datetime objects makes chronological sorting, date filtering, and time-series analysis much easier.</p><h3>B. Visualising Data and Working with Dates</h3><p>Data visualisation helps us move beyond rows and columns to identify patterns, trends, and unusual observations.</p><p>I learnt how to set a datetime column as the DataFrame index:</p><pre>london = london.set_index(&quot;Date&quot;)</pre><p>Once the index is a datetime index, selecting a particular period becomes straightforward:</p><pre>summer = london.loc[&quot;2014-06-01&quot;:&quot;2014-08-31&quot;]</pre><p>This was particularly useful when analysing London’s summer weather.</p><p>I also practised plotting data directly with Pandas:</p><pre>summer[[&quot;Mean Temperature&quot;]].plot(<br>    grid=True,<br>    figsize=(10, 5)<br>)</pre><p>Using grid=True improves readability, while figsize=(10, 5) provides enough space to examine the trend.</p><p>My key lesson was that effective visualisation starts with properly prepared data. A chart can be technically correct but still misleading if its underlying dates, values, or data types are inconsistent.</p><h3>C. Combining Datasets with Pandas</h3><p>Another important topic was combining datasets using pd.merge().</p><p>Real-world analysis frequently requires information from multiple sources. For example, economic analysis may require GDP figures from one dataset and life expectancy figures from another.</p><p>I explored four common join types:</p><ul><li><strong>Inner join:</strong> Returns only records with matching keys in both datasets.</li><li><strong>Left join:</strong> Retains every record from the left dataset and adds matching records from the right.</li><li><strong>Right join:</strong> Retains every record from the right dataset and adds matching records from the left.</li><li><strong>Outer join:</strong> Retains records from both datasets, including unmatched records.</li></ul><p>For my GDP and life expectancy project, I used an inner join on the Country column:</p><pre>combined = pd.merge(<br>    gdp,<br>    life_expectancy,<br>    on=&quot;Country&quot;,<br>    how=&quot;inner&quot;<br>)</pre><p>This allowed me to focus on countries with matching records in both datasets.</p><p>I also learnt the value of using uppercase variable names for constants, such as:</p><pre>START_DATE = &quot;2014-06-01&quot;<br>END_DATE = &quot;2014-08-31&quot;</pre><p>This convention makes important configuration values easier to identify and maintain.</p><h3>D. Exploring Statistical Relationships</h3><p>Data science is not only about cleaning and plotting data; it also involves understanding relationships between variables.</p><p>I learnt how to calculate the <strong>Spearman rank correlation coefficient</strong> using SciPy:</p><pre>from scipy.stats import spearmanr</pre><pre>correlation, p_value = spearmanr(<br>    combined[&quot;GDP_GBP_Millions&quot;],<br>    combined[&quot;Life Expectancy&quot;]<br>)</pre><pre>print(&quot;Spearman correlation:&quot;, correlation)<br>print(&quot;P-value:&quot;, p_value)</pre><p>Spearman correlation measures the strength and direction of a monotonic relationship between two variables using their ranks. It is useful when a relationship may not be linear or when rank-based analysis is more appropriate.</p><p>I also explored logarithmic scales for visualising heavily skewed data:</p><pre>combined.plot(<br>    x=&quot;GDP_GBP_Millions&quot;,<br>    y=&quot;Life Expectancy&quot;,<br>    kind=&quot;scatter&quot;,<br>    logx=True<br>)</pre><p>GDP values can vary enormously between countries. A logarithmic x-axis can make differences among countries with smaller economies easier to inspect without removing the influence of larger economies.</p><p>This reinforced my understanding that choosing an appropriate visualisation is an analytical decision, not merely a design choice.</p><h3>2. What I Solved</h3><p>The practical projects helped me apply these concepts to real datasets rather than treating them as isolated coding exercises.</p><h3>Project A: Finding the Best Time for a London Holiday</h3><p>The first project involved historical weather data for London Heathrow in 2014, originally sourced from Weather Underground.</p><p>The objective was to clean the weather records, examine the summer months, and identify a suitable two-week period for a holiday based on temperature and precipitation.</p><p>My workflow involved:</p><ol><li><strong>Inspecting the raw data:</strong> I examined the column names, values, and data types to identify inconsistencies.</li><li><strong>Cleaning the dataset:</strong> I removed unwanted whitespace from column headers and dealt with HTML artefacts attached to values.</li><li><strong>Preparing the dates:</strong> I converted the date column into a datetime format and used it as the index.</li><li><strong>Filtering summer records:</strong> I selected the relevant summer period for further analysis.</li><li><strong>Visualising weather patterns:</strong> I plotted temperature and precipitation to compare conditions across the season.</li><li><strong>Evaluating possible holiday windows:</strong> I examined the patterns to identify a two-week period that appeared favourable.</li></ol><p>This project showed me how data analysis can support an everyday decision.</p><p>Instead of relying entirely on assumptions about the British summer, I could use historical observations to investigate when conditions were more likely to be suitable.</p><p>However, historical weather patterns are not guarantees of future conditions. They provide useful context rather than a reliable forecast for a specific future holiday.</p><h3>Project B: Do Richer Countries Have Longer Life Expectancies?</h3><p>My second project explored the relationship between GDP and life expectancy using two World Bank datasets from 2013.</p><p>The question was straightforward:</p><p><strong>Do countries with higher GDP tend to have higher life expectancy?</strong></p><p>To investigate this, I needed to prepare and combine the datasets carefully.</p><p><strong>Step 1: Clean the datasets</strong></p><p>I removed aggregated regional records so that the analysis focused on individual countries rather than groups such as regions or income categories.</p><p>I also handled null values and checked that the country identifiers and numerical columns were suitable for analysis.</p><p><strong>Step 2: Convert GDP into British pounds</strong></p><p>The GDP figures needed to be converted from US dollars into millions of British pounds to make the units more suitable for the intended analysis.</p><p>This required applying an appropriate currency conversion rate and scaling the resulting values into millions. The exchange-rate assumption matters because it affects the resulting figures and should be documented in a reproducible analysis.</p><p><strong>Step 3: Merge the datasets</strong></p><p>I combined the prepared datasets using an inner join on the Country column.</p><p>This ensured that the resulting dataset contained countries with matching GDP and life expectancy records.</p><p><strong>Step 4: Measure the relationship</strong></p><p>I calculated the Spearman rank correlation coefficient and used visualisation to investigate the relationship between GDP and life expectancy.</p><p>The analysis indicated a <strong>moderate positive relationship</strong>: countries with higher GDP tended to have higher life expectancy in the dataset.</p><p>However, this finding requires careful interpretation.</p><p>GDP alone does not determine how long people live. Healthcare access, education, public infrastructure, income distribution, nutrition, political stability, and other socioeconomic factors may also influence life expectancy.</p><p>Furthermore, correlation does not establish causation.</p><p>This project reminded me that the responsibility of a data scientist extends beyond calculating a statistic. We must understand what the result means, what it does not mean, and which limitations should accompany our conclusions.</p><h3>Project C: Reproducing the Analysis with SQL</h3><p>Alongside Pandas, I practised running SQL queries against the cohort database schemas, including week07 and week08.</p><p>The objective was to reproduce data preparation, joining, filtering, and aggregation tasks directly in the database.</p><p>This included working with:</p><ul><li>Data cleaning and column selection.</li><li>Date filtering and summer-period analysis.</li><li>Joins between related datasets.</li><li>Aggregations to summarise observations.</li><li>Window functions for analysing consecutive periods.</li></ul><p>One particularly interesting exercise involved identifying the driest seven-day summer period using SQL window functions.</p><p>Rather than examining each week manually, a window-based approach can calculate rolling totals over seven consecutive days and help identify the period with the lowest precipitation.</p><p>Conceptually, a rolling seven-day total can be expressed as:</p><pre>SUM(precipitation) OVER (<br>    ORDER BY observation_date<br>    ROWS BETWEEN 6 PRECEDING AND CURRENT ROW<br>)</pre><p>This expression calculates a seven-row rolling sum when the records are ordered by date. In practice, I would also ensure that the dataset contains one record per calendar day, with missing dates handled appropriately, so that seven rows truly represent seven consecutive days.</p><p>This exercise helped me appreciate SQL as more than a language for retrieving records. It can perform substantial analytical work directly within a database.</p><h3>3. Challenges I Faced and How I Overcame Them</h3><p>Learning to work with data also means learning how to troubleshoot errors. Two challenges stood out during the week.</p><h3>Challenge 1: ModuleNotFoundError: No module named &#39;db&#39;</h3><p>While working in my Jupyter notebook, I encountered this error when attempting to import run_query from the project&#39;s database connection module.</p><p>The problem was related to the project structure.</p><p>My notebook was running inside the Week-7/ subfolder, while the database connection script was located in the parent project directory. Python could not find the module because the project root was not available on the import search path.</p><p><strong>How I solved it</strong></p><p>I used Python’s pathlib and sys modules to locate the project root dynamically and add it to sys.path.</p><p>The approach was to search upwards through the directory hierarchy until the expected project files or folders were found, then add that directory to Python’s module search path before importing the database function.</p><p>A simplified example is:</p><pre>from pathlib import Path<br>import sys</pre><pre>current_dir = Path.cwd().resolve()</pre><pre>for directory in (current_dir, *current_dir.parents):<br>    if (directory / &quot;db.py&quot;).exists():<br>        sys.path.insert(0, str(directory))<br>        break</pre><pre>from db import run_query</pre><p>This example assumes that db.py is located at the project root. If the module is stored in a package or another subdirectory, the locator condition must be adjusted accordingly.</p><p>The key lesson was that import errors do not always indicate a problem with the module itself. Sometimes, the issue is how Python resolves file paths and package locations.</p><p>I also learnt why understanding project structure is an important skill when working with notebooks and shared codebases.</p><h3>Challenge 2: Messy Column Headers and HTML Artefacts</h3><p>The second challenge came directly from the raw weather CSV file.</p><p>Some column headers contained leading spaces, while some numerical values had HTML fragments attached to them.</p><p>For example:</p><ul><li>&#39; Mean Humidity&#39; instead of &#39;Mean Humidity&#39;.</li><li>&#39;186&lt;br /&gt;&#39; instead of &#39;186&#39;.</li></ul><p>These inconsistencies made it difficult to reference columns consistently and convert values into numerical types.</p><p><strong>How I solved it</strong></p><p>I stripped whitespace from the column headers:</p><pre>london.columns = london.columns.str.strip()</pre><p>I then converted the affected values to strings, removed the unwanted trailing HTML fragment, and converted the cleaned values into integers:</p><pre>london[&quot;Mean Humidity&quot;] = (<br>    london[&quot;Mean Humidity&quot;]<br>    .astype(str)<br>    .str.rstrip(&quot;&lt;br /&gt;&quot;)<br>    .astype(int)<br>)</pre><p>One important technical detail is that .str.rstrip(&quot;&lt;br /&gt;&quot;) treats its argument as a collection of characters to remove from the right-hand end of a string, not as an exact HTML substring. It is suitable for this particular pattern, but a more robust solution for varied HTML artefacts would use an exact replacement or a regular expression.</p><p>For example:</p><pre>london[&quot;Mean Humidity&quot;] = (<br>    london[&quot;Mean Humidity&quot;]<br>    .astype(&quot;string&quot;)<br>    .str.replace(r&quot;&lt;br\s*/?&gt;&quot;, &quot;&quot;, regex=True)<br>    .str.strip()<br>    .astype(int)<br>)</pre><p>This version targets the HTML line-break tag more explicitly.</p><p>These experiences taught me to inspect the actual structure of a dataset before applying transformations. A cleaning method that works for one type of inconsistency may not be appropriate for every dataset.</p><h3>4. Insights and Takeaways</h3><h3>Raw Data Is Rarely Ready for Analysis</h3><p>My biggest takeaway from Week 7 is that <strong>data preparation is not a minor step before analysis; it is a fundamental part of the analytical process.</strong></p><p>A dataset can contain thousands of records and still be unsuitable for meaningful analysis if its dates are inconsistent, its values are stored as text, its column names are unreliable, or its records contain irrelevant aggregates.</p><p>I now have a stronger appreciation for the need to:</p><ul><li>Inspect data before modifying it.</li><li>Standardise column names and data types.</li><li>Identify missing values and unexpected strings.</li><li>Validate transformations before proceeding.</li><li>Check that joins produce the expected number of records.</li><li>Interpret statistical results in their proper context.</li></ul><p>These practices improve the reliability and reproducibility of the entire workflow.</p><h3>Pandas and SQL Solve Related Problems in Different Environments</h3><p>Another important insight was understanding the value of performing data preparation directly in SQL rather than automatically transferring every raw dataset into Pandas.</p><p>Pandas is powerful for exploratory analysis, flexible transformations, statistical calculations, and visualisation. SQL is particularly effective for filtering, joining, aggregating, and transforming data stored in relational databases.</p><p>In many professional data environments, analysts and data engineers clean and combine data inside the database before loading a smaller, analysis-ready dataset into Pandas.</p><p>This approach can reduce unnecessary data transfers and take advantage of database query engines. It is especially useful when working with large datasets that may be inefficient to load entirely into memory.</p><p>However, the best approach depends on the data volume, database capabilities, transformation requirements, and the tools available.</p><p>The goal is not to choose Pandas over SQL, or SQL over Pandas. It is to understand where each tool fits into an effective data workflow.</p><h3>Good Analysis Requires More Than Good Code</h3><p>The GDP and life expectancy project reinforced another lesson: a technically correct result does not automatically lead to a valid conclusion.</p><p>A correlation coefficient can describe a relationship, but it cannot independently explain the mechanisms behind that relationship.</p><p>Similarly, a weather chart can reveal historical patterns, but it cannot guarantee future conditions.</p><p>As I continue developing my data science skills, I want to become better at combining technical accuracy with critical thinking, clear communication, and an awareness of analytical limitations.</p><p>That is what turns code and statistics into useful insights.</p><h3>Conclusion</h3><p>Week 7 of the DataraFlow Data Science Internship strengthened my practical understanding of data cleaning, transformation, visualisation, statistical analysis, and SQL.</p><p>From cleaning unexpected HTML artefacts in London weather records to combining World Bank datasets and investigating the relationship between GDP and life expectancy, each exercise demonstrated how careful preparation supports more reliable analysis.</p><p>I also learnt to troubleshoot Python import errors, work with datetime indexes, use different join types, and apply SQL window functions to time-series problems.</p><p>Most importantly, I learnt that becoming a data scientist is not simply about learning more libraries or writing more complex queries. It is about asking better questions, understanding the data, validating the results, and communicating findings responsibly.</p><p>I look forward to building on these lessons as I continue my journey in data science.</p><p><strong>Now, I would love to hear from you: What is your favourite data-cleaning trick, Pandas method, or SQL technique for turning messy data into something useful?</strong></p><p>Share it in the comments. Your approach might help another learner solve a problem more efficiently.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=50930d2338e4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Bridging the Gap Between SQL Warehouses and Python: My Week 6 Data Science Journey at DataraFlow]]></title>
            <link>https://medium.com/@austinebri/bridging-the-gap-between-sql-warehouses-and-python-my-week-6-data-science-journey-at-dataraflow-8ea387a0d958?source=rss-67d950ffa5e------2</link>
            <guid isPermaLink="false">https://medium.com/p/8ea387a0d958</guid>
            <category><![CDATA[pandas]]></category>
            <category><![CDATA[datara-flow]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[sql]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Austin Ebri]]></dc:creator>
            <pubDate>Sun, 18 Oct 2026 11:49:35 GMT</pubDate>
            <atom:updated>2026-10-18T11:49:35.497Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_yY9Lcayf5rXL3TZW66OZQ.png" /></figure><p>How I went from writing basic SELECT statements and array checks to debugging live cloud database schemas, catching silent SQL bugs, and auditing real-world datasets using T-SQL, NumPy, and Pandas.</p><p>In the real world, data rarely arrives as a spotless CSV file sitting in your local downloads folder. Most data analysts and data scientists spend their days connecting to read-only cloud data warehouses, profiling unfamiliar schemas, shaping messy tables with SQL, and pulling right-sized datasets into Python for deeper exploration.</p><p>Week 6 of my data science training with <strong>DataraFlow (Cohort 2)</strong> was all about mastering that exact production workflow. Across a packed curriculum covering <strong>Intermediate SQL</strong>, <strong>NumPy</strong>, and <strong>Pandas</strong>, I transitioned from isolated coding exercises to querying a live Azure SQL Server database (assessmentDB) directly from <strong>Antigravity IDE</strong>.</p><p>Here is a deep dive into what I learned, how I approached the tasks, the roadblocks I hit along the way, and the lessons I am taking forward.</p><h3>1. What I Learned: Key Concepts, Tools, and Techniques</h3><p>This week’s learning path brought together three foundational pillars of the modern data science stack:</p><h3>Vectorised Array Computing with NumPy</h3><p>NumPy (“Numerical Python”) is the engine underneath the Python data ecosystem. Instead of writing sluggish Python for loops, I learned how to inspect and compare arrays element-by-element in C-optimised memory:</p><ul><li><strong>IEEE 754 Floating-Point Realities:</strong> Why standard equality (== np.nan) always fails and why np.isnan() is essential for missing-value detection, alongside np.allclose() and np.isclose() to compare floating-point arrays within a tolerance.</li><li><strong>Type &amp; Memory Profiling:</strong> Distinguishing complex, real, and scalar values (np.iscomplex(), np.isreal(), np.isscalar()) and calculating exact array buffer footprints using .size * .itemsize.</li><li><strong>Array Dimensionality:</strong> Understanding array shapes (arr.shape) and reshaping 1-D arrays into multi-dimensional matrices.</li></ul><h3>Series &amp; DataFrame Manipulation with Pandas</h3><p>Moving from raw arrays to labelled tables, I explored how <strong>Pandas Series</strong> and <strong>DataFrames</strong> streamline tabular analysis:</p><ul><li><strong>Series Operations:</strong> Converting between Python lists, dictionaries, NumPy arrays, and Pandas Series; flattening nested Series of lists with .explode(); and safely coercing corrupted string entries (like &#39;python&#39; mixed into numeric strings) into NaN using pd.to_numeric(..., errors=&#39;coerce&#39;).</li><li><strong>DataFrame Indexing &amp; Boolean Masking:</strong> Using zero-based integer indexing (.iloc[]), previewing subsets with .head() and .tail(), and chaining compound bitwise filters (&amp; and |) to isolate high-burden tuberculosis records from the World Health Organisation (week06.who_pop_tb_all) dataset.</li></ul><h3>Production-Grade T-SQL (Intermediate SQL)</h3><p>The highlight of the week was working against a live <strong>Microsoft SQL Server (Azure SQL)</strong> database using a read-only account (db_datareader). Because write operations (INSERT, UPDATE, DELETE, CREATE) are blocked by design in production analytics environments, <strong>Common Table Expressions (CTEs)</strong> became my workspace. Key SQL techniques I mastered included:</p><ul><li><strong>Schema Discovery:</strong> Querying INFORMATION_SCHEMA.TABLES and INFORMATION_SCHEMA.COLUMNS to profile unfamiliar schemas before writing a single analytical query.</li><li><strong>Conditional Aggregation &amp; </strong><strong>HAVING:</strong> Using SUM(CASE WHEN ... THEN 1 ELSE 0 END) to pivot multiple filtered counts in a single table scan, and filtering grouped aggregates with HAVING.</li><li><strong>Relational Joins &amp; Anti-Joins:</strong> Combining datasets across different organizations (WHO tuberculosis data vs. World Bank population figures) and running LEFT JOIN ... WHERE right.key IS NULL (or NOT EXISTS) anti-joins to audit unmatched records.</li><li><strong>Set Operators (</strong><strong>UNION ALL):</strong> Stacking separate tables vertically—the SQL equivalent of pd.concat—to unify multi-city weather logs.</li><li><strong>Window Functions:</strong> Using ROW_NUMBER() OVER (PARTITION BY ... ORDER BY ...) to solve the classic &quot;Top-1 per group&quot; interview problem, and SUM() OVER (ORDER BY ... ROWS UNBOUNDED PRECEDING) for year-to-date running totals.</li></ul><h3>2. How I Approached the Weekly Tasks and Assessments</h3><p>The Week 6 take-home was divided into four technical stages:</p><ol><li><strong>Environment &amp; Database Architecture Setup:</strong> Rather than hardcoding database credentials into my notebook (a major security anti-pattern), I configured a root .env file with our read-only Azure SQL credentials and built the db/dataraflow_db.py module using SQLAlchemy, pymssql, and python-dotenv. This exposed a clean run_query(sql) helper function that executes T-SQL queries and returns results directly as a Pandas DataFrame.</li><li><strong>NumPy &amp; Pandas Exercises (Tasks Q1–Q11 &amp; Assignment Q1–Q12):</strong> I adopted a strict clean-code standard: every variable name had to be self-documenting, and every code comment had to explain <em>why</em> a decision was made rather than restating <em>what</em> the syntax did.</li><li><strong>Six-Airport Weather Exploratory Data Analysis (</strong><strong>week06 schema):</strong> Instead of pulling six separate DataFrames for <strong>Beijing, Brasília, Cape Town, Delhi, London, and Moscow</strong> (365 daily rows each for 2014) and cleaning them one by one in Python, I followed the golden rule from Section 13.2 of our SQL guide: <em>use SQL to get a tidy, right-sized table, then use Pandas for analysis and visuals</em>. I normalised the schema quirks inside a single UNION ALL query to produce a clean 2,190 × 9 DataFrame (6 cities × 365 days).</li><li><strong>The 10-Task SQL Assessment (</strong><strong>s01–</strong><strong>s10):</strong> Each SQL task was graded automatically against a SHA-256 fingerprint of the resulting DataFrame, requiring exact row counts, column ordering, rounding, and sorting. I approached each query by first verifying the expected output grain (e.g., 12 rows × 3 columns for monthly hottest days) before layering on CTEs and window functions.</li></ol><h3>3. Challenges I Faced (What Didn’t Work at First)</h3><p>No real-world data project goes smoothly on the first try, and Week 6 threw several practical curveballs my way:</p><h3>Challenge 1: The NameError and Azure SQL Firewall Block (Error 40615)</h3><p>When I first ran my schema inspection query in Antigravity IDE, Python threw NameError: name &#39;run_query&#39; is not defined. After fixing the cell execution order to ensure db/dataraflow_db.py was imported, I immediately hit a second wall:</p><blockquote><em>MSSQLDatabaseException: (40615, &quot;Cannot open server &#39;df-cohort2&#39; requested by the login. Client with IP address &#39;...&#39; is not allowed to access the server.&quot;)</em></blockquote><h3>Challenge 2: Invisible Leading Spaces &amp; Inconsistent Column Names</h3><p>While exploring the 2014 weather tables in week06, queries referencing [Events] or [Mean Humidity] failed with invalid column errors. Furthermore, stacking the six cities failed initially because London&#39;s date column was named GMT, whereas Beijing, Brasília, Cape Town, Delhi, and Moscow named theirs Date.</p><h3>Challenge 3: Silent SQL Bugs (Integer Division &amp; Aggregate Rows)</h3><p>In SQL Server, AVG(Churn) on a 0/1 integer column returned 0 instead of the actual churn rate, and AVG([Mean TemperatureC]) silently threw away decimal precision. Similarly, when calculating UK milk export totals from week09.comtrade_milk_uk_monthly_14, a naive SUM([Trade Value (US$)]) double-counted all trade values because the table included a Partner = &#39;World&#39; summary row alongside individual partner countries.</p><h3>4. How I Overcame Them</h3><ul><li><strong>Resolving the Firewall &amp; Path Discovery:</strong> I learned how the pathlib.Path.cwd() while loop walks up parent directories to locate db/dataraflow_db.py regardless of which subfolder the notebook runs from. For the 40615 Azure firewall error, instead of repeatedly retrying failed logins, I extracted my public IP address and shared it with the cohort admin to be whitelisted on the server firewall.</li><li><strong>Defeating Dirty CSV Headers with </strong><strong>INFORMATION_SCHEMA:</strong> Because the weather tables were originally ingested from CSV files that had a space after the comma in their header rows, column names actually started with a space ([ Events], [ Mean Humidity], [ CloudCover]). Running a diagnostic query (WHERE COLUMN_NAME LIKE &#39; %&#39;) revealed every hidden leading space immediately. I also used TRY_CONVERT(date, GMT) and TRY_CONVERT(date, [Date]) inside my UNION ALL CTE so all six cities aligned into a single standardised date column.</li><li><strong>Eliminating Silent Math &amp; Join Bugs:</strong> To beat the T-SQL integer-division trap (integer ÷ integer = integer), I multiplied integer columns by 1.0 inside aggregates (AVG(1.0 * Churn)) before scaling to percentages and rounding. And by checking SELECT DISTINCT Partner early, I knew to always include AND Partner &lt;&gt; &#39;World&#39; in my WHERE clause.</li></ul><h3>5. Key Insights &amp; Real-Data Gotchas: Always Sanity-Check Your Data!</h3><p>Two analytical discoveries from this week stood out as reminders that <strong>a good data scientist never blindly trusts a table just because a query ran without syntax errors</strong>:</p><ol><li><strong>The Delhi Weather Anomaly (</strong><strong>week06.delhi_del_2014):</strong> When I grouped the six cities by month to compare seasonal temperature curves, Northern Hemisphere cities (London, Beijing, Moscow) peaked in July–August, Tropical Brasília stayed warm year-round (~20°C–24°C), and Southern Hemisphere Cape Town inverted cleanly with its coolest months in June–August. However, delhi_del_2014 recorded <strong>sub-zero winter temperatures</strong>! In reality, Delhi rarely drops below 2°C. Comparing its monthly profile against Moscow revealed that the Delhi table had a high-latitude continental temperature curve—a classic data-ingestion or file-labeling bug that would ruin a downstream model if left unchecked.</li><li><strong>Entity Resolution in Multi-Source Joins (</strong><strong>WHO vs. </strong><strong>World Bank) &amp; Loan Bias (</strong><strong>week17):</strong> When joining week06.who_pop_tb_all with week08.wb_pop_2013 on Country, an anti-join (LEFT JOIN ... WHERE p.country IS NULL) revealed that several countries failed to match simply because different organizations spell names differently (e.g., <em>&quot;Bolivia (Plurinational State of)&quot;</em> vs. <em>&quot;Bolivia&quot;</em>). In production, joins across external sources should always use standardized <strong>ISO 3166-1 alpha-3 codes</strong> or a canonical mapping table rather than free-text names. Similarly, auditing week17.loan_approval_prediction by Property_Area × Credit_History exposed suspiciously uniform/missing segment distributions that would need to be flagged to a Head of Credit before training any predictive model.</li></ol><h3>Advice for Fellow Data Science Learners</h3><ol><li><strong>Comment the <em>Why</em>, Not the <em>What</em>:</strong> Anyone can read WHERE Partner &lt;&gt; &#39;World&#39;. Writing -- Exclude the &#39;World&#39; aggregate row to prevent double-counting exports helps your future self—and your teammates—understand the business logic.</li><li><strong>Always Run an Anti-Join After an Inner Join:</strong> Whenever you join two tables from different sources, check what got dropped using NOT EXISTS or LEFT JOIN ... WHERE key IS NULL. Unnoticed row loss is one of the most common sources of bad analytics.</li><li><strong>Peek, Don’t Dump:</strong> Use SELECT TOP 5 * in SQL and df.head() in Pandas before pulling or transforming millions of rows.</li></ol><p>Week 6 pushed my workflow much closer to how industry data teams actually operate. Onward to Week 7!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8ea387a0d958" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Learning Data Science to Asking Better Questions: My DataraFlow Weeks 4 and 5 Journey]]></title>
            <link>https://medium.com/@austinebri/from-learning-data-science-to-asking-better-questions-my-dataraflow-weeks-4-and-5-journey-53a0ff5bff44?source=rss-67d950ffa5e------2</link>
            <guid isPermaLink="false">https://medium.com/p/53a0ff5bff44</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[learning-journey]]></category>
            <category><![CDATA[research]]></category>
            <dc:creator><![CDATA[Austin Ebri]]></dc:creator>
            <pubDate>Sun, 11 Oct 2026 12:19:59 GMT</pubDate>
            <atom:updated>2026-10-11T12:19:59.132Z</atom:updated>
            <content:encoded><![CDATA[<p><em>How research, curiosity, and a real-world agricultural problem are shaping my journey into data science.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UY_uPfwu6IFkY4TNyNQIXg.png" /></figure><p>When I started the DataraFlow Data Science Internship, I knew I wanted to improve my ability to work with data. I wanted to move beyond collecting information, organising spreadsheets, and preparing reports to understanding how data could be used to solve problems, uncover patterns, and support better decisions.</p><p>However, Weeks 4 and 5 introduced me to an important part of that journey that I had not fully appreciated before: <strong>research</strong>.</p><p>These two weeks challenged me to slow down, think critically, ask better questions, and approach a problem with the mindset of a researcher rather than simply someone looking for a dataset to analyse.</p><p>My research topic became:</p><p><strong>“Predicting Cocoa Production Using Historical Climate and Agricultural Data: A Machine Learning Approach.”</strong></p><p>It is a topic that connects my growing interest in data science with my professional exposure to the agricultural sector. More importantly, it gave me an opportunity to explore how technology could contribute to solving a real-world problem.</p><h3>The Shift from Learning Tools to Defining a Problem</h3><p>One thing I am gradually learning about data science is that the work does not begin with writing Python code or building a machine learning model.</p><p>It begins with a problem.</p><p>Before Weeks 4 and 5, it was easy to think about data science in terms of the tools involved: Python, spreadsheets, visualisation, statistical analysis, and machine learning algorithms. These are important skills, but I began to understand that knowing the tools is not enough. I also need to know what problem I am trying to solve, why it matters, what information is required, and how I will determine whether my approach is useful.</p><p>The research activities pushed me to think through these questions.</p><p>Rather than rushing into a project because it sounded technically interesting, I needed to identify a problem that was relevant, researchable, and suitable for my current learning stage.</p><p>That was where the real work began.</p><h3>Choosing a Research Topic That Means Something to Me</h3><p>Agriculture is an area where data can make a meaningful difference. Weather conditions, rainfall patterns, temperature, farming practices, soil conditions, and other agricultural factors can influence crop performance.</p><p>Having gained professional exposure to the cocoa value chain, I became particularly interested in the relationship between climate conditions and cocoa production.</p><p>Cocoa is an important agricultural commodity in Nigeria, supporting farmers, businesses, and livelihoods. Yet agricultural production is influenced by factors that are not always easy to predict. Changes in rainfall, temperature, and seasonal conditions can affect crop development and ultimately influence how much farmers produce.</p><p>This led me to an important question:</p><p><em>Can historical climate and agricultural data be used to predict cocoa production?</em></p><p>I found this question interesting because it connects a practical agricultural challenge with a technical area I am working to understand: machine learning.</p><p>The idea is to investigate whether historical observations of climate conditions and agricultural production can reveal patterns that may help estimate future cocoa production.</p><p>I also made a deliberate decision not to use Unifacto Cocoa Ltd’s internal data for this research. I wanted to develop a study that could stand independently, using appropriately sourced data and a research design that could potentially be applied beyond one organisation.</p><p>That decision made the project more challenging in some respects, but it also encouraged me to think more carefully about publicly available data, data quality, and the feasibility of my proposed approach.</p><h3>The Challenges Behind the Topic</h3><p>Choosing a promising research topic was only the beginning. Defining a study that could realistically be carried out was a different challenge.</p><h3>1. Turning a broad interest into a focused research question</h3><p>At first, the relationship between climate and cocoa production seemed straightforward. Climate affects agriculture, and machine learning can identify patterns in data. It appeared natural to bring the two together.</p><p>But a broad idea is not yet a research project.</p><p>I needed to think about what I actually intended to predict. Was I looking at cocoa production at the national level, across particular cocoa-producing states, or within a more local agricultural area? Which climate variables would be relevant? What period of historical data would be appropriate?</p><p>These questions matter because the scope of a study determines the data required and the methods that can reasonably be applied.</p><p>I worked through the topic by breaking it down into its main components: the outcome I wanted to predict, the possible explanatory variables, the sources of historical information, and the machine learning approach I hoped to investigate.</p><p>This helped me move from a general interest in agricultural technology towards a more structured research idea.</p><p>My biggest lesson here was that a good research topic should not simply sound impressive. It should be clear enough to guide the work that follows.</p><h3>2. Understanding what the data would need to contain</h3><p>Another challenge was thinking beyond the availability of data to its suitability for the study.</p><p>It is easy to say that I want to use historical climate and agricultural data. It is more difficult to establish whether the available records contain the right variables, cover a sufficiently useful period, and correspond to the same geographical areas and time intervals.</p><p>For example, rainfall data and cocoa production figures may come from different sources. Before they can be combined meaningfully, their dates, locations, units, and definitions need to be examined.</p><p>Missing observations, inconsistent reporting, and differences in the way agricultural statistics are collected could also affect the quality of a model.</p><p>This made me appreciate that data preparation is not merely a technical step performed before modelling. It is part of the research itself.</p><p>A machine learning model cannot compensate for poorly understood data. Even a technically sophisticated model can produce misleading results if the underlying information is incomplete or unsuitable.</p><p>I therefore began to see data availability and data quality as questions to investigate early, rather than problems to postpone until the implementation stage.</p><h3>3. Connecting the research idea to machine learning</h3><p>Because my topic includes machine learning, I also needed to think about the analytical approach.</p><p>I am still developing my data science skills, so I did not want to select an algorithm simply because it is popular or appears advanced.</p><p>Instead, I needed to understand the nature of the prediction problem and how different methods might fit it.</p><p>Historical climate and agricultural data could potentially be organised as a supervised learning problem, with cocoa production as the target variable and selected climate and agricultural indicators as predictor variables.</p><p>Depending on the structure and volume of the available data, regression models could provide a useful starting point. More advanced approaches could then be considered if the data justified them.</p><p>I also began to recognise that model evaluation would be important. A model that fits historical observations well is not necessarily capable of predicting future production accurately.</p><p>For time-dependent data, the evaluation method must respect the chronological order of observations. Randomly mixing past and future records between training and testing sets could give an overly optimistic impression of performance.</p><p>This is an area I intend to approach carefully as the research progresses.</p><p>The challenge was not to know every answer immediately. It was to identify the questions I would need to answer before building a credible model.</p><h3>How I Worked Through the Challenges</h3><p>I did not overcome these challenges by suddenly becoming an expert in research or machine learning. I worked through them by breaking the process into smaller, more manageable decisions.</p><p>First, I concentrated on defining the problem rather than rushing towards implementation. This helped me understand what the proposed study was intended to achieve.</p><p>Second, I considered the practical requirements of the research. I began thinking about data sources, relevant variables, geographical scope, and the relationship between the information I hoped to obtain and the prediction I wanted to make.</p><p>Third, I approached the topic as a learning opportunity. Rather than expecting myself to have all the answers at the proposal stage, I treated unanswered questions as things to investigate.</p><p>Finally, I kept the intended application in view. I wanted the research to address an agricultural problem, not simply demonstrate that I could apply a machine learning algorithm.</p><p>This approach made the work feel more manageable. It also helped me understand that research is an iterative process: a question leads to investigation, investigation reveals gaps, and those gaps help refine the original question.</p><h3>What Weeks 4 and 5 Taught Me About Data Science</h3><p>Looking back, the most valuable lesson from these activities is that data science requires more than technical competence.</p><p>It requires curiosity, critical thinking, patience, and the ability to distinguish between what the data might show and what we would like it to show.</p><p>I have also come to appreciate several important principles.</p><p><strong>First, the problem should guide the technology.</strong> It is tempting to begin with an algorithm and search for a problem to solve with it. A stronger approach is to understand the problem first and select methods that suit the available data and the research objectives.</p><p><strong>Second, research requires discipline.</strong> A promising idea must be supported by relevant literature, a feasible methodology, appropriate data, and a clear plan for evaluating the findings.</p><p><strong>Third, data quality matters as much as model selection.</strong> The quality, consistency, and relevance of the information used in a study can determine how useful its findings will be.</p><p><strong>Finally, uncertainty is part of learning.</strong> Not knowing how to answer a research question immediately does not mean the question is wrong. It means there is work to do.</p><p>These lessons have changed how I think about my development as a data professional. I am learning to place greater emphasis on asking the right questions before looking for technical solutions.</p><h3>Why Cocoa Production Prediction Matters to Me</h3><p>My interest in this research goes beyond completing an internship activity.</p><p>Agriculture generates complex problems that require information from different sources. Climate observations, farm records, historical production figures, and agricultural statistics can each tell part of the story. Analysing them together may reveal patterns that are difficult to identify through observation alone.</p><p>If a reliable predictive approach can be developed, it could potentially support agricultural planning, production forecasting, resource allocation, and discussions about climate-related risks in cocoa-producing regions.</p><p>For farmers, better production estimates could contribute to more informed planning. For agricultural businesses, they could help improve expectations around supply. For policymakers and agricultural development organisations, they could provide additional evidence for planning and decision-making.</p><p>However, I recognise that these are potential applications, not results I have already achieved. Whether the proposed approach can produce reliable and useful predictions will depend on the quality of the data, the suitability of the methodology, and the results of the eventual evaluation.</p><p>That distinction is important to me. I want to learn how to use data to produce defensible insights, not make claims that the evidence cannot support.</p><h3>From Research Proposal to Practical Application</h3><p>With the research topic approved, my next task is to turn the idea into a well-defined study.</p><p>That means reviewing existing research, identifying appropriate data sources, refining the research questions and objectives, and establishing a methodology that I can realistically implement.</p><p>I will also need to examine the limitations of the available data, select suitable baseline models, and evaluate the results using appropriate performance measures.</p><p>I expect that this next stage will introduce new challenges. Finding suitable historical data may take time. Some variables may not be available at the geographical level I initially have in mind. The data may require considerable cleaning and preparation before any meaningful analysis can begin.</p><p>I am learning not to interpret these possibilities as reasons to abandon the project. Instead, they are practical considerations that should shape the study from the beginning.</p><p>The objective is not simply to finish a project. It is to understand the process well enough to explain the decisions made, the limitations encountered, and the conclusions supported by the evidence.</p><h3>A Step Forward in My Data Science Journey</h3><p>Weeks 4 and 5 of the DataraFlow Data Science Internship have reminded me that professional growth often involves changing how we approach familiar problems.</p><p>I came into the activities wanting to strengthen my data science skills. I am coming out of them with a better appreciation of the thinking that must happen before the technical work begins.</p><p>My research topic, <em>Predicting Cocoa Production Using Historical Climate and Agricultural Data: A Machine Learning Approach</em>, is still a work in progress. I have not yet demonstrated that a particular model can accurately predict cocoa production. That is part of the work ahead.</p><p>But I now have a clearer direction, a practical problem to investigate, and a better understanding of the questions that need to be answered.</p><p>For me, this is what learning should look like: not pretending to know everything, but becoming better equipped to investigate, test, question, and improve.</p><p>As I continue this journey, I hope to build the technical skills needed to turn historical data into useful insights, particularly in agriculture and sustainability, where better decisions can have real consequences for people and businesses.</p><p>The journey from data collection to meaningful prediction is not always straightforward. Sometimes, the most important progress happens before the first line of code is written.</p><p>And that is the lesson I am taking from Weeks 4 and 5.</p><p><strong>About the author</strong></p><p>I am a Data and ICT professional developing my skills in data science, analytics, and digital transformation. My interests lie at the intersection of technology, agriculture, and sustainability, particularly in how data-driven approaches can improve operational decisions and address practical challenges across agricultural value chains.</p><p>I am documenting my learning journey through the DataraFlow Data Science Internship, one challenge and one lesson at a time.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=53a0ff5bff44" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[My DataraFlow Week Three Journey: Learning to Work with Files, Modules, SQL and Error Handling in…]]></title>
            <link>https://medium.com/@austinebri/my-dataraflow-week-three-journey-learning-to-work-with-files-modules-sql-and-error-handling-in-b5d51277275b?source=rss-67d950ffa5e------2</link>
            <guid isPermaLink="false">https://medium.com/p/b5d51277275b</guid>
            <category><![CDATA[datara-flow]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[learning-in-public]]></category>
            <category><![CDATA[sql]]></category>
            <dc:creator><![CDATA[Austin Ebri]]></dc:creator>
            <pubDate>Fri, 25 Sep 2026 14:46:14 GMT</pubDate>
            <atom:updated>2026-09-25T14:46:14.844Z</atom:updated>
            <content:encoded><![CDATA[<h3>My DataraFlow Week Three Journey: Learning to Work with Files, Modules, SQL and Error Handling in Python</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fbnxtuynrPbnCS5NiEOtzQ.png" /></figure><p>Week Three of my DataraFlow learning journey was another important step in developing my Python and data skills. Compared with the previous weeks, this week exposed me to more practical concepts that showed me how Python interacts with files, external libraries, structured data, databases, dates and user input.</p><p>The major topics I covered during the week were <strong>reading and writing files in Python, Python modules, mathematics, JSON, dates, an introduction to SQL, PIP, Try-Except, user input and virtual environments</strong>.</p><p>What made this week particularly interesting was that I started seeing Python as more than a programming language for solving simple exercises. I began to understand how it can be used to manage real data, interact with files and databases, handle errors, and build applications that accept information from users.</p><h3>Reading and Writing Files in Python</h3><p>One of the first things I learnt was how Python works with files.</p><p>Previously, most of the data I used in my programmes was written directly inside the code. Learning file handling showed me that Python can read information stored outside a programme and can also create or update files.</p><p>I practised opening files using open(), reading their contents and writing information into text files.</p><p>I also learnt about different file modes such as:</p><ul><li>r for reading</li><li>w for writing</li><li>a for appending</li><li>x for creating a new file</li></ul><p>Using the with open() approach also helped me understand a safer and cleaner way of handling files because Python automatically closes the file after the operation is completed.</p><p>At first, I had to pay close attention to file paths and file modes. Using the wrong path or mode could easily result in an error or even overwrite existing information. With repeated practice, however, file handling became much clearer.</p><h3>Understanding Python Modules</h3><p>Another important topic was <strong>Python modules</strong>.</p><p>I learnt that a module is simply a Python file containing functions, variables or other pieces of code that can be reused in another programme.</p><p>Instead of writing everything from scratch, Python allows me to import existing functionality.</p><p>For example:</p><pre>import math</pre><p>or:</p><pre>from math import sqrt</pre><p>This introduced me to the idea of reusable and organised code.</p><p>I also began to understand why large software projects are divided into different modules rather than putting thousands of lines of code into one file.</p><h3>Working with Mathematics in Python</h3><p>The math module introduced me to several mathematical operations that Python already provides.</p><p>I worked with functions and constants such as:</p><pre>math.sqrt()<br>math.ceil()<br>math.floor()<br>math.pow()<br>math.pi</pre><p>This helped reinforce an important lesson for me: programmers do not always need to create every solution themselves. Python already provides powerful libraries that can simplify many tasks.</p><p>Learning how to locate and use these built-in functions is just as important as knowing how to write code manually.</p><h3>Learning JSON</h3><p>JSON was another interesting part of Week Three.</p><p>I learnt that <strong>JSON — JavaScript Object Notation — is one of the most common formats used for storing and exchanging structured information</strong>.</p><p>What made JSON easier for me to understand was its similarity to Python dictionaries.</p><p>For example, data containing a person’s name, age and occupation can easily be represented using key-value pairs.</p><p>Using Python’s json module, I learnt how information can be converted between Python objects and JSON data.</p><p>Some of the functions I encountered included:</p><pre>json.dumps()<br>json.loads()<br>json.dump()<br>json.load()</pre><p>At first, remembering the difference between dump, dumps, load and loads was slightly confusing. I overcame this by practising each operation separately and observing whether I was working with a Python object, a JSON string or a JSON file.</p><h3>Working with Dates and Time</h3><p>I also explored Python’s datetime module.</p><p>Dates are something we encounter constantly in real-world data, so learning how Python handles them was particularly useful.</p><p>I practised getting the current date, extracting the year, month and day, formatting dates and calculating differences between dates.</p><p>For example:</p><pre>from datetime import date</pre><pre>today = date.today()<br>print(today)</pre><p>I realised that date handling will be very useful in areas such as reports, transaction records, data analysis, scheduling and tracking activities over time.</p><h3>My Introduction to SQL</h3><p>One of the biggest additions to my learning journey this week was my introduction to <strong>SQL — Structured Query Language</strong>.</p><p>This was particularly interesting because SQL connects directly with the world of data management.</p><p>I learnt that databases store information in organised structures called tables and that SQL allows us to interact with those tables.</p><p>I was introduced to concepts such as:</p><pre>SELECT<br>GROUP BY<br>WHERE</pre><p>Learning SQL helped me understand that data analysis is not only about Python. In many real organisations, large amounts of data are stored in relational databases, and SQL is one of the main tools used to retrieve and manage that information.</p><p>Although I am still at the introductory stage, I can already see how combining Python and SQL will become very valuable as I continue developing my skills in data analytics and data management.</p><h3>Learning About PIP</h3><p>I also learnt about <strong>PIP</strong>, Python’s package management tool.</p><p>Before this lesson, I had used some Python libraries without fully understanding how external packages were installed.</p><p>Learning commands such as:</p><pre>pip install package_name</pre><p>helped me understand how Python’s capabilities can be extended through third-party packages.</p><p>This made me realise how large the Python ecosystem really is. There are libraries available for data analysis, web development, machine learning, automation, visualisation and many other areas.</p><h3>Handling Errors with Try-Except</h3><p>One of my favourite lessons from Week Three was <strong>exception handling using </strong><strong>try and </strong><strong>except</strong>.</p><p>When learning programming, errors are unavoidable.</p><p>Earlier in my journey, seeing an error message could sometimes make me think something had gone seriously wrong. I am gradually learning that errors are actually part of programming.</p><p>With try and except, I learnt how to anticipate certain errors and allow a programme to respond properly instead of crashing.</p><p>For example:</p><pre>try:<br>    number = int(input(&quot;Enter a number: &quot;))<br>except ValueError:<br>    print(&quot;Please enter a valid number.&quot;)</pre><p>This helped me understand how developers make applications more reliable and user-friendly.</p><p>Instead of assuming that users will always provide the correct information, good programmes should be prepared for unexpected input.</p><h3>Working with User Input</h3><p>I also practised collecting information from users using the input() function.</p><p>This may look simple, but it introduced an important concept: <strong>interaction between the programme and the user</strong>.</p><p>I learnt that values collected using input() are initially treated as strings, meaning that they sometimes need to be converted using functions such as:</p><pre>int()<br>float()</pre><p>I encountered several errors when trying to perform mathematical operations on values that had not been converted correctly.</p><p>Through practice, I became more careful about checking the type of data my programme was receiving.</p><h3>Understanding Virtual Environments</h3><p>Another concept that initially felt slightly technical was <strong>virtual environments</strong>.</p><p>I learnt that a virtual environment creates an isolated Python environment for a particular project.</p><p>This allows different projects to use different versions of packages without creating conflicts.</p><p>For example, a virtual environment can be created using:</p><pre>python -m venv myenv</pre><p>At first, I wondered why this was necessary when packages could simply be installed globally.</p><p>After learning more about project dependencies, I understood the importance.</p><p>If Project A requires one version of a library while Project B requires another version, virtual environments allow both projects to work independently.</p><p>This was one of those concepts that seemed unnecessary at first but became much more meaningful once I understood the problem it solves.</p><h3>Challenges I Faced During Week Three</h3><p>One major challenge this week was the number of new concepts introduced within a short period.</p><p>Moving from file handling to JSON, dates, SQL, package management and virtual environments required me to constantly adjust my thinking.</p><p>I also encountered several coding errors.</p><p>Sometimes they came from using the wrong file path. Other times, they resulted from incorrect data types, syntax mistakes, missing modules or invalid user input.</p><p>JSON functions such as load, loads, dump and dumps also required some additional practice before the differences became clear.</p><p>SQL introduced a different syntax from Python, which meant I had to begin learning another way of expressing instructions to a computer.</p><p>Virtual environments also required me to understand commands outside the Python code itself.</p><h3>How I Overcame the Challenges</h3><p>The main strategy that helped me was <strong>practice</strong>.</p><p>Instead of only reading through examples, I tried writing the programmes myself.</p><p>Whenever I encountered an error, I read the error message carefully and attempted to understand what Python was telling me.</p><p>I also broke larger problems into smaller steps.</p><p>For example, instead of trying to build a complete file-processing programme immediately, I first tested whether I could create a file. Then I tested writing into it. Afterwards, I practised reading the same file.</p><p>I applied the same approach to JSON, dates and exception handling.</p><p>Another thing that helped me was repeating exercises until I could understand the logic rather than simply memorising the syntax.</p><p>I am gradually becoming more comfortable with the fact that getting errors does not mean I am failing. Debugging those errors is actually one of the ways I learn the most.</p><h3>My Biggest Lesson from Week Three</h3><p>One of my biggest lessons from Week Three is that <strong>Python becomes much more powerful when it interacts with other technologies and external data</strong>.</p><p>Files allow Python to store and retrieve information.</p><p>JSON allows programmes to exchange structured data.</p><p>Modules allow us to reuse existing functionality.</p><p>PIP gives us access to thousands of external Python packages.</p><p>Virtual environments help us manage projects properly.</p><p>SQL allows us to communicate with databases.</p><p>Try-Except helps us build programmes that can respond intelligently when something goes wrong.</p><p>User input makes programmes interactive.</p><p>When these concepts are combined, Python begins to move from simple classroom exercises towards real application development and data solutions.</p><h3>Moving Forward</h3><p>Week Three of DataraFlow has strengthened my interest in Python, data management and analytics.</p><p>I still have a lot to learn, especially with SQL, databases and more advanced Python libraries, but I am becoming more confident with each week.</p><p>The journey has also taught me that learning programming is not about memorising every command.</p><p>It is about understanding concepts, practising regularly, learning how to solve problems and knowing how to find the right tools when they are needed.</p><p>Every error I solve adds something to my understanding.</p><p>Every programme I complete improves my confidence.</p><p>Every new concept gives me another tool that I can apply to real problems.</p><p>Week Three was another important milestone in my DataraFlow journey, and I am looking forward to seeing how the skills I am developing will connect with data analytics, automation and real-world data projects in the weeks ahead.</p><p><strong>The journey continues.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b5d51277275b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[My Week Two Journey at DataraFlow: Understanding OOP, Inheritance, Iterators, Polymorphism and…]]></title>
            <link>https://medium.com/@austinebri/my-week-two-journey-at-dataraflow-understanding-oop-inheritance-iterators-polymorphism-and-4d1cb05a1384?source=rss-67d950ffa5e------2</link>
            <guid isPermaLink="false">https://medium.com/p/4d1cb05a1384</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[datara-flow]]></category>
            <dc:creator><![CDATA[Austin Ebri]]></dc:creator>
            <pubDate>Sat, 19 Sep 2026 14:52:17 GMT</pubDate>
            <atom:updated>2026-09-19T14:52:17.675Z</atom:updated>
            <content:encoded><![CDATA[<h3>My Week Two Journey at DataraFlow: Understanding OOP, Inheritance, Iterators, Polymorphism and Scope in Python</h3><p>My second week at <strong>DataraFlow</strong> took me deeper into Python programming. Unlike Week One, which focused mainly on the fundamentals of Python and basic problem-solving, Week Two introduced me to some of the core concepts of <strong>Object-Oriented Programming (OOP)</strong>.</p><p>The major topics I worked on were <strong>OOP, inheritance, iterators, polymorphism and scope</strong>.</p><p>At first, these concepts felt more abstract than the basic Python exercises I had worked on previously. I was no longer just writing short programs with variables, loops and conditions. I now had to understand how different parts of a program could be organised into classes, how objects could interact with one another, and how code could be structured in a more reusable and scalable way.</p><p>It was challenging, but it was also one of the most interesting parts of my Python learning journey so far.</p><h3>Moving from Simple Programs to Object-Oriented Programming</h3><p>One of the biggest changes for me in Week Two was learning to think in terms of <strong>objects and classes</strong>.</p><p>Previously, I mostly wrote programs as a sequence of instructions. With OOP, I had to start thinking about how real-world entities could be represented in Python.</p><p>I learnt that a <strong>class</strong> can serve as a blueprint, while an <strong>object</strong> is an instance created from that blueprint.</p><p>For example, instead of storing information about a book in separate variables, I could create a Book class with attributes such as:</p><ul><li>Title</li><li>Author</li><li>Year of publication</li></ul><p>From that class, I could then create different book objects.</p><p>This helped me understand how OOP makes programs more organised, especially when working with larger systems where many related objects need to be managed.</p><h3>Working with Inheritance</h3><p>Another major concept I learnt was <strong>inheritance</strong>.</p><p>Inheritance allows one class to acquire attributes and methods from another class. This was initially confusing because I had to understand the relationship between a parent class and its child classes.</p><p>A practical exercise that helped me understand this was building a simple <strong>Library Management System</strong>.</p><p>I created a base Book class and then worked with subclasses such as:</p><ul><li>EBook</li><li>PrintedBook</li></ul><p>The subclasses inherited the general properties of a book while also having their own unique properties.</p><p>For example, an eBook could have attributes such as file size and format, while a printed book could have weight and dimensions.</p><p>I also worked with a base Member class and subclasses such as StudentMember and TeacherMember.</p><p>This exercise made the concept of inheritance much clearer to me because I could see how common features could be defined once in a parent class instead of repeating the same code several times.</p><p>It taught me an important principle of programming: <strong>reuse what is common and specialise only what is different</strong>.</p><h3>Understanding Polymorphism</h3><p>Polymorphism was another concept that required some time for me to fully understand.</p><p>The basic idea is that different objects can respond differently to the same method.</p><p>For example, I worked with a method such as get_description() in the Book class. The same method could behave differently depending on whether the object was an EBook or a PrintedBook.</p><p>Similarly, a borrow_book() method could behave differently for a student and a teacher because they may have different borrowing limits.</p><p>At first, I wondered why I would use the same method name if the behaviour was different.</p><p>As I practised, I began to understand the advantage.</p><p>Instead of writing completely separate code for every type of object, I could use a common interface while allowing each class to define its own behaviour.</p><p>This gave me a better appreciation of how larger software systems are designed.</p><h3>Learning About Iterators</h3><p>I also worked with <strong>iterators</strong>, which introduced another way of accessing data in Python.</p><p>Before this, I was already familiar with using loops to go through lists. But Week Two helped me understand what is happening behind the scenes when Python moves from one item to another.</p><p>I learnt about the iterator protocol and how methods such as __iter__() and __next__() can be used to control iteration.</p><p>In the Library Management System exercise, I implemented iteration so that members or books in the library could be accessed one after another.</p><p>This concept was initially difficult because the special methods looked unfamiliar to me.</p><p>However, when I related them to a normal for loop, the idea started making more sense.</p><p>I realised that many Python features that appear simple on the surface are supported by deeper mechanisms underneath.</p><h3>Understanding Scope</h3><p>Another important topic during the week was <strong>scope</strong>.</p><p>Scope helped me understand where variables can be accessed within a program.</p><p>I learnt about concepts such as:</p><ul><li>Local scope</li><li>Global scope</li><li>Class-level variables</li><li>Instance variables</li></ul><p>This was important because I had previously used variables without always thinking deeply about where they existed or how long they remained accessible.</p><p>I encountered situations where a variable worked perfectly inside a function but could not be accessed outside that function. At first, this could be confusing.</p><p>After studying scope more carefully, I began to understand that Python deliberately controls where variables are available.</p><p>This helps prevent different parts of a program from interfering with one another.</p><h3>My Main Challenges During Week Two</h3><p>Week Two was definitely more challenging than Week One.</p><p>One of my main challenges was understanding the relationship between <strong>classes, objects, methods and attributes</strong>.</p><p>The syntax itself was not always the difficult part. The bigger challenge was understanding how everything connected.</p><p>I also found inheritance and polymorphism confusing at first because both concepts involve relationships between classes.</p><p>There were moments when I had to ask myself:</p><p>Why should this method be placed in the parent class?</p><p>Should this attribute belong to the parent or the subclass?</p><p>Should this method be overridden?</p><p>What exactly should be inherited?</p><p>Another challenge was understanding iterators because methods such as __iter__() and __next__() were very different from the Python syntax I had become used to during Week One.</p><h3>How I Overcame the Challenges</h3><p>One approach that helped me a lot was breaking each concept into smaller pieces.</p><p>Instead of trying to understand the entire Library Management System at once, I focused on individual parts.</p><p>I first tried to understand the Book class.</p><p>Then I looked at how EBook inherited from it.</p><p>After that, I examined how methods could be overridden.</p><p>Gradually, the relationships became clearer.</p><p>I also found it useful to create small examples before applying the concepts to a bigger project.</p><p>For instance, before fully understanding inheritance in the library system, I experimented with simple parent and child classes. Once the basic relationship made sense, I returned to the larger exercise.</p><p>Another important part of my learning process was <strong>running the code repeatedly and observing what changed</strong>.</p><p>Whenever something did not work as expected, I checked:</p><ul><li>The class definition</li><li>The indentation</li><li>The constructor</li><li>The parameters being passed</li><li>Whether I was using self correctly</li><li>Whether the method belonged to the correct class</li><li>Whether an attribute was local, global or instance-based</li></ul><p>These small debugging steps helped me understand the structure of OOP better.</p><h3>The Importance of Asking “Why?”</h3><p>One thing I noticed about my learning during Week Two was that I kept asking myself why a concept was necessary.</p><p>Why use inheritance instead of creating separate classes?</p><p>Why override a method?</p><p>Why use an iterator when a loop already exists?</p><p>Why should a variable be local instead of global?</p><p>Finding answers to these questions helped me move beyond simply memorising syntax.</p><p>I started to understand the purpose behind the concepts.</p><p>For me, this has been one of the most important parts of learning Python.</p><p>I do not just want to know what code to write. I want to understand <strong>why the code is structured that way</strong>.</p><h3>Connecting the Lessons to Real-World Systems</h3><p>As someone who works with <strong>data, ICT systems, digital processes and traceability</strong>, the ideas behind OOP started to feel very practical.</p><p>Many real-world systems involve different entities that share common characteristics but also have their own unique behaviours.</p><p>For example, in a traceability system, there could be farmers, field officers, processing staff, warehouse staff and administrators. They may all be users of the same system, but each role may have different permissions and responsibilities.</p><p>OOP provides a structured way of representing such relationships.</p><p>This made Week Two particularly interesting to me because I could begin to see how the concepts I was learning could eventually be applied to larger software and data systems.</p><h3>What I Took Away from Week Two</h3><p>By the end of Week Two, I had gained a much better understanding of how Python programs can be structured beyond simple scripts.</p><p>I learnt how classes and objects can represent real-world entities.</p><p>I learnt how inheritance can reduce code duplication.</p><p>I learnt how polymorphism allows related objects to behave differently using a common interface.</p><p>I learnt how iterators control the way objects are accessed sequentially.</p><p>And I learnt how scope determines where variables are available within a program.</p><p>I still have more to learn, and I know these concepts will become clearer as I continue to practise.</p><p>But Week Two gave me an important foundation.</p><p>The biggest lesson for me was that programming is not just about making code work.</p><p>It is also about <strong>organising code in a way that is logical, reusable, maintainable and scalable</strong>.</p><p>My journey with DataraFlow continues, and with each week, I am beginning to understand not only how to write Python, but also how to think more like a programmer.</p><p><strong>Week Two completed. More learning ahead.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4d1cb05a1384" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[My Week One Journey with DataraFlow: Learning Python One Challenge at a Time]]></title>
            <link>https://medium.com/@austinebri/my-week-one-journey-with-dataraflow-learning-python-one-challenge-at-a-time-8d4c2c8ff608?source=rss-67d950ffa5e------2</link>
            <guid isPermaLink="false">https://medium.com/p/8d4c2c8ff608</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[datara-flow]]></category>
            <category><![CDATA[python-programming]]></category>
            <dc:creator><![CDATA[Austin Ebri]]></dc:creator>
            <pubDate>Thu, 10 Sep 2026 20:36:59 GMT</pubDate>
            <atom:updated>2026-09-10T20:36:59.715Z</atom:updated>
            <content:encoded><![CDATA[<p>Starting my journey with <strong>DataraFlow</strong> has been both exciting and challenging. As someone with a background in data, ICT and technology, I have always been interested in developing stronger programming and data science skills. Week One gave me the opportunity to move beyond simply being interested in Python to actually writing programs, making mistakes, debugging them and gradually understanding how programming works.</p><p>One of the biggest lessons I learnt during my first week was that <strong>learning Python is not just about understanding syntax — it is about learning how to think logically</strong>.</p><h3>Getting Started with Python</h3><p>During Week One, I worked through several Python exercises designed to introduce fundamental programming concepts.</p><p>Some of the activities included writing programs to:</p><ul><li>Calculate the factorial of a number</li><li>Reverse a string without using Python slicing</li><li>Generate Fibonacci numbers</li><li>Accept marks for five subjects, calculate the average and assign a grade</li><li>Build a basic calculator for addition, subtraction, multiplication and division</li><li>Create a simple shopping cart using a Python dictionary</li><li>Develop a number-guessing game using randomly generated numbers</li><li>Accept student names, save them to a text file and read them back</li></ul><p>Although these exercises looked simple at first, each one introduced a different way of thinking.</p><p>For example, generating Fibonacci numbers helped me understand how values can be updated repeatedly inside a loop. Creating the shopping cart introduced me to dictionaries and how data can be stored using <strong>key-value pairs</strong>. The guessing game helped me understand conditional statements, loops and Python’s random module.</p><p>The file-handling exercise was also particularly interesting because it demonstrated how a Python program can interact with information outside the program itself.</p><h3>My Biggest Challenge: Understanding the Logic</h3><p>My main challenge during the week was not necessarily remembering Python commands. The bigger challenge was understanding <strong>why a particular approach worked</strong>.</p><p>There were situations where I could see a working solution but still wanted to know whether there was another way of solving the same problem.</p><p>This became an important part of my learning process.</p><p>Instead of simply accepting one solution, I started exploring alternative approaches. I discovered that programming problems can often be solved in several ways. Some solutions may use loops, others may use built-in functions, while another programmer might structure the same problem differently.</p><p>This helped me understand that learning programming should not be about memorising code.</p><p>It should be about understanding the logic behind the code.</p><h3>Learning to Break Problems into Smaller Steps</h3><p>Another challenge I encountered was translating a written question into programming instructions.</p><p>A question such as:</p><p><em>“Create a program where the computer randomly selects a number between 1 and 20, and the user must guess it.”</em></p><p>sounds straightforward.</p><p>But when writing the program, I had to think about several smaller questions:</p><p>How do I generate the random number?</p><p>How do I collect the user’s input?</p><p>How do I compare the user’s guess with the computer’s number?</p><p>How do I tell the user whether the guess is too high or too low?</p><p>How do I keep asking until the correct answer is entered?</p><p>Breaking the problem into smaller steps made it much easier to solve.</p><p>This is probably one of the most valuable lessons I gained from Week One.</p><p><strong>Before writing code, understand the problem. Then divide the problem into smaller logical operations.</strong></p><h3>Using Jupyter Notebook as a Learning Environment</h3><p>I also worked with <strong>Jupyter Notebook</strong>, organising my exercises into Markdown and code cells.</p><p>I found this particularly useful because it allowed me to combine explanations with executable Python code.</p><p>Instead of having only a collection of scripts, I could write a short explanation of a concept and place the corresponding Python code directly underneath it.</p><p>This approach feels especially relevant to my wider journey into <strong>data science and machine learning</strong>, where documenting the reasoning behind an analysis can be just as important as writing the code itself.</p><h3>How I Worked Through the Challenges</h3><p>Whenever I encountered difficulty, I tried not to stop at simply getting the correct answer.</p><p>I asked questions.</p><p>I explored different versions of the same program.</p><p>I compared alternative solutions.</p><p>I ran the programs and observed their outputs.</p><p>Most importantly, I tried to understand what each part of the code was doing.</p><p>Sometimes a concept became clearer after seeing a simpler version of the solution. At other times, rewriting the same program using another approach helped me understand it better.</p><p>Gradually, concepts such as variables, loops, conditional statements, lists, dictionaries, functions, user input and file handling started connecting together.</p><p>The experience reinforced something important for me:</p><p><strong>You do not need to understand everything immediately. What matters is continuing to experiment until the concept begins to make sense.</strong></p><h3>From ICT and Data Management to Programming</h3><p>My interest in Python is closely connected to the work I already do around <strong>data management, ICT, digital systems, analytics and traceability</strong>.</p><p>A lot of my day-to-day work involves data collection, Excel, databases, reporting, digital systems and solving technology-related problems.</p><p>Learning Python is more than completing a course.</p><p>I see it as another tool that can eventually help me automate repetitive tasks, clean and analyse larger datasets, build smarter data workflows and develop more advanced solutions.</p><p>As I progress further into data science and machine learning, I expect Python to become increasingly important.</p><p>Week One was therefore a small but necessary foundation.</p><h3>What Week One Taught Me</h3><p>By the end of the week, I realised that progress in programming does not always mean writing complicated applications.</p><p>Sometimes progress is finally understanding why a for loop behaves differently from a while loop.</p><p>Sometimes it is fixing a small error.</p><p>Sometimes it is writing a five-line program completely on your own.</p><p>And sometimes it is looking at code that seemed confusing yesterday and suddenly understanding what it does today.</p><p>Those small improvements add up.</p><p>My Week One experience with DataraFlow has strengthened my determination to continue learning Python and developing my data science skills.</p><p>There is still a lot ahead — more Python concepts, data analysis, statistics, machine learning and probably many errors along the way.</p><p>But that is part of the process.</p><p>For now, my biggest takeaway from Week One is simple:</p><p><strong>Do not just learn how to write code. Learn how to think through problems, ask questions, experiment with different solutions and keep practising.</strong></p><p>Week One completed.</p><p>The journey continues!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8d4c2c8ff608" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>