<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ibukun Irinyenikan on Medium]]></title>
        <description><![CDATA[Stories by Ibukun Irinyenikan on Medium]]></description>
        <link>https://medium.com/@ibukunirinyenikan?source=rss-7ab15db4bcd2------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*4upF2_-Uax7CEtNT.jpg</url>
            <title>Stories by Ibukun Irinyenikan on Medium</title>
            <link>https://medium.com/@ibukunirinyenikan?source=rss-7ab15db4bcd2------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 08:43:38 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ibukunirinyenikan/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Experimental Tracking with MLflow]]></title>
            <link>https://medium.com/@ibukunirinyenikan/experimental-tracking-with-mlflow-11d791ac3a7c?source=rss-7ab15db4bcd2------2</link>
            <guid isPermaLink="false">https://medium.com/p/11d791ac3a7c</guid>
            <category><![CDATA[experiment]]></category>
            <category><![CDATA[tracking]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[mlflow]]></category>
            <dc:creator><![CDATA[Ibukun Irinyenikan]]></dc:creator>
            <pubDate>Tue, 01 Sep 2026 19:21:00 GMT</pubDate>
            <atom:updated>2026-09-01T19:21:00.837Z</atom:updated>
            <content:encoded><![CDATA[<h4>A practical example showing how MLflow can help you track your training experiment</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Ii-dTVefjEyPSmcA.jpg" /><figcaption>MLflow</figcaption></figure><p>In machine learning (ML), no data scientist knows the exact algorithm or hyperparameter that best solves a problem before training. The best-performing model is discovered through a trial-and-error process — experimenting with different algorithms and hyperparameter settings.</p><p>Earlier in my ML career, keeping track of different algorithms and their hyperparameters was time-consuming and tedious. Sometimes, I manually wrote them out in another code cell in Jupyter Notebook. Each time I needed to change a setting, I would copy and paste the code into another code cell before changing the settings so I wouldn’t forget the hyperparameters and results of previously trained model. You can imagine how messy my notebook would be.</p><p>This is the reality for many data scientists when we first learn to train models in Jupyter Notebook.</p><p>I can now experiment with various algorithms and hyperparameters without worrying which produce certain result. Why?</p><p>MLflow is the answer.</p><p>In this article, we will learn about the concept of experimental tracking and how you can use MLflow to track model training outputs with a practical code example.</p><p><strong>What is Experimental Tracking?</strong></p><p>Experimental tracking is the process of tracking and recording every information about model training, along with its outputs, to help reproduce the model. Machine learning experiments can be tracked using MLflow and Weights &amp; Biases. MLflow is widely adopted because it is open-source and easy to learn.</p><p>In this article, we will learn how to use MLflow in our ML projects to keep a record of training results with a practical code example. Before that, let’s learn some MLflow concepts that will help us follow through the tutorial.</p><p>MLflow is an open-source experimental tracking tool that allows us to track and record every training trial. It also provides us with an interactive web interface where we can view and compare the results from various trials.</p><p><strong>Important features of MLflow</strong></p><p><strong>Run:</strong> A run is a single execution of ML code. A script may contain training code alongside evaluation code. Each time you execute that script, that’s a run. If you choose a different learning rate in the script and run again, that’s another run. Each of the trials you perform is called a run.</p><p><strong>Experiment:</strong> An experiment groups related runs together in one place. You can imagine an experiment as a folder that keeps related files(runs). An experiment allows you to track and compare results from all the runs. In an ML project, we have several training trials; all of these trials (runs) are stored and organised in an experiment.</p><p><strong>Artifact:</strong> An artifact is a file generated in a training run. An artifact could be a model file, a visualisation plot, or a configuration file.</p><p><strong>Metadata:</strong> A metadata in MLflow is an information that describes a model, its run and artifacts. A metadata includes the following:</p><ul><li>Hyperparameters: learning rate, tree depth, etc.</li><li>Evaluation metrics: Root Mean Square, precision, recall, etc.</li><li>Others include run names, artifact names, GitHub commit.</li></ul><p>Unlike metadata that can be stored in a database like SQLite and PostgreSQL. Artifacts are files; they are unstructured data and cannot be fit into rows and columns of a relational database. Artifacts can be stored in a local folder or stored in AWS S3 for production.</p><p><strong>MLflow Model Registry:</strong> The registry is where the best-performing model is stored. After you compare all the model results from the different runs, you move the best-performing model to the model registry for deployment.</p><p><strong>The UI: </strong>The UI is the interactive web User Interface (UI) that allows you to view and compare training runs. Later in the tutorial, I will explain what tracking URI means. Now, let’s learn how to implement MLflow to track our experiments.</p><p><strong>Practical-Based Tutorial to Help You Track Your Experiment with MLflow</strong></p><p>In this section, we will build an ARIMA model with Tesla stock data to demonstrate how to track and record experiments in MLflow.</p><p>Although this tutorial uses a time-series data; it is focused on experimental tracking using MLflow. I will write an article in the future on how to build an ARIMA model with a time-series step by step.</p><p>This tutorial assumes you have a Python interpreter running on your PC. If you don’t, please follow this link to install it: <a href="https://www.python.org/downloads/">Download Python</a>.</p><p>The first thing to do in a machine learning project is to create a virtual environment. Creating a virtual environment creates a separate directory (a folder) in our working directory where all our Python packages for this project will be installed. The virtual environment prevents packages installed in other projects or system-wide packages from conflicting with the packages installed in our project.</p><p><strong>How to create and activate a virtual environment on Windows OS.</strong></p><p>Open the terminal and run the code below. It will create a virtual environment as a directory named venv in your working directory.</p><pre>python3 -m venv venv</pre><p>Run the next code to activate the virtual environment.</p><pre>venv\Scripts\activate</pre><p><strong>How to create and activate a virtual environment for macOS/Linux</strong></p><p>First, run the code below to create it:</p><pre>python3 -m venv venv</pre><p>To activate, run:</p><pre>source venv/bin/activate</pre><p>The next thing is to install all the necessary libraries.</p><pre># Install necessary libraries<br># To run on Jupyter Notebook, add an ! like this: !pip install...<br>pip install -r yfinance pandas numpy statsmodels mlflow matplotlib ipykernel</pre><p>Copy and paste the code above in the terminal and run.</p><p>After activating the virtual environment and installing the necessary libraries, open a Jupyter Notebook, then select the venv python environment to give it access to the installed packages.</p><p>The first thing we will do on Jupyter Notebook is to import all the packages we will be using in this tutorial:</p><pre># import all necessary libraries<br><br>import pandas as pd<br>import mlflow<br>import yfinance as yf<br>import matplotlib.pyplot as plt<br>from statsmodels.tsa.stattools import adfuller<br>from statsmodels.tsa.arima.model import ARIMA<br>import os<br>from sklearn.metrics import root_mean_squared_error, mean_absolute_error</pre><p>Let’s download a 20-year historical Tesla stock data using a FREE API provided through the Yahoo Finance package. And you don’t need an API Key for this.</p><pre># Download 20 years of historical Tesla (TSLA) stock data from Yahoo Finance<br>df = yf.download(tickers=&quot;TSLA&quot;, period=&quot;20Y&quot;, progress=False)<br>df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/475/1*2ijO98DdL4qylH6-8Yd_tg.png" /></figure><p>After loading the data, in the following code, we perform a light preprocessing:</p><ul><li>Remove the second column levels: “TSLA” and “Ticker”,</li><li>Remove the column name: “Price”</li><li>Finally, convert the column names to lowercase.</li></ul><pre># Remove the extra column level created by yfinance<br>df = df.droplevel(axis=1, level=1)<br><br># Remove the column index names<br>df.columns.name = None<br><br># Convert the column names to lowercase<br>df.columns = df.columns.str.lower()<br><br>print(f&quot;The observations collected spans between {str(df.index.min().date())} to {str(df.index.max().date())}.&quot;)<br>df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/528/1*DS1ZRug2FyxIG1bXjB4kLQ.png" /><figcaption>Image by Author</figcaption></figure><p>Instead of forecasting the daily price close, we will forecast the weekly average return because it is less noisy and also stationary. We will understand the concept of stationarity next.</p><pre># Average weekly return <br>weekly_avg = df.close.resample(&quot;W&quot;).mean()<br>returns = weekly_avg.pct_change().dropna()<br>returns.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/327/1*1LZO9uctK3VRbgnpIXxnsg.png" /></figure><p>Building a model with the ARIMA algorithm requires that the time-series is stationary. A series is stationary when the mean, variance, and autocorrelation stay constant over time — that is, the data is neither trending up nor down and its spread remains constant over time.</p><p>The Adfuller test is used to check for stationarity. If the series is stationary, we expect the Adfuller test to return a p-value less than 0.05.</p><pre># Test for stationarity using Adfuller test.<br>result = adfuller(returns)<br>if result[1] &lt; 0.05:<br>    print(f&quot;p-value: {result[1]} Reject null hypothesis. The data is likely stationary&quot;)<br>else:<br>    print(f&quot;p-value: {result[1]}: Fail to reject null hypothesis. The data is likely not stationary&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/527/1*wqWu0Yej3kkVwh1qxCAE_g.png" /></figure><p>Since the result shows that the series is stationary, we can split our data into a train and test set for training.</p><p>The train_test_split method provided by the sklearn library is unsuitable for splitting a time series because a time-series data is chronological. It is ordered and cannot be shuffled. The first date in the test set should continue from the last date in the train set. For simplicity, we will use a simple splitting method to split the data into a train and test set using a cutoff date. In the near future, I will update the post with a better splitting approach.</p><pre># Split data into train and test set<br><br>cutoff_date = &quot;2026-01-01&quot;<br>train = returns[returns.index &lt; cutoff_date]<br>test = returns[returns.index &gt;= cutoff_date]<br><br>assert len(train) + len(test) == len(returns)<br>print(f&quot;Train: {len(train)} observations&quot;)<br>print(f&quot;Test: {len(test)} observations&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/576/1*8A34Mz5Bg70qKnvkRbkFfg.png" /><figcaption>Image by Author</figcaption></figure><p>It’s time for training the model and tracking the experiments.</p><p>We need to first set our tracking URI (Uniform Resource Identifier). The tracking URI is the address the MLflow sends our logged data to.</p><p>To view our experiment on the MLflow web interface, we need a backend store that serves the logged data to a tracking server.</p><p>The backend store may be a local filesystem or a database like a SQLite database. When the backend store is a SQLite database, the metadata are stored in an mlflow.db file created automatically in our working directory. while the artifact is stored in a /mlruns folder in our working directory.</p><pre># Set the tracking uri.<br><br>mlflow.set_tracking_uri(&quot;sqlite:///mlflow.db&quot;)</pre><p>When the backend store is a local filesystem, both metadata and artifact are stored in the /mlruns folder in the working directory.</p><p>In earlier MLflow versions, if you do not set a tracking URI, the backend store defaults to a /mlruns folder. However, recent versions of MLflow now default the backend store to an SQLite database.</p><p><strong>Model Training and Experimental Tracking</strong></p><p>Let’s understand the ARIMA algorithm briefly:</p><p>ARIMA (Autoregressive Integrated Moving Average) is a classical forecasting algorithm that predicts future values based on the data’s past behavior. ARIMA uses three components: p, d, and q for forecasting. The p, d. and q are available through the order parameter. The appropriate values for p, d, and q are found by experimenting with different combination of the values (hyperparameter tuning) to optimise model performance.</p><p><strong>Order (p, d, q):</strong></p><p><strong>p</strong> — The Autoregressive (AR) component of ARIMA that determines how many past values (lags) are used to predict the current value.</p><p><strong>d </strong>— The Integrated (I) component of ARIMA that determines how many times the time-series is differenced to make it stationary (to eradicate trends).</p><p><strong>q</strong> — The Moving Average (MA) component of ARIMA determines how many forecast errors are used to improve the current prediction.</p><p>We first perform a single run by training a model. Later, we will perform multiple runs by performing a simple hyperparameter tuning. We created a separate experiment for educational purposes.</p><p>The following code only executes a single run. Each time the</p><p>When the <strong><em>with mlflow.start_run() </em></strong>code block is executed, a single run is executed, and the outputs are logged into the experiment set.</p><p>At this run, the following data are logged:</p><ul><li>The Root Mean Square (RMSE)</li><li>Model hyperparameters: p, d, q</li><li>Residual plot allowing us to confirm how the model performs by viewing the residual distribution.</li><li>The trained ARIMA model</li></ul><pre># Uncomment the two lines below if you do not want to set sqlite db as the tracking uri<br># but want to use your local file system as the backend store.<br><br># mlflow.set_tracking_uri(&quot;file:./mlruns&quot;)<br># os.environ[&quot;MLFLOW_ALLOW_FILE_STORE&quot;]=&quot;true&quot;<br><br>mlflow.set_experiment(&quot;simple_experiment&quot;)<br><br>with mlflow.start_run(run_name=&quot;arima_2_0_1&quot;):<br>    # Train the model<br>    arima = ARIMA(train, order=(2, 0, 1))<br>    model = arima.fit()<br>    <br>    # Predict returns<br>    forecasts = model.forecast(steps=len(test))<br><br>    # Evaluate the model<br>    rmse = root_mean_squared_error(test, forecasts)<br>    <br>    # Log metric and parameters in mlflow<br>    mlflow.log_metric(&quot;rmse&quot;, rmse)<br>    mlflow.log_params({&quot;p&quot;: 2, &quot;d&quot;: 0, &quot;q&quot;: 1})<br>    <br>    # Log residual plot in mlflow<br>    residual_diagnostic_plot = model.plot_diagnostics(figsize=(12, 7))<br>    mlflow.log_figure(residual_diagnostic_plot,artifact_file= &quot;residual_diagnostic.png&quot;)<br>    <br>    # Log models in mlflow<br>    mlflow.statsmodels.log_model(model, name=&quot;model&quot;)</pre><p>After performing the single run, we can view the logged data on the web interface by running the following code in the terminal. To execute the code in Jupyter Notebook, add a “!” before the mlflow command like this: !mlflow ui…</p><pre># Launch the mlflow UI using sqlite database as the backend store <br><br>mlflow ui --backend-store-uri sqlite:///mlflow.db --port 5000</pre><ul><li>ui — means user interface.</li><li>setting sqlite db as the backend-store-uri tells the server to display the metadata stored in mlflow.db on the web user interface.</li><li>port 5000 — The web interface will run on a localhost server. 5000 is the port we are connecting to on the server. 5000 is the default on MLflow but, you can set another port.</li></ul><p>After running the <strong><em>mlflow ui.. </em></strong>command, wait a few seconds for it to execute.</p><p>Then, you can open your web browser and type localhost:5000</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jjGUZvOcNevLMv1HH90lTw.png" /><figcaption>Image by Author</figcaption></figure><p>When the user interface is loaded successfully, click on <strong>Experiment i</strong>n the left pane to view the experiments as shown in the screenshot above. To view the runs for this experiment:</p><ul><li>Click on the <strong>simple_experiment</strong>,</li><li>After opening <strong>simple_experiment</strong>, the <strong>Training runs </strong>option will appear in the left pane.</li><li>Click on the <strong>Training run </strong>to view all the runs in the experiment as shown in the following image.</li><li>You can then click on the run to view the logged metrics, model and plot.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uIgiiRFQnu8gXZlXEjvCPQ.png" /></figure><p>In machine learning, a single trial is not enough, we need to run several trials using hyperparameter tuning to find the best hyperparameters that achieve the best model.</p><p>In the following code, we modify the previous trial code and wrap it in a function to allow us to tune the model hyperparameter. We will set “<strong>full_experiment</strong>” as the name for this experiment.</p><p>Plural methods in MLflow, like log_metrics, and log_params are usually used to store multiple values in a dictionary while singular methods like log_metric, and log_param log a single value. It takes the name of the value as the first parameter (the key) and the value itself as the second.</p><pre><br>def run_experiment(train, test, order):<br>    &quot;&quot;&quot;Train, evaluate and run an Arima model experiment with MLflow&quot;&quot;&quot;<br>    mlflow.set_experiment(&quot;full_experiment&quot;)<br>    with mlflow.start_run(run_name=f&quot;arima_{order}&quot;):<br>        <br>        arima = ARIMA(train, order=order)<br>        model = arima.fit()<br>        <br>   <br>        forecasts = model.forecast(len(test))<br><br>        rmse = root_mean_squared_error(test, forecasts)<br>        mae = mean_absolute_error(test, forecasts)<br>        <br>        # Residual plot<br>        residual_diagnostic_plot = model.plot_diagnostics(figsize=(12, 7))<br><br>        # Log metrics and params<br>        mlflow.log_metrics({&quot;mae&quot;: mae, &quot;rmse&quot;:rmse})<br>        mlflow.log_metric(&quot;aic&quot;, model.aic)<br>        mlflow.log_params({&quot;p&quot;: order[0], &quot;d&quot;: order[1], &quot;q&quot;: order[2]})<br>        <br>        # Log residual plot<br>        mlflow.log_figure(residual_diagnostic_plot, artifact_file=&quot;residual_diagnostic.png&quot;);<br>        <br>        # Log model<br>        mlflow.statsmodels.log_model(model, name=f&quot;model_{order}&quot;)</pre><p>The following code performs a simplified hyperparameter tuning using four sets of different hyperparameters in the list. The loops will run four times, implying that we will have four runs. Each run is tracked alongside their respect artifacts and metadata.</p><pre># Perform hyperparameter tuning<br>for order in [(1, 0, 0), (1, 0, 1), (2, 1, 2), (4, 0, 2)]:<br>    run_experiment(train, test, order)  </pre><p>Finally, we can reload localhost:5000 to view the results of our experiment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0w-WCR-ukZpjwVRsxIyQ7g.png" /><figcaption>Image by Author</figcaption></figure><p>We now have all the results for each run tracked and stored under the experiment. You can compare the results from each run to know which model from the runs performs best. When you click on the first small box beside <strong>Run Name </strong>as shown in the screenshot above, the <strong>compare</strong> option will appear as shown in the screenshot above.</p><p>You can then compare the metrics across runs as shown in the following screenshot; you can also view and compare the residual plots across runs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uQ78s02niTsKV8xb25FrnA.png" /></figure><p>If we want to view the metadata logged on the UI from the mlflow.db from the Notebook, we can connect to the SQLite database like this.</p><pre>import sqlite3<br>conn = sqlite3.connect(&quot;mlflow.db&quot;)</pre><p>We can then query the database to view the logged data in the metrics:</p><pre>df_metrics = pd.read_sql(&quot;SELECT * FROM metrics&quot;, conn)<br>df_metrics.head(10)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/654/1*taizywvzK_LmQhBnBBIz1g.png" /></figure><p>We can check all the tables available in the database by running the following code:</p><pre>cursor = conn.cursor()<br>cursor.execute(&quot;SELECT name FROM sqlite_master WHERE type=&#39;table&#39;;&quot;)<br>tables = cursor.fetchall()<br><br>for table in tables:<br>    print(table[0])</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*P3jx9JjwmyE93ccnsZZ5Zw.png" /></figure><p>The result above is an excerpt of the result from the code. We can then view the contents of any table as we did with the metrics table previously.</p><p><strong>Conclusion</strong></p><p>MLflow does not only tracks the models and metrics, but it also tracks the GitHub commit, dataset version and many others that we will treat when this post is updated in the nearest future.</p><p>In this post, we have learnt the concept of experimental tracking and how MLflow can be used to track experiments. Next time you are training a model, ensure to make your work easy by tracking the experiment with MLflow.</p><p>I’m Ibukun Irinyenikan, a data and machine learning professional. I write articles that simplify complex technical topics through practical, real-world examples. I’m currently open to freelance and full-time opportunities in technical writing, data analytics and machine learning. If you’d like us to work together, feel free to connect on <a href="http://linkedin.com/in/ibukunirinyenikan">LinkedIn</a> or <a href="https://wa.link/yhip68">WhatsApp</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=11d791ac3a7c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[MLOps Explained]]></title>
            <link>https://medium.com/@ibukunirinyenikan/mlops-explained-8415e0bbf34e?source=rss-7ab15db4bcd2------2</link>
            <guid isPermaLink="false">https://medium.com/p/8415e0bbf34e</guid>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[ai-engineering]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[towards-data-science]]></category>
            <dc:creator><![CDATA[Ibukun Irinyenikan]]></dc:creator>
            <pubDate>Fri, 28 Aug 2026 19:06:01 GMT</pubDate>
            <atom:updated>2026-08-28T19:06:01.316Z</atom:updated>
            <content:encoded><![CDATA[<h4>Practices employed in production-based machine learning development</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/901/1*jXCkW2dxuX9qDbxZWktJsA.jpeg" /><figcaption>source: <a href="https://www.datascience-pm.com/machine-learning-operations/">Data Science PM</a></figcaption></figure><p><strong>What is MLOps?</strong></p><p>Machine Learning Operations (MLOps) is a set of practices employed to make machine learning (ML) model development deployable, usable, and maintainable. It is a software engineering practice responsible for taking model development from experimentation to production, where the model is used to solve real-life problems.</p><p>MLOps is inspired by Development Operations (DevOps). DevOps is a set of software engineering practices applied to making software applications usable and maintainable.</p><p>MLOps applies DevOps practices to machine learning development, as well as new practices specific to machine learning, such as model retraining and model monitoring. MLOps practices allow model development to be automated, reproducible, and maintainable.</p><p>In production-based ML, MLOps practices are implemented in each stage of the workflow, from data ingestion to deployment.</p><p><strong>Why is MLOps Important?</strong></p><p>Let’s explore the characteristics of machine learning systems built with MLOps practices.</p><p><strong>Automation</strong></p><p>MLOps allows each stage of the MLOps lifecycle to be automated, eliminating unnecessary manual intervention in production. Automating the pipelines makes the workflow faster. Workflow orchestration tools such as Airflow and Prefect can be used to automate each pipeline from ingestion to deployment. GitHub Actions or Jenkins can be used as Continuous Integration/Continuous Deployment (CI/CD) tools to automate testing, model training, and deployment.</p><p><strong>Reproducibility</strong></p><p>MLOPS allows machine learning experiments to be reproducible by applying certain MLOps practices to every stage of the pipeline. Reproducing the result is possible when everything involved in the training is tracked and recorded. Applying a seed to model training is also crucial for reproducibility. The trained model, training seed value, model parameters, evaluation metrics, and references to the code and dataset must all be tracked. Reproducibility promotes collaboration among team members. Tracking and storing all this information makes it possible for a data scientist who joins the team today to reproduce the exact result of a model that was trained three months ago.</p><p>Tracking and storing this information is possible through version control and experimental tracking.</p><p>· Code versioning uses Git to track and store code versions on GitHub.</p><p>· Data versioning uses the Data Version Control (DVC) library to track dataset versions.</p><p>· Experimental tracking uses tools like MLflow and Weights &amp; Biases to track and record model artifacts, including the ML models, parameters, training seed, and evaluation metric results.</p><p><strong>Stages in the MLOps Lifecycle</strong></p><p>MLOps lifecycle is an end-to-end process that applies MLOps in building, deploying, monitoring, and maintaining machine learning models in a production environment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k4hMo7bhiYkGSJxZi433JA.jpeg" /><figcaption>AI-generated</figcaption></figure><p><strong>Data Ingestion</strong></p><p>Ingestion is the first stage of the MLOps lifecycle responsible for extracting data from necessary sources for training machine learning models in production. The data can be extracted by batch and streaming processing. Before raw data is pre-processed, the data must pass through a data validation stage to confirm that the structure and quality of the ingested data matches what is expected from the source. Great Expectation library can be used to validate data and provide a report on the validation status of the attributes and data values.</p><p>The data can be validated for the following:</p><p>· The column matches what is expected</p><p>· The missing data are within the expected range.</p><p>· The number of categories in the categorical variables matches the expected.</p><p>· Outliers in the numerical variables are within a specified threshold.</p><p>When a validation check fails, it can cause the system to terminate the system depending on the severity of the issue and the response action configured by the machine learning engineer. If all validation checks pass, the pipeline continues to the next stage. Before proceeding to the next stage, the dataset must be versioned. Versioning the dataset allows machine learning experiments to be reproducible. DVC is used to version and track the raw dataset. A copy of the dataset can be stored remotely in cloud storage such as AWS S3, while a DVC metadata file containing information about the dataset is created, can be tracked, and pushed to GitHub using Git.</p><p><strong>Data Transformation/Feature Engineering</strong></p><p>The feature engineering pipeline is responsible for creating the required input for model training and inference in a production environment. At this stage, data transformation techniques such as onehot encoding can be performed on categorical variables to convert them to numbers. Standardisation or normalisation can be applied to numerical variables to bring the data to the same scale in order to eliminate model bias towards features with higher unit. The data is not only transformed at this stage, but new features derived from other features in the dataset can also be created.</p><p>The transformed data can be stored and managed in a feature store like Feast. Storing the transformed dataset in a feature store allows the model to receive the same input structure at inference as it does at training. The exact cleaning and transformation techniques applied to the training data must also be applied to the data at inference to eliminate the training-serving skew where input structure of dataset at training differs from the data served at inference.</p><p><strong>Model Training</strong></p><p>In production MLOps systems, model training can be part of the automated retraining pipeline. The model is trained applying hyperparameter tuning. Multiple versions of a model are trained on different hyperparameter settings. The models are evaluated with a validation set or through cross-validation. All model versions alongside its hyperparameters, metrics result are tracked and stored using experimental tracking tools like MLflow.</p><p><strong>Model Evaluation</strong></p><p>During evaluation, the model with the best performance after hyperparameter tuning is evaluated on a test set in a production environment. Evaluation metrics such as precision, recall, F1-score, accuracy, and confusion matrix are used for evaluating classification tasks, while Root Mean Square Error (RMSE), Mean Absolute Error (MAE), r2 score are used for evaluating regression tasks. Tracking and storing the evaluation results of each experimental run with MLflow allows different model results to be compared. The model with the best performance is stored in a model registry like MLflow Model Registry or a data lake like AWS S3.</p><p><strong>Model Deployment</strong></p><p>In a production environment, before a retrained model is deployed. Its performance is first compared with the performance of the currently deployed model. A/B testing may be performed to confirm that the newly trained model statistically outperforms the model currently deployed in production. If the performance of the retrained model is statistically significant, the newly retrained model is promoted to production, replacing the currently deployed model. Otherwise, the system continues running with the current model. MLOps automates the deployment process in a continuous deployment pipeline.</p><p>An application can then be used to serve the model with APIs like FastAPI or Flask to the end-user. Instead of the best-performing model being deployed as a standalone, it is deployed in a container. The model, alongside the serving application code and dependencies are packaged into a Docker container so it can run consistently across different environment.</p><p>The model that best solves the problem can be deployed after it passes the organisation’s required governance and approval checks to ensure fairness, compliance, and security. This is usually the practice for offline-trained models and production-based retrained models.</p><p><strong>Monitoring</strong></p><p>In production, monitoring describes tracking both the model performance and the health of the infrastructure where the model is running. Monitoring the model is essential for the ML model to continue producing desirable results.</p><p>Evidently AI can be used to monitor model performance. With Evidently AI, the data is continuously monitored to discover when the distribution of the data at production environment differs from the training data — known as data drift.</p><p>The model performance can also be assessed when ground-truth data is available for evaluation.</p><p>In addition, the health of the production environment is also being tracked at this stage. The system is checked for latency, throughput, and infrastructure cost using Prometheus and Grafana.</p><p><strong>Model Retraining</strong></p><p>Retraining occurs on schedules or triggers in a continuous training pipeline. A model in the production environment can be scheduled to retrain on a monthly basis when enough data is available. The model can also be triggered to retrain when significant data drift and model performance degradation are detected.</p><p><strong>Conclusion</strong></p><p>Rather than a linear implementation of the pipelines, the MLOps lifecycle is cyclical. It goes from data ingestion to deployment, and when new data is available, the lifecycle returns to ingestion, extracting the data, transforming it, training a model, evaluating it, and deploying the model.</p><p>An orchestration tool like Airflow is responsible for automating how the pipeline is executed. Airflow not only orchestrates the entire pipeline from one stage to the other, but it is also responsible for scheduling what time or when a certain pipeline, like ingestion, needs to run or when the model should be retrained. This could occur at a specified time or when an event is triggered.</p><p>MLOps skills may not always be a must if you are aspiring to become a data science or AI researcher. However, if you want to become a data scientist in organisations where machine learning is applied in developing software applications that solve real-life problems, a practical understanding of MLOps is crucial and cannot be overemphasized.</p><p>PS: This article will be continually updated in the coming weeks.</p><p>I’m Ibukun Irinyenikan, a data and machine learning professional. I write articles that simplify complex technical topics for beginners through practical, real-world examples. I’m currently open to freelance and full-time opportunities in technical writing, data analytics and machine learning. If you’d like us to work together, feel free to connect on <a href="http://linkedin.com/in/ibukunirinyenikan">LinkedIn</a> or <a href="https://wa.link/yhip68">WhatsApp</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8415e0bbf34e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Data Science Lifecycle]]></title>
            <link>https://medium.com/@ibukunirinyenikan/data-science-life-cycle-053585e8e768?source=rss-7ab15db4bcd2------2</link>
            <guid isPermaLink="false">https://medium.com/p/053585e8e768</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-scientist]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science-life-cycle]]></category>
            <dc:creator><![CDATA[Ibukun Irinyenikan]]></dc:creator>
            <pubDate>Mon, 24 Aug 2026 22:11:01 GMT</pubDate>
            <atom:updated>2026-08-24T23:02:47.217Z</atom:updated>
            <content:encoded><![CDATA[<h4>The stages involved in the data science lifecycle</h4><p>Data science is responsible for converting raw data into actionable insights that solve business problems. There are series of steps data scientists follow while solving a problem. These steps are referred to as data science life cycle.</p><p>Data science life cycle is a structural and iterative process which comprises various stages data scientists take to uncover insight from data.</p><p><strong>The Data Science Lifecycle</strong></p><p><strong>Problem Definition and Planning</strong></p><p>Starting a project without a proper understanding of the business problem is a journey in the wrong direction, often leading to waste of time, energy and resources. The first thing to do before embarking on a project is to critically analyse the business problem, understand it and propose possible solutions. It is at this stage that the team discovers whether data analysis is sufficient or building a model is necessary to reach the goal.</p><p>The team breaks the project into milestones and sets a deadline the team should strive to meet utilising project management tools like Jira.</p><p><strong>Data Collection</strong></p><p>Data collection explains how data is extracted from different sources for processing. Raw data may be collected from various sources — data warehouse, data lake or from websites.</p><p>Data can be collected from a dedicated warehouse using Structured Query Language (SQL).</p><p>To collect data from a third-party website, an Application Programming Interface (API) offered by the website can be used. In the absence of that, web scraping using Selenium and the BeautifulSoup Python library is an alternative.</p><p><strong>Data Preprocessing</strong></p><p>Raw data are rarely clean. Raw data may contain inconsistencies like missing data, incorrect data, incorrect data types, etc. During data preprocessing, the inconsistencies are eliminated for the data to be useful for analysis or model training.</p><p>For machine learning task, data transformation techniques such as onehot encoding and label encoding are applied to convert categorical variables to numbers — the format suitable for mathematical calculations by machine learning algorithms. The numerical data is standardized or normalised to bring data with different units to the same scale. This prevents the model from being biased towards variables with higher units.</p><p>As part of the data preprocessing stage, the data is split into training, validation and a test set before transformation is applied on the dataset. The training set is used for training the model. The validation set is used for evaluating the model performance during hyperparameter tuning. The test set is used to evaluate the model performance after training.</p><p><strong>Exploratory Data Analysis</strong></p><p>After data cleaning, the data scientist analyses the data to understand it and extract insights that may be useful to create new features during feature engineering. Data exploration is not only performed to draw insights but also to check the distribution of data, which may help discover any anomalies such as outliers in the dataset.</p><p><strong>Feature Engineering</strong></p><p>Feature engineering explains how new features are created to train or improve models. The data scientist applies their knowledge of the business to aggregate multiple features to create a new one or derive new features by applying certain transformation techniques to an existing feature. The transformed data is then stored and managed using a feature store which ensures that the model receives consistent features in both training and inference. Feast is an open-source example.</p><p><strong>Model Training and Hyperparameter Tuning</strong></p><p>Model training is performed in a trial-and-error approach where the data scientist initially trains multiple models and selecting one for hyperparameter tuning.</p><p>Instead of training a model with a single hyperparameter setting, data scientists apply hyperparameter tuning to experiment with different hyperparameter settings to find the settings that best optimise model performance. The model performance is evaluated with the validation set kept aside during data splitting during hyperparameter tuning.</p><p><strong>Model Evaluation</strong></p><p>Evaluation is performed on the test set to simulate the model performance on real-world data because the test set is not exposed to the model during training. The model performance is assessed by the data scientist not only to confirm its accuracy on the test set but also to confirm that the model does not have a bias that can be detriment to users at deployment. If the model is cleared by the data scientist, it can then be selected for deployment.</p><p><strong>Model Deployment</strong></p><p>Deployment defines how the model will be utilized in the real-world to solve the problem identified at the first stage of the lifecycle.</p><p>Depending on the use-case, the model can be served with an API like FastAPI or Flask for users to make predictions on a web browser.</p><p><strong>Monitoring</strong></p><p>Monitoring means continuous tracking of the model performance and the system health at production environment. Model monitoring ensures that the model continues to perform as expected.</p><p>At this stage, the data is continuously monitored to discover when the distribution of the data at production environment differs from the training data — known as data drift. It also continually tracks model quality — by evaluating the model performance with a ground-truth data.</p><p>Deployment and monitoring can be implemented as part of the continuous Integration/Continuous Deployment pipeline (CI/CD).</p><p><strong>Conclusion</strong></p><p>The data science lifecycle explains the stages the data scientists follow to solve a problem. In practice, the lifecycle is iterative — meaning, implementation does not always follow a forward path. For instance, after model training, the data scientist may return to the data collection stage to collect more data to improve model performance. It is also possible to return to the data transformation or feature engineering stage to create new features or transformations to assess their impact on model performance on retraining.</p><p>My name is Ibukun Irinyenikan. I am a data and machine learning professional. I write articles that simplify complex technical topics for beginners through practical, real-world examples. I’m currently open to freelance and full-time opportunities in technical writing, data analytics and machine learning. If you’d like us to work together, feel free to connect on <a href="http://linkedin.com/in/ibukunirinyenikan">LinkedIn</a> or <a href="https://wa.link/yhip68">WhatsApp</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=053585e8e768" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Skills Required to Become a Data Scientist]]></title>
            <link>https://medium.com/@ibukunirinyenikan/the-skills-required-to-become-a-data-scientist-be554ee97cba?source=rss-7ab15db4bcd2------2</link>
            <guid isPermaLink="false">https://medium.com/p/be554ee97cba</guid>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[towards-data-science]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[soft-skills]]></category>
            <dc:creator><![CDATA[Ibukun Irinyenikan]]></dc:creator>
            <pubDate>Fri, 21 Aug 2026 19:06:01 GMT</pubDate>
            <atom:updated>2026-10-16T17:27:44.309Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wLUieqqhM6ljDLviX-6XRg.jpeg" /><figcaption>AI-generated</figcaption></figure><p>The role of a data scientist is to uncover insights from data. These insights can be used to solve business problems. Becoming a data scientist requires both technical and soft skills. Let’s take a look at the key skills required to become a data scientist.</p><p><strong>1.</strong> <strong>Programming</strong></p><p>Programming languages such as Python or R are essential for data cleaning, statistical analysis and machine learning modelling. Python is highly recommended as it is considered the ‘consensus’ programming language in the data science and machine learning community. It is the most popular language for data analytics and machine learning libraries.</p><p><strong>2.</strong> <strong>Probability and Statistics</strong></p><p>Probability and statistics are essential for performing data analysis and statistical tests. It is also required to understand the distribution of data. If you are aspiring to become a data scientist, you should learn both descriptive and inferential statistics. A proper knowledge of descriptive statistics is key.</p><p>Descriptive statistics concepts such as mean, median and mode are required in summarising data. Variance and standard deviation are important in understanding dataset distribution.</p><p>Knowledge of probability is required to understand inferential statistics concepts such as z-score, t-test and p-values, hypothesis testing, among others.</p><p><strong>3.</strong> <strong>Mathematics</strong></p><p>Mathematical concepts like linear algebra and calculus are crucial for understanding how machine learning algorithms work. Machine learning algorithms are developed with concepts from linear algebra and calculus. Having a proper understanding of these topics will help you develop intuition on how the algorithms work.</p><p><strong>4.</strong> <strong>Data Visualisation</strong></p><p>Python libraries such as Matplotlib, Seaborn and Plotly are required for creating visualisations while performing exploratory data analysis. Visualisation tools like Power BI or Tableau are essential for building dashboards to communicate insights to stakeholders.</p><p><strong>5.</strong> <strong>Machine learning</strong></p><p>Knowledge of machine learning is critical for data scientists. Data scientists have an in-depth understanding of how machine learning algorithms work. You should at least have an intuition of how the algorithms work.</p><p><strong>6.</strong> <strong>Database Management</strong></p><p>Data are often stored in databases. Structured Query Langugage (SQl) is the language of the database. You should know how relational and non-relational databases work. The knowledge of SQL and NoSQL is important for querying, storing and manipulating data.</p><p><strong>7.</strong> <strong>Model Deployment</strong></p><p>After data scientists achieve a model that solves the problem. The model needs to be deployed to a platform where it can be used in real-life. You need to learn how to deploy models not just on a laptop but in a real production environment. That is the skill that that solves problems.</p><p>8. <strong>Machine Learning Operations (MLOps):</strong> MLOps is a concept responsible for taking machine learning projects from experimentation to production. It is strongly required by companies. Understand MLOps concepts such as code versioning, data versioning, model tracking, retraining, monitoring and Continuous Integration and Continuous Deployment (CI/CD) among others.</p><p>9. <strong>Cloud Computing: </strong>Cloud computing is a sought-after skill. Real-world data science projects are not carried out on a personal computer (PC) because building models requires compute resources beyond what a PC offers. Models are built using the compute resources of third-party cloud providers.</p><p>Amazon Web Services (AWS), Google Cloud Platform (GCP) and Microsoft Azure are the leading cloud platforms. Pick one and learn.</p><p>When you sign up for the first time on any of the three cloud platforms, you get a free trial for you to start learning. Make good use of it.</p><p><strong>Soft skills required to become a data scientist</strong></p><p>These soft skills are not specific to data science. They are cross-functional; they cut across many tech fields. The skills are also very important for the success of data science projects.</p><p><strong>1.</strong> <strong>Communication</strong></p><p>Communication skill is necessary for data scientists to communicate complex findings to both technical and non-technical stakeholders in a clear and concise way.</p><p><strong>2.</strong> <strong>Critical Thinking and Problem Solving</strong></p><p>As a data scientist, you should know how to approach problems logically and break down complex problems into smaller ones before embarking on a project.</p><p><strong>3.</strong> <strong>Collaboration</strong></p><p>As a data scientist, you will collaborate with data engineers, software engineers and non-technical teams. Data science projects often require a teamwork. As a data scientist, you must know how to collaborate with team members to achieve the desired objective.</p><p><strong>4.</strong> <strong>Domain knowledge</strong></p><p>Having knowledge of the kind of company you are passionate about working for will be crucial in getting you the job. Knowledge of the business helps you know how each variable in the dataset is related to the business. Having domain knowledge will help you communicate findings to non-technical stakeholders.</p><p>Now that we have understood the skills required to become a data scientist. Let us see how some of these skills are applied in data science.</p><p><strong>What do data scientists do?</strong></p><p>Data scientists perform various kinds of tasks depending on the nature of the business problem they want to solve. Let’s take a look at the category of problems data scientists solve and the tools required.</p><p><strong>Descriptive analytics</strong></p><p>Data scientists apply descriptive analytics to inquire what happened in the business. With descriptive analytics, they are able to analyse historical data to help the stakeholders know how the business performed. Your knowledge of statistics is vital here.</p><p>To perform descriptive analytics, the Python Pandas library is crucial for analysing data. Knowledge of visualisation libraries like Matplotlib, Seaborn and Plotly are required for understanding the data during exploratory data analysis. They help you create charts like bar charts, histograms, line charts, boxplots, and violin plots to understand the data.</p><p><strong>Diagnostic analytics</strong></p><p>Knowing what happened in a business is not enough. It is also important to know why it happened. This will help stakeholders correct problems discovered or improve working strategies. This is what diagnostic analytics seek to solve. Unlike descriptive analytics where data scientists find out what happened in the business, data scientists apply diagnostic analytics to find out why it happened by digging deep into the root of what has happened. This will help the stakeholders make informed decisions. Similar tools used for descriptive analytics are also applicable for performing diagnostic analytics.</p><p>Your knowledge of statistical concepts in descriptive and inferential statistics is vital and applicable in performing both descriptive and diagnostic analytics.</p><p><strong>Predictive analytics</strong></p><p>Predictive analytics helps the business know what will happen in the future. This is the aspect of the data scientists’ job that requires machine learning. In predictive analytics, data scientists build machine learning models using historical data to know what will happen in the future. Your knowledge of linear algebra and calculus will be applied for training and improving machine learning models to achieve the desired results.</p><p><strong>Prescriptive analytics</strong></p><p>After data scientists discover events that happened using descriptive statistics, and what could likely happen in the future through predictive analytics, data scientists will now apply prescriptive analytics to recommend actions and measures that will help the business meets its goals and objectives based on information discovered from the analytics discussed above.</p><p>To make recommendations to stakeholders, results can be communicated through presentation slides or visualisation dashboards. This is where the knowledge of PowerPoint and visualisation tools such as Power BI and Tableau comes in handy. Communication skills are required here for effective communication of results.</p><p><strong>Conclusion</strong></p><p>In this article, we have covered both the technical and soft skills required to become a data scientist. It is easy for learners to focus on the technical skills while ignoring the soft skills.</p><p>Technical skills are required to get hands-on and build projects. As a data scientist, you will always be working in a team. Therefore, soft skills are required for the success of every project.</p><p>The lists are by no means exhaustive, but they are sufficient to get you started in your career. In the last section of the articles, we not only explored the kind of tasks a data scientist performs, but we also highlight the skills required to perform them.</p><p>As a data and machine learning professional, I write articles that simplify complex technical topics for beginners through practical, real-world examples. I’m currently open to freelance and full-time opportunities in technical writing, data analytics and machine learning. If you’d like us to work together, feel free to connect on <a href="http://linkedin.com/in/ibukunirinyenikan">LinkedIn</a> or <a href="https://wa.link/yhip68">WhatsApp</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=be554ee97cba" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Machine Learning is Applied in Data Science]]></title>
            <link>https://medium.com/@ibukunirinyenikan/how-machine-learning-is-applied-in-data-science-9d01087f2fb0?source=rss-7ab15db4bcd2------2</link>
            <guid isPermaLink="false">https://medium.com/p/9d01087f2fb0</guid>
            <category><![CDATA[data-scientist]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-analyst]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Ibukun Irinyenikan]]></dc:creator>
            <pubDate>Wed, 19 Aug 2026 19:16:01 GMT</pubDate>
            <atom:updated>2026-08-20T10:23:53.244Z</atom:updated>
            <content:encoded><![CDATA[<h4>The machine learning tasks data scientists perform and how the machine learning algorithm learns</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4hIzNYWmaX48nlqk6Cuv7Q.jpeg" /><figcaption>AI-generated</figcaption></figure><p><strong>What is data science?</strong></p><p>Data science is the process of using algorithms and statistical tools to uncover meaningful information from data that helps organisations make informed decisions.</p><p>Data science is data-driven — allowing stakeholders to make decisions based on facts derived from data instead of personal feelings or trial and error.</p><p>Statistical tools are used in data science to unveil insights from data. Data scientists train machine learning models on large amounts of data in order to make prediction about the future.</p><p><strong>What is machine learning?</strong></p><p>Machine learning is a subset of artificial intelligence where a computer uses statistical techniques and mathematical principles to learn patterns from data without being explicitly programmed. Unlike traditional programming, where the system follows a step-by-step written logic and rules to create solutions. Machine learning instead learns patterns from data during training and uses these patterns to create solutions.</p><p><strong>How is machine learning related to data science?</strong></p><p>Data science is concerned with drawing useful information from data by applying various tools and techniques based on the business problem. One of such techniques is machine learning. Data scientists train machine learning models to learn patterns from data to predict future events.</p><p><strong>Types of Machine Learning</strong></p><p>Data scientists build models using different learning approaches. The approach employed is determined by the kind of problem to be solved. Such learning approach includes supervised, unsupervised, semi-supervised and reinforcement learning.</p><p><strong>Supervised machine learning</strong></p><p>In supervised machine learning, the model is trained with a labelled dataset — the features along with their corresponding labels known as the target variable. The label or target is the variable to be predicted.</p><p>The features are the predictors; they are responsible for predicting the target. The model learns to associate the features to their corresponding labels so the model can make predictions on data it was not trained on.</p><p>Depending on the nature of the target, supervised machine learning can be grouped into classification and regression.</p><p><strong>Classification</strong>: It is applied to datasets where the values of the target are discrete — that is, they are distinct and countable. Temperature, as a target variable, for example, is distinct and countable. Because temperature can be cold, warm or hot.</p><p>Classification tasks are applied in image classification, spam filtering, fraud detection and sentiment analysis.</p><p><strong>Regression</strong>: Regression tasks are applied in datasets where the target variable is continuous — that is, measurable and can take any value within a range, even taking on decimal values. Height and price, for example, are measurable. Regression tasks are applied in e-commerce to predict sales revenue and in time series analysis to forecast stock returns.</p><p><strong>Unsupervised machine learning</strong></p><p>This task is applied to datasets without labels. The machine learning algorithm learns to figure out patterns and relationships in the data all on its own without labels to map them to. Unsupervised leaning is employed in clustering, dimensionality reduction and association rule.</p><p><strong>Types of Unsupervised learning</strong></p><p><strong>Clustering</strong></p><p>This task is applied to a dataset without a target. The machine learning algorithm learns to group datapoints with similar characteristics in the same cluster. It is applicable in e-commerce for performing customer segmentation — meaning the model groups customers with similar buying traits together. KMeans, hierarchichal and density-based clustering algorithms are among the top algorithms for performing clustering tasks.</p><p><strong>Dimensionality Reduction</strong></p><p>It is the process of reducing the number of features in a dataset while retaining enough information to make sense of the data.</p><p>Dimensionality reduction is an unsupervised task because the algorithms take features (without a target) with larger dimensions to find relationships and patterns to reduce them to a smaller dimension.</p><p>Dimensionality reduction is a feature engineering technique used by data scientists when a dataset contains too many features, often resulting in higher computational cost.</p><p>To train a model on such data, data scientists may apply dimensionality reduction to eradicate redundant features in the dataset in order to improve computational efficiency. When a dataset contains 50 features, dimensionality reduction may be applied to reduce them to 2 or 3 features.</p><p>It is also used to reduce overfitting and increase model reliability by eradicating multicollinearity — correlations between multiple independent variables that can lead to misleading model accuracy.</p><p>However, the downside of applying dimensionality reduction is that models trained on such transformed features are difficult to interprete relative to the original features. A popular algorithm used for performing dimensionality reduction is Principal Components Analysis (PCA).</p><p><strong>Association Rule</strong></p><p>Association rule is applied using a rule-based technique to find relationships between independent variables in a dataset. It looks for patterns where the occurrence of an event implies another.</p><p>In e-commerce, association rule is applied in market basket analysis — a technique used to discover products that customers frequently purchase together.</p><p>For example, if customers who buy iPhones always buy a charger alongside. Association rule algorithms capture this pattern so that the next time a potential buyer clicks on an iPhone, the algorithm recommends a charger alongside.</p><p>Machine learning algorithms such as Apriori and FP-Growth are popular among data scientists for performing association rule tasks.</p><p><strong>Semi-supervised machine learning</strong></p><p>In semi-supervised learning, a model learns from a dataset containing a small amount of labelled data and a large amount of unlabelled data.</p><p>Although it sits between supervised and unsupervised learning, only supervised tasks are performed.</p><p>It is applied when labelling is expensive or time-consuming. Let’s understand it better as we explore self-training and co-training forms of semi-supervised learning.</p><p><strong>Types of semi-supervised learning</strong></p><p><strong>Self-Training: </strong>Suppose you have a dataset where a smaller percentage is pre-labelled, and the larger percentage is unlabelled. In this task:</p><p>· The model is trained on the pre-labelled dataset so that the resulting model can predict the label for the unlabelled dataset.</p><p>· After labelling, the data from the newly labelled data where the model’s confidence is high is separated and then combined with the initial pre-labelled dataset for retraining.</p><p>The resulting model can then be used to label the remaining data where the model’s confidence was low.</p><p><strong>Co-Training: </strong>Two or more models are trained on different feature subsets of the labelled dataset. The resulting models label the unlabelled dataset for one another. What that means is that, say the dataset has two features, the small pre-labelled dataset feature may be divided into two subsets.</p><p>· A first model is trained on the first feature subset.</p><p>· A second model is trained on the second feature subset.</p><p>· To label the unlabelled dataset, the first feature of the unlabelled dataset is used by the first model to label it.</p><p>· The second feature of the newly labelled dataset with high prediction confidence is then extracted alongside its labels and joined to the second pre-labelled subset for retraining.</p><p>· Similarly, the second model labels the unlabelled dataset using its second feature only.</p><p>· The first feature is then extracted from the newly labelled dataset and included in the first pre-labelled subset for retraining.</p><p><strong>Reinforcement learning:</strong> This is a machine learning technique where a computer agent interacts with an environment by learning to perform a task through repetitive trial and error. The aim of reinforcement learning is to enable the agent to make decisions that maximise the reward.</p><p>When the agent makes wrong decisions, it is penalised for it. Reinforcement learning is applied in improving AI chatbots like ChatGPT and in autonomous cars.</p><p>Forms of reinforcement learning include value-based, model-based and policy-based reinforcement learning.</p><p><strong>Conclusion</strong></p><p>In this article, we learnt that machine learning is a tool applied by data scientists for helping organisations make informed decisions about the future.</p><p>We also explored the various tasks performed by data scientists. This article covers most of the approach data scientists apply to solve problems in day-to-day task.</p><p>Data scientists do not just choose any of these machine learning approaches to solve problems. The problems determine the kind of approach appropriate for solving them.</p><p>For a labelled dataset, supervised learning is applied.</p><p>Unsupervised learning is applied when the dataset is unlabelled and we want to find relationships in the data. Semi-supervised learning is applied for labelling a large dataset where a small portion of the dataset is already labelled.</p><p>Lastly, reinforcement learning is applied in an environment where improvements are achieved based on a reward system.</p><p><strong><em>About Me</em></strong></p><p><em>As a data and machine learning professional, I write articles that simplify complex technical topics for beginners through practical, real-world examples. I’m currently open to freelance and full-time opportunities in technical writing, data analytics and machine learning. If you’d like us to work together, feel free to connect on </em><a href="http://linkedin.com/in/ibukunirinyenikan"><em>LinkedIn</em></a><em> or </em><a href="https://wa.link/yhip68"><em>WhatsApp</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9d01087f2fb0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Artificial Intelligence Beyond ChatGPT]]></title>
            <link>https://medium.com/@ibukunirinyenikan/artificial-intelligence-beyond-chatgpt-64133514d274?source=rss-7ab15db4bcd2------2</link>
            <guid isPermaLink="false">https://medium.com/p/64133514d274</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Ibukun Irinyenikan]]></dc:creator>
            <pubDate>Tue, 18 Aug 2026 04:06:01 GMT</pubDate>
            <atom:updated>2026-10-12T21:45:28.330Z</atom:updated>
            <content:encoded><![CDATA[<h4>How Artificial Intelligence started before machine learning and LLM chatbots</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*VateDo-HniVlwDUkb9vdSQ.jpeg" /><figcaption>source: Britannica</figcaption></figure><p>As kids, when we heard the word AI, for many of us, the first thing that came to mind was humanoid robots — robots that look like humans. Of course, that perception came from movies.</p><p>Artificial intelligence (AI) has been used interchangeably with machine learning in the last decade by many people in tech.</p><p>The word took another shift when OpenAI released ChatGPT in 2022. Today, AI is almost always used to refer to ChatGPT, and chatbots built with Large Language Models (LLMs) — machine learning models that learn from almost all data on the internet.</p><p>The way AI is being used interchangeably with AI chatbots, it is becoming hard to identify other AI systems apart from LLM chatbots.</p><p>So, I decide to write this article to expand our knowledge of AI beyond ChatGPT so that the next time you see an AI system other than ChatGPT, you will be able to recognise it as one.</p><h4><strong>What is Artificial Intelligence?</strong></h4><p>Artificial intelligence refers to the ability of making computers smart enough to perform complex tasks that humans do, or better.</p><p>Instead of performing a task ourselves, a system is given the ‘intelligence’ to perform it like us. Often faster and sometimes achieves better results than us. The word AI is used in different context to describe the field, concept and systems. In this article, I use the word to describe systems that perform tasks that require human intelligence.</p><h4><strong>How a system is given AI capability</strong></h4><p>There are several ways a system can be given an AI capability, but we will discuss the two dominant approaches — rule-based and machine learning.</p><h4><strong>The Rule-based Approach</strong></h4><p>In a rule-based approach, the system is developed to make human-level decisions by following specified rules defined by the programmer. The rule-based method is applicable in expert systems.</p><h4><strong>What is an Expert system?</strong></h4><p>An expert system is an old subset of artificial intelligence that simulates human intelligence by applying a set of rules to make decisions like a human expert would. Let’s understand an expert system better by considering its two main components:</p><ul><li>Knowledge base</li><li>Inference engine</li></ul><p><strong>The knowledge base of an Expert System</strong></p><p>For a human expert to solve a particular problem, they rely on information and experience they acquired over the years in that area. They usually follow logical reasoning steps to arrive at a solution.</p><p>To give a system similar problem-solving capability, relevant information that helps the expert makes decisions are fed to the system alongside the logical reasoning steps the human expert would follow. These logical reasoning steps are hardcoded as rules often in the form of IF-THEN statements.</p><p>The relevant information fed to the system is known as the <strong>facts,</strong> while the IF-THEN conditional statements that simulate the reasoning steps are the <strong>rules</strong>. The facts and rules form the knowledge base of an expert system.</p><p><strong>The Inference engine of an Expert system</strong></p><p>The inference engine is the component of an expert system responsible for decision-making. When the system receives a problem to solve. The inference engine performs logical reasoning similar to how a human expert reasons by applying the rules in the knowledge base to reason through the facts available in order to arrive at a solution for the problem.</p><p>Let’s use a rule-based approach to build an email spam classifier.</p><p><strong>Email Spam Classifier AI Built with the Rule-based Approach</strong></p><p>With a rule-based approach, a programmer can use conditional statements to build rules so that when an email contains certain words or phrases like “won $1,000,000!” or “cash prize”, it is classified as <strong>spam </strong>and such an email is sent to the spam box. In the absence of those phrases, the email is classified as <strong>not spam.</strong></p><p><strong>The Machine Learning Approach</strong></p><p>When an AI system is built using machine learning, the system is built using machine learning models to make decisions.</p><p><strong>What is machine learning?</strong></p><p>Machine learning (ML) is the process of applying mathematical and statistical methods to learn patterns from data in order to draw insight useful for making predictions on data it was not trained on.</p><p><strong>Email Spam Classifier AI Built with the Machine Learning Approach</strong></p><p>To build an AI system that uses machine learning to send promotional emails to the spam box, thousands of emails are collected that contain promotional and non-promotional emails.</p><p>A sample dataset is represented in the following table.</p><p>A human annotator then labels the promotional emails as spam and the non-promotional ones as not spam.</p><p>The dataset is cleaned and converted to numbers for the algorithm to perform mathematical operations on it.</p><p>The transformed data is then passed to a machine learning algorithm — a mathematical rules and logic that learns pattern from data.</p><p>The algorithm learns patterns from the dataset that are associated with the corresponding label to achieve a model that can make predictions on data it was not trained on.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jTMd6vnXxxe7uZjE9XuFww.jpeg" /><figcaption>Email dataset</figcaption></figure><p>During training, the algorithm may learn to associate emails that contains the phrases like “click”, “won $1,000,000!” and “cash prize” as <strong>spam,</strong> while emails without such words as <strong>not spam. </strong>After training, a model is achieved that can now decide if an email is spam because it learnt patterns associated to them during training.</p><p><strong>Top Domains where Artificial Intelligence is applied</strong></p><p>Because of the association of machine learning with computer vision and NLP, many think that the two domains are inherently machine learning field. Although machine learning is popularly applied to solve problems in both domains. However, both computer vision and NLP have been in existence for several decades, long before machine learning came. Decades ago, scientists applied a rule-based approach to build computer vision and NLP AI systems.</p><p>Let’s consider briefly what computer vision and NLP are and the approach that’s being used to develop AI systems in both domains.</p><p><strong>Computer vision</strong></p><p>This is a field of AI where systems are given the capability to process, analyse and recognise images and video data like humans. Computer vision is applicable in building facial recognition systems for security systems, traffic sign detections in autonomous cars and brain tumour detections in the health sector.</p><p>Computer vision systems can be built by both rule-based and machine learning approaches.</p><p><strong>Rule-based Computer vision</strong>: Edge detection applies mathematical operations to identify edges and boundaries in images. Template matching is also used for comparing parts of images with a predefined template to find matching object.</p><p><strong>Machine learning-based computer vision</strong>: Traditional ML algorithms like Support Vector Machine (SVM) and Random Forest algorithms are still used to develop computer vision models. However, neural network algorithms like Convolutional Neural Network (CNN) and Vision Transformers are achieving remarkable performance in computer vision.</p><p><strong>Natural language processing (NLP)</strong></p><p>NLP is the field of AI that helps computers process and interpret human text and audio. NLP is applied for developing speech recognition systems to distinguish human speech. It is also applicable in sentiment analysis to find the sentiment or emotions behind texts.</p><p>Examples of NLP applications are Amazon Alexa and Apple Siri.</p><p><strong>Rule-based NLP</strong>: Regular expressions are used to find and extract specific texts for classification. Grammar parsing uses grammar rules to identify the structure of a text. Conditional statements are also used to find relevant words in texts for classification like the rule-based email spam classifier described earlier.</p><p><strong>Machine learning-based NLP:</strong> All the above-mentioned computer vision algorithms are applied in NLP. In addition, Generative Pretrained Transformers (GPT) — models that power LLMs like ChatGPT, are the best performing models for text generation tasks today.</p><p>We just saw that AI systems are not only developed using machine learning techniques but are also developed using the rule-based techniques.</p><p><strong>Is ChatGPT an Artificial Intelligence?</strong></p><p>Simple answer: Yes. ChatGPT is indeed an artificial intelligence system because it performs human tasks and arguably does it better. Thanks to Generative Pretrained Transformer — a deep learning model behind ChatGPT’s powerful capability.</p><p>However, AI is not limited to ChatGPT and other LLM chatbots. There are many AI systems that were not developed with machine learning.</p><p><strong>Distinctions between Rule-Based AI and Machine Learning AI</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Nd3kd0X_59qqUvWEm5JxYQ.png" /><figcaption>Rule-based AI vs Machine learning AI</figcaption></figure><p><strong>Relationship between Artificial Intelligence and Machine learning</strong></p><p>AI is a broader field focused on making computers solve problems by mimicking human intelligence in decision-making. Machine learning is a recent subset of AI that makes decisions based on patterns learnt from data. Rule-based systems like expert systems however, are the old-fashioned subset of AI that make decisions based on hardcoded logics and conditions. Machine learning and the rule-based approach are techniques used for building AI.</p><p>Although the arrival of machine learning has made the rule-based approach less dominant, especially in areas where machine learning excels. However, the rule-based approach is still applicable today especially, in areas where hardcoding rules is enough to get the job done.</p><p><strong>Conclusion</strong></p><p>Techniques do not define an AI system; capability does. Whether an AI system is developed using machine learning or developed using the rule-based technique, it does not matter.</p><p>An AI system is not defined by the mechanism through which it is developed. It is rather defined by its decision-making ability. If a system can make a human-level decision or perform difficult tasks that humans do, or better, it is an AI system.</p><p>All systems developed with machine learning are AI systems, but not all AI systems are developed using machine learning.</p><p>If, in the future, researchers discover a better approach than machine learning for building AI systems, those systems will indeed be considered AI.</p><p><strong><em>About Me</em></strong></p><p><em>As a data and machine learning professional, I write articles that simplify complex technical topics for beginners through practical, real-world examples. I’m open to freelance and full-time opportunities in technical writing, data analytics and machine learning. If you’d like us to work together, feel free to connect on </em><a href="http://linkedin.com/in/ibukunirinyenikan"><em>LinkedIn</em></a><em> or </em><a href="https://wa.link/yhip68"><em>WhatsApp</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=64133514d274" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI Job Applicant System]]></title>
            <link>https://medium.com/@ibukunirinyenikan/ai-job-applicant-system-e3357af2b254?source=rss-7ab15db4bcd2------2</link>
            <guid isPermaLink="false">https://medium.com/p/e3357af2b254</guid>
            <category><![CDATA[job-search]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[automation]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Ibukun Irinyenikan]]></dc:creator>
            <pubDate>Fri, 14 Aug 2026 14:51:01 GMT</pubDate>
            <atom:updated>2026-08-20T10:19:18.143Z</atom:updated>
            <content:encoded><![CDATA[<h4>An AI system that searches and applies for job all on its own</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lH3l_Pan7l2ZSLeORmFEzQ.jpeg" /><figcaption>Credit: Magnific</figcaption></figure><p>There is this frustration that job seekers go through that no one talks about. It is not a rejection email. It is the exhausting process of applying for jobs. You can’t tell when the job that matches your skills is posted. So, you keep watching LinkedIn, Upwork, and Indeed for the next job postings. How about the distraction that comes with checking your phone every now and then, pausing your current task to confirm if a new job has come in?</p><p>If you apply after hundreds of applications have already been sent, your chance is slim. If you saw it too late, you may be applying after the job has already been filled. How about the stress of rewriting every application email, tailoring it to the job requirement for that specific job?</p><p>So, I thought, how about automating this process with Artificial Intelligence (AI) using Large Language Models (LLM). Besides, AI is proven to be excellent at automating repetitive tasks. So, I built myself this system to monitor job postings on Nairaland and automate the application process. If the job posting matches my skill set, it identifies the recruiter’s email and applies on my behalf.</p><p>Instead of manually applying for jobs, the system submits my application for me by:</p><p>· Scraping job postings from Nairaland</p><p>· Understanding what the job is about</p><p>· Deciding if the job matches my skill set</p><p>· Extracting the recruiter’s email</p><p>· Writing a proper application letter for the job</p><p>· Sending the application letter to the recruiter — all without human intervention.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-c-uRyK3qc8O289d2OpJZg.jpeg" /><figcaption>System Architecture (Designed by AI)</figcaption></figure><p><strong>How the System Works</strong></p><p>The system is a Python-based automation pipeline that combines web scraping, LLM, prompt engineering, and email automation. Next is a non-technical explanation of how the system works.</p><p><strong>1.</strong> <strong>Scraping job posts automatically</strong></p><p>The first component uses Selenium, a program that interacts with a website like a human, to scrape job postings from the Nairaland job forum. The system extracts:</p><p>· Job headlines</p><p>· Job links</p><p>· Job description</p><p>Instead of manually watching job postings, the system automatically monitors them for me at intervals.</p><p><strong>2.</strong> <strong>Deciding Relevant Jobs</strong></p><p>After scraping, not every job matches my skills. The job headline is fed to an LLM, which then decides whether the job posting is relevant to my skill sets: data analytics, data science, and writing. If it matches any of them, the system proceeds to the next stage. Otherwise, it skips the job. This stage serves as the decision layer because it mimics what human applicants do mentally before applying for jobs.</p><p><strong>3.</strong> <strong>Extracting Recruiter Emails</strong></p><p>Job postings may contain email addresses other than the recruiter’s email. Using regular expressions to match the recruiter’s email is unreliable because it can lead to submissions to the wrong email address. An LLM is used to analyse the email addresses and determine which one is actually meant for the job application. Once the recruiter’s email is identified, the system proceeds to the next stage. However, if the recruiting process is not email-based, the system skips the job posting.</p><p><strong>4.</strong> <strong>Generating an Application Letter</strong></p><p>Once the system finds a matching job and successfully extracts the recruiter’s email, the LLM will then generate a custom application email in the context of the job requirements, my skills and experience. The LLM is prompted to be concise (150 words max), professional, and highlight relevant skills in the application letter. The content of the application letters depends on the role. A data analytics role receives a different application letter from an AI engineering role.</p><p><strong>5.</strong> <strong>Sending Application Letter to Recruiter</strong></p><p>The final stage uses a Python built-in email system to send the application letter directly from my Gmail to the recruiter’s email. Once this stage is complete, the entire application cycle is complete and fully automated. From discovering the jobs to sending applications, the entire process happens without human intervention.</p><p>The codebase is written in a modular structure to make experimentation with different LLMs easier. The default LLM employed in this project is the OpenAi open-source model, GPT-OSS.</p><p><strong>Features Planned for Future Versions</strong></p><p>Some improvements I plan to add:</p><p>· Including resumes for various jobs so the system can attach my resume to the application email depending on the role.</p><p>· Fine-tuning the model on my write-ups so the model can learn to mimic my writing style when writing application letters for me.</p><p>· Scaling the system for other users to automate their job applications.</p><p>· Expanding the system so that it can apply to other job websites.</p><p>What started as a frustration that comes from applying for jobs has evolved into building an intelligent system that finds opportunity, understands requirement and sends applications all on its own. This project is not just about job applications. It shows how AI can handle repetitive tasks that take our attention every day. As AI continues to advance, we will have more digital personal assistants like this, working silently in the background for us, helping us save time while we focus our attention on tasks that require creative thinking.</p><p><strong>PS</strong>: I will update this post in the future with real-world results showing how the system performs over time.</p><p><strong><em>About Me</em></strong></p><p><em>As a data and machine learning practitioner, I write articles that simplify complex technical topics for beginners through practical, real-world examples. I’m currently open to freelance and full-time opportunities in technical writing, data analytics and machine learning. If you’d like us to work together, feel free to connect on </em><a href="http://linkedin.com/in/ibukunirinyenikan"><em>LinkedIn</em></a><em> or </em><a href="https://wa.link/yhip68"><em>WhatsApp</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e3357af2b254" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why You Don’t Need a Big Laptop To Start]]></title>
            <link>https://medium.com/@ibukunirinyenikan/why-you-dont-need-a-big-laptop-to-start-a-career-in-data-science-and-analytics-271fa7a5180b?source=rss-7ab15db4bcd2------2</link>
            <guid isPermaLink="false">https://medium.com/p/271fa7a5180b</guid>
            <category><![CDATA[gaming-laptop]]></category>
            <category><![CDATA[kaggle]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[google-colab]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Ibukun Irinyenikan]]></dc:creator>
            <pubDate>Wed, 12 Aug 2026 16:21:01 GMT</pubDate>
            <atom:updated>2026-10-07T19:22:50.845Z</atom:updated>
            <content:encoded><![CDATA[<h4>An alternative to powerful laptops for starting a career in data science</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*s4FWHrpcDOeRoswT" /><figcaption>Credit: Unsplash</figcaption></figure><p>Last Christmas, I spoke with a former classmate who had been interested in data science for over two years but had not started. When I asked why, he said he’s been saving to upgrade his Core i3 laptop to a Core i7 before enrolling in a data science course.</p><p>This mindset is common among aspiring data scientists. Many prospects delay or abandon their interest in data science because they believe they need a high-end laptop to begin.</p><p>Interestingly, practising data scientists with powerful laptops often face the same limitation — running out of compute resources when working on compute-intensive real-life projects. Whether you are just starting out or already working in this field, hardware constraints often push many data tasks from laptop to the cloud.</p><p>This article aims to correct the misconception that you need an expensive laptop to start a career in data science while also helping practising data scientists understand when and why platforms like Google Colab and Kaggle Notebook are better suited for their project.</p><h4><strong>Powerful Laptops Are Good but Not Always Enough</strong></h4><p>Not all data science tasks require the cloud. Some data science tasks such as data cleaning, exploratory data analysis, feature engineering and training traditional machine learning models, can be performed on a laptop depending on the size of the dataset.</p><p>For beginners, a mid-range laptop is usually adequate for learning Python, building machine learning models, and completing entry-level projects.</p><p>However, as you work on larger datasets and more complex models, laptop-level compute resources quickly reach their limit. In larger organisations that deal with gigabytes and even terabytes of data, and GPU-intensive deep learning workloads, the cloud becomes essential. For example, fine-tuning a 120-billion parameter variant of GPT-OSS on millions of tokens require compute resources beyond the compute resources of a laptop. This is where the cloud comes as a lifesaver.</p><h4><strong>Where the Cloud Comes in</strong></h4><p>The cloud allows data scientists to access powerful compute resources such as CPU, GPU, TPU, RAM, and storage from cloud servers through the internet. Where local compute resources are insufficient, data scientists often rely on cloud-based platforms like AWS, Azure, and GCP that provide on-demand access to compute resources beyond what a local machine can offer.</p><p>Since beginners may not have the financial capacity to pay for compute on cloud platforms like AWS. Google Colab and Kaggle Notebook are free options that make compute resources available through a browser.</p><p>Instead of delaying your career because you do not have a powerful laptop, you can start learning data science with your current laptop and take advantage of cloud-based platforms when your project demands it.</p><p>In the next section, we discuss the two cloud platforms and highlight where each one best fit depending on your project.</p><h4><strong>Comparing Available Compute Resources for Free Version of Colab and Kaggle Notebook</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0OA3hn4ruTkCdXNyT7kaEg.jpeg" /><figcaption>Colab vs Kaggle Notebook</figcaption></figure><p>Kaggle offers better compute resources than Colab, except for storage, where Colab wins it. However, the resources assigned to users are not constant. Different resources are assigned to users based on the collective users’ workload at that particular time.</p><h4><strong>Other Features of Colab and Kaggle Notebook</strong></h4><p><strong>1.</strong> <strong>Runtime Stability:</strong></p><p>· Colab runtime may reset unexpectedly during model training, which often leads to loss of files and waste of time to start re-training.</p><p>· Kaggle has a stable runtime for long training task and are more likely to complete.</p><p><strong>2.</strong> <strong>GPU Access:</strong></p><p>· Colab may revoke GPU access for over-usage</p><p>· Kaggle provides 30 hours of GPU usage per week without interruption. It resets every Friday.</p><p><strong>3.</strong> <strong>Google Drive Integration:</strong></p><p>· Colab is easy to integrate with Google Drive. Files can be stored permanently on Google Drive.</p><p>· Although Kaggle does not have a straightforward integration with Google Drive. Files can still be downloaded and uploaded to Google Drive from Kaggle notebook. In addition, Kaggle also provides permanent hosting for dataset and models.</p><p><strong>4.</strong> <strong>Paid version:</strong></p><p>· Colab offers more powerful compute resources through its Colab Pro and Colab Pro+ subscriptions.</p><p>· Kaggle is entirely free to use. However, Kaggle allows Colab Pro and Pro+ subscribers to get an additional 15 hours and 30 hours of GPU usage, respectively, every week.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*8YSHcJgAPwO1n9t92QnaIg.jpeg" /><figcaption>Colab(Top) and Kaggle Notebook(down)</figcaption></figure><h4><strong>Google Colab vs Kaggle Notebook — Which Should You Choose?</strong></h4><p>For data science learners, Colab is the better option because of its seamless integration with Google Drive. With Colab, you can explore data, train models and save result to Google Drive.</p><p>For data science professionals, Colab can be the better option where projects require a more powerful GPU than what Kaggle offers, by subscribing to Colab Pro/Pro+.</p><p><strong>Choose </strong><a href="https://colab.research.google.com/"><strong>Colab</strong></a><strong> if:</strong></p><p>· You are learning data science because of its Google Drive integration.</p><p>· You are building a computationally expensive model, and want to invest in Colab Pro/Pro+ to train faster than Kaggle.</p><p>Kaggle Notebook is the better option when you want better compute resources without spending money on compute. Because Kaggle offers more powerful GPU than Colab Free, neural network trains faster on Kaggle.</p><p><strong>Choose </strong><a href="https://www.kaggle.com/"><strong>Kaggle</strong></a><strong> if:</strong></p><p>· You want a better GPU to train neural network, stable runtime for longer training and do not want to spend on Colab subscription.</p><p>· You want to participate in a data science competition.</p><p>Instead of training models on Kaggle and Colab platform, you can train models on VS Code using their compute resources. This allows project files to be easier to manage in one place.</p><p>Next, I am going to explain step by step process of how to use the compute resources of Kaggle and Colab from VS Code.</p><h4><strong>How to Run Google colab on VS Code</strong></h4><p><strong>Step 1: Install Google Colab on VS Code:</strong></p><ol><li>Open VS Code and go to the <strong>Extensions</strong> Tab</li><li>Search for <strong>Google Colab</strong> and install the one published by Google (With a google.com verified badge).</li><li>Ensure you have Jupyter Notebook installed. If not, Install Jupyter Notebook from the <strong>Extensions</strong> tab.</li></ol><p><strong>Step 2: Connect VS Code to a Colab runtime:</strong></p><ol><li>Create and open a notebooke file (colab_remote.ipnyb).</li><li>Click <strong>Select Kernel</strong> in the top-right corner of the notebook.</li><li>Choose <strong>Colab</strong> from the list of option and sign in with your Google account when prompted</li><li>Click <strong>Select Kernel </strong>again and choose <strong>New Colab Server</strong></li><li>A list of compute resources (CPU, GPU, TPU) will show. Choose GPU if you want to train a neural network model. Otherwise, select CPU.</li><li>An input box will open after selecting the resource. Input an alias name for the runtime (say, colab_gpu)and press Enter.</li></ol><p><strong>Summary</strong></p><p><strong>Step 1:</strong></p><p>Open VS Code &gt;&gt; Extenstions &gt;&gt; Install Google Colab</p><p><strong>Step 2:</strong></p><p>Create a colab_remote.ipynb &gt;&gt; Select Kernel &gt;&gt; Colab &gt;&gt; New Colab Server &gt;&gt; GPU &gt;&gt; T4 &gt;&gt; latest &gt;&gt; Input a name for the runtime and press Enter</p><pre>import torch<br>print(&quot;CUDA available:&quot;, torch.cuda.is_available())<br>print(&quot;Device name:&quot;, torch.cuda.get_device_name(0) if torch.cuda.is_available() else &quot;No GPU&quot;)</pre><p>If the code run sucessfully, then the connection is successful.</p><h4><strong>How to Connect Kaggle Notebook to VS Code</strong></h4><p><strong>Step 1: How to copy Kaggle remote server link:</strong></p><ol><li>Sign in to <a href="https://www.kaggle.com/">Kaggle.com</a> and open the Notebook sections.</li><li>Click <strong>Settings </strong>in the Menu bar, then select <strong>Accelerator</strong>. Select GPU, if required. Otherwise, select None.</li><li>Click <strong>Run </strong>in the Menu bar, then select <strong>Kaggle Jupyter Server</strong>.</li><li>The <strong>Kaggle Jupyter Server</strong> panel will appear on right-hand side of the notebook.</li><li>Click <strong>Start session</strong>. Wait till the session starts (Could take a few seconds).</li><li>A remote server link will appear at the bottom of the panel under <strong>VSCode Compatible URL</strong></li><li>Copy the link for step 2.</li></ol><p><strong>Step 2: Connect VS Code to a Kaggle server:</strong></p><ol><li>Go to VS Code and create a new notebook file (kaggle_remote.ipynb).</li><li>Click <strong>Select Kernel</strong> in the top-right corner and select <strong>Existing Jupyter Server</strong>. If you can’t find it, click <strong>Select Another Kernel</strong> and <strong>Existing Jupyter Server</strong> should appear.</li><li>Then paste the remote server URL from kaggle in the box and click Enter.</li><li>After connecting succesfully, Enter a name for the runtime (say, kaggle_gpu) and press Enter.</li><li>Lastly select your preferred promming language.</li></ol><p>If the above code runs successfuly. Congratulate you can now train ML models on your PC using Kaggle compute resources.</p><p>If any of the steps break, it may be due to bad network. Just retry.</p><p><strong>Conclusion</strong></p><p>A powerful laptop makes data science comfortable because dataset will load faster and model will train quickly. However, no matter how powerful a machine is, it will eventually hit its limit when faced with project that requires more compute resources than the laptop can offer.</p><p>A decent laptop is enough to start learning Python, explore data, and build models. As projects grow in size and complexity, platforms like Colab and Kaggle Notebook come in handy. They allow data scientists to scale compute resources when the project demands it without the need to upgrade their machine.</p><p>However, if you can afford a powerful laptop, it will certainly improve workflow efficiency. But it is not a prerequisite for learning data science. Just start with what you have and take advantage of the cloud when your project demands more compute power.</p><p><strong><em>About Me</em></strong></p><p><em>As a data and machine learning practitioner, I write articles that simplify complex technical topics for beginners through practical, real-world examples. I’m currently open to freelance and full-time opportunities in technical writing, data analytics and machine learning. If you’d like us to work together, feel free to connect on </em><a href="http://linkedin.com/in/ibukunirinyenikan"><em>LinkedIn</em></a><em> or </em><a href="https://wa.link/yhip68"><em>WhatsApp</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=271fa7a5180b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Cloud Simplified]]></title>
            <link>https://medium.com/@ibukunirinyenikan/the-cloud-simplified-5886c356f7a1?source=rss-7ab15db4bcd2------2</link>
            <guid isPermaLink="false">https://medium.com/p/5886c356f7a1</guid>
            <category><![CDATA[cloud-services]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[servers]]></category>
            <category><![CDATA[data-center]]></category>
            <category><![CDATA[cloud]]></category>
            <dc:creator><![CDATA[Ibukun Irinyenikan]]></dc:creator>
            <pubDate>Mon, 10 Aug 2026 18:16:01 GMT</pubDate>
            <atom:updated>2026-08-20T10:17:24.674Z</atom:updated>
            <content:encoded><![CDATA[<h4>The cloud explained in a simple way so that you not only understand it but also able to explain it to your friend</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/650/0*sjW6wmSAT0J2pqR4.jpg" /><figcaption>source: Racksolutions</figcaption></figure><p><strong>Why the Cloud?</strong></p><p>In the early days of the internet, owning a website was not as easy as it is today. In those days, you needed a physical server to host your website. These servers were usually housed in an office, and the cost of purchasing and maintaining them was high. On top of that, specialised IT skills were required to maintain them. As website traffic grew, scaling became much more difficult and expensive as additional servers were required to handle the increased demand. For big companies, it sometimes meant owning their own data centre. Today, cloud computing has made it super easy for almost anyone to build and host a website.</p><p>Data is a crucial part of every business. Startups in their early years may store and process data on personal computers (PC). However, as they grow, they often run out of storage and compute resources that a PC offers. This limitation usually pushes them to migrate their data and tasks to the cloud.</p><p>Even on our phones, when we run out of storage, we move our data to services like Google Drive and iCloud.</p><p>This leads us to the question: What is the cloud?</p><p>In this article, I will explain the concept of cloud computing so that you not only understand it but can also explain it easily to your friends.</p><p><strong>What is the cloud?</strong></p><p>The cloud is a network of powerful computers called servers that are connected together, and made accessible to users over the internet. These computers are called servers because they ‘serve’ data to other computers. These servers are responsible for distributing data to our devices, storing it and running applications on the internet. These servers live in <strong>data centres</strong> located in remote locations around the world.</p><p><strong>What is a Data Centre?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lGBmbSqlNsd1pb7s" /><figcaption>AI-generated</figcaption></figure><p>A data centre is a physical building that houses several powerful servers, networking devices, and provides constant electricity to keep them running 24/7.</p><p>Just like laptops use a fan to prevent the CPU and GPU from overheating, data centres have better cooling systems to regulate temperature so that the servers are always running efficiently.</p><p>When we are talking about the cloud, we are not just talking about one server or one data centre. We are talking about the network of several data centres located around the world.</p><p><strong>Why Is It Called the Cloud?</strong></p><p>The name cloud comes from old network diagrams where engineers used a cloud-shaped symbol to show that the inner details of a network were omitted.</p><p>The name is applied to cloud computing today. When you store files or run applications in the cloud, you do not need to know which servers or data centre is handling your task. The cloud provider handles all the inner details for you behind the scenes so that you can simply use the service.</p><p>Think of cloud as a metaphor to show us that just like the atmospheric “cloud” up there is entirely hidden from us, everything that happens in cloud data centres is all hidden from the users.</p><p><strong>What is Cloud Computing?</strong></p><p>Cloud computing is the delivery of computing resources — such as servers, storage, databases, software and networking — over the internet, allowing users to use and pay for resources they use instead of owning and maintaining the underlying infrastructure.</p><p><strong>How Computers store data</strong></p><p>To better understand how the cloud works, it is important to first understand how data is stored and processed on a computer.</p><p>To store data, a PC uses storage devices like Hard Disk Drives (HDD) or Solid-State Drives (SSD). To process data, computer relies on RAM, CPU and sometimes GPU. Together, these components form the basic building block of a computer. Every piece of data is stored on storage devices and processed with processing components like RAM, CPU and GPU.</p><p><strong>Cloud servers are powerful computers</strong></p><p>Cloud servers also have storage and processing components similar to those of a PC, but they are much more powerful and have significantly larger capacity.</p><p>All the data you access on your device over the internet is usually stored on the storage devices of servers and processed by their CPU and RAM in data centres located far from you.</p><p>Your Instagram photos, X tweets, YouTube videos and even this article you are currently reading are all stored in the storage of computers in a remote location. The remote location is not the satellite up there in space, but physical data centres built right here on earth on-ground.</p><p><strong>Is the cloud and the internet the same?</strong></p><p>Many people confuse the cloud with the internet but, they are different. Let’s take a look at this scenario.</p><p>Lharry Tech is a phone store with branches in most states in Nigeria and delivers nationwide. Suppose you order an iPhone 17 on their website. Their delivery man picks up the phone from their nearest store in Lagos and delivers it to you in Akure. Imagine the road paths through which the delivery man travels from Lagos to Akure as the internet and the phone store as the cloud server. The phone cannot get to you without the road and, the store is where the phone comes from. Similarly, the internet is the pathway where data moves to and from the cloud, while the cloud is where the data is stored.</p><p><strong>How the Internet relates to the Cloud</strong></p><p>Let’s take a look at this Google search example to understand how the cloud and internet work together.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MdzlWK963yw-ah8bBChUmw.jpeg" /><figcaption>AI-generated</figcaption></figure><p>1. You make a Google search on your phone/PC.</p><p>2. Your device sends a request through your browser to Google’s cloud server.</p><p>3. The request travels through the internet to the google data centre where its cloud servers are located.</p><p>4. Once the request reaches the Google data centre, the servers process it using their CPUs.</p><p>5. The servers fetch the result from their storage (HDD/SSD)</p><p>6. Google servers send the response back to you through the internet.</p><p>7. Finally, the result appears on your browser.</p><p><strong>Cloud Providers and Cloud Services</strong></p><p>Data centres are usually owned and managed by companies referred to as cloud providers. In cloud computing, the cloud providers are responsible for managing the cloud infrastructures of their data centres.</p><p>Cloud infrastructure refers to the virtual and physical resources that power cloud computing services. They are the combination of software and hardware — storage, servers and networking devices that allow cloud providers to provide on-demand computing, storage and powerful resources to users through the internet. Google, Amazon and Microsoft are the top cloud providers in the cloud computing industry. Each of these companies operates multiple data centres across the world.</p><p>Cloud providers offer cloud services through their cloud platforms:</p><p>· Google through Google Cloud Platform (GCP)</p><p>· Amazon through Amazon Web Services (AWS)</p><p>· Microsoft through Microsoft Azure</p><p>To host a website, store data or build an application with cloud resources, you need to choose a cloud provider and select the cloud service that best fit your need. The kind of cloud service you choose depends on what you want to do. In the next section, we will explore various forms of cloud services so you can understand which one is suitable for your use case.</p><p><strong>Types of Cloud Services</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZTmplWH-KPA1sxSWKEWOrQ.jpeg" /><figcaption>Cloud services</figcaption></figure><p><strong>Infrastructure as a Service (IaaS): </strong>IaaS refers to renting computer resources from cloud servers. Imagine IaaS as renting an office space in a duplex; you furnish the office space to your own taste and move in to start working.</p><p>With IaaS, you are renting a portion of the cloud provider’s infrastructure. That portion is called a Virtual Machine (VM).</p><p>The VM is a software emulation of a computer allocated its own compute resources — HDD, RAM, CPU and sometimes GPU -by the server. The process of achieving that is often known as virtualisation. IaaS allows you to get a virtual machine where you install your operating system, configure the server and install necessary libraries to build your application.</p><p>IaaS gives you the kind of flexibility and control that you get when working on your own laptop while leveraging the power and scalability of cloud infrastructure.</p><p>If you want to build an application that requires more compute resources than your PC can offer, IaaS allows you to rent a more powerful computer over the internet, where you only pay for what you consume. Examples of IaaS are AWS EC2, Google Compute Engine and Azure Virtual Machines.</p><p><strong>Platform as a Service (PaaS): </strong>PaaS is a cloud service where the cloud provider provides a ready-to-use platform for developers to build, deploy and run their applications. Unlike IaaS, developers do not need to worry about installing an operating system, configuring servers and installing libraries. All of these are handled by the cloud providers.</p><p>PaaS allows developers to focus on writing their code and deploying their applications without worrying about the underlying infrastructure.</p><p>Google offers Google App Engine for developers to build and run their applications. Amazon offers Amazon SageMaker for data scientists to build machine learning models and, Microsoft offers Power Apps for users to build applications with little or no coding.</p><p><strong>Software as a Service (SaaS):</strong> SaaS is a cloud service that most of us use every day without even realising it. It is a cloud model where software applications are made available to users through the internet and easily accessible through a web browser. The software vendor hosts their application on a cloud provider’s platform. The application is built to perform a specific function and, the users just use it without worrying about installations, maintenance and security, as they are handled by the provider. It is offered to the users for free or on a subscription.</p><p>Examples of SaaS include GMAIL for sending and receiving email, Google Drive for storing files and social media applications such as Facebook, Instagram and X.</p><p><strong>Function as a Service (FaaS): </strong>FaaS is a serverless cloud service that allows developers to upload pieces of code so that it runs when certain events are triggered. It is not ‘serverless’ because the code does not run on a server; it is ‘serverless’ because the developer is not responsible for running and managing the server. The cloud provider is.</p><p>FaaS helps to reduce cost because the servers only run to execute the code when a particular event is triggered. Unlike other cloud models, where the servers continue to run even when they are idle.</p><p>For instance, if a developer wrote a function on AWS lambda to send a notification message to a seller each time a customer makes an order on their ecommerce store, the code is executed only when an order is made.</p><p><strong>Container as a Service (CaaS): </strong>Unlike PaaS, where developers upload application code for deployment, CaaS allows developers to upload and run containers in the cloud. A container is a standardised unit of software that package application code along with its libraries and dependencies so that the application is able to run on any environment. Examples of CaaS are Amazon ECS, Azure Kubernetes Services and Google Kubernetes Engine.</p><p>CaaS allows developers to upload docker containers to the cloud platform so that they can deploy, manage and scale application inside the container without worrying about underlying servers or architecture.</p><p><strong>Types of cloud deployment</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5PlSA1n0BVFS_gXn9ImC7g.jpeg" /><figcaption>Cloud deployment models</figcaption></figure><p><strong>Public Cloud: </strong>In a public cloud, cloud services are provided over the internet, and servers are shared among multiple organisations and individuals.<strong> </strong>The services are offered through a subscription or pay-for-what-you-use plan. The cloud provider is responsible for the management and security of the infrastructure and services.</p><p><strong>Private Cloud: </strong>In a private cloud, the cloud computing environment is exclusively dedicated to a single organisation. Unlike a public cloud where resources are shared among many users, private cloud offers the organisation greater privacy security and control. It is either hosted on-premise — the organisation’s own data centre or at a third-party cloud provider’s data centres like AWS.</p><p>When private cloud is hosted on-premise, it is usually expensive and complex to manage as it requires skilled professionals to maintain its cloud infrastructure.</p><p><strong>Hybrid Cloud: </strong>Hybrid cloud combines the benefits of both private and public cloud. With a hybrid cloud, an organisation may host sensitive data on their own on-premise data centres to ensure data privacy and security while deploying a public cloud provider to host less-sensitive data such as marketing websites.</p><p><strong>Community Cloud: </strong>In a community cloud, organisations that share common goals come together to build and run a data centre for their collective use. With community cloud, the organisations shares the financial and operational burden. Community cloud may be used by banks, fintech or government organisations.</p><p><strong>Conclusion</strong></p><p>The cloud is not a space satellite in the sky. It is a network of powerful servers housed in data centres that delivers computing resources over the internet when they are needed.</p><p>By eliminating the need of owning and maintaining servers, cloud computing has made powerful computing resources accessible to users, startups and large enterprise.</p><p>The next time you are watching a TikTok live, reading a Medium blog, listening to a podcast on Spotify, understand that this data is not floating in outer space above us; it is stored and running in the storage of physical servers housed in data centres located in a remote location.</p><p><strong><em>About the Author</em></strong></p><p><em>As a data and machine learning practitioner, I write articles that simplify complex technical topics for beginners through practical, real-world examples. I’m currently open to freelance and full-time opportunities in technical writing, data analytics and machine learning. If you’d like us to work together, feel free to connect on </em><a href="http://linkedin.com/in/ibukunirinyenikan"><em>LinkedIn</em></a><em> or </em><a href="https://wa.link/yhip68"><em>WhatsApp</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5886c356f7a1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Data Analysis vs Data Analytics— The Simple Difference Most People Get Wrong]]></title>
            <link>https://medium.com/@ibukunirinyenikan/data-analysis-vs-data-analytics-the-simple-difference-most-people-get-wrong-fc7c989d2005?source=rss-7ab15db4bcd2------2</link>
            <guid isPermaLink="false">https://medium.com/p/fc7c989d2005</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[data-analytics]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[careers]]></category>
            <dc:creator><![CDATA[Ibukun Irinyenikan]]></dc:creator>
            <pubDate>Thu, 02 Jul 2026 15:27:31 GMT</pubDate>
            <atom:updated>2026-08-20T10:14:06.994Z</atom:updated>
            <content:encoded><![CDATA[<h4>A real-world phone store case study that finally makes the difference obvious — even if you have been using the words interchangeably</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tauQf-CxF0kjOIo2MMKY6A.png" /><figcaption>Generated with AI</figcaption></figure><blockquote>“They are the same; data analysis is a verb while data analytics is a noun.”</blockquote><p>That was a colleague’s answer when asked to distinguish between data analysis and data analytics in a Zoom job interview.</p><p>He’s not alone.</p><p>Because the two terms are spelt similarly and sound alike, many people assume they are the same thing.</p><p>Even tech professionals use them interchangeably in everyday conversation. But while they are closely related, they are not exactly the same.</p><p>Confusing the two terms with each other is like saying petrol is the same as petroleum. As petrol is a component of petroleum, data analysis, too is a component of data analytics.</p><p>Data analysis is like a chapter in a novel, while data analytics is the novel.</p><p><strong>What is Data Analysis?</strong></p><p>Data analysis is the process of examining data to find insights and patterns. It answers the questions: <strong>“What happened?”</strong> and <strong>“Why did it happen?”</strong></p><p><strong>What is Data Analytics?</strong></p><p>Data analytics is the bigger picture. It includes data analysis but goes further — using those insights to solve business problems. It covers the entire workflow — from identifying the problem, collecting and cleaning data, analysing it and finally making recommendations to stakeholders. Depending on the project goal, the data analytics workflow may also include forecasting — building and deploying predictive models.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*VJSi6pBjtKkAF0XhUT2F7g.png" /><figcaption>Table by author, designed with AI</figcaption></figure><h4>A Real-World Example to Make the Distinction Clearer</h4><p>Instead of just explaining the difference between the two terms, we will start by using a practical business scenario to clear up the confusion and make the distinction very obvious.</p><h4><strong>The business problem</strong></h4><blockquote>“Our sales are dropping. Nobody wants to buy our phones anymore and I need to know why today.”, your manager said, as he bursts into your office on a Monday morning, looking like someone who had just got off a bad phone call.</blockquote><p>As a data analyst working at Lharry Tech, a phone store in Lagos, your job is to find out what went wrong and recommend to your manager ways to increase sales.</p><p>As we solve the problem, we will also learn the four types of data analytics — descriptive, diagnostic, predictive and prescriptive — and where they fit in the data analytics workflow.</p><p>We will follow a simplified workflow that I used AI to convert into a diagram in the following image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gB633kGlh9cQvic3ptwC6A.png" /><figcaption>Generated with AI</figcaption></figure><h4>Identifying the problem</h4><p>Because this matter is urgent, you immediately paused your current task to start investigating the problem.</p><p>The business problem is clear.</p><p>“Sales are dropping.”</p><p>Defining the problem is the first thing to do before you start solving the problem.</p><h4><strong>Data Collection</strong></h4><p>To investigate the problem, you quickly write SQL queries to retrieve monthly sales records from the database and save them in a spreadsheet.</p><h4>Data Cleaning and Preparation</h4><p>It’s time to make the data ready for analysis. Real-world data is rarely clean. You notice some inconsistencies in the dataset. Some product names were inconsistently spelt and there was a missing value in the <strong>Price</strong> column.</p><p>The product names were corrected, the missing value in the <strong>Price</strong> column was filled after confirming the appropriate price from the manager and a <strong>Revenue</strong> column was created.</p><p>Revenue is calculated as: <em>Quantity * Price.</em></p><p>The following table is the clean version of the dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/461/1*Z7NdRSi4oUwao2uthPx6AQ.png" /><figcaption>Phone store dataset after cleaning and preparation</figcaption></figure><p>At this point, the data is now ready for analysis.</p><p>Up to this point, we have identified the business problem, collected the data and also cleaned it in preparation for analysis. Notice that we haven’t analyse the data yet. Everything we have done so far is part of the broader data analytics process.</p><h4><strong>Data Analysis</strong></h4><p>This is where the actual <strong>data analysis</strong> takes place. At this stage, cleaned data is examined using statistical methods to extract insights, identify patterns, and understand relationships.</p><p>Data analysis aligns well with descriptive and diagnostic analytics. Data analysis uses <strong>descriptive analytics </strong>to find what happened and <strong>diagnostic analytics</strong> to know why it happened.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*QQtHbrMyVqZZik5JokGyHw.png" /><figcaption>Monthly revenue including next month’s forecast</figcaption></figure><p><strong>What happened? — Descriptive Analytics</strong></p><p>After examining the dataset, you discover that:</p><p><strong>·</strong> Sales revenue increased during discount periods.</p><p><strong>·</strong> The units of phones sold increased significantly during discount periods.</p><p><strong>·</strong> Higher revenue is generated from the iPhone because it is more expensive than Samsung even though it records lower unit sales.</p><p><strong>·</strong> The business made the highest revenue during discount periods.</p><p><strong>·</strong> Revenue started dropping in April when the discount was removed.</p><p><strong>Why is revenue declining? — Diagnostic Analytics</strong></p><p>After discovering what happened, the manager also needs to know why revenue is declining.</p><p>On comparing revenue generated during the discount and non-discount periods, you discovered that higher revenue was generated during the discount periods.</p><p>To confirm that the increase in revenue was not due to chance, you ran a statistical test. The result shows that the increase in revenue during discount periods is statistically significant, suggesting that the observed increase in revenue is unlikely due to random variation. This shows that discounts are important factors contributing to revenue performance.</p><h4><strong>Forecasting the future — Predictive Analytics</strong></h4><p>Historical data becomes more valuable when used to predict the future. After discovering what happened in the past months, you now build a model to predict what happens in the following months.</p><p>The model predictions are represented by the dotted lines in the line chart above. They represent the revenue forecast from May to July.</p><p>Your forecasts show that sales revenue may start increasing from May if discounts are re-introduced.</p><p>At this stage, we used <strong>predictive analytics</strong> — a kind of data analytics, to predict what may likely happen in the future.</p><h4><strong>Recommendation — Prescriptive Analytics</strong></h4><p>After discovering why sales are reducing and what is likely to happen in the coming months through forecasting, you can now make your recommendation to the manager on what to do next.</p><p>Based on your findings, you recommend that:</p><p><strong>·</strong> The store should introduce discounts during low-sales periods.</p><p><strong>·</strong> Discounts should be more focused on iPhones because iPhones’ higher pricing makes them generate more revenue.</p><p>Recommending the next action goes beyond data analysis; it is <strong>prescriptive analytics</strong>.</p><h4>Data Analysis vs Data Analytics — The key difference</h4><p>Everything we did to solve this business problem, from <strong>identifying the problem </strong>to making <strong>recommendations</strong>, is part of the data analytics process. <strong>Data analysis</strong> is just one stage of the data analytics process.</p><p>As a data analyst, next time you are asked the question, what are you into? Don’t answer “data analysis”; instead, say “data analytics.” As a data analyst, you will identify the problem, collect and clean data, uncover patterns, build dashboards, perhaps build/deploy machine learning models, and even make recommendations to stakeholders.</p><p>Data analytics is your field. It describes everything you do — the entire project workflow. Data analysis is the aspect of your job where you examine data to uncover insights.</p><p>If you want to become a data analyst, you must learn data analytics, not just data analysis.</p><h4><strong>Why are Data Analysis and Data Analytics often mixed up?</strong></h4><p>Many people confuse data collection, data cleaning, building dashboards, recommendations among many others with data analysis.</p><p>Why?</p><p>Without data collection and data cleaning, analysing the data is impossible. They are the roads that lead to analysing the data. This is why many people group them under the “<strong>data analysis process</strong>” even though they are not <strong>data analysis </strong>itself.</p><p>So, today, when many people use the term data analysis, they use it to describe everything data analysts do.</p><p>Strictly speaking, only the aspect where the data is examined to uncover insights is <strong>data analysis. </strong>All other tasks performed before and after analysis can be classified under <strong>data analytics.</strong></p><h4><strong>Final Takeaway</strong></h4><p>So, next time you hear a colleague use the term <strong>data analysis</strong> to describe a data project or the entire process/field of using data to solve problems, don’t frown at them.</p><p>Just know that they actually meant “<strong>data analytics</strong>”.</p><p>But when they use the word to just mean unveiling insight from data — calculating average sales, creating bar charts, etc. in Excel, they indeed meant “<strong>data analysis</strong>”.</p><p>If you are analysing data, it is <strong>data analysis;</strong> otherwise, it is <strong>data analytics</strong>. And don’t forget that <strong>data analysis</strong> is also a subset of <strong>data analytics</strong>.</p><p>If this article clarifies the difference, consider sharing it with a friend who might find it helpful. I’d also love to hear your feedback and suggestions in the comment section. Thanks for reading.</p><p><strong><em>About the Author</em></strong></p><p><em>As a data and machine learning practitioner, I write articles that simplify complex technical topics for beginners through practical, real-world examples. I’m currently open to freelance and full-time opportunities in technical writing, data analytics and machine learning. If you’d like us to work together, feel free to connect on </em><a href="http://linkedin.com/in/ibukunirinyenikan"><em>LinkedIn</em></a><em> or </em><a href="https://wa.link/yhip68"><em>WhatsApp</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fc7c989d2005" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>