<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mzameli Mashiyi on Medium]]></title>
        <description><![CDATA[Stories by Mzameli Mashiyi on Medium]]></description>
        <link>https://medium.com/@khathithi?source=rss-e15173b70f2a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*xayxkahgsQkhYUR2</url>
            <title>Stories by Mzameli Mashiyi on Medium</title>
            <link>https://medium.com/@khathithi?source=rss-e15173b70f2a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 21:15:05 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@khathithi/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[From raw maritime signals to actionable intelligence: Building an end-to-end machine learning…]]></title>
            <link>https://medium.com/@khathithi/from-raw-maritime-signals-to-actionable-intelligence-building-an-end-to-end-machine-learning-f667a8e4998f?source=rss-e15173b70f2a------2</link>
            <guid isPermaLink="false">https://medium.com/p/f667a8e4998f</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Mzameli Mashiyi]]></dc:creator>
            <pubDate>Wed, 11 Nov 2026 21:16:00 GMT</pubDate>
            <atom:updated>2026-11-11T21:16:00.782Z</atom:updated>
            <content:encoded><![CDATA[<h3>From raw maritime signals to actionable intelligence: Building an end-to-end machine learning pipeline for vessel classification.</h3><p><strong>By Mzameli Mashiyi | Computer Engineering · Data Science · Artificial Intelligence</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DxfKf3spBmP3VHshhD7-bA.jpeg" /></figure><h3>Introduction: The ocean is a data environment</h3><p>The world’s oceans are among the most important economic and ecological resources on the planet. They facilitate international trade, support fishing industries, sustain coastal communities and provide livelihoods to millions of people.</p><p>However, monitoring what happens across these vast maritime environments remains a considerable challenge.</p><p>Illegal, Unreported and Unregulated (IUU) fishing is one of the most persistent threats to marine ecosystems and ocean economies. An estimated 11 to 26 million tons of fish are removed from the world’s oceans through IUU fishing every year. Its consequences extend beyond the depletion of marine resources, affecting food security, coastal livelihoods and the sustainability of fisheries.</p><p>In South Africa, the consequences are particularly concerning. Organized syndicates have contributed to the collapse of wild abalone resources and placed significant pressure on West Coast rock lobster populations.</p><p>The challenge is not simply identifying vessels at sea. It is determining what those vessels are actually doing and whether the information they transmit can be trusted.</p><p>This is where data science and machine learning become particularly interesting.</p><p>I explored this problem through a research project titled MACHINE LEARNING CLASSIFICATION OF VESSEL TYPES FROM AUTOMATIC IDENTIFICATION SYSTEM DATA: A TRANSFERABLE APPROACH TO DETECTING ILLEGAL, UNREPORTED AND UNREGULATED FISHING IN SOUTH AFRICAN WATERS.</p><p>The objective was to develop an end-to-end machine learning pipeline capable of independently inferring a vessel’s operational class using the physical and movement-related characteristics contained in its Automatic Identification System (AIS) messages.</p><p>The broader motivation is to contribute towards more intelligent maritime domain awareness, with potential applications in South African waters.</p><h3>1. Understanding the problem: What is AIS?</h3><p>The Automatic Identification System (AIS) is a ship-borne transponder system that allows vessels to broadcast information about their identity, position and movement.</p><p>Depending on the message type and available fields, AIS data can contain information such as:</p><ul><li>Maritime Mobile Service Identity (MMSI).</li><li>Navigational status.</li><li>Speed over ground (SOG).</li><li>Course over ground (COG).</li><li>Heading.</li><li>Vessel type.</li><li>Vessel dimensions.</li><li>Draught.</li></ul><p>These attributes provide a valuable source of information for maritime monitoring.</p><p>However, there is an important limitation: the vessel-type field is self-declared.</p><p>This means that the classification transmitted by a vessel is not necessarily an independently verified representation of its actual operational type. Information can be missing, incorrectly entered or deliberately manipulated.</p><p>Consider a scenario where a vessel broadcasts its identity and position but declares itself as a cargo vessel when its physical characteristics and movement patterns are more consistent with a fishing vessel.</p><p>A monitoring system that relies exclusively on the declared vessel type may fail to identify this inconsistency.</p><p><strong>The research question therefore becomes: Can we use machine learning to infer vessel type independently from the characteristics of AIS data?</strong></p><p>Rather than trusting the classification provided by the vessel, we can develop models that learn relationships between vessel dimensions, movement patterns and operational classes.</p><h3>2. The dataset: Learning from real maritime traffic</h3><p>One of the initial challenges was data availability.</p><p>At the time of this research, an open, labelled South African AIS dataset suitable for supervised vessel classification was not available.</p><p>To establish and evaluate the methodology, I used publicly available AIS data from the Danish Maritime Authority.</p><p>The dataset covered vessel traffic through the Kattegat Strait, a maritime passage between Denmark and Sweden.</p><p>The dataset contained:</p><ul><li><strong>358,351 AIS records</strong></li><li><strong>3,894 individual vessels</strong></li><li><strong>17 original ship-type categories</strong></li><li>Physical and kinematic vessel attributes.</li></ul><p>The dataset provided an opportunity to investigate classification under real maritime operating conditions rather than relying on artificially generated observations.</p><p>An important distinction is that the Kattegat Strait dataset was used to develop and validate the methodology. Its results should not automatically be interpreted as measured performance in South African waters. That requires further local validation.</p><h3>3. Building the end-to-end data science pipeline</h3><p>Rather than approaching the problem as an isolated model-training exercise, I developed a complete data science pipeline.</p><p>The objective was to establish a workflow that could take raw AIS records, process them, store them efficiently, extract meaningful information, train machine learning models and expose predictions through an API and interactive dashboard.</p><p>The architecture consisted of eight major stages.</p><h3>Stage 1: Data ingestion</h3><p>The process begins with raw AIS data loaded from CSV files.</p><p>At this stage, the objective is to bring the source data into the processing environment while retaining the information required for subsequent analysis.</p><p>Python was used as the primary programming language.</p><h3>Stage 2: ETL and data preprocessing</h3><p>Raw maritime data requires careful preprocessing before it can be used for machine learning.</p><p>The ETL pipeline incorporated:</p><ul><li>Data validation.</li><li>Data cleaning.</li><li>Missing-value handling.</li><li>Data transformation.</li><li>Standardization.</li></ul><p>The purpose was to improve data consistency and prepare the observations for storage and analysis.</p><h3>Stage 3: Database design</h3><p>After preprocessing, the cleaned data was stored in PostgreSQL.</p><p>A relational database structure was developed around several key entities:</p><ul><li>ships</li><li>ship_types</li><li>ais_observations</li><li>navigational_statuses</li></ul><p>This separation makes it possible to manage vessel information, classification categories and individual AIS observations more systematically.</p><p>PostgreSQL also provided an efficient environment for querying and aggregating maritime observations.</p><h3>Stage 4: SQL analytics</h3><p>Before developing the machine learning models, I used SQL to explore the data and investigate its underlying structure.</p><p>Example analytical questions included:</p><ul><li>What is the distribution of ship types?</li><li>What is the average speed for each vessel category?</li><li>Are there missing observations?</li><li>How do vessel characteristics differ between operational classes?</li><li>How does vessel activity change over time?</li></ul><p>This stage was important because machine learning should not be treated as a substitute for understanding the underlying data.</p><p>SQL provided the foundation for structured exploration and feature extraction.</p><h3>Stage 5: Exploratory data analysis</h3><p>Exploratory Data Analysis (EDA) was used to understand the distributions, relationships and potential anomalies within the dataset.</p><p>The analysis focused on:</p><ul><li>Feature distributions.</li><li>Correlations.</li><li>Outliers.</li><li>Patterns across vessel categories.</li><li>Data quality.</li></ul><p>Visualisations helped identify relationships that could potentially distinguish one vessel category from another.</p><p>For example, the physical dimensions of a vessel may provide useful information about its likely operational class, while movement characteristics can offer additional behavioural context.</p><h3>Stage 6: Feature engineering and machine learning</h3><p>This was the central modelling component of the project.</p><p>The original 17 AIS ship types were mapped into seven operational classes.</p><p>I then engineered 14 features, divided into two broad groups:</p><p><strong>Geometric features</strong></p><p>These represent the physical characteristics of a vessel, including its dimensions.</p><p><strong>Kinematic features</strong></p><p>These represent the movement-related characteristics of a vessel, including speed and course-related information.</p><p>The distinction was important because vessel classification is not exclusively a behavioral problem. The physical structure of a vessel can contain substantial information about its intended operational role.</p><p>Three supervised learning algorithms were evaluated:</p><ol><li>Logistic Regression.</li><li>Random Forest.</li><li>XGBoost.</li></ol><p>The models were trained and evaluated to determine how effectively they could distinguish between the seven operational classes.</p><h3>Stage 7: Model deployment</h3><p>A trained machine learning model is considerably more useful when its predictions can be accessed by other applications.</p><p>For this reason, I developed an API using FastAPI.</p><p>The deployment architecture exposed endpoints for:</p><ul><li>Predicting vessel type.</li><li>Retrieving vessel information using MMSI.</li><li>Accessing summary statistics.</li></ul><p>Swagger UI provided an interactive interface for testing the API endpoints.</p><h3>Stage 8: Interactive dashboard</h3><p>The final component was an interactive Streamlit dashboard.</p><p>The dashboard was designed to make the results accessible beyond the model-training environment.</p><p>Its components included:</p><ul><li>Data overview.</li><li>Ship-type distribution.</li><li>Model performance.</li><li>Feature importance.</li><li>Vessel prediction interface.</li></ul><p>The intention was to connect the analytical and machine learning components to a practical interface through which users could explore the data and obtain predictions.</p><h3>4. The most important engineering decision: Preventing data leakage</h3><p>One of the most important aspects of this research was the model validation strategy.</p><p>AIS datasets contain multiple observations associated with the same vessel. This creates a potential problem when randomly splitting individual records into training and testing datasets.</p><p>Imagine that a vessel contributes hundreds of AIS observations.</p><p>If some observations from that vessel appear in the training dataset while others appear in the test dataset, the model may effectively encounter a vessel it has already learned from.</p><p>This can inflate performance measurements and create an unrealistic representation of how the model will perform on previously unseen vessels.</p><p>To address this, I implemented <strong>vessel-grouped validation</strong>.</p><p>Instead of randomly splitting individual records, all observations belonging to a particular vessel were kept entirely within either the training partition or the test partition.</p><p>This ensured that the evaluation measured the model’s ability to classify vessels it had not encountered during training.</p><p>The difference was significant.</p><p>A conventional random record-level split increased XGBoost’s macro F1-score from 0.781 to 0.960.</p><p>That is a substantial difference.</p><p>It demonstrates why validation methodology matters just as much as algorithm selection. A model can achieve impressive evaluation scores while still providing an unrealistic estimate of its generalisation capability.</p><p>In applied machine learning, the objective should not be to produce the highest possible score. It should be to produce a score that reflects how the system is expected to perform in practice.</p><h3>5. Model performance: Which algorithm performed best?</h3><p>The research compared three supervised learning algorithms.</p><p>The results on previously unseen vessels were:</p><p>AlgorithmAccuracyMacro F1Logistic Regression66.3%Not reported hereRandom Forest86.2%0.792XGBoost86.3%0.781</p><p>XGBoost achieved the highest accuracy, although Random Forest achieved a slightly higher macro F1-score.</p><p>This distinction is important.</p><p>Accuracy measures the proportion of correctly classified observations, while macro F1 calculates the F1-score for each class and gives every class equal weight.</p><p>Consequently, macro F1 provides an additional perspective on performance when the classification problem contains multiple categories.</p><p>The results indicate that both ensemble-based algorithms substantially outperformed Logistic Regression.</p><p>However, the difference between XGBoost and Random Forest was marginal. It would therefore be inaccurate to conclude that XGBoost was unequivocally superior across all evaluation criteria.</p><h3>Fishing vessel detection</h3><p>An especially important result concerned fishing vessels.</p><p>The multiclass classification model achieved an F1-score of <strong>0.838 for fishing vessels</strong>.</p><p>A separate binary fishing-vessel detector achieved a precision-recall Area Under the Curve (PR-AUC) of <strong>0.898</strong>.</p><p>This is particularly relevant to the broader motivation behind the research.</p><p>If a maritime monitoring system can independently estimate whether a vessel is likely to belong to a fishing category, that information could support additional screening and risk-assessment processes.</p><p>It does not, by itself, establish that a vessel is engaged in illegal fishing. Rather, it provides another source of information for maritime analysts.</p><h3>6. What did the features reveal?</h3><p>Perhaps one of the most interesting findings was the difference between geometric and kinematic information.</p><p>I evaluated the contribution of these two feature groups independently.</p><p>The results were:</p><p>Feature groupMacro F1Geometry only0.720Kinematics only0.355</p><p>The geometry-only model substantially outperformed the kinematics-only model.</p><p>This suggests that the physical characteristics of a vessel provide considerably more discriminative information for this classification task than movement-related attributes alone.</p><p>In other words, the dimensions of a ship can tell us a great deal about its likely operational category.</p><p>Kinematic features still provide useful information, particularly when combined with physical characteristics. However, the results suggest that vessel geometry should receive considerable attention when designing AIS-based classification systems.</p><p>This finding also demonstrates the importance of feature engineering.</p><p>Selecting an algorithm is only one part of developing a successful machine learning system. Understanding which features contain meaningful information can be equally important.</p><h3>7. From the Kattegat Strait to South African waters</h3><p>The broader motivation for this research is its potential application in South African maritime domain awareness.</p><p>South Africa has an extensive Exclusive Economic Zone (EEZ), valuable marine resources and significant maritime security challenges.</p><p>Monitoring such a large maritime environment requires effective ways of collecting, processing and analyzing information about vessel activity.</p><p>AIS provides one important source of information, but relying exclusively on self-declared vessel classifications introduces uncertainty.</p><p>The proposed next step is to investigate transferring the developed methodology to South African maritime environments.</p><p>This would involve integrating multiple information sources, including:</p><ul><li>National AIS data sources.</li><li>Vessel registration information.</li><li>Physical vessel characteristics.</li><li>Satellite-derived maritime observations.</li><li>Analyst feedback.</li></ul><p>An interesting opportunity exists in combining AIS information with data from CPUT’s own MDASat nanosatellite constellation.</p><p>Satellite-based maritime observations could provide additional information to complement terrestrial AIS reception and help analysts investigate vessel activity beyond the coverage of coastal receiving infrastructure.</p><p>A future system could compare a vessel’s declared classification against the classification inferred by a machine learning model.</p><p>For example:</p><p>A vessel declares itself as a cargo ship, but its dimensions and other observed characteristics are more consistent with a fishing vessel.</p><p>The system could flag this inconsistency for further investigation.</p><p>Such a framework could support:</p><ul><li>Maritime domain awareness.</li><li>Fisheries monitoring.</li><li>Identification of potentially misclassified vessels.</li><li>Risk-based vessel screening.</li><li>Protection of marine resources.</li></ul><p>The objective is not to replace maritime authorities or human analysts. It is to provide them with additional analytical tools that can help prioritise investigations.</p><h3>8. Building beyond a machine learning model</h3><p>One of the biggest lessons from this project is that developing a machine learning model is only one component of an engineering solution.</p><p>A practical data-driven system requires much more.</p><p>It needs reliable data ingestion, robust preprocessing, database design, analytical querying, feature engineering, model evaluation, API development and an interface through which users can interact with the results.</p><p>This is why I approached the project as an end-to-end engineering pipeline.</p><p>The combination of Python, PostgreSQL, SQLAlchemy, pandas, NumPy, Scikit-learn, XGBoost, FastAPI, Streamlit and Docker provided a foundation for building a reproducible and deployable system.</p><p>Each component served a purpose within the broader architecture.</p><p>More importantly, the project reinforced a principle that I consider fundamental to applied artificial intelligence:</p><p><strong>A model is only as useful as the engineering system that supports it.</strong></p><h3>9. Conclusion: Smarter maritime data, safer oceans</h3><p>This research demonstrates that supervised machine learning can provide a useful methodology for independently classifying vessel types from AIS information.</p><p>Using 358,351 AIS records from 3,894 vessels, the developed pipeline mapped 17 original ship types into seven operational classes and evaluated three supervised learning algorithms.</p><p>XGBoost and Random Forest achieved accuracies of 86.3% and 86.2%, respectively, when evaluated on previously unseen vessels using vessel-grouped validation.</p><p>The research also demonstrated that geometric features were substantially more informative than kinematic features when considered independently.</p><p>However, the most important contribution extends beyond the classification scores.</p><p>It is the development of a complete methodology that can potentially be transferred and further validated within South African maritime environments.</p><p>There is an opportunity to combine machine learning, national AIS infrastructure, satellite observations and human expertise to strengthen maritime intelligence.</p><p>For a country with valuable marine resources and significant ocean-based economic activity, the ability to turn maritime data into actionable intelligence could have considerable value.</p><p>The ocean is vast, but the information it generates is becoming increasingly accessible.</p><p>The challenge is learning how to use that information responsibly, intelligently and effectively.</p><p><strong>Smarter maritime data. Better intelligence. Safer oceans.</strong></p><p><em>Research note: This project was developed using open AIS data from the Danish Maritime Authority covering vessel traffic in the Kattegat Strait. The reported results represent evaluation on that dataset and do not constitute validation on South African maritime traffic. Local validation and integration with authoritative vessel records are necessary before operational deployment.</em></p><p><strong>Tags:</strong> Machine Learning · Data Science · Maritime Technology · Artificial Intelligence · AIS · South Africa · Maritime Domain Awareness · XGBoost · Python · Ocean Economy</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f667a8e4998f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Beginner Data Science Projects: A Practical Guide to Getting Started.]]></title>
            <link>https://medium.com/@khathithi/beginner-data-science-projects-a-practical-guide-to-getting-started-9eb60e97eb2d?source=rss-e15173b70f2a------2</link>
            <guid isPermaLink="false">https://medium.com/p/9eb60e97eb2d</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-analysis]]></category>
            <dc:creator><![CDATA[Mzameli Mashiyi]]></dc:creator>
            <pubDate>Wed, 06 May 2026 22:30:25 GMT</pubDate>
            <atom:updated>2026-05-06T22:30:25.194Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oNu0U0mlmF_RxyJk5rhBqg.png" /></figure><p>Data science can feel overwhelming at first — there are tools, algorithms, statistics, and programming concepts all competing for attention. The best way to cut through that noise is simple: build projects. Projects turn theory into skill, help you build a portfolio, and teach you how real-world data behaves (spoiler: it’s messy).</p><p>This article explores beginner-friendly data science projects that are practical, achievable, and valuable for learning.</p><p><strong>Why Projects Matter in Data Science</strong></p><p>Before diving into examples, it’s important to understand why projects are essential:</p><ul><li>They reinforce concepts like data cleaning, visualization, and modeling</li><li>They build problem-solving skills</li><li>They demonstrate your abilities to employers</li><li>They expose you to real-world data challenges</li></ul><p>Even simple projects can teach powerful lessons if done properly.</p><p><strong>1. Exploratory Data Analysis (EDA) Project</strong></p><p>Goal: Understand and summarize a dataset using statistics and visualization.</p><p>Example Ideas:</p><ul><li>Analyze student performance data</li><li>Explore COVID-19 trends</li><li>Study housing prices</li></ul><p>Skills You’ll Learn:</p><ul><li>Data cleaning (handling missing values)</li><li>Summary statistics (mean, median, standard deviation)</li><li>Visualization (histograms, box plots, scatter plots)</li></ul><p>Tools:</p><ul><li>Python (Pandas, Matplotlib, Seaborn)</li></ul><p>Why It’s Great for Beginners:</p><p>EDA is the foundation of all data science work. It teaches you how to “read” data before making predictions.</p><p><strong>2. Data Visualization Dashboard</strong></p><p>Goal: Communicate insights through visuals.</p><p>Example Ideas:</p><ul><li>Sales dashboard for a fictional company</li><li>Personal finance tracker</li><li>Population growth visualization</li></ul><p>Skills You’ll Learn:</p><ul><li>Data storytelling</li><li>Chart selection (bar, line, pie, etc.)</li><li>Dashboard design</li></ul><p>Tools:</p><ul><li>Tableau, Power BI, or Python (Plotly)</li></ul><p>Beginner Tip:</p><p>Focus on clarity, not complexity. A clean, simple dashboard is more powerful than a cluttered one.</p><p><strong>3. Predictive Modeling Project (Regression)</strong></p><p>Goal: Predict a continuous value.</p><p>Example Ideas:</p><ul><li>Predict house prices</li><li>Forecast sales revenue</li><li>Estimate fuel consumption</li></ul><p>Skills You’ll Learn:</p><ul><li>Linear regression</li><li>Feature selection</li><li>Model evaluation (R², MAE, RMSE)</li></ul><p>Tools:</p><ul><li>Scikit-learn</li></ul><p>Why It’s Useful:</p><p>Regression is one of the most widely used techniques in business and economics.</p><p><strong>4. Classification Project</strong></p><p>Goal: Predict categories or classes.</p><p>Example Ideas:</p><ul><li>Spam email detection</li><li>Customer churn prediction</li><li>Disease diagnosis (basic datasets)</li></ul><p>Skills You’ll Learn:</p><ul><li>Logistic regression</li><li>Decision trees</li><li>Confusion matrices and accuracy metrics</li></ul><p>Tools:</p><ul><li>Scikit-learn</li></ul><p>Beginner Insight:</p><p>Start simple. Don’t jump into complex models like neural networks right away.</p><p><strong>5. Data Cleaning Project</strong></p><p>Goal: Transform messy data into usable form.</p><p>Example Ideas:</p><ul><li>Clean a scraped dataset</li><li>Standardize inconsistent formats</li><li>Handle missing or duplicate values</li></ul><p>Skills You’ll Learn:</p><ul><li>Data wrangling</li><li>Handling null values</li><li>String and date manipulation</li></ul><p>Why This Matters:</p><p>In real-world data science, 80% of time is spent cleaning data.</p><p><strong>6. SQL Data Analysis Project</strong></p><p>Goal: Query and analyze data stored in databases.</p><p>Example Ideas:</p><ul><li>Analyze e-commerce transactions</li><li>Customer segmentation queries</li><li>Sales performance reports</li></ul><p>Skills You’ll Learn:</p><ul><li>SELECT, JOIN, GROUP BY</li><li>Aggregations</li><li>Data filtering</li></ul><p>Tools:</p><ul><li>MySQL, PostgreSQL, SQLite</li></ul><p>Career Tip:</p><p>SQL is one of the most demanded skills in data roles.</p><p><strong>7. Time Series Analysis Project</strong></p><p>Goal: Analyze data over time.</p><p>Example Ideas:</p><ul><li>Stock price trends</li><li>Weather data analysis</li><li>Website traffic patterns</li></ul><p>Skills You’ll Learn:</p><ul><li>Trend and seasonality</li><li>Moving averages</li><li>Forecasting basics</li></ul><p>Tools:</p><ul><li>Pandas, statsmodels</li></ul><p><strong>8. Web Scraping Project</strong></p><p>Goal: Collect data from websites.</p><p>Example Ideas:</p><ul><li>Scrape product prices</li><li>Gather job listings</li><li>Collect news headlines</li></ul><p>Skills You’ll Learn:</p><ul><li>HTML structure</li><li>Data extraction</li><li>Automation</li></ul><p>Tools:</p><ul><li>BeautifulSoup, Selenium</li></ul><p><strong>9. End-to-End Project (Best Portfolio Piece)</strong></p><p>Goal: Combine everything into one project.</p><p>Example Workflow:</p><ol><li>Collect data</li><li>Clean and preprocess</li><li>Perform EDA</li><li>Build a model</li><li>Visualize results</li><li>Present findings</li></ol><p>Example Idea:</p><p>Predict customer churn and present insights in a dashboard.</p><p>Why It’s Powerful:</p><p>This shows you can handle a complete data science pipeline, which is exactly what employers want.</p><p>Tips for Beginners</p><ul><li>Start small — don’t aim for perfection</li><li>Use publicly available datasets (e.g., Kaggle)</li><li>Document your work (GitHub + README)</li><li>Focus on explaining your results, not just coding</li><li>Revisit and improve your projects over time</li></ul><p>Final Thoughts</p><p>You don’t need advanced math or complex algorithms to start in data science. What you need is consistency and practical experience. Beginner projects — when done thoughtfully — can build a strong foundation and open doors to more advanced work.</p><p>Start with one project, finish it, learn from it, and then move on to the next. That’s how real progress happens.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9eb60e97eb2d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Agentic AI: The Next Evolution of Artificial Intelligence.]]></title>
            <link>https://medium.com/@khathithi/agentic-ai-the-next-evolution-of-artificial-intelligence-02482a752199?source=rss-e15173b70f2a------2</link>
            <guid isPermaLink="false">https://medium.com/p/02482a752199</guid>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[prompt-engineering]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Mzameli Mashiyi]]></dc:creator>
            <pubDate>Mon, 27 Apr 2026 12:44:46 GMT</pubDate>
            <atom:updated>2026-04-27T12:44:46.204Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Bb3LWz9XYlwFUa6PsyUxGA.png" /></figure><p>Artificial Intelligence is no longer just about prediction — it’s about action. The rise of <em>Agentic AI</em> marks a significant shift from systems that simply analyse data to systems that can make decisions, plan tasks, and execute them autonomously. This evolution is redefining how we interact with technology, pushing AI closer to functioning as a true digital “agent” rather than just a tool.</p><p><strong>What is Agentic AI?</strong></p><p>Agentic AI refers to AI systems designed to operate with a degree of autonomy. Unlike traditional AI models that require explicit instructions for every step, agentic systems can:</p><ul><li>Understand goals</li><li>Break them into smaller tasks</li><li>Execute those tasks</li><li>Adapt based on feedback or changing conditions</li></ul><p>In simple terms, instead of asking AI <em>“what should I do?”</em>, you tell it <em>“achieve this outcome”</em> — and it figures out the steps.</p><p><strong>How Agentic AI Works</strong></p><p>Agentic AI systems are typically built on a combination of:</p><p><strong>1. Large Language Models (LLMs)</strong></p><p>These provide reasoning and understanding capabilities, allowing the agent to interpret instructions and generate responses.</p><p><strong>2. Memory Systems</strong></p><p>Agents can store and recall past interactions, enabling context-aware decisions and long-term planning.</p><p><strong>3. Tool Integration</strong></p><p>Agentic systems can interact with APIs, databases, or software tools — allowing them to perform real-world actions (e.g., sending emails, querying data, or deploying code).</p><p><strong>4. Planning and Reasoning Loops</strong></p><p>Instead of a single response, agents operate in cycles:</p><ul><li>Plan → Act → Observe → Reflect → Repeat</li></ul><p>This loop enables continuous improvement and adaptation.</p><p><strong>Key Characteristics of Agentic AI</strong></p><p>What sets Agentic AI apart?</p><ul><li><strong>Autonomy:</strong> Minimal human intervention once a goal is set</li><li><strong>Goal-Oriented Behavior:</strong> Focused on outcomes, not just outputs</li><li><strong>Adaptability:</strong> Can adjust strategies dynamically</li><li><strong>Multi-step Reasoning:</strong> Handles complex workflows</li><li><strong>Tool Usage:</strong> Executes actions beyond text generation</li></ul><p><strong>Real-World Applications</strong></p><p>Agentic AI is already reshaping multiple industries:</p><p><strong>1. Data Science &amp; Analytics</strong></p><p>Agents can:</p><ul><li>Clean datasets</li><li>Run models</li><li>Interpret results</li><li>Generate reports</li></ul><p>Instead of manually coding each step, a data scientist can delegate entire workflows.</p><p><strong>2. Software Development</strong></p><p>AI agents can:</p><ul><li>Write and debug code</li><li>Run tests</li><li>Deploy applications</li></ul><p>This is accelerating development cycles and enabling “AI-assisted engineering.”</p><p><strong>3. Customer Support Automation</strong></p><p>Agentic systems can:</p><ul><li>Handle complex queries</li><li>Escalate when necessary</li><li>Learn from past interactions</li></ul><p>They go beyond chatbots by managing full conversations and resolutions.</p><p><strong>4. Business Process Automation</strong></p><p>From scheduling meetings to managing supply chains, Agentic AI can orchestrate workflows across systems with minimal oversight.</p><p><strong>Agentic AI vs Traditional AI</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/695/1*vOPp-W8mdFHPs2uyhWns2Q.png" /></figure><p><strong>Challenges and Risks</strong></p><p>While powerful, Agentic AI introduces new challenges:</p><p><strong>1. Reliability</strong></p><p>Autonomous systems can make mistakes, especially in complex environments.</p><p><strong>2. Safety andControl</strong></p><p>Ensuring agents act within boundaries is critical.</p><p><strong>3. Ethical Concerns</strong></p><p>Questions around accountability, bias, and transparency become more complex.</p><p><strong>4. Over-Automation</strong></p><p>Delegating too much control can reduce human oversight.</p><p><strong>The Future of Agentic AI</strong></p><p>We are moving toward a world where AI doesn’t just assist — it collaborates. Future systems may:</p><ul><li>Manage entire projects independently</li><li>Act as personal digital assistants with long-term memory</li><li>Coordinate with other agents in multi-agent ecosystems</li></ul><p>This shift could redefine productivity, creativity, and even the nature of work itself.</p><p><strong>Final Thoughts</strong></p><p>Agentic AI represents a fundamental leap in artificial intelligence — from passive models to proactive systems. For data scientists, developers, and businesses, it opens up a new paradigm: <em>delegation to intelligent agents</em>.</p><p>The question is no longer <em>“What can AI do?”</em><br> It’s <em>“What can AI do on its own?”</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=02482a752199" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introduction to SQL: The Language of Data]]></title>
            <link>https://medium.com/@khathithi/introduction-to-sql-the-language-of-data-da61408c539d?source=rss-e15173b70f2a------2</link>
            <guid isPermaLink="false">https://medium.com/p/da61408c539d</guid>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[mysql]]></category>
            <category><![CDATA[sql-server]]></category>
            <category><![CDATA[sql]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Mzameli Mashiyi]]></dc:creator>
            <pubDate>Mon, 20 Apr 2026 10:18:56 GMT</pubDate>
            <atom:updated>2026-04-20T10:18:56.292Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U0yZrzMaIvR3DaJE2zD2wg.png" /><figcaption>Infographic illustrating core SQL concepts, including database tables, essential commands, and how SQL fits into the data science workflow.</figcaption></figure><h3><strong>Introduction to SQL: The Language of Data</strong></h3><p><strong>In today’s data-driven world, almost every application — from banking systems to social media platforms — relies on databases. At the heart of interacting with these databases is SQL, one of the most important skills for anyone working in tech, data science, or analytics.</strong></p><h3><strong>What is SQL?</strong></h3><p><strong>SQL (Structured Query Language) is a standard programming language used to store, manipulate, and retrieve data from relational databases. It allows you to communicate with databases in a simple, readable way.</strong></p><p><strong>Think of SQL as a bridge between you and your data. Instead of manually searching through tables, you write a query, and SQL returns exactly what you need.</strong></p><h3><strong>Why is SQL Important?</strong></h3><p><strong>SQL is widely used across industries because:</strong></p><ul><li><strong>It’s easy to learn compared to many programming languages</strong></li><li><strong>It works with major databases like MySQL, PostgreSQL, Oracle Database, and Microsoft SQL Server</strong></li><li><strong>It’s essential for data analysis, reporting, and backend development</strong></li><li><strong>It integrates well with tools like Python, Power BI, and Tableau</strong></li></ul><p><strong>If you’re interested in data science or analytics, SQL is not optional — it’s foundational.</strong></p><h3><strong>Understanding Databases and Tables</strong></h3><p><strong>A database is a collection of organized data. Inside a database, data is stored in tables.</strong></p><p><strong>A table looks like a spreadsheet:</strong></p><pre><strong>id        name        age        city<br>1         Alice       25         Cape Town<br>2         Bob         30         Johannesburg</strong></pre><ul><li><strong>Rows represent records</strong></li><li><strong>Columns represent attributes (or fields)</strong></li></ul><h4><strong>1. SELECT — Retrieve Data</strong></h4><pre>SELECT name, age FROM users;</pre><p><strong>This query retrieves the </strong><strong>name and </strong><strong>age columns from the </strong><strong>users table.</strong></p><h4><strong>2. WHERE — Filter Data</strong></h4><pre>SELECT * FROM users WHERE age &gt; 25;</pre><p><strong>This filters results to only include users older than 25.</strong></p><h4><strong>3. INSERT — Add Data</strong></h4><pre>INSERT INTO users (name, age, city)<br>VALUES (&#39;John&#39;, 28, &#39;Durban&#39;);</pre><h4><strong>4. UPDATE — Modify Data</strong></h4><pre>UPDATE users<br>SET age = 29<br>WHERE name = &#39;John&#39;;</pre><h4><strong>5. DELETE — Remove Data</strong></h4><pre>DELETE FROM users<br>WHERE name = &#39;John&#39;;</pre><h3><strong>Key SQL Concepts</strong></h3><ul><li><strong>Primary Key: A unique identifier for each record</strong></li><li><strong>Foreign Key: Links tables together</strong></li><li><strong>JOINs: Combine data from multiple tables</strong></li><li><strong>Indexes: Improve query performance</strong></li></ul><p><strong>Example of a JOIN:</strong></p><pre>SELECT users.name, orders.amount<br>FROM users<br>JOIN orders ON users.id = orders.user_id;</pre><h3><strong>Real-World Use Cases</strong></h3><p><strong>SQL is used everywhere:</strong></p><ul><li><strong>E-commerce: Track orders and customers</strong></li><li><strong>Finance: Manage transactions and accounts</strong></li><li><strong>Healthcare: Store patient records</strong></li><li><strong>Telecom: Analyze usage patterns</strong></li></ul><p><strong>For example, a telecom company in Cape Town might use SQL to analyze customer call data and improve service quality.</strong></p><h3><strong>SQL and Data Science</strong></h3><p><strong>In a typical data science workflow:</strong></p><ol><li><strong>SQL is used to extract data from databases</strong></li><li><strong>Tools like Python process and analyze the data</strong></li><li><strong>Visualization tools like Tableau present insights</strong></li></ol><p><strong>Without SQL, accessing real-world data becomes extremely difficult.</strong></p><h3><strong>Getting Started with SQL</strong></h3><p><strong>Here are simple steps to begin:</strong></p><ol><li><strong>Install a database system like MySQL or PostgreSQL</strong></li><li><strong>Use a GUI tool like DBeaver</strong></li><li><strong>Practice basic queries daily</strong></li><li><strong>Work on small projects (e.g., customer database, sales analysis)</strong></li></ol><h3><strong>Final Thoughts</strong></h3><p><strong>SQL is one of the most practical and powerful skills you can learn. Whether you want to become a data analyst, data scientist, or software engineer, mastering SQL will open doors to countless opportunities.</strong></p><p><strong>Start simple, stay consistent, and soon you’ll be querying data like a pro.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=da61408c539d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Preparing a Python Environment to conduct Exploratory Data Analysis (EDA) on a Jupyter Notebook.]]></title>
            <link>https://medium.com/@khathithi/preparing-a-python-environment-to-conduct-exploratory-data-analysis-eda-on-a-jupyter-notebook-5c8f9baa7fb0?source=rss-e15173b70f2a------2</link>
            <guid isPermaLink="false">https://medium.com/p/5c8f9baa7fb0</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[anaconda-navigator]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[jupyter-notebook]]></category>
            <category><![CDATA[exploratory-data-analysis]]></category>
            <dc:creator><![CDATA[Mzameli Mashiyi]]></dc:creator>
            <pubDate>Sun, 22 Mar 2026 20:34:38 GMT</pubDate>
            <atom:updated>2026-03-22T20:34:38.380Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tPvhVh2O0xHH09xKB4oVxA.png" /></figure><p>Preparing a Python Environment to start working on a Jupyter Notebooks.</p><p>Before performing a brief Exploratory Data Analysis (EDA), it is essential to ensure that your development environment is correctly configured.</p><p>First, verify whether Python is installed on your machine. Open the Command Prompt and run:</p><p>python -- version</p><p>If Python is not installed, download it from the official website: <a href="https://www.python.org/downloads/">https://www.python.org/downloads/</a>. Install Python 3.14.3, and ensure that the “Add Python to PATH” option is selected during installation.</p><p>After installation, rerun the version command to confirm successful setup.</p><p>Next, check if Anaconda Navigator is available on your system. If not, download and install it from: <a href="https://www.anaconda.com/products/navigator">https://www.anaconda.com/products/navigator</a>. You may be required to provide an email address during the download process.</p><p>Once installed, prepare your working directory by creating a dedicated folder for your project. Download your dataset and place it inside this folder to maintain an organized workflow. Launch Jupyter Notebook via Anaconda Navigator. This will serve as your interactive environment for performing EDA.</p><p>Before proceeding, ensure that the required Python libraries — pandas, numpy, and matplotlib — are installed. You can check this by attempting to import them in a notebook or by running the following commands in the Command Prompt:</p><p>python -c “import pandas”</p><p>python -c “import numpy”</p><p>python -c “import matplotlib”</p><p>If no errors appear, the libraries are installed. If a ModuleNotFoundError occurs, install the missing packages using:</p><p><em>pip install pandas numpy matplotlib</em></p><p>Alternatively, if using Anaconda:</p><p><em>conda install pandas numpy matplotlib</em></p><p>Once these dependencies are verified, your environment, you are ready to start working on your Jupyter Notebook.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5c8f9baa7fb0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Artificial Intelligence: A Beginner’s Guide to the Technology Shaping Our Future.]]></title>
            <link>https://medium.com/@khathithi/artificial-intelligence-a-beginners-guide-to-the-technology-shaping-our-future-6dad40e4d2d5?source=rss-e15173b70f2a------2</link>
            <guid isPermaLink="false">https://medium.com/p/6dad40e4d2d5</guid>
            <category><![CDATA[superintelligent-ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-engineering]]></category>
            <dc:creator><![CDATA[Mzameli Mashiyi]]></dc:creator>
            <pubDate>Thu, 19 Mar 2026 12:14:09 GMT</pubDate>
            <atom:updated>2026-03-19T12:14:09.526Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mKek5LIaHnD4HJ90SNO6fg.png" /><figcaption>AI and the future of technology</figcaption></figure><p>Artificial Intelligence (AI) is no longer a futuristic concept reserved for science fiction, it is a powerful, real-world technology transforming how we live, work, and interact with the world. From personalized recommendations on streaming platforms to self-driving cars, AI is quietly embedded in many aspects of our daily lives.</p><h3>What is Artificial Intelligence?</h3><p>At its core, Artificial Intelligence refers to the ability of machines to perform tasks that typically require human intelligence. These tasks include learning from data, recognizing patterns, understanding language, making decisions, and even solving complex problems.</p><p>Unlike traditional programming, where systems follow fixed rules, AI systems can adapt and improve over time by learning from experience. This ability to learn is what makes AI so powerful.</p><h3>Key Types of AI</h3><p>AI can be broadly categorized into three types:</p><h3>1. Narrow AI (Weak AI)</h3><p>This is the most common form of AI today. Narrow AI is designed to perform a specific task, such as voice recognition, image classification, or language translation. Examples include virtual assistants, recommendation systems, and spam filters.</p><h3>2. General AI (Strong AI)</h3><p>General AI refers to machines that can perform any intellectual task that a human can do. This type of AI remains theoretical and has not yet been achieved.</p><h3>3. Superintelligent AI</h3><p>This is a hypothetical form of AI that surpasses human intelligence in all areas. While it is widely discussed, it is still a topic of research and debate.</p><h3>How Does AI Work?</h3><p>AI systems rely heavily on data and algorithms. The most common approach is Machine Learning (ML), a subset of AI that allows systems to learn from data without being explicitly programmed.</p><p>Here are the main components:</p><ul><li>Data: The foundation of AI. More data often leads to better performance.</li><li>Algorithms: Mathematical models that process data and make predictions.</li><li>Training: The process of teaching the model using data.</li><li>Inference: Using the trained model to make predictions on new data.</li></ul><p>Deep Learning, a subset of Machine Learning, uses neural networks inspired by the human brain to process large amounts of data and identify complex patterns.</p><h3>Real-World Applications of AI</h3><p>AI is transforming industries across the globe:</p><ul><li>Healthcare: Disease diagnosis, drug discovery, and personalized treatment plans.</li><li>Finance: Fraud detection, credit scoring, and algorithmic trading.</li><li>Retail: Customer recommendations and demand forecasting.</li><li>Transportation: Autonomous vehicles and traffic optimization.</li><li>Education: Personalized learning experiences and intelligent tutoring systems.</li></ul><h3>Benefits of AI</h3><ul><li>Efficiency: Automates repetitive tasks, saving time and resources.</li><li>Accuracy: Reduces human error in data-heavy processes.</li><li>Scalability: Handles large volumes of data effortlessly.</li><li>Innovation: Enables new products and services.</li></ul><h3>Challenges and Ethical Concerns</h3><p>Despite its benefits, AI also raises important concerns:</p><ul><li>Bias: AI systems can inherit biases from data.</li><li>Privacy: Large amounts of personal data are often required.</li><li>Job Displacement: Automation may replace certain jobs.</li><li>Accountability: Determining responsibility for AI decisions can be complex.</li></ul><p>Addressing these challenges is crucial to ensuring that AI is developed and used responsibly.</p><h3>The Future of AI</h3><p>The future of AI is both exciting and uncertain. As technology continues to evolve, AI is expected to become more integrated into everyday life, driving innovation across industries. However, it is equally important to focus on ethical development, transparency, and regulation.</p><h3>Final Thoughts</h3><p>Artificial Intelligence is one of the most transformative technologies of our time. While it offers immense potential to improve our lives, it also requires careful consideration and responsible implementation. Whether you’re a beginner or an experienced professional, understanding AI is essential in today’s digital world.</p><p>As we move forward, one thing is clear: AI is not just the future; it is the present.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6dad40e4d2d5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Data Science Life Cycle: A Succinct Practical Python Cheat Sheet.]]></title>
            <link>https://medium.com/@khathithi/the-data-science-life-cycle-a-succinct-practical-python-cheat-sheet-e080ae0a8585?source=rss-e15173b70f2a------2</link>
            <guid isPermaLink="false">https://medium.com/p/e080ae0a8585</guid>
            <category><![CDATA[jupyter-notebook]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Mzameli Mashiyi]]></dc:creator>
            <pubDate>Wed, 18 Mar 2026 11:52:37 GMT</pubDate>
            <atom:updated>2026-03-18T11:52:37.017Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pTu1WCcB6fbofo13whxWtQ.png" /><figcaption>Data Science Life Cycle Cheat Sheet</figcaption></figure><p>If you’re starting your journey in data science or building real-world projects, one thing becomes clear very quickly: having a structured workflow is everything.</p><p>The Data Science Life Cycle provides that structure, guiding you from defining a problem all the way to deploying and maintaining a model. In this article, I’ll walk you through each step of the life cycle, paired with simple, practical Python “cheat codes” you can use immediately.</p><p><strong>What is the Data Science Life Cycle?</strong></p><p>The Data Science Life Cycle is a step-by-step process that helps you turn raw data into actionable insights and intelligent systems.</p><p>It typically includes:</p><ol><li>Problem Definition</li><li>Data Collection</li><li>Data Understanding (EDA)</li><li>Data Cleaning</li><li>Feature Engineering</li><li>Data Visualization</li><li>Model Building</li><li>Evaluation</li><li>Deployment</li><li>Monitoring</li></ol><p>Let’s break each step down with Python examples.</p><p><strong>1. Problem Definition</strong></p><p>Before writing a single line of code, define your objective.</p><p>problem = “Predict house prices”</p><p>target_variable = “price”</p><p>features = [“size”, “location”, “bedrooms”]</p><p>A clear problem saves you hours of confusion later.</p><p><strong>2. Data Collection</strong></p><p>Data can come from files, databases, or APIs.</p><p>import pandas as pd</p><p># Load CSV</p><p>df = pd.read_csv(“data.csv”)</p><p># Load API data</p><p>import requests</p><p>response = requests.get(“https://api.example.com/data&quot;)</p><p>data = response.json()</p><p><strong>3. Data Understanding (EDA)</strong></p><p>Understand what you’re working with before modelling.</p><p>df.head()</p><p>df.info()</p><p>df.describe()</p><p># Missing values</p><p>df.isnull().sum()</p><p># Correlation</p><p>df.corr()</p><p>This step helps you detect patterns, errors, and relationships.</p><p><strong>4. Data Cleaning</strong></p><p>Real-world data is messy so we have to clean it before using it.</p><p># Handle missing values</p><p>df.fillna(method=’ffill’, inplace=True)</p><p># Remove duplicates</p><p>df.drop_duplicates(inplace=True)</p><p># Fix data types</p><p>df[‘date’] = pd.to_datetime(df[‘date’])</p><p># Rename columns</p><p>df.columns = df.columns.str.lower()</p><p><strong>5. Feature Engineering</strong></p><p>Create better inputs for your model.</p><p># New feature</p><p>df[‘price_per_room’] = df[‘price’] / df[‘bedrooms’]</p><p># Encode categorical variables</p><p>df = pd.get_dummies(df, drop_first=True)</p><p><strong>6. Data Visualization</strong></p><p>Visuals help you understand trends quickly.</p><p>import matplotlib.pyplot as plt</p><p>import seaborn as sns</p><p>df[‘price’].hist()</p><p>plt.show()</p><p>sns.heatmap(df.corr(), annot=True)</p><p>plt.show()</p><p><strong>7. Train-Test Split</strong></p><p>Separate your data to evaluate performance properly.</p><p>from sklearn.model_selection import train_test_split</p><p>X = df.drop(“price”, axis=1)</p><p>y = df[“price”]</p><p>X_train, X_test, y_train, y_test = train_test_split(</p><p>X, y, test_size=0.2, random_state=42</p><p>)</p><p><strong>8. Model Building</strong></p><p>Train your first model.</p><p>from sklearn.linear_model import LinearRegression</p><p>model = LinearRegression()</p><p>model.fit(X_train, y_train)</p><p><strong>9. Model Evaluation</strong></p><p>Measure how well your model performs.</p><p>from sklearn.metrics import mean_squared_error, r2_score</p><p>y_pred = model.predict(X_test)</p><p>print(“MSE:”, mean_squared_error(y_test, y_pred))</p><p>print(“R2:”, r2_score(y_test, y_pred))</p><p><strong>10. Hyperparameter Tuning</strong></p><p>Improve performance by tuning parameters.</p><p>from sklearn.model_selection import GridSearchCV</p><p>from sklearn.ensemble import RandomForestRegressor</p><p>params = {</p><p>“n_estimators”: [100, 200],</p><p>“max_depth”: [None, 10, 20]</p><p>}</p><p>grid = GridSearchCV(RandomForestRegressor(), params, cv=5)</p><p>grid.fit(X_train, y_train)</p><p>best_model = grid.best_estimator_</p><p><strong>11. Model Saving</strong></p><p>Save your model for reuse.</p><p>import joblib</p><p>joblib.dump(model, “model.pkl”)</p><p># Load later</p><p>model = joblib.load(“model.pkl”)</p><p><strong>12. Deployment</strong></p><p>Make your model usable in real applications.</p><p>def predict(input_data):</p><p>return model.predict([input_data])</p><p>predict([1200, 2, 1])</p><p><strong>13. Monitoring and Maintenance</strong></p><p>Models degrade over time — keep checking them.</p><p>from sklearn.metrics import mean_absolute_error</p><p>mae = mean_absolute_error(y_test, model.predict(X_test))</p><p>print(“MAE:”, mae)</p><p><strong>Final Thoughts</strong></p><p>The Data Science Life Cycle isn’t just theory, it’s your roadmap to building real, production-ready solutions.</p><p>If you follow these steps consistently, you’ll:</p><ul><li>Avoid common mistakes</li><li>Build cleaner models</li><li>Deliver more reliable results</li></ul><p><strong>Bonus Tip</strong></p><p>As you grow, start using pipelines:</p><p>from sklearn.pipeline import Pipeline</p><p>from sklearn.preprocessing import StandardScaler</p><p>This keeps your workflow clean, scalable, and production-ready.</p><p><strong>Closing</strong></p><p>Whether you’re a beginner or transitioning into real-world projects, this cheat sheet gives you a solid foundation to build upon.</p><p>If you found this helpful, consider turning it into your own template for future projects because it’ll save you a lot of time.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e080ae0a8585" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Data Science Life Cycle]]></title>
            <link>https://medium.com/@khathithi/the-data-science-life-cycle-e6bd35613071?source=rss-e15173b70f2a------2</link>
            <guid isPermaLink="false">https://medium.com/p/e6bd35613071</guid>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science-life-cycle]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Mzameli Mashiyi]]></dc:creator>
            <pubDate>Tue, 17 Mar 2026 22:22:28 GMT</pubDate>
            <atom:updated>2026-03-17T22:22:28.631Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o7WozDYKZhDEMvDx3BtHJQ.png" /><figcaption>Data Science Life Cycle</figcaption></figure><p>Data Science is not just about building models; it’s a structured process that transforms raw data into meaningful insights and actionable decisions. Whether you’re working on a simple analysis or deploying a machine learning system, understanding the Data Science Life Cycle is essential.</p><p>In this article, we’ll walk through each stage of the lifecycle in a clear, practical way.</p><p><strong>1. Problem Definition</strong></p><p>Every successful data science project starts with a well-defined problem.</p><p>This step involves:</p><ul><li>Understanding business objectives</li><li>Translating goals into data questions</li><li>Defining success metrics</li></ul><p>Example:<br> Instead of saying <em>“build a model”</em>, define:<br> <em>“Predict customer churn with at least 80% accuracy.”</em></p><p>A poorly defined problem leads to wasted effort, so take your time here.</p><p><strong>2. Data Collection</strong></p><p>Once the problem is clear, the next step is gathering relevant data.</p><p>Sources may include:</p><ul><li>Databases (SQL, NoSQL)</li><li>APIs</li><li>CSV/Excel files</li><li>Web scraping</li><li>Sensors or IoT devices</li></ul><p>Tip: Always collect more data than you think you need but ensure it’s relevant.</p><p><strong>3. Data Cleaning (Data Preprocessing)</strong></p><p>Raw data is often messy. Cleaning it is one of the most time-consuming but critical steps.</p><p>Tasks include:</p><ul><li>Handling missing values</li><li>Removing duplicates</li><li>Fixing data types</li><li>Dealing with outliers</li><li>Standardizing formats</li></ul><p>Reality check: Data scientists spend 70–80% of their time here.</p><p><strong>4. Exploratory Data Analysis (EDA)</strong></p><p>EDA helps you understand the data before modelling.</p><p>You’ll:</p><ul><li>Visualize distributions</li><li>Identify patterns and trends</li><li>Detect anomalies</li><li>Explore relationships between variables</li></ul><p>Common tools:</p><ul><li>Pandas</li><li>Matplotlib</li><li>Seaborn</li></ul><p>Goal: Turn raw data into insights.</p><p><strong>5. Feature Engineering</strong></p><p>This step improves model performance by transforming data into useful inputs.</p><p>Examples:</p><ul><li>Creating new variables (e.g., “age” from birthdate)</li><li>Encoding categorical variables</li><li>Scaling numerical features</li><li>Aggregating data</li></ul><p>Key idea: Better features translate to better models.</p><p><strong>6. Model Building</strong></p><p>Now comes the exciting part, building models.</p><p>Depending on your problem:</p><ul><li>Regression → Predict continuous values</li><li>Classification → Predict categories</li><li>Clustering → Find hidden groups</li></ul><p>Common algorithms:</p><ul><li>Linear Regression</li><li>Decision Trees</li><li>Random Forest</li><li>Neural Networks</li></ul><p><strong>7. Model Evaluation</strong></p><p>You need to measure how well your model performs.</p><p>Common metrics:</p><ul><li>Accuracy</li><li>Precision and Recall</li><li>F1 Score</li><li>Mean Squared Error</li></ul><p>Important:<br> Always use separate training and testing datasets to avoid overfitting.</p><p><strong>8. Model Deployment</strong></p><p>A model is only useful if it’s used in the real world.</p><p>Deployment options:</p><ul><li>Web applications (Flask, Streamlit)</li><li>Cloud platforms (AWS, Azure)</li><li>APIs</li></ul><p>This step turns your model into a real product.</p><p><strong>9. Monitoring and Maintenance</strong></p><p>The lifecycle doesn’t end after deployment.</p><p>You must:</p><ul><li>Monitor performance</li><li>Detect data drift</li><li>Retrain models when needed</li><li>Update features and pipelines</li></ul><p>Why?<br> Because real-world data changes over time.</p><p><strong>Final Thoughts</strong></p><p>The Data Science Life Cycle is iterative, not linear. You’ll often go back and refine earlier steps as you learn more about the data and the problem.</p><p>Key Takeaways:</p><ul><li>Start with a clear problem</li><li>Clean data thoroughly</li><li>Focus on understanding before modelling</li><li>Continuously improve after deployment</li></ul><p><strong>Bonus: Real-World Workflow</strong></p><p>In practice, your workflow might look like this:</p><ol><li>Define problem</li><li>Collect data</li><li>Clean data</li><li>Explore data</li><li>Engineer features</li><li>Train model</li><li>Evaluate model</li><li>Deploy</li><li>Monitor and improve</li></ol><p>By mastering this lifecycle, you’ll move from simply writing code to actually solving real-world problems which is what Data Science is all about.</p><p>If you’re starting your data science journey, focus on mastering each stage one at a time. The tools may change, but the lifecycle remains the same.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e6bd35613071" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is a Data Card? A simple guide for Data Science projects.]]></title>
            <link>https://medium.com/@khathithi/what-is-a-data-card-a-simple-guide-for-data-science-projects-6df9b6bf0212?source=rss-e15173b70f2a------2</link>
            <guid isPermaLink="false">https://medium.com/p/6df9b6bf0212</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[card-data]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Mzameli Mashiyi]]></dc:creator>
            <pubDate>Mon, 16 Mar 2026 09:55:15 GMT</pubDate>
            <atom:updated>2026-03-16T09:55:15.624Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1zbBa84_F5wOLj4h-Xsdpg.png" /></figure><p>In modern data science and machine learning projects, documenting datasets properly is extremely important. One useful tool for doing this is a Data Card. A data card helps researchers, engineers, and stakeholders clearly understand where data comes from, how it is structured, and how it should be used.</p><p>If you are building data science projects or publishing work online, for example on GitHub or Kaggle, creating a data card makes your work more transparent and professional.</p><p><strong>What Is a Data Card?</strong></p><p>A Data Card is a structured document that describes a dataset in detail. It provides important information about the dataset’s source, collection process, structure, intended use, and limitations.</p><p>Think of a data card as the documentation for a dataset.</p><p>It helps answer questions such as:</p><ul><li>Where did the data come from?</li><li>How was the data collected?</li><li>What do the variables mean?</li><li>Are there biases or limitations?</li><li>How should the dataset be used responsibly?</li></ul><p>Data cards are especially important in machine learning, where poor data understanding can lead to incorrect conclusions or biased models.</p><p><strong>Why Data Cards Are Important</strong></p><p>1. Transparency</p><p>Data cards clearly explain the origin and structure of a dataset. This helps other researchers understand how the data was created.</p><p>2. Reproducibility</p><p>Good documentation allows others to reproduce experiments and results, which is a key principle in scientific research.</p><p>3. Responsible AI</p><p>Machine learning systems can inherit biases from data. Data cards help identify potential ethical issues or biases before using the data.</p><p>4. Collaboration</p><p>When working in teams, a data card ensures that everyone understands the dataset.</p><p><strong>Key Components of a Data Card</strong></p><p>A typical data card includes the following sections:</p><p>1. Dataset Overview</p><p>This section gives a general description of the dataset.</p><p>Example:</p><ul><li>Dataset name</li><li>Description</li><li>Domain (finance, healthcare, transportation, etc.)</li></ul><p><strong>2. Data Source</strong></p><p>This explains where the data comes from.</p><p>Example:</p><ul><li>Public dataset</li><li>Government database</li><li>Sensor data</li><li>Survey responses</li></ul><p><strong>3. Data Collection Process</strong></p><p>This section describes how the data was collected.</p><p>Important details may include:</p><ul><li>Data collection methods</li><li>Time period</li><li>Tools or systems used</li></ul><p><strong>4. Data Structure</strong></p><p>This explains the features (variables) in the dataset.</p><p>Example table:</p><p><strong>Feature -&gt; Description</strong></p><p>Age <strong>-&gt; </strong>Annual income</p><p>Age of the individual <strong>-&gt; </strong>CreditScore</p><p>Income <strong>-&gt; </strong>Numerical credit rating</p><p><strong>5. Data Cleaning and Processing</strong></p><p>This section describes preprocessing steps such as:</p><ul><li>Handling missing values</li><li>Removing duplicates</li><li>Normalization or scaling</li><li>Feature engineering</li></ul><p><strong>6. Intended Use</strong></p><p>Explain how the dataset is meant to be used.</p><p>Examples:</p><ul><li>Training machine learning models</li><li>Statistical analysis</li><li>Educational purposes</li></ul><p><strong>7. Limitations and Bias</strong></p><p>All datasets have limitations. A good data card should mention:</p><ul><li>Missing or incomplete data</li><li>Sampling bias</li><li>Measurement errors</li><li>Limited geographic coverage</li></ul><p><strong>8. Licensing and Access</strong></p><p>This section explains how the dataset can be used.</p><p>Example:</p><ul><li>Open dataset</li><li>Academic use only</li><li>Commercial restrictions</li></ul><p><strong>Example of a Simple Data Card</strong></p><p><strong>Dataset Name:</strong> Credit Score Prediction Dataset</p><p><strong>Description:</strong><br> This dataset contains financial and demographic information used to predict a customer’s credit score.</p><p><strong>Number of Records:</strong><br> 10,000 rows</p><p><strong>Features:</strong></p><ul><li>Age</li><li>Income</li><li>Employment Status</li><li>Debt Ratio</li><li>Credit Score</li></ul><p><strong>Data Source:</strong><br> Public financial dataset from Kaggle.</p><p><strong>Use Case:</strong><br> Training machine learning models for credit score prediction.</p><p><strong>Limitations:</strong><br> The dataset contains mostly data from North American customers and may not generalize globally.</p><p><strong>Data Cards in Modern AI Systems</strong></p><p>Large technology organizations such as Google and Microsoft encourage the use of data documentation practices like Data Cards and Model Cards. These practices improve transparency in AI systems and help ensure responsible use of machine learning.</p><p><strong>Conclusion</strong></p><p>A Data Card is a simple but powerful tool for documenting datasets. It improves transparency, reproducibility, and responsible use of data in machine learning projects.</p><p>Whether you are building a data science portfolio, publishing a dataset online, or writing about your work on Medium, including a data card makes your project more professional and easier for others to understand.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6df9b6bf0212" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Download, Install, and Configure VS Code for Data Science (Step-by-Step)]]></title>
            <link>https://medium.com/@khathithi/how-to-download-install-and-configure-vs-code-for-data-science-step-by-step-f4d100866289?source=rss-e15173b70f2a------2</link>
            <guid isPermaLink="false">https://medium.com/p/f4d100866289</guid>
            <category><![CDATA[github]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Mzameli Mashiyi]]></dc:creator>
            <pubDate>Fri, 13 Mar 2026 17:16:18 GMT</pubDate>
            <atom:updated>2026-03-13T17:16:18.480Z</atom:updated>
            <content:encoded><![CDATA[<p>If you are starting your Data Science journey, one of the best tools you can use is Visual Studio Code (VS Code). It is a free, lightweight, and powerful code editor developed by Microsoft.</p><p>VS Code supports Python, Jupyter notebooks, Git, and many data science libraries, making it ideal for building Machine Learning and Data Science projects.</p><p>In this guide, you will learn how to download, install, and configure VS Code for Data Science projects step-by-step.</p><h3>1. Download Visual Studio Code</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*HCa3uhNhAlYjwzkV.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8Vo84PXzJzPcyuWF.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wIl2e_yJ4NwOzbKb.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*0QxutTRRs6cKf0Q1.jpg" /></figure><p>Follow these steps:</p><ol><li>Open your browser.</li><li>Go to the official VS Code website:<br><a href="https://code.visualstudio.com/">https://code.visualstudio.com</a></li><li>Click Download for Windows (or Mac/Linux depending on your system).</li><li>The installer file will download automatically.</li></ol><p>Tip: Always download VS Code from the official website to avoid security risks.</p><h3>2. Install Visual Studio Code</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/499/0*cMmZMofE3EruIdXs.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/529/0*F-HUhFKc8-c0Ixz3.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/589/0*GKXOTxpK6GuAigQw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vDHBvCGWJfufTIMC.png" /></figure><p>Once the installer is downloaded:</p><ol><li>Double-click the VSCodeSetup.exe file.</li><li>Accept the License Agreement.</li><li>Click Next until you reach Select Additional Tasks.</li><li>Make sure you check the following boxes:</li></ol><p>✔ Add “Open with Code” action<br>✔ Add to PATH (important)<br>✔ Create a desktop icon</p><p>5. Click Install.</p><p>6. Once the installation is complete, click Finish.</p><p>VS Code will now open on your computer.</p><h3>3. Install Python</h3><p>Data Science projects require Python, so you must install it before configuring VS Code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/200/0*JR29wOiT2-ao1qky.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/661/0*U7vrMe9P1XiwLjdF.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/658/0*nilkwbt8dxjNJoln.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/666/0*6hE5IgxaDlF-qVLO.jpg" /></figure><p>Steps:</p><ol><li>Go to <a href="https://www.python.org/">https://www.python.org</a></li><li>Click Downloads.</li><li>Download the latest version of Python.</li><li>Run the installer.</li><li>Important: Check the box:</li></ol><p>✔ Add Python to PATH</p><p>6. Click Install Now.</p><p>To confirm installation, open Command Prompt and type:</p><pre>python --version</pre><p>You should see the installed Python version.</p><h3>4. Install Essential VS Code Extensions for Data Science</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*XGwQeEEK1o60WEX6.gif" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*39ToE5xBbuSkkUCv.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wwQILdDCMSEHg7Vz.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*365Ahrw54oMWeF0R.gif" /></figure><p>Extensions add powerful features to VS Code.</p><p>Open VS Code and follow these steps:</p><ol><li>Click the Extensions icon on the left sidebar.</li><li>Search and install the following extensions:</li></ol><h3>1. Python</h3><p>Install the Python extension by Microsoft.</p><p>Features:</p><ul><li>Python IntelliSense</li><li>Code linting</li><li>Debugging</li><li>Environment management</li></ul><h3>2. Jupyter</h3><p>The Jupyter extension allows you to run notebooks directly inside VS Code.</p><p>Features:</p><ul><li>Create .ipynb notebooks</li><li>Run cells interactively</li><li>Visualize outputs and plots</li></ul><h3>3. Pylance</h3><p>Provides:</p><ul><li>Fast IntelliSense</li><li>Type checking</li><li>Better Python performance</li></ul><h3>4. GitHub Copilot (Optional)</h3><p>If you want AI coding assistance, you can install:</p><p>GitHub Copilot</p><p>It suggests code as you type.</p><h3>5. Install Data Science Libraries</h3><p>Open the VS Code terminal:</p><pre>Terminal → New Terminal</pre><p>Install the core Data Science libraries:</p><pre>pip install numpy pandas matplotlib seaborn scikit-learn jupyter</pre><p>These libraries are essential for:</p><p>NumPy for Numerical computing.</p><p>Pandas for Data manipulation.</p><p>Matplotlib for Data visualization.</p><p>Seaborn for Statistical visualization.</p><p>Scikit-learn for Machine learning.</p><p>Jupyter for Interactive notebooks.</p><h3>6. Create Your First Data Science Project</h3><ol><li>Create a new folder called:</li></ol><pre>DataScienceProjects</pre><p>2. Open the folder in VS Code:</p><pre>File → Open Folder</pre><p>3. Create a new Python file:</p><pre>analysis.py</pre><p>Example code:</p><pre>import pandas as pd<br>import matplotlib.pyplot as plt</pre><pre>data = pd.DataFrame({<br>    &quot;Year&quot;:[2020,2021,2022,2023],<br>    &quot;Sales&quot;:[100,150,200,250]<br>})</pre><pre>data.plot(x=&quot;Year&quot;, y=&quot;Sales&quot;, kind=&quot;line&quot;)<br>plt.show()</pre><p>Run the file using:</p><pre>Run ▶</pre><p>You have now executed your <strong>first data science script in VS Code</strong>.</p><h3>7. Using Jupyter Notebooks in VS Code</h3><p>You can also create notebooks for analysis.</p><p>Steps:</p><ol><li>Click File → New File</li><li>Save it as:</li></ol><pre>analysis.ipynb</pre><p>Now you can:</p><ul><li>Run code cells</li><li>Plot graphs</li><li>Perform data exploration</li></ul><p>This workflow is widely used in Machine Learning and Data Science.</p><h3>8. Connect VS Code with GitHub (Recommended)</h3><p>For version control and project sharing, connect VS Code with GitHub.</p><p>Steps:</p><ol><li>Install Git.</li><li>Open VS Code.</li><li>Go to Source Control.</li><li>Sign in to GitHub.</li><li>Initialize a repository.</li></ol><p>This allows you to:</p><ul><li>Track code changes</li><li>Collaborate with others</li><li>Showcase projects.</li></ul><h3>Final remarks</h3><p>By completing this setup, you now have a fully configured Data Science development environment using:</p><ul><li>Visual Studio Code</li><li>Python</li><li>Essential data science libraries</li><li>Jupyter notebooks</li><li>GitHub version control</li></ul><p>This setup is used by many professional Data Scientists for building Machine Learning models, analyzing datasets, and deploying projects. Does this make sense?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f4d100866289" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>