<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Krishnanayakbluezone on Medium]]></title>
        <description><![CDATA[Stories by Krishnanayakbluezone on Medium]]></description>
        <link>https://medium.com/@krishnanayak902bluezone?source=rss-9958de5a3787------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*XYaC07Bj-LAzvMdT</url>
            <title>Stories by Krishnanayakbluezone on Medium</title>
            <link>https://medium.com/@krishnanayak902bluezone?source=rss-9958de5a3787------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 07:16:47 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@krishnanayak902bluezone/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[“ AgTech-Project” Crop Recommendation]]></title>
            <link>https://medium.com/@krishnanayak902bluezone/agtech-project-crop-recommendation-143c2b124ce3?source=rss-9958de5a3787------2</link>
            <guid isPermaLink="false">https://medium.com/p/143c2b124ce3</guid>
            <dc:creator><![CDATA[Krishnanayakbluezone]]></dc:creator>
            <pubDate>Tue, 23 Apr 2024 20:32:46 GMT</pubDate>
            <atom:updated>2024-04-23T20:32:46.081Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/650/1*SI0wHJvEPE7RXR7pqAnT4A.png" /></figure><h4>🌱 Welcome to the World of Green: A Journey into Crop Recommendation 🌾</h4><p>In this blog post, we’re diving into crop recommendation, where we blend modern technology with ancient farming knowledge. 🌱 Using Python and machine learning, we’ll explore datasets packed with details about soil, weather, and different crops. 📊</p><p>We’ll closely examine our code, where clever algorithms make sense of all this data. 🔍 We’ll talk about models like Logistic Regression, Random Forests, Support Vector Machines, and Decision Trees. These models help us predict which crops are best suited for specific conditions. 💻</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/563/1*q8GAjqCMt3ale7-Ttu_c_A.jpeg" /><figcaption>Let’s Learn!! <em>🚀</em></figcaption></figure><blockquote>🌍 Let’s explore together and find out which model works best for predicting crops. 🚜</blockquote><h4>STEP — 1 (Loading Data and Essential Libraries)</h4><p>In this section, we’ve imported crucial libraries for our crop recommendation project. We’re using NumPy for mathematical operations, Pandas for handling data, Matplotlib for creating plots, and Seaborn for enhanced visualizations. 🌟 Additionally, we’ve imported modules from Scikit-learn for model selection, preprocessing, and evaluation. 🛠️ Finally, we’ve loaded our dataset named “crop_recommendation.csv” using Pandas and displayed the first 8 rows to get a glimpse of our data. 🌾</p><pre># Importing necessary libraries for data manipulation, visualization, and machine learning<br>import numpy as np  # For numerical operations<br>import pandas as pd  # For data manipulation and analysis<br>import matplotlib.pyplot as plt  # For creating plots and visualizations<br>import seaborn as sns  # For enhanced data visualization<br><br># Importing modules for model selection, preprocessing, and evaluation<br>from sklearn.model_selection import train_test_split  # For splitting data into training and testing sets<br>from sklearn.preprocessing import StandardScaler  # For standardizing numerical features<br>from sklearn.preprocessing import LabelEncoder  # For encoding categorical labels<br>from sklearn.preprocessing import MinMaxScaler  # For scaling numerical features to a specific range<br>from sklearn.metrics import accuracy_score  # For evaluating model performance<br><br># Importing machine learning models<br>from sklearn.linear_model import LogisticRegression  # For logistic regression model<br>from sklearn.ensemble import RandomForestClassifier  # For random forest classifier model<br>from sklearn.tree import DecisionTreeClassifier  # For decision tree classifier model<br>from sklearn.svm import SVC  # For support vector machine classifier model<br><br>#loading dataset<br>crop_df = pd.read_csv(&quot;crop_recommendation.csv&quot;)<br>crop_df.head(8)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/390/1*y1RcOEwLvgIuXTtHZm5Aag.png" /><figcaption>This sample dataset contains information about various agricultural features, including Nitrogen (N), Phosphorus (P), and Potassium (K) levels in the soil, as well as temperature, humidity, pH, and rainfall conditions. Each row represents a specific crop, such as rice, along with corresponding values for these features.</figcaption></figure><h4>STEP — 2 (Initial Exploration &amp; Data Cleaning)</h4><p>Initial Exploration &amp; Data Cleaning is like laying the foundation for a sturdy building — without it, the entire structure could collapse.</p><p>🏗️ Through initial exploration, we get a glimpse into the raw form of our dataset, understanding its shape, size, and basic characteristics.</p><p>📊 However, data often comes with imperfections — missing values, outliers, inconsistencies — that can skew our analysis and lead to erroneous conclusions.</p><p>🧹 Hence, data cleaning becomes essential, where we meticulously scrub, polish, and refine our dataset, ensuring that it’s ready for analysis.</p><pre># Displaying the dimensions of the dataset (number of rows and columns)<br>crop_df.shape<br><br># Checking for missing values in each column and summing them up<br>crop_df.isnull().sum()<br><br># Displaying concise information about the dataset including data types and non-null values<br>crop_df.info()<br><br># Counting the number of duplicate rows in the dataset<br>crop_df.duplicated().sum()<br><br># Generating descriptive statistics for numerical columns like count, mean, standard deviation, etc.<br>print(crop_df.describe())<br><br># Counting the frequency of each unique value in the &#39;label&#39; column, which represents different crop types<br>crop_df[&#39;label&#39;].value_counts()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/467/1*owAM6lyuvNpxNik1Woo35A.png" /><figcaption>The dataset consists of 2200 entries and 8 columns. Also, data types include integers (int64), floating-point numbers (float64), and one object type (object) for the ‘label’ column representing crop types.</figcaption></figure><h4>STEP — 3 (Exploratory Data Analysis)🕵️‍♂️</h4><p>EDA is like peering through a magnifying glass into the heart of our dataset, uncovering hidden patterns, anomalies, and insights. Through visualizations, summary statistics, and data manipulation techniques, EDA helps us understand the underlying structure of our data, paving the way for more informed decisions and model building.</p><pre># Assuming crop_df is your DataFrame containing both numerical and categorical columns<br># Separate numerical and categorical features<br>numerical_features = crop_df.select_dtypes(include=[&#39;float64&#39;, &#39;int64&#39;]).columns<br>categorical_features = crop_df.select_dtypes(include=[&#39;object&#39;]).columns<br><br># Create a pie chart for each categorical feature<br>plt.figure(figsize=(12, 8))<br>for i, feature in enumerate(categorical_features, 1):<br>    plt.subplot(1, 1, i)<br>    crop_df[feature].value_counts().plot(kind=&#39;pie&#39;, autopct=&#39;%1.1f%%&#39;, startangle=90)<br>    plt.title(f&#39;{feature} Distribution&#39;)<br>plt.tight_layout()<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/818/1*u4qWKCmm2r4PVwEecVo0AQ.png" /><figcaption>We observe a fascinating phenomenon: the equal distribution of crops across our dataset. It’s as if nature itself has meticulously arranged each crop type to occupy approximately 4.5% of our data, creating a harmonious balance.</figcaption></figure><pre># Create a count plot to visualize the distribution of crop types<br>plt.figure(figsize=(10, 6))<br>sns.countplot(x=&#39;label&#39;, data=crop_df)<br>plt.title(&#39;Distribution of Labels&#39;)  # Set title for the plot<br>plt.xlabel(&#39;Crop Type&#39;)  # Set label for x-axis<br>plt.ylabel(&#39;Count&#39;)  # Set label for y-axis<br>plt.xticks(rotation=45)  # Rotate x-axis labels for better readability<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/853/1*R__mWa7cHRPHkhMwsAA0Hg.png" /><figcaption>The same is plotted in a bar graph for better visualisation!!</figcaption></figure><pre>numeric_crop_df = crop_df.select_dtypes(include=[&#39;float64&#39;, &#39;int64&#39;])<br><br># Visualize the correlation matrix<br>plt.figure(figsize=(12, 8))<br>sns.heatmap(numeric_crop_df.corr(), annot=True, cmap=&#39;coolwarm&#39;)<br>plt.title(&#39;Correlation Matrix&#39;)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/908/1*65wWHCeDqdabpLaPghcpAA.png" /><figcaption>Visualizing the correlation matrix is unravelling the intricate connections within our dataset, illuminating the relationships between different numerical features. In this heatmap, each cell represents the correlation coefficient between two features, ranging from -1 to 1. A value close to 1 indicates a strong positive correlation and a value near -1 signifies a strong negative correlation.</figcaption></figure><pre>for col in numeric_crop_df:<br>    # Create a boxplot for each numerical feature grouped by crop type<br>    plt.figure(figsize=(8, 6))<br>    sns.boxplot(x=&#39;label&#39;, y=col, data=crop_df)<br>    plt.title(f&#39;Boxplot of {col} by Crop Type&#39;)  # Set title for the plot<br>    plt.xlabel(&#39;Crop Type&#39;)  # Set label for x-axis<br>    plt.ylabel(col)  # Set label for y-axis<br>    plt.xticks(rotation=45)  # Rotate x-axis labels for better readability<br>    plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/702/1*s-ah9As0RrOgP3xLx2hfww.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/702/1*af8MoxX94mEs9CKDPfY74w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/702/1*9jXBJVT0cDPgx-_ygUAokg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/702/1*VlsghvnouT6OVOYINO-yZg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/693/1*gG65QEJAVDWLoKavgLyOgw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/693/1*P5rMpJU-jmTMDcQqH6E7hg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/702/1*8Fu28ETvXgCJLhGBoC2YrA.png" /><figcaption>Each boxplot serves as a window into the distribution of a specific numerical feature across different crop types, offering a vivid snapshot of their variability and central tendencies. As we traverse through these graphical representations, we witness the diversity among crops, discerning how factors like Nitrogen content, Phosphorus levels, and other environmental variables vary across different agricultural contexts.</figcaption></figure><h4>STEP — 4 (Splitting Train &amp; Test Data)</h4><blockquote>NOTE-</blockquote><blockquote>LabelEncoder(): In agriculture, we often deal with categorical variables like crop types. Machine learning algorithms typically work with numerical data, so we use LabelEncoder() to convert categorical labels into numerical format. This transformation make it easier for machine learning algorithms to understand and process the data.</blockquote><blockquote>StandardScaler(): Agricultural datasets often contain numerical features with varying scales and units. StandardScaler() is essential for ensuring that all numerical features are standardized to have a mean of 0 and a standard deviation of 1. By scaling the features, we bring them onto a common scale, preventing features with larger magnitudes from dominating the learning process during model training.</blockquote><pre>label_encoder = LabelEncoder()<br>crop_df[&#39;label&#39;] = label_encoder.fit_transform(crop_df[&#39;label&#39;])  # Encode categorical &#39;label&#39; column to numerical values<br><br>scaler = StandardScaler()<br>crop_df[[&#39;N&#39;, &#39;P&#39;, &#39;K&#39;, &#39;temperature&#39;,&#39;humidity&#39;,&#39;ph&#39;,&#39;rainfall&#39;]] = scaler.fit_transform(crop_df[[&#39;N&#39;, &#39;P&#39;, &#39;K&#39;, &#39;temperature&#39;,&#39;humidity&#39;,&#39;ph&#39;,&#39;rainfall&#39;]])  # Standardize numerical features using StandardScaler<br><br>X = crop_df.drop([&#39;label&#39;], axis=1)  # Extract features (X) by removing the &#39;label&#39; column<br>Y = crop_df[&#39;label&#39;]  # Extract target variable (Y)<br><br>X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.2, random_state=42)  # Split data into training and testing sets with 80% training and 20% testing</pre><pre>print(X)<br>print(Y)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/659/1*kb7lXqCaHl2ppwtEmjsvPA.png" /></figure><pre>ms = MinMaxScaler()  # Initialize MinMaxScaler object for feature scaling<br><br>X_train = ms.fit_transform(X_train)  # Fit and transform the training data to scale features between 0 and 1<br>X_test = ms.transform(X_test)  # Transform the testing data using the same scaler to maintain consistency<br><br>X_train  # Display the scaled training data</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/627/1*_pUFZvu7PhSSsjhNXPI2nw.png" /></figure><pre>sc = StandardScaler()  # Initialize StandardScaler object for feature scaling<br><br>sc.fit(X_train)  # Fit the scaler to the training data to compute the mean and standard deviation of each feature<br><br>X_train = sc.transform(X_train)  # Transform the training data using the computed mean and standard deviation<br>X_test = sc.transform(X_test)  # Transform the testing data using the same scaler to maintain consistency<br><br>X_train  # Display the scaled training data</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/515/1*sOL8tE8l3OrlldUllmRP5A.png" /></figure><h4>STEP 5 — (Model Building)</h4><h4>🚀 In this phase, we harness the might of algorithms like Logistic Regression, Random Forests, Support Vector Machines, and Decision Trees, each offering unique strengths in deciphering the complexities of crop recommendation. 🌾</h4><blockquote>Logistic Regression Model</blockquote><pre># Initialize and train the logistic regression model<br>log_reg = LogisticRegression()<br>log_reg.fit(X_train, Y_train)<br><br># Predict on the training and test set<br>Y_train_pred = log_reg.predict(X_train)<br>Y_test_pred = log_reg.predict(X_test)<br><br># Calculate accuracy for training and test set<br>train_accuracy = accuracy_score(Y_train, Y_train_pred)<br>test_accuracy = accuracy_score(Y_test, Y_test_pred)<br><br># Print accuracy values<br>print(f&quot;Train Accuracy: {train_accuracy}&quot;)<br>print(f&quot;Test Accuracy: {test_accuracy}\n&quot;)</pre><p>Train Accuracy: 0.977840909090909<br>Test Accuracy: 0.9636363636363636</p><blockquote>Random Forest Model</blockquote><pre># Initialize and train the Random Forest model<br>rf_classifier = RandomForestClassifier()<br>rf_classifier.fit(X_train, Y_train)<br><br># Predict on the training and test set<br>Y_train_pred = rf_classifier.predict(X_train)<br>Y_test_pred = rf_classifier.predict(X_test)<br><br># Calculate accuracy for training and test set<br>train_accuracy = accuracy_score(Y_train, Y_train_pred)<br>test_accuracy = accuracy_score(Y_test, Y_test_pred)<br><br># Print accuracy values<br>print(f&quot;Train Accuracy: {train_accuracy}&quot;)<br>print(f&quot;Test Accuracy: {test_accuracy}\n&quot;)</pre><p>Train Accuracy: 1.0<br>Test Accuracy: 0.9931818181818182</p><blockquote>SVM model</blockquote><pre># Initialize and train the SVM model<br>svm_classifier = SVC()<br>svm_classifier.fit(X_train, Y_train)<br><br># Predict on the training and test set<br>Y_train_pred = svm_classifier.predict(X_train)<br>Y_test_pred = svm_classifier.predict(X_test)<br><br># Calculate accuracy for training and test set<br>train_accuracy = accuracy_score(Y_train, Y_train_pred)<br>test_accuracy = accuracy_score(Y_test, Y_test_pred)<br><br># Print accuracy values<br>print(f&quot;Train Accuracy: {train_accuracy}&quot;)<br>print(f&quot;Test Accuracy: {test_accuracy}\n&quot;)</pre><p>Train Accuracy: 0.9880681818181818<br>Test Accuracy: 0.9681818181818181</p><blockquote>Decision Tree Model</blockquote><pre># Initialize and train the Decision Tree model<br>dt_classifier = DecisionTreeClassifier()<br>dt_classifier.fit(X_train, Y_train)<br><br># Predict on the training and test set<br>Y_train_pred = dt_classifier.predict(X_train)<br>Y_test_pred = dt_classifier.predict(X_test)<br><br># Calculate accuracy for training and test set<br>train_accuracy = accuracy_score(Y_train, Y_train_pred)<br>test_accuracy = accuracy_score(Y_test, Y_test_pred)<br><br># Print accuracy values<br>print(f&quot;Train Accuracy: {train_accuracy}&quot;)<br>print(f&quot;Test Accuracy: {test_accuracy}\n&quot;)</pre><p>Train Accuracy: 1.0<br>Test Accuracy: 0.9886363636363636</p><h4>STEP — 6 (Best Fit Model)</h4><pre># Initialize classifiers<br>classifiers = {<br>    &#39;Logistic Regression&#39;: LogisticRegression(),<br>    &#39;Random Forest&#39;: RandomForestClassifier(),<br>    &#39;Support Vector Machine&#39;: SVC(),<br>    &#39;Decision Tree&#39;: DecisionTreeClassifier()<br>}<br><br># Train and evaluate each classifier<br>accuracy_scores = {}<br>for name, classifier in classifiers.items():<br>    classifier.fit(X_train, Y_train)<br>    Y_test_pred = classifier.predict(X_test)<br>    accuracy_scores[name] = accuracy_score(Y_test, Y_test_pred)<br><br># Find the best performing model<br>best_model = max(accuracy_scores, key=accuracy_scores.get)<br>best_accuracy = accuracy_scores[best_model]<br><br># Plot the accuracy scores<br>plt.figure(figsize=(10, 6))<br>plt.bar(accuracy_scores.keys(), accuracy_scores.values(), color=&#39;lightgreen&#39;)<br>plt.xlabel(&#39;Classifier&#39;)<br>plt.ylabel(&#39;Accuracy&#39;)<br>plt.title(&#39;Accuracy of Different Classifiers&#39;)<br>plt.xticks(rotation=45)<br>plt.ylim(0, 1)<br>plt.axhline(best_accuracy, color=&#39;red&#39;, linestyle=&#39;--&#39;, label=f&#39;Best Model: {best_model} ({best_accuracy:.2f})&#39;)<br>plt.legend()<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/846/1*3ksyZ3gQhGd9fMfmZHJynw.png" /><figcaption>Based on the comprehensive analysis of various machine learning algorithms applied to our agricultural dataset, the evidence overwhelmingly points to Random Forest as the most adept model for our crop recommendation project.</figcaption></figure><pre>rfc = RandomForestClassifier()  # Initialize RandomForestClassifier object for building a random forest model<br><br>rfc.fit(X_train,Y_train)  # Train the random forest classifier model on the training data (X_train and Y_train)<br><br>ypred = rfc.predict(X_test)  # Predict the target variable (Y) for the testing data (X_test)<br><br>accuracy_score(Y_test,ypred)  # Evaluate the accuracy of the model predictions by comparing them with the true labels (Y_test)</pre><p>Accuracy — 0.9931818181818182</p><h4>STEP — 7 (USER INPUT)</h4><p>As it is successfully predicting right!! We can check through the dataset given below in the drive link.</p><pre># Take user input for features<br>user_input = {<br>    &#39;N&#39;: float(input(&quot;Enter Nitrogen content (N): &quot;)),<br>    &#39;P&#39;: float(input(&quot;Enter Phosphorus content (P): &quot;)),<br>    &#39;K&#39;: float(input(&quot;Enter Potassium content (K): &quot;)),<br>    &#39;temperature&#39;: float(input(&quot;Enter temperature: &quot;)),<br>    &#39;humidity&#39;: float(input(&quot;Enter humidity: &quot;)),<br>    &#39;ph&#39;: float(input(&quot;Enter pH value: &quot;)),<br>    &#39;rainfall&#39;: float(input(&quot;Enter rainfall: &quot;))<br>}<br><br># Preprocess user input data (scale using StandardScaler)<br>user_input_scaled = scaler.transform([[user_input[&#39;N&#39;], user_input[&#39;P&#39;], user_input[&#39;K&#39;],<br>                                        user_input[&#39;temperature&#39;], user_input[&#39;humidity&#39;],<br>                                        user_input[&#39;ph&#39;], user_input[&#39;rainfall&#39;]]])<br><br># Predict crop label using the trained Random Forest model<br>predicted_crop_label = rfc.predict(user_input_scaled)<br><br># Decode the predicted label using LabelEncoder<br>predicted_crop = label_encoder.inverse_transform(predicted_crop_label)<br><br>print(&quot;Predicted Crop:&quot;, predicted_crop[0])</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/660/1*iSe8RYIZaiQ9YKf0IEkjOw.png" /></figure><h4>STEP — 8 (DEPLOYMENT)</h4><p>💼 By deploying our models, we empower farmers with the ability to access crop recommendations effortlessly, leveraging the power of technology to optimize crop selection, maximize yields, and mitigate risks. 🌱 PyCharm, our trusted development environment, plays a crucial role in this process, providing a robust platform for model deployment and integration with other software components.</p><blockquote>Frontend Code</blockquote><pre>from flask import Flask, render_template, request, jsonify<br>import numpy as np<br>import pandas as pd<br>from sklearn.preprocessing import LabelEncoder, StandardScaler<br>from sklearn.ensemble import RandomForestClassifier<br><br>app = Flask(__name__)<br><br># Load the crop recommendation dataset<br>crop_df = pd.read_csv(&quot;Crop_recommendation.csv&quot;)<br><br># Preprocess the dataset<br>label_encoder = LabelEncoder()<br>crop_df[&#39;label&#39;] = label_encoder.fit_transform(crop_df[&#39;label&#39;])<br>scaler = StandardScaler()<br>crop_df[[&#39;N&#39;, &#39;P&#39;, &#39;K&#39;, &#39;temperature&#39;, &#39;humidity&#39;, &#39;ph&#39;, &#39;rainfall&#39;]] = scaler.fit_transform(<br>    crop_df[[&#39;N&#39;, &#39;P&#39;, &#39;K&#39;, &#39;temperature&#39;, &#39;humidity&#39;, &#39;ph&#39;, &#39;rainfall&#39;]])<br><br># Split features and target variable<br>X = crop_df.drop([&#39;label&#39;], axis=1)<br>Y = crop_df[&#39;label&#39;]<br><br># Train the Random Forest model<br>rf_classifier = RandomForestClassifier()<br>rf_classifier.fit(X, Y)<br><br>@app.route(&#39;/&#39;)<br>def index():<br>    return render_template(&#39;index.html&#39;)<br><br>@app.route(&#39;/predict&#39;, methods=[&#39;POST&#39;])<br>def predict():<br>    data = request.json<br>    user_input = {<br>        &#39;N&#39;: data[&#39;N&#39;],<br>        &#39;P&#39;: data[&#39;P&#39;],<br>        &#39;K&#39;: data[&#39;K&#39;],<br>        &#39;temperature&#39;: data[&#39;temperature&#39;],<br>        &#39;humidity&#39;: data[&#39;humidity&#39;],<br>        &#39;ph&#39;: data[&#39;ph&#39;],<br>        &#39;rainfall&#39;: data[&#39;rainfall&#39;]<br>    }<br>    # Preprocess user input data (scale using StandardScaler)<br>    user_input_scaled = scaler.transform([[user_input[&#39;N&#39;], user_input[&#39;P&#39;], user_input[&#39;K&#39;],<br>                                           user_input[&#39;temperature&#39;], user_input[&#39;humidity&#39;],<br>                                           user_input[&#39;ph&#39;], user_input[&#39;rainfall&#39;]]])<br><br>    # Predict crop label using the trained Random Forest model<br>    predicted_crop_label = rf_classifier.predict(user_input_scaled)<br><br>    # Decode the predicted label using LabelEncoder<br>    predicted_crop = label_encoder.inverse_transform(predicted_crop_label)<br><br>    return jsonify({&#39;predicted_crop&#39;: predicted_crop[0]})<br><br><br>if __name__ == &#39;__main__&#39;:<br>    app.run(debug=True, port=5001)</pre><blockquote>Backend Code</blockquote><pre>&lt;!DOCTYPE html&gt;<br>&lt;html lang=&quot;en&quot;&gt;<br>&lt;head&gt;<br>    &lt;meta charset=&quot;UTF-8&quot;&gt;<br>    &lt;meta name=&quot;viewport&quot; content=&quot;width=device-width, initial-scale=1.0&quot;&gt;<br>    &lt;title&gt;Crop Recommendation&lt;/title&gt;<br>&lt;/head&gt;<br>&lt;body&gt;<br>    &lt;h1&gt;Crop Recommendation System&lt;/h1&gt;<br>    &lt;form id=&quot;cropForm&quot;&gt;<br>        &lt;label for=&quot;N&quot;&gt;Nitrogen content (N):&lt;/label&gt;<br>        &lt;input type=&quot;number&quot; id=&quot;N&quot; name=&quot;N&quot; required&gt;&lt;br&gt;&lt;br&gt;<br>        &lt;label for=&quot;P&quot;&gt;Phosphorus content (P):&lt;/label&gt;<br>        &lt;input type=&quot;number&quot; id=&quot;P&quot; name=&quot;P&quot; required&gt;&lt;br&gt;&lt;br&gt;<br>        &lt;label for=&quot;K&quot;&gt;Potassium content (K):&lt;/label&gt;<br>        &lt;input type=&quot;number&quot; id=&quot;K&quot; name=&quot;K&quot; required&gt;&lt;br&gt;&lt;br&gt;<br>        &lt;label for=&quot;temperature&quot;&gt;Temperature:&lt;/label&gt;<br>        &lt;input type=&quot;number&quot; id=&quot;temperature&quot; name=&quot;temperature&quot; required&gt;&lt;br&gt;&lt;br&gt;<br>        &lt;label for=&quot;humidity&quot;&gt;Humidity:&lt;/label&gt;<br>        &lt;input type=&quot;number&quot; id=&quot;humidity&quot; name=&quot;humidity&quot; required&gt;&lt;br&gt;&lt;br&gt;<br>        &lt;label for=&quot;ph&quot;&gt;pH value:&lt;/label&gt;<br>        &lt;input type=&quot;number&quot; id=&quot;ph&quot; name=&quot;ph&quot; required&gt;&lt;br&gt;&lt;br&gt;<br>        &lt;label for=&quot;rainfall&quot;&gt;Rainfall:&lt;/label&gt;<br>        &lt;input type=&quot;number&quot; id=&quot;rainfall&quot; name=&quot;rainfall&quot; required&gt;&lt;br&gt;&lt;br&gt;<br>        &lt;button type=&quot;submit&quot;&gt;Predict Crop&lt;/button&gt;<br>    &lt;/form&gt;<br>    &lt;div id=&quot;result&quot;&gt;&lt;/div&gt;<br><br>    &lt;script src=&quot;https://code.jquery.com/jquery-3.6.0.min.js&quot;&gt;&lt;/script&gt;<br>    &lt;script&gt;<br>        $(document).ready(function () {<br>            $(&#39;#cropForm&#39;).submit(function (e) {<br>                e.preventDefault();<br>                $.ajax({<br>                    type: &#39;POST&#39;,<br>                    url: &#39;/predict&#39;,<br>                    contentType: &#39;application/json&#39;,<br>                    data: JSON.stringify({<br>                        N: $(&#39;#N&#39;).val(),<br>                        P: $(&#39;#P&#39;).val(),<br>                        K: $(&#39;#K&#39;).val(),<br>                        temperature: $(&#39;#temperature&#39;).val(),<br>                        humidity: $(&#39;#humidity&#39;).val(),<br>                        ph: $(&#39;#ph&#39;).val(),<br>                        rainfall: $(&#39;#rainfall&#39;).val()<br>                    }),<br>                    success: function (response) {<br>                        $(&#39;#result&#39;).text(&#39;Predicted Crop: &#39; + response.predicted_crop);<br>                    },<br>                    error: function (error) {<br>                        console.log(error);<br>                    }<br>                });<br>            });<br>        });<br>    &lt;/script&gt;<br>&lt;/body&gt;<br>&lt;/html&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/665/1*OlrqPWTa4h4OZNl3vkOVzA.png" /><figcaption>Our project has reached its final stage, where users can interact with our crop recommendation system. With a user-friendly interface, individuals can input relevant data into our application.</figcaption></figure><h3>Conclusion</h3><p>In conclusion,💡 By leveraging predictive models like Random Forest, we’ve empowered farmers to optimize crop selection, maximize yields, and mitigate risks, paving the way for sustainable agricultural growth. 🌟 As we bid farewell to this project, we embrace the profound impact of data-driven solutions in shaping the future of farming, fostering resilience, efficiency, and prosperity in agricultural communities worldwide.</p><p>Dataset &amp; Code</p><p><a href="https://drive.google.com/drive/u/0/folders/1Rp3pdHHtviGGOAKxqRYwrqZP7KYBCuz4">crop recommendation - Google Drive</a></p><h3>Thank You!!</h3><p><em>Let’s keep the good vibes rolling! 🚀</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=143c2b124ce3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[“ AgTech-Project” Crop Yield Prediction]]></title>
            <link>https://medium.com/@krishnanayak902bluezone/agtech-project-crop-yield-prediction-e9c2bde3b6be?source=rss-9958de5a3787------2</link>
            <guid isPermaLink="false">https://medium.com/p/e9c2bde3b6be</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[html]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[web-development]]></category>
            <dc:creator><![CDATA[Krishnanayakbluezone]]></dc:creator>
            <pubDate>Sun, 21 Apr 2024 04:30:53 GMT</pubDate>
            <atom:updated>2024-04-21T04:30:53.240Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/650/1*SI0wHJvEPE7RXR7pqAnT4A.png" /></figure><h4>🌱 Welcome to the world of Crop Yield Prediction!! 🌾</h4><p>Ever wondered how farmers predict their harvests? Or how do scientists forecast agricultural productivity? Or how do beginners dive into the world of agrigulture+machine learning?? Get ready for this exciting project where data meets agriculture, and algorithms sow the seeds of insight! 🚜💻</p><p>From analyzing historical data to building predictive models, we’ll explore every step of the process with hands-on examples and colourful visualizations. 📊🌿</p><p>But why should you care about crop yield prediction, you ask? Imagine a world where farmers know exactly when to harvest their crops, governments can give resources just right, and everyone has enough food. It’s like having a magic tool for farming that ensures we all have plenty to eat, no worries needed. 🌍🌾</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/563/1*vZWy5HkexS0a3CMReaxf3g.jpeg" /></figure><h4>Step-1</h4><h3>Importing the important libraries and dataset</h3><p>We start by importing some essential Python libraries: NumPy, Pandas, Seaborn, and Matplotlib. These libraries are crucial for data manipulation and visualization. Next, we prepare for model building by importing specific modules from sci-kit-learn, a powerful machine-learning library in Python. We’re importing functionalities for data preprocessing, such as OneHotEncoder, ColumnTransformer, and StandardScaler, along with tools for model evaluation, like mean_absolute_error, mean_squared_error, and r2_score.</p><p>Finally, we load our dataset, named “yield_df.csv”, into a Pandas DataFrame named “df”. We then sample 8 random rows from the DataFrame to glimpse our data’s structure and contents. This step helps us understand our dataset before diving deeper into analysis and modelling.</p><pre>#import important libraries <br>import numpy as np<br>import pandas as pd<br>import seaborn as sns<br>import matplotlib.pyplot as plt<br><br>#model building <br>from sklearn.preprocessing import OneHotEncoder<br>from sklearn.compose import ColumnTransformer<br>from sklearn.preprocessing import StandardScaler<br>from sklearn.model_selection import train_test_split<br>from sklearn.linear_model import LinearRegression<br>from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score<br>from sklearn.ensemble import RandomForestRegressor<br>from sklearn.tree import DecisionTreeRegressor<br>from sklearn.svm import SVR<br><br>#import datase<br>df = pd.read_csv(&quot;yield_df.csv&quot;)<br>df.sample(8)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/810/1*W2tQlMLJYl6XATKekIMYug.png" /><figcaption>In this dataset, we have information on crop yields across different countries and crop types over multiple years. Each row represents a specific crop entry, with details including the country (“Area”), crop type (“Item”), year of observation (“Year”), and various agricultural factors such as yield (“hg/ha_yield”), average rainfall (“average_rain_fall_mm_per_year”), pesticide usage (“pesticides_tonnes”), and average temperature (“avg_temp”).</figcaption></figure><h4>Step-2</h4><h3>Initial Exploration &amp; Data Cleaning</h3><pre>df.isnull().sum()<br>df.drop(&#39;Unnamed: 0&#39;,axis=1,inplace=True) <br>df.duplicated().sum()<br>df.drop_duplicates(inplace=True)<br>df.duplicated().sum()<br>df = df.rename(columns={&quot;hg/ha_yield&quot;: &quot;yield&quot;})<br>df.info()<br>df.head()<br>df[&quot;Area&quot;].unique()<br>df[&quot;Area&quot;].value_counts()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/631/1*LcChESBYZCSHIHzlS_RM6Q.png" /><figcaption>Here, we’re cleaning up our data to get it ready for analysis. First, we check if there are any missing values. Then, we remove a column called ‘Unnamed: 0’ that we don’t need. Next, we get rid of any duplicate rows in the dataset. After that, we rename a column to make it easier to understand. Finally, we check the basic info about our data and take a quick look at the first few rows. Even we also looked for unique “Area” data to get a better understanding. These steps help us make sure our data is neat and organized for further analysis.</figcaption></figure><h4>Step-3</h4><h3>Exploratory Data Analysis (EDA)</h3><p>EDA reveals insights about crop distribution across countries, contributing significantly. Analysis of yield trends over the years uncovers fluctuations and patterns, while pesticide usage varies among crop types. Understanding these dynamics informs agricultural strategies for improved productivity and sustainability.</p><h4>1. Count of Entries in Each Area</h4><pre><br>plt.figure(figsize=(15, 20))<br># Create a count plot to visualize the distribution of crop entries across different countries<br>sns.countplot(y=df[&#39;Area&#39;])<br>plt.xlabel(&quot;Count&quot;)<br>plt.ylabel(&quot;Area&quot;)<br># Set the title of the plot<br>plt.title(&quot;Count of Entries in Each Area&quot;)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xclNMy7CSByxgZIpDWK3Yg.png" /><figcaption>The count plot gives us a picture of where crop data comes from. India has the most entries, then Brazil, Mexico, Australia, and Pakistan. This means these countries share a lot of agricultural information. On the flip side, places like Estonia, Belgium, Bahrain, Sudan, and Montenegro have fewer entries. Knowing this helps us focus research and help where it’s needed most, which could make a big difference in feeding people around the world.</figcaption></figure><h4>2. Trend of Yield Over the Years</h4><pre># Group the data by Year and calculate the average yield<br>yield_over_years = df.groupby(&#39;Year&#39;)[&#39;yield&#39;].mean()<br><br># Plotting the trend of yield over the years<br>plt.figure(figsize=(10, 6))<br>plt.plot(yield_over_years.index, yield_over_years.values, marker=&#39;o&#39;, linestyle=&#39;-&#39;)<br>plt.title(&#39;Trend of Yield Over the Years&#39;)<br>plt.xlabel(&#39;Year&#39;)<br>plt.ylabel(&#39;Average Yield (hg/ha)&#39;)<br>plt.grid(True)<br>plt.xticks(rotation=45)<br>plt.tight_layout()<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/990/1*C42o3KIgfb6AChWtCCYRLw.png" /><figcaption>The trend of crop yield over the years reveals interesting insights. Overall, we observe fluctuations in yield, with some years showing higher average yields compared to others.</figcaption></figure><h4>3. Distribution of Pesticide Usage by Crop Type</h4><pre>pesticide_by_crop = df.groupby(&#39;Item&#39;)[&#39;pesticides_tonnes&#39;].sum().reset_index()<br><br># Create a pie chart<br>plt.figure(figsize=(10, 8))<br>plt.pie(pesticide_by_crop[&#39;pesticides_tonnes&#39;], labels=pesticide_by_crop[&#39;Item&#39;], autopct=&#39;%1.1f%%&#39;, startangle=140)<br>plt.title(&#39;Distribution of Pesticide Usage by Crop Type&#39;)<br>plt.axis(&#39;equal&#39;)  # Equal aspect ratio ensures that pie is drawn as a circle.<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/794/1*XUgoeOKDUFKtuef96mQ-xA.png" /><figcaption>The pie chart shows how pesticides are used across different crops. Potatoes use the most pesticides, followed by wheat and maize. By knowing this, farmers can use pesticides more wisely, helping crops grow better while being mindful of the environment.</figcaption></figure><h4>4. Country-wise Maximum Yield for Each Crop</h4><pre># Plotting the bar graph<br>plt.figure(figsize=(12, 8))<br>plt.bar(max_yield_per_country[&#39;Area&#39;], max_yield_per_country[&#39;yield&#39;], color=&#39;red&#39;)<br>plt.xlabel(&#39;Country&#39;)<br>plt.ylabel(&#39;Total Yield (hg/ha)&#39;)<br>plt.title(&#39;Country-wise Maximum Yield for Each Crop&#39;)<br>plt.xticks(rotation=90)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1001/1*em1HwylQWTCQ-rcQ6ZZ21Q.png" /><figcaption>In the dataset, India stands out with a high yield, indicating that it produces a significant amount of crops per hectare. Following India, Australia emerges as the second-largest contributor to crop yield.</figcaption></figure><h4>5. Finding the index of maximum yield for each country</h4><pre>country_crop_yield = df.groupby([&#39;Area&#39;, &#39;Item&#39;])[&#39;yield&#39;].sum().reset_index()<br><br># Finding the index of maximum yield for each country<br>idx_max_yield = country_crop_yield.groupby(&#39;Area&#39;)[&#39;yield&#39;].idxmax()<br><br># Getting the rows with maximum yield for each country<br>max_yield_per_country = country_crop_yield.loc[idx_max_yield]<br><br># Displaying the result<br>print(max_yield_per_country)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/416/1*ZA-O7alFGuPkBEk37GDdrA.png" /></figure><h4>6. Yield Variability Across Different Crop Types</h4><pre># Boxplot of yield variability across different crop types<br>plt.figure(figsize=(10, 6))<br>df.boxplot(column=&#39;yield&#39;, by=&#39;Item&#39;, grid=False, rot=90)<br>plt.title(&#39;Yield Variability Across Different Crop Types&#39;)<br>plt.xlabel(&#39;Crop Type&#39;)<br>plt.ylabel(&#39;Yield (hg/ha)&#39;)<br>plt.suptitle(&#39;&#39;) <br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/614/1*-sHYY5ilb8YRcO2whg1WCg.png" /><figcaption>Some crops exhibit wider ranges of yield, indicating greater susceptibility to fluctuations, while others show more consistent yields. For farmers and policymakers, understanding these variations is crucial for planning and decision-making. Crops with higher variability may require more robust management practices to mitigate risks and ensure stable production.</figcaption></figure><h4>7. Relationship between Yield and Average Rainfall per Year</h4><pre># Calculate the correlation between crop yield and average rainfall per year<br>correlation = df[&#39;yield&#39;].corr(df[&#39;average_rain_fall_mm_per_year&#39;])<br><br># Print the correlation coefficient<br>print(&quot;Correlation between yield and average rainfall per year:&quot;, correlation)<br><br># Create a scatter plot to visualize the relationship between yield and average rainfall<br>plt.figure(figsize=(8, 6))<br>plt.scatter(df[&#39;average_rain_fall_mm_per_year&#39;], df[&#39;yield&#39;], color=&#39;lightpink&#39;, alpha=0.5)<br>plt.title(&#39;Relationship between Yield and Average Rainfall per Year&#39;)<br>plt.xlabel(&#39;Average Rainfall (mm/year)&#39;)<br>plt.ylabel(&#39;Yield (hg/ha)&#39;)<br>plt.grid(True)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/722/1*LdG89o7zlV-vp8zkQW1QMg.png" /><figcaption>Correlation between yield and average rainfall per year: -0.00406709743340259. The correlation between crop yield and average rainfall per year is very weak, with a coefficient of approximately -0.004. This suggests that there is almost no discernible relationship between these two variables in the dataset. In simpler terms, it means that changes in average rainfall don’t seem to have a significant impact on crop yield.</figcaption></figure><h4>8. Relationship between Yield and Pesticides</h4><pre># Calculate correlation between yield and pesticides_tonnes<br>correlation = df[&#39;yield&#39;].corr(df[&#39;pesticides_tonnes&#39;])<br><br>print(&quot;Correlation between yield and pesticides_tonnes:&quot;, correlation)<br>plt.figure(figsize=(8, 6))<br>plt.scatter(df[&#39;pesticides_tonnes&#39;], df[&#39;yield&#39;], alpha=0.5)<br>plt.title(&#39;Relationship between Yield and Pesticides&#39;)<br>plt.xlabel(&#39;Pesticides (tonnes)&#39;)<br>plt.ylabel(&#39;Yield (hg/ha)&#39;)<br>plt.grid(True)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/722/1*fC68J5KGMHaPjR40ID457g.png" /><figcaption>Correlation between yield and pesticides_tonnes: 0.06552335650655518. The correlation coefficient between crop yield and pesticide usage, at 0.0655, suggests a weak positive relationship between the two variables. This means that there is a slight tendency for higher pesticide usage to be associated with slightly higher crop yields, but the relationship is not strong.</figcaption></figure><h4>9. Relationship between Yield and Average Temperature</h4><pre># Calculate correlation between yield and pesticides_tonnes<br>correlation = df[&#39;yield&#39;].corr(df[&#39;avg_temp&#39;])<br><br>print(&quot;Correlation between yield and avg_temp:&quot;, correlation)<br>plt.figure(figsize=(8, 6))<br>plt.scatter(df[&#39;avg_temp&#39;], df[&#39;yield&#39;], alpha=0.5)<br>plt.title(&#39;Relationship between Yield and Average Temperature&#39;)<br>plt.xlabel(&#39;Average Temperature&#39;)<br>plt.ylabel(&#39;Yield (hg/ha)&#39;)<br>plt.grid(True)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/722/1*s3zBjyypj-Oyn80-dFvS2A.png" /><figcaption>Correlation between yield and avg_temp: -0.1103998244302233. The correlation coefficient of -0.11 indicates a weak negative relationship between crop yield and average temperature. This means that as the average temperature increases, there is a slight tendency for crop yields to decrease, although the correlation is not very strong.</figcaption></figure><blockquote>{ Note — In addition more visualization, have been conducted for each crop individually, providing a deeper and more comprehensive understanding of the relationship between yield and environmental factors, that you will get in the code file. Kindly go though it. }</blockquote><h3>Preprocessing Data</h3><p>Preprocessing data plays a crucial role in this project by standardizing and transforming the input features to a suitable format for machine learning algorithms. Scaling numerical features ensures that variables with different scales contribute equally to the model, preventing dominance by larger values. One-hot encoding categorical variables converts them into a numerical format understandable by algorithms, avoiding bias towards numerical representations.</p><pre># Selecting the relevant columns for analysis<br>col = [&#39;Year&#39;, &#39;average_rain_fall_mm_per_year&#39;,&#39;pesticides_tonnes&#39;, &#39;avg_temp&#39;, &#39;Area&#39;, &#39;Item&#39;, &#39;yield&#39;]<br>df = df[col]<br><br># Separating features (X) and target variable (y)<br>X = df.iloc[:, :-1]  # Features (independent variables)<br>y = df.iloc[:, -1]   # Target variable (dependent variable)<br><br># Displaying the first 5 rows of the modified DataFrame<br>df.head(5)<br><br># Splitting the data into training and testing sets<br># train_size=0.8 specifies that 80% of the data will be used for training, and random_state=0 ensures reproducibility<br>X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=0.8, random_state=0, shuffle=True)</pre><pre># Initialize OneHotEncoder to encode categorical variables and drop the first category to avoid multicollinearity<br>ohe = OneHotEncoder(drop=&#39;first&#39;)<br><br># Initialize StandardScaler to scale numerical features to have zero mean and unit variance<br>scale = StandardScaler()<br><br># Create a ColumnTransformer to apply transformations to different columns<br>preprocesser = ColumnTransformer(<br>    transformers = [<br>        (&#39;StandardScale&#39;, scale, [0, 1, 2, 3]),  # Apply StandardScaler to numerical features<br>        (&#39;OHE&#39;, ohe, [4, 5]),  # Apply OneHotEncoder to categorical features<br>    ],<br>    remainder=&#39;passthrough&#39;  # Passthrough remaining columns without any transformation<br>)</pre><pre># Apply the preprocessing transformations to the training data<br>X_train_dummy = preprocesser.fit_transform(X_train)<br><br># Apply the same transformations to the testing data<br>X_test_dummy = preprocesser.transform(X_test)<br><br># Get the feature names after preprocessing<br># col[:-1] selects all columns except the target variable &#39;yield&#39;<br>preprocesser.get_feature_names_out(col[:-1])</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/667/1*Wpd67weml601yYDBCNwyWA.png" /></figure><h3>Model Building</h3><h4>— Linear Regression</h4><pre># Transform the data using the preprocesser<br>X_train_dummy = preprocesser.fit_transform(X_train)<br>X_test_dummy = preprocesser.transform(X_test)<br><br># Initialize and train the linear regression model<br>lr = LinearRegression()<br>lr.fit(X_train_dummy, y_train)<br><br># Predict on the training and test set<br>y_train_pred = lr.predict(X_train_dummy)<br>y_test_pred = lr.predict(X_test_dummy)<br><br># Calculate RMSE for training and test set<br>train_rmse_lr = mean_squared_error(y_train, y_train_pred, squared=False)<br>test_rmse_lr = mean_squared_error(y_test, y_test_pred, squared=False)<br><br># Print RMSE values<br>print(f&quot;Train RMSE: {train_rmse_lr}&quot;)<br>print(f&quot;Test RMSE: {test_rmse_lr}\n&quot;)<br><br># Calculate R squared for training and test set<br>train_r_squared = lr.score(X_train_dummy, y_train) * 100<br>test_r_squared = lr.score(X_test_dummy, y_test) * 100<br><br># Print R squared values<br>print(&quot;R squared for train is :&quot;, train_r_squared)<br>print(&quot;R squared for test is :&quot;, test_r_squared)</pre><p>Train RMSE: 42143.63169112969<br>Test RMSE: 42630.26417111315</p><p>R squared for train is : 75.18631987907828<br>R squared for test is : 74.7313043466613</p><h4>— RandomForestRegressor</h4><pre>rf = RandomForestRegressor()<br>rf.fit(X_train_dummy, y_train)<br><br># Predict on the training and test set<br>y_train_pred = rf.predict(X_train_dummy)<br>y_test_pred = rf.predict(X_test_dummy)<br><br># Calculate RMSE for training and test set<br>train_rmse_rf = mean_squared_error(y_train, y_train_pred, squared=False)<br>test_rmse_rf = mean_squared_error(y_test, y_test_pred, squared=False)<br><br># Print RMSE values<br>print(f&quot;Train RMSE: {train_rmse_rf}&quot;)<br>print(f&quot;Test RMSE: {test_rmse_rf}\n&quot;)<br><br># Calculate R squared for training and test set<br>train_r_squared = rf.score(X_train_dummy, y_train) * 100<br>test_r_squared = rf.score(X_test_dummy, y_test) * 100<br><br># Print R squared values<br>print(&quot;R squared for train is :&quot;, train_r_squared)<br>print(&quot;R squared for test is :&quot;, test_r_squared)</pre><p>Train RMSE: 3596.5011618917506<br>Test RMSE: 9285.621143890698</p><p>R squared for train is : 99.81928771657799<br>R squared for test is : 98.80113937242379</p><h4>— DecisionTreeRegressor</h4><pre>dt = DecisionTreeRegressor()<br>dt.fit(X_train_dummy, y_train)<br><br># Predict on the training and test set<br>y_train_pred = dt.predict(X_train_dummy)<br>y_test_pred = dt.predict(X_test_dummy)<br><br># Calculate RMSE for training and test set<br>train_rmse_dt= mean_squared_error(y_train, y_train_pred, squared=False)<br>test_rmse_dt = mean_squared_error(y_test, y_test_pred, squared=False)<br><br># Print RMSE values<br>print(f&quot;Train RMSE: {train_rmse_dt}&quot;)<br>print(f&quot;Test RMSE: {test_rmse_dt}\n&quot;)<br><br># Calculate R squared for training and test set<br>train_r_squared = dt.score(X_train_dummy, y_train) * 100<br>test_r_squared = dt.score(X_test_dummy, y_test) * 100<br><br># Print R squared values<br>print(&quot;R squared for train is :&quot;, train_r_squared)<br>print(&quot;R squared for test is :&quot;, test_r_squared)</pre><p>Train RMSE: 0.0<br>Test RMSE: 11769.989000124096</p><p>R squared for train is : 100.0<br>R squared for test is : 98.07381103375783</p><h4>— Support Vector Regression (SVR)</h4><pre>svr = SVR()<br>svr.fit(X_train_dummy, y_train)<br><br># Predict on the training and test set<br>y_train_pred = svr.predict(X_train_dummy)<br>y_test_pred = svr.predict(X_test_dummy)<br><br># Calculate RMSE for training and test set<br>train_rmse_svr= mean_squared_error(y_train, y_train_pred, squared=False)<br>test_rmse_svr= mean_squared_error(y_test, y_test_pred, squared=False)<br><br># Print RMSE values<br>print(f&quot;Train RMSE: {train_rmse_svr}&quot;)<br>print(f&quot;Test RMSE: {test_rmse_svr}\n&quot;)<br><br># Calculate R squared for training and test set<br>train_r_squared = svr.score(X_train_dummy, y_train) * 100<br>test_r_squared = svr.score(X_test_dummy, y_test) * 100<br><br># Print R squared values<br>print(&quot;R squared for train is :&quot;, train_r_squared)<br>print(&quot;R squared for test is :&quot;, test_r_squared)</pre><p>Train RMSE: 92312.4932040058<br>Test RMSE: 93125.93379849613</p><p>R squared for train is : -19.055160551004448<br>R squared for test is : -20.583547930779588</p><h3>Finding the Best Fit Model</h3><pre>import matplotlib.pyplot as plt<br><br># Store performance metrics of each model<br>models_performance = {<br>    &quot;Decision Tree&quot;: (test_rmse_dt, dt.score(X_test_dummy, y_test) * 100),<br>    &quot;Random Forest&quot;: (test_rmse_rf, rf.score(X_test_dummy, y_test) * 100),<br>    &quot;Linear Regression&quot;: (test_rmse_lr, lr.score(X_test_dummy, y_test) * 100),<br>    &quot;Support Vector Regressor&quot;: (test_rmse_svr, svr.score(X_test_dummy, y_test) * 100)<br>}<br><br># Sort the models based on RMSE and R squared<br>sorted_models = sorted(models_performance.items(), key=lambda x: (x[1][0], -x[1][1]))<br><br># Print sorted models and their performance metrics<br>print(&quot;Models sorted by RMSE and R squared (Test Set):&quot;)<br>for model, (rmse, r2) in sorted_models:<br>    print(f&quot;{model}: RMSE = {rmse}, R squared = {r2}&quot;)<br><br># Choose the best model<br>best_model_name, (best_rmse, best_r2) = sorted_models[0]<br>print(f&quot;\nBest Model: {best_model_name}&quot;)<br>print(f&quot;RMSE: {best_rmse}&quot;)<br>print(f&quot;R squared: {best_r2}&quot;)<br><br># Visualize the performance of the models using a bar graph<br>model_names = [model[0] for model in sorted_models]<br>r2_values = [model[1][1] for model in sorted_models]<br><br>plt.figure(figsize=(10, 6))<br>plt.barh(model_names, r2_values, color=&#39;skyblue&#39;)<br>plt.xlabel(&#39;R squared (%)&#39;)<br>plt.title(&#39;Model Performance Comparison&#39;)<br>plt.gca().invert_yaxis()<br>plt.show()</pre><p>Best Model: Random Forest<br>RMSE: 9285.621143890698<br>R squared: 98.80113937242379</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/981/1*xKHbEY7_ZsQYeuVd7bggyw.png" /><figcaption>The performance evaluation of four regression models reveals significant differences in predictive accuracy. Among them, the Random Forest model emerges as the most suitable choice, demonstrating the lowest RMSE of 10970.15 and the highest R squared value of 98.34%.</figcaption></figure><h3>Taking User Input</h3><pre>def prediction(Year, average_rain_fall_mm_per_year, pesticides_tonnes, avg_temp, Area, Item):<br>    # Assuming preprocesser and rf are defined elsewhere in your code<br>    # Create an array of the input features<br>    features = np.array([[Year, average_rain_fall_mm_per_year, pesticides_tonnes, avg_temp, Area, Item]], dtype=object)<br><br>    # Transform the features using the preprocessor<br>    transformed_features = preprocesser.transform(features)<br><br>    # Make the prediction<br>    predicted_yield = rf.predict(transformed_features).reshape(1, -1)<br><br>    return predicted_yield[0]<br><br># Taking user input<br>Year = int(input(&quot;Enter the year: &quot;))<br>average_rain_fall_mm_per_year = float(input(&quot;Enter the average rainfall in mm per year: &quot;))<br>pesticides_tonnes = float(input(&quot;Enter the pesticides in tonnes: &quot;))<br>avg_temp = float(input(&quot;Enter the average temperature: &quot;))<br>Area = input(&quot;Enter the area: &quot;)<br>Item = input(&quot;Enter the item: &quot;)<br><br># Calling the prediction function with user input<br>result = prediction(Year, average_rain_fall_mm_per_year, pesticides_tonnes, avg_temp, Area, Item)<br>print(&quot;Predicted yield:&quot;, result)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/634/1*8NngpA5zTInnvVBf8khKEw.png" /><figcaption>In our analysis, the model’s predictions closely align with the actual values, indicating a high level of accuracy and reliability. This alignment suggests that the model has effectively captured the underlying patterns and relationships within the dataset.</figcaption></figure><h3>Deployment (PyCharm)</h3><pre>import pickle<br>pickle.dump(rf,open(&#39;rf.pkl&#39;,&#39;wb&#39;))<br>pickle.dump(preprocesser,open(&#39;preprocessor.pkl&#39;,&#39;wb&#39;))</pre><p>For deployment, we utilize the pickle module in Python to serialize our trained Random Forest model (rf) and preprocessing pipeline (preprocessor). Serialization converts the objects into byte streams, which can be easily stored and later reconstructed. We save these serialized objects as files with the .pkl extension. During deployment, we extract these pickle files and load them into memory within our Flask application, which serves as the backend.</p><blockquote>#for backend</blockquote><pre><br>from flask import Flask,request, render_template<br>import numpy as np<br>import pandas as pd<br>import pickle<br>import sklearn<br><br>#loading models<br>rf = pickle.load(open(&#39;rf.pkl&#39;,&#39;rb&#39;))<br>preprocessor = pickle.load(open(&#39;preprocessor (1).pkl&#39;,&#39;rb&#39;))<br><br>#flask app<br>app = Flask(__name__)<br><br><br>@app.route(&#39;/&#39;)<br>def index():<br>    return render_template(&#39;index.html&#39;)<br>@app.route(&quot;/predict&quot;,methods=[&#39;POST&#39;])<br>def predict():<br>    if request.method == &#39;POST&#39;:<br>        Year = request.form[&#39;Year&#39;]<br>        average_rain_fall_mm_per_year = request.form[&#39;average_rain_fall_mm_per_year&#39;]<br>        pesticides_tonnes = request.form[&#39;pesticides_tonnes&#39;]<br>        avg_temp = request.form[&#39;avg_temp&#39;]<br>        Area = request.form[&#39;Area&#39;]<br>        Item  = request.form[&#39;Item&#39;]<br><br>        features = np.array([[Year,average_rain_fall_mm_per_year,pesticides_tonnes,avg_temp,Area,Item]],dtype=object)<br>        transformed_features = preprocessor.transform(features)<br>        predicted_value = rf.predict(transformed_features).reshape(1,-1)<br><br>        return render_template(&#39;index.html&#39;,predicted_value=predicted_value)<br><br>#python main<br>if __name__==&quot;__main__&quot;:<br>    app.run(debug=True)</pre><p>This Python code implements a Flask web application for crop yield prediction. It loads a trained Random Forest model and a preprocessor using pickle. The app defines routes for rendering the index page and handling form submissions for prediction.</p><blockquote>#for frontend</blockquote><pre>&lt;!DOCTYPE html&gt;<br>&lt;html lang=&quot;en&quot;&gt;<br>&lt;head&gt;<br>    &lt;meta charset=&quot;UTF-8&quot;&gt;<br>    &lt;meta name=&quot;viewport&quot; content=&quot;width=device-width, initial-scale=1.0&quot;&gt;<br>    &lt;title&gt;Crop Yield Prediction&lt;/title&gt;<br>    &lt;link href=&quot;https://cdn.jsdelivr.net/npm/bootstrap@5.3.0/dist/css/bootstrap.min.css&quot; rel=&quot;stylesheet&quot; integrity=&quot;sha384-9ndCyUaIbzAi2FUVXJi0CjmCapSmO7SnpJef0486qhLnuZ2cdeRhO02iuK6FUUVM&quot; crossorigin=&quot;anonymous&quot;&gt;<br>&lt;/head&gt;<br>&lt;body&gt;<br>    &lt;div class=&quot;container mt-5&quot;&gt;<br>        &lt;h1 class=&quot;text-center text-success&quot;&gt;Crop Yield Prediction Per Country&lt;/h1&gt;<br>        &lt;div class=&quot;row justify-content-center mt-5&quot;&gt;<br>            &lt;div class=&quot;col-md-6&quot;&gt;<br>                &lt;form action=&quot;{{ url_for(&#39;predict&#39;) }}&quot; method=&quot;post&quot;&gt;<br>                    &lt;div class=&quot;mb-3&quot;&gt;<br>                        &lt;label for=&quot;Year&quot; class=&quot;form-label&quot;&gt;Year&lt;/label&gt;<br>                        &lt;input type=&quot;number&quot; class=&quot;form-control&quot; id=&quot;Year&quot; name=&quot;Year&quot; step=&quot;any&quot; value=&quot;2013&quot;&gt;<br>                    &lt;/div&gt;<br>                    &lt;div class=&quot;mb-3&quot;&gt;<br>                        &lt;label for=&quot;average_rain_fall_mm_per_year&quot; class=&quot;form-label&quot;&gt;Average Rain Fall (mm per year)&lt;/label&gt;<br>                        &lt;input type=&quot;number&quot; class=&quot;form-control&quot; id=&quot;average_rain_fall_mm_per_year&quot; name=&quot;average_rain_fall_mm_per_year&quot; step=&quot;any&quot;&gt;<br>                    &lt;/div&gt;<br>                    &lt;div class=&quot;mb-3&quot;&gt;<br>                        &lt;label for=&quot;pesticides_tonnes&quot; class=&quot;form-label&quot;&gt;Pesticides (Tonnes)&lt;/label&gt;<br>                        &lt;input type=&quot;number&quot; class=&quot;form-control&quot; id=&quot;pesticides_tonnes&quot; name=&quot;pesticides_tonnes&quot; step=&quot;any&quot;&gt;<br>                    &lt;/div&gt;<br>                    &lt;div class=&quot;mb-3&quot;&gt;<br>                        &lt;label for=&quot;avg_temp&quot; class=&quot;form-label&quot;&gt;Average Temperature&lt;/label&gt;<br>                        &lt;input type=&quot;number&quot; class=&quot;form-control&quot; id=&quot;avg_temp&quot; name=&quot;avg_temp&quot; step=&quot;any&quot;&gt;<br>                    &lt;/div&gt;<br>                    &lt;div class=&quot;mb-3&quot;&gt;<br>                        &lt;label for=&quot;Area&quot; class=&quot;form-label&quot;&gt;Area&lt;/label&gt;<br>                        &lt;input type=&quot;text&quot; class=&quot;form-control&quot; id=&quot;Area&quot; name=&quot;Area&quot;&gt;<br>                    &lt;/div&gt;<br>                    &lt;div class=&quot;mb-3&quot;&gt;<br>                        &lt;label for=&quot;Item&quot; class=&quot;form-label&quot;&gt;Item&lt;/label&gt;<br>                        &lt;input type=&quot;text&quot; class=&quot;form-control&quot; id=&quot;Item&quot; name=&quot;Item&quot;&gt;<br>                    &lt;/div&gt;<br>                    &lt;button type=&quot;submit&quot; class=&quot;btn btn-primary&quot;&gt;Predict&lt;/button&gt;<br>                &lt;/form&gt;<br>            &lt;/div&gt;<br>        &lt;/div&gt;<br>        {% if predicted_value %}<br>        &lt;div class=&quot;row justify-content-center mt-5&quot;&gt;<br>            &lt;div class=&quot;col-md-6&quot;&gt;<br>                &lt;h2 class=&quot;text-center&quot;&gt;Predicted Yield:&lt;/h2&gt;<br>                &lt;h3 class=&quot;text-center&quot;&gt;{{ predicted_value }}&lt;/h3&gt;<br>            &lt;/div&gt;<br>        &lt;/div&gt;<br>        {% endif %}<br>    &lt;/div&gt;<br>&lt;/body&gt;<br>&lt;/html&gt;<br></pre><p>This HTML code creates a web form for users to input data for crop yield prediction. It includes fields for the year, average rainfall, pesticide usage, average temperature, area, and crop items. Upon submission, the form sends a POST request to the server. If a predicted yield value is available, it is displayed on the page. This form is styled using Bootstrap for responsiveness and aesthetic appeal, with input validation for numeric fields.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/499/1*naUhE5N_uK4IkyZSX4sBYA.jpeg" /><figcaption>Our project has reached its final stage, where users can interact with our crop yield prediction system. With a user-friendly interface, individuals can input relevant data, such as year, rainfall, pesticides, and temperature, into our application.</figcaption></figure><h3>Conclusion</h3><p>This crop yield prediction project offers a valuable opportunity for both learning and practical application. By leveraging machine learning techniques and web development skills, you can build a user-friendly interface that provides insightful predictions for agricultural outcomes. Whether you’re a beginner or an experienced developer, diving into this project can be a rewarding journey of learning and innovation.🌾💡</p><p><strong>Dataset &amp; Code</strong></p><p><a href="https://drive.google.com/drive/u/0/folders/14ZNuwIyH5z2HcamwrRgpK9gcFBRfJgMz">https://drive.google.com/drive/u/0/folders/14ZNuwIyH5z2HcamwrRgpK9gcFBRfJgMz</a></p><h3>Thank YOU!!</h3><blockquote>Let’s keep the good vibes rolling! 🚀</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e9c2bde3b6be" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[“Hotel Rating Classification” — Machine Learning Project]]></title>
            <link>https://medium.com/@krishnanayak902bluezone/hotel-rating-classification-machine-learning-project-eb7af7a7d0f1?source=rss-9958de5a3787------2</link>
            <guid isPermaLink="false">https://medium.com/p/eb7af7a7d0f1</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[tech-writer]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Krishnanayakbluezone]]></dc:creator>
            <pubDate>Thu, 04 Apr 2024 07:05:57 GMT</pubDate>
            <atom:updated>2024-04-05T05:25:26.701Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>“Hotel Rating Classification” — Machine Learning Project!!</strong></h3><p>Welcome to my very first project blog! 🎉 Step into the fascinating world of hospitality, where understanding guests’ thoughts and feelings is key to ensuring unforgettable experiences. I’m beyond excited to share with you my data science project, “Hotel Rating Classification.” Together, we’ll embark on a journey into the realm of hotel reviews, uncovering insights into guests’ sentiments. Armed with clever machine learning techniques and the power of sentiment analysis, I’ve delved into vast datasets of reviews and ratings, deciphering whether they’re glowing with positivity, tinged with negativity, or somewhere in between. Come along as we unravel the mysteries behind what truly makes guests happy (or not so happy) during their hotel stays!</p><h3><strong>Importing important Libraries and Dataset</strong></h3><p>This code brings important tools to help us effectively work with our data. We use Pandas and NumPy to handle our data, Matplotlib and Seaborn to create visual graphs, and Plotly Express for more interactive visuals. With NLTK, we can process our text data by breaking it down into words, removing unnecessary words, and simplifying words to their base form. The Imbalanced Learn library helps us deal with unequal groups in our data. And scikit-learn provides various tools for machine learning, like turning our text data into numbers that our models can understand. These tools are essential for our “Hotel Rating Classification” project, making our data prep and model building smooth and efficient.</p><pre># import Libraries<br>import pandas as pd<br>import numpy as np<br>import matplotlib.pyplot as plt<br>import seaborn as sns<br>import plotly.express as px<br>import warnings<br>warnings.filterwarnings(&#39;ignore&#39;)<br>%matplotlib inline<br><br># data preprocessing<br>import nltk<br>nltk.download(&#39;wordnet&#39;)<br>nltk.download(&#39;stopwords&#39;)<br>nltk.download(&#39;punkt&#39;)<br>import string<br>from nltk.corpus import stopwords<br>from nltk.stem import WordNetLemmatizer<br>from nltk.tokenize import sent_tokenize, word_tokenize<br>import re<br>from collections import Counter<br>from wordcloud import WordCloud<br>from imblearn.over_sampling import RandomOverSampler<br>from sklearn.feature_extraction.text import TfidfVectorizer<br>from sklearn.model_selection import train_test_split<br>from sklearn.preprocessing import LabelEncoder<br>from sklearn.linear_model import LogisticRegression<br>from sklearn.naive_bayes  import MultinomialNB<br>from sklearn.svm import LinearSVC<br>from sklearn.metrics import accuracy_score,confusion_matrix,classification_report,precision_score<br><br>#Load dataset<br>data=pd.read_excel(&#39;hotel_reviews (2).xlsx&#39;)<br>data.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/722/1*brg0uG9aRe_WuAQs7qyGkg.png" /><figcaption>This line of code removes a column labelled ‘@’ from your dataset ‘data’. By specifying, the modification is applied directly to your original dataset, without creating a new one. After executing this code, the column named &#39;@&#39; in your dataset, will no longer be present in the data. This action effectively alters the structure of your dataset by eliminating the specified unwanted column, which could impact any subsequent analysis or modelling.</figcaption></figure><h3>Descriptive Statistics</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/650/1*5vhmUQaYwIMv1okkT_nFxQ.png" /><figcaption>This code offers a snapshot of our hotel reviews dataset, providing key insights into its structure and content. We learn that our dataset consists of 20,491 entries (rows) with 2 columns: “Review” and “Rating.” The “Review” column holds text data, likely guest comments or feedback, while the “Rating” column contains numerical data, likely star ratings. It’s reassuring to know that there are no missing values or duplicate entries in our dataset, ensuring its integrity. These descriptive statistics lay the groundwork for our analysis, clearly understanding our data’s composition before diving deeper.</figcaption></figure><h3>EDA (Exploratory Data Analysis)</h3><p>Now coming to the main part of the project!! It is exploratory data analysis (EDA), we take a deep dive into our hotel reviews dataset to find most important patterns and trends. When we look at how ratings are spread out, we see that most reviews give high ratings, showing that guests are generally happy. By checking which booking sites people use the most, we learn about their preferences. We also look at specific feedback about parking, costs, and types of trips to understand what guests like or don’t like. Through graphs and comparing different aspects, like review length and ratings, we get a better idea of what matters most to guests. This helps us make smarter decisions in our project.</p><p>So, let’s go into the insights one by one:</p><h4><strong>1. Distribution Of Ratings</strong></h4><pre>data[&#39;Rating&#39;].value_counts()<br><br># Plotting the distribution of ratings as a pie chart<br>plt.figure(figsize=(8, 8))<br>labels = data[&#39;Rating&#39;].value_counts().index<br>sizes = data[&#39;Rating&#39;].value_counts().values<br>plt.pie(sizes, labels=labels, autopct=&#39;%1.1f%%&#39;, startangle=140, colors=[&#39;blue&#39;, &#39;pink&#39;, &#39;yellow&#39;, &#39;green&#39;, &#39;red&#39;])<br>plt.title(&#39;Distribution of Ratings&#39;)<br>plt.axis(&#39;equal&#39;) <br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/544/1*CBDVZ1HVWNodqiHXS2Mp-Q.png" /><figcaption>Most reviews give hotels high ratings, like 4 or 5 stars, which make up about 75% of all the feedback. This suggests that guests mostly feel very positive about their experiences. But even though there’s a lot of positivity, it’s important to remember that some people leave negative reviews too. Because there are so many good ratings, it might mean that people are more likely to leave positive reviews. So, it’s super important for hotels to pay attention to both the good and bad comments. By listening to what guests say, hotels can make sure everyone has a great experience, keeping their reputation strong in the long run.</figcaption></figure><h4>2. Top 5 Booking Sites</h4><pre>booking_sites = data[&#39;Review&#39;].str.extract(r&#39;(\w+\.com)&#39;)[0].value_counts()<br>plt.figure(figsize=(10,5))<br>sns.barplot(y=booking_sites.head(5).index, x=booking_sites.head(5))<br>plt.title(&#39;top 5 websites for booking an hotel&#39;)<br>plt.xlabel(&#39; No of customers using the website to book the hotel&#39;)<br>plt.ylabel(&#39;websites name&#39;)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/761/1*B6xhbuc1mI1mJamilk8rqQ.png" /><figcaption>Analyzing the graph above, we observe that Hotels.com and TripAdvisor.com emerge as the top choices for booking hotels, with 41 and 33 mentions respectively. This indicates a strong trust among travellers towards these platforms for their travel needs. Interestingly, we also notice the presence of Hotmail.com and Yahoo.com, suggesting the influence of email promotions in travellers’ booking decisions. This underscores the significance of user reviews and user-friendly booking processes in shaping travellers’ choices. Overall, the data underscores the pivotal role of online platforms and reviews in the modern travel landscape.</figcaption></figure><h4>3. Parking Feedback</h4><pre>parking_feedback = data[data[&#39;Review&#39;].str.contains(&#39;parking&#39;, case=False)]<br><br># Count positive and negative parking feedback<br>positive_parking_feedback = parking_feedback[parking_feedback[&#39;Rating&#39;] &gt;= 3]<br>negative_parking_feedback = parking_feedback[parking_feedback[&#39;Rating&#39;] &lt; 3]<br><br># Plot the graph<br>plt.figure(figsize=(8, 5))<br>plt.bar([&#39;Positive&#39;, &#39;Negative&#39;], [len(positive_parking_feedback), len(negative_parking_feedback)], color=[&#39;skyblue&#39;, &#39;red&#39;])<br>plt.xlabel(&#39;Feedback&#39;)<br>plt.ylabel(&#39;Number of Reviews&#39;)<br>plt.title(&#39;Parking Feedback&#39;)<br>plt.grid(axis=&#39;y&#39;, linestyle=&#39;--&#39;, alpha=0.7)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/619/1*VTyzKJRmlLLiArds2lx1Zg.png" /><figcaption>Looking at our graph, it’s obvious that many people have opinions about the hotel parking. Over 900 guests gave positive feedback, while around 218 weren’t satisfied. This means most guests were happy with the parking, but some weren’t. The hotel needs to consider both positive and negative feedback to ensure everyone has a good parking experience. Plus, they can use the positive feedback to highlight their parking facilities and attract more guests. It’s all about making sure every guest feels valued and comfortable during their stay.</figcaption></figure><h4>4. Expensive Vs Affordable</h4><pre>expensive_reviews = data[data[&#39;Review&#39;].str.contains(&#39;expensive|pricey&#39;, case=False)]<br>affordable_reviews = data[data[&#39;Review&#39;].str.contains(&#39;affordable|cheap&#39;, case=False)]<br><br># Count the number of reviews mentioning each sentiment<br>expensive_count = expensive_reviews.shape[0]<br>affordable_count = affordable_reviews.shape[0]<br><br>print(&quot;Number of people saying it&#39;s expensive:&quot;, expensive_count)<br>print(&quot;Number of people saying it&#39;s affordable:&quot;, affordable_count)<br><br>plt.figure(figsize=(8, 5))<br>plt.bar([&#39;Expensive&#39;, &#39;Affordable&#39;], [expensive_count, affordable_count], color=[&#39;green&#39;, &#39;yellow&#39;])<br>plt.title(&#39;Perceived Cost of the Hotel&#39;)<br>plt.ylabel(&#39;Number of Reviews&#39;)<br>plt.grid(axis=&#39;y&#39;)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/649/1*LvW-nO_1hBf63xlsgsGOFg.png" /><figcaption>Looking at the graph, we see that many people, around 2,302, think the hotel is pricey. On the flip side, about 1,686 people consider it affordable. This difference shows that everyone has their own idea of what’s expensive or not. While some guests think the prices are fair, others think they’re too high. This tells us that hotels need to consider different guests’ views on pricing. By doing this, they can adjust their prices to make sure they’re fair for everyone. It’s all about finding the right balance between what guests think is worth paying and what they can afford.</figcaption></figure><h4>5. Family Trip Vs Business Trip</h4><pre>business_trip_count = data[data[&#39;Review&#39;].str.contains(&#39;business trip&#39;, case=False)].shape[0]<br>family_trip_count = data[data[&#39;Review&#39;].str.contains(&#39;family trip&#39;, case=False)].shape[0]<br><br>print(&quot;Number of people coming for a business trip:&quot;, business_trip_count)<br>print(&quot;Number of people coming for a family trip:&quot;, family_trip_count)<br><br>plt.figure(figsize=(8, 5))<br>plt.bar([&#39;Business Trip&#39;, &#39;Family Trip&#39;], [business_trip_count, family_trip_count], color=[&#39;purple&#39;, &#39;pink&#39;])<br>plt.title(&#39;Number of People by Trip Type&#39;)<br>plt.ylabel(&#39;Count&#39;)<br>plt.grid(axis=&#39;y&#39;)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/606/1*paqaQkWuXrnVhM0MMXCgsg.png" /><figcaption>After analyzing the hotel reviews, it’s clear that more people, about 155, are visiting for business purposes compared to just 16 for family trips. This indicates that the majority of guests are here for work rather than leisure with their families. Knowing this is crucial for hotels to tailor their services accordingly. They can focus on providing amenities that cater to business travellers, such as reliable Wi-Fi and meeting spaces. However, it’s also essential to consider the needs of families by offering kid-friendly activities and accommodations. Finding the right balance between these two types of guests ensures that hotels can satisfy everyone’s preferences and encourage repeat visits.</figcaption></figure><h4>6. Plotting Histograms for review length based on Rating</h4><pre>data[&#39;Length&#39;] = data[&#39;Review&#39;].apply(len)<br>data[&#39;num_words&#39;] = data[&#39;Review&#39;].apply(word_tokenize).apply(len)<br><br># Create subplots for each rating category<br>fig, axes = plt.subplots(1, data[&#39;Rating&#39;].nunique(), figsize=(15, 5), sharey=True)<br><br># Iterate over each subplot and plot the histogram<br>for ax, (rating, sub_data) in zip(axes, data.groupby(&#39;Rating&#39;)):<br>    ax.hist(sub_data[&#39;Length&#39;], color=&#39;#973aa8&#39;)<br>    ax.set_title(f&#39;Rating {rating}&#39;)<br>    ax.set_xlabel(&#39;Length&#39;)<br>    ax.set_ylabel(&#39;Frequency&#39;)<br><br>plt.tight_layout()<br>plt.show()<br>correlation = data[&#39;Rating&#39;].corr(data[&#39;Length&#39;])<br><br>print(&quot;Correlation between Rating and Length:&quot;, correlation)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/788/1*JdgZwPUAmv-6JU7YvDtfxw.png" /><figcaption>The histogram tells us that when reviews are longer, they tend to have higher ratings. It seems like happier customers share more about their experiences. However, when we calculated the correlation coefficient, it showed a weak negative connection between review length and rating. This means that longer reviews don’t always lead to higher ratings. Other things, like how good the service is and people’s tastes, also affect ratings. So, while longer reviews often mean positive experiences, businesses need to look at different factors to understand customer feedback better and make improvements that keep everyone happy.</figcaption></figure><blockquote>Create sentiment target variable, If rating is 4 or 5, sentiment will be POSITIVE. If rating is 1 or 2, sentiment will be NEGATIVE. Otherwise, sentiment will be NEUTRAL!!</blockquote><pre>#  Sentiment Analysis it categorizes the ratings into sentiment labels (&#39;Positive&#39;, &#39;Negative&#39;, or &#39;Neutral&#39;)<br>data[&#39;Sentiment&#39;] = data[&#39;Rating&#39;].apply(lambda x: &#39;Positive&#39; if x &gt;= 4 else &#39;Negative&#39; if x &lt;= 2 else &#39;Neutral&#39;)<br>data[&#39;Sentiment&#39;].value_counts()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/207/1*ZTSqHfTVfnkFPw7FbCLeYA.png" /><figcaption>This code sorts out the hotel ratings into three groups: ‘Positive’, ‘Negative’, or ‘Neutral’. It checks each rating and decides which group it belongs to. If a rating is 4 or higher, it’s called ‘Positive’. If it’s 2 or lower, it’s named ‘Negative’. Anything in between gets labelled as ‘Neutral’. After sorting, it counts how many ratings fall into each group. The results show that most reviews are ‘Positive’ (15,093), followed by ‘Negative’ (3,214), and ‘Neutral’ (2,184). This helps us understand the overall sentiment of the reviews and what guests think about their hotel experiences.</figcaption></figure><p><strong><em>NOTES</em></strong><em>:- GUYS from above we get to know that our data is imbalance data, so we have to take care of these before model building</em></p><h4>7. Sentiment Analysis</h4><pre>#  Sentiment Analysis (Pie chart)<br>plt.figure(figsize=(8,8))<br>plt.subplot(4, 4, 5)<br>data[&#39;Sentiment&#39;].value_counts().plot(kind=&#39;pie&#39;, autopct=&#39;%1.1f%%&#39;, colors=[&#39;red&#39;, &#39;green&#39;, &#39;yellow&#39;])<br>plt.title(&#39;Sentiment Analysis&#39;)<br>plt.tight_layout()<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/490/1*X_o2OfpphtnguPckYg1WtA.png" /><figcaption>In our analysis of hotel reviews, we discovered that the majority, around 73.7%, were positive, meaning guests had good experiences. Conversely, about 15.7% were negative, showing some guests weren’t happy. Around 10.7% were neutral, meaning they didn’t feel strongly either way. Knowing these feelings helps hotels enhance their services and ensure guest satisfaction. It’s like listening to feedback to make things better. By paying attention to these sentiments, hotels can keep guests happy and continuously improve their offerings.</figcaption></figure><h3>Text Cleaning &amp; Preprocessing</h3><pre># Define stop words<br>stop_words = set(stopwords.words(&quot;english&quot;))<br><br># Function to remove emojis<br>def remove_emoji(text):<br>    emoji_pattern = re.compile(&quot;[&quot;<br>                               u&quot;\U0001F600-\U0001F64F&quot;  # emoticons<br>                               u&quot;\U0001F300-\U0001F5FF&quot;  # symbols &amp; pictographs<br>                               u&quot;\U0001F680-\U0001F6FF&quot;  # transport &amp; map symbols<br>                               u&quot;\U0001F1E0-\U0001F1FF&quot;  # flags (iOS)<br>                               u&quot;\U00002702-\U000027B0&quot;<br>                               u&quot;\U000024C2-\U0001F251&quot;<br>                               &quot;]+&quot;, flags=re.UNICODE)<br>    return emoji_pattern.sub(r&#39;&#39;, text)<br><br># Function for data processing<br>def data_processing(text):<br>    text = text.lower()  # Converting the entire text in the reviews to lowercase<br>    text = re.sub(r&quot;http\S+|www\S+|https\S+&quot;, &#39;&#39;, text, flags=re.MULTILINE)  # Removing url&#39;s<br>    text = re.sub(r&#39;\S+@\S&#39;, &#39;&#39;, text)  # Removing the emails from reviews<br>    text = re.sub(r&#39;\d+&#39;, &#39;&#39;, text)  # Removing the digits from reviews<br>    text = text.strip()  # removing extra space from the reviews<br>    text = remove_emoji(text)  # Removing emojis<br>    text = re.sub(r&#39;[^\w\s]&#39;, &#39;&#39;, text)  # remove all the punctuations from the reviews<br>    text_tokens = word_tokenize(text)<br>    filtered_text = [w for w in text_tokens if not w in stop_words]<br>    return &quot; &quot;.join(filtered_text)<br><br># Apply data processing function to &#39;Review&#39; column<br>data[&#39;transform_text&#39;] = data[&#39;Review&#39;].apply(data_processing)<br><br># Function for cleaning<br>def cleaning(text):<br>    clean_text = text.translate(str.maketrans(&#39;&#39;, &#39;&#39;, string.punctuation)).lower()<br>    clean_text = [word for word in clean_text.split() if word not in stopwords.words(&#39;english&#39;)]<br><br>    # Lemmatize the word<br>    sentence = []<br>    for word in clean_text:<br>        lemmatizer = WordNetLemmatizer()<br>        sentence.append(lemmatizer.lemmatize(word, &#39;v&#39;))<br><br>    return &#39; &#39;.join(sentence)<br><br># Apply cleaning function to &#39;Clean_Review&#39; column<br>data[&#39;transform_text&#39;] = data[&#39;transform_text&#39;].apply(cleaning)</pre><p>This code is super important because it helps us get our hotel reviews ready for studying. It does a bunch of things to clean up the reviews. First, it gets rid of stuff we don’t need like web links, emails, numbers, and emojis. Then, it takes out common words that don’t tell us much (like “and” or “the”). After that, it cleans up the text even more by removing punctuation marks. Finally, it makes everything lowercase and organizes it neatly. Once we run this code on our reviews, we get a new column called ‘transform_text’ with all the cleaned-up text. This makes it easier for us to figure out how people feel about their hotel stays. So, basically, this code helps us get our data ready for finding out what guests really think about their experiences!</p><h4>8. Total Text Length Before and After Cleaning</h4><pre>data[&#39;T_length&#39;] = data[&#39;transform_text&#39;].apply(len)<br><br>original_length=data[&#39;Length&#39;].sum()<br>new_length = data[&#39;T_length&#39;].sum()<br><br>print(&#39;Total text length before cleaning: {}&#39;.format(original_length))<br>print(&#39;Total text length after cleaning: {}&#39;.format(new_length))<br>plt.figure(figsize=(8, 6))<br>plt.bar([&#39;Before Cleaning&#39;, &#39;After Cleaning&#39;], [original_length, new_length], color=[&#39;skyblue&#39;, &#39;lightgreen&#39;])<br>plt.title(&#39;Total Text Length Before and After Cleaning&#39;)<br>plt.ylabel(&#39;Total Length&#39;)<br>plt.xlabel(&#39;Cleaning Process&#39;)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*hf27Xkxy8diiVYnMgk7xkg.png" /><figcaption>After we cleaned up the text, we noticed a big change in how long it was. Before cleaning, it was a huge number — 14,861,007. But after cleaning, it went down to 13,911,757. This drop shows that we got rid of a bunch of unnecessary stuff from the text. Cleaning probably meant tossing out things like strange symbols or unimportant words. By doing this, we made the text simpler and easier to understand. When the text is cleaner, it’s way easier to study and find important stuff in it. So, by cleaning it up, we made our data more useful and ready for analysis.</figcaption></figure><h4>9. Most Common Words in Positive</h4><pre>pos_reviews=data[data.Sentiment==&#39;Positive&#39;]<br>neg_reviews=data[data.Sentiment==&#39;Negative&#39;]<br>from collections import Counter<br>count=Counter()<br>for text in pos_reviews[&#39;Review&#39;].values:<br>  for word in text.split():<br>    count[word] +=1<br>count.most_common(15)<br><br># Get the top 15 positive words<br>top_positive_words = count.most_common(15)<br><br># Create a DataFrame for the top positive words<br>pos_words = pd.DataFrame(top_positive_words, columns=[&#39;word&#39;, &#39;count&#39;])<br><br># Plot the top positive words using Plotly Express<br>fig = px.bar(pos_words, x=&#39;count&#39;, y=&#39;word&#39;, title=&#39;Common words in positive reviews&#39;)<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/961/1*GaRa6ycd8vfZ5BTi8jLiwQ.png" /><figcaption>Looking at the graph, it’s clear that in positive hotel reviews, guests often talk about certain things a lot. Words like ‘hotel’, ‘room’, ‘great’, ‘staff’, ‘good’, ‘nice’, ‘location’, and ‘breakfast’ pop up frequently. This means that they really like these parts of the hotels they stay in. They appreciate stuff like the overall hotel experience, comfy rooms, helpful staff, good service, nice vibe, convenient location, and yummy breakfasts. These words give us a good idea of what guests care about most when picking a hotel. Hotels can focus on making these things even better to keep guests happy and make a good impression on travellers.</figcaption></figure><h4>10. Most Common Words in Negative</h4><pre>from collections import Counter<br>count=Counter()<br>for text in neg_reviews[&#39;Review&#39;].values:<br>  for word in text.split():<br>    count[word] +=1<br>count.most_common(15)<br><br># Get the top 15 Negative words<br>top_negative_words = count.most_common(15)<br><br># Create a DataFrame for the top positive words<br>neg_words = pd.DataFrame(top_negative_words, columns=[&#39;word&#39;, &#39;count&#39;])<br><br># Plot the top positive words using Plotly Express<br>fig = px.bar(neg_words, x=&#39;count&#39;, y=&#39;word&#39;, orientation=&#39;h&#39;, title=&#39;Common words in Negative reviews&#39;)<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/891/1*d0XiOI7MfM1AmTaZA2hN5w.png" /><figcaption>Looking at the graph, we notice that some words and phrases pop up a lot in negative reviews, like “not,” “hotel,” and “room.” These words hint at areas where guests aren’t happy. Phrases like “did not” and “no” point out complaints about service and room quality. Guests often face problems with staff, service, and room conditions. This analysis helps hotels understand what they need to work on, like making rooms better and improving service. By focusing on these areas, hotels can make guests happier and enhance their reputation.</figcaption></figure><h3>“Preparing Data for Machine Learning: Label Encoding, Vectorization train test split”</h3><pre># Split the data into X and Y<br>X = data[&#39;transform_text&#39;]<br>Y = data[&#39;Sentiment&#39;]<br><br># Vectorizing the text data<br>vect = TfidfVectorizer()<br>X = vect.fit_transform(data[&#39;transform_text&#39;])<br><br># Splitting the dataset into training and testing sets<br>from sklearn.model_selection import train_test_split<br>x_train, x_test, y_train, y_test = train_test_split(X, Y, test_size=0.2, random_state=42)<br><br># Encoding the target variable<br>labelEncoder = LabelEncoder()<br>y_train = labelEncoder.fit_transform(y_train)<br>y_test = labelEncoder.transform(y_test)</pre><p>This code segment preprocesses the text data for sentiment analysis. It splits the dataset into input features (X) and target labels (Y), where X contains the processed text reviews and Y contains the corresponding sentiment labels. The text data is then vectorized using TF-IDF (Term Frequency-Inverse Document Frequency) vectorization, converting the text into numerical representations. Next, the dataset is divided into training and testing sets using 80–20 split ratio. Finally, the target labels are encoded into numerical values using LabelEncoder, enabling compatibility with machine learning algorithms. This preprocessing prepares the data for training sentiment classification models effectively.</p><h3>Handling imbalanced data</h3><pre># Plotting the distribution of ratings<br>plt.figure(figsize=(10, 5))<br>plt.subplot(1, 2, 1)<br>data[&#39;Sentiment&#39;].value_counts().sort_index().plot(kind=&#39;bar&#39;, color=&#39;purple&#39;)<br>plt.title(&#39;Distribution of Sentiment&#39;)<br>plt.xlabel(&#39;Rating&#39;)<br>plt.ylabel(&#39;Count&#39;)<br>np.bincount(y_train)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/417/1*6qDBF2Q1kDoYJvGXzD0Lfg.png" /><figcaption>In our dataset, most reviews are positive, followed by negative ones, and the least are neutral. To address this imbalance, we use a method called RandomOverSampler. It helps by duplicating less common reviews until all types are balanced. This ensures our model learns from all kinds of reviews equally. By doing this, we prevent our model from favouring the majority of reviews and make fair predictions. In simple words, RandomOverSampler helps us train a better model that considers all types of reviews equally, leading to more accurate results.</figcaption></figure><h3>Checking class distribution “Before” and “After” oversampling</h3><pre># Checking class distribution before and after oversampling<br>print(&quot;Before sampling class distribution:&quot;, Counter(y_train))<br>ros = RandomOverSampler()<br>ros_X_train, ros_y_train = ros.fit_resample(x_train, y_train)<br>print(&quot;After sampling class distribution:&quot;, Counter(ros_y_train))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/858/1*VS_jm3TsfQUXNJWPwVIjEw.png" /><figcaption>Before we did oversampling, there were more samples for one sentiment class and fewer for others. But after using RandomOverSampler, each sentiment class had the same number of samples. This means all types of sentiments had equal data for learning. Balancing the dataset this way helps machine learning models to understand and predict all sentiment categories equally well. Simply put, oversampling ensured that each sentiment group had enough data for training, making the dataset fairer and more reliable for building accurate sentiment analysis models.</figcaption></figure><h3>Model Building</h3><p>After meticulously preparing our data, we embarked on crafting and evaluating four distinct models: Logistic Regression, Multinomial Naive Bayes, Linear SVC, and Random Forest. These sophisticated algorithms are designed to delve into hotel reviews, unraveling customer sentiments with precision. Our ultimate goal is to empower hotels with invaluable insights into guest feelings, enabling them to refine their services proactively. Through the lens of these models, we aim to uncover nuanced aspects of customer sentiment embedded within text reviews. Ultimately, our mission is to equip hotels with the tools to decipher and respond to guest feedback adeptly, thereby elevating the overall customer experience to new heights.</p><blockquote><strong>Logistic Regression</strong></blockquote><pre>logistic_reg=LogisticRegression(random_state=0)<br>logistic_reg.fit(ros_X_train,ros_y_train)<br>logistic_reg_pred=logistic_reg.predict(x_test)<br>logistic_reg_acc=accuracy_score(logistic_reg_pred,y_test)<br>print(&quot;Test accuracy: {:.2f}%&quot;.format(logistic_reg_acc*100))</pre><h4>Test accuracy: 83.39%</h4><pre>print(confusion_matrix(y_test,logistic_reg_pred))<br>print(&quot;\n&quot;)<br>print(classification_report(y_test,logistic_reg_pred))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/686/1*ruU-c4l1UgpNxQMKoxocKg.png" /><figcaption>In this code, we’re using a Logistic Regression model to guess how people feel in hotel reviews. First, we teach the model using the extra training data we made (ros_X_train) and their labels (ros_y_train). Then, we test the model with the test data (x_test) to see if it’s right. It turns out the model is about 83.39% accurate, which is pretty good! The confusion matrix tells us how many times it guessed right and wrong for each sentiment. And the classification report gives us more details about how well it did overall and for each sentiment.</figcaption></figure><blockquote><strong>MultinomialNB Model</strong></blockquote><pre>mnb=MultinomialNB()<br>mnb.fit(ros_X_train,ros_y_train)<br>mnb_pred=mnb.predict(x_test)<br>mnb_acc=accuracy_score(mnb_pred,y_test)<br>print(&quot;Test accuracy: {:.2f}%&quot;.format(mnb_acc*100))</pre><h4>Test accuracy: 76.48%</h4><pre>from sklearn.metrics import classification_report<br>print(confusion_matrix(y_test,mnb_pred))<br>print(&quot;\n&quot;)<br>print(classification_report(y_test,mnb_pred))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/674/1*dRUsqNadH0F-xSkls9xUJQ.png" /><figcaption>This code applies a Multinomial Naive Bayes (MNB) model to analyze sentiments in hotel reviews. After training the model, we tested it on new data and found its accuracy to be 76.48%. This means it’s pretty good at predicting how people feel. The confusion matrix shows us how many predictions were right and wrong for each sentiment category. Moreover, the classification report gives us more detailed metrics like precision, recall, and F1-score for each sentiment, helping us understand how well the model performed overall and for each sentiment category.</figcaption></figure><blockquote><strong>LinearSVC</strong></blockquote><pre>svc=LinearSVC()<br>svc.fit(ros_X_train,ros_y_train)<br>svc_pred=svc.predict(x_test)<br>svc_acc=accuracy_score(svc_pred,y_test)<br>print(&quot;Test accuracy: {:.2f}%&quot;.format(svc_acc*100))</pre><h4>Test accuracy: 83.34%</h4><pre>from sklearn.metrics import classification_report<br>print(confusion_matrix(y_test,svc_pred))<br>print(&quot;\n&quot;)<br>print(classification_report(y_test,svc_pred))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/774/1*THGB43LCQLA3GeGYBLLsbw.png" /><figcaption>This code uses a Logistic Regression model to guess how guests feel based on hotel reviews. First, we teach the model using some training data and labels. Then, we test it using new data to see how accurate it is. It got an 83.34% accuracy, which is quite good. The confusion matrix shows us how many times it got each sentiment right or wrong. The classification report gives us more details about how well it did for each sentiment, including precision, recall, and F1-score. Overall, it helps us understand how good the model is at predicting sentiments in hotel reviews.</figcaption></figure><blockquote><strong>RandomForestClassifier</strong></blockquote><pre>from sklearn.ensemble import RandomForestClassifier<br>rf=RandomForestClassifier(n_estimators=100,criterion=&#39;entropy&#39;,random_state=0)<br>rf.fit(ros_X_train,ros_y_train)<br>rf_pred=rf.predict(x_test)<br>rf_acc=accuracy_score(rf_pred,y_test)<br>print(&quot;Test accuracy: {:.2f}%&quot;.format(rf_acc*100))</pre><h4>Test accuracy: 80.90%</h4><pre>from sklearn.metrics import classification_report<br>print(confusion_matrix(y_test,rf_pred))<br>print(&quot;\n&quot;)<br>print(classification_report(y_test,rf_pred))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/610/1*ALFRExMwzhCOZ_Q-XT5rfA.png" /><figcaption>This code uses a Random Forest Classifier to understand feelings in reviews. It creates 100 decision trees and uses the ‘entropy’ criterion to measure impurity. We train the model with some data and label, and then test it to see how well it predicts. It got an accuracy of 80.90%, which is pretty good. The confusion matrix shows how many predictions were right or wrong for each feeling. Also, the classification report gives us more details about how well it performed for each feeling, like precision, recall, and F1-score. Overall, it helps us understand how good the model is at analyzing sentiments in reviews.</figcaption></figure><h3>Best Model For Prediction</h3><pre># Calculate accuracies of all models<br>models = [&quot;Logistic Regression&quot;, &quot;Multinomial Naive Bayes&quot;, &quot;Linear SVC&quot;, &quot;Random Forest&quot;]<br>accuracies = []<br>for model in [logistic_reg, mnb, svc, rf]:<br>    y_pred = model.predict(x_test)<br>    accuracy = accuracy_score(y_test, y_pred)<br>    accuracies.append(accuracy)<br><br># Determine the best model based on accuracy<br>best_model_index = np.argmax(accuracies)<br>best_model_name = models[best_model_index]<br><br># Plotting the accuracies<br>plt.figure(figsize=(10, 5))<br>plt.bar(models, accuracies, color=&#39;skyblue&#39;)<br>plt.title(&#39;Accuracy of Different Models&#39;)<br>plt.xlabel(&#39;Model&#39;)<br>plt.ylabel(&#39;Accuracy&#39;)<br>plt.xticks(rotation=45)<br>plt.ylim(0, 1)<br>plt.grid(axis=&#39;y&#39;)<br><br># Annotating the best model<br>plt.text(best_model_index, accuracies[best_model_index], f&#39;Best Model: {best_model_name}&#39;, ha=&#39;center&#39;, va=&#39;bottom&#39;)<br><br># Show plot<br>plt.tight_layout()<br>plt.show()</pre><p>In this code, we’re checking how well four models perform by calculating their accuracies using a function called accuracy_score. These accuracies are saved in a list, and then we figure out which model did the best by finding the one with the highest accuracy. After that, we make a bar plot to show the accuracies of all the models, with a label pointing out which one is the best. This helps us see which model is most accurate at predicting sentiments in hotel reviews.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/799/1*0s5ZwjNnY07T0Dr-MjmMTw.png" /><figcaption>Upon reviewing the graph, it’s evident that the Logistic Regression model surpasses the others in accuracy, emerging as the top-performing model for sentiment analysis on our dataset. Its accuracy outshines that of the other models, making it the most reliable choice for predicting sentiments in hotel reviews.</figcaption></figure><h3>Prediction on Review — By User!!</h3><pre># Function to preprocess user input and make predictions using a logistic regression model<br>def predict_sentiment_logistic(model, vectorizer, user_input):<br>    # Preprocess user input using the same vectorizer used during training<br>    user_input_vectorized = vectorizer.transform([user_input])<br>    <br>    # Make prediction<br>    prediction = model.predict(user_input_vectorized)<br>    <br>    # Convert predicted label back to the original sentiment<br>    predicted_sentiment = labelEncoder.inverse_transform(prediction)<br>    <br>    return predicted_sentiment[0]  # Return the predicted sentiment as a string<br><br><br>user_input = input(&quot;Enter your review: &quot;)<br>predicted_sentiment_logistic = predict_sentiment_logistic(logistic_reg, vect, user_input)<br>print(&quot;Predicted sentiment (Logistic Regression):&quot;, predicted_sentiment_logistic)</pre><p>This code helps users analyze sentiments in their reviews by predicting if they’re positive, negative, or neutral. It uses a logistic regression model trained on hotel review data to process user input and make predictions. It’s handy for people or businesses who want to understand sentiments in feedback quickly. By using this code, users can make better decisions and improve customer satisfaction by addressing concerns in reviews effectively.</p><h4>Negative input!!</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SW5h5EQjtwQG3HxUbv3T0g.png" /></figure><h4>Positive input!!</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1017/1*YmfGAhvxteKjlWKeliC7Mw.png" /></figure><h4>Neutral input!!</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/994/1*lO9cduNFIPK0X0j23TAgMg.png" /></figure><h3>CONCLUSION -</h3><p>In wrapping up, 🎉this project delved deep into hotel reviews, tapping into machine learning to understand guest sentiments. With thorough data prep, analysis, and model creation, we’ve grasped the essence of guest feelings. The star of the show? The logistic regression model, using sentiment predictions. These insights are golden for hotels, guiding them to tweak services and exceed guest hopes, ensuring a stellar stay every time. It’s a testament to the power of data-driven strategies in making guests smile wider and hotels shine brighter.</p><h4>“Acknowledgment: Project Mate and Mentor Appreciation”</h4><p>I extend my heartfelt gratitude to <a href="https://medium.com/u/8c218cf6f5d1">Azim Khan</a>, my project mate, for his unwavering support and collaboration throughout this project journey. Additionally, I am immensely grateful to <a href="https://medium.com/u/f45293d3f3bf">Iftekarpatel</a> Sir, my mentor, whose guidance and expertise have been invaluable in steering us in the right direction and overcoming challenges. Furthermore, I would like to express my sincere appreciation to <a href="https://medium.com/u/b7d15b8bd717">ExcelR Solutions Hyderabad</a> for their continuous support, guidance, and provision of insightful resources.</p><p>Drive Link: <a href="https://drive.google.com/drive/folders/1xshHxy3ffJcNhbGal9y3Q9KdHtaqg_lB?usp=sharing">https://drive.google.com/drive/folders/1xshHxy3ffJcNhbGal9y3Q9KdHtaqg_lB?usp=sharing</a></p><h3>Thank YOU!!</h3><blockquote>Let’s keep the good vibes rolling! 🚀</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eb7af7a7d0f1" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>