<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Almogklein  on Medium]]></title>
        <description><![CDATA[Stories by Almogklein  on Medium]]></description>
        <link>https://medium.com/@almogklein.16?source=rss-57479f4e239d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*xc_Q7sBm9R8uqpjX</url>
            <title>Stories by Almogklein  on Medium</title>
            <link>https://medium.com/@almogklein.16?source=rss-57479f4e239d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 01:45:09 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@almogklein.16/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Vision Meets Precision: Using Image Detection to Label Tabular Data for Enhanced Accuracy — The…]]></title>
            <link>https://medium.com/@almogklein.16/vision-meets-precision-using-image-detection-to-label-tabular-data-for-enhanced-accuracy-the-aeb6eee4ec2a?source=rss-57479f4e239d------2</link>
            <guid isPermaLink="false">https://medium.com/p/aeb6eee4ec2a</guid>
            <category><![CDATA[fall-detection]]></category>
            <category><![CDATA[vision]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[yolov8]]></category>
            <category><![CDATA[tabular-data]]></category>
            <dc:creator><![CDATA[Almogklein ]]></dc:creator>
            <pubDate>Thu, 25 Apr 2024 09:26:42 GMT</pubDate>
            <atom:updated>2024-04-25T09:26:42.770Z</atom:updated>
            <content:encoded><![CDATA[<h3>Vision Meets Precision: Using Image Detection to Label Tabular Data for Enhanced Accuracy — The complete guide</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*04328nuq2TFLawOu3lKyvg.png" /></figure><p>In an era where technology continuously reshapes our approach to health and safety, a fascinating challenge has surfaced in health monitoring — accurately identifying fall events through accelerometer data from smartwatches. This persistent issue highlights the urgent need for more precise and automated methods to interpret complex time-series data efficiently, especially in scenarios like elderly care where accuracy can have significant implications.</p><h4>Overview of the Challenge</h4><p>The core challenge lies in the inherent limitations of conventional data analysis when applied to the nuanced patterns of accelerometer data. Traditional manual labeling techniques often fall short, causing models trained on these datasets to struggle to distinguish actual falls from regular activities. This variability and the errors introduced by subjective manual labeling have driven a search for a more robust solution.</p><h4>The Innovative Solution: Image Modeling Meets Tabular Data</h4><p>Recently, I embarked on a unique journey that melded the realms of image processing and tabular data analysis. Inspired by a concept I discovered in an article titled “<a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6210545/">SmartFall: A Smartwatch-Based Fall Detection System Using Deep Learning</a>”, I used image detection to enhance the accuracy of labeling accelerometer data. The idea was to use video footage synchronized with the accelerometer data to train an image model that could recognize fall events. These events could then be correlated with the accelerometer data to improve the accuracy of the labels.</p><p>By employing the YOLO (You Only Look Once) model, known for its exceptional capability in real-time object detection, it became possible to train an image model to accurately identify fall events from video.</p><p>This journey represents a blend of technology and creativity, leveraging the advancements in image processing to tackle the limitations of analyzing accelerometer data. The synchronization of video footage with data from wearable devices opens up new avenues for enhancing data accuracy, demonstrating a promising path forward in the realm of health monitoring and beyond.</p><h3>Table of Contents:</h3><blockquote>- Setting the Stage &amp; Preparing Your Toolkit</blockquote><blockquote>- Step 1: Checking and Preparing the Data</blockquote><blockquote>- Step 2: Training the YOLO Model</blockquote><blockquote>- Step 3: Video Annotation Using YOLO</blockquote><blockquote>- Step 4: Labeling the Accelerometer Data</blockquote><blockquote>- Step 5: Evaluating the Results</blockquote><blockquote>- Conclusion</blockquote><h3>Setting the Stage &amp; Preparing Your Toolkit</h3><p>Before diving into the technical aspects of this innovative approach, it’s crucial to understand the type of data we’re dealing with. Accelerometer data from smartwatches captures motion along three axes — X, Y, and Z. This data is a time series that records movements at different moments, providing insights into the wearer’s activity levels and motions.</p><p>By analyzing these patterns, we can detect anomalies that might indicate a fall. However, the subtlety and variety of motion captured by these devices make it difficult for traditional analysis methods to differentiate between a fall and other movements, such as sitting down quickly or jumping.</p><h4>Why Video Synchronization?</h4><p>Synchronizing the accelerometer data with video footage provides a visual confirmation of what the data is supposed to represent. For instance, if the accelerometer data indicates a sudden movement, the corresponding video can confirm whether this movement was a fall or another activity. This method enhances the data’s reliability before it even reaches the labeling phase, ensuring that the input for our image model is as accurate as possible.</p><p>To implement this solution, you’ll need several tools and technologies. Here’s a rundown of what you’ll need:</p><h4>— YOLO Model:</h4><blockquote>I use the YOLO object detection system because of its speed and efficiency in real-time scenarios.</blockquote><h4>— Programming Environment:</h4><blockquote>Python is the preferred language due to its extensive support for data science and machine learning libraries.</blockquote><h4>— Libraries and Frameworks:</h4><blockquote><em>* OpenCV</em>: For handling video processing and image operations.</blockquote><blockquote><em>* PyTorch</em>: As the backbone for running YOLO.</blockquote><blockquote><em>* Pandas</em>: For managing and analyzing tabular data.</blockquote><blockquote><em>* NumPy</em>: For numerical operations.</blockquote><blockquote><em>* Ultralytics YOLO</em>: A Python package for YOLO models.</blockquote><h4>— Data Files:</h4><blockquote><em>* Accelerometer Data</em>: Collected from smartwatches.</blockquote><blockquote><em>* Video Footage</em>: Synchronized with the accelerometer data.</blockquote><blockquote>* <em>data.yaml</em>: Configuration file for training the YOLO model with paths to your dataset.</blockquote><h4>Setting Up Your Environment:</h4><ul><li>Install Python: Ensure you have Python installed on your system. Python 3.8 or later is recommended.</li><li>Install Libraries: Use pip to install necessary Python libraries. You can install them using the command:</li></ul><pre>pip install opencv-python torch pandas numpy</pre><ul><li>Prepare the Data: Organize your video and accelerometer data. Ensure the timestamps are synchronized and the data formats are consistent.</li><li>Configure YOLO: Set up the YOLO model according to your specific requirements in the data.yaml file, ensuring all paths are correct and accessible.</li></ul><h3>Step 1: Checking and Preparing the Data</h3><p>Proper data preparation is crucial for the success of any machine learning project. The accuracy and efficiency of our model depend heavily on how well the data is organized and processed before training begins. In this step, I’ll guide you through acquiring, checking, and preparing the fall detection dataset, ensuring it’s ready for use with our Fine-Tune YOLO model.</p><h4>Downloading the Dataset</h4><p>The dataset we’ll be using is tailored specifically for fall detection and is available on Roboflow at the following URL: <a href="https://universe.roboflow.com/hero-d6kgf/yolov5-fall-detection/dataset/1">Roboflow — YOLOv5 Fall Detection Dataset</a>. This dataset is organized into three categories: training, testing, and validation, which are essential for training robust machine learning models.</p><ol><li>Navigate to the URL: Open your web browser and go to the provided link.</li><li>Create an Account or Log In: You may need to create an account or log in to access the dataset.</li><li>Download the Dataset: Look for a download option that allows you to download the dataset in a format compatible with YOLO (usually a ZIP file containing images and annotation files).</li></ol><h4>Verifying and Organizing the Data</h4><p>Once you have downloaded the dataset, the next steps involve verifying the integrity of the data and organizing it in a structure that your model can easily access.</p><ol><li>Extract the Dataset: Unzip the downloaded file. You should see folders labeled ‘train’, ‘test’, and ‘validation’, each containing images and their corresponding annotations.</li><li>Check the Data Structure: Open the data.yaml file included with the dataset. This file contains paths to the training, validation, test images, and other configuration details such as class names and the number of classes. Ensure the paths specified in the data.yaml file correctly point to the respective directories. If the paths are relative, they should be relative to the location of the data.yaml file. Here’s a typical structure you might see:</li></ol><pre>train: ./train/images<br>val: ./valid/images<br>test: ./test/images<br>nc: 2<br>names: [&#39;no fall&#39;, &#39;fall&#39;]</pre><p>3. Adjust Paths if Necessary: If the paths are incorrect or if you prefer a different directory structure, update the paths in the data.yaml file accordingly.</p><p>4. Inspect the Data: It’s a good practice to visually inspect some of the images along with their annotations to ensure that everything is in order. You can use image viewing software or write a simple script in Python using OpenCV to display images and their bounding boxes.</p><pre>import cv2<br><br>def display_image(image_path):<br>    # Load an image from the specified path<br>    image = cv2.imread(image_path)<br><br>    # Check if the image was loaded successfully<br>    if image is None:<br>        print(&quot;Failed to load image.&quot;)<br>        return<br><br>    # Create a window to display the image<br>    cv2.imshow(&#39;Image Display&#39;, image)<br><br>    # Wait for any key to be pressed before closing the window<br>    print(&quot;Press any key to close the image window.&quot;)<br>    cv2.waitKey(0)<br>    cv2.destroyAllWindows()<br><br># Example usage<br>if __name__ == &quot;__main__&quot;:<br>    # Replace &#39;path_to_image.jpg&#39; with the actual file path of your image<br>    image_path = &#39;path_to_image.jpg&#39;<br>    display_image(image_path)</pre><h4>Setting Up the Data for Training</h4><p>With the data now organized and verified, the final step is setting up your environment for training:</p><ol><li>Place the data.yaml File Appropriately: Ensure that the data.yaml file is accessible from your training script. The YOLO model will use this file to find the data and understand the dataset’s structure.</li><li>Prepare the Environment: Confirm that your Python environment has all the necessary libraries installed (as mentioned in the previous chapter). This includes libraries for handling data formats, image processing, and machine learning operations.</li><li>Backup the Data: Always keep a backup of the original data in a safe location before starting the manipulation or training process. This allows you to revert to the original dataset if something goes wrong during preparation.</li></ol><h3>Step 2: Training the YOLO Model</h3><p>After ensuring that your data is correctly prepared and verified, the next crucial phase is training the YOLO model. This step will transform your annotated image data into a robust model capable of recognizing fall events from video footage. Here’s how to proceed:</p><h4>1. Initialize the Model</h4><p>Start by loading the base YOLO model, which will be fine-tuned with your specific dataset. This involves loading a pre-trained model that is generally trained on a large and diverse dataset, providing a good starting point for detecting generic features before it learns to specialize in fall detection.</p><pre>֫from ultralytics import YOLO<br><br>def load_base_model(model_path):<br>    model = YOLO(model_path)<br>    return model</pre><h4>2. Configure Training Parameters</h4><p>Before beginning the training, you’ll need to configure several parameters that control how the model learns. These include the path to your dataset, image size, number of epochs, and batch size. These parameters can significantly affect the training efficiency and the model’s final performance.</p><ul><li>data: Path to the data.yaml file that describes your dataset.</li><li>imgsz: The size of the images on which the model will train. Typically, this is set to 640x640 for YOLO.</li><li>epochs: The total number of passes the model will make over the entire dataset. A higher number of epochs can lead to better learning but risks overfitting if too high.</li><li>batch: The number of training examples used to estimate the gradient in each training step.</li></ul><pre>֫def configure_training(model, data_yaml, img_size=640, epochs=100, batch_size=8):<br>    results = model.train(data=data_yaml, imgsz=img_size, epochs=epochs, batch=batch_size)<br>    return results</pre><h4>3. Start the Training Process</h4><p>With the model initialized and training parameters set, you can start the training process. This step involves feeding the annotated images into the model, allowing it to learn and adapt to the specifics of fall detection.</p><pre>֫def train_model(model, data_yaml):<br>    results = configure_training(model, data_yaml)<br>    return results</pre><p>4. Monitor Training Progress</p><p>Monitoring the training process is vital. The YOLO framework provides a real-time view of various metrics such as loss and accuracy during training. Keep an eye on these metrics to ensure the training is progressing as expected. A decreasing loss and increasing accuracy are good indicators that your model is learning effectively.</p><p>5. Save the Trained Model</p><p>Once the model has been trained, it’s important to save it. This allows you to reuse the model without needing to retrain it from scratch in the future. It also enables you to deploy the model in different environments for further testing or in production settings.</p><pre>֫def save_trained_model(model, save_path):<br>    model.save(save_path)</pre><p>6. Evaluate the Model</p><p>After training, evaluate your model using the validation set to see how well it performs on unseen data. This evaluation helps you gauge the model’s generalization capabilities and ensures it performs well in real-world scenarios.</p><pre>֫if __name__ == &quot;__main__&quot;:<br>    base_model_path = &#39;yolov8n.pt&#39;  # Path to the pre-trained YOLO model<br>    data_yaml_path = &#39;data.yaml&#39;    # Configuration and path to your dataset<br>    save_model_path = &#39;trained_fall_detection_model.pt&#39;  # Where to save the trained model<br><br>    model = load_base_model(base_model_path)<br>    training_results = train_model(model, data_yaml_path)<br>    save_trained_model(model, save_model_path)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9ofYGbrktTbZVSxAR_6MzQ.png" /></figure><h3>Step 3: Video Annotation Using YOLO</h3><p>Once the YOLO model is trained to detect fall events accurately, the next step is to utilize this model to annotate video footage. This process involves running the trained model on video data to identify and mark fall events in real time or from recorded footage. Here’s a step-by-step guide on how to annotate video using the trained YOLO model.</p><h4>1. Load the Trained Model</h4><p>Begin by loading your trained YOLO model, which has been specifically tuned to recognize fall events from the training process outlined in Step 2.</p><h4>2. Setup Video Processing</h4><p>To process a video, you’ll need to handle video input and output streams. This step involves reading from a video file or a camera feed, processing each frame through the model, and then saving or displaying the annotated output.</p><pre>֫def setup_video_processing(input_video_path, output_video_path):<br>    cap = cv2.VideoCapture(input_video_path)<br>    if not cap.isOpened():<br>        print(&quot;Error: Could not open video.&quot;)<br>        return None, None<br><br>    # Define the codec and create VideoWriter object to save the output<br>    fourcc = cv2.VideoWriter_fourcc(*&#39;mp4v&#39;)<br>    fps = cap.get(cv2.CAP_PROP_FPS)<br>    width, height = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)), int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))<br>    out = cv2.VideoWriter(output_video_path, fourcc, fps, (width, height))<br><br>    return cap, out</pre><h4>3. Annotate Each Frame</h4><p>For each frame in the video, use the YOLO model to detect fall events. The model will provide bounding boxes, class IDs, and confidence scores for each detection. These detections need to be drawn onto the frame to visually annotate them.</p><pre>֫def annotate_frame(frame, detections):<br>    for x1, y1, x2, y2, conf, class_id in detections:<br>        if class_id == 1 and conf &gt; 0.75:  # Assuming &#39;1&#39; is the class ID for falls<br>            cv2.rectangle(frame, (x1, y1), (x2, y2), (0, 0, 255), 2)  # Draw rectangle with red color<br>            label = f&quot;Fall: {conf:.2f}&quot;<br>            cv2.putText(frame, label, (x1, y1 - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.9, (0, 0, 255), 2)<br>    return frame<br><br>def process_video(model, cap, out):<br>    while True:<br>        ret, frame = cap.read()<br>        if not ret:<br>            break<br>        <br>        detections = model.predict(frame)<br>        annotated_frame = annotate_frame(frame, detections)<br>        out.write(annotated_frame)<br><br>    cap.release()<br>    out.release()<br>    print(&quot;Video processing completed and saved.&quot;)</pre><h4>4. Putting It All Together</h4><p>Now, integrate these components to annotate a video from start to finish.</p><pre>֫def annotate_video(input_video_path, trained_model_path, output_video_path):<br>    model = load_trained_model(trained_model_path)<br>    cap, out = setup_video_processing(input_video_path, output_video_path)<br>    if cap is not None and out is not None:<br>        process_video(model, cap, out)<br>    else:<br>        print(&quot;Failed to set up video processing.&quot;)<br><br>if __name__ == &quot;__main__&quot;:<br>    input_video_path = &#39;path_to_input_video.mp4&#39;<br>    trained_model_path = &#39;trained_fall_detection_model.pt&#39;<br>    output_video_path = &#39;annotated_output_video.mp4&#39;<br><br>    annotate_video(input_video_path, trained_model_path, output_video_path)</pre><p>This allows for real-time or post-event analysis of fall incidents, making it an invaluable tool for monitoring and improving safety protocols, especially in elderly care scenarios.</p><h3>Step 4: Labeling the Accelerometer Data</h3><p>After successfully annotating the video footage using the trained YOLO model to detect fall events, the next essential step is to label the accelerometer data. This process aligns the detected events from the video with the time-stamped accelerometer data from a smartwatch or similar device. The objective is to accurately label the accelerometer data with ‘fall’ or ‘no fall’ based on the video annotations, thus enhancing the dataset’s reliability for further analysis or model training.</p><h4>1. Synchronize Timestamps</h4><p>The first task is to synchronize the timestamps from the video with the accelerometer data. This requires a precise understanding of when the video was started relative to the accelerometer data collection. This synchronization ensures that each frame from the video corresponds accurately to a specific point in time in the accelerometer data.</p><h4>2. Calculate Time Frames for Falls</h4><p>Using the timestamps and frame indices where falls were detected (fall_frames), calculate the exact moments in time these falls occurred:</p><pre>֫def calculate_fall_timestamps(start_time, fall_frames, fps):<br>    fall_timestamps = [start_time + timedelta(seconds=i / fps) for i in fall_frames]<br>    return fall_timestamps</pre><h4>3. Label the Accelerometer Data</h4><p>With the fall timestamps calculated, the next step is to label the accelerometer data. This involves marking data points that correspond to the time frames of detected falls:</p><pre>֫def label_accelerometer_data(csv_file, fall_timestamps, start_time, end_time, fps, output_csv):<br>    df = pd.read_csv(csv_file)<br>    df[&#39;createdAt&#39;] = pd.to_datetime(df[&#39;createdAt&#39;]).dt.tz_localize(None)  # Ensure time is timezone-naive for comparison<br><br>    # Initialize all entries as &#39;no fall&#39;<br>    df[&#39;target&#39;] = &#39;no fall&#39;<br><br>    # Labeling for falls within the specific time windows around each fall event<br>    for ts in fall_timestamps:<br>        frame_time_window_start = ts - timedelta(seconds=1 / fps)<br>        frame_time_window_end = ts + timedelta(seconds=1 / fps)<br>        mask = (df[&#39;createdAt&#39;] &gt;= frame_time_window_start) &amp; (df[&#39;createdAt&#39;] &lt;= frame_time_window_end)<br>        df.loc[mask, &#39;target&#39;] = &#39;fall&#39;<br><br>    # Label data outside the video duration as &#39;out of the scope of the video&#39;<br>    df.loc[df[&#39;createdAt&#39;] &lt; start_time, &#39;target&#39;] = &#39;out of the scope of the video&#39;<br>    df.loc[df[&#39;createdAt&#39;] &gt; end_time, &#39;target&#39;] = &#39;out of the scope of the video&#39;<br><br>    # Save the labeled data<br>    df.to_csv(output_csv, index=False)</pre><h4>4. Execute the Labeling</h4><p>Combine the functions to execute the entire labeling process from video analysis to data labeling:</p><pre>def get_video_creation_time(video_file):<br>    cmd = f&quot;ffmpeg -i {video_file} -f ffmetadata -&quot;<br>    process = subprocess.run(cmd, capture_output=True, text=True, shell=True)<br>    match = re.search(r&quot;creation_time\s*:\s*(.+)&quot;, process.stderr)<br>    if match:<br>        return match.group(1).strip()<br>    else:<br>        print(&quot;No creation time found in video metadata.&quot;)<br>        return None<br>֫<br>def main(video_path, model_path, csv_path, output_video_path, output_csv_path):<br>    creation_time = get_video_creation_time(video_path)<br>    if not creation_time:<br>        print(&quot;Failed to find video creation time.&quot;)<br>        return<br><br>    fall_frames, fps, total_frames = annotate_video(video_path, model_path, output_video_path)<br>    start_time = datetime.fromisoformat(creation_time[:-1]).replace(tzinfo=utc)<br>    end_time = start_time + timedelta(seconds=total_frames / fps)<br>    fall_timestamps = calculate_fall_timestamps(start_time, fall_frames, fps)<br><br>    label_accelerometer_data(csv_path, fall_timestamps, start_time, end_time, fps, output_csv_path)</pre><h3>Step 5: Evaluating the Results</h3><p>After implementing a novel approach to labeling accelerometer data using video-annotated fall detection, it is crucial to evaluate the effectiveness and reliability of your labeling methodology. This step involves reviewing the labeled data, assessing potential discrepancies, and understanding the model’s performance to ensure the approach meets your analytical needs.</p><h4>1. Analyze the Labeled Data</h4><p>Begin by examining the labeled accelerometer data closely. Look for clear indications that the timestamps marking ‘fall’ events correlate logically with the expected behavior during falls, as indicated by sudden changes or anomalies in the accelerometer data.</p><pre>֫def review_labeled_data(labeled_csv):<br>    # Load the labeled data<br>    labeled_data = pd.read_csv(labeled_csv)<br>    <br>    # Display some statistics<br>    fall_events_count = labeled_data[labeled_data[&#39;target&#39;] == &#39;fall&#39;].count()<br>    no_fall_events_count = labeled_data[labeled_data[&#39;target&#39;] == &#39;no fall&#39;].count()<br>    out_of_scope_count = labeled_data[labeled_data[&#39;target&#39;] == &#39;out of the scope of the video&#39;].count()<br><br>    print(f&quot;Total &#39;fall&#39; events labeled: {fall_events_count}&quot;)<br>    print(f&quot;Total &#39;no fall&#39; events labeled: {no_fall_events_count}&quot;)<br>    print(f&quot;Total &#39;out of the scope of the video&#39; events: {out_of_scope_count}&quot;)<br><br>    return labeled_data</pre><h4>2. Visual Inspection</h4><p>Conduct a visual inspection of random samples from the labeled data alongside the video clips to confirm that the falls are being correctly identified and labeled. This could involve plotting the accelerometer data points and highlighting those labeled as ‘fall’ to see if they coincide with significant movements or impacts.</p><pre>֫import matplotlib.pyplot as plt<br>import matplotlib.dates as mdates<br><br>def plot_labeled_falls(labeled_csv):<br>    # Load labeled data<br>    labeled_data = pd.read_csv(labeled_csv)<br>    labeled_data[&#39;createdAt&#39;] = pd.to_datetime(labeled_data[&#39;createdAt&#39;])<br><br>    # Set up the figure and subplots<br>    fig, axes = plt.subplots(4, 1, figsize=(15, 35), sharex=True)  # Increased figure height<br><br>    # Titles for each subplot<br>    titles = [&#39;X Acceleration&#39;, &#39;Y Acceleration&#39;, &#39;Z Acceleration&#39;, &#39;Unified XYZ Acceleration&#39;]<br>    colors = {&#39;x0&#39;: &#39;blue&#39;, &#39;y0&#39;: &#39;green&#39;, &#39;z0&#39;: &#39;red&#39;}<br><br>    # Iterate through each axis data for individual plots<br>    for i, (axis, color) in enumerate(colors.items()):<br>        # Plot all movements with default settings<br>        axes[i].scatter(labeled_data[&#39;createdAt&#39;], labeled_data[axis], label=f&#39;{axis.upper()} Movement&#39;, alpha=0.5,<br>                        color=color)<br><br>        # Highlight &#39;fall&#39; events with more opacity and black edges<br>        falls = labeled_data[labeled_data[&#39;target&#39;] == &#39;fall&#39;]<br>        axes[i].scatter(falls[&#39;createdAt&#39;], falls[axis], color=&#39;black&#39;, edgecolors=&#39;white&#39;, s=150,<br>                        label=&#39;Falls Detected&#39;, alpha=0.7, marker=&#39;v&#39;)<br><br>        # Highlight &#39;out of scope&#39; with a transparent overlay<br>        out_of_scope = labeled_data[labeled_data[&#39;target&#39;] == &#39;out of the scope of the video&#39;]<br>        if not out_of_scope.empty:<br>            min_time = min(out_of_scope[&#39;createdAt&#39;])<br>            max_time = max(out_of_scope[&#39;createdAt&#39;])<br>            axes[i].axvspan(min_time, max_time, color=&#39;gray&#39;, alpha=0.2, label=&#39;Out of Scope&#39;)<br><br>        # Axis labels and title<br>        axes[i].set_ylabel(f&#39;{axis.upper()} Values&#39;)<br>        axes[i].set_title(titles[i])<br>        axes[i].legend()<br><br>    # Unified XYZ plot<br>    for axis, color in colors.items():<br>        axes[3].scatter(labeled_data[&#39;createdAt&#39;], labeled_data[axis], label=f&#39;{axis.upper()} Acceleration&#39;, alpha=0.5,<br>                        color=color)<br><br>    # Highlight &#39;fall&#39; events in the unified plot<br>    for axis in colors.keys():<br>        axes[3].scatter(falls[&#39;createdAt&#39;], falls[axis], color=&#39;black&#39;, edgecolors=&#39;white&#39;, s=150, alpha=0.7,<br>                        marker=&#39;v&#39;)<br><br>    axes[3].set_ylabel(&#39;Acceleration Values&#39;)<br>    axes[3].set_title(titles[3])<br>    axes[3].legend()<br><br>    # General plot settings<br>    plt.xlabel(&#39;Time&#39;)<br>    plt.suptitle(&#39;2D Scatter Plots of Accelerometer Data&#39;, y=1.000001)  # Adjust title spacing<br>    plt.xticks(rotation=0)<br>    plt.grid(True)<br>    plt.tight_layout(pad=4.0)  # Adjust layout to prevent crowding<br>    plt.subplots_adjust(top=0.91)  # Adjust top spacing to ensure title visibility<br>    plt.show()<br><br><br>if __name__ == &quot;__main__&quot;:<br><br>    labeled_csv_path = &#39;labeled_events.csv&#39;<br>    plot_labeled_falls(labeled_csv_path)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7SPrId6ViCuyzyw0V9in4Q.png" /></figure><h4>3. Feedback Collection</h4><p>Solicit feedback from potential users or domain experts who understand the context in which the data will be used. This qualitative feedback can help identify any practical issues with the data’s usability or suggest areas for refinement.</p><h3>Conclusion</h3><p>Through this integration of video annotation and accelerometer data labeling using the YOLO model, we’ve taken a significant step forward in enhancing the accuracy and reliability of fall detection systems. This interdisciplinary approach not only addresses the limitations of traditional data analysis methods but also opens up new avenues for research and application in health monitoring and beyond. By harnessing the power of image recognition and precise time-series analysis, we are poised to make substantial advancements in how we interpret complex data, ultimately leading to better outcomes in care settings and other fields where accurate event detection is crucial. As technology continues to evolve, continually improving our ability to safeguard and enhance human lives.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aeb6eee4ec2a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Pandas 2.0 vs Polars 0.17.7: Battle for Supremacy in Speed and Syntax]]></title>
            <link>https://medium.com/better-programming/data-duel-pandas-2-0-and-polars-0-17-7-battle-for-supremacy-in-speed-and-syntax-87f062995550?source=rss-57479f4e239d------2</link>
            <guid isPermaLink="false">https://medium.com/p/87f062995550</guid>
            <category><![CDATA[pandas]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[polar]]></category>
            <category><![CDATA[data-manipulation]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Almogklein ]]></dc:creator>
            <pubDate>Mon, 24 Apr 2023 21:30:12 GMT</pubDate>
            <atom:updated>2023-05-17T09:26:38.965Z</atom:updated>
            <content:encoded><![CDATA[<h4>A data duel</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/953/1*kxDzAzyLb09jrdscFJsBUw.jpeg" /><figcaption><a href="https://www.google.com/search?q=pandas+vs+polars&amp;rlz=1C1CHZN_enIL1006IL1006&amp;sxsrf=APwXEdeoGE6RpxeQBvjplPyI8zotRbrW2A:1682372876414&amp;source=lnms&amp;tbm=isch&amp;sa=X&amp;ved=2ahUKEwjhp7atv8P-AhX7iv0HHYnKCwIQ3_AUbiho=Awbiho=Aw6&amp;obiho=ECA15&amp;obiw96%2067&amp;dpr=1.38#imgrc=yevUNmMtBw14_M">Google Images</a></figcaption></figure><p>When it comes to data analysis, choosing the right tool for the job is essential. Pandas is a popular Python library for data manipulation and analysis, but it can sometimes be slow when working with large datasets. Polars is a newer library that provides a fast and efficient alternative to Pandas, with a similar API.</p><p>We will compare the syntax and performance of Pandas and Polars by creating, manipulating, and analyzing large datasets.</p><h3><em>Importing Libraries</em></h3><p>In this blog post, we will compare the performance of two popular data analysis libraries — Pandas and Polars. Pandas is a widely used library for data manipulation and analysis in Python, while Polars is a relatively new library that aims to provide similar functionality but with improved performance. We will first import the required libraries and then compare the syntax of various operations in Pandas and Polars.</p><p>Finally, we will compare the performance of these libraries for different operations and provide theoretical and probabilistic explanations for our observations.</p><p>First, let’s import the necessary libraries for our analysis. We will be using Pandas version 2.0.0 and Polars version 0.17.7.</p><pre>֫֫!pip install pandas==2.0.0<br>!pip install polars==0.17.7</pre><p>Additionally, we will import other libraries such as NumPy, Matplotlib, and IO. NumPy is a library for numerical computing in Python, while Matplotlib is a plotting library that can be used to visualize data. IO is a library for working with file I/O operations.</p><pre>import pandas as pd <br>import polars as pl<br><br>import io<br>import time<br>import numpy as np<br>import matplotlib.pyplot as plt</pre><p>Next, we will compare Pandas and Polars to see how similar their syntax is for common data manipulation tasks. We will create a DataFrame, select columns, and filter rows using both libraries and compare their syntax.</p><h3>Syntax Comparison</h3><p>Let’s start with the Syntax Comparison by creating a DataFrame using both libraries.</p><h4>Creating a DataFrame</h4><p>To create a DataFrame in Pandas and Polars, we can use the DataFrame() constructor. We will create a DataFrame with randomly generated data for the purpose of comparison.</p><p>The Pandas DataFrame will have the following columns: name, age, city, income, gender, is_married, children, zip_code, interest_rate, and is_default. The Polars DataFrame will have the same columns and data types as the Pandas DataFrame.</p><pre>֫def create_dataframe(n_rows, library):<br>  if library == &#39;pandas&#39;:<br>    data = {<br>      &#39;name&#39;: np.random.choice([&#39;Alice&#39;, &#39;Bob&#39;, &#39;Charlie&#39;, &#39;David&#39;, &#39;Eva&#39;], n_rows),<br>      &#39;age&#39;: np.random.randint(18, 65, size=n_rows),<br>      &#39;city&#39;: np.random.choice([&#39;New York&#39;, &#39;San Francisco&#39;, &#39;Los Angeles&#39;], n_rows),<br>      &#39;income&#39;: np.random.normal(50000, 10000, size=n_rows),<br>      &#39;gender&#39;: np.random.choice([&#39;Male&#39;, &#39;Female&#39;], n_rows),<br>      &#39;is_married&#39;: np.random.choice([True, False], n_rows),<br>      &#39;children&#39;: np.random.randint(0, 5, size=n_rows),<br>      &#39;zip_code&#39;: np.random.randint(10000, 99999, size=n_rows),<br>      &#39;interest_rate&#39;: np.random.uniform(0, 0.1, size=n_rows),<br>      &#39;is_default&#39;: np.random.choice([True, False], n_rows)<br>    }<br>    return pd.DataFrame(data)<br><br>  elif library == &#39;polars&#39;:<br>    data = {<br>      &#39;name&#39;: np.random.choice([&#39;Alice&#39;, &#39;Bob&#39;, &#39;Charlie&#39;, &#39;David&#39;, &#39;Eva&#39;], n_rows).astype(&#39;str&#39;),<br>      &#39;age&#39;: np.random.randint(18, 65, size=n_rows).astype(&#39;int&#39;),<br>      &#39;city&#39;: np.random.choice([&#39;New York&#39;, &#39;San Francisco&#39;, &#39;Los Angeles&#39;], n_rows).astype(&#39;str&#39;),<br>      &#39;income&#39;: np.random.normal(50000, 10000, size=n_rows).astype(&#39;float&#39;),<br>      &#39;gender&#39;: np.random.choice([&#39;Male&#39;, &#39;Female&#39;], n_rows).astype(&#39;str&#39;),<br>      &#39;is_married&#39;: np.random.choice([True, False], n_rows).astype(&#39;bool&#39;),<br>      &#39;children&#39;: np.random.randint(0, 5, size=n_rows).astype(&#39;int&#39;),<br>      &#39;zip_code&#39;: np.random.randint(10000, 99999, size=n_rows).astype(&#39;str&#39;),<br>      &#39;interest_rate&#39;: np.random.uniform(0, 0.1, size=n_rows).astype(&#39;float&#39;),<br>      &#39;is_default&#39;: np.random.choice([True, False], n_rows).astype(&#39;bool&#39;)<br>    }<br>    return pl.DataFrame(data)</pre><h4>Saving and Reading a DataFrame</h4><p>To save a DataFrame to a CSV file, we can use the to_csv() method in Pandas and the write_csv() method in Polars. To read a DataFrame from a CSV file, we can use the read_csv() function in Pandas and the read_csv() method in Polars.</p><pre>֫def save_dataframe_to_csv(df, file_name):<br>  &quot;&quot;&quot;<br>  Saves a Pandas or Polars DataFrame to a CSV file.<br>  :param df: The DataFrame to save.<br>  :param file_name: The name of the file to save to.<br>  &quot;&quot;&quot;<br>  if isinstance(df, pd.DataFrame):<br>    df.to_csv(file_name)<br>  elif isinstance(df, pl.DataFrame):<br>    df.write_csv(file_name)<br><br>def read_dataframe_from_csv(file_name, library):<br>  &quot;&quot;&quot;<br>  Reads a Pandas or Polars DataFrame from a CSV file.<br>  :param file_name: The name of the CSV file to read from.<br>  :param library: The library to use - &#39;pandas&#39; or &#39;polars&#39;.<br>  <br>  :return: The DataFrame read from the CSV file.<br>  &quot;&quot;&quot;<br>  if library == &#39;pandas&#39;:<br>    return pd.read_csv(file_name)<br>  elif library == &#39;polars&#39;:<br>    return pl.read_csv(file_name)</pre><h4>Sorting a DataFrame</h4><p>To sort a DataFrame in Pandas, we can use the sort_values() method. In Polars, we can use the sort() method. Both methods take the column name to sort by and a Boolean flag to specify whether to sort in ascending or descending order.</p><pre>def sort_dataframe(df, column_name, ascending=True):<br>  &quot;&quot;&quot;<br>  Sorts a Pandas or Polars DataFrame by the values in a given column.<br>  :param df: The DataFrame to sort.<br>  :param column_name: The name of the column to sort by.<br>  :param ascending: Whether to sort in ascending or descending order. Default is ascending.<br>  <br>  :return: The sorted DataFrame.<br>  &quot;&quot;&quot;<br>  if isinstance(df, pd.DataFrame):<br>    return df.sort_values(by=column_name, ascending=ascending)<br>  elif isinstance(df, pl.DataFrame):<br>    return df.sort(column_name, descending=not ascending)</pre><h4>Filtering a DataFrame</h4><p>To filter a DataFrame in Pandas, we can use boolean indexing by specifying a condition in square brackets. In Polars, we can use the filter() method with a col() function to specify the column and a condition in parentheses.</p><pre>def filter_dataframe(df, column, value):<br>  &quot;&quot;&quot;<br>  Filters a Pandas or Polars DataFrame based on a given condition.<br>  :param df: The DataFrame to filter.<br>  :param column: The column to filter on.<br>  :param value: The value to filter on.<br>  :return: The filtered DataFrame.<br>  &quot;&quot;&quot;<br>  if isinstance(df, pd.DataFrame):<br>    return df[df[column] == value]<br>  elif isinstance(df, pl.DataFrame):<br>    return df.filter(pl.col(column) == value)</pre><h4>Grouping a DataFrame</h4><p>We can use the method to group a DataFrame in Pandas, which creates a groupby object. In Polars, we can also use the groupby() method, which returns a groupby object. We can then perform aggregate operations on the groupby object, such as count(), mean(), and sum().</p><pre>def groupby_dataframe(df, column_name):<br>  &quot;&quot;&quot;<br>  Groups a Pandas or Polars DataFrame by a given column name.<br>  :param df: The DataFrame to group.<br>  :param column_name: The name of the column to group by.<br>  :return: The grouped DataFrame.<br>  &quot;&quot;&quot;<br>  start_time = time.time()<br>  result = df.groupby(column_name)<br>  end_time = time.time()<br>  return end_time - start_time</pre><h3>Performance Comparison</h3><p>We can use the functions we defined earlier to compare the performance of Pandas and Polars. We’ll start by generating DataFrames with a million rows using both libraries and then time each operation for both libraries.</p><p>Here is the code:</p><pre>֫n_rows = 1,000,000<br>file_name_pan = &#39;data_pandas.csv&#39;<br>file_name_pol = &#39;data_polar.csv&#39;</pre><pre># Creating DataFrames<br>start_time = time.time()<br>df_pandas = create_dataframe(n_rows, &#39;pandas&#39;)<br>pandas_times[&#39;Create&#39;] = time.time() - start_time<br><br>start_time = time.time()<br>df_polars = create_dataframe(n_rows, &#39;polars&#39;)<br>polars_times[&#39;Create&#39;] = time.time() - start_time<br><br># Saving DataFrames<br>start_time = time.time()<br>save_dataframe_to_csv(df_pandas, file_name_pan)<br>pandas_times[&#39;Save&#39;] = time.time() - start_time<br><br>start_time = time.time()<br>save_dataframe_to_csv(df_polars, file_name_pol)<br>polars_times[&#39;Save&#39;] = time.time() - start_time<br><br># Reading DataFrames<br>start_time = time.time()<br>df_pandas = read_dataframe_from_csv(file_name_pan, &#39;pandas&#39;)<br>pandas_times[&#39;Read&#39;] = time.time() - start_time<br><br>start_time = time.time()<br>df_polars = read_dataframe_from_csv(file_name_pol, &#39;polars&#39;)<br>polars_times[&#39;Read&#39;] = time.time() - start_time<br><br># Sorting DataFrames<br>start_time = time.time()<br>sort_dataframe(df_pandas, &#39;gender&#39;, ascending=True)<br>pandas_times[&#39;Sort&#39;] = time.time() - start_time<br><br>start_time = time.time()<br>sort_dataframe(df_polars, &#39;gender&#39;, ascending=True)<br>polars_times[&#39;Sort&#39;] = time.time() - start_time<br><br># Filtering DataFrames<br>start_time = time.time()<br>filter_dataframe(df_pandas, &#39;gender&#39;, &#39;Male&#39;)<br>pandas_times[&#39;Filter&#39;] = time.time() - start_time<br><br>start_time = time.time()<br>filter_dataframe(df_polars, &#39;gender&#39;, &#39;Male&#39;)<br>polars_times[&#39;Filter&#39;] = time.time() - start_time<br><br># GroupBy DataFrames<br>pandas_times[&#39;GroupBy&#39;] = groupby_dataframe(df_pandas, &#39;zip_code&#39;)<br>polars_times[&#39;GroupBy&#39;] = groupby_dataframe(df_polars, &#39;zip_code&#39;)</pre><p>We have used the functions create_dataframe, save_dataframe_to_csv, read_dataframe_from_csv, sort_dataframe, filter_dataframe, and groupby_dataframe to perform the operations on the DataFrames generated by both libraries. The runtimes of each operation have been stored in two dictionaries, pandas_times and polars_times.</p><p>Now we can use the compare_runtimes function to plot a bar graph comparing the runtimes of each operation for Pandas and Polars. Here is the code:</p><pre>֫def compare_runtimes(pandas_times, polars_times):<br>  &quot;&quot;&quot;<br>  Compares runtimes of Pandas and Polars for various operations.<br>  :param pandas_times: A dictionary containing the runtimes for various operations for Pandas.<br>  :param polars_times: A dictionary containing the runtimes for various operations for Polars.<br>  &quot;&quot;&quot;<br>  fig, ax = plt.subplots(figsize=(12, 4))<br><br>  # Setting the title and labels for the graph<br>  ax.set_title(&quot;Comparison of Runtimes for Pandas and Polars&quot;)<br>  ax.set_xlabel(&quot;Operations&quot;)<br>  ax.set_ylabel(&quot;Time (Seconds)&quot;)<br><br>  # Creating a bar graph for Pandas and Polars runtimes<br>  operations = pandas_times.keys()<br>  num_operations = len(operations)<br>  bar_width = 0.35<br>  pandas_x = np.arange(num_operations)<br>  polars_x = pandas_x + bar_width<br><br>  pandas_y = list(pandas_times.values())<br>  polars_y = list(polars_times.values())<br><br>  pandas_bar = ax.bar(pandas_x, pandas_y, bar_width, color=&#39;tab:blue&#39;, label=&#39;Pandas&#39;)<br>  polars_bar = ax.bar(polars_x, polars_y, bar_width, color=&#39;tab:orange&#39;, label=&#39;Polars&#39;)<br><br>  # Setting the x-axis ticks and labels<br>  ax.set_xticks(pandas_x + bar_width / 2)<br>  ax.set_xticklabels(operations)<br><br>  # Setting the y-axis limit as the maximum of the two maximum values of runtimes<br>  ax.set_ylim(0, max(max(pandas_y), max(polars_y)) * 1.2)<br><br>  # Adding values on top of the bars<br>  for i, bars in enumerate(zip(pandas_bar, polars_bar)):<br>    for bar in bars:<br>      height = bar.get_height()<br>      ax.annotate(f&quot;{height:.4f}&quot;, xy=(bar.get_x() + bar.get_width() / 2, height),<br>                  xytext=(0, 3), textcoords=&quot;offset points&quot;, ha=&#39;center&#39;, va=&#39;bottom&#39;)<br><br>  # Adding a legend to the graph<br>  ax.legend()<br><br>  # Displaying the graph<br>  plt.show()</pre><pre>compare_runtimes(pandas_times, polars_times)</pre><p>Here is the graph comparing the runtimes of Pandas and Polars for various operations:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cRKPevCMbpaBiR4rFLetPQ.png" /></figure><p>It is clear from the graph that Polars perform faster than Pandas for most operations. This is particularly true for the GroupBy operation, where Polars is nearly 20 times faster than Pandas. The Filter operation is also significantly faster in Polars, while the Create and Read operations are somewhat faster in Pandas. Overall, Polars seems to be a more performant library for data manipulation, particularly for large datasets.</p><p>The x-axis shows the different operations we performed, and the y-axis shows the time taken in seconds. The blue bars represent the runtimes for Pandas, while the orange bars represent the runtimes for Polars. The graph shows that Polars consistently perform better than Pandas in most operations. Although there are some cases where Pandas perform slightly better than Polars, the overall trend favors Polars.</p><p>It is important to note that these results are specific to the dataset and operations used in this example. Different datasets and operations may yield different results, and it is always recommended to benchmark different libraries for specific use cases to determine the optimal choice for performance.</p><h3>Theoretical and Probabilistic Explanations</h3><p>The performance differences between Pandas and Polars can be attributed to using different underlying data structures. Pandas uses a DataFrame, which is based on a 2-dimensional numpy array, while Polars uses a DataFrame based on Rust’s Arrow data format. This difference can explain why Polars is generally faster than Pandas, as Rust’s Arrow is designed to be more efficient for data manipulation tasks.</p><p>Another factor that can affect the performance of data manipulation libraries is the probabilistic time complexity of their algorithms. Polars has been optimized to use algorithms with lower time complexity, which means that they perform better for large datasets. On the other hand, Pandas uses algorithms with higher time complexity, making it slower for large datasets.</p><h3>Conclusion</h3><p>In this blog post, we compared the performance of Pandas and Polars for various data manipulation operations. We found that Polars was generally faster than Pandas, particularly for the GroupBy and Filter operations. However, Pandas was faster than Polars for the Create operation.</p><p>The performance differences can be attributed to the fact that Pandas and Polars use different underlying data structures and the probabilistic time complexity of their algorithms.</p><p>Overall, if you are working with large datasets and need to perform complex data manipulation tasks, Polars might be a more performant library to use.</p><p>However, if you work with smaller datasets, Pandas might be a more convenient option as it has a more user-friendly API.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=87f062995550" width="1" height="1" alt=""><hr><p><a href="https://medium.com/better-programming/data-duel-pandas-2-0-and-polars-0-17-7-battle-for-supremacy-in-speed-and-syntax-87f062995550">Pandas 2.0 vs Polars 0.17.7: Battle for Supremacy in Speed and Syntax</a> was originally published in <a href="https://betterprogramming.pub/">Better Programming</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building Your Portfolio Website with GitHub Pages]]></title>
            <link>https://medium.com/@almogklein.16/building-your-portfolio-website-with-github-pages-f67b9645284a?source=rss-57479f4e239d------2</link>
            <guid isPermaLink="false">https://medium.com/p/f67b9645284a</guid>
            <category><![CDATA[website]]></category>
            <category><![CDATA[job-hunting]]></category>
            <category><![CDATA[github]]></category>
            <category><![CDATA[portfolio]]></category>
            <dc:creator><![CDATA[Almogklein ]]></dc:creator>
            <pubDate>Sat, 22 Apr 2023 12:50:50 GMT</pubDate>
            <atom:updated>2023-04-22T12:50:50.508Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-adW0D8Bg9to-FR3fkMxtg.png" /></figure><p>In today’s digital age, having a professional online presence is more important than ever. A polished and professional portfolio website is essential for showcasing your work and attracting potential employers. GitHub Pages offer a simple and free way to create and host your portfolio website.</p><p>In this step-by-step guide, I&#39;ll walk you through the process of building your portfolio website using GitHub Pages and a website template. I’ll also share some insightful tips and tricks to make your website truly stand out.</p><h4>Table of content:</h4><ul><li><strong><em>Importance of a Portfolio Website</em></strong></li><li><strong><em>Choosing the Right Website Template</em></strong></li><li><strong><em>Customizing the Website Template</em></strong></li><li><strong><em>Deploying the Portfolio Website on GitHub Pages</em></strong></li><li><strong><em>Promoting Your Portfolio Website</em></strong></li><li><strong><em>Keeping Your Portfolio Website Updated</em></strong></li><li><strong><em>summary</em></strong></li></ul><h4><strong><em>Importance of a Portfolio Website</em></strong></h4><p>A well-designed portfolio website serves as a digital showcase for your skills, accomplishments, and best works, helping you stand out and make a lasting impression on potential clients or employers. Much like a workshop for an artisan, your portfolio website is a personal space where you can display your achievements and demonstrate your capabilities to a global audience.</p><p>Creating and maintaining a portfolio website also encourages you to learn new things, flex your creative muscles, and refine your design process, ultimately making you a better professional in your field.</p><h4><strong><em>Choosing the Right Website Template</em></strong></h4><p>Selecting the right website template is a crucial step in building your portfolio website, as it sets the foundation for showcasing your work in the best possible light. To choose a suitable template, consider browsing through platforms like <a href="https://html5up.net/">HTML5 UP</a> or other template providers that offer a variety of modern, visually appealing designs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_Bk592auVWxB2VFM4TNPuA.png" /><figcaption><a href="https://html5up.net/">HTML5 UP</a></figcaption></figure><p>When evaluating templates, pay attention to their responsiveness, ensuring that they display well on various devices and screen sizes.</p><p>Additionally, look for templates that offer customization options, allowing you to personalize the design and make it uniquely yours easily. Remember that the template you choose should reflect your personal brand and align with the overall aesthetic and message you want to convey to your audience. By choosing the right template, you’ll create a strong foundation for a portfolio website that effectively highlights your skills, projects, and expertise, ultimately capturing the attention of potential clients and employers.</p><h4><strong><em>Customizing the Website Template</em></strong></h4><p>When you download a website template, you typically receive a compressed folder containing all the necessary files to build the website. These files usually include the following:</p><ol><li>HTML files — The main structure and content of your web pages.</li><li>CSS files — Style and design elements, such as colors, fonts, and layout.</li><li>JavaScript files — Scripts for adding interactivity and dynamic features.</li><li>Images — Any images used in the template design.</li><li>Documentation — Instructions or guides on how to use and customize the template.</li></ol><p>To customize the template, focus primarily on the HTML files. The HTML files contain the content and structure of your web pages, while the CSS files define their appearance and style.</p><p>To understand and edit the HTML files, familiarize yourself with the following concepts:</p><ol><li>Tags: HTML is built using tags, which are enclosed in angle brackets &lt; and &gt;. Tags are used to create and define elements, such as headings, paragraphs, lists, and images. Most tags have an opening tag, like &lt;p&gt;, and a closing tag, like &lt;/p&gt;, with the content placed in between.</li><li>Attributes: Attributes are used within opening tags to provide additional information about the element. For example, the href attribute in an anchor tag &lt;a&gt; specifies the link destination: &lt;a href=&quot;https://example.com&quot;&gt;Visit Example&lt;/a&gt;.</li><li>Nested elements: HTML elements can be nested within one another to create a hierarchy. For example, a list item element &lt;li&gt; is nested within an unordered list element &lt;ul&gt;.</li></ol><p>To customize the content of your template, locate and edit the relevant HTML tags. For example, to update the main heading of your homepage, look for the &lt;h1&gt; tag and replace the existing text with your desired content:</p><pre>&lt;h1&gt;Your Name - Portfolio Website&lt;/h1&gt;</pre><p>Similarly, to update the text in a paragraph, locate the &lt;p&gt; tag and replace the existing content:</p><pre>&lt;p&gt;Welcome to my portfolio website!&lt;/p&gt;</pre><p>Once you’ve customized your website template to your liking, proceed with uploading the updated files to your GitHub repository and configuring the GitHub Pages settings to publish your portfolio website.</p><h4><strong><em>Deploying the Portfolio Website on GitHub Pages</em></strong></h4><p>After downloading and adjusting the template, the first step to hosting your portfolio website on GitHub Pages is to create a new repository dedicated to your website. Follow these steps to set up your repository and deploy your website:</p><ol><li>Log in to your GitHub account: If you don’t already have an account, create one by following the instructions on the <a href="https://docs.github.com/en/get-started">GitHub website</a>.</li><li>Create a new repository: Click on the “+” icon in the top-right corner and select “New repository” from the dropdown menu. Name your repository in the following format: &lt;your-username&gt;.github.io. Be sure to replace &lt;your-username&gt; with your actual GitHub username. This naming convention is essential for GitHub Pages to correctly recognize and serve your website.</li><li>Add a brief description: Describe your repository, such as “My personal portfolio website.”</li><li>Choose between public or private: Note that the repository must be public for GitHub Pages to serve your website publicly.</li><li>Initialize the repository with a README file: Check the “Add a README file” box.</li><li>Complete the process by clicking the “Create repository” button.</li><li>Upload your website files: Clone the newly created repository to your local machine using a Git client or the command line. Copy all the customized website files (HTML, CSS, JavaScript, images, etc.) into the cloned repository folder. Add, commit, and push the changes to the remote repository on GitHub.</li><li>Enable GitHub Pages: Go to your repository&#39;s settings page on GitHub. In the sidebar under Code and automation section, hit on “Pages”. Under “Source,” select the branch you want to deploy your website from (usually the main or master branch). Save the settings.</li><li>Access your live website: After a few minutes, your portfolio website will be live at https://your-username.github.io. Check the deployment status in your repository settings&#39; &quot;GitHub Pages&quot; section. It may take a few minutes for the website to become accessible.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*waGsLq1mgfVRGoWo57tq6w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1006/1*mUBCcf7MyRQ74RYkGgSMzw.png" /></figure><p>Your portfolio website is now live on GitHub Pages! You can continue to update and improve your website by editing the files in your repository and pushing the changes. GitHub Pages will automatically update your live website with the latest changes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pqg8biqSHP6x8U-ezzg-aQ.png" /></figure><h4><strong><em>Promoting Your Portfolio Website</em></strong></h4><p>Once your portfolio website is live, it’s crucial to promote it and reach your target audience. Here are a few tips to help you achieve this:</p><ol><li>Leverage social media: Share your portfolio website on platforms like LinkedIn, Twitter, and Facebook to showcase your work to your network and potential clients.</li><li>Include your portfolio link in your email signature: This simple step ensures that every email you send becomes an opportunity for recipients to discover your work.</li></ol><h4><strong><em>Keeping Your Portfolio Website Updated</em></strong></h4><p>To maintain a fresh and relevant portfolio, it’s essential to keep your website updated:</p><ol><li>Add new projects: As you complete new projects or case studies, showcase them on your website to reflect your most recent work.</li><li>Update your skills and experience: Regularly review and update your skills, experience, and any relevant certifications to demonstrate your continuous growth and development.</li><li>Refresh the design: Periodically update the design and layout of your portfolio website to keep it modern and visually appealing.</li></ol><h4><strong><em>Summary</em></strong></h4><p>A well-crafted portfolio website is a valuable tool for showcasing your work and brand. Start by choosing a suitable template, customizing it, and deploying it on GitHub Pages. Promote your website through various channels, such as social media and networking, and regularly update your portfolio with new projects and achievements.</p><p>Following these steps, you’ll have a unique portfolio website demonstrating your skills, experience, and capabilities to potential clients and employers.</p><p>With dedication and creativity, your portfolio website will serve as a testament to your professional achievements and become a dynamic, ever-evolving platform that captivates visitors and opens doors to exciting new opportunities.</p><p>Happy showcasing!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f67b9645284a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Maximizing AI Potential: Develop an Advanced Chatbot with Rasa, ChatGPT, and Google Sheets…]]></title>
            <link>https://medium.com/@almogklein.16/maximizing-ai-potential-develop-an-advanced-chatbot-with-rasa-chatgpt-and-google-sheets-3cd641967237?source=rss-57479f4e239d------2</link>
            <guid isPermaLink="false">https://medium.com/p/3cd641967237</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[gpt-3]]></category>
            <category><![CDATA[google-sheets]]></category>
            <category><![CDATA[rasa]]></category>
            <category><![CDATA[chatbots]]></category>
            <dc:creator><![CDATA[Almogklein ]]></dc:creator>
            <pubDate>Sat, 25 Feb 2023 15:18:41 GMT</pubDate>
            <atom:updated>2023-02-28T08:57:42.770Z</atom:updated>
            <content:encoded><![CDATA[<h3>Maximizing AI Potential: Develop an Advanced Chatbot with Rasa, ChatGPT, and Google Sheets Integration</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1EuH0ycArrFn7qdr8OWKUw.png" /><figcaption>Taken from:<br><a href="https://elearningindustry.com/rise-ai-powered-chatbots-elearning">https://elearningindustry.com/rise-ai-powered-chatbots-elearning</a></figcaption></figure><p>Chatbots have become essential to many businesses’ digital strategies, providing 24/7 customer service and automating repetitive tasks. The latest advancements in AI have made it possible to create chatbots that can mimic human conversation more accurately than ever before. With the recent hype surrounding ChatGPT, the advanced language model from OpenAI, there’s never been a better time to explore the potential of AI-powered chatbots.</p><blockquote><strong><em>Table of content</em></strong></blockquote><blockquote><em>- Introduction</em></blockquote><blockquote><em>- Understanding Rasa</em></blockquote><blockquote><em>- Google Sheets Integration</em></blockquote><blockquote><em>- Implementing ChatGPT</em></blockquote><blockquote><em>- Component integration, training and evaluation the Chatbot</em></blockquote><blockquote><em>- Conclusion</em></blockquote><h4><em>Introduction</em></h4><p>This guide will walk you through the steps to create your own basic chatbot using Rasa, ChatGPT, and Google Sheets Integration. I’ll provide a detailed explanation of each component and show you how to integrate them to build an interactive chatbot that can choose structured answers through Google Sheets or reference ChatGPT’s responses. My goal is to make this process accessible to everyone and show that anyone can create their own chatbot with these powerful AI tools.</p><h4>Understanding Rasa</h4><p>Rasa is an open-source framework that provides tools for building advanced chatbots. Rasa provides Natural Language Understanding (NLU) and Dialogue Management to understand user inputs and respond with appropriate actions. NLU is understanding the user’s input and extracting relevant information, such as intent and entities. Dialogue management decides what actions to take based on the user’s input and the chatbot’s current state. Rasa also provides tools for training and evaluating your chatbot’s performance.</p><p>To get started with Rasa, you will need to install it on your system. Create a virtual environment using tools like Anaconda. A simulated environment is a self-contained environment that allows you to install and manage packages separately from your system Python installation.</p><p>Create a new environment using Anaconda, and run the following command:</p><pre>conda create --name myenv python=3.8</pre><p>Activate your chat environment by running the following commands:</p><pre>conda activate myenv<br>pip install openai==0.4.0<br>pip install pandas==1.1.5<br>pip install pathlib==1.0.1<br>pip install requests==2.25.1<br>pip install ruamel.yaml==0.16.12</pre><p>Install Rasa and its dependencies. Once Rasa is installed, you can create a new Rasa project using the following command:</p><pre>pip install rasa<br>rasa init</pre><p>This command will create a new Rasa project in your current directory with the following files:</p><ul><li>actions/: This directory contains Python scripts that define custom actions for your chatbot.</li><li>config.yml: This file contains the configuration settings for your Rasa project, such as the language model to use and the pipeline for processing user inputs.</li><li>credentials.yml: This file contains the credentials for external services, such as API keys for ChatGPT or Google Sheets.</li><li>data/: This directory contains the training data for your chatbot, including stories and NLU data.</li><li>domain.yml: This file defines the domain of your chatbot, including the intents, entities, and responses it can understand and generate.</li><li>endpoints.yml: This file defines the endpoints for external services, such as the API endpoint for ChatGPT or the Google Sheets API.</li></ul><p>After installing Rasa using the Rasa CLI, you can start building your chatbot project by defining the intents, entities, and responses in your domain file, training the language model with your training data, and defining custom actions for your chatbot.</p><p>The Rasa CLI provides tools for training and evaluating your chatbot and generating the necessary files for deployment.</p><h4>Implementing ChatGPT</h4><p>Now that we have set up Rasa and created a basic chatbot, we can integrate ChatGPT to provide more sophisticated responses. In this section, I’ll show you how to implement ChatGPT in the actions.py file of your Rasa project.</p><p>ChatGPT is a language model developed by OpenAI that uses machine learning to generate human-like responses to user inputs. It is based on the GPT (Generative Pre-trained Transformer) architecture, which has achieved state-of-the-art results in various natural language processing tasks.</p><p>First, we need to import the necessary libraries and modules. These include io, os, openai, random, requests, pandas, and the load_dotenv function from the dotenv module. We will also import the Action, Tracker, CollectingDispatcher, Text, Dict, Any, and List classes from the Rasa SDK.</p><pre>֫import io<br>import os<br>import openai<br>import random<br>import requests<br>import pandas as pd<br>from dotenv import load_dotenv<br>from rasa_sdk import Action, Tracker <br>from typing import Any, Text, Dict, List<br>from rasa_sdk.executor import CollectingDispatcher</pre><p>Next, we will define a function called <em>get_answers_from_chatgpt</em>. This function takes user_text as input and returns the response generated by ChatGPT.</p><p>To get a response from ChatGPT, we first need to set our OpenAI API key using the <em>os.getenv</em> function. We will then create a new instance of the OpenAI Completion API and set the engine to “text-davinci-003”, which is one of the most advanced language models available.</p><pre>def get_answers_from_chatgpt(user_text):<br><br>    # OpenAI API Key<br>    openai.api_key = os.getenv(&quot;GPT_API_KEY&quot;)<br><br>    # Use OpenAI API to get the response for the given user text and intent<br>    response = openai.Completion.create(<br>        engine=&quot;text-davinci-003&quot;,<br>        prompt= user_text,<br>        max_tokens=1024,<br>        n=1,<br>        stop=None,<br>        temperature=0.5,<br>    ).choices[0].text<br><br>    # Return the response from OpenAI<br>    return response</pre><p>We will then use the OpenAI API to generate a response to the user_text input. The max_tokens parameter controls the maximum number of tokens (words) in the response, and the n parameter controls the number of responses generated. The stop parameter controls the stopping sequence (if any) for the response generation, and the temperature parameter controls the “creativity” of the response.</p><p>Once we have the response from ChatGPT, we will return it to the calling function.</p><p>Next, we will define a new class called <em>Simple_ChatGPT_Action</em>. This class will inherit from the Action class and implement a run method that takes in a dispatcher, tracker, and domain as inputs.</p><pre>֫class Simple_ChatGPT_Action(Action):<br><br>    &quot;&quot;&quot;Rasa action to parse user text and pulls a corresponding answer <br>    from ChatGPT.&quot;&quot;&quot;<br><br>    def name(self) -&gt; Text:<br>        return &quot;action_gpt_default_fallback&quot; <br><br>    def run(self, dispatcher: CollectingDispatcher,<br>            tracker: Tracker,<br>            domain: Dict[Text, Any]) -&gt; List[Dict[Text, Any]]:<br><br>        # Get the latest user text <br>        user_text = tracker.latest_message.get(&#39;text&#39;)<br>    <br>        # Dispatch the response from OpenAI to the user<br>        dispatcher.utter_message(&#39;ChatGPT (custom_action): &#39; + get_answers_from_chatgpt(user_text))<br><br>        return []</pre><p>The run method will get the tracker object&#39;s latest user text and intent. We will then pass the user text to the <em>get_answers_from_chatgpt </em>function and use the CollectingDispatcher object to send the response generated by ChatGPT to the user.</p><p>Finally, we will return an empty list indicating the completed action. Once we have integrated this code into our actions.py file, we can use it to generate responses to user queries using ChatGPT.</p><h4>Google Sheets Integration</h4><p>Google Sheets is a web-based spreadsheet program that allows users to create and edit spreadsheets online while collaborating in real-time with other users. It can be used as a database for your chatbot to store responses for various intents and entities.</p><p>To set up Google Sheets integration with Rasa and ChatGPT, create a new Google Sheet and add your intents, entities, and responses to the sheet.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/877/1*U-zCTe4VrW21R5C-tGKqyg.png" /></figure><p>Once you have added the information, you need to make the sheet public by clicking “Share” and selecting “Anyone with the link can view.” Once the Google Sheet is created, the URL of the sheet needs to be added to the .env file in the Rasa project.</p><p>Next, you need to set up your Rasa environment to read the contents of the Google Sheet. You can do this by adding the following code to your actions.py file:</p><pre>class Simple_Google_sheet_or_ChatGPT_Action(Action):<br><br>    &quot;&quot;&quot;Rasa action to parse user text and pulls a corresponding answer <br>    from google sheet based on the intent and entities.<br>    If there is no answer in the google sheet, it will use the ChatGPT API&quot;&quot;&quot;<br><br>    def name(self) -&gt; Text:<br>        return &quot;simple_google_sheet_or_chatgpt_action&quot; <br><br>    def run(self, dispatcher: CollectingDispatcher,<br>            tracker: Tracker,<br>            domain: Dict[Text, Any]) -&gt; List[Dict[Text, Any]]:<br><br>        # Get the latest user text and intent<br>        user_text = tracker.latest_message.get(&#39;text&#39;)<br>        intent = tracker.latest_message.get(&#39;intent&#39;).get(&#39;name&#39;)<br>        entities = tracker.latest_message.get(&#39;entities&#39;)<br>        <br>        # Dispatch the response from OpenAI to the user<br>        dispatcher.utter_message(&#39;Google Sheets (custom_action): &#39; + str(self.get_answers_from_sheets(intent, entities, user_text)))<br><br>        return []<br>    <br>    def get_answers_from_sheets(self, intent, entity, user_text):<br><br>        # Connect to Google Sheets<br>        sheet_url = os.getenv(&quot;SHEET_URL&quot;)<br><br>        GOOGLE_SHEET_URL = f&quot;https://docs.google.com/spreadsheets/d/{sheet_url}/export?format=csv&amp;gid=0&quot;<br>        s = requests.get(GOOGLE_SHEET_URL).content<br>        <br>        # Read the contents of the URL as a CSV file and store it in a dataframe<br>        proxy_df = pd.read_csv(io.StringIO(s.decode(&#39;utf-8&#39;)))        <br>        <br>        if entity:<br>            # Filter the dataframe by the intent column and retrieve the answer list<br>            filtered_df = proxy_df[(proxy_df[&#39;Intent&#39;] == intent) &amp; (proxy_df[&#39;Entity&#39;] == entity[0][&#39;value&#39;])]<br><br>            if filtered_df.empty:<br>                answer = get_answers_from_chatgpt(user_text) <br>            else:<br>                answers = filtered_df[&#39;Answer&#39;].tolist()<br>                answer = random.choice(answers)<br>        else:<br>            answer = get_answers_from_chatgpt(user_text)<br><br>        # Return the answer list<br>        return answer</pre><p>The <em>Simple_Google_sheet_or_ChatGPT_Action </em>class is used to parse the user text and retrieve a corresponding answer from the Google Sheet based on the intent and entities.</p><p>To use Google Sheets as a database for your chatbot, the <em>get_answers_from_sheets </em>function is used to connect to the Google Sheet and retrieve the answer list. The function filters the data frame by the intent column and retrieves the answer list. If an entity exists in the user query, the function filters the data frame by the entity column. If there is a match, the function retrieves the answer list. If there is no match, the ChatGPT API is used to retrieve a response.</p><p>With Google Sheets Integration, chatbots can be easily updated with new responses without having to modify the code. This makes it easy for non-technical users to maintain and update the chatbot.</p><h4><em>Component integration, training, and evaluation</em></h4><p>We must first design the conversation flow to build and test the chatbot. This involves creating the chatbot’s stories, intents, entities, and responses. Once our conversation flow is designed, we can integrate Rasa, ChatGPT, and Google Sheets to create a fully functional chatbot that can respond to user inputs and queries.</p><p>In order to create a fully functional chatbot, the YAML files for stories, rules, nlu, and domain need to be adjusted. The file nlu.ymlcontains examples of user inputs with their corresponding intents, such as “utter_intent” and “what_is_x”. The stories.ym file contains a sequence of actions and intents used to test the chatbot’s conversation flow. The domain.ym file lists the intents, entities, and responses used in the chatbot and the available actions.</p><pre>ֶ֫nlu:<br>- intent: utter_intent<br>  examples: |<br>    - can you utter?<br>    - do you know how to utter?<br>    - show me an example of utter?<br>    - how to utter?<br>- intent: what_is_x<br>  examples: |<br>    - what is a [dog](object_entity)?<br>    - what is a [chair](object_entity)?<br>    - what a [dog](object_entity) is?<br>    - what a [chair](object_entity) is</pre><pre>domain:<br>intents:<br>  - greet<br>  - utter_intent<br>  - what_is_x<br><br>entities:<br>  - object_entity<br><br>responses:<br>  utter_response:<br>  - text: &quot;This is direct utter, no custom action&quot; <br><br>actions:<br>- utter_response<br>- simple_action_hello_world<br>- simple_google_sheet_or_chatgpt_action <br>- action_gpt_default_fallback<br><br>session_config:<br>  session_expiration_time: 60<br>  carry_over_slots_to_new_session: true</pre><pre>stories:<br>- story: simple utter<br>  steps:<br>  - intent: utter_intent<br>  - action: utter_response<br><br>- story: simple action use - &#39;hello_world&#39;<br>  steps:<br>  - intent: greet<br>  - action: simple_action_hello_world<br><br>- story: simple google sheet action use<br>  steps:<br>  - intent: what_is_x<br>  - action: simple_google_sheet_or_chatgpt_action</pre><pre>rules:<br>- rule: out-of-scope<br>  steps:<br>  - intent: nlu_fallback<br>  - action: action_gpt_default_fallback</pre><p>After creating the necessary YAML files, the Rasa, ChatGPT, and Google Sheets components must be integrated. For example, the <em>simple_google_sheet_or_chatgpt_action</em> function pulls answers from Google Sheets or ChatGPT based on the user’s intent and entities. The chatbot can then be trained on this integrated model using the Rasa framework and evaluated for any issues or errors.</p><p>To train the bot with the new configuration, we can run the following command:</p><pre>rasa train</pre><p>In order to allow access to the server of the actions, you need to activate it in a separate terminal window. To do this, navigate to the main directory and run the following command:</p><pre>rasa run actions --actions actions </pre><p>After training, we can test the bot in the shell using the following:</p><pre>rasa shell</pre><p>This will open up the chat interface where we can interact with the bot and test its responses.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-w_RXsrpbuG9nKvhYws_Pg.png" /></figure><p>To access the chat on an HTML page, you can include the Rasa Chat Widget code in your HTML file. Create a new index.html file in the main folder and add the following content:</p><pre>֫&lt;!DOCTYPE html&gt;<br>&lt;html&gt;<br>    &lt;head&gt;<br>        &lt;style&gt;<br>            body {<br>                padding: 0;<br>                width: 100vw;<br>                height: 100vh;<br>                margin: 0;<br>                background-image: url(https://medium.com/&#39;https://www.w3schools.com/w3images/ocean.jpg&#39;);<br>                background-size: contain;<br>            }<br>        &lt;/style&gt;<br>    &lt;/head&gt;<br>    &lt;body&gt;<br>        &lt;div <br>            id=&quot;rasa-chat-widget&quot; <br>            data-websocket-url=&quot;http://localhost:5005/&quot;  <br>            data-height=475&gt;<br>        &lt;/div&gt;<br>        &lt;script src=&quot;https://unpkg.com/@rasahq/rasa-chat&quot; type=&quot;application/javascript&quot;&gt;&lt;/script&gt;<br>    &lt;/body&gt;<br>&lt;/html&gt;</pre><p>The code creates a div element with the id “rasa-chat-widget” and specifies the URL of the Rasa server using the data-websocket-url attribute. You can also adjust the chat window&#39;s height by changing the data-height attribute&#39;s value. Finally, we can test the bot using the REST API by running the following command:</p><pre>֫rasa run -m models --enable-api --cors &quot;*&quot;</pre><p>This will start the Rasa server, and we can send HTTP requests to the server to interact with the bot.</p><p>Once the index.html is included, you can access the chat widget by opening the HTML file in a web browser. The chat widget should appear as a window on the bottom right corner of the page.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Gq6Ec2Uk85ttSmzuijFlew.png" /></figure><p>If you have deployed the chatbot on a server and want to access it via requests, you can use the requests library in Python.</p><p>First, you need to specify the URL of the Rasa server using the rasa_url variable. Then, you can create a dictionary containing the message information with the keys &quot;sender&quot; and &quot;message&quot;.</p><p>Finally, you can use the requests.post() method to send a POST request to the Rasa server with the message information and headers containing the content type and accept type.</p><p>The server will then return a response, which you can print out as a JSON object using response.json(). In the example code you provided, the message variable contains a test message &quot;hello&quot; sent by a user with ID &quot;test&quot;.</p><pre>import requests<br><br>rasa_url = &#39;http:/localhost:5005/webhooks/rest/webhook&#39;<br><br>header = {&#39;Content-Type&#39;: &#39;application/json&#39;, <br>          &#39;Accept&#39;: &#39;application/json&#39;}<br><br>message = {&#39;sender&#39;: &#39;test&#39;, <br>           &#39;message&#39;: &#39;hello&#39;}<br><br><br>response = request.post(rasa_url, header, message)<br>print(response.json())</pre><h4>Conclusion</h4><p>In this guide, we have covered the basics of Rasa, Google sheets, and ChatGPT integration, including setting up a Rasa project, defining intents, entities, and responses, training the language model, and implementing ChatGPT to provide more sophisticated responses. By following the steps outlined in this guide, anyone can develop their own advanced chatbot.</p><p>As AI technology continues to evolve, the possibilities for chatbots are endless. In the future, we can expect to see chatbots with more advanced natural language processing capabilities, as well as the ability to understand emotions and respond accordingly. The use of AI chatbots is also likely to expand into new industries, such as healthcare and finance, providing even more benefits to businesses and their customers.</p><p>If you’re interested in developing your own advanced chatbot, I encourage you to explore Rasa, ChatGPT, and other AI tools and technologies.</p><p>By doing so, you can create a chatbot that not only provides excellent customer service but also sets your business apart from the competition.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3cd641967237" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[In a good smell-Part 2]]></title>
            <link>https://medium.com/@almogklein.16/in-a-good-smell-part-2-c9a1eb22a7de?source=rss-57479f4e239d------2</link>
            <guid isPermaLink="false">https://medium.com/p/c9a1eb22a7de</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[classification]]></category>
            <category><![CDATA[visualization]]></category>
            <category><![CDATA[senate]]></category>
            <category><![CDATA[k-means]]></category>
            <dc:creator><![CDATA[Almogklein ]]></dc:creator>
            <pubDate>Sun, 05 Feb 2023 06:53:31 GMT</pubDate>
            <atom:updated>2023-02-05T06:53:31.716Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qPGgFApYNe2ouDoB6bVc7g.png" /><figcaption>Taken from “phys.org” — <a href="https://phys.org/news/2022-06-friends-people-drawn.html">Link</a></figcaption></figure><p>Welcome back to the second part of my research journey into the world of scent analysis and classification. In Part 1, we discussed the initial steps of cleaning, pre-processing, and analyzing the unique odor data collected from digital sensors. We also gained insights into the numerical features of the dataset, which helped inform feature selection and transformation.</p><p>In this phase, we continued our research by diving deeper into the data and applying dimension extraction algorithms such as PCA and LDA. This allowed us to visualize the odor samples in 2D and 3D. We also used K-means to cluster the data into groups after the dimensionality reduction transformation and tested the model’s performance in low-dimensional space.</p><blockquote><strong><em>Table of Contents:</em></strong></blockquote><blockquote>- Dimension extraction using PCA and LDA</blockquote><blockquote>- Visualizing the odor samples in 2D and 3D</blockquote><blockquote>- Clustering the data using K-means</blockquote><blockquote>- Model evaluation on low-dimensional space</blockquote><blockquote>- Results and discussion</blockquote><blockquote>- Summary</blockquote><h4>Dimension extraction using PCA and LDA:</h4><p>We applied PCA and LDA to extract the most significant dimensions from the data and reduce the number of features. Both are dimensionality reduction techniques that help to project high-dimensional data into a lower-dimensional space while preserving as much information as possible.</p><p>PCA is an unsupervised technique that finds the most significant directions of variation in the data, while LDA is a supervised technique that maximizes the separation between different classes.</p><p>After applying both to the data, we could visualize the odor samples in 2D and 3D by plotting the transformed data on a scatter plot. But wait, the real excitement comes when we compare the results and see how LDA outperforms PCA in this task, giving us a clearer and more separable cluster.</p><h4>Visualizing the odor samples in 2D and 3D:</h4><p>After applying PCA and LDA to extract the most significant dimensions from the data, we used visualization techniques to represent the transformed data. We created a 2D scatter plot to visualize the odor samples in a low-dimensional space using the first two principal components obtained from PCA and LDA. The scatter plot was colored based on class labels, allowing us to observe the separation between different classes in reduced-dimension space.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ojET7FsUvU7bi-GB-CsvoA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*s6yVPtJUmTppLLTNjCy2vg.png" /></figure><p>These plots give us a clear visual representation of the differences between the methods. PCA plot shows data points spread out and needing better separation into distinct clusters. LDA plot separates data points into two large central blocks, with separation more pronounced and distinct making it easier to identify and analyze different clusters, highlighting the superiority of the LDA method in cluster separation, crucial for accurate and effective data analysis.</p><p>The LDA method gives a more informative and meaningful representation of the data, allowing us to gain a deeper understanding of underlying patterns and relationships in odor samples.</p><p>We also created a 3D scatter plot to visualize the odor samples in a low-dimensional space using the first three principal components obtained from LDA, providing a complete representation of the data and allowing us to observe a separation between different classes in three dimensions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/813/1*w4bKajBity0nj4M0JQ_3DA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/892/1*OoEeBlSE3CrG1OGcvcoRSw.png" /></figure><h4>Clustering the data using K-means:</h4><p>In this section, we used the K-means algorithm to cluster the data into groups based on similarity. Clustering is an unsupervised learning technique that groups similar data points together, and K-means is one of the most widely used clustering algorithms.</p><p>The actual cluster number for the odor samples was used (k=9). The resulting clusters were compared with the class labels to evaluate the performance of the K-means algorithm in low-dimensional space.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CECHFtIuswGvvJZET-E6lQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8BjiqVttlw2UvuwBGGPV6A.png" /></figure><p>We can also evaluate the clustering results by comparing the predicted clusters with the actual class labels. One standard method uses a confusion matrix, which compares the predicted clusters with the actual class labels and shows the number of correct and incorrect predictions. This can help us understand the K-means algorithm&#39;s performance and identify areas for improvement.</p><p>In conclusion, K-means is a practical algorithm for clustering high-dimensional data. It can be used with visualization techniques and performance metrics to better understand the data. Using K-means to cluster the odor samples in low-dimensional space, we could identify patterns and similarities between the samples and evaluate the algorithm&#39;s performance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tVAOaN9_hmfBUrS8Ad7flQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1004/1*PQN1m_QpP3eCh0ZWjF3muQ.png" /></figure><p>Additionally, by combining LDA with K-means clustering, we can further improve the analysis of the odor data. K-means clustering groups similar data points together based on their features, which can help to identify patterns and similarities within the data.</p><p>By visualizing the K-means clustering results on the LDA-transformed data, we can see how the samples are grouped into clusters and gain a deeper understanding of the underlying relationships within the data. Overall, LDA and K-means are powerful tools for analyzing odor data and can lead to more accurate and effective analysis. Together, these methods can accurately identify and classify different odors, making them useful for applications such as quality control in perfumes and fragrances.</p><h4>Model evaluation on low-dimensional space:</h4><p>To evaluate the performance of the K-means algorithm in low-dimensional space, we used classification indices such as F1-score, precision, and accuracy.</p><p>These metrics provide a quantitative measure of the performance of the clustering algorithm and allow us to compare the results with other methods. All these metrics can be calculated using the Sklearn library’s metrics module.</p><h4>Results and discussion:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/471/1*gaXp07-xfiBMquXl5MZoOg.png" /></figure><p>The results of the classification report indicate that our model performed well, with high precision, recall, and F1-score values. The model&#39;s accuracy was also high, indicating that the model could correctly classify the samples into the correct clusters.</p><p>The silhouette coefficient was also high, indicating that the samples were well separated from other clusters. The adjusted Rand index also indicates the good performance of the model.</p><blockquote><strong><em>Discussion:</em></strong></blockquote><blockquote><strong>1. How does the LDA method compare to PCA for odor analysis?</strong></blockquote><blockquote>The LDA method is better than PCA for analyzing odors because it maximizes the separation between different classes. It can better identify and separate clusters of odors based on chemical composition, resulting in a more accurate data representation.</blockquote><blockquote><strong>2. How could the results of this analysis be applied in a real-world scenario?</strong></blockquote><blockquote>The results of this analysis could be applied in a real-world scenario such as quality control in the perfumes and fragrances industry to identify and classify different odors. LDA method can be used to analyze the chemical composition of fragrance samples and classify them into groups based on similarity and K-means method can be used to group them into subgroups for identifying patterns or similarities between the samples.</blockquote><h4>Summary:</h4><ul><li>Use of dimensionality reduction techniques such as PCA and LDA</li><li>Visualization of data in 2D and 3D</li><li>Clustering of data using the K-Means method to classify and group the samples</li><li>Valuable insights into the patterns and relationships present in the dataset</li></ul><p>This research journey has provided a deeper understanding of odor samples&#39; underlying patterns and relationships. We have successfully cleaned and pre-processed the data, visualized the data in 2D and 3D, and applied dimensionality reduction techniques such as PCA and LDA. Used K-means clustering to group the samples into subgroups based on their similarity and evaluated the model’s performance in low-dimensional space.</p><p>We have seen that LDA is a powerful tool for odor analysis, allowing us to understand the underlying patterns and relationships in the odor samples. The results of this analysis demonstrate the potential of LDA for use in real-world applications such as quality control, product development, and industrial process monitoring.</p><p>The journey has been exciting, and I hope this research will inspire others to further explore the scent analysis and classification field.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c9a1eb22a7de" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[In a good smell — Part 1]]></title>
            <link>https://medium.com/@almogklein.16/in-a-good-smell-part-1-42adbd9e01d8?source=rss-57479f4e239d------2</link>
            <guid isPermaLink="false">https://medium.com/p/42adbd9e01d8</guid>
            <category><![CDATA[exploratory-data-analysis]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[smells]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Almogklein ]]></dc:creator>
            <pubDate>Sat, 07 Jan 2023 20:33:37 GMT</pubDate>
            <atom:updated>2023-01-25T21:07:10.790Z</atom:updated>
            <content:encoded><![CDATA[<h3>In a good smell-Part 1</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qPGgFApYNe2ouDoB6bVc7g.png" /><figcaption>Taken from “phys.org” — <a href="https://phys.org/news/2022-06-friends-people-drawn.html">Link</a></figcaption></figure><p>Welcome to the first part of my research journey into the world of scent analysis and classification. I recently had the opportunity to work with unique odor data collected from digital sensors, which has opened up a new realm of possibilities for me to explore.</p><p>This post will focus on cleaning, pre-processing, data analysis, and labeling the different smells in the data. In the second part of the post, we will continue our research by visualizing an odor sample through dimension extraction algorithms and associating new samples with subgroups.</p><p>Before diving into the data, it’s essential to understand the distinction between “smell” and “odor.”</p><p>“Smell” refers to the general sense of detecting and identifying odors, while “Odor” specifically refers to the characteristic or quality of a smell. Different smells can be pleasant or unpleasant to humans, depending on individual preferences and past experiences. For example, the smell of freshly baked cookies may be pleasant to some, while the smell of cigarette smoke may be unpleasant to others.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/874/1*M3XYb092wHL8a60a_Ze28Q.png" /><figcaption>Google images with the quarry, “smell and odor”</figcaption></figure><h4>alright, alright, alright — DATA!</h4><p>Now, let’s take a look at the data. When I received it, the data had no column names, so I had to manually clean it up.</p><p>The resulting table contained records of values from various odor sensors, with ten columns representing each sensor (labeled with an “S” and a number). Each row represented a sample from each sensor for the same odor sample.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/537/1*RNp_ESJ6qSdFFFZ4MxXAvg.png" /></figure><p>The data was collected by exposing digital sensors to an odor sample for about a minute, with a resolution of one sample per second. After exposure to the material, the sensors were exposed to clean air, and this process was repeated a few times (called a “stroke”).</p><p>Each material was sampled with about 3–5 strokes, and a sample of smell was called a “Contegees Number,” which had a “Base” and “Batch Index.”</p><h4>Structure of numerical features</h4><p>Now that we have a basic understanding of the data let’s delve into the numerical features and see how many unique values each has.</p><p>This will give us an idea of the number of binary, ordinal, and continuous features in the dataset. By examining this information, we can gain valuable insights that will help us better understand the data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/939/1*8MXRnk9J4n9mLOfCF29ucQ.png" /></figure><p>In addition, the black background tells us that the data is complete and does not contain null values.</p><h4>Numerical features</h4><p>One tool we can use to visualize the numerical features is the pandas’ .plot() function, which creates a plot for each sample in the dataset, with the y-axis displaying the feature value and the x-axis representing the sample index. These plots can be very helpful for data cleaning and exploratory data analysis.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FaBRfakYxagjt6abhCNbHA.png" /></figure><h4>Feature distribution</h4><p>Another helpful technique for gaining insights into the data is examining the distribution of values for each feature. One quick way to do this for numerical features is using histogram plots.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TExaLRt0UOYrNuSKyccP6w.png" /></figure><p><strong>Most frequent entry:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RsRTwFlmFjWrJ06gi6y9WA.png" /></figure><p>The above image shows the most frequent entry in the data, which can give us an idea of the dominant values in the dataset.</p><h4>Correlation:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/805/1*b9gxmwT5CqDrijer1AqdHw.png" /></figure><p>The above image shows the correlation between different features in the dataset. This can help us identify relationships between features and inform feature selection or transformation.</p><h4>PairPlot:</h4><p>Finally, we have the pairplot created using seaborn’s pairplot function. This allows us to visualize the relationships between the continuous features we selected. However, it’s important to note that it may take a long time to create all the subplots if we use it for more than approximately ten features at once.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/910/1*VQAJwI5Atv87Sj9-9KKt-A.png" /></figure><h4>Summary of the post:</h4><ul><li>Analyzing odors using data collected from digital sensors.</li><li>Initial steps included cleaning and pre-processing the data and analyzing and labeling the different smells.</li><li>Numerical features were examined to gain insights into the dataset and inform feature selection and transformation.</li><li>Tools used for data visualization included pandas’ .plot() function, histogram plots, and seaborn’s pairplot function.</li></ul><p>In the next part of this research journey, we will continue exploring this fascinating world of odors and the potential of digital sensors to detect and analyze them.</p><p>Stay tuned!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=42adbd9e01d8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[NASA space apps hackathon: Exploring Venus Together]]></title>
            <link>https://medium.com/@almogklein.16/nasa-space-apps-hackathon-exploring-venus-together-4ce2009ba788?source=rss-57479f4e239d------2</link>
            <guid isPermaLink="false">https://medium.com/p/4ce2009ba788</guid>
            <category><![CDATA[renewable-energy]]></category>
            <category><![CDATA[energy]]></category>
            <category><![CDATA[space]]></category>
            <category><![CDATA[venus]]></category>
            <category><![CDATA[nasa]]></category>
            <dc:creator><![CDATA[Almogklein ]]></dc:creator>
            <pubDate>Mon, 03 Oct 2022 15:34:20 GMT</pubDate>
            <atom:updated>2022-10-03T18:26:39.673Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6N_M6UptY1EaFZq375P2Hw.png" /><figcaption>The Challenge page — NASA’s official <a href="https://2022.spaceappschallenge.org/challenges/2022-challenges/exploring-venus/details">website</a></figcaption></figure><p>We are VStars Energy, a group of engineering students (<a href="https://www.linkedin.com/in/reem-ghanim-446709228/">Reem</a>, <a href="https://www.linkedin.com/in/nir-gavriel/">Nir</a>, and <a href="https://www.linkedin.com/in/almog-klein-562b6a15b/">myself</a>) who are passionate about space and have joined forces to enter the NASA challenge competition. In the next post, I will describe our experience during the hackathon, some of the things we learned, and our final products.</p><blockquote>Post structure:</blockquote><blockquote><em>- SPACE APPS hackathon<br>- Our Challenge: Exploring Venus Together<br>- Venus environment<br>- Energy systems<br>- Introducing our concept<br>- Conclusion</em></blockquote><h3>SPACE APPS hackathon:</h3><p>NASA International Space Apps Challenge (Space Apps) is a hackathon that brings together coders, scientists, designers, storytellers, makers, builders, technologists, and others from around the world to solve real-world problems on Earth and in space by leveraging NASA’s free and open data.<br>Each October, for two days, Space Apps brings participants from around the world together at hundreds of in-person and virtual local events to solve challenges submitted by NASA experts. After the hackathon, project submissions are judged by space agency experts, and winners are selected for one of 10 Global Awards.<br>Space Apps provides problem-solvers worldwide with NASA’s free and open data, allowing teams to learn how to use these resources to solve each year’s challenges.<br>Space Apps also inspire collaboration, creativity, and critical thinking. <br>Using NASA’s open data to build innovative solutions for Earth’s and space’s challenges, NASA’s International Space Apps Challenge has engaged 180,000+ registrants from 150+ countries/territories since 2012.</p><h4>Our Challenge: Exploring Venus Together</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/828/1*im8AF1R7t252fTwFG2peiA.png" /><figcaption>The Challenge page - NASA’s official <a href="https://2022.spaceappschallenge.org/challenges/2022-challenges/exploring-venus/details">website</a></figcaption></figure><p>We chose this challenge since it is among the most challenging, considering that the longest mission inside the Venus atmosphere lasted just over an hour (due to extreme environmental conditions — also known as the evil twin of Earth).</p><h3>Basic background</h3><h4>Venus environment:</h4><p>It’s not for nothing that Venus is called Earth’s evil twin. While their dimensions are similar, their environmental conditions are very different, such as extreme temperatures and gas compounds in the atmosphere which contain potent acids. Without mentioning the compressed atmosphere, all of these conditions surely entitle this planet’s name.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/697/1*4-o2yVuysToMBFkvHNgMTw.png" /><figcaption>From an article that NASA published, “Solar Powered Flight on Venus” — <a href="https://drive.google.com/file/d/16F5MkuAeRifF99ZYZLGmkVFkbi8ZEB1E/view?usp=sharing">Link</a></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/471/1*iKeMvlh8WTMd_To0PcziSQ.png" /><figcaption>Temperature and pressure variables throughout the layers of the atmosphere on the planet Venus.</figcaption></figure><h4>Energy systems:</h4><p>A general energy system usually contains two parts: a generative part that produces the energy and a storage part which stores it.</p><h4>Generation</h4><p>For the part of creating the energy, we decided to use renewable energy since we do not want to depend on a distant factor (for example, rocket fuel which needs to be brought from the earth and be carefully calculated beforehand for specific purposes) and to be able to provide energy with the help of the existing renewable resources.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c3mMsUaGfKkfRW5ExLnqiA.png" /><figcaption>Breakdown of renewables in the energy — <a href="https://ourworldindata.org/renewable-energy">Link</a></figcaption></figure><h4>Storage</h4><p>Energy Storage System (ESS) components are grouped according to function into battery components, parts required for reliable system operation, and grid connection components.</p><p>A battery system consists of a battery pack that connects multiple cells at the appropriate voltage and capacity. The BMS protects the cells from harmful operation in terms of voltage, temperature, and current to achieve reliable and safe operation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*1iG0mUrgW-TsgGWnNUtBiQ.png" /><figcaption>From an article that NASA published, “Energy Storage Technologies<br>for Future Planetary Science Missions” — <a href="https://drive.google.com/file/d/136jyGXS6qDtitYXSV6uA8e-8QoRV5DNo/view?usp=sharing">Link</a></figcaption></figure><h3><em>Introducing our concept</em></h3><p>Our solution is a Wind Turbine that relies on the strong winds on Venus. The turbine technology is based on our earth’s technology, the ability to adjust its direction by the wind direction for maximum efficiency. <br>We suggested materials that have been tested by NASA for the turbine body, mechanical parts, electronics, and the battery, which can handle Venus’s extreme conditions. The purpose of the turbine is to supply the energy with a wireless charger, and any extra energy will be saved in the turbine battery.</p><p>To perform our research, we assumed that Venus missions would rely on the same systems as those used for Mars missions allowing us to simulate a mission using the same methods. We assume that the power is 100 W for 4 hours.<br>Throughout our research on Venus’s past missions, we’ve realized that the batteries do not operate as needed. The first step to finding a solution was to understand if there was a possibility to use already made batteries that could survive in the Venus environment. We found that NASA can use batteries that are explicitly meant for future missions to Venus.</p><blockquote><em>There are three kinds of materials that can be used for those kinds of batteries :<br></em><strong><em>1. Lil-FeS2<br>2. Sodium-Sulfur (Na-S)<br>3. Sodium-Nickel Chloride (Na-NiCl2)</em></strong></blockquote><p>We decided to use the existing batteries developed by NASA and establish a system that can create energy in the most innovative way possible, depending on the natural resources on Venus. Our system’s primary resource that will be used is the strong wind currents close to the planet’s surface.</p><p>We developed a wind turbine to harvest the energy from the strong surface winds as we found this to be the most effective. The turbine can open and close by leveraging its folding pole. The base of the turbine uses SpaceX’s folding legs technology, and the turbine blades can also be folded so that the turbine structure becomes modular.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/956/1*eivaEVZPG_309uhvFbWJFw.png" /><figcaption>Illustrating the idea of folding the turbine blades in order to reduce accelerated wear during strong winds and full load — A 3D solid model of our energy system.</figcaption></figure><p>The energy created by the turbine then charges the already made NASA batteries. Once the batteries are set, the rover can drive onto a platform and get charged wirelessly without any mechanics, lowering poseable issues.</p><blockquote><em>The materials needed to build our system are based on NASA’s research and include the following:</em></blockquote><blockquote><strong><em>1. Carbon Graphite</em></strong></blockquote><blockquote><strong><em>2. Silicon Carbide</em></strong></blockquote><p>We believe using Carbon Graphite would be more efficient as the structure will be lighter (3-ton max).</p><h3><em>Conclusion</em></h3><p>The hackathon was the perfect journey for us. We learned much about energy production and energy storage solutions, considering extreme and very different conditions. Very different from what we know on earth, by performing deep research of suitable materials, venus data, lessons from previous venus missions, etc.<br>Our approach was to find a realistic solution to an existing problem, using our knowledge and eagerness to learn and explore the space field.<br>The evolution of our project had several stages, each one differing from the previous by a minor or major change we thought of by new insights we discovered, by the mentors and judge’s help, and by help from other team members.</p><blockquote>The model is only a concept. Due to a limited time frame, we didn’t perform stress tests and optimization of aerodynamics, structure planning, mechanics, and electronics.</blockquote><p>Overall it was a great opportunity and wonderful experience,</p><p>See you next year! :)</p><h3>References</h3><p>For the basic data about Venus, we used the followings :</p><p><a href="https://www.researchgate.net/publication/24381363_Solar_Powered_Flight_on_Venus">Solar Powered Flight on Venus</a></p><p><a href="https://solarsystem.nasa.gov/news/1519/venus-resources/?page=0&amp;per_page=40&amp;order=created_at+desc&amp;search=&amp;tags=Venus&amp;category=324">Venus Resources</a></p><p>for the technology we developed, we used the followings :</p><p><a href="https://ssed.gsfc.nasa.gov/davinci/roundtable">DAVINCI In Situ Capability Roundtable</a></p><p><a href="https://solarsystem.nasa.gov/resources/549/energy-storage-technologies-for-future-planetary-science-missions/">Energy Storage Technologies for Future Planetary Science Missions</a></p><p><a href="https://www.nasa.gov/feature/automaton-rover-for-extreme-environments-aree/">Automaton Rover for Extreme Environments (AREE)</a></p><p>One of the most important things was to learn from others with challenges similar to ours, so we used :</p><p><a href="https://www.nasa.gov/exploring-hell-venus-rover-challenge">Exploring Hell: Avoiding Obstacles on a Clockwork Rover</a></p><p><a href="https://www.youtube.com/watch?v=y83KXU26Bjg&amp;ab_channel=MichelvanBiezen">Winds speed vs. height on venus</a></p><p><a href="https://mars.nasa.gov/mer/mission/rover/energy/">Mars rover energy</a></p><p><a href="https://www.engineeringtoolbox.com/wind-power-d_1214.html">Wind turbine size calculation</a></p><p><a href="https://www.hou.usra.edu/meetings/vexag2017/pdf/8037.pdf">Turbine for power generation on venus</a></p><p><a href="https://sci.esa.int/web/venus-express/-/33010-summary">https://sci.esa.int/web/venus-express/-/33010-summary</a></p><p><a href="https://www.sciencedirect.com/book/9780128053232/high-temperature-thermal-storage-systems-using-phase-change-materials">https://www.sciencedirect.com/book/9780128053232/high-temperature-thermal-storage-systems-using-phase-change-materials</a></p><p><a href="https://solarsystem.nasa.gov/planets/venus/overview/#:~:text=Venus%20has%20a%20thick%2C%20toxic,is%20closer%20to%20the%20Sun">https://solarsystem.nasa.gov/planets/venus/overview/#:~:text=Venus%20has%20a%20thick%2C%20toxic,is%20closer%20to%20the%20Sun</a>.</p><p><a href="https://solarsystem.nasa.gov/planets/venus/in-depth/">https://solarsystem.nasa.gov/planets/venus/in-depth/</a></p><p><a href="https://ssed.gsfc.nasa.gov/davinci/roundtable">https://ssed.gsfc.nasa.gov/davinci/roundtable</a></p><p><a href="https://solarsystem.nasa.gov/planets/venus/overview/">https://solarsystem.nasa.gov/planets/venus/overview/</a></p><p><a href="https://iopscience.iop.org/article/10.3847/PSJ/ac63c2/pdf">https://iopscience.iop.org/article/10.3847/PSJ/ac63c2/pdf</a></p><p><a href="https://ssed.gsfc.nasa.gov/davinci/roundtable">https://ssed.gsfc.nasa.gov/davinci/roundtable</a></p><p><a href="https://www2.jpl.nasa.gov/adv_tech/ballutes/Blut_ppr/vnusmatl.pdf">https://www2.jpl.nasa.gov/adv_tech/ballutes/Blut_ppr/vnusmatl.pdf</a></p><p><a href="https://www.airscort.me/">https://www.airscort.me/</a></p><p><a href="https://www.roccarbon.com/what-is-carbon-graphite/">https://www.roccarbon.com/what-is-carbon-graphite/</a></p><p><a href="http://www.shadetreephysics.com/vel/1918vpt.htm">http://www.shadetreephysics.com/vel/1918vpt.htm</a></p><p><a href="https://arc.aiaa.org/doi/pdf/10.2514/1.A34617">Experimental Study of Structural Materials for Prolonged Venus Surface Exploration Missions</a></p><p><a href="https://drive.google.com/drive/folders/1CKRx7fHXqwEiw9FVZQbRd814TWS87bzR?usp=sharing">https://drive.google.com/drive/folders/1CKRx7fHXqwEiw9FVZQbRd814TWS87bzR?usp=sharing</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4ce2009ba788" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>