<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Aishwary Singh Chauhan on Medium]]></title>
        <description><![CDATA[Stories by Aishwary Singh Chauhan on Medium]]></description>
        <link>https://medium.com/@imaishwary007?source=rss-1dc6b645fb2------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*vuPH0BrfFLWdrBy-dqM4-w.jpeg</url>
            <title>Stories by Aishwary Singh Chauhan on Medium</title>
            <link>https://medium.com/@imaishwary007?source=rss-1dc6b645fb2------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 06:01:21 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@imaishwary007/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Easily Load Excel File To Snowflake (Part 2 — using Streamlit App deployed in Snowflake)]]></title>
            <link>https://medium.com/@imaishwary007/easily-load-excel-file-to-snowflake-part-2-using-streamlit-app-deployed-in-snowflake-7d2f67ae58c2?source=rss-1dc6b645fb2------2</link>
            <guid isPermaLink="false">https://medium.com/p/7d2f67ae58c2</guid>
            <category><![CDATA[snowflake]]></category>
            <category><![CDATA[excel]]></category>
            <category><![CDATA[load]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[streamlit]]></category>
            <dc:creator><![CDATA[Aishwary Singh Chauhan]]></dc:creator>
            <pubDate>Mon, 05 May 2025 11:53:57 GMT</pubDate>
            <atom:updated>2025-05-05T11:53:57.683Z</atom:updated>
            <content:encoded><![CDATA[<h3>Easily Load Excel File To Snowflake (Part 2 — using Streamlit App deployed in Snowflake)</h3><p>In Snowflake we can use Snowsight or Classic Console to load files with structured data (e.g. CSV, TSV) or semi-structured data (e.g. JSON, Avro, ORC, Parquet, or XML).</p><p>However currently there is no native ingestion of excel files .</p><p>Wouldn’t it be great if we can directly load excel data as well using a simple User Interface ? Moreover we wouldn’t have to worry about file formats (field delimiter , encoding , row delimiter etc.) which we typically have to deal with while loading data using CSV ?</p><p>Well we have already discussed how to achieve this in the previous post — <a href="https://medium.com/@imaishwary007/easily-load-excel-file-to-snowflake-using-streamlit-ab3ac5636950">https://medium.com/@imaishwary007/easily-load-excel-file-to-snowflake-using-streamlit-ab3ac5636950</a></p><p>This is an Part 2 of the above Post, with the updated code that can be deployed in the Snowflake itself just with few lines of code.</p><p><em>Code tested with below dependencies -</em></p><blockquote><em>openpyxl==3.1.5<br>snowflake-connector-python==1.30.0<br>streamlit==1.42.0</em></blockquote><h4>1. Create an new Streamlit application in Snowflake</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NJb4C9SQc01YrWDM8q7JIA.png" /><figcaption>Snowflake Streamlit apps section</figcaption></figure><h4>2. Add Below Code</h4><pre># Import python packages<br>import streamlit as st<br>from snowflake.snowpark.context import get_active_session<br>import pandas as pd<br>from datetime import datetime<br>import traceback<br><br># Get the current credentials<br>session = get_active_session()<br><br>def get_data_str(data_list):<br>    try:<br>        data_str = &#39;&#39;<br>        for ind , row_list in enumerate(data_list) :<br>            for index, value in enumerate(row_list) :<br>                if index == 0:<br>                    data_str += &#39;(&#39;<br>                data_str += f&quot;&#39;{str(value)}&#39;&quot; if value!= &#39;NULL&#39; else value<br>                if index == len(row_list) - 1:<br>                    data_str += &#39;)&#39;<br>                else :<br>                    data_str += &#39;,&#39;<br>            if ind != len(data_list) - 1:<br>                data_str += &#39;,&#39;<br>        return data_str<br>    except Exception as e:<br>        print(f&#39;failed to get data... {traceback.print_exc}&#39;)<br>        raise(e)<br><br>target_table_name = st.text_input(&quot;Enter Table Name as &lt;Database&gt;.&lt;Schema&gt;.&lt;Table&gt; eg: streamlit_db.test.hello_word&quot;)<br><br>st.header(&#39;:blue[Snowflake Excel Data Loader]&#39;)<br>uploaded_file = st.file_uploader(&quot;Drop your xlsx here&quot;, type={&quot;xlsx&quot;})<br><br>if uploaded_file is not None:<br>    xls = pd.ExcelFile(uploaded_file)<br>    sheet_names = xls.sheet_names<br><br>    with st.form(key=&#39;sheet_selector_form&#39;):<br>        selected_sheet = st.selectbox(&quot;Select a sheet to load data&quot;, sheet_names)<br>        selected_skip_lines = st.text_input(&quot;Number of lines to skip (including header if present)&quot;)<br>        submit_button = st.form_submit_button(label=&#39;Submit&#39;)<br><br>        if submit_button:<br>            file_df = pd.read_excel(uploaded_file, sheet_name=selected_sheet , skiprows = int(selected_skip_lines) , header= None) <br>            file_df = file_df.fillna(&#39;NULL&#39;)<br>            <br>            data_list = list(file_df.values.tolist())<br>            data_str = get_data_str(data_list)<br>            <br>    <br>            sql_query = f&quot;&quot;&quot;insert into {target_table_name} values {data_str} &quot;&quot;&quot;<br>            print(sql_query)<br><br>            session.sql(sql_query).collect()<br>            st.success(&#39;Data Loaded SuccessFully !! Please check the data in the table&#39;, icon=&quot;✅&quot;)</pre><h4>3. Run the application!! You have successfully created the Streamlit application in Snowflake</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UTSmYGcKN7J1ib6N1XZ3lg.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7d2f67ae58c2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Easily Load Excel File To Snowflake using Streamlit]]></title>
            <link>https://medium.com/@imaishwary007/easily-load-excel-file-to-snowflake-using-streamlit-ab3ac5636950?source=rss-1dc6b645fb2------2</link>
            <guid isPermaLink="false">https://medium.com/p/ab3ac5636950</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[excel]]></category>
            <category><![CDATA[dataloader]]></category>
            <category><![CDATA[snowflake]]></category>
            <category><![CDATA[streamlit]]></category>
            <dc:creator><![CDATA[Aishwary Singh Chauhan]]></dc:creator>
            <pubDate>Tue, 21 May 2024 06:30:54 GMT</pubDate>
            <atom:updated>2024-05-25T02:32:22.849Z</atom:updated>
            <content:encoded><![CDATA[<p>In Snowflake we can use Snowsight or Classic Console to load files with structured data (e.g. CSV, TSV) or semi-structured data (e.g. JSON, Avro, ORC, Parquet, or XML).</p><p>However currently there is no native ingestion of excel files .</p><p>Wouldn’t it be great if we can directly load excel data as well using a simple User Interface ? Moreover we wouldn’t have to worry about file formats (field delimiter , encoding , row delimiter etc.) which we typically have to deal with while loading data using CSV ?</p><p>Well let’s have some fun. Here is an easy <strong>Streamlit</strong> UI based approach that can be used to load data from excel file to snowflake tables in 4 simple steps !!</p><blockquote>Code tested with below dependencies -</blockquote><blockquote>pandas==2.2.2<br>snowflake-connector-python==3.9.1<br>streamlit==1.33.0</blockquote><h3>1. Create creds.json and add the connection details</h3><pre>{<br>    &quot;account&quot;: &quot;&quot;,<br>    &quot;user&quot;: &quot;&quot;,<br>    &quot;role&quot;: &quot;&quot;,<br>    &quot;password&quot;: &quot;&quot;,<br>    &quot;warehouse&quot;: &quot;&quot;,<br>    &quot;database&quot;: &quot;&quot;,<br>    &quot;schema&quot;: &quot;&quot;<br>  }</pre><h3>2. Create utils.py</h3><pre>import snowflake.connector<br>import traceback<br><br>def get_data_str(data_list):<br>    try:<br>        data_str = &#39;&#39;<br>        for ind , row_list in enumerate(data_list) :<br>            for index, value in enumerate(row_list) :<br>                if index == 0:<br>                    data_str += &#39;(&#39;<br>                data_str += f&quot;&#39;{str(value)}&#39;&quot; if value!= &#39;NULL&#39; else value<br>                if index == len(row_list) - 1:<br>                    data_str += &#39;)&#39;<br>                else :<br>                    data_str += &#39;,&#39;<br>            if ind != len(data_list) - 1:<br>                data_str += &#39;,&#39;<br>        return data_str<br>    except Exception as e:<br>        print(f&#39;failed to get data... {traceback.print_exc}&#39;)<br>        raise(e)<br><br><br>def connect_to_sf(connection_parameters):<br>    try:<br>        con = snowflake.connector.connect(<br>                user=connection_parameters[&#39;user&#39;],<br>                password=connection_parameters[&#39;password&#39;],<br>                account=connection_parameters[&#39;account&#39;],<br>                warehouse=connection_parameters[&#39;warehouse&#39;],<br>                database=connection_parameters[&#39;database&#39;],<br>                schema=connection_parameters[&#39;schema&#39;],<br>                role = connection_parameters[&#39;role&#39;]<br>                )<br>        print(f&#39;connection to snowflake successful&#39;)        <br>        return con<br>    except Exception as e:<br>        print(f&#39;connection to snowflake failed with error ... {traceback.print_exc}&#39;)<br>        raise(e)</pre><p><strong>3. Create main.py and update the target_table_name variable with the table name to which you want to load data </strong>.</p><pre>import streamlit as st<br>import pandas as pd<br>import json<br>from utils import get_data_str,connect_to_sf<br>from datetime import datetime<br><br>target_table_name = &#39;TEST_EXCEL_LOAD&#39;<br><br># connect to Snowflake<br>with open(&#39;creds.json&#39;) as f:<br>    connection_parameters = json.load(f)  <br><br>st.header(&#39;:blue[Snowflake Excel Data Loader]&#39;)<br>uploaded_file = st.file_uploader(&quot;Drop your xlsx here&quot;, type={&quot;xlsx&quot;})<br><br>if uploaded_file is not None:<br>    xls = pd.ExcelFile(uploaded_file)<br>    sheet_names = xls.sheet_names<br><br>    with st.form(key=&#39;sheet_selector_form&#39;):<br>        selected_sheet = st.selectbox(&quot;Select a sheet to load data&quot;, sheet_names)<br>        selected_skip_lines = st.text_input(&quot;Number of lines to skip (including header if present)&quot;)<br>        submit_button = st.form_submit_button(label=&#39;Submit&#39;)<br><br>        if submit_button:<br>            file_df = pd.read_excel(uploaded_file, sheet_name=selected_sheet , skiprows = int(selected_skip_lines) , header= None) <br>            file_df = file_df.fillna(&#39;NULL&#39;)<br>            <br>            data_list = list(file_df.values.tolist())<br>            data_str = get_data_str(data_list)<br>            con =  connect_to_sf(connection_parameters)<br>            cur = con.cursor()<br>            sql_query = f&quot;&quot;&quot;insert into {target_table_name} values {data_str} &quot;&quot;&quot;<br>            print(sql_query)<br>            cur.execute(sql_query)<br>            st.success(&#39;Data Loaded SuccessFully !! Please check the data in the table&#39;, icon=&quot;✅&quot;)<br>            cur.close()<br>            con.close()</pre><h3>4. Run the streamlit application</h3><pre>streamlit run main.py</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h1COFIZF-N5PlKfSCckwhA.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ab3ac5636950" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[SNOWSIGHT — Dashboard Creation for Snowflake Monitoring Using Custom Warehouse Filter]]></title>
            <link>https://medium.com/@imaishwary007/snowsight-dashboard-creation-for-snowflake-monitoring-using-custom-warehouse-filter-a21ee6ade1c3?source=rss-1dc6b645fb2------2</link>
            <guid isPermaLink="false">https://medium.com/p/a21ee6ade1c3</guid>
            <dc:creator><![CDATA[Aishwary Singh Chauhan]]></dc:creator>
            <pubDate>Sun, 14 May 2023 09:04:55 GMT</pubDate>
            <atom:updated>2023-05-14T09:04:55.182Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>SNOWSIGHT — Dashboard Creation for Snowflake Monitoring Using Custom Warehouse Filter</strong></h3><h3><strong>Overview</strong></h3><blockquote><strong>Snowsight </strong>— The new Snowflake Web Interface, comes up with dashboarding capabilities. This post will guide the readers to create a Monitoring Dashboard that gives insights at an individual Warehouse level, with the option to choose the warehouse for insight from the custom filter in the dashboard.</blockquote><h3>Final Admin Dashboard With Warehouse Filter</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8QZ_OUvp4gs9wYSKx6B5SQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eueWBeJvY3r-NpwNGmNYpg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z-fO8wWX9497eU4zcdFu0Q.png" /></figure><h3>Setting up the Warehouse level Admin Dashboard on Snowflake</h3><blockquote>1. Go to Dashboards in Snowsight and click on + Dashboard</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0RzF-LnTkLgKiYBKl7xnug.png" /></figure><blockquote>2. Give the name for the Dashboard and Click on Create Dashboard</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/759/1*O-SpyuTzKW_CKb4f-PdDMQ.png" /></figure><blockquote>3. Click on filter icon at the top left</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/130/1*pWVcWJTsGwO9Vz8PfUeSXg.png" /></figure><blockquote>4. Two system-created filters (: datebucket and:daterange are already available ). Select Create new filter button to create our custom warehouse filter</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GJBXBZKwovKOfGn3aH7bEQ.png" /></figure><blockquote>5. Configure the details as below and leave other fields as default. Click on Edit query to write the query for the custom warehouse filter</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/376/1*Qpxazpya5n8o0EFKjJmVFw.png" /></figure><blockquote>7. Write and execute the query to get the list of Warehouse names for the custom filter.</blockquote><pre>select distinct warehouse_name from snowflake.account_usage.warehouse_metering_history; </pre><blockquote>8. Click on Done, and the custom warehouse filter is created, and we are ready to create tiles for the dashboard.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/969/1*jg_3u0KPZBBH74rrKcZjyA.png" /></figure><blockquote>9. Add the below queries in each tile to be added in the Dashboard</blockquote><p><strong>Credit Used by Warehouse</strong></p><pre>select<br>    sum(credits_used)<br>from<br>    snowflake.account_usage.metering_history<br>where<br>    start_time = :daterange<br>    and name = :filter_warehouse<br>    ;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/896/1*6wCFgh5p5NDii7ZownySUQ.png" /></figure><p><strong>Total Number of Jobs Executed on Warehouse</strong></p><pre>select<br>    count(*) as number_of_jobs<br>from<br>    snowflake.account_usage.query_history<br>where<br>    start_time =:daterange<br>    and warehouse_name = :filter_warehouse;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/783/1*vIjHa3vTCFq2Kr_39M0bUA.png" /></figure><p><strong>Warehouse Usage Greater than 7-Day Average</strong></p><pre>SELECT<br>    WAREHOUSE_NAME,<br>    DATE(START_TIME) AS DATE,<br>    SUM(CREDITS_USED) AS CREDITS_USED,<br>    AVG(SUM(CREDITS_USED)) OVER (<br>        PARTITION BY WAREHOUSE_NAME<br>        ORDER BY<br>            DATE ROWS 7 PRECEDING<br>    ) AS CREDITS_USED_7_DAY_AVG,<br>    (TO_NUMERIC(SUM(CREDITS_USED)/CREDITS_USED_7_DAY_AVG*100,10,2)-100)::STRING || &#39;%&#39; AS VARIANCE_TO_7_DAY_AVERAGE<br>FROM<br>    &quot;SNOWFLAKE&quot;.&quot;ACCOUNT_USAGE&quot;.&quot;WAREHOUSE_METERING_HISTORY&quot;<br>WHERE<br>    START_TIME = :daterange<br>    and warehouse_name = :filter_warehouse<br>GROUP BY<br>    DATE,<br>    WAREHOUSE_NAME<br>ORDER BY<br>    DATE DESC;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2h8-bbNRQlO7cb4zJk4gDw.png" /></figure><p><strong>Execution Time by Query Type (Avg Seconds)</strong></p><pre>select<br>    query_type,<br>    warehouse_size,<br>    avg(execution_time) / 1000 as average_execution_time<br>from<br>    snowflake.account_usage.query_history<br>where<br>    start_time = :daterange<br>    and warehouse_name =:filter_warehouse<br>group by<br>    1,<br>    2<br>order by<br>    3 desc;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bljNoZb2dMmO2sleUF59Fg.png" /></figure><p><strong>Top 25 Longest Queries</strong></p><pre>select<br>    query_id,<br>    query_text,<br>    (execution_time / 60000) as exec_time<br>from<br>    snowflake.account_usage.query_history<br>where<br>    execution_status = &#39;SUCCESS&#39;<br>    and warehouse_name = :filter_warehouse<br>order by<br>    execution_time desc<br>limit<br>    25</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DmYRJ4izj5rxuGtLSsO4HQ.png" /></figure><p><strong>Total Execution Time by Repeated Queries</strong></p><pre>select<br>    query_text,<br>    (sum(execution_time) / 60000) as exec_time<br>from<br>    account_usage.query_history<br>where<br>    execution_status = &#39;SUCCESS&#39;<br>    and warehouse_name =:filter_warehouse<br>group by<br>    query_text<br>order by<br>    exec_time desc<br>limit<br>    25;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LTIR7AaE1fJX95shrq8SpQ.png" /></figure><p><strong>Credits Billed by Month</strong></p><pre><br>select to_char(start_time,&#39;YYYY-MM&#39;) as Usage_Month, Warehouse_name , sum(credits_used)<br>    from snowflake.account_usage.warehouse_metering_history <br>    where warehouse_name =:filter_warehouse<br>     group by 1,2 order by 3 desc ;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bj69jvt9n6r737Fnn06yuQ.png" /></figure><p><strong>Average Query Execution Time (By User)</strong></p><pre>select<br>    user_name,<br>    (avg(execution_time)) / 1000 as average_execution_time<br>from<br>    account_usage.query_history<br>where<br>    start_time = :daterange<br>    and warehouse_name =:filter_warehouse<br>group by<br>    1<br>order by<br>    2 desc;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WpjUuzNAphXksYSlQa7vwQ.png" /></figure><p><strong>Total number of queries executed by the user</strong></p><pre>select<br>    user_name,<br>    count(*) as query_count<br>from<br>    snowflake.account_usage.query_history<br>where<br>    start_time = :daterange<br>    and warehouse_name =:filter_warehouse<br>group by<br>    1<br>order by<br>    2 desc;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FPf2EPRhQ5u0uzd2DRVfiA.png" /></figure><p><strong>Cloud Service Utilization by Query Type (Top 10)</strong></p><pre>select<br>    query_type,<br>    sum(credits_used_cloud_services) cs_credits,<br>    count(1) num_queries<br>from<br>    snowflake.account_usage.query_history<br>where<br>    true<br>    and start_time = :daterange<br>    and warehouse_name =:filter_warehouse<br>group by<br>    1<br>order by<br>    2 desc<br>limit<br>    10;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*__b5Noz43zquyB57UO4q-w.png" /></figure><p><strong>Compute and Cloud Services by Warehouse</strong></p><pre>select<br>    warehouse_name,<br>    sum(credits_used_cloud_services) credits_used_cloud_services,<br>    sum(credits_used_compute) credits_used_compute,<br>    sum(credits_used) credits_used<br>from<br>    snowflake.account_usage.warehouse_metering_history<br>where<br>    true<br>    and start_time = :daterange<br>    and warehouse_name =:filter_warehouse<br>group by<br>    1<br>order by<br>    2 desc<br>limit<br>    10;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CHONzFhMHYzZOvcPZvD1TQ.png" /></figure><h3>References</h3><p><a href="https://medium.com/snowflake/monitoring-snowflake-with-snowsight-e9990a2898f1">https://medium.com/snowflake/monitoring-snowflake-with-snowsight-e9990a2898f1</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a21ee6ade1c3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Read File from S3 as Pandas Dataframe with just two lines of code !!]]></title>
            <link>https://medium.com/@imaishwary007/read-file-from-s3-as-pandas-dataframe-with-just-two-lines-of-code-3669c1119b7?source=rss-1dc6b645fb2------2</link>
            <guid isPermaLink="false">https://medium.com/p/3669c1119b7</guid>
            <category><![CDATA[pandas]]></category>
            <category><![CDATA[s3]]></category>
            <category><![CDATA[pandas-dataframe]]></category>
            <category><![CDATA[s3-bucket]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Aishwary Singh Chauhan]]></dc:creator>
            <pubDate>Mon, 21 Nov 2022 12:46:00 GMT</pubDate>
            <atom:updated>2022-11-21T12:46:00.014Z</atom:updated>
            <content:encoded><![CDATA[<h3>Read File from S3 as Pandas Dataframe with just two lines of code !!</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/241/1*2x6pxOnufErjSPkkRVSNJQ.png" /></figure><p>If you enjoy working with Pandas Dataframes and want to read large datafiles stored on S3 , this story would be what all you need to get started directly with your work without having to deal with much efforts .</p><h3>What we would be using ?</h3><h4>Python Smart Open Library</h4><p><strong>smart_open</strong> is well-tested, well-documented, and has a simple Pythonic API .</p><p>smart_open is a Python 3 library for efficient streaming of very large files from/to storages such as S3, GCS, Azure Blob Storage, HDFS, WebHDFS, HTTP, HTTPS, SFTP, or local filesystem. It supports transparent, on-the-fly (de-)compression for a variety of different formats.</p><p>Working with large remote files, for example using Amazon’s <a href="https://boto3.amazonaws.com/v1/documentation/api/latest/index.html">boto3</a> Python library, is a pain. boto3’s Object.upload_fileobj() and Object.download_fileobj() methods require gotcha-prone boilerplate to use successfully, such as constructing file-like object wrappers. smart_open shields you from that. It builds on boto3 and other remote storage libraries, but offers a clean unified Pythonic API. The result is less code for you to write and fewer bugs to make.</p><p>Reference — <a href="https://pypi.org/project/smart-open/">https://pypi.org/project/smart-open/</a></p><h4>Install smart_open Library</h4><pre>pip install smart_open</pre><h4><strong>Sample code to read a pkl file from s3 as pandas Dataframe</strong></h4><pre>import pandas as pd<br>from smart_open import open<br><br>with open(&#39;s3://path_to_file.pkl&#39;,&#39;rb&#39;) as f:<br>  df_s3_pkl = pd.read_pickle(f)</pre><p>Voila ! That&#39;s all you have to do . You got the file from S3 location as a pandas dataframe in df_s3_pkl .</p><p>Include these two lines of your code and carry on your work by utilizing the power of pandas dataframe !!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3669c1119b7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Python Script to keep your Laptop screen always On]]></title>
            <link>https://medium.com/@imaishwary007/python-script-to-keep-your-laptop-screen-always-on-c8c5cb675a89?source=rss-1dc6b645fb2------2</link>
            <guid isPermaLink="false">https://medium.com/p/c8c5cb675a89</guid>
            <category><![CDATA[automation]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[python3]]></category>
            <category><![CDATA[sleep]]></category>
            <category><![CDATA[awake]]></category>
            <dc:creator><![CDATA[Aishwary Singh Chauhan]]></dc:creator>
            <pubDate>Thu, 17 Nov 2022 03:19:09 GMT</pubDate>
            <atom:updated>2022-11-17T03:19:09.420Z</atom:updated>
            <content:encoded><![CDATA[<p>This is a simple Python script that will enable you be online and keep your screen always on and stop it from going to sleep .</p><p>Without spending much time , let’s start directly with the code !!</p><p><strong><em>1.) Importing the Required Modules</em></strong></p><pre>import time, pyautogui<br>import PySimpleGUI as sg<br>import multiprocessing</pre><p><strong><em>2.) Define KeepUI () function</em></strong></p><pre>def KeepUI():<br>    <br>    sg.theme(&#39;Dark&#39;)<br>    layout = [<br>        [sg.Text(&#39;Keep-Me-Up is now running.\nYou can keep it minised, and it will continue running.\nClose it to disable it.&#39;)]<br>    ]<br>    window = sg.Window(&#39;Keep-Me-Up&#39;, layout)<br>    <br>    p2 = multiprocessing.Process(target = dontsleep)<br>    p2.start()<br>    <br>    while True:<br>        try:<br>            event, values = window.read()<br>            if event == sg.WIN_CLOSED: # if user closes window or clicks cancel<br>                if p2.is_alive(): <br>                    p2.terminate()<br>                break<br>        except :<br>            time.sleep(30)</pre><p><strong><em>3.) Define dontsleep() function</em></strong></p><pre>def dontsleep():<br>    while True:<br>        try :<br>            pyautogui.press(&#39;volumedown&#39;)<br>            time.sleep(1)<br>            pyautogui.press(&#39;volumeup&#39;)<br>            time.sleep(30)<br>        except :<br>            time.sleep(30)</pre><p><strong><em>4.) Finally Call KeepUI() function</em></strong></p><pre>if __name__ == &#39;__main__&#39;:<br>    p1 = multiprocessing.Process(target = KeepUI)<br>    p1.start()</pre><p><strong><em>5.) Save the file and run</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/955/1*LNF5qxV7YyIXNaVfKEwA3A.jpeg" /><figcaption>save file as keep_awake.py and run through terminal</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/704/1*dto7R56xLcB1tIOnTp8lGQ.png" /><figcaption>A dialog appears confirming the script execution started</figcaption></figure><h3>Congratulations !! Just this , you are done with the set up :)</h3><p>You will see Volume Up / Down in screen after every 30 seconds .</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/225/1*ns0Q-RdNP9vmne8xPyNJGA.png" /></figure><blockquote><strong>Relax and Let Python handle the rest .</strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c8c5cb675a89" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>