<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Avril Aysha on Medium]]></title>
        <description><![CDATA[Stories by Avril Aysha on Medium]]></description>
        <link>https://medium.com/@avrilaysha?source=rss-b016782a8709------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*pgiPiqFaPTnCHGpHL61bqg.png</url>
            <title>Stories by Avril Aysha on Medium</title>
            <link>https://medium.com/@avrilaysha?source=rss-b016782a8709------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 16:06:00 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@avrilaysha/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Hello world — meet Avril Aysha ]]></title>
            <link>https://avrilaysha.medium.com/hello-world-meet-avril-aysha-0f556b903683?source=rss-b016782a8709------2</link>
            <guid isPermaLink="false">https://medium.com/p/0f556b903683</guid>
            <dc:creator><![CDATA[Avril Aysha]]></dc:creator>
            <pubDate>Wed, 31 Jan 2024 09:59:56 GMT</pubDate>
            <atom:updated>2024-01-31T09:59:56.989Z</atom:updated>
            <content:encoded><![CDATA[<h3>Hello world — meet Avril Aysha💃 🥳</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AMDxl3i-hVOk-njEsAaSlg.jpeg" /></figure><p>Exciting news! I am changing my name and pronouns. You can call me <strong>Avril Aysha</strong> and <strong>she/her</strong> from now on 💃 🥳</p><p>Practicalities first, then cake and prosecco later:</p><p>This means that <strong>I’ll be changing my URL and handle here on Medium</strong>. If you have bookmarked any of my posts, <strong>these links will likely break</strong>. You will need to redirect them to `avrilaysha.medium.com/&lt;blog-title&gt;`.</p><p>Sorry but not so sorry for the inconvenience! 🤷‍♀️</p><p>Now for the cake and prosecco:</p><p><a href="https://www.linkedin.com/posts/avrilaysha_hello-world-meet-avril-aysha-sheher-activity-7158164533478404096-ky53?utm_source=share&amp;utm_medium=member_desktop">Avril Aysha on LinkedIn: Hello world, meet Avril Aysha (she/her)! 💃 🥳 Today is my (re)birthday!... | 13 comments</a></p><p>Today I’m stepping into a brighter, happier, kinder, and more confident form of myself that’s been a few years in the making. It’s a super exciting day and I would love to virtually celebrate it with you all here :)</p><p>I am transforming.</p><p>We all are, all the time.</p><p>Stable identities are a cozy illusion: comfortable but imaginary nonetheless.</p><p>We are all in continuous growth and evolution and our potential is limitless 🚀</p><p>I’m still the same cheeky, geeky human on the inside (OK, maybe a bit more cheeky 💁‍♀️). And I will keep sharing my journey as I continue exploring the beautiful binary world of data and code…and the beyond-binary world of delightful gender expression and freedom 🤩</p><p>A big, BIG thank you to my family, friends and colleagues who have supported me throughout this journey. I wouldn’t be here without you all!</p><p>☀️ The future is bright, folx! Be whoever the f-score you want to be ;)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*pgiPiqFaPTnCHGpHL61bqg.png" /></figure><p>Find Avril Aysha on:</p><p>LinkedIn: <a href="https://www.linkedin.com/in/avrilaysha/">https://www.linkedin.com/in/avrilaysha/</a></p><p>Github: <a href="https://github.com/avriiil">https://github.com/avriiil</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0f556b903683" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Real-time data streaming in Jupyter]]></title>
            <link>https://python.plainenglish.io/real-time-data-streaming-in-jupyter-81372dfaa790?source=rss-b016782a8709------2</link>
            <guid isPermaLink="false">https://medium.com/p/81372dfaa790</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[real-time-analytics]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[jupyter-notebook]]></category>
            <dc:creator><![CDATA[Avril Aysha]]></dc:creator>
            <pubDate>Wed, 17 Jan 2024 04:16:16 GMT</pubDate>
            <atom:updated>2024-02-06T14:01:37.773Z</atom:updated>
            <content:encoded><![CDATA[<h4>Build dashboards for live data streams in Jupyter Notebooks</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HAWD_atBKkQ7pyBEbzooJg.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@chrisliverani?utm_content=creditCopyText&amp;utm_medium=referral&amp;utm_source=unsplash">Chris Liverani</a> on <a href="https://unsplash.com/photos/turned-on-flat-screen-monitor-dBI_My696Rk?utm_content=creditCopyText&amp;utm_medium=referral&amp;utm_source=unsplash">Unsplash</a></figcaption></figure><h3>Jupyter notebooks for streaming data</h3><p><a href="https://jupyter.org/">Jupyter</a> notebooks are a great tool for working with data. They give you a friendly interactive interface in which you can process data comfortably and visualise it in intuitive, human-readable ways. It’s no surprise that the tool is extremely popular among data professionals, with <a href="https://analyticsindiamag.com/why-jupyter-notebooks-are-so-popular-among-data-scientists/">millions</a> of public notebooks shared on Github every month.</p><p>Most data analysts use Jupyter notebooks with static (or “batch”) workflows. Jupyter has historically been less popular for streaming workflows. Using Jupyter with realtime data streams can feel intimidating or have mixed results due to inadequate or overly complicated tooling.</p><p>This article aims to change that. By working through this tutorial you will learn how to visualise a live data stream in real-time from the familiar context of a Jupyter Notebook. You will use <a href="https://bokeh.org/">Bokeh</a>, <a href="https://panel.holoviz.org/">Panel</a> and <a href="https://pathway.com/">Pathway</a> to build a real-time data visualisation dashboard that will alert you when the data hits a critical threshold. You can follow along on this page or run the code for yourself <a href="https://colab.research.google.com/github/pathwaycom/pathway-examples/blob/main/showcases/live-data-jupyter.ipynb">in Colab</a> or <a href="https://github.com/pathwaycom/pathway-examples/blob/main/showcases/live-data-jupyter.ipynb">from Github</a>.</p><p>Let’s jump in! 🪂</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CItV2kKyXpVe8mkm9mApAg.gif" /></figure><h3>What we’ll do 👩🏽‍💻</h3><p>In this tutorial, you will take a stream of financial data and implement a simple trading algorithm using <a href="https://en.wikipedia.org/wiki/Bollinger_Bands">Bollinger Bands</a>. This will be a helpful example of visualising streaming data and will also show the usefulness of getting critical alerts in real-time.</p><p>You don’t need to fully understand the algorithm or the financial terminology. Focus on the steps you will do to take a live data stream, perform some computation on it in real-time, and output a live visualization dashboard in your Jupyter Notebook that updates on-the-fly. <strong>These steps are generalizable to any data streaming use case.</strong></p><p>Concretely, we will compute the 1-minute running mean of Volume Weighted Average Price (vwap) and the 20-minute volatility, the Volume Weighted Standard Deviation (vwstd) on the price time series. This creates two bands around the mean price, with most price movements happening between the bands. Intuitively, when a price approaches the upper band, it is abnormally high and may likely drop — it is a good moment to <em>SELL</em> and we’ll raise an alert. Likewise, when the price approaches the lower band, it indicates it is low and may grow to revert to the mean — we’ll raise an alert that it’s a good moment to <em>BUY</em>. For further reliability, the BUY/SELL actions are performed only when there is a significant volume of trades, indicating that the outlying price is not a one-off event.</p><p>First, import the necessary libraries:</p><ul><li>pathway for data processing,</li><li>datetime for date manipulation,</li><li>bokehand panel for dashboarding.</li></ul><pre>import datetime<br><br>import bokeh.models<br>import bokeh.plotting<br>import panel<br><br>import pathway as pw</pre><p>And then fetch the sample data from Github. This data was generated using <a href="https://polygon.io/">polygon.io</a>.</p><pre>!wget -nc https://gist.githubusercontent.com/janchorowski/e351af72ecd8d206a34763a428826ab7/raw/ticker.csv</pre><h3>Data source setup 🛠</h3><p>Create a streaming data source that replays data in a CSV file. This is an easy way to simulate a live data stream without any infrastructure hassle. You can of course also use Pathway with a real, production-grade data stream, for example <a href="https://pathway.com/developers/user-guide/from-jupyter-to-deploy/#kafka-integration">from Kafka</a> or <a href="https://pathway.com/developers/tutorials/connectors/switching-to-redpanda/">Redpanda</a>.</p><p>The input_rate parameter controls how fast the data is replayed.</p><p><em>💡 No data processing actually happens when you run this cell. We are building a computational graph that will only be executed at the end of the tutorial. This allows the streaming engine to optimise the computations and perform them as fast as possible when the data starts streaming.</em></p><pre>fname = &quot;ticker.csv&quot;<br>schema = pw.schema_from_csv(fname)<br>data = pw.demo.replay_csv(fname, schema=schema, input_rate=1000)<br><br># # For static data exploration use<br># data = pw.io.csv.read(fname, schema=schema, mode=&quot;static&quot;)<br><br># Parse the timestamps<br>data = data.with_columns(<br>    t=pw.apply_with_type(<br>        datetime.datetime.fromtimestamp, pw.DATE_TIME_NAIVE, data.t / 1000.0<br>    )<br>)</pre><p>As you probably noticed, the code block above also includes a commented-out section. You can use this line instead of data = pw.demo.replay_csv(…) to test the workflow with static data. <strong>This is the only change you need to make in your code to </strong><a href="https://pathway.com/developers/user-guide/getting-started/switch-from-batch-to-streaming/"><strong>switch between static and stream processing</strong></a><strong>.</strong></p><h3>Let’s Trade! 💸</h3><p>Now it’s time to build your trading algorithm. There is no need to fully understand the terminology or the math here. What’s most important to grasp is how you are taking a stream of data and performing a windowing transformation to get more analytical value out of the raw data.</p><h4>Create the 20-minute Bollinger Band</h4><p>Start by creating the first of our two Bollinger Bands: the 20-minute volatility measured as the Volume Weighted Standard Deviation. Use a sliding window to compute at every minute the volume weighted price mean and standard deviation aggregate on the past 20 minutes of data. The behavior option tells Pathway that the window should emit the statistics only when it is finished — we do not want to see incomplete results.</p><p>To compute the standard deviation, use the identity:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*HXdHQCfoteAaq9lDRl7p2g.png" /></figure><p>which is easily expressible using Pathway reducers: we first compute the total volume, price, and price2. We then post-process them to obtain the mean (vwap), standard deviation (vwstd), and Bollinger Bands places at vwap±2⋅vwstd.</p><p><strong>Or in simpler terms: the code block below takes your incoming data stream and calculates important statistics in real-time. These statistics are continually updated as the data comes in so that you can identify critical moments as they happen.</strong></p><pre>minute_20_stats = (<br>    data.windowby(<br>        pw.this.t,<br>        window=pw.temporal.sliding(<br>            hop=datetime.timedelta(minutes=1), duration=datetime.timedelta(minutes=20)<br>        ),<br>        behavior=pw.temporal.exactly_once_behavior(),<br>        instance=pw.this.ticker,<br>    )<br>    .reduce(<br>        ticker=pw.this._pw_instance,<br>        t=pw.this._pw_window_end,<br>        volume=pw.reducers.sum(pw.this.volume),<br>        transact_total=pw.reducers.sum(pw.this.volume * pw.this.vwap),<br>        transact_total2=pw.reducers.sum(pw.this.volume * pw.this.vwap**2),<br>    )<br>    .with_columns(vwap=pw.this.transact_total / pw.this.volume)<br>    .with_columns(<br>        vwstd=(pw.this.transact_total2 / pw.this.volume - pw.this.vwap**2) ** 0.5<br>    )<br>    .with_columns(<br>        bollinger_upper=pw.this.vwap + 2 * pw.this.vwstd,<br>        bollinger_lower=pw.this.vwap - 2 * pw.this.vwstd,<br>    )<br>)</pre><h4>Create the 1-minute Bollinger Band</h4><p>Now it’s time to compute the second Bollinger Bands: the 1-minute running mean. You will need to compute the mean price over the last minute of trades.</p><p>The code is analogous to the 20-minute statistics but simpler: you can use a tumbling window and don’t have to compute the standard deviation.</p><pre>minute_1_stats = (<br>    data.windowby(<br>        pw.this.t,<br>        window=pw.temporal.tumbling(datetime.timedelta(minutes=1)),<br>        behavior=pw.temporal.exactly_once_behavior(),<br>        instance=pw.this.ticker,<br>    )<br>    .reduce(<br>        ticker=pw.this._pw_instance,<br>        t=pw.this._pw_window_end,<br>        volume=pw.reducers.sum(pw.this.volume),<br>        transact_total=pw.reducers.sum(pw.this.volume * pw.this.vwap),<br>    )<br>    .with_columns(vwap=pw.this.transact_total / pw.this.volume)<br>)</pre><h4>Joining the Bollinger Bands</h4><p>Now join the 20-minute and 1-minute statistics, gathering all the information needed for alerting in one place. Alert triggering is now a breeze.</p><pre>joint_stats = (<br>    minute_1_stats.join(<br>        minute_20_stats, pw.left.t == pw.right.t, pw.left.ticker == pw.right.ticker<br>    )<br>    .select(<br>        *pw.left,<br>        bollinger_lower=pw.right.bollinger_lower,<br>        bollinger_upper=pw.right.bollinger_upper,<br>    )<br>    .with_columns(<br>        is_alert=(pw.this.volume &gt; 10000)<br>        &amp; (<br>            (pw.this.vwap &gt; pw.this.bollinger_upper)<br>            | (pw.this.vwap &lt; pw.this.bollinger_lower)<br>        )<br>    )<br>    .with_columns(<br>        action=pw.if_else(<br>            pw.this.is_alert,<br>            pw.if_else(pw.this.vwap &gt; pw.this.bollinger_upper, &quot;sell&quot;, &quot;buy&quot;),<br>            &quot;hodl&quot;,<br>        )<br>    )<br>)<br>alerts = joint_stats.filter(pw.this.is_alert)</pre><h3>Visualize your real-time data stream 🔮</h3><p>Now let’s create a Bokeh plot and panel table visualization: the plot shows the Bollinger Bands along with the price running mean and indicates the price of buy and sell decisions. The table gathers all the decisions conveniently for further processing, such as reducing it to compute a historical evaluation of the gains of the strategy.</p><p>When the cell is executed, placeholder containers are created for the plot and table dashboard. They will be populated with live data when the computation is started (i.e. when running the final pw.run() at the end of this tutorial).</p><pre>def stats_plotter(src):<br>    actions = [&quot;buy&quot;, &quot;sell&quot;, &quot;hodl&quot;]<br>    color_map = bokeh.models.CategoricalColorMapper(<br>        factors=actions, palette=(&quot;#00ff00&quot;, &quot;#ff0000&quot;, &quot;#00000000&quot;)<br>    )<br><br>    fig = bokeh.plotting.figure(<br>        height=400,<br>        width=600,<br>        title=&quot;20 minutes Bollinger bands with last 1 minute average&quot;,<br>        x_axis_type=&quot;datetime&quot;,<br>        y_range=(188.5, 191),<br>    )<br>    fig.line(&quot;t&quot;, &quot;vwap&quot;, source=src)<br>    band = bokeh.models.Band(<br>        base=&quot;t&quot;,<br>        lower=&quot;bollinger_lower&quot;,<br>        upper=&quot;bollinger_upper&quot;,<br>        source=src,<br>        fill_alpha=0.3,<br>        fill_color=&quot;gray&quot;,<br>        line_color=&quot;black&quot;,<br>    )<br><br>    fig.scatter(<br>        &quot;t&quot;,<br>        &quot;vwap&quot;,<br>        color={&quot;field&quot;: &quot;action&quot;, &quot;transform&quot;: color_map},<br>        size=10,<br>        marker=&quot;circle&quot;,<br>        source=src,<br>    )<br><br>    fig.add_layout(band)<br>    return fig<br><br><br>viz = panel.Row(<br>    joint_stats.plot(stats_plotter, sorting_col=&quot;t&quot;),<br>    alerts.select(pw.this.ticker, pw.this.t, pw.this.vwap, pw.this.action).show(<br>        include_id=False, sorters=[{&quot;field&quot;: &quot;t&quot;, &quot;dir&quot;: &quot;desc&quot;}]<br>    ),<br>)<br>viz</pre><h3>Run your pipeline in real-time 👨🏻‍🔧</h3><p>All the hard work is done! The final step is to start the Pathway data processing engine using the command:</p><pre>pw.run()</pre><p>Watch how the dashboard is updated in realtime. The Bollinger Bands action trigger seems to be working — the green buy decision markers are frequently followed by the red sell markers at a slightly higher price.</p><p>While the computation is running, pathway prints important statistics such as message processing latency.</p><p>Successful evaluation of the code should result in the animation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CItV2kKyXpVe8mkm9mApAg.gif" /><figcaption>animation by author</figcaption></figure><h3>Jupyter Notebooks for Streaming Data</h3><p>Congratulations! You have successfully built a live data streaming pipeline with useful data visualisations and real-time alerts, right from a Jupyter notebook 😄</p><p>This is just a taste of what is possible. If you’re interested in diving deeper and building a production-grade data science pipeline all the way from data exploration to deployment, you may want to check out the full-length <a href="https://pathway.com/developers/user-guide/from-jupyter-to-deploy/">From Experimenting in Jupyter to Production</a> tutorial.</p><p><a href="https://www.linkedin.com/in/avrilaysha/">Follow me on LinkedIn</a> for regular data hacks and content 🚀</p><p>Happy streaming! 🦦</p><p><em>Originally published at </em><a href="https://pathway.com/developers/showcases/live_data_jupyter/"><em>https://pathway.com</em></a><em>.</em></p><h3>In Plain English 🚀</h3><p><em>Thank you for being a part of the </em><a href="https://plainenglish.io/"><strong><em>In Plain English</em></strong></a><em> community! Before you go:</em></p><ul><li>Be sure to <strong>clap</strong> and <strong>follow</strong> the writer ️👏<strong>️️</strong></li><li>Follow us: <a href="https://twitter.com/inPlainEngHQ"><strong>X</strong></a><strong> | </strong><a href="https://www.linkedin.com/company/inplainenglish/"><strong>LinkedIn</strong></a><strong> | </strong><a href="https://www.youtube.com/channel/UCtipWUghju290NWcn8jhyAw"><strong>YouTube</strong></a><strong> | </strong><a href="https://discord.gg/in-plain-english-709094664682340443"><strong>Discord</strong></a><strong> | </strong><a href="https://newsletter.plainenglish.io/"><strong>Newsletter</strong></a></li><li>Visit our other platforms: <a href="https://stackademic.com/"><strong>Stackademic</strong></a><strong> | </strong><a href="https://cofeed.app/"><strong>CoFeed</strong></a><strong> | </strong><a href="https://venturemagazine.net/"><strong>Venture</strong></a></li><li>More content at <a href="https://plainenglish.io/"><strong>PlainEnglish.io</strong></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=81372dfaa790" width="1" height="1" alt=""><hr><p><a href="https://python.plainenglish.io/real-time-data-streaming-in-jupyter-81372dfaa790">Real-time data streaming in Jupyter</a> was originally published in <a href="https://python.plainenglish.io/">Python in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Sliding Windows in Pandas]]></title>
            <link>https://medium.com/data-science/sliding-windows-in-pandas-40b79edefa34?source=rss-b016782a8709------2</link>
            <guid isPermaLink="false">https://medium.com/p/40b79edefa34</guid>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[pandas]]></category>
            <dc:creator><![CDATA[Avril Aysha]]></dc:creator>
            <pubDate>Wed, 29 Mar 2023 18:39:51 GMT</pubDate>
            <atom:updated>2024-02-06T14:00:50.408Z</atom:updated>
            <content:encoded><![CDATA[<h4>Identify Patterns in Time-Series Data with Overlapping Window Techniques</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NOUaYUNUJpqB0o91y0eHnQ.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/it/@eugenechystiakov?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Eugene Chystiakov</a> on <a href="https://unsplash.com/photos/B-h3so_5UKA?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>Windowing techniques enable data analysts to identify valuable patterns in time-series data. Sliding windows are particularly powerful because they allow you to spot patterns earlier than other techniques. This is an important feature in situations when making a key decision a few minutes (or seconds) earlier can save you money.</p><p>This article will show you 3 ways to perform windowing in pandas and will discuss the tradeoffs and benefits of each approach. By the end, you’ll be able to create a sliding window visualisation that looks something like this:</p><figure><img alt="line plot showing zones with peak demand in NYC rideshare data" src="https://cdn-images-1.medium.com/max/1024/1*ZxhJvyZ9sN-_oTcgGRijCA.png" /><figcaption>image by author</figcaption></figure><p>Imagine you’re an Uber or Lyft driver. You’re driving around New York waiting for your next ride…and burning expensive gas in the meantime. Not cool. You need to find a way to maximise your chance of getting a new ride quickly. How could you do this?</p><p>You could take an experience-driven approach and test out different strategies over a period of time to find out which parts of New York have high demand. You try starting in one neighbourhood the first day, another the second, and so on… But New York City is BIG and this way it’s going to take you a <em>loooong</em> time before you can start to identify patterns with any degree of confidence.</p><p>It would be much more efficient to take <strong>a data-driven approach</strong>. Which is exactly what we’ll be doing in this article.</p><p>Let’s get to it 🚀</p><h4>The Dataset</h4><p>We’ll be working with 1 hour of rideshare (Uber/Lyft) data from the New York City Taxi and Limousine Commission (TLC) Trip Record* dataset, which is subsampled from the dataset for January 2022. You can <a href="https://d37ci6vzurychx.cloudfront.net/trip-data/fhvhv_tripdata_2022-01.parquet">download the complete Parquet file </a>for January 2022. This contains data for more than 14.5 million rides (!) To replicate the dataset I’m using in this article, subsample to the first 100K rows. You can also <a href="https://github.com/rrpelgrim/sliding-windows-pandas">access the data and notebook directly</a> from a dedicated Github repo.</p><pre>df = pd.read_parquet(&quot;fhvhv_tripdata_2022-01.parquet&quot;).head(100_000)</pre><p>Remember: our objective is to <strong>identify zones with peaks in demand, defined as number of requests per unit of time.</strong></p><h3>1. Quick-n-Dirty: a GroupBy over the Entire Dataset</h3><figure><img alt="diagram showing a line representing the dataset and a box representing the groupby operation" src="https://cdn-images-1.medium.com/max/922/1*7FCuWRtLzyi-rUJvpoHquQ.png" /><figcaption>image by author</figcaption></figure><p>We’ll start by doing a very rough pass to identify high-demand zones over the entire 1-hour dataset.</p><p>After using a time_mask to subset the data for a specific hour and sorting the values by the request_datetime column, this is essentially just a groupby operation.</p><pre># define time mask<br>time_mask = (df[&#39;request_datetime&#39;].dt.month == 1) &amp; \<br>            (df[&#39;request_datetime&#39;].dt.day == 1) &amp; \<br>            (df[&#39;request_datetime&#39;].dt.hour == 0) <br><br># sort values<br>one_hour = df[time_mask].sort_values(by=[&#39;request_datetime&#39;])<br><br># perform a simple groupby with .count()<br>high_demand = one_hour[[&#39;PULocationID&#39;,&#39;hvfhs_license_num&#39;]].groupby([&#39;PULocationID&#39;]).count()</pre><p>We now have a list of the number of requests per pickup location. Let’s clean up the column names and sort the values to get a meaningful output.</p><pre>high_demand = high_demand.rename(columns={&#39;hvfhs_license_num&#39;:&#39;n_requests&#39;})<br>high_demand.sort_values(by=&#39;n_requests&#39;, ascending=False, inplace=True)<br>high_demand.head(10)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/234/1*ccvXmUBYIBfTWHQpUJwnzA.png" /><figcaption>data output generated by author from [this notebook]</figcaption></figure><p>Sweet. We can now identify the top-10 pick-up zones with highest demand during the hour for which we have data. We could run this code in batch once a hour. This is great…but only sort of.</p><p>Processing the data in 1-hour batches means that drivers will only receive the signal for a high-demand area <em>at the end of the hour. </em>What if the peak was actually happening during the first 15 or even 5 minutes of that hour? Then there would be absolutely no use in rushing to that part of town only to realise that the peak demand was long over.</p><h3>2. Tumbling Windows in Pandas</h3><figure><img alt="diagram showing a line representing the dataset and 4 boxes representing the 4 tumbling windows, one for each 15-minute window" src="https://cdn-images-1.medium.com/max/932/1*Yb4LlLkMXtdo1UP5D4CzkQ.png" /><figcaption>image by author</figcaption></figure><p>The drivers aren’t happy. We need to do better. Let’s build some code that can chop up our hour of data into <a href="https://pathway.com/glossary/tumbling-window">tumbling windows</a> of 15 minutes each. This way we will get some more granularity and be able to give the drivers more helpful instructions on when to be in which part of town.</p><p>We can use the pandas pivot_table() and .resample() methods to do this. We’ll need the index of our DataFrame to be set to a datetime column. Since we’re interested in the number of requests, we’ll use the request_datetime column.</p><p>Note that the dataset is not sorted correctly by incoming requests and includes some trips that were requested before midnight on the year before. So let’s clean those up first.</p><pre># subset df to requests from 2022 only<br>requests = df[[&#39;request_datetime&#39;, &#39;on_scene_datetime&#39;, &#39;pickup_datetime&#39;, &#39;dropoff_datetime&#39;, &#39;PULocationID&#39;, &#39;DOLocationID&#39;]][df.request_datetime.dt.year==2022]<br><br># sort by request_datetime<br>requests.sort_values(by=[&#39;request_datetime&#39;], inplace=True)<br><br># set request_datetime as index<br>requests.set_index(&#39;request_datetime&#39;, inplace=True)<br><br>requests.head()</pre><p>Our DataFrame now looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/891/1*ra5wi3wdimD0g-ESGZSiug.png" /><figcaption>data output generated by author</figcaption></figure><p>Remember that we are looking to group our dataset into 15-minute windows with the number of requests per pickup zone for each window. This means we want our output to have 4 rows (15 x 4 = 60 minutes) and a column for each PULocationID.</p><p>We need a pivot:</p><pre># expand df to have a unique column for each PULocationID<br>pivot = pd.pivot_table(<br>    requests, <br>    values=&#39;pickup_datetime&#39;, <br>    index=requests.index,<br>    columns=[&#39;PULocationID&#39;], <br>    aggfunc=&#39;count&#39;<br>)<br><br>pivot.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k9dZnYvLDX8UgjhiXveHhw.png" /><figcaption>data output generated by author</figcaption></figure><p>This is a sparse DataFrame with simply a count (1) for each row (request) in the column corresponding to that requests’ pickup zone.</p><p>Let’s resample our DataFrame with .sum()to group this data into 15-minute windows and sum up the counts for each PULocationID:</p><pre>tumbling = pivo.resample(&#39;15min&#39;).sum()<br>tumbling</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Up6KMsTTWobILbD0dhLkiQ.png" /><figcaption>data output generated by author</figcaption></figure><p>Nice! We now have the number of requests per pickup zone for each 15-minute window. All that’s left is to define exactly what we mean with “high-demand” — let’s say 40 requests per 15-minute window — and then we can use this DataFrame to filter out the windows and pickup zones that exceed that high-demand threshold.</p><p>We can also inspect the data visually to identify the peak-demand zones:</p><pre>res.resample(&#39;15min&#39;).sum().plot(legend=False)</pre><figure><img alt="plot of the sums of requests per pickup zone, the plot is dense and not very legible or useful" src="https://cdn-images-1.medium.com/max/636/1*nxnQ3s6TpyXDNPI3Xj9GLA.png" /><figcaption>plot generated by author</figcaption></figure><p>This is great…but still only sort of. We’re still missing out on lots of granularity here. We need to wait for each 15-min window to finish before we can aggregate the data. What if the spike happens in the 15-minute window between minute 1 and minute 16 (rather than minute 0 and minute 15)? We will have missed the spike by just a minute but will need to wait 14 minutes before receiving the signal. By that time it may be too late to instruct drivers to that pickup zone.</p><blockquote>Do not fear, perfection is near!</blockquote><h3>3. Sliding Windows in Pandas</h3><figure><img alt="diagram showing a line representing the dataset and boxes representing each 5-minute window. The 5-minute window boxes overlap by four-fifths." src="https://cdn-images-1.medium.com/max/934/1*NmrHVa8WCIkhdkMu17nt4Q.png" /><figcaption>image by author</figcaption></figure><p>Ideally, we want a signal every minute that tells us which zones are experiencing peak demand. We also want to reduce the size of our windows to enable our drivers to react to the incoming data faster. So let’s work with 5-minute windows and define “peak demand” as <strong>more than 50 requests per 5-minute window</strong>.</p><p>We can accomplish this using the pandas .rolling() method to create <strong>sliding windows</strong>: 5-minute windows with a 4-minute overlap, so that we can track the rate of incoming requests every minute.</p><p>You can create sliding windows in pandas using the .resample() and .rolling() methods. Make sure to .resample() to <strong>the size of your desired signal interval</strong> instead of the size of your window:</p><pre># create sliding windows in pandas<br>res = pivot.resample(interval_size).sum()<br>windows = res.rolling(window_size).sum()</pre><p>Let’s pick it apart step by step.</p><p>We want our output to be a series of rows in which each row is a 5-minute window and the columns contain the sum of requests per pickup zone for that 5-minute window. However, instead of starting the second row at the end of the first 5-minute window, we want it to start at the end of the first minute of that first 5-minute window.</p><p>Still with me here? If not, maybe this diagram below will help:</p><figure><img alt="Diagram showing how the 5-minute window boxes overlap by four-fifths to give us a signal every minute." src="https://cdn-images-1.medium.com/max/868/1*E5HZ55eyvrwPg0eWI9Y-6g.png" /><figcaption>image by author</figcaption></figure><p>What we really want is 56 rows of 5-minutes each, with a 4-minute overlap between each row. This will give us a signal every minute which zones (if any) are getting more than 50 requests per 5 minutes.</p><p>To do this, we’ll have to resample our DataFrame first in order to get a single row for every minute:</p><pre>res_1m = res.resample(&#39;1min&#39;).sum()<br>res_1m.head(3)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/997/1*qzTf0LxtS3dsJY5hyf4V-g.png" /><figcaption>data output generated by author</figcaption></figure><p><em>(If you wanted to get a signal say every second, you would resample to </em><em>1s here.)</em></p><p>Now that every row represents a single minute we can use rolling() and sum() to get the sum of the number of requests for every 5 rows (minutes):</p><pre>windows = res_1m.rolling(5).sum()<br>windows.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wbYoEk9c7EnVRkuTILw39w.png" /><figcaption>data output generated by author</figcaption></figure><p>😱 Yikes, that’s a lot of NaNs.</p><p>But that’s only because .rolling() by default places the output of the window operation (in this case the sum) at the right edge of the window. This is technically correct since we will only get the first signal at the end of minute 4, when the first window has been completed.</p><p>We can drop the first 4 columns if we want to get rid of the first 4 empty rows (minutes):</p><pre>windows_drop = windows.iloc[4:]<br>windows_drop.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1019/1*p2IO0FHPwpxayDRMxCQjsQ.png" /><figcaption>data output generated by author</figcaption></figure><h4>Side note: Forward-Looking Windows</h4><p>In some cases, you might want to move the values to the left edge to make it more easily readable. If you do so, be careful not to make the mistake of thinking that you will actually have data for window 1 at minute 1 (!) — you’re really just shifting the results over to the first row.</p><p>You can use the FixedForwardWindowIndexer object to place the window output on the left edge and create forward-looking windows:</p><pre># create custom window indexer<br>indexer = pd.api.indexers.FixedForwardWindowIndexer(window_size=5)<br><br># use indexer to create rolling window<br>windows_f = res_1m.rolling(window=indexer).sum()<br><br>windows_f.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1018/1*3O0yZoRSvx39-5MlYMYwQg.png" /><figcaption>data output generated by author</figcaption></figure><p>The last 4 columns of our DataFrame are now empty (because there is no full 5-minute window available after minute 55) and we can drop them.</p><pre># drop the last 14 columns<br>windows_f_drop = windows.iloc[:45]<br>windows_f_drop.tail()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9E5mYwEydsiadlZakRgDXg.png" /><figcaption>data output generated by author</figcaption></figure><p>This way of using the .rolling()method is not very well-known and is a little <a href="https://pandas.pydata.org/docs/user_guide/window.html#custom-window-rolling">buried in the docs</a> but once you understand how it works it’s pretty easy to use.</p><h3>Visualise Sliding Windows in Pandas</h3><p>Let’s visualise our data to identify some patterns and spot the pickup zones with the highest peaks in demand. We’ll work with windows to make it extra clear that we don’t have any data for the first 5 minutes:</p><pre>import matplotlib.pyplot as plt<br><br>windows.plot(kind=&#39;bar&#39;, legend=False);</pre><figure><img alt="Plot of all the rows in our 5-minute sliding window dataframe. The plot is messy and hard to read." src="https://cdn-images-1.medium.com/max/639/1*_g30fvnxp1tYLkfqsaUvFg.png" /><figcaption>plot generated by author</figcaption></figure><p>This is a very rough visualisation but even here we can already clearly see that:</p><ol><li>We don’t have any data the first 5 minutes</li><li>There are clear peaks in the rate of pickup requests in specific zones.</li></ol><p>From here, we can now define a cutoff/threshold and identify zones with above-threshold requests in specific windows. We can then motivate drivers to go to that zone.</p><p>One way to clarify the plot is to remove any columns with values below the threshold. Let’s say we want to identify zones with more than 50 requests per 5-minute window:</p><pre>signal = windows[(windows &gt; 50)].dropna(axis=1, how=”all”)<br>signal.plot(figsize=(10,8))<br>plt.legend(bbox_to_anchor=(1.0,1.0))<br>plt.show()</pre><figure><img alt="Plot of all the rows in our 5-minute sliding window dataframe with requests above the threshold. The plot is clearly legible and we see 11 high-demand pickup zones listed in the legend." src="https://cdn-images-1.medium.com/max/1024/1*yo6UVUGGfJkByElSGQ5gYw.png" /><figcaption>plot generated by author</figcaption></figure><p>There’s much more we could do here. Play around with the code in <a href="https://github.com/rrpelgrim/sliding-windows-pandas/blob/main/pandas-sliding-windows-nyc-uber-lyft.ipynb">the notebook</a> to perfect this further.</p><h3>Sliding Windows in Pandas: Conclusion</h3><p>This article has introduced you to the powerful capabilities of performing sliding windows on time-series data. By chopping up your dataset into small, overlapping windows you are now able to get valuable data signals <em>at an actionable rate</em>.</p><p>Of course, ideally you would be doing all of this in real-time. Watch this space for more on that soon, but in the meantime you might want to check out <a href="https://www.dataengineeringweekly.com/p/unlocking-data-stream-processing">this real-time monitoring tutorial</a>. Be prepared for a slight context change from New York traffic to website traffic instead ;)</p><p>I hope you found this post helpful and engaging. <a href="https://www.linkedin.com/in/avrilaysha/">Follow me on LinkedIn</a> to catch my latest content.</p><p>Happy data crunching! 👋</p><p><em>*New York City Taxi and Limousine Commission (TLC) Trip Record Data was accessed on March 15, 2023 from </em><a href="https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page"><em>https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page</em></a><em>. License: </em><a href="https://www.nyc.gov/home/terms-of-use.page"><em>https://www.nyc.gov/home/terms-of-use.page</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=40b79edefa34" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/sliding-windows-in-pandas-40b79edefa34">Sliding Windows in Pandas</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Not Everyone Can Become a Data Scientist]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://avrilaysha.medium.com/not-everyone-can-become-a-data-scientist-433f10c2e5d0?source=rss-b016782a8709------2"><img src="https://cdn-images-1.medium.com/max/2600/1*cVK-MMHfc8arqnmv5sQiWw.jpeg" width="4671"></a></p><p class="medium-feed-snippet">Why we need to talk more openly about privilege</p><p class="medium-feed-link"><a href="https://avrilaysha.medium.com/not-everyone-can-become-a-data-scientist-433f10c2e5d0?source=rss-b016782a8709------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://avrilaysha.medium.com/not-everyone-can-become-a-data-scientist-433f10c2e5d0?source=rss-b016782a8709------2</link>
            <guid isPermaLink="false">https://medium.com/p/433f10c2e5d0</guid>
            <category><![CDATA[development]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[money]]></category>
            <category><![CDATA[careers]]></category>
            <dc:creator><![CDATA[Avril Aysha]]></dc:creator>
            <pubDate>Wed, 01 Jun 2022 14:07:03 GMT</pubDate>
            <atom:updated>2022-06-01T14:07:03.727Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Accessing the NYC Taxi Data in 2022]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/data-science/accessing-the-nyc-taxi-data-in-2022-8363adfda76a?source=rss-b016782a8709------2"><img src="https://cdn-images-1.medium.com/max/2600/1*P444fv3gmbNzbCOKk203pg.jpeg" width="4000"></a></p><p class="medium-feed-snippet">Everything you need to know about the recent changes</p><p class="medium-feed-link"><a href="https://medium.com/data-science/accessing-the-nyc-taxi-data-in-2022-8363adfda76a?source=rss-b016782a8709------2">Continue reading on TDS Archive »</a></p></div>]]></description>
            <link>https://medium.com/data-science/accessing-the-nyc-taxi-data-in-2022-8363adfda76a?source=rss-b016782a8709------2</link>
            <guid isPermaLink="false">https://medium.com/p/8363adfda76a</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[dataset]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[nyc]]></category>
            <category><![CDATA[big-data]]></category>
            <dc:creator><![CDATA[Avril Aysha]]></dc:creator>
            <pubDate>Tue, 17 May 2022 08:42:04 GMT</pubDate>
            <atom:updated>2022-05-18T19:39:36.418Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Julia vs Python for Data Science in 2022]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://avrilaysha.medium.com/julia-vs-python-for-data-science-in-2022-1f5f6f38f3ac?source=rss-b016782a8709------2"><img src="https://cdn-images-1.medium.com/max/2600/1*6JR-zIOrnMD38TKT7gQivQ.jpeg" width="5470"></a></p><p class="medium-feed-snippet">Comparing Programming Languages for Data Science</p><p class="medium-feed-link"><a href="https://avrilaysha.medium.com/julia-vs-python-for-data-science-in-2022-1f5f6f38f3ac?source=rss-b016782a8709------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://avrilaysha.medium.com/julia-vs-python-for-data-science-in-2022-1f5f6f38f3ac?source=rss-b016782a8709------2</link>
            <guid isPermaLink="false">https://medium.com/p/1f5f6f38f3ac</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Avril Aysha]]></dc:creator>
            <pubDate>Fri, 01 Apr 2022 01:45:48 GMT</pubDate>
            <atom:updated>2022-04-01T01:45:48.893Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[5 Rookie Mistakes to Avoid when Using Dask]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/data-science/5-rookie-mistakes-to-avoid-when-using-dask-592e4a909de6?source=rss-b016782a8709------2"><img src="https://cdn-images-1.medium.com/max/2600/1*omZHp_hhe-CICEoEsAMDKg.jpeg" width="2832"></a></p><p class="medium-feed-snippet">Strategies for Successful Distributed Computing in Python</p><p class="medium-feed-link"><a href="https://medium.com/data-science/5-rookie-mistakes-to-avoid-when-using-dask-592e4a909de6?source=rss-b016782a8709------2">Continue reading on TDS Archive »</a></p></div>]]></description>
            <link>https://medium.com/data-science/5-rookie-mistakes-to-avoid-when-using-dask-592e4a909de6?source=rss-b016782a8709------2</link>
            <guid isPermaLink="false">https://medium.com/p/592e4a909de6</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[dask]]></category>
            <category><![CDATA[data-science-training]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Avril Aysha]]></dc:creator>
            <pubDate>Thu, 10 Feb 2022 19:23:06 GMT</pubDate>
            <atom:updated>2022-02-22T09:21:53.111Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[The Beginner’s Guide to Distributed Computing]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/data-science/the-beginners-guide-to-distributed-computing-6d6833796318?source=rss-b016782a8709------2"><img src="https://cdn-images-1.medium.com/max/600/1*B5lGA3Th--hsYXoo7_r38Q.gif" width="600"></a></p><p class="medium-feed-snippet">7 Fundamental Concepts to Succeed With Distributed Computing in Python</p><p class="medium-feed-link"><a href="https://medium.com/data-science/the-beginners-guide-to-distributed-computing-6d6833796318?source=rss-b016782a8709------2">Continue reading on TDS Archive »</a></p></div>]]></description>
            <link>https://medium.com/data-science/the-beginners-guide-to-distributed-computing-6d6833796318?source=rss-b016782a8709------2</link>
            <guid isPermaLink="false">https://medium.com/p/6d6833796318</guid>
            <category><![CDATA[distributed-computing]]></category>
            <category><![CDATA[deep-dives]]></category>
            <category><![CDATA[big-data]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Avril Aysha]]></dc:creator>
            <pubDate>Thu, 10 Feb 2022 14:55:51 GMT</pubDate>
            <atom:updated>2024-02-06T14:00:18.895Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[How to Build Powerful Airflow DAGs for Big Data Workflows in Python]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/data-science/how-to-build-powerful-airflow-dags-for-big-data-workflows-in-python-14eb2b66f280?source=rss-b016782a8709------2"><img src="https://cdn-images-1.medium.com/max/2600/1*MsE29S_Zo2fWhQzaucZLoA.jpeg" width="3824"></a></p><p class="medium-feed-snippet">Scale your Airflow pipelines to the cloud</p><p class="medium-feed-link"><a href="https://medium.com/data-science/how-to-build-powerful-airflow-dags-for-big-data-workflows-in-python-14eb2b66f280?source=rss-b016782a8709------2">Continue reading on TDS Archive »</a></p></div>]]></description>
            <link>https://medium.com/data-science/how-to-build-powerful-airflow-dags-for-big-data-workflows-in-python-14eb2b66f280?source=rss-b016782a8709------2</link>
            <guid isPermaLink="false">https://medium.com/p/14eb2b66f280</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[airflow]]></category>
            <dc:creator><![CDATA[Avril Aysha]]></dc:creator>
            <pubDate>Fri, 07 Jan 2022 16:30:15 GMT</pubDate>
            <atom:updated>2022-02-11T12:29:14.686Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Why You Should Save NumPy Arrays with Zarr]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/data-science/why-you-should-save-numpy-arrays-with-zarr-dabff4ae6c0c?source=rss-b016782a8709------2"><img src="https://cdn-images-1.medium.com/max/2600/1*ikaD6ot28DacO_O-AnA5vw.jpeg" width="4914"></a></p><p class="medium-feed-snippet">Read and Write Arrays Faster with Dask</p><p class="medium-feed-link"><a href="https://medium.com/data-science/why-you-should-save-numpy-arrays-with-zarr-dabff4ae6c0c?source=rss-b016782a8709------2">Continue reading on TDS Archive »</a></p></div>]]></description>
            <link>https://medium.com/data-science/why-you-should-save-numpy-arrays-with-zarr-dabff4ae6c0c?source=rss-b016782a8709------2</link>
            <guid isPermaLink="false">https://medium.com/p/dabff4ae6c0c</guid>
            <category><![CDATA[numpy]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Avril Aysha]]></dc:creator>
            <pubDate>Wed, 05 Jan 2022 16:49:32 GMT</pubDate>
            <atom:updated>2022-02-11T12:32:55.536Z</atom:updated>
        </item>
    </channel>
</rss>