<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Pranav Badami on Medium]]></title>
        <description><![CDATA[Stories by Pranav Badami on Medium]]></description>
        <link>https://medium.com/@pranavbadami?source=rss-9aa99fb1d5e7------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Np1Dm-0E9hd0i6CUeoYR5Q.png</url>
            <title>Stories by Pranav Badami on Medium</title>
            <link>https://medium.com/@pranavbadami?source=rss-9aa99fb1d5e7------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 21:04:47 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@pranavbadami/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Developer’s Annotation: Downtown Brooklyn Case Study]]></title>
            <link>https://medium.com/numina/developers-annotation-downtown-brooklyn-case-study-4c9b1f13e283?source=rss-9aa99fb1d5e7------2</link>
            <guid isPermaLink="false">https://medium.com/p/4c9b1f13e283</guid>
            <category><![CDATA[transportation]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[api]]></category>
            <category><![CDATA[product]]></category>
            <category><![CDATA[civictech]]></category>
            <dc:creator><![CDATA[Pranav Badami]]></dc:creator>
            <pubDate>Mon, 23 Sep 2019 12:46:01 GMT</pubDate>
            <atom:updated>2019-09-23T12:46:01.182Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0GT-6kXtIl9RHTJgzINp7A.png" /><figcaption>To learn more about the Numina API, head to <a href="https://www.numina.co/api">numina.co/api</a></figcaption></figure><p>Hi there! This post is a developer annotation of a <a href="https://numina.co/downtown-brooklyn-case-study-scaffolding-and-safety/">previous analysis</a> published on the Numina blog. In this post, we’ll go through how a developer can use the <a href="http://numina.co/?page_id=1066">Numina API</a> to analyze detailed street-level data. If you’re interested in using the Numina API, please request access to our developer sandbox <a href="https://numina.co/contact-apps/">here</a>.</p><p><em>In New York City, construction and scaffolding are familiar backdrops of daily life, from the drone of jackhammers as an early morning wake-up call to commute detours due to an unexpected sidewalk closure or delivery truck parked in a bike lane. With over </em><a href="http://gothamist.com/2019/01/30/city_scaffolding_sidewalk_sheds.php"><em>300 miles of scaffolding in the city</em></a><em>, at scale, these street level changes present a safety concern for pedestrians and disrupt the flow of traffic across all modes (e.g. pedestrian, bicycle, car).</em></p><p><em>At Numina, we measure a wide range of indicators of mobility and quality of life on our streets, including traffic volume counts, pedestrian road crossing paths, and incidents of illegal activity such as cars driving in the bus lane. This year, we measured the impact of a month of construction and scaffolding at a section of the Fulton Mall in Downtown Brooklyn.</em></p><h3>Annotation: Devices Query</h3><p>Let’s start by getting some basic geographic information about the Numina sensors deployed for our organization. First, follow the instructions in <a href="https://developer.numina.co/">the Getting Started documentation</a> to authenticate with the Numina API. You will notice that the Numina API is a GraphQL API; if you’re unfamiliar or rusty with GraphQL, you can get caught up <a href="https://graphql.org/">here</a>.</p><p>You can get information about the sensors that your user has access to by issuing the devices query like so:</p><p>For this analysis we’ll focus on the device with alias dbp-4 and serialno NUM200B001S0004.</p><p><strong>Query:</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d5e1ccfe1250b5f5f5384efce03597b9/href">https://medium.com/media/d5e1ccfe1250b5f5f5384efce03597b9/href</a></iframe><p><strong>Response:</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2659f64eb6c9f12ce79fdb657f99479c/href">https://medium.com/media/2659f64eb6c9f12ce79fdb657f99479c/href</a></iframe><p><em>As a major shopping destination and at the heart of the City’s third largest Central Business District, Fulton Mall is a hotspot for commuters and shoppers, and sees high pedestrian and cyclist activity throughout the area. However, it also hosts several bus stops and receives a significant amount of truck and car activity.</em></p><p><em>When we installed our sensors on Fulton Mall, with support from </em><a href="http://newlab.com/"><em>New Lab</em></a><em> and </em><a href="http://downtownbrooklyn.com/"><em>Downtown Brooklyn Partnership</em></a><em>, we noticed the prevalence of scaffolding in our image views, foreshadowing the high levels of construction in the area. A few weeks after, the first project, a building demolition, began, closing down an entire stretch of sidewalk, and on some days, even the road.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/0*r4AlhS1yQz3UZz4O.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/0*Img0ymUw2xwhwte1.png" /><figcaption>One of our sensor’s field of view at Fulton Mall. Note: Our sensors transmit sample images once per hour, at random times each hour; the images are then de-identified. See our <a href="https://numina.co/our-privacy-principles/">privacy policy</a> for more details.</figcaption></figure><h3>Annotation: Sample Images Query</h3><p>We can access sample images for a given sensor’s field of view over time using the getSampleImages query in the Numina API like so:</p><p><strong>Query:</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/924ab5733987e3516ff54401f50d1f4d/href">https://medium.com/media/924ab5733987e3516ff54401f50d1f4d/href</a></iframe><p><strong>Response:</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2a48936570cfb3c686823aae10a39289/href">https://medium.com/media/2a48936570cfb3c686823aae10a39289/href</a></iframe><p>Currently, users need to request for an updated sample image to be used starting on a given date. If you believe that your sensor view has changed significantly due to construction or changes to the built environment, please contact <a href="/service/mailto:support@numina.co">support@numina.co</a> and we will facilitate adding a new timestamped sample image.</p><p><em>After a few days, the road was cleared, but only a small opening connected the sidewalk across either side of the construction site. This forced pedestrians to step into the road often and even walk in the road regularly. Seasons passed, and even during snowy days, pedestrians found themselves sharing the road with vehicles, often large trucks and buses.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/0*W0G6Eab5bTNlFhB5" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/0*7v20gzM3Wgqsx8Gj.png" /></figure><h3>Annotation: Create Behavior Zone Mutation</h3><p>Let’s try to understand how pedestrian road crossings was affected by the presence of construction and scaffolding on Fulton Mall. First, we need to define “the road” as a spatial region of our image; the Numina API refers to these regions as <strong>Behavior Zones</strong>. The easiest way to create a Behavior Zone is via the sensor view in the Numina dashboard. However, if you are part of our Developer Sandbox you can use our in-browser <a href="http://developer.numina.co/behavior-zone-creator/">Behavior Zone Creator</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/570/0*Qgu2tquv8cu7LAJO.png" /><figcaption>Access the Behavior Zone creator tool <a href="https://developer.numina.co/behavior-zone-creator/index.html">here</a></figcaption></figure><p>Grab the coordinates produced by the drawing tool and use the createBehaviorZone mutation like so:</p><p><strong>Mutation:</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/79e49fadcc7cc40513731d1f002a20b9/href">https://medium.com/media/79e49fadcc7cc40513731d1f002a20b9/href</a></iframe><p><strong>Response:</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a1ab22e1d146afcc980154cc37dedc40/href">https://medium.com/media/a1ab22e1d146afcc980154cc37dedc40/href</a></iframe><p><em>Over a month after construction began, the sidewalk was partially reopened, but the scaffolding remained, and pedestrians continued to walk in the street occasionally. We compared pedestrian activity during the construction period, 10/11/18–11/19/18, to that before and after construction. By applying spatial analysis techniques to our pedestrian movement data, we were able to analyze how many more pedestrians walked on the street when the sidewalk was obstructed by construction. The data showed that pedestrians were 53.3% more likely to walk in the road during that time frame.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/695/0*ckGKrmCbdgpF-Oj5" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/0*M4zShSwIYgd3za1M.png" /></figure><h3>Annotation: Using the CountMetrics analytics queries</h3><p>To perform this analysis we need to use analytics queries in the Numina API. First, we will want to know how many pedestrians were detected by this sensor over the given time period. For this purpose, we can use the feedCountMetrics query by passing in our sensor’s serial number, a date range, a time interval, and a list of object classes. The feedCountMetrics query will return a response similar to the following:</p><p><strong>Query:</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/596d0440fddf659a2ca931f5dbd9834e/href">https://medium.com/media/596d0440fddf659a2ca931f5dbd9834e/href</a></iframe><p><strong>Response:</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bb610a0fef1d7bfca7fbaf0792bdb18a/href">https://medium.com/media/bb610a0fef1d7bfca7fbaf0792bdb18a/href</a></iframe><p>Then, we can issue another zoneCountMetrics query to the Numina API, passing the rawId of the “Roadway” Behavior Zone we created earlier with our mutation:</p><p><strong>Query:</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a87ce37cf1f08cf82af7ab50d9e3938f/href">https://medium.com/media/a87ce37cf1f08cf82af7ab50d9e3938f/href</a></iframe><p><strong>Response:</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2f4263713e61a1a3a2651c9949507faf/href">https://medium.com/media/2f4263713e61a1a3a2651c9949507faf/href</a></iframe><p>We can use the resulting counts data from the two analytics queries to compare the counts of pedestrians in the road versus total pedestrians. This normalizes the counts data and lets us understand how anomalous counts are for a given time period.</p><p><em>Some other troubling trends we observed occurred when delivery trucks parked on the side of the two-lane road, effectively shutting down a lane and transforming the street into a one lane road. With pedestrians, cyclists, and two directions of car, bus, and truck traffic sharing one lane, risk of collision between any of these modes increased significantly.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/599/0*ZGNDGrVvWmpXuoZH" /></figure><h3>Annotation: Visualizing street-level activity with heatmaps</h3><p>This specific animation was created with in-house software, but we can still generate data visualizations using the Numina API. To get an understanding of how pedestrians moved about this scene over a period of time, we can call the feedHeatmaps analytics query with our serialno:</p><p><strong>Query:</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/680227fa25187186f70012dc2ba4a992/href">https://medium.com/media/680227fa25187186f70012dc2ba4a992/href</a></iframe><p><strong>Response:</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/40fec2ca31d48e1458b3eef52e056675/href">https://medium.com/media/40fec2ca31d48e1458b3eef52e056675/href</a></iframe><p>Note that the heatmap data is returned as a list of [x, y, data] objects. We can overlay this data on top of a sample image for the time period (retrieved in the Sample Images Query annotation) using a visualization library of our choice (such as <a href="https://opencv.org/">opencv-python</a>) to better understand pedestrian patterns in this streetscape:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/0*Ruf-JEPw43TWM5Kt" /></figure><p>Similarly, we can also use the zoneHeatmap queries with our “Roadway” Behavior Zone from a few annotations above. This helps us hone in on the movement patterns of pedestrians who crossed the road:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/0*qe7aNbWSCvQiI6V9" /></figure><p>These visualizations (and our Behavior Zone filtering) are computed on the fly, over hundreds of thousands of objects. If you want to learn more about how we perform these computations quickly, see the talk we gave at <a href="https://numina.co/numina-presents-at-transit-techies-nyc/">Transit Techies NYC</a>.</p><p><em>Two months after this construction project ended, another began at the end of the block, at the Fulton &amp; Flatbush intersection, shutting down part of the sidewalk on the other side of the street. While building and street development is an inevitable part of New York City life, efforts can be made to improve safety conditions during these projects, such as better signage to inform pedestrians early on of a road closure, temporary pedestrian refuges or mid-block crosswalks at busy intersections, or better enforcement of idling vehicles during this period.</em></p><p><em>One of Numina’s goals is to provide responsible, secure, objective intelligence about the impact of street changes on everyone who uses our public realm, to identify opportunities for positive interventions improving safety, efficiency, and enjoyment of streets. Better understanding of construction’s impact on streets invites new ideas for temporary pedestrian infrastructure and ways to reimagine the streetscape during times of change.</em></p><h3>How you can try the Numina API</h3><p>We hope that this set of developer annotations serves as a helpful springboard on using the Numina API to dive into street-level data. We believe there’s plenty of work that could be done with the Numina API, such as anomaly detection, and analysis of weather and planned events. If you’re interested in using the Numina API to conduct data analyses of your own, please <a href="https://numina.co/contact-apps/">request access</a> to our <a href="https://developer.numina.co/#api-sandbox">API Sandbox</a>. The sandbox contains ten days of real street-level data from a sensor deployment in Downtown Brooklyn in New York City. Thanks for reading!</p><p><a href="https://www.numina.co/"><em>Numina</em></a><em> empowers cities with data to become more responsive and more equitable. With a </em><a href="https://numina.co/our-privacy-principles/"><em>privacy-first approach</em></a><em>, we develop technology that measures all kinds of activity in streets and provides intelligence without surveillance.</em></p><p><em>If you enjoyed this post, please :clap: below! To see more from Numina, please follow us on Medium, </em><a href="https://twitter.com/numina"><em>Twitter</em></a><em>, and </em><a href="https://www.instagram.com/numina/"><em>Instagram</em></a><em>!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4c9b1f13e283" width="1" height="1" alt=""><hr><p><a href="https://medium.com/numina/developers-annotation-downtown-brooklyn-case-study-4c9b1f13e283">Developer’s Annotation: Downtown Brooklyn Case Study</a> was originally published in <a href="https://medium.com/numina">Numina</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Recap — Transit Techies NYC #7: 7 Minute Delay]]></title>
            <link>https://medium.com/@pranavbadami/recap-transit-techies-nyc-7-7-minute-delay-37d334026fa3?source=rss-9aa99fb1d5e7------2</link>
            <guid isPermaLink="false">https://medium.com/p/37d334026fa3</guid>
            <category><![CDATA[transportation]]></category>
            <category><![CDATA[new-york-city]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Pranav Badami]]></dc:creator>
            <pubDate>Thu, 28 Mar 2019 16:43:40 GMT</pubDate>
            <atom:updated>2019-03-28T16:43:40.194Z</atom:updated>
            <content:encoded><![CDATA[<h3>Recap: Transit Techies NYC #7: 7 Minute Delay</h3><p><em>A quick recap of </em><a href="https://www.meetup.com/Transit-Techies-NYC/events/259203820/"><em>Transit Techies NYC #7: 7 Minute Delay</em></a><em>, a Meetup held on Wednesday, March 20, 2019</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8hODZ1nvri1uOg2ytddy4Q.jpeg" /><figcaption>The Vessel itself</figcaption></figure><p>As usual, wonderful hosts <a href="https://twitter.com/greent_tyler?lang=en">Tyler Green</a> and <a href="https://twitter.com/kohzy">Kohzy Koh</a> kicked off the evening with a funny bit. This time, they launched into a quick trivia session about the history behind the newly completed Hudson Yards (where <a href="https://www.intersection.com/">Intersection</a> hosted this Meetup). Amongst other things, I learned that the new Vessel sculpture has been… let’s say, compared to many different things. Great stuff as always!</p><h3>goodservice.io — Sunny Ng</h3><p>First up was <a href="https://twitter.com/_blahblahblah">Sunny Ng</a>, who presented his <a href="https://www.goodservice.io/trains">goodservice.io</a> project. A few years ago, the MTA started to publish real time service status tables for the subway system. As these things go, New Yorkers began to mock what the MTA data considered “good service” so Sunny set out to build a better alternative.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5lZ2uQOKA_2hkSkk_Cj5Zw.jpeg" /><figcaption>Sunny Ng demoing goodservice.io</figcaption></figure><p>goodservice.io compares real-time headways (the time between trains arriving at a platform) to scheduled headways in order to provide a better understanding of current subway performance. Sunny then refined his implementation to incorporate local vs express trains, segment subway lines into different service sections (that may be active concurrently), and account for any ad hoc service changes. The final output is a more accurate notion of subway status based on headway at that exact moment:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HFoIsAYn09_2fI_Jw2ovRg.jpeg" /><figcaption>The different statuses outputted by goodservice.io</figcaption></figure><p>He also built a nifty <a href="https://ctylives.slack.com/apps/AGK7ZP2AH-goodserviceio">Slack integration</a> so you can quickly check service status before leaving work. Finally, a <a href="https://twitter.com/goodservice_io">Twitter bot</a> posts any service interruptions to Twitter in real-time and Sunny showed us a few examples where it’s even announced delays before the MTA’s official account. You can learn more and follow goodservice.io <a href="https://twitter.com/goodservice_io">here</a>. 🚋</p><h3>Accessibility Principles at Sidewalk Labs — Samara Trilling</h3><p><a href="https://www.linkedin.com/in/samaratrilling/">Samara Trilling</a>, the accessibility lead at <a href="https://www.sidewalklabs.com/">Sidewalk Labs</a>, was up next to discuss the SWL’s efforts to incorporate accessibility by design into their <a href="https://medium.com/sidewalk-toronto/quayside-a-new-vision-for-torontos-waterfront-60d969d16c5f">Quayside</a> (TIL: pronounced “key-side”) development. SWL’s perspective is that intentionally designing cities for underrepresented and marginalized experiences makes the final product better for everyone. Towards this end, SWL aims to co-create accessibility principles for future cities by working with accessibility groups that have lived experience in their cities. 🙌</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g439R7Qqj--HVpthRIJOzQ.jpeg" /><figcaption>Samara Trilling presenting on accessibility</figcaption></figure><p>What do accessibility and transit entail in smart cities of the future? To answer this question, Samara’s team has been leading many co-design brainstorming sessions with accessibility groups from the Toronto area. These co-design sessions have touched on topics including wayfinding for public transit and the physical design of transit infrastructure.</p><p>Ideas from these sessions were then prototyped and iterated upon at a community hackathon at SWL’s experimental workspace in Toronto. One idea I found particularly interesting was an augmented reality feedback interfaces for public parks; this could make it easier for certain users to acquire important information about their immediate surroundings.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*korIkuT8lC4l-L9Zr9yZug.jpeg" /><figcaption>Some ideas that came out of the hackathon</figcaption></figure><p>The feedback and ideas from these sessions has been distilled into a set of <a href="https://medium.com/sidewalk-toronto/co-designing-a-more-accessible-community-d6377599f4ce">accessibility principles</a> that SWL released as a public draft for further review and feedback.</p><h3>Wayfinder3D — Will Geary</h3><p>Next up was <a href="https://twitter.com/wgeary">Will Geary</a>, Transit Techies NYC’s first repeat speaker 👏, to discuss his <a href="http://willgeary.github.io/data/2018/03/07/Wayfinder3D.html">Wayfinder3D</a> project. Will’s newest project takes multimodal directions from Google Maps API and compares them by “racing” different modes against each other in a nifty animation.</p><h3>Will Geary on Twitter</h3><p>Driving vs. subway vs. bike, uptown Manhattan -&amp;gt; downtown. Made with https://t.co/ZMngimDaPV https://t.co/h9okdtSm3e</p><p>Will also created a simple command line API to run Wayfinder3D. He demonstrated the app live by inputting Hudson Yards as a starting point and the George Washington Bridge as the destination. The command launched a visualization of car, subway and bike labels racing to the GW bridge. Initially, the car had a sizable lead, but the crowd went wild when the subway burst forward to the front of the pack. Ultimately though, the car won the race. 😕</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W64NWWR43OlaTsnEaHkopw.jpeg" /><figcaption>How Wayfinder3D works, presented by Will Geary</figcaption></figure><p>Will mentioned some additional visualization ideas he’s considering adding to Wayfinder3D, including representing CO2 emissions for each mode with a growing bubble or bar chart. In terms of environmental, time, and monetary cost, it may be interesting to evaluate ride-share in this format as well. If you want to check out or contribute to the project, you can do so <a href="https://github.com/willgeary/Wayfinder3D">here</a>.</p><h3>Brooklyn Bus Network Redesign — Eric Goldwyn</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Pan-0uKQJiXNfNf3I_xZPg.jpeg" /><figcaption>Eric Goldwyn presenting on redesigning Brooklyn’s bus network</figcaption></figure><p>Last up was <a href="https://twitter.com/ericgoldwyn">Eric Goldwyn</a> from NYU’s Marron Institute to talk about his research with <a href="https://twitter.com/alon_levy">Alon Levy</a> on <a href="https://www.citylab.com/perspective/2018/11/brooklyn-bus-route-redesign-mta-new-york/575716/">redesigning the MTA bus network</a> in Brooklyn. Bus ridership has been steadily in decline for many years, while subway usage in the city has increased. Often, transit planners share best practices for increasing bus utilization and service quality, so Eric’s group set out to understand how such best practices could be used to actually increase bus ridership in Brooklyn.</p><p>To do this, the team modeled the current bus system and tweaked their model to measure the potential impact of several best practices in transit network design. For example, they considered how enforcing riders to prepay their transit fare, instead of paying when boarding the bus, could streamline services by reducing the time a bus is idling at a stop.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-saf45yG7ONbbYS177hTgw.jpeg" /><figcaption>Demonstrating improved connectivity in the network</figcaption></figure><p>Interestingly, the group found that consolidating tightly bunched stops was the most impactful way to improve bus performance; their model suggested that doubling the average distance between stops to 1500 ft in Brooklyn could lead to a 7% increase in time efficiency. You can learn more about Eric’s work <a href="https://www.citylab.com/perspective/2018/11/brooklyn-bus-route-redesign-mta-new-york/575716/">here</a>.</p><p><em>Thanks for reading! If you enjoyed this post, please 👏 and share to any transit enthusiasts or NUMTOTS in your life.</em></p><p><em>If you like projects like these, please come say hi at the next </em><a href="https://www.meetup.com/Transit-Techies-NYC/events/260083847/"><em>Transit Techies Meetup</em></a><em>! You can find me on </em><a href="https://twitter.com/Pranav_Badami"><em>Twitter</em></a><em>, </em><a href="https://www.linkedin.com/in/pranavbadami/"><em>LinkedIn</em></a><em>, and Medium.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=37d334026fa3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Four projects improving transportation in New York City]]></title>
            <link>https://medium.com/hackernoon/four-projects-improving-transportation-in-new-york-city-905fb4cd8bac?source=rss-9aa99fb1d5e7------2</link>
            <guid isPermaLink="false">https://medium.com/p/905fb4cd8bac</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[new-york]]></category>
            <category><![CDATA[sustainability]]></category>
            <category><![CDATA[transportation]]></category>
            <dc:creator><![CDATA[Pranav Badami]]></dc:creator>
            <pubDate>Thu, 19 Jul 2018 20:01:01 GMT</pubDate>
            <atom:updated>2019-05-16T12:00:27.826Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A quick recap of the second </em><a href="https://www.meetup.com/Transit-Techies-NYC/"><em>Transit Techies NYC</em></a><em> Meetup: </em><a href="https://www.meetup.com/Transit-Techies-NYC/events/249954509/"><em>Second Avenue Segfaults</em></a><em>, held on Wednesday, July 11, 2018</em></p><p>Transportation systems in New York City are critically important for millions of people. Beyond MTA subways and yellow cabs, commuters and travelers rely on regional commuter rail and the MTA’s bus network to get in and around the city. The past five years has seen bike sharing programs and on-demand transportation burst onto New York’s transportation scene.</p><p>These were all topics of conversation at the second Transit Techies NYC Meetup, held at Intersection HQ in Hudson Yards. Approximately 100 Transit Techies gathered to discuss and learn how <a href="https://hackernoon.com/tagged/technology">technology</a> can make these transportation systems better in NYC. After a solid dinner spread featuring dessert from Baked by Melissa, we settled in for four scheduled talks. Here’s a quick recap of the event!</p><h3>LIRR signal data</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tpKxOu7Q8DBb0vcrnNuYMg@2x.jpeg" /></figure><p><a href="https://twitter.com/wafisher">Will Fisher</a> from the MTA explained how the Long Island Railroad (LIRR) is working with legacy rail signal systems to better inform staff and customers about current train performance. The LIRR is the <a href="https://en.wikipedia.org/wiki/List_of_United_States_commuter_rail_systems_by_ridership">busiest commuter rail network</a> in the country and has, over time, accumulated many interconnected systems to manage rail signals. Signals and interlockings are controlled by seven systems, implemented by various vendors at various times.</p><p>The LIRR is in a predicament: some of these critical systems are no longer maintained and can’t be modified to add features or extract data. Legacy systems can be difficult to work with and the LIRR has been forced to get creative.</p><p>To solve this problem, Will’s team is working on a program that uses optical character recognition (OCR) to glean track data from a real-time graphical user interface (GUI). Then, track segments and interlockings are modeled as nodes in a graph; this graph can then be used to calculate and report performance data, such as expected arrival times.</p><h3>Destination recommender for on-demand mobility</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UFDyJ8x5vOy6duiSZcN9aA@2x.jpeg" /></figure><p><a href="https://www.linkedin.com/in/assel-dmitriyeva">Assel Dimitriyeva</a>, a graduate student at NYU, presented next about building a dynamic recommendation engine to improve on-demand mobility solutions. Elderly or disabled citizens who cannot access conventional public transit rely on paratransit mobility services (e.g. Access-a-Ride in NYC) to travel around cities. On-demand paratransit services are extremely expensive and making them more efficient could mean affordable mobility for these citizens.</p><p>Assel’s project recommends alternative, closer destinations for users of paratransit services using a <a href="https://en.wikipedia.org/wiki/Multi-armed_bandit">multi-armed bandit</a> (MAB) approach and Yelp data. Dynamically recommending similar destinations closer to a user can reduce distance traveled and cost. Considering a set of users traveling to a set of destinations, the problem is modeled using MAB. Then, Yelp data provides destinations as input to a contextually-aware recommender engine that solves the MAB problem and recommends relevant alternative destinations.</p><h3><strong>Bike Inspector by Motivate</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TLuNLyiQyNvtmm5lAO9BDg@2x.jpeg" /></figure><p>Alex Hill, David Bromwich, and Josselin Philippe from <a href="https://www.motivateco.com/">Motivate</a>, a major bike share operator in 9 US cities, presented an internal tool called Bike Inspector. Across all programs, including Citi Bike in NYC, Motivate (recently acquired by Lyft) is responsible for 30,000+ bikes, which need to be kept safe and operational for riders. These bikes need to be serviced periodically, or when a rider reports a problem.</p><p>Motivate’s bike mechanics then have to service these bikes in the field or their in-house shop; locating, inspecting, and repairing bikes across large cities was a big logistical challenge. Bike Inspector, initially developed by Josselin Philippe while he was a mechanic, provides an intuitive map interface for bike mechanics to use in the field. They can see which docks contain bikes in need of service, diagnose issues, and report repairs all within Bike Inspector.</p><p>With Bike Inspector, Motivate can dispatch mechanics more efficiently to docks with bikes that need repairs or inspections. Bike Inspector is in currently in beta in NYC.</p><h3>Data-driven advocacy for better NYC buses</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dOFjyDdVHnP1vqIf0dgyng@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UJxpqSvEel34mpU223afow.png" /><figcaption>Right: <a href="http://busturnaround.nyc/#bus-report-cards">Report cards</a> from Bus Turnaround Coalition</figcaption></figure><p><a href="https://www.linkedin.com/in/mary-buchanan-41980a7a/">Mary Buchanan</a> from Transit Center discussed how New Yorkers have joined forces to advocate for better buses in NYC. A quarter of New Yorkers do not live within walking distance of a subway station and rely on buses to get around. Unfortunately, ridership has been steadily declining for years, possibly due to ride sharing services winning customers. As ridership goes down, funding decreases which causes service to worsen, pushing ridership even lower in a vicious cycle.</p><p>The <a href="http://busturnaround.nyc/">Bus Turnaround Coalition</a>, a group of New Yorkers working with Transit Center, are using data to fight against poor bus service. The Coalition uses real-time bus data from the MTA to compute speed, on-time performance and headway for bus lines in NYC. They use these metrics to generate report cards on the bus system, assigning a letter grade (A-F) to each bus line. According to their methodology, 183 of the 246 (74%) bus routes get either a D or F grade.</p><p>If you’d like to support the Bus Turnaround Coalition, learn more about helping out the #BusTurnaround campaign <a href="http://busturnaround.nyc/#take-action">here</a>.</p><p>Many thanks to <a href="https://twitter.com/greent_tyler">Tyler Green</a> and <a href="https://twitter.com/kohzy">Kozhy Koh</a> for organizing this Meetup! You can learn about (and join) Transit Techies NYC <a href="https://www.meetup.com/Transit-Techies-NYC/">here</a>.</p><p><em>If you enjoyed this recap, please 👏 this article below. Please follow my Medium profile for recaps of </em><a href="https://hackernoon.com/tagged/future"><em>future</em></a><em> transit and sustainability Meetups!</em></p><p><em>Check out my recap of the first Transit Techies NYC Meetup:</em></p><ul><li><a href="https://medium.com/@pranavbadami/transittechiesnyc-a-quick-recap-of-the-inaugural-meetup-b7c3a81428f2"><em>#TransitTechiesNYC: A quick recap of the inaugural Meetup</em></a></li></ul><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2Fdde502%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2Fhackernoon%2F&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/3c851dac986ab6dbb2d1aaa91205a8eb/href">https://medium.com/media/3c851dac986ab6dbb2d1aaa91205a8eb/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=905fb4cd8bac" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hackernoon/four-projects-improving-transportation-in-new-york-city-905fb4cd8bac">Four projects improving transportation in New York City</a> was originally published in <a href="https://medium.com/hackernoon">HackerNoon.com</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What makes a city "smart" in 2018?]]></title>
            <link>https://medium.com/hackernoon/what-makes-a-city-smart-in-2018-d2fdf43082de?source=rss-9aa99fb1d5e7------2</link>
            <guid isPermaLink="false">https://medium.com/p/d2fdf43082de</guid>
            <category><![CDATA[sustainability]]></category>
            <category><![CDATA[smart-cities]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Pranav Badami]]></dc:creator>
            <pubDate>Thu, 28 Jun 2018 12:31:00 GMT</pubDate>
            <atom:updated>2019-05-16T12:00:28.059Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UAWNKtBCLoHB7WfqMDskkQ.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/photos/xbusTJ7IUu0?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Luca Bravo</a> on <a href="https://unsplash.com/search/photos/brooklyn-navy-?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><h3>What makes a city “smart” in 2018?</h3><p><em>Current trends, technologies, and companies you need to know about in the smart city space</em></p><p>I recently attended the <a href="https://smartcitiesny.com/">Smart Cities New York</a> conference as a volunteer. SCNY operated at the intersection of technology, policy and urban living, bringing together mayors, startups, big industry and… me. As someone just beginning my journey in the sustainability field, I was blown away by the work being done to make cities more modern, connected, and resource-efficient.</p><p>At the conference, I noticed several key areas that leaders and technologists are focusing on to make cities smarter in 2018: <strong><em>technological equity, open data, and sustainability</em></strong>. I’ve been researching companies and public initiatives working on these focus areas and thought I’d share some of what I’ve learned.</p><h3>Smart cities are thinking about inclusiveness and equity</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NpyNi4SkUVCbS36qAYGfuA.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/photos/5cxIBFFpZEg?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Evan Krause</a> on <a href="https://unsplash.com/search/photos/inclusiveness?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>Making technology inclusive and accessible for all people in cities was a major topic of discussion. Nearly every workshop, panel, and keynote discussed the impact and potential of technology to make cities more (or less) equitable for marginalized populations.</p><ul><li><a href="http://www1.nyc.gov/site/analytics/index.page"><strong>NYC Mayor’s Office of Data Analytics</strong></a><strong>:</strong> MODA is interested in how automated decision systems (basically, algorithms) used by public officials have ethical considerations on low-income or minority populations. For example, municipalities have used predictive policing software such as <a href="https://www.hunchlab.com/">HunchLab</a>, which attempts to inform law enforcement of where crimes are likely to occur. Neighborhoods with minority populations may be disproportionately targeted by such technologies, and public officials need to consider these impacts.</li><li><a href="https://www.dollaride.com/"><strong>Dollaride</strong></a><strong>:</strong> As transportation options in cities grow, how can we ensure that lower income neighborhoods do not get left behind? One company trying to bridge this divide is Dollaride, which provides low-cost transit vans in urban transit deserts, areas that lack accessible public transit options despite being in cities.</li></ul><h3>Smart cities are publicly sharing data</h3><p><a href="https://hackernoon.com/tagged/data">Data</a> for days. As IoT devices and connectivity grows, so does the amount of data being collected by local governments. Public transit, utilities, and permits account for millions of public records published across hundreds of data streams. There’s a lot of creative <a href="http://mbtaviz.github.io/">work</a> <a href="http://iquantny.tumblr.com/">being</a> <a href="https://www.nytimes.com/interactive/2018/05/09/nyregion/subway-crisis-mta-decisions-signals-rules.html">done</a> with this data:</p><ul><li><a href="http://www.opendatanation.com/"><strong>Open Data Nation</strong></a><strong>:</strong> Based out of Washington D.C., Open Data Nation applies machine learning to publicly available data. They’re able to assess risk for things like car crashes and food safety violations, better informing cities and third-party insurers.</li><li><strong>City data portals:</strong> Lots of <a href="http://us-city.census.okfn.org/">cities release data</a> to the public via online portals. This data often includes subway/transit performance, environmental surveys and even detailed cartographical data. For example, check out <a href="https://opendata.cityofnewyork.us/">NYC Open Data</a> and <a href="https://www.citibikenyc.com/system-data">Citi Bike System Data</a>, which I’ll be using to write an article soon!</li></ul><h3><strong>Smart cities are building for a sustainable future</strong></h3><p>It’s estimated that cities account for roughly 70% of global emissions. Cities need to lead a worldwide push to transform urban infrastructure and reduce emissions. The battle to make urban impacts on the environment more sustainable is fought on four main frontiers: transportation, energy generation, building efficiency, and waste management.</p><h4><strong>Transportation:</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*PXqd81sFZO5mFVeM.jpg" /><figcaption>Credit: carmera.com</figcaption></figure><p>Both public and private entities in cities know that urban transportation is undergoing a revolution. Autonomous vehicles (AVs) and electric vehicles (EVs) are on our doorstep, while ride sharing has revolutionized how city dwellers think about transportation.</p><p>Smart cities are preparing for a new transportation future by building support for efficient, next-generation transit into their infrastructure. Cities are working with companies to support <a href="https://www.its.dot.gov/v2i/index.htm">vehicle-to-infrastructure</a> (V2I) systems that can wirelessly communicate with AVs to make operations safer and more efficient.</p><ul><li><a href="https://www.carmera.com/"><strong>Carmera</strong></a>: Carmera provides AVs with real-time, HD maps of city streets, which augment on-board cameras and sensors. AVs can use these maps to see what their on-board cameras cannot and navigate cities better.</li><li><a href="http://www.numina.co/"><strong>Numina</strong></a>: Similarly, Numina provides a simple infrastructure solution for urban planners to gather data on how various modes of traffic traverse intersections. Data-driven urban planning can make cities easier to navigate by foot or bicycle.</li></ul><h4><strong>Energy generation</strong>:</h4><p>The DOE estimates that 62% of the energy consumed to generate electricity is lost somewhere in the electrical grid. Many components of existing electrical grids, such as generators and transmission wires, contribute to these losses. Smart cities are creating better grids that incorporate distributed energy resources, such as solar grids and wind farms, and eliminate wasted energy.</p><ul><li><a href="https://www.opusonesolutions.com/"><strong>Opus One</strong></a><strong>:</strong> As renewable energy sources become economically feasible, solar and wind farms will need to integrate with the existing electrical grid. <a href="https://www.opusonesolutions.com/solutions/">GridOS</a>, by Opus One, provides a grid management platform to plan and monitor how renewable energy assets can integrate with a utility’s infrastructure.</li><li><a href="https://3dfs.com/"><strong>3DFS</strong></a><strong>:</strong> 3DFS is working on software defined electricity (SDE). The company is <a href="https://www.vox.com/energy-and-environment/2018/6/5/17373314/electricity-technology-efficiency-software-waste-3dfs">using software</a> to (nearly) instantaneously modulate electric power to the instantaneous load of an electrical system, such as a data center. The idea is that optimizing power delivery to an electric load can greatly reduce waste in the form of heat and vibration in circuits.</li></ul><h4><strong>Building efficiency</strong>:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/969/0*ILEv7PEV8Y_nJggz.jpg" /><figcaption>Credit: <a href="http://fentrend.com/portfolio-item/the-brooklyn-grand/">Fentrend</a></figcaption></figure><p>New York City <a href="https://www.dec.ny.gov/docs/administration_pdf/nycghg.pdf">estimates</a> that 58% of its emissions are caused by residential and commercial buildings. Heating, ventilation, and cooling (HVAC) systems consume lots of energy. Aging buildings often have inefficient HVAC systems and poor insulation, which compounds the energy demands of these buildings. Making buildings more resource-efficient, or replacing old buildings with more modern construction, is crucial.</p><ul><li><a href="https://www.sapient.industries/"><strong>Sapient Industries</strong></a><strong>:</strong> Sapient is working on an autonomous building management system that identifies how and where energy is being used in a building. Then, via smart outlets, they want to automatically turn off appliances or devices that aren’t being used.</li><li><a href="http://fentrend.com/"><strong>Fentrend</strong></a>: Old windows and doors lead to poorly insulated buildings, causing heating and cooling demands. Fentrend is creating a marketplace between buildings and suppliers to make it easier and cheaper for buildings to purchase modern windows and doors.</li></ul><h4><strong>Waste management</strong>:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*983MQ5dBT-gN60UC.png" /><figcaption>Linear vs circular economy, Credit: <a href="https://nerc.org/news-and-updates/blog/nerc-blog/2015/05/12/materials-management-and-the-circular-economy">Northeast Recycling Council</a></figcaption></figure><p>Currently, most cities are linear economies: raw materials are used to make products, products are consumed, and waste is discarded. Smart cities are thinking about and implementing circular economies, where a sizable portion of materials are reused or recycled back into the economy.</p><ul><li><strong>City of Amsterdam:</strong> Amsterdam is a world leader in transitioning to a circular economy. The city is already recycling <a href="https://www.iamsterdam.com/en/living/everyday-essentials/waste-and-recycling/recycling-in-amsterdam">nine waste streams</a>, including batteries and cooking oils (in addition to the paper, plastic and glass streams us Americans are familiar with). Amsterdam has also devised a long-term <a href="https://www.circle-economy.com/case/developing-a-roadmap-for-the-first-circular-city-amsterdam/#.WyvDiBJKhTZ">roadmap</a> (<a href="https://www.circle-economy.com/wp-content/uploads/2016/04/Circular-Amsterdam-EN-small-210316.pdf">link to PDF</a>) to become “more circular” by focusing on recycling construction and high-value materials.</li><li><a href="https://industrialorganic.com/"><strong>Industrial/Organic</strong></a><strong>: </strong>Industrial/Organic is a <a href="https://hackernoon.com/tagged/startup">startup</a> trying to recover water, energy, and nutrients from organic waste. They’re currently operating a facility in Newark, NJ which uses both biological and mechanical process to turn food waste into cleaning products.</li></ul><p>There’s lots of exciting work going on to make cities more inclusive, data-driven, and sustainable. Most of the companies mentioned here are from the NYC area (where I’m currently based), so please feel free to share other relevant companies or initiatives in the comments or via <a href="https://twitter.com/Pranav_Badami">Twitter</a>.</p><p>If you found this overview interesting or helpful, please 👏 the article below and follow me (<a href="https://medium.com/u/9aa99fb1d5e7">Pranav Badami</a>) on Medium to read similar work in the future!</p><p>Similar articles by me:</p><ul><li><a href="https://medium.com/@pranavbadami/transittechiesnyc-a-quick-recap-of-the-inaugural-meetup-b7c3a81428f2">#TransitTechiesNYC: A quick recap of the inaugural Meetup</a>3</li></ul><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2Fdde502%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2Fhackernoon%2F&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/3c851dac986ab6dbb2d1aaa91205a8eb/href">https://medium.com/media/3c851dac986ab6dbb2d1aaa91205a8eb/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d2fdf43082de" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hackernoon/what-makes-a-city-smart-in-2018-d2fdf43082de">What makes a city &quot;smart&quot; in 2018?</a> was originally published in <a href="https://medium.com/hackernoon">HackerNoon.com</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[TrendFinder: Educational Trends from the Past Decade (guest post)]]></title>
            <link>https://medium.com/making-donorschoose-org/trendfinder-the-most-impactful-and-interesting-educational-trends-of-the-last-decade-d8bc1403e084?source=rss-9aa99fb1d5e7------2</link>
            <guid isPermaLink="false">https://medium.com/p/d8bc1403e084</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[education]]></category>
            <dc:creator><![CDATA[Pranav Badami]]></dc:creator>
            <pubDate>Tue, 05 Jun 2018 19:38:53 GMT</pubDate>
            <atom:updated>2018-06-05T20:19:30.209Z</atom:updated>
            <content:encoded><![CDATA[<p><em>How tech and furniture have changed the American classroom</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GeWOzQ-cJ8pRQ2KcurxJHg.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/photos/qESmLLXAmWs?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Patricia Prudente</a> on <a href="https://unsplash.com/search/photos/kids-ipad?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p><em>This blog post is part of a series of guest posts from the CKM Advisors Pro Bono Team recapping a recent 4-month collaboration, where CKM Advisors designed and developed a product for DonorsChoose.org called TrendFinder.</em></p><p><em>TrendFinder is a self-sufficient, interactive business intelligence dashboard built to detect and analyze trends on the DonorsChoose.org platform in real-time. In this article, we cover major educational trends we found using TrendFinder.</em></p><h3>What is TrendFinder?</h3><p>TrendFinder is a tool that can alert DonorChoose.org about trending words in their underlying data; it also includes several demographic and geographic analyses to break trends down and ask deeper questions. Here are some questions (and answers) TrendFinder can explore:</p><p><em>What words are trending across DonorsChoose.org data over the past two weeks?</em></p><p>TrendFinder displays 50+ trending keywords in a table, sorted by how trendy the word is.</p><p><em>If the word “insect” is trending, for example, we can dig deeper into this trend:</em></p><ul><li>What subjects or grade levels are primarily driving the “insect” trend?</li><li>Is the “insect” trend correlated with a recent spike in Google searches for the word “insect”?</li><li>Where is “insect” trending? In what cities or geographical regions? How has the “insect” spread geographically over time?</li></ul><p><em>We can also look at how “insect” has behaved historically:</em></p><ul><li>Has the number of “insect” projects been growing over time?</li><li>Is the “insect” trend seasonal?</li><li>Have multiple subjects been driving the “insect” trend over time, or just one (e.g. science)?</li></ul><p>In the process of building TrendFinder, we found lots of trends across many different types of classrooms. We asked questions like the ones above to see how large these trends were, as well as when and where they took place. Certain prominent trends kept coming up in our analyses and we wanted to share the most interesting and impactful insights we found.</p><h3>The “Big 3” Trends in Education</h3><p>The three biggest, unmistakable trends we discovered using TrendFinder on the DonorsChoose.org data (2008–2018) were the rise of iPads, Chromebooks and … wobble chairs. What is a wobble chair, you ask? Read on to find out, as we unpack these trends with TrendFinder.</p><h4>iPad: The Granddaddy of them All</h4><p>Without question, iPads were the most prominent trend we found over the last ten years of DonorsChoose.org data. As of 2018, iPad projects represent over 10% of all DonorsChoose.org projects since 2008. While this in itself may not come as a surprise (who hasn’t seen kids begging for their parents’ Apple devices?), we can use TrendFinder’s features to examine how the iPad’s presence grew in classrooms over time.</p><p>The following graph is an output of TrendFinder. It illustrates what fraction of projects in the DonorsChoose.org dataset, over time, contained the word “ipad” in various text fields:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/680/1*56eFfpVlH84xev-GIFBGeQ.png" /><figcaption>A trend proportion of 0.2 means 20% of projects contained “ipad” at that particular time.</figcaption></figure><p>Based on the graph, the initial release of the iPad in April 2010 didn’t make a massive splash on the DonorsChoose.org platform. The iPad really took off in classrooms with the debut of the iPad mini in late 2012. The iPad mini was the most affordable iPad option to date, which means that teachers were able to get more devices for students with a smaller donation amount. It’s interesting to note that requests for iPads have fallen in recent years, possibly due to cheaper options on the market.</p><p>Looking at demographic shifts over time in “ipad” projects reveals interesting insights. The following TrendFinder output visualizes how under or over represented a particular demographic is among “ipad” projects. The y-axis of the graph is labeled “ratio difference”; without getting too technical, the higher the ratio difference, the more overrepresented a demographic is in “ipad” projects.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/758/1*jEaZ7JYkkENclcXilDddNw.png" /></figure><p>There is a widening gulf between the grades PreK-2 (blue line) and grades 3–5 (pink line) demographics. Over time, iPads are becoming much more heavily represented in classrooms with the youngest kids (note: iPads are still big in Grades 3–5 relative to older kids). Is this because iPads are an easier technology to use than laptops? We may have a hunch that younger and younger kids are using technology, but it is entirely different to see it laid out in the DonorsChoose data.</p><h4>Chromebooks: An Affordable Classroom Computer</h4><p><a href="https://www.google.com/chromebook/">Chromebooks</a> are no-frills laptops that run Google’s ChromeOS. Chromebooks are sold at a lower price point than Macs or PCs , which means that teachers can get more Chromebooks for their classrooms than premium laptops with the same funding. The first wave of Chromebooks, released starting June 2011, didn’t gain very significant traction on the platform. However, around November of 2012, we see a massive spike in the proportion of Chromebooks being requested:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/696/1*mc-jG8B9VtqSl76j0FKBuA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RpUueXa57nznpiYONqNA4A.png" /><figcaption>Left: Trend proportion graph for “chromebook”, Right: Screenshot of “Chromebooks for Classrooms!” campaign</figcaption></figure><p>As it turns out, DonorsChoose ran a promotional campaign with Google called “Chromebooks for Classrooms” in December 2012. The promotion let teachers request Chromebooks on the platform for $99 each. This campaign had a massive impact; in the month of the promotion, over 25% of projects on the platform were Chromebook projects compared to less than 2% in the months before. Why was this growth difficult to sustain? This case is typical of the canonical “hype cycle”:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E2U2S0gOqYsu2yq4cMt-Xw.png" /><figcaption>Source: <a href="https://en.wikipedia.org/wiki/Hype_cycle">Wikipedia</a></figcaption></figure><p>Teachers, as a whole, may have needed some time to incorporate Chromebooks into their lesson plans and classrooms. After the initial success of the Chromebook promotion, we see sustained growth in Chromebook projects over the past 6 years.</p><h4>Wobble Chairs: Flexible Seating for Kids</h4><p>TrendFinder picked up a variety of high-proportion trends that we didn’t expect: words such as “wiggle”, “wobble”, and “flex”:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/808/1*txOZtvsErUaOUYlpmJgaPg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/216/1*pc6klCaFRy1_2cR1Qd2nuA.jpeg" /></figure><p>As non-educators, we had to do some research into these words in the educational context to realize that they all referred to flexible seating options, chairs that move and wobble so that children aren’t stuck fidgeting in rigid chairs. You can read more about flexible seating in education <a href="https://www.edutopia.org/blog/flexible-seating-student-centered-classroom-kayla-delzer">here</a>.</p><p>Interestingly, flexible seating seems to have taken off in some of the Southwestern cities before it grew as a trend nationwide. We can use TrendFinder’s geographical analyses features to see where the “wobble” trend, for example, really picked up steam. The following graph shows, at any given point, the fraction of “wobble” projects took place in the city of Dallas up to that point in time (cumulative proportion):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/554/1*_sZPvs7wAt43LPF9KWP9Qw.png" /></figure><p>There was a flurry of activity early in the life-cycle of the “wobble” trend in Dallas, which was even covered in <a href="https://www.dallasnews.com/news/education/2016/10/19/wobble-chairs-bouncy-balls-let-students-wiggle-work">local press</a>. Cities seen as trendsetters, such as New York and Los Angeles, lagged quite a bit behind Dallas and only caught on in 2017. TrendFinder’s geographical features present an interesting opportunity to spotlight and promote trends in unexpected areas.</p><h3><strong>Seasonal Trends: Falling down and Springing back up</strong></h3><p>iPads, Chromebooks, and flexible seating all exhibited some degree of seasonality, or repeating pattern, on a yearly basis. We can see a really clear example of seasonality when we look at basic resources, such as sheets and markers:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mmxDih72yFjQk7a4WzIlVg.png" /></figure><p>Teachers need to stock up on these basic supplies at the beginning of the school year, which explains why we see many more sheet and marker projects in the late summer. Similarly, school sports programs need equipment before their seasons start, which is reflected in when certain sports start trending on the DonorsChoose platform.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gMz3POFN3SG0dl_9Hm44qA.png" /><figcaption>Learn more about Google Trends <a href="https://trends.google.com/trends/?geo=US">here</a></figcaption></figure><p>The professional and youth football seasons begin in the fall, and football projects on DonorsChoose (green line) predictably rise along with the start of the season. TrendFinder lets us compare the behavior of “football” in the DonorsChoose data with its behavior in the Google Trends index (red line), which exhibits clear periodicity. Google Trends gives us a rough measure of how popular a keyword is at any given point on the broader internet.</p><h3>Pop-culture and education</h3><p>Often, pop cultural phenomena become hot topics of discussion on the internet and also start trending on the DonorsChoose platform. TrendFinder, with its Google Trend analysis feature, is a useful tool for identifying and making sense of keywords pertaining to popular books and movies.</p><p>The film <em>Bully</em>, for example, was a documentary following the lives of American students. It premiered globally in July 2011 and was released in the US in March 2012. TrendFinder identified “bully” trending for several months prior to the US release of the film. Even more interestingly, “bully” briefly trended again on DonorsChoose in late 2012; this timing corresponded to the DVD release of the movie.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/821/1*VICJE6gBoSSrq1bTr5ZgjQ.png" /></figure><p>Disney’s <em>Frozen</em> is perhaps the biggest children’s movie of the past five years. The word “frozen” has trended a few times, around the theatrical and DVD releases in the series. This isn’t necessarily surprising, but we found something really interesting looking at TrendFinder’s demographic breakdown for “frozen”:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bgkCIRmnIcF7rCEaD61kJg.png" /><figcaption>Ratio difference graph over time for “frozen”. Note: high y-value indicates overrepresentation.</figcaption></figure><p>Since 2015, teachers of the youngest classrooms (Grades PreK-2, blue line) were heavily overrepresented in “frozen” projects, but recently, things have changed. Grades 3–5 (orange line) have been requesting far more “frozen” projects, accompanied by a decline in Grades K — 2. This begs the question: <em>Are the kids who fell in love with Frozen growing up? Are these kids more receptive to Frozen-related content as 3rd through 5th graders?</em> Sometimes, TrendFinder poses more questions than it presents answers.</p><h3>Moving Forward</h3><p>Finding these thought-provoking observations was one of the most rewarding parts of building TrendFinder for the CKM Pro Bono Team. As non-educators, we definitely learned a lot about how trendsetting teachers are adopting innovative products for the classroom.</p><p>TrendFinder lives on as a deployed dashboard populated by a frequently running data analysis pipeline. Every week or two, TrendFinder will deliver an email full of the latest trending keywords to the awesome folks at DonorsChoose.org. And, every week or two, we hope DonorsChoose.org can use it to find something or someone interesting to prepare for, talk about, or connect with. After all, those chairs aren’t going to wobble themselves.</p><p><em>TrendFinder was a collaboration between DonorsChoose.org and the </em><a href="http://ckmadvisors.com/"><em>CKM Advisors</em></a><em> Pro Bono team. If you like this work, please support </em><a href="https://www.donorschoose.org/"><em>DonorsChoose.org</em></a><em> and their mission of building a future by supporting classrooms.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Fz6j6mja3RfYMLaVN6emHw.jpeg" /></figure><p><em>Made with ❤ by the CKM Advisors Pro Bono Team — </em><a href="https://medium.com/u/9aa99fb1d5e7"><em>Pranav Badami</em></a><em>, </em><a href="https://medium.com/u/c3bb24bdd66f"><em>Lorena De La Parra Landa</em></a><em>, </em><a href="https://medium.com/u/4949dbc31189"><em>Elya Pardes</em></a><em>, </em><a href="https://medium.com/u/5379b385a1eb"><em>Lex Spirtes</em></a><em>, and </em><a href="https://medium.com/u/f43daa5e7896"><em>Michael Zhang</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d8bc1403e084" width="1" height="1" alt=""><hr><p><a href="https://medium.com/making-donorschoose-org/trendfinder-the-most-impactful-and-interesting-educational-trends-of-the-last-decade-d8bc1403e084">TrendFinder: Educational Trends from the Past Decade (guest post)</a> was originally published in <a href="https://medium.com/making-donorschoose-org">Making DonorsChoose</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[#TransitTechiesNYC: A quick recap of the inaugural Meetup]]></title>
            <link>https://medium.com/@pranavbadami/transittechiesnyc-a-quick-recap-of-the-inaugural-meetup-b7c3a81428f2?source=rss-9aa99fb1d5e7------2</link>
            <guid isPermaLink="false">https://medium.com/p/b7c3a81428f2</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[new-york-city]]></category>
            <category><![CDATA[meetup]]></category>
            <category><![CDATA[transportation]]></category>
            <dc:creator><![CDATA[Pranav Badami]]></dc:creator>
            <pubDate>Thu, 24 May 2018 21:09:21 GMT</pubDate>
            <atom:updated>2018-05-24T21:09:21.281Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A quick recap of the first </em><a href="https://www.meetup.com/Transit-Techies-NYC/"><em>Transit Techies NYC Meetup: Christening the Code Train</em></a><em>, held on Wed, May 23, 2018</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C6-6l5uUuiMpnyVDbLUCpw.jpeg" /><figcaption>Taken from Unsplash… Just kidding, this is the view from <a href="https://www.intersection.com/">Intersection</a> taken by me.</figcaption></figure><p>Last night, coders, transit geeks, and transit-geeks-turned-coders alike descended upon Hudson Yards for the first <a href="https://www.meetup.com/Transit-Techies-NYC/">Transit Techies NYC Meetup</a> held at <a href="http://intersection.com/">Intersection</a>. Beside the stunning view from Intersection’s office and a sizable pile of pizza boxes, the Meetup featured a really cool lineup of speakers covering a wide array of projects relating to transit and technology. <a href="https://medium.com/u/f43daa5e7896">Michael Zhang</a> and I have been <a href="https://towardsdatascience.com/the-5-stages-of-a-system-breakdown-on-nj-transit-8258127e31e9">blogging about data and NJ Transit</a> recently and thought this Meetup would be a great place to learn about interesting work and people in the transit space. Here’s a quick recap of the event!</p><h3><strong>Introduction</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OaU7uVwKoTEIaj8btezbgw.jpeg" /></figure><p>When we got to Intersection, there were already 30–40 techies already in the space hanging out and chatting. Intersection shares a multi-level space with Alphabet’s Sidewalk Labs so some Sidewalkers (is this a thing?) were also in the house. There were probably around 80 people at the event by the time it started, which is really encouraging for the Meetup moving forward.</p><p>Organizers <a href="https://twitter.com/kohzy">Kozhy Koh</a> and <a href="https://twitter.com/greent_tyler">Tyler Green</a> got things started with a humorous Powerpoint; with that, the first Transit Techies NYC Meetup was underway! Four speakers were scheduled to present their transit tech projects to the group.</p><h3><strong>TransitFlow</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BeIjr1mhVew8bSts2TKsiA.jpeg" /><figcaption>Will Geary talking about TransitFlow</figcaption></figure><p><a href="https://twitter.com/wgeary">Will Geary</a> took the floor first to discuss his open-source <a href="https://github.com/transitland/transitland-processing-animation">TransitFlow</a> project. The project takes General Transit Feed Specification (GTFS) data from transit agencies and produces animated transit maps using <a href="https://processing.org/">Processing</a>, a coding language for the visual arts.</p><p>Timetables are the primary means of communicating transit information and they are decidedly not visual; they’re often confusing and it’s difficult to tell what times of the day have more or less train activity. TransitFlow tries to make transit communication more visual and intuitive by animating trains; I wish I’d found it before my <a href="https://towardsdatascience.com/the-5-stages-of-a-system-breakdown-on-nj-transit-8258127e31e9">previous article</a> where I animated the NJ Transit system…using Plotly.</p><p>Will also shared Rami Saab’s super interesting <a href="https://github.com/rami-codes/transitland-matlab-audiolizer">“audiolizer” project </a>which uses TransitFlow to produce music from transit schedules.</p><h3><strong>GoPhillyGo</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/846/1*WevrAFPbKi6dCZ44TrRaSw.png" /><figcaption>Screenshot from a GoPhillyGo <a href="https://www.youtube.com/watch?time_continue=26&amp;v=RwmeD3Hv9w8">YouTube video</a></figcaption></figure><p><a href="https://twitter.com/Banderkat">Kathryn Killebrew</a> was up next to talk about <a href="https://gophillygo.org/">GoPhillyGo</a>, a multi-modal, non-car, transit planning tool for residents of Philadelphia. Given two locations, GoPhillyGo tells you how to get from point A to point B without using a car. With clear bike, transit, and bike+transit routing, Philadelphians don’t have to fumble through different modes of Google Maps on the go when they leave their cars behind.</p><p>GoPhillyGo takes a really interesting community-inspired approach to its website as well. It features local events in the Philly area, which users can then click on and see how to get to without using a car. With bike share, electric skateboards, and electrical scooters becoming more mainstream, I can see a larger need for people wanting to combine alternative personal and public transportation options to go carless. GoPhillyGo’s routing is done using <a href="https://github.com/opentripplanner/OpenTripPlanner">OpenTripPlanner</a>.</p><h3><strong>Citibikes in the Outer Boroughs</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C6lRTtKmLQ-oxWWEWkfWhg.jpeg" /><figcaption>Concluding slide of Jiaxu Zhao’s presentation</figcaption></figure><p><a href="https://www.linkedin.com/in/jiaxu-zhou/">Jiaxu Zhou</a>, a former Masters student at NYU, talked about using data to predict Citibike usage in the outer NYC boroughs. Citibike is <a href="http://nycdotbikeshare.info/maps_plans">expanding its coverage</a> in 2018 and Jiaxu’s work (conducted in 2015–16) measured potential usage of new stations in the outer boroughs, notably Brooklyn. Transportation availability and demographic features are combined to quantify how much a census block group will utilize a proposed Citibike dock nearby. The potential usage is color mapped and plotted on a map using a GIS library.</p><p>This project was really interesting from both a technical and societal perspective. While data was used to model and tackle a difficult problem, the project also touches on the inclusiveness and equity of new-age transit options in cities.</p><h3><strong>Transit Explorer by Sidewalk Labs</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/982/1*7rcbsstXhNdVcBz0RG0alg.png" /><figcaption>Screenshot from <a href="https://transit.sidewalklabs.com/">Transit Explorer</a></figcaption></figure><p><a href="https://twitter.com/danvdk">Dan Vanderkam</a> was up last to discuss <a href="https://transit.sidewalklabs.com/">Transit Explorer</a>, a tool built by <a href="https://www.sidewalklabs.com/">Sidewalk Labs</a> to help New Yorkers figure out how long it takes to get to regions of the city via public transit. Transit Explorer uses isochrones, or maps that are colored according to values, to show you travel times to different parts of Manhattan from your apartment in, say, Williamsburg.</p><p>With the impending closure of the L train connecting Manhattan and Brooklyn (the ominous “L-maggedon”), TransitExplorer has allowed users to remove the L train and see how badly the <a href="https://medium.com/sidewalk-talk/new-map-demo-how-the-l-train-shutdown-will-impact-your-commute-6a1dc74f65f5">closure will impact them</a>. Sidewalk Labs is currently building a revamped transit explorer for Toronto <a href="http://totx.sidewalklabs.com/">here</a>.</p><p>All in all, the first Transit Techies NYC Meetup set the stage nicely for the community to grow in the future. I’m interested in experimenting with (and possibly contributing to) some of the work I learned about, such as TransitFlow. We talked about some super cool projects, as well as new public initiatives and technologies; I’m really looking forward to the next one! You can join the techies on <a href="https://www.meetup.com/Transit-Techies-NYC/">Meetup</a>.</p><p><em>If you enjoyed this recap, please 👏 this article below. I’ll try to make it to more transit and sustainability related Meetups in the coming months, so follow my Medium profile to stay posted on future recaps!</em></p><p><em>Here are some transit tech-y articles I’ve written recently:</em></p><ul><li><a href="https://towardsdatascience.com/the-5-stages-of-a-system-breakdown-on-nj-transit-8258127e31e9"><em>The 5 Stages of a System Breakdown on NJ Transit</em></a></li><li><a href="https://medium.com/@pranavbadami/how-data-can-help-fix-nj-transit-c0d15c0660fe"><em>How Data can Help Fix NJ Transit</em></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b7c3a81428f2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The 5 Stages of a System Breakdown on NJ Transit]]></title>
            <link>https://medium.com/data-science/the-5-stages-of-a-system-breakdown-on-nj-transit-8258127e31e9?source=rss-9aa99fb1d5e7------2</link>
            <guid isPermaLink="false">https://medium.com/p/8258127e31e9</guid>
            <category><![CDATA[public-transit]]></category>
            <category><![CDATA[new-jersey]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[towards-data-science]]></category>
            <dc:creator><![CDATA[Pranav Badami]]></dc:creator>
            <pubDate>Tue, 17 Apr 2018 13:05:07 GMT</pubDate>
            <atom:updated>2018-04-17T13:05:07.889Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This article is part of an ongoing collaboration between </em><a href="https://medium.com/u/f43daa5e7896"><em>Michael Zhang</em></a><em> and me analyzing real-time NJ Transit data that we’ve been scraping. You can find the introductory article </em><a href="https://medium.com/@pranavbadami/how-data-can-help-fix-nj-transit-c0d15c0660fe"><em>here</em></a><em> and the previous article </em><a href="https://medium.com/@mzhang13/what-are-the-chances-that-nj-transit-will-cause-you-to-miss-the-dinky-bfeacd11ebc6"><em>here</em></a><em>.</em></p><p>On Friday, March 2nd, 2018, New York City was on the tail end of the <a href="https://en.wikipedia.org/wiki/March_1%E2%80%933,_2018_nor%27easter">first of four Nor’easters</a> to sweep through the region in March. A wintry mix had been steadily falling from the previous night through the early afternoon, accompanied by gusty winds in the late morning hours.</p><p>As I wrapped up my day at work and headed to New York Penn Station for my commute home, I became increasingly worried as texts from friends and family informed me that NJ Transit service was heavily backed up.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*ejsh-XmlVVPcqTewGxZjeA.jpeg" /><figcaption>An ominous screenshot of Departure Vision for New York Penn at 4:39 pm on 3/2/18. Note the “STATUS” column.</figcaption></figure><p>Nothing could have prepared me for my actual arrival at Penn Station, where a literal mass exodus of frustrated commuters flooded out onto 7th Avenue. The NJ Transit system had completely broken down and 9 trains were simultaneously cancelled, followed by several more slightly after, due to weather related complications.</p><p>While these sorts of massive, system-wide breakdowns are rare, they leave a dramatic imprint on the memories of thousands of commuters who had their entire day or weekend ruined. Let’s take a look at the impact of this service breakdown: the following plots show how delayed each of the NJ Transit trains scheduled in and out of New York Penn Station was on March 2nd.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-ITgtGXwmKVr1q_0x4ApXw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UqL7KYF3kubtV0fjmLnxRQ.png" /><figcaption>Each dot is a train colored by delay severity. Note the symmetry in volume about hour 14 or 15. (Left: Inbound trains into New York Penn, Right: Outbound trains from New York Penn)</figcaption></figure><p>The lateness of trains into New York Penn is determined by their cumulative delay up to New York Penn (their last stop), while the lateness of trains out of New York Penn is determined by their delay getting to their first stop. Of the 32 trains scheduled to depart from New York Penn between 5 pm and 7pm, 18 trains were cancelled and several others heavily delayed.</p><p>How did the system end up in this state? What happened to cause this and what were the resulting system dynamics that resulted in this breakdown? We can examine the data to visualize how the system reacts to an interruption and identify distinct phases that lead to system breakdowns.</p><h3>Visualizing the NJ Transit system in action</h3><p>For the uninitiated, the NJ Transit system is a non-cyclical rail network that services 162 train stations in New Jersey and connects travelers to New York Penn Station. As a reference, here is a zoomed in system rail map, capturing the stations around New York Penn Station, including Newark Penn Station, Secaucus Junction (Upper Level), Newark Broad St, and Hoboken:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1011/1*LsNXlx12Ar6BAV4hhAYFBQ.png" /><figcaption>Abbreviated NJ Transit system map. Pink inset: The stations we will focus on. Note the overlap at Secaucus Junction, where the lines crossing above stop at Secaucus Upper Level; the lines crossing below stop at Secaucus Lower Level.</figcaption></figure><p>As a result of many line intersections and high volume to New York Penn Station, this region of NJ Transit operations has complex system dynamics that affect tens of thousands of people everyday.</p><p>We’ll investigate the March 2 system breakdown by focusing on this stretch of track because the catalyzing service interruption (covered later) occurred between Newark Penn Station and Secaucus Junction; also, the high volume of trains will demonstrate system dynamics at play.</p><h3>Phase 1: The Calm Before (or During) the Storm</h3><p>Let’s begin our analysis by looking at the state of the system on the morning of March 2, as it dealt with persistent freezing rain starting the night before. The system found some stability immediately following the morning rush hour, after dealing with a handful of minor service interruptions earlier in the morning due to broken down trains and Amtrak track maintenance.</p><p>The following animation shows the behavior of the system as scheduled on the left versus the actual observed behavior on the right. Stop-level train data gathered by our scraper is converted into a stateful data structure, from which train positions are then interpolated and animated.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-l1QTFt380iJYhteR6SN6g.gif" /><figcaption>System animation at 10 am. Each dot here corresponds to a train. Left: Scheduled trains, Right: Observed trains. Stations: NWP (Newark Penn), SEC (Secaucus Upper Level), NYP (New York Penn), ND (Newark Broad St), HOB (Hoboken)</figcaption></figure><p>Trains are traveling at roughly the same average velocity between stops. Further, we see that trains are departing out of the New York Penn origin very close to their scheduled departure time. For the moment, the system is stable.</p><h3>Phase 2: Something Breaks</h3><p>As the day progressed into the early afternoon, precipitation from the storm began to tail off and was replaced by gusty winds. Then, this happened:</p><h3>NJ TRANSIT - NEC on Twitter</h3><p>NEC train #3853, the 3:26pm from NPS, is delayed due to an Amtrak overhead wire problem. Update to follow.</p><p>An overhead wire between Newark Penn and Secaucus Junction went down in the early afternoon. Overhead power lines that supply electricity to trains along their journey are one of the more vulnerable parts of fixed rail infrastructure, as they are susceptible to damage from high winds and constant wintry precipitation.</p><p>In this case, we see that winds really picked up in the region around the time the wire went down. Here’s the wind data for Newark, NJ on March 2:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OCCzDunEE4iNgLThETsiVg.png" /><figcaption>Wind speed in Newark on 3/2/18 (courtesy of the Time Machine on darksky.net)</figcaption></figure><p>Despite the tweet above being timestamped 4:47 pm, several trains were cancelled earlier in the afternoon due to inclement weather; for the purpose of this analysis, we’ll assume that the wire went down between 1 pm to 2 pm. This down wire caused only a single track to be available for a limited stretch between Newark Penn and Secaucus (according to announcements I heard from NJ Transit). Let’s see how the affected part of the system reacted to this service interruption moving forward.</p><h4><strong>Phase 3: Queueing</strong></h4><p>The early afternoon is generally a relatively slow period of the NJ Transit service cycle but, as the late afternoon approaches, the number of trains both into and out of New York Penn starts growing rapidly. As a simplifying assumption, we can state that enough trains need to arrive at New York Penn in the late afternoon such that they can be reused to service the evening commute out of New York Penn. The exact configuration of these trains is unclear, but we see that the schedule aims to maintain an equal number of trains going into and out of the terminal.</p><p>On March 2, however, due to the downed wire between Newark and Secaucus, trains in either direction between these stations had to share a single track. Trains slow to a crawl heading towards New York Penn as they are held in a queue, while earlier trains use the single track.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*byXAB5eDdUmI-HHl_qp4VA.gif" /><figcaption>System animation at 4 pm. Note the queueing trains heading towards NWP at 4:00 pm.</figcaption></figure><h4><strong>Phase 4: Cancellations</strong></h4><p>Let’s examine the effect of prior service interruptions and queueing on the number of trains that have arrived at New York Penn. To do this, we’ll look at the difference in cumulative number of trains arrived into New York Penn versus the cumulative number of trains that were scheduled to depart out of New York Penn up to any given point. In effect, this is the number of trains available at New York Penn to service the scheduled outgoing demand at any given point. Again, this may be a simplifying assumption of how the system works, but it captures its general behavior.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D0Xkrq3ci8LWT5S7a0Ve2A.png" /><figcaption>The cumulative difference between how many trains have arrived to New York Penn, versus how many have been scheduled to go out. The “actual” line incorporates observed delays and cancellations on 3/2/18.</figcaption></figure><p>In the graph above, when the “scheduled” (blue) line diverges from the “actual” (orange) line and recovers later in the day, this indicates delays caused by queueing. The “actual” line shows us that, at 6:30 pm on March 2, the number of trains scheduled to depart from New York Penn eclipsed the number of trains that had arrived at New York Penn. Also, cancellations earlier in the day are reflected by a sharp dip in the number of reserve trains around 2 pm.</p><p>If all had gone according to schedule, 9 trains would have been available at New York Penn at 6:30 pm to service the tail end of the commute. Instead, earlier cancellations and queuing between 4pm and 6pm made the system fall severely behind. At precisely 6:29 pm, our scraped data shows that NJ Transit simultaneously cancelled, you guessed it, 9 trains out of New York Penn. The system appears to depend on having enough trains in “reserve” at a terminal to meet outgoing demand.</p><h4><strong>Phase 5: Resolution</strong></h4><p>This bulk cancellation at 6:30 pm was precisely when I found myself entering Penn Station on March 2. While I can empathize with how frustrated commuters were with the cancellations, I see now that NJ Transit physically might not have had enough trains in the yard at Penn Station to service the complete evening schedule. The bulk cancellations at 6:30 pm essentially served as a mechanism for the system to catch-up to outgoing demand.</p><p>Later in the evening, queueing is still present and there’s a lot of residual train traffic in the affected stretch of track, but trains are going both into and out of New York Penn again. We can visualize the system around 8:30 pm to see the sluggish recovery:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*kiq2leXSYI-6aOdMewVZIw.gif" /><figcaption>System animation around 8:30 pm.</figcaption></figure><h4><strong>Takeaways</strong></h4><p>This major service interruption impacted the performance of the system from roughly 12 pm to 9 pm on March 2. Thousands of commuters were stranded and had to find alternate ways to get home after the bulk cancellations.</p><p>Analyzing the system response after the downed wire between Newark and Secaucus reveals perhaps why there was a lack of available trains at New York Penn Station. Based on incoming supply and outgoing demand, it was apparent that there simply weren’t enough trains at least an hour or so before the bulk cancellations were officially announced.</p><p>I have lots of sympathy for NJ Transit in these situations. They are under tremendous pressure from factors not in their control, namely the weather, Amtrak infrastructure, and a lack of funding. As a starting point, here are a couple things I think NJ Transit could do to mitigate the frustrations of commuters when it comes to large scale delays:</p><ul><li><strong>Severe weather scheduling</strong>: While the Nor’easter was forecasted to reduce in precipitation intensity on March 2, perhaps NJ Transit could have opted to use their Severe Weather Level 1 alternate schedule due to forecasted high winds.</li><li><strong>Warn commuters of a looming breakdown</strong>: The NJ Transit Twitter sends automated tweets when trains are cancelled or delayed heavily. In a case such as March 2, the Twitter page could warn customers of degrading conditions system-wide and encourage travelers to make different plans. It’s not clear that bulk cancellations are about to happen from tweets about individual trains.</li><li><strong>Provide commuters a way to look at the state of the system as a whole</strong> (similar to the animations in this article): Commuters would be aware of the magnitude of the problem and have a realistic sense of how long it will take to resolve.</li></ul><p>In any case, it’s clear that infrastructure improvements are much needed, especially in crucial, high-volume stretches of track. Without funding and more resilient infrastructure, we can expect these 5 phases to repeat themselves time and again.</p><p><em>If you enjoyed this bit of analysis, please 👏 this article below. Also, follow my Medium profile, as well as </em><a href="https://medium.com/u/f43daa5e7896">Michael Zhang</a><em>, to get access to this data (soon!) and read more NJ Transit analysis in the coming months! Thank you!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8258127e31e9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/the-5-stages-of-a-system-breakdown-on-nj-transit-8258127e31e9">The 5 Stages of a System Breakdown on NJ Transit</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How data can help fix NJ Transit]]></title>
            <link>https://medium.com/@pranavbadami/how-data-can-help-fix-nj-transit-c0d15c0660fe?source=rss-9aa99fb1d5e7------2</link>
            <guid isPermaLink="false">https://medium.com/p/c0d15c0660fe</guid>
            <category><![CDATA[new-jersey]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[transportation]]></category>
            <category><![CDATA[public-transit]]></category>
            <dc:creator><![CDATA[Pranav Badami]]></dc:creator>
            <pubDate>Mon, 05 Mar 2018 12:27:42 GMT</pubDate>
            <atom:updated>2018-03-18T19:11:20.058Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This article is the first of an ongoing collaboration between </em><a href="https://medium.com/u/f43daa5e7896">Michael Zhang</a> <em>and I using data to examine the performance of NJ Transit. You can find the second article by Michael </em><a href="https://medium.com/@mzhang13/what-are-the-chances-that-nj-transit-will-cause-you-to-miss-the-dinky-bfeacd11ebc6"><em>here</em></a><em>. If you’re interested in more like this, or getting your hands on the data, please follow me to get notified of new articles in the future. Thanks.</em></p><p>I commute to New York City everyday on NJ Transit. At times, negotiating the trade off between packing into an overflowing train or possibly being late to work leads to moments of peak anxiety. But then, I step back and thank myself that I’m not in one of the infinite timelines where I’m <a href="https://www.youtube.com/watch?v=1eAP-BoxVkg">commuting around Mumbai</a>. Thank you, parents, for moving to New Jersey.</p><h4><strong>NJ Transit in February 2018</strong></h4><p>In any case, as far as public transportation in America is concerned, NJ Transit is one of the largest commuter rail networks in the country. With over <a href="https://en.wikipedia.org/wiki/List_of_United_States_commuter_rail_systems_by_ridership">88 million annual riders</a>, NJT is sandwiched in second place between the MTA’s Long Island Rail Road and Metro North.</p><p>Despite so many constituents depending on the system, NJT has fallen into disrepair over the past decade. Years of neglect have turned NJT into an unpredictable and unreliable system to the thousands of commuters that use it everyday. I’d like to use data to identify issues with NJ Transit and provide potential solutions to improve rail performance.</p><p>The only thing is… I couldn’t find any particularly helpful public data to use for my analysis. NJ Transit provides aggregate data that simply classifies trains as “on-time” or “late”. I was looking for granular (train-level), high resolution (~1–2 min accuracy) records to dive in to. So, I wrote a program that’s been scraping and parsing NJT and Amtrak data for the past month. I’ll be writing an explanatory post on how this works and releasing the scraped data openly in the coming weeks.</p><p>Every month or so, NJT publishes <a href="http://www.njtransit.com/AdminTemp/rail_otp.pdf">summary statistics</a> about the high-level performance of the system. In January 2018, 87.5% of trains were “on time”, which seems pretty rough right off the bat:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UkK2Jej5jm5caBEmHT5wwA.png" /><figcaption>Fig 1. Source: <a href="http://www.njtransit.com/AdminTemp/rail_otp.pdf">NJ Transit</a>. Their goal is 94.7% on time performance.</figcaption></figure><p>Sure, it sounds reasonable that about 1 in 8 trains is a bit late due to some issue or the other. What you may not realize, however, is that NJT defines “on time” as trains that arrive at their final destination <em>within 6 minutes</em> of their scheduled time. To really gain an understanding of system performance, I’m going to use the data set I’ve been collecting for the past month, which captures train performance on a stop-by-stop level with a resolution of ~1–2 minutes.</p><p>First, let’s define a more reasonable on-time metric and see how delayed trains have really been. Here’s my “actual train delay” number line:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SGiHuw1JsjTHdbqswEibLQ.png" /><figcaption>Fig 2. I think it’s reasonable…</figcaption></figure><p>How does NJ Transit stack up against this scale? Let’s take a look at system-wide delay this past month (Feb 5 — Mar 4). Each bar here tracks what percent of trains on a given day fall into the categories on our “actual train delay” scale:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EUTD7W5AdDImlrOJlzoT-g.png" /><figcaption>Fig 3. Reference line at 80%. Weekends/holidays marked with * . Interactive Plotly <a href="https://plot.ly/~pranavbadami/17/">link</a>. Note: Excludes Princeton Shuttle.</figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/370ca243a5ce2fc3f53655765d4f6063/href">https://medium.com/media/370ca243a5ce2fc3f53655765d4f6063/href</a></iframe><p>Of the approximately 16,267 NJ Transit trips made, 74.9% of trains were on time across the entire system. In the past month, the stacked bar chart (Fig. 3) reveals that there were only six days where &gt; 80% of trains were on time.</p><h4><strong>The Rush Hour Commute</strong></h4><p>While this is bad enough in it’s own right, system-wide performance includes trains at all hours of the day across all NJ Transit lines. Most passengers aren’t taking trains in the wee hours of the morning; no, they’re commuting to and from New York City during peak commuting hours. Let’s look at the performance of trains scheduled to arrive at New York Penn station within the morning rush hour (8 am — 10 am):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*92iSAbR0VPa9g13sorZmUA.png" /><figcaption>Fig 4. Reference line at 50%. Interactive Plotly <a href="https://plot.ly/~pranavbadami/13/">link</a>.</figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/230840b14a89a24d5fc0ed233a9b6641/href">https://medium.com/media/230840b14a89a24d5fc0ed233a9b6641/href</a></iframe><p>Approximately 43% of rush hour trains into New York Penn Station were delayed in the month of February. Further, the stacked bar chart (Fig. 4) indicates that the system failed to deliver 50% of trains on-time to Penn Station on seven days. Let’s try putting these numbers into perspective. <strong>In a given five day work week, you have a 94% chance of being delayed into New York Penn at least once</strong>. The likelihood of you experiencing a delay of greater than 6 minutes in a week of commuting is 73%. Perhaps most telling of all, over 2 weeks of commuting, you have a 77% chance of experiencing at least one delay of greater than 10 minutes. Basically, if you commute on NJ Transit to NYC, you’re gonna be late… and you’re gonna be late often.</p><p>Another important time for riders is the rush hour commute home after work. Is the system serving commuters going out of NYC better than it serves them going in? Here’s the performance of evening trains departing NY Penn between 4pm and 7pm:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VF8UAu5t1YjCumbGmWwWog.png" /><figcaption>Fig 5. Reference line at 50%. Interactive Plotly <a href="https://plot.ly/~pranavbadami/15/">link</a>.</figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/35aa68bef70314ebe3991e6f97776a0f/href">https://medium.com/media/35aa68bef70314ebe3991e6f97776a0f/href</a></iframe><p>On the bar chart (Fig. 5), we see that on-time performance dips below 50% on 11 of the 19 work days this past month. While this could indicate that performance may be worse during the evening, over 40% of trains are only slightly delayed. In fact, only 14.8% of trains are delayed for longer than 5 minutes leaving NY Penn in the evening, compared to 23% for inbound morning trains. It seems that longer delays are more prevalent among morning trains, while evening trains are being held for a few extra minutes before departure.</p><p>Then, there’s that massive gray bar on March 2 (Fig. 5) when a Nor’Easter blew through the region and brought the system to a standstill. That evening, 18 of 32 trains out of NY Penn got cancelled and many others were incredibly delayed.</p><p>This is why commuters are so frustrated with NJ Transit. The majority of riders are using NJ Transit precisely when it is most prone to getting delayed. Visualizing rush hour performance across the past month displays how vulnerable the system is to service interruptions. Next time, we’ll dive deeper into the March 2 service interruption as a case study of how the system breaks down causing delays to pile up.</p><h4><strong>Where do we go from here?</strong></h4><p>This article is a preliminary, descriptive exploration into the performance of the NJ Transit system. My end goal is to integrate various other streams of data in an ongoing effort to identify the most major issues plaguing the system. Here’s a preview of analyses that can further describe weaknesses in the system, possibly predict failures of the system, and hopefully prescribe fixes to improve performance:</p><ol><li><strong>Weather data (predictive)</strong> may reveal how different conditions can affect and predict delays, as well as identifying factors such as high winds or precipitation that lead to system failures.</li><li><strong>Service advisories (prescriptive)</strong> can be text mined to see what type of infrastructure issues are consistently leading to delays and where. NJ Transit could use this information to improve resource allocation for repairs.</li><li><strong>Amtrak data (descriptive, predictive) </strong>can provide a more precise understanding on how the two systems interact with each other. For example, we may find a threshold number of concurrently running trains over which delays are more likely.</li></ol><p>Beyond this, it’s personally interesting for me to use this data to help frustrated commuters understand how the system can be fixed moving forward. I think public transit is a great service that makes our daily lives more sustainable, and I’d like to do what I can to help fix my local transit network.</p><p><em>If you enjoyed this bit of analysis, please 👏 this article below. Also, follow my Medium profile, as well as </em><a href="https://medium.com/u/f43daa5e7896">Michael Zhang</a><em>, to get access to this data (soon!) and read more NJ Transit analysis in the coming months! Thank you!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c0d15c0660fe" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>