<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ram Avni on Medium]]></title>
        <description><![CDATA[Stories by Ram Avni on Medium]]></description>
        <link>https://medium.com/@ram.avni?source=rss-f09377c2c098------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*gm3obf6HAoNsTOh6us00hg.jpeg</url>
            <title>Stories by Ram Avni on Medium</title>
            <link>https://medium.com/@ram.avni?source=rss-f09377c2c098------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 14:26:35 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ram.avni/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Advances in AI Boost Diabetes Care Quality]]></title>
            <link>https://medium.com/@ram.avni/advances-in-ai-boost-diabetes-care-quality-be4811eac513?source=rss-f09377c2c098------2</link>
            <guid isPermaLink="false">https://medium.com/p/be4811eac513</guid>
            <category><![CDATA[diabetes]]></category>
            <category><![CDATA[closed-loop]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[closed-loop-system]]></category>
            <category><![CDATA[naïve-bayes]]></category>
            <dc:creator><![CDATA[Ram Avni]]></dc:creator>
            <pubDate>Tue, 24 Sep 2019 22:45:37 GMT</pubDate>
            <atom:updated>2019-09-24T22:45:37.367Z</atom:updated>
            <content:encoded><![CDATA[<p>As wearable technology is becoming more popular, diabetic medical equipment is catching up — many devices are getting smaller, more sophisticated, and more comfortable to wear. This, in turn, increases the <strong>amount of digitized health data</strong> from diabetic populations, and coupled with Artificial Intelligence, Artificial Pancreas, and personalized medicine, opens up a new page in diabetes management.</p><p>This new outlook aims to deliver tailored care and medical decisions to individuals. However, the inclusion of machine learning in decision making has ethical ramifications, such as the release of personal data; for instance, a frequent use of health apps and the potential use of tools based on AI by insurance companies could lead to discrimination or the exclusion (or both) of some citizens from health services.</p><p><strong>A Primer to Diabetes</strong></p><p>There are three basic types of diabetes:<br>1- Type I<br>2- Type II<br>3- Gestational diabetes</p><p><strong>Type 1</strong> diabetes (T1D) results from the pancreas’s failure to produce enough insulin due to loss of beta cells. This form was previously referred to as “insulin-dependent diabetes mellitus” (IDDM) or “juvenile diabetes”. The cause is unknown.</p><p><strong>Type 2</strong> diabetes begins with insulin resistance, a condition in which cells fail to respond to insulin properly. As the disease progresses, a lack of insulin may also develop. This form was previously referred to as “non insulin-dependent diabetes mellitus” (NIDDM) or “adult-onset diabetes”. The most common cause is a combination of excessive body weight and insufficient exercise.</p><p><strong>Gestational</strong> diabetes is the third main form, and occurs when pregnant women without a previous history of diabetes develop high blood sugar levels.</p><p><strong>Long term effects:</strong></p><p>Since blood is flowing through every organ in the body, poor care of diabetes can potentially have severe effects everywhere, from your toes to your kidneys and eyes. However, with disciplined monitoring and measurement, it is possible to stay absolutely healthy and active.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*31xDbZQ6OMCTxg4jzoSi5A.png" /></figure><p><strong>Typical Blood Glucose Range:<br></strong>Pre-meal 80–100<br>Post-meal 120–150<br>Long-term Monitoring: <strong>HbA1c</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/950/1*sOpu90dSwPD-4N1hop_3Zg.png" /></figure><p><strong>How do you treat diabetes?</strong></p><p>This blog will focus on T1D. Besides a good healthcare system, a sufficient level of education on available treatment, and self discipline, the following devices and drugs are fundamental and are used on a continuous, daily basis:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/225/1*aDFSiDN5PgXSheHYZiEhvw.jpeg" /></figure><p><strong>Glucometer</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/275/1*K-lm42YS3SuE4RZbN9a6fw.jpeg" /></figure><p><strong>CGM</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/275/1*JU1050OoQ56ZOAQZYLsytA.jpeg" /></figure><p><strong>Insulin pump</strong></p><p><strong>Open-loop and closed-loop systems</strong></p><p>An open-loop system consists of a GCM and an insulin pump, and the person has to manually calibrate the pump to deliver the amount of required insulin. On the other hand, a closed-loop system integrates artificial intelligence, so the insulin pump “talks” to the GCM and can calibrate automatically how much insulin to dispense in real time.</p><p><strong>Efforts and advances in the application of AI methodologies to diabetes management and decision support</strong></p><p>AI is attracting increased attention in this field because the amount of data acquired electronically from patients suffering from diabetes has grown exponentially. By means of complex and refined methods, AI has been shown to provide useful management tools to deal with these incremental repositories of data. Thus, AI has played a key role in the recognition of these systems as routine therapeutic aids for patients with diabetes.</p><h4>The most important families of techniques are artificial neural networks (ANNs), support vector machines (SVMs), random forest (RF), evolutionary algorithms (EAs), deep learning, Naïve Bayes, decision trees, and regression algorithms.</h4><p><strong>What does it take to predict?</strong></p><p>There are numerous factors that affect one’s blood sugar. Here are some factors that can be considered as features in building a useful model:</p><p><strong>Features<br>- </strong>Physiological factors<br>- Meal intake<br>-Current BG level<br>- Last meal<br>- Meal pattern — range last to next 12 hrs<br>- Physical activity<br>- Stress<br>- General health (blood pressure, overweight, other conditions etc.)<br>- Mental / emotional — anxiety (adrenaline, caffeine etc.)<br>- Fatigue</p><h3>Dividing AI’s impact into specific categories</h3><h3>Blood Glucose Control Strategies</h3><p>Development of the artificial pancreas (AP) has been intensively pursued over the past decade. An AP consists of an automated system that mimics islet physiology, including a glucose sensor, a closed-loop control algorithm, and an insulin infusion device. The ultimate goal of an AP system is to improve overall diabetes management and reduce the frequency of life-threatening events associated with T1D. The algorithms used by the AP to calculate insulin dosage have been intensively investigated, either using data from diabetic patients or computer simulated patients, commonly named virtual patients (VP).</p><h3>Blood Glucose Prediction</h3><p>The ability to anticipate BG excursions could provide early warnings regarding ineffective or poor treatments. Thus, information collected from new technologies for diabetes management, such as the CGM devices, could lead to real-time predictions of future glucose levels. Prediction of BG levels is challenging due to the number of physiological factors involved, such as delays associated with absorption of food and insulin, and the lag associated to measurements in the interstitial tissue. Errors of the CGM also increase the difficulty of predicting BG values (approximately 9% of the mean absolute relative difference for the best sensors [<a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6000484/#ref38">38</a>]).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fuXEg6nR3Z94dna18zZMvg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/364/1*MxB90M6xlFLjqC6DI9y2_A.png" /></figure><h3>Detection of Adverse Glycemic Events</h3><p>As with BG prediction, glycemic episode detection encompasses a set of tools that deal with the complexity of effective BG control. However, in this section we will not address glucose values, but instead focus on the appearance of hyperglycemic or hypoglycemic events. These tools enable us to detect the occurrence of glycemic episodes and give us the opportunity to respond promptly to their effects.</p><p>S<strong>ensitivity and specificity</strong> were the most common outcomes used to assess the quality of approaches to glycemic detection.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Zma-oq60pZDp1VIHAwYZ9Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/560/1*s_jGdmAwrx5si6WTe6e2FA.png" /></figure><h3>Insulin Bolus Calculators and Advisory Systems</h3><p>The calculation of correct insulin doses and the estimation of the amount of carbohydrates is a regular task in the daily life of many insulin-dependent patients. Bolus advisors are based on previous insulin doses, BG measurements, planned carbohydrate estimates, and other patient-specific parameters, including insulin-to-carbohydrate ratio and insulin sensitivity. Manually calculating bolus doses and counting carbohydrates can be complex and challenging because individuals must consider multiple parameters to achieve satisfactory glucose control, and miscalculation of these values may result in persistent glycemic episodes.</p><p>To support carbohydrate estimation and determination of insulin doses by patients, tools for providing bolus advice and carbohydrate estimates are increasingly being adopted. These tools seek to increase the accuracy of mealtime and correction boluses. AI has been used to provide sets of tools to improve the accuracy of carbohydrate estimates and to calculate the optimal insulin bolus for the ingested meal</p><h3>Risk and Patient Stratification</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/458/1*ZyvsQa5QWJFlrzGWSEDJNw.png" /></figure><p>Treatment of diabetes is governed by diverse factors, implying high intra- and interpatient variability [<a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6000484/#ref94">94</a>]. Exercise, nutrition disturbances, age, and cardiovascular complications are just some of the long list of factors that can dramatically impact quality of life and undermine medication adherence even when patients follow their treatment regimen strictly. Such patient variability severely limits the use of general models, which cannot capture the specific physiological behaviors of individuals. Thus, an important step toward better risk detection and intervention is <strong>personalization</strong> of the system. Over the past decade, major research efforts have been devoted to developing management tools capable of stratifying patients in different segments of the population. Risk assessment and patient stratification methods are important to improving the management of diabetes, and therefore the overall health outcomes of diabetic patients, and consequently have attracted a greater share of attention from the medical community.</p><h3>Detection of Meals, Exercise, and Faults</h3><p>Because people with both types of diabetes need support to successfully manage their disease, solutions with higher accuracy that require less user interaction are associated with higher-quality diabetes treatments. Tools or algorithms capable of early detection of critical events affecting glycemic control, such as exercise, a meal, or an infusion set failure, are critical for systematic automation of both closed-loop and open-loop systems. Insulin-dependent patients monitoring their glucose with CGM devices use BG measurements to calculate insulin infusion rates. Consequently, failure of these devices can lead to episodes of hyperglycemia or hypoglycemia.</p><h3>Lifestyle and Daily-Life Support in Diabetes Management</h3><p>Lifestyle management is a fundamental aspect of diabetes care. Sedentary living, stress, nonadherence to medication, lack of regular medical examinations, and bad habits can lead to discontinuation of treatment for patients with diabetes. From the time of diagnosis, patients are required to optimize their lifestyles to manage complications and other comorbid conditions, with the overall goal of enhancing their own care. Current technologies and data warehouses enable solutions that model data and make quality decisions based upon them. Decision support systems (DSSs) consist of tools focused on helping patients or doctors to manage diabetes therapies. These systems usually have monitoring features that facilitate systematic recording of information about diet, physical activity, medication, glucose measurements, etc and combine it with tools to support both patients and clinicians, with the overall goal of enhancing therapeutic outcomes.</p><h3>Cecelia Health</h3><p>Cecelia Health is a great example of a digital healthcare startup. The company was founded by David Weingard, a programmer and executive at Microsoft, an Ironman Triathlon competitor, and a T1D patient.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WeGRoQLDcU5hW8ltw9N2_Q.png" /></figure><p>Cecelia Health delivers a personalized, technology-based approach to improving medical adherence and health outcomes for people living with diabetes and related chronic conditions.</p><p>References:</p><p><a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6000484/">https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6000484/</a></p><p><a href="https://diatribe.org/whats-normal-a1c-when-it-misleading">https://diatribe.org/whats-normal-a1c-when-it-misleading</a></p><p><a href="https://en.wikipedia.org/wiki/Diabetes">https://en.wikipedia.org/wiki/Diabetes</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=be4811eac513" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Saving the Whales]]></title>
            <link>https://medium.com/@ram.avni/saving-the-whales-d9f2dde511e3?source=rss-f09377c2c098------2</link>
            <guid isPermaLink="false">https://medium.com/p/d9f2dde511e3</guid>
            <category><![CDATA[environment]]></category>
            <category><![CDATA[bayesian-statistics]]></category>
            <category><![CDATA[bayesian-machine-learning]]></category>
            <category><![CDATA[whales]]></category>
            <category><![CDATA[bayesian]]></category>
            <dc:creator><![CDATA[Ram Avni]]></dc:creator>
            <pubDate>Sat, 07 Sep 2019 22:58:05 GMT</pubDate>
            <atom:updated>2019-09-07T22:58:05.249Z</atom:updated>
            <content:encoded><![CDATA[<h4>Environmental Research Challenges in Times of Rapid Changes</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/771/1*B1TMFSlZ5m8byhGrDbuHaQ.jpeg" /></figure><p>With a widespread global commercial maritime traffic, there has been an increasing number of collisions between whales and boats, leading to many whale injuries and deaths.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q8EX6vurtyHnFefB4plvUw.jpeg" /></figure><p>Because of unregulated shipping patterns and lack of legislation and supervision, this is a chronic problem in many areas of the world.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TcbD6tTZXQawdjVwdb1S8A.jpeg" /></figure><p>However, in this blog I’ll talk about an inspiring success story of harnessing scientific research to change maritime regulations and protect the whales.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/920/1*XGKXFZVZb9Vddwtl65a0ng.jpeg" /></figure><p><strong>Special Habitat</strong></p><p>Thanks to Panama’s unique weather, the country hosts a variety of habitats with rich flora and fauna. The Gulf of Panama, on the pacific side of the Panama Canal, has warm and rich waters, offering a refuge for the southern hemisphere’s mighty whales, where hundreds of humpback whales gather to feed and mate. Humpbacks migrate northwards from their summer feeding grounds around Antarctica, arriving in the region around late June. They breed in the fertile Las Perlas waters, where an upwelling of nutrient-rich water produces an annual plankton bloom. Altogether, about 900 animals are thought to be involved. There are also visitors from a Northern Hemisphere humpback population.</p><p><strong>The Canal</strong></p><p>Panama is also famous for its canal, and the Gulf of Panama is the Pacific end of the Panama Canal. About 17,000 large ships pass through Panamanian waters each year, the majority international cargo vessels using the canal. This shipping superhighway sees more than 36 ships ply its waters each day, including massive cargo vessels weighing hundreds of thousands of tonnes. Until recently, unregulated shipping patterns caused whales and ships to frequently collide, damaging vessels and injuring or killing the animals. From 2009 to 2012, there were 13 whale deaths in the gulf, many thought to be the result of ship strikes. Data shows one particular vessel routinely passing right through the Las Perlas protected area in order to excavate sand from the sea bed and bring it to land for construction.</p><p><strong>STRI — The Smithsonian Tropical Research Institute</strong></p><p>STRI is the only bureau of the Smithsonian Institution based outside of the United States, in Panama. It is dedicated to understanding the past, present and future of tropical ecosystems and their relevance to human welfare. Héctor Guzmán, a marine biologist at the Smithsonian Tropical Research Institute in Panama, led a research that was crucial to the development of the new regulations to protect the whales.</p><p>Guzmán’s research focuses on highly migratory species including whales, manatees, sharks, sea birds, billfishes and turtles to better understand their basic ecology and to encourage scientifically supported conservation policies. A major challenge in this endeavor is, as Guzmán’s states:</p><h4><strong><em>“Rapid change and lack of pristine marine environments skew basic research. Instead of studying tropical systems unchanged for millennia, scientists must ask questions about ongoing human impacts and conservation to inform policy.”</em></strong></h4><p><strong>The Research</strong></p><p>“This started as basic, fundamental research,” Guzmán says. The next step was to find partners to do something about it. Guzmán worked with Panama Canal pilot Fernando Jaén, the Panama Canal Authority, and a range of other stakeholders for two years to refine a proposal to keep ships away from the majority of the known whale concentrations.</p><p>Similar to how roads are now sometimes built to curve around the natural habitats of land creatures, Traffic Separation Schemes (<strong>TSS</strong>) create shipping lanes that restrict marine traffic to certain areas. But in order to get all shipping to abide by this system, countries need the approval of the International Maritime Organization (<strong>IMO</strong>), a UN body that regulates shipping safety and navigation around the world.</p><p>So Guzmán and his colleagues began to use satellite beacons to track 15 humpback whales in the Gulf of Panama, and overlaid their tracks with transit records for nearly 1,000 ships. They found that because there was no set path for ships to take as they entered the canal, whales and ships frequently came into contact. Further research showed that during one 11-day period, whales and ships moved within 200 meters of each other 98 times.</p><p><strong>Solution and Legislation</strong></p><p>In 2013, the proposed TSS was submitted to the IMO. The proposal was adopted in December 2014, and global shipping charts were updated to reflect the now-mandatory shipping lanes. The IMO also requires ships to move no faster than 10 knots in the area from August to November, when humpback whales tend to congregate in the gulf.</p><p>Fortunately, the combination of solid whale biology and regulations passed by the IMO has resulted in a scheme that has lessened the frequency of those ship strikes; these regulations are estimated to have cut the potential for whale strikes by as much as 92 percent. Guzman is now using his whale biology studies and the lessons learned from the Panama project to help create similar schemes all along the Pacific coast.</p><p><strong><em>Digging Deeper into the Data Science behind the Research</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*bvhqkMUzEaBV2Lr4lIexQg.jpeg" /></figure><p>Time period: 2003 to 2009</p><p>Area: Las Perlas Archipelago off the Pacific coast of Panama</p><p>Observations: 53 times between the months of August and October</p><p><strong>Main goals:</strong></p><p>1. To estimate the size of the humpback whale population at Las Perlas Archipelago in Pacific Panama.</p><p>2. To gain a better understanding of the migratory connections between the whales from the Southern Hemisphere.</p><p>3. To acquire the knowledge needed to determine if protection is warranted for the species in Panama from the effects of increasing coastal development, maritime traffic, and tourism.</p><p><strong>Identification / Classification:</strong></p><p>1. We identified 295 individuals using photo-identification of dorsal fins, including 58 calves, and the population estimate for a single season was 100–300 solitary adults plus 25–50 mothers with calves; the estimated population of animals across all seasons using a mark and recapture model was over 1,000.</p><p>2. Eight of the 139 fluke identifications were matched to whales in photograph catalogues from the Antarctic Peninsula and a ninth was matched to a whale sighted in Chilean waters; four of these nine individuals have also been sighted in Colombia.</p><p>3. We conclude that Panama (Las Perlas Archipelago in particular) is an important calving area for humpback whales in the Southern Hemisphere. These data should provide a foundation for monitoring of population change and to increase awareness in Panama about the need to manage vessel traffic and tourism related to the whales at Las Perlas.</p><p><strong>Methods:</strong></p><p>Study Area and Photo-identification / Classification:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/461/1*IpatD9mBMaB4eFTiSMHveQ.png" /></figure><ul><li>168,771 ha, of which 135,618 ha are marine environments.</li><li>averaging 15 m depth and all &lt;50 m.</li></ul><p>Site preference and whale distribution within the archipelago:</p><ul><li>carried out with monthly aerial surveys (was <strong>not</strong> used in any population analyses)</li></ul><p><strong>Whales were photographed for later identification using ventral fluke and dorsal fin marks:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YagxzvJcnAlhGc2lkFO_RQ.png" /></figure><ul><li>Images catalogued with IMatch3. The flukes and dorsal fins were first compared internally for each year, then compared externally.</li></ul><p><strong>Whale Satellite Tracking:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*afcegpAJKpi-YFdE9BRoqA.jpeg" /></figure><p>Real-time satellite transmitters, constant transmission.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/313/1*9EKZqEl8daxz-Bxk5mNaUA.png" /></figure><p>We used tag-derived positions from Argos location classes 3, 2, 1, 0, A, and B, providing positional errors of 150 m to 5 km radius.</p><p><strong>The open-population mark-recapture model:</strong></p><ul><li>A sample of 295 individuals (based on the photo-identifications)</li><li>Daily sighting histories were constructed for those animals,</li><li>50 observation days over the 5 yr</li><li>Data are vectors of length 36, <br>1 = days it was photographed <br>0 = on days it was not.</li><li>Animals known by flukes alone were excluded because they might overlap with those known by dorsal fin alone.</li><li>The sighting histories formed the basis of an analysis of population size and residency, using the Cormack-Jolly-Seber mark-recapture method.</li></ul><p><strong>Estimating population size requires estimating first two key parameters:</strong></p><ol><li>The probability dt of detecting an animal in the study area on day t,</li></ol><p>2. the probability et that an animal in the study area on day t departs before day t+1.</p><p>The standard method is to find parameters which best account for observations based on <strong><em>maximum likelihood</em></strong>.</p><p>We used a <strong><em>Bayesian method</em></strong>, based on the same likelihood formulations but producing also 95% credible intervals for every parameter, and we employed a multilevel, hierarchical model, in which daily variation in the parameters was permitted but constrained to follow overarching hyperdistributions. This improves statistical power relative to unconstrainted day-to-day variation.</p><p><strong>Results and Discussion</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pu-wGBawApvnDkrnZke9dA.png" /></figure><ul><li>Sightings and Resightings: we identified 295 distinct animals based on dorsal fin markings, including 185 solitary adults, 52 mothers (with a calf), and 58 calves (with a mother). — table 2</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7ApWuWlgip0-Y45gIPqFrA.png" /></figure><ul><li>Migratory Connections: Panama is an important breeding area for some whales feeding in Chile. Fig 2</li><li>Capture-recapture Estimate: by pooling our data across seasons, we estimated that the population size at Las Perlas is just over 1,000 animals.</li><li>Further collaborative studies are needed to provide better estimates of population size in Pacific Panama, and long-term monitoring is essential in order to detect and understand changes and potential risks to the population, particularly calf production. Our current study provides an antecedent to such monitoring.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d9f2dde511e3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[MySQL JOINs]]></title>
            <link>https://medium.com/@ram.avni/mysql-joins-110bb151d689?source=rss-f09377c2c098------2</link>
            <guid isPermaLink="false">https://medium.com/p/110bb151d689</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[mysql-join]]></category>
            <category><![CDATA[mysql]]></category>
            <dc:creator><![CDATA[Ram Avni]]></dc:creator>
            <pubDate>Sun, 25 Aug 2019 16:47:09 GMT</pubDate>
            <atom:updated>2019-08-25T16:47:09.185Z</atom:updated>
            <content:encoded><![CDATA[<p>This blog will discuss one particular feature, the JOIN, starting with the four basic types of joins. We’ll use some examples mainly for MySQL, and then move on to more advanced joins, with a few examples involving three tables.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*v4H1pgk_mNDH8nT1XoKoXA.png" /><figcaption>Venn Diagrams — 4 basic JOIN types</figcaption></figure><p>The Venn Diagram is mostly used to depict the joins, but some argue that it is actually confusing and even misleading representation (<a href="https://blog.jooq.org/2016/07/05/say-no-to-venn-diagrams-when-explaining-joins/">Say NO to Venn Diagrams When Explaining JOINs</a>). For an alternative representation, consider the Join Diagram, as depicted below for a CROSS JOIN:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2O86OSW6xKml85QGeggS0g.png" /><figcaption>Cross JOIN</figcaption></figure><p>The cross join name is derived from the cross product (remember linear algebra and matrices) — you basically take every item on the left side, and combine it with every item on the right side. When you CROSS JOIN a table of 4 rows with a table of 5 rows, the result will be a new table of 4×5=20 rows.</p><p>Note that the cross join is basically the same as the inner join, but without any condition. In other words, the inner join has to specify the clause “using (t1.key)” , but this clause cannot be used with the cross join. Also note that all other types of JOIN can be derived from the cross join.</p><p>Let’s take a look again at the four JOIN types with this row representation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kNCXqI8zzl9Cw1F4l_A2tw.png" /><figcaption>JOIN Diagram</figcaption></figure><p>We’ll use the following two tables — Customers and Orders — to illustrate the syntax and use case of the four basic join types:</p><p>Customers table:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cO5T0IjsnEz2AH9-iboMig.png" /></figure><p>Orders table:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lXMsXJdwzVrWPGHrrOWBTg.png" /></figure><h3>Inner Join:</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0IX4Ge8kwqZwc9c233NL6g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GUr48gY7E32DD4iGHKHJtw.png" /></figure><p>An inner join is a perfect solution if we want to have the names of all the customers who ordered some products and all the details about their orders.</p><h3>Left Join (also referred to as Left Outer Join):</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zK6Otr6IdrM6fUA5oF2_gw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wlHWuaKcCruGo-gh3rrxgA.png" /></figure><p>Think of left join as <strong>appending </strong>information from the second table into the first table (i.e. which is on the left). In this example we see every customer from Customers, with any matching record from Orders. If there is no matching record in Orders, it will simply show as Null. This result is useful if we want to have a list of customers who have not ordered any products. We can easily do that with the query: “where order_date is NULL”</p><h3>Right Join (also referred to as Right Outer Join):</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5GgjVFCKaHAmc14aLX8Iyw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ELHQx7JIQL8gU1AlzJ8AyA.png" /></figure><p>As the name implies, it’s the opposite of Left join, so think about it as getting a list of all orders, and appending customer information. This time around we get Null values for first and last name, which could be useful if we want to get a list of those products that were purchased, but for some reason the customer name was not entered into the system (<strong>“where first_name is NULL”</strong>).</p><p>Since it is a mirror image of the left join, we could get the exact same result by querying: “Orders left join Customers.”</p><h3>Full Join:</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kTdYUeO8heYT4gHShJ3ApQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BaTeF3ohc3s09xNVbqhKEA.png" /></figure><p>We use this join to get a full list of all records from both tables, but have some caution — it’s NOT the same as CROSS Join. Also note that this time we get rows with the Null values that appear in both Left and Right joins (whereas in inner join we don’t get any Null.</p><p>Additional good resource to visualize these four basic types is the <a href="https://www.youtube.com/watch?v=9yeOJ0ZMUYw&amp;vl=en">Socratica video</a>, which illustrates the examples with row representation and minimal use of Venn Diagrams.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qH3JDwgjqvb_jKfu7q8H2g.png" /></figure><h3>Three Additional Join Types:</h3><ul><li><strong>Left Outer Join with Exclusion</strong></li><li><strong>Right Outer Join with Exclusion</strong></li><li><strong>Full Outer Join with Exclusion</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IUVfcYxSleXObPpo_vdHtQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ks_vGDO8NZcvvqwE0jT41w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jUygZOf4Fmll071WUzjj8g.png" /></figure><p>Basically, the Left Outer Join with Exclusion will do the same as a Left Join, with the query “WHERE t2.key is Null” (as we saw above with the Customers and Orders tables). however , as we may want to do a subquery into the second table, it can come handy as in the following library / book collections example:</p><p><strong>Example of Left Outer Join with Exclusion:</strong></p><p>Consider the following Library and Collections tables:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/796/1*4NEN_UCmrLZt51vlxaLv1Q.png" /></figure><p>We want to show all the books that are <em>not</em> in a user’s collection. This query to show all the books not in <em>any</em> user’s collection:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NSTfdZ_r-emyrzo2Xrc3ag.png" /></figure><p>How can we find out only those books which are not in a <strong><em>specific</em></strong> user’s collection?</p><p>Let’s try it with Yish:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qAKXJhMiO-Ul06ZhfPdIVA.png" /></figure><p>Let’s break it down:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*L_EdOT9rv_Br_25_TO2bwQ.png" /></figure><h3>Finally, joining three tables:</h3><ul><li>Two INNER joins</li><li>Two LEFT OUTER joins</li><li>INNER join and LEFT OUTER join</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*16CX31PBFEKF_HMA5gjqlQ.png" /></figure><p><strong>Two INNER joins</strong>: it is similar to the regular join, but with three tables, and the query will result only in elements that exists in all three (no field will have a Null value).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iSiPEYNbEWSTAu7rTwYadA.png" /></figure><p><strong>Two LEFT OUTER joins</strong>:</p><p>Let’s illustrate it with</p><p>Table A = <em>Customers</em>, Table B = <em>Orders</em>, and Table C = <em>Payments</em>.</p><p>This join will result in all customers, with some Null values if they haven’t made a purchase or haven’t made a payment yet. So if for example we want all customers who both haven’t made a purchase or haven’t made a payment, we would specify “WHERE t2.key is Null” AND “t3.key is Null”. If we want only customers who paid in advance or made a deposit but actually haven’t ordered yet, we would query “WHERE t2.key is Null” AND “t3.key is NOT Null”. And we’d do vice versa for customers who ordered but haven’t paid yet.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yYfKwlQ-3CM5U5tjo9Ta6w.png" /></figure><p><strong>INNER join and LEFT OUTER join</strong>:</p><p>Let’s illustrate it with the Flatiron School.</p><p>Let Table A = <strong><em>Data Science cohort 1</em></strong>, Table 2 = <strong><em>Data Science cohort 2</em></strong>, and Table 3 = <strong><em>Software Engineering cohort</em></strong>.</p><p>This join will return students who deferred from DS1 to DS2, and SW students who changed program type (DS to SW and vice versa) AND deferred from DS1 to DS2. If we want to get only those who deferred, we can subtract from this list the Two INNER joins (by querying for elements with “t3.key = Null”).</p><p><strong>References:</strong></p><p>Say NO to Venn Diagrams When Explaining JOINs, <a href="https://blog.jooq.org/2016/07/05/say-no-to-venn-diagrams-when-explaining-joins/">https://blog.jooq.org/2016/07/05/say-no-to-venn-diagrams-when-explaining-joins/</a></p><p>Wikipedia, MySQL: <br><a href="https://en.wikipedia.org/wiki/MySQL">https://en.wikipedia.org/wiki/MySQL</a></p><p><a href="https://stevestedman.com/2015/03/mysql-join-types-poster/">https://stevestedman.com/2015/03/mysql-join-types-poster/</a></p><p>Socratica — SQL Tutorial,<br><a href="https://www.youtube.com/watch?v=nWyyDHhTxYU">https://www.youtube.com/watch?v=nWyyDHhTxYU</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=110bb151d689" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Reshaping in Pandas with stack() and unstack() Functions]]></title>
            <link>https://medium.com/swlh/reshaping-in-pandas-with-stack-and-unstack-functions-bb169f64467d?source=rss-f09377c2c098------2</link>
            <guid isPermaLink="false">https://medium.com/p/bb169f64467d</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[flatiron-school]]></category>
            <dc:creator><![CDATA[Ram Avni]]></dc:creator>
            <pubDate>Wed, 14 Aug 2019 03:29:18 GMT</pubDate>
            <atom:updated>2019-08-26T07:32:39.184Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*VAf3C69Ssd1ZSqSN.jpeg" /></figure><p>Pandas is a software library written for the Python programming language for data manipulation and analysis. In particular, it offers data structures and operations for manipulating numerical tables and time series.</p><p>Pandas stands for “Python Data Analysis Library”. According to the Wikipedia page on Pandas, “the name is derived from the term “panel data”, an econometrics term for multidimensional structured data sets.” It is one of the most popular Python libraries, and provides interface with other popular libraries like Numpy and Matplotlib. It provides abstractions of dataframes and series, and in this blog I’ll focus on particular functions to reshape these objects.</p><p>The concept of stacking comes in handy when we have data with multi indices. Using the <em>stack()</em> function will reshape the dataframe by converting the data into a stacked form. Since we are having multiple indices, that means converting (also called rotating or pivoting) the innermost column index into the innermost row index. <em>Unstacking</em>, as the name implies, does exactly the inverse operation — it will convert the innermost row index back into the innermost column index.</p><p>Let’s take a look at the diagram below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*xygBiATSiXlreNWV" /><figcaption>Reshaping with stack() and unstack()</figcaption></figure><p>As you can see, stacking means rearranging the data vertically (or stacking it on top of each other, hence the name “stacking”), making the shape of the dataframe as a taller and narrower stack (with fewer columns and more rows). And as you probably could figure out, unstacking will do just the opposite, spreading out the data and reshaping it into a shorter but wider dataframe (with fewer rows but more columns).</p><p>Note that if you stack and unstack the same dataframe, you will get back the original dataframe, but you have a choice to stack on different indices and get a different shape of dataframe.</p><p>Note that the concepts of multi indexing, hierarchical indexing, pivoting, stacking and unstacking are all related, but let’s start by looking at a sample dataframe, and let’s create a sample dataframe with a typical example we are all familiar with — a grading table.</p><p><strong>Methods of multiIndex creation</strong></p><p>There are various ways to create multiple indices. One of the simplest ways to create a multi index <em>Series</em> or <em>DataFrame</em> object is by passing a list of two or more arrays to the constructor:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cG7QkzFnsleawPru4lMc0Q.png" /><figcaption>List of arrays</figcaption></figure><p>Other options include passing a dictionary where the keys are spelled out as tuples:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xNlz1CqWCT6o3jX5B6HG-w.png" /><figcaption>Tuples</figcaption></figure><p>Finally, you can explicitly create a multi index by using the class method constructors available in the <em>pd.MultiIndex</em>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*akZ2OCm54wS8yeRH8gnAyw.png" /><figcaption>Class method constructor</figcaption></figure><p>For this blog, I used the first method. Note that I had to convert the original data from list of lists into an np.array type:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p3Kn-v3zzBBYTFxV6eP3Qw.png" /><figcaption>An np.array</figcaption></figure><p>And here is how our multi index table looks like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xhJgwPEZlMEj51XB5PbXlQ.png" /><figcaption>Original Dataframe — Data Science Grading</figcaption></figure><p>At first glance, it seems that there are quite a few students with an “Incomplete” grade in the Labs section, so let’s throw in a quick regex to clean up those values by accessing the innermost column index for labs, and now we have:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uAtULkd62tEtv2fIECIliw.png" /><figcaption>Cleaning data with regex</figcaption></figure><p>Before we even run some statistics, let’s think about how we would like to see the data. This view can give us some idea about the overall performance of the class, but as an individual student you might want to quickly see all your grades across every subject and activity since the beginning of the course — so let’s unstack the original dataframe to get the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DNRGFs54ml1yKW6gdKejBg.png" /><figcaption>First unstack</figcaption></figure><p>Note that we can go back to the original dataframe by running the inverse function, stack():</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-zUXJL9BrYdrnv4tPedMyA.png" /><figcaption>Original dataframe</figcaption></figure><p>Now, we can further unstack the new dataframe, using df2 = df1.unstack(), and let’s see what we get:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/930/1*H68ViB2V8IXcLL38x3jYvA.png" /><figcaption>Dataframe 2x unstacked</figcaption></figure><p>Here, all the columns indices shifted to the row axis, and we can now see the overall performance of all students across every subject and activity since the beginning of the course.</p><p>When I tried to stack the last result back, I got a surprise: Series’ object has no attribute ‘stack.’</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YflIMi7vpGS8cAM18LbyOw.png" /><figcaption>Error message — cannot stack</figcaption></figure><p>The reason is that once we “run out” of row indices, the dataframe becomes a series, and if we try to stack we’ll get an error. But, since it’s a series, we can unstack this series:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*4I8psAhBBsTq6aDHiTCPDw.png" /><figcaption>Unstacking series</figcaption></figure><p>Note how the indices now flipped from the original df! In other words, it is stacking the dataframe back but in a different order.</p><p>And if we unstack one more time, it will flip the innermost row index (activity type — Labs, Projects, etc.), so we completely flipped the indices of the original df:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/670/1*e8g5G-FbQNqxEMoRdZ6QtQ.png" /><figcaption>Unstacking twice</figcaption></figure><p>Alright, enough flipping, let’s go back to the original dataframe. Let’s stack, replace the Nan, and see what happens:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/972/1*rZCjU-BvJnHBrZlQfLcElw.png" /><figcaption>Stacking the original dataframe</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/956/1*N4O7V8msewgJ5t8Fjstm1Q.png" /><figcaption>Cleaning NaN values</figcaption></figure><p>We got missing values since the subjects change every period (in this case every week). However, this view could be useful for an annual program (with 2 semesters etc.), where the subjects repeat throughout the year.</p><p>And lastly, let’s stack the last result, and we’ll get:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/886/1*1UjG0JJXhTGa67P-EY2a0A.png" /><figcaption>Stacking 2x</figcaption></figure><p>Note that it’s again a series type object, very similar to what we got by unstacking the original dataframe two times. The difference however is that in this case the order of the indices is the reverse of the double unstacking, as expected.</p><p><em>References</em>:</p><p><a href="https://en.wikipedia.org/wiki/Pandas_(software)"><em>https://en.wikipedia.org/wiki/Pandas_(software)</em></a></p><p><a href="https://nikgrozev.com/2015/07/01/reshaping-in-pandas-pivot-pivot-table-stack-and-unstack-explained-with-pictures/"><em>https://nikgrozev.com/2015/07/01/reshaping-in-pandas-pivot-pivot-table-stack-and-unstack-explained-with-pictures/</em></a></p><p><a href="http://www.datasciencemadesimple.com/reshape-using-stack-unstack-function-pandas-python/"><em>http://www.datasciencemadesimple.com/reshape-using-stack-unstack-function-pandas-python/</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bb169f64467d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/reshaping-in-pandas-with-stack-and-unstack-functions-bb169f64467d">Reshaping in Pandas with stack() and unstack() Functions</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Committee to Save Cooper Union]]></title>
            <link>https://medium.com/@ram.avni/the-committee-to-save-cooper-union-70bcfa164939?source=rss-f09377c2c098------2</link>
            <guid isPermaLink="false">https://medium.com/p/70bcfa164939</guid>
            <category><![CDATA[education]]></category>
            <dc:creator><![CDATA[Ram Avni]]></dc:creator>
            <pubDate>Wed, 31 Jul 2019 22:43:20 GMT</pubDate>
            <atom:updated>2019-07-31T22:43:20.039Z</atom:updated>
            <content:encoded><![CDATA[<p>(or: Is it the end of free education?)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*wi5obbKVD4LUHhmhc9XFpA.jpeg" /></figure><h3>Chapter 1:</h3><h3>The Code Broke!</h3><p>Cooper Union is a private college in NYC. It is located at Cooper Square on the border of the East Village. It is my alma mater, and I was lucky to receive a 4-year, full tuition scholarship. As a matter of fact, all my classmates, and all students, received the same full scholarship.</p><p>Well, it turns out that Cooper is the only free college in the country. If you google it you’ll surely get a list of 14 free higher ed options, but each one with a catch — they are either not open to all students (location, gender), are not ranked, or have other strings attached.</p><p>And the beautiful thing about Cooper is its tradition — a tradition of free tuition for over 100 years, in Engineering, Architecture, and the Arts. Alas, in 2014 this legacy, this code, broke; the college announced, out of the blue, that it is going to charge tuition effective immediately.</p><p>So what really happened here??</p><p>Well, to understand what happened, and why the Attorney General and the court decided what they decided, we need to go back in time and find who was Peter Cooper.</p><h3>Chapter 2:</h3><h3>The Legacy of Peter Cooper</h3><p>Who was Peter Cooper?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*q3ZXt9FnfSOVt6OzQAqa0A.jpeg" /></figure><p>An inventor, engineer, self-made billionaire, and most compassionate person and philanthropist. Peter Cooper was born in 1791 in New York, and grew up in poverty. He even had to quit school in order to help his family make a living. Nevertheless, those circumstances did not slow down his drive, tenacity, and genius, and at the age of 13 he already invented a mechanical clothes-washing machine. He was a truly self-taught student and budding businessman; while working long hours during the day, he managed to borrow books and continued to study on his own at night.</p><p>His first commercial success was a glue factory, and then, thanks to his technical genius and foresight, he made incredible fortune from numerous inventions and investments, including:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/370/1*lE7cYS2WxbuMMDMATG5SDQ.gif" /></figure><ul><li>Iron and steel</li><li>Railroads</li><li>The first steam-engine locomotive — “Tom Thumb”</li><li>Telecommunication</li><li>And of course Jell-O !</li></ul><p>Nearing sixty, Peter Cooper was one of the richest and most celebrated men in the United States. His passion was philanthropy in its most literal sense; he wanted to share his good fortune by helping others to develop their gifts and talents, and to improve society as a whole. The vehicle he chose was The Cooper Union for the Advancement of Science and Art, which he founded in 1859.</p><h3>Chapter 3:</h3><h3>The History of Cooper Union</h3><p>From the start, Cooper Union was a unique institution, dedicated to Peter Cooper’s proposition that education is the key not only to personal prosperity but to civic virtue and harmony. His goal — and this is a <strong><em>key point as we jump back to 2014</em></strong> — was to provide free education for all, drive innovation, and instill a sense of social justice.</p><p>The school started as adult continuing education, open to women, minorities, people of color, and immigrants. Some of the famous people that passed through Cooper include Thomas Edison (chemistry classes), future supreme court justices, and Bob Kane, the creator of Batman.</p><p>Gradually, the institution evolved to a full-time degree programs in Engineering, Art, and Architecture. For over 100 years it maintained the spirit and vision of its founder, Peter Cooper, providing free education and promoting his values. That is, until 2014.</p><p>So what happened in 2014? We need to peek under the hood, and inspect the school endowment and finances which were tightly concealed by the board of trustees.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/852/1*_O2PFiU6zAKKF07ZdkXL4g.jpeg" /></figure><p>Turns out that Peter Cooper children’s gave the school a gift — the land under the Chrysler building.</p><p>It accounted for 80% of the endowment, and by leasing this land, the school got most of its income. With the financial and real estate crisis of 2008, prices plummeted and the school suffered a serious blow in its operational income. Coupled with bad management decisions, Cooper found itself all of a sudden on the brink of bankruptcy.</p><p>Looking at history, the Cooper Union endured 2 world wars, the great depression, multiple stock market crashes, and financial crises. The simplest rule of asset management, one familiar to even novice investors, is diversification. Yet Cooper Union’s endowment is highly unusual in that it’s concentrated in a single asset — the land under the Chrysler Building.</p><p>So let’s reiterate the grave circumstances:</p><p>- 80% of the endowment invested in one asset type — real estate</p><p>- The rest 20% were astonishingly invested in hedge funds</p><p>- This ill advised combo led to a serious depletion of Copper’s endowment.</p><p>If it’s hard to believe, just run a simple python / SQL query to verify:</p><p>&gt;&gt;&gt; type (endowment)</p><p>Class (real estate)</p><p>&gt;&gt;&gt; count (real_estate_assets)</p><p>1</p><p>&gt;&gt;&gt; type (NOT real_estate)</p><p>Class (hedge funds)</p><p>&gt;&gt;&gt; “””SELECT <strong><em>Common_Sense</em></strong> from <strong><em>BOARD_OF_TRUSTEES</em></strong>”””</p><p>NULL</p><p>***</p><p>And now, very unexpectedly, was brought to its knees and decided to start charging tuition. Usually universities can leverage it with 3 tools, none of which was available to Cooper:</p><ul><li>raise tuition</li><li>cut scholarships</li><li>dark secret — admitting higher % of wealthy students</li></ul><p>Guess what happened next? An upheaval. The Cooper community revolted against the president and board of trustees. Angry students, alums, parents, and professors protested and took the case to court, and legal battles ensued. There were even various occupations of the administrative building and a sit-in at the president’s office.</p><h3>Chapter 4:</h3><h3>So now what??</h3><p>First off, the president (of whom I am totally not fond of) resigned. And after legal battles and long negotiations, the school and The Committee to Save Cooper Union arrived at the following agreement:</p><p>- Current students will be paying 50% tuition (~ $20k).</p><p>- Cooper announces 10-year plan to reinstate full-tuition scholarship.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/685/1*W2yYJwsz9nRB50-S_H5tBw.jpeg" /></figure><h3>Thank You!</h3><p>References:</p><p>Wikipedia contributors. (2019, July 27). Cooper Union. In <em>Wikipedia, The Free Encyclopedia</em>. Retrieved 23:53, July 30, 2019, from <a href="https://en.wikipedia.org/w/index.php?title=Cooper_Union&amp;oldid=908169808">https://en.wikipedia.org/w/index.php?title=Cooper_Union&amp;oldid=908169808</a></p><p>Wikipedia contributors. (2019, July 11). Peter Cooper. In <em>Wikipedia, The Free Encyclopedia</em>. Retrieved 23:54, July 30, 2019, from <a href="https://en.wikipedia.org/w/index.php?title=Peter_Cooper&amp;oldid=905768417">https://en.wikipedia.org/w/index.php?title=Peter_Cooper&amp;oldid=905768417</a></p><p><a href="http://www.cooper.edu/">www.cooper.edu</a></p><p>The New York Times, “Cooper Union Announces Plan to Reinstate Free Tuition,” March 15, 2018, <a href="https://www.nytimes.com/2018/03/15/nyregion/cooper-union-free-tuition.html?login=google">https://www.nytimes.com/2018/03/15/nyregion/cooper-union-free-tuition.html</a></p><p>The New York Times, “How Cooper Union’s Endowment Failed in Its Mission,” May 10, 2013, <a href="https://www.nytimes.com/2013/05/11/business/how-cooper-unions-endowment-failed-in-its-mission.html?module=inline">https://www.nytimes.com/2013/05/11/business/how-cooper-unions-endowment-failed-in-its-mission.html?module=inline</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=70bcfa164939" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>