<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by How Khang on Medium]]></title>
        <description><![CDATA[Stories by How Khang on Medium]]></description>
        <link>https://medium.com/@howkhang?source=rss-f52b5c7a1fb5------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*xcINjdV67nXQtaj1iaWhQw.jpeg</url>
            <title>Stories by How Khang on Medium</title>
            <link>https://medium.com/@howkhang?source=rss-f52b5c7a1fb5------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 21:04:43 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@howkhang/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Singapore’s Legal Profession: Data Analysis of Lead Counsel Appearance Numbers]]></title>
            <link>https://medium.com/@howkhang/singapores-legal-profession-data-analysis-of-lead-counsel-appearance-numbers-eee245dc7e0f?source=rss-f52b5c7a1fb5------2</link>
            <guid isPermaLink="false">https://medium.com/p/eee245dc7e0f</guid>
            <category><![CDATA[law]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[singapore]]></category>
            <dc:creator><![CDATA[How Khang]]></dc:creator>
            <pubDate>Thu, 06 Dec 2018 07:32:18 GMT</pubDate>
            <atom:updated>2018-12-06T13:14:52.759Z</atom:updated>
            <content:encoded><![CDATA[<h4>Building and Exploring a Data Set of Singapore Advocates</h4><p>Recently, in an unprecedented move, 21 local law firms pledged to provide more opportunities for their younger lawyers to represent their clients in court. The backdrop to this has been the suggestion that a lack of opportunities for young lawyers to appear in court may have contributed to the high numbers of mid-level lawyers dropping out of legal practice. <a href="https://journalsonline.academypublishing.org.sg/Portals/0/EFirst/Html/0851_[2018]%20SAL%20Prac%201_Nicholas%20Poon%20-%20Decline%20of%20Oral%20Advocacy%20(Published%20on%206%20September%202018).htm">Calls</a> have also been made for a study to be conducted on the careers of advocates.</p><p>As a member of Singapore’s legal profession and an advocate who used to appear in court regularly, I became interested in compiling data that could possibly shed light on the following questions:</p><ul><li>What are the admission dates of the advocates in private practice who have appeared as lead counsel before the Court of Appeal and High Court of Singapore?</li><li>If we group them by admission date (1980s, 1990s, 2000s etc.), what are their relative proportions and how have these changed over time?</li></ul><p>After some online searches and data wrangling, I managed to compile a list of lead counsel names mentioned in written grounds/judgments issued by the Supreme Court of Singapore (Jan 2000 to Oct 2018) and matched most of these names to their respective call dates. Covering over 6,700 cases with over 14,000 lead counsel mentions, this should be the largest data set of its kind. An analysis of this data led to the following observations.</p><h4>Findings and observations</h4><ul><li>Advocates called to the bar in the 1990s have the highest share of appearances in written grounds/judgments since 2001 and are especially dominant when it comes to civil trial cases.</li><li>Advocates called to the bar in the 1980s have seen their share of appearances decline continually since 2000 but they still maintain a good share for number of appearances in Court of Appeal judgments.</li><li>Advocates called to the bar in the 2000s have seen their share of appearances slowly increase over the years, and in the past two years, they have had more individuals appear in written judgments than their 1980s counterparts.</li><li>Advocates called to the bar in the 2010s have also seen their share of appearances increase, but most of their appearances are still limited to non-Court of Appeal and non-trial cases.</li><li>The 2000s and 2010s cohorts appear to have lagged behind the 1990s cohort in making lead counsel appearances during their initial years of practice.</li></ul><p>In the sections below, I set out the steps I took to build the data set and the analysis that was done to arrive at these observations. I have also <a href="https://github.com/howkhang/leadcounsels/blob/master/cases.csv">published</a> the data set online so that anyone can verify the data or use it for further analysis. Please feel free to leave a comment if you find any error in the data or in my analysis.</p><h3>Collecting the Data</h3><h4>Singapore Law Watch’s archive of written grounds/judgments</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fW9ntLs2_GgZypTTev2iLA.png" /><figcaption>Sample judgment page with counsel names and topic tags</figcaption></figure><p>The Singapore Law Watch <a href="https://www.singaporelawwatch.sg/Judgments">archive</a> includes all the written grounds/judgments issued by both the Court of Appeal and the High Court of Singapore since January 2000 and it is the largest publicly available data set containing mentions of lead counsel appearances. A total of 6,716 written judgments from January 2000 to October 2018 were used for this analysis.</p><p>We know that not every case results in a written grounds/judgment. By plotting the number of written grounds/judgments and the number of cases that have been disposed of over the years, we can observe that both sets of numbers have been fairly constant over the years but only a small fraction of all cases end up with written grounds/judgment being issued:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/953/1*S3igzSbnnZfschYJ5fHhvw.png" /></figure><p>Based on our knowledge of how the courts operate, we also know that there is a higher likelihood of written grounds being issued if the case contains complex or novel issues, or if it involves the wider public interest and/or the consequences are more profound (e.g. in criminal cases).</p><p>We are able to observe the effect of this practice when we plot the number of written judgments in the civil and criminal categories (a written judgment is a criminal case if the Public Prosecutor appears as a party and is a civil case otherwise) in proportion to the total number of cases in each of these categories (source: <a href="https://www.supremecourt.gov.sg/publications/annual-reports">Supreme Court Annual Reports</a> workload statistics):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/844/1*VlWmStjD76dO3N-IsKtUog.png" /><figcaption>This chart was edited after publication due to an error in the numerator used that resulted in a higher %.</figcaption></figure><p>These ratios clearly show that the set of written judgments is a skewed sample, with a higher proportion of criminal cases ending up with a written judgment compared to civil cases. From this we conclude that advocates who specialize in areas where the courts are more likely to issue written grounds/judgments, are more likely to be over-represented in our data as compared to their counterparts who do not practise in these areas.</p><p>This means that if we are comparing the appearance numbers between individual advocates, we should try to do so <em>within</em> categories rather than across categories, so that the comparisons are more likely to be meaningful.</p><p>However, if we are comparing appearance numbers for different groups of advocates across time periods, this skew is arguably not as problematic if we assume that the courts’ practice has been consistent across time and this skew equally affects the numbers for all the different groups being compared.</p><h4>Dates of admission</h4><p>Having compiled a list of lead counsel appearances, the next step is to obtain the dates on which these advocates were admitted to the Singapore bar. A few Google searches surfaced the following online resources:</p><ul><li>The Ministry of Law’s Legal Services Regulatory Authority search page containing the list of current practitioners with a Singapore practising certificate:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OlAqUoZd3MopB2lPA9DBHQ.png" /><figcaption>MinLaw’s LSRA Search Engine</figcaption></figure><ul><li>The Law Society of Singapore’s list of practitioners who voted/did not vote in the Election of Council by category (<a href="https://www.lawsociety.org.sg/portals/0/resourcecentre/runningyourpractice/pdf/practitioners_who_voted_2004.pdf">2004-middle</a>, <a href="https://www.lawsociety.org.sg/portals/0/resourcecentre/runningyourpractice/pdf/practitioners_who_voted_2005.pdf">2005-senior</a>, <a href="https://www.lawsociety.org.sg/portals/0/resourcecentre/runningyourpractice/pdf/practitioners_who_voted_2006.pdf">2006-middle</a>, <a href="https://www.lawsociety.org.sg/portals/0/resourcecentre/runningyourpractice/pdf/practitioners_who_voted_2007.pdf">2007-middle</a>, <a href="https://www.lawsociety.org.sg/portals/0/resourcecentre/runningyourpractice/pdf/practitioners_who_voted_2010_junior.pdf">2010-junior(1)</a> and <a href="https://www.lawsociety.org.sg/portals/0/resourcecentre/runningyourpractice/pdf/practitioners_who_did_not_vote_2010_junior.pdf">(2)</a>, <a href="https://www.lawsociety.org.sg/ejus_news/pdf/2012_election_list.pdf">2012</a>, <a href="https://www.lawsociety.org.sg/Portals/0/election/2013/pdf/Election%20List%20of%20Senior%20Lawyers.pdf">2013-senior</a>, <a href="https://www.lawsociety.org.sg/Portals/0/election/2013/pdf/Election%20List%20of%20Junior%20Lawyers.pdf">2013-junior)</a>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2GQlH-P1pwxVa9FwNaTryg.png" /><figcaption>Law Society’s list of practitioners who voted in 2004 (middle category)</figcaption></figure><p>Matching the advocate names in the written judgments against the names in these lists (with the admission dates) was not a trivial exercise due to the different ways individual names can be styled. Furthermore, different individuals could share similar names, which meant that it became necessary to try and match their firm names as well in order to avoid mismatches.</p><p>After a number of trials, the best method was an iterative process that involved normalizing the names, and using a number of different heuristics with the help of the distance-based scoring<em> </em>library <a href="https://github.com/seatgeek/fuzzywuzzy">FuzzyWuzzy</a><em> </em>to find the closest matching names. For a number of cases that had no match, the admission dates were manually added if their occurrences were high enough. Using this method, a match-rate of over 96% was achieved for a total 14,280 lead counsel mentions in the written judgments.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G55RnCJTlQNABjFH4GsnuA.png" /><figcaption>Sample list of names matched with their admission dates</figcaption></figure><p>Plotting the 3% or so remaining missing values over the years shows that there were more missing values for the years 2000–2004:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/754/1*_9vSJNzA6BTv4NDaCFckkQ.png" /></figure><p>One likely reason why this might be so is because we do not have any data for advocates and their admission dates for the years preceding 2004 (our earliest resource is a list of middle-category practitioners who voted in the 2004 Election of Council). If any of the advocates whose names are mentioned in the written judgments in 2000–2004 had retired, left practice, or switched firms by 2004–2005, our heuristics would not have picked them up.</p><p>Despite some missing values, this is still a decent end-result as we now have a sizable number of lead counsel appearances along with their admission dates from which we can make our observations and findings, as long as we account for the fact that the numbers for the years from 2000 to 2004 have a higher margin of error.</p><h3>Exploring the Data</h3><p>We start with some quick exploratory data analysis by listing the names of the advocates with the highest number of appearances. (Note: the firm names or instructing firm names were taken from the written judgments the advocates last appeared in and might not necessarily represent the names of their current practices.)</p><h4>All Categories</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/863/1*TwxiYlEkpN2ZzGaboQc5_g.png" /><figcaption>As expected, the top 20 list is dominated by Senior Counsels</figcaption></figure><h4>Admiralty and Shipping</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-joVZTFP2HTgkNCJyr4uuA.png" /><figcaption>Top 10 advocates with most appearances in written judgments tagged with ‘Admiralty Law’</figcaption></figure><h4>Arbitration</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TF7-BI71u_C9HLRyaDrzgQ.png" /><figcaption>Top 10 advocates with most appearances in written judgments tagged with ‘Arbitration’</figcaption></figure><h4>Building and Construction</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k_Xivzwna_k9obNFCde9dA.png" /><figcaption>Top 10 advocates with most appearances in written judgments tagged with ‘Construction’</figcaption></figure><h4>Civil Procedure</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fVDk_UL_31zl3GIJ_2nskg.png" /><figcaption>Top 10 advocates with most appearances in written judgments tagged with ‘Civil Procedure’</figcaption></figure><h4>Companies</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oNy9Lx2Y-Mq6s9RlNQq8Mg.png" /><figcaption>Top 10 advocates with most appearances in written judgments tagged “Companies”</figcaption></figure><h4>Criminal Law</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*okdqR5KCXeH40-1nWEbk-g.png" /><figcaption>Top 10 advocates with most appearances in written judgments that have the Public Prosecutor as a party</figcaption></figure><h4>Equity and Trusts</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b8L7KogoKJ3XhRYifduLrA.png" /><figcaption>Top 10 advocates with most appearances in written judgments tagged with “Equity” or “Trusts”</figcaption></figure><h4>Family Law</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jiJl7FHlPbM8m5N6Qh5MSA.png" /><figcaption>Top 10 advocates with most appearances in written judgments tagged with ‘Family Law’</figcaption></figure><h4>Insolvency</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WqD33PuHiwpvziSUzefSzw.png" /><figcaption>Top 10 advocates with most appearances in written judgments tagged with ‘Insolvency’</figcaption></figure><h4>Intellectual Property</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y2RkiL69PCO5uDTbpY1YhA.png" /><figcaption>Top 10 advocates with most appearances in written judgments tagged “Trade Marks” or “Copyright” or “Intellectual Property”</figcaption></figure><h4>Tort</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dWQY3RSIoskf4sVstOOA-w.png" /><figcaption>Top 10 advocates with most appearances in written judgments tagged “Tort”</figcaption></figure><h3>Data Analysis</h3><p>Once we are reasonably confident that our data is generally reliable, we can now attempt to answer the questions of interest.</p><h4>Which group of advocates made the most appearances in written judgments?</h4><p>To answer this question, we plot, for every year, the number of lead counsels who appeared in written judgments after normalizing the numbers over the total number of represented parties for that year (the normalization is necessary because the raw numbers would vary by the number of represented parties each year).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/858/1*yEsl18uKh70C-fJgr8bXQw.png" /><figcaption>Proportion of appearances by group in all written judgments</figcaption></figure><ul><li>Advocates admitted in the 1990s have the highest share of appearances in written judgments since they overtook their 1980s counterparts in the year 2001.</li><li>They continue to be the dominant group even though their overall share appears to be declining since their peak in 2013.</li><li>The proportion of appearances by advocates admitted in the 1970s and 1980s has been steadily declining since 2000 while that of the younger advocates admitted in the 2000s/2010s has continually trended upwards.</li><li>The proportion of appearances by the 2000s group has nearly caught up with that of the 1980s group for written judgments in 2018.</li></ul><p><strong>Court of Appeal</strong>: The picture changes slightly if we limit the analysis to Court of Appeal judgments (those with ‘SGCA’ in the neutral citation):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/907/1*gUCdTYQeV0f6PFY2PmDctw.png" /><figcaption>Proportion of appearances by group in Court of Appeal judgments</figcaption></figure><ul><li>Here we see that the 1980s cohort’s share is a few percentage points higher than that of the 2000s cohort in 2018. This gap has narrowed over the years but at a slower pace compared to the earlier numbers when High Court appearances were included.</li><li>The 2010s cohort’s share drops significantly when it comes to appearances in Court of Appeal judgments.</li></ul><p><strong>Civil Trials</strong>: The picture again changes slightly if we limit the analysis to just civil trials (a written judgment is classified as a civil trial if its case number has “suit” but no “summons”, “appeal” or any other description that makes it clear that it is not a trial.)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/871/1*I9WD9EL0uSeas43t1V1Jmg.png" /><figcaption>Proportion of appearances by group in the written judgments identified as civil trial-related.</figcaption></figure><ul><li>We notice that the 1990s group are even more dominant when only written judgments involving a civil trial are included.</li><li>Similar to what we saw in the Court of Appeal analysis, the 2010s group’s share drops significantly here as well.</li></ul><h4>Which group of advocates has the highest number of different individuals appearing in written judgments?</h4><p>To answer this question, we only take into account, at most, one appearance per advocate per year (instead of counting all appearances as was done above). This tells us how many different advocates appeared for each year.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/808/1*BOPqyD2-pf_DsL_8mH5Mww.png" /><figcaption>Proportion of individuals by group in all written judgments</figcaption></figure><ul><li>The 1990s cohort has the highest number of different individuals appearing in written judgments since 2000.</li><li>The number of individuals from the 2000s cohort has surpassed the number of individuals from the 1980s cohort since 2017.</li><li>The number of individuals from the 2010s cohort is now nearly as high as the number of individuals from the 1980s.</li><li>This tells us that the advocates from the 1980s tend to make more appearances per advocate as compared to those from the 2000s/2010s.</li></ul><p><strong>Multiple appearances</strong>:<strong> </strong>To find out how many advocates from the 2000s and 2010s made multiple appearances, we plot the number of appearances together with the number of unique appearances and observe the differences:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/772/1*J8J_GjnOmlIUquiL_BN3Xw.png" /><figcaption>Unique vs non-unique appearances made by the 2000s and 2010s groups in all written judgments.</figcaption></figure><ul><li>From this plot we observe that the 2000s cohort have increasingly made more appearances per advocate.</li><li>The vast majority of the 2010s cohort have been limited to one appearance per advocate.</li></ul><h4>Have the appearance numbers for the 2000s and 2010s cohorts risen at a slower pace compared to their 1990s counterparts?</h4><p>To meaningfully compare the rate of change between the groups, we need to measure their appearance rates at comparable points in time. One way is to take the 10-year mark since the first batch of the group entered practice as the appropriate time-slice, and compare (1) the appearance rate of the 1990s group at year 2000; (2) the appearance rate of the 2000s group at year 2010; and (3) the appearance rate of the 2010s group at year 2018 (since we are not at 2020 yet). Like every other snapshot method of comparison, this may not perfectly capture all the relevant information but it gives us a basic idea of how the groups match up at roughly comparable points in time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/790/1*gsgjhCd4MRwTu0ZWB30ZlQ.png" /></figure><ul><li>The 1990s cohort had nearly 40% share of all lead counsel appearances in written judgments for the year 2000.</li><li>In contrast, the 2000s cohort barely had 9% share of all lead counsel appearances in written judgments for the year 2010.</li><li>The 2010s cohort currently has about 10% share of all lead counsel appearances in written judgments for the year 2018. (We do not yet have the numbers for 2020 but it is difficult to imagine how their appearance numbers could drastically increase over the next year or two based on its current trajectory.)</li><li>Therefore, we could conclude that the 2000s and 2010s cohorts have lagged behind the 1990s cohort in making lead counsel appearances during their initial years of practice.</li></ul><h3>Concluding Remarks</h3><p>This has been a challenging and satisfying exercise at the same time. If time permits, I may improve on this data set and attempt to answer other interesting and worthwhile questions about the Singapore legal profession, so if you have any ideas, feel free to reach out in the comments section.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eee245dc7e0f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Ultimate guide to setting up a Google Cloud machine for fast.ai course (deprecated)]]></title>
            <link>https://medium.com/ai-saturdays/ultimate-guide-to-setting-up-a-google-cloud-machine-for-fast-ai-version-2-f374208be43?source=rss-f52b5c7a1fb5------2</link>
            <guid isPermaLink="false">https://medium.com/p/f374208be43</guid>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[gpu]]></category>
            <dc:creator><![CDATA[How Khang]]></dc:creator>
            <pubDate>Wed, 03 Jan 2018 15:34:33 GMT</pubDate>
            <atom:updated>2019-02-13T02:26:23.576Z</atom:updated>
            <content:encoded><![CDATA[<h4>Latest: 13 Feb 2019 — This guide has been deprecated. Please refer to the official guide (https://course.fast.ai/start_gcp.html)</h4><h3>Official Guide:</h3><p><a href="https://course.fast.ai/start_gcp.html">GCP</a></p><p>– – – – – – – – – – – – – – – – – – – – – –</p><p>The guide below is archived and no longer works.</p><h3>1. <a href="https://cloud.google.com/free/">Sign up</a> on Google Cloud Platform Free Tier.</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oW2KzH3sh4AhzbqiUZaHUw.png" /><figcaption>12 months worth of US$300 credit (3 Jan 2018).</figcaption></figure><h3>2. Upgrade to a paid account (credit card required).</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xvvfPfSJgDp3y5vIpHKoXA.png" /><figcaption><a href="https://cloud.google.com/free/docs/frequently-asked-questions#upgrading-billing">https://cloud.google.com/free/docs/frequently-asked-questions#upgrading-billing</a></figcaption></figure><p>Note 1: Please note that some people have reported that Google Cloud does not accept debit cards.</p><p>Note 2: If you are unable to find an Upgrade button at this point, just continue with the next steps. The Upgrade button should appear at Step 5 below when you request for an increase in quota.</p><h3>3. Create a new project for the fast.ai course</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UvFX5sl4THJ2BIqj4Omonw.png" /><figcaption>Click on “Create Project”.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VPXnbttBuyVcSmm_1F0RzA.png" /><figcaption>Give your project an identifiable name and click “Create”.</figcaption></figure><h3>4. Set a default Compute Engine region</h3><p>Compute Engine → Settings</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EzbWC637IDh1YW2zW0P1OQ.png" /><figcaption>Set default region and zone (choose one from list).</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*37K7qqUwgWP2BhBTZG486g.png" /><figcaption>Zones with NVIDIA K80 GPU and Intel Broadwell CPU (3 Jan 2018).</figcaption></figure><h3>5. Request for increase in quota for GPU</h3><p>IAM &amp; Admin → Quotas</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qOTETSUTF4_C508z4NCrnA.png" /><figcaption>Select your project’s region.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eyq3rgYHcMqlY62fnoGHrg.png" /><figcaption>Select NVIDIA K80 GPUs (without “preemptible”) → Edit Quotas → Change to “1” → Submit Request.</figcaption></figure><h3>6. Receive email approval of quota increase</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mUbTqkLVpZab9C9BCTn2-g.png" /><figcaption>Approval email from Google Cloud Platform Support.</figcaption></figure><h3>7. Navigate to the Google Cloud Marketplace</h3><p>Compute Engine → Marketplace</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T3Ef7oiOQ8JULZ9NEFTqjQ.png" /><figcaption>Marketplace page</figcaption></figure><h3>8. Select the Deep Learning VM</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KiUMwH4wYMoh0rpJyCqk0g.png" /><figcaption>Filter by “Virtual Machines” and “Compute”</figcaption></figure><h3>9. Launch Deep Learning VM on Compute Engine</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dRGCGQbfexF5y08xYrZl0w.png" /><figcaption>Note that the OS is Debian 9 (and not Ubuntu)</figcaption></figure><h3>10. Fill in VM specs and select Pytorch 1.0 Preview</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-EW36Dq_dSN2Gwrv0LBkKQ.png" /><figcaption>Select GPU and check install GPU driver automatically</figcaption></figure><p>It is up to you to decide the number of cores and the amount of memory to allocate for your VM instance but if you have the budget for it, it is a good idea to go with 4 cores and &gt; 12 GB memory, as the Tesla K80 GPU comes with 12 GB memory.</p><h3>11. Deploying the VM</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GV-D-FFNNKDknaniaczFcA.png" /><figcaption>Warnings due to deprecated image being used</figcaption></figure><p>There appear to be warnings due to a deprecated image being used but so far I have not encountered any downstream problems so it may be safe to just ignore them for now.</p><h3>12. Check that VM is now running</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zkCRPOzpjI9H1YjdC_3rfQ.png" /><figcaption>Compute Engine → VM Instances</figcaption></figure><h3>13. Connecting to the VM via SSH</h3><p>There are a few methods of connecting to your Debian VM according to Google Cloud Platform’s official documentation (which can be found <a href="https://cloud.google.com/compute/docs/instances/connecting-to-instance">here</a>). Two of those methods will be mentioned briefly.</p><h4>Method 1: SSH from the browser:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mkMkb1vlrCnBSLzkV02dEw.png" /><figcaption>Requires <a href="https://www.google.com.sg/url?sa=t&amp;rct=j&amp;q=&amp;esrc=s&amp;source=web&amp;cd=14&amp;cad=rja&amp;uact=8&amp;ved=0ahUKEwi3pKbE8bzYAhUVSI8KHR79DhIQFghpMA0&amp;url=https%3A%2F%2Fkb.mailchimp.com%2Fcampaigns%2Fpreviews-and-tests%2Fenable-pop-ups-in-your-browser&amp;usg=AOvVaw1CeX6uS0IAdVJ2sRUTTBxj">enabling</a> popups on your browser (usually disabled by default).</figcaption></figure><p>While this method is the most convenient, it comes with a list of known issues that are officially documented <a href="https://cloud.google.com/compute/docs/ssh-in-browser">here</a>, which is why I recommend using the Google Cloud SDK command line method below.</p><h4>Method 2: Command line using Google Cloud SDK (recommended)</h4><p><strong>Step a</strong>. <a href="https://cloud.google.com/sdk/docs/">Download</a> and install Cloud Tools for your OS to use the Cloud SDK. (Windows users: (1) verify that the option to install bundled Python is checked and (2) ensure that you have <a href="https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html">PuTTY</a> installed).</p><p><strong>Step b</strong>. Run “gcloud init” to link your account, and select project and region.</p><p><strong>Step c</strong>. Run “gcloud compute ssh &lt;<em>your instance name</em>&gt; -- -L 8080:localhost:8080<em>”</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7uVOAt6g6L5FAqDJEaCM_A.png" /><figcaption>Successful login to Debian 9.5 VM and running nvidia-smi command</figcaption></figure><p>Note: You should always stick to the same method of connecting to your VM because your login usernames for SSH from browser and for Cloud SDK may be different. This may result in confusion as you will end up in completely different home directories after connecting to your VM. If you intend to alternate between the two methods, first configure your <a href="https://cloud.google.com/compute/docs/ssh-in-browser#login_username">SSH from browser</a> and <a href="https://cloud.google.com/sdk/docs/configurations#default_configuration">Cloud SDK</a> login usernames to be identical.</p><p>*To switch users when connecting via Cloud SDK, use the command: “gcloud compute ssh &lt;<em>your user name&gt;</em>@&lt;<em>your instance name&gt;”</em></p><h3>14. Connect to JupyterLab</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KG5HVZQ1JzmBNMDyEHHGSg.png" /><figcaption>Connected to JupyterLab</figcaption></figure><p>Launch your browser and connect using “localhost:8080”.</p><h3>**Stopping your VM instance</h3><h4>Important: Remember to stop your VM instance when you are done with your notebooks or you will continue to incur charges.</h4><h4>Method 1: Online console</h4><p>Compute Engine → VM instances</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UPkc3vwN-yBpUnZUjFMYcw.png" /><figcaption>Stopping your instance via the online console.</figcaption></figure><h4>Method 2: Command line on Cloud SDK</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YvRs7I5OjGFxTnri9Mu5JQ.png" /><figcaption>Enter the command: gcloud compute instances stop &lt;insert instance name&gt;</figcaption></figure><h4>Method 3: Use the Cloud Console <a href="https://cloud.google.com/console-app/">mobile app</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mJ8D1CWPlcYAcYqiwttD8w.png" /><figcaption>Allows you to check on your instances anytime on your smartphone.</figcaption></figure><h3>The instructions beyond this point were for the fast.ai course part 1 v 2 and no longer apply as of October 2018. They are archived here in case there is a need to update them to provide another option to the Deep Learning Image option above.</h3><p>— — — — — — — — — — — — — — — — — — — — — — — — — — — — — —</p><h3>(Ignore) 7. Configure firewall for Jupyter notebook</h3><p>VPC Network → Firewall rules</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jojDM-gEvA-h419yNoJhaw.png" /><figcaption>The default firewall rules.</figcaption></figure><p>Create a firewall rule if your project does not already have one for tcp:8888.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MP9eCdIKx-xoByE6GMg9Wg.png" /><figcaption>Name: “default-allow-jupyter” and Target tags: “jupyter”</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*55nKSZ5In94DJRUfJxi7xQ.png" /><figcaption>Source IP ranges:“0.0.0.0/0” and Specified protocols and ports: “tcp:8888-8889”</figcaption></figure><p>You may choose some other unused port number but the typical number is 8888. It is permissible to specify a range of ports such as in the above case.</p><h3>(Ignore) 8. Create your virtual machine instance for fast.ai</h3><p>Compute Engine → VM instances → Create</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CIYg6DPccLAb7Gkhs0B9Yg.png" /><figcaption>CPU platform: “Intel Broadwell or later” and Number of GPUs: “1”</figcaption></figure><p>It is up to you to decide the number of cores and the amount of memory to allocate for your VM instance but if you have the budget for it, it is a good idea to go with 4 cores and 26 GB memory (ideally &gt; 12 GB, as the Tesla K80 GPU comes with 12 GB memory).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b2qdmLZEfPDWxxvOsGQFPw.png" /><figcaption>Boot disk: “Ubuntu 16.04 LTS” and “50 GB SSD”, Firewall: Check both, Network tags: “jupyter”</figcaption></figure><p>Again, it is up to you to decide the type of persistent disk to use as the boot disk but SSD is recommended for its higher speed if you have the budget for it. (For the fast.ai version 2 course, you minimally need 20 GB.)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rnRZSZvgCSyEpdgKxCG_qw.png" /><figcaption>Newly created VM instance is running.</figcaption></figure><h3>(Ignore) 9. (Optional) Convert the IP address to static</h3><p>VPC network → External IP addresses</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PgwsPPxMDUzLFud-PQyNNg.png" /><figcaption>Convert IP address to “Static” and give it a name.</figcaption></figure><p>**Following this step results in <a href="https://cloud.google.com/compute/pricing#ipaddress">ongoing charges</a> for the static IP address. Skip this step if you wish to save on the cost of a static IP address (US$0.01/hour at time of writing).</p><h3>(Ignore) 10. Connecting to the VM instance</h3><p>There are a few methods of connecting to your Ubuntu VM according to Google Cloud Platform’s official documentation (which can be found <a href="https://cloud.google.com/compute/docs/instances/connecting-to-instance">here</a>). Two of those methods will be mentioned briefly.</p><h4>Method 1: SSH from the browser:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mkMkb1vlrCnBSLzkV02dEw.png" /><figcaption>Requires <a href="https://www.google.com.sg/url?sa=t&amp;rct=j&amp;q=&amp;esrc=s&amp;source=web&amp;cd=14&amp;cad=rja&amp;uact=8&amp;ved=0ahUKEwi3pKbE8bzYAhUVSI8KHR79DhIQFghpMA0&amp;url=https%3A%2F%2Fkb.mailchimp.com%2Fcampaigns%2Fpreviews-and-tests%2Fenable-pop-ups-in-your-browser&amp;usg=AOvVaw1CeX6uS0IAdVJ2sRUTTBxj">enabling</a> popups on your browser (usually disabled by default).</figcaption></figure><p>While this method is the most convenient, it comes with a list of known issues that are officially documented <a href="https://cloud.google.com/compute/docs/ssh-in-browser">here</a>, which is why I recommend using the Google Cloud SDK command line method below.</p><h4>Method 2: Command line using Google Cloud SDK (recommended)</h4><p><strong>Step a</strong>. <a href="https://cloud.google.com/sdk/docs/">Download</a> and install Cloud Tools for your OS to use the Cloud SDK. (Windows users: (1) verify that the option to install bundled Python is checked and (2) ensure that you have <a href="https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html">PuTTY</a> installed).</p><p><strong>Step b</strong>. Run “gcloud init” to link your account, and select project and region.</p><p><strong>Step c</strong>. Run “gcloud compute ssh &lt;<em>your instance name</em>&gt;<em>”</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rDyYpxZ1s27dMsLBCLoo5Q.png" /><figcaption>Successful connection to VM running on Ubuntu 16.04.3 LTS.</figcaption></figure><p>Note: You should always stick to the same method of connecting to your VM because your login usernames for SSH from browser and for Cloud SDK may be different. This may result in confusion as you will end up in completely different home directories after connecting to your VM. If you intend to alternate between the two methods, first configure your <a href="https://cloud.google.com/compute/docs/ssh-in-browser#login_username">SSH from browser</a> and <a href="https://cloud.google.com/sdk/docs/configurations#default_configuration">Cloud SDK</a> login usernames to be identical.</p><p>*To switch users when connecting via Cloud SDK, use the command: “gcloud compute ssh &lt;<em>your user name&gt;</em>@&lt;<em>your instance name&gt;”</em></p><h3>(Ignore) Step 11: Run the bash setup script</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hOkBPvmfCxXlEYj4c5z4QQ.png" /></figure><p>curl https://raw.githubusercontent.com/howkhang/fastai-v2-setup/master/setup.sh | bash</p><p>You will be automatically disconnected after the script has finished running in order for the VM to do a reboot.</p><h3>(Ignore) Step 12: Reconnect to the VM instance</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0LLQlBx2xSBv5J0Kef7v2g.png" /><figcaption>Run “jupyter notebook”</figcaption></figure><h3>(Ignore) Step 13: Connect to Jupyter notebook</h3><h4>Method 1: Connecting using your VM’s static IP address</h4><p>Use this method only if you went through step 9 to obtain a static IP address.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l-GZCinWk_erS7MhO8bdZg.png" /></figure><p>Launch your browser and connect using your VM’s static IP address followed by the port number (in this case, “:8888”). For the token, copy the long string of characters shown after the “token=” .</p><h4>Method 2: Connecting using ssh and port forwarding</h4><p>Use this method if you skipped obtaining a static IP address in step 9 or if you encounter difficulties connecting to your Jupyter notebook using method 1.</p><p><strong>Step a. </strong>Open another terminal (or Google Cloud SDK Shell) on your local machine and run the following command for port forwarding:</p><p>gcloud compute ssh &lt;<em>your instance name</em>&gt; --ssh-flag=“-L” --ssh-flag=“8888:localhost:8888”</p><p>You should see a new VM terminal pop up. You should now have a total of two local terminals (both showing your gcloud compute ssh commands) and two VM terminals (one showing the jupyter notebook server running and the other just showing the command line prompt).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eDvwkM6Jx1F9fYJoxPum2w.png" /></figure><p><strong>Step b. </strong>Connect to Jupyter notebook using localhost:8888</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ELUsABXE_H6Y_A1Xif578Q.png" /></figure><p>Launch your browser and connect using “localhost” followed by the port number (in this case, “:8888”). For the token, copy the long string of characters shown after the “token=”.</p><h3>(Ignore) Step 14: Get ready to do some serious learning!</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qTSecq4KZZgLOc8piJ0mzQ.png" /></figure><p>And we are good to go! Let me know if you face any issues or have any feedback on this guide.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f374208be43" width="1" height="1" alt=""><hr><p><a href="https://medium.com/ai-saturdays/ultimate-guide-to-setting-up-a-google-cloud-machine-for-fast-ai-version-2-f374208be43">Ultimate guide to setting up a Google Cloud machine for fast.ai course (deprecated)</a> was originally published in <a href="https://medium.com/ai-saturdays">AI Saturdays</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Windows guide to setting up a Google Cloud instance for the fast.ai course]]></title>
            <link>https://medium.com/ai-saturdays/the-windows-guide-to-setting-up-a-google-cloud-instance-for-the-fast-ai-course-fe2d61f098ca?source=rss-f52b5c7a1fb5------2</link>
            <guid isPermaLink="false">https://medium.com/p/fe2d61f098ca</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[gpu]]></category>
            <category><![CDATA[windows-10]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[How Khang]]></dc:creator>
            <pubDate>Fri, 29 Dec 2017 16:31:34 GMT</pubDate>
            <atom:updated>2018-01-06T14:49:15.073Z</atom:updated>
            <content:encoded><![CDATA[<h4>Updated for fast.ai version 2</h4><p>This is a quick guide for the Windows users who are attending <a href="https://nurture.ai/ai-saturdays">AI Saturdays</a> and need a cloud instance with GPU to run the <a href="http://course.fast.ai/">fast.ai</a> notebooks.</p><h4>Step 1: Follow the steps <a href="https://medium.com/@howkhang/ultimate-guide-to-setting-up-a-google-cloud-machine-for-fast-ai-version-2-f374208be43">here</a> until you complete Step 6.</h4><h4>Step 2: Download and install the <a href="https://cloud.google.com/sdk/docs/#windows">Google Cloud SDK</a> interactive installer for Windows with Python bundled (link edited).</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6Co4fngOl5u0khm1ZX0kGQ.png" /><figcaption>Final screen for installation</figcaption></figure><p>Run the interactive installer with all the default options (ensure option to install Python is checked) until installation is complete.</p><h4>Step 3: Link your Google account to the Cloud SDK</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9PLEajpdu1ufWuf9T1G3Wg.png" /></figure><p>Log in to your Google account and select the project you created in Step 1. Select ‘Y’ to configure Google Compute Engine and check your zone.</p><h4>Step 4: Click <a href="https://raw.githubusercontent.com/howkhang/fastai-v2-setup/master/windows/gpc-gpu.ps1">here</a> to download the setup script (link updated)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bApd20RZ6J-TnG-t7l2acw.png" /><figcaption>Updated on 2 Jan 2018</figcaption></figure><p>Right-click on the page, “<em>Save as…</em>” and select a folder on your machine. But before you hit <em>Save</em>, add double quotes <strong>“ ”</strong> around the file name and choose <em>Save as type: All Files</em>.</p><blockquote><strong>Edited </strong>(1 Jan 2018): I have updated the script based on feedback from the last AI Saturday session. The later version now obtains your Google Cloud SDK default configuration settings, checks for your regional quota limits and includes error handling. I will also be updating the screenshots as a result.</blockquote><h4>Step 5: Run the setup script using PowerShell</h4><p>Go to the folder where you downloaded the script. Select the file, right-click and choose “<em>Run with PowerShell</em>”. The PowerShell terminal will launch and create a Google Cloud instance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XsL1JjhKUnRl2Ky4et8Dlw.png" /><figcaption>Updated 2 Jan 2018</figcaption></figure><h4>Step 6: Connecting to your Google Cloud instance</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TiIg3270fGKHBR_VPt_kaA.png" /></figure><p>There should now be two new files in the folder. Open the file <em>fastai-instance-&lt;#&gt;-commands.txt</em> and copy the command for connecting to your VM instance and paste it into a new PowerShell terminal (to launch, click the Windows Start Menu and type PowerShell).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4_S5A_l1ZAFW1YWmReDNcA.png" /></figure><p>A PuTTY terminal should launch and connect you to the VM instance.</p><h4>Step 7: Run the bash script to install all the drivers and libraries</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hOkBPvmfCxXlEYj4c5z4QQ.png" /></figure><p>curl https://raw.githubusercontent.com/howkhang/fastai-v2-setup/master/setup.sh | bash</p><h4>Step 8: Reconnect to your Jupyter notebooks for fast.ai</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a4Rf7oS9VkweZlP_INOSxQ.png" /></figure><p>Reconnect and in the VM terminal, type <em>jupyter notebook </em>and hit Enter.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PEL-Bt4endxYLTY15AVJdg.png" /></figure><p>Copy the IP address, paste it into your browser with an added <em>:8888 </em>behind.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l-GZCinWk_erS7MhO8bdZg.png" /></figure><p>For the token, copy the long string of characters shown after the “<em>=</em>” .</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qTSecq4KZZgLOc8piJ0mzQ.png" /></figure><p>And we are good to go! Happy training!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fe2d61f098ca" width="1" height="1" alt=""><hr><p><a href="https://medium.com/ai-saturdays/the-windows-guide-to-setting-up-a-google-cloud-instance-for-the-fast-ai-course-fe2d61f098ca">The Windows guide to setting up a Google Cloud instance for the fast.ai course</a> was originally published in <a href="https://medium.com/ai-saturdays">AI Saturdays</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>