<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Eric Turner’s Blog - Medium]]></title>
        <description><![CDATA[Eric Turner’s Blog - Medium]]></description>
        <link>https://medium.com/etdev?source=rss----5688d702725d---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Eric Turner’s Blog - Medium</title>
            <link>https://medium.com/etdev?source=rss----5688d702725d---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 20:58:02 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/etdev" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Finding success as a “Gaijin Engineer” in Japan]]></title>
            <link>https://medium.com/etdev/https-medium-com-etdev-how-to-find-a-job-as-a-software-developer-in-japan-1f364da15502?source=rss----5688d702725d---4</link>
            <guid isPermaLink="false">https://medium.com/p/1f364da15502</guid>
            <category><![CDATA[careers]]></category>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[japan]]></category>
            <category><![CDATA[job-hunting]]></category>
            <category><![CDATA[software]]></category>
            <dc:creator><![CDATA[Eric Turner]]></dc:creator>
            <pubDate>Sat, 23 Nov 2019 09:30:38 GMT</pubDate>
            <atom:updated>2019-11-23T12:07:27.491Z</atom:updated>
            <content:encoded><![CDATA[<h3>Finding Success as a “Gaijin Engineer” in Japan</h3><figure><img alt="Group viewing computer" src="https://cdn-images-1.medium.com/max/720/0*4IG6N85BzeMLmz2k" /></figure><p><em>Originally published on the </em><a href="https://japan-dev.com/blog/how-to-find-a-job-as-a-software-developer-in-japan"><em>Japan Dev Blog</em></a><em>. Note: “Gaijin Engineer” is a nod to </em><a href="https://medium.com/@xevix/gaijin-engineer-in-tokyo-aaa9be8919b2"><em>Alejandro’s excellent post</em></a><em> about working as a developer in Japan.</em></p><p>Finding a job as a software developer in Japan can be tough for foreigners. There are a lot of great opportunities, but they’re often hard to find. Especially for developers searching for their first job in Japan. The language gap combined with a lack of online resources makes separating the good tech companies from the less desirable ones a challenge.</p><p>It’s not that there are no resources available. In fact there’s a myriad of Japan-focused job boards and recruitment agencies out there. The problem is that <em>the data you actually want</em> can rarely be found online at all, let alone in English. And while the reports of “black companies” may be overstated at times, it’s true that you need to be selective when choosing an IT job in Japan. Especially when you’re a foreigner.</p><p>In places like the US software engineering has become one of the most popular careers, but this isn’t the case in Japan. Many Japanese companies see programming as a necessary evil. To them it’s low-level work that’s not worth investing in. However, as Japan’s internet tech industry matures, more and more companies are seeing the value of software development and things have improved rapidly in the last 5–10 years. I feel the industry has reached a point where there’s a critical mass of good software jobs for foreigners that want to work here.</p><p>So the question is: <em>how do you find the good jobs?</em> In this post I’ll explore this question and offer some resources and advice based on my own experiences searching for jobs as a developer in Tokyo. I know for a fact that working in Japan can be a deeply fulfilling experience, so my hope is that I can help other foreigners avoid the pitfalls I’ve faced and achieve success here in Japan.</p><h3>What I won’t cover</h3><p>This is not a guide on the mechanics of physically moving to Japan. That’s not specific to software engineers, and many such guides have been written already. You can find them online, and the requirements will be pretty much the same for everyone. So if you’re looking for general info about the logistics of your move to Japan, then I recommend consulting them instead.</p><p>For example, <a href="https://www.tokyodev.com/2016/06/22/first-job-programmer-japan/">TokyoDev’s guide</a> on finding your first programming job in Japan is a good place to start. You can also check out <a href="https://www.bfftokyo.com/ultimate-guide-for-a-japan-visa/">BFFTokyo</a> for anything else related to getting settled and thriving in Japan (learning Japanese, making Japanese friends, visas, credit cards etc.). These topics are important too, but in this post I’m going to try to avoid them. I won’t go into how much money you should bring or what visa you should get. Instead I’d like to focus on how to find a software development job <em>that you’ll actually enjoy</em> and avoid the ones you won’t.</p><figure><img alt="Educate yourself" src="https://cdn-images-1.medium.com/max/720/0*Lwsq7VzcwaRGnsKl" /></figure><h3>First things first: Educate yourself</h3><p>If you want to find a software job in Japan (that’s actually good), the first step is research. In order to thrive as a developer here, it’s crucial for you to understand what kind of job you want. You’ll then need to determine which tech companies in Japan are actually willing to hire foreign engineers for that position, and most importantly, which ones can actually provide the type of environment you expect.</p><p>Maybe in part due to Japan’s historic culture of “lifetime employment”, I feel that tech recruiting here is still in its infancy. There are a few general Glassdoor-like sites like <a href="https://www.vorkers.com/">OpenWork</a> and <a href="https://en-hyouban.com/">EN-Hyouban</a>, but they’re all in Japanese. The data collected there is somewhat useful, but it’s often not relevant to non-Japanese engineers. That’s why I built <a href="https://japan-dev.com/">Japan Dev</a> as a resource aimed directly at foreigners who want to work at tech companies here.</p><p>I recommend browsing the Japan Dev company list page to get a holistic view of the foreigner-friendly companies hiring developers in Japan. Everyone has their own criteria, so you should decide yours and then research which companies meet them. Whether you’re looking for an English-speaking environment, remote work or visa sponsorship, you should be able to come up with a list of tech companies in Japan that fulfill your requirements.</p><blockquote>Pro tip:<em> Know what kind of job you want, and spend time researching the best companies offering those jobs. Don’t set yourself up for failure by assuming you’ll be happy with any old job as long as you’re in Japan.</em></blockquote><h3>Foreigner-friendly resources for developers</h3><p>Here are a few more hand-picked resources that you can use to improve your understanding of the programming jobs available to you:</p><p><a href="https://japan-dev.com/">Japan Dev</a> — We offer the most exhaustive list of top foreigner-friendly tech companies hiring developers in Japan. There’s also salary data for most companies as well as “pros” and “cons” for each.</p><p><a href="https://linkedin.com/">LinkedIn</a> — Many top companies in Japan advertise software jobs here. LinkedIn also recommends companies based on the info in your profile, which is useful.</p><blockquote><strong><em>Pro tip:</em></strong><em> Set your LinkedIn location to Tokyo even if you’re not here yet. You’ll get more interest from recruiters.</em></blockquote><p><a href="https://www.tokyodev.com/">TokyoDev</a> — Great resource for programming jobs in Tokyo. Pretty much all jobs posted here include visa support and most don’t require Japanese skills.</p><p><a href="https://angel.co/location/tokyo">AngelList</a> — More startup jobs. Most are globally-minded companies.</p><p><a href="https://justa.io/">Justa.io</a> — Focused on Japanese startups (and not just for software developers), but leaning toward English-friendly companies.</p><p><a href="https://hntokyo.slack.com/">HN Tokyo Slack</a> — Connect directly with fellow developers in Tokyo. This is especially valuable for people who are not in Japan yet.</p><p><a href="https://stackoverflow.com/jobs">StackOverflow Jobs</a> — StackOverflow is an English site, so the companies in Japan posting jobs here are typically pretty global.</p><h3>General-purpose Japan job boards</h3><p>There are tons of other more generalized job boards (Gaijinpot, Daijob etc.), but they can be overwhelming. The above resources either curate jobs for English-speaking developers or attract the type of company interested in hiring foreign developers, so I recommend focusing your efforts on these sites (and others like them; let me know if you have others!) as they provide a better signal to noise ratio for those specifically seeking programming jobs.</p><p>In general, the more you educate yourself at this stage, the better off you’ll be when you’re talking to recruiters and actually meeting with companies. So definitely take the time to vet companies and gather as much information as you can.</p><figure><img alt="Group of people blackboard" src="https://cdn-images-1.medium.com/max/720/0*1aHaJUq_0ctiIX7F" /></figure><h3>Utilizing developer events</h3><p>The above set of resources should provide you with quite a lot of data about companies as well as specific job opportunities. Once you’re confident that a given company is a good fit, applying directly through one of those sites can be a good option. But this should not be your only plan of attack. If you’re in Japan already, then <strong>events are invaluable</strong> for finding jobs. It’s a well-known fact that many of the best jobs will never be posted online, and this definitely holds true in the Japan developer community.</p><p>There’s just no replacement for physically coming to Tokyo and meeting people in person. You can get the latest information from fellow software developers already working in Japan, and they’ll be much more likely to refer you to their company if they have a chance to meet you first.</p><h3>Finding someone to vouch for you</h3><p>Did I mention that companies in Japan love referrals? Because they do. A lot of companies here get 50%+ of their employees from employee referrals because it’s a win-win for the company and the employee. Even if the company has to pay a fee to the employee, it’s usually much less than what they would’ve paid to a job board or recruiter. With a referral, the employee gets money and the company gets to reward one of their members and create some good will in them towards the company. This means that many companies’ employees <em>want</em> to find people looking to work with them and are even financially incentivized to do so.</p><p>In other words, the more people you can talk to at your target companies, the better your chances will be of landing a development position with them. And referrals are one of the best ways to make connections with many companies.</p><h3>Finding the most impactful English-friendly events for developers</h3><p>Tokyo has a never-ending stream of programming and IT related events, so there should pretty much always be <em>something</em>, but here are a few of my personal recommendations for software developers interested in attending events in Japan:</p><p><a href="https://tokyotech.com/">TokyoTech</a> — This isn’t actually a meetup, but it’s my favorite resource for <em>finding meetups</em> in Tokyo. It showcases most of the upcoming tech-related Tokyo meetups in one place so you don’t need to go hunting for them across the internet.</p><p><a href="https://www.meetup.com/tokyotechmeetup/">Tokyo Tech Meetup</a> — A recent addition, this bilingual event is great for meeting fellow developers — plus it’s free!</p><p><a href="https://hntokyo.doorkeeper.jp/">Hacker News Tokyo</a> — This is a great event for networking in Tokyo. There are always plenty of software engineers and designers as well as entrepreneurs and anyone else interested in tech or startups.</p><p><a href="https://www.meetup.com/devjapan/">Dev Japan</a> — Originally an event where programmers could go to work on projects, Dev Japan has been expanding lately and sponsoring more and more events in Tokyo.</p><p><a href="https://www.meetup.com/">Meetup.com</a> — Many of Tokyo’s developer events can be found here. There’s something pretty much every night.</p><p><a href="https://www.doorkeeper.jp/?locale=en">Doorkeeper</a> — Doorkeeper is another site for organizing meetups; it’s home to events like the venerable Tokyo Rubyist meetup.</p><figure><img alt="Recruiter meeting" src="https://cdn-images-1.medium.com/max/720/0*zjtit_G_7iPyrTUC" /></figure><h3>What about recruiters?</h3><p>Recruiting firms can be a valuable tool for finding software engineering jobs in Japan. In fact, anecdotally speaking, I think this is one of the most popular ways to find jobs here. However, I feel it’s important to understand how recruiting in Japan works before enlisting their help. I would also advise against using recruiters as your <em>only</em> avenue when searching for jobs.</p><p>The thing about recruiters is that <strong>each recruiter only works with a subset of companies</strong>. Recruiting firms have contracts with specific partner companies, and those are the ones they’ll introduce you to. Thus, it’s usually a good idea to at least talk to a few different recruiters to get as wide of a view as possible of the jobs available to you before signing a contract.</p><p>Good recruiters will be honest with you about your options and not try to push you toward a company unless it’s a genuinely good fit, but you do need to be a little careful. Recruiting fees can be <strong>25–50% (or more) of an applicant’s yearly salary</strong>, which means recruiters have a massive financial incentive to convince you to join one of <em>their</em> companies. This is why you shouldn’t put all your eggs in one basket and make sure you talk to several different ones.</p><p>When working with recruiters, always do your own research and only agree to deals that are mutually beneficial for you from a business standpoint. And make sure you arm yourself with knowledge so you can avoid falling for sales tactics.</p><figure><img alt="Books frustrated" src="https://cdn-images-1.medium.com/max/720/0*xFEMxpiFnsQ0zC00" /></figure><h3>Do I need to speak Japanese?</h3><p>No, but the more Japanese you speak, the easier finding a programming job in Japan will be. This may sound obvious, but it’s profoundly true in my experience. The number of software development opportunities available to you will correlate directly to your Japanese level. As your skills improve, doors will open. However, this doesn’t mean you’re totally sunk if you <em>don’t</em> speak any Japanese. It just means that you’ll need to compensate by being stronger in another area.</p><p>For example, the stronger your technical skills, the more acceptable it will be for you to not speak Japanese. In fact if your Japanese skills are non-existent and you want to move to Japan and work as a programmer ASAP, then it might make more sense to drill <a href="https://leetcode.com/">Leetcode</a> and brush up on your technical interviewing skills than to try and learn Japanese. You can also work on improving your communication skills and ability to sell yourself, as these will also have a direct impact on your performance in technical interviews with Japanese companies.</p><blockquote><strong><em>Pro tip:</em></strong><em> No one in Japan cares about the JLPT.</em></blockquote><p>Well, some other foreigners might care a little bit. But very few Japanese companies even know what the JLPT is, let alone what the different levels actually mean. And if it matters, yes I’m speaking from experience here (I have N1). What I found was that the higher levels <em>might</em> help you get from the resume stage to the interview stage at certain companies, but it’s literally useless after that (not that you’d need to rely on a resume screening anyway since you got a referral, right?). The JLPT still has a lot of value as a way to measure your learning progress and keep you motivated, you just shouldn’t expect companies to roll out the red carpet for you because you passed it.</p><p>The only thing that actually matters after the resume screening stage is your communication during the interviews themselves. At the end of the day, companies have requirements, and either you fulfill them or you don’t. And when it comes to Japanese skills, this means you need to show one of two things to get hired: (1) enough Japanese skill to do the job or (2) enough passion that they’re willing to trust that you’ll learn. If a company requires Japanese and you can’t show either of these, they won’t hire you. So if I were you I’d focus on practical speaking skills and company research rather than test prep.</p><figure><img alt="Tokyo office building" src="https://cdn-images-1.medium.com/max/720/0*Xa00_pCxXZdgR4Ak" /></figure><h3>Do I need to move to Tokyo?</h3><p>This really depends. I personally believe that being in Tokyo (as opposed to somewhere else in Japan) increases the chances of finding a fulfilling job as a development job by magnitudes. Which is not to say that there are no good jobs elsewhere. There are tech companies throughout Japan, it’s just that the vast majority of foreigner-friendly ones are in Tokyo.</p><p>There are a handful of big companies with offices in places like Fukuoka (<a href="https://japan-dev.com/companies/line">LINE</a>, <a href="https://japan-dev.com/companies/mercari">Mercari</a>) but the number of programming jobs that provide the type of environment sought by most foreigners will be much smaller.</p><p>Outside of Tokyo, salaries will also usually be lower, even relative to the lower cost of living. There will be fewer events and the ones there are will have fewer foreigners and less English. Having said that, there <em>are</em> still foreign software developers working happily from places like Fukuoka and Osaka, and the number of progressive startups is continuing to increase and the situation outside Tokyo seems like it will continue to improve. Fukuoka recently introduced a special visa to encourage people to open startups there, and Kyoto has also become somewhat of a hot spot in the last couple of years.</p><p>So it’s up to you. Being in Tokyo will make it a lot easier to progress in your career, but it’s at least worth searching elsewhere for those who don’t want to (or can’t) move to here.</p><h3>What about remote jobs at foreign companies</h3><p>While it can be tricky to pull off, working remotely for a foreign company while living in Japan is a viable option for some people. This opens up an entire new set of companies, and is also worth looking into if you can get a visa that allows it and you don’t mind the full-remote lifestyle. Keep in mind that while the number of companies allowing remote work is definitely on the rise, most companies still require remote workers to be in the same country as the rest of their team (or at least the same time zone).</p><h3>Resources for remote work in Japan</h3><p>It may be fairly rare, but I have heard of people working remotely from Japan, so here are a few resources for finding remote jobs for developers:</p><p><a href="https://remotive.io/remote-companies">Remoteive.io</a></p><p><a href="https://remotehub.io/">RemoteHub</a></p><p><a href="https://news.ycombinator.com/item?id=21126014">Hacker News “Who’s Hiring”</a> threads (Google the one for the current month)</p><p><a href="https://github.com/remoteintech/remote-jobs">Github — remote-jobs</a></p><p><a href="https://github.com/lukasz-madon/awesome-remote-job">Github — awesome-remote-job</a></p><p><a href="https://weworkremotely.com/">WeWorkRemotely</a></p><figure><img alt="Man in suit" src="https://cdn-images-1.medium.com/max/720/0*_4ysiXz_vO6Tl8Uc" /></figure><h3>Japan’s interview process for developers</h3><p>I’m not sure there’s much to say about the actual interview process for developer jobs here in japan. If you’re applying to companies from abroad they’ll typically do at least the first few interviews via video chat, in fact flying candidates out to Japan for interviews is still relatively rare (with a few exceptions). Interview dress code depends on the company just like in the US and most other places. Banks will typically expect you to wear a suit whereas startups won’t care what you’re wearing.</p><p>Hardcore algorithm and data structure-focused interviews are rare, although programming challenges and take-home tests are common. Every company will ask you why you want to work there so you should have a good reason lined up. Many will also ask you specific questions about their product or service so you should actually use it beforehand. Most companies will have 3–4 rounds of interviews before making an offer, and the vast majority will ask you your current and expected salary pretty early in the process, so have your negotiation strategy ready (I’m a proponent of the <a href="https://www.kalzumeus.com/2012/01/23/salary-negotiation/">Patio11 method</a> personally, but that’s a whole topic on its own).</p><h3>Lastly: Try to stay realistic</h3><p>Pretty much any semi-skilled, English-speaking software engineer who wants a job as a job in Japan could probably find one if they just search. But foreigners should be careful when looking for a job in Japan, especially in the software industry. It’s crucial that you do your own research and find a company that you can genuinely see yourself working for.</p><p>A lot of people dream about living in Japan, and some of them get so focused on that dream that they forget to do their due diligence and end up in jobs they don’t really want. Working in Japan can be an amazing experience, but the novelty of being here will wear off, and when it does you need to make sure you’re in an environment where you’ll be okay with that. Being in Japan doesn’t remove the need to keep moving forward in your career and learning new skills.</p><p>We’ve all heard the horror stories from engineers who moved to Japan and had bad experiences at tech companies here. The truth is that if you’re not careful, it’s easy to end up in a poor environment as a programmer here. And this is especially true if you accept an offer from a company just because it’s in Japan. As someone who has spent my entire career as an engineer in Japan, the job landscape here for foreign developers has never been better, but you still need to be discerning.</p><p>The truth is there are still a huge number of companies out there that wouldn’t be a good fit for the majority of English speakers, yet may be willing to hire them anyway. So if you choose a company because you’re desperate, or move here as an escape, there’s a good chance you’ll end up at one of them. And then you’ll be the next foreigner telling horror stories about Japan’s IT industry.</p><p>Knowledge is power when it comes to working in IT in Japan. If you do your research and don’t compromise until you find somewhere you can thrive, you’ll likely have an amazing time in Japan. On the other hand, if you exhaust all of the progressive, foreigner-friendly companies and still can’t find a position, consider putting your move to Japan off and spending some time to improve your Japanese or your tech skills. Japan’s not going anywhere, and if you really want to end up here, you will.</p><p>I think the best litmus test when deciding whether to accept a tech job in Japan is this: Would you be genuinely excited about this job <em>even if it weren’t in Japan?</em></p><p>If the answer is yes, then go for it! That’s what I did, and I’ve never regretted it a day in my life.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1f364da15502" width="1" height="1" alt=""><hr><p><a href="https://medium.com/etdev/https-medium-com-etdev-how-to-find-a-job-as-a-software-developer-in-japan-1f364da15502">Finding success as a “Gaijin Engineer” in Japan</a> was originally published in <a href="https://medium.com/etdev">Eric Turner’s Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Graphs in Ruby Pt. 1.5 — Refactoring]]></title>
            <link>https://medium.com/etdev/graphs-in-ruby-pt-1-5-refactoring-933a198f150?source=rss----5688d702725d---4</link>
            <guid isPermaLink="false">https://medium.com/p/933a198f150</guid>
            <category><![CDATA[refactoring]]></category>
            <category><![CDATA[ruby]]></category>
            <category><![CDATA[graph-theory]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Eric Turner]]></dc:creator>
            <pubDate>Wed, 31 Jan 2018 15:31:04 GMT</pubDate>
            <atom:updated>2018-01-31T15:30:09.117Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Yx7KNaTteVDA1XyMD37mjQ.jpeg" /></figure><p>Since my <a href="https://etdev.me/graphs-in-ruby-pt-1-representation-26b0750be2d4">previous post</a> on representing graphs in ruby, I’ve changed my mind about a few things. I went through and did some refactoring to achieve what I believe is a superior ruby implementation of the two graph types: Adjacency Matrix and Adjacency List. In this post I’d like to quickly go over how I changed the code, so that my later posts on this topic make sense.</p><h3>The situation pre-refactor</h3><p>Just as a reminder, here’s what the project’s directory structure looked like when I wrote my original post introducint the code:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/730/0*UIUHD2K-nCAsiMKW.png" /></figure><p>As you can see, I used the concept of Edge Strategies and Storage Strategies and created factory classes to create these for a single Graph base class that mostly just delegated to one of these strategies to achieve various behavior. I had versions of these that were tied to List and Matrix Graphs directly, and folders to store these.</p><p>Here’s the original Graph class:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/865a6e8cda42aead7224ba10702f6024/href">https://medium.com/media/865a6e8cda42aead7224ba10702f6024/href</a></iframe><p>This is one way of achieving some level of decoupling, but through refactoring I came up with a different method that I think I like better.</p><h3>The updated implementation</h3><p>Here’s the new file structure I ended up with:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/408/0*_ip8IxZAjLqDaAIv.png" /></figure><p>Clearly this is much simpler than the previous screenshot, which reflects the fact that the code has been simplified as well. Despite this, one thing that has not changed is the fact that we have one central Graph class:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/358f567b8f8efefd29e75d3c528b0117/href">https://medium.com/media/358f567b8f8efefd29e75d3c528b0117/href</a></iframe><p>The class has grown somewhat, but its behavior requires less delegation and instead it contains most of the logic needed to represent a graph. There is still some polymorphism happening, but I’ve encapsulated it to the vertex_list_class method, which controls what kind of list we&#39;re using for the vertexes in the base element array.</p><p>Doing this removes the need to do all of the stuff with factories and strategies that I was doing in my previous post. Now all we need to do is add an ArrayVertexList class that fulfills the same interface as our ListNode class, which allows us to use them interchangeably. This is now the only difference between an adjacency list-based graph and an adjaceny matrix-based graph.</p><p>Here’s what the ArrayVertexList looks like:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/601875b2e980d2ead9297cfae969c764/href">https://medium.com/media/601875b2e980d2ead9297cfae969c764/href</a></iframe><p>This class keeps the relatively low-level HAS_VERTEX and NO_VERTEX in one place, which limits the changes necessary in other classes as a result of these constants&#39; behavior changing.</p><p>So that’s the main change that I made. This allowed me to remove the concepts of Edge Strategy and Storage Strategy and generally delete a lot of code. One other small change is that I created a Components namespace to store the above class and the ListNode class.</p><p>I’ve also given the whole project a new namespace of Graphs, which I think is a better overall description for the library, and I&#39;ve switched from using require_relative to adding the base directory to the LOAD_PATH in graphs.rb and using the normal require.</p><p>I did also expand a Graph’s functionality a bit, by adding the following methods to its public interface:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f6499a442c09a380618c662f2f8e0321/href">https://medium.com/media/f6499a442c09a380618c662f2f8e0321/href</a></iframe><p>The add_vertex and remove_vertex make constructing new graphs easier and more flexible. The incident_vertices method should be useful for traversals, which is what I will discuss in part 2 of this series!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=933a198f150" width="1" height="1" alt=""><hr><p><a href="https://medium.com/etdev/graphs-in-ruby-pt-1-5-refactoring-933a198f150">Graphs in Ruby Pt. 1.5 — Refactoring</a> was originally published in <a href="https://medium.com/etdev">Eric Turner’s Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Graphs in Ruby Pt. 1 — Representation]]></title>
            <link>https://medium.com/etdev/graphs-in-ruby-pt-1-representation-26b0750be2d4?source=rss----5688d702725d---4</link>
            <guid isPermaLink="false">https://medium.com/p/26b0750be2d4</guid>
            <category><![CDATA[graph]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[computer-science]]></category>
            <category><![CDATA[ruby]]></category>
            <category><![CDATA[linked-lists]]></category>
            <dc:creator><![CDATA[Eric Turner]]></dc:creator>
            <pubDate>Wed, 10 Jan 2018 06:34:02 GMT</pubDate>
            <atom:updated>2018-01-10T03:55:02.694Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NG0bzk0wtQcBdMYAnXKeBQ.jpeg" /></figure><p>I’ve been brushing up on my CS fundamentals recently. Specifically, I’ve been thinking a lot about <a href="https://encrypted.google.com/search?q=graphs+CS+wikipedia&amp;hl=en&amp;gl=en">graphs</a>. And one thing that’s always bothered me is the relative lack of resources for learning about them in Ruby. As a result, while I work through various graph-related problems in ruby, I’ve decided to document the process on this blog, in the hopes that it might be helpful to others.</p><p>In this first part of my series on graphs in Ruby, I’ll just be introducing a couple of methods of representing graphs, implemented in Ruby. These implementations are the building blocks that we’ll need to start working on solving problems in Part 2. Let’s jump right in.</p><h3>Representation #1: The Adjacency Matrix</h3><p>What is an adjacency matrix? Well, let’s say we have a graph with N vertices. The idea behind an Adjacency Matrix is that we store an NxN matrix, where each element (i,j) is 0 when there is no edge between vertex i and vertex j, and 1 when there is such an edge. This is probably the simplest way to represent a graph in code. I want to talk a little bit about the pros and cons of this representation, but first I&#39;ll introduce what I came up with for an implementation in Ruby.</p><p>I began by deciding on a basic interface for my graph. I wanted to be able to do the following operations:</p><ul><li>add_edge - Add an edge to the graph</li><li>remove_edge - Remove an edge from the graph</li><li>edge?(i, j) - Check if there is an edge connecting two vertices, i and j</li><li>to_s - Pretty-print the graph&#39;s contents</li><li>element(i, j) - Get the element at (i, j)</li><li>row(i) - Get the ith row</li></ul><p>I also wanted to support undirected and directed implementations, and check for OutOfBounds errors.</p><p>Let’s look at some RSpec tests I created to ensure the implementation works properly. First, for creation:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5b7d3a4c4f2df4cc3a66ea67ea6a2cb2/href">https://medium.com/media/5b7d3a4c4f2df4cc3a66ea67ea6a2cb2/href</a></iframe><p>Basically the above ensures that we can create a blank MatrixGraph with all vertices initialized to zero.</p><p>Now for the tests about adding and removing edges:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b35ecc3e0e118c7533864407de0558ea/href">https://medium.com/media/b35ecc3e0e118c7533864407de0558ea/href</a></iframe><p>This is the basic functionality I hoped to get out of my initial adjacency matrix implementation. Here’s the code I ended up with:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c81ca180cb5368f12a48196c1d33b6ce/href">https://medium.com/media/c81ca180cb5368f12a48196c1d33b6ce/href</a></iframe><p>As you can see the class itself is extremely simple. It creates a graph with a supplied number of vertices, it allows you to add and remove edges. It allows you to check if edges exist and get specific elements and rows, and it prints the current graph.</p><p>Note that I’ve employed the <a href="https://en.wikipedia.org/wiki/Strategy_pattern">Strategy Pattern</a> to separate the difference in logic between undirected and directed graphs:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e19cb3528f7df5cc5fc3e542128660c1/href">https://medium.com/media/e19cb3528f7df5cc5fc3e542128660c1/href</a></iframe><p>As you can see this class encapsulates the idea that when we add or remove an edge, we must also add or remove its corresponding edge. For example, if you do graph.add_edge(1, 2), then in an undirected graph, you must also do graph.add_edge(2, 1), since having the first one implies that the second must also be included.</p><p>Here’s the corresponding class for directed graphs, where this extra step is not necessary:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6e9d1095737b961b21250bb22aec344f/href">https://medium.com/media/6e9d1095737b961b21250bb22aec344f/href</a></iframe><p>I have also used the with_bounds_check method to ensure that we raise an error when any of our methods are called with invalid vertex indices. Its implementation looks like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/761b64061f9ee05eebcbf3231324406e/href">https://medium.com/media/761b64061f9ee05eebcbf3231324406e/href</a></iframe><p>So that’s our implementation. It’s really very straightforward, but this is all you need to represent a graph in Ruby.</p><h3>Visualizing our graphs</h3><p>It can be a bit tough to visualize a graph based purely on code, so I also wrote a small ruby script using <a href="http://shoesrb.com/">Shoes</a> to print a geometric representation of a graph on a computer screen.</p><p>Let’s look at an example of how we can view one representation of a graph created with my MatrixGraph class.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c68d648d61795d6d4ebb563e8280e09a/href">https://medium.com/media/c68d648d61795d6d4ebb563e8280e09a/href</a></iframe><p>This one-liner is enough to create a simple image:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nwj6T9ADLVeuvE1Y.png" /></figure><p>Not too interesting. But this is our graph as it stands. It is just a collection of nine vertices without any edges between them. Now let’s try adding an edge:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/aa5c160b8fd7f1595bfd8d7abeca77ff/href">https://medium.com/media/aa5c160b8fd7f1595bfd8d7abeca77ff/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7a9u41yWAxHo19Pv.png" /></figure><p>There we go, now we can see that an edge has been added betwen the first two vertices. By the way, here’s the output of graph.to_s (array-based implementation):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/386/0*-J_DajD2AB3bTpgm.png" /></figure><p>As you can see, the elements at (0, 1) and (1, 0) are set to 1, representing the edge shown above.</p><p>Let’s add a few more edges:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/74370fa9fa67ecd7c1f03309fa572352/href">https://medium.com/media/74370fa9fa67ecd7c1f03309fa572352/href</a></iframe><p>And here’s the resulting graph:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*L_yr70J6p2_CxS6C.png" /></figure><h3>Pros and cons of the Adjacency Matrix</h3><p>The MatrixGraph approach has some benefits. For one thing, checking the inclusion of an edge is dead simple and runs in O(1) time, since all you need to do is get elements[i][j]. Additionally, when we have dense graphs (where a given element has edges to a significant portion of all other elements), this is more efficient than other representations. The operation of adding new edges is also a constant-time, simple operation.</p><p>Unfortunately, there are also some cons to the Adjacency Matrix approach. Namely, when dealing with sparse graphs (like the ones shown above), we’re wasting a lot of memory by storing zeros for every possible edge location, when there are only a handful of ones. And if you wanted to go through the graph and find all of the edges, that operation would be O(N^2), which is particularly unfortunate when we have a sparse graph where most of this traversal is useless.</p><p>There is one other major way to represent graphs, which we’ll look at now. It’s called an Adjacency List.</p><h3>Graph Representation #2: Adjacency List</h3><p>The basic idea behind the <a href="https://en.wikipedia.org/wiki/Adjacency_list">Adjacency List</a> is that, instead of having an NxN array of arrays, we have an array of linked lists. This way, when we have a sparse graph, the total amount of memory being used can be lower than with adjacency matrices (O(N) instead of O(N^2)).</p><p>In order to look at how to represent this structure in Ruby, we first need an implementation of a Linked List. So I went ahead and began by creating a class to accomplish that.</p><p>First of all, here are the specs governing this ListNode class:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/934a58e08b9dc52fccf124790cb30ec8/href">https://medium.com/media/934a58e08b9dc52fccf124790cb30ec8/href</a></iframe><p>Basically we need to be able to add and remove elements, print the elements in a list, and check if a given element is contained in the list.</p><p>Here’s the actual implementation for this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/69435b6ff598b06b0062b0a42b313bdf/href">https://medium.com/media/69435b6ff598b06b0062b0a42b313bdf/href</a></iframe><p>I believe this implementation is pretty straightforward, so now let’s introduce the Adjacency List representation of a graph that uses the above class. The external interface is pretty much the same (though slightly slimmed-down) as the one for the Adjacency Matrix so I won’t show the test cases, but here’s the class itself:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1e013703d5ed5ad32319c185081d1bd9/href">https://medium.com/media/1e013703d5ed5ad32319c185081d1bd9/href</a></iframe><p>Once again we end up with a tiny, compact class that looks very similar to the previous one. The main difference is that we’re using an array of linked lists under the hood. This isn’t overly clear when looking at the code, but it can make a difference in our runtime and space complexities.</p><h3>Adjacency List Pros and Cons</h3><p>Let’s think again about how switching to Linked Lists changes our class’s behavior. The biggest pro is the fact that now when we have a sparse graph (such as a friendship graph, road network etc), we can save a lot of memory by only storing a value in the list if an edge exists, and assuming it doesn’t exist otherwise. This can make a huge speed difference when we start actually using these graphs to run algorithms. With the Adjacency Matrix, we had no choice but to always store O(N^2) elements. Now with the Adjacency List we can often store O(N) elements, which can make the difference between an algorithm being slow or fast.</p><p>Having said that, there are also some drawbacks to this approach. For one thing, adding a new edge requires traversal of the linked list for the given vertex, and is therefore O(N) instead of O(1) as it was before. Checking inclusion of a given edge is also no longer possible in constant-time, since again we must traverse a linked list and search for the given vertex.</p><p>Despite these drawbacks, <strong>adjacency lists are superior to adjacency matrices in the majority of applications</strong>. So in the upcoming posts in this series, I will typically use my adjacency list representation by default, unless there’s a specific reason not to. And that about sums up the ruby graph implementations that I’ll be using as I continue to explore graphs and CS concepts in ruby. H<a href="https://github.com/etdev/algorithms/tree/master/6_graphs">ere is a link</a> to all of the source code for this post, and thanks for reading!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=26b0750be2d4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/etdev/graphs-in-ruby-pt-1-representation-26b0750be2d4">Graphs in Ruby Pt. 1 — Representation</a> was originally published in <a href="https://medium.com/etdev">Eric Turner’s Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Taking the guesswork out of iPhone X purchasing with ruby]]></title>
            <link>https://medium.com/etdev/taking-the-guesswork-out-of-iphone-x-purchasing-with-ruby-922d720863f6?source=rss----5688d702725d---4</link>
            <guid isPermaLink="false">https://medium.com/p/922d720863f6</guid>
            <category><![CDATA[apple]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[ruby]]></category>
            <category><![CDATA[iphone-x]]></category>
            <category><![CDATA[japan]]></category>
            <dc:creator><![CDATA[Eric Turner]]></dc:creator>
            <pubDate>Sun, 26 Nov 2017 16:03:51 GMT</pubDate>
            <atom:updated>2017-12-03T15:15:39.377Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*j_tz5rQpd9D5WjzD.png" /><figcaption><a href="https://www.flickr.com/photos/williamhook/26388870449">https://www.flickr.com/photos/williamhook/26388870449</a></figcaption></figure><p>I’m not exactly what you’d call a patient man.</p><p>So when I decided I wanted an iPhone X, I was a bit disappointed by the 2–3 week delivery estimate on Apple’s online store. I didn’t want to wait that long if I could avoid it. So I searched for alternate solutions and it turns out there was indeed another way.</p><p>You see, Apple occasionally opens up reservations for same-day pickup of new products at their retail stores. But this doesn’t happen often, so you need to be quick and notice immediately if you want a chance of getting the device you want.</p><p>Now, as a programmer, the idea of having to constantly check Apple’s site to see if the stores in Tokyo had stock wasn’t very appealing, so I decided to see if I could take out some of the guesswork involved and automate this process. In short, I set out to create a program that would repeatedly check for iPhone X stock at the Apple stores near me, and notify me if it found any.</p><p>Here’s how I went about this using my programming language of choice, ruby.</p><h3>Step 1 — Find your data source</h3><p>The first thing I needed to do was find a reliable source of stock data at my chosen Apple stores. So I visited Apple’s website:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*NobFcBI0hI-EMcq2.png" /></figure><p>This is the page for purchasing an iPhone X. Notice the “Check Availability” link. This is the key to finding the data we need. See, when you enter your zip code here, it will do a search for your chosen iPhone at all Apple Stores near you. For example, if you’re in New York city:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*CZaKqGblhQG1_wHv.png" /></figure><p>Cool. But this by itself isn’t a great data source, since we’d need some fairly complex HTML/JS scraping to reliably extract the data we want from here.</p><p>However, the key to simplifying this data extraction lies in the “Network” tab of our browser’s dev tools:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lKOkAObvI1E-MReo.png" /></figure><p>Bingo. We can see that the site makes an XHR request to the following address:</p><p><a href="https://www.apple.com/shop/retail/pickup-message?pl=true&amp;cppart=ATT/US&amp;parts.0=MQAK2LL/A&amp;location=10001">https://www.apple.com/shop/retail/pickup-message?pl=true&amp;cppart=ATT/US&amp;parts.0=MQAK2LL/A&amp;location=10001</a></p><p>Note that this isn’t the actual link I used, just an example of the endpoint.</p><p>And let’s take a look at a subset of the response to this request:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9c00d89b1dca11fb81a6b9e74e8a70fb/href">https://medium.com/media/9c00d89b1dca11fb81a6b9e74e8a70fb/href</a></iframe><p>It shows a list of stores in JSON format, with entries per store for the iPhone type we selected (in my case iPhone X 64GB Silver), including whether it’s currently available for pickup or not.</p><p>Whenever I do any kind of web scraping, I first look for API endpoints being used in the background, and checking the “XHR” requests under your browser’s “Network” tab is a great way to find them.</p><p>Now that we know where to find the data we want, let’s actually write some code!</p><h3>Step 2 — Set up your project directory</h3><p>The first thing we need to do is create the basic files for our ruby project. I’m a big believer in conventions, so I used the following hierarchy, inspired by the structure typically used in ruby gems.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/604/0*AfmsSSgPyi2dfdh-.png" /></figure><p>The first step is to create a new project directory, which I callediphone_x_checker.</p><p>Then within that directory, I created a simple Gemfile, a blank README and a LICENCE.txt file.</p><p>In the end the only gem I needed was “rest-client”, so here’s what my Gemfile looked like:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/20fb4e41722d54ee55668d09fbac400e/href">https://medium.com/media/20fb4e41722d54ee55668d09fbac400e/href</a></iframe><p>I then ran bundle install to install this gem locally.</p><p>Next, I created a lib directory and inside that added a file called iphone_x_checker.rb:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4ac14e652922ee6b333c016162755259/href">https://medium.com/media/4ac14e652922ee6b333c016162755259/href</a></iframe><p>This file is the entryway to the project and just requires all the other files and runs a simple command to begin checking Apple’s site.</p><p>You can also add a bin directory with a separate file to run the project, but because this was a quick one-off just for personal use, I didn&#39;t add one and instead started the checker by simply running ruby iphone_x_checker.rb.</p><p>Now let’s talk about the actual components themselves.</p><h3>Step 3 — Build out the required components</h3><p>As you’ve seen briefly in the above code, I created several components in order to accomplish the task at hand. While I believe the names make them fairly self-explanatory, here’s a quick description of each one:</p><ul><li>Checker - Keeps track of how and when to check Apple&#39;s site for the latest stock data.</li><li>Notifier - Base class for notifying the user (me) when stock is found.</li><li>Notifiers::EmailNotifier - Concrete class implementing notification via Email (SMTP); it&#39;s used in Notifier.</li><li>Fetcher - Simple class to hit the API endpoint using rest-client and return the raw response data.</li><li>Parser - Parse the raw response from Fetcher into a ruby hash.</li><li>Response - Contains the methods for extracting what we actually care about from our parsed response data.</li><li>Printer - I had some puts statements with similar formatting littered throughout, so I extracted this class to DRY that up somewhat.</li></ul><p>So that’s the basic architecture I went with; it might not be perfect, but I think it’s a relatively DRY design, seeing as it uses the Strategy Pattern forNotifiers, which allows me to swap them without affecting unrelated classes. Also the EmailNotifer class is fairly generic, so I could easily re-use this in other projects with only minor changes.</p><p>Next, I’d like to dive a bit deeper and show how I actually implemented the various classes. Let’s start with actually fetching the JSON from Apple’s server.</p><h4>Fetcher</h4><p>This ended up being very simple; I just imported the excellent rest-client gem, stored the URL in a (frozen) string constant, and added a simple fetch_data method to fetch the data.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/91451059512e7acfe09e200198207c5c/href">https://medium.com/media/91451059512e7acfe09e200198207c5c/href</a></iframe><p>It’s not the most generic class ever, seeing as it hardcodes the link for the exact product I want, but it is simple and focused, and it fulfills my use case.</p><p>So now we have the data itself, but in the form of a huge JSON string that’s not very much use to us yet. The next step? Let’s parse that data into something we can actually use.</p><h4>Parser</h4><p>The Parser class turns out to be another very simple class; it just takes some data, and uses the supplied concrete parser (by default, the JSON class, which I&#39;m categorizing as a parser here since it fulfills my required interface, i.e. it has a parse method).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d6e7a383c110fd980a9238605d8a0c47/href">https://medium.com/media/d6e7a383c110fd980a9238605d8a0c47/href</a></iframe><p>As you can see, all we need to get a hash version of our data is something like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/70a0a9d9b9a71e6dd314bc6401a42296/href">https://medium.com/media/70a0a9d9b9a71e6dd314bc6401a42296/href</a></iframe><p>Now we have our data in a form that we can actually work with (a hash). But there are still some things I want to do to this object. In particular, most of the data in this hash is irrelevant to me. All I want to know is <em>whether any of these stores have stock or not</em>, so I need to do some processing of this data. I could do this in one of my existing classes, but I think it makes sense to make a new one. I’ve chosen to call this class “Response”.</p><h4>Response</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5d9645bde2ad0fa7caece6b920f2ccfd/href">https://medium.com/media/5d9645bde2ad0fa7caece6b920f2ccfd/href</a></iframe><p>This class contains the specifics about how to navigate our parsed data. It knows what stores I’m interested in, and how to extract the things I care about. It gets rid of all the noise and reduces our data to a simple available? interface that is either true or false.</p><p>Note that everything else is private, including the data instance variable accessor. As a rule, I don&#39;t expose methods or data publicly unless I absolutely have to.</p><p>Next let’s talk about how we can actually notify users when this class tells us that yes, the iPhone <em>is available</em>.</p><h4>Notifier</h4><p>There are many ways to notify people of things these days. Email, SMS, push notifications etc. Because of this, I decided to write the Notifier class in such a way as to not be coupled to any specific method of notification. So the Notifier class itself is fairly generic:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3be28f8ff6db04805731b30cf4d7c334/href">https://medium.com/media/3be28f8ff6db04805731b30cf4d7c334/href</a></iframe><p>Notice that we have a notifier attribute here. This might seem a bit weird since we&#39;re <em>in</em> the Notifier class, but as I said, I had a good reason for this: it allows us to swap out concrete notification methods easily, without changing the callers of this class. I might’ve been able to name this a bit less confusingly however.</p><p>Ultimately this class just calls out to another concrete notifier to send the actual notification, and then sleeps for an hour (since if we do find stock, we don’t want to keep spamming the user; this gives me time to turn off the checker after I’ve successfully purchased the phone).</p><p>I decided to notify myself via email since it seemed the simplest, so now let’s look at the EmailNotifier itself.</p><h4>EmailNotifier</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4620b2a6f0910d1651062c7d8ede0d04/href">https://medium.com/media/4620b2a6f0910d1651062c7d8ede0d04/href</a></iframe><p>So this is probably the most complicated class in this project; it contains all the logic surrounding creation of an email via SMTP.</p><p>It also makes a few assumptions:</p><ul><li>You’re probably using Gmail</li><li>You’re just going to send the email to yourself (i.e. you can’t set the sender and receiver to different addresses)</li></ul><p>These limit the usefulness of this class a bit, but I <em>am</em> using gmail and wanted the email to be sent from my own account, so this was enough for my use case.</p><p>Another thing you’ll notice is that I’m using ENV to access environmental variables. Since we&#39;re dealing with things like emails and especially passwords, it&#39;s not suitable to store this in the source code itself, which is why I&#39;ve used environmental variables here. That way you can use your preferred solution to store them securely on the server that runs the code and still keep the source code in version control.</p><p>In the Github repo for this project I&#39;ve included an example of how to run the file if you want to set environmental variables via the command line.</p><p>As I mentioned before, one thing I like about this class is how easy it would be to make a bit more generic and break off into a general-purpose SMTP sending library. I may do this at some point, since I often have small projects like this that involve being notified when things happen.</p><p>Next I want to dive down into the Printer class, which has already made a few appearances in the above files.</p><h4>Printer</h4><p>I wanted to print a message every time the checker failed to find stock, and I wanted it to include a timestamp so I could login to my server at any point and ensure it was still running.</p><p>Here’s the code I wrote to accomplish this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/58af9047faac847b3226be204b85fa62/href">https://medium.com/media/58af9047faac847b3226be204b85fa62/href</a></iframe><p>I think the behavior of this class is pretty straightforward, so let’s move on to the class that puts it all together and actually initiates checking Apple’s site.</p><h4>Checker</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/48fb18ed7bec46a1fb313af05a8344ec/href">https://medium.com/media/48fb18ed7bec46a1fb313af05a8344ec/href</a></iframe><p>Once again we have a class with a very simple public interface: the begin_checking method loops indefinitely, checking the availability of the iPhone X using our Fetcher and the other classes I outlined above. Once again everything that doesn&#39;t absolutely need to be exposed for use in other classes is hidden in private attr_readers and methods.</p><p>Now getting back to our base iphone_x_checker.rb file, we simply create an instance of this class and tell it to begin checking:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/881459818dc361d9f311c225fb29ad26/href">https://medium.com/media/881459818dc361d9f311c225fb29ad26/href</a></iframe><p>And that’s it! This will check Apple’s site every minute, and email me as soon as it finds that there’s a silver iPhone X 64GB in any of the Apple stores in Tokyo, for in-store pick-up.</p><h3>Conclusion</h3><p>So how did the code perform in practice? Well I wrote this code on Thursday, November 23rd, and it ran for about three days without a peep until the fateful morning of Sunday the 26th, when sure enough I got an email at 6:00 AM sharp telling me that the iPhone X was currently available to register for in-store pickup at one of my local Apple stores.</p><p>So in the end, I was able to get my new iPhone a little bit earlier thanks to this project. My online order (which I had also made as a way of hedging my bets) still listed a 1–2 week wait time on the day I picked up my new phone in the Apple store.</p><p>By the way my code is available on <a href="https://github.com/etdev/iphone-x-checker">Github</a>, so feel free to check it out!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1-KYhTgyXmyWfMgJp4UkJg.png" /></figure><p>Huzzah!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=922d720863f6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/etdev/taking-the-guesswork-out-of-iphone-x-purchasing-with-ruby-922d720863f6">Taking the guesswork out of iPhone X purchasing with ruby</a> was originally published in <a href="https://medium.com/etdev">Eric Turner’s Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Getting IRC working properly on the iPhone]]></title>
            <link>https://medium.com/etdev/getting-irc-working-properly-on-the-iphone-50ca4f6ef175?source=rss----5688d702725d---4</link>
            <guid isPermaLink="false">https://medium.com/p/50ca4f6ef175</guid>
            <category><![CDATA[iphone]]></category>
            <category><![CDATA[deployment]]></category>
            <category><![CDATA[irc]]></category>
            <dc:creator><![CDATA[Eric Turner]]></dc:creator>
            <pubDate>Sun, 22 Jan 2017 08:53:36 GMT</pubDate>
            <atom:updated>2017-01-22T08:53:36.520Z</atom:updated>
            <content:encoded><![CDATA[<h4>Overcoming Apple’s boundaries with Palaver and ZNC</h4><p>Recently I’ve been searching for an IRC app for the iPhone. But I found none of them deliver a very good experience, since Apple limits how long you can keep connections open in the background. So about 10 minutes after closing the app it stops sending push notifications for new messages.</p><p>However, it turns out there’s a fairly straightforward way of getting around this limitation using something called an <a href="http://en.wikipedia.org/wiki/BNC_%28software%29#IRC">IRC bouncer</a>. Here are the steps that I took to get IRC working properly on my iPhone.</p><ul><li><a href="http://wiki.znc.in/ZNC">ZNC</a> (IRC bouncer, <strong>free</strong>)</li><li>A place to run the ZNC server (ideally a VPS like <a href="https://www.linode.com/">Linode</a>, <a href="https://www.digitalocean.com/">Digital Ocean</a> or <a href="http://aws.amazon.com/ec2/">EC2</a>).</li><li>The <a href="https://github.com/cocodelabs/znc-palaver">ZNC-Palaver</a> ZNC module (to enable push notifications).</li><li><a href="http://palaverapp.com/">Palaver</a> (IRC client for iOS, <strong>$2.99</strong>).</li><li><a href="https://github.com/Homebrew/homebrew">Homebrew</a> (if you’re installing locally on a mac, which I don’t actually recommend)</li></ul><h4>Installing ZNC</h4><p>First things first, we have to install ZNC.</p><p>If you’re on <strong>Linux</strong> (Ubuntu/Debian or CentOS/Fedora/RedHat), please follow my instructions <a href="https://gist.github.com/etdev/666355a6879804d75095"><strong>here</strong></a> to install ZNC from source. For other distritbutions, please see the <a href="http://wiki.znc.in/Installation#Linux">ZNC Wiki</a> for install instructions.</p><p>If you’re running <strong>OS X</strong>, the easiest way is to use <a href="https://github.com/Homebrew/homebrew">homebrew</a>.</p><pre>brew install znc</pre><p>and you’re done.</p><h4>Configuring ZNC</h4><p>Now it’s time to setup ZNC how we want it. The best way to do this is to make a new user account that we can use to administrate the server (run as root or using sudo):</p><pre>adduser znc-admin</pre><p>Next, let’s set a password for the user:</p><pre>passwd znc-admin</pre><p>and then follow the prompts to set the password to whatever you like.</p><p>Switch to the new user:</p><pre>su znc-admin</pre><p>You’ll be asked to enter the password you just set.</p><p>Next, CD into znc-admin’s home directory:</p><pre>cd ~</pre><p>And create your ZNC config file:</p><pre>vnc --makeconf</pre><p>This will begin an interactive dialogue for setting up the server configuration. Choose what port you want the server to listen on (I chose 4321 but any unused port should work):</p><pre>What port would you like ZNC to listen on? (1025 to 65535): 4321</pre><p>Enter ‘yes’ to use SSL (and enter yes again if you get an error about a missing pem file):</p><pre>Would you like ZNC to listen using SSL? (yes/no) [no]: yes</pre><p>Enter ‘yes’ to allow both IPv4 and IPv6 addresses:</p><pre>Would you like ZNC to listen using both IPv4 and IPv6? (yes/no) [yes]: yes</pre><p>Depending on your VPS, you may be better off setting this to ‘no’ (for example <a href="https://www.digitalocean.com/">Digital Ocean</a> doesn’t support IPv6), but it shouldn’t really matter.</p><p>Load the partyline module (it’s required if you want to use IRC):</p><pre>Load global module &lt;partyline&gt;? (yes/no) [no]: yes</pre><p>Load the webadmin module (for controlling ZNC from your web browser) :</p><pre>Load global module &lt;webadmin&gt;? (yes/no) [no]: yes</pre><p>Create a user/password for logging into ZNC itself (you’ll need this for the web admin panel):</p><pre>Username (AlphaNumeric): testuser  <br>Enter Password: <strong>***************</strong><em>***</em>  <br>Confirm Password: <strong>***************</strong><em>***</em></pre><p>Enter yes to set the user as an admin :</p><pre>Would you like this user to be an admin? (yes/no) [yes]: yes</pre><p>Then you can set your IRC settings. I used the following settings but you’ll want to replace them with your own personal info:</p><pre>Nick [testuser]: testuser  <br>Alt Nick [testuser]: testuser_  <br>Ident [testuser]: testuser  <br>Real Name [Got ZNC?]: Test User  <br>Bind Host (optional):  &lt; (you can leave this blank) &gt;  <br>Number of lines to buffer per channel [50]: 50  <br>Would you like to clear channel buffers after replay? (yes/no) [yes]: yes  <br>Default channel modes [+stn]: +stn</pre><p>Next it will ask if you want to load some more modules; I would choose yes for all of them:</p><pre>Load module &lt;chansaver&gt;? (yes/no) [no]: yes  <br>Load module &lt;controlpanel&gt;? (yes/no) [no]: yes  <br>Load module &lt;perform&gt;? (yes/no) [no]: yes</pre><p>Now it’s time to enter the info for the network you want to connect to. I’ll use the #atom channel at irc.freenode.net as an example, but you’ll want to choose your own channel(s) here:</p><pre>Would you like to set up a network? (yes/no) [no]: yes  <br>Network (e.g. &#39;freenode&#39; or &#39;efnet&#39;): freenode</pre><p>It will then ask you if you want to enable modules for this specific network. You can read up on the various modules <a href="http://wiki.znc.in/Modules#Network_Modules">here</a> and choose which ones you want to install. Again I chose to install each of them:</p><pre>Load module ? (yes/no) [no]: yes  <br>Load module &lt;keepnick&gt;? (yes/no) [no]: yes  <br>Load module &lt;kickrejoin&gt;? (yes/no) [no]: yes  <br>Load module &lt;nickserv&gt;? (yes/no) [no]: yes  <br>Load module &lt;perform&gt;? (yes/no) [no]: yes  <br>Load module &lt;simple_away&gt;? (yes/no) [no]: yes</pre><p>Next, we set the address and port, and the password (if required) for the server:</p><pre>IRC server (host only): irc.freenode.net  <br>[irc.freenode.net] Port (1 to 65535) [6667]: 6667<br>[irc.freenode.net] Password (probably empty): &lt;(leave blank)&gt;</pre><p>Choose whether you want to connect using SSL (I do):</p><pre>Does this server use SSL? (yes/no) [no]: yes</pre><p>It will then ask if you want to connect to any other servers. I don’t, so I answered no:</p><pre>Would you like to add another server for this IRC network? (yes/no) [no]: no</pre><p>Then choose what channel(s) you want to automatically connect to and finish:</p><pre>Would you like to add a channel for ZNC to automatically join? (yes/no) [yes]: yes  <br>Channel name: #atom  <br>Would you like to add another channel? (yes/no) [no]: no  <br>Would you like to set up another network? (yes/no) [no]: no  <br>Would you like to set up another user? (yes/no) [no]: no  <br>Launch ZNC now? (yes/no) [yes]: yes</pre><p>Now ZNC should be running. You can confirm this by accessing the web admin panel at <a href="https://:.">https://:.</a> For example if you were running on your local machine and port 4321 it would be <a href="https://localhost:4321/">https://localhost:4321</a>.</p><p>If you’re using SSL you’ll probably see a screen that looks something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*re60XRCyfO0HPlBV.png" /></figure><p>Just proceed to the next page, and you should end up here:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*3CB78xww2vkUkyUJ.png" /></figure><p>If you want you can login and check out the settings, but for now we don’t need to change anything, we’re just making sure it works.</p><p>That’s the end of the ZNC setup. Next, let’s download and setup the iOS app that we’ll use to connect to our new server!</p><h4>Installing Palaver</h4><p>Simply download the app from the <a href="https://itunes.apple.com/us/app/palaver-irc-client/id538073623?mt=8">app store</a> and install it.</p><h4>Setting up Palaver</h4><p>This is pretty straightforward too. When you first startup the app, it will look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/0*JeiVauP7rdMsRbKx.png" /></figure><p>Not very exciting. So let’s add a connection to our server. Click the plus sign in the upper left, and use the following settings (replacing my info with your own):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/0*5nTZYiO6npHzOzXl.png" /></figure><p>Then click on “Authentication” and enter your settings. Here’s what I used:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/0*nZxwE9ZGWxSD5TWd.png" /></figure><p>Then click “Done” and it should automatically connect to your server and join the channels you’ve specified:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/0*IHVcHvdmfCnHBsG8.png" /></figure><p>Voila! We now have a functioning connection to the server and can use Palaver normally without dropped connections, but there’s still one more problem: in our current state we still won’t receive push notifications. Which brings us to our last step, installing the <a href="https://github.com/cocodelabs/znc-palaver">palaver</a> ZNC module.</p><h4>Enabling push notifications with ZNC-Palaver</h4><p>Download the <a href="https://github.com/cocodelabs/znc-palaver">znc-palaver</a> source files via git:</p><pre>$ git clone <a href="https://github.com/Palaver/znc-palaver">https://github.com/Palaver/znc-palaver</a></pre><p>Cd to the created directory:</p><pre>$ cd znc-palaver</pre><p>Compile the package:</p><pre>make</pre><p>Copy the compiled .so file to .znc/modules</p><pre>cp palaver.so ~/.znc/modules</pre><p>Then load the module from within palaver:</p><pre>/msg *status loadmod palaver</pre><p>Now, all messages that get sent while the app is closed should be available when you re-open the app, and push notifications should work:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/0*54XFrp7ZphQCQ3wD.png" /></figure><p>Beautiful.</p><p>If you have any feedback or questions about this post, please feel free to leave me a comment!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=50ca4f6ef175" width="1" height="1" alt=""><hr><p><a href="https://medium.com/etdev/getting-irc-working-properly-on-the-iphone-50ca4f6ef175">Getting IRC working properly on the iPhone</a> was originally published in <a href="https://medium.com/etdev">Eric Turner’s Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Passing a block to String#gsub in Ruby]]></title>
            <link>https://medium.com/etdev/passing-a-block-to-string-gsub-in-ruby-63c81a54cae0?source=rss----5688d702725d---4</link>
            <guid isPermaLink="false">https://medium.com/p/63c81a54cae0</guid>
            <category><![CDATA[ruby]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Eric Turner]]></dc:creator>
            <pubDate>Sun, 22 Jan 2017 08:48:10 GMT</pubDate>
            <atom:updated>2017-01-22T08:48:09.610Z</atom:updated>
            <content:encoded><![CDATA[<p>Let’s say you have the following string:</p><pre>“the quick brown fox jumped over the lazy dog”</pre><p>and you want to make the following substitutions:</p><ul><li>“e” with “3”</li><li>“o” with “0”</li><li>“l” with “1”</li></ul><p>In ruby, the most naive way to do this would be to call <a href="http://ruby-doc.org/core-2.2.2/String.html#gsub-method">String#gsub!</a> three times:</p><pre>str = “the quick brown fox jumped over the lazy dog”</pre><pre><strong>str.gsub!(“e”, “3”)</strong></pre><pre>=&gt; “th3 quick brown fox jump3d ov3r th3 lazy dog”</pre><pre><strong>str.gsub!(“o”, “0”)</strong></pre><pre>=&gt; “th3 quick br0wn f0x jump3d 0v3r th3 lazy d0g”</pre><pre><strong>str.gsub!(“l”, “1”)</strong></pre><pre>=&gt; “th3 quick br0wn f0x jump3d 0v3r th3 1azy d0g</pre><p>Or, if you don’t want to mutate the original string, by chaining the non-bang version, <a href="http://ruby-doc.org/core-2.2.2/String.html#gsub-method">String#gsub</a>:</p><pre><a href="http://ruby-doc.org/core-2.2.2/String.html#gsub-method):nn%60%60%60Rubynstr">str</a> = “the quick brown fox jumped over the lazy dog”</pre><pre><strong>str.gsub(“e”, “3”)</strong></pre><pre><strong>.gsub(“o”, “0”)</strong></pre><pre><strong>.gsub(“l”, “1”)</strong></pre><pre>=&gt; “th3 quick br0wn f0x jump3d 0v3r th3 1azy d0g”</pre><p>This approach works, but it has two problems:</p><ol><li>We’re hardcoding the replacements in the gsub method; I’d prefer to have a list of the replacements I want to make, and then iterate through that list.</li><li>We’re iterating through the string three times.</li></ol><p>First, to solve problem #1 we could use a Hash, and then iterate through it with <a href="http://ruby-doc.org/core-2.2.3/Hash.html#method-i-each">Hash#each</a>:</p><pre>str = &quot;the quick brown fox jumped over the lazy dog&quot;  </pre><pre>mappings = { &quot;e&quot; =&gt; &quot;3&quot;, &quot;o&quot; =&gt; &quot;0&quot;, &quot;l&quot; =&gt; &quot;1&quot; }  </pre><pre><strong>mappings.each do |key, val|     <br>  str.gsub!(key, val) <br>end   </strong></pre><pre>=&gt; &quot;th3 quick br0wn f0x jump3d 0v3r th3 1azy d0g&quot;</pre><p>I like this better since there’s now less mental overhead involved in changing the list of values I want to replace. Instead of having to look in the code that’s doing the replacements itself, I have one central place I can go to make changes.</p><p>This is a good thing, but we’re still iterating through the string three times, which seems inefficient.</p><p>It may not be obvious at first glance, but the <a href="http://ruby-doc.org/core-2.2.3/String.html#gsub-method">String#gsub</a> method allows you to specify a hash argument, so you can do the following:</p><pre>str = &quot;the quick brown fox jumped over the lazy dog&quot;</pre><pre>mappings = { &quot;e&quot; =&gt; &quot;3&quot;,&quot;o&quot; =&gt; &quot;0&quot;, &quot;l&quot; =&gt; &quot;1&quot; }</pre><pre><strong>str.gsub(/[eol]/, mappings)  </strong></pre><pre>=&gt; &quot;th3 quick br0wn f0x jump3d 0v3r th3 1azy d0g&quot;</pre><p>The /[eol]/ regex matches a single occurrence of any of e, o, or l, and passing the mappings hash tells gsub to check the matched value, and <strong>if the hash has that value as a key, replace it with the hash value at that key</strong>. Now we&#39;re only doing a single pass of the string, and we&#39;re relying on fast (O(1)) hash lookups instead.</p><p>If you want, you could also go one step further to DRY up this code and do something like the following:</p><pre>str = &quot;the quick brown fox jumped over the lazy dog&quot;</pre><pre>mappings = { &quot;e&quot; =&gt; &quot;3&quot;, &quot;o&quot; =&gt; &quot;0&quot;, &quot;l&quot; =&gt; &quot;1&quot; }</pre><pre>mapping_keys = mappings.keys.join(&quot;&quot;)</pre><pre><strong>str.gsub(/[#{mapping_keys}]/, mappings)</strong></pre><pre>=&gt; &quot;th3 quick br0wn f0x jump3d 0v3r th3 1azy d0g&quot;</pre><p>That way the only place you need to make changes is the mappings hash.</p><pre>str = &quot;the quick brown fox jumped over the lazy dog&quot;</pre><pre><strong>str.tr(&quot;eol&quot;, &quot;301&quot;)</strong></pre><pre># =&gt; &quot;th3 quick br0wn f0x jump3d 0v3r th3 1azy d0g&quot;</pre><p>but I digress.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=63c81a54cae0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/etdev/passing-a-block-to-string-gsub-in-ruby-63c81a54cae0">Passing a block to String#gsub in Ruby</a> was originally published in <a href="https://medium.com/etdev">Eric Turner’s Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Useful Commands Cheatsheet]]></title>
            <link>https://medium.com/etdev/useful-commands-cheatsheet-a78ee3830171?source=rss----5688d702725d---4</link>
            <guid isPermaLink="false">https://medium.com/p/a78ee3830171</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[terminal]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[bash]]></category>
            <dc:creator><![CDATA[Eric Turner]]></dc:creator>
            <pubDate>Sun, 22 Jan 2017 08:46:59 GMT</pubDate>
            <atom:updated>2017-01-22T08:46:58.910Z</atom:updated>
            <content:encoded><![CDATA[<p>For the past few months I’ve been building a personal programming cheatsheet as a <a href="https://gist.github.com/etdev/128f61cb94a8f0c23f36/edit">gist</a>. It’s basically a list of commands, snippets and other info that I’ve found useful to have as a reference when coding. But I’ve decided to move this content to my blog and maintain it here instead. Feel free to take a look and see if you find anything useful.</p><h3>What’s included?</h3><ul><li><a href="https://github.com/etdev/dotfiles/blob/master/CHEATSHEET.md#mysql">MySQL</a></li><li><a href="https://github.com/etdev/dotfiles/blob/master/CHEATSHEET.md#postgres">Postgresql</a></li><li><a href="https://github.com/etdev/dotfiles/blob/master/CHEATSHEET.md#ag">ag</a> (the silver searcher)</li><li><a href="https://github.com/etdev/dotfiles/blob/master/CHEATSHEET.md#awk">awk</a></li><li><a href="https://github.com/etdev/dotfiles/blob/master/CHEATSHEET.md#curl">curl</a></li><li><a href="https://github.com/etdev/dotfiles/blob/master/CHEATSHEET.md#sed">sed</a></li><li><a href="https://github.com/etdev/dotfiles/blob/master/CHEATSHEET.md#shell">shell</a> (bash/zsh scripting)</li><li><a href="https://github.com/etdev/dotfiles/blob/master/CHEATSHEET.md#sort">sort</a></li><li><a href="https://github.com/etdev/dotfiles/blob/master/CHEATSHEET.md#uniq">uniq</a></li><li><a href="https://github.com/etdev/dotfiles/blob/master/CHEATSHEET.md#wc">wc</a></li><li><a href="https://github.com/etdev/dotfiles/blob/master/CHEATSHEET.md#xargs">xargs</a></li><li><a href="https://github.com/etdev/dotfiles/blob/master/CHEATSHEET.md#paste">paste</a></li><li><a href="https://github.com/etdev/dotfiles/blob/master/CHEATSHEET.md#git">Git</a></li><li><a href="https://github.com/etdev/dotfiles/blob/master/CHEATSHEET.md#gitignore-patterns">Gitignore patterns</a></li><li><a href="https://github.com/etdev/dotfiles/blob/master/CHEATSHEET.md#javascript">Javascript</a></li><li><a href="https://github.com/etdev/dotfiles/blob/master/CHEATSHEET.md#one-liners">One-liners and Misc.</a></li></ul><h3>Databases</h3><h4>MySQL</h4><p>Create a new user</p><pre>CREATE USER &#39;newuser&#39;@&#39;localhost&#39; IDENTIFIED BY &#39;password&#39;;</pre><p>Grant permissions to user</p><pre>GRANT ALL PRIVILEGES ON *.* TO &#39;newuser&#39;@&#39;localhost&#39;;</pre><p>Reload privileges:</p><pre>FLUSH PRIVILEGES;</pre><p>Update record(s):</p><pre>UPDATE `potluck`<br>SET<br>`confirmed` = &#39;Y&#39;<br>WHERE `potluck`.`name` = &#39;Sandy&#39;;</pre><p>Delete record(s):</p><pre>DELETE FROM `potluck` WHERE `potluck`.`name` = &#39;John&#39;;</pre><p>Add column to table:</p><pre>ALTER TABLE potluck ADD email VARCHAR(40) AFTER name;</pre><p>Remove column from table:</p><pre>ALTER TABLE potluck DROP email;</pre><p>Analyze (explain) query:</p><pre>EXPLAIN SELECT COUNT(*) FROM salaries WHERE salary BETWEEN 60000 AND 70000;</pre><p>Change a column’s data type,options:</p><pre>ALTER TABLE t1 MODIFY a TINYINT NOT NULL;</pre><p>Add an index to existing column:</p><pre>ALTER TABLE t1 ADD index (d);</pre><p>View the indexes on a table:</p><pre>SHOW INDEXES FROM table_name;</pre><p>Connect to database from the command line:</p><pre>mysql -u USERNAME -pPASSWORD -h HOSTNAMEORIP DATABASENAME</pre><p>Check currently-connected database:</p><pre>SELECT DATABASE();</pre><h4>Postgresql</h4><p>Add column to table</p><pre>ALTER TABLE distributors ADD COLUMN address varchar(30);</pre><p>Drop a column from table:</p><pre>ALTER TABLE distributors DROP COLUMN address RESTRICT;</pre><p>Add a not-null constraint to a column:</p><pre>ALTER TABLE distributors ALTER COLUMN street SET NOT NULL;</pre><p>Add a foreign key constraint:</p><pre>ALTER TABLE distributors ADD CONSTRAINT distfk FOREIGN KEY (address) REFERENCES addresses (address) MATCH FULL;</pre><p>Add a (multi-column) unique constraint:</p><pre>ALTER TABLE distributors ADD CONSTRAINT dist_id_zipcode_key UNIQUE (dist_id, zipcode);</pre><h3>Command line</h3><h4>ag (Silver Searcher)</h4><p>Agignore patterns</p><ul><li>Basically the same as gitignore patterns, but has some issues</li><li>Use the latest version if possible</li></ul><p>E.g. Minified css or js files anywhere</p><pre># .agignore<br>**.min.*<br># matches script.min.js, styles.min.css etc.</pre><p>More info: <a href="https://github.com/ggreer/the_silver_searcher/issues/385">https://github.com/ggreer/the_silver_searcher/issues/385</a></p><h4>awk</h4><p>Basic syntax</p><pre>awk &#39;/search_pattern/ { action_to_take_on_matches; another_action; }&#39; file_to_parse</pre><p>E.g. print each line in etc/fstab beginning with UUID</p><pre>awk &#39;/^UUID/&#39; /etc/fstab</pre><p>E.g. print the 6th column of netstat output:</p><pre>sudo netstat -pant | awk &#39;{print $6}&#39;</pre><p>Full syntax:</p><pre>awk &#39;BEGIN { action; }<br>/search/ { action; }<br>END { action; }&#39; input_file</pre><p>E.g. set FieldSeparator to : and then print first column of /etc/passwd</p><pre>awk &#39;BEGIN { FS=&quot;:&quot;; }<br>{ print $1; }&#39; /etc/passwd</pre><p>Given favorite_foods.txt:</p><pre>1 carrot sandy<br>2 wasabi luke<br>3 sandwich brian<br>4 salad ryan<br>5 spaghetti jessica</pre><p>E.g. Match based on regex per-field</p><pre>awk &#39;/$2 ~ ^sa/&#39; favorite_food.txt<br># =&gt; 3 sandwich brian<br># =&gt; 4 salad ryan</pre><p>E.g. combined matching, including !~ to NOT match a regex per-field, and &amp;&amp; to combine conditions</p><pre>awk &#39;BEGIN { print &quot;NUM\tFOOD&quot;; } $2 ~ /^(c|w)/ &amp;&amp; $1 !~ /1/ { print $1,&quot;\t&quot;,$2; }&#39; favorite_foods.txt<br># =&gt; NUM   FOOD<br># =&gt; 2     wasabi</pre><p>E.g. Set ORS (output record separator) var to use e.g. commas instead of newlines</p><pre>awk &#39;ORS=&quot;, &quot; { print $1; }&#39; favorite_foods.txt<br># =&gt; 1, 2, 3, 4, 5,</pre><p>Built-in variables:</p><pre>ARGC = input argument count (cols)<br>ARGV = array storing the command-line args<br>FILENAME = current filename<br>FS = input field separator<br>NF = number of fields in current record{<br>NR = number of current record<br>OFS = output field separator (def. space)<br>RS = input record separator (def. newline)</pre><p>More info: <a href="https://www.digitalocean.com/community/tutorials/how-to-use-the-awk-language-to-manipulate-text-in-linux">https://www.digitalocean.com/community/tutorials/how-to-use-the-awk-language-to-manipulate-text-in-linux</a></p><h4>curl</h4><p>E.g. Get your current IP</p><pre>curl icanhazip.com</pre><p>E.g. Get http headers for URL</p><pre>curl -I example.com</pre><p>E.g. Follow redirects</p><pre>curl -L google.com</pre><p>E.g. Continue downloading partially-downloaded file</p><pre>curl -C - example/com/file.zip</pre><p>E.g. Pass HTTP Basic Auth credentials</p><pre>curl -u etdev:pass example.com/members</pre><p>E.g. Download files in range</p><pre>curl ftp://ftp.uk.debian.org/debian/pool/main/[a-z]/</pre><p>E.g. POST to a page</p><pre>curl -d &quot;name=etdev&amp;password=hunter2&quot; <a href="http://www.example.com/process.php">www.example.com/process.php</a></pre><p>E.g. Set referer</p><pre>curl -e &quot;http://someReferringSite.com&quot; <a href="http://www.example.com/">http://www.example.com</a></pre><p>E.g. Set request method</p><pre>curl -X POST quiet-waters-1228.herokuapp.com/echo</pre><p>E.g. Add parameters to body</p><pre>curl -X POST -d &quot;fname=Mark&amp;lname=Bates&quot; quiet-waters-1228.herokuapp.com/echo</pre><p>E.g. Add parameters to body by specifying a file</p><pre>curl -X POST -d &quot;fname=Mark&amp;lname=Bates&quot; quiet-waters-1228.herokuapp.com/echo</pre><p>E.g. POST as a form</p><pre>curl -X POST -F user[fname]=Mark -F user[lname]=Bates -F foo=bar \<br>  quiet-waters-1228.herokuapp.com/echo -H &quot;Accept: application/json&quot;</pre><p>E.g. Set headers</p><pre>curl -X POST -d @form_data.json quiet-waters-1228.herokuapp.com/echo \<br>  -H &quot;Accept: application/json&quot;</pre><p>Adding the Accept: application/json header will return JSON instead of HTML.</p><h4>sed</h4><p>Sed is a stream editor.</p><p>Basic syntax:</p><pre>sed [options] commands [file-to-edit]</pre><p>E.g. print first line of test.txt</p><pre>sed -n &#39;1p&#39; test.txt</pre><ul><li>The -n suppresses the default print each line behavior</li><li>The 1 tells sed what line(s) to operate on -- line 1</li><li>The p means the print command; so print line 1, suppressing default print</li></ul><p>E.g. print lines 1–5 of test.txt</p><pre>sed -n &#39;1,5p&#39; test.txt</pre><pre>sed -n &#39;1,+4p&#39; test.txt</pre><p>E.g. print every other line of test.txt</p><pre>sed &#39;1~2p&#39; test.txt</pre><p>E.g. delete every other line of test.txt</p><pre>sed &#39;1~2d&#39; test.txt</pre><ul><li>Doesn’t affect the actual file</li><li>You can make it change the source file with the -i option though.</li></ul><p>E.g. delete every other line of test.txt, actually changing original file</p><pre>sed -i.bak &#39;1~2d&#39; file.txt</pre><ul><li>The -i.bak means it will create a backup file with the extension .bak</li></ul><p>Given annoying.txt:</p><pre>this is the song that never ends<br>yes, it goes on and on, my friend<br>some people started singing it<br>not knowing what it was<br>and they&#39;ll continue singing it forever<br>just because...</pre><p>E.g. Substitute on with forward</p><pre>sed &#39;s/on/forward&#39; annoying.txt</pre><ul><li>Only replaces first match in each line</li><li>Replaces any instance of on, even in the middle of other words</li></ul><p>E.g. Substitute all instances of on in first 3 lines with forward</p><pre>sed &#39;1,3s/on/forward/g&#39; annoying.txt</pre><p>E.g. Substitute case-insensitively all instances of SINGING with saying, and only print changed lines</p><pre>sed -n &#39;s/SINGING/saying/gip&#39; annoying.txt</pre><p>E.g. Substitute from beginning of line to last instance of at with parenthesized version</p><pre>sed &#39;s/^.*at/(&amp;)/&#39; annoying.txt</pre><ul><li>The &amp; represents the captured text (which is captured despite lack of parentheses in the regex)</li></ul><p>E.g. Switch the first two words in each line</p><pre>sed &#39;s/\([^ ][^ ]*\) \([^ ][^ ]*\)/\2 \1/&#39; annoying.txt</pre><ul><li>You can use escaped parentheses for capture groups, which you can then refer to with \1, \2, ...</li></ul><p>E.g. Substitute it with it loudly but only on lines that match /singing/</p><pre>sed &#39;/singing/s/it/&amp; loudly/&#39; annoying.txt</pre><p>E.g. Delete all empty lines</p><pre>sed &#39;/^$/d&#39; annoying.txt</pre><p>E.g. Delete all lines that AREN’T empty</p><pre>sed &#39;/^$/!d&#39; annoying.txt</pre><p>E.g. Delete all lines between lines starting with START and END (including the start, end lines)</p><pre>sed &#39;/^START/,/^END/d&#39; test.txt</pre><p>E.g. Replace all instances of routeParams with stateParams (with ag, xargs)</p><pre>ag &#39;routeParams&#39; -l | xargs sed -i &#39;s/routeParams/stateParams/&#39; {}</pre><h4>shell</h4><p>Simple for loop</p><pre>for i in 1 2 3 .. N<br>do<br>&lt;commands&gt;<br>done</pre><p>E.g. create files test1.txt through test5.txt</p><pre>for i in 1 2 3 4 5<br>do<br>touch test$i.txt<br>done</pre><p>E.g. print files in directory, but preceeded by some message</p><pre>for i in $(ls); do<br>echo item: $i;<br>done</pre><p>E.g. rename files in current dir with a 2 in their name, to item_&lt;old_name&gt;</p><pre>for i in $(ls | grep 2); do<br>mv $1 item_$1;<br>done</pre><p>More info: <a href="http://www.tecmint.com/wc-command-examples/">http://www.tecmint.com/wc-command-examples/</a></p><h4>sort</h4><p>Sorts text by line</p><pre>echo &#39;c\nb\na&#39; | sort<br># =&gt; a<br># =&gt; b<br># =&gt; c</pre><p>More info: <a href="http://www.computerhope.com/unix/usort.htm">http://www.computerhope.com/unix/usort.htm</a></p><h4>uniq</h4><p>Get the unique values from some text stream/file</p><p>E.g. simply remove duplicates</p><pre>echo &#39;a\na\nb&#39; | uniq<br># =&gt; a<br># =&gt; b</pre><p>E.g. show counts</p><pre>echo &#39;a\na\nb&#39; | uniq -c<br># =&gt; 2 a<br># =&gt; 1 b</pre><h4>wc</h4><p>Count lines / words / characters in a file or stream</p><p>E.g. Show line count, word count, and char count for test.txt</p><pre>wc test.txt<br># =&gt; 15  229 1529 test.txt</pre><p>E.g. Show line count for test.txt</p><pre>wc -l test.txt<br># =&gt; 15 test.txt</pre><p>E.g. Show word count for test.txt</p><pre>wc -w test.txt<br># =&gt; 229 test.txt</pre><p>E.g. Show char count for test.txt</p><pre>wc -c test.txt<br># =&gt; 1529 test.txt</pre><h4>xargs</h4><p>Get args from standard input and do things with them</p><p>E.g. remove files matching test3</p><pre>find . -name &quot;*test3*&quot; | xargs rm</pre><p>E.g. remove files matching test3 #2</p><pre>find . -name &quot;*test3*&quot; | xargs -i rm {}</pre><p>The -i basically replaces {} with the current parameter</p><p>E.g. Print filenames + ‘is a file!’ for files matching /test[1,2]{2}$/</p><pre>ag -g &#39;test[1,2]{2}\.txt$&#39; | xargs -Iresult echo result is a file!</pre><p>The -I lets you refer to the current matched param in the command.</p><h4>MySQL</h4><p>E.g. List the files in current directory in three columns</p><pre>ls | paste - - -</pre><p>E.g. Combine pairs of lines from a file into single lines</p><pre>paste -s -d &#39;\t\n&#39; - - myfile</pre><h3>Git</h3><h4>gitignore patterns</h4><p>E.g. Any file or directory called doc, recursive</p><pre># .gitignore<br>doc</pre><p>E.g. Any directory called doc, recursive</p><pre># .gitignore<br>doc/</pre><p>E.g. Any file or directory called doc, root dir only</p><pre># .gitignore<br>/doc</pre><p>E.g. Directory called doc, root dir only</p><pre># .gitignore<br>/doc/</pre><p>E.g. Files anywhere ending with .tmp</p><pre># .gitignore<br>*.tmp</pre><p>E.g. Files under the /doc directory ending with .tmp</p><pre># .gitignore<br>/doc/*.tmp<br># matches /doc/test.tmp, NOT /doc/stuff/test.tmp</pre><p>E.g. File or directory bar whenever its parent directory is foo</p><pre># .gitignore<br>**/foo/bar</pre><p>E.g. All files and directories inside foo directory, anywhere</p><pre># .gitignore<br>foo/**</pre><p>E.g. baz directory with ancestor directory foo</p><pre># .gitignore<br>foo/**/baz/</pre><p>E.g. All .html files except index.html</p><pre># .gitignore<br>*.html<br>!index.html</pre><p>More info: <a href="https://git-scm.com/docs/gitignore">https://git-scm.com/docs/gitignore</a></p><h3>One-liners and misc.</h3><p>Smartresize function with imagemagick</p><pre>smartresize() {<br>   mogrify -path $3 -filter Triangle -define filter:support=2 -thumbnail $2 -unsharp 0.25x0.08+8.3+0.045 -dither None -posterize 136 -quality 82 -define jpeg:fancy-upsampling=off -define png:compression-filter=5 -define png:compression-level=9 -define png:compression-strategy=1 -define png:exclude-chunk=all -interlace none -colorspace sRGB $1<br>}</pre><p>Usage example:</p><pre>smartresize inputfile.png 300 outputdir/</pre><p>Pyenv/Virtualenv — create</p><pre>pyenv virtualenv &lt;python_version&gt; &lt;virtualenv_name&gt;</pre><p>E.g.</p><pre>pyenv virtualenv 3.5.1 my_virtualenv</pre><p>Pyenv/Virtualenv — activate</p><pre>pyenv activate &lt;virtualenv_name&gt;</pre><h3>Javascript</h3><h4>Iteration</h4><p>Array — Select elements from array where condition is true</p><pre>arr.filter(element =&gt; element.enabled);</pre><p>Array — Get array with 5 added to all elements (non-destructive)</p><pre>arr.map(element =&gt; element + 5);</pre><p>Object — Iterate through an Object</p><pre>for (var key in p) {<br>  if (p.hasOwnProperty(key)) {<br>    console.log(key + &quot; -&gt; &quot; + p[key]);<br>  }<br>}</pre><p>String — Iterate through string</p><pre>const str = &#39;myString&#39;;<br>for ( const chr of str ){<br>  do_something(chr);<br>}</pre><p>You can also use the for-of loop for Maps, Sets, and most other iterable objects.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a78ee3830171" width="1" height="1" alt=""><hr><p><a href="https://medium.com/etdev/useful-commands-cheatsheet-a78ee3830171">Useful Commands Cheatsheet</a> was originally published in <a href="https://medium.com/etdev">Eric Turner’s Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>