<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Geoffrey Bradway on Medium]]></title>
        <description><![CDATA[Stories by Geoffrey Bradway on Medium]]></description>
        <link>https://medium.com/@geoff.bradway?source=rss-ab02e41ba4d8------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*HtRUB1G_cDfnWPqZpipVMw.jpeg</url>
            <title>Stories by Geoffrey Bradway on Medium</title>
            <link>https://medium.com/@geoff.bradway?source=rss-ab02e41ba4d8------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 06:20:58 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@geoff.bradway/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Commoditization of Deep Learning]]></title>
            <link>https://medium.com/@geoff.bradway/the-commoditization-of-deep-learning-92e65197e879?source=rss-ab02e41ba4d8------2</link>
            <guid isPermaLink="false">https://medium.com/p/92e65197e879</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[business-strategy]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[economy]]></category>
            <dc:creator><![CDATA[Geoffrey Bradway]]></dc:creator>
            <pubDate>Mon, 03 Oct 2016 18:55:11 GMT</pubDate>
            <atom:updated>2016-10-04T00:16:30.952Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OjX1YIyBie2pAHOLFiWUhQ.jpeg" /></figure><p>It is getting easier and easier to do deep learning (DL). There exist <a href="http://deeplearning.net/reading-list/">papers</a>, <a href="http://colah.github.io/">blogs</a>, <a href="https://en.wikipedia.org/wiki/Comparison_of_deep_learning_software">frameworks</a>, <a href="https://www.quora.com/What-are-some-good-books-papers-for-learning-deep-learning">books</a>, <a href="https://www.google.com/search?q=deep%20learning%20courses&amp;rct=j">courses</a>, newsletters, conferences, and many more resources. If you don’t want to implement it yourself, there are machine learning (ML) API services on AWS, GCE, Azure and companies like Clarifai and Bonsai, to name a few. Thinking back to a few years ago, neural nets were sometimes regarded as “just a fad” —I can remember skipping this topic in my grad school ML class because the professor didn’t like them and thought they were hyped. So let’s talk about some of things that have happened to the field of DL in the last few years that have shifted this view.</p><p>While initially most DL papers came from a few academics, most of the new papers, results, tools, and datasets now come from big companies. And much of the current state of everything can be attributed to big companies. So the natural question is “What’s in it for them?” Why would Google open source something like Tensorflow, or why would Microsoft publish a paper detailing how to make state-of-the-art vision algorithms? On the surface it seems like developing this technology provides a competitive advantage, so it doesn’t make sense why you might want to make it free or widely available. As it turns out, this actually represents some pretty nifty business strategy.</p><p>First, let’s profile a big company to see what components it has, and to see things from its perspective. Let’s start by thinking of Amazon, what they do and where they make money.</p><p>Let’s start with what they do:</p><ol><li>They make a website that lets you buy a lot of stuff from many people</li><li>They have physical infrastructure in place to sell you things</li><li>Consumer electronics [we are going to ignore this… ]</li><li>AWS</li></ol><p>In the process of doing business, they are generating one more critical resource, data. The data is super important because not only does this help to make Amazon better, but it is also proprietary, so other web sellers can’t get better. Amazon can analyze the data to figure out better recommendations, predict who will want what and preship items, figure out consumption patterns, A/B test, and the list goes on and on. Then, when Amazon is doing a better job as a marketplace, more people will buy and sell, and they will collect more data, creating a positive feedback loop.</p><p>This isn’t unique to Amazon. The more Amazon sells, the better it gets at selling. The more searches Google provides, the better searches it can provide. The more movies Netflix recommends, the better recommendations it can make. So effectively by collecting data, you&#39;re raising the barrier to entry for competition. This is what makes data super duper valuable.</p><p>You can think of Microsoft, Google, and Facebook, all working roughly in the same way. They all have some version of AWS and some software cash-cow that makes money with the help of data. Their business models are thus roughly the same.</p><p>Now, the bulk of Amazon’s profit comes from AWS, and less from their retail division. For Google, Facebook, and Microsoft, it is the complete opposite. Most of Google and Facebook’s money comes from ads, and Microsoft’s from selling software.</p><p>The two largest money makers consist of some sort of software and of the computing infrastructure. In contrast, a small portion of money comes from companies selling data indirectly, via machine learning APIs.</p><p>Now that we have an idea of the rough profile of one of these companies, let’s talk about deep learning, but as a collection of products.</p><p>The argument I am about to present takes its roots from <a href="http://www.joelonsoftware.com/articles/StrategyLetterV.html">this highly recommended blog post</a> by Joel Spolsky. The relevant takeaways are:</p><ol><li>Think of products as either complements or substitutes. Complement products go with other products (e.g., when you buy peanut butter, you often buy jelly). Substitute products act as replacements (e.g., I might buy a PB&amp;J instead of a turkey sandwich for lunch because it is cheaper).</li><li>When the cost of a complementary product decreases (all else equal), the price of the other product can increase (e.g., if peanut butter were free, you could charge more for jelly, and the cost of a PB&amp;J would either stay the same or decrease).</li><li>“Smart companies try to commoditize their products’ complements.” If you can commoditize your products’ complements, then you can charge more and make more money.</li></ol><p>Let’s now think about DL as related to five different products: data, compute power, research, DL tools, and software (that could benefit from DL — for example, Netflix recommendation). Let’s analyze these in terms of their relationship with each other.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6qb_AdLQtE8AXDuoiAjfTg.png" /></figure><p>Let’s talk about this chart a bit. As you can see, most of these products complement each other pretty well — cheaper GPUs make it so you can do more research, deploy more DL helped software, crunch more data, etc. Basically, more of any one of these components makes the other ones better. If you talk to people about why “AI has taken off in the last five years,” they will invariably mention it is because of Moore’s Law. I argue that having more computation and cheaper computation is not in itself the propelling force; rather, it provides the necessary momentum to accelerate data, research, tools, and — most importantly — profitability.</p><p>It makes sense that the diagonal in the above chart would be mostly substitutes. If AWS is cheaper the GCE, then I will use AWS since these are commodities. The one exception to this is research. Research complementing other research makes sense (the more research is widely disseminated, the more focused it can be). Furthermore, releasing research is beneficial even between competitors, since data — the main fuel for ML — is proprietary.</p><p>With this in mind, think of where big companies make money from:</p><ol><li>Data in an indirect way (e.g., making ads better) — basically every tech company</li><li>Data in a direct way (e.g., selling you ML APIs) — Google, Microsoft, Amazon</li><li>Compute power — Amazon, Google, NVIDIA, Intel</li><li>Software — Facebook, Google, Amazon, Netflix</li></ol><p>Think of what the big companies give out for free:</p><ol><li>Research</li><li>Tools — Tensorflow, Paddle, Torch</li><li>Data — limited scope data (e.g., MS Coco, Webscope), sponsoring data (e.g., ImageNet)</li></ol><p>By giving these away for free, companies want to commoditize tools and research (aka the difficult parts), and monetize the resources needed for ML (which is a large capital investment that creates a high barrier of entry).</p><p>Given where companies make money, I’d predict that DL research is going to continue to look at more resource intensive models so that really only big companies can take advantage of it.</p><p>Finally, I want to thank my friends Eric Bakan, Stedman Hood, and Corina Grigore for helping me flesh out this idea.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=92e65197e879" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Humans aren’t as smart as Pokemon, but we might be in 7.7 million years]]></title>
            <link>https://medium.com/@geoff.bradway/humans-arent-as-smart-as-pokemon-but-we-might-be-in-7-7-million-years-e0a1409331bd?source=rss-ab02e41ba4d8------2</link>
            <guid isPermaLink="false">https://medium.com/p/e0a1409331bd</guid>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[mathematics]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[pokemon]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Geoffrey Bradway]]></dc:creator>
            <pubDate>Fri, 02 Sep 2016 18:09:02 GMT</pubDate>
            <atom:updated>2016-09-02T18:09:02.968Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Co Written with <a href="https://twitter.com/ericbakan">Eric Bakan</a></blockquote><p>So someone on my facebook posted that according to the Pokedex that, my favorite type of pokemon, Alakazam has an IQ of 5000. Now, for humans, IQ follows a distribution that is normal with mean 100, and standard deviation of 15. So that means that Alakazam is about 327 standard deviations above normal! Now, 327 std deviations above the normal is pretty rare, but exactly how rare?</p><p>So we wanted to ask, how smart has the smartest human ever been? And how many humans would there need to match Alakazam’s staggering intellect?</p><p>So time for statistics!</p><p>First things first, let’s normalize our average person, so that on average they have an normalized IQ of 0, with standard deviation 1. This means that instead of thinking about 5000, we care about (5000–100)/15 ~= 327. [So instead of thinking of the IQ we are thinking of the z-score]. And if we denote Xi to be the normalized IQ of the ith person we have</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/233/1*8YMcV2Z5_7QvA_zp02W_Jw.png" /></figure><p>And we care about</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/401/1*-LmkNTBUMz6URJT2caWt-w.png" /></figure><p>More specifically we care about the expected value of Y, and we care about it as a function of N (so right now, about<a href="http://www.prb.org/Publications/Articles/2002/HowManyPeopleHaveEverLivedonEarth.aspx"> 100 billion people have ever lived</a>, so N = 10¹¹ )</p><p>Okay, so now we claim that Y is always (well almost always) positive for a large N.</p><p>The way to see this is that a standard normal random variable is symmetric about zero, so there is a 50/50 chance it will be positive or negative (and a 0 chance it will be zero!)</p><p>So then the only way for Y to be negative is if all the X’s are negative, since Y is just the maximum of all the X’s. You can think of this like a coin flip, where for Y to be negative all the coin flips have to be heads. Each X has a 1/2 chance of that, so the chance they are all negative is (1/2)^N . And so we know that</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/505/1*g3V0KDb5A9KcexoBHPEsuw.png" /></figure><p>Which goes to 1 as N goes to infinity.</p><p>So the reason we want to use is because of this nifty identity for positive random variables</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/335/1*kB4rkLzbi0tNw_glA3KIKA.png" /><figcaption>Details <a href="https://en.wikipedia.org/wiki/Expected_value#Continuous_distribution_taking_non-negative_values">here</a></figcaption></figure><p>So we know that</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/342/1*BaZA7kNGyqQaizP5NcxOyA.png" /></figure><p>Now we just have to figure out that probability. This can be done with simple trick about maxes (also works for mins)[in general these things are called the order statistics, but order statistics that aren’t the max and min are hard to work with].</p><p>The trick is this: the maximum of a bunch of iid random variables is less than some number if and only if each random variable was less than that number.</p><p>So we can actually write it out explicitly as</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/553/1*aaX1WrwOpof2kDGH5t57DA.png" /></figure><p>Where Phi is the CDF for a standard normal. So now we have</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*Ih9lgQ-RBkCFrXmkA3xhxA.png" /></figure><p>So if we plug this into a numerical integral estimator with N = 100 billion, we get that E[Y] = 6.78, so the smartest person who has ever lived is expected to have an IQ of 100 + 15*6.78 = 202.</p><p>So now we want to figure out how many humans will need to have lived for us to expect someone 327 std deviations away from the mean, which means solving for N</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/297/1*sL_xVaEUjV1b5jlr44rYxQ.png" /></figure><p>Here is when things start to get tricky — because if you start raising numbers to the 10⁸⁰ power and such, they will underflow and after a certain point any N will give you the same number. So what we did was switch to an <a href="http://mpmath.org/">arbitrary precision arithmetic library</a> in python to try and figure out the population size needed.</p><p>Since these integral takes a bit to compute, and you have to keep increasing the precision (which makes it take longer and longer), we decided to regress on</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/328/1*u1sISuX6Tv1ILfcW5XgMVA.png" /></figure><p>Getting the graph, and estimating F as follows</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*ACT1dE0DA9aL8F6Nszn0Bw.png" /><figcaption>X-axis is log(N), and Y axis is number of standard deviations</figcaption></figure><p>From this we can estimate F(N) = 327, which solves to N = 10³⁸⁰⁶³, which is quite a lot!!!</p><p>For some perspective, there are ~10⁸⁰ atoms in the universe. So we need more people than atoms by about 10³⁸⁰⁰⁰. Now, if humans kept growing at our current rate, into different universes, how many years would it take us to get there?</p><p>Well, if we assume that humanity is growing at our current rate of <a href="http://www.worldometers.info/world-population/#growthrate">1.13%</a> we get</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/545/1*7prMVnv_SPGjBhJ3K6aucg.png" /></figure><p>Which solves to around T = 7.7 million years.</p><p>So in conclusion</p><ul><li>The smartest person to ever live up until now had an IQ of around 202</li><li>We will need about 10³⁸⁰⁰⁰ people to get to the 5000 IQ level</li><li>It will take 7.7 million years to match up with Alakazam</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e0a1409331bd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Primer on Quantum Computing and Algorithms]]></title>
            <link>https://medium.com/@geoff.bradway/a-primer-on-quantum-computing-and-algorithms-d64033abfb52?source=rss-ab02e41ba4d8------2</link>
            <guid isPermaLink="false">https://medium.com/p/d64033abfb52</guid>
            <category><![CDATA[computer-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[mathematics]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[quantum-computing]]></category>
            <dc:creator><![CDATA[Geoffrey Bradway]]></dc:creator>
            <pubDate>Mon, 18 Jul 2016 22:29:09 GMT</pubDate>
            <atom:updated>2016-07-19T19:41:04.299Z</atom:updated>
            <content:encoded><![CDATA[<h3>Intro</h3><p>The best way your average computer scientist could describe a quantum algorithm is “a riddle, wrapped in a mystery, inside an enigma”. In this article I hope to answer the fundamental question every computer scientist asks themselves from time to time “wtf is going on?”, but on the more limited scope of quantum algorithms.</p><p>Useful resources I have found along the way are</p><ul><li><a href="https://medium.com/@rigetticomputing/the-quantum-software-challenge-1a86eec1ce47#.sjy4ea5ea">An awesome introductory blog series on quantum computing</a></li><li>Quantum Algorithms via Linear Algebra [A ~200 page book which builds the theory up from scratch. Overall I thought it was pretty good, and readable, and most of this article is drawn from that book.]</li><li><a href="http://www.nature.com/articles/npjqi201523">Quantum Algorithms: an overview</a></li><li>Quantum Computation and Quantum Information [Described as the bible for quantum computing, but much like the real bible, it is fairly long and so I haven’t read all of it yet.]</li></ul><h3>Background</h3><h4>What should you know before reading this?</h4><p>It turns out that Quantum Computing looks a lot like machine learning (read: linear algebra). That being said, to get the most out of this this post it would be nice if you knew some linear algebra— but I want to make this accessible, so I will try to limit the technical background needed and explain everything as I go.</p><h4>What level of abstraction are we talking about?</h4><p>Typically, when someone says algorithm, most people think of something like</p><p>def cool_sort(my_list):</p><p>if is_sorted(my_list): return my_list</p><p>return cool_sort(random_shuffle(my_list))</p><p>And this would do something like</p><p>cool_sort([3,2,1]) = [1, 2, 3]</p><p>But for this post we are considering algorithms on a much lower level — the circuitry itself. The pieces at our disposal are more analogous to adders, ALUs, etc. For example, a half adder is a boolean circuit that just adds two bits together. The function looks something like</p><p>Half_Add(0, 0) = 00</p><p>Half_Add(0, 1) = 01</p><p>Half_Add(1, 0) = 01</p><p>Half_Add(1, 1) = 10</p><p>And this is done via circuits, and looks something like</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/180/1*Fnv8Kw-vyQ3rY9M6QHzc4Q.png" /></figure><p>Never mind exactly what the circuit physical does, that part is abstracted away from us, just keep in mind that we are doing the equivalent of a very low level algorithm, not the high level sorting we were doing before.</p><p>Later on when we talk about these U matrices, just keep in mind these are physical circuits operating on physical qubits. These are algorithms too, and I will be using “quantum algorithms” and “quantum circuits” interchangeably.</p><p>The question of designing a good quantum chip, programming language, etc are still very open. If you are curious about designing a full stack quantum computer, definitely check out the blog series I linked to above.</p><h3>Creating a Quantum Circuit</h3><h4>Mechanics</h4><p>In general, a function takes in some sort of input, does some useful calculations, and spits out an output. Quantum algorithms are no exception to this. In the quantum case — the input is some quantum bits, our program is some matrices U1, U2, U3,…, and the output is the same quantum bits we had before, but hopefully we have done some useful work, and the new state of those quantum bits is something meaningful.</p><p>The basic mechanics of quantum circuit are actually quite simple — it is just linear algebra and a tiny bit of probability.</p><p>A basic circuit follows a simple outline, but it’s okay to not understand exactly what this is doing right now. In fact I am trying to divorce it from what is going on physically to show that the math is easy. Once you are convinced it is easy, we will build up some intuition for what it is doing.</p><p>The outline of a quantum circuit looks like this</p><ul><li>Start out with a basic input vector, let’s call it X.</li><li>Multiply it by a bunch of square matrices U1, U2, U3, …, Un. So we get something that looks like <br>Un Un-1…U3 U2 U1 X = Y</li><li>Sampling an output proportionally to |Y|**2, where **2 is the element wise square. So if Y^T was [1/2, 1/2, 1/2, 1/2], then the element-wise square is [1/4, 1/4, 1/4, 1/4], and you have an equal probability of each outcome. This is also true if Y was [1/2, 1/2, -1/2, -1/2] or even [1/2, 1/2, 1/2 i, 1/2 i]. And this sampled outcome of Y is the result of your algorithm!</li><li>Voila! Pat yourself on the back because you just performed a quantum algorithm!</li></ul><p>There are a few rules we need to stick by to make this a valid algorithm</p><ul><li>||X||^2 = 1</li><li>U is unitary [meaning (U^T)U = I or U*U = I if U has complex entries].</li></ul><p>See? Super easy. Now we will build up some math intuition, and then I will tell you how this relates to everything, and then we will retouch on the outline with our brand new awesome understanding.</p><p>So let’s first do a little review of some linear algebra, and we will see how that ties into these unitary matrices.</p><p>If we remember back, the determinant of a matrix measures how a matrix changes the space it operates on. So if we have a matrix A with det(A) = 2, then one unit of space before will double in size once you apply A. Or if det(A) = 0, then one unit of space will get completely destroyed when you apply A, and that is why A isn’t invertible iff det(A) = 0, because once the space gets destroyed and you can’t un-destroy it. This effect on the change of the size of the space is why the determinant makes an appearance in the multivariable change of variables equation, because you need to re-scale based on how the system of equations resizes the space.</p><p>Well, one property that unitary matrices have is that |det(U)| = 1. This means that our unitary matrix will preserve things nicely, and it won’t stretch or shrink anything — it will just move stuff around. In fact, if we are talking about R^2, then the only possible unitary matrices we have are the identity, rotations about the origin, and twists about other vectors, or some combination of rotations and twists. But while the vector points somewhere else now, it is still the same length.</p><p>So geometrically, we can interpret our algorithm as this — we are starting with a vector who lies on a unit circle, and our transformation has to keep it on the unit circle. So by the end of applying all these matrices we have ||Y||^2 = 1, and when we sample via the square-normed elements, we are sampling from a legitimate probability distribution. Hooray!</p><p>Visually, this looks something like the blue arrows in <a href="https://www.researchgate.net/figure/263048802_fig5_Figure-4Example-Bloch-sphere-trajectories-for-a-Bloch-vector-starting-from-1-with-a">this picture</a>, the red arrows aren’t important for what we need to visualize.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*VWc4GwtQPRLJUppPXpnW4A.png" /><figcaption>If you want to sound *very very fancy* this circle is called the Bloch Sphere. And if you study really really hard, maybe one day you can have a trivial mathematical object named after yourself!</figcaption></figure><p>As we can see, we are applying various Us to the blue vector, but it is staying on the circle, it is just pointing in different directions!</p><p>Now we are ready to start talking about what is actually going on. These vectors are basically just fancy bookkeeping for the physical state the system could be in.</p><p>Let’s start simple— if we have a qubit, then we can measure it to be a 0 or a 1. But the qubit doesn’t have to be deterministically 0 or 1. We can have a qubit so that there is some chance it will be 0 or 1. Your first guess might be to represent these directly as probabilities, which would look like a vector [a, b], where a+b =1, and a,b&gt;=0. But as you may have figured out from what I was talking about above, we can also represent it [x, y], where |x|^2+|y|^2 = 1, and our qubit is 0 with probability |x|^2 and 1 with probability |y|^2. And it actually turns out, this is better.</p><p>One thing to note is that for 1 bit we needed two state variables, one for 0 and one for 1, and in general for n bits, we need vectors of size 2^n, because we need one row for each possible state.</p><p>If you look at the 2 bit case, then we care about vectors that represent [00, 01, 10, 11] or in the 3 bit case [000, 001, 010, 011, 100, 101, 110, 111].</p><p>[Technical aside, feel free to ignore this part: if these were probabilities, we would need only the specify (2^n)-1 values. This is because the probabilities have to add up to one and so you can always figure out what the last one is if you have all the other values. But in general in the quantum case we use complex numbers, and so you are looking at |x1|^2+|x2|^2+…. = 1. This means we that while the magnitude of the last value can always be figured out, for the same reason we could always figure it out for the probabilities, the value still needs to be specified because it can be any vector with that magnitude].</p><p>So the basis of the vector space we are considering looks like this (in the 2 bit case), [E00, E01, E10, E11]. A vector of [1, 0, 0,0] means that we for-sure have a qubit of 00, and a vector of [1/2, 1/2, 1/2, 1/2] means that if we measured our qubit, then 25% of the time it would be 00, 01, 10, or 11.</p><p>So let’s retouch on the outline of a quantum algorithm</p><ul><li>Start out with a basic input vector, let’s call it X, where ||X||^2 = 1</li></ul><p>X is the vector representation of the state of our input qubits, which is why ||X||^2 = 1 . In almost all of our quantum algorithms we want to start with qubits which we know the state. Since there is no uncertainty about what the state is the vector representation would be something like [1, 0, 0, 0] or [0, 1, 0, 0].</p><ul><li>Apply our unitary matrices U1, U2,… Un to get the result <br>Un-1…U3 U2 U1 X = Y</li></ul><p>Here we are running our “program”. Each one of these matrices had to be unitary, so it changes the state of our qubits, but they are still valid qubits.</p><ul><li>Figure out the new state of our qubits by sampling an output proportionally to |Y|**2</li></ul><p>Y is the new state representation of our qubits, and when we measure our qubits we are forcing them to take a specific value like 00 or 11. This is the same thing as picking one of our basis vectors E00, E01, etc. Then |Y|**2 just tells us how likely we are to see each possible configuration of qubits, and so this sampling procedure is equivalent to measuring the system.</p><h4>Deutsch-Jozsa Algorithm</h4><p><a href="https://en.wikipedia.org/wiki/Deutsch%E2%80%93Jozsa_algorithm#Motivation">From wikipedia</a>, the Deutsch-Jozsa Algorithm is the bee’s knees because:</p><blockquote>The Deutsch–Jozsa problem is specifically designed to be easy for a quantum algorithm and hard for any deterministic classical algorithm. The motivation is to show a black box problem that can be solved efficiently by a quantum computer with no error, whereas a deterministic classical computer would need exponentially many queries to the black box to solve the problem.</blockquote><p>The basic idea is that we want an algorithm that can figure out if some boolean function, F, is a constant function. This function F takes as input n-bits and returns as output a single bit or alternatively true or false.</p><p>Mathematically, this looks like</p><p>F:{0,1}^n -&gt;{0, 1}</p><p>So far so good. But because this is quantum computing, we can do something much fancier, and we can tell if F is constant only calling F once. Intuitively, this makes absolutely zero sense whatsoever , like seriously zero sense, because it seems like you would have to evaluate it on at least two inputs to at least tell if it wasn’t constant. But the trick (which we will talk about more in the “the magic of quantum” section) is that we can do one function call but over a distribution of states.</p><p>For this blog post we are going to talk about the simple case, where F is a one-bit function, so F:{0,1} -&gt;{0,1}.</p><p>First things first, there is a little bit of busy work to get out of the way before we talk about the algorithm, and it will be nice to have it out of the way so we don’t have to do it in the middle, so let’s do that. Now there are four possible one-bit F’s, F(x) = x, F(x) = not x, F(x) = 0, and F(x) =1. But we care about a different object related to F. For this section, xy means x concat y, and they are both bits. The function we care about is G where</p><p>G_F(xy) = x concat (F(x) XOR y)</p><p>So if F is the identity, <br>G_I(00) = 0 concat (F(0) XOR 0) = 0 concat (0 XOR o) = 00</p><p>Let’s make some G truth table matrices for each possible F. Remember, our basis [E00, E01, E10, E11]^T. So if we look at G*[1, 0, 0, 0]^T, that is the same as G(00), and G*[0, 0, 1, 0]^T will tell us what happened if did G(10). Since one specific input to G will have one specific output, we expect that each row will just have a single 1, indicating what the output bits will be.</p><p>Let’s work through a tangible example. Let us consider the case where F(x) = not x</p><p>Then G looks like</p><p>G(00) = 0 concat (F(0) XOR o) = 0 concat (1 XOR 0) = 01</p><p>G(01) = 0 concat (F(0) XOR 1) = 0 concat (1 XOR 1) = 00</p><p>G(10) = 1 concat (F(1) XOR o) = 1 concat (0 XOR 0) = 10</p><p>G(11) = 1 concat (F(1) XOR 1) = 1 concat (0 XOR 1) = 11</p><p>So the G_not matrix looks like</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/231/1*ReDPwlLLtYWW5jcmaiBMgA.gif" /></figure><p>And similarly, the other values of G look like (G_T is G true, G_F is G false, and G_I is G identity)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/543/1*wsXntlQp87Dn124-q7-9mA.gif" /></figure><p>Now, constructing G is where it seems like we are cheating. I said we could do this algorithm with one function call, but clearly we had to evaluate all possible function values to make G, and we are going to use G in our algorithm — so did I do more than one call in the algorithm?</p><p>The answer is no. G is only for bookkeeping. If we look at the circuit for the Deutsch-Jozsa Algorithm itself it looks like</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*r__epFtcvI1DdzqxqIFt9w.png" /><figcaption>Ignore the Hs and weird looking boxes</figcaption></figure><p>Where the part that they call Uf we are calling G, but you can see that it’s just circuit that operates on the two bit inputs. We needed to fill in all of the values for our analysis, just like you would have had to do with a normal truth table, but when the circuit runs, the two qubits go through that part only once.</p><p>With all the funness out of the way, we have that out of the way, we can present our algorithm</p><ol><li>Take X = [0, 1, 0, 0]^T (So we are starting with qubits 01)</li><li>Apply H4</li><li>Apply G</li><li>Apply R</li><li>Measure the output and look at the first bit</li></ol><p>Where H4 and R are [Note: I forgot a normalization constant on H4 to make it unitary, but that just means that the norm of all these vectors will be c instead of 1, and we were really missing a 1/c]</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/564/1*s5YvfT66CsHSSK3Ogp6dyg.gif" /></figure><p>Let’s do a step by step walkthrough of this</p><p>If we look at H4X we get the following results</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/497/1*v4-cTByJZMtNHWExj8sMPQ.gif" /></figure><p>The resulting vector of [1, -1, 1, -1]^T means that if we were to measure now, we have a 25% chance of seeing a qubit in any state — 00, 01, 10, 11. So we have taken our starting vector an we have almost “uniformly” distributed it across all possible states —but even though we would measure uniformly, there is more structure in the space since some entries have different signs!</p><p>Now if we look at GH4X for all possible G’s we get</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/523/1*C6IAEOTz_xRqZSiYbRgnhA.gif" /></figure><p>Which looks like a jumbled mess — but it will make sense in after we apply the next matrix.</p><p>RGH4X then looks like the following</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/571/1*H3Zlc-8mXlkPToxA4YKvpA.gif" /></figure><p>Now something really cool just happened — If you notice True and False, which are the only ways that F is constant, map only to E00 and E01, and Not and Identity map to E10 and E11. So if F was the True or False function, we don’t know if the output to our program will be 00 or 01, since it will happen with a 50/50 chance, but we know that the first bit will always be 0. Similarly for I and Not, there is a 50/50 chance it will be 10 or 11, but again the first bit is always 1. This means that if we look at the left-most bit of the output, we can tell if F was constant or not!</p><h4>The Magic of Quantum</h4><p>Now that we have all of the setup out of the way there are two reasons that quantum algorithms are cool</p><ol><li>The representation we picked (well mother nature picked for us), is a lot stronger than simple probability estimates, and can express more complicated things</li><li>Qubits store “distributions” and not instances</li></ol><p>The reason that I did the algorithm before this section is because we can see both reasons in the algorithm itself.</p><p>First let’s look at the representation but in the context of the Deutsch-Jozsa Algorithm. If we were to measure after GHX, it wouldn’t be very exciting. Each vector is different, but each component of each vector has the same magnitude. Any F function would make this act the same way at that point in the algorithm — an equal chance of measuring any state. So from a probabilistic perspective, this isn’t very interesting, they are all just uniform distributions. So we couldn’t do anything at this point, because each distribution is the same, and so there is nothing we could do to an observation to figure out which distribution it came from.</p><p>Here is a good example of the added expressiveness of this representation, because we aren’t representing it directly as probabilities, instead complex numbers, there is a lot more stuff we can do with it. In this particular case, we were able to use the R matrix to actually distinguish which class of function it came from, because there was more structure.</p><p>And if you think about it, there is one way to represent a uniform distribution over N numbers, namely a probability vector [1/N, 1/N,…, 1/N], but the space of x where ||x||^2 = 1, there are infinitely many points that could represent that distribution all of the form [e^(ia)/sqrt(N), e^(ib)/sqrt(N), e^(ic)/sqrt(N), …] and it even includes what we had before with the point [1/sqrt(N), 1/sqrt(N), … ]. So that is cool because you have distributions that from a probabilistic standpoint are the same, but mean different things, and interact differently.</p><p>The second really cool part is that qubits store “distributions” and not instances. To examine this, let us consider the step where we applied G to H4X</p><p>Let’s remember what H4X looks like</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/497/1*v4-cTByJZMtNHWExj8sMPQ.gif" /></figure><p>And we can see that part of our qubits are in each possible configuration. Then when we multiply G by H4X, its like we get to do a G call on each possible state. <strong>Four for the price of one! </strong>(Or in the general case, 2^n for the price of 1!)</p><p>To better see this, let’s look at what would be the classical analog. Remember before when I mentioned that if you have a deterministic boolean function, then using our truth-table matrix you can do one look up it at once — this is is the same as doing G * E, where E^T is one of the following [1, 0, 0, 0], [0, 1, 0, 0],[0, 0, 1, 0], or [0, 0, 0, 1]. So the equivalent classical operation is</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/442/1*9NRo5zOjzghdoKj4-1P5ZA.gif" /><figcaption>Decomposing H4X</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/557/1*OKkPnndpFyfjFu3Yf_HUMQ.gif" /></figure><p>Which is more inline with what we would have expected — us needing 4 evaluations of F to tell if all outputs are equal.</p><p>But, because qubits don’t take on a value until we measure them, we can get away with some crazy stuff.</p><p>So in conclusion, by using a better representation and by being able to carry more state, you can get away with some pretty cool stuff!</p><p>Lastly, I want to thank my friends who I conscripted to give me feedback — Matt, Eric, Yunus, and Cathy.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d64033abfb52" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to be a Data Scientist]]></title>
            <link>https://medium.com/@geoff.bradway/how-to-be-a-data-scientist-583779bfbe61?source=rss-ab02e41ba4d8------2</link>
            <guid isPermaLink="false">https://medium.com/p/583779bfbe61</guid>
            <dc:creator><![CDATA[Geoffrey Bradway]]></dc:creator>
            <pubDate>Sun, 26 Jul 2015 13:56:36 GMT</pubDate>
            <atom:updated>2015-07-26T13:56:36.908Z</atom:updated>
            <content:encoded><![CDATA[<p>The other day I was talking with my friend Connor, and he was curious about an optimization class he wanted to do. He was wondering what sort of background he would need to do well in optimization, and how that played into the machine learning community as a whole.</p><p>So why listen to me?</p><p>I’m just some guy on the internet. But I have a degree in math and another in stats, I’ve worked on experimentation software in some form or another for 2.5 years, and now I’m doing AI research. That being said, my background is more academic, and I love structure and theory, so that definitely colors my opinion on the subject.</p><p>What is a data scientist?<br>A data scientist is simply a person who is good at statistics as well as programming and does something that requires both. That’s it.</p><p>So what topics are there, and how do they relate?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g6iKmHXA0nACYwcHEKTmyQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xzmISFPErzKnXSMllSGLuQ.png" /></figure><p>Some justification for weird things on there:</p><p>Real analysis — Honestly if there was one subject that I think is systematically undervalued for data scientists, it’s real analysis. I think this gets undervalued because most data scientists usually don’t come from academia, and if they do, it’s from the CS realm. Real analysis is a pretty theory heavy branch of math, but it is the theory that we care about — it’s the theory that is trying to describe all the quantitive things you do in data science. It is essentially the backbone of all applied math (optimization, linear alg, stats, prob). In real analysis you study things like what it means for <strong>two objects to be close </strong>(how do we know our model’s estimated parameters are close to the ‘true’ parameters?), <strong>what it means for a function to converge to another function</strong> (how do we know our estimated distribution gets close to the real distribution? How far off of the real distribution is our estimate?), <strong>you learn why making the distinction of discrete random variables and continuous random various is stupid</strong> (its because you study intro probability theory with the wrong type of integral! — it turns out most probability and statistics boils down to integrating things, and estimating those integrals), <strong>functional approximation </strong>(how do I approximate my function as a polynomial, with sines and cosines, complex exponentials, with exploding polynomials [okay, thats actually complex analysis], on Fourier space. How do I know the types of functions that I can use to approximate another function, and what will the error be?)</p><p>OS/compilers — OS is on there because I think people who like data science would like OS, and it’s a ton of fun. It’s basically how do I manage all the resources on a computer in a coherent way? I would describe it as operations research for a computer, and operations research is just data science for business. Compilers teaches you basically how to write awesome code, and what as a programmer you need to do and what you don’t need to do, which is needed for people who want to do high-performance stats or scientific computing.</p><p>Programming Languages — Most of the time you will be using python, R, SAS, matlab. And the fact is, while those are fine for the occasional homework project, they suck for writing large projects. PL gives you exposure to the wide variety of CS stuff out there, and how to pick the best language for the best project. It also exposes you to hidden gems you wouldn’t necessarily come across out in the wild, like Prolog or Church [and probabilistic programming seems pretty awesome].</p><p>Some nice resources I’ve found:</p><p>Real analysis:<br>Hard, but good — <a href="http://www.maa.org/publications/maa-reviews/principles-of-mathematical-analysis">http://www.maa.org/publications/maa-reviews/principles-of-mathematical-analysis</a></p><p>OS — <a href="http://pages.cs.wisc.edu/~remzi/OSTEP/">http://pages.cs.wisc.edu/~remzi/OSTEP/</a></p><p>Deep Learning — <a href="https://www.coursera.org/course/neuralnets">https://www.coursera.org/course/neuralnets</a></p><p>Robotics (I haven’t actually tried this one, but it seem cool) — <a href="http://ocw.mit.edu/courses/aeronautics-and-astronautics/16-412j-cognitive-robotics-spring-2005/index.htm">http://ocw.mit.edu/courses/aeronautics-and-astronautics/16-412j-cognitive-robotics-spring-2005/index.htm</a></p><p>AI — <a href="http://aima.cs.berkeley.edu/">http://aima.cs.berkeley.edu/</a></p><p>ML — <a href="http://research.microsoft.com/en-us/um/people/cmbishop/prml/">http://research.microsoft.com/en-us/um/people/cmbishop/prml/</a></p><p>Blogs/papers —<br><a href="http://matt.might.net/">http://matt.might.net/</a><br><a href="http://colah.github.io/">http://colah.github.io/</a><br><a href="http://worrydream.com/refs/Shannon%20-%20A%20Mathematical%20Theory%20of%20Communication.pdf">http://worrydream.com/refs/Shannon%20-%20A%20Mathematical%20Theory%20of%20Communication.pdf</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=583779bfbe61" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Down the Rabbit Hole]]></title>
            <link>https://medium.com/@geoff.bradway/down-the-rabbit-hole-744a2549b648?source=rss-ab02e41ba4d8------2</link>
            <guid isPermaLink="false">https://medium.com/p/744a2549b648</guid>
            <dc:creator><![CDATA[Geoffrey Bradway]]></dc:creator>
            <pubDate>Tue, 16 Dec 2014 03:38:16 GMT</pubDate>
            <atom:updated>2014-12-16T03:38:16.050Z</atom:updated>
            <content:encoded><![CDATA[<p>I would like to take a minute to reflect on some of the most important things I&#39;ve figured out in life, and how I figured them out.</p><p>I remember the first time I was truly blown away by learning. I was taking calculus and we were learning about a special type of circuit, called an LC circuit. We wrote down the equations that described the circuit, and when we solved it, I saw something astonishing. The solution was just sines and cosines. The current oscillated back and forth, exactly like a weight on the end of a spring. That amazed me, and I decided I wanted to be an electrical engineer. I saw something cool, and I decided that I wanted to do it.</p><p>And I remember the first time I bit off more than I could chew. It was tightly coupled with the LC circuit incident. I decided to take a physics class about electricity before I had the one that teaches you how things fall down(a strict prerequisite you see…), and concurrently with the math class it needed. I worked 10 hours on the first problem set to get a 50%. It was the hardest I had ever worked on something, and it was the worst I have ever done. I was devastated, and I decided that I wasn&#39;t going to let that happen again. So I started spending 15 hours, 20 hours, as many hours as needed to succeed. Then at the end of it I realized something important. It was a Herculean task, but that only made besting it better.</p><p>I decided that I didn&#39;t want to be an engineer shortly after I decided I did. What happened was I wanted to do all the engineering math before I got to college, so I would kick ass, and so one semester I ended up taking two math classes. The first math class was an engineering math class, and the second (to get my math minor) was a math math class. What I found was that the engineering math class was spoon fed. It was just trying to teach you which equations to use where. The math math class (introduction to analysis, for the curious) was challenging and fun. It made me think hard and it gave me a chance to excel, and I really loved that. I decided that I wanted to do math. I saw something cool, and I decided I wanted to do it.</p><p>And so that’s what I did for a while. It was fun. I learned why the infinite sum of 1/n^2 is pi^2/6. I learned why the predatory prey population is cyclical [ it’s because when you write down the math that describes it, the solution is sines and cosines! aka imaginary eigenvalues!]. I learned dozens of neat ways to approximate functions. It was a good time.</p><p>But I decided I didn&#39;t really want to be a professor, so I needed a different pursuit, something that would get me an industry job. The thing I really loved was called real analysis. I thought I would try out something similar, called statistics, and it turned out I loved that even more because it is really analysis under the hood, but with way more cool real world application.</p><p>I also debated taking a programming class. A few of my closest friends really loved doing that sort of stuff, but it didn&#39;t seem particularly relevant. I decided what the hell and took it anyways. This led to another programming class. And another. And another. Soon some machine learning and AI followed, and hey, it turns out it did have to do with statistics. Programming empowered me more than the other statisticians, and math empowered me more than the other programmers. It was awesome. Now computer science is central to what I do, and who I am, all because of a why not.</p><p>The same types of things cropped up in CS though. The exact same process of exploring and discovering new intellectual loves happened again. The same process of “insurmountable” challenges. I took a parallel programming class with any real systems knowledge (and zero C/C++ experience), and it was tough, but I focused and did it, and I put hundreds of hours into it. In the end it really solidified a lot of knowledge about computers for me and fundamentally changed the way I see solving big problems.</p><p>What made me do well is that I consumed everything that the university could offer me. I took as many internships as I could get, I often talked my way into Phd classes I was 4 or 5 prereqs away from (if a class sounds cool, it is and you should take it. If you don&#39;t have the prereqs, talk to the professor. They want passionate people above all else.). I would sit in the hardest seminars I could find, and attend, listen and ask questions until it all seemed obvious. I went to student conferences, professional conferences, any conference I could get my hands on. I would pester professors with hard questions. I would do anything to learn and take advantage of what the university could offer. There was no shortage of resources at my university. Just like there is no shortage of resources in the world. There is a shortage of people who take advantage of them though.</p><p>Here’s the kicker though. While I love what I do, there is so much more to everything than that. If you ask me what my favorite class in high school was, it was my small metals class. Some of my favorite college classes were drawing, creative writing, a cross discipline paleontology class. They were all unexpected opportunities for me to try new things, and do things I normally wouldn&#39;t.</p><p>If you told me 4 years ago when I started exploring I would have ended up here, I wouldn&#39;t have believed you. I started trying to engineer, then went to math, then statistics, and finally to data science, trying and failing on the way. Now I work on data projects that affect billions of people, something I would have thought to be impossible 4 years ago.</p><p>Plus I&#39;ve just got a fancy new circuit board called an FPGA to mess around with in my free time, which brings me full circle back to electrical engineering.</p><p>So here are the main things I&#39;ve learned about life up until this point</p><ul><li>Do things that are absurdly challenging. It will make you grow.</li><li>Don&#39;t be afraid to try and fail. Failure teaches you a valuable lesson about what works and what does, and failure is almost always a necessary and sufficient condition for success.</li><li>Explore things. If it sounds interesting, it probably is, but you won&#39;t know unless you put effort into it.</li><li>Be an opportunist. Opportunity is littered everywhere.</li><li>Most importantly: Don&#39;t be afraid to go down the rabbit hole. This is how you learn the most about yourself.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=744a2549b648" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>