<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Anton Muehlemann on Medium]]></title>
        <description><![CDATA[Stories by Anton Muehlemann on Medium]]></description>
        <link>https://medium.com/@antonmu?source=rss-4e398f0112fa------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*bZwsKreqPEzdXscFjjTmpQ.jpeg</url>
            <title>Stories by Anton Muehlemann on Medium</title>
            <link>https://medium.com/@antonmu?source=rss-4e398f0112fa------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 21:43:26 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@antonmu/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[US Census 2020]]></title>
            <link>https://medium.com/data-science/us-census-2020-868e8555313?source=rss-4e398f0112fa------2</link>
            <guid isPermaLink="false">https://medium.com/p/868e8555313</guid>
            <category><![CDATA[usa]]></category>
            <category><![CDATA[census]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[towards-data-science]]></category>
            <category><![CDATA[data-visualization]]></category>
            <dc:creator><![CDATA[Anton Muehlemann]]></dc:creator>
            <pubDate>Tue, 25 Feb 2020 14:18:47 GMT</pubDate>
            <atom:updated>2020-02-25T14:18:47.138Z</atom:updated>
            <content:encoded><![CDATA[<h4>Visualizing US census data over the past 230 years!</h4><p>2020 is a special year for all Americans — it is the year of the US Census. The US census is one of the biggest population surveys in the world and is mandated by the US constitution. It takes place every ten years and helps the government decide where to build and maintain schools, hospitals, transportation infrastructure, and police departments to name just a few. All in all, the various censuses determine the allocation of over $400 billion in federal funds every year.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fpublic.flourish.studio%2Fvisualisation%2F1322083%2Fembed&amp;display_name=Flourish&amp;url=https%3A%2F%2Fpublic.flourish.studio%2Fvisualisation%2F1322083%2F&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=flourish" width="700" height="575" frameborder="0" scrolling="no"><a href="https://medium.com/media/55d2312a2373223aa2036e6e49be2d72/href">https://medium.com/media/55d2312a2373223aa2036e6e49be2d72/href</a></iframe><p>To celebrate this occasion, I collected 230 years of US census data and turned them into a bar chart race. I <a href="https://github.com/AntonMu/Census2020/blob/master/USCensus2020.ipynb">documented all the steps</a> that went into this data project — so that you can follow them if you would like to tackle a similar project in the future.</p><h3>230 Years of US Census</h3><p>Back in 1790, the US only had 16 states and the total population was just a little shy of 4 million — less than the city of Los Angelos today! Less than 350,000 people lived in the entire state of New York, and Virginia — with over 700,000 inhabitants — was by far the most populous state.</p><p>But this dominance didn’t last for long — with the founding of Wall Street in 1792 and the establishment of strategic coal and lumber ports, New York quickly became the economic leader of the US. In 1810, the state of New York reached close to 1 million inhabitants and became the most populous US state — a title which it was able to keep for the next 150 years!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c9O8SW3iO6Nv4kVETNwdBw.png" /></figure><p>So who took the first place? You guessed it — California! Starting in 1940, with the rise of advanced technology such as aerospace and electronics, California became the new US economic powerhouse. On average, California’s population grew by more than 4 million people per decade — a trend that continues to this day.</p><p>These are just some examples of what the data can show us, but there are many more things to explore. For instance, what caused the sudden rise of population in Ohio starting in 1820 and that in Illinois in 1960? Why did Virginia’s population drop after 1860? What caused the rapid rise in Florida’s population in the mid 20th century?</p><h3>Creating a US Census Bar Chart Race</h3><p>Apart from sharing some interesting facts about the US, this post will also help you tackle your own data project and teach you how to use some of the most popular tools for data scientists. For an in-depth description of all steps, you can check out my tutorial on <a href="https://github.com/AntonMu/Census2020/blob/master/USCensus2020.ipynb">github.com</a>. To present the data, I chose a bar chart race, a data animation tool that became extremely popular in 2019 — with trending posts on Twitter, Facebook and LinkedIn alike.</p><h4>Data Acquisition</h4><p>Like almost all data science projects, I started with acquiring the data. I decided to use Wikipedia’s well-curated articles on <a href="https://en.wikipedia.org/wiki/2010_United_States_Census">the US Census</a>. Even though parsing data from the web is generally not recommended, in this case, Wikipedia’s US census page layout was relatively consistent and thus easy to parse. Nonetheless, as with almost all parsed data, some cleaning up was needed. To obtain the raw data, I used <a href="https://pandas.pydata.org/">pandas</a>’ read HTML function.</p><pre>pd.read_html(<br>“https://en.wikipedia.org/wiki/1790_United_States_Census”)</pre><h4>Data Cleaning</h4><p>Once I got the raw data, it was time to clean up different formatting and make sure that all numbers look reasonable. Consistency checks are often overlooked in data science projects and can lead to a lot of trouble down the road. In my case, the year <a href="https://en.wikipedia.org/wiki/1800_United_States_Census#State_and_regional_populations">1800</a> didn’t play well with the others and used a completely different table layout. Unlike every other year, in 1800, the census data was broken down into separate districts.</p><pre><strong>State</strong><br>New York (excluding Duchess, Ulster, Orange co...<br>New York (Duchess, Ulster, Orange counties)</pre><p>In this case, I was lucky enough to spot these inconsistencies early on. But in practice, you may not always be that lucky. But worry not — it is totally okay to just carry on. Once you start visualizing your data, these small hiccups should become more apparent. For instance, in our example, this error would result in a sudden apparent drop in New York’s population in 1800. Once you see such an anomaly, there are two possible scenarios: either something cool happened or your data is faulty. It is best to not mix the two!</p><p>To fix the problem of separate districts, I matched the row names with the names of the current 50+1 states and then summed up all populations that shared the same row name. After some additional cleaning up, the data was good to go.</p><h4>Data Visualization</h4><p>Once the data was in a good format, the bulk of the work was over. For creating the bar chart animation, I used an app called Flourish studio which seemed to offer a friendly interface and layout for this task.</p><h3>Conclusion</h3><p>I hope that this abbreviated walkthrough of a data project was helpful for some people who are just getting started with data science. For the full tutorial head over to <a href="https://github.com/AntonMu/Census2020">github.com/AntonMu/Census2020</a>.</p><p>Even from this small data project, we can take away two important lessons:</p><blockquote>Data cleaning matters!</blockquote><p>Data cleaning and consistency checks are often the most time-consuming part of any data science project and are rarely the most glamorous. However, to become a successful data scientist, you need to invest the time to do them properly or else <a href="https://en.wikipedia.org/wiki/Garbage_in,_garbage_out">even the fanciest model won’t save the day</a>.</p><blockquote>Use powerful visualizations!</blockquote><p>Once you are done with data cleaning — have some fun! I chose to visualize the data with a bar chart race, but there are many different ways to visualize data and it is worth exploring some options before settling. Sometimes a simple table is enough but often there are better and more engaging ways to present and learn from your data. The benefits of powerful visualizations are twofold. It can improve <em>your understanding</em> of the data and guide you to some potential flaws in your data that need some further cleaning up. It can improve <em>your audience’s understanding</em> of the data. A great visualization can captivate your audience and guarantee that your message gets delivered — whether this is to your boss, your family or your best friend.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=868e8555313" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/us-census-2020-868e8555313">US Census 2020</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[There is No Noise — Only Bias]]></title>
            <link>https://medium.com/insight-data/there-is-no-noise-only-bias-f68f8b8e423a?source=rss-4e398f0112fa------2</link>
            <guid isPermaLink="false">https://medium.com/p/f68f8b8e423a</guid>
            <category><![CDATA[bias]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[insight-ai]]></category>
            <category><![CDATA[variance]]></category>
            <category><![CDATA[noise]]></category>
            <dc:creator><![CDATA[Anton Muehlemann]]></dc:creator>
            <pubDate>Thu, 20 Feb 2020 18:18:02 GMT</pubDate>
            <atom:updated>2020-04-21T16:28:30.801Z</atom:updated>
            <content:encoded><![CDATA[<h3>There is No Noise — Only Bias</h3><h4>Why noise in machine learning is nothing but bias</h4><p>In this post, we explain how bias and noise in machine learning are two sides of the same coin.</p><blockquote>God does not play dice. — Albert Einstein</blockquote><p>Einstein famously gave this statement in reaction to the emerging theory of quantum mechanics, which seemed to defeat the fundamental laws of physics. Read on to see how the same goes for the fundamental laws of machine learning.</p><h3>The Classical Bias-Variance-Noise Tradeoff</h3><p>Before getting into it, let’s briefly review the classical bias-variance-noise tradeoff. The prediction error of a machine learning model, that is, the difference between ground truth and the learned model, is traditionally composed of three parts:</p><blockquote>Error = Variance + Bias + Noise</blockquote><p>Here, <strong>variance</strong> measures the fluctuation of learned functions given different datasets, <strong>bias</strong> measures the difference between the ground truth and the best possible function within our modeling space, and <strong>noise</strong> refers to the irreducible error due to non-deterministic outputs of the ground truth function itself. If you want to learn more about the bias-variance tradeoff, you can check out <a href="https://blog.insightdatascience.com/bias-variance-tradeoff-explained-fa2bc28174c4">this article</a>.</p><h3>Example: Playing Dice</h3><p>We are not God — so let’s play some dice! In the following three experiments, we repeat the same steps each time — we roll a die and record the resulting number. The only difference between the experiments is the number of features we are considering.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*btlbvnjxOUWVlRTSk0fkyA.png" /></figure><h4><strong>1. No Features</strong></h4><p>More often than not, rolling a die is associated with generating a random number <strong>M</strong> between 1 and 6 with an equal probability of 16.6%. From this perspective, the output function (the number rolled) is completely non-deterministic, and the error is fully characterized by the noise term.</p><blockquote>Error = Noise</blockquote><p>Therefore, the expected error and standard deviation are simply those of rolling a die (<strong>E = 3.5 </strong>and <strong>σ = 105/36</strong>). Since we are not using any features, there is no model to learn and thus no variance. But is there also no bias?</p><h4><strong>2. Some Features</strong></h4><p>Let’s repeat the <strong>same </strong>experiment, but this time, let’s record the number <strong>N</strong> facing up at the moment the die is released and the height <strong>h</strong> it is dropped from. Based on these two features, we can generate pairs of training data <strong>x = (h, N)</strong> and <strong>y = M</strong>, where <strong>M</strong> is the result of the die roll. Armed with enough training data and a good model, we expect that for two new inputs <strong>h</strong> and <strong>N</strong> our model is able to predict <strong>M</strong> better than chance. For example, if <strong>N = 1 </strong>and <strong>h = 5 cm</strong>, our model may predict <strong>M = 1</strong> with 60% confidence and thus show an improvement over our first model which would have predicted <strong>M = 1 </strong>with only 16.6% confidence. Thus, we managed to reduce the overall prediction error by reducing the noise term. By training a machine learning model we also introduced a bias and variance term in our overall error.</p><blockquote>Error = Variance + Bias + Noise</blockquote><p>But didn’t we just repeat the same experiment?</p><h4><strong>3. All Features</strong></h4><blockquote>Rolling a die is completely deterministic and there is absolutely no randomness in it.</blockquote><p>As long as we keep track of all relevant quantities such as initial speed, angular momentum, air resistance, drop height, etc., we can predict the outcome of the roll with 100% accuracy. We don’t even need a machine learning model to predict the outcome. Instead, we can apply the laws of physics. But the laws will get complicated, so for the sake of our example, let’s train a machine learning model instead. In this case, we expect that noise is completely eliminated and we are left with just bias and variance.</p><blockquote>Error = Variance + Bias</blockquote><p>If we consider a very complex (and suitable) modeling space, we will have almost no bias. If we further assume a huge amount of training data, our variance will also be very small. In such a case, our overall prediction error will be close to zero.</p><h4>Putting it all Together</h4><p>In each of the three experiments above, we threw a single die. Nonetheless, in the first example, our “irreducible error” (or noise) was very large, whereas in the last example it was completely gone. How is that possible?</p><p>The answer is simple: there is no such thing as “irreducible error”. When we threw the die in the first experiment, we simply ignored most of our reality. It was ignorance in our experimental design that led to this apparent noise in our output. In the last experiment, in contrast, our awareness was at 100%, and with all that information, we eliminated the apparent noise. But of course, each time the reality was the same; all that changed was our perspective (or <strong>bias</strong>)<strong> </strong>when observing the rolling die.</p><p>Put differently, noise is never inherent to any observed phenomena but is merely a consequence of an observer’s bias to not collect enough features.</p><h4>Practical Implications</h4><p>The above thought experiment may seem purely academic, but it has important practical implications.</p><p>When training a machine learning model, we often apply tricks and tips that help reduce bias or variance, and reducing one often increases the other. But what about noise? Is it more like bias or more like variance, and which techniques should we apply to reduce it? The answer is: noise is bias! Therefore, the same <a href="https://www.youtube.com/watch?v=EQWr3GGCdzw">techniques that reduce bias also reduce noise</a>, and vice versa.</p><p>In particular, techniques that reduce variance such as collecting more training samples won’t help reduce noise. Instead, adding more features and considering more complex models will help reduce both noise and bias.</p><p>For illustration, let’s arrange our training samples in a matrix form, with each row representing one training sample and each column representing one feature. Extending this matrix on the bottom by adding more training samples will reduce variance. Extending this matrix to the right by adding more features will reduce noise and bias.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6fIwKZ5PjSLBl5QomDq3rg.png" /></figure><h4>Conclusions</h4><p>In pretty much all practical situations, we find ourselves in the second scenario where we have some (but not all) possible features and thus have some apparent noise. The takeaway message is not that (apparent) “noise” doesn’t exist but rather that “noise” is a misleading term. Perhaps it would be more honest to call it <strong>“feature bias</strong>,” as it stems from our bias of ignoring parts of reality. In this language, what is classically called “bias” should really be called <strong>“modeling bias,</strong>” as it stems from our particular choice of one model over another.</p><p>Next time you face a machine learning problem and find yourself trying to reduce “noise”, pause for a second and ask yourself if you aren’t, in fact, after bias.</p><p>Finally, I hereby challenge you to find a machine learning problem in which noise is not bias, meaning that there truly is an “irreducible error.” If you think you’ve found one, leave a comment below!</p><p><strong><em>Interested in more content like this?</em></strong><em> </em><a href="https://notify.insightdatascience.com/partner?utm_source=dataiku-noisebias&amp;utm_medium=blog"><em>Sign up</em></a><em> for the latest resources on how to find and hire the best talent for specialized roles in data and tech, and </em><a href="https://insightfellows.com/partner#get-in-touch?utm_source=dataiku-noisebias&amp;utm_medium=blog"><em>reach out</em></a><em> to learn more about Insight.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f68f8b8e423a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/insight-data/there-is-no-noise-only-bias-f68f8b8e423a">There is No Noise — Only Bias</a> was originally published in <a href="https://medium.com/insight-data">Insight</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Bias-Variance Tradeoff Explained]]></title>
            <link>https://medium.com/insight-data/bias-variance-tradeoff-explained-fa2bc28174c4?source=rss-4e398f0112fa------2</link>
            <guid isPermaLink="false">https://medium.com/p/fa2bc28174c4</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[insight-data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[variance-analysis]]></category>
            <category><![CDATA[bias]]></category>
            <dc:creator><![CDATA[Anton Muehlemann]]></dc:creator>
            <pubDate>Thu, 19 Dec 2019 15:37:23 GMT</pubDate>
            <atom:updated>2020-02-20T18:26:55.341Z</atom:updated>
            <content:encoded><![CDATA[<h4>Understanding the Bias-Variance tradeoff at three different levels: simple, intermediate and advanced.</h4><p>In this post, we explain the bias-variance tradeoff in machine learning at three different levels: simple, intermediate and advanced. We will follow up with some illustrative examples and discuss some practical implications in the end.</p><blockquote>If you can’t explain it simply, you don’t understand it well enough. — Albert Einstein</blockquote><p>Understanding the bias-variance tradeoff can be a bit tricky at first as it involves several quantities that are <em>never </em>available in practice. Nonetheless, it provides insights into best practices for optimizing real-world machine learning applications.</p><h3>Simple</h3><p>The prediction error of a machine learning model, that is the difference between the ground truth and the trained model, can be decomposed as the sum of two error terms: the bias term and the variance term. Since the total error is the <em>sum </em>of these two error terms, there is a trade-off between the two.</p><blockquote><strong>Total Error = Bias + Variance</strong></blockquote><p>The bias term measures how well our choice of a model, for example linear models, can approximate the ground truth in a best-case scenario. The variance term measures how tightly clustered trained models are around this best-case scenario. Low variance means that all trained models are very close to the best-case scenario and high variance means that they are spread out.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IiSfk79msX9BOPIB_m4i8g.png" /><figcaption>The bias-variance tradeoff visualized. The <strong>bias </strong>is the distance between the ground truth <strong>f</strong> and the best possible approximation within our modeling space (<strong>ğ</strong>). The <strong>g</strong>{<strong>D</strong>}‘s are the results of training on different datasets <strong>D</strong>. The <strong>variance </strong>measures how much each trained function fluctuates about the best possible approximation <strong>ğ</strong>.</figcaption></figure><h3>Intermediate</h3><p>The bias-variance tradeoff refers to a decomposition of the prediction error in machine learning as the sum of a bias and a variance term.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M1Sorb9-nT3R3SleSbwL3g.png" /></figure><p>Here <strong>f</strong> is the ground truth function and <strong>g</strong>{<strong>D</strong>}<strong><em> </em></strong>is the learned function given a dataset <strong>D</strong> and given your choice of the modeling space. Examples of modeling spaces could be linear models, neural networks or random forests.</p><p>The term <strong>ğ</strong> is interpreted as the “best we can do” given our modeling choice. In other words, even if we had an infinite amount of training data, we would still end up with <strong>ğ</strong> because we can only find models that lie within our modeling space.</p><p>The first term (<strong>f — ğ</strong>)<strong>² </strong>is the<em> bias </em>term and measures the distance between the “best we can do” and the ground truth.</p><p>The second term (<strong>ğ — g</strong>{<strong>D</strong>})<strong>² </strong>is the<em> variance </em>term and measures how well we can “zoom in” on the “best we can do” given different training data sets <strong>D</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MH7Zix9Lj1KdrUx5ODslwg.png" /><figcaption>An example of the bias-variance tradeoff in practice. On the top left is the ground truth function <strong>f</strong> — the function we are trying to approximate. To fit a model we are only given two data points at a time (<strong>D</strong>’s). Even though <strong>f</strong> is not linear, given the limited amount of data, we decide to use linear models. On the bottom left, we see <strong>ğ </strong>— the best linear approximation to <strong>f</strong>. The error between <strong>f</strong> and <strong>ğ </strong>represents the <strong>bias</strong>. On the right, we see three different linear functions (<strong>g</strong>{<strong>D</strong>}’s) which were learned from the three different datasets (<strong>D</strong>’s). The fluctuation of the <strong>g</strong>{<strong>D</strong>}’s about <strong>ğ </strong>is responsible for the <strong>variance</strong>.</figcaption></figure><h3>Advanced</h3><p>The bias-variance tradeoff is a decomposition of the expected prediction error of a machine learning model into a bias term and a variance term. To fully understand this decomposition we need to first define three concepts.</p><p>First, there is the ground truth function <strong>f</strong> (“God’s function”). The ultimate goal is to recover God’s function purely based on its data outputs. Think of “God’s function” as the function that lets you find outputs to any given input. For instance, this could be a cat expert that tells us if there is a cat in an image.</p><p>Second, there is the domain <strong>X</strong> on which God’s function is defined. With <strong>X</strong>, we can associate an (unknown) probability distribution <strong>P</strong>. By drawing samples <strong>x</strong> from <strong>X </strong>we can generate a dataset <strong>D</strong>. Each dataset <strong>D</strong> is sampled from the collection of all possible datasets <strong>Đ</strong>. Just like <strong>X</strong>, there is another (unknown) probability distribution on the set of all datasets <strong>Đ</strong>. For example, in our cat examples, <strong>X</strong> is the set of all possible pictures and there is some probability of encountering any particular image. Typically <strong>D</strong> would be a training or test dataset that contains a number of different images and the set <strong>Đ</strong> would be the set of all sets of images.</p><p>Third, there is the hypothesis set <strong>H</strong>, which specifies the class of functions (models) we would like to consider when approximating God’s function <strong>f</strong>. Unlike all previous notions, the choice of modeling space explicitly depends on your own choice. Unsurprisingly, this choice is responsible for the <em>bias </em>term. Examples of hypothesis sets could be neural networks, linear models or random forests.</p><p>The goal of machine learning is to find an element <strong>g</strong> within our hypothesis set <strong>H</strong> that matches the given dataset <strong>D </strong>best. We make this dependence explicit by calling the learned function <strong>g</strong>{<strong>D</strong>}.</p><p>Now suppose that we had access to all possible datasets <strong>Đ</strong>. Computing the expectation of <strong>g</strong>{<strong>D</strong>} with respect to those datasets gives us an “average” hypothesis <strong>ğ</strong>.<strong> </strong>We can interpret <strong>ğ </strong>as the best possible function within our hypothesis set <strong>H</strong>. With <strong>ğ</strong> at hand, we are ready to write the bias-variance tradeoff (see <a href="https://github.com/AntonMu/BiasVarianceTradeoff/blob/master/Bias_Variance_Tradeoff.pdf">pdf</a> for details on the derivation):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8K9BIwRGHLhhETdRs7jWfw.png" /></figure><blockquote>Error = Bias + Variance</blockquote><p>On the left, we have the expected prediction error as the squared difference between the ground truth function <strong>f</strong> and the learned function <strong>g</strong>{<strong>D</strong>}. The expectations are taken with respect to both the domain and the datasets.</p><p>Typically, the performance would only be measured on a test set which is a subset of <strong>X</strong>. But instead of changing <strong>X</strong>, it is easier to think about changing the probability distribution <strong>P</strong> in such a way that it only considers elements in <strong>X</strong> that are contained in the test set. However, it is worth noting that the bias-variance tradeoff holds true irrespectively of your model design and whether you measure your error in the training, validation or test set.</p><p>The first summand is the <em>bias </em>term and tells us how far the ground truth <strong>f</strong> is from <strong>ğ</strong>. Recall that <strong>ğ </strong>can be interpreted as the best possible approximation of <strong>f </strong>within the hypothesis set <strong>H</strong>. Notably, neither the ground truth nor the best possible approximation depends on the dataset <strong>D</strong>.</p><p>The second summand is the <em>variance </em>term and measures the expected fluctuation of each hypothesis <strong>g</strong>{<strong>D</strong>} about <strong>ğ</strong>.</p><p>As a side note, if god’s function was non-deterministic, that is if an <strong>x</strong> has more than one label, there would be an additional noise term in the decomposition. To learn more about the role of noise, you can check out this <a href="https://blog.insightdatascience.com/there-is-no-noise-only-bias-f68f8b8e423a">article on noise and bias</a>.</p><h3>Examples</h3><p>Let’s look at some examples of the bias-variance tradeoff.</p><h4>No Bias and no Variance</h4><p>Since the prediction error is the sum of bias and variance, the only certain way to have no error is if our modeling space only contains God’s function <strong>f</strong>, meaning <strong>H</strong>={<strong>f</strong>}. Since our modeling space contains God’s function <strong>f</strong>, there is no bias. Furthermore, since the modeling space only has one element, we don’t need to train our model and no matter the training data, the “learned” function is always <strong>f</strong> and thus there is no variance.</p><h4>High Variance</h4><p>Assume we only have 100 training examples. If we attempt to fit these few data points to a neural network with 10000+ parameters, even the slightest change in the input data is likely to lead to a completely different trained model. Thus, we have high variance.</p><p>It is tempting to say that in this case we automatically have low bias. This is false. Recall that the bias does not depend on data. In fact, we don’t know the complexity of the underlying ground truth function <strong>f</strong> and it may itself have one million parameters. In this case, we would also have high bias.</p><p>The truth is that we were only given very few training examples and even though we don’t know the bias, we know that our total error is the sum of bias and variance. Considering that our goal is to reduce the prediction error, we need to find a model whose complexity matches the low number of training examples. In other words, choosing a very complex model for only 100 data points will always give a high variance and thus make our overall prediction poor. To maximize our chances of a low prediction error, we have to pick a simpler model.</p><h4>Low Bias?</h4><p>Let’s consider an extreme example where our training points are of the form <strong>(k, f(k))</strong> and <strong>f</strong> is a random number generator. If our goal was to have no bias, we would be forced to learn a new value for each training point and thus learn infinitely many parameters. Of course, in this case, the variance would be very high as the algorithm doesn’t learn any pattern and each change in input gives a new model.</p><p>In the above situation, learning is simply not feasible. The point of a (good) random number generator is that no pattern can be learned and thus the prediction error of any model must be high.</p><p>The takeaway is that, since we never know <strong>f</strong>, we need to use our domain knowledge and the data to get an idea of a good model to use. Once, we decided on our model, for example neural network, random forest, linear regression, etc., we can use the training data to fix the free parameters.</p><h3>Practical Implications and Conclusions</h3><p>Despite its abstract nature, the bias-variance tradeoff has important practical implications.</p><p>Just by looking at the equation, we readily conclude that the bias term does <em>not </em>depend on training data. Thus, in particular, collecting more data cannot reduce bias. Instead, the only way to reduce bias is by changing the modeling space <strong>H</strong>.</p><p>In contrast, the variance term depends on data and collecting more data will lead to a closer clustering of learned functions <strong>g</strong>{<strong>D</strong>} about the best possible approximation <strong>ğ</strong> and thus to lower variance. Also, using regularization restricts the fluctuations of each trained function and thus also helps to reduce variance.</p><p>Ultimately, as the prediction error is the sum of bias and variance, a good model needs to strive for both low bias and low variance. Thus, as machine learning practitioners, our goal is to find a modeling space that has the right complexity for both the problem <em>and </em>the data we are given.</p><p><strong><em>Are you interested in transitioning to a career in tech? </em></strong><a href="https://notify.insightdatascience.com/notify?utm_source=bias-variance&amp;utm_medium=blog">Sign up to learn more</a> about the <a href="https://www.insightdatascience.com/?utm_source=bias-variance&amp;utm_medium=blog">Insight Fellows programs</a> and <a href="https://apply.insightdatascience.com/?utm_source=bias-variance&amp;utm_medium=blog">start your application today</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fa2bc28174c4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/insight-data/bias-variance-tradeoff-explained-fa2bc28174c4">Bias-Variance Tradeoff Explained</a> was originally published in <a href="https://medium.com/insight-data">Insight</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to train your own YOLOv3 detector from scratch]]></title>
            <link>https://medium.com/insight-data/how-to-train-your-own-yolov3-detector-from-scratch-224d10e55de2?source=rss-4e398f0112fa------2</link>
            <guid isPermaLink="false">https://medium.com/p/224d10e55de2</guid>
            <category><![CDATA[insight-ai]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[yolo]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Anton Muehlemann]]></dc:creator>
            <pubDate>Fri, 04 Oct 2019 16:18:28 GMT</pubDate>
            <atom:updated>2019-11-18T07:57:21.411Z</atom:updated>
            <content:encoded><![CDATA[<p>This comprehensive and easy three-step tutorial lets you train your own custom object detector using YOLOv3. The only requirement is basic familiarity with Python.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/929/1*H_0MBlwdC9SfM9EZacKCLw.png" /><figcaption>Our input data set are images of cats (without annotations).</figcaption></figure><p>As an example, we learn how to detect faces of cats in cat pictures. Given the omnipresence of cat images on the internet, this is clearly a long-awaited and extremely important feature! But even if you don’t care about cats, by following these exact same steps, you will be able to build a YOLO v3 object detection algorithm for your own use case.</p><h3>Training Data</h3><p>If you already have an image dataset, you are good to go and can proceed to the next step! If you need to create an image dataset first, consider using a Chrome extension such as <a href="https://chrome.google.com/webstore/detail/fatkun-batch-download-ima/nnjjahlikiabnchcpehcpkdeckfgnohf?hl=en">Fatkun Batch Downloader</a> which lets you build your own dataset easily. For instance, if you’d like to detect fidget spinners in images with fidget spinners, do a Google Image search for “fidget spinner” and save the resulting images.</p><p>If you just want to learn more about training YOLO v3, you can also use the cat images already contained in the accompanying GitHub repo.</p><h3>Clone the Repo</h3><p>Before getting started, clone the repo at <strong>github.com/AntonMu/TrainYourOwnYOLO </strong>to your local machine. Make sure to set up a virtual environment and install the requirements.</p><p>This YOLO tutorial is designed to work for Windows, Mac, and Linux operating systems. To make things run smoothly, it is highly recommended to <strong>keep the</strong> <strong>original folder</strong> structure of the cloned GitHub repo. The GitHub repo also contains further details on each of the steps below, as well as lots of cat images to play with.</p><h3>Step 1: Annotate Images</h3><p>In order for our detector to learn to detect objects in images, such as cat faces in pictures, it needs to be fed with labeled training data. In our cat example, this means manually labeling cat faces for pictures located in <strong>TrainYourOwnYOLO/Data/Source_Images/Training_Images</strong>. If you are using your own image dataset, replace the cat images in that folder with your own images. For decent results, label at least 100 objects — the more the better!</p><p>To label images, I recommend using Microsoft’s <a href="https://github.com/microsoft/VoTT">Visual Object Tagging Tool (VoTT)</a> which has release packages for Windows, Mac and Linux available at:</p><blockquote><a href="https://github.com/Microsoft/VoTT/releases">https://github.com/Microsoft/VoTT/releases</a></blockquote><p>For Mac, download and install vott-2.x.x-darwin.dmg; for Windows, download and install vott-2.x.x-win32.exe; and for Linux, download and install vott-2.x.x-linux.snap.</p><p>Once installed, create a <strong>new project</strong> and call it <strong>Annotations</strong>. Then choose <strong>TrainYourOwnYOLO/Data/Source_Images/Training_Images</strong> as the <strong>Source</strong> and <strong>Target Connection</strong>. Under <strong>Export Settings</strong> choose <strong>Comma Separated Values (CSV) </strong>as <strong>Provider</strong> and hit <strong>Save Export Settings</strong>. Now, start labeling cat faces by drawing bounding boxes around cat faces.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fw1-HB5VITXBowEv6tprDQ.gif" /><figcaption>Labeling cat faces with Microsoft’s VoTT.</figcaption></figure><p>Once you have labeled enough cat faces, press <strong>CRTL+E</strong> to export the project. Inside the <strong>TrainYourOwnYOLO/Data/Source_Images/Training_Images</strong>, you should now see a new folder called <strong>vott-csv-export </strong>containing <strong>Annotations-export.csv</strong>.</p><p>Next, navigate to <strong>TrainYourOwnYOLO/1_Image_Annotation</strong> and run the conversion script to convert the annotations to YOLO format.</p><pre>python Convert_to_YOLO_format.py</pre><p>That’s all for image annotation. You are now ready to train your YOLOv3 model.</p><h3>Step 2: Train your YOLOv3 Model</h3><p>Before getting started you need to download the pre-trained dark-net weights and convert them to YOLO format. To do this, navigate to <strong>TrainYourOwnYOLO/2_Training</strong> and run:</p><pre>python Download_and_Convert_YOLO_weights.py</pre><p>Once finished, train the detector by running:</p><pre>python Train_YOLO.py</pre><p>Depending on your set up, this process can take a few minutes to a few hours. I recommend using a GPU to speed up training. The final weights are saved in <strong>TrainYourOwnYOLO/Data/Model_Weights</strong>.<strong> </strong>This concludes the training step and you are now ready to detect objects in new images!</p><h3>Step 3: Try your Detector</h3><p>To test your object detector, navigate to <strong>TrainYourOwnYOLO/3_Inference</strong> and run:</p><pre>python Detector.py</pre><p>This will apply your freshly trained YOLOv3 object detector on test images located in <strong>TrainYourOwnYOLO/Data/Source_Images/Test_Images</strong>. In our example, we detect cat faces in new images of cats. To test the detector on your own images, populate the folder with your own images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4rDs1Pw7Fsxq-svkI5_U8w.png" /><figcaption>Detected cat faces on test images.</figcaption></figure><p>If you made it this far — <strong>congratulations</strong>! You have successfully trained your own YOLOv3 computer vision model!</p><p>To explore more options and to customize your code head over to <a href="http://github.com/AntonMu/TrainYourOwnYOLO">github.com/AntonMu/TrainYourOwnYOLO</a>. All python scripts above have optional command-line arguments that help you adapt the image detector to your use case and to tweak performance. To list command-line options run:</p><pre>python &lt;script_name.py&gt; -h</pre><p>Trying your own project? Leave a comment describing how this story made a difference for you and what you have achieved. If you encounter these pesky bugs, write your question below and I will do my best to answer them.</p><p><strong><em>Are you interested in transitioning to a career in data? </em></strong><a href="https://notify.insightdatascience.com/notify?utm_source=yolov3&amp;utm_medium=blog&amp;source=post_page">Sign up</a> to learn more about the <a href="https://www.insightdatascience.com/?utm_source=yolov3&amp;utm_medium=blog&amp;source=post_page">Insight Fellows</a> programs and start your application today.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=224d10e55de2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/insight-data/how-to-train-your-own-yolov3-detector-from-scratch-224d10e55de2">How to train your own YOLOv3 detector from scratch</a> was originally published in <a href="https://medium.com/insight-data">Insight</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Evolution of the Digital Asset Market in 2019 — Q2 Update]]></title>
            <link>https://medium.com/blockchain-at-berkeley/the-evolution-of-the-digital-asset-market-in-2019-q2-update-fa89e509dc22?source=rss-4e398f0112fa------2</link>
            <guid isPermaLink="false">https://medium.com/p/fa89e509dc22</guid>
            <category><![CDATA[trade]]></category>
            <category><![CDATA[ether]]></category>
            <category><![CDATA[crypto]]></category>
            <category><![CDATA[cryptocurrency]]></category>
            <category><![CDATA[bitcoin]]></category>
            <dc:creator><![CDATA[Anton Muehlemann]]></dc:creator>
            <pubDate>Fri, 26 Jul 2019 03:36:38 GMT</pubDate>
            <atom:updated>2019-07-26T03:51:53.445Z</atom:updated>
            <content:encoded><![CDATA[<h3>The Evolution of the Digital Asset Market in 2019 — Q2 Update</h3><blockquote>Crypto is back and Bitcoin leads the way!</blockquote><p>Slowly but steadily, crypto climbed back towards its old records. Even though prices are still well below the Jan 2018 crypto bubble hype, Q2/19 trading volumes are three times as high as Q1/18!</p><p>Apart from record trading activity, by crunching numbers of 184 digital asset exchanges we identified the following global trends:</p><ul><li>Q2/2019 best quarter for crypto investors since 2017,</li><li>Bitcoin outperforms smaller cryptocurrencies,</li><li>USA’s global market share continues to rise while Korean trading activity slows down,</li><li>Global trading is becoming more diverse and shows less concentration on just a few exchanges.</li></ul><h3>An All Winner Quarter</h3><p>Between April 1 and July 1, <strong>all </strong>top ten most traded digital assets increased in value. The best performing asset was good old Bitcoin with +155% and was closely followed by its largest fork, Bitcoin Cash, which gained +148%. Ether and Litecoin both doubled in value and the ‘worst performers’ were Ripple (XRP) and DASH with a plus of about +30%. Along with the prices volumes also shot up. For instance, Bitcoin Cash and Tether USD were traded almost five times as much as in Q1/19 and most other cryptos more than doubled in volume. The one exception to the rule was DASH which became 20% less popular due to a lack of interest by Korea.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fe.infogram.com%2Fd3f3baba-c98d-4600-9c47-e796921eb47b%3Fsrc%3Dembed&amp;url=https%3A%2F%2Finfogram.com%2Fvolume-and-price-change-1hmr6g3gn9894nl&amp;image=https%3A%2F%2Finfogram-thumbs-1024.s3.amazonaws.com%2F467c608d-2d83-4b74-9fd2-2b495468ac8b.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=infogram" width="550" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/1d92d299de39f6136b9936b49d035fdd/href">https://medium.com/media/1d92d299de39f6136b9936b49d035fdd/href</a></iframe><h4>Bitcoin Yay, Altcoins Meh</h4><p>Unlike Q1/19, were Altcoin portfolios beat just holding Bitcoin, simply holding Bitcoin turned out to be the best strategy in Q2/19. Even when looking at year-to-date performance, Bitcoin still beats holding baskets of altcoins. This trend is a possible reason for the termination of several crypto basket products such as the <a href="https://www.theblockcrypto.com/tiny/coinbase-shuts-down-the-coinbase-bundle-product/">Coinbase bundle</a> and the <a href="https://www.coindesk.com/coinbase-confirms-shutdown-of-crypto-index-fund-product">Coinbase Index Fund</a>.</p><p>To visualize the performance of altcoins, we compared 5 different equal-weighted portfolio strategies. The first strategy invested only in Bitcoin and the remaining strategies invested in the Top 5, 10, 50 or 100 most traded digital assets.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fe.infogram.com%2F8a5d8ad5-d657-4f55-b5ef-bf2c33ab6f6a%3Fsrc%3Dembed&amp;url=https%3A%2F%2Finfogram.com%2Fportfolio-comparison-1h706ejrz0y745y&amp;image=https%3A%2F%2Finfogram-thumbs-1024.s3.amazonaws.com%2Ff3d0a18c-edf9-406f-b4ac-e432fba08cea.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=infogram" width="550" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/c3e15418d1a27a20509815783d7e75b4/href">https://medium.com/media/c3e15418d1a27a20509815783d7e75b4/href</a></iframe><p>In Q2, Bitcoin continued its long outperformance run that started in 2018. This dominance signals that in uncertain market situations, investors seek safety in the biggest, oldest and most well-established cryptocurrency. While in 2017, a <a href="https://medium.com/blockchain-at-berkeley/can-you-still-beat-the-crypto-market-196255da0dc2">Top 100 ‘S***-coin’ portfolios skyrocketed</a>, this time around its performance is sub-par. Since Jan 1, 2019, a Top 100 portfolio only gained +68% whereas Bitcoin is up more than +170%.</p><h3>Global Trading Hits New All-Time High</h3><p>At the peak of the crypto bubble in Q1/18, <a href="https://medium.com/blockchain-at-berkeley/the-evolution-of-the-digital-asset-market-in-2018-9af4c5de176c">$7B worth of crypto was traded worldwide</a> and trading activity has been significantly lower ever since — until now! In Q2/19, global trading crushed the previous record with an average daily trading volume of $14B. For comparison, this is around 3 times the daily trading volume of <a href="https://www.nasdaq.com/symbol/aapl">Apple (APPL) on the Nasdaq</a>.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fe.infogram.com%2Fe66ee58e-624b-411a-a924-c28292690dc2%3Fsrc%3Dembed&amp;url=https%3A%2F%2Finfogram.com%2Fglobal-trading-volume-1h706ejrzmj745y&amp;image=https%3A%2F%2Finfogram-thumbs-1024.s3.amazonaws.com%2F52a00f96-894c-4704-aa5d-0ceaa0231e8b.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=infogram" width="550" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/dc13f2df88a427fc9af3e303944caffd/href">https://medium.com/media/dc13f2df88a427fc9af3e303944caffd/href</a></iframe><h4>USA’s Comeback and Koreas Decline</h4><p>After its <a href="https://blockchainatberkeley.blog/the-evolution-of-the-digital-asset-market-in-2018-9af4c5de176c">steady decline starting in 2017</a>, the US continues its strong year by moving up the ranks to 4th place in global digital asset trading. In Q1/19, $450M worth of crypto changed hands in the US every day, in Q2/19 this number more than doubled to $1.1B/day. In contrast, former crypto powerhouse South Korea fell from rank 3 in Q1/19 to rank 6 in Q2/19. This fall was in spite of an overall increase in trading volume from $700M/day to $800M/day — it just couldn’t keep up with the overall market that more than tripled in the same time period.</p><h4>Increasing Market Diversity</h4><p>Exchange trading still follows a typical exponential distribution but has become more diverse <a href="https://medium.com/blockchain-at-berkeley/the-evolution-of-the-digital-asset-market-in-2019-q1-update-f285f65f7d59">compared to Q1/19</a>. In Q2/19, 90% of global trading is processed at the Top 21 (+5) exchanges and 99% at the Top 49 (+8) exchanges. The highest-ranked US exchange is Bitmart coming in 12th place. The largest individual exchange is OKEX with $1.8B worth of cryptocurrencies traded every day.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fe.infogram.com%2F478d2bd6-7bb3-42ee-8971-dcaf741e141e%3Fsrc%3Dembed&amp;url=https%3A%2F%2Finfogram.com%2Fdistribution-1hnp27jvgxz82gq&amp;image=https%3A%2F%2Finfogram-thumbs-1024.s3.amazonaws.com%2F10f498d6-aff0-4b5a-b852-63e1ff6cd295.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=infogram" width="550" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/76fd982fcc4ed31faf2056ebe7c6b3c9/href">https://medium.com/media/76fd982fcc4ed31faf2056ebe7c6b3c9/href</a></iframe><h3>Korea Remains Adventurous but Loses Influence</h3><p>In previous quarters, Korea proved to be one of the <a href="https://medium.com/blockchain-at-berkeley/the-evolution-of-the-digital-asset-market-in-2019-q1-update-f285f65f7d59">most influential crypto markets</a> capable of driving global prices. This was not at last due to their frequently changing interest in different Altcoins. However, in Q2/19, Korea fell behind in global market share and saw its influence dip as a result. DASH remained extremely popular in Korea but did not impress the rest of the world.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fe.infogram.com%2Fab4b5f58-5f64-484a-8f6f-626efde676fd%3Fsrc%3Dembed&amp;url=https%3A%2F%2Finfogram.com%2Fby-currency-1hnp27jvg1x82gq&amp;image=https%3A%2F%2Finfogram-thumbs-1024.s3.amazonaws.com%2F87000b8f-c830-4c4c-a989-5836bead30d0.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=infogram" width="550" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/26e193bf38e43ab3c09dfe51dbf8aa22/href">https://medium.com/media/26e193bf38e43ab3c09dfe51dbf8aa22/href</a></iframe><p>In all other currencies, Bitcoin was by far the most popular crypto asset and Japan remained true to its alleged Japanese founder and almost exclusively traded Bitcoin for Japanese Jen (apart from some XRP).</p><h3>Conclusions</h3><p>Q2/19 has been a record quarter for the global crypto market. We saw the highest ever recorded global trading activity and Bitcoin stayed above $8000 for the longest time in history.</p><p>These new records are the result of tireless efforts of the crypto community to raise awareness, build better infrastructure and making trading easier and more accessible. After many recent setbacks, it is great to see how these combined efforts come to bear fruits. At last, crypto is on its way to unfold its true potential and make a dent in the global financial market.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fairtable.com%2Fembed%2FshrK1YtPYYG8phQLU&amp;url=https%3A%2F%2Fairtable.com%2FshrK1YtPYYG8phQLU&amp;image=https%3A%2F%2Fstatic.airtable.com%2Fimages%2Foembed%2Fairtable.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=airtable" width="800" height="533" frameborder="0" scrolling="no"><a href="https://medium.com/media/b450ea8660af6319b82dbedf907c8df4/href">https://medium.com/media/b450ea8660af6319b82dbedf907c8df4/href</a></iframe><p><strong><em>Disclaimer:</em></strong><em> Nothing contained in this article constitutes investment, accounting, tax or legal advice or a recommendation to buy, or sell any (alt-)coin or other investment, management product or service or pursue any investment strategy. You should not buy any (alt-) coin unless you are prepared to sustain a total loss of the money you have invested plus any commission or other transaction charges. Moreover, past performance is not indicative of future results.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/979/1*iq8PIxmtAj5nhAMBwK_rAg.png" /><figcaption>Cover Photo</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fa89e509dc22" width="1" height="1" alt=""><hr><p><a href="https://medium.com/blockchain-at-berkeley/the-evolution-of-the-digital-asset-market-in-2019-q2-update-fa89e509dc22">The Evolution of the Digital Asset Market in 2019 — Q2 Update</a> was originally published in <a href="https://medium.com/blockchain-at-berkeley">Blockchain at Berkeley</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Evolution of the Digital Asset Market in 2019 — Q1 Update]]></title>
            <link>https://medium.com/blockchain-at-berkeley/the-evolution-of-the-digital-asset-market-in-2019-q1-update-f285f65f7d59?source=rss-4e398f0112fa------2</link>
            <guid isPermaLink="false">https://medium.com/p/f285f65f7d59</guid>
            <category><![CDATA[ethereum]]></category>
            <category><![CDATA[bitcoin]]></category>
            <category><![CDATA[cryptocurrency]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[coinbase]]></category>
            <dc:creator><![CDATA[Anton Muehlemann]]></dc:creator>
            <pubDate>Tue, 16 Apr 2019 18:35:09 GMT</pubDate>
            <atom:updated>2019-04-25T16:25:33.629Z</atom:updated>
            <content:encoded><![CDATA[<h3>The Evolution of the Digital Asset Market in 2019 — Q1 Update</h3><blockquote>After 2017’s Crypto-Optimism and 2018’s Crypto-Pessism, it is time for Crypto-Realism.</blockquote><p>After a year of lower and lower prices, crypto has made its long-awaited come-back. In Q1/19, both trading volumes and overall digital asset prices increased and show a strong upward trend. Most notably, prices of four of the Top 10 most traded digital assets increased by more 25% — gains which haven’t been seen since 2017.</p><p>Crunching numbers of 192 digital asset exchanges worldwide led to the following key findings:</p><ul><li>Q1/2019 best quarter for crypto since 2017,</li><li>USA’s global market share increased for the first time since 2017 but is still far behind Asian markets,</li><li>Korean trading activity shows the highest correlation to (global) digital asset prices.</li></ul><h3>Best Quarter Since 2017</h3><p>Between Jan 1 and April 1, prices of four of the top ten most traded digital assets saw significant gains. The best performing asset was Litecoin with +89%. Bitcoin saw a moderate gain of 7% whereas Ether and Bitcoin Cash remained almost constant. The only losers were Ripple (XRP) and Ether Classic (ETC) with a decline of 15% and 10%. For the first time, the stable coin Tether (USDT) entered the ranks of the most-traded assets.</p><h4>Altcoin Portfolios Outperform Bitcoin</h4><p>During 2017, investors rushed into altcoins (i.e. digital assets other than Bitcoin) fueling the creating of a crypto bubble. Once the bubble burst, the reverse phenomenon happened and investors sought refuge in the relatively stable Bitcoin or stable coins such as USDT.</p><p>To visualize the performance of altcoins, we compared 5 different equal-weighted portfolio strategies. The first strategy invested only in Bitcoin and the remaining strategies invested in the Top 5, 10, 50 or 100 most traded digital assets.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fe.infogram.com%2F2ea88195-d0f5-4e22-a556-46af9f063217%3Fsrc%3Dembed&amp;url=https%3A%2F%2Finfogram.com%2Fportfolio-performance-1hnq41y9qlwd43z&amp;image=https%3A%2F%2Finfogram-thumbs-1024.s3.amazonaws.com%2F2f39ecfb-aea7-45c4-84c1-0ad743e81829.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=infogram" width="550" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/ed70c94ebe63255428f6c63ed618f0a6/href">https://medium.com/media/ed70c94ebe63255428f6c63ed618f0a6/href</a></iframe><p>While in the past year, Bitcoin outperformed all other strategies in almost every quarter, in Q1/19, Bitcoin’s performance is far behind. The biggest gain was seen by an investment in the Top 5 cryptocurrencies with a gain of 27%. Unlike in 2017, smaller altcoins did not see exorbitant gains signaling that this time, investors are less starry-eyed when it comes to altcoin investments.</p><h3>US Trading Volumes on the Rise</h3><p>After its <a href="https://blockchainatberkeley.blog/the-evolution-of-the-digital-asset-market-in-2018-9af4c5de176c">steady decline starting in 2017</a>, for the first time, the US was able to gain market share. Reaching an all-time low of just 8% global market share ($280M/day) in January, it has rebounded to 10% in March ($650M/day).</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fe.infogram.com%2Fce95fbcd-2fe9-43e2-9cc3-f5eae2602c7d%3Fsrc%3Dembed&amp;url=https%3A%2F%2Finfogram.com%2Fglobal-trading-volume-1h0r6rozjrg86ek&amp;image=https%3A%2F%2Finfogram-thumbs-1024.s3.amazonaws.com%2Ff986b895-894b-4a31-8550-31a387acd456.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=infogram" width="550" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/c070bae3826b06429761142f55f09c2a/href">https://medium.com/media/c070bae3826b06429761142f55f09c2a/href</a></iframe><p>Globally, trading activity increased throughout the quarter from $3.7B/day in January to $6.6B a day in March. As in Q4/18, Asia still dominates the market with a share of more than 80% and (formerly) Chinese exchanges account for more than 25% of it.</p><h4>16 Exchanges Responsible for 90% of Global Trading</h4><p>Breaking down the global ranking by exchanges reveals a typical exponential distribution. The Top 16 exchanges account for more than 90% of global trading and the Top 41 account for more than 99.9% of all assets traded. The highest ranked US exchange is Bitmart coming in on 9th place. The largest individual exchange is OKEX with $600M worth of cryptocurrencies traded per day.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fe.infogram.com%2Fd083e1a6-ec0a-47d6-9d05-6f606b922ad3%3Fsrc%3Dembed&amp;url=https%3A%2F%2Finfogram.com%2Fdistribution-1h0r6rozjyv76ek&amp;image=https%3A%2F%2Finfogram-thumbs-1024.s3.amazonaws.com%2Fd9f7ecd1-7388-43f9-8752-7e306ddb5deb.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=infogram" width="550" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/566ac4c68a28c401b0f94c2d25fe7d58/href">https://medium.com/media/566ac4c68a28c401b0f94c2d25fe7d58/href</a></iframe><h3>Korea Dominates FIAT Market and Drives Prices</h3><p>The vast majority of trades are from digital-to-digital such as e.g. ETH/BTC or BCH/BTC and the resulting trading patterns provide little information about price development. However, if one excludes all virtual markets (/USDT, /BTC and /ETH) a more insightful pattern emerges.</p><p>Even though Korea only accounts for <a href="https://infogram.com/fiat-trading-volumes-1hxj4803k1yr6vg?live">15% of global trading, it accounts for the majority of fiat trades</a>. The record holds DASH with almost 100% of all fiat trades denoted in Korean Won (₩). At the end of last year, also 97% of all ETC/fiat and 90% of all LTC/fiat trades took place in Korea.</p><blockquote>Unlike the rest of the world, Korean’s like to use ‘real’-money to buy virtual money.</blockquote><p>So does ‘real’ money drive ‘real’ prices?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ow-MXWPfbj5ixvXpYFIwzQ.png" /><figcaption>Weekly correlation between cryptocurrency prices and their volumes in different fiat markets. Comparison is from Oct 1, 2018 to April 1, 2019. Blanks signify the absence of a particular trading pair, e.g. there is no XRP/GBP or LTC/JPY market. *Note that all volume is calculated in ‘Coins’ traded rather than fiat value traded.</figcaption></figure><p>To find out, we determined the weekly correlation between the prices of the Top 10 (excluding USDT) most traded digital assets vs their trading volume (in coins) in different fiat markets. The results are impressive. Korean trading activity is significantly more correlated to prices than any other fiat market. Perhaps surprisingly, DASH’s price doesn’t seem to be highly correlated to Korean trading. However, there is a simple reason, DASH only had negligible trading volume in Korea in October and November 2018 and has become increasingly popular ever since. In fact, if the pattern continues, DASH is probably going to be the next crypto to moon because of its introduction to Korean markets.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fe.infogram.com%2Fd6159072-04ca-4ba0-8f0a-cea19703bd7a%3Fsrc%3Dembed&amp;url=https%3A%2F%2Finfogram.com%2Fkrw-trading-1h984ww0pwkg4p3&amp;image=https%3A%2F%2Finfogram-thumbs-1024.s3.amazonaws.com%2F5d72771b-1285-4f82-ad08-bbd9aa450e10.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=infogram" width="700" height="1018" frameborder="0" scrolling="no"><a href="https://medium.com/media/be05792221781e4a65b00b01541c6b18/href">https://medium.com/media/be05792221781e4a65b00b01541c6b18/href</a></iframe><h3>Conclusions</h3><p>For the first time in more than a year, the crypto market is seeing a general upturn. The <a href="https://blockchainatberkeley.blog/the-evolution-of-the-digital-asset-market-in-2018-9af4c5de176c">fundamental improvements</a> of the crypto ecosystem in 2018 such as Google’s and Facebook’s reversal of their crypto bans, more legal clarity and interest from endowments and banks, seem to finally bear fruits.</p><p>If 2017 was the year of excessive crypto optimism, then 2018 was the year of excessive crypto pessimism. In 2019, crypto seems to finally have found its rightful place. It is an innovative technology with high potential that will only be successful through long hard work and through small steps in the right direction. No matter how ingenious the idea, there is no overnight success — welcome to adulthood crypto!</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fairtable.com%2Fembed%2Fshr5TMkHhIelgymG4&amp;url=https%3A%2F%2Fairtable.com%2Fshr5TMkHhIelgymG4&amp;image=https%3A%2F%2Fstatic.airtable.com%2Fimages%2Foembed%2Fairtable.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=airtable" width="800" height="533" frameborder="0" scrolling="no"><a href="https://medium.com/media/a7dcb560e7535bdc570abe10ebc4043a/href">https://medium.com/media/a7dcb560e7535bdc570abe10ebc4043a/href</a></iframe><p><strong><em>Disclaimer:</em></strong><em> Nothing contained in this article constitutes investment, accounting, tax or legal advice or a recommendation to buy, or sell any (alt-)coin or other investment, management product or service or pursue any investment strategy. You should not buy any (alt-) coin unless you are prepared to sustain a total loss of the money you have invested plus any commission or other transaction charges. Moreover, past performance is not indicative of future results.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XQLnrk5PqmUvjTszVcKwvg.png" /><figcaption>Cover Photo</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f285f65f7d59" width="1" height="1" alt=""><hr><p><a href="https://medium.com/blockchain-at-berkeley/the-evolution-of-the-digital-asset-market-in-2019-q1-update-f285f65f7d59">The Evolution of the Digital Asset Market in 2019 — Q1 Update</a> was originally published in <a href="https://medium.com/blockchain-at-berkeley">Blockchain at Berkeley</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Evolution of the Digital Asset Market in 2018]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/blockchain-at-berkeley/the-evolution-of-the-digital-asset-market-in-2018-9af4c5de176c?source=rss-4e398f0112fa------2"><img src="https://cdn-images-1.medium.com/max/1459/1*6_ewqHJjDCwaY1TjGcv4CA.png" width="1459"></a></p><p class="medium-feed-snippet">When Moon and when Lambo? Not in 2018!</p><p class="medium-feed-link"><a href="https://medium.com/blockchain-at-berkeley/the-evolution-of-the-digital-asset-market-in-2018-9af4c5de176c?source=rss-4e398f0112fa------2">Continue reading on Blockchain at Berkeley »</a></p></div>]]></description>
            <link>https://medium.com/blockchain-at-berkeley/the-evolution-of-the-digital-asset-market-in-2018-9af4c5de176c?source=rss-4e398f0112fa------2</link>
            <guid isPermaLink="false">https://medium.com/p/9af4c5de176c</guid>
            <category><![CDATA[ethereum]]></category>
            <category><![CDATA[eos]]></category>
            <category><![CDATA[bitcoin]]></category>
            <category><![CDATA[coinbase]]></category>
            <category><![CDATA[cryptocurrency]]></category>
            <dc:creator><![CDATA[Anton Muehlemann]]></dc:creator>
            <pubDate>Mon, 21 Jan 2019 19:01:00 GMT</pubDate>
            <atom:updated>2019-02-18T21:31:07.133Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[The Bitcoin Cash Hash War: ABC vs SV]]></title>
            <link>https://medium.com/ironchain-capital/the-bitcoin-cash-hash-war-abc-vs-sv-afef73a569dc?source=rss-4e398f0112fa------2</link>
            <guid isPermaLink="false">https://medium.com/p/afef73a569dc</guid>
            <category><![CDATA[war]]></category>
            <category><![CDATA[cryptocurrency]]></category>
            <category><![CDATA[bitcoincash]]></category>
            <category><![CDATA[bitcoin]]></category>
            <dc:creator><![CDATA[Anton Muehlemann]]></dc:creator>
            <pubDate>Mon, 19 Nov 2018 22:58:05 GMT</pubDate>
            <atom:updated>2018-11-20T08:21:56.443Z</atom:updated>
            <content:encoded><![CDATA[<p>Last week’s contentious Bitcoin Cash hard fork was the likely reason for one of the biggest sell-offs in the cryptocurrency market in the past year. In this short article, we recap what happened and where we are right now.</p><p><strong>What is A Hard Fork?</strong></p><p>In simple terms, a hard fork of a digital asset network happens whenever the underlying protocol is changed in a way that makes it incompatible with the existing protocol. In this case, any miner that runs the original software will no longer accept blocks created by miners running the “updated” mining software and vice versa. Hence, from this point onwards, the blockchain branches into two distinct blockchains that produce blocks according to their own rules.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*j39VHgk_eLgb5DU8" /><figcaption>The evolution of Bitcoin Cash forks. © AM</figcaption></figure><p>If the vast majority of miners switch to the updated protocol (or stay with the original protocol) the other branch will eventually be abandoned and the main protocol continues under the new rules (or under the original rules).</p><p>Bitcoin Cash (BCH) was itself a hard fork that split away from Bitcoin in August 2017 in an attempt to increase transaction throughput by increasing the size of each block. In May 2018, BCH underwent a scheduled (non-contentious) network upgrade which increased the block size even further. Since this update was accepted by all miners, the network did not fork.</p><p><strong>What are BCH ABC and SV?</strong></p><p>Bitcoin ABC stands for Adjustable Blocksize Cap and is seen by many as the continuation of the original BCH protocol. The main points are to keep the current blocksize of 32MB and to introduce certain code that would allow the blockchain to access external data. Bitcoin SV stands for Satoshi’s Vision and aims to increase the block size to 128MB. It also wants to re-enable some code proposed in the original whitepaper that was later disabled due to security concerns. Furthermore, Bitcoin SV plans to remove the cap on the number of instructions that can be executed in one go. More technical details can be found <a href="https://cryptoslate.com/explained-bitcoin-cash-hash-wars-where-the-factions-stand-on-the-issues/">here</a>.</p><p><strong>What is the Controversy?</strong></p><p>In short, there is no real controversy and the war seems mostly politically motivated. In fact, there is a third proposal, called Bitcoin Unlimited, that would incorporate features from both organizations. However, it seems that reaching a compromise is not in the interest of either party. Instead, both parties openly announced that they fight until one chain dies.</p><p><strong>The Fight to Death</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*MJOOH7HuH-7YILUS" /><figcaption>Source: <a href="https://cash.coin.dance/blocks/hashrate">Coin.dance</a></figcaption></figure><p>The main problem of the November 15 hard fork is that both <a href="https://cash.coin.dance/">camps are relatively evenly matched</a> and thus neither side is willing to back down. BCH ABC is supported by Bitmain, the largest BCH mining manufacturer in the world, and BCH SV is supported by CoinGeek, one of the world’s largest BCH mining pools. Craig Wright, <a href="https://hackernoon.com/nchain-in-the-brain-return-of-the-fake-satoshi-1415a8b07f04">a.k.a. ‘Fake-Satoshi’</a>, the driving force behind BCH SV, announced on Twitter to engage in “Continuous competition until one dies as we do not stop.” Jihan Wu, co-founder of Bitmain, responded with:</p><blockquote>“I have no intention to start a hash war with [Craig Wright], because if I do […] BTC price will dump below yearly support […] But since [Craig Wright] is relentless, I am all in to fight till death!”.</blockquote><p><strong>No Truce Without Replay Protection</strong></p><p>What makes the Nov 15 fork particularly dire is the lack of replay protection. Since both camps claim to be the true and only successor of Bitcoin Cash (or even Bitcoin), both camps refused to implement a replay protection in their protocol update. Without replay protection, any valid transaction on one chain is also a valid transaction on the other chain. This can be used to perform a so-called replay attack. In a replay attack, a receiver asks the sender to send them BCH SV coins. The sender then uses their private key to sign the message and publish the signed transaction. Since both chains use the same transaction format, the sender can then take this signed transaction and include it also in the BCH ABC blockchain to receive an equal amount of BCH ABC coins.</p><p><strong>Conclusions</strong></p><p>The BCH hard fork is the first ‘real’ war between two big players in the blockchain space. Even though the underlying changes are relatively minor and easy to reach a compromise on, neither party is willing to end the war in the interest of the rest of the blockchain world. Like any real war, this does not come cheaply. It is estimated that both sides currently <a href="https://twitter.com/BitMEXResearch/status/1063407625805664256">burn between $300,000 to $400,000</a> a day to fuel the hash war. Additionally, just as in the real world, the conflict spread to the rest of the blockchain economy and wiped out almost $40 billion of the total digital asset market.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=afef73a569dc" width="1" height="1" alt=""><hr><p><a href="https://medium.com/ironchain-capital/the-bitcoin-cash-hash-war-abc-vs-sv-afef73a569dc">The Bitcoin Cash Hash War: ABC vs SV</a> was originally published in <a href="https://medium.com/ironchain-capital">IronChain</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Volatility in Digital Asset Market Reaches New All-Time Low]]></title>
            <link>https://medium.com/mithrilx/volatility-in-digital-asset-market-reaches-new-all-time-low-9be89ff77007?source=rss-4e398f0112fa------2</link>
            <guid isPermaLink="false">https://medium.com/p/9be89ff77007</guid>
            <category><![CDATA[volatility]]></category>
            <category><![CDATA[dow-jones]]></category>
            <category><![CDATA[ethereum]]></category>
            <category><![CDATA[bitcoin]]></category>
            <category><![CDATA[digital-asset]]></category>
            <dc:creator><![CDATA[Anton Muehlemann]]></dc:creator>
            <pubDate>Mon, 29 Oct 2018 19:19:01 GMT</pubDate>
            <atom:updated>2018-10-30T02:07:59.872Z</atom:updated>
            <content:encoded><![CDATA[<iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fe.infogram.com%2F52b0a188-aaea-4efc-8352-14ff23de99c3%3Fsrc%3Dembed&amp;url=https%3A%2F%2Finfogram.com%2Fvolatility-in-digital-asset-market-reaches-new-all-time-low-1hxj488yyxnr4vg&amp;image=https%3A%2F%2Finfogram-thumbs-1024.s3.amazonaws.com%2F53578cd7-4128-4838-b46f-beeb7cc183a5.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=infogram" width="698" height="1192" frameborder="0" scrolling="no"><a href="https://medium.com/media/8781fa74567bf18e572af7b617d4a6f5/href">https://medium.com/media/8781fa74567bf18e572af7b617d4a6f5/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lwqzgUp9oAQMfIkgE-OYrg.png" /><figcaption>Cover Photo</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9be89ff77007" width="1" height="1" alt=""><hr><p><a href="https://medium.com/mithrilx/volatility-in-digital-asset-market-reaches-new-all-time-low-9be89ff77007">Volatility in Digital Asset Market Reaches New All-Time Low</a> was originally published in <a href="https://medium.com/mithrilx">MithrilX</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Digital Asset Markets — Q3 Update]]></title>
            <link>https://medium.com/mithrilx/digital-asset-markets-q3-update-d71ab12509b4?source=rss-4e398f0112fa------2</link>
            <guid isPermaLink="false">https://medium.com/p/d71ab12509b4</guid>
            <category><![CDATA[digital-asset]]></category>
            <category><![CDATA[bitcoin]]></category>
            <category><![CDATA[eos]]></category>
            <category><![CDATA[ripple]]></category>
            <category><![CDATA[cryptocurrency]]></category>
            <dc:creator><![CDATA[Anton Muehlemann]]></dc:creator>
            <pubDate>Sun, 14 Oct 2018 17:11:36 GMT</pubDate>
            <atom:updated>2018-10-14T17:11:36.991Z</atom:updated>
            <content:encoded><![CDATA[<h3>Digital Asset Markets — Q3 Update</h3><blockquote>The wait continues ….</blockquote><p>To put it short — Q3 left a lot to be desired. Both trading volumes and digital asset prices continued their 2018 downward trend. Overall, trading volumes are now only half as high as in Q1. Nonetheless, they are still almost twice as high as in Q3/2017.</p><p>Crunching numbers of 159 digital asset exchanges* worldwide led to the following key findings:</p><ul><li>Altcoin prices decreased significantly while Bitcoin saw a small gain of +4%,</li><li>Global trading volumes dropped by 20% from Q2 to Q3; trading in the US is down by over 40%,</li><li>ETH saw the sharpest price decline (-49%) and EOS almost halved its trading volume,</li><li>USA’s global market share dropped below 10% and is now level with the UK; Asia now makes up 77% of the global digital asset market,</li><li>Volatilities reached new all-time lows but remain high compared to classical markets,</li><li>Tether USD (USDT) trading volume now twice as high as that of USD.</li></ul><h3>US Trading Volume Down 43%</h3><p>On a global level, digital asset trading volume dropped by almost 20% (measured across USD, EUR, KRW, JPY, GBP, RUB, BTC, ETH and USDT markets) from Q2 to Q3 2018. Nonetheless, (formerly) Chinese based exchanges (+8%) and Japanese exchanges (+12%) saw an increase in trading volume. In contrast, digital asset trading dropped by 58% in South Korea and by 33% in Hong Kong. The US (-43%) was able to hold on to its third place internationally but the gap to Hong Kong and China widened. The US market is now only marginally bigger than that of the UK which saw a 5% decline in trading activity.</p><p>Compared to last year (Q3/2017), trading in the US has decreased by 55% whereas global trading volume has almost doubled (+90%). The decline of USA’s importance in global digital asset trading is worrying given that the US has a long history of leading tech revolutions and is also home to the biggest classical markets.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fe.infogram.com%2F344e268c-6db8-4155-a1cc-ecbf02a879b0%3Fsrc%3Dembed&amp;url=https%3A%2F%2Finfogram.com%2Fglobal-trading-volume-1h8j4xxzw3zd4mv&amp;image=https%3A%2F%2Finfogram-thumbs-1024.s3.amazonaws.com%2F7f7996c6-8e17-4627-9477-f9974c89817c.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=infogram" width="550" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/c90127b281308573183f6212bf6af5c5/href">https://medium.com/media/c90127b281308573183f6212bf6af5c5/href</a></iframe><h3>Prices and Volumes Keep Falling</h3><p>Between July 1 and October 1, prices of eight out of the top ten digital assets decreased significantly (ETH -49%, EOS -29%, BCH -28%, LTC -25%, ETC -30%, IOTA -48%, NEO -40%, ZEC -25%) and only Bitcoin (+4%) and XRP (+25%) were able to gain in value.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fe.infogram.com%2F268500ea-e360-429c-8635-2ce2f5a80a5b%3Fsrc%3Dembed&amp;url=https%3A%2F%2Finfogram.com%2Fquarterly-price-and-volume-change-1h0r6rkqo0km4ek&amp;image=https%3A%2F%2Finfogram-thumbs-1024.s3.amazonaws.com%2F4d96d627-2246-4b9d-8292-7196b7a3552e.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=infogram" width="903" height="588" frameborder="0" scrolling="no"><a href="https://medium.com/media/413fc8b42b757a5b5fd9af9f13d3019e/href">https://medium.com/media/413fc8b42b757a5b5fd9af9f13d3019e/href</a></iframe><p>When looking at trading volumes, a similar picture emerges. The biggest loser was EOS with a loss of 44% in trading volume. The <a href="https://blog.coinbase.com/final-testing-for-etc-support-on-coinbase-cdccaedb2ae3">listing of ETC on Coinbase</a> seems to have resulted in increased activity and a boost in trading volumes by almost 20%. Zcash (+89%) saw a spike in trading interest following the release of the first major upgrades (<a href="https://z.cash/support/schedule/">Overwinter and Sapling</a>) since its original release in October 2016.</p><h3>Volatilities Reach All-Time Lows</h3><p>Q3/2018 saw the lowest volatilities for almost all of the Top 10 digital assets. The daily volatility for Bitcoin dropped below 3% for the first time since Q1/2017. As expected, the older and more established digital assets show a lower volatility than the younger protocols.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fe.infogram.com%2F533946d4-ccbb-4759-9cf0-85ccdf5e946a%3Fsrc%3Dembed&amp;url=https%3A%2F%2Finfogram.com%2Fquarterly-volatility-1h17498j0vkg2zj&amp;image=https%3A%2F%2Finfogram-thumbs-1024.s3.amazonaws.com%2F251a37c0-2dc7-43ce-9fde-d49e59b35020.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=infogram" width="550" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/ae23096abdfb7b9337d6da7720021d91/href">https://medium.com/media/ae23096abdfb7b9337d6da7720021d91/href</a></iframe><p>It is striking how sharply the volatility of EOS and BCH dropped every quarter since their original launch in Q3/2017. Whereas both new protocols started off with more than 15% daily volatility in their first quarter, this number dropped below 12% in the next and then continued to decline to the current levels around 6%. To put these numbers in perspective, the S&amp;P 500&#39;s daily volatility usually hovers around 0.6% with a peak of around 1.25% in Q1/2018. Thus, even though the volatility of digital assets is constantly decreasing, they are still a far cry from classical markets.</p><h3>Coin Volumes Are Slowly Recovering</h3><p>If one considers the <a href="https://infogram.com/price-vs-volume-1h984w8gwxk84p3">volume of coins traded</a> (rather than their value in USD), we can see a general upward trend in Q3. It is perhaps little surprise that ETH, XRP, and BCH saw some of the largest gains in coins traded due to their price decline. The chart is available <a href="https://infogram.com/price-vs-volume-1h984w8gwxk84p3">here</a>.</p><p>The story becomes more interesting if one only looks at the traded volume in Korea. Just as in the Q1 and Q2/2018, Korea is radically different from the rest of the world. Whereas globally the number of coins traded seems completely uncorrelated to the price development, in Korea, a stronger correlation can be observed. Particularly the prices of EOS and XRP seem to be heavily influenced by their un/popularity in Korea. As in the previous quarters, in any other market Bitcoin is by far the most traded digital asset. In Korea however, the appetite to gamble with new coins is unparalleled and depending on the time of the year, EOS, XRP or BCH can easily overtake Bitcoin in terms of popularity.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fe.infogram.com%2Fd1bbe0ac-d486-48fe-852c-29bca06b5e3b%3Fsrc%3Dembed&amp;url=https%3A%2F%2Finfogram.com%2Fprice-vs-volume-in-krw-1hxj488dopy74vg&amp;image=https%3A%2F%2Finfogram-thumbs-1024.s3.amazonaws.com%2Fd619a1b4-fa00-45b8-ac27-0a7badb8a04c.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=infogram" width="740" height="1271" frameborder="0" scrolling="no"><a href="https://medium.com/media/ff44ae03c78b1ded174196f94023eade/href">https://medium.com/media/ff44ae03c78b1ded174196f94023eade/href</a></iframe><h3>Korea Sets the Tone Again</h3><blockquote>“If you want to know what’s going on in crypto — keep looking at South Korea.”</blockquote><p>Just as in Q1 and Q2/2018, trading activity in Korean Won (₩) varies greatly from month to month and a decrease in popularity in Korea seems to trigger a world-wide sell-off (or vice versa). This variability is particularly striking when compared to USD and USDT markets that seem completely unaffected by price changes and trading volume is in almost perfect alignment with the market capitalization of the coin.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fe.infogram.com%2F708bbf42-b473-4093-85ab-26eccdaddf3e%3Fsrc%3Dembed&amp;url=https%3A%2F%2Finfogram.com%2Fby-currency-1hxj488j03074vg&amp;image=https%3A%2F%2Finfogram-thumbs-1024.s3.amazonaws.com%2F16dedf7b-9856-4adb-8438-c320b5ccde7e.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=infogram" width="550" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/4f5ed3b2e2a36bfadacf840f8da9dba8/href">https://medium.com/media/4f5ed3b2e2a36bfadacf840f8da9dba8/href</a></iframe><p>One of the few exceptions from that rule is the ZCash/Ether (ZEC/ETH) market. Trading volume almost quadrupled every month since April 2018. Whereas in April only 1000 ETH (Ξ) were exchanged for ZEC per day, this value jumped to almost 300,000 ETH (Ξ) in September 2018. A plausible interpretation is that only tech-savvy investors use ETH to buy other digital assets and those tech-savvy investors are also more likely to know about the recent <a href="https://z.cash/support/schedule/">ZCash protocol upgrades</a>.</p><h3><strong>Tether USD on the Rise</strong></h3><p>In Q2, we first saw USDT (Tether USD) trading overtake that of USD. This trend intensified in Q3/2018 and in August and September 2018, USDT was almost twice as popular as USD.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fe.infogram.com%2F62f62340-2d98-4d9f-8d74-b0663da7dc6c%3Fsrc%3Dembed&amp;url=https%3A%2F%2Finfogram.com%2Fusd-vs-usdt-1hxj488zekzd4vg&amp;image=https%3A%2F%2Finfogram-thumbs-1024.s3.amazonaws.com%2Fa5be679a-f205-4bcd-833c-911cd1d73354.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=infogram" width="550" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/55c6730bb1c24fb285cfe951d324a774/href">https://medium.com/media/55c6730bb1c24fb285cfe951d324a774/href</a></iframe><h3>Conclusions</h3><p>Based on the collected quantitive price and volume data, Q3/2018 failed to impress. Even though trading volumes are still about twice as high as one year ago, the continued declined from Q1/2018 is worrisome.</p><p>However, on a fundamental level, the future looks brighter. Q3/2018 saw several improvements of the digital asset ecosystem such as the offering of new custody solutions by Kingdom Trust, Coinbase, BitGo and Gemini, the (partial) reversal of bans of crypto-related advertising on Facebook and Google, and the entering of university endowments (Yale, Harvard, MIT and Stanford) in the digital asset space.</p><p>In hindsight, it is clear that one year ago, neither the legal nor the technological infrastructure was ready for the storm and the hype was almost impossible to sustain. As bad as the inevitable fall may have been for many investors, it was a necessary step to gain enough attention from government, industry, media and academia to bring this technological revolution on track to becoming a story of success.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fairtable.com%2Fembed%2FshrMwpbWbKWM025lf&amp;url=https%3A%2F%2Fairtable.com%2FshrMwpbWbKWM025lf&amp;image=https%3A%2F%2Fstatic.airtable.com%2Fimages%2Foembed%2Fairtable.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=airtable" width="800" height="533" frameborder="0" scrolling="no"><a href="https://medium.com/media/ba2468c2d8f2bdf11fda8720faa51efa/href">https://medium.com/media/ba2468c2d8f2bdf11fda8720faa51efa/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9_VNogdC9oMJ2-EV7TXj9Q.png" /><figcaption>Cover Photo: Trading activity of the top ten most traded digital assets in Korean Won (₩).</figcaption></figure><h3>*Data Acquisition and Data Analysis</h3><p>All data has been obtained from <a href="https://www.cryptocompare.com/">cryptocompare</a>, saved as an sqlite database, queried with <a href="https://pandas.pydata.org/">pandas</a> and visualized with <a href="https://infogram.com/">infogram</a>. All charts are interactive and can be customized by (un-)selecting items. When hovering over data points additional information is displayed. Selecting different tabs shows additional data sets. We excluded exchanges that reported unrealistic values.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d71ab12509b4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/mithrilx/digital-asset-markets-q3-update-d71ab12509b4">Digital Asset Markets — Q3 Update</a> was originally published in <a href="https://medium.com/mithrilx">MithrilX</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>