<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Matthew R Finch on Medium]]></title>
        <description><![CDATA[Stories by Matthew R Finch on Medium]]></description>
        <link>https://medium.com/@FinchMF?source=rss-f583b68d1c6c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*op79WfMamZcc4hXf-pFpGQ.jpeg</url>
            <title>Stories by Matthew R Finch on Medium</title>
            <link>https://medium.com/@FinchMF?source=rss-f583b68d1c6c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 02:39:47 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@FinchMF/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Engineering a MultiLayer Perceptron]]></title>
            <link>https://medium.com/data-science/engineering-a-multilayer-perceptron-8a177b60e6cf?source=rss-f583b68d1c6c------2</link>
            <guid isPermaLink="false">https://medium.com/p/8a177b60e6cf</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Matthew R Finch]]></dc:creator>
            <pubDate>Mon, 27 Apr 2020 02:58:52 GMT</pubDate>
            <atom:updated>2020-04-27T03:09:10.235Z</atom:updated>
            <content:encoded><![CDATA[<h4>A brief review of neural mathematics in Numpy</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-XkdxmRcRazPLaP5n8L7vA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0iNG-3gqglj1ZpdjNGvF_A.jpeg" /><figcaption>drawings by author</figcaption></figure><p>In this article, I will <em>briefly </em>outline the mathematical developments that lead from the single layer perceptron depicted in the image on the left to becoming a multilayer structure, depicted in the image on the right; now known as a <em>neural net: </em>the foundational unit in modern artificial intelligence.</p><p>For an in depth look at the history and social impact of the perceptron’s development, read <a href="https://tripleampersand.org/kernelled-connections-perceptron-diagram/">Kernelled Connections: The Perceptron as Diagram</a> by Michael Eby.</p><p>Theano, TensorFlow, Keras, PyTorch — Each offer resources that allow deep learning to be more accessible. I spend a lot of time working with these modules. However, sometimes I lose track of what is really happening mathematically. Each library has its own syntax. I find the lens of each syntax shapes my conceptualization of how an architecture handles a signal through functions, such as: forward and backwards propagation; or how an activation is being applied. Like any utility library, there are many elements neatly packaged and stored under the hood. Let’s unpack a few of those tools.</p><p>Using Keras and a toy dataset as a baseline.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f6d23930c9849aa3f4386d83a92c6882/href">https://medium.com/media/f6d23930c9849aa3f4386d83a92c6882/href</a></iframe><p>Perhaps I’m jaded, but that was too easy. It may just be an exercise to get a model up and running, but you should be exercising more than just your typing speed when engineering a model. Below is a notebook where I obtained the same dataset, raw, and was able to produce a better score, more insight from the data, as well as engineer custom functionality to a custom multilayer perceptron built from Numpy.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/48cf698defc94b24df7260c5b45a6d51/href">https://medium.com/media/48cf698defc94b24df7260c5b45a6d51/href</a></iframe><h3>Taking a Step Back</h3><p>How do we get here? How do we take something like a perceptron as a simple logical operator and develop it into a deep learning model?</p><h4>What is a Perceptron?</h4><p>A perceptron is a linear, binary classifier. It finds the most articulate boundary between two classes. To instantiate a more intuitive description, a perceptron is able to perceive the difference between two given classes and detect what class a given data point belongs to. It is able to perceive representative space that separates one data point from another, linearly.</p><h4>As Logical Operators</h4><p>Logical operators are functions that control signal flow. There are three main logical operators that result in binary output.</p><ol><li>AND</li><li>OR</li><li>NOT</li></ol><p>In programming, a logical operator parses a signal of multiple inputs into a resultant singular output after being processed through a linear activation. This is particularly useful if your data is a point on a chart; say (3,4) on a graph. (3,4) is an identity, it is a location. The perceptron can evaluate the location of that point and distinguish from what class that identity is part of. Again, assuming that the class properties are divided linearly.</p><h3>How logical Operators Work</h3><h4><strong>AND</strong></h4><p>AND is the Truth operator. AND is only True if both inputs are True. In other words, an AND operator will only produce 1 if both A and B are 1. If A =1 and B =0, then the result = 0. This means, out of the four possible outcomes between 1 and 0, AND is only True once: when both points are 1.</p><h4>OR</h4><p>OR is the opposite of AND. OR is always True unless both operands are 0. This means A = 1 and B = 0 is True. Additionally, A = 0 and B = 1 is True. A = 1 and B = 1 is in thrice, True! Only A = 0 and B = 0 is False.</p><p>Whereas AND is only True 25% of the time, OR is True 75% of the time.</p><h4>NOT</h4><p>NOT is simply a contrarian. If A and B = 1 then NOT = 0; and if A and B = 0 then NOT = 1.</p><p><em>*there are direct examples of each operator in the next notebook</em></p><p>The perceptron was the first algorithm and instance of hardware that was developed modeling biological, neurological functionality. However, it was widely realized, roughly 20 years later, in the 80’s, that the perceptron grossly lacked in interpreting any data not tailored to its linear receptors. Reality is dimensional and non linear.</p><blockquote>Additionally, Perceptrons argues, correctly, that a single-layered perceptron is unable to classify non-linear patterns; its classificatory capacities are limited to patterns that are linearly separable. In Euclidean geometry, linear separability refers to the clustering of two sets of data into A and B regions. This is most easily visualized with a two-dimensional plane. If a line or decision surface is able to neatly divide the data into A and B, the pattern is considered linearly separable. If data distribution is not so clearly delimited, the pattern is considered non-linear.</blockquote><p><em>Michael Eby</em></p><h4><strong>Step to Sigmoid</strong></h4><p>Another important idea with the development of the perceptron: there is not only 0 and 1, but an infinity of continuous possibilities between 0 and 1. Through sigmoid, and the recognition of sigmoid’s derivative, a perceptron is able to generate the likelihood of a class, as well as update the perceptron’s confidence in the prediction of likelihood.</p><p>The notebook below gives examples of logical operators and walks through sigmoid implementation. Additionally, it shows how sigmoid’s derivative is used in backpropagation.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/70ec2f2dc39fda0fbea1ef0f8514ce4b/href">https://medium.com/media/70ec2f2dc39fda0fbea1ef0f8514ce4b/href</a></iframe><p>Let’s look more closely at the process of gradient descent using the functions from the above notebook.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f0e539b625799041b217bad8eb47928c/href">https://medium.com/media/f0e539b625799041b217bad8eb47928c/href</a></iframe><p>Now let’s look more closely at the architecture of SENTI_NET, <em>the sentiment classifying multilayered perceptron</em>.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ec2d020cade70fcd0c8deb4d3f75d574/href">https://medium.com/media/ec2d020cade70fcd0c8deb4d3f75d574/href</a></iframe><p>We can see that Senit_Net is not only an MLP, but also has a text preprocessing unit in it. While this can be done externally, I wanted to build the model to specifically deal with text. Senti_Net is hardcoded to only discern between positive and negative sentiment, but can be trained on any labeled text data. We can see at the end of the notebook utilizing a trained Senti_Net, after passing through text data from political tweets that it had never seen, it was able to accurately classify the sentiment.</p><p>After I built Senti_Net, I wanted to see if I could make another binary classifier, but rather than classify the sentiment as<em> ‘positive’</em> or <em>‘negative’</em>, I tasked the MLP to classify whether a tweet was from a democrat or republican. Results can be found in the notebook below.</p><p><em>*The twitter data is comprised of tweets from:</em></p><ul><li>Joe Biden</li><li>Barack Obama</li><li>Bernie Sanders</li><li>Hillary Clinton</li><li>Mike Pence (both @Mike_Pence and @VP handles)</li><li>Donald Trump</li><li>Sean Hannity</li></ul><p><em>* from each account - starting from Tuesday April 21st- chronologically looking back, 3000 tweets were extracted</em></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c99794dba0f636fa4cbee00be311fb66/href">https://medium.com/media/c99794dba0f636fa4cbee00be311fb66/href</a></iframe><p>Tw_Net performs pretty well! Add these two models together and you could explore the sentiment of each party’s tweets.</p><p>The point of this exercise was to inspire a curiosity as to what is occurring underneath the hood in deep learning libraries. We went through a simple MLP, but now ask your self how a CNN, or RNN is mathematically performing. What is the process replicating that makes it <em>intelligent</em>? For example, the mathematics of an MLP is replicating the process of recognition. The ability to perceive between <em>THIS</em> and <em>THAT</em>. AI may seem otherworldly, or magical, at times, but when paid close attention to, AI reveals not only the beauty in mathematics, but the beauty in how humans have used mathematics as a communicative tool expressing internal processes of our minds.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8a177b60e6cf" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/engineering-a-multilayer-perceptron-8a177b60e6cf">Engineering a MultiLayer Perceptron</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Constructing, Manipulating, Classifying and Generating Audio with Digital Signal Processing and…]]></title>
            <link>https://medium.com/data-science/constructing-manipulating-classifying-and-generating-audio-with-digital-signal-processing-and-2c5a252dbab9?source=rss-f583b68d1c6c------2</link>
            <guid isPermaLink="false">https://medium.com/p/2c5a252dbab9</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[music]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[philosophy]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Matthew R Finch]]></dc:creator>
            <pubDate>Fri, 17 Apr 2020 20:56:03 GMT</pubDate>
            <atom:updated>2020-04-28T03:47:16.358Z</atom:updated>
            <content:encoded><![CDATA[<h3>Constructing, Manipulating, Classifying and Generating Audio with Digital Signal Processing and Machine Learning</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pEQzuHcxEUcGYBkLg5BpLg.png" /><figcaption>image by author</figcaption></figure><blockquote>“We live in a world where there is more and more information, and less and less meaning.”</blockquote><blockquote>“And so art is everywhere, since artifice is at the very heart of reality. And so art is dead, not only because its critical transcendence is gone, but because reality itself, entirely impregnated by an aesthetic which is inseparable from its own structure, has been confused with its own image. Reality no longer has the time to take on the appearance of reality. It no longer even surpasses fiction: it captures every dream even before it takes on the appearance of a dream.”</blockquote><p>Written in 1981, both of these quotes come from Jean Baudrillard’s <em>Simulation and Simulacra</em>. Adjacent is the idea of <em>Hyperreality; </em>defined as:</p><blockquote><strong>Hyperreality</strong>, in semiotics and postmodernism, is an inability of consciousness to distinguish reality from a simulation of reality, especially in technologically advanced postmodern societies</blockquote><p>If you <a href="http://google.com/">Google</a> ‘<em>What is Audio?’ — </em>you will get a few results.</p><ol><li>It originates from the latin word: ‘<em>audire’. </em>Which means: ‘<em>to hear’</em></li><li>It wasn’t until the late 1800’s and early 1900’s that the word was widely used in the english corpus. There is a slight upward trend starting a little before 1880, and then a sudden rise right before it takes off between 1900 and 1910.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5Z1E_-rdJLKLeQfQ1VbjcQ.png" /><figcaption>google books ngram viewer, word: <strong>audio</strong> 1800–2019</figcaption></figure><p>The <strong>Phonautograph</strong> was first patented in France during March of 1857, by Édouard-Léon Scott de Martinville. It is the earliest known device to record sound. It would represent sound waves as lines traced onto smoke blackened paper or glass. The representation was a direct reaction from the physical phenomenon of sound waves moving through the air. Initially used to study acoustics, it was not a method of distributing sound as a media, but was used, in some cases, to detect musical pitch by transcribing a person playing a pitch on an instrument or singing and then comparing the transcription to another transcription made from a tuning fork. Using a representation to classify a representation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/321/1*P6ewbR3XDxLAfhZnHz-g2A.jpeg" /><figcaption>É.-L. Scott, “Application for Patent №31470” dated 25 March 1857. 6 pp. including certificate and one phonautogram | <a href="http://webistem.com/acoustics2008/acoustics2008/cd1/data/articles/001974.pdf">source</a></figcaption></figure><p>Below is a recording made with the Phonautograph on April 9th, 1860. Believed to be Scott, himself, singing a French folk song: ‘<em>Au clair de la lune</em>’</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F796559899%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2F1860-scott-au-clair-de-la-lune-05-09&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-HDys395HbMTrQQyB-3Dez1g-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/dac54bda938771f57b16be9dc604a934/href">https://medium.com/media/dac54bda938771f57b16be9dc604a934/href</a></iframe><p><em>An interesting note: when Scott made the recordings, it is said that he had no intention to have them heard by anyone, but rather to simply be an analytical tool. It wasn’t until 1877 that someone thought a sound could be reproduced by a representation.</em></p><p>Prior to 2008, Scott’s recordings were not audibly realized and the oldest recordings available were Edison’s <strong>Phonograph — yellow paraffine cylinder </strong>captures during June 29th, 1888’s concert of 4000 voices performing ‘<em>Israel in Egypt</em>’ at the Handel Festival in London, England.</p><p>The very first in-ear device, however, was invented in the 1850s. A medical <strong>Stethoscope</strong>. The first stethoscope was noted in 1816. At that time it was not an in-ear device that we know now. It was a tool to amplify the sounds of the chest. Rather than using electricity, the stethoscopes of 1816, were paper funnels the doctor would place against a patient’s chest. The stethoscope of 1850 wasn’t too different but had become more refined as a bi-aural device; allowing the sounds from a patient’s chest to enter directly to both of the doctor’s ears.</p><p>In 1895 we get the first transmissions using <em>radio signals</em>. Although, the first transmission was a telegraph of the letter ‘S’. So, not really dealing with sound. But in 1910, Nathaniel Baldwin invents, what we now know as the modern <strong>Headphones</strong>. Initially intended for the Navy’s use as a method of communication, it was designed containing a mile of copper wiring connected to the operator’s headband. This allowed for an analog signal to travel through the copper via electricity.</p><p>The third result you’ll find when Googling <em>‘What is Audio’, </em>is this definition:</p><blockquote>An audio signal is a representation of sound, typically using a level of electrical voltage for analog signals, and a series of binary numbers for digital signals. Audio signals have frequencies in the audio frequency range of roughly 20 to 20,000 Hz, which corresponds to the lower and upper limits of human hearing.</blockquote><p>For the purposes of this article, lets explore digital audio and the methods used by a machine to represent sound.</p><h4>How is Sound Simulated with Digital Signal Processing?</h4><p>A digital signal, like an analog signal, is a representation of sound. There are two main aspects to a digital signal that allows a machine to render a representation. These two aspects can be thought of as X and Y; Time and Amplitude, respectively: <strong>Sample Rate</strong> and <strong>Bit Depth.</strong></p><h4>Sample Rate</h4><p>If you’ve ever opened a Digital Audio Workspace (DAW), or inspected an audio file, you will be familiar with the number 44100. If you have not, 44100Hz is the standard sampling rate for digital rendering of audio. There are different rates, such as 88200, 96000 and 196000, as well. But 44100 is a specific number and the lowest threshold of sampling a signal.</p><p>Sampling rate is the number of times per second that the machine is analyzing incoming sound or an analog signal while recording. Defined online as: ‘<em>The spatial </em><strong><em>frequency</em></strong><em> of the digital </em><strong><em>sampling</em></strong><em>. The reciprocal of the center-to-center distance between adjacent pixels</em>’. What this means is that for each sample <em>(1/44100 of a second) </em>the machine records the signal and represents each bin as a digit. Audio can also be constructed with these principles as well.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/352/1*SoOLWEbsEFskxS-yeJyj5Q.gif" /><figcaption>illustration of low to high (left to right) sampling rate’s effect on rendering a digital representation | <a href="http://monicandreatic.blogspot.com/2016/05/digital-sound.html">source</a></figcaption></figure><p>In the image above, you can see that with a low sampling rate the representation misses a lot of information in the signal; and as the sampling rate increases, produces a more accurate representation. To break it down: <em>the task while converting sound, or analog signals, to digital signals is to take a continuous signal and convert it into discrete values, that when put together over time, render a representation of the continuous signal that can then be converted back</em>. Basically, the higher the sampling rate, the more accurate and precise the machine can render. But why 44100Hz?</p><p>Nyquist’s theorem of sampling states: <em>to achieve a sampling rate that most accurately renders the signal of a frequency, the sampling rate must be double the frequency</em>.</p><pre>Sampling_Rate_Needed = frequency * 2</pre><p>So if you have the frequency of 440Hz, then the sampling rate needed to most accurately render a digital representation of the frequency, would be a sampling rate of 880Hz. Extrapolating on that, the human range of hearing has an upper limit of, roughly, 20,000Hz. Therefore, to accurately render any digital representation of sound that humans can hear, we need a minimum of 40,000Hz sampling rate. And it seems that for safe measure, 4,100Hz were added on. The additional sampling rates are used in specific cases such as, surround sound, audio in films, music production and other hi-fidelity experiences that require more precise manipulation of the rendered signal. But to accurately represent anything a human can hear, 44100Hz is the necessary rate.</p><h4><strong>Bit Depth</strong></h4><p>Bit depth is similar to sample rate in that it is a type of resolution. But rather than rendering along time, it is rendering along amplitude. A low resolution bit depth of 1 can only render amplitude as two dynamics: On or Off. Sound or Silence. Analogous to an image in Black and White.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/366/1*s_8UFg69z4jiOLUjSMVScw.jpeg" /><figcaption>illustration of different bit depth quantization’s effect on rendering a digital representation | <a href="http://monicandreatic.blogspot.com/2016/05/digital-sound.html">source</a></figcaption></figure><p>Here you can see that with more bit depth, there are more discrete values available for the machine to render the representation of amplitude. We can calculate the number of discrete values per bit depth with this formula:</p><pre>2^bit_depth</pre><p>For a depth of 8 bits, there is a range of 256 discrete values <em>(0–255)</em>. For 16 bits, there is a range of 65536 discrete values. And with 24bits, there is a range of 16777216 discrete values. Another way to conceptualize this range of discrete values is: the possible dynamic range of amplitudes available for the machine to render volume of a given sample. Using the below formula, we can understand the dynamic range of a bit depth in decibels.</p><pre>n = bit_depth<br>20 * log10(2^n) = db</pre><p>With a bit depth of 8, we have 48db in dynamic range. However, with a bit depth of 16, we have 96db; and when we factor in dithering, we can reach 115db. For a human, 140db is the limit before our ears become painfully damaged. Therefore 115db is plenty of dynamic range for most production needs when it comes to rendering sound.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C23ts09BHhyHBGcU7WZPkg.png" /><figcaption>basic overview of sound signal being represented into discrete digits as an array | Original © Aquegg | Wikimedia Commons</figcaption></figure><p>The above image shows the signal as a red line, being analyzed by the machine with blue dots and simulated with a digital rendering as an array to the right. Assuming this graph is one second, the sampling rate would be 21Hz with a bit depth of 4bits. Not sure that would sound too good.</p><p>Most music we listen to is at 44.1KHz with a 16bit depth. So next time you’re listening to audio from a digital source, try to imagine all the discrete values along frequency<em> (44100 samples per second)</em> and amplitude <em>(65536 levels of dynamic range)</em> that are tightly rendered into a composite that simulates the experience of listening to a continuous acoustic sound wave.</p><h3>Constructing Digital Audio</h3><h4>Engineering Frequencies</h4><p>In my last article, <a href="https://towardsdatascience.com/sonifying-%CF%80-c453c20a7acd">Sonifying PI</a>, I constructed an overtone series based off the frequency 3.14Hz and generated audio files from the series. Here I will go more in depth about the module’s functionality that I built to accomplish this. Furthermore, I’ll explain how to construct the western musical system as well as more complex, unique musical systems.</p><p>When we are thinking about what a musical pitch is, we often think of their letter names.‘ A, A#, G, G#’ … etc. But these are only abstract representations to help us understand music as a language and navigate through the sonic landscape symbolically. ‘A’ represents a class of frequencies. To reference a more specific instance of ‘A’, we can call the pitch along with the octave number associated with its position in the series. This is called Scientific Notation. The most common instance of ‘A’ is ‘A4’. This specific instance represents the frequency 440Hz and is in the position of the 4th octave. The current standard of tuning is based on 440Hz. Each octave position is exactly 1/2 of the octave above it and exactly double the octave below it. For example, ‘A5’ represents the frequency 880Hz, and ‘A3’ represents the frequency 220Hz. And in between each octave, there is an instance of all twelve notes in the western system. The minimum interval between two frequencies is called a minor second. A minor second is also a symbolic abstraction that holds connotations of quality and definition within the western context; but is only a representation of a numerical distance between two frequencies. There are many ways to evaluate this distance.</p><p>In the notebook below are examples of the mathematical relations necessary to develop an octave as used in the <em>Western Equal Tempered Twelve Tone System of Music.</em></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/90b727189d067301ef3b594190d35a85/href">https://medium.com/media/90b727189d067301ef3b594190d35a85/href</a></iframe><p>The point of this notebook it to expose a few simple truths.</p><ol><li>An octave is a representation of the space between 1 and 2.</li><li>Dividing the octave is the division of the space between 1 and 2 into equal intervals.</li><li>The intervals that are equivalent to the those divisions are represented by an abstracted characterization and are unique respective to the octave they are native to — but share common factorization.</li></ol><p>But let’s use this logic and investigate some alternative musical systems.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4a6464bbca32d7bd1fccd6f008b388bb/href">https://medium.com/media/4a6464bbca32d7bd1fccd6f008b388bb/href</a></iframe><p>For those who are not familiar with the western musical system, the aim of this notebook is to present an exercise that depicts the mathematical process of taking a symbolic structure of discrete values, deconstruct it and nest itself, in its entirety, within the space of each discrete value. Embedding a mirroring substructure within the spaces of the superstructure.</p><h4>Building a Wav</h4><p>From here, let’s use the frequencies from these systems to make a digital representation of their tones.</p><p>The below notebook briefly overviews the process of starting with a digit and constructing tones, then constructing systems based off those tones and analyzing a couple of the files to verify they are accurately rendered.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b2835ee66fb52fb5b60ab0fb7078acfb/href">https://medium.com/media/b2835ee66fb52fb5b60ab0fb7078acfb/href</a></iframe><h4>Building Functional Music with Digital Representations of Tones</h4><p>Figure 1 is three scales built from tones rendered from the above code.</p><ol><li>chromatic quarter tone scale moving from 440Hz to 880Hz</li><li>chromatic semi tone scale moving from 880Hz to 440Hz</li><li>diatonic semi tone scale moving from 440Hz to 880Hz</li></ol><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F798711799%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fconstructed-scales-u-d&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-2baYZz73moPnL7Hj-1bxmXw-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/914a8e784cdb793f2fe59d0e3a4a5e43/href">https://medium.com/media/914a8e784cdb793f2fe59d0e3a4a5e43/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*46vCzXjUov8qgORLELf96Q.png" /><figcaption>graph fig 1 | image generated by author’s code</figcaption></figure><p>With the tones at our disposal, we could make <em>any</em> ‘A’ scale. I used these three just as a proof of concept. Along with scales, we can construct chords and progressions. Figure 2 is a progression built from the rendered tones.</p><ol><li>The progression starts with a I ii V6 I</li><li>Then enters microtonal chords and ends with mixed chords utilizing tones from the diatonic and microtonal systems</li></ol><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F798713572%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fconstructed_progressions&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-oglUm7qfPKxwIJvK-4hCd6A-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/ab1a680b95ca81f6b31c652519f5a00d/href">https://medium.com/media/ab1a680b95ca81f6b31c652519f5a00d/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8qtJ4F5thHO-pNxSoNLVXA.png" /><figcaption>graph fig 2 | image generated by author’s code</figcaption></figure><h4>Novel Construction of Digital Audio</h4><p>Below is a scan of an ink and pencil drawing I made some years ago. Below that is the same image converted in digital audio and displayed through a spectrograph. The data in each pixel is used to construct the frequency and amplitude for the audio information. In a future blog I can break down how this process works, but for now, I just wanted to show this to further elaborate that digital audio is a representation and not sound itself.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*I_R763otD9wkviEi7zAf8w.jpeg" /><figcaption>scan of an ink and pencil drawing by author | image by author</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*76x5SwM2OJRZvEjnsnSrNw.jpeg" /><figcaption>image rendered as a series of frequencies over time | image by author</figcaption></figure><p><em>ps. the actual sound it makes when played through speakers is unbearable</em></p><h3>Manipulating Digital Audio</h3><p>Developing digital renderings that mimic musical mathematical relations is one thing. It’s nice to know the machine can instantiate the tones. But when a machine is using purely mathematical precision, the perception of the sound is familiar but lacks qualities that we are expecting when hearing sound in the physical world.</p><p>Let’s explore more of the module’s functionality and use recorded audio. For this section, we will be using the soothing sounds of public domain mall muzak from the 1970s.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F798758821%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fstimulus-elevator-muzak-1970s-mall&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-ypm4TQ8HFZeyvKtJ-FtzJLw-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/ef6902da304c0b4b03f270fda63436f7/href">https://medium.com/media/ef6902da304c0b4b03f270fda63436f7/href</a></iframe><p>To manipulate the audio is similar to basic string and array manipulation in python.</p><pre>#to know how many elements are in an given array of audio</pre><pre>num_of_ele = seconds*sample_rate</pre><p>Each element is a sample containing a snapshot of audio data.</p><p>Below is a notebook that walks through a series of single manipulations on audio arrays and concludes in an operation of chaining the effects sequentially and passing the audio through. All resulting examples can be found beneath the notebook.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2a681cd1ee41a48e696c135990ac3065/href">https://medium.com/media/2a681cd1ee41a48e696c135990ac3065/href</a></iframe><h3>Results of Audio Manipulation</h3><h4>Reversed Audio</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F799089958%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Freveresed-audio-2&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-ex5vOvvyCYIgyxEg-jTOa5w-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/eaf9d5b4966e60db7dd0bd04338bb908/href">https://medium.com/media/eaf9d5b4966e60db7dd0bd04338bb908/href</a></iframe><p>There are a couple ways to reverse audio in python, but the main principle is to extract the audio data into an array and reverse the array then convert it back to an audio file. One simple way to do this:</p><pre>sr, y = read(wav)<br>reverse_y = y[::-1]<br>write(outfile, sr, reverse_y)</pre><h4>Time Stretch Audio</h4><p>Here are three examples of Time Stretch Algorithms that slow the audio. Depending how you factor the change in the sample rate, you can achieve different sounding stretches.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F799092343%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Ftime-stretch-slow-algorithm&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-2baYZz73moPnL7Hj-1bxmXw-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/137272fd02bd22fdd9b373469831d260/href">https://medium.com/media/137272fd02bd22fdd9b373469831d260/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F799093801%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Ftime-stretch-algorithm-2&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-HDys395HbMTrQQyB-3Dez1g-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/872b7a5a3a58fe4e48b689402e5f87e4/href">https://medium.com/media/872b7a5a3a58fe4e48b689402e5f87e4/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F799094698%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Ftime-down-3&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-oglUm7qfPKxwIJvK-4hCd6A-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/c59a264d7badf46f909affee32e3203a/href">https://medium.com/media/c59a264d7badf46f909affee32e3203a/href</a></iframe><p>The fundamental idea is that you want to make the sample rate longer in order to stretch the musical content over longer intervals. To speed up the audio, you do the opposite.</p><p>Below are two examples of speeding up the audio.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F799097731%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Ftime-stretch-speed-up-algorithm-1&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-ex5vOvvyCYIgyxEg-jTOa5w-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/446d3975051042dee3b077d3f65f0ebd/href">https://medium.com/media/446d3975051042dee3b077d3f65f0ebd/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F799100287%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Ftime-stretch-speed-up-algorithm-2&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-2baYZz73moPnL7Hj-1bxmXw-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/f7c67eb9c9b3b4e683727805e3637ea3/href">https://medium.com/media/f7c67eb9c9b3b4e683727805e3637ea3/href</a></iframe><h4>Pitch Shift Audio</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F799102159%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fpitch-up-algorithm&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-gtvFwwlWw5cxy2qW-3ZByng-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/ff0ffa03812f2802f07aa9a4716b1245/href">https://medium.com/media/ff0ffa03812f2802f07aa9a4716b1245/href</a></iframe><p>The key to finding a method of pitch shifting is to change the speed of the sample rate, without causing the wav to time stretch. To achieve this, I divided the amount of Hz I wanted to shift the pitch by the sample rate then separated the file into left and right channels. Once I did that, I used the <em>Fast Fourier Transform</em> on each sample in each channel and transposed the frequencies then reversed the process and stitched the two channels back together.</p><h4>Delay Audio</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F799106383%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fdelay-algorithm-1000ms-05f-3n&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-oglUm7qfPKxwIJvK-4hCd6A-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/5b4d77bcf27b9b18250675a74c08268e/href">https://medium.com/media/5b4d77bcf27b9b18250675a74c08268e/href</a></iframe><p>If you look at the spectrograph plotting the audio from this file, you can see how the delay blurs the middle of the file, but has distinct repetitions of three at the beginning and end where there is considerably less overlapping iterations of the signal. In order to delay digital audio, you must manipulate the bytes by generating an offset. The logic to do so is below:</p><pre><strong>def</strong> delay(bytes,params,offset_ms):<br>    # generate offset<br>    offset= params.sampwidth*offset_ms*int(params.framerate/1000)<br>    <br>    # add silence in the beginning<br>    beginning= b&#39;<strong>\0</strong>&#39;*offset</pre><pre>    <em>#remove space from the end</em><br>    end= bytes[:-offset]</pre><pre>    # concat bytes with both beginning and end with sample width<br>    <strong>return</strong> add(bytes, beginning+end, params.sampwidth)</pre><h4>Using Code Blocks like Chained Effects Pedals</h4><p>After structuring algorithms in the sequence:</p><p><em>Reverse -&gt; Delay -&gt; Stretch -&gt; Delay half Speed -&gt; Layer Reverse</em></p><p>And passing the digital signal through, the below audio resulted.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F799110358%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fsequential-algorithms&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-5GkEzGI60L4x8sv4-bZRaeA-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/0e659d219cef478f7a0a01b2f63ed36a/href">https://medium.com/media/0e659d219cef478f7a0a01b2f63ed36a/href</a></iframe><h3>Classifying Digital Audio</h3><p>Everything up until this point has been meddling with the mathematical mechanisms that render digital audio. Essentially playing with the audio arrays. For the rest of this article, we will examine more in depth analysis of audio with feature extraction and how a machine can model audio using these features.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/295/1*NE-aprONxrYI5tp7K5ferg.png" /><figcaption>visualization of FFT | <a href="https://www.ritchievink.com/blog/2017/04/23/understanding-the-fourier-transform-by-example/">source</a></figcaption></figure><p>To extract these features, we will use methods found in Fourier Analysis; mainly: the Fast Fourier Transform. FFT is a method of taking a complex wave and parsing it into subsequent simpler waves. It allows us to access features based in the frequency domain. In the notebook below we will explore:</p><ol><li>The Spectral Centroid</li><li>Spectral Bandwidth</li><li>Mel-Frequency Cepstral Coefficients</li><li>Chroma Features</li></ol><p>To briefly explain these:</p><h4>Spectral Centroid</h4><p>The spectral centroid can be thought of as measuring the ‘<em>brightness</em>’ of a given sound. It calculates the weighted mean of frequencies in a given sample.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/486/1*ZITDC_Gpd30IOuS1zuSGbQ.png" /><figcaption>where <em>x(n)</em> represents the weighted frequency value, or magnitude, of bin number <em>n</em>, and <em>f(n)</em> represents the center frequency of that bin.</figcaption></figure><h4>Spectral Bandwidth</h4><p>The spectral bandwidth is related to the spectral ‘<em>resolution’</em> of a sound. It is defined as <em>the width of the band at one-half the peak maximum frequency per sample</em>.</p><p><strong>Mel-Frequency Cepstral Coefficients</strong></p><p>Built from a power cepstrum, the Mel-Frequency Cepstral is a representation of the short-term power spectrum of a sound. The coefficients are derived by examining a sample via FFT, then mapping the powers onto a mel scale. Then taking the log of each mel frequency. After that, performing as discrete cosine transform as if the list of log mel powers were an isolated signal. The resultant amplitudes of <em>that</em> ‘signal’ are the coefficients. The coefficients accurately describe the shape of the spectral envelope of a given sample.</p><p><strong>Chroma</strong></p><p>Chroma extractions show intensity along a vector of 0–11. This vector represents pitch class set and reveals the amount of each pitch class that is present. In other words, it is telling us what harmonic frequencies are present in the signal.</p><p>This is only a handful of features that are present in a signal, but with these and a few others shown in this notebook, we can begin modeling signals.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d3e673b4180fd3345f4489ec247d522a/href">https://medium.com/media/d3e673b4180fd3345f4489ec247d522a/href</a></iframe><p>It’s interesting, to model a digital audio signal is to extract representations of the characteristics of what is already a representation.</p><h4>Modeling</h4><p>Using the <a href="http://marsyas.info/downloads/datasets.html">G. Tzanetakis </a>collection, lets model music genre using these feature extractions. The collection is comprised of 1000 audio tracks each 30 seconds long. It contains 10 genres, each represented by 100 tracks. The tracks are all 22050 Hz monophonic 16-bit wav files. Once we have extracted all the data from the collection, let’s start with our simple but good friend, the artificial deep neural network.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*Q8DyD7WWqspjiCyWIKYgWQ.jpeg" /><figcaption>DeepANN Classifier | <a href="https://www.shutterstock.com/image-vector/neural-net-neuron-network-deep-learning-484275199">source</a></figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/14af7f131a29bbc885cdec7c98777e7f/href">https://medium.com/media/14af7f131a29bbc885cdec7c98777e7f/href</a></iframe><p>The model achieves an accuracy of 69.50%.</p><p>Considering that modeling is a method of engineering representations of the signals, let’s look at other ways we could model the data. Rather than extracting features representing different values within a sample, let’s use the image of the digital audio. To do this, we can take each wav file and generate a spectrograph. We can then use a convolutional neural network (CNN) on the spectrographs.</p><p>Here is an example of the spectrographs we will train the CNN on.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/576/1*2qGqmEtO1URPR9Ne88FGfg.png" /><figcaption>spectrograph of a 10 second wav file from gtzan collection, genre: hip hop | image generated by author’s code</figcaption></figure><p>Rather than training on the extracted features themself to understand the musical profile of a genre, the CNN will learn the visual impression of the files to understand what characterizes a genre.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*hvEjb4lP3XCqDWko3Uc10g.png" /><figcaption>CNN classifier | <a href="https://www.researchgate.net/figure/Convolutional-neural-networks-structure_fig3_326884617">source</a></figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/78e5b20816e00ee378b901b36305f79b/href">https://medium.com/media/78e5b20816e00ee378b901b36305f79b/href</a></iframe><p>While the idea of using a spectrograph to represent the audio is interesting, using it to train a classifier doesn’t seem to be the most effective method. Not to mention the training time on the CNN was much more expensive in memory and time.</p><p>Speaking of memory and time, let’s revert back to using feature extraction from raw audio and try out a long short term memory (LSTM) model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Uc9Bsw7PY2k3FrXer-o76g.png" /><figcaption>LSTM | <a href="http://colah.github.io/posts/2015-08-Understanding-LSTMs/">source</a></figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c3d87de6507522511c71b46e2427ce8e/href">https://medium.com/media/c3d87de6507522511c71b46e2427ce8e/href</a></iframe><p>We reach an accuracy of 66.66% — a little less than 3% from the original neural net model we started this section with. Here the LSTM is effective, but I’d still go with the ANN for a classification problem. But let’s see what else LSTMs can do.</p><h3>Generate Digital Audio</h3><p>In my article <a href="https://medium.com/swlh/generative-ai-malfunctions-imperfections-irrationality-and-expression-9f50c81a6291">Generative AI: Malfunctions, Imperfections, Irrationality and Expression</a> I discuss some opinions about if and how an AI can ‘<em>generate</em>’. Briefly, to overview, I hold the opinion that AI <em>can</em> generate, thus create, under certain circumstances. Before we attempt having an AI generate music, let’s use the predict function to see how well it can replicate music.</p><p>In this notebook, I take three audio files.</p><ol><li><em>‘Magic Spells’ by Crystal Castles</em></li><li><em>‘Two Can Win’ by J Dilla</em></li><li><em>‘Music in Twelve Parts, Part 3’ by Philip Glass Ensemble (live performance in 1975)</em></li></ol><p>The task here is to use the higher dimension audio data, the array of amplitudes that we used in manipulating digital audio, to see if the LSTM can predict the proper sequence of amplitude per sample.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b73ecaa278a7e4d5bd6ed11490bdf4ec/href">https://medium.com/media/b73ecaa278a7e4d5bd6ed11490bdf4ec/href</a></iframe><p>As we can see, the LSTM can be trained to do a fairly accurate job. Generated audio below:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F800381938%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=http%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Foriginal-1&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-ex5vOvvyCYIgyxEg-jTOa5w-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/156f399706fac49ad994064468e236f9/href">https://medium.com/media/156f399706fac49ad994064468e236f9/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F800383270%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Foutput-1-model-at-its-worst&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-ex5vOvvyCYIgyxEg-jTOa5w-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/990dd57a30d2763164f8386b65af3767/href">https://medium.com/media/990dd57a30d2763164f8386b65af3767/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/432/1*BQZHSi-nthUikehPG_-ehw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/432/1*jknqXrbzoZs0wm-WPHiT2A.png" /><figcaption>training performance of first model | images generated by author’s code</figcaption></figure><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F800384425%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Foutput-2-model-almost-there&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-ex5vOvvyCYIgyxEg-jTOa5w-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/77505e912d7c5253ba3d284cf891914a/href">https://medium.com/media/77505e912d7c5253ba3d284cf891914a/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/432/1*L4WnhbZlQpBNBXEC5gUqTw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/432/1*8etZ4D9vbovZxPP4vaftiQ.png" /><figcaption>training performance of second model | images generated by author’s code</figcaption></figure><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F800385535%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Foutput-3-model-at-peak-performance&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-ex5vOvvyCYIgyxEg-jTOa5w-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/4d006b6b346860ce53e143f76c8502f5/href">https://medium.com/media/4d006b6b346860ce53e143f76c8502f5/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/432/1*rea4Q3A2mfz8lei9xsIbKA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/432/1*CuBPYF2DHxuhf2No29KIdA.png" /><figcaption>training performance of third model | images generated by author’s code</figcaption></figure><p>For this next experiment in generating audio, I won’t be showing a notebook, because each model took several days if not longer to train (one model took close to a week with a GPU!). But for those who are interested to read the model’s code, here is the final model’s architecture. Depending on how I preprocessed the data, the model ending up producing some interesting results.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/12e53450c20688bf84b4791345f2e22f/href">https://medium.com/media/12e53450c20688bf84b4791345f2e22f/href</a></iframe><p>One of the main differences of this model, besides the numerous added layers, is the Time Distributed Dense Layering that everything is built within. This allows for the LSTM to consistently return full sequences for every step within and between layers while training. It also produces an output that is a full sequence, instead of a single sample at a time.</p><p>Rather than remain at a surface level of amplitude, in this experiment I wanted to train models on lower dimensional spectral features extracted via Fourier Analysis. And instead of training on a single audio file with a complex form, I trained on full albums and live performances of an artist. I continued to use Crystal Castles, J Dilla and Philip Glass Ensemble. Initially, I split entire albums into 10 second sections of audio, and eventually trained one model on entire tracks uncut.</p><p>During the training process, I saved various stages of the different model’s developments and built a random sequence generator from each artist’s extracted feature data. I then tasked each model from various stages of learning to generate audio to the best it could. Below are outputs from different stages strung together. The audio is not altered in any fashion and is placed in chronological order respective to the model’s stages of learning.</p><p><em>*warning: it is a little noisy at beginning</em></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F800415631%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fai_lstm_generated_crystal_castles&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-212GkypQAjGVjcU9-dHxZ4w-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/6822e23036b8cbf5e2d0cb5d4b401e1d/href">https://medium.com/media/6822e23036b8cbf5e2d0cb5d4b401e1d/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P4EvMv5NcPc-qg1ENdgoow.png" /><figcaption>spectrograph of AI_LSTM_Generated_Crystal Castle | image generated by author’s code</figcaption></figure><p><em>*there&#39;s something beautiful and imperfect occurring between :50s and 1:40s</em></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F800420149%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fai_lstm_generated_j_dilla&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-ex5vOvvyCYIgyxEg-jTOa5w-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/cc3f622dd2127af1b910500dd7d16df5/href">https://medium.com/media/cc3f622dd2127af1b910500dd7d16df5/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FbkywqBzQib67rVWT9qLLg.png" /><figcaption>spectrograph of AI_LSTM_Generated_J Dilla | image generated by author’s code</figcaption></figure><p>What really amazes me, is: what parts of audio the models learn first, given the data, and how quickly it goes from seemingly just tones and sounds to a defined understanding of rhythms. The output reminded me greatly of tuning through fm radio signals, looking for the right station, while driving upstate.</p><p>Now, in these two examples, there is some serious overfitting, as well some nuances. In fact, with the J Dilla track, it is interesting that the model kept spitting out loops. I wonder if that has anything to do with J Dilla’s use of samples. Whereas in Crystal Castles’ music, there is repetition, but it is people repeating words, rhythms and melodies while J Dilla tracks are verbatim sampled repetitions of records and tape. I also found it fascinating that in the J Dilla track, the human voice was clear and defined words could be heard — you can see the vocal overtones in the later half of the spectrograph; whereas in the Crystal Castles track, the voice was present but appears as a vague silhouette of words that blends seamlessly in with the instrumentation, rather than defined utterances.</p><p>Lastly, let’s listen to the Philip Glass Ensemble model. Something interesting is happening here. Rather than listening through the evolution from white noise to intelligent simulations, I want to zoom in and listen to the musical choices occurring here, as well as the timbre. In both Crystal Castles and J Dilla, each mature AI would oscillate between verbatim simulation and an edge of error that allowed for some glitchy imperfection that is reminiscent of a decision. With the Glass model, though, all the outputs are passages of music that cannot be exactly found in the live recording.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F800547535%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fai_lstm_glass_2_1_6&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-2baYZz73moPnL7Hj-1bxmXw-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/9e068f7aeb2304b25f1c62ecfeaaee68/href">https://medium.com/media/9e068f7aeb2304b25f1c62ecfeaaee68/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*zrn3UlBjPtC6uLhNKzqBGw.png" /><figcaption>spectrograph of AI_LSTM_Glass_2_1_6 | image generated by author’s code</figcaption></figure><p>Below are a handful of spectrographs that I find beautiful. The AI is understanding frequency and rhythm and generating signals that are not present in the actual recordings it was trained on. However, it is lacking in timbre and sounds like an analog synth; but to me, feels oddly genuine and has a ghostly influence of Philip Glass, but is something augmented, creative and unique.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*GTxZuuoFs0PCrhQKSpzFFw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*YF0LjPV7DEr8NZ9F38DIAg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*zrn3UlBjPtC6uLhNKzqBGw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*LnOI_Hp430oB_b_T49Tj0A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*wMWctL53Kv5O9nt1ZKgP2Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*rYui7XZ0QOP3YIxr3S2pgg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*Rh2JDWkqaTwsN5D77ov14g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*NYjtE6L9ueRobGagDxLksg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*u5quSOskr3Ql5AoHmIFKHQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*GGZZMf0cVcT0YEwVz-dTCw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*SnT7KOiDysj9zGieKD71dw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*E4Lo3c3zwzwjV8u7k4BYCw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*0p4jtXE8psfNotjC_QAiuw.png" /><figcaption>all spectrographs generated by author’s code</figcaption></figure><p>Below is a compilation of outputs I found interesting. The audio is unaltered.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F800568484%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Faverage-loss-glass-ai&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-HDys395HbMTrQQyB-3Dez1g-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/3a6e48e421205afdbfd89a783c61d003/href">https://medium.com/media/3a6e48e421205afdbfd89a783c61d003/href</a></iframe><h3>Conclusion</h3><blockquote>“And so art is everywhere, since artifice is at the very heart of reality. And so art is dead, not only because its critical transcendence is gone, but because reality itself, entirely impregnated by an aesthetic which is inseparable from its own structure, has been confused with its own image. Reality no longer has the time to take on the appearance of reality. It no longer even surpasses fiction: it captures every dream even before it takes on the appearance of a dream.”</blockquote><p>This, along with the concept of <strong><em>Hyperreality </em></strong>has been in the back of my mind throughout. And while I know Baudrillard is referring to a theoretical discussion of reality as a system of objects, something about this resonated with me when I began exploring DSP. I thought of digital audio as a representation, a simulation of sound. One that could not be distinguished from a naturally occurring sound. And technically, it is. But ending with the GlassAI, left me feeling that through simulation and the intentional development of replicant mediums, we may find a dimension of creativity that we simply could not reach without a technological bridge. Circling back to the idea that AI can be a creative and collaborative tool. I conclude with this notebook.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/33153a30474274421c341fe6588bea6c/href">https://medium.com/media/33153a30474274421c341fe6588bea6c/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F800580997%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fslow-glass&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-gOJupMSztjpYTLQb-tjnnwA-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/77306b3e329d7f6f9c6c52413a63ad97/href">https://medium.com/media/77306b3e329d7f6f9c6c52413a63ad97/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2c5a252dbab9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/constructing-manipulating-classifying-and-generating-audio-with-digital-signal-processing-and-2c5a252dbab9">Constructing, Manipulating, Classifying and Generating Audio with Digital Signal Processing and…</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Sonifying π]]></title>
            <link>https://medium.com/data-science/sonifying-%CF%80-c453c20a7acd?source=rss-f583b68d1c6c------2</link>
            <guid isPermaLink="false">https://medium.com/p/c453c20a7acd</guid>
            <category><![CDATA[music]]></category>
            <category><![CDATA[creative-coding]]></category>
            <category><![CDATA[dsp]]></category>
            <category><![CDATA[object-oriented]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Matthew R Finch]]></dc:creator>
            <pubDate>Fri, 20 Mar 2020 02:04:18 GMT</pubDate>
            <atom:updated>2020-03-20T04:11:37.047Z</atom:updated>
            <content:encoded><![CDATA[<p><em>(using object oriented programming to explore transcendental numbers with digital signal processing)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uWnKnPWXbmoP1Fap8MIOrQ.png" /><figcaption>mod_1193.0116_sq_duty_cycle.wav</figcaption></figure><p><em>In this article, I’ll briefly review what a transcendental number is and how the harmonic series can interact with these types of numbers. I’ll also give examples of using object oriented programming to generate and sonify Pi. Lastly, I’ll use the output of the class objects to build a set of synthesizers.</em></p><p>March 14 just passed. In celebration of Pi day, I found myself thinking about how composers have often put pi to music. Normally the process is a literal conversion from numbers to pitch name. Sometimes the conversion is diatonic. And sometimes the conversion is chromatic. But what if we used the literal conversion of 3.14 to 3.14 Hz as the fundamental of a harmonic sequence? What is the resultant series, and what would it sound like?</p><p><strong>π</strong></p><p>To review the idea of a transcendental number let’s play a game. The point of the game is to reduce any given number to zero. The rules of the game are simple:</p><ol><li>you can only use whole numbers</li></ol><p>2. you can ADD, SUBTRACT, MULTIPLY or take the given number to any POWER — granted the power is a whole number.</p><p>The day prior to Pi day, I was eating at an Italian corner bistro with a friend. She was a mathematician and introduced the game.</p><blockquote>‘What number would you like to start with?”</blockquote><p>I responded: ‘15’</p><p>We can take 15 and multiply it by zero, but let’s make it more interesting. She looked at me and said,</p><blockquote>“Let’s subtract it by itself. ’</blockquote><pre>15 - 15 = 0</pre><p>Okay — that seemed trivial. But it’s a good start.</p><blockquote>‘How about a different type of number?”</blockquote><p>I responded: ‘2/5’ — (<em>thats two fifths — not 2 divided by five)</em></p><p>2/5 — multiple it by 5 and subtract 2. We arrive at zero.</p><pre>5(2/5) - 2 = 0</pre><p>Okay — makes sense.</p><blockquote>‘How about something crazier? What about the √2’</blockquote><p>I wasn’t sure what number multiplied by itself was equivalent to 2, but maybe that didn’t matter. We could raise √2 to the power of 2 and subtract 2 to reach zero.</p><pre>(√2)^2 - 2 = 0</pre><p>We continued the game with √-1.</p><pre>(√-1)^2 + 1 = 0</pre><p>Simple. We finished the game with:</p><blockquote>‘How would you reduce √2 + √3 down to zero?’</blockquote><p>First:</p><pre>(√2 + √3)^2 = 2 + 2√2√3 + 3</pre><pre>which reduces down to</pre><pre>5 + 2√2√3</pre><p>Second:</p><pre>5 + 2√2√3 - 5 = 2√2√3</pre><pre>and if we (2√2√3)^2</pre><pre>we get:</pre><pre>4x2x3 = 24</pre><p>So naturally, the third step is to subtract 24 and you arrive at zero.</p><p>Interestingly, if you play the game backwards, starting with zero and setting the initial number to X; the game becomes solve for X. The game is an exercise in algebra. It reveals what numbers are subject to algebraic operations. These can be called algebraic numbers. But what about numbers like 2.71828?</p><h4>e</h4><blockquote><em>‘</em>e has a few special properties. One of them being, it’s growth potential. It’s uses the ‘<strong>Natural’</strong> Exponential Function, where its slope IS its value.<em>’</em></blockquote><pre>f(x) = e^x</pre><p>In short, algebraic operations can not reduce it down to zero. It holds a unique property of being constant.</p><p>Similar to e, Pi is the ratio of the circumference of a circle to its diameter. It is also a constant. These two numbers are believed to be infinite in size. Both numbers transcend algebraic operations. They contain exponential qualities and can not be reduced. This is what is meant by a transcendental number.</p><h4>The Harmonic Series</h4><p>The Harmonic Series is ‘<em>a set of frequencies consisting of a fundamental and the harmonics related to it by an exact fraction</em>.’ If you put in a quick google search, you’ll quickly come across this definition:</p><blockquote>The <strong>harmonic series</strong> is the sum from n = 1 to infinity with terms 1/n. If you write out the first few terms, the <strong>series</strong> unfolds as follows: 1 + 1/2 + 1/3 + 1/4 + 1/5 +. . .etc. As n tends to infinity, 1/n tends to 0.</blockquote><p>To help understand this, here is a visualization of the series as layered frequencies.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P_spPpxHNN3i7RnvVzzOBw.png" /><figcaption>visualizing the harmonic series in frequencies</figcaption></figure><p>Musically, the equation is simple:</p><pre>f = Fundamental Frequency</pre><pre>1xf, 2xf, 3xf, 4xf, 5xf....etc</pre><p>Each position along the series is a harmonic overtone related to the fundamental. Functional harmony and the construction of music is derived from these overtones. Within the series, the elements of every diatonic chord is present, as well as elements of the melodic minor scale. With the harmonic series, an entire toolkit of harmonic intervals is available. To activate the toolkit, one only needs to set a fundamental.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/550/1*N5QJkVQ207OnTjjb5oyP3g.png" /><figcaption>harmonic series in musical pitches</figcaption></figure><h4>Generating Raw Data and Wav Signals with π</h4><p>If you want to know more about what is happening under the hood with the classes, both modules are on my <a href="https://github.com/FinchMF/wav_utilities">Github</a> with detailed documentation. But essentially, the two modules, <strong><em>wav_table</em></strong> and <strong><em>freq_table </em></strong>are interactive DSP modules with classes and functions that calculate Pi, calculate octaves, calculate the harmonic series and transform frequencies into raw signals, as well as wav signals. Additionally, there are functions for stacking and concatenating signals to make intervals, chords and progressions. You can output these signals as wav files or raw data files directly. There are also functions to classify a random frequency’s nearest diatonic pitch. Lastly, there are also tools to visualize both raw and wav signals. Then just for fun, once you create a wav or raw signal object from a given frequency, you can store the objects for later use as well as print inspection reports.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/73da01d3cd2745bb8c782c7eb3af07ce/href">https://medium.com/media/73da01d3cd2745bb8c782c7eb3af07ce/href</a></iframe><h4><strong>Passing π through the Harmonic Series</strong></h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/22b98e8d18a358ef6232199b12f02ca3/href">https://medium.com/media/22b98e8d18a358ef6232199b12f02ca3/href</a></iframe><p><em>note: the code in this notebook generates more than 320 wav files.</em></p><h4>Examining Class Objects</h4><p>If you look directly under <em>‘Lets look at the Pi_Wav_objs dictionary of pi harmonics’</em> you’ll see one object from the Wav_Signal dictionary inspected, followed by an inspection of a Hz class object. The read out details the entire class object and describes its unique properties and functionality.</p><h4>Examining some Audio Outputs</h4><p>After generating a variety of Pi’s wav signals — here are a set of visualizations to further explore them.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d433d69804b9b65f9cf903394d5d4486/href">https://medium.com/media/d433d69804b9b65f9cf903394d5d4486/href</a></iframe><p>There are four main categories of wav files being visualized here.</p><ol><li>Simple</li><li>Modulated</li><li>Stacked</li><li>Concatenated</li></ol><p>The <em>‘Simple’ </em>category are wav files made with no alterations. They are simply a frequency with a duration of ten seconds. Within the ‘<em>Simple’ </em>category there are four types.</p><ol><li>Sine Wave</li><li>Square Wave</li><li>Square Wave with a Duty Cycle</li><li>Sawtooth Wave</li></ol><p>The ‘Modulated’ category has the same four types, but differs from the <em>‘Simple’ </em>wav because the amplitude modulates and shapes the signal. We see and hear this as the signal getting bigger and smaller, louder and quieter.</p><p><em>‘Stacked’</em> is a set of wav files that increasingly become more distorted and noisy as the process continues. The first stacked audio wave looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hhFGQNXVdh8aEahulI5gyQ.png" /><figcaption>stack_pair_0.wav</figcaption></figure><p>Both signals that were stacked were exactly the same frequency, but different types of waves. Here we are seeing a sine wave layered with a sawtooth.</p><p>As the stacking continued, it reached more unique shapes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LxYnBFxAN6mLOiIsn9Apzg.png" /><figcaption>stack_oct_2.wav</figcaption></figure><p>Here we have many layers occurring. There is a slower sine wave, layered with two types of square waves. Both of the square waves have different cycles, but a similar frequency. There is a high pitched sawtooth layered in there as well. The sound is pretty jarring. But the signal has an almost artistic, visual quality.</p><p>Lastly, we have the <em>‘Concatenated’</em> category. In the notebook, you can clearly see the different sections of timbre and wave form. Additionally, you can spot what frequencies are most present. In some cases, you can see how a strong frequency is present through the juxtaposed sections; represented as a saturated line in the foreground of the spectrograms. Below is an example from the end of the notebook.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yod8Sft6K8JrBcO8PHA6vA.png" /><figcaption>very vapour wave</figcaption></figure><h4>Listening in</h4><p>Below are three frequencies from Pi’s harmonic series sonified. They were taken in chronological order from the series. They are Harmonics 2, 3 and 4 of the base series transposed up 3 octaves.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F778688908%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fsimple-394384-sin&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-9Xeqyw68ayZz9w3R-C3kz2w-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/de47b12f624fb6c84e73e48218be246d/href">https://medium.com/media/de47b12f624fb6c84e73e48218be246d/href</a></iframe><p>Here the frequency is barely audible. If you had speakers that were big enough, you’d be able to feel the signal more distinctly than you could hear it.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F778689367%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fsimple-887364-sin&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-wejWL9PzXa5acpon-eu3Cgg-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/e2b7a021b6217ff234d336d66a5db574/href">https://medium.com/media/e2b7a021b6217ff234d336d66a5db574/href</a></iframe><p>Moving roughly at an interval of a 9th above. The signal is closer to audibility but still pretty low. These pitches are very close to the fundamental of the series.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F778689757%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fsimple-1577536-sin&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-1yp3A7cUFzpnwRGn-tFcaRg-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/e941a2b3396cca65187f72601af9f968/href">https://medium.com/media/e941a2b3396cca65187f72601af9f968/href</a></iframe><p>Just below mid-range and another minor 7th up, we land on a pitch that is roughly two octaves higher than the initial.</p><p>In terms of pitch names, these frequencies are very close to &lt;Eb<em>1_F2_Eb3&gt;. </em>The actual pitched frequencies are only cents difference.</p><p>Eb1 =38.89hz</p><p>F2=87.31hz</p><p>Eb3=155.56hz</p><p>It’s interesting having a range in what we can hear and can’t hear. Even when it’s out of our range, sound is still occurring. For example, listening to the 157.7536hz by itself is quite different than listening to these three frequencies together. Once together, the overall tone, possesses qualities produced by the relation of each signal interacting with one another. Even though we cannot hear each signal when isolated, we can sonically notice the mathematical effects between them when together. Below is an example of the three frequencies functioning as a triad.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F778699360%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fsimple-384384-887364-1577563-sine-triad&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-E2GOQ1BL4EzBWE0x-eV0Rlg-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/f5750e3f50a92a2889564508967aa21d/href">https://medium.com/media/f5750e3f50a92a2889564508967aa21d/href</a></iframe><h3>Intersecting Science and Art</h3><p>The following three audio files are three synthesizers I made. The first one is using <em>pi’s harmonic series in sine wav</em>; the second is using <em>pi’s harmonic series in square wav</em>; and the third is using the <em>Simple 38.4384 88.7364 157.7563 Sine Triad wav</em>. Below each audio file is the same audio data in a spectrogram. Each timbre has a distinct character to it. In the last synthesizer, the same sample is used to pitch each tone. This means that each pitch is a transposition of the original sample, rather than each sample being separate mathematical entities as with the first two.</p><blockquote>Interesting note: turns out pi is close to G natural’s harmonic series.</blockquote><h4>Outcome</h4><p>At points soothing, and at other points very jarring and abrasive. The sound is reminiscent of the mid-1900s impressionistic computer music. Feel free to skip around on the tracks to hear the different combinations of timbres and frequencies.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F778758328%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fpi-sine-wav-synth&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-mBcVNlspWD2helhi-hh4TZg-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/01e7a0555bd36800768518324a0f24b5/href">https://medium.com/media/01e7a0555bd36800768518324a0f24b5/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6MZpZdqON1TvOLb-eVGBSQ.png" /><figcaption>Pi Sine Wav Synthesizer</figcaption></figure><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F778758649%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fpi-square-8-wav-synth&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-mBcVNlspWD2helhi-hh4TZg-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/ed611b05d3fd8c264a56ee901339b80e/href">https://medium.com/media/ed611b05d3fd8c264a56ee901339b80e/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Muq7WNtNgwQbMy8DT7AKdA.png" /><figcaption>Pi Square 8 Wav Synthesizer</figcaption></figure><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F778758857%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fstacked-sine-triad-synth&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-mBcVNlspWD2helhi-hh4TZg-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/b8f9e41f5b29a099de6db89c82cf1c87/href">https://medium.com/media/b8f9e41f5b29a099de6db89c82cf1c87/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F7Ya-XEcykQ0laGDsQtS_A.png" /><figcaption>Stack Sine Triad Synthesizer</figcaption></figure><p>Both modules are still being developed and will include more diverse functionality as the project continues. Stay tuned for a future post reviewing the documentation of the modules as well as a brief introduction to DSP.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c453c20a7acd" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/sonifying-%CF%80-c453c20a7acd">Sonifying π</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Generative AI: Malfunctions, Imperfections, Irrationality and Expression]]></title>
            <link>https://medium.com/swlh/generative-ai-malfunctions-imperfections-irrationality-and-expression-9f50c81a6291?source=rss-f583b68d1c6c------2</link>
            <guid isPermaLink="false">https://medium.com/p/9f50c81a6291</guid>
            <category><![CDATA[music]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[creative-coding]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Matthew R Finch]]></dc:creator>
            <pubDate>Thu, 12 Mar 2020 22:30:40 GMT</pubDate>
            <atom:updated>2020-03-13T15:18:15.160Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JgsUOhoMcPW3-blwaw9FRQ.png" /><figcaption>untitled image from an old hard drive of mine</figcaption></figure><p>A week or two ago I was speaking with a fellow machine learning engineer. They were telling me that they didn’t believe that AI could truly ‘create’. That there was an essential human element that was lacking from a model’s output. That there was no intention. No decision. Only algorithm. And that an unsupervised algorithm was only a sophisticated set of instructions that could retain and display learned information but that the output was not an expression. At most a disguised regurgitation. They explained the output as only a product, a result. They got me thinking about what the output was a result of: Should the result be considered a product of the model’s functionality? Maybe the mathematical construction of the model? Or perhaps it was a product of the coding language used and its native compiling methods? Or is it a product of the deep learning engineer who learned the coding language, studied the model’s functionality and construction, and intentionally tuned the hyper-parameters to its most optimal state for the model to train?</p><p>We both couldn’t deny that when using deep learning models, the experience still feels novel. There’s an excitement when the model finishes the training phase and outputs text, midi patterns, images or predictions. In my experience, it always appears new. Sometimes comical, non-sensical, or incoherent; sometimes outright wrong, but other times the output is poetically sophisticated, harmonically functional and uniquely composed with a clear influence of the works it was trained on. As if the model had really grasped underlying patterns in the decision making process, the expressive signature, of the creator it was trained on and made the next logical iteration of that creator’s process.</p><p>Our conversation made me consider what expression was. Can human expression be isolated and represented in data? If so, can’t a model retain the principles of that data’s structure? They rebutted with the example of market analysis to suggest that what is retained by a model is not the expression itself, but rather an insight describing the expression. An insight categorizing and understanding the expression. But not an insight making the expression. Let’s say we gather a data set on a marketing campaign. The data may well inform us where to invest our marketing budget, between which platforms to maximize our ROI based on current cultural practice — but can the model truly understand the consumer or cultural context of why the market campaign is working or not? Can it isolate and retain the ethos of the market? Can the model make the decision or anticipate unexpected social trends? They asked me to consider why a model can’t predict if or when a tweet, or some youtube video will go viral. They asked me to consider what is missing.</p><p>After reflecting through a few minutes of silence, I admitted I couldn’t articulate but innately understood what they were getting at. They simply said: “Irrationality” Maybe the model can predict the next logical iteration of a process, but humans do not follow a logical path with absolute precision. People change, and sometimes unexpectedly. As the loss function cascades down to zero and accuracy achieves a position close to one hundred precent, the model becomes more and more enclosed in a narrowing path of rationality and remains a process embedded in the logical context of its training.</p><p>Interestingly enough I found myself in another conversation with a fellow composer and producer not too long after. Convinced with a satirical pessimistic tone, they hyperbolized that there would be no human composers in the future. No human authors. No human philosophers. No human scientist or doctors. No human police or soldiers. That the state of our emerging modern societies would be automated to a point of social classes defined by owners of the AI, operators of the AI and those subdued and entertained by the AI. While the sarcasm was noted, they’re not the first to hold this idea. Entire cyberpunk genre’s were developed from this notion. Whether it be 1999’s <em>The Matrix </em>or 1921’s <em>R.U.R</em> the concept of ‘<em>The Second Renaissance’ </em>has been part of our social lexicon for sometime. Reminiscent of aliens, monsters born of advancements in science and superstitions or the destruction of nature from the industrial revolution — the concept of technologies and unknowns dominating with anthropomorphic motives, in my opinion, can largely be traced to not understanding. Then reacting with a defense mechanism.</p><p>Both of my friends might laugh at one another’s position believing the other, respectively, is too optimistic and underestimating or too pessimistic and overreacting. In some sense, it’s a trade off. Rather than take a side, I wanted to venture into the curiosity with an aim to find the middle ground. A place where AI is an extension of human intelligence. A tool to enhance how we interact with ourselves and our environment; not a tool of dependence or oppression.</p><p>In 1956, at University of Illinois at Urbana-Champaign, Lejaren Hiller and Leonard Isaacson developed the first work composed by AI. It was titled, ‘<em>Illiac Suite for String Quartet’</em>. The work is made up of four movements, each a product of an experiment. Illiac is an acronym for: Illinois Automatic Computer.</p><p>The first experiment focused on setting an algorithm that resulted in a Cantus Firmus. Cantus Firmus, meaning ‘fixed song’ refers to a process where a composer uses a pre-existing melody as the outlining structure of the work, as well as the structure for the interior voices in order to achieve polyphony. It is a technique that is replicable and often used to teach species counterpoint.</p><p>Movement 1 below:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fn0njBFLQSk8%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dn0njBFLQSk8&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fn0njBFLQSk8%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/37d17178a9ea14f07c7d3508622db25c/href">https://medium.com/media/37d17178a9ea14f07c7d3508622db25c/href</a></iframe><p>The second experiment is an algorithm focused on resulting four voices, where each voice follows a set of rules.</p><p>Movement 2 below:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FMrN0pOnA1x4%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DMrN0pOnA1x4&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FMrN0pOnA1x4%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/a865131ca69f1adb7f4ae6708642195c/href">https://medium.com/media/a865131ca69f1adb7f4ae6708642195c/href</a></iframe><p>The third experiment is an algorithm that results in varied rhythm, dynamics and playing instructions.</p><p>Movement 3 below:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fcuq4smO_4Js%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dcuq4smO_4Js&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fcuq4smO_4Js%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/6653227f24009aed6b097fd63182d2bf/href">https://medium.com/media/6653227f24009aed6b097fd63182d2bf/href</a></iframe><p>Finally, the fourth experiment is an algorithm that I like to call a musical expression of probabilistic models.</p><p>Movement 4 below:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FQyqiSbbwHIs%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DQyqiSbbwHIs&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FQyqiSbbwHIs%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/e63cb5d03058b564c8c747f59f319599/href">https://medium.com/media/e63cb5d03058b564c8c747f59f319599/href</a></iframe><p>The interesting thing about this Suite, is not that Illaic created new music. Here I would have to agree with my first friend. What is occurring is that the Illiac is precisely executing an algorithm set by Hiller and Isaacson. Rather than the computer creating, it is more so translating a defined set of rules into musical expression. Each experiment is focused around techniques and aesthetics that were very prevalent in academic music during modernism. In a sense, a human would have probably written something similar, but perhaps the Illiac just wrote it quicker through a series of automated process. Here, the computer isn’t creating, only executing.</p><p>Not until David Cope in the1980s do we get Emmy. Emmy is a nickname for <em>Experiments in Musical Intelligence</em>. For Cope, Emmy is about style simulation. Recreation of what already exist. Distilling a style into an algorithmic replication. There is an excellent interview with him called: <em>Composing with Algorithms: An Interview with David Cope</em>. He essentially talks about music in terms that it is already a natural system. We organize it into styles. Styles become records of operations within music that emerged in varying cultural contexts. The emergent operations popularized at any given time are more so an expression of cultural aesthetic.</p><p>When asked by Keith Muscutt, “How would you define algorithmic music?”</p><blockquote>Cope replies, “An algorithm is a step-by-step recipe for achieving a specific goal. Breathing, heartbeats, blinking, and so on, are all naturally occuring algorithms that help keep us alive. DNA, the building block of life, represents another good example of a common algorithm”</blockquote><p>When asked, “In your opinion, who are the most important historical algorithmic composers?”</p><p>He goes much further than 1950’s Illiac.</p><blockquote>Cope replies, “When I consider the history of algorithmic music, I think almost immediately of Guillaume de Machaut, the 14th-century composer who plotted isorhythms — different lengths of interlinked pitches and durations — …..I also think of J.S. Bach, whose fugues, canons and similar algorithmic music form a substantial part of his musical output”</blockquote><p>At another point Cope speaks of how Xenakis used mathematical principles to develop his patterned stochastic music. Often using grids rather than traditional ledger lines. Further more, there is something I find to be one of the most insightful interpretations of music in this interview. Cope says,</p><blockquote>“Algorithms can be as vague or explicit as those who invoke them desire. Classical sonata-allego form is an example of a vague algorithm, where the general types of materials are denoted, but not the actual musical materials themselves. On the other hand, Baroque figured bass is a fairly explicit algorithm….should be followed precisely”</blockquote><p>Cope interprets the creative process as an algorithm itself. Music in his interpretation is a medium to express our creative algorithms. The music we make becomes an output of our human interaction with the natural system of music. Resulting compositions reflect structural decisions within our creative algorithms. In this sense, a deep learning model could distill the structural principles a composer leaves in their work. The composition is a sequence in time. The musical patterns are the data. Converting it to an input to train a model doesn’t seem that far out.</p><p>Before I talk about how 5 LSTMs and a Linear SVC get involved, I want to suggest an idea. Returning to the initial question. It seems evident that generative AI is a combined expression depicting how the model works, how the architecture is structured, how the data is used to train it; and the engineer who tunes the hyper-parameters. For generative AI to be a collaborative extension to the “algorithm” of human creativity seems to be the middle ground. The question becomes, which aspects of the creative algorithm do we want to experiment automating when collaborating with AI?</p><p>When talking about machine learning models, there is a concept that is irresponsible to neglect when considering the evaluation of a model’s performance. The concept of bias / variance trade off. For those who need a refresher, I’ll briefly explain.</p><p><strong>Bias</strong></p><p>Bias is the difference between the average prediction of the model and the correct value. Having a high bias is also known as <em>underfitting</em>. This means that the model is over generalizing the training data and is unable to detect the underlying pattern within the dataset. Therefore, is inaccurate when predicting on test data.</p><p><strong>Variance</strong></p><p>Variance is the variability of the model’s prediction for a given data point. Having high variance is also know as <em>overfitting. </em>This means that the model performs very well with the training data. Actually, too well. So much so that it also inaccurate when predicting on the test data, but in the opposite fashion. Rather than not detecting an underlying pattern in the training data, it is so tightly fit to the training data that it cannot predict what is not already show to it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_7OPgojau8hkiPUiHoGK_w.png" /><figcaption>High Bias (left) / High Variance (right)</figcaption></figure><p><strong>Bias / Variance Tradeoff</strong></p><p>The trade off is an achieved balance between simplicity and complexity of parameters. If the model is too simple, it will be underfit and over generalized. If the model is too complex, it will be overfit and too specific. Finding a balance between the two will produce a model that is more accurate. Detecting the underlying pattern in the data but also generalized enough to predict something that it has not already learned.</p><p>How does this apply to Generative AI? Let’s stack another concept on this. High Bias as more <em>generative</em> and High Variance as more <em>replicative. </em>In this sense, a <em>generative </em>AI would be a model that outputs something similar the composer it is trained on, but if passed through a classifier, not able to be classified as the composer it is trained on. Like a musician who absorbs influence, yet manages to generate their own algorithm. <em>Replicative,</em> in this sense, would be an AI similar to a model that if given the beginning of a piece of music, would output a replica of the composer’s piece as it is already written. Somewhat like a performer who is lost on their instrument without sheet music in front of them. Unable to produce anything without explicit instruction.</p><p>The trade off, here, is the balance achieved when the model can produce a new composition in the <em>style of</em>. Or as David Cope might think, a new composition in the <em>operation of</em> the composer it was trained on. Meaning, when passed through a classifier, the piece would be classified as the composer the model was trained on but, would result in a composition that the composer never wrote.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*QXP9gLbkRbF1I-kSawSl2A.png" /><figcaption>Generative (left) / Replicative (right)</figcaption></figure><p>Thinking of the initial conversation with the fellow machine learning engineer, I would say: intentional malfunctions, celebrated imperfections and accepted irrationality — built in margins of error as methods to achieve AI expression.</p><h3>Collecting Files</h3><p>I chose to go with MIDI files for a few reasons. In a technical sense, MIDI would be easy to convert into model inputs. But in a theoretical sense, having the model learn MIDI is analogous to having a composer learn sheet music. In this experiment, I did not want to simulate a performance, but rather simulate the act of composing.</p><p>To obtain the files I used four libraries:</p><pre>import requests<br>import re<br>from bs4 import BeautifulSoup<br>import wget</pre><p>I scrapped from three websites. The first was Piano Midi Page. The second was Midi World. The third was Classical Archives. From each site I collected as many MIDI files as I could for five composers: Bach, Beethoven, Chopin, Debussy and Ravel. For example, to pull all the Chopin MIDI files from Piano Midi Page, I built this simple function:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fe5a468c1ddc622e619cbaccc190d8a4/href">https://medium.com/media/fe5a468c1ddc622e619cbaccc190d8a4/href</a></iframe><p>This function will pull each piano MIDI file for each composer that you set. Try it out.</p><h4>Exploring the data in a Composer’s Style</h4><p>Once I had a dataset for each composer, I used libraries PrettyMIDI and Music21 to visualize and explore the bodies of work. Here’s an example of the visualization. Similar to a piano roll read out that you might find in a DAW, below is a MIDI representation of Chopin’s Op. 9 №2 in Eb. We can use this to obtain a snapshot of the overall structure and form of the piece.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HjBkYbvQKYB0ghmumJTbHQ.png" /><figcaption>Chopin’s Op.2 №2 in Eb</figcaption></figure><p>But what could we learn from the composer’s body of work? How would the results from one body of work compare to another, and where would they contrast?</p><h4>Chopin’s Harmonic Tools</h4><p>Below is a set of visualizations of what Key, Pitch, PitchSet and Interval Vector and the use count.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/795/1*ejHnoipiQG1aqbC1q73BiA.png" /><figcaption>Chopin’s Pitch Count through out whole Body of Work</figcaption></figure><p>For those not familiar, the reason Pitch axis is 0–11 is because the chart is abstracting pitch out of register into PitchSet where 0 = C and 11 = B, with all twelve pitches in between. Some argue that abstracting away octave diminishes the analysis. The argument has merit, but, in this exploratory stage, the insights we are after can still be derived without acknowledging register. The first thing I see here is that Chopin used these three pitches the most: <em>Eb, Ab, Db</em>. Being familiar with Chopin’s work, I’m not surprised with that. Many of his pieces are written in a flat key. However, I do find it interesting that the three pitches most present in his body of work could be stacked to make an undefined mode of a <em>Db9 chord</em> or interpreted as one of the most common progressions in all of music: <em>ii-V-I</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jMSMtJdlllOU2QUd_KTRew.png" /><figcaption>Chopin’s Key Count through out his Body of Work</figcaption></figure><p>Initial glances here tell me the keys used in his body of work are evenly spread in between key names with accidentals and natural key names. There are twenty-four different keys he is activating in his works. <em>Ab </em>being the most common. With that information, it might be more appropriate to look at the pitch count as representing not, <em>ii-V-I</em>, but rather <em>iv-V-I</em>. Also, maybe it would be best to interpret the three pitches as an undefined mode of an <em>Ab11chord. </em>Additionally we see that out of the twelve Keys he uses and equal amount of major and minor modes, but engages with more minor modalities than major.</p><p>Lets move on to Pitch Set and Interval Vectors</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*v6geqVOAb-4koXQ29_C3iQ.png" /><figcaption>Chopin’s PitchSet Count through out his Body of Work</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hL1Ou9d9maFQFdKP0UlrxQ.png" /><figcaption>Chopin’s Interval Vector Count through out his Body of Work</figcaption></figure><p>Both of these graphs are hard to read and essentially tell us one main bit of information. Chopin used a lot of different types of chords. This tells us that his music was very colorful and made use of many different, unique harmonic quality combinations. The immediate suggestion I interpret is that one of the defining characteristics of Chopin’s music is going to be is harmonic and melodic structures. With out getting too lost in music theory, lets contextualize Chopin’s music, historically. He is an iconic Romantic composer. Following Beethoven, whose work may be considered the transition between Classical into Romanticism, Chopin is one of the earliest composers to explore melodic chromaticism and mixed modal progressions that pass through elegant dissonance and dismantle the pillars of functional harmony. All while retaining a poetic balance. His tools are not as far out as <em>the second school of viennese,</em> but his operations are distant from the mechanics of Bach’s harmonic tools. And when we look at Bach’s Harmonic data, it displays that difference.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/795/1*zD_bbu3YFjYGjaefk1-hnw.png" /><figcaption>Bach Pitch Count through out Body of Work</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WEPm-FLLKmrwU6KXNxWsnw.png" /><figcaption>Bach Key Count through out Body of Work</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FmuLTN_KQjgg2sqt6PC0Xg.png" /><figcaption>Bach Pitch Set Count through out Body of Work</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CcuuHf2ksEF6iKjv03AVnw.png" /><figcaption>Bach Interval Vector Count through out Body of Work</figcaption></figure><p>Immediately it is clear that Bach gravitates to key centers, is using less chord types, remaining in smaller intervallic structures and writing more consistently in natural key names; primarily in D Major, D minor, G Major and C Major. However, there is something interesting about about these three key names, mode excluded. <em>D, G, C</em> has a connection to <em>Eb, Ab, Db. </em>At the core of each composer’s body of work is the same intervallic structure. That is to say: both &lt;<em>Eb, Ab, Db&gt; </em>and &lt;<em>D, G, C&gt; </em>are instances of &lt;<em>ii-V-I&gt;</em>.</p><p>Now this is only two composers, and to go further into this, I’ll have to wait to write another blog focused more on musicology and the history of western music to test out this hypothesis. However, I find this pattern interesting. Perhaps Cope would say that &lt;<em>ii-V-I&gt; </em>or &lt;<em>iv-V-I&gt; </em>and both an 11th chord and 9th chord have a lot to do with the DNA of western music.</p><p>One last thing about historical context. Read Ross W. Duffin’s <em>How Equal Temperament Ruined Harmony (and why you should care).</em> The basic insight of that text: instruments now and instruments then did not access the natural system of music in the same method. Based on different tuning technologies that instrument engineers used, it was a different sonic experience both limiting and opening different musical activations.</p><h3>Models and Evaluations</h3><p>Earlier I mentioned 5 LSTMs and a Linear SVC. Below is the visual representation of the my final model and evaluation flow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*5EGGHW_cYg8Yq2bYDAv4TQ.png" /><figcaption>5 LSTMs and 1 Linear SVC</figcaption></figure><p>In short, I used each composer’s body of work to train a single LSTM. Then used the entire body of each composer’s work to train an SVC. I wanted to build both an AI model that could compose, then a classifier that could evaluate the work of each model. Having a separate classifier that evaluates the composition allows me to experiment with the hyper-parameters of the LSTM in search of the generative/replicative trade off that I wanted to collaborate with. Just because I have a high loss function during a training session might not necessarily mean an outside classifier can’t recognize the output as the composer that trained the model.</p><h3>Why Input Matters</h3><p>My first attempt at this was comical. Listen to result.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F774515770%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fnonsense-chopin_1&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-VeNTyNjqEHjXbEKz-XQpgsg-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/acd46af9b077b959771a4c8983e439e6/href">https://medium.com/media/acd46af9b077b959771a4c8983e439e6/href</a></iframe><p>The only thing this model learned after roughly eight hours was to predict <em>Ab. </em>So in a sense, it learned the most used key in Chopin’s body of work, but did not learn anything about how Chopin sequenced his pitches. There seemed to be an attempt in the beginning with the flourish of seemingly random notes. But when we take a closer look, the pitches are: <em>F, Gb and A</em>. Certainly not excusing this poor example, but these pitches do make some sense and other than <em>A</em> all fall within the key of <em>Ab.</em></p><p>How did I get here? Two problems in my process. First, how I generated the input for the LSTM and secondly, how I built the LSTM.</p><p>Let’s see why my input didn’t capture accurate sequential data.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/925d0e1b54fe995b89261cae79d229d7/href">https://medium.com/media/925d0e1b54fe995b89261cae79d229d7/href</a></iframe><p>After parsing the pitches, transforming them in to integers, isolating the patterns and normalizing the input — I’m left with a vector of sequences and pitch relations. Other than over generalize the learning process, there is another thing to observe about the model’s output. Listen to how mechanical the rhythm is. Each note is exactly the same duration and each offset is mostly one after the other. There are vey few instances of where notes occur at the same time. Let’s imagine that this model did learn a more diverse process of predicting notes. There would still be a pretty big problem if they hardly every occurred simultaneously. That type of model would only be able to output melody but completely miss harmony.</p><p>The deep learning model’s architecture was a total of 3 LSTM layers and, 3 drop out layers, 2 Dense layers with an activation layer using <em>softmax</em> and compiled with <em>categorical</em> <em>cross entropy</em> as the loss and an <em>rms prop</em> optimizer.</p><p>Here’s the summary.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jUDqiE83ekhDtx-4eTbLxQ.png" /><figcaption>first deep learning model’s architecture</figcaption></figure><p>It turns out for the model to understand polyphony and rhythmic data, duration and offset are crucial.</p><p>Below is an image depicting how I converted the midi data into a matrix that accounted for duration and offset. In this example it is referring to Ravel’s midi data. The process worked the same for each composer though.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*cGnZXWxOGNEqc5lc-9PlmQ.png" /><figcaption>transforming midi into LSTM inputs</figcaption></figure><p>The matrix is 128 x 300. The y axis accounts for all the notes possible 0–127. The x axis is time. Where there is a 1, the note is on. Where there is a 0, the note is off. When a 1 is followed by .5, the note remains on for the entirety of its duration. Once the note ends, a 0 is put in place to represent that the note is again off. This is analogous to interpreting the matrix as a piano roll. In order to normalize the input, there are 300 steps per matrix. If a piece is divided into such that its final matrix does not equal 300 steps, then the rest of the matrix is padded with rests in order to fit the shape. This allows for the model to not only learn the note’s position and its duration, but also to understand their position in relation to each other as chordal intervals and progressive intervals.</p><p>I changed the deep learning architecture by adding more layers of activation and dropout, introducing layers of batch normalization, and flattening, changing the optimizer to <em>stochastic gradient descent, </em>specifying the learning rate as well as adding in a handful of layers of <em>leaky relu</em> and <em>tanh </em>functions in order to better control the LSTM signals and memory gates between layers. In addition, in order to amplify attention to significant steps in a composer’s musical patterns, I set the model to use 15 steps in the matrix to predict the next. Further more, I took advantage of keras’ RepeatVector layers and Permute layers to build out and control the feedback loop within the model. This was to optimize the model’s <em>select</em> and <em>ignore</em> gates.</p><p>The model summary below:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5d64e2b06c8047d301354f8024d483ea/href">https://medium.com/media/5d64e2b06c8047d301354f8024d483ea/href</a></iframe><p>AI compositions are at the bottom of the article.</p><p>As for the Linear Support Vector Classifier, I did not use matrices at all. Rather I converted the midi into xml files and treated the musical literature as text literature and implemented NLP. I did this by taking the chord sequences in a composer’s work and transforming them in ngrams. In a high dimensional sense, this image below describes the process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*wEUav56aiumQUomO-DEJUQ.png" /><figcaption>transforming midi in chord ngrams for SVC</figcaption></figure><p>To do this, there are essentially two steps after converting the midi into numerical representations:</p><ol><li>Use the twelve tone PitchClass, in the form of a list, to build a function that can detect the distance and abstract the intervallic relation in PitchSets</li><li>Make an index of 192 chords and label each chord from 0–191. Then use the chord index to identify the chords and represent the sequences in a lists of numbers.</li></ol><p>Let’s dive into some intervallic mathematics to show how this can be done.</p><pre>#  12 tone scale in pictchclass  <br>roots = list(range(12))<br>print(roots)</pre><p><em>output: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]</em></p><p>Okay, cool. We have the twelve tone scale. Now let’s look at a problem and see what we can do to solve it.</p><p>Given two notes, what is their intervallic relation?</p><p>We are going to start with <em>A </em>and <em>Eb</em>, and <em>Ab</em> and <em>D.</em></p><p>Here are the two sets of notes given as numbers: (9, 3) &amp; (8, 2). First it’s important to note that both PitchSets contain equivalent intervallic distances between PitchClasses. So lets focus on (9, 3) and come back to (8, 2)</p><p>We are asking, what is the relationship between PitchClass 9 and 3? A diminished fifth is the answer. A diminished fifth has an interval of six steps. In other words, a diminished fifth is equivalent to PitchSet &lt;06&gt;.</p><p>Here’s the solution:</p><pre>(roots[9-6:]+roots[:(9+6)%12])[6+3], (roots[9-6:]+roots[:(9+6)%12])[6-3]</pre><p><em>output: (0, 6)</em></p><p>The variable <strong>roots</strong> = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]</p><p>Below is a notebook that takes you step by step to arriving at this line of code.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5bc67e0dbc52c9c62fa93df2eeb22356/href">https://medium.com/media/5bc67e0dbc52c9c62fa93df2eeb22356/href</a></iframe><p>Above, this allows us to take converted midi data and see what interval qualities are at the heart of the work. Similar to what is happening under the hood with Music21, this allows us to extract this information and train the Support Vector Classifier.</p><p>Secondly, to make the chord index, I took the twelve tone system and multiplied the abstracted system by the sixteen most common chord qualities. Essentially 12x16 = 192 PitchSets. I did not convert the PitchSets to their PrimeForm, rather left them in NormalOrder in order to retain information about what key the composer was using in addition to the intervallic qualities.</p><p>Below is a notebook that takes you step by step to building the index.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/87aa843da5cb24f19f468ce8de690c89/href">https://medium.com/media/87aa843da5cb24f19f468ce8de690c89/href</a></iframe><p>Now let’s look at the Support Vector Classifier itself. When using the chord ngrams and these parameters for the SVC:</p><pre>Parameters:</pre><pre>          Loss: ‘Hinge’</pre><pre>          Penalty: ‘L2’</pre><p>the model reached a performance level of:</p><pre>Fscores:</pre><pre>Micro-Weighted: <strong>97.41%</strong></pre><pre>Macro-Weighted: <strong>95.4%</strong></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/343/1*geFGlPpTK1qpDKKGHRfe2A.png" /><figcaption>SVC for four composers classified by ngrams</figcaption></figure><p><em>Note: I had to let go of Ravel during the classification training due to how little works I was able to retrieve of his. I was still able to to make an AI_Composer model, but the music was not enough be effective in the classifier.</em></p><p>While I was happy with the result of the classifier’s training, my fellow machine learning engineer friend tasked me with the question: “If incorporating rhythm had a positive affect on performance of you’re AI_Composer, shouldn’t it increase accuracy here?”</p><p>It was a good insight. After making ngrams on the durations and offsets of each chord and training the classifier on both chord ngrams and duration ngrams, I was able to push the accuracy to nearly 100%. With same parameters, the model’s performance jumped to:</p><pre>Fscores:</pre><pre>Micro-Weighted: <strong>98.63%</strong></pre><pre>Macro-Weighted: <strong>98.15%</strong></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/343/1*vhErhgmkp7IKwevm3oWGUw.png" /><figcaption>SVC for four composers classified by chord and chord duration ngrams</figcaption></figure><p>Lastly, in order to not leave Ravel out of the mix completely, I made a binary classification model that was trained to classify whether a composition was from a Ravel or Debussy AI_Composer. Using the same parameters and method of generating inputs, I reached a model that performed:</p><pre>Fscores:</pre><pre>Micro-Weighted: <strong>91.67%</strong></pre><pre>Macro-Weighted: <strong>90.0%</strong></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/334/1*dHTWNoL7yJhSrSFuw0FvRg.png" /><figcaption>SVC for two composers using chord and chord duration ngrams</figcaption></figure><p>Choosing SVC seemed like the obvious chose do to the nature of how the model interacts with features per data point, but I also evaluated the performances of Logistic Regression, KNN, Multinomial Naive Bayes and a Multi Layer Perceptron Classifier. All of them did well, but The Support Vector Machine was consistently the most accurate through out the different variations of ngram types and different hyper-parameter settings.</p><p>Each AI_Composer model is set to generate a new piece each time the code is run. This is due to using a random generator for the inputs to set the generative process in motion. Below are 5 that I felt were interesting…</p><p>Chopin_Bot:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F775142779%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fchopincompose&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-jVGkBRNgOgJ7xXy4-KfgtpA-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/0f492e007ee5588931db7640e6800e18/href">https://medium.com/media/0f492e007ee5588931db7640e6800e18/href</a></iframe><p>Ravel_Bot:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F775143841%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fravelcompose&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-jVGkBRNgOgJ7xXy4-KfgtpA-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/e92825a692e71ac01abae6a03e98daf2/href">https://medium.com/media/e92825a692e71ac01abae6a03e98daf2/href</a></iframe><p>Bach_Bot:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F775141642%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fbachcompose&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-jVGkBRNgOgJ7xXy4-KfgtpA-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/db946b71bb1254a6350666ecdd7b3bac/href">https://medium.com/media/db946b71bb1254a6350666ecdd7b3bac/href</a></iframe><p>Beethoven_Bot:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F775144708%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fbeethovencompose&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-jVGkBRNgOgJ7xXy4-KfgtpA-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/49ab76857719a67db771bb2a76131b9c/href">https://medium.com/media/49ab76857719a67db771bb2a76131b9c/href</a></iframe><p>Debussy_Bot:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F775145878%26show_artwork%3Dtrue&amp;display_name=SoundCloud&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fdebussycompose&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-jVGkBRNgOgJ7xXy4-KfgtpA-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/b0de1cfbb31e0f200991c50cd43c7996/href">https://medium.com/media/b0de1cfbb31e0f200991c50cd43c7996/href</a></iframe><p>When passed through the classifier, the AI’s output is classified correctly between, roughly, 57% — 70% of the time. The margin of error built into the AI model could be seen as a deviation, or could be interpreted as the AI’s own creative decision making. The expression of the AI. Or perhaps an expression of the engineer. However you decide to think about it, the SVC could detect to some degree, an AI’s output had <em>something</em> in common, with the composer’s data that trained it. But these 5 examples are times when the SVC misclassified the compositions.</p><p>There is something mechanical when listening. I can tell that it is not a human playing. But I wonder what it might sound like to sample these works and use them as a starting point for a project? Or to print the sheet music and have a human perform them rather than a computer? How might the model and I be able to work together?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9f50c81a6291" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/generative-ai-malfunctions-imperfections-irrationality-and-expression-9f50c81a6291">Generative AI: Malfunctions, Imperfections, Irrationality and Expression</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is Post-Internet Glitch Art?]]></title>
            <link>https://medium.com/lotus-fruit/what-is-post-internet-glitch-art-7baba654345a?source=rss-f583b68d1c6c------2</link>
            <guid isPermaLink="false">https://medium.com/p/7baba654345a</guid>
            <category><![CDATA[databending]]></category>
            <category><![CDATA[art]]></category>
            <category><![CDATA[data-art]]></category>
            <category><![CDATA[philosophy]]></category>
            <category><![CDATA[machine-language]]></category>
            <dc:creator><![CDATA[Matthew R Finch]]></dc:creator>
            <pubDate>Thu, 05 Mar 2020 00:07:48 GMT</pubDate>
            <atom:updated>2020-05-28T16:53:43.987Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/616/1*f36NdLmbRqxsmm8dG72h8w.png" /><figcaption>detail of data moshed image, ‘untitled’</figcaption></figure><p>Data-moshing was a term I became aware of later. Initially it was just an aesthetic tool at my disposable. I didn’t consider opening a jpeg in a text or hex editor to be in anyway philosophical, non the less political. At no point did I find myself reflecting on the dimensional implications of turning a png or bmp file into raw data and importing it into audacity. Glitch aesthetic was a technique oriented interaction. Digital material abstracted for the sake of form. Form over content or meaning. Changing bytes or misusing applications never meant a resistance, nor was it an attempt to fracture a simulation. Similar to the Romantic Era of adding chromaticism and extended mixed modal progressions into a composer’s harmonic vocabulary and complexity of expression, glitch was just a method of interacting with and extending a medium. Did <em>Clare de Lune </em>have a social statement about the nature of <em>Power</em> embedded in the use of chromatic motion in both the harmonic outline and melody? Was there a critique on hyperreality inferred somewhere in a transition or dynamic shift? To me, glitch was just another dialect of a visual language used both independently and commercially. I associated no deeper meaning than an aesthetic result from an agnostic process.</p><p>Glitch Art has been a hobby of mine for sometime. Before I was interested in data bending raw data and file formats, I was experimenting with circuit bending. Often experimenting to make something from a broken keyboard of an electronic toy with the idea of sampling the textures. When I came across the terms: <em>“Post-Internet” , “Post-Data” , “Post-Digital” </em>— I was intrigued to find a whole community of people who glitched files and rerouted circuit boards. It didn’t surprise me to recognize a strong <em>Post-Modern</em> , <em>Post-Structuralist</em> ideological origin at the base of the <em>Post-Internet</em> mission.</p><p>In this blog I take a look at some main figures in the <em>Post-Internet</em> glitch movement and the overall messaging of the aesthetic. And ask: What is the point? How does it help? What is its impact?</p><p><em>Post-Internet</em> simply means since the internet has been around. With art it more specifically refers to Internet Art that was made <em>of </em>or<em> about </em>the effects internet has on society and culture in the early to mid 2000&#39;s.</p><h4><strong>Influential Thinkers in Glitch</strong></h4><ol><li><a href="https://beyondresolution.info/">Rosa Menkmen:</a></li></ol><p>Explained in her 2011 <a href="https://networkcultures.org/_uploads/NN%234_RosaMenkman.pdf"><em>The Glitch Moment(um)</em></a> — glitch art is something which both exists as artifact and process. That while there may be a movement within the aesthetic’s development, it is not the defining results that hold any significance, but rather the process and the method of interaction. Aesthetic is a result of process, result of the machine language used to store the pre-glitched data. It is about what is learned in the moment of the glitch. A fleeting moment which illuminates, via something unexpected, an insight and furthered access to understanding the digital environment.</p><p>To Menkmen, <em>glitch </em>means an interruption to a conventional stream. The glitch is a symbol for a way of life. A way of interacting with digital power structures, both psychologically and architecturally, which surround and assimilate us while interacting with a digital environment. Glitch is not just an aesthetic. Glitch is an operation to Menkmen, a process. The results are irrelevant, as the experiment is the act and investigation of a medium, format or file type through deviance of context.</p><blockquote>“[a glitch teaches us]the taxonomy of the machine language’s system to store data” — Menkmen.</blockquote><p>She stresses the <em>positive outcome of imperfection. </em>She describes a glitch as an imperfection. To see the positive outcome is to recognize new possibilities revealed by the imperfection.</p><p>2. <a href="http://nickbriz.com/">Nick Briz:</a></p><p>Nick has a series of <a href="https://www.youtube.com/user/NickBriz">video essays</a> that very clearly explains the aims and mission of <em>glitch</em>. His <a href="https://www.youtube.com/playlist?list=PLE7F731BE271DC5D4">Glitch Tutorial</a> is a refined elaboration and tutorial to get one introduced to glitching.</p><p>Rather than glitch being a mere temporal manipulation of raw data, Nick Briz takes the act beyond the limited formalist view point and assigns a larger ideological function for glitch.</p><p>Briz’s main focus is that people come to understand digital literacy and want to learn it. He stresses that digital literacy is prerequisite to digital agency. Through glitch, Briz states, it is brought to light the politics embedded within digital systems. He goes to explain that file types, applications, languages and operating systems are all embedded with cultural and social conventions. These conventions begin to assimilate us. He explains that if we do not disrupt the conventions then we agree to them.</p><blockquote>“An iphone is a package of conventions…culture politics masquerading as modern technology”</blockquote><p>Social media has developed a type of self awareness, a set of actions and social perceptions that did not exist prior to the early 2000’s. It is not a single post that matters, but rather that the act of posting is a social communication now.</p><p>Through glitch, two things are accomplished:</p><p><strong>First</strong>, a convention is challenged. Knowledge is gained through disruption. Knowledge is power. Embedded culture exposed in the structures.</p><p><strong>Second</strong>, the disruption also exposes the exploits in the system. Interacting with the exploits teaches digital literacy over time.</p><p>Together these two results empower a tactical user of digital environments to both learn and master digital agency.</p><p>Agency empowers a user to harness the modalities of a digital environment.</p><p>3. <a href="http://jonsatrom.com/">Jon Satrom:</a></p><p>Satrom has a big influence within the glitching community. He’s very active. He is known for live desktop melt downs and lectures about the process of <em>‘Creatively Problem Creating’ .</em></p><p>In general, between these two, the main <em>glitch</em> philosophy is outlined. First, that glitch is a systematic way to disrupt streamlined conventions. In order to examine imperfect results and to discover new possibilities that were, beforehand, unnoticeable. Secondly, that the digital environment is a product of the cultural politics of those who developed it. Therefore through cognizant, tactical interactions one is able to achieve agency. Through agency, one can help transform the digital environment’s landscape. <em>glitch</em> becomes a way to further learn about and master digital agency.</p><h4><strong>Doubts</strong></h4><p>Okay, okay, okay. I get the idea. I can appreciate it. Infact I like it. But is <em>glitch </em>the academic appropriation of ‘Hacktivism’ or a companion aesthetic to help people understand theoretical and ethical views of ‘Hacktivist’ mentality? It is still difficult for me to latch onto the philosophy as an active component. The philosophy feels more like an after thought; a ribbon tying the ideas together, justifying the relevance of <em>glitch</em>. Is <em>Post-Internet</em>, just an ironic reference to taking this mechanical abstraction of computer malfunctions and casting a <em>Post-modern</em> explanation on the process of glitching?</p><p>What is the ideal outcome? Where is the application of this digital agency? When I try to imagine this glitched post-internet content’s future, I see an abundance of absurdity. I see <em>glitch</em> commoditized and digital agency forfeited in order to get a name brand glitch. Is digital agency analogous to accessing the html of your myspace and changing the color of the font and margins — or is it more? Where does the agent take action? Is the aim to have a <em>glitch</em> inspire someone to build a personalized operating system or challenge the bias in a machine learning model?</p><h3><strong>A Few Ways To Glitch</strong></h3><p>The first way I learned how to glitch was by opening a jpeg in TextEdit. You get the ASCII characters of all the bytes and can disrupt the file in any way you’d like. You might find yourself copy and pasting sections of ASCII or writing in your own messages. The results vary but here are some examples.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yGsGiZFvqO5SEPLoEz5-IQ.png" /><figcaption>ASCII and jpeg databend side by side</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CqPN1LVCZUPd02STlcisTg.png" /><figcaption>ASCII and jpeg databend side by side</figcaption></figure><p>The image is a building in downtown manhattan. There are three methods of glitching that I used. Here is the final result.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*KAAd0s9zGcwt1EGIGoGqNQ.jpeg" /><figcaption>final version_glitched image ‘untitled’</figcaption></figure><p>The first method I showed above. Accessing the ASCII and rearranging, adding and omitting characters. The second method I went in a little deeper and used a hex editor to more manually change with intention rather than purely by chance. The third method, more reliant on chance, was running the file through audacity as an audio wave file then exporting back out as a jpeg.</p><h3><strong>The Image I Started With</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EwArRs99HaEwPCbbAHAh_w.jpeg" /><figcaption>original image</figcaption></figure><p>Here’s an example of using the first two methods side by side.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zdeKhTZ3UzZu8WRTPYvZNA.png" /><figcaption>left to right: hex code, ASCII, Image.</figcaption></figure><h4>For those unfamiliar</h4><p>Hexadecimal looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zpQLBXJUx_7Roo6z78hpZQ.png" /><figcaption>Hexadecimal code of Image in a Hex editor</figcaption></figure><p>Here’s how Hexadecimal works.</p><p>In the picture above you will see 4 columns. Starting on the left:</p><ol><li>Line number from my IDE (VS Code)</li><li>Data Position</li><li>Binary Data as Hexadecimal</li><li>ASCII encoding of Data</li></ol><p>Each hex represents one byte, which is binary — so each hex is divided in two. Both of the characters in hex represent half of the byte with 4 bits, together equaling eight bits<em>. </em>In short, the reason for using hexadecimal instead of decimal is as follows:</p><pre>Highest number 8 bits can express</pre><pre>2^8 = 256</pre><pre>0 -&gt; 255 = 256</pre><p>Here is how hexadecimal looks alongside decimal.</p><pre>Decimal<br>0,1,2,3,4,5,6,7,8,9 = 10</pre><pre>Hexadecimal <br>0,1,2,3,4,5,6,7,8,9,A,B,C,D,E,F = 16</pre><pre>A = 10<br>B = 11<br>C = 12<br>D = 13<br>E = 14<br>F = 15</pre><p>The relationship between 0–255 = 256 (highest number expressed with 8 bits). The hexadecimal system is so convenient. It’s beautiful.</p><p>So if,</p><p>0x0 = 0</p><p>0xF = 15</p><p>0xC0 = C0 = 192</p><p>Which hex equals 255?</p><p>0xFF!</p><p>This allows for the byte to represent the values of possible numerical expressions (0–255 ) in two digits rather than using three. So each half of the byte can be represented for its entire numerical possibilities 0–0xF side by side creating a larger scale representing all possible numerical expressions in the total byte, 0–0xFF.</p><h4>ASCII</h4><p>ASCII is another level of encoding. More readable. It results symbols of the english alphabet, punctuation etc. However, it can not read this file in a way that informs us. Well, at least not easily.</p><p>I have two images above that show the glitch and ASCII side by side. Here I want to show the ASCII read out closer.</p><p>Editing is essentially the same in theory as it would be in a hex editor, but rather manipulating ASCII representation of bytes:<em> ‘Copy, Paste, Delete, Write in Your Own’</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/984/1*52oDtaRgXEthbuPmx6ZJuA.png" /><figcaption>Image being read in ASCII</figcaption></figure><p>In these two types of glitching, you’re doing the same thing; but with hex, you’re accessing interpretable code that allows you to see the structure of a file. ASCII does not appear easily interpretable. However, I do feel they produce different results. The file type also effects how the data is bent. A .jpeg and a .png corrupt differently.</p><h4>A Third Method</h4><p>Beside inspecting and manipulating the structure of a file to corrupt expressively, there is another process that leaves more to chance but is interesting. Rather than changing the design, this process has you change the file type all together and run the file through an application that would have not been able to read the file as its original type. For example, taking a .jpeg and putting reverb effects on it via Audacity. Below is a brief tutorial how to that.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/436/1*452ioyO_kgDSQXbrD1MPJw.png" /><figcaption>Image saved as jepg</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/436/1*1yWp1BbtD0Dzl_m9J2NMiA.png" /><figcaption>change file type to raw</figcaption></figure><p>Then in Audacity:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/1*CsPVIV05hHgUHuKDsw3cHw.png" /><figcaption>import the file</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nHzfmZTl2ctXIUTBbVNLIQ.png" /><figcaption>IMG_0048_databend.jpeg being interpreted as a wav file.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ijZe-h5OmCNJLA4c20lXCQ.png" /><figcaption>a closer look</figcaption></figure><p>Now I want to use functions that are native to Audacity and effect the data as if it were audio, then convert it back to an image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RA5ttTNtgT_Hfte_s8AlPQ.png" /><figcaption>using the wahwah effect on the image.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/586/1*esVVU1WSIQSHHSwAZv7nbw.png" /><figcaption>image processed with a wahwah along with other effects</figcaption></figure><p>I continued to play with the three processes until I came across something that struck me. The final_version, shown earlier, to me is an accident — however supervised it may be. Im not sure it represents anything. The process of it was entertaining as usual. And to some degree I can see the silhouette of the Post-Internet Glitch Art’s impact on literacy and awareness in the digital environment. It has the potential to empower agency. I think the next thing I would do is save the .jpeg as a .wav file, then open the hex editor and see how many layers of transitions can be bent, and how far you can push the imperfection to reveal something beautiful in the machine language.</p><h3>PhotoShop a WAV file</h3><p>Let’s take the process from the third method and reverse file type. We’re going to start with Debussy’s <em>‘Clar de Lune’. </em>Rather than take an image and send it through audio effects, let take audio and put it through an image editor.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F734881708%26show_artwork%3Dtrue&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fclaire-de-lune-claude-debussy&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-000659195245-0ih0rw-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/16854557cd1c4071954d00a2f9dc95b0/href">https://medium.com/media/16854557cd1c4071954d00a2f9dc95b0/href</a></iframe><p>From there, change the file into raw data then open it in Photoshop.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vVsxhOZ-4xK7mYIg9n5z_g.png" /><figcaption>Clar de Lune audio as a jpeg</figcaption></figure><p>Each pixel is a byte. The beginning of the piece is at the top left corner, and the end of the piece is at the bottom right corner. Thinking of the image as a surface, like the magnetic dust on tape or the bottom of a cd, you can imagine the idea of scraping or scratching that surface. The same idea works here.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8FaYtz6SEsDt4Npq3XNFuQ.png" /><figcaption>small scratches</figcaption></figure><p>Eerie and somewhat comical once returned to a .wav — the glitches start at 1 min and have a duration of ~2mins35s. Turn sound on low at first. There are some spikes</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F734883814%26show_artwork%3Dtrue&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fclaire-de-lune-claude-debussy-copy-copy&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-000659197060-v35wox-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/c4e859d41207f9f0a33e56988572937f/href">https://medium.com/media/c4e859d41207f9f0a33e56988572937f/href</a></iframe><p>While I didn’t find the glitched audio particularly listenable, I wanted to look a little closer at the data. I wanted to more clearly see where the data had been corrupted and where in the audio file would it be visible — and how it would be visible.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/af411595b659db7824cda97ecb54eaa3/href">https://medium.com/media/af411595b659db7824cda97ecb54eaa3/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/308ddc8f3daf6576146b3ce72a7d9a72/href">https://medium.com/media/308ddc8f3daf6576146b3ce72a7d9a72/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3c52891667265bb661c5c7c72b484c1e/href">https://medium.com/media/3c52891667265bb661c5c7c72b484c1e/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6KD-OZklHILw8WXIi3K9kg.png" /><figcaption>glitch occurs from 1min to just shy of the 2min45s marking.</figcaption></figure><p>Here you can see the disruption pretty well. Below, I’ve changed the color to see if a different contrast would clarify where the glitch is occurring.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OU7Oix6utI6G-zPxR5C_wQ.png" /></figure><p>You can see the piano notes of the piece pretty clearly. You can see that at 1 min the glitch begins. You can see all the spikes. And where the corruption ends and the coherent music returns. I still wanted a closer more detailed look.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1df41b4932e26d43992e5b0d79feb68d/href">https://medium.com/media/1df41b4932e26d43992e5b0d79feb68d/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/899/1*xHfR-4VqPB1FX9cuZheBlA.png" /><figcaption>first 15s of the piece</figcaption></figure><p>Above is the beginning. This can act as the base model of normalcy. Each note is distinct. The shapes and spacing are very clear. Overall there is a sense of <em>simple</em> and relatively uniformity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/899/1*5y_q495f0uqDUoTwHzFHBA.png" /><figcaption>75s–90s</figcaption></figure><p>Depicting a fifteen second frame shortly after the glitch begins, we can see a drastic change in the data. There is much more activity. Seems more chaotic and choppy. There seems to be a change in timbre. We can see that it is louder as well.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/899/1*csnwxGo4wt32l_9Im9Iskg.png" /><figcaption>135s-150s</figcaption></figure><p>2min15s to 2min30s. These 15s show the tail of the glitch. You can see indications of uniformity the more you look right, while the left is still distorted.</p><h3>So What is Post-Internet Glitch Art?</h3><p>While I find a lot of this very interesting to play with, I struggle to find meaning. I’m vaguely reminded of Michael Foucault and Jacques Derridas. I’m entertained with the notion of digital agency. But I lack to connect any sense of empowerment. To me, Post-Internet Glitch Art is an aesthetic resulting in our current time and based in a deconstructionist interaction with machine language. The act is enjoyable, but I feel the niche is somewhat exclusive and not inviting or sensical to anyone who doesn’t have a grasp of binary or file formats.</p><p>I notice more and more a commercial accessibility of glitching via phone applications and photoshop tutorials to help get the ‘glitched’ look. If there is something empowering, then maybe the commoditization of the aesthetic is good. Good in that in that it makes available for everyone to break their jpegs and signal a messaging of resistance. Or maybe it undermines the idea completely because the fracture is not made with intention, but rather automated by a third party. Nonetheless, I’m sure I’ll continue to play and build pythonic utilities to distort and rearrange bytes.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7baba654345a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lotus-fruit/what-is-post-internet-glitch-art-7baba654345a">What is Post-Internet Glitch Art?</a> was originally published in <a href="https://medium.com/lotus-fruit">Lotus Fruit</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Praise, Questions and Critique: Spotify API]]></title>
            <link>https://medium.com/@FinchMF/praise-questions-and-critique-spotify-api-38e984a4174b?source=rss-f583b68d1c6c------2</link>
            <guid isPermaLink="false">https://medium.com/p/38e984a4174b</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[music]]></category>
            <category><![CDATA[librosa]]></category>
            <category><![CDATA[spotify]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Matthew R Finch]]></dc:creator>
            <pubDate>Wed, 18 Dec 2019 15:42:24 GMT</pubDate>
            <atom:updated>2020-03-05T16:43:54.431Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dzh55-fMF8KNnS0cGnYKxg.jpeg" /><figcaption>opening 13 bars of Erik Satie’s Gymnopedie №1</figcaption></figure><p>Recently I built a Multiclass Music Genre Classification Model with Python using audio data from Spotify’s API. I was satisfied with the outcome of the model having reached an 80.4% accuracy using Random Forest with 6 classifiers. But after further inspection of the data provided by Spotify’s API, I found myself left with questions and an ambiguous understanding of what Spotify’s data interprets. In this blog I’m going to break down Spotify’s interpretation of Audio Features and Audio Analysis and compare it to old fashioned, hand written theoretical analysis as well as use the Python library: Librosa, for audio analysis. I’m going to investigate one piece of music: 1888’s Erik Satie’s solo piano piece, ‘Gymnopedie №1’.</p><p>Let’s start with what the music sounds like.</p><h4>Gymnopedie №1</h4><p>From Erik Satie’s <em>The Essential Collection</em> here is <em>Gymnopedie №1</em>.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F729824446%26show_artwork%3Dtrue&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fgymnopedie-no1&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-000654096361-xi6ujn-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/d34d235b36b76b7a940305892feaef49/href">https://medium.com/media/d34d235b36b76b7a940305892feaef49/href</a></iframe><h4>Spotify’s API</h4><p>Spotify breaks down their audio data into mainly two categories. The first is comprised of 11 high dimensional audio data titled <em>Audio Features</em>. The second is a more in depth lower dimensional analysis of the audio titled <em>Audio Analysis</em>. Here’s the break down of both:</p><p><strong>AUDIO FEATURES:</strong></p><p>{ <br> &quot;danceability&quot;: <br> &quot;energy&quot;: <br> &quot;key&quot;: <br> &quot;loudness&quot;: <br> &quot;mode&quot;: <br> &quot;speechiness&quot;: <br> &quot;acousticness&quot;: <br> &quot;instrumentalness&quot;: <br> &quot;liveness&quot;: <br> &quot;valence&quot;: <br> &quot;tempo&quot;:</p><p>}</p><p>These are described, by Spotify, as:</p><ol><li>Acousticness:</li></ol><blockquote>A confidence measure from 0.0 to 1.0 of whether the track is acoustic. 1.0 represents high confidence the track is acoustic.</blockquote><p>2. Danceability:</p><blockquote>Danceability describes how suitable a track is for dancing based on a combination of musical elements including tempo, rhythm stability, beat strength, and overall regularity. A value of 0.0 is least danceable and 1.0 is most danceable.</blockquote><p>3. Key:</p><blockquote>The key the track is in. Integers map to pitches using standard Pitch Class notation . E.g. 0 = C, 1 = C♯/D♭, 2 = D, and so on.</blockquote><p><em>Quick note: Pitch Class Notation is a 21st Century method of theoretical music analysis, departing from diatonic and functional harmonic analysis from earlier methods of analyzing music which used Roman Numerals to label progression and harmonic form of a piece in regards to harmonic consonance. Pitch Class abstracts the pitch elements into numerical sets removed from octave and spectral placement in order to describe the underlying intervalic relation of pitches in a musical body rather than describe the functional consonance of a pitch’s harmonic relation to its chordal context in a musical body. In Pitch Class Notation:</em></p><blockquote>C = 0, C#/Db = 1, D = 2, D#/Eb = 3, E =4,</blockquote><blockquote>F = 5, F#/Gb = 6, G = 7, G#/Ab = 8</blockquote><blockquote>A = 9, A#/Bb = 10, B =11</blockquote><p><em>In Pitch Class, rather than </em>C Major, Roman Numeral<em> </em>I = root major triad: [C, E,G] — <em>C Major, a major chord = </em>[0,4,7]<em>. All major chords = </em>[0,4,7]<em>, and </em>[0,4,7] <em>explicitly expresses the intervalic relation of M3,m3. The importance and value of Pitch Class is better explained </em><a href="https://en.wikipedia.org/wiki/Set_theory_(music)"><em>here</em></a><em>. But to sum it up, Pitch Class reveals the underlying pitch relation, thus giving insight into the fundamental identity of body of music</em>.</p><p>4. Loudness:</p><blockquote>The overall loudness of a track in decibels (dB). Loudness values are averaged across the entire track and are useful for comparing relative loudness of tracks. Loudness is the quality of a sound that is the primary psychological correlate of physical strength (amplitude). Values typical range between -60 and 0 db.</blockquote><p>5. Mode:</p><blockquote>Mode indicates the modality (major or minor) of a track, the type of scale from which its melodic content is derived. Major is represented by 1 and minor is 0.</blockquote><p><em>An important side note: Major and Minor modalities are related by a minor 3rd(m3). For example: C major, which has 0#/b is the same Key as A minor, otherwise known as a diatonic collection. In terms of Pitch Class that is to say 0 major is the same as 9 minor. This is important to note because if a musical body is intentionally designed to function in a Minor modality, it will use all the elements of its related Major mode, but be arranged in a way that will affect the the emotional perception of the musical body and cause the listener to perceive the body as different than a musical body designed to function in a Major modality of a diatonic collection. This functionality is a produce of Harmonic structure. There are actually seven modes to each diatonic collection, but we can leave that discussion to another time.</em></p><p>6. Speechiness:</p><blockquote>Speechiness detects the presence of spoken words in a track. The more exclusively speech-like the recording (e.g. talk show, audio book, poetry), the closer to 1.0 the attribute value. Values above 0.66 describe tracks that are probably made entirely of spoken words. Values between 0.33 and 0.66 describe tracks that may contain both music and speech, either in sections or layered, including such cases as rap music. Values below 0.33 most likely represent music and other non-speech-like tracks.</blockquote><p>7. Insturmentalness:</p><blockquote>Predicts whether a track contains no vocals. “Ooh” and “aah” sounds are treated as instrumental in this context. Rap or spoken word tracks are clearly “vocal”. The closer the instrumentalness value is to 1.0, the greater likelihood the track contains no vocal content. Values above 0.5 are intended to represent instrumental tracks, but confidence is higher as the value approaches 1.0.</blockquote><p>8. Energy:</p><blockquote>Energy is a measure from 0.0 to 1.0 and represents a perceptual measure of intensity and activity. Typically, energetic tracks feel fast, loud, and noisy. For example, death metal has high energy, while a Bach prelude scores low on the scale. Perceptual features contributing to this attribute include dynamic range, perceived loudness, timbre, onset rate, and general entropy.</blockquote><p>9. Liveness:</p><blockquote>Detects the presence of an audience in the recording. Higher liveness values represent an increased probability that the track was performed live. A value above 0.8 provides strong likelihood that the track is live.</blockquote><p>10. Tempo:</p><blockquote>The overall estimated tempo of a track in beats per minute (BPM). In musical terminology, tempo is the speed or pace of a given piece and derives directly from the average beat duration.</blockquote><p>11. Valence:</p><blockquote>A measure from 0.0 to 1.0 describing the musical positiveness conveyed by a track. Tracks with high valence sound more positive (e.g. happy, cheerful, euphoric), while tracks with low valence sound more negative (e.g. sad, depressed, angry).</blockquote><p>Founded June, 2005 to be acquired by Spotify API in 2014, Echo Nest released this statement on March 29, 2016:</p><blockquote>We’re always working to make improvements to our Web APIs and today, we have some updates to share that will impact both The Spotify and The Echo Nest Web APIs.</blockquote><blockquote>Over the past two years, following Spotify’s acquisition of The Echo Nest, it’s become clear that rather than have two separate APIs with considerable overlap in features, it makes sense to migrate functionality and focus all of our future development on the Spotify API.</blockquote><blockquote>… As of today, We are announcing three new APIs (details below) and The Echo Nest API will no longer be issuing new API keys</blockquote><blockquote>… The Echo Nest platform will no longer serve requests and developers will need to move over to the Spotify API…</blockquote><blockquote>As part of our migration of many of The Echo Nest API features over to the <a href="https://developer.spotify.com/web-api">Spotify Web API</a>, we’re announcing three new APIs today. The new <a href="https://developer.spotify.com/web-api">Spotify Web APIs</a> ..</blockquote><blockquote>If you currently have an app that uses The Echo Nest API, check out the <a href="https://developer.spotify.com/web-api">Spotify Web API</a> as we’ve already migrated the most-used features….</blockquote><p>AUDIO_ANALYSIS:</p><p>It’s rather long, but seems promising.</p><pre>{<br>  &quot;bars&quot;: [<br>    {<br>      &quot;start&quot;: 251.98282,<br>      &quot;duration&quot;: 0.29765,<br>      &quot;confidence&quot;: 0.652<br>    }<br>  ],<br>  &quot;beats&quot;: [<br>    {<br>      &quot;start&quot;: 251.98282,<br>      &quot;duration&quot;: 0.29765,<br>      &quot;confidence&quot;: 0.652<br>    }<br>  ],<br>  &quot;sections&quot;: [<br>    {<br>      &quot;start&quot;: 237.02356,<br>      &quot;duration&quot;: 18.32542,<br>      &quot;confidence&quot;: 1,<br>      &quot;loudness&quot;: -20.074,<br>      &quot;tempo&quot;: 98.253,<br>      &quot;tempo_confidence&quot;: 0.767,<br>      &quot;key&quot;: 5,<br>      &quot;key_confidence&quot;: 0.327,<br>      &quot;mode&quot;: 1,<br>      &quot;mode_confidence&quot;: 0.566,<br>      &quot;time_signature&quot;: 4,<br>      &quot;time_signature_confidence&quot;: 1<br>    }<br>  ],<br>  &quot;segments&quot;: [<br>    {<br>      &quot;start&quot;: 252.15601,<br>      &quot;duration&quot;: 3.19297,<br>      &quot;confidence&quot;: 0.522,<br>      &quot;loudness_start&quot;: -23.356,<br>      &quot;loudness_max_time&quot;: 0.06971,<br>      &quot;loudness_max&quot;: -18.121,<br>      &quot;loudness_end&quot;: -60,<br>      &quot;pitches&quot;: [<br>        0.709,<br>        0.092,<br>        0.196,<br>        0.084,<br>        0.352,<br>        0.134,<br>        0.161,<br>        1,<br>        0.17,<br>        0.161,<br>        0.211,<br>        0.15<br>      ],<br>      &quot;timbre&quot;: [<br>        23.312,<br>        -7.374,<br>        -45.719,<br>        294.874,<br>        51.869,<br>        -79.384,<br>        -89.048,<br>        143.322,<br>        -4.676,<br>        -51.303,<br>        -33.274,<br>        -19.037<br>      ]<br>    }<br>  ],<br>  &quot;tatums&quot;: [<br>    {<br>      &quot;start&quot;: 251.98282,<br>      &quot;duration&quot;: 0.29765,<br>      &quot;confidence&quot;: 0.652<br>    }<br>  ],<br>  &quot;track&quot;: {<br>    &quot;duration&quot;: 255.34898,<br>    &quot;sample_md5&quot;: &quot;&quot;,<br>    &quot;offset_seconds&quot;: 0,<br>    &quot;window_seconds&quot;: 0,<br>    &quot;analysis_sample_rate&quot;: 22050,<br>    &quot;analysis_channels&quot;: 1,<br>    &quot;end_of_fade_in&quot;: 0,<br>    &quot;start_of_fade_out&quot;: 251.73333,<br>    &quot;loudness&quot;: -11.84,<br>    &quot;tempo&quot;: 98.002,<br>    &quot;tempo_confidence&quot;: 0.423,<br>    &quot;time_signature&quot;: 4,<br>    &quot;time_signature_confidence&quot;: 1,<br>    &quot;key&quot;: 5,<br>    &quot;key_confidence&quot;: 0.36,<br>    &quot;mode&quot;: 0,<br>    &quot;mode_confidence&quot;: 0.414,<br>    &quot;codestring&quot;: &quot;eJxVnAmS5DgOBL-ST-B9_P9j4x7M6qoxW9tpsZQSCeI...&quot;,<br>    &quot;code_version&quot;: 3.15,<br>    &quot;echoprintstring&quot;: &quot;eJzlvQmSHDmStHslxw4cB-v9j_A-tahhVKV0IH9...&quot;,<br>    &quot;echoprint_version&quot;: 4.12,<br>    &quot;synchstring&quot;: &quot;eJx1mIlx7ToORFNRCCK455_YoE9Dtt-vmrKsK3EBsTY...&quot;,<br>    &quot;synch_version&quot;: 1,<br>    &quot;rhythmstring&quot;: &quot;eJyNXAmOLT2r28pZQuZh_xv7g21Iqu_3pCd160xV...&quot;,<br>    &quot;rhythm_version&quot;: 1<br>  }<br>}</pre><p>Spotify describes as:</p><ol><li>Bars:</li></ol><blockquote>The time intervals of the bars throughout the track. A bar (or measure) is a segment of time defined as a given number of beats. Bar offsets also indicate downbeats, the first beat of the measure</blockquote><p>2. Sections:</p><blockquote>Sections are defined by large variations in rhythm or timbre, e.g. chorus, verse, bridge, guitar solo, etc. Each section contains its own descriptions of tempo, key, mode, time_signature, and loudness</blockquote><p>3. Section Key:</p><blockquote>The estimated overall key of the section. The values in this field ranging from 0 to 11 mapping to pitches using standard Pitch Class notation (E.g. 0 = C, 1 = C♯/D♭, 2 = D, and so on). If no key was detected, the value is -1.</blockquote><p>3. Segments:</p><blockquote>Audio segments attempts to subdivide a song into many segments, with each segment containing a roughly consistent sound throughout its duration.</blockquote><p>Below is the call to the Spotify API for Audio Features and Audio Analysis for Erik Satie’s Gymnopedie №1.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1dd547c0cd081f116c3f8c9f6068285a/href">https://medium.com/media/1dd547c0cd081f116c3f8c9f6068285a/href</a></iframe><p>RETURNS:</p><pre>&#39;{&quot;access_token&quot;:&quot;BQA0Uo0ftYE6McrLxtfuIXWF-lV5Qb2lRPLGbviO4XU3j4pik2Q1naeEaGIWX6evN1K_SQx4hj4AMP223Y4&quot;,&quot;token_type&quot;:&quot;Bearer&quot;,&quot;expires_in&quot;:3600,&quot;scope&quot;:&quot;&quot;}&#39;</pre><blockquote>*Note the token shown is inactive.</blockquote><p>The token is only active for 3600 seconds. One hour.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9d1e50a91a6c5d3b80bb557158bcf371/href">https://medium.com/media/9d1e50a91a6c5d3b80bb557158bcf371/href</a></iframe><p>RETURNS</p><pre>{&#39;danceability&#39;: 0.469,<br> &#39;energy&#39;: 0.0128,<br> &#39;key&#39;: 7,<br> &#39;loudness&#39;: -36.856,<br> &#39;mode&#39;: 1,<br> &#39;speechiness&#39;: 0.119,<br> &#39;acousticness&#39;: 0.994,<br> &#39;instrumentalness&#39;: 0.937,<br> &#39;liveness&#39;: 0.0941,<br> &#39;valence&#39;: 0.354,<br> &#39;tempo&#39;: 72.765,<br> &#39;type&#39;: &#39;audio_features&#39;,<br> &#39;id&#39;: &#39;5NGtFXVpXSvwunEIGeviY3&#39;,<br> &#39;uri&#39;: &#39;spotify:track:5NGtFXVpXSvwunEIGeviY3&#39;,<br> &#39;track_href&#39;: &#39;https://api.spotify.com/v1/tracks/5NGtFXVpXSvwunEIGeviY3&#39;,<br> &#39;analysis_url&#39;: &#39;https://api.spotify.com/v1/audio-analysis/5NGtFXVpXSvwunEIGeviY3&#39;,<br> &#39;duration_ms&#39;: 205867,<br> &#39;time_signature&#39;: 3}</pre><blockquote>Main take aways: Mode : 1 = ‘Major’ and Tempo = 72.765. It is also interesting the to note the Valence = 0.354 along with Insturmentalness = 0.937, Acousticeness = 0.994 and Time_Signature = 3. These are all very accurate. But the Key = 7 caught my attention.</blockquote><p><em>Gymnopedie</em> <em>№1 </em>is in D major. This is common knowledge to the repertoire. That is to say that <em>Gymnopedie</em> <em>№1 </em>should have Key = 3. D major = 3. 7 = G major. While classifying this body of music as Key = 7 is inaccurate, it is understandable. The piece begins on a G Major chord. G and D share a <em>Perfect 5th/Perfect 4th</em> inverse intervalic relationship. Whether in either Key, both G and D are Major Chords. The only harmonic change is the C# until measure 17. It makes sense that this could be heard as I — V in G major or IV — I in D major.</p><p>Below is an example of the floating chord structure for the first <em>17 bars</em>. The <em>17th</em> and <em>18th</em> are a subtle transition technique using secondary dominance as a cadence leading to melodic change. The melody remaining in the ambient diatonic tones. The melody is a fleeting yet stable flow of <em>8 quarter notes </em>moving mostly stepwise. Notice how the <em>F#</em> is the most consistent note for these measures. Beginning a piece on a <em>IV7maj</em> and sustaining the <em>Maj7 </em>of the<em> IV7maj, but using the 3rd of diatonic scale as a continious sound Maj7 of the IV7maj </em>is a clever technique to obscure sonority. Below is a <em>Roman Numeral Analysis</em> of the piece.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/964/1*82GHQlMlGzLxBHd0LSkGdQ.png" /><figcaption>Roman Numeral Analysis of first 18 bars</figcaption></figure><blockquote>When you call for the audio analysis it looks likes this:</blockquote><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4f85e9a87656020136c4f136785f5bcd/href">https://medium.com/media/4f85e9a87656020136c4f136785f5bcd/href</a></iframe><p>PARTIAL RETURN:</p><p><em>(there were 7032 lines of code)</em></p><p>These 60 lines are only .87751ms of time.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7b7eae9198b4dabc927a2600a7ce0b4e/href">https://medium.com/media/7b7eae9198b4dabc927a2600a7ce0b4e/href</a></iframe><p><strong>ROMAN NUMERAL ANALYSIS</strong></p><p>Roman Numeral Analysis is simple. In the most basic sense, its a way of denoting simple harmonic entities which define the Key. It is a way of compartmentalizing harmonic structure and relating the compartment throughout a work.</p><p>In Dmaj:</p><p>D : I</p><p>E : ii</p><p>F#: iii</p><p>G: IV</p><p>A: V</p><p>B: vi</p><p>C#: vii/o</p><p>The Roman Numerals describe three basic things.</p><ol><li>The pitches in the chord.</li><li>The Mode of the chord.</li><li>The function of the chord.</li></ol><p><em>Quick note — For example: I is the first chord of the diatonic collection spaced by tonal 3rds. The pattern becomes M3m3 for I, making I a major chord spelt as C,E,G. ii is a minor chord spelt as D, F, A. As for the principles of functional harmony in regards to diatonic collections, I’ll explain in another blog. For now, all you’ll need to know is that chords have relations based on intervals and that relation is functional in shaping the harmonic outline of a musical body.</em></p><p>A generalized harmonic outline of <em>Gymnopedie</em> <em>№1 </em>should be shaped like so:</p><p><em>IV7.maj — I7.maj — IV7.maj — I7.maj — iii.min — vi.min — ii.min — v.min — ii.min — v.min — ii.min — V7.dom — i.min</em></p><p>In order to analyze the harmony with Roman Numerals, I made a dictionary of diatonic collections. The dictionary can take in Spotify Audio Analysis and return the proper Roman Numerals. The dictionary is responsive to Major and Minor Mode, Key of Track and Key of Section.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e771013f9b025e23969023917de7529b/href">https://medium.com/media/e771013f9b025e23969023917de7529b/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/290c8e52cc82f3c1abfeb29c3cd3a330/href">https://medium.com/media/290c8e52cc82f3c1abfeb29c3cd3a330/href</a></iframe><p>Spotify breaks this track up into 9 sections. This is inaccurate to me. Just by listening, I’d say there are closer to 16 sections.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SwKG5Qkfk-A8OMq5eNOtiA.png" /><figcaption><em>Gymnopedie</em> <em>№1 divded into 16 sections by ear.</em></figcaption></figure><p>Taking Spotify’s 9 sections, I passed Track Mode, Track Key and Harmonic Progression through the function below. Harmonic Progression I engineered using Track Key, Track Mode and Key of Sections. As we discussed earlier, Spotify’s data tells the function to retrieve a Roman Numeral Analysis based on G Major.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/763deb2750facaee4f3735f89ed292f0/href">https://medium.com/media/763deb2750facaee4f3735f89ed292f0/href</a></iframe><p>RETURN:</p><pre>[&#39;I&#39;, &#39;I&#39;, &#39;ii&#39;, &#39;I&#39;, &#39;V&#39;, &#39;V&#39;, &#39;V&#39;, &#39;vi&#39;, &#39;ii&#39;]</pre><p>This is close, but not accurate. This suggests the body has a tonal sonority of G Major and has a generalized harmonic outline of G, D, A before repeating. The shape is admittedly close. However it is a vague resemblance compared to the accuracy of detecting Tempo. Also, this is referring to the wrong key. If we manually pass the correct Key, D Major, through the function, something interesting happens and the accuracy lessens. While the accuracy is still off, the harmonic shape appears closer to the proper spelling.</p><p>RETURN:</p><pre>[&#39;IV&#39;, &#39;IV&#39;, &#39;V&#39;, &#39;IV&#39;, &#39;I&#39;, &#39;I&#39;, &#39;I&#39;, &#39;ii&#39;, &#39;V&#39;]</pre><p>The intervalic relationship stays the same, but it is evident that the Audio Analysis is in general interpreting the track incorrectly by the distance of a <em>Perfect 4th</em>.</p><p>To further coerce the Analysis into a more acceptable return, I manually made a list of 9 modes that better delineated the functionality of the harmonic struture to accompany the interpreted intervalic relations. First in G Major.</p><p>RETURN:</p><pre>[&#39;i&#39;, &#39;i&#39;, &#39;ii/o&#39;, &#39;i&#39;, &#39;v&#39;, &#39;v&#39;, &#39;v&#39;, &#39;vi&#39;, &#39;ii/o&#39;]</pre><p>This is more accurate in interval and addresses the Minor Modality of the body, but overall is still wrong. Now let us run the function with D Major and the curated list of 9 modes.</p><p>RETURN:</p><pre>[&#39;iv&#39;, &#39;iv&#39;, &#39;v&#39;, &#39;iv&#39;, &#39;i&#39;, &#39;i&#39;, &#39;i&#39;, &#39;ii/o&#39;, &#39;v&#39;]</pre><p>Here we retrieve a Roman Numeral Analysis that better represents the functional structure of the harmony. By knowing the inaccuracy of the Spotify Audio Analysis and curating the modal progression necessary, I was able to coerce a return with the correct interpretation of Key and Chord Progression.</p><blockquote>There is still an inaccuracy in Mode, but is steps closer than the original in form. It seems that Spotify’s use of Echo Nest can, with relative accuracy, interpret the audio signal of the file, but then inaccurately casts an interpretation of Key and subsequently, Harmonic Progression.</blockquote><p>I never had the chance to use Echo Nest, but I wanted to compare Librosa to Spotify’s Audio Analysis.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3cfc49a6713f14248c4219c3d5313b71/href">https://medium.com/media/3cfc49a6713f14248c4219c3d5313b71/href</a></iframe><p>Below is a run through of analyzing pitch and harmonic form using Librosa. There is more explanation of these methods in my last blog, <strong><em>LibRosa: Data Driven Music Analysis</em></strong>. For this post, I’m going to show the coding steps, the graphs and only make brief comments. But the point is to show that, in my opinion, Librosa is more accurate than Spotify at gauging an audio signal as well as its Harmonic context.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/28c6e911297b50f26bbc2d26c54334bb/href">https://medium.com/media/28c6e911297b50f26bbc2d26c54334bb/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u7y9KRRazY7LH-6mUwngxQ.png" /><figcaption>Entire body.</figcaption></figure><p>You can see here that the pitches gravitate around D andG until 50 seconds in, where E and A become more prevalent. Then around 1min40s the preceding pattern repeats. You can see it repeats for until roughly minute 3 before another section occurs and the piece ends.</p><p>Here’s a look at the midi. Note the similarity is form.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6G16YWPR4fi5XxlLUilk3Q.png" /><figcaption>midi form matching the pitch analysis usingLibrosa</figcaption></figure><p><strong>ZOOMING IN</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/87d0f01ab491a65f2fe67a174a94e3b0/href">https://medium.com/media/87d0f01ab491a65f2fe67a174a94e3b0/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/769/1*NHnztn7IhTLmQkiNhzDe-Q.png" /><figcaption>zoom in on first 15-second chunck</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/769/1*p4AlWhXSvlTLaOdYCr-3Aw.png" /><figcaption>zoom in on 15-second chunck between second 45 and 60</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/769/1*SRQOoO9L6RUP5A8fA0pgag.png" /><figcaption>zoom in on 15-second chunck between second 115 and 130</figcaption></figure><p>In the first image you can see the steady rhythmic structure starting at G and oscillating between a D and G. In the second image, you can see the shift in the musical body. Now steadily moving between E and A with a return to D and G in the third image.</p><p><strong>TAKING FORM</strong></p><p>By ear, this piece of music seems to have 16 sections. Spotify observes 9. Librosa reaches a more accurate interpretation in relation to the auditory experience and observes 12.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e96ffcd6d838dc227d02f7592865127c/href">https://medium.com/media/e96ffcd6d838dc227d02f7592865127c/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/856/1*bvMFjK6kMnXTc-Df1yS9Hw.png" /><figcaption>making bins</figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fe9bbcd1d389dd1c0461b314520017c6/href">https://medium.com/media/fe9bbcd1d389dd1c0461b314520017c6/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/856/1*4RSpyqnTg5yQRDSW-jMBTQ.png" /><figcaption>detecting beat timings</figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/58e4946e3cf9857aff003ebc73ddb20d/href">https://medium.com/media/58e4946e3cf9857aff003ebc73ddb20d/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/568/1*Bu0jhMY8XyN2rvPsUo2lCA.png" /><figcaption>mapping</figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/46bdffa4786d1122b29d91beaf88da7d/href">https://medium.com/media/46bdffa4786d1122b29d91beaf88da7d/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/568/1*q5fOPnkXaZkEE75z8GInoA.png" /><figcaption>mapping form</figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9d8a3e48442dfd36a360b2ff3cafa917/href">https://medium.com/media/9d8a3e48442dfd36a360b2ff3cafa917/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/843/1*cENlmdptw3N7TMm1B8RFVQ.png" /><figcaption>using KMeans Clustering to predict Sections</figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0cb6c60627845505005ac318559b0454/href">https://medium.com/media/0cb6c60627845505005ac318559b0454/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/856/1*xPGeAdDbj4SHxWbi1020_A.png" /><figcaption>binding beat into bound surfaces using predicted sections and graphing the model onto the wav file</figcaption></figure><p>Compared to what is observed by the ear, not too bad.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SwKG5Qkfk-A8OMq5eNOtiA.png" /></figure><p>You can see that the model captured more nuanced sections than Spotify’s API. Ultimately the ear, caught the most. What do you hear?</p><h4><strong>ENDING NOTE</strong></h4><p>While I like Spotify’s API, I would like to see more accurate analysis of pitch and harmony. This is certainly not the easiest task, but with some added methods and feature engineering of both the audio itself along with the data already provided by Spotify, retrieving an accurate pitch and harmonic description of a music body is not out of reach.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=38e984a4174b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Sound Data]]></title>
            <link>https://medium.com/@FinchMF/sound-data-f2390fd7c05?source=rss-f583b68d1c6c------2</link>
            <guid isPermaLink="false">https://medium.com/p/f2390fd7c05</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[sonification]]></category>
            <category><![CDATA[creative-coding]]></category>
            <category><![CDATA[data-sonification]]></category>
            <dc:creator><![CDATA[Matthew R Finch]]></dc:creator>
            <pubDate>Tue, 12 Nov 2019 18:04:08 GMT</pubDate>
            <atom:updated>2019-11-12T18:04:08.626Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9ZXxlwX6we246ijNNrZ4qA.jpeg" /><figcaption>an image databent via audacity</figcaption></figure><p>Visualization is the standard method of interpreting data. It is something we are all familiar with. However, it is not the only way to represent data. Sonification is a technique that can be traced back to 1666. Robert Hooke was reported by Samuel Pepys in one of his diaries,</p><blockquote>“<em>he is able to tell how many strokes a fly makes with her wings (those flies that hum in their flying) by the note that it answers to in musique during their flying</em>.”</blockquote><p>There are a number of theories on how Robert Hooke accomplished this, but he never published any of his findings.</p><p>Sonification has been present in both the aesthetic and scientific. In the early 1900’s, the optophone was created by Dr. Edmund Fournier. Black print was detected by selenium photosensors. The print would be converted in to audible output by the optophone through the sensors. In 1970, Charles Dodge used sonfication via synthesizers to express data from earth’s magnetic fields. Today, composer John Luther Adam is using natural data to compose large scale works, such as <em>Become Ocean. </em>With access to more data, there are more chances to find streams that can aesthetically express the objective through amplitude, tone, timbre, duration and repetition. In the same way that visualization uses conventions of graph type for certain information, ie. a line graph will never be used to express something categorical, as well as subjective decisions of color, line, bar or dot boldness — including composition of the graph’s components, it is consistent subjective principle which allow for object pattern recognition to emerge.</p><p>This blog is a quick survey of the ways Python deals with sound. Let’s start with with graphing a wave file.</p><h3><strong>Graphing a Wav File</strong></h3><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F711746743%26show_artwork%3Dtrue&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fmoth-wonder_m&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-000636089080-tuw3b9-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/98b175615d4d0a18c118cc912ae954a8/href">https://medium.com/media/98b175615d4d0a18c118cc912ae954a8/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ef9a6860cccf5069e12487613557a56e/href">https://medium.com/media/ef9a6860cccf5069e12487613557a56e/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X644XywWAkfMmyYx0-WIUg.png" /><figcaption>waveform and spectrogram for wav file</figcaption></figure><h3><strong>Generating a Wav File using Python</strong></h3><p>Using a combination of Scipy, Numpy and Struct I wanted to use Python to generate a wav file rather than interpret a wav file.</p><p>With Numpy’s mathematical methods, I was able to take the sample_rate, frequency and sampling as parameters and, essentially, draw out the function into raw data via Struct and export as a wav file by the running the Python script from the command line. Scipy’s signal library allowed me to do the same thing but with functions not inherent to Numpy, namely: the Square waves and the Sawtooth wave.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/623ab7e9b12d5cd201d05e71a031bff7/href">https://medium.com/media/623ab7e9b12d5cd201d05e71a031bff7/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*n2-KLywLmJMuBskC3SHDIA.png" /><figcaption>sine_wave wav file</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AAKAMhT4MW_j1x9wDk7LGA.png" /><figcaption>sine_wave wav file (closer)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9UGPphC8j1Y_6_whDFYMUA.png" /><figcaption>sine_wave file (closer)</figcaption></figure><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F711758563%26show_artwork%3Dtrue&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fsine_test&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-000636104035-wrnm6s-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/88e3e76f055d051f18f3a860d73a25e3/href">https://medium.com/media/88e3e76f055d051f18f3a860d73a25e3/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9f0c1800223bf9befc0bdc0b1862b97b/href">https://medium.com/media/9f0c1800223bf9befc0bdc0b1862b97b/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_yJPbsx9Gau52KoEHLCGQQ.png" /><figcaption>square_wave wav file</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3En1FratD9IfHFdpSOY9Ow.png" /><figcaption>square_wave wav file (closer)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vKc9av0lJa5_vPQx8PQWVg.png" /><figcaption>square_wave wav file (closer)</figcaption></figure><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F711759511%26show_artwork%3Dtrue&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fsquare_test&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-000636104797-xdq6qf-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/40c5aec24efc869b106293ba332aef64/href">https://medium.com/media/40c5aec24efc869b106293ba332aef64/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cdef18f91296654efa6647fb2fb140d7/href">https://medium.com/media/cdef18f91296654efa6647fb2fb140d7/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M2f_wfwJZyG8psORgLSWxQ.png" /><figcaption>sqaure_wave (duty cycle) wav file</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0ZYs10CNIslCM7jeAaKTKQ.png" /><figcaption>square_wave (duty cycle) wav file (closer)</figcaption></figure><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F711760624%26show_artwork%3Dtrue&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fsquare-duty-test&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-000636105664-m40vo8-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/4d634816c370d33a03ae2c12593bb5c9/href">https://medium.com/media/4d634816c370d33a03ae2c12593bb5c9/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/46c8e2c1b109d21a1545e5be35abc9f9/href">https://medium.com/media/46c8e2c1b109d21a1545e5be35abc9f9/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XWCXlTL8ysGFwyX1_84GBQ.png" /><figcaption>sawtooth_wave wav file</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1dWqfOiiRt0jn0-J-d-a1A.png" /><figcaption>sawtooth_wave wav file (closer)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2Prv6XqblJCqtAPv9CORRw.png" /><figcaption>sawtooth_wave wav file (closer)</figcaption></figure><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F711761947%26show_artwork%3Dtrue&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fsawtooth-test&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-000636106825-4jpepn-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/b6c2b0b9f4903653f6e517b8c7f3e586/href">https://medium.com/media/b6c2b0b9f4903653f6e517b8c7f3e586/href</a></iframe><h3><strong>Sonification of Weather Data</strong></h3><p>After realizing that data produced by functions could be translated into wav files and generate a single pitch, I began to wonder what other data could be translated into series of pitches. Through the use of Dark Sky’s API, I made 3 calls to collect weather data on New York, Chicago and LA.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/537da5aa4028ad1de401091d6a1c22da/href">https://medium.com/media/537da5aa4028ad1de401091d6a1c22da/href</a></iframe><h4><strong>Translating to Midi</strong></h4><p>Once I’d collected and parsed 48 instances of recorded data (48 hours from November 11th, 2019 at 4:00pm) I needed to convert the data into something that could be read by MIDI (Music Insturment Digital Interface) and converted to a time series.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/35042ecb60c70677be48e5aa95f4f703/href">https://medium.com/media/35042ecb60c70677be48e5aa95f4f703/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/94afb4424d881aa9662333d993ef19ef/href">https://medium.com/media/94afb4424d881aa9662333d993ef19ef/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f30f7d107250949c8cb30930f098e8fe/href">https://medium.com/media/f30f7d107250949c8cb30930f098e8fe/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/22672d90a2a259745ef64647a1294ead/href">https://medium.com/media/22672d90a2a259745ef64647a1294ead/href</a></iframe><p>Below is the resultant list of lists for each element of weather from a 48 forecast starting in New York. Each list a description of the each data point as a pitch, or MIDI point. Using MIDItime library, one of a handful of libraries for MIDI conversion, I was able to translate the data points into MIDI data points.</p><p>Each list is formated to these paramaters:</p><pre>[beat placement, pitch placement, velocity, duration]</pre><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9d24fc20bdb2f7a49893e4fa19f263a8/href">https://medium.com/media/9d24fc20bdb2f7a49893e4fa19f263a8/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/649e8f82136872a37850cfea4eb6df22/href">https://medium.com/media/649e8f82136872a37850cfea4eb6df22/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/23348c63008e5e580cd6d27cd0d91c6b/href">https://medium.com/media/23348c63008e5e580cd6d27cd0d91c6b/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cef707b5863dc50d25d80e2f9e7bca94/href">https://medium.com/media/cef707b5863dc50d25d80e2f9e7bca94/href</a></iframe><p>After each file was saved, individually, I made a collected file housing each time series to see how they occurred together along the piano roll.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kZbOV0jzcthInhM_VT_zcg.png" /><figcaption>NYC weather data mapped into one midi file</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3f3T6tvPVRw-mLLtebTyVA.png" /><figcaption>NYC weather data mapped into one midi track</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HgcutQbhzFA8FZYOabAqGw.png" /><figcaption>NYC weather data mapped into separate midi tracks</figcaption></figure><h3><strong>Sonic Results</strong></h3><p>Below are two examples of the same data used in separate methods of expression. The first example is using the western convention of chromatic scales, incorporating all intervals. The purpose was to generate an objective as possible mapping process. However, the sound, to me, is rather unpleasing and reminiscent of 12-tone serial music from the early 1900&#39;s from the Second Viennese School of composition. This is an instance of using the wrong compositional conventions to sonifiy the data. My next go in this method will be to isolate the data in functional harmonic systems, taking advantage of the natural functional relationships in harmonic series.</p><p>The second example is more aesthetic than it is scientific, but it was fun none the less. Using subjective abstracting processes, where I let the data inform how I recorded some samples, programmed beats and layered two types of wav files (piano and guitar) that I recorded. Then I made that recording into sampler instrument and used the weather data to play the sampler.</p><p>What patterns do you hear?</p><p>1.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F711706288%26show_artwork%3Dtrue&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fweather_data_nyc_chro_191111&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-000636044617-7d0nwd-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/13f618ff2e18512d638c9420814126f3/href">https://medium.com/media/13f618ff2e18512d638c9420814126f3/href</a></iframe><p>2.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttps%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F711696397%26show_artwork%3Dtrue&amp;url=https%3A%2F%2Fsoundcloud.com%2Fstudio-night-at-grbc%2Fnyc_weather_data_191111&amp;image=https%3A%2F%2Fi1.sndcdn.com%2Fartworks-000636036547-waln48-t500x500.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/e472e9462fbce20e9e5e242fa59881e9/href">https://medium.com/media/e472e9462fbce20e9e5e242fa59881e9/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f2390fd7c05" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Functional Creativity]]></title>
            <link>https://medium.com/@FinchMF/functional-creativity-9289ae8bf6c3?source=rss-f583b68d1c6c------2</link>
            <guid isPermaLink="false">https://medium.com/p/9289ae8bf6c3</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[killer-rabbits]]></category>
            <category><![CDATA[creativity]]></category>
            <category><![CDATA[terminal]]></category>
            <category><![CDATA[unittest]]></category>
            <dc:creator><![CDATA[Matthew R Finch]]></dc:creator>
            <pubDate>Wed, 23 Oct 2019 06:42:47 GMT</pubDate>
            <atom:updated>2019-10-23T07:01:15.706Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*EMZ6wPwtXURNm9duciIbzQ.png" /></figure><p>For the longest time — for me — creativity has been framed with words like spontaneous, uninhibited and natural. For the most part, creativity was a method of process. A method of imagination towards an infinite end. No real consequence if the final product was never really finished and could be philosophically wrapped in an aesthetic of ‘nothingness’. Creativity — for me — was an engagement similar to the endless fractals of Jackson Pollock’s action paintings; or the meditative routine of Agnes Martin’s mark making. It was for its own sake and fundamentally an exercise in the act of creating. But who was that actually for?</p><p>Ever since I’ve been working with Python to analyze data I’ve found myself in an interesting mindset. First, creativity became more than an exercise. It became a method of problem solving; one that can serve as a solution to others in addition to practice for the self. Secondly, Python engaged creativity as a function with an end: to systematically develop a process which provides insights. The method to solutions and recognition of trends is a fascinating art when it’s designed accurately. But how can we assure an accurate design?</p><p>This blog is a brief introduction to a method of testing Python code by creating a test module with unittest.</p><p><strong>WHAT IS UNITTEST?</strong></p><p>Simply put: unittest is a module in the standard library of Python. Using it parallel with a script you’re working on can help you create reliable, flexible and intentional code.</p><p><a href="https://docs.python.org/3/library/unittest.html">Here is a link to Python unittest documentation.</a></p><p>Below is a set of elementary functions, followed by a unittest module designed to test the accuracy of the functions.</p><h3>Functions</h3><p>Note: I saved the below five functions as filename: BLOG_CODE_1.py. This is important as you’ll see I’ve imported BLOG_CODE_1 along with unittest in the following module.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ae717af0d1f3f517bb61e3c4f1352135/href">https://medium.com/media/ae717af0d1f3f517bb61e3c4f1352135/href</a></iframe><h3>Test Module with unittest</h3><p>Note: I saved this module as filename: test_funct.py. This is important due to how you will run the test in your terminal. See comments in bottom of code as well as Step 1.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/68869f8cfb5f6c2c7f4567d7d4d7e7fd/href">https://medium.com/media/68869f8cfb5f6c2c7f4567d7d4d7e7fd/href</a></iframe><p>Okay — so let’s break this down.</p><p><strong>Step 1: Naming your Module.</strong></p><p>How you name the test module is crucial. Here it is named: test_funct.py. Using this naming convention allows your terminal to read the file as a test. You will find that if you name the file something else, say: funct_test.py, you’re terminal will not run the test accurately if at all, rendering no useful results. Naming convention: <em>‘test_filename.py’</em></p><p><strong>Step 2: Import</strong></p><p>Unittest is native in Python. Simply import it. Additionally, import the script you’d like to test as well. If you don’t know the name you’d like to use for your script file, then add it later.</p><p><strong>Step 3: Basic Order and Procedure</strong></p><ol><li><em>Designing the Test</em></li></ol><p>Due to the elementary nature of BLOG_CODE_1, it may not be obvious that I designed the module first. This is where the functional creativity comes in. Using comments, I outlined how I imagined the purpose for each function in the test. From there I designed how the tests would interact with the code I would write to test the accuracy of the intention behind the code. That being said, while you build your class, it’s okay to leave a placeholder in the class name where you’ll see the script’s name proceeding def (see below):</p><pre><strong>def</strong> Test[scriptname](unittest.testcase)</pre><p>This may be seem tedious, and if anyone else has some other suggestions, please comment at the end of the blog — but there’s a reasoning behind this. To me, a test is the blueprint to code. As I mentioned, with comments in your text editor, you’ll make a rough sketch how each test will shape the code’s functionality. Then begin to fill in the logic and design the intention behind each line. In unittest’s documentation, you’ll find a variety of assertions that will be the syntax of your tests. I used mostly ‘assertEqual’ due to the nature of the code I knew I would write for this example.</p><p>Imagination is key here. You will not want to limit your assertions to standard use of your code. Get that mind spinning up some possibilities of how an input could potentially break your code and be sure to account for edge cases in your assertions. If you’re building scripts for open source domains, you won’t want messages bombarding you that your script broke because the user thought it would be able handle something you didn’t think of! Additionally, this is even more important if you’re designing customizations for code specific to a product for a client. Either way — to me, this is fun the part.</p><p>2. <em>Writing the Script</em></p><p>The script should come easy at this point, like coloring in a template. However, there’s still creativity here. As I was writing the subsequent script, I found my self continuing to question how I could test the functions I was defining. Nothing too fancy surfaced, as the example for this blog didn’t call for it; but I had the test screen parallel in a right split in VSCODE the whole time, and it was a last minute addition to place <em>result_edge_2. </em>Continuing the metaphor: as you color the template, you may realize there are more sides or angles to your architecture. By having the module open along side your script, making quick redesigns or additions to your test is simple — and at times, inspiring.</p><p>3. <em>Running the Test</em></p><p>What a moment of excitement — or potential anxiety. To run the test, just open your terminal and from the directory your files are located in, run:</p><pre>python3 -m unittest filename.py</pre><p>If the test was a success you’ll see:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/882/1*35Gfsx7blZw5DsVslhswAA.png" /></figure><p>Note the number of dots equal the number of successful tests run and correspond to each function in the test case.</p><p>If the test spotted an error, you’ll receive Python’s lovely feedback. I broke the code on the last test in this example. Here is the error message. Notice that the first 4 tests ran smoothly, noted by the dots, until <em>‘E’</em> on the last test:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/888/1*FRUwCZ_lysdMTtmAfPdl8g.png" /></figure><p>There are other testing methods and I encourage you to dive into them. PyTest is another one I like.</p><p>While it may seem like just an added step to what can already be a lengthy process, testing your code will not only assure its accuracy, allow you to make it flexible and custom when needed as well as give deeper understanding to how your script is operating — more importantly, it will save so much time when debugging, and more interestingly, allow you to make further creative decisions in modulating, updating and evolving your code to newly imagined functionality as new solutions are needed — without running into errors.</p><p>One last thing. You’ll notice I’m running:</p><pre>python3 test_funct.py</pre><p>By adding:</p><pre>if__name__==&#39;__main__&#39;:<br>    unittest.main()</pre><p>at the bottom of you’re test module, you’ll be able to run your test with fewer keystrokes in the terminal.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9289ae8bf6c3" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>