<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by victorbserra on Medium]]></title>
        <description><![CDATA[Stories by victorbserra on Medium]]></description>
        <link>https://medium.com/@locopornatureza?source=rss-4147f045df60------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*AMR73wcA2-ZuvuZgVwLmUA.jpeg</url>
            <title>Stories by victorbserra on Medium</title>
            <link>https://medium.com/@locopornatureza?source=rss-4147f045df60------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 09:28:25 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@locopornatureza/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Exploring the Intersection of Deep Learning and Music: My Journey in Creating a Chord Recognition…]]></title>
            <link>https://medium.com/@locopornatureza/exploring-the-intersection-of-deep-learning-and-music-my-journey-in-creating-a-chord-recognition-9d2f3b1c1be8?source=rss-4147f045df60------2</link>
            <guid isPermaLink="false">https://medium.com/p/9d2f3b1c1be8</guid>
            <category><![CDATA[music]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[wav2vec]]></category>
            <category><![CDATA[transformers]]></category>
            <dc:creator><![CDATA[victorbserra]]></dc:creator>
            <pubDate>Sun, 02 Jun 2024 22:49:53 GMT</pubDate>
            <atom:updated>2024-06-02T22:49:53.233Z</atom:updated>
            <content:encoded><![CDATA[<h3>Exploring the Intersection of Deep Learning and Music: My Journey in Creating a Chord Recognition Model</h3><p>Hello everyone, how are you?</p><p>I have always been an enthusiast of Machine Learning and Neural Networks since I first heard about them. In recent years, I distanced myself a bit from these advancements, focusing on my career as a Data Analyst. However, with the rise of Transformers, I dove back into studying with dedication and began applying some of these models to personal projects.</p><p>My first endeavor combines this passion for Machine Learning with another of my long-standing passions: music. Playing songs by ear is one of the biggest challenges for beginner guitarists. With this in mind, I thought of creating an AI with “perfect pitch” capable of recognizing chords played in an audio track and, from there, generating chord charts.</p><h3>Data Set Development</h3><p>The first challenge was to develop the data set. Among various ideas, the one that stood out was web scraping guitar chord websites, resulting in over 37,000 different combinations in various tunings. With this data in hand, my first idea was to transform all the chords into images and run convolutional models I had been studying.</p><p>The process of generating the data set to finalizing the first model was exhaustive, and the initial frustration was significant when I realized that the model was completely incapable of making predictions, clearly suffering from overfitting. Comparing spectrograms from external audio tracks, I noticed they were entirely different from the data in the base.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j2Blqxnso06tdxMW9N0vgA.png" /><figcaption>Original Input</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*mPXim7-MaOZ9ond8O9mn7A.png" /><figcaption>Auto-Record Input</figcaption></figure><h3>Data Augmentation and Training</h3><p>The solution was to search for various data sets to add different types of noise to the samples, performing data augmentation. With city noise, nature sounds, and various work-related noises, I increased those 37,000 combinations to over 500,000 data points. Additionally, I applied white noise randomly, doubling the data set.</p><p>With this work done, I used convolutional networks again, and the results improved significantly. I then decided to abandon building my neural network and move on to fine-tuning pre-trained models, like Facebook’s Wav2Vec 2.0, which showed promising results.</p><h3>Adjustments and Results</h3><p>To continue, I had to adapt my scripts to run on Google Colab, Cloud Functions, and Google Cloud Storage. I simplified the original model, which tried to predict 374 classes, by dividing it into several models, each with a specific function, such as predicting the tone, identifying accidents, and recognizing the chord type (5, 7, 9, 13).</p><p>After 3 epochs of training, the first model achieved 62% accuracy, reaching 78% with 8 epochs. I noticed that beyond this point, the model started to overfit, and performance on new data declined.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/647/0*WM8DL0oVVO2DLFU-" /></figure><h3>Challenges and Future Steps</h3><p>After obtaining these results, I wanted to see how the model would perform with different data compared to those captured from chord sites in MIDI format. Surprisingly, the model reasonably identified the tonic of various chords, even in voice and guitar combinations. However, several challenges arose.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/517/0*ndWQcjDczbuYaVma" /></figure><p><strong>Key Observations:</strong></p><ul><li><strong>Data Duration: </strong>The model’s input was designed for 1-second durations, often capturing more than one chord.</li><li><strong>Strumming Patterns: </strong>The model struggled with different strumming directions and complex rhythms, succeeding only with constant rhythms.</li></ul><p>Despite relative success in recognizing tones, the goal of creating a chord chart automator has not yet been achieved. I found many master’s theses dedicated to achieving less than what I am developing as a hobby, which encourages me to continue. This is not a simple task; many master’s theses focus on less than what I’m developing as a hobby.</p><p>Creating an AI with “perfect pitch” might be too demanding. For example, consider playing a G major chord and a C9 chord at the lower part of a guitar neck; the notes played are the same, which could confuse even the best models. Given this, working with chord combinations or musical theories like circle of fourths and secondary dominants might be a promising path.</p><p>I hope to return soon with a finished model and a detailed account of this journey. For now, I’m sharing some organized files and results, as shown in the screenshots, indicating that the model can predict chords from test audio files downloaded from YouTube. I acknowledge the need to become more organized throughout this process.</p><p><a href="https://github.com/victorbserra/projects/tree/main/chords-predict">Chord Project’s on Github</a></p><p>Thank you for your attention, and see you next time!</p><p>#MachineLearning, #DeepLearning, #DataScience, #AI, #NeuralNetworks, #DataScience, #MusicTech</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9d2f3b1c1be8" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>