Processing Audio with Python!
How to get rid of noises and add some cool effects.
This may sound crazy, but in 2022’s Xmas night, I decided to play around with wine glass sounds for fun (I was doing the dishes, you know?). That time, the difference was that I decided to record it and analyze it with Python. It led to the cool experiments we see here.
Audio is everywhere. We do not only listen to them but produce them a lot. How do they look and what can we do with them? That’s the kind of questions we are going to have answers for.
Disclaimer
The field of audio processing makes use of several concepts. Though important, I am not going to elaborate on them in here. I will instead point them out for the infovores out there:
- Fourier Transform/Series;
- How audio recording works;
- Impulse Response and its applications;
- Signals and Systems Analysis.
For the time being, I’ll turn to the following:
- What is an audio (from a coding perspective);
- Sine waves and the notion of spectrums;
- Filters and effects
That said, we can start this post for real. And, by the way, for those out there who want to replicate what I did here, you can pop in my Github repo. All the codes and examples are there. If you like it, please consider dropping a star ⭐. I would really appreciate it 🥹.
Picture yourself sending a voice message
Imagine you are listening to a tune that brings childhood memories back and you definitely need to share this nostalgic moment with a friend. You pick up your phone and start recording. You send them this voice message and naturally decide to listen to what was recorded. But to your surprise, the quality is not that great.
It happens a noisy whistle messed up with the recording. Dang! What should we do? If the source is now off, there is no way to record again! Or maybe the whistle will pop up again.
But wait… what if we manage to remove it someway. In the end, this is an audio file and people play around with audio all the time! But what is an audio to start with?
Ok, but what is an ‘audio’ anyway?
With Python, we can open an audio file using Scipy’s Wav utilities. Once this is done, we can see the samples that make it up.
As I said before, I’ll not dive in how a sound (mechanic wave) is converted to a digital file, but here goes an idea: this process usually involves some preprocessing to dumb the noise and highlight what we want, followed by a converter that breaks the continuous signal into short units (samples) that are stored as numbers.
A’ight, these things are cool and whatnot, but do not look really useful. How can we process audio with it? Well, we actually need a new way of looking at it.
A new visualization
Keeping in mind we need to find clues on how to get rid of the noise, the previous graph is a letdown: it does not hint anything. But luckly, some Signal Processing geeks have developed neat tools for us to use. The first one is the Fourier Transform.
In a Signal Processing course you’ll probably see the math behind it. It is nice, but a bit of out of scope here. So let’s jump to the application.
The graph above shows a 20 Hz sine wave (that is, it oscillates 20 times in a sec). We can understand sound waves as sines and vice-versa. For instance, you can listen to some in Chrome Music Lab’s Oscillators experiment. Now the trick is… let’s apply the Fourier Transform to the 20 Hz toy wave and see what happens.
The Fourier Transform modified our signal such that its frequencies are highlighted. The original signal oscillated 20 times per second, then we have a peak at 20 Hz (ignore the peak at -20 Hz, it is just a reflection).
We can visualize the frequencies that comprises our audio on the fly by extracting a spectrogram. This type of transformation applies Fourier Transform (or some other transform) on patches of audio, creating a 2D graph of the frequencies that make up our signal. You can play with it in Chrome Music Lab’s Spectrogram experiment.
This is cool because it underlines the ‘information’ contents of our signal. To elaborate on that and get some intuition of what can be done, let’s play around more with some signals. First, let’s create a wave by weighting several frequencies together. To be more specifically, we have:
Get Mateus de Assis Silva’s stories in your inbox
Join Medium for free to get updates from this writer.
resulting_wave = 1*sine_20_hz + 0.5*sine_30_hz + 0.1*sine_50_hz
Sweet! The Fourier Transform can see the individual frequencies in the input signal even when we are clueless which they are. Moreover, it notices the different sines have different weights!
Could we remove some frequencies? Actually we can, using a Notch Filter. Such filter works by attenuating a band of frequencies (other frequencies are left unchanged).
Ideally, we would like to remove only the frequency we are mad at. But the nearby are always affected too (that’s why we say the filter ‘attenuates a band of frequencies’). We can control how heavily they are altered, but can never make it perfect. The nomenclature for the band of frequencies that are modified is ‘stop band’.
I’ll remove the frequencies 30 and 50 of our toy example. The result can be seen below.
We can qualitatively compare how good is this reconstruction…
We now have a good theory to help us out. Let’s see how it fares in our real-world example.
Hands on!
Applying the same logic I introduced above, let’s first how our corrupted audio looks like upon applying our Fourier Transform to it.
Interesting… we see those twin peaks again! This implies the noise infecting our audio has a sinusoidal form. According to the zoomed-in picture on the right, such a noise signal has a frequency of 936 Hz, approximately.
The next step is to filter it. By removing the frequency of 936 Hz we have another audio, which can be listened to, and seen, below.
The first conclusion we have with such results is: the noise is not that bothersome anymore (we call it ‘atenuated’, remember?). Ok, it is still there, but compared to our previous audio, this is a good result. Visually, we notice the profile of our audio has changed. It does not have that wide volume of signal, and now it is more smooth and concentrated around the X-axis. As for the Fourier Transform, we see the peak (and its twin) around 936 has disappeared, as expected.
BONUS: Transfering effects
As a bonus, let’s apply an effect to our audio. This works by mixing two signals: our audio and the sound effect signature. Such a signature is called in the technical realm as impulse response, but this is a whole other story.
The effect I’ll be using here mimics the echo we expect to hear in a large, long hall. Here is its signature:
It might be surprising or counterintuitive to take such blast sound as a hall echo (which tends to be smoother), but this is due to the nature of the signature. An impulse response is just a system’s response to an impulse signal. Such an impulse is basically a short-term, high-intensity, signal. In other words, a blast.
The mixing process is called a convolution. I’ll not dive deeper in it, but to keep you curious, I’ll say it is the basis of a lot technologies, including how image processing works. Whatever… let’s see what does such a mixing result in:
We see the effect makes our audio thicker, and it does sound like was recorded on a large, long hall. It has to do with how the mixing is done. Next we see a side-to-side comparison.
Conclusion
This is a cool experiment, but often audio processing is done in dedicated audio editing softwares. The goal was to introduce the concepts and methodology, so, if you are not in a research project, or a college subject, or developing audio programs, I think you’re not going to deal with audio the way I did.
Anyway, I hope you liked it… my first contact with these concepts was in a college subject and I wanted to try them out on a real-world setting ever since. If you build on this work and want to share, please come back! I’ll appreciate talking about it with you! Moreover, consider talking about it in an event! I am sure the community will like it. I did the same in a Python Meetup and it was amazing! If you want to know more what we had there, you can read about it here.
See you in the next post!










