Teaching Computers to Recognise Traditional Instruments

Have you ever heard a beautiful yet mysterious instrument in a world music piece, and had no idea what it was? While Shazam can tell you the name of any pop song in moments, we found nothing similar for identifying the incredible variety of traditional musical instruments from around the globe. We’re trying to change that, however, with some clever AI – and this is about to become reality.

When Cultural Sounds Go Unrecognised

Mostly, we can spot a piano or guitar pretty quickly but ask us to identify a Chinese Hulusi or Armenian duduk and we’re very quickly lost! This is a real blow to our global culture. As fewer and fewer people learn these traditional instruments, our ability to recognise and appreciate them fades away.

Things get even trickier when multiple instruments are played together. If you’ve ever tried to pick out individual sounds in a busy orchestral piece, you’ll know how tough this can be, even for trained ears.

The Experiment

Here at Cultural Infusion, we decided to see if artificial intelligence could accomplish what our human ears struggle with. We’ve trained computer programmes to recognise 63 traditional musical instruments from places such as Armenia, Bolivia, Brazil, China, Egypt and Germany, through the creation of 10,000 five-second audio clips, digitally mixing 2-5 instruments together.

Imagine combining a Chinese two-stringed violin with Brazilian drums and a medieval German wind instrument – it sound chaotic, right? That was exactly the point; real music is messy and layered.

How It Actually Works

We tested six different AI approaches (ie, tying different keys until one turns the lock). The winner in the end was a Convolutional Neural Network (CNN) – the same tech that helps your phone identify faces in photos, applied to sound.

We fed the system various audio clues: visual sound patterns, compressed audio “fingerprints,” and mathematical descriptions of pitch and rhythm. Essentially, we taught the computer to listen like a very analytical musician.

The Results

Best case scenario, our system got things right around 55% of the time – quite impressive when you consider the obscure instruments we’re working with, most of which are unknown to the general public. In many cases, there are many of these playing simultaneously. The system excelled especially with some German Renaissance instruments and various Chinese stringed instruments.

Why This Actually Matters

This isn’t just a cool tech demo. The UN recognises cultural diversity as something worth protecting, and this research could genuinely help in:

• Music education: Students learning about unfamiliar cultural instruments
• Cultural preservation: Documenting instruments before the knowledge disappears
• Pure curiosity: Finally getting answers to “what is that beautiful sound?”

From Research to Reality

We don’t intend for this tech to stay locked away in the vaults of academic journals. Cultural Infusion is preparing to launch our instrument recognition app very soon, bringing this groundbreaking research directly to music lovers worldwide. Soon, you’ll be able to hold up your phone to any music recording and discover the cultural treasures hidden therein.

The Bigger Picture

We have been honest about the challenges we’ve faced as there’s simply not much data on traditional instruments compared to mainstream ones. But our work offers hope that we won’t lose the incredible variety of sounds different cultures have created over centuries.

We’re honest about the primary challenge we’ve faced: there simply isn’t much data on traditional, more obscure instruments, as compared to Western mainstream ones. Our work, however, offers the hope that we won’t lost the incredible variety of sounds different cultures have created over the centuries.

As the world gets more connected, teaching computers to recognise these instruments is, in a way, teaching ourselves to value the beautiful complexity of world music.

---