One of the most interesting modules that I've  been studying this semester has been Artificial Intelligence.

I've been shown how to do lots of very complex, intelligent things with computers including the use of Search Trees to find a solution to a complicated problem, how to use Genetic Algorithms and Genetic Programming to 'evolve' complex solutions and how to cluster data into groups and classify labelled data.

As a revision exercise (but also just to see if I could) I decided to try and cluster my music collection into genres using K-Means and Agglomerative Hierachical clustering. I originally planned to use the actual shape of the music (through wave form analysis) as my metric to find the distance between tracks. However, I simply don't have the mathematical expertise for this job. Therefore, I've been implementing the system using Last.FM metadata and comparing tags assigned to songs.

The biggest problem I've had so far is establishing some rules and some form of distance measure using the tagging system. However, after some lengthy googling, I came across this little gem of information. Essentially, it suggests assigning each possible tag a weight based on the frequency of its use. If a tag is used 5 times and the overall number of taggings (that is assignments of a word to a track, not the number of words themselves) is 100, then the tag's weight is 1/20.

With this system in place, its really easy to start building a picture of a song's 'weight' and matching tracks. The next part was the clustering of the tracks using a Hierarchical Agglomerative approach. This basically means, taking the Euclidean value of each cluster and merging it with its closest neighbor until there is only one big cluster left. Unfortunately, my python code really doesn't want to play ball, so I've written off to the python mailing list to see if any of them can solve it. But if you are interested in seeing the program I came up with, take a look at this python code.

I'm hoping to figure out why this isn't working myself, but any input would be very valuable to me.

Share on Facebook

agglomerative · Artificial Intelligence · clustering · hierarchical · python · research · revision