One of the most interesting modules that I've  been studying this semester has been Artificial Intelligence.

I've been shown how to do lots of very complex, intelligent things with computers including the use of Search Trees to find a solution to a complicated problem, how to use Genetic Algorithms and Genetic Programming to 'evolve' complex solutions and how to cluster data into groups and classify labelled data.

As a revision exercise (but also just to see if I could) I decided to try and cluster my music collection into genres using K-Means and Agglomerative Hierachical clustering. I originally planned to use the actual shape of the music (through wave form analysis) as my metric to find the distance between tracks. However, I simply don't have the mathematical expertise for this job. Therefore, I've been implementing the system using Last.FM metadata and comparing tags assigned to songs.

The biggest problem I've had so far is establishing some rules and some form of distance measure using the tagging system. However, after some lengthy googling, I came across this little gem of information. Essentially, it suggests assigning each possible tag a weight based on the frequency of its use. If a tag is used 5 times and the overall number of taggings (that is assignments of a word to a track, not the number of words themselves) is 100, then the tag's weight is 1/20.

With this system in place, its really easy to start building a picture of a song's 'weight' and matching tracks. The next part was the clustering of the tracks using a Hierarchical Agglomerative approach. This basically means, taking the Euclidean value of each cluster and merging it with its closest neighbor until there is only one big cluster left. Unfortunately, my python code really doesn't want to play ball, so I've written off to the python mailing list to see if any of them can solve it. But if you are interested in seeing the program I came up with, take a look at this python code.

I'm hoping to figure out why this isn't working myself, but any input would be very valuable to me.

Share on Facebook

agglomerative · Artificial Intelligence · clustering · hierarchical · python · research · revision

Well I know its a little late, but its time for that obligatory 'reflective' and 'philosophical'  post that comes but once a year.

Firstly, the last year has seen a wonderfully diverse series of events that have been enlightening, entertaining, educational and occasionally embarassing. I've seen the coming and going of some really good friends, the success of several relationships and the failures of others.

In 2010, I managed to secure myself a sandwich year with IBM after several months of searching for a job. I applied to some really well known companies and organisations such as CERN and Intel but to no avail. Eventually in September, I was invited by IBM to attend an interview and was offered a junior programmer's position. At IBM Hursley, near Winchester in the UK. I'm scheduled to start in July this year and I'm really psyched!

I also passed the first year of my degree with a 1st! I was very excited and it gave me a great opportunity to work as a demonstrator within the Computer Science department and get some experience teaching computer science to the first years.

So what's happening this year? Well, aside from finishing my 2nd year at uni, I'll be starting my year out at IBM, having purchased a car and an appartment on my own for the first time.

New years resolutions? Keep this blog updated with a new post at least weekly and to keep my personal diary up to date too.

See you in the next week readers!

Share on Facebook

No tags