focused data-mining

[cellio]

There is a service that I wish someone -- Amazon is an obvious candidate -- would provide to me, and I wonder how realistic it is. (It seems like it should be easy given the data, and they've surely got the data.)

Here's an example: I've bought three CDs from a particular artist. I really like two of them, and I don't care for the third. She has more CDs. Which (if any) am I likely to like? If I could find ratings for those CDs from people who liked the two I liked and didn't like the one I didn't like, that would be informative. Overall ratings of those products are irrelevant, because I don't know anything the raters. Reviews can be more informative but, again, I don't know how the reviewers' tastes match my own and most reviews are not well-written. User-submitted "if you like this buy that" links can be helpful but are under-used; they're also not annotated. Knowing that people also bought them isn't helpful; I bought the one I didn't like, too. I want to tap into the actual ratings (not purchase history) from a subset of raters -- the ones who match my own profile. Ratings are more reliable than the other options available because Amazon uses them to push things at you, so you're motivated to get them right.

I'm casting this in terms of Amazon because they're doing part of the problem already, but in theory this could be done by anyone who can mine the data.

Is anyone providing a service like this (with a large-enough user base to be relevant)? Does any Amazonian reading this want to run with it? :-)

Comments

[geekosaur]

Last.FM, supposedly, but you need to drink their koolaid.

[filkerdave.livejournal.com]

Who's the artist, BTW?

[xthlcm.livejournal.com]

Hmm. Well, there are a number of group-rating systems out there, but none I know of that will actually recommend particular CDs. I've become a huge Pandora junkie lately, although because I'm picky my "custom radio stations" tend to have only a small set of songs in rotation. :p

Sadly, I don't work at Amazon anymore.

[ralphmelton.livejournal.com]

I have such an artist of my own. (Nine Inch Nails. The first album was well-nigh perfect for me; the later albums have not pleased me nearly so much. I would love to find more albums like the first one.)

[dmnsqrl.livejournal.com]

Launchcast? Mind, reccomendations tend to be more on a song-by-song basis than album-by-album... but the thing is that you rate songs, artists, albums, and they make reccomendations based on things liked by people who had similar sets of ratings

I can't find it

[(Anonymous)]

On my blog (quite possibly before Christmas, back before the latest reboot (sigh), there was talk of a computer system that analyzed music based on computer analysis. For once, no humans were involved. The idea is that, being able to analyze large numbers of songs, the computer might find good stuff that's being overlooked.

As I remember, though, it wasn't ready for prime time. I also remember something about it using neural nets to do the analysis. That's actually problematic. Neural nets can find connections between things that humans don't care about. For example, a data set comparing images with and without tanks worked fine in training, but presented with new images not in either data set, the system couldn't determine if there was a tank in the picture. It seems that all the non-tank images were taken on sunny days and all the tank images on cloudy days. The computer completely missed the tanks and paid attention to sunny vs. non-sunny. Oops.

Rob of UnSpace (http://www.unspace.net/)

PS: Jupiter has a new red spot! (http://www.scienceblog.com/cms/jupiters_got_a_new_red_spot_10142.html) Wheee! Ok, so I get excited over really stupid stuff....

[gclectic.livejournal.com]

I tried writing a small essay explaining all of the reasons why this sort of collaborative filtering problem is hard in general, and got bogged down. Suffice it to say that (a) you are right that this would be really nifty; and (b) there are a lot of tricky details in creating an appropriate dataset for such a query.

What I find most interesting about your suggestion is the desire to take a broad user profile and emphasize a selected small portion of it in order to produce a query. This is, indeed, an idea that seems incredibly obvious and useful -- and is entirely new to me. Maybe someone will pick up on it and run with it.