There are at least 3 different programs you could consider using: FreeTrack, FaceTrackNoIR, and OpenTrack. I’ve had at least some experience with all three, but I’ve only really used one for the most part.
You’ll want to stay away from FreeTrack outright, its an old program that hasn’t seen an update for a very long time and its got some quirks with the PS3 Eye and modern operating systems that make it prone to crashing, so it’s not worth trying to use it. This is the one I started out on and it was okay while I was still messing around with my set up and making it all work, but the constant random crashes was irritating and I moved onto a better solution.
FaceTrackNoIR is the one I personally use with my setup. I don’t recall having to do anything fancy to get the PS3 eye working with this one, but its been a while since I touched BMS so I could be wrong. Its got pretty much everything you’ll need for customizing your tracker, and its receiving updates, although kind of slowly last I remember. There’s also support for a ton of other kinds of tracking methods, in case you decide to switch in the future.
Actually, I didn’t know this until I just checked out their website, but FaceTrack is no longer free… they now charge 2.85 euros. Well, I guess its up to you to decide if it sounds worthwhile to purchase. I’m running an older version so I’m not sure exactly whats new in this version. But I can say the program does work well, and I like it.
OpenTrack is, as the name suggests, an open-source head tracking software. I’ll be honest, I have very limited use with this program, so my word may not be the most representative. From my experience in trying this quite some time ago, its pretty much an open-source version of FaceTrack; its got all the same features from what I can remember. I don’t know enough to say for sure how this differs from FaceTrack, but it is free and receiving updates at a good pace. Personally I had some issues in setting up my stuff using OpenTrack, and it just felt like my setup with FaceTrack was “better”. Plus I don’t play BMS very regularly, so it didn’t seem worth it to me to spend too much time trying to get OpenTrack working perfectly for me when I had a good enough setup with FaceTrack.
In any case, since FaceTrack is apparently no longer free, you may want to start off using OpenTrack and if you don’t like that, try out FaceTrack… or maybe see if you can find one of the older versions of FaceTrack that were free (I’m using 1.7). As a final note, I found that no matter what program I was using, it took a lot of tweaking the profile settings to get the head tracking motion to feel good.