To create the interactive system for Stardust I needed a tool that would allow me to track people in real time, and then project back the particle system I would create (the stars) on people’s bodies. I had a go at Processing before (for my AR Hockey project) but after discussing it with my tutors, the best option was to use openFrameworks (OF) instead: an open source toolkit that allows you to work with code in order to create interactive work.
OF comes with some examples, but to truly explore its potential you can download addons, or libraries created by the members of the OF community. These addons do specific things, like tracking bodies, for instance. You can adapt and build from these addons into whatever your project needs, and then you can put your project up on a repository (Github, for example) so others can see what you’ve done and build up on it, and so on.
The working ethos of sharing and building up from each other’s work was really helpful for me at stage. I could have never been able to start from scratch, but by using OF I had a play and tried to make sense of the addons I needed in order to materialize my idea.
First, I downloaded OF. To use it, I needed an environment to run it from. As an OS user, I got Xcode, and got the necessary installs (Macports, Homebrew) to get going with OF.
When everything is installed, it looks something like this:
I really recommend following the tutorials on the OF site and Github site. There are also a couple of video tutorials that explain step-by step how to get openFrameworks for your computer.
I then started to play around with the examples and addons. I explored the OpenCV addon, and later the ofxCV (the one wrapped by Kyle McDonald), ofxKinect, openNi, among others.
OpenCV blob detection example.
OfxKinect addon (pointcloud view).
Trying an example of OfxCV (unexpected glitch).
OpenNI. This addon uses the kinect as a depth camera and detects and tracks the body, placing a skeleton and some joints. I used this as a base to create the silhouette as an alpha mask, and place an image of stars “inside” for my screen-tests.
With these tests, then I made my own little app (project) and started making Stardust. I started by making the silhouette (using openNi, ofxCV, ofxKinect), the one that is shown on the full-scale sceen tests. Here is how the first experiments with openFrameworks for stardust turned out:
After, it was iteration after iteration, tweaking and adding code to make a particle system (flocking), make it look like my aesthetic benchmark (like stardust), and adjusting the tracking and of course, the projection!