Animation is foundational to pretty much all computer generated special effects, the entire video game industry, and even a great deal of entertainment. Part of animation, of course, is creating effective renderings of the motions of things—and often those things happen to be human beings, or at least ape human motion. Numerous extremely expensive tools now exist to aid this sort of animation from 3D simulations of human musculature and skeletal systems that can be plotted to both 3D and 2D animated characters in order to give them more lifelike animation. However, often the best is to map it from the real thing: that’s where Kinect motion capture comes in.
We can look to extensive use of motion capture from movies such as Tron: Legacy and The Incredible Hulk—where director Luis Leterrier provided some of the gestures and movements for the motion capture to portray hulked-out Bruce Banner smashing up objects and fighting tanks, aircraft, and other hulk monsters. Motion capture takes the movements and gestures of real-live actors and translates them into the animated movements of 3D and 2D animated characters on screen, giving animators a foundation for making tweaks to human motion.
Here we have an example of the Kinect hacked to combine with a choreography tool called MikuMikuDance, which is combined into a project called Vocalaid Promotion Video Project (VPVP). The software places a big-eyes-small-mouth animé type avatar on screen and translates the motions of the user into motions for the character, developing a motion-capture system letting them choreograph their own dance moves to songs. The rest of the project also involves a music mixer and the ability to produce a somewhat eerie sounding singing voice, but we’re only looking at the motion capture right now.
Does this remind anyone else of William Gibson’s book Idoru? A story in which an entirely there exists an entirely synthetic pop start whose personality is run by an A.I. We’ve seen our own real life emergence of this phenomena in both Japan and Korea—animators produce extremely realistic animations, give them computerized voices, songs, and choreography and then let them loose with their own personas and songs.
Perhaps we’re right now seeing the next viral celebrity on YouTUBE where the ugly duckling, sitting in her dark room with her computer can become the next smash hit on the Internet with just a Kinect, a little grace and style, and a synthesizer.
We’re probably going to see a lot more really interesting use of this by amateur and even casual animators, probably cropping up on YouTUBE and other video social sites as this software becomes more prevalent. In fact, we’ve already seen shadow-puppet shows done with projectors and brought to life with the Microsoft Kinect. Also there’s a very interesting video of the Kinect being hacked with the OpenNI Motion Capture drivers for the popular 3D renderer 3ds Max (video here)