A MadMapper Kinect-ion
Upon first hearing this request, I immediately thought of somehow using the MadMapper application that allows video to be mapped to practically any object. Luckily, MadMapper also has a companion app called Mad KinectMasker that enables a Kinect camera (yes, the same camera all you gamers use with your Xboxes) to be mapped as an input layer within MadMapper. So I proceeded to hook it all up and see how it worked and what the limitations might be for using this technology in a rather outside the norm application.
The KinectMasker application allows the camera to be connected to the same computer that MadMapper runs on, and using Syphon, the camera’s image is sent into MadMapper as an input source. This source can then be layered over an object and then, through the use of an additional content server application (MadMapper has an app called Mad GraphicPatterns, but I used ArKaos MediaMaster since it is Syphon ready), you can easily select a piece of content and overlay it with the camera’s input to create the masked layer/image combination. Was this effective? Yes, but there were a couple of issues that I feel prevent this setup of technology from being used in a professional application for now. The main thing was that my Kinect camera tended to freeze often, and the only way to make everything work again was to restart every single application that I needed to use, each time this happened. This, of course, takes too much time to do on a job site, so, in my opinion, reliability would be the biggest drawback to using a Kinect camera in a live performance.
Distance Matters
A second issue while using the Kinect camera is sensitivity range/depth of image. I found that as long as my moving person was within about 10 feet of the camera, the camera had no problems discerning the movements. However, if the person moved back to 20 feet, the camera began to lose detail, and the result was a grainy, intermittent image. This, of course, resulted in MadMapper not being able to tell where to map the video content to within the camera’s input, and the results were sloppy and stuttered.
Since an infrared camera measures the variances in heat or heat energy to create an image (called infrared radiation), there are some details to consider when using an IR camera, including distance from camera to target, camera operating temperatures, the temperature of the target, and the camera’s accuracy and sensitivity to high and low temperatures.
Some digital lighting fixtures like the DL.3 from High End Systems/Barco have a built-in IR-capable camera already onboard. As the company notes, the DL.3’s integrated Sony camera with Super HAD image sensor technology and infrared illuminator “provides live video input and output from fixture location,” and this IR illuminator sensor “makes it capable of ‘seeing’ people and other subject matter in dark environments.” This effect can be output from one DL.3 and back into another DL3 via SVideo on a layer, and then images and effects can be applied to this layer as any other layer, making this a “live feedback” image that needs no special equipment. More info at highend.com/products/digital_lighting/dl_3.asp.
Overall, IR technology does have its limitations, including the fact that it works more effectively in darker environments. However, with a little experimenting and some creative thinking, it can be used in typical performances with moderate levels of stage light as well. So, next time you’re working out in front of your Xbox Kinect camera, think about all the other ways you could be using that piece of equipment — and don’t just dance, do it!
Warm Object Mapping: Key Considerations
• Distance from Camera to Target:
If the distance is 20 feet or less from the target, then most standard lenses will work. If it’s greater than 20 feet, then a telephoto lens will be needed to capture detail.
• Camera Operating Temperatures:
Normal operating temperatures for the camera will be listed on the spec sheets, and they are important to know. These indicate the maximum and minimum temperatures at which the camera can accurately function.
• Temperature of Target:
Infrared technology uses variances in temperature to define an image. Therefore, your target must have some difference in temperature to relative to its surroundings.
• Temperature Resolution/Accuracy:
This refers to the level of accuracy of the temperature measurements being recorded by the camera. Like operating temperatures, cameras come with differing abilities to measure high and low temperatures.
• Thermal Resolution/Sensitivity:
Thermal resolution or thermal sensitivity is how close in temperature separate objects in the image can be with the camera still able to detect and measure these differences.
