Motion capture – solution to boost-up animation

Írta: Matúš Repka

Olvasóink értékelése: / 1
ElégtelenKitűnő 

Motion Capturing is a technology for digitally recording a real actor motions and getting the data set so as to use for various purposes. There are several interesting technologies available to capture the motion, most of which are based on tracking markers placed on the actor's body (but e.g. exoskeleton systems work without markers). The motion capture computer software records and calculates (2) the positions, angles, velocities, accelerations and impulses, providing an accurate digital representation of the motion.

Motion capture (AKA Mocap) used to be considered a fairly controversial tool for creating animation. In the early days, the effort required to clean up the motion capture data often took as long as if the animation was created by an animator, from scratch. Thanks to developments by the manufacturers of motion capture systems, as well as numerous software developers, motion capture has become a feasible tool for the generation of animation.

In entertainment applications this can reduce the costs of animation which otherwise requires the animator to draw each frame, or with more sophisticated software, key frames which are interpolated by the software. Motion capture saves time and creates more natural movements than manual animation, but is limited to motions that are anatomically possible. In biomechanics, sports and training, real time data can provide the necessary information to diagnose problems or suggest ways to improve performance.

How motion capturing works

Most of the motion capturing systems works on the basis one of the three principles – optical capturing, magnetic capturing and the exoskeleton systems.

Optical systems use reflective markers illuminated from strobes on the camera and triangulate each marker from its relative location. The markers  may be passive or active. Passive markers are small balls (from several millimeters to several centimeters) covered by reflective tape made of a foil covered with microscopic glass blobs; active markers are typically LED diodes. These systems typically use sensors where the camera captures an image of the scene, reduces it to bright spots and finds their centers. A high speed 4 megapixel sensor costs around $1,000 USD and can run at 640,000,000 pixels per second. This means at resolution 640 x 480 these cameras can sample at 2,000 frames per second, but then trade off spatial resolution for temporal resolution causing blurring or jitter which requires heavy filtering to correct. Typical 1.2 megapixel sensors work up to 600,000,000 pixels per second what results to 500 frames per second at 1.2 megapixel resolution. Anyway, the usual capturing speed is 100 – 120 fps.

Optical systems can be used to capture the motion in a larger space, however they are physically limited just to visible markers and this constraint cannot be always fulfilled especially for more difficult motions or when the actor just simply lies down on the floor.

Most large MC installations today are based on optical technology by Vicon and Motion Analysis companies – they generate precise noise-free data, are reliable and less prone to a external interference.

Magnetic systems calculate position and orientation by the relative magnetic flux of three orthogonal coils on both the transmitter and each receiver. The relative intensity of the voltage or current of the three coils allows these systems to calculate both range and orientation by extremely precise mapping the tracking volume. The sensor output is 6-degrees of freedom so useful results can be obtained with two-thirds the number of markers required in optical systems. This system is able to recognize the rotation of the marker what optical systems can’t, but it is limited to a smaller space than optical systems are and require special metal-free environment. Both “AC” and “DC” magnetic systems are used, but there is a distinction between them: one uses square pulses, the other uses sine wave pulse.

Magnetic MC systems are not so popular today, as the sensors on the actor's body are active, that means the use of cables.  Also, the active volume of these systems can't be as large as those of optical systems; magnetic MC data also exhibit more noise, which must be filtered out.

Exoskeleton motion capture systems directly track body joint angles. The sensors are attached to the performing actor body. As the actor moves, the articulated mechanical parts measure the performer’s relative motion. Typically, they are rigid structures of jointed, straight metal or plastic rods linked together with potentiometers that articulate at the joints of the body. The disadvantage of these systems is that they physically constrain the performing actor and as such are not suitable for all kind of motions.

The special cases

Facial motion capturing is utilized to record the complex movements in a human face, especially while speaking with emotion. This is generally performed with an optical setup using multiple cameras arranged in a hemisphere at close range, with small markers glued or taped to the actor’s face. Invisible infrared strobes are used, so actor is not disturbed by close strong light sources.

For hand fingers the mechanical capturing is a much better solution due to the fact the markers on the fingers are usually covered and cannot be viewed by cameras. This mechanical capturing is typically performed using special data gloves. Data gloves range in price from roughly $1100 per pair up to approximately $25,000 per pair. Differentiating factors include the number of joints sensed. A 5-sensor glove measures the curl of each finger but cannot differentiate the degree of movement per knuckle. Basically, the 5 sensor gloves record the curl of each finger, from open palm to closed fist. 16-sensor gloves record each finger joint and add adduction (recording how close or far apart each finger is). 22-sensor gloves measure motions of the palm, including more complex motions of the palms.

Motion capture in live

We had a unique opportunity to visit the motion capture studio. The Vicon Motion Capture system that we visited is operated by a Czech game development studio known as Bohemia Interactive. This studio focuses on developing computer entertainment software and the research of advanced real time 3D graphics, artificial intelligence and physical simulation technologies for real-time interactive environments. Their base is situated in the countryside nearby Prague; in a purpose built house by the forest with offices and a 15x15x5 metre hall inside.

As the motion capture engineer Št?pán Kment explained to us, inside this hall the 15 cameras (infrared and classic) are installed to cover cca 7x7 meters area. Bohemia Interactive uses 1MPixel high-speed cameras (1000x1000 pixels resolution – there are up to 4MPixel cameras available today). High frame rate cameras are one of the key factors for motion capturing. The higher speed you capture the markers, the better output you get. When you use faster camera, the differences of the marker positions between the shots are smaller, it means less computation and smoother trajectories in the output. However, the hardware capabilities of the cameras have their limitations thus very high frame rates usually reduce the output image resolution. “We use 120fps for our purposes. We found this rate high enough for good capturing balanced with high output image resolution,” Št?pán Kment stated.

Inside the mocap hall

Bohemia Interactive uses a B&W capturing system so that the captured image is assembled based on the crossings of the bright threshold – if the brightness of the point at image exceeds the threshold its represented by a white color pixel, otherwise it stays black. These days grayscale capturing systems are also available, their advantage being the reduced jittering usually caused by the low brightness of the edge marker points, which are considered sometimes under and sometimes above the threshold in B&W systems.

The captured image from the cameras is transferred into channel concentrators and then to Vicon proprietary cards including channel A/D converter, where the analog image is turned into a digital representation, processed and saved.

Even though motion capture is quite complex and a pretty expensive solution, there are several such systems in Central Europe.  However some of them are far smaller than the one described above. The second mocap in the Czech Republic can be found in Brno, used by another game development company called Illusion Softworks. There are also other mocaps in Budapest, Munich or Frankfurt.

Data processing

What happens after the data is captured? The data processing begins. First of all the software finds and computes the marker centers. Intersections of the lines between different cameras and marker centers represent the tracked point in 3D. These first two steps are important to obtain a static position point for each shot. The trajectories of each of the tracked points are then assembled from the shot sequence.

After this automatic job is done the second phase starts when the mocap engineer must map the captured animation to a model. He starts with defining the markers on the model skeleton representing the captured body. Then he performs labeling – captured trajectories are assigned to defined markers on the skeleton model.

Although it could seems that the animation is ready, usually it is not. In the real life situation some of the captured data is missing, so the captured animation must be cleaned up and gaps in trajectories filled. Data can be cleaned up with the aid of cinematic constraints and predictive gap filling algorithms. Gaps are solved by computing the missing trajectory information when the tracked markers were covered and any camera that couldn’t catch their position (e.g. bottom side markers on the actor lying on the floor). This can be performed using a rigid body definition (used e.g. for computing covered markers on the chest of lying actor) or using similar trajectories of the adjacent markers. After gaps are filled, the filters are applied to reduce jittering.

Every company in the MC business develops their unique software for processing the data – actually it may be the case that the quality of the algorithms used is even more important than the features of the hardware and that is where major MC companies differ from their smaller counterparts – advanced SW packages are stable, workflow oriented, fast or even real-time and provide reliable high quality data that one can depend on.