An introduction to the beginning of motion capture technology.
The science of human motion analysis is fascinating because of its highly interdisciplinary nature and a wide range of applications. Histories of science usually begin with the ancient Greeks, who first left a record of human inquiry concerning the nature of the world in relation to our powers of perception. Aristotle (384-322 B.C.) might be considered the first biomechanician. He wrote the book called ’De Motu Animalium’ – On the Movement of Animals. He not only saw animals’ bodies as mechanical systems, but pursued such questions as the physiological difference between imagining performing an action and actually doing it.
Nearly two thousand years later, in his famous anatomic drawings, Leonardo da Vinci (1452-1519) sought to describe the mechanics of standing, walking up and down hill, rising from a sitting position, and jumping. Galileo (1564-1643) followed a hundred years later with some of the earliest attempts to mathematically analyze physiologic function. Building on the work of Galilei, Borelli (1608-1679) figured out the forces required for equilibrium in various joints of the human body well before Newton published the laws of motion. He also determined the position of the human center of gravity, calculated and measured inspired and expired air volumes, and showed that inspiration is muscle-driven and expiration is due to tissue elasticity. The early work of these pioneers of biomechanics was followed up by Newton (1642-1727), Bernoulli (1700-1782), Euler (1707-1783), Poiseuille (1799-1869), Young (1773-1829), and others of equal fame. Muybridge (1830-1904) was the first photographer to dissect human and animal motion (see figure at heading ‘human motion analysis’). This technique was first used scientifically by Marey (1830-1904), who correlated ground reaction forces with movement and pioneered modern motion analysis. In the 20th century, many researchers and (biomedical) engineers contributed to an increasing knowledge of human kinematics and kinetics. This paper will give a short overview of the technologies used in these fields.
Human motion analysis
Many different disciplines use motion analysis systems to capture movement and posture of the human body. Basic scientists seek a better understanding of the mechanisms that are used to translate muscular contractions about articulating joints into functional accomplishment, e.g. walking. Increasingly, researchers endeavor to better appreciate the relationship between the human motor control system and gait dynamics.
In the realm of clinical gait analysis, medical professionals apply an evolving knowledge base in the interpretation of the walking patterns of impaired ambulators for the planning of treatment protocols, e.g. orthotic prescription and surgical intervention and allow the clinician to determine the extent to which an individual’s gait pattern has been affected by an already diagnosed disorder. With respect to sports, athletes and their coaches use motion analysis techniques in a ceaseless quest for improvements in performance while avoiding injury. The use of motion capture for computer character animation or virtual reality (VR) applications is relatively new. The information captured can be as general as the position of the body in space or as complex as the deformations of the face and muscle masses. The mapping can be direct, such as human arm motion controlling a character’s arm motion, or indirect, such as human hand and finger patterns controlling a character’s skin color or emotional state. The idea of copying human motion for animated characters is, of course, not new. To get a convincing motion for the human characters in Snow White, Disney studios traced animation over film footage of live actors playing out the scenes. This method, called rotoscoping, has been successfully used for human characters. In the late’70’s, when it began to be feasible to animate characters by computer, animators adapted traditional techniques, including rotoscoping.
Generally, motion analysis data collection protocols, measurement precision, and data reduction models have been developed to meet the requirements for their specific settings. For example, sport assessments generally require higher data acquisition rates because of increased velocities compared to normal walking. In VR applications, real-time tracking is essential for a realistic experience of the user, so the time lag should be kept to a minimum. Years of technological development has resulted in many systems and can be categorized in mechanical, optical, magnetic, acoustic and inertial trackers. The human body is often considered as a system of rigid links connected by joints. Human body parts are not actually rigid structures, but they are customarily treated as such during studies of human motion.
How does Mechanical Motion capture work?
Mechanical trackers utilize rigid or flexible goniometers which are worn by the user. Goniometers within the skeleton linkages have a general correspondence to the joints of the user. These angle measuring devices provide joint angle data to kinematic algorithms which are used to determine body posture. Attachment of the body-based linkages as well as the positioning of the goniometers presents several problems. The soft tissue of the body allows the position of the linkages relative to the body to change as motion occurs. Even without these changes, alignment of the goniometer with body joints is difficult. This is specifically true for multiple degrees of freedom (DOF) joints, like the shoulder. Due to variations in anthropometric measurements, body-based systems must be recalibrated for each user.
How does Optical motion capture work?
Optical sensing encompasses a large and varying collection of technologies. Image-based systems determine position by using multiple cameras to track predetermined points (markers) on the subject’s body segments, aligned with specific bony landmarks. The position is estimated through the use of multiple 2D images of the working volume. Stereometric techniques correlate common tracking points on the tracked objects in each image and use this information along with knowledge concerning the relationship between each of the images and camera parameters to calculate position. The markers can either be passive (reflective) or active (light emitting). Reflective systems use infrared (IR) LED’s mounted around the camera lens, along with IR pass filters placed over the camera lens and measure the light reflected from the markers. Optical systems based on pulsed-LED’s measure the infrared light emitted by the LED’s placed on the body segments. Also, camera tracking of natural objects without the aid of markers is possible, but in general less accurate. It is largely based on computer vision techniques for pattern recognition and often requires high computational resources. Structured light systems use lasers or beamed light to create a plane of light that is swept across the image. They are more appropriate for mapping applications than dynamic tracking of human body motion. Optical systems suffer from occlusion (line of sight) problems whenever a required light path is blocked. Interference from other light sources or reflections may also be a problem which can result in so-called ghost markers.
How does Magnetic Motion Capture work?
Magnetic motion capture systems utilize sensors placed on the body to measure the low-frequency magnetic fields generated by a transmitter source. The transmitter source is constructed of three perpendicular coils that emit a magnetic field when a current is applied. The current is sent to these coils in a sequence that creates three mutually perpendicular fields during each measurement cycle. The 3D sensors measure the strength of those fields which is proportional to the distance of each coil from the field emitter assembly. The sensors and source are connected to a processor that calculates position and orientation of each sensor based on its nine measured field values. Magnetic systems do not suffer from
Magnetic motion capture systems utilize sensors placed on the body to measure the low-frequency magnetic fields generated by a transmitter source. The transmitter source is constructed of three perpendicular coils that emit a magnetic field when a current is applied. The current is sent to these coils in a sequence that creates three mutually perpendicular fields during each measurement cycle. The 3D sensors measure the strength of those fields which is proportional to the distance of each coil from the field emitter assembly. The sensors and source are connected to a processor that calculates position and orientation of each sensor based on its nine measured field values. Magnetic systems do not suffer from line of sight problems because the human body is transparent for the used magnetic fields. However, the shortcomings of magnetic tracking systems are directly related to the physical characteristics of magnetic fields. Magnetic fields decrease in power rapidly as the distance from the generating source increases and so they can easily be disturbed by (ferro)magnetic materials within the measurement volume.
What is Acoustic Motion Capture?
Acoustic tracking systems use ultrasonic pulses and can determine position through either time-of-flight of the pulses and triangulation or phase coherence. Both outside-in and inside-out implementations are possible, which means the transmitter can either be placed on a body segment or fixed in the measurement volume. The physics of sound limit the accuracy, update rate and range of acoustic tracking systems. A clear line of sight must be maintained and tracking can be disturbed by reflections of the sound.
What is Inertial Motion capture?
Inertial sensors use the property of bodies to maintain constant translational and rotational velocity, unless disturbed by forces or torques, respectively. The vestibular system, located in the inner ear, is a biological 3D inertial sensor. It can sense angular motion as well as linear acceleration of the head. The vestibular system is important for maintaining balance and stabilization of the eyes relative to the environment. Practical inertial tracking is made possible by advances in miniaturized and micromachined sensor technologies, particularly in silicon accelerometers and rate sensors. A rate gyroscope measures angular velocity, and if integrated over time provides the change in angle with respect to an
Inertial sensors use the property of bodies to maintain constant translational and rotational velocity, unless disturbed by forces or torques, respectively. The vestibular system, located in the inner ear, is a biological 3D inertial sensor. It can sense angular motion as well as linear acceleration of the head. The vestibular system is important for maintaining balance and stabilization of the eyes relative to the environment. Practical inertial tracking is made possible by advances in miniaturized and micromachined sensor technologies, particularly in silicon accelerometers and rate sensors. A rate gyroscope measures angular velocity, and if integrated over time provides the change in angle with respect to an initially known angle. An accelerometer measures accelerations, including gravitational acceleration g. If the angle of the sensor with respect to the vertical is known, the gravity component can be removed and by numerical integration, velocity and position can be determined. Noise and bias errors associated with small and inexpensive sensors make it impractical to track orientation and position for long time periods if no compensation is applied. By combining the signals from the inertial sensors with aiding/complementary sensors and using knowledge about their signal characteristics, drift and other errors can be minimized.
Commercial optical systems such as Vicon (reflective markers) or Optitrak (active markers) are often considered as a ‘standard’ in human movement analysis. Although these systems provide accurate position information, there are some important limitations. The most important factors are the high costs, occlusion problems and limited measurement volume. The use of a specialized laboratory with fixed equipment impedes many applications, like monitoring of daily life activities, control of prosthetics or assessment of workload in ergonomic studies. In the past few years, the health care system trend toward early discharge to monitor and train patients in their own environment. This has promoted a large development of non-invasive portable and wearable systems. Inertial sensors have been successfully applied for such clinical measurements outside the lab. Moreover, it has opened many possibilities to capture motion data for athletes or animation purposes without the need for a studio.
The orientation obtained by present-day micromachined gyroscopes typically shows an increasing error of degrees per minute. For accurate and drift free orientation estimation Xsens has developed an algorithm to combine the signals from 3D gyroscopes, accelerometers and magnetometers. Accelerometers are used to determine the direction of the local vertical by sensing acceleration due to gravity. Magnetic sensors provide stability in the horizontal plane by sensing the direction of the earth magnetic field like a compass. Data from these complementary sensors are used to eliminate drift by continuous correction of the orientation obtained by angular rate sensor data. This combination is also known as an attitude and heading reference system (AHRS).
For human motion tracking, the inertial motion trackers are placed on each body segment to be tracked. The inertial motion trackers give absolute orientation estimates which are also used to calculate the 3D linear accelerations in world coordinates which in turn give translation estimates of the body segments. Since the rotation from sensor to body segment and its position with respect to the axes of rotation are initially unknown, a calibration procedure is necessary. An advanced articulated body model constraints the movements of segments with respect to each other and eliminates any integration drift.
A single axis accelerometer consists of a mass, suspended by a spring in a housing. Springs (within their linear region) are governed by a physical principle known as Hooke’s law. Hooke’s law states that a spring will exhibit a restoring force which is proportional to the amount it has been expanded or compressed. Specifically, F = kx, where k is the constant of proportionality between displacement x and force F. The other important physical principle is that of Newton’s second law of motion which states that a force operating on a mass which is accelerated will exhibit a force with a magnitude F = ma. This force causes the mass to either compress or expand the spring under the constraint that F = ma = kx. Hence an acceleration a will cause the mass to be displaced by x = ma/k or, if we observe a displacement of x, we know the mass has undergone an acceleration of a = kx/m. In this way, the problem of measuring acceleration has been turned into one of measuring the displacement of a mass connected to a spring. In order to measure multiple axes of acceleration, this system needs to be duplicated along each of the required axes.
Gyroscopes are instruments that are used to measure angular motion. There are two broad categories: (1) mechanical gyroscopes and (2) optical gyroscopes. Within both of these categories, there are many different types available. The first mechanical gyroscope was built by Foucault in 1852, as a gimbaled wheel that stayed fixed in space due to angular momentum while the platform rotated around it. Mechanical gyroscopes operate on the basis of conservation of angular momentum by sensing the change in direction of an angular momentum. According to Newton’s second law, the angular momentum of a body will remain unchanged unless it is acted upon by a torque. The fundamental equation describing the behavior of the gyroscope is:
where the vectors tau and L are, the torque on the gyroscope and its angular momentum, respectively . The scalar I is its moment of inertia, the vector omega is its angular velocity, and the vector alpha is its angular acceleration.
Gimbaled and laser gyroscopes are not suitable for human motion analysis due to their large size and high costs. Over the last few years, microelectromechanical machined (MEMS) inertial sensors have become more available. Vibrating mass gyroscopes are small, inexpensive and have low power requirements, making them ideal for human movement analysis. A vibrating element (vibrating resonator), when rotated, is subjected to the Coriolis effect that causes secondary vibration orthogonal to the original vibrating direction. By sensing the secondary vibration, the rate of turn can be detected. The Coriolis force is given by:
where m is the mass, v the momentary speed of the mass relative to the moving object to which it is attached and omega the angular velocity of that object. Various micro machined geometries are available, of which many use the piezo-electric effect for vibration exert and detection.
The traditional application area of inertial sensors is navigation as well as guidance and stabilization of military systems. Position, velocity and attitude are obtained using accurate, but large gyroscopes and accelerometers, in combination with other measurement devices such as GPS, radar or a baro altimeter. Generally, signals from these devices are fused using a Kalman filter to obtain quantities of interest (see figure below).
About this figure:
Complementary Kalman filter structure for position and orientation estimates combining inertial and aiding measurements. The signals from the IMU (a − g and w) provide the input for the INS. By double integration of the accelerations, the position is estimated at a high frequency. At a feasible lower frequency, the aiding system provides position estimates. The difference between the inertial and aiding estimates is delivered to the Kalman filter. Based on the system model the Kalman filters estimates the propagation of the errors. The outputs of the filter are fed back to correct the position, velocity, acceleration and orientation estimates.
The Kalman filter is useful for combining data from several different indirect and noisy measurements. It weights the sources of information appropriately with knowledge about the signal characteristics based on their models to make the best use of all the data from each of the sensors. There is no perfect sensor; each type has its strong and weak points. The idea behind sensor fusion is that characteristics of one type of sensor are used to overcome the limitations of another sensor. For example, magnetic sensors are used as a reference to prevent the gyroscope integration drift about the vertical axis in the orientation estimates. However, iron and other magnetic materials will disturb the local magnetic field and as a consequence, the orientation estimate. The spatial and temporal features of magnetic disturbances will be different from those related to gyroscope drift errors. Using this a priori knowledge, the effects of both drift and disturbances can be minimized. The inertial sensors of the inertial navigation system (INS) can be mounted on vehicles in such a way that they stay leveled and pointed in a fixed direction. This system relies on a set of gimbals and sensors attached on three axes to monitor the angles at all times. Another type of INS is the strapdown system that eliminates the use of gimbals which is >suitable for human motion analysis. In this case, the gyros and accelerometers are mounted directly to the structure of the vehicle or strapped on the body segment. The measurements are made in reference to the local axes of roll, pitch, and heading (or yaw). The clinical reference system provides anatomically meaningful definitions of main segmental movements (e.g. flexion-extension, abduction-adduction or supination-pronation).