MPEG2Event

home :: motion_vectors

Decoding Motion Vectors

Motion vectors are encoded at the macroblock level in P or B frames. Motion vectors are encoded differentially relative to previous motion vectors encoded in the frame. Furthermore, the syntax and scale of the motion vectors depends on information that is included in the picture header and picture coding extension header. Because of this, motion vector decoding is a bit complicated. If you are interested in decoded motion vector information, the MotionVectorDecoder class is provided to help you.

The constructor for a MotionVectorDecoder takes a BitStream object as a parameter. The MotionVectorDecoder registers a number of callbacks with various coding elements needed to decode motion vectors. The motion vector decoder expects that parsing the video stream will be driven by some other process (see here for more on managing the parsing process). As coding elements are parsed and published, the motion vector decoder maintains the various motion vector predictors required. After a macroblock has been decoded, the motion vector decoder can be queried in order to get the current value of any motion vector. This information is no longer available once the next macroblock is decoded because it is replaced by whatever motion vector information is associated with the next macroblock instead. So to capture motion vectors, you need to collect them from the motion vector decoder after each macroblock is decoded. Furthermore, the motion vector decoder does not maintain which (if any) motion vectors are valid (i.e., which were actually encoded within the macroblock and are associated with the macroblock). This information is held in the macroblock mode coding element and can be queried through properties of Macroblock element event object.

In short, to capture motion vector information at the macroblock level, you need to:

• Create a MotionVectorDecoder object.
• Drive the video parsing process (usually with a VideoParser object
• For every Macroblock coding element event that is published, query the Macroblock event to find out which motion vectors are valid and then query the MotionVectorDecoder for those motion vectors.

The easiest way to do the last step of the above is to create and register a handler for Macroblock events. Unfortunately, this may result in a bit of a race condition. Since the motion vector decoder needs the information in a Macroblock event in order to properly decode the motion vectors, your handler needs to be guaranteed that it is called after the motion vector decoder has done whatever processing it needs to do. But since the order of execution for handlers registered with an event can not be guaranteed, we encounter a possible race condition in which your handler is invoked before the handler installed by the motion vector decoder. The solution is that the motion vector decoder maintains a list of handlers for Macroblock events of its own and republishes these events to this list after it has completed its processing. This allows you to register your callback for processing macroblocks which needs the motion vector information with the motion vector decoder instead of with the Macroblock class and thus be assured that the motion vector decoder has completed its necessary processing before your handler is called.

MotionVectorDecoder Details

The constructor looks like:

MotionVectorDecoder mvd = new MotionVectorDecoder(bs); // bs is a BitStream object

The constructor simply requires the BitStream object from which the video stream will be parsed. A side result of the constructor is that handlers for various coding elements will be registered so that the motion vector decoder object can do its job.

The motion vector decoder maintains a list of Macroblock event handlers to republish these events to after motion vector decoding has taken place as a publicly accessible instance member. This is declared as:

public event Macroblock.Handler macroblockEvent;

Notice the type of the event is the same as the event maintained by the Macroblock class so any handler that could have been registered there can be registered here just as easily. The advantage of registering it here is that you are assured that all motion vector decoding has completed before your handler is called.

To access the motion vector information, use the methods listed below. For all of these methods, the first parameter selects whether the first or second encoded motion vector is retrieved. This parameter is constrained to be one of the two enumerated values defined as MotionVectorDecoder.Select.FIRST and MotionVectorDeocder.Select.SECOND. To determine how many motion vectors were encoded with the macroblock and are valid, query properties of the Macroblock element event. The second parameter to these methods is the motion vector direction (i.e., whether it is a forward motion vector from the past reference frame or a backward motion vector from the future reference frame). Again, properties of the Macroblock event will determine which are valid for that particular macroblock. The values of this parameter are enumerated as MotionVectorDecoder.Direction.FORWARD and MotionVectorDecoder.Direction.BACKWARD. All of these methods return the horizontal and vertical components of the motion vector in half-pixel resolution through the reference parameters h and v.

• GetPMV(Select r, Direction s, out int h, out int v)
  This method retrieves the current value for the specified motion vector predictor. This may or may not be the same as the decoded motion vector.
• GetLumVector(Select r, Direction s, out int h, out int v)
  This method retrieves the current value of the specified motion vector decoded appropriately for use with the luminance plane.
• GetChrVector(Select r, Direction s, out int h, out int v)
  This method retrieves the current value of the specified motion vector decoded appropriately for use with the chrominance plane.

Dual Prime Motion Vectors

It should be noted that dual prime motion vectors are not supported at this time.

2004.11.23-09:24.00