MPEG2Event

Download Link

Thanks for signing in. Here's the link to the package:

2004.11.23-11:33.00

Download

Thanks for your interest in MPEG2Event. Although MPEG2Event is freely and publicly available, I am trying to keep track of who has downloaded it. After submitting the form below, you will get a link to a compressed tarball which contains all the sources and documentation and associated Visual Studio .NET project files.

Thanks again and please feel free to send any and all feedback, comments, suggestions, etc.

2004.11.23-11:01.00

Overview

MPEG2Event is a C# library intended to facilitate rapid prototyping of MPEG-2 analysis tools. Unlike other MPEG-2 decoding libraries which are designed for performance, MPEG2Event sacrifices parsing speed in order maximize flexibility and expose the coding elements contained within an MPEG-2 stream at a number of different granularities.

MPEG2Event provides an event-based architecture. As the media stream is parsed, the library constructs and publishes an event for each coding element encountered. By using the C# publish/subscribe event model, your code is notified via delegate methods when these events occur. Event types are organized in a hierarchy allowing you to subscribe in a number of different ways including:

• A very specific atomic coding element (e.g., the horizontal size field of a picture header, the vertical component of a motion vector).
• A compound coding element comprised of smaller coding elements (e.g., the entire picture header, a motion vector, a macroblock).
• A general class of coding elements (e.g., all headers, all atomic coding elements, etc.).

Currently, MPEG2Event only works with MPEG-2 elementary video streams. You must use other tools to demultiplex the video stream out from a transport or program stream. The MPEG2Event architecture, however, is fairly general and could be easily extended to work with transport, program, or audio streams.

Basic Usage

For most uses of the library, the following basic approach can be followed:

• Wrap the source of bits into a "BitStream" object.
  The BitStream class provides a bit parser interface to any C# Stream object. Simply pass the source of the MPEG2 stream data (i.e., file stream, network stream, etc.) as a parameter to the BitStream constructor. See the BitStream class documentation for more details.
• Register delegate methods as callbacks for the coding elements that you would like to process.
  The callback method signature must match the delegate definition within the coding element event class. Register the callback by adding the delegate to the statically defined "handlers" event for each event class.
• Create a "VideoParser" object to parse the video.
  The VideoParser does the work of actually parsing the video stream. It does so by keeping track of decoding state and knowing what syntactic structure to expect next. It uses the BitStream object created above to retrieve the MPEG2 stream.
• Call the parse_picture() method of the VideoParser object in an infinite loop (or for however many pictures you want to parse).
  This method will throw an exception if an error in the bitstream is found or if the end of the stream is reached.
• As coding elements are parsed, events are constructed and published to your callback methods as appropriate.

Coding Element Hierarchy

Events are generated for each coding element as it is encountered in the bitstream. Each coding element is associated with a class which defines the coding element event generated. These coding element events are organized into a relatively flat hierarchy. This hierarchy allows you to subscribe to more general abstract classes of coding events that may encompass several specific subtypes. The design of the coding element class hierarchy is described here, and the event mechanism is more fully described here.

Library Organization

MPEG2Event is organized into a few different namespaces in anticipation of extensions for other parts of the MPEG-2 standard. Currently, the following namespaces are defined:

• MPEG2Event
  This namespace contains all of the general classes that are not specific to a particular media type including the BitStream class, the parent coding element classes such as CodingElement, AtomicCodingElement, and Header, as well as various utility classes such as ZigZag.
• MPEG2Event.Video
  This namespace contains all of the classes that are specific to MPEG2 video including all of the coding elements that comprise an MPEG2 video stream, the video parser class VideoParser, and the motion vector decoder class MotionVectorDecoder.
• MPEG2Event.Systems
  This namespace will contain all of the classes that are part of the MPEG2 systems layer. This is work in progress and is not very useful just yet.

2004.11.23-09:24.00

Video Parsing

Each class in the coding element event hierarchy implements a static class method called GetNext(BitStream bs). This method assumes that the next bit in the bitstream passed as a parameter is the first bit of the coding element. The method then parses the coding element, constructs a new instance of the codinge element event, and publishes the event to all registered handlers (see here for more on the event mechanism).

While each coding element event class knows how to parse an instance of itself, determining the order of the coding elements to be parsed and managing the video parsing process is not provided by these methods per se. A helper class called VideoParser is provided for this task.

Creating a Video Parser

To create a new video parser, simply call the constructor, providing a BitStream object as the source of bits for the video stream like so:

VideoParser vp = new VideoParser(bs); // bs is an instance of BitStream

Using the Video Parser

The VideoParser object provides a very simple public interface. There are only two public methods which are:

public void parsePicture();
public void skipPicture();

These methods do pretty much what they say they do (i.e., parse the next picture or skip the next picture). When either is called for the first time, the parser skips over all bits until it encounters a MPEG-2 sequence header. These skipped bits are published as OpaqueBits events. The sequence header, sequence extension header, any user data and other extension headers, group of pictures header, picture header, and picture coding extension header are then parsed and published. If the picture is to be parsed (as opposed to skipped), the picture data is then parsed and published. If the picture is to be skipped, slice headers are parsed and published and all of the coding elements between the slice headers (i.e., the actual coded picture information) is simply parsed off as OpaqueBits. Parsing ends at the end of the picture data. Subsequent calls to either method picks up parsing where the last call left off.

The skipPicture() method is useful to avoid parsing and publishing the coding elements of a picture you know that you are not interested in. For example, if you are only interested in I-frames, then you can use skipPicture() to avoid parsing P and B frames and use parsePicture() only when the picture is an I frame.

Dealing with the End of the Stream

After the sequence end code has been encountered, both methods will simply return without doing anything. This condition can be checked using the EOF property of the VideoParser object. If the bitstream runs out of bits, the BitStream object will throw an exception which can be caught to detect this situation.

Typical Use Example

The following example shows the typical way of using the VideoParser object.


// Create and set up a BitStream object. Here we assume that
// input is a Stream object where the bytes are coming from.

BitStream bs = new BitStream(input);

// Create and register our event delegates that will process the
// coding elements we are interested in. Here we assume that
// we have methods for processing PictureHeader, PictureCoding,
// and Macroblock elements called my_pict_header_handler,
// my_pict_coding_handler, and my_mb_handler

PictureHeader.handlers += new PictureHeader.Handler(my_pict_header_handler);
PictureCoding.handlers += new PictureCoding.Handler(my_pict_coding_handler);
Macroblock.handlers += new Macroblock.Handler(my_mb_handler);

// Create a VideoParser object.
VideoParser vp = new VideoParser(vp);

// Parse the video stream until either end of sequence or an
// out of bits exception occurs.

try {

while (!vp.EOF) {

vp.parsePicture();

}

}
catch (OutOfBitsException e) {

// Handle ungraceful end to stream here.

}

2004.11.23-09:24.00

Decoding Motion Vectors

Motion vectors are encoded at the macroblock level in P or B frames. Motion vectors are encoded differentially relative to previous motion vectors encoded in the frame. Furthermore, the syntax and scale of the motion vectors depends on information that is included in the picture header and picture coding extension header. Because of this, motion vector decoding is a bit complicated. If you are interested in decoded motion vector information, the MotionVectorDecoder class is provided to help you.

The constructor for a MotionVectorDecoder takes a BitStream object as a parameter. The MotionVectorDecoder registers a number of callbacks with various coding elements needed to decode motion vectors. The motion vector decoder expects that parsing the video stream will be driven by some other process (see here for more on managing the parsing process). As coding elements are parsed and published, the motion vector decoder maintains the various motion vector predictors required. After a macroblock has been decoded, the motion vector decoder can be queried in order to get the current value of any motion vector. This information is no longer available once the next macroblock is decoded because it is replaced by whatever motion vector information is associated with the next macroblock instead. So to capture motion vectors, you need to collect them from the motion vector decoder after each macroblock is decoded. Furthermore, the motion vector decoder does not maintain which (if any) motion vectors are valid (i.e., which were actually encoded within the macroblock and are associated with the macroblock). This information is held in the macroblock mode coding element and can be queried through properties of Macroblock element event object.

In short, to capture motion vector information at the macroblock level, you need to:

• Create a MotionVectorDecoder object.
• Drive the video parsing process (usually with a VideoParser object
• For every Macroblock coding element event that is published, query the Macroblock event to find out which motion vectors are valid and then query the MotionVectorDecoder for those motion vectors.

The easiest way to do the last step of the above is to create and register a handler for Macroblock events. Unfortunately, this may result in a bit of a race condition. Since the motion vector decoder needs the information in a Macroblock event in order to properly decode the motion vectors, your handler needs to be guaranteed that it is called after the motion vector decoder has done whatever processing it needs to do. But since the order of execution for handlers registered with an event can not be guaranteed, we encounter a possible race condition in which your handler is invoked before the handler installed by the motion vector decoder. The solution is that the motion vector decoder maintains a list of handlers for Macroblock events of its own and republishes these events to this list after it has completed its processing. This allows you to register your callback for processing macroblocks which needs the motion vector information with the motion vector decoder instead of with the Macroblock class and thus be assured that the motion vector decoder has completed its necessary processing before your handler is called.

MotionVectorDecoder Details

The constructor looks like:

MotionVectorDecoder mvd = new MotionVectorDecoder(bs); // bs is a BitStream object

The constructor simply requires the BitStream object from which the video stream will be parsed. A side result of the constructor is that handlers for various coding elements will be registered so that the motion vector decoder object can do its job.

The motion vector decoder maintains a list of Macroblock event handlers to republish these events to after motion vector decoding has taken place as a publicly accessible instance member. This is declared as:

public event Macroblock.Handler macroblockEvent;

Notice the type of the event is the same as the event maintained by the Macroblock class so any handler that could have been registered there can be registered here just as easily. The advantage of registering it here is that you are assured that all motion vector decoding has completed before your handler is called.

To access the motion vector information, use the methods listed below. For all of these methods, the first parameter selects whether the first or second encoded motion vector is retrieved. This parameter is constrained to be one of the two enumerated values defined as MotionVectorDecoder.Select.FIRST and MotionVectorDeocder.Select.SECOND. To determine how many motion vectors were encoded with the macroblock and are valid, query properties of the Macroblock element event. The second parameter to these methods is the motion vector direction (i.e., whether it is a forward motion vector from the past reference frame or a backward motion vector from the future reference frame). Again, properties of the Macroblock event will determine which are valid for that particular macroblock. The values of this parameter are enumerated as MotionVectorDecoder.Direction.FORWARD and MotionVectorDecoder.Direction.BACKWARD. All of these methods return the horizontal and vertical components of the motion vector in half-pixel resolution through the reference parameters h and v.

• GetPMV(Select r, Direction s, out int h, out int v)
  This method retrieves the current value for the specified motion vector predictor. This may or may not be the same as the decoded motion vector.
• GetLumVector(Select r, Direction s, out int h, out int v)
  This method retrieves the current value of the specified motion vector decoded appropriately for use with the luminance plane.
• GetChrVector(Select r, Direction s, out int h, out int v)
  This method retrieves the current value of the specified motion vector decoded appropriately for use with the chrominance plane.

Dual Prime Motion Vectors

It should be noted that dual prime motion vectors are not supported at this time.

2004.11.23-09:24.00

Organization of an MPEG-2 Bitstream

Here we provide a brief overview of how an MPEG-2 video bitstream is organized. This should give you an idea of how different coding elements are related to each other and what to expect when the parser starts throwing events at you. In the text below, highlighted fixed-font text refers to specific classes in the coding element event hierarchy.

In general, an MPEG-2 bitstream is comprised of the following coding structures in this order:

• Sequence header and sequence extension header
• Possibly user data and/or other extension headers
• Group of pictures header
• Possibly user data
• For each frame in the group of pictures...
  • Picture header and picture coding extension header
  • Possibly user data and/or other extension headers
  • Picture data

At the end of a group of pictures, this structure is repeated starting either at the sequence header level or the group of pictures header level. If the sequence header is repeated, then all values in the sequence header and sequence extension header must match the values previously encoded in any previous sequence header and/or sequence extension header. At the end of the entire sequence (i.e., at the end of the video file), should be a sequence end header. Once the sequence end header is encountered, the video stream is considered finished (i.e., any and all bytes after the sequence end header are meaningless relative to this video sequence).

Below are more detailed descriptions of these coding structures including the exact coding element events that are published when they are encountered.

Sequence Header and Sequence Extension

The bitstream should start with a sequence header. If you use the VideoParser object, everything before the sequence header will be parsed off and published as a sequence of OpaqueBits events. The sequence header will result in a StartCode event (the value of which should indicate that a sequence header follows) followed by the events published for each atomic coding element that is part of the sequence header. These will be, in order:

• HorizontalSizeValue
• VerticalSizeValue
• AspectRatioInformation
• FrameRateCode
• BitRateValue
• MarkerBit
• VBVBufferSizeValue
• ContrainedParametersFlag
• LoadIntraQuantiserMatrix
• IntraQuantiserMatrix
  Only if previous load intra quantiser matrix flag value is set to one, otherwise not present.
• LoadNonIntraQuantiserMatrix
• NonIntraQuantiserMatrix
  Only if previous load non-intra quantiser matrix flag value is set to one, otherwise not present.

All of this (not including the start code) is collected together and published as a SequenceHeader compound coding element event.

Following the sequence header will be an extension start code which indicates the presence of the sequence extension header. For MPEG-2 video, this is mandatory. The sequence header must be followed by the sequence extension header. The extension start code results in a StartCode event. This StartCode must have the appropriate value to indicate that an extension header follows. Following this will be an ExtensionStartCodeIdentifier event. This 4-bit atomic coding element determines the type of extension header that follows. In this case, its value must indicate that the extension header is a sequence extension header. The sequence extension header is comprised of the following atomic coding elements in this order:

• ProfileAndLevelIndication
• ProgressiveSequence
• ChromaFormat
• HorizontalSizeExtension
• VerticalSizeExtension
• BitRateExtension
• MarkerBit
• VBVBufferSizeExtension
• LowDelay
• FrameRateExtensionN
• FrameRateExtensionD

All of this is part of the compound element SequenceExtension which is published after its component parts are parsed and published individually. Again, note that the start code and the extension start code identifier are not part of the SequenceExtension coding element.

Group of Pictures Header

The group of pictures header is preceded by a start code with the appropriate value indicating that what follows is a group of pictures header. The start code is published as a StartCode event and is not considered to be part of the group of pictures header itself. The group of pictures header is comprised of the following atomic coding elements in this order:

• TimeCode
• ClosedGOP
• BrokenLink

These are collected together in the compound coding element GroupOfPicturesHeader which is published after all of its component parts are publised.

Picture Header and Picture Coding Extension Header

The picture header follows a StartCode event with the appropriate value. It is comprised of the following atomic coding elements:

• TemporalReference
• PictureCodingType
• VBVDelay
• FullPelForwardVector
  Present only if previously encountered PictureCodingType indicates that this picture is a P or B frame.
• ForwardFCode
  Present only if FullPelForwardVector element is present.
• FullPelBackwardVector
  Present only if previously encountered PictureCodingType indicates that this picture is a B frame.
• BackwardFCode
  Present only if FullPelBackwardVector element is present.
• ExtraBitPicture
• ExtraInformationPicture
  Present only if previous ExtraBitPicture value is set to 1. This two element sequence is repeated until ExtraBitPicture element is set to 0.

The compound PictureHeader element event is published after all of its components are parsed and published individually.

The picture header must be followed by a picture coding extension header. This is introduced with a StartCode indicating the extension followed by a ExtensionStartCodeIdentifier give the extension type as a picture coding extension. The picture coding extension is comprised of the following atomic coding elements which follow in this order:

• Four FCode elements
  These provide parameters required for decoding motion vectors in different types of frames.
• IntraDCPrecision
• PictureStructure
• TopFieldFirst
• FramePredFrameDCT
• ConcealmentMotionVectors
• QScaleType
• IntraVLCFormat
• AlternateScane
• RepeatFirstField
• Chroma420Type
• ProgressiveFrame
• CompositeDisplayFlag
  If CompositeDisplayFlag is set to 1 then the following coding elements will follow:

  • VAxis
  • FieldSequence
  • SubCarrier
  • BurstAmplitude
  • SubCarrierPhase

The picture coding extension is then published as PictureCodingExtension event.

Picture Data

Picture data is comprised of the coding elements that actually encode the frame's visual content. MPEG-2 frames may be organized either as a single progressive frame or two fields (odd and even). If the frame is organized as two fields, each field is encoded as a two consecutive pictures with the same temporal reference value. Various values in the picture header and picture coding extension header should provide all the information you need about the type (i.e., I, P, or B) and structure (i.e., field or frame, odd first or even first, etc.) of a particular picture.

Picture data is organized into slices. Each slice starts with a StartCode. The slice start codes range from 0x00000101 through 0x000001AF. The last byte of the slice start code provides the macroblock row number of the first macroblock encoded in this slice.

After the slice start code, if the vertical size of the frame encoded in the picture header and picture coding extension header exceeds 2800 pixels, a SliceVerticalPositionExtension element is parsed. This is rarely used, but if needed, provides the additional info needed to properly encode the macroblock row number of the first macroblock in the slice.

At this point, if certain scalability options are in use, there may be 7 bits encoding the priority breakpoint value. At this time, MPEG2Event doesn't support scalability options and their use would probably break the VideoParser object at just this point since these 7 bits would not be interpreted correctly.

Next a QuantiserScaleCode element is parsed. This resets the quantiser value for the slice. This value is allowed to change on a per macroblock value so this element may be encountered again if the value is changed at any given macroblock.

The next bit is parsed as the SliceExtensionFlag. If set to 1, then the following coding elements will be parsed:

• IntraSlice
• SlicePictureIdEnable
• slicePictureId
• ExtraBitSlice
• ExtraInformationSlice
  Only if ExtraBitSlice is set to 1. If so set, then ExtraBitSlice and ExtranInformationSlice elements repeatedly parsed until an ExtraBitSlice with value 0 is encountered.

Everything after the slice start code until this point is then published as a SliceHeader. Following this is one or more macroblocks. Each macroblock is comprised of some subset of the following coding elements:

• MacroblockEscape
  One or more of these elements are parsed if the difference between the current macroblock address and the previous macroblock address is greater than 33.
• MacroblockAddressIncrement
  The combination of zero or more MacroblockEscape elements and this element are published as part of a compound element called MacroblockAddressDelta that provides the actual encoded macroblock address difference.
• MacroblockMode
  This element signals the presence of an encoded quantiser scale code, forward motion vectors, backward motion vectors, and the coded block pattern. If not encoded, then these elements will not appear in the bitstream.
• QuantiserScaleCode
• MotionVector
  Multiple MotionVector elements may be present depending on the type and mode of motion compensation in use. This is a compound element that encompasses a number of atomic coding elements that actually encode the various parts of each motion vector. A more detailed discussion of motion vector decoding can be found here.
• CodedBlockPattern
• Block
  Zero or more blocks are encoded according to the type of frame and the value of the previous CodedBlockPattern element if present. A Block is a compound element that encapsulates the following atomic elements:

  • DiffEncodedCoeff
    Present if the DC coefficient is encoded differentially with respect to a previous DC coefficient (i.e., I-frames, and I-macroblocks in P and B frames).
  • HuffmanEncodedCoeff
    One for each DCT coefficient encoded as a (run-length, value) pair.

All of these elements are encapsulated in the compound Macroblock element.

Other Extension Headers and User Data

After the sequence extension header and picture extension header, zero or more other extension headers may be present or user data may be present. Also, after the group of pictures header, user data may be present. Currently MPEG2Event only recognizes, parses, and publishes the quantization matrix extension header which is published as a QuantMatrixExtension element. User data, if present is parsed and published as a UserData element. All other extension headers are parsed as OpaqueBits.

2004.11.23-09:24.00

DCT Coefficients

DCT Coefficients are encoded in one of two ways. The DC term of an intracoded macroblock are differentially encoded relative to a plane-specific (i.e., Y, U, or V) DC predictor. The predictor is reset whenever a non-intracoded macroblock is encountered. For all other terms (i.e., all AC terms and the DC term of intercoded macroblocks), the coefficient is encoded as a run/value pair using a Huffman code. The run specifies the number of zero-valued coefficients before the encoded coefficient in the zigzag pattern and the value is the quantized value of the coefficient. The coding element events that are generated by these encodings are called DiffEncodedCoeff and HuffmanEncodedCoeff. The end of block marker that indicates no more coefficients exists for a particular block is parsed and published as a special case of HuffmanEncodedCoeff.

Because coefficients may be encoded in one of two different ways (not to mention the end of block marker is parsed as a coefficient but is not actually a coefficient), a coefficient interface is defined for obtaining general information about coefficients regardless of type. This interface, called ICoeff, is implemented by both DiffEncodedCoeff and HuffmanEncodedCoeff.

The ICoeff interface specifies 6 public read-only properties which are listed below with their return types.

• int DequantizedValue
  Returns the dequantized value of the coefficient.
• int Value
  Returns the quantized value that was encoded.
• int ZigPos
  Returns the index of the coefficient in the zigzag order. This will be a value in the range [0,63].
• int RelPos
  Returns the position of the coefficient relative to the other coefficients encoded for this block. A value of 0 means that this was the first encoded coefficient. A value of 1 means that this was the second encoded coefficient, and so on. This encoding position is not necessarily related to the the zigzag position.
• bool IsEOB
  Returns true if this coefficient is really the end of block marker. If true, then the other property values do not have meaning.
• bool IsImplicit
  The Block coding element maintains a value for the DC term even if, as may happen in an intercoded block, it is implicitly defined via its absense. When this occurs, the coefficient is stored in the block as an instance of the ImplicitDCCoeff class which implements ICoeff but is not in the coding element event hierarchy because technically it was not actually coded. If you are accessing coefficient information through a Block object, however, you may need to differentiate between DC coefficients that were actually coded versus those that were implicitly defined. This property of the ICoeff interface allows you to do so. If the coefficient is implicit, then the ZigPos and RelPos properties have no meaning.

The Block coding element event maintains an array of all the coefficients defined for that block and provides methods and properties for getting information about those coefficients. A block object also maintains a representation for the DC coefficient regardless of whether it was implicitly defined or explicitly coded. The properties and methods of a Block object that provide this access are listed and described below:

• ICoeff DC
  A property that returns the DC coeffient of the block. Note, this may have been implicitly defined (see discussion above where IsImplicit property of the ICoeff interface is described).
• IEnumerator GetACEnumerator()
  A method that returns a standard C# enumeration instance that can be used to get each encoded AC coefficient in turn. The objects returned by the enumerator must be cast to ICoeff.
• int ACCount
  A property that returns the number of non-zero AC coefficients defined for this block.
• ICoeff GetAC(int index)
  A method that returns a specific AC coefficient. The index parameter refers to the relative order (i.e., first encoded AC coefficient, second encoded AC coefficient, etc.). Note, this is different from its index in zigzag order.
• ICoeff[] ACArray
  A property that returns an array of all the encoded non-zero AC coefficients in relative encode order (see note about relative vs. zigzag order above).
• ulong ZigPosMask
  A property that returns an unsigned long integer which can be used to indicate which zigzag positions are associated with defined non-zero coefficients. The least significant bit corresponds to zigzag position zero (i.e., the DC coefficient). The second least significant bit corresponds to zigzag postion 1 and so on. The bit that corresponds to the DC coefficient is only set if the DC coefficient was explicitly coded (i.e., not implicit).

2004.11.23-09:24.00

Coding Element Class Hierarchy

The CodingElement Class

At the root of the coding element class hierarchy is the "CodingElement" class. This is an abstract class with no direct instantiations. This class encapsulates information common to every coding element including:

• BitAddress
  
  A read only property returning the bit count of the first bit of this coding element. In other words, the number of bits read from the BitStream object before this coding element was encountered. NOTE: this is the count of *bits* not bytes.
• Length
  
  A read-only property returning the length of this coding element in bits as encoded in the bitstream.
• isContiguous()
  
  A method that returns a boolean value indicating whether all of the bits that encode this coding element are contiguous in the bitstream. The was intended to support coding elements which are compound elements which encapsulate other coding elements that are not necessarily back-to-back within the bitstream. In practice, all coding elements, including compound ones like the picture header, are contiguous.

Atomic vs. Compound Coding Elements

At this point, it may be useful to discuss the difference between an "atomic" coding element and a "compound" coding element. An atomic coding element is a sequence of bits that encode a specific value with a specific purpose. For example, a 32-bit start code is an atomic coding element, the 10 bits that make up the temporal reference number in a picture header is an atomic coding element, and so on. A compound coding element is a collection of one or more atomic coding elements that form a higher level of syntax for the video stream. For example, all of the atomic coding elements that go into a picture header are collected together in a compound coding element called PictureHeader. Atomic coding elements are published as events as they are encountered. Compound coding elements are published after the last of its components is published.

Compound coding element events generally provide an interface to retrieve its component parts if you need them. This allows you to subscribe to the level of detail that makes the most sense. If, for example, you are interested in a number of different pieces of information that all reside in the picture header (e.g., the temporal reference, the picture coding type I, P, or B, etc.), then it would make sense to subscribe to the entire picture header event and then get to the specific pieces of information you need by going through the published picture header event. If you really just need the temporal reference, then it would make more sense to just subscribe to the temporal reference event.

Atomic coding element events are all subclasses of the abstract class AtomicCodingElement, which in turn is a direct subclass of CodingElement. Most compound element events are direct subclasses of CodingElement. The notable exceptions are the SequenceHeader, SequenceExtension, PictureHeader, PictureCodingExtension, GroupOfPicturesHeader, SliceHeader, and QuantMatrixExtension which are all grouped together as subclasses of Header which in turn is a direct subclass of CodingElement. The Header class does not provide much functionality other than give you a convenient way to subscribe to all the header type events without having to specify a handler for each specific header type. Note that the start code coding element that announces the presence of a particular header is not considered part of the header (i.e., the StartCode atomic coding element is not contained by the header that the StartCode identifies).

It is important to note that the organization of these coding element event types does not reflect the organization of the MPEG-2 bitstream. In other words, the Slice coding element contains Macroblock coding elements which in turn contain MacroblockAddressDelta, MacroblockMode, and Block coding elements. However, all of these coding elements mentioned are just direct subclasses of CodingElement since they are all compound coding elements. For more information on the organization of an MPEG-2 bitstream (i.e., which compound elements contain which other compound and atomic coding elements, what order things arrive in, etc.) see here for an overview. Full details, of course, can be found in the MPEG-2 standard specification.

The Hierarchy

Below is an abbreviated map to the coding element class hierarchy. The numerous atomic coding elements are not all individually listed, but it should give you a general idea for how things are structured and lists some of the more important atomic coding elements that you may find useful. See the class documentation for more details.

• CodingElement
  • AtomicCodingElement
    • StartCode
    • OpaqueBits
    • HuffmanEncodedCoeff
    • TemporalReference
    • PictureStructure
    • PictureCodingType
    • Other subclasses of AtomicCodingElement too numerous to list invidually
  • Header
    • SequenceHeader
    • SequenceExtension
    • PictureHeader
    • PictureCodingExtension
    • GroupOfPicturesHeader
    • SliceHeader
    • QuantMatrixExtension
  • Macroblock
  • MacroblockAddressDelta
  • MacroblockMode
  • MotionVector
  • Block
  • DiffEncodedCoeff
  • MatrixCodingElement
    • IntraQuantiserMatrix
    • NonIntraQuantiserMatrix
    • ChromaIntraQuantiserMatrix
    • ChromaNonIntraQuantiserMatrix
  • UserData
  • Slice

2004.11.23-09:24.00

Simple Video Parsing Example

In this example, we use the VideoParser object to parse the coding elements of a video file specified on the command line. We create a single event handler for all CodingElement events (essentially all of the coding elements). The handler simply prints out a string representation of each coding element encountered.

using System;
using System.IO;

// Declare the MPEG2Event namespaces.
using MPEG2Event;
using MPEG2Event.Video;

public class SimpleParsingExample {

public static void Main(String[] args) {

// Check for filename passed on command line.
if (args.Length != 1) {

System.Console.WriteLine("Must specify filename");
return;

}
// Open the file for reading and wrap a
// BitStream object around the file stream.
BitStream bs = new BitStream(File.OpenRead(args[0]));
// Create the video parser
VideoParser vp = new VideoParser(bs);
// Register our callback
CodingElement.handlers += new CodingElement.Handler(element_handler);
try {

// Parse pictures until the end of the stream
while (!vp.EOF) {

vp.parsePicture();

}

}
catch (OutOfBitsException e) {

// Handle an ungraceful end to the video stream
System.Console.WriteLine("Video file ended without sequence end code");

}

}
// This is the callback that will process the coding element
// events as they are published.
public static void element_handler(BitStream bs, CodingElement e) {

// All we do with them is print them to the console.
System.Console.WriteLine(e.ToString());

}

}

2004.11.23-09:24.00

Event Mechanism

Each class in the coding element event class hierarchy implements an event-based mechanism for informing your code via callbacks when a particular coding element has been parsed. To explain how this mechanism works, let's take a closer look at how it is implemented in the root CodingElement class.

The CodingElement class defines a delegate with the following signature:

public delegate void Handler(BitStream bs, CodingElement e)

This is the signature that your callback must adhere to. The first argument BitStream bs will be set to the BitStream object that the coding element was parsed from. This is useful if you are trying to process more than one stream at a time and need a handler to disambiguate which stream the coding element came from. The second argument, CodingElement e is set to the actual coding element that was parsed.

Given a callback with the appropriate signature, you construct a delegate for the callback in the normal C# way. As a reminder, this means if your callback is defined as:

void my_callback(BitStream bs, CodingElement e) {...}

Then you can create the delegate with code like:

CodingElement.Handler callback_delegate = new CodingElement.Handler(my_callback);

The delegate serves as a type-safe wrapper for the function pointer to your callback. Your callback can be an instance method of an object or a statically defined class method. The only requirement is that it match the signature given by the delegate definition. Once you have the delegate defined, you add it to the list of handlers for the coding element class using the statically defined event named handlers like so:

CodingElement.handlers += callback_delegate;

Now when a coding element is parsed, it publishes itself to all of the registered callbacks by invoking the registered delegates via the virtual protected method publish. This method first publishes the event to any parent classes. In this way, any and every callback registered with a particular coding element class or any ancestor of that class will be called for that coding element with the more general handlers being called before the more specific handlers.

Every subclass in the hierarchy under CodingElement implements this publication mechanism and by convention, each defines the delegate as Handler and maintains the registered callbacks in handlers. So, for example, the coding element class PictureCodingType, which is created when the picture coding type code is encountered in a picture header, defines the PictureCodingType.Handler delegate and maintains registered callbacks in the class variable PictureCodingType.handlers.

To remove a callback, simply "subtract" the delegate from the list of handlers like so:

CodingElement.handlers -= callback_delegate;

2004.11.23-09:24.00

This is a test.

Testing.

2004.11.23-09:24.00