Added in API level 16

public final class

MediaCodec

extends Object

java.lang.Object
↳	android.media.MediaCodec

Class Overview

MediaCodec class can be used to access low-level media codec, i.e. encoder/decoder components.

MediaCodec is generally used like this:

 MediaCodec codec = MediaCodec.createDecoderByType(type);
 codec.configure(format, ...);
 codec.start();

 // if API level <= 20, get input and output buffer arrays here
 ByteBuffer[] inputBuffers = codec.getInputBuffers();
 ByteBuffer[] outputBuffers = codec.getOutputBuffers();
 for (;;) {
   int inputBufferIndex = codec.dequeueInputBuffer(timeoutUs);
   if (inputBufferIndex >= 0) {
     // if API level >= 21, get input buffer here
     ByteBuffer inputBuffer = codec.getInputBuffer(inputBufferIndex);
     // fill inputBuffers[inputBufferIndex] with valid data
     ...
     codec.queueInputBuffer(inputBufferIndex, ...);
   }

   int outputBufferIndex = codec.dequeueOutputBuffer(timeoutUs);
   if (outputBufferIndex >= 0) {
     // if API level >= 21, get output buffer here
     ByteBuffer outputBuffer = codec.getOutputBuffer(outputBufferIndex);
     // outputBuffer is ready to be processed or rendered.
     ...
     codec.releaseOutputBuffer(outputBufferIndex, ...);
   } else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
     // no needed to handle if API level >= 21 and using getOutputBuffer(int)
     outputBuffers = codec.getOutputBuffers();
   } else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
     // Subsequent data will conform to new format.
     // can ignore if API level >= 21 and using getOutputFormat(outputBufferIndex)
     MediaFormat format = codec.getOutputFormat();
     ...
   }
 }
 codec.stop();
 codec.release();
 codec = null;

Each codec maintains a number of input and output buffers that are referred to by index in API calls.

For API levels 20 and below: The contents of these buffers are represented by the ByteBuffer[] arrays accessible through getInputBuffers() and getOutputBuffers().

After a successful call to start() the client "owns" neither input nor output buffers, subsequent calls to dequeueInputBuffer(long) and dequeueOutputBuffer(MediaCodec.BufferInfo, long) then transfer ownership from the codec to the client.

The client is not required to resubmit/release buffers immediately to the codec, the sample code above simply does this for simplicity's sake. Nonetheless, it is possible that a codec may hold off on generating output buffers until all outstanding buffers have been released/resubmitted.

Once the client has an input buffer available it can fill it with data and submit it it to the codec via a call to queueInputBuffer(int, int, int, long, int). Do not submit multiple input buffers with the same timestamp (unless it is codec-specific data marked as such using the flag BUFFER_FLAG_CODEC_CONFIG).

The codec in turn will return an output buffer to the client in response to dequeueOutputBuffer(MediaCodec.BufferInfo, long). After the output buffer has been processed a call to releaseOutputBuffer(int, boolean) will return it to the codec. If a video surface has been provided in the call to configure(MediaFormat, Surface, MediaCrypto, int), releaseOutputBuffer(int, boolean) optionally allows rendering of the buffer to the surface.

Input buffers (for decoders) and Output buffers (for encoders) contain encoded data according to the format's type. For video types this data is all the encoded data representing a single moment in time, for audio data this is slightly relaxed in that a buffer may contain multiple encoded frames of audio. In either case, buffers do not start and end on arbitrary byte boundaries, this is not a stream of bytes, it's a stream of access units.

Most formats also require the actual data to be prefixed by a number of buffers containing setup data, or codec specific data, i.e. the first few buffers submitted to the codec object after starting it must be codec specific data marked as such using the flag BUFFER_FLAG_CODEC_CONFIG in a call to queueInputBuffer(int, int, int, long, int).

Codec specific data included in the format passed to configure(MediaFormat, Surface, MediaCrypto, int) (in ByteBuffer entries with keys "csd-0", "csd-1", ...) is automatically submitted to the codec, this data MUST NOT be submitted explicitly by the client.

Once the client reaches the end of the input data it signals the end of the input stream by specifying a flag of BUFFER_FLAG_END_OF_STREAM in the call to queueInputBuffer(int, int, int, long, int). The codec will continue to return output buffers until it eventually signals the end of the output stream by specifying the same flag (BUFFER_FLAG_END_OF_STREAM) on the BufferInfo returned in dequeueOutputBuffer(MediaCodec.BufferInfo, long). Do not submit additional input buffers after signaling the end of the input stream, unless the codec has been flushed, or stopped and restarted.

Seeking & Adaptive Playback Support

You can check if a decoder supports adaptive playback via isFeatureSupported(String). Adaptive playback is only supported if you configure the codec to decode onto a Surface.

For decoders that do not support adaptive playback (including when not decoding onto a Surface)

In order to start decoding data that's not adjacent to previously submitted data (i.e. after a seek) one must flush() the decoder. Any input or output buffers the client may own at the point of the flush are immediately revoked, i.e. after a call to flush() the client does not own any buffers anymore.

It is important that the input data after a flush starts at a suitable stream boundary. The first frame must be able to be decoded completely on its own (for most codecs this means an I-frame), and that no frames should refer to frames before that first new frame. Note that the format of the data submitted after a flush must not change, flush does not support format discontinuities, for this a full stop(), configure(), start() cycle is necessary.

For decoders that support adaptive playback

In order to start decoding data that's not adjacent to previously submitted data (i.e. after a seek) it is not necessary to flush() the decoder.

It is still important that the input data after the discontinuity starts at a suitable stream boundary (e.g. I-frame), and that no new frames refer to frames before the first frame of the new input data segment.

For some video formats it is also possible to change the picture size mid-stream. To do this for H.264, the new Sequence Parameter Set (SPS) and Picture Parameter Set (PPS) values must be packaged together with an Instantaneous Decoder Refresh (IDR) frame in a single buffer, which then can be enqueued as a regular input buffer. The client will receive an INFO_OUTPUT_FORMAT_CHANGED return value from dequeueOutputBuffer() or onOutputBufferAvailable() just after the picture-size change takes place and before any frames with the new size have been returned.

Be careful when calling flush() shortly after you have changed the picture size. If you have not received confirmation of the picture size change, you will need to repeat the request for the new picture size. E.g. for H.264 you will need to prepend the PPS/SPS to the new IDR frame to ensure that the codec receives the picture size change request.

States and error handling

During its life, a codec conceptually exists in one of the following states: Initialized, Configured, Executing, Error, Uninitialized, (omitting transitory states between them). When created by one of the factory methods, the codec is in the Initialized state; configure(MediaFormat, Surface, MediaCrypto, int) brings it to the Configured state; start() brings it to the Executing state. In the Executing state, decoding or encoding occurs through the buffer queue manipulation described above. The method stop() returns the codec to the Initialized state, whereupon it may be configured again, and release() brings the codec to the terminal Uninitialized state. When a codec error occurs, the codec moves to the Error state. Use reset() to bring the codec back to the Initialized state, or release() to move it to the Uninitialized state.

The factory methods createByCodecName(String), createDecoderByType(String), and createEncoderByType(String) throw IOException on failure which the caller must catch or declare to pass up. MediaCodec methods throw IllegalStateException when the method is called from a codec state that does not allow it; this is typically due to incorrect application API usage. Methods involving secure buffers may throw MediaCodec.CryptoException(int, String), which has further error information obtainable from getErrorCode().

Internal codec errors result in a MediaCodec.CodecException, which may be due to media content corruption, hardware failure, resource exhaustion, and so forth, even when the application is correctly using the API. The recommended action when receiving a MediaCodec.CodecException can be determined by calling isRecoverable() and isTransient(). If isRecoverable() returns true, then a stop(), configure(MediaFormat, Surface, MediaCrypto, int), and start() can be performed to recover. If isTransient() returns true, then resources are temporarily unavailable and the method may be retried at a later time. If both isRecoverable() and isTransient() return false, then the MediaCodec.CodecException is fatal and the codec must be reset or released. Both isRecoverable() and isTransient() do not return true at the same time.

Summary

Nested Classes
class	MediaCodec.BufferInfo	Per buffer metadata includes an offset and size specifying the range of valid data in the associated codec (output) buffer.
class	MediaCodec.Callback	MediaCodec callback interface.
class	MediaCodec.CodecException	Thrown when an internal codec error occurs.
class	MediaCodec.CryptoException	Thrown when a crypto error occurs while queueing a secure input buffer.
class	MediaCodec.CryptoInfo	Metadata describing the structure of a (at least partially) encrypted input sample.

Constants
int	BUFFER_FLAG_CODEC_CONFIG	This indicated that the buffer marked as such contains codec initialization / codec specific data instead of media data.
int	BUFFER_FLAG_END_OF_STREAM	This signals the end of stream, i.e.
int	BUFFER_FLAG_KEY_FRAME	This indicates that the (encoded) buffer marked as such contains the data for a key frame.
int	BUFFER_FLAG_SYNC_FRAME	This constant was deprecated in API level 21. Use `BUFFER_FLAG_KEY_FRAME` instead.
int	CONFIGURE_FLAG_ENCODE	If this codec is to be used as an encoder, pass this flag.
int	CRYPTO_MODE_AES_CTR
int	CRYPTO_MODE_UNENCRYPTED
int	INFO_OUTPUT_BUFFERS_CHANGED	This constant was deprecated in API level 21. This return value can be ignored as `getOutputBuffers()` has been deprecated. Client should request a current buffer using on of the get-buffer or get-image methods each time one has been dequeued.
int	INFO_OUTPUT_FORMAT_CHANGED	The output format has changed, subsequent data will follow the new format.
int	INFO_TRY_AGAIN_LATER	If a non-negative timeout had been specified in the call to `dequeueOutputBuffer(MediaCodec.BufferInfo, long)`, indicates that the call timed out.
String	PARAMETER_KEY_REQUEST_SYNC_FRAME	Request that the encoder produce a sync frame "soon".
String	PARAMETER_KEY_SUSPEND	Temporarily suspend/resume encoding of input data.
String	PARAMETER_KEY_VIDEO_BITRATE	Change a video encoder's target bitrate on the fly.
int	VIDEO_SCALING_MODE_SCALE_TO_FIT	The content is scaled to the surface dimensions
int	VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING	The content is scaled, maintaining its aspect ratio, the whole surface area is used, content may be cropped

Public Methods
void	configure(MediaFormat format, Surface surface, MediaCrypto crypto, int flags) Configures a component.
static MediaCodec	createByCodecName(String name) If you know the exact name of the component you want to instantiate use this method to instantiate it.
static MediaCodec	createDecoderByType(String type) Instantiate a decoder supporting input data of the given mime type.
static MediaCodec	createEncoderByType(String type) Instantiate an encoder supporting output data of the given mime type.
final Surface	createInputSurface() Requests a Surface to use as the input to an encoder, in place of input buffers.
final int	dequeueInputBuffer(long timeoutUs) Returns the index of an input buffer to be filled with valid data or -1 if no such buffer is currently available.
final int	dequeueOutputBuffer(MediaCodec.BufferInfo info, long timeoutUs) Dequeue an output buffer, block at most "timeoutUs" microseconds.
final void	flush() Flush both input and output ports of the component, all indices previously returned in calls to `dequeueInputBuffer(long)` and `dequeueOutputBuffer(MediaCodec.BufferInfo, long)` become invalid.
MediaCodecInfo	getCodecInfo() Get the codec info.
ByteBuffer	getInputBuffer(int index) Returns a `cleared`, writable ByteBuffer object for a dequeued input buffer index to contain the input data.
ByteBuffer[]	getInputBuffers() This method was deprecated in API level 21. Use the new `getInputBuffer(int)` method instead each time an input buffer is dequeued. Note:As of API 21, dequeued input buffers are automatically `cleared`.
final MediaFormat	getInputFormat() Call this after `configure(MediaFormat, Surface, MediaCrypto, int)` returns successfully to get the input format accepted by the codec.
Image	getInputImage(int index) Returns a writable Image object for a dequeued input buffer index to contain the raw input video frame.
final String	getName() Get the component name.
ByteBuffer	getOutputBuffer(int index) Returns a read-only ByteBuffer for a dequeued output buffer index.
ByteBuffer[]	getOutputBuffers() This method was deprecated in API level 21. Use the new `getOutputBuffer(int)` method instead each time an output buffer is dequeued. This method is not supported if codec is configured in asynchronous mode. Note:As of API 21, the position and limit of output buffers that are dequeued will be set to the valid data range.
final MediaFormat	getOutputFormat() Call this after dequeueOutputBuffer signals a format change by returning `INFO_OUTPUT_FORMAT_CHANGED`.
final MediaFormat	getOutputFormat(int index) Returns the output format for a specific output buffer.
Image	getOutputImage(int index) Returns a read-only Image object for a dequeued output buffer index that contains the raw video frame.
final void	queueInputBuffer(int index, int offset, int size, long presentationTimeUs, int flags) After filling a range of the input buffer at the specified index submit it to the component.
final void	queueSecureInputBuffer(int index, int offset, MediaCodec.CryptoInfo info, long presentationTimeUs, int flags) Similar to `queueInputBuffer(int, int, int, long, int)` but submits a buffer that is potentially encrypted.
final void	release() Make sure you call this when you're done to free up any opened component instance instead of relying on the garbage collector to do this for you at some point in the future.
final void	releaseOutputBuffer(int index, boolean render) If you are done with a buffer, use this call to return the buffer to the codec.
final void	releaseOutputBuffer(int index, long renderTimestampNs) If you are done with a buffer, use this call to update its surface timestamp and return it to the codec to render it on the output surface.
final void	reset() Returns the codec to its initial (Initialized) state.
void	setCallback(MediaCodec.Callback cb) Sets an asynchronous callback for actionable MediaCodec events.
final void	setParameters(Bundle params) Communicate additional parameter changes to the component instance.
final void	setVideoScalingMode(int mode) If a surface has been specified in a previous call to `configure(MediaFormat, Surface, MediaCrypto, int)` specifies the scaling mode to use.
final void	signalEndOfInputStream() Signals end-of-stream on input.
final void	start() After successfully configuring the component, call `start`.
final void	stop() Finish the decode/encode session, note that the codec instance remains active and ready to be `start()`ed again.

Protected Methods
void	finalize() Invoked when the garbage collector has detected that this instance is no longer reachable.

[Expand]

Inherited Methods

From class java.lang.Object

Object	clone() Creates and returns a copy of this `Object`.
boolean	equals(Object o) Compares this instance with the specified object and indicates if they are equal.
void	finalize() Invoked when the garbage collector has detected that this instance is no longer reachable.
final Class<?>	getClass() Returns the unique instance of `Class` that represents this object's class.
int	hashCode() Returns an integer hash code for this object.
final void	notify() Causes a thread which is waiting on this object's monitor (by means of calling one of the `wait()` methods) to be woken up.
final void	notifyAll() Causes all threads which are waiting on this object's monitor (by means of calling one of the `wait()` methods) to be woken up.
String	toString() Returns a string containing a concise, human-readable description of this object.
final void	wait() Causes the calling thread to wait until another thread calls the `notify()` or `notifyAll()` method of this object.
final void	wait(long millis, int nanos) Causes the calling thread to wait until another thread calls the `notify()` or `notifyAll()` method of this object or until the specified timeout expires.
final void	wait(long millis) Causes the calling thread to wait until another thread calls the `notify()` or `notifyAll()` method of this object or until the specified timeout expires.

Constants

public static final int BUFFER_FLAG_CODEC_CONFIG

Added in API level 16

This indicated that the buffer marked as such contains codec initialization / codec specific data instead of media data.

Constant Value: 2 (0x00000002)

public static final int BUFFER_FLAG_END_OF_STREAM

Added in API level 16

This signals the end of stream, i.e. no buffers will be available after this, unless of course, flush() follows.

Constant Value: 4 (0x00000004)

public static final int BUFFER_FLAG_KEY_FRAME

Added in API level 21

This indicates that the (encoded) buffer marked as such contains the data for a key frame.

Constant Value: 1 (0x00000001)

public static final int BUFFER_FLAG_SYNC_FRAME

Added in API level 16

This constant was deprecated in API level 21.
Use BUFFER_FLAG_KEY_FRAME instead.

This indicates that the (encoded) buffer marked as such contains the data for a key frame.

Constant Value: 1 (0x00000001)

public static final int CONFIGURE_FLAG_ENCODE

Added in API level 16

If this codec is to be used as an encoder, pass this flag.

Constant Value: 1 (0x00000001)

public static final int CRYPTO_MODE_AES_CTR

Added in API level 16

Constant Value: 1 (0x00000001)

public static final int CRYPTO_MODE_UNENCRYPTED

Added in API level 16

Constant Value: 0 (0x00000000)

public static final int INFO_OUTPUT_BUFFERS_CHANGED

Added in API level 16

This constant was deprecated in API level 21.
This return value can be ignored as getOutputBuffers() has been deprecated. Client should request a current buffer using on of the get-buffer or get-image methods each time one has been dequeued.

The output buffers have changed, the client must refer to the new set of output buffers returned by getOutputBuffers() from this point on.

Constant Value: -3 (0xfffffffd)

public static final int INFO_OUTPUT_FORMAT_CHANGED

Added in API level 16

The output format has changed, subsequent data will follow the new format. getOutputFormat() returns the new format. Note, that you can also use the new getOutputFormat(int) method to get the format for a specific output buffer. This frees you from having to track output format changes.

Constant Value: -2 (0xfffffffe)

public static final int INFO_TRY_AGAIN_LATER

Added in API level 16

If a non-negative timeout had been specified in the call to dequeueOutputBuffer(MediaCodec.BufferInfo, long), indicates that the call timed out.

Constant Value: -1 (0xffffffff)

public static final String PARAMETER_KEY_REQUEST_SYNC_FRAME

Added in API level 19

Request that the encoder produce a sync frame "soon". Provide an Integer with the value 0.

Constant Value: "request-sync"

public static final String PARAMETER_KEY_SUSPEND

Added in API level 19

Temporarily suspend/resume encoding of input data. While suspended input data is effectively discarded instead of being fed into the encoder. This parameter really only makes sense to use with an encoder in "surface-input" mode, as the client code has no control over the input-side of the encoder in that case. The value is an Integer object containing the value 1 to suspend or the value 0 to resume.

Constant Value: "drop-input-frames"

public static final String PARAMETER_KEY_VIDEO_BITRATE

Added in API level 19

Change a video encoder's target bitrate on the fly. The value is an Integer object containing the new bitrate in bps.

Constant Value: "video-bitrate"

public static final int VIDEO_SCALING_MODE_SCALE_TO_FIT

Added in API level 16

The content is scaled to the surface dimensions

Constant Value: 1 (0x00000001)

public static final int VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING

Added in API level 16

The content is scaled, maintaining its aspect ratio, the whole surface area is used, content may be cropped

Constant Value: 2 (0x00000002)

format	The format of the input data (decoder) or the desired format of the output data (encoder).
surface	Specify a surface on which to render the output of this decoder.
crypto	Specify a crypto object to facilitate secure decryption of the media data.
flags	Specify `CONFIGURE_FLAG_ENCODE` to configure the component as an encoder.

IllegalArgumentException	if the surface has been released (or is invalid), or the format is unacceptable (e.g. missing a mandatory key), or the flags are not set properly (e.g. missing `CONFIGURE_FLAG_ENCODE` for an encoder).
IllegalStateException	if not in the Initialized state.

IOException	if the codec cannot be created.
IllegalArgumentException	if name is not valid.
NullPointerException	if name is null.

IllegalStateException	if not in the Executing state, or codec is configured in asynchronous mode.
MediaCodec.CodecException	upon codec error.

info	Will be filled with buffer meta data.
timeoutUs	The timeout in microseconds, a negative timeout indicates "infinite".

IllegalStateException	if not in the Executing state.
MediaCodec.CodecException	upon codec error.

IllegalStateException	if not in the Executing or Configured state.
MediaCodec.CodecException	upon codec error.

index	The index of a client-owned input buffer previously returned in a call to `dequeueInputBuffer(long)`.
offset	The byte offset into the input buffer at which the data starts.
size	The number of bytes of valid input data.
presentationTimeUs	The presentation timestamp in microseconds for this buffer. This is normally the media time at which this buffer should be presented (rendered).
flags	A bitmask of flags `BUFFER_FLAG_CODEC_CONFIG` and `BUFFER_FLAG_END_OF_STREAM`. While not prohibited, most codecs do not use the `BUFFER_FLAG_KEY_FRAME` flag for input buffers.

index	The index of a client-owned output buffer previously returned from a call to `dequeueOutputBuffer(MediaCodec.BufferInfo, long)`.
render	If a valid surface was specified when configuring the codec, passing true renders this output buffer to the surface.

index	The index of a client-owned output buffer previously returned from a call to `dequeueOutputBuffer(MediaCodec.BufferInfo, long)`.
renderTimestampNs	The timestamp to associate with this buffer when it is sent to the Surface.

MediaCodec.CodecException	if an unrecoverable error has occured and the codec could not be reset.
IllegalStateException	if in the Uninitialized state.

IllegalArgumentException	if mode is not recognized.
IllegalStateException	if in the Uninitialized state.

IllegalStateException	if not in the Configured state or just after `flush()` for a codec that is configured in asynchronous mode.
MediaCodec.CodecException	upon codec error. Note that some codec errors for start may be attributed to future method calls.

Results

Interfaces

Classes

Exceptions

MediaCodec

Class Overview

Seeking & Adaptive Playback Support

For decoders that do not support adaptive playback (including when not decoding onto a Surface)

For decoders that support adaptive playback

States and error handling

Summary

Constants

public static final int BUFFER_FLAG_CODEC_CONFIG

public static final int BUFFER_FLAG_END_OF_STREAM

public static final int BUFFER_FLAG_KEY_FRAME

public static final int BUFFER_FLAG_SYNC_FRAME

public static final int CONFIGURE_FLAG_ENCODE

public static final int CRYPTO_MODE_AES_CTR

public static final int CRYPTO_MODE_UNENCRYPTED

public static final int INFO_OUTPUT_BUFFERS_CHANGED

public static final int INFO_OUTPUT_FORMAT_CHANGED

public static final int INFO_TRY_AGAIN_LATER

public static final String PARAMETER_KEY_REQUEST_SYNC_FRAME

public static final String PARAMETER_KEY_SUSPEND

public static final String PARAMETER_KEY_VIDEO_BITRATE

public static final int VIDEO_SCALING_MODE_SCALE_TO_FIT

public static final int VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING

Public Methods

public void configure (MediaFormat format, Surface surface, MediaCrypto crypto, int flags)

Parameters

Throws

public static MediaCodec createByCodecName (String name)

Parameters

Throws

public static MediaCodec createDecoderByType (String type)

Parameters

Throws

public static MediaCodec createEncoderByType (String type)

Parameters

Throws

public final Surface createInputSurface ()

Throws

public final int dequeueInputBuffer (long timeoutUs)

Parameters

Throws

public final int dequeueOutputBuffer (MediaCodec.BufferInfo info, long timeoutUs)

Parameters

Throws

public final void flush ()

Throws

public MediaCodecInfo getCodecInfo ()

Throws

public ByteBuffer getInputBuffer (int index)

Parameters

Returns

Throws

public ByteBuffer[] getInputBuffers ()

Throws

public final MediaFormat getInputFormat ()

Throws

public Image getInputImage (int index)

Parameters

Returns

Throws

public final String getName ()

Throws

public ByteBuffer getOutputBuffer (int index)

Parameters

Returns

Throws

public ByteBuffer[] getOutputBuffers ()

Throws

public final MediaFormat getOutputFormat ()

Throws

public final MediaFormat getOutputFormat (int index)

Parameters

Returns

public Image getOutputImage (int index)

Parameters

Returns