背景

最近公司项目涉及到音视频的播放，之前采用的是两个 github 的开源项目直接使用，正常来说，是没啥问题的。这里推荐下这两个开源库，非常的 Nice，这两个库都有很详细的文档，接入使用都很方便:

音频播放库：StarrySky：A Powerful and Streamline MusicLibrary
视频播放库：DKVideoPlayer：A video player for Android.

可随着业务的发展，需要播放用户的音视频，随之而来的就是出现了不支持的音视频格式，这下好玩了。要是更改依赖，改动就有些大了，那么要怎么解决这个问题呢？

思考

既然要扩展，那么要综合考虑了。首先是视频播放，先看 DKVideoPlayer，支持 mediaplayer，exoplayer，ijkplayer 解码。mediaplayer 用作者的话来说：

不推荐使用，兼容性较差

那么就回到 exoplayer，ijkplayer 这两个，先说 ijkplayer 吧，bilibili 出品，目前31.4k star，很多人使用的音视频播放依赖库。上去看了下，上次的更新维护时间已经是两年前了，2.8k 个 open issues，2.5k 个 closed issues，应该是被抛弃了吧。这个库的好处呢，依赖于 ffmpeg，使用软解，支持格式较多，可通过自编译so扩展格式。基本上是可以满足绝大多数的视频正常播放的，库里面默认的支持格式并不全，要解决问题需要重新编译出so文件，替换了原本的so就好了，流程也是非常的简单。可以作为待选方案吧。再看看 exoplayer，这个库由Google 出品，目前 20.7k star，0.7k 个 open issues，9.2k 个 closed issues，上次更新在一周前，还是很及时的。正常来说 exoplayer 解码为硬解，支持格式得看设备 CPU，对于多年前或者低端 CPU，一些格式不支持。估计是 Google 考虑到这个问题，在查看官方文档 > Exoplayer官方文档 < 时候发现，Exoplayer 支持扩展，可以直接扩展 ffmpeg，从而得到更多的格式支持，并且可以选择多种扩展模式，官方说明如下 ffmpeg扩展：

Once you’ve followed the instructions above to check out, build and depend on
the module, the next step is to tell ExoPlayer to use FfmpegAudioRenderer. How
you do this depends on which player API you’re using:

• If you’re passing a DefaultRenderersFactory to ExoPlayer.Builder, you
  can enable using the module by setting the extensionRendererMode parameter
  of the DefaultRenderersFactory constructor to
  EXTENSION_RENDERER_MODE_ON. This will use FfmpegAudioRenderer for
  playback if MediaCodecAudioRenderer doesn’t support the input format. Pass
  EXTENSION_RENDERER_MODE_PREFER to give FfmpegAudioRenderer priority over
  MediaCodecAudioRenderer.
• If you’ve subclassed DefaultRenderersFactory, add an FfmpegAudioRenderer
  to the output list in buildAudioRenderers. ExoPlayer will use the first
  Renderer in the list that supports the input media format.
• If you’ve implemented your own RenderersFactory, return an
  FfmpegAudioRenderer instance from createRenderers. ExoPlayer will use
  the first Renderer in the returned array that supports the input media
  format.
• If you’re using ExoPlayer.Builder, pass an FfmpegAudioRenderer in the
  array of Renderers. ExoPlayer will use the first Renderer in the list
  that supports the input media format.

Note: These instructions assume you’re using DefaultTrackSelector. If you have
a custom track selector the choice of Renderer is up to your implementation,
so you need to make sure you are passing an FfmpegAudioRenderer to the player,
then implement your own logic to use the renderer for a given track.

第一条说的是设置 extensionRendererMode 的时候，EXTENSION_RENDERER_MODE_ON 是原本 Exoplayer 不支持的时候，ffmpeg 作为扩展。而 EXTENSION_RENDERER_MODE_PREFER 则是覆盖默认解码。
第二条则是将 FfmpegAudioRenderer 添加到列表，那么 Exoplayer 将会使用列表中第一个支持接码的解码器。
第三条和第二条说的一样，只是使用方法略有差异。
关于这个说明，在源码中可以看到逻辑：

protected void buildAudioRenderers(
     Context context,
     @ExtensionRendererMode int extensionRendererMode,
     MediaCodecSelector mediaCodecSelector,
     boolean enableDecoderFallback,
     AudioSink audioSink,
     Handler eventHandler,
     AudioRendererEventListener eventListener,
     ArrayList<Renderer> out) {
   MediaCodecAudioRenderer audioRenderer =
       new MediaCodecAudioRenderer(
           context,
           getCodecAdapterFactory(),
           mediaCodecSelector,
           enableDecoderFallback,
           eventHandler,
           eventListener,
           audioSink);
   out.add(audioRenderer);

   if (extensionRendererMode == EXTENSION_RENDERER_MODE_OFF) {
     return;
   }
   int extensionRendererIndex = out.size();
   if (extensionRendererMode == EXTENSION_RENDERER_MODE_PREFER) {
     extensionRendererIndex--;
   }

   try {
     Class<?> clazz = Class.forName("com.google.android.exoplayer2.decoder.midi.MidiRenderer");
     Constructor<?> constructor = clazz.getConstructor();
     Renderer renderer = (Renderer) constructor.newInstance();
     out.add(extensionRendererIndex++, renderer);
     Log.i(TAG, "Loaded MidiRenderer.");
   } catch (ClassNotFoundException e) {
     // Expected if the app was built without the extension.
   } catch (Exception e) {
     // The extension is present, but instantiation failed.
     throw new RuntimeException("Error instantiating MIDI extension", e);
   }

   try {
     // Full class names used for constructor args so the LINT rule triggers if any of them move.
     Class<?> clazz = Class.forName("com.google.android.exoplayer2.ext.opus.LibopusAudioRenderer");
     Constructor<?> constructor =
         clazz.getConstructor(
             android.os.Handler.class,
             com.google.android.exoplayer2.audio.AudioRendererEventListener.class,
             com.google.android.exoplayer2.audio.AudioSink.class);
     Renderer renderer =
         (Renderer) constructor.newInstance(eventHandler, eventListener, audioSink);
     out.add(extensionRendererIndex++, renderer);
     Log.i(TAG, "Loaded LibopusAudioRenderer.");
   } catch (ClassNotFoundException e) {
     // Expected if the app was built without the extension.
   } catch (Exception e) {
     // The extension is present, but instantiation failed.
     throw new RuntimeException("Error instantiating Opus extension", e);
   }

   try {
     // Full class names used for constructor args so the LINT rule triggers if any of them move.
     Class<?> clazz = Class.forName("com.google.android.exoplayer2.ext.flac.LibflacAudioRenderer");
     Constructor<?> constructor =
         clazz.getConstructor(
             android.os.Handler.class,
             com.google.android.exoplayer2.audio.AudioRendererEventListener.class,
             com.google.android.exoplayer2.audio.AudioSink.class);
     Renderer renderer =
         (Renderer) constructor.newInstance(eventHandler, eventListener, audioSink);
     out.add(extensionRendererIndex++, renderer);
     Log.i(TAG, "Loaded LibflacAudioRenderer.");
   } catch (ClassNotFoundException e) {
     // Expected if the app was built without the extension.
   } catch (Exception e) {
     // The extension is present, but instantiation failed.
     throw new RuntimeException("Error instantiating FLAC extension", e);
   }

   try {
     // Full class names used for constructor args so the LINT rule triggers if any of them move.
     Class<?> clazz =
         Class.forName("com.google.android.exoplayer2.ext.ffmpeg.FfmpegAudioRenderer");
     Constructor<?> constructor =
         clazz.getConstructor(
             android.os.Handler.class,
             com.google.android.exoplayer2.audio.AudioRendererEventListener.class,
             com.google.android.exoplayer2.audio.AudioSink.class);
     Renderer renderer =
         (Renderer) constructor.newInstance(eventHandler, eventListener, audioSink);
     out.add(extensionRendererIndex++, renderer);
     Log.i(TAG, "Loaded FfmpegAudioRenderer.");
   } catch (ClassNotFoundException e) {
     // Expected if the app was built without the extension.
   } catch (Exception e) {
     // The extension is present, but instantiation failed.
     throw new RuntimeException("Error instantiating FFmpeg extension", e);
   }
 }

这里有一点需要注意，这里使用反射去获取解码器，其包名定死了，正如注释所说可以定义 Link 规则。那么要么使用 Link 定义，要么使用 com.google.android.exoplayer2.ext.ffmpeg 包名。

这样看起来，exoplayer 和 ijkplayer 对于视频播放来说都是可以的。

那么这时候再来看看音频播放。StarrySky 默认使用的是 Exoplayer 解码，理论上来说是可以改成 ijkplayer 的，但本着能不改就不改的原则，这里就使用默认的就好了，把 Exoplayer 扩展加进去就完事儿。

想到这，方案就已经有了。那么接下来就开造~

方案

StarrySky + Exoplayer + ffmpeg扩展
DKVideoPlayer + Exoplayer + ffmpeg扩展
这时候对照着 Exoplayer ffmpeg 扩展文档操作就行了
没啥难的，也没卡点，偷个懒，把 Step to do 放这了。要注意一点 ENABLED_DECODERS 是设置需要解码的格式，具体支持可以看链接

Build instructions (Linux, macOS)

To use the module you need to clone this GitHub project and depend on its
modules locally. Instructions for doing this can be found in the
top level README. The module is not provided via Google’s Maven repository
(see #2781 for more information).

In addition, it’s necessary to manually build the FFmpeg library, so that gradle
can bundle the FFmpeg binaries in the APK:

• Set the following shell variable:

cd "<path to project checkout>"
FFMPEG_MODULE_PATH="$(pwd)/extensions/ffmpeg/src/main"

• Download the Android NDK and set its location in a shell variable.
  This build configuration has been tested on NDK r21.

NDK_PATH="<path to Android NDK>"

• Set the host platform (use “darwin-x86_64” for Mac OS X):

HOST_PLATFORM="linux-x86_64"

• Fetch FFmpeg and checkout an appropriate branch. We cannot guarantee
  compatibility with all versions of FFmpeg. We currently recommend version 4.2:

cd "<preferred location for ffmpeg>" && \
git clone git://source.ffmpeg.org/ffmpeg && \
cd ffmpeg && \
git checkout release/4.2 && \
FFMPEG_PATH="$(pwd)"

• Configure the decoders to include. See the Supported formats page for
  details of the available decoders, and which formats they support.

ENABLED_DECODERS=(vorbis opus flac)

• Add a link to the FFmpeg source code in the FFmpeg module jni directory.

cd "${FFMPEG_MODULE_PATH}/jni" && \
ln -s "$FFMPEG_PATH" ffmpeg

• Execute build_ffmpeg.sh to build FFmpeg for armeabi-v7a, arm64-v8a,
  x86 and x86_64. The script can be edited if you need to build for
  different architectures:

cd "${FFMPEG_MODULE_PATH}/jni" && \
./build_ffmpeg.sh \
  "${FFMPEG_MODULE_PATH}" "${NDK_PATH}" "${HOST_PLATFORM}" "${ENABLED_DECODERS[@]}"

Build instructions (Windows)

We do not provide support for building this module on Windows, however it should
be possible to follow the Linux instructions in Windows PowerShell.

到这里，就已经生成需要的所需要的 .a 静态库。接下来稍作修改就可以运用到 StarrySky 和 DKVideoPlayer 了。

由于笔者有点强迫症，对目录和 CMakeLists.txt 做了调整，做成一个依赖库，以便使用。当然读者可以直接依赖这个库，可以直接完成 Exoplayer 扩展。
Github 地址：Exoplayer FFmpeg Extension

在 StarrySky 中使用

emmm~ 这个在 ExoPlayback 已经设置了，因此只需要引用就行了

@Synchronized
private fun createExoPlayer() {
    if (player == null) {
    	// 这里已经设置了扩展，引入依赖就行了
        @ExtensionRendererMode val extensionRendererMode = DefaultRenderersFactory.EXTENSION_RENDERER_MODE_PREFER
        val renderersFactory = DefaultRenderersFactory(context)
            .setExtensionRendererMode(extensionRendererMode)

        trackSelectorParameters = ParametersBuilder(context).build()
        trackSelector = DefaultTrackSelector(context)
        trackSelector?.parameters = trackSelectorParameters as DefaultTrackSelector.Parameters

        player = SimpleExoPlayer.Builder(context, renderersFactory)
            .setTrackSelector(trackSelector!!)
            .build()

        player?.addListener(eventListener)
        player?.setAudioAttributes(AudioAttributes.DEFAULT, isAutoManagerFocus)
        if (!isAutoManagerFocus) {
            player?.playbackState?.let { focusManager.updateAudioFocus(getPlayWhenReady(), it) }
        }
    }
}

在 DKVideoPlayer 中使用

在示例 Sample 中将 ExoVideoView 和 CustomExoMediaPlayer 拷到项目中的合适位置，初始化完成时候调用：

···
mExoVideoView.setRenderersFactory(new DefaultRenderersFactory(this)
  .setExtensionRendererMode(DefaultRenderersFactory.EXTENSION_RENDERER_MODE_PREFER));

或者使用库里面的 FfmpegRenderersFactory：

···
mExoVideoView.setRenderersFactory(new FfmpegRenderersFactory(this));

效果

ExoPlayer       D  tracks [eventTime=0.18, mediaPos=0.00, window=0, period=0
ExoPlayer       D    group [
ExoPlayer       D      [X] Track:0, id=1, mimeType=video/avc, codecs=avc1.640029, res=848x480, language=en, selectionFlags=[default], supported=YES
ExoPlayer       D    ]
ExoPlayer       D    group [
ExoPlayer       D      [X] Track:0, id=2, mimeType=audio/ac3, channels=6, sample_rate=48000, language=und, selectionFlags=[default], supported=YES
ExoPlayer       D    ]
ExoPlayer       D    group [
ExoPlayer       D      [X] Track:0, id=3, mimeType=application/x-subrip, language=en, label=英文字幕, selectionFlags=[default], supported=YES
ExoPlayer       D    ]
ExoPlayer       D    group [
ExoPlayer       D      [ ] Track:0, id=4, mimeType=application/x-subrip, language=und, label=中英双字幕, supported=YES
ExoPlayer       D    ]
ExoPlayer       D    group [
ExoPlayer       D      [ ] Track:0, id=5, mimeType=application/x-subrip, language=zh, label=中文字幕, supported=YES
ExoPlayer       D    ]
ExoPlayer       D  ]

可以看到原本不支持的 AC3 已经支持了，完美收场~