HTML5 Live Audio Streaming with ffserver

Recently I've successfully set up an HTML5 live audio streaming by using ffmpeg and ffserver. It streams well on most HTML5 browsers including latest IE9, Firefox and Chrome. The streaming source comes from an PC running Windows, and the streaming server runs on FreeBSD.

At streaming source, to make all your sounds (i.e. what you can hear) capable of being captured by ffmpeg, you need to create an virtual device which is DirectShow-compatible so that the latest ffmpeg can recognize it as an audio capturer.

• download and install virtual-audio-capture-device.
• download latest 32-bit Zeranoe build ffmpeg. (must 32bit one)

There should exist an virtual-audio-capturer entry after checking with the command,

ffmpeg -f dshow -list_devices true -i dummy

and the following is the command to stream what you hear out to the remote server.

ffmpeg -f dshow -i audio=virtual-audio-capturer http://server:8090/feed.ffm

At server side, we need to configure two kinds of audio streams in order to provide HTML5 audio compatibilities among mainstream browsers,

• set up an mp3 stream with libmp3lame audio codec. (IE, Safari, Chrome)
• set up an webm stream with libvorbis audio codec. (Firefox, Opera, Chrome)

Adding one more line to both of the streams you just set up,

AVOptionAudio flags +global_header

to make them live streaming capable.

You can directly open the streaming in VLC(webm, mp3) or foobar2000(mp3) to test whether it works or not. And note that there are still some issues between ffmpeg and ffserver while streaming.

The MP3 Stream Cannot be Played on Internet Explorer 9

After googling, I just know that IE9 accepts only server response of audio/mpeg as mime-type to mp3 stream, whereas ffserver(N-42472-gd3abbb1) responds the old audio/x-mpeg one.

Here is the dirty one-line fix to the issue,

diff --git a/libavformat/mp3enc.c b/libavformat/mp3enc.c
index 8a9f788..14ec70c 100644
--- a/libavformat/mp3enc.c
+++ b/libavformat/mp3enc.c
@@ -499,7 +499,7 @@ static int mp3_write_trailer(AVFormatContext *s)
 AVOutputFormat ff_mp3_muxer = {
     .name              = "mp3",
     .long_name         = NULL_IF_CONFIG_SMALL("MPEG audio layer 3"),
-    .mime_type         = "audio/x-mpeg",
+    .mime_type         = "audio/mpeg",
     .extensions        = "mp3",
     .priv_data_size    = sizeof(MP3Context),
     .audio_codec       = CODEC_ID_MP3,

then ffserver feeds IE9 happily.

OGG Stream Format Plays Abnormal on Most Players

When specifying the libvorbis stream with OGG format, audio stream being played on Firefox or Opera will come off and on. The problem exists by directly opening it using VLC or foobar2000. Hence I think it maybe an unknown bug comes with ffserver's OGG format.

For the same libvorbis stream, I changed it to using WEBM format, and everything is OK. So I recommend to choose webm as your encapsulation format for libvorbis streams.

Libvorbis Transcoding Fails in a Ranged Bitrate

For the libvorbis codecs, I encountered a strange behavior of transcoding failed at source ffmpeg(N-42347-g299387e). It failed at bitrate 128kb, 160kb and any value lower than 192kb.

I haven't find out how to fix it yet, so just setting bitrate to 192kb at the moment to prevent the failure transcoding.

Realtime Buffer X% Full

if you see this at source point occasionally while streaming stop,

[dshow @ 01f1c000] real-time buffer 1412% full! frame dropped!

You may increase buffer size by adding rtbufsize option to prevent this error,

ffmpeg -rtbufsize 512000k -f dshow ...