Skip navigation
Currently Being Moderated

FP 11+ Audio Encoding Issue With H.264 Video

Nov 4, 2011 10:37 AM

Tags: #h.264 #fms_4.5 #nellymoser

FP 11+ Audio Encoding Issue With H.264 Video

 

 

I'm not sure if this issue is with the Flash Player when encoding the stream or with the Flash Media Server that is saving the stream so I'll be posting in both forums.

 

The problem is when publishing an audio and video stream from Flash Player 11 (Windows, FF) to the Flash Server 4.5 using the H.264 video codec. The audio portion of the recorded stream doesn't appear to be playable by anything and FFMPEG cannot process the audio stream. Here are the differences in the stream:

 

When using Sorenson as the video codec and recording the stream (flv) from the Flash server, the information about the audio stream from FFMPEG is this:

 

Stream #0.0: Audio: nellymoser, 22050 Hz, mono, s16

 

This clip can be played and processed by various players and FFMPEG with no problem.

 

When using H.264 as the video codec and recording the stream (f4v) from the Flash server, the information about the audio stream from FFMPEG is this:

 

Stream #0.1(eng): Audio: nmos / 0x736F6D6E, 22050 Hz, 1 channels

 

This clip does not play the audio and cannot be understood by FFMPEG.

 

I tried using the Speex codec as well and I have a similar issue as above (it doesn't play the audio stream and FFMPEG can't understand it).

 

Note the version of FFMPEG I'm using supports both Nellymoser and Speex so there should not be an issue.

 

1) Any idea why the codec appears different and won't play.

 

2) How to achieve the goal: send H.264 video stream with audio from the Flash Player and have a recorded stream with playable audio.

 

Thanks!

 
Replies
  • Currently Being Moderated
    Nov 7, 2011 9:13 PM   in reply to ImAStreamer

    After you record the stream to Flash Media Server, process it with the F4V Post Processor tool to play it back on the desktop or using progressive download (you can stream it without using the Post Processor):

    http://www.adobe.com/go/fms_tools

     
    |
    Mark as:
  • Currently Being Moderated
    Nov 9, 2011 12:10 AM   in reply to ImAStreamer

    My FP11 H264 recording content plays fine, when live and as vod content via FMS. However nothing recognises it like, media info, mp4info, mpui or FFmpeg when inspected and even VLC Player can not play it. FLVCheck says its ok. MP4UI did say the atom size was too small, so I can only think FP11 is not creating the container within the MP4 spec.

     

    I would like this raised as a bug! Point me to the correct url please.

     
    |
    Mark as:
  • Currently Being Moderated
    Nov 9, 2011 4:45 AM   in reply to Adrian_NA

    Not sure about FMS, but I find that when using Wowza, if I publish h.264 / speex from the flashplayer, and record to an FLV container, FFMPEG is able to process the file.

     

    Keep in mind that neither Nellymoser Asao nor Speex are MP4 compliant, so I would expect most applications to fail to read those files.

     
    |
    Mark as:
  • Currently Being Moderated
    Nov 9, 2011 4:57 AM   in reply to JayCharles

    Thanks. I can process now, not sure I was doing wrong before

     
    |
    Mark as:
  • Currently Being Moderated
    Nov 9, 2011 6:17 AM   in reply to ImAStreamer

    @ImAStreamer - Yes, that's what I'm suggesting. FFMPEG should be able to convert to a proper mp4 container. Just be sure to change the output audio codec to something mp4 compliant (mp3 or AAC)

     
    |
    Mark as:
  • Currently Being Moderated
    Nov 9, 2011 6:43 AM   in reply to ImAStreamer

    One more bit of advice...

     

    The AAC encoder in FFMPEG (FAAC) is terrible, and tends to result in poor quality at low bitrates. If you're transcoding to AAC, you'll likely want to use 128kbps or higher. If you workflow and endpoint targets allow for it, I'd go with MP3 audio.

     
    |
    Mark as:
  • Currently Being Moderated
    Nov 9, 2011 6:58 AM   in reply to ImAStreamer

    From a quality perspective, I prefer Nellymoser. Speex encoding in FP is limited to 16kHz, where Nellymoser is capable of 44.1khz. For voice, the difference isn't all that noticeable, but if there's any music involved, you'll find that the sounds is a bit dull when using speex, as the higher frequencies aren't represented.

     

    Also, if your application involves multiway communication and you want to take advantage of AEC, your only option is Speex, as AEC doesn't work with Nellymoser.

     

    From a licensing standpoint, Speex is the clear winner. To be completely legal, you'd need a license from Nellymoser to decode the audio on the server side using FFMPEG, and that license is $7500 per year per server the last time I checked. Speex on the other hand is not patent encumbered, and (AFAIK) requires no licensing.

     

    That said, if you're in a position where codec licensing is an issue, you'll want to look into an MP3 encoding license for your implementation as well.

     
    |
    Mark as:
  • Currently Being Moderated
    Nov 9, 2011 11:58 AM   in reply to ImAStreamer

    None that I'm aware of.

     

    As I mentioned before, I've been using Wowza lately (and h.264/speex to flv works fine there), so perhaps this is just something that FMS won't do. Maybe someone from the FMS engineering team will have suggestions about how to record a file that anything other than FMS can read.

     
    |
    Mark as:
  • Currently Being Moderated
    Nov 10, 2011 4:42 AM   in reply to ImAStreamer

    Using my latest H264 recording application I was able to use FFmpeg to transcode to AAC. I understand you need to use the flvpackager to make the recorded mp4 playable direct from the desktop.

     

    JC I suggest posting your application code (bits you are using for h264 recording) for forum users to debug.

     
    |
    Mark as:
  • Currently Being Moderated
    Nov 10, 2011 4:59 AM   in reply to Adrian_NA

    As I mentioned previously, my implementation uses Wowza, not FMS, so I don't think my Java code is going to be of use to anyone here.

     
    |
    Mark as:
  • Currently Being Moderated
    Nov 10, 2011 5:16 AM   in reply to JayCharles

    Sorry I ment for ImAStreamer

     
    |
    Mark as:
  • Currently Being Moderated
    Nov 10, 2011 10:25 AM   in reply to ImAStreamer

    I know this thread has moved beyond the F4V Post Processor (thanks for all the info, Jay), but I wanted to let you know that the audio bug mentioned in the previous thread has been fixed. Unfortunately, I didn't grab the fix build number and I don't have access to the bug base anymore, but maybe someone from Adobe could grab it for us. 

     
    |
    Mark as:
  • Currently Being Moderated
    Nov 11, 2011 1:22 AM   in reply to ImAStreamer

    I am still unable to play my videos outside of FMS using H264 settings, unless transcoded, I am using FFmpeg.

     
    |
    Mark as:
  • Currently Being Moderated
    Nov 11, 2011 4:35 AM   in reply to Adrian_NA

    I think what Jody was getting a was the inclusion of ulaw and alaw (pcm) audio in the flashplayer. The trouble there is, unless you're going to use a really low sample rate, the bitrate required for pcm is going to be very high, and (if I'm not mistaken), the flashplayer will not perform AEC for pcm.

     

    @AdrianWilkinsUK - I would expect that you can't play those files outside of FMS without transcoding. Speex and nellymoser aren't mp4 compliant. They -might- play in the flashplayer (haven't tried that... but maybe the FP eng team has something in place there), but other programs won't.

     
    |
    Mark as:
  • Currently Being Moderated
    Nov 11, 2011 5:51 AM   in reply to JayCharles

    Yeah I saw post on that JC, thanks.

     
    |
    Mark as:
  • Currently Being Moderated
    Nov 11, 2011 12:44 PM   in reply to ImAStreamer

    I have the feeling this is an FMS issue. When I play the H.264/speex FLV recorded through Wowza in flashplayer 11.0.1.152, it plays back just fine for me, both streaming from Wowza, and via progressive download.

     
    |
    Mark as:
  • Currently Being Moderated
    Nov 17, 2011 12:39 AM   in reply to ImAStreamer

    Hi ImAStreamer,

     

    I see your bug report has been closed without resolution.  I raised the same problem in this thread:

     

    http://forums.adobe.com/message/3858528

     

    And got the reply:

     

    Thank you for the info. Apparently this is a bug in the Post Processor tool. The responsible team is aware of it and they are working on a fix. Of course I can't tell you when it will be fixed or released but it is in the works.

     

    that was in August and nothing since.  It seems a fairly major bug that you can't have audio in your saved video.  I can't find anywhere to contact that team for an update, and the problem seems to have been forgotten or ignored.   If anyone from Adobe is looking in, an update would be appreciated. 

     
    |
    Mark as:
  • Currently Being Moderated
    Nov 24, 2011 12:03 AM   in reply to ImAStreamer

    Has anyone got a work around for this issue?  Does anyone from Adobe know when it will be resolved, or if it will? 

     
    |
    Mark as:
  • Currently Being Moderated
    Jan 4, 2012 2:57 AM   in reply to nonsensemidget

    anyone?

     
    |
    Mark as:
  • Currently Being Moderated
    Jan 4, 2012 2:02 PM   in reply to nonsensemidget

    You can transcode the flatten streams using ffmpeg.  I've noted it here;

     

    http://forums.adobe.com/message/4053414#4053414

     
    |
    Mark as:
  • Currently Being Moderated
    Calculating status...
    Mar 12, 2012 4:40 AM   in reply to ImAStreamer

    Hi,

     

    When encoding live in H264 with the FP 11  then playing it live on another client (through FMS r) the video sometimes (1/3 of times) has major skips or jerks. It's almost as if the video goes back to some older keyframes and then returns to live. Same experience with non-h264 produces smooth video.

    If the server (FMS ) records the video, the replay looks smooth, which is strange because I would have assumed the FP was not encoding well with H264.

     

    Has anyone had this issue? please reply

     

    Thank you.

     
    |
    Mark as:
  • Currently Being Moderated
    May 7, 2012 2:58 AM   in reply to Arvind Mehta

    @arnold.25: yes i am also facing similar issues.  when i googled for it,  found out that its because of either the buffer or bandwidth causing the issue. so need to set some buffertime on your playback client end or the bandwidth should be sufficeint enough to play the stream. but when i tried the first method it intriduced a lot of delay and the second method is expensive. so i am in search of some other solutions for this.

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 11, 2012 6:39 AM   in reply to ImAStreamer

    I tried this out, ffmpeg is rendering the file fine at my end. I used ffplay to try and play this, I am getting video and audio.

    I have tried h264 with both nellymoser and speex.

    If still you are having this problem please pass me your file so that I can investigate further.

     

    Thankyou.

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 11, 2012 7:03 AM   in reply to TanujM

    Hi simpleton,

     

    Thanks for your time. I could resolve this at my end. It was nothing but the swf version mismatch on my compiler-settings. I was using the Flash player 11.2 and the compilet option was setting  -swf-version to 14. So H264 encoding had this jerking issue. But once give the appropriate version # it got resolved. I got the version detatils from here

    http://sleepydesign.blogspot.in/2012/04/flash-swf-version-meaning.html

     

    Vivek

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 11, 2012 11:20 PM   in reply to TanujM

    @simpleton how can i send you my files.send me ur email id so that i can send u my files.

     
    |
    Mark as:
  • Currently Being Moderated
    Nov 7, 2012 10:22 PM   in reply to Arvind Mehta

    Hi Arnold

    Sorry I forgot mentioning my email id.

     
    |
    Mark as:

More Like This

  • Retrieving data ...

Bookmarked By (0)

Answers + Points = Status

  • 10 points awarded for Correct Answers
  • 5 points awarded for Helpful Answers
  • 10,000+ points
  • 1,001-10,000 points
  • 501-1,000 points
  • 5-500 points