audioSampleAccess - capture audio from RTMFP subscribed stream

[tomfoo]

I'd like to capture the audio data from an RTMFP stream to which the client is subscribed (so I get a bytearray of audio samples).

The presence of the audioSampleAccess propery on the NetStream class certainly makes that sounds possibe:

For RTMFP connections, specifies whether peer-to-peer subscribers on this NetStream are allowed to capture the audio stream. When FALSE, subscriber attempts to capture the audio stream show permission errors.

http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/fla sh/net/NetStream.html#audioSa...

But I dont see how I would do it. In the case of video, I see I could use BitmapData.draw(), and discucced here:http://forums.adobe.com/message/1066374#1066374

But in the case of audio, I dont know how to address the audio data to get it into a bytearray.

Any ideas?

My instinct said this wasnt possible, but the presence of the 'audioSampleAccess' property makes me think it might be..

thanks

tom

[seth_hodgson]

I assume you should be using the SoundMixer class: http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/media/SoundMixer.html?filte...

Best,
Seth

[tomfoo]

Thanks the suggestion Seth, but I'm not trying to compute the spectrum, I'm trying to get the audio samples (so that I can store/record it, and play it back later).

thanks

tom

[seth_hodgson]

Take a few minutes to read through the ASDocs for that method, and the example that follows. I think this is doing the same sort of thing as you can do with video sample access and BitmapData.draw() to generate periodic graphic snapshots of an underlying video stream.

In the case of video or audio, you're only being given access to take samples/snapshots. There is no API in the Player/AIR that lets you extract the complete audio/video byte stream from a source NetStream.

Best,
Seth

[tomfoo]

Thanks Seth.

I had already read the asDoc, and now I have read them again. I still dont think it does what I need it to. I need to be able to record the _entire_ audio stream, for playback later. Do you think it does that?

It absolutely is possible to do this with video: you just take a snapshot of the video object on every ENTER_FRAME, and then reconstruct those images into a video - it's a near-exact replica of the original video. It's possible because video is a single image per frame, which can be precisely captured. Whereas with audio, there is ~33ms of audio per frame, and if compute spectrum only returns 512 bytes (per the asDoc) then there is no way that all 33ms of audio are captured at 44Khz in 512 bytes. At 11 Khz it might be possible, but even then I doubt the ENTER_FRAME is reliable enough to construct a continuous audio stream. I will give it a try..

tom

[seth_hodgson]

Hi Tom,

I'm afraid this is as close as you can currently get. As I mentioned in my previous post, there is no client API that will give you the full AV byte stream. We do have an API going in the other direction (i.e. NetStream.appendBytes()), and that's used for HTTP dynamic streaming but you could source your bytes from anywhere.

Outside of that, I'd suggest investigating other ways to get your source audio to the client indirectly (e.g. using URLStream to pull down an MP3 rather than pulling it over a NetConnection/NetStream). You could then do processing on those bytes and feed them into NetStream.appendBytes() for playback.

I think what you're really looking for would be something like an "extract bytes" event on NetStream that would give you a way to pull the full byte stream into your own ByteArray for processing. This could then be routed back into a NetStream (using appendBytes()) to display in the UI or publish to another host.

Feel free to log an enhancement request here: http://bugs.adobe.com/flashplayer/

Best,
Seth

[Robert Mc Dowell]

Really soundMixer should have an extractByte like the Sound object