Problem With Synchronizing Sounds In AS3

[Guest]

I am building a 10-channel mixer/remixer and I`m having a problem with synchronizing the sounds.  When a user clicks on the "Play" button, it calls a method in the main (document) class which runs through a "for" loop and calls a method in each of the channel objects which play their respective sounds.

Here is the method in the document class which calls the playTrack() method in each channel object:

public static const MAXTRACKS:uint = 10;

protected function playTracks():void

{

     var clip;

     for(var n:uint=1; n<=MAXTRACKS; n++)

     {

          clip = getChildByName("Track_" + n.toString());

          clip.playTrack();

     }

}

Here is the playTrack() method which is in each channel object:

public function playTrack():void

{

     stopTrack();

     activeChannel = activeSound.play(0);

     activeChannel.soundTransform = activeTransform;

     isPlaying = true;

}

The problem is that the sounds, particularly the ones which are triggered toward the end of the "for" loop, are triggering out of sync with each other.  When I click repeatedly on the "Play" button, I can get them to play in closer synch with each other.  I built a previous mixer in AS1 which had this same problem.  However, there was a workaround (it had an Indian sounding name) at the time where, inside the master "play" method you triggered all the sounds, then stopped them all, then triggered them all again.  For the most part it worked well.  I thought that this problem had long been addressed by recent Flash compilers/players, but apparently it hasn`t.  I`ve read in other forums that people have had the same issues.

Granted, playing 10 mp3s simultaneously might place a heavy load on the Flash Player which might cause signal dropout or something (and there is none), however, the fact that they all trigger late is simply unacceptable.  This just seems like it should be elementary.  What am I doing wrong?  Are there any secret tweeks or ways I can re-code this in order to get them to play in synch?  The whole application rests on this!

Thanks for any help!

[Andrei1-bKoviI]

Sorry to disappoint you but you are observing the correct behavior. I think you expect something that AS3 language and, thus, Flash cannot deliver by nature. The code is executed line-by-line, so there is always a delay in the next loop iteration comparing to the previous one because it, well, happened LATER. In other words, loop waits until all the code, including the code inside instances, is executed and only then it moves to the next iteration. In your case it get child, then waits until code in playTrack() is done, etc. - this takes a lot of time. I mean, event one millisecond is a lot of time to to expect ideal sync.

You may want to experiment with time to see how much delay happens and how code optimization can remedy it. On the other hand even trace and getTimer() take some time to execute.

for(var n:uint=1; n<=MAXTRACKS; n++)
{
     trace("begin", i, getTimer());
     clip = getChildByName("Track_" + n.toString());
     clip.playTrack();
     trace("end", i, getTimer());
}

[Guest]

Thanks Andrei1

I`ll try to clean up the code a bit, perhaps by abandoning the "for" loop and making direct calls to the various channel objects`s playTrack() methods:

Track_1.playTrack();

Track_2.playTrack();

Track_3.playTrack();

etc...

Or even making the activeSound object variables public and calling the play() method on them from the document class.

I`ve seen Flash sequencers that seem to do this multi-sound synching without a hitch.  I wonder if there is another way to do this that involves the Sound class`s Extract() method - something along the lines of Andre Michelle style Flash sound programming.  I would imagine that this would not be easy (and perhaps would entail enough research/work to earn one a computer science degree), but it might be worth looking into.

[Andrei1-bKoviI]

I feel that one of the ways would be to detect whether there is enough buffer time (perhaps via progress event listeners) in all of the sounds and then reset them to 0 once all of them meet buffering requirements.

As for the loops, I think there is no way around them. The only thing is to make sure each loop iteration takes an absolute minimum of time. This, of course, is a trial and error business but if you find an array of references that when looped through executes the fastest - you should be good.

[Guest]

Thank you very much Andrei1!  This is very insightful.

I follow what you`re saying about the buffer time settings, but I`m a little confused.  These are not streamed sounds.  They are completely loaded in (via an external SWF file) and assigned to their respective track objects before the "Play" button is even enabled.  Is buffering even necessary?  If I`m just misinformed about this area of sound, then how would I put what you recommended into practice?  Even a little pseudocode would be tremendously helpful!  I`m pretty much a NOOB when it comes to messing around with audio buffers, and I`m not sure I could implement this properly.

Thanks again!

[Guest]

Hi phaseblue.

Are you taking into account the silence at the start of MP3's.

Depending on the source of the mp3's the silence may be different lengths.

I've been struggling with a similar issue, which is more of a sequencer than a mixer but I use wav files in my library and set to raw in the sounds library properties. I have a stable timer now but it only works on my laptop running XP, my desktop with vista still has glitches which suggests that an accurate timer is possible but very much depends on the machine its running on.

Whilst it is true that AS is sequential I doubt that human ears can "hear" or percept a difference of less than 5ms between two sounds that are supposed to be in sync. Any more than this and you start to notice a "flanging" effect, particularly with drums, somewhere above 10-15 milliseconds its starts to sound just plain wrong.

If you do have any sequencing going on in your remixing I would suggest that you consider using the SOUND_COMPLETE event as the timing engine using an empty sound (silence) of a specific length. You can then use the play offset to get your tempo. It is not perfect in that although the resolution of of this offset is supposedly 1ms, I have been finding that it only seems to have any effect on timing in increments of around 50ms. I am no expert at AS3 but the fact that I can now get a solid timer at about 140 BPM at sixteenth note resolution (albeit on one machine) suggests that it is possible. My app has very little graphics going on apart from highligted note heads and an led style flashing metronome all done in code. Even if I allow the event listeners for buttons that change the drum pattern to be active whilst the sequencer is playing it will redraw the pattern without affecting the timing. My guess is that flash player is happier with raw  .wav data than mp3 and that AS3 whilst not as fast as some other languages is no slouch.

Good luck

[Guest]

Thanks for the reply lynchylynch!

As far as the sound files, they are all imported into the library as WAV files.  I then set them to compress as mp3s from within Flash and export the swf as a "song" file to be loaded into the mixer swf.  I do the mp3 compression from within Flash because it removes the beginning silence on them.

The problem I am facing is that the sounds are all being triggered slightly offset from one another, so much so that one can easily hear at least a 100 ms lag between the loops in track 1 and track 10.  The cause of this must be, Andrei1 said, caused by time consuming function calls within each iteration of the loop.  I just need to find a good way to get around this, or somehow "fool" Flash into triggering them more in synch with each other.  I`m starting to realize that Flash apps. like ours are very much like video games in the sense that they are processor intensive, and therefore require certain OOP principles to be tossed out the window in areas where performance is key!

Thanks for the timer advice!  I`ll look more into that!

[Andrei1-bKoviI]

I am not sure I can say more about the sound. lynchylynch has, obviously, much more experience with it. I would like to point out in addition to his/her comments that Flash ALWAYS depends on frame rate. It sounds counterintuitive but event Timer() class relies on frame rate. This means that there is the smallest possible increment for the next iteration (screen refresh or sound, or whatever is at stake). I don't remember exact numbers but, approximately, if frame rate it 20 fps, Timer() delay, I believe, cannot be set to, say, 10 ms. It still will fire in 50 ms. Of course it can be set to 10, but it will not abide.

I belive that maximum frame rate possible is 120 fps. This means that the lowest delay can be only 8-9 ms. 1 ms is never possible.

I think this is important in terms of frame rate dependency. Flash goes to the next frame ONLY after all the code designed to execute in the current frame. The implication is that ACTUAL frame rate may be (and it is in a lot of cases) lower. If, say, frame rate is 20 and code for current frame takes 100 ms to execute, actual frame rate for the current second is 19 fps even if the rest of the frame's code was executed in a timely manner. Thus, all the timers are affected adversely (not animations only).

I don't know how sound is aligned against frame rate but, it least, it is logical that if timers are used for syncing, discrepancies will always be there.

[Guest]

I totaly agree with Andrei1 about the frame rate issue

IF you are using flash for its main design purpose which has historically been superb interactive visual web development.

The minute you even consider creating audio specific apps in flash I think you have to throw a lot of  the normal rules of flash development out the window. Including , as you mentioned, a lot of the principles of OOP. My motivation for doing so has been to learn AS3  by porting an app I wrote in VB6 that used midi for the sound.

I'm a newbie at flash and AS3 but for audio apps I think this means avoiding animation on the stage timelime wherever possible, especially if you want to sync audio and visual.

For me this has mean't coding all the animations that highlight note heads as they are playing and a simple flashing "LED" style metronome.

All graphical symbols are sprites instead of movieclips and all the actionscript is in a 1700 line single .as document (including blank lines and comments to make it look pretty .). Obviously if you want realtime interactivety for your users then buttons/sliders are a must, but I think these are a little different to any graphics that need to sync to audio where frame rate would be an issue.

I did a lot of reading on timers in flash all over the web, and for audio specific apps it seems that timers are too unreliable to use. The first mention, i noticed, of the use of the SOUND_COMPLETE event as a timer was found on the Computus website (do a search for "accurate timers in AS3").

It was there I read that the flash sound engine runs on a separate thread, which if true???, means that as Andrei1 said, as long as your code is not running lots of other tasks in loops and the timeline is not too busy, AS3 should be able to cope with simple audio apps.

If your remixer app is playing 10 loops, and the remixing is done by muting or altering the pan or volume of those 10 loops then my gut feeling is that you would still be better off letting the SOUND_COMPLETE event handle the looping. If all your loops are exactly the same length in time I would just add a SOUND_COMPLETE listener to the main drum loop sound and let this event retrigger all the sounds. You can still do the "live" mixing by storing the sound transform values in variables and calling them as you use sound play for your loops.

If your loops are of different lengths then just use the shortest one as your trigger engine and code when the longer loops should restart. I doubt very much that AS3 can do realtime stretching or squashing of loops like full blown audio apps can.

As I said in the previous post, SOUND_COMPLETE is not perfect because it has its own latency, but this latency seems to be pretty consistent which means in theory you can allow for it.

Another point, if you ever use a calculated offset for playing sounds in code, beware of the possibility of negative offset numbers. Through my own bad coding I had an awful sounding metronome that sounded like a fax machine because I had allowed the code to create a negative offset number. I guess the flash sound engine was just playing the data from the header data of the wave file. It didn't sound too good and I don't think my speakers or my neighbours liked it too much.

Steve.

[Guest]

Just another thought I've had about your sounds in an external swf. I don't have an answer as I'm a newbie too but does this mean that flash player is trying to load the audio files from this external swf each time you call them in your code. 10ms delay per loop seems an awful lot.

[Guest]

lynchylynch,

Sorry for the late reply!  No, the sounds are loaded in once a "song" is selected.  From then on, the song (sounds) can be played/stopped or muted/ un-muted as the user chooses.  I just need to get them all to trigger in time with each other!

Anyway, thanks for all your help!  I really appreciate the link to The Computus Engine!  Very insightful source!

Here is a link I found to a class library called KichenSync:

http://code.google.com/p/kitchensynclib/

It looks like it uses the "system clock" to sync everything together!  What`s your take on it?

I also found this (Sync multiple SoundChannels and animation):

http://www.actionscript.org/forums/showthread.php3?p=936222

...and this (Flex Music Maker):

http://www.few.vu.nl/~eliens/media/student-mma-16-documentation.pdf

Without a doubt I love these forums, but in all honesty I wish there was a place where all the Flash "sound" people could get together and share ideas, resources, and code. I own a domain name, "psychosound.com", and I`ve never put it to any use!  Hmm...

[Guest]

Sorry to bring this thread back from the dead, but I figured that I owed it to everyone to share what I`ve found out regarding the "synchronizing multiple sounds in AS" matter!

Although there are many ways to accomplish this - some better than others (and none of them particularly "easy") - I think I have found the "mother load" solution!  It is Pixel Bender!

Since PB is basically a number cruncher at it`s core, and PB kernels run in a separate thread than the Flash Player, it is perfect for tasks that require large amounts of math like audio mixing/manipulation!  I already have a basic 10-sound "test" mixer built using some code from one of the bottom links.

For anyone who`s interested, these 2 links will start you off:

http://www.kaourantin.net/2008/10/audio-mixing-with-pixel-bender.html

http://blogs.adobe.com/kevin.goldsmith/2009/08/pixel_bender_au.html

Anyone else messed around with any of this?

[Alfred PS]

Hi All there,

are you know!!!

why i cannot heard my swf https://goo.gl/mI4pW1  ?

THANKS