15 Replies Latest reply on Jan 19, 2012 6:46 AM by Lars Renklint

    Unpause animation if key value equals X

    cheerbot Level 1

      tldr: I need an expression that plays a sequence of images on a layer (and leave it on the last image) when the value of a key (keyframe?) in another layer of the same comp equals a specific value.


      I am fairly new to AE and trying to port a project I made exclusively using Processing. Wrapping my head around how Expressions work is proving difficult, and I learn best by example, and am even finding it hard to piece together how to do this from what I've found online.


      The situation and what I need:

      I have imported key frames from a midi file (using this script) into a layer called "midi" and a layer that is an animation of frames . The idea is that if midi's effect ch_0_pitch has a value of, in this case, 36, that the frames in layer "ds_bass" animate when they come across one of these keys.


      This is basically a "bass drum" animating on a "bass note" pitch. The midi file contains references to the entirety of drum midi needs (ie hi hat, snare, bass), each having a different pitch value. While I could simply render each individual "pitch" midi sequence to a midi file and import it as keys and then use a time remap expression to unpause and pause the animation on markers, which has worked, I need the capability to specify pitches for other instruments and run specific animations based on what pitch is being hit. As well, going through and adding a marker for each midi note is tedius and inaccurate (as there are very small differences visually between two keys with different values played synchronously, ie a bass hit and a hi hat hit).


      I found something online that accomplished this at first, with markers, placed in TimeRemap of the ds_bass layer. I would go note by note in the midi and add markers to the ds_bass layer for this to work, and it perfectly paused and ran the animation when I needed it:


      try {
            m = thisLayer.marker.nearestKey(time);
            linear(time, m.time, outPoint, 0, outPoint - m.time);
      } catch (e) {



      But seeing as I needed more to automate this process (this is a four minute piece with a lot of instrumentation and I'd like to be able to use this as a template for another project), I looked at the reference and other resources and the best I could come up with was something like this, which doesn't cause errors but also doesn't do anything (as well on TimeRemap of ds_bass layer in the same comp as midi):


      try {

           if (thisComp.layer("midi").effect("ch_0_pitch").nearestKey(value) == 36) {

                m = thisComp.layer("midi").nearestKey(time);

                linear(time, m.time, outPoint, 0, outPoint - m.time);


      } catch (e) {




      In Processing this was all handled with a switch in a method that brought in midi info on noteOn signals, and that checked for pitch values and ran methods based on pitch which ran sequences of images. Maybe I am going about this the wrong way, admittedly, as I am terribly new to AE, but this seems like it should be completely possible to automate all of this for very solid and automated timing mechanisms based directly off of the midi data.


      Many of the other animations will be dependent on running a sequence based on pitch, in the sense of having a puppet that can animate ArmL to ArmLPlaySnare OR ArmLPlayHat - having a single "resting" frame that executes different animations (these are sets of 1-5 .pngs) dependent on midi pitch. Again, how it was managed in Processing.


      If anyone can give some insight I would greatly appreciate it. I moved to AE as I found I'd need to render visuals in non-realtime (from midi data), and it seemed superior to learn AE as it has render capabilities that are more inline and intuitive with this project. At the moment I am stuck. Thanks for your time.


      Edit - If I could simply copy the key frames I need to the ds_bass layer and then use the marker script (first above) with nearest key instead I figure it would work fine, though that's merely a more satisfactory convenience, not a proper fix. I'd still have to easily render dozens of midi files and do this for each. Best case scenario would be that I have one keyframed midi file with all channels in it (in my "master comp?") that can be referenced by the individual compositions. At the moment I'm putting instrument specific midi-keyed layers into each instrument specific comp.


      Message was edited by: cheerbot: added references online to resources used

        • 1. Re: Unpause animation if key value equals X
          Dan Ebberts Level 5

          Interesting. So will your ch_0_pitch effect have keys for multiple drums? If so, the the bass time remapping expression needs to start at the current time and go backwards in time, key-by-key, until it finds a key that matches that drum. Then it will know how far along in the animation the time remapping should be. If there's only one instrument in the set of keys, the expression just needs to find the most recent, previous key, which is pretty easy.  Are there keys for both note on and note off events? With a little more info on the nature of the keyframe values, we can probably zero in on this.



          • 2. Re: Unpause animation if key value equals X
            cheerbot Level 1

            Short reply: The midi files can be structured with one instrument (one channel), one drum (one pitch of a channel) or the entirety of the song with different referenced channels (say 10 channels, with multiple pitches/ key values for each). The first is current and is do able as most instruments other than the drum set are structured this way, the second is inconvenient but doable and the third is the most preferable. In each case a pitch is tied to the keyframe as its value. There are three different effects for each channel when imported to AE: pitch, velocity and duration. Pitch and Velocity both have a keyframe created for noteOn and noteOff while duration only has a keyframe for noteOn.


            Long reply:

            This should illustrate it better.


            This was done with Processing using real time midi data coming from Logic as it also played the music. After rendering this I ended up fleshing out both actors with a lot more frames for smoother actions, but ended up dropping them in favor of a different aesthetic once I implemented environmental visuals (a more current version can be found on vimeo though it's synced terribly - the main reason I am trying AE as I just can't do this in real time and had to use screen cap for Processing with several memory eaters).


            Let me see if I can answer you, Dan, without being too verbose, using a few examples.


            There is a different external midi track that sums up each instrument in Logic and assigns it to a different channel (Channel 0 would be for beats only, Channel 1 for guitar notes, Channel 2 for drum set, 3 = bass, 4 = keyboard, Channel 5+ for environment graphics). So, four instruments, each with a different midi file for triggering animations.


            This is how two of them worked in Processing and how the midi is set up:


            The drum set midi track would have a different pitch associated with each drum: A pitch of 36 would be a bass drum, a pitch of 42 a hi hat, a pitch of 38 a snare. When a noteOn comes from Channel 2, it triggers both the specific drum to animate as well as the arm to animate a sequence of images that show it hitting that drum.


            If two arms hit the same drum, I have the drum image cycle each time but with an appropriate arm sequence (so I'd have pitch 42 and 43 both hit the same drum, but would animate different arms).


            The guitar midi track (, however, simply has pitches that are all relative to the guitar instrument, pitches it plays (so 60 is simply middle C, 72 is one octave above). If any Channel 1 noteOns come in, it triggers the left arm to do a play animation, while triggering the right arm to "play a specific note" by playing a different sequence of images.


            Environmental graphics have pitches like the guitar but they were used to relay positions on the screen (which I'll probably just try to do with regular AE animations at this point).


            Thus ultimately any instrument needs to animate only when a key has a specific value. I think the way you put it at first seems appropriate then?


            In AE, I render the individual midi tracks for each instrument, and import it using the script to a composition that is a singular instrument. So, a composition per midi channel, instead, although ultimately it would be great to just dump one large midi of the entire song into a main comp file and reference it by midi channel for each individual instrument comp.


            The way the script seems to work is that it drops the midi channel in with three "effects" per channel: ch_0_pitch, ch_0_velocity and ch_0_duration. Pitch and Velocity both create a key for noteOn and noteOff, while duration seems to only have a noteOn. If I bring in a midi with more than one channel, they are simply numbered appropriately and have keys created for each.


            As I have not been using the noteOff values (I simply cycle the animation at a defined rate, and format the notes to be as small as possible), I really just need the noteOn to trigger it, so duration would most likely be used. I would however like to be able to properly use noteOff for environmental graphics or better timed instrument animations.


            Many of these animations, as well, have the "down beat" about halfway through the cycle. I've been cycling them within about 150ms so it's unnoticable, but if I could interpolate a ~5 frame sequence to hit the 3rd frame right on the needed key that would be fantastic, since the only way to do that previously would be to read a midi file through text and sequence things individually through time codes so I've forgone the ability to show "the future" in any of these projects.


            I hope that helps clarify.



            Just as an aside the animations I keep refering to are numbered cells in .png format that I import as a sequence and then assign to a layer. I do wonder if I'm thinking too along the lines of working in Processing and that there might be a better way to do all of this within AE, but I also have been working on this project for awhile learning code and would like to just finish it and then learn more on a new project.

            • 3. Re: Unpause animation if key value equals X
              Dan Ebberts Level 5

              I think it's all very do-able, but you have a number of issues to deal with. This is the basic expression that will calulate how long it has been since a note 36 keyframe:


              myId = 36;

              note = thisComp.layer("midi").effect("ch_0_pitch")("Slider");

              n = 0;

              if (note.numKeys > 0){

                n = note.nearestKey(time).index;

                if (note.key(n).time > time) n--;


              if (n > 0){

                while (n > 0){

                  if (note.key(n).value == myId) break;





              if (n > 0)

                time - note.key(n).time




              The problem is that the script generates keys for note on and note off and there doesn't seem to be a good way to know which is which. So the note off keys are going to mess up the timing.


              So maybe you need to modify the script to leave those out. That should work for animations that start when the note sounds (like animating the drum, for example), but your other animations need to start before the note (like an arm moving to strike the drum). So you probably need a marker on the animation layer to denote where the animation needs to be when the note fires. Then your expression needs to calculate the timing that will syncronize the drum key to the marker on the animation.  I did a tutorial over at Creative Cow a while back that discusses synchronizing with markers that might be of some help:






              • 4. Re: Unpause animation if key value equals X
                cheerbot Level 1

                Thanks Dan, that tutorial is much along the same lines as what I'm trying to accomplish. The script you gave me works, but it is horrifyingly true that the noteOn/Off screws with it. While I certainly understand some of the scripting in the script I'm using (again it's from http://omino.com/pixelblog/2011/12/26/ae-hello-again-midi/ ) I'm not a proficient enough programmer to properly mess with it. Well, maybe, I'll keep taking a look at it, in my head I keep considering it's just removing some references to get rid of noteOff.


                I figure I could maybe restrict midi notes to smallest values possible (which can be automated in Logic) but it may have enough of a double animation to distract. Unluckily that's not the only issue, as I see I could certainly utilize noteOffs in the future, and interpolate frames (with linear()?) to keep animation sequences brief.


                I can't think of a decent option, though. I doesn't seem efficient to simply map noteOffs to a separate effect along with ch_x_pitch and velocity, going between the two for reference would be a hassle. I guess on some level I'm hesitant to start a complicated and time consuming process without first seeing if there's an efficient alternative - marking each note seems to work, it's just time consuming.


                Is there a way to copy keyframes to markers? I assumed after looking through resources that there must be a way to reference keyframes instead of markers (may be my unfamiliarity with AEE), and I was struck by not being able simply copy key frames between two layers (is this actually easily doable? Parenting maybe?). I could probably render a midi file for each pitch then and just move it over, use the script in the tutorial (modified slightly to use key frames instead of markers if doable) to sync everything up pretty nicely with nothing but a few figurative copy pastes.


                And that'd be more efficient, I suppose. Though I still have the same pitch issue when I jump to other instruments that are needful of identification between noteOn and noteOff. For that to work in this quickly outlined method I'd need to render each pitch in midi and then assign markers, but most of the instrumentation isn't confined to a single octave of course, and some pitches would share animations (which I guess I could reformat the midi files prior so that all of one animation's pitches, ie if one arm position is "C3, C4, C5", are on a single pitch).


                After looking more at this ch_x_duration and ch_x_velocity might have use in the code if you'd be so kind to continue to help me. When I was using Processing I fixed this problem by only parsing any information above a certain velocity, as the library I was using didn't utilize noteOff but instead used a velocity of 0 to signify noteOff. I thought that was a bug, but I guess it was more of a workaround, just as this script is utilizing it. If that's possible it wouldn't be too difficult (I think) to check velocity as well and only recycle animation if velocity > 0.


                Something like (building off your Expression while trying to understand it, please correct me if I'm wrong, referencing time is odd to me):


                pitch = 36; // keyframe value to look for - what if multiples?

                vel = 10; // ignores ch_x_vel keyframe is less than vel

                notePit = thisComp.layer("midi").effect("ch_0_pitch") ("Slider");

                noteVel = thisComp.layer("midi").effect("ch_0_vel") ("Slider");

                n = 0; // keyframe index reference?


                // numKeys = "Returns the number of keyframes on a property."

                // nearestKey = "Returns the Key or MarkerKey object nearest to a designated time."


                // find current key in notePit

                if (notePit.numKeys > 0 ) {                // if effect.ch_0_pitch has keyframes,

                     n = notePit.nearestKey(time).index;   // n set to nearest key (to layer time)

                     if (notePit.key(n).time > time)       // if the nearest key's time is later than this layer's current time,

                          n--;                             // n to set to previous keyframe.



                // find current key in noteVel

                if (noteVel.numKeys > 0) {

                     n = noteVel.nearestKey(time).index;

                     if (noteVel.key(n).time > time)




                if (n > 0) {                                             // if the selected effect.ch_0_pitch is greater than 0

                     while (n > 0) {                                     // while true,

                          if (notePit.key(n).value == pitch) break;      // if effect.ch_0_pitch keyframe N is needed pitch, break loop.

                          if (noteVel.key(n).value > vel) break;         // if effect.ch_0_vel keyframe N is greater than zero, break loop.

                          n--;                                           // else try a previous keyframe.



                if (n > 0)                                               // if effect.ch_0_pitch's selected keyframe is greater than zero

                     time - noteVel.key(n).time                          // start animation at current time - notePit.key(n).


                     0                                                   //




                Well. I am not sure. I just spent the last few hours trying to figure it out but I am really not understanding the utilization of key frames in code and am pretty much grasping at the moment, maybe I'm just overly complicating it. I am still a fairly amateur coder. But it should be doable, any ideas? I guess I'll need the markers anyway, though I was considering interpolating the frames so that they hit the middle of the sequence during the keyframe. Which I only vaguely understand.


                edit - I guess I'm not assuming that key(n) is an array. hmm.

                • 5. Re: Unpause animation if key value equals X
                  Dan Ebberts Level 5

                  I think you're close, but it occured to me that it might be better to modify the script to generate layer markers instead of keyframes. Markers can have parameters associated with them so a single marker could contain the pitch, the duration, and the velocity. That would get rid of the pesky note off keys. I think all you'd be losing would be the note-off velocity (I don't know if that's critical to your plans). I'll take a look at the script tomorrow if I have time.




                  • 6. Re: Unpause animation if key value equals X
                    cheerbot Level 1

                    Actually that sounds like a perfect way to circumvent some of these issues. noteOn tends to register velocity as 0 as a way of signifying noteOff, apparently, so a noteOff velocity wouldn't be needed in general, I think. I'm looking forward to what you can do (though I wonder if it'd be more prudent to simply write the author of that script a suggestion).


                    Honestly I'm a little surprised that midi-based keyframing specific data doesn't have more accessability in AE. I've been wondering if people usually go through Flash with any midi related items, it seems more often people tend to just use midi purely as a beat finder in AE, leaving out a lot of useful information that can really be tied into the dynamics of a presentation. I've seen very little that takes advantage of pitch or velocity (in an automated or precise way, anyway. I have of course seen utilizing sound volume in the same way that one could use aspects of velocity).


                    That is to say, I'm sure others would appreciate it too.

                    • 7. Re: Unpause animation if key value equals X
                      Dan Ebberts Level 5

                      If you go into the script's applyChannelToLayer() function and wrap the piece of code that populates the picth, val, and dur arrays with a test for non-zero velocity, I think that will eliminate the keyframes that mess up the expression:



                      if (vel > 0){






                        var durS = "" + dur;

                        if(vel && typeof(dur) != "undefined")






                        lastT = t;



                      Give that a try and see if it helps.



                      1 person found this helpful
                      • 8. Re: Unpause animation if key value equals X
                        cheerbot Level 1

                        That works, all velocity 0 is ignored which leaves pitch fine. The drum set is completely automated now, though there's still the issue of actually utilizing noteOff/velocity = 0 for other instruments.


                        I made a comment on the original script page with a link to here - I figure since the script is actually only a few weeks old could try to get some further abilities from the author (if warranted). Though admittedly I know very little of what I'm doing!


                        I guess my original question still stands, though. Beyond getting the Expression you were so helpful to write, I guess there's still the issue that I'm not familiar enough with scripting to appropriately fix the expression to disregard (or dignify when needed) the 0 velocity noteOff references for instruments beyond the drum set.


                        In consideration of utilizing a more complicated instrument, I'm wondering how I would set up something like an arm that would have to handle multiple animations but keep a consistent "resting" image. I'll take a closer look at your tutorial and see if there's more insight I can gain from it in that regard. I think I'm just inexperienced enough with AE to see how to automate several of these sequences in the appropriate way, but I'm definitely getting a better understanding of this through this discussion so I'm appreciative.


                        Maybe I'm just a little unsure of what questions to be specific about in asking to get exactly what I need.

                        • 9. Re: Unpause animation if key value equals X
                          Dan Ebberts Level 5

                          I'm not sure if you really need the note off events, because you have the note duration. Note off just gives the velocity of the key release, doesn't it? If you can live without that, you should be able to calculate note off time from note on and duration. Of course, I may be missing something (it wouldn't be the first time)...


                          I don't know if you've see this on my site, bu it has some info on multiple animations on the same layer (deliniated by markers) that might be useful:




                          I did try to convert the script to save all the data in markers, but it hung AE. I don't know if it's a bug I introduced or if it chocked on so many markers, but I may try it again, because that seems like the way to go if it works.



                          1 person found this helpful
                          • 10. Re: Unpause animation if key value equals X
                            david van brink

                            Cheerbot -- I love the animation you linked http://andsuch.org/pub/saturn_v4_test.mov !


                            Dan -- Looks like you found a good fix for his needs, eliminating the note-offs. I'll definitely add a checkbox for that (or maybe just remove all the note-offs, they're not so interesting with the Duration track...)


                            Is there any benefit to using Markers instead of Keyframes? The Keyframes carry values, which seems nice.


                            Cheerbot -- my thought was that in some cases one could use the music authoring tool to construct a reduced or altered MIDI track that was specifically for the AE import. Polyphony makes for confusing keyframes. Very simple scores (like the Two Part Invention example I used) work ok as-is.

                            • 11. Re: Unpause animation if key value equals X
                              Dan Ebberts Level 5

                              I was thinking markers might be a good choice because you can stuff the note, velocity, and duration in the parameters of one marker. That way you wouldn't have to correlate separate streams of keyframes.



                              • 12. Re: Unpause animation if key value equals X
                                cheerbot Level 1

                                Thanks David, as you can see I'm trying to bring that to a higher level, heh. After putting together some midi files for the project, the issue of polyphony certainly arises. I never did recognize a possible way of dealing with polyphony when I was working on this project with Processsing, though I did see with your script that you're finding polyphony using a quick delayed check. This issue seems to exist pretty normally, I suppose because polyphony can be expressed very easily in audio but it doesn't have an alegorical usage with visuals, or even an easily identifiable methodology to encapsulate that information in a way that can be utilized independently of one another. So, unless a more intelligent method comes along I guess I'll have to continue with modified midi tracks.


                                I agree with Dan about markers, simply as beyond having multiple values to single layers in channels, there are a lot of resources online (including Dan's tutorial) that make excellent use of markers for exactly the purposes I - and others? - would utilize them for. If nothing else it seems more efficient for the user (though I could be wrong).


                                Dan, in my experience noteOff (as in a velocity of 0) has usually been a preferable way to work with media, as I can usually just change a boolean, end a thread or animation by checking velocity. I suppose using duration would work the same way, but I can't really say, though it does seem less complicated. I'm always up for finding a better way to do something.

                                • 13. Re: Unpause animation if key value equals X
                                  david van brink Level 1

                                  Oh! I... I never used markers. They can hold multiple attributes. Ok. Awesome. Got it. Thanks.

                                  • 14. Re: Unpause animation if key value equals X
                                    cheerbot Level 1

                                    Thanks, looks like this is successful but will just take some work and tweaking. I still need some experience with rendering and uploading to Vimeo/YouTube apparently, hah, not sure why it doesn't sync properly after uploading...



                                    With some time echo and slight wiggles on body and instrument aligned to the tempo of the song, if comes off as a really fantastic effect, despite its simplicity. Anyway I'm amazed but I'm an AE amateur.


                                    Dan, I really appreciate you going through my verbosity. That last tutorial link ( http://www.motionscript.com/design-guide/marker-sync.html ) was exactly what I needed for the majority of the actor's animations. I ended up restructuring the precomp animations with an extra "rest" marker after each animation otherwise it would oddly jump to the first frame of the next marked sequence as a rest (or "norm" as you have it). Otherwise I had to change some of the timing on the codes, adding 0.1 of a second to make sure all animations hit at the right times. So again, thanks! This was indeed the answer I needed, though I wonder if it could be automated moreso to read the values of the markers without setting comments.


                                    David, thanks for stopping in, I'll keep checking your blog to see when/if you put up a marker based version to help with the workflow at the moment. The original script has helped significantly, though.


                                    I'll put together a little page that just shows all the steps involved and what's needed and update this post with a link once I put out the full project so if anyone's curious they can replicate it.


                                    You guys were completely helpful, thanks again.



                                    Edit - I found three useful free scripts online after trying to brute force 4 minutes of instrumentation. These definitely help if anyone is trying to do something along the same lines. With segmented, specific midi files this is a breeze.

                                    rd_CopyMarkers - copy markers from one layer to another.

                                    rd_keyMarkers - copy keys to markers (+ assign comment).

                                    Layer Marker Batch Editor - batch edit markers on a layer (in this case used for changing all markers to the same word).

                                    • 15. Re: Unpause animation if key value equals X
                                      Lars Renklint Level 1

                                      This has been very helpful. Nice project.

                                      I had just discovered Davids script and also wanted to create keyframes for the note on event only. I'm just painting notes like a pianoroll, but with just one keyframe for every pitch you can decide to paint a bezier curve or linear curve instead. Thanks everyone involved.