Particles with synthesia

[Robertomr00]

Hi, I was wondering if you could figure out how the particle effect of this video is made. Thank you in advance. This is the video: VIOLET MELODY - Patrik Pietschmann (Original Piano Music) - YouTube

[Roland Kahlenberg]

I highly doubt this was done with a particle system - the elements falling down are too well-placed and perfectly timed for them to be left to particle simulation. If I were to do this I'd begin with creating one animation for each of the two types of elements, in a precomp. Then nest them in a 'final' comp; duplicate as many as required and put them into position and time to suit my needs. HTH.

[Towerguy]

Hi. It's called Piano Roll Notation which some audio editing programs can show. It takes the MIDI information from the digital input/score and instead of showing notes on a staff  it shows the pitch and length of the notes as bars. It's the system pianolas or self playing pianos used to use where the 'bars' were cut out of a long paper roll which caused the notes to sound.

[P.M.B]

In the video's description there's a link to an app called Flowkey.  That might have something to do with it.  If I were you I would reach out to the person who's channel it is and just ask.

[Towerguy]

On further research it's actually an app called Synthesia. I suspect the footage was shot of the player recording a MIDI file (on a light-up keyboard), then the file was imported into Synthesia and rendered, then overlayed above the real hands playing (color corrected with some glow added?). The particles might possibly be made with trapcode, using a masked section of  the piano roll as layer emitter (?).

[Robertomr00]

Indeed, I already knew it was made synthesia, sorry to not have mentioned it. I'm specially interested in the particles and their behaviour, because I don't think they were made one by one for each note and I would love to know how to construct this. If you can specify a little bit more I would love it. Thank you all for trying, too.

[P.M.B]

Dumb question.  Have you tried asking in the synthesia forums?

[Towerguy]

Hi Gutterfish, there are some additional particles which are not part of the synthesia app - the ones that generate when the bars touch the keyboard.

Robertomr00, if you duplicate the synthesia footage and precompose the top layer you can draw a thin rectangular mask around the bottom where the bars hit the keyboard - this is the bit that will generate the particles. In your main comp set the precomp to 3D. On a new solid add Trapcode Particular. In Emitter settings: Particles/sec very high (say 24,000); Emitter Type - Layer; Layer Emitter - set to your precomposed layer, Layer Sampling - Current Time. Then it's down to creating the particles to look and behave as you want.

You should try to get the synthesia bars onto an alpha background in the precomp as the current background will also be generating particles - you'll see what I mean.

[Robertomr00]

Hi Towerguy, firstly thank you for the explanation, it's been so helpful for me. But I'm not an expert in after effects and I'm having a few issues. The first, if you take the thin bar from the precomp as the emitter, it generates the particles all over the screen, not only in the little place I want it to. Probably there's a way to change this, but I can't find it. What I've tried to resolve this is to take one point as the emitter (I know is not very profesional) and the next proble I find is to take the particles under the synthesia bars. I've tried with removing the gray color from the video with Keylight in an adjustment layer, but that's not working. I would try what you said about the alpha background, but I don't know how to do that. The screenshots I attach are for you to get a more visual image of the problem. I may be asking for too much explaination, and I'd understand if you didn't want to continue. Thank you for your time

[Towerguy]

Hi, happy to help but today is a bit frantic. As a starting point put your Keylight effect on the layers themselves, don't use adjustment layers. To get the particles underneath the green bars move the video layer to the above the Particular layer, and turn the eye icon off for the precomp so it becomes invisible. For the rest I will get back to you asap.

[Towerguy · Accepted solution]

Hi. To put the particles behind the bars adds quite a level of complexity. However, let's start from scratch:

In your main comp have your video, duplicate it (Ctrl +D) and Precompose it - call it Precomp ( make sure you Move All Attributes and Open New Composition for the precompositions we're going to be doing).

In Precomp add the Keylight effect and key out the green bars (toggle the Transparency Grid on at the bottom of the screen, it's the checkerboard symbol next to Active Camera). You should see a checkerboard pattern where the green used to be which shows that it is transparent. Add a new solid and make it the same green as the bars (use the eyedropper tool and sample the green from the Keylight swatch). Put the green solid below the video. You should see a TrkMat pulldown box beside the pickwhip symbol - if not press the Toggle Switches/Modes button at the very bottom of the screen and it should appear. Toggle the box open and select Alpha Inverted Matte. This will leave you with only green bars on a transparent background.

Precompose Precomp - it will automatically be named Precomp 2.

In Precomp 2 turn on the eyeball switch beside the video - the video will reappear. Use the pen tool and draw a wide thin box just above the top of the keyboard - this is the area that the particles will appear from. Toggle open the video layer and you will see Masks listed. Select the heading "Masks" and copy it (Ctrl +C). Select the Green Solid layer and paste the mask (Ctrl + V). Turn off the eyeball for the video layer and you should have a thin slit which the green bars will move through with a transparent background.

Now, in your Main comp: make a solid and add Particular to it and move it below your video layer

The layers should now be:

Video

Particular

Precomp 2 (turn off the eyeball - make it 3D)

Precomp

Black Solid (you'll need to make this)

Using the pen tool draw a tight mask around the keyboard only.

Now go to your Particular settings and set it as per my earlier post using Precomp 2 as the Layer emitter.

You should now have little green balls coming our from behind the green bars - now is the time to work on the actual particles to get them to look like you want.

I've tried to be as clear as I can, if there is anything that is unclear let me know.

PS: Getting some knowledge about Track Mattes is one of the most useful things you can learn in After Effects - they have a lot of uses.

[Robertomr00]

Hi again. Firstly, I cannot say how thankful I am to you for having taken the time to explain this step by step and so clearly. I've followed everything you said and it worked almost perfectly. The few problems I haven't been able to solve after a few hours trying appear at the end, with the particles themselves. As you can see in the video, particles appear (although they bright less) all over the screen, and the ones that come from the green bars disapear immediately with the bar. I think the problem comes from the layer sampling setting, but I don´t know how to solve it. Thank you in advance again.

[Towerguy]

Hi. You still appear to have the grey background in your Precomp 2 - see how at the beginning there are grey particles floating up, there should be nothing until the green bars hit the mask. Double check in Precomp 2 that you have switched off the eyeball and that the background is invisible. Try using a larger number of particles and play with the life settings if they die off too quickly.

[Robertomr00]

Hi. I have triple checked that the eyeball is turned off and the background is invisible, as you can see in this screenshot. About the other problem, if I set the life time even to 30 sec. they have the same lifetime as the bars they come from. I've tried also setting the particle number to 100.000 but changes nothing. By the way, the particles in the begining are slightly green, so I think they are generated by the particle system for some reason. If you want I can send you the .aep file for you to understand better the problem os how to solve it, because I've run out of ideas. Thank you

(In the second screenshot I've reduced the velocity for you to notice the changes )

[Towerguy]

Hi, my mistake. In your Particular settings change the Layer Sampling to Still (or try the other one if that doesn't work when you make your own type of particles). Solo the Particular layer and see where the strange particles are coming from (Press the Dot icon near the eye - they might be ones that are floating up from the green bar's particle spread). If not then, in your Precomp 2 drag the ends of your mask inwards so that the box just covers the area that the green bars go through. I would also suggest that you clean your cache (Edit/Purge/Image Cache) in case that's causing it. Sorry I can't use your aep, I'm on CS6 and have an older version of Particular.

You may also want to mute the sound on Precomp and Precomp 2.

[Robertomr00]

Hi towerguy. Yesterday I was the entire day attemping to solve all the problems, and I've made it! The problem which consisted in the particles disapearing too fast was solved with the layer sampling, as you said, set to from birth. The particles floating with no sense were created because in the precomp 2 the mask took maybe one pixel of purple along the screen, enough to generate particles. Now I'm rendering the entire video and my only problem is that it's taking too long, due to the number of particles, I imagine. It's been 11 hours and says there are 25 left. Indeed, I've done some math and it's going to take about 3.5 days! Do you know any way for making this time shorter. Anyway, I'm so thankful to you for helping me with the entire process.

[Towerguy]

Hi, glad you got it sorted out. Sorry I don't know any way to speed up the render for, as you say, there are a huge number of particles (when you use a layer emitter the whole layer emits particles which include alpha particles that you can't see, hence why you have to have such a high birth rate).

[Robertomr00]

Hi Towerguy, I've finished the video. Your nickname is in the description and at the end of the video of course .

You can watch it here: Goodbye Hachiko - Jan A. P. Kaczmarek - YouTube

[Towerguy]

Nice piece of music, glad to have helped.

[Robertomr00]

Hi again. If you don't mind, from time time to time I will post some questions in order to improve the quality of my videos. One thing I've been wondering is whether I can set the particles to interact with themselves, like in the video I linked in the very first question here, where the new particles push the older ones. Thank you in advance

[Towerguy]

Particular doesn't allow particles to interact with themselves but I suspect in the example they used Particular's Auxiliary Particles which is sort-of similar. I think the new version of Stardust does allow actual interaction;

Stardust: The New Standard In After Effects Particle Plug-Ins | Motion Array

[P.M.B]

Looking closely at that, whatever's going on can't be done with stardust either, I don't think.  Yes the latest version has collisions but if you look closely there's also a hazy fog around the particles that's also being pushed in a way that looks very much like a fluid simulation (although maybe it's just a glow)

As I stated way earlier it might be worth trying to ask the person who made the video or even generally in the Synthesia forums how that effect was done.  Seems there's a good chance somebody lurking around those forums might know.

It's awesome that you got it working with Towerguy's help but it's still possible that whoever did it, didn;t do it with After Effects.

[lucar45957726]

Hi, First of all thanks for the good explanation.

Everything worked, but I can not change the particles.

and do you know how it is possible to add the turbulence field on the particular effect. When I trie to add this nothing happens, only when i change the emitter type from layer to sphere.

For example like this->     Fluid Simulations w Particular | After Effects TUTORIAL - YouTube 

thank you in advance

[Towerguy]

Hi. Have you done any basic tutorials on Particular (it is not an easy program to understand, let alone master)? If so, how do you want to "change" the particles? What is not working? To add the turbulence field go to Physics>Air>Turbulence Field. Click on the Visualise Fields checkbox to see the guides.

[Phillip1234]

I finally made the whole particle thing work but unfortunately the rendering time for the 4min video is around 200h, which is absolutely crazy. Does anyone have a real solution for reducing the rendering time without making the quality worse? I've watched many youtube tutorials on this topic and nothing helped till now...