4 Replies Latest reply: Sep 24, 2013 7:12 AM by sinious RSS

    Help with Video Performance Issues using Flash

    TriciaLawrence Adobe Employee

      Asking on behalf of a customer who has been unable to get any answers so far - are you able to help?



      We have a port of our Game Development Kit which allows us to recompile all our games using Crossbridge (http://adobe-flash.github.io/crossbridge/) into SWF without any code modifications.



      Our framework is using OpenGL for rendering and we have successfully ported it along with the audio and video to run in Flash.


      We are experiencing performance issues using Video. We cannot use image sequence as some of the video animations are too long and would increase the download to an unacceptable size.  Assets vary between 256x256 - 1024x1024 videos.

        • 1. Re: Help with Video Performance Issues using Flash
          sinious MVP

          What is the target, devices or desktop?


          Are they using StageVideo or literally the old Video classes?


          What are they applying this video to? The square base 16 aspect makes me think they're using them as a texture. Is this in Stage3D?

          • 2. Re: Help with Video Performance Issues using Flash
            TriciaLawrence Adobe Employee

            Hi! - I'm just the messenger, but have more information in their email which I'd like to forward to you - can you connect with me by email at palawren@adobe.com?  Thank you!

            • 3. Re: Help with Video Performance Issues using Flash
              TriciaLawrence Adobe Employee

              Here's the rest of the story.  Let me know if you can see any resolution, and I will connect him to the forums.  Thank you.


              Current Video Solution:

              We create an instance of NetConnection, NetStream, and Video according to most samples out there, and invoke draw to rasterize the Video DisplayObject into a BitmapData instance.


              The BitmapData instance has a fixed color component layout which is not compatible with Stage3D texture and is therefore has to be reformatted before uploaded to Stage3D Texture (See Code Listing below).


              Our Problems:


              1. Performance issues with RGBA conversion (either using copyChannel or manually reformatting is not fast enough) natively in as3; this required for stage3d texture.
                1. Copying each channel individual using bitmapdata.Copychannel seems faster, but not significantly faster.
              2. Cannot detect when video frame has been updated, therefore we may copy pixels that are not needed in enterframe (processpixel).
              3. Looping video, our current solution uses the NET_STATUS event Buffer empty; Is there a better way to loop videos than checking buffer and seeking to 0.
              4. Stepping video, loading FLV or MP4 side by side assets from HTTP or embedded does not support stepping? Is there another way?


              ActionScript Code Listing:


              video_nc = new NetConnection();

              video_nc.addEventListener(NetStatusEvent.NET_STATUS , onConnect);

              video_nc.addEventListener(AsyncErrorEvent.ASYNC_ERROR , trace);




              // OnConnect Event:

              1. this.ns = new NetStream(e.target as NetConnection);

              eventclient = new Object();

              1. eventclient.onMetaData = onMetaData;

              1. this.ns.client = eventclient;
              2. ns.play(flvfile);
              3. ns.pause();



              //onMetaData event:


              1. this.width = infoObject.width;
              2. this.height = infoObject.height;
              3. this.textureWidth = NextPowerOfTwo(this.width);
              4. this.textureHeight = NextPowerOfTwo(this.height);

              cliprect = new Rectangle(0, 0, this.width ,this.height);

              1. cliprect.x = 0;
              2. cliprect.y = 0;
              3. cliprect.width = this.textureWidth;
              4. cliprect.height = this.textureHeight;

              totalFrames = infoObject.duration * infoObject.fps;


              1. this.hasAlpha = true;

              if(infoObject.videocodecid == 5)

              this.hasAlpha = true;


              1. this.bitmapData = new BitmapData(this.textureWidth, this.textureHeight, hasAlpha, 0xff000000);

              1. this.video = new Video(this.width, this.height);
              2. this.video.attachNetStream(ns);
              3. this.video.addEventListener(Event.ENTER_FRAME, processPixels);


              // processPixel method:


              1. BitmapData.draw(video);
              2. GLAPI.instance.glBindTexture(GLAPI.GL_TEXTURE_2D,this.textureId);

              var fmt:uint = GLAPI.GL_ARGB;


              // converting pixels using copychannel or loop through pixels


              1. GLAPI.instance.glBindTexture(GLAPI.GL_TEXTURE_2D,this.textureId);
              2. GLAPI.instance.glTexImage2D(GLAPI.GL_TEXTURE_2D, 0, fmt, this.textureWidth, this.textureHeight, 0,fmt, GLAPI.GL_UNSIGNED_BYTE, 0, convBitmapData.getPixels(cliprect));


              • 4. Re: Help with Video Performance Issues using Flash
                sinious MVP

                Just to get the basics started, hopefully flasCC users will assist..


                On 1) Alpha is expensive. Have they tried any calculations without it to see the performance difference? Depending on the target, by any means the calculations would perform much better without the middle men (NetStream->Video->BitmapData->Upload). A custom decoder library or (woo) any possible native extension directly leveraging hardware decoding on the GPU with a direct channel to Stage3D if it's even possible would be best. There's just no speedy path between the flash layer and Stage3D (here's to hoping for a buffer hook between StageVideo and stage3D!)


                On 2) If you're talking per-pixel changes on spatial compression, I've never seen it. The comparison algo would probably be just as costly as just processing every pixel.


                On 3) It has been a life long wishlist item. Seek(0) always studders and has just never looped seamless. I've always has to use clips on a timeline embedded in a SWF to be seamless.


                On 4) You can only seek to keyframes unless the content is delivered from a streaming server or played on a timeline.


                There's a few good Starling debates on video to textures worth googling. Strategies based on cutting videos into small aligned chunks, sampling strategies, etc.