0 Replies Latest reply: Apr 2, 2012 4:41 PM by juerg3n RSS

    Realtime video processing (filters etc.) performance issues

    juerg3n

      I'm developing a simple AS3 game right now. The background of my game consists of the player's webcam image modified through some filters (blur and color matrix). However,  the realtime video processing  drains my CPU and I'm wondering if I can somehow improve my code to make it more efficient and CPU friendly (e.g. by using PixelBender or something ... ).

       

      I've already done the following tricks to improve performance:

      • Use Bitmap Data class etc. instead of video class
      • Set blur filter quality to low
      • Set mode of webcam to capture at only 1/4 of the availalbe resolution
      • Only capture webcam images at 15fps
      • Activated hardware acceleration (2 GPU) - However, that doesn't seem to have any effect on the performance.

       

      I tried using a semi-transperent blue overlay instead of the color matrix filter but that doesn't look as good. Below you can find a screenshot of what the background looks like and my current "VideoBkr" class handling the video processing. I'm gratefull for every hint in the right direction ...

      Bildschirmfoto 2012-04-03 um 01.32.50.png

       

          import flash.display.Bitmap;

          import flash.display.BitmapData;

          import flash.display.Sprite;

          import flash.events.Event;

          import flash.events.TimerEvent;

          import flash.filters.BitmapFilterQuality;

          import flash.filters.BlurFilter;

          import flash.filters.ColorMatrixFilter;

          import flash.geom.Point;

          import flash.geom.Rectangle;

          import flash.media.Camera;

          import flash.media.Video;

          import flash.utils.Timer;

       

          public class VideoBkr extends Sprite {

              var video : Video;

              var colorMat : ColorMatrixFilter;

              var rect : Rectangle;

              var point : Point;

              var bkrwidth:uint =  Main.instance.stage.stageWidth;

              var bkrheight:uint =  Main.instance.stage.stageHeight;

              var blurFilter : BlurFilter = new BlurFilter(64, 64, BitmapFilterQuality.LOW);

              var bmd : BitmapData = new BitmapData(bkrwidth, bkrheight, false, 0);

       

              public function VideoBkr() : void {

                  // Define color matrix for filter

                  var matrix : Array = [0, 0, 0, 0, -255, 0, 0.5, 0, 0, -120, 0, 0, 0.4, 0, 90, 0, 0, 0, 0.5, 0];

                  colorMat = new ColorMatrixFilter(matrix);

       

                  // Get webcam and attach it to video object

                  var cam : Camera;

                  cam = Camera.getCamera();

                  cam.setMode(bkrwidth / 2, bkrheight / 2, 15);

                  // Capture image with 1/4 of the original resolution for better performance

                  video = new Video(bkrwidth, bkrheight);

                  video.attachCamera(cam);

       

                  // rectangle and pointer for applying filter

                  rect = new Rectangle(0, 0, bmd.width, bmd.height);

                  point = new Point(0, 0);

       

                  // link BitmapData to Bitmap

                  var canvasBitmap : Bitmap = new Bitmap(bmd);

       

                  // Mirror webcam image and add it to stage

                  canvasBitmap.scaleX = -1;

                  canvasBitmap.x = bkrwidth;

                  canvasBitmap.y = 0;

                  addChild(canvasBitmap);

       

                  // timer for refreshing webcam image: 15fps

                  var refreshTimer : Timer = new Timer(1000 / 15);

                  refreshTimer.start();

                  refreshTimer.addEventListener(TimerEvent.TIMER, imageRefresh);

              }

       

              function imageRefresh(evt : Event) : void {

                  // Refresh webcam image and apply filters

                  bmd.draw(video);

                  // apply colormatrixfilter (blue-ish ...)

                  bmd.applyFilter(bmd, bmd.rect, new Point(), colorMat);

                  // apply blur

                  bmd.applyFilter(bmd, bmd.rect, new Point(), blurFilter);

              }

          }