Video playback options for stage3D + AIR mobile?

[Unsanctioned]

We have an existing app on iOS and Android that has videos playing with FLV on mobile, since H.264 has conflicts with stage3D and doesn't render. The current code uses flash.media.Video with NetStream/NetConnection to render the video.

We're looking at releasing a sequel, which includes new videos. However, now that Adobe has removed FLV support from the tools and it is apparently no longer possible to access the older versions of Media Encoder / Premier that still supported FLV, we can't see any options for getting the new videos to play.

Things we've tried so far:

- Use H.264 : doesn't render, presumably due to bug Bug#3626740 - H.264 playback conflicts with Stage3D (Android)

- Use VideoTexture : not supported on Android

What alternatives are left at this point?

[Julien26]

VideoTexture is now supported everywhere so you can use it in your project with stage3D. It kind of work, but it still has some bugs =>

Bug#4150401 - VideoTexture on iOS - Sometimes the event TEXTURE_READY isn't dispatched

Bug#4142301 - StageVideo and VideoTexture - Video and Audio out of sync

And if you want to continue to use FLV, there's a way to (officially, from Adobe servers) download the latest version of Adobe Premiere that has Media Encoder with FLV support. Here's the tutorial :

Adobe CC 2013 Direct Download Links: Creative Cloud 2013 Release | ProDesignTools

But we don't know for how long it will work

[Unsanctioned]

Unfortunately, VideoTexture is not supported everywhere. The first few Android devices I attempted return Context3D.supportsVideoTexture = false.

We did finally get FLV working by using a third-party tool to convert videos, but the same implementation has *terrible* rendering performance on iOS. On that note, the video implementation with FLV *used* to perform fine on iOS under a previous version of AIR. That problem snuck past our QA department and made it into a public release of our older product using the same engine, since we hadn't made any video-related changes with the update - though we *did* update the version of AIR.

[Colin Holgate]

You don't mention it, but I assume you're talking about a section of the app that shows video, and not that you want video rotoscoped into a 3D scene? If that's the case why not use StageVideo? It will definitely step on the graphics card, but at least you know that, and can reset any Stage3D context before going back to the Stage3D part of the app.

[Unsanctioned]

Unfortunately, we can't use StageVideo because we have localized subtitle and UI overlays on top of the video that are rendered with Stage3D.

[Colin Holgate]

There are two things you can do to help performance:

If everything that is seen is a bitmap texture, like, there are no vectors or filters, what you see is in the bitmap image, you could say stage.quality = "low"; It should look just the same.

If you're getting poor performance on older Retina devices, you can tell them not to run in Retina. You add info to the app descriptor file. This for example would only let iPhone 6 and above run in Retina:

<requestedDisplayResolution excludeDevices="iPad2 iPad3 iPad4 iPad5 iPhone4 iPhone5">high</requestedDisplayResolution>

That would go into the <iPhone> part of the XML.

[Unsanctioned]

The original iOS app that now has the performance problem when playing FLV video, but across all devices - has been out for a few years, and performs well outside of the video playback. This is definitely a performance degradation introduced into one of the more recent AIR SDK release, since the performance of the videos was acceptable in previous builds. The update where this was introduced was forced by a need to support 64-bit iOS devices.

I still haven't managed to get a working implementation on iOS that can play MP4 files. Mind you, this is all intertwined with a years-old codebase.

The basic non-VideoTexture code is fairly straightforward, at least as much as using NetConnection/NetStream is straightforward for playing a full-screen video within a much larger engine. At the root, the video player object is a flash.display.Sprite:

v = new Flash.Media.Video(2731, 1536);

nc = new NetConnection();

nc.connect(null);

ns = new NetStream(nc);

ns.client = {

   onMetadata: function(metadata:Object) :void {}

};

ns.inBufferSeek = true;

v.attachNetStream(ns);

addChild(v);

v.x = -v.width / 2;

v.y = -v.height / 2;

v.smoothing = true;

ns.addEventListener(NetStatusEvent.NET_STATUS, onNetStatus);

ns.soundTransform = new SoundTransform(volume, 0);

ns.play(this.videoURL);

...

function onNetStatus(event : NetStatusEvent):void {

   if (event.info.code == "NetStream.Play.Start") {

       startVoiceOver();

       startSubtitles();

   }

   else if (event.info.code == "NetStream.Play.Failed") {

       trace("Video play failed");

   }

   else if (event.info.code == "NetStream.Play.Stop") {

       cleanupVideo();

   }

}