Hi Damon,
I was able to do something similar to what you are describing. However, instead of building ffmpeg with Alchemy I used the native process feature found in Air 2.0 to interact with ffmpeg. In short, native process in Air 2.0 allows you to write to stdout and read from stdIn. FFMPEG also has the ability to listen to take input on stdin "-i -" and write to stdout. Furthermore, the the flash.net.NetStream change in Flash version 10.1 allow you to append bytes to bytes to a NetStream. As a result, I'm able to play an MPEG2 video in an Air video player by transcoding on the fly from MPEG2 to flv. Currently, I encounter memory issues with large files but it does work with small files.
If your intention is simply to transcode and not play files then the memory issues I described likely wouldn't be an issue. If I can help further, let me know.
Hmm, I am not so sure anymore whether I actually did succeed building ffmpeg.
Let me give you the whole story:
About a month ago Scott asked me to look at an Alchemy problem reported by a client. He was trying to build a custom version of ffmpeg and got this error:
Cannot yet select: 0x2035780: i32 = ConstantPool < i64 6881500230622117888> 0
I spent a few days just trying to reproduce that problem and I believe during my attempts I successfully built ffmpeg. But that was not what I wanted at the time! After a while I did find a way to reproduce the problem. I ended up using a simple ./configure for ffmpeg 0.5 plus zlib-1.2.5 and bzip2-1.0.5. After excluding mpegvideo_xvmc.c in libavcodec's Makefile I was able to build everything up to linking ffmpeg_g.
That's where I got this error:
gcc -L"/Users/bparadie/Research/ffmpeg-0.5"/libavdevice -L"/Users/bparadie/Research/ffmpeg-0.5"/libavformat -L"/Users/bparadie/Research/ffmpeg-0.5"/libavcodec -L"/Users/bparadie/Research/ffmpeg-0.5"/libavutil -L/Users/bparadie/Research/zlib-1.2.5 -L/Users/bparadie/Research/bzip2-1.0.5 -Wl,-dynamic,-search_paths_first -rdynamic -o ffmpeg_g ffmpeg.o cmdutils.o -lavdevice -lavformat -lavcodec -lavutil -lz -lbz2 -lm
WARNING: While resolving call to function 'main' arguments were dropped!
Cannot yet select: 0xb64eaa0: i32 = ConstantPool < i64 6881500230622117888> 0
0 llc 0x00636dfe _ZNSt8_Rb_treeIN4llvm3sys4PathES2_St9_IdentityIS2_ESt4lessIS2_ESaIS2_ EE13insert_uniqueERKS2_ + 6078
1 llc 0x006373a2 _ZNSt8_Rb_treeIN4llvm3sys4PathES2_St9_IdentityIS2_ESt4lessIS2_ESaIS2_ EE13insert_uniqueERKS2_ + 7522
2 libSystem.B.dylib 0x952422bb _sigtramp + 43
3 ??? 0xffffffff 0x0 + 4294967295
4 libSystem.B.dylib 0x952b623a raise + 26
5 libSystem.B.dylib 0x952c2679 abort + 73
6 llc 0x002f862b _ZN98_GLOBAL__N__Volumes_data_dev_FlaCC_llvm_2.1_lib_Target_AVM2_AVM2 ISelDAGToDAG.cpp_00000000_F04616B616AVM2DAGToDAGISel10SelectCodeEN4llv m9SDOperandE + 187
7 llc 0x002fa193 _ZN98_GLOBAL__N__Volumes_data_dev_FlaCC_llvm_2.1_lib_Target_AVM2_AVM2 ISelDAGToDAG.cpp_00000000_F04616B616AVM2DAGToDAGISel10SelectRootEN4llv m9SDOperandE + 819
8 llc 0x002e6a2c _ZN4llvm19X86_64TargetMachineD0Ev + 65116
9 llc 0x003de4ca _ZN4llvm11StoreSDNodeD1Ev + 1610
10 llc 0x0040d3fe _ZN4llvm11StoreSDNodeD1Ev + 193918
11 llc 0x0040f92e _ZN4llvm11StoreSDNodeD1Ev + 203438
12 llc 0x005d1926 _ZN4llvm12FunctionPassD1Ev + 20998
13 llc 0x005d1f3a _ZN4llvm12FunctionPassD1Ev + 22554
14 llc 0x005d20c5 _ZN4llvm12FunctionPassD1Ev + 22949
15 llc 0x00002e44 _mh_execute_header + 7748
16 llc 0x00001f36 _mh_execute_header + 3894
make: *** [ffmpeg_g] Error 6
Here is a technique that I told the client and it helped him tremendously isolating and then eventually solving this problem.
It turns out that the error above (Cannot yet select:...) is a little bit misleading. When I used llc -debug I saw a different picture.
First I set ACHACKS_TMPS to 1 and rebuilt the project in order to get to the temporary bc files before calling llc:
export ACHACKS_TMPS=1
make
llc -debug -f -o 68122.achacks.as 68122.achacks.exe.bc
After that I got this error:
...
=== asin
Couldn't allocate output reg for contraint 'r'!
The llvm code for asin() looks like this:
define internal double @asin(double %X) {
entry:
%tmp2 = tail call double asm "$0 Math.asin($1);\0A", "=r,r,~{dirflag},~{fpsr},~{flags}"( double %X ) ; <double>
[#uses=1]
ret double %tmp2
}
The problem with "Couldn't allocate output reg for contraint 'r'" seems to be an old Alchemy problem, see these posts:
http://forums.adobe.com/thread/525968?tstart=0
http://forums.adobe.com/message/1871533#1871533
The client isolated the problem further and identified an Alchemy bug related to casting uint64 to double. From what I understand he then wrote his own uint64_to_double() function in C and went through the whole ffmpeg code base and replaced uint64 to double casts with calls to his custom function. That's pretty advanced stuff and that's why I think that Lame or the method that Joe described might be the better way to go.
Well, that's the whole story.
What you might learn from this is the trick with llc -debug and that building ffmpeg with Alchemy is most likely going to be a rocky adventure.
Cheers,
- Bernd
Thanks for the detailed info, Bernd. Yes, it sounds rocky indeed.
I'm not positive, but it sounds like the method Joe describes is calling the ffmpeg executable (in Windows, for example) from ActionScript? I've done this with C# and that's fine, but I'm wondering about distribution of an implementation like that to multiple platforms, since AIR capitalizes on this. If this is the case, it sounds like I would need to include third-party executables in my chosen install method, for example. Is this correct?
I agree with you, if you want to deliver your AIR app on multiple platforms then you either have to create some sort of "fat binary" for ffmpeg, or you bundle your app with platform versions of ffmpeg. Alternatively you bite the bullet and port ffmpeg using Alchemy. But then you'll most likely get a slower app.
I think those are the trade-offs.
- Bernd
Hi Bernd,
When you say:
The problem with "Couldn't allocate output reg for contraint 'r'" seems to be an old Alchemy problem, see these posts:
[...posts...]
The client isolated the problem further and identified an Alchemy bug related to casting uint64 to double.
Does this mean that the "Couldn't allocate output reg for contraint 'r'" problem can happen even if there is no (non-avm2) inline assembly in the code? All of the posts you point to refer to inline assembly, but I've tried the tricks for finding this and I'm not seeing any.
I'm seeing:
Cannot yet select: 0x9b1e990: ch,flag = AVM2ISD::CALL - A call instruction 0xc4d49d0, 0xc67f3c0
My llc -debug output has:
[...]
cond_true881.cond_next948_crit_edge: 0x42dd850, LLVM BB @0x41e2450, ID#107:
Predecessors according to CFG: 0x41d9000 (#106)
JMP mbb<cond_next948,0x42dd9f0>
Successors according to CFG: 0x42dd9f0 (#111)
Total amount of phi nodes to update: 1
Node 0 : (0x42dda80, 1089)
Couldn't allocate output reg for contraint 'r'!
The last function name in the output is pretty large and has no inline assembly, so it's hard to know where the issue is. (The llc -debug output is huge! 140MB!)
Could you expand on what the customer did to determine that this was a uint64 to double problem?
Any additional suggestions you could offer for tracking this down would be most helpful!
Thanks!
-c
Hello C,
the "Couldn't allocate output reg for contraint 'r'" problem usually only occurs if the C source code uses pretty wild casts, i.e.
Toaster* toaster = (Toaster*)bunny;
There are quite a few wild casts like that in the ffmpeg source code, which is to me ... well let's just say it rhymes with "happy mode".
When llc processes this kind of source code it just passes those cast requests to the backends. In this case Alchemy would receive an expression that asks for a toaster return value from a bunny input value. Unfortunately Alchemy 0.5 does not handle exotic casts like that well. Future versions of Alchemy will be more robust.
I believe Joe will reply to this post later with more details on how to deal with the problems in ffmpeg. From what I understand narrowing the ./configure flags (thus minimizing your exposure to questionable code) helps. But if you do end up with code that triggers the error above then the best thing you can do is cleaning up the code - which is a good thing to do anyways. Joe might be able to present some examples. When we looked at the code we decided that removing the casts and adding temporary variables increased readability and gave the compiler a chance to come up with somewhat reasonable types to convert to.
In regards to how our client solved the problem I am afraid I can't really say very much about it. He only described the method, which I shared, but I didn't get to see the source code.
Hope this helps,
- Bernd
With a little guidance from Bernd, I've been trying to build FFMPEG from Alchemy over the past month. Here is the approach I'm taking in detail.
Get FFMPEG building and debugging with gcc (/usr/bin/gcc).
1. Create a CTD (C/C++) project in Eclipse.
2. Create a new makefile project
- Use Mac OS GCC toolchain
3. Install Lame
4. Run configure with the following options:
./configure --disable-shared --disable-vhook --disable-mmx --enable-debug --disable-ffserver
5. Build
7. Setup a debugging configuration to use ffmpeg_g
8. Set the debugger to gdb/mi
9. Set the GDB command set to "Standard (Mac OS)"
Experiment with configure to isolate the code related to the features you need.
1. Create a test that accomplishes what you want to with FFMPEG. In my case, a subset of FFMPEG will accomplish what I need it to.
2. Note that if you need a specific component (demuxed, parser, decoder, etc…) you can disable all components and then enable specific components. This example will enable only mpegvideo:
--disable-demuxers --enable-demuxer=mpegvideo
Try to build it with alchemy gcc and if (when) it fails, isolate the function that is failing with llc -debug.
1. When the debug fails it will leave behind build artifacts (*.bc) files. Find the bc file left behind.
2. Print the debug output to text file with a command that looks something like this:
llc -debug -march=avm2 -avm2-use-memuser -o=26006.achacks.as -avm2-package-name=cmodule.ffmpeg_g <build artifact file>.bc >& ~/Desktop/out.txt
Note: I think the exact arguments will differ from platform to platform. To find the argument for your platform enable logging for gcc and check the gcc.log file.
Note: The output files will be quite large. I've had one exceed 2GB.
3. grep the log for "^===" to get a list of functions that compiled.
cat out.txt | grep "^==="
The last function in the list is the function that caused the failure.
4. Clean up that function. This is the step that I can't give exact steps for. Here are some general tricks I've used to get around this:
- As Bernd said look for wild casts. In addition to what has already been said, I've fixed a few functions by eliminating the following casts:
(uint16_t*)
(const uint16_t*)
Note: While this fixed a few functions for me I don't think it would be correct to say all casts like this need to be eliminated. I was in the middle of investigating these when I got diverted to other things. At this point consider it something to look for and experiment with in a failing function.
- Another clue that may help is to look for warnings thrown by your compiler which can be seen in your IDE.
- I've also found that for my purposes some of the functions (specifically logging functions) can be eliminated entirely.
- Any time I make and edit to the code I build and test it with Eclipse (/usr/bin/gcc) to ensure that the functionality I needed wasn't broken.
At this point I think I'm close but I haven't had time finish in the past few weeks. If I finish it, I will post exactly what I did.
Hey guys
I have also been trying to port ffplay through Alchemy. My objective is to play DivX or mjpegs in AIR. But lately, I realized that ffplay(or ffmpeg) can only act as a decoder or transcoder for the video which means that there needs to be a special video component as well in Flex which can understand the decoded stream. Also can a function in C pass the decoded stream fast enough to a function in ActionScript to play a video in real time.So transcoding on the fly is possible but direct playback seems unlikely.
Am I missing something??
Dude,
I dont think its an easy task. I tried to follow the steps mentioned in this forum and got stuck mid way. You have to dig inside the code of ffmpeg an figure what functionality u want to use to tune ur actionscript functions. My objective was to directly play other video formats but for that I will have to build a netstream class which will understand these streams.Not really a part of my project but would have helped if it was any easier. So mostly i will be using server side encoding. Best of luck.
@Froffs
I have made some progress but nothing I'm ready to report quite yet. Since you are using AIR you may consider looking at Native Processes if you haven't already. (See above) I'm convinced what you're trying to do can be done but, I'm not sure what the performance would look like since you will be transcoding on the fly.
@Vineet89
Exactly! Your not missing anything. FFMPEG is only a decoder or transcoder and transcoding on the will likely run into performance issues if you tried to transcode and play at the same time.
@focusppokus
I agree that it would be nice of FFMPEG could be provided as a swc. When I first tackled this project I was hoping we could do exactly that. It turns out that providing a complete implementation of FFMPEG as a swc is a bigger project than I originally realized. It would be one thing to get FFMPEG to compile. It's quite another to write and test an ActionScript interface that exposes all the FFMPEG functionality. Its much easier to limit your swc to the subset of FFMPEG that you need. Unfortunately, this doesn't benefit the community as much.
@Bruce.Lane06
I would be glad to give samples. What specifically would you like to see?
I've been away on vacation for the past few weeks. I'll respond more quickly in the future.
Thanks for your response, Joe!
I would like to send BitmapData from an AIR app to ffmpeg so it can encode the images to create a video file (for instance an avm2 swf in my case).
Even if slow performance...
Any other (already working) code samples would be appreciated to get started.
Have a nice day!
@Bruce.Lane06
Just wanted to let you know I'm still working on this when time permits. You and I are using FFMPEG for different purposes so I'm not sure that my code will apply. Still, I'll pass on any tricks I learn along the way related to FFMPEG in general. Since this is more of a personal project for me progress has been slow. I hope to have more info info to share in about 2-3 weeks.
FYI - ffmpeg-0.6 seems to compile, without code modification. I'd post the SWFs, except: (1) they're completely untested, and (2) I can't see any way here to attach a .zip file.
Note that I did cheat a little, by using standard ./configure (with standard gcc) to create the executables in Cygwin, then:
There is some bug in the final copying of the SWFs / SWCs (including the creation of SWC files) which I can't figure out. However, with ACHACKS_TMPS=1, it does, in fact, create the SWFs.
Once I get a chance to test them (which may not be for a while, as I have no use for ffmpeg right now), I'll be happy to post them somewhere more public.
Or, if anyone else would like to test them, I'd be happy to send you a .zip file containing the relevant items.
(And if that doesn't work for you, then let me know, and I can also send along my slightly-modified "gcc" and "achacks.pl" file. I've long since forgotten what I did to them, but the changes were fairly simple, related more to some weirdness with the script or debugging or somesuch, and not to what C code does / doesn't compile. I could make .diffs, but again: where to post? I'm hesitant to put them up in an "official" location without testing, but would be willing to toss 'em in a forum post.)
Hi, I would be very interested in testing this ffmpeg port.
I need it to extract the codec information.
You could upload it to rapidshare or hotfile or something similar.
Or just send it to my e-mail addres: crockn@gmail.com
thx,
Sam
Could you please send me your Alchemy (as a zip folder for example) and some step by step compilation instructions (because I try http://stackoverflow.com/questions/6317816/how-to-compile-ffmpeg-via-a lchemy-gcc and http://stackoverflow.com/questions/6333275/adobe-alchemy-compiler-prob lem-it-generates-l-bc-files-how-to-turn-them-into-s but get really bad output=() Oleg.Jakushkin@gmail.com pleeeease=)
I finally have a portion of FFMPEG compiling with Alchemy and running in a Flash App. For a handful of reasons, I'm not going to dump my work to a zip file but, I will share everything I learned along the way. In addition to what I mentioned earlier here are a few additional steps I would advise:
1. Use the latest version of FFMPEG.
When I started using version 6.1 many of the earlier compile issues I had disappeared. I didn't dig too deeply into why that was the case. I suspect that much of the code Alchemy struggled with has been modified to Alchemy's liking.
2. Disable Inline Assembly
Be sure to include --disable-asm in your configure statement.
3. Math Function Improvisation
Some of the math functions were missing so I had to work around that. One example, I replaced cbrtf (Cube Root) references were replaced with pow(value, .333)
4. Allocate enough flash memory
Here's how you would set flash memory to 30MB:
ApplicationDomain.currentDomain.domainMemory.length = 31457280;
5. Glue Code
FFMPEG ultimately uses freed, fwrite, and seek to write contents to files and similar functions to write to streams. I replaced these references to alchemy equivalents of AS3_ByteArray_readBytes, AS3_ByteArray_writeBytes, and AS3_ByteArray_seek. Since function pointers were used in this section of the code, it was relatively non-intrusive to do. The following was my basic approach when interacting with the FFMPEG with Flex/Flash.
- Read a file into a byte array
- Transcod the contents of the source byte array into a destination byte array.
- Play or save the destination byte array.
6. Ensure that compilation in C continues to work as you modify FFMPEG.
I used a small MakeFile change to switch back and forth between compiling for C and Alchemy.
7. I recommend rewriting the MakeFile
I encountered issues compilation that I solved by building the library directly from the object files (Bypassing the static library phase). Furthermore, I found the FFMPEG make files difficult to work with. Here's excerpt from the file I wrote:
CC=gcc
CFLAGS=-DCOMPILE_FOR_ALCHEMY -DHAVE_AV_CONFIG_H -I. -std=c99 -mdynamic-no-pic -O3 -g -Wall
LIBAVCODEC_OBJS=libavcodec/aandcttab.o libavcodec/adpcm.o libavcodec/allcodecs.o ...
LIBAVDEVICE_OBJS=libavdevice/alldevices.o libavdevice/avdevice.o
LIBAVFORMAT_OBJS=libavformat/allformats.o libavformat/avc.o libavformat/avio.o …
LIBAVUTIL_OBJS=libavutil/adler32.o libavutil/aes.o libavutil/avstring.o …
LIBSWSCALE_OBJS=libswscale/options.o libswscale/rgb2rgb.o libswscale/swscale.o ..
FFMPEG_OBJS=ffmpeg.o
ARTIFACT=ffmpeg_g.swc
OBJS_LIGHT=ffmpeg.o libavformat/utils.o
all: ${LIBAVCODEC_OBJS} ${LIBAVDEVICE_OBJS} ${LIBAVFORMAT_OBJS} ${LIBAVUTIL_OBJS} ${LIBSWSCALE_OBJS} ${FFMPEG_OBJS}
@echo "starting the ffmpeg build ..."
${CC} ${LIBAVCODEC_OBJS} ${LIBAVDEVICE_OBJS} ${LIBAVFORMAT_OBJS} ${LIBAVUTIL_OBJS} ${LIBSWSCALE_OBJS} ${FFMPEG_OBJS} ${CFLAGS} -swc -o ${ARTIFACT}
@echo "finished the ffmpeg build"
libavcodec/*.o : libavcodec/*.c
@echo "compiling $< ..."
${CC} ${CFLAGS} -c $< -o $@
libavdevice/*.o : libavdevice/*.c
@echo "compiling $< ..."
${CC} ${CFLAGS} -c $< -o $@
libavformat/*.o : libavformat/*.c
@echo "compiling $< ..."
${CC} ${CFLAGS} -c $< -o $@
libavutil/*.o : libavutil/*.c
@echo "compiling $< ..."
${CC} ${CFLAGS} -c $< -o $@
libswscale/*.o : libswscale/*.c
@echo "compiling $< ..."
${CC} ${CFLAGS} -c $< -o $@
*.o : *.c
@echo "compiling $< ..."
${CC} ${CFLAGS} -c $< -o $@
I'm not finished yet. I still have a few bugs to work through and hope to get pthreads working with it so that the app is not seized up while the transcode is taking place. I hope this helps. Let me know if you have any questions. I'll do my best to respond in a timely manner.
Hi Joe,
Regarding "FFMPEG also has the ability to listen to take input on stdin '-i -' and write to stdout."
So you mean that instead of passing the file path on the native process that invokes ffmpeg I have the option to read the video file content with air api and pass the video data to the ffmpeg native process on the stdin?
thanks a lot.
That is correct. The following command will read from stdin and write to stdout which can access with Native Processes in Air. The ffmpeg command will look something like this:
ffmpeg -i - (other options) -
The "-i -" tells ffmpeg to read from stdin and the "-" at the end tells ffmpeg to write to stdout. To test this with files do something like this:
ffmpeg -i - (other options) - < (path to input file) > (path to output file)
The Flex code will look something like this:
//declare the process
private var _process:NativeProcess = new NativeProcess();
//start the process
_process.start(nativeProcessCmdInfo);
//write to stdin
_process.standardInput.writeBytes(_buffer, 0, _buffer.length);
//read from stdout
_process.addEventListener(ProgressEvent.STANDARD_OUTPUT_DATA, writeToStream);
Let me know if this works for you.
Thanks,
Joe
I finally have a portion of FFMPEG compiling with Alchemy and running in a Flash App. For a handful of reasons, I'm not going to dump my work to a zip file but, I will share everything I learned along the way.
I wonder if we could make a google code project for this (ffmpeg-alchemy?). Ideally, it would just be the modified makefiles and such and (possibly) built .a/.l.bc files.
Hi Joe,
In have gave it a try but does not seem to work. When the method onFileReadProgress attempts to write to the stdin it throws an io error and does not generate output.
(I have been looking at this issue a bit more and looks like the stdin of ffmpeg is overrun with data or something like that, I'm trying to look into it a bit further. Maybe increasing the ffmpeg input buffer size could work but don't know how to do that yet.)
thanks a lot,
Polaco.
I don't remember running into an error like that. Can you tell me at a high level what your trying to do? Feel free to send me an email directly.
Also, I forgot to mention earlier, you don't have to use LAME when compiling FFMPEG with Alchemy. Atleast I didn't for the task I was trying to accomplish.
Here someone seems to have had the same issue:
http://answerpot.com/showthread.php?2767596-AIR+NativeProcess+standard Input+problems
North America
Europe, Middle East and Africa
Asia Pacific