Skip navigation

Alchemy (READ ONLY)

paleozogt

Currently Being Moderated

upgrading llvm?

Jul 21, 2009 3:30 PM

I'm trying to compile the Speex library with Alchemy 0.5a, but LLVM is crashing:

if /bin/sh ../libtool --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.. -I../include -I../include -I..      -g -O2 -fvisibility=hidden -MT quant_lsp.lo -MD -MP -MF ".deps/quant_lsp.Tpo" -c -o quant_lsp.lo quant_lsp.c; \
     then mv -f ".deps/quant_lsp.Tpo" ".deps/quant_lsp.Plo"; else rm -f ".deps/quant_lsp.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I.. -I../include -I../include -I.. -g -O2 -fvisibility=hidden -MT quant_lsp.lo -MD -MP -MF .deps/quant_lsp.Tpo -c quant_lsp.c -o quant_lsp.o
Constants.cpp:1548: failed assertion `C->getType()->isInteger() && Ty->isInteger() && "Invalid cast"'
quant_lsp.c:382: internal compiler error: Abort trap
Please submit a full bug report,
with preprocessed source if appropriate.
See <URL:http://llvm.org/bugs> for instructions.
mv: rename .deps/quant_lsp.Tpo to .deps/quant_lsp.Plo: No such file or directory

The LLVM used by alchemy is pretty old (v2.1), so I tried upgrading the llvm-gcc compilers (as mentioned here) However, this doesn't work, as trying to compile a simple "hello world" code results in

opt: Invalid bitcode signature

Any tips on upgrading Alchemy's LLVM?

3374 Views 12 Replies Latest reply: artguate, Mar 21, 2011 5:15 PM

Replies

Currently Being Moderated

1. paleozogt,

Jul 22, 2009 12:57 PM in reply to paleozogt

Report

So, the reason that it wasn't working was that I had only partially upgraded LLVM. Not only the gcc stuff has to be upgrade, but all the llvm-*, opt, gccas,gccld, etc, stuff must be upgraded as well.

This will compile things fine-- until it gets to the link step. Using LLVM v2.3 with a 'hello world' results in

llc: for the -march option: : Cannot find option named 'avm2'! llc: Unknown command line argument '-avm2-use-memuser'. Try: 'llc --help' llc: Unknown command line argument '-avm2-package-name=cmodule'. Try: 'llc --help'

llc out of the box doesn't support the 'avm2' architecture. It appears that adobe branched llc to support avm (ActionScript Virtual Machine). It seems that the source code available to us is for gcc and the C++ standard lib. There doesn't seem to be any source for the branched llvm stuff (like llc). Why?

So: LLVM v2.1 is crashing in the opt tool. I could upgrade LLVM to v2.1+ (which doesn't crash in opt), except that it requires an upgraded llc that has avm2 support, but there's no source for that. And now I'm stuck.

|
Mark as:
Currently Being Moderated

2. zazzo9,

Jul 22, 2009 1:34 PM in reply to paleozogt

Report

I'm afraid that's the deal. Adobe created their own, closed source, llc, to compile from llvm 2.1 bytecode into actionscript. Open sourcing lcc has been mentioned, but it has not happened. Hence, there are three, horrible, solutions:
Try to backport the bugfix to llvm 2.1, or a compatible llvm version, and use adobe's llc will the fixed llvm version.
Examine the output of adobe's llc and write your own actionscript compiler for a newer llvm release.
Use some kind of binary commenting search to identify the area of code causing the internal error, and rewrite or remove it so as to get the thing to compile.

|
Mark as:
Currently Being Moderated

3. paleozogt,

Jul 22, 2009 1:47 PM in reply to zazzo9

Report

(1) Seems doable, if i had any idea of what the bug was (I *think* its fixed in v2.2). Searching through the LLVM bug database hasn't been very revealing. I could do a patch with the v2.2 changes, but I think this might bring in bytecode format changes... Has anyone done this?
(2) omg!
(3) I'm not sure how to go about doing this. What LLVM settings would allow me to see the 'bad' area of code?

|
Mark as:
Currently Being Moderated

4. zazzo9,

Jul 22, 2009 2:00 PM in reply to paleozogt

Report

(1) I haven't managed to fix any of the internal compiler errors llvm 2.1 sometimes spews out. You could try just using 2.2 -- maybe it's close enough. Otherwise, I wish you luck -- and would be quite interested in your results. If doing a full blown patch, you'd probably have to comprehend any bytecode changes and make sure they're not included.

(3) What I mean here is, trimming down the code you're asking llvm to compile until you find a very small change which triggers the crash, and then either isolating what causes it or tweaking the change so the code doesn't exist. I.e. originally I have a massive cpp file referencing a number of .h files which is crashing the compiler. I comment out half of the entire file with #if 0 / #endif, in such a way that it still compiles. If the crash still occurs, I delete the commented half and comment out half of what's left. If the crash doesn't occur, I try commenting out the other half. Eventually things narrow down to one or more key areas, the presence of which the crash hinges on. If you're lucky, you don't need one of the things that makes it crash and you can just leave it cut out -- otherwise, you can try to rewrite it in some other way so that there is no crash.

I've done this by hand, but I bet llvm could help somehow. For example, if you got a stack trace on the crash by running the compilation under gdb, it might help reveal where the code was parsing and what was going on. Of course, once you start understanding that stuff, #1 and #2 are going to look a little more enticing.

|
Mark as:
Currently Being Moderated

5. paleozogt,

Jul 22, 2009 3:31 PM in reply to zazzo9

Report

I would've bet you that (3) wouldn't work, because it would involve some kind of deep loop-unrolling optimization code and who knows why it makes llvm crash?

I would've lost that bet.

Using your "#ifdef binary search" idea, I found the lines of code that were causing the crash in llvm's opt. Its lines like these (in libspeex's quant_lsp.c):
id = lsp_quant(qlsp, cdbk_nb, NB_CDBK_SIZE, order);
Apparently the bug in llvm's opt has to do with numeric literals being passed to functions (NB_CDBK_SIZE is a #define macro for 64). The workaround was to create a global value to hold that literal (local temp vars won't work):
// goofball workaround for llvm bug (can't be a const or static) int nb_cdbk_size= NB_CDBK_SIZE;
and pass it into the function:
id = lsp_quant(qlsp, cdbk_nb, nb_cdbk_size, order);
And now its compiling.

So, if anyone has gets strange llvm crashes-- you can't upgrade llvm, but you can try a "#ifdef binary search"!

|
Mark as:
Currently Being Moderated

6. fozzter,

Nov 17, 2009 11:58 AM in reply to paleozogt

Report

I tried the Alchemy toolkit on various versions of Ubuntu and 7.10 seems to work the best. I'm guessing 7.10 is the version the llvm tools were built against. The compile errors seem to be related to specific shared libraries in the newer releases, such as libc (/lib/tls/i686/cmov/libc.so.6).

This may not be your case but going with 7.10 seemed to make a lot of my compile problems go away.

|
Mark as:
Currently Being Moderated

7. artguate,

Mar 18, 2011 3:06 PM in reply to paleozogt

Report

Hi Aaron,

Did you ever got speex to work in Alchemy?

Thanks for your tip, I got it to compile. Had the same issue you did. Just changed the lines of code with the "issues" and it compiled fine.

But once I ran it, it didn't produce the proper output. I'm wondering if you did get it to work.

I took the input from the microphone, converted it to PCM, as it's in float format, and sent it to the encoder.

The enconder runs fine, returns a size that I would expect, but the data isn't just right.

When I compare the output from the encoder built running alchemy, the first 4 bytes of each frame are identical to the bytes returned by the win32 executable supplied by speex, but after the 4th byte of each frame the result varies.

So I'm wondering if you got yours to work properly?

Thanks,

-Art

|
Mark as:
Currently Being Moderated

8. paleozogt,

Mar 18, 2011 4:14 PM in reply to artguate

Report

I was able to get speex working.

Most likely the problem you're seeing is a byte-order issue. For some strange historical reason the bytes coming out of the microphone are Big Endian, but Alchemy (and Flash) are really Little Endian. So you need to flip each float.

This is true for encoding (Mic -> speex) or decoding (speex -> Sound).

Also, I would recommend doing the byte-order flipping in C code.

|
Mark as:
Currently Being Moderated

9. artguate,

Mar 19, 2011 9:11 AM in reply to paleozogt

Report

Thanks,

Do you mind if I ask you a couple of questions, so that I can avoid lunacy?

So that I can me sure the library is working correctly and I'm the one causing the problem.

1) What version of Speex did you get it working with?
2) Did you have to set any option when you ran the ./configure or did you ran it without options.

Now the Byte order issue.

It's my understanding that the output from the microphone is in float format, and it's also my understanding that the input speex is expecting is short format.

Did you send the Float input to the speex codec without converting it to short PCM? Just flipping the byte order of the float?

Thank you for your time.

|
Mark as:
Currently Being Moderated

10. paleozogt,

Mar 21, 2011 8:41 AM in reply to artguate

Report

1) I'm using speex v1.2 beta3
2) I'm using speex in a slightly non-standard way-- its source is included directly in my project. So I've never actually had to run configure. :/
3) I'm converting from big-endian float to little-endian short.

|
Mark as:
Currently Being Moderated

11. artguate,

Mar 21, 2011 9:03 AM in reply to paleozogt

Report

Thanks,

I also convert the big-endian float to little-short first multiplying it by 32767.

I'm getting something that sort of resemples the original sound.

I've flipped byte orders, and the impact is dramatic, so I belive I've got that part fine, and if I save the 16 bit short with a WAVE header, it plays just fine in anything else.

I'm either not doing something right in the implementation, or I'm just not seeing some silly mistake, Debugging in Alchemy isn't the easiest thing, but I've got traces everywhere and all the pointers are where they "should" be.

So I'm wondering if the ./configure script breaks something. I'm going to have to try compiling everything in myself, see if that fixes it.

When I look at the pure raw output, it does resemple's alot the output of the .spx file created by the supplied binaries, but it's not exactly the same, the first 4 bytes of every frame are identical, and the difference increases as more frames go by. If I change the byte order or anything else, it's dramatically different. So I'm thinking that the ./configure script misinterprests something and makes a "wrong" adjustment somewhere.

Thanks again.

|
Mark as:
Currently Being Moderated

12. artguate,

Mar 21, 2011 5:15 PM in reply to paleozogt

Report

Thanks, I got it working.

Turns out that you were correct from the beggining. My issue was with the byteorder, but not with speex itself, as all the enconding and decoding was fine, it was when outputing to the audio card. I recalled something you wrote in a different thread. Although Alchemy itself is little-endian, the byte arrays returned by Alchemy are big-endian (Flash's default).

Thanks,

|
Mark as:

Forum guidelines

Answers + Points = Status

10 points awarded for Correct Answers
5 points awarded for Helpful Answers
10,000+ points
1,001-10,000 points
501-1,000 points
5-500 points