system.callSystem on macOS with UTF-8 characters produces unexpected results

[Arie Stavchansky]

Hi There,

I'm running into an issue on macOS when passing strings with UTF-8 characters into the system.callSystem() method.  I'm using AE 15.1.1 and macOS 10.13.5.  On Windows, this test code works fine.

This is the very simple and basic code I'm using within the ESTK:

var cmd = "echo 'Does this character work? => x'"
$.writeln(cmd)
system.callSystem(cmd)

Here are some test strings and the results as listed in the ESTK Javascript Console window.

1.

Test string   

"echo 'Does this character work? => x'"

ESTK Javascript Console Result

echo 'Does this character work? => x'
Result: Does this character work? => x

2.

Test string

"echo 'Does this character work? => ©'"

ESTK Javascript Console Result

echo 'Does this character work? => ©'
Result: Does this character work? => ÔøΩ

3.

Test string

"echo 'Does this character work? => ב'"

ESTK Javascript Console Result

echo 'Does this character work? => ב'
Result: Does this character work? => ?

4.

Test string

"echo 'Does this character work? => Ă'"

ESTK Javascript Console Result

echo 'Does this character work? => Ă'
Result: Does this character work? => ?

5.

Test string

"echo 'Does this character work? => ”'"

(note that this is a closed curly quote)

ESTK Javascript Console Result

None.  Both ESTK hangs until the "stop button" is pressed.  When the "run button" is pressed again, ESTK reports a very strange error, then and AE and ESTK freeze and require a force quit:

After Effects error: NSInvalidArgumentException: *** [__NSArrayM InsertObject:atIndex:]: object cannot be nil

As you can see above, in some cases the character is returned perfectly.  In other cases, the character is shown as a question mark, and then in others it crashes both ESTK and AE.

Does anybody have any information as to why this happens?  Am I missing something in either my code or my OS config?  Any help is appreciated!

Thanks!

[Alex White · Accepted solution]

Yep, that problem has been there for a while and the only way to fix that is to use .sh scripts with UTF-8 encoding and Mac .lineFeed.

[Arie Stavchansky]

Thanks Alex, was going nuts for a while trying to understand the issue.  I’ve moved over to placing the code in a .sh file and then invoking that bash file using the system.callSystem method.