Forums

What is getting overwritten?

Have you tried compiling and running your code outside of alchemy to determine whether the problem is with your code or with alchemy?

Are you saying that

verts = new float[MAX_VERTICES*3];
norms = new float[MAX_VERTICES*3];
uvts = new float[MAX_VERTICES*2];

is giving different results than

m_pVertices = new SVertex[MAX_VERTICES];

?  That's very strange.  I've never encountered memory problems like that in alchemy.

Do you have some relatively short amount of complete, compilable, code that demonstrates the problem?  Is the library you are porting available on the internet?

Hi _divy,

I'm not sure exactly how your string example illustrates your memory problem, but I did see one little oversight that might be the root of your problem. When you malloc the char buffer, you're only allowing for the number of bytes in the file. C strings must be null terminated so you'll actually need 1 more byte, and that byte must be set to zero. Here's my change to your loadmaps function (changes in RED)

//Allocate buffer
// cm.. added +1 for null terminator
buffer = (char*) malloc(sizeof(char)*fileSize+1);

//Read file into buffer
fread(buffer, 1, fileSize, file);

// cm.. terminate the string before converting to std::string
buffer[fileSize] = 0;
   
//Cast the buffer as a string
string sfile = (string)buffer;

I'm new to Alchemy, but in general there is no guarantee what values are in memory that you malloc or where they are aligned relative to all the other memory in your application.

When I run your code unmodified, I get "1 2 3 4 5 6 7 8 9 26" which illustrates that the 10th line is terminated somewhere other than expected. Looking at the memory debug window, that last line is (hex): 0x31 32 33 34 35 36 37 38 39 30 FD FD FD FD AB AB AB AB AB AB AB AB EE FE EE FE 00, which is 26 characters long up to the NULL terminator. The 0xFDFDFD, 0xABABABAB and 0xFEEEFEEE are standard Microsoft "no-mans-land" memory barriers for debugging, in case you haven't seen those before. I have no idea how this memory will look in Alchemy, but it'll most likely be very different!

I don't, however, see anything in this example that illustrates your memory overwrite problem. Can you explain step by step, or point to the location in code where the memory is overwritten?

Also, I'm confused about your struct example. In the first you have an Array of Structures (AoS) and the second is a Structure of Arrays (SoA). These are clearly different. If you had 3 verts, the first example (AoS) should look like:

     (first memlocation) vvvnntt vvvnntt vvvnntt

And the second example (SoA) should look like:

     (first mem location) vvvvvvvvv

     (second mem location) nnnnnn

     (third mem location) tttttt

These two techniques are completely different. The first does a single memory allocation and crams all the data for the vertex together. SoA requires 3 allocations and groups the position, normal, and texture coordinates independant of each other. I'm not sure I understand zazzo9's surprise that these would give different results. They do allocate the same amount of memory but the memory is 1) organized differently and 2) split into different numbers of chunks.

Hopefully your issue was the missing NULL terminator!

I do love bugs in graphics code. Your inside out sphere is particularly entrancing! Is it supposed to morph out of shape after a while? If not, you've got some floating point representation problems as well!

I'd be up for checking out your code offline, via NDA if necessary.

/cm

I think I found your error.

vector<string> map(10);

this creates a vector with 10 null elements.

map.push_back(line);

this adds extra elements to the list.  It does not fill in your already-allocated null elements.

So, when you iterate over them, you are iterating over junk memory.  You are lucky it doesn't throw an exception (or perhaps unlucky, as that would show your problem).

You should use a default constructor, or else use a size of 0.

(P.S. I notice in your original post that you mentioned using the default constructors -- but the code example here is wrong.  Fix it, and it will likely work).

_divy, there are two problems here. The first is a major bug in Alchemy that you've discovered. The second is a misundersanding of STL.

Alchemy std::string Bug

Wow, this is a huge problem: ALCHEMY does not handle std::string copy constructors correctly! One of the basic underlying premises of all STL classes is that data is copied, and since std::string in Alchemy isn't working correctly, you should expect any container that uses std::string to fail. I've been playing around all morning with different scenarios and I'm getting crazy results. My basic suspicion is that the underlying buffers are being shared between std::string instances, and that there are some optimizations for identical strings. .NET does something like this as well. If you have two strings "abcd" then there isn't any reason that they can't point at the same memory location. When one of the strings is changed, though, it should get new memory and point to a different location. Something deep in std::string isn't working right.

Here is a small test showing the underlying problem:

string s1;

string s2;

s1 = "a";

s2 = s1; // copy constructor

s1 = "b";

// use your favorite TRACE function here

TRACE(s1); // expected: "b", actual: "b"

TRACE(s2); // expected: "a", actual: "b", ERROR

When compiled with alc-off and standard g++, this routine will print "b, a" as expected. When compiled with alc-on under Alchemy it outputs "b, b" showing that the two strings are sharing an underlying buffer, which is incorrect.

I've tried a bunch of workarounds, and I can't seem to get anything to work consistently.

First I tried using std::string::substr() to create a forced copy, which worked for my simple test example but failed to fix _divy's problem. Even worse, values set in my "a,b" test function were propogating out into the parent function, causing the output from _divy's code to start with "b,b,b"!

I tried copying the data into temporary char buffer before creating a new string but the output from walking the vector<string> still showed 3x "b" and 7x "1234567890".

Since I couldn't come up with any workaround, I dug into the Alchemy version of std::string in "ALCHEMY/avm2-libc/include/c++/3.4/bits/basic_string.h" but didn't see anything immediately that could cause the problem.

Besides that, there is one other issue with the code.

Repetitive Constructor

You need to use the default constructor/reserve technique. The parameterized constructor std::vector<X>(int N) is called a "repetitive construtor" which creates N instances of the default object (which is why there are no crashes). You have this problem with your map and rawmaps variables. Using reserve is the right way to preallocate the space for your memory.

This mistake results in an error further down in your code when you try to trace the rawmaps values. You're accessing rawmaps[0] which is actually a default constructed vector<string> which has no elements. I'm surprised that your code printed anything at all. When I ran it I only got a blank line. I'm guessing that when you ran it you actually used rawmap[1] instead?

Recommendations / Options

1. Do NOT use std::string with Alchemy (yes this sucks for a port project)

2. Find a replacement string class (maybe try boost? I'm not sure if that will work with Alchemy or not, and it doesn't look like BOOST has a replacement string class, only a string algorithms library)

3. Code your own string class and remove all references to <string> and "using std::string", replacing them with your own string class.

4. Get Adobe to fix the std::string class

All these options are nasty and major changes, but you really don't have an option. If you can get Adobe to fix this, let me know!

References

Description of the Repetitive Constructor (see the second ctor)

http://www.cplusplus.com/reference/stl/vector/vector/

BOOST (probably not helpful)

http://www.boost.org/doc/libs/1_40_0/libs/libraries.htm

Is this your blog? Describes this exact problem

http://www.peternitsch.net/blog/?p=445

I posted a quick workaround to std::string here:

http://forums.adobe.com/thread/487570?tstart=0

I tried putting this into your example but encountered another bug. Since I was using my new class cod3monk3y::string instead of std::string I couldn't use std::sstream because it requires std::stream. So either I could write a replacement for std::sstream that uses my string, or I could switch from using FILE to using std::ifstream. I chose the latter, and ran into two new problems. Basically, std::ifstream doesn't work either. Here is my code rewritten to use cod3monk3y::string and std::ifstream, with comments where the errors occur:

#include <vector>
#include <iostream>
#include <fstream>
#include "cod3monk3y/string.h"
#include "stdincl.h"
#include "AS3.h"

using cod3monk3y::string;
using std::vector;
using std::ifstream;
using std::ios;

AS3_Val loadmaps (void *data, AS3_Val args) {

     char * fileName;
     AS3_ArrayValue(args, "StrType", &fileName);

     TRACE("loadmaps(%s)", fileName);

     std::ifstream mapfile(fileName);

     if(!mapfile.is_open()) {
         TRACE("ERROR: map file is NOT open!");
         return AS3_Int(-1);
     }
     TRACE("Map file opened successfully.");

     // Trying to get the length of the file throws an error:
     // Error #1006: value is not a function.
     // at: basic_filebuf::char_traits::seekoff()
#if 0
     mapfile.seekg(0,ios::end);
     int length = mapfile.tellg();
     mapfile.seekg(0,ios::beg);
     TRACE("File length is: %d bytes", length);
#endif

     // These were using the repetitive constructor.
     // The proper way to reserve memory is to use:
     //     vector<>::reserve(int)
     vector< vector<string> > rawmaps;
     vector< string > map;
     rawmaps.reserve(1);
     map.reserve(10);

     int linenum = 0;
     AS3_Trace(AS3_String( "\nline by line:" ));
     while (mapfile.good()) {
        char buffer[256];

        // ERROR: RangeError: Error #1125: The index 1092156 is out of range 721.
        // at basic_filebuf::char_traits::underflow::work()
        mapfile.getline( buffer, 256, '\n');

        // This is using cod3monk3y::string
        string line(buffer);
        map.push_back(line);

        // Show the string details line by line
        TRACE("Line %d: len=%d c_str=%s", linenum++, line.length(), line.c_str());
     }

     // Trace maps values
     TRACE( "\nmaps values:" );
     for (unsigned int i = 0; i < map.size(); i++) {
         string& s = map[i];
         TRACE("len=%d c_str=%s", s.length(), s.c_str());
     }

     // Store map vector in rawmaps vector, which should
     // store a copy of the current vector including copies
     // of all strings.
     rawmaps.push_back(map);

     // Trace all dimensions of the vector<vector<>>
     TRACE( "\nrawmaps values:" );
     for (unsigned int i = 0; i < rawmaps.size(); i++) {
         TRACE("[i = %d]", i);
         vector<string>& v = rawmaps[i];

         for (unsigned int j = 0; j<v.size(); j++) {
             TRACE("[%d,%d] len=%d c_str=[%s]", i, j, v[i].length(), v[i].c_str());
         }
     }

     return AS3_Int(0);
}