Saving objects data to txt

[Guest]

Hi

I'm populating a datagGrid with external csv data, by loading it, converting it to an object, then loading into the data provider.

Would I be better off (and is it possible) to not use csv, but objects instead?

So have in a txt file...

{id:128, name:"Moi"},{id:156, name:"You"} // my actual objects have about 10 keys each

...load the txt file, split it at the commas into an array of objects, and use that as the dataprovider?

Or do you have to serialize from dataprovider -> objects -> csv (or xml), then csv -> objects -> dataprovider?

I have a feeling AS3 has a max string size it'll load, so maybe that would kill things.

Cheers for taking a look.

[kglad · Accepted solution]

you could use json or xml in your text file and directly assign the loaded string to your dp.

[sinious]

Max string length should be very tolerably long, but consider any multi-byte characters and the ramping up of urlencoded values using up multiple characters. The string length I recall is 2^30 - 1 (or 1,073,741,823 bytes). I think you'd hit your network transfer timeout before transferring a string that big . Another vote for JSON here as well and your CSV is eerily already similar to it, like a faux JSON. 

[Guest]

Thanks you both.  I'll load the latest sdk so I can try the new JSON class.

[kglad]

you're welcome.

[Guest]

I'm not getting the result I want, and was wondering if someone could explain the basic process?

Say I have a txt file with this:

{ "firstName":"John", "lastName":"Parker", "age":"32", "country":"Canada", "job":"Programmer" },

{ "firstName":"Peter", "lastName":"Anderson", "age":"30", "country":"USA", "job":"System administrator" },

{ "firstName":"Bob", "lastName":"Johnson", "age":"35", "country":"Canada", "job":"Coder" }

My aim  is to load it into a datagrid, edit it, then save it back to the file.

I've browsed and selected the txt file, and can successfully trace the above data with...

trace(myLoader.data);

How do I get that into a dataProvider?

I was successfully able to import the following data using var myData:Object = JSON.parse(myLoader.data); but not the above  multiple object dataset:

{

   "firstName": "Chris",

   "lastName": "Griffith",

   "education":

   {

      "elementary": "Franklin",

      "jrhighSchool": "Curran",

      "highSchool": "Bakersfield High",

      "college": "University of California, Santa Barbara"

   }

}

Do you have to do something different to store multiple objects?

Thanks again guys

[sinious]

What are you using to parse the original data into a CSV? Is it a server-side script? If so, languages have native JSON parsers or parser libs (like PHP, ASP, etc) that can take the original object you're creating the CSV from, convert it to JSON and transmit that directly to Flash, who can then just JSON.parse() it.

If you're exporting this from a spreadsheet or application manually then you will need to program the first step, which is to parse the CSV into an object. In that situation unfortunately there's no benefits to converting it from what you have to JSON or any other format.

Are you getting the CSV data from a server script?

[Guest]

It's for an AIR desktop app - so no server side.

Originally I was using a csv class to save the datagrid's contents into a .txt file (just values), and when loading the txt file back into the datagrid, was using a for loop to build an object and add keys to the values.  But I figured there was a better way.  That's when I posted my first question above, and you guys gave thumbs up for JSON, which I've been playing with.

So nothing is set in stone.

I have since managed to load JSON data in, but with a problem.  Here's the data:

{"songs":[{"Title":"Bad Influence","Artist":"Pink","DiscID":"EMI8988"},{"Title":"Bad Influence","Artist":"Pink","DiscID":"Sony2002"}]}

After loading it from txt, I successfuly use:

var myData:Object = JSON.parse(myLoader.data);

trace(myData.songs[0].Title);

...to access the data.

The problem is that when I get up over 2500-ish objects (I tested adding 1000s of copies of {"Title":"Bad Influence","Artist":"Pink","DiscID":"EMI8988"}, to the "songs" array in the txt file), JSON throws the error: SyntaxError: Error #1132: Invalid JSON parse input.

Using the csv method was a bit more work, but it handles large datasets fine.

I'll be working with datasets that will result in up to 150 000 rows, so I need to get the JSON parser working with larger datasets.

Any ideas why the error is thrown with larger sets?

Thanks for your reply.

[sinious]

You have some other error, probably a trailing comma on your last array entry. Make sure you didn't do:

[{obj},{obj},] // error trailing comma

It needs to end:

[{obj},{obj}] // no trailing comma

A common mistake you probably made. Here's a quick code snippit this 5 year old 4GB ram laptop has no issue with:

import flash.utils.getTimer;

// start time

var startTime:int = getTimer();

// make ridiculos 1 MILLION length object

var str:String = '{"songs":[';

for (var i:int = 0; i < 1000000; i++)

{

          if (i > 0) { str += ','; }

          str += '{"Title":"Bad Influence","Artist":"Pink","DiscID":"EMI8988"}';

}

str += ']}';

var myData:Object = JSON.parse(str);

trace(myData.songs[999999].Title);

trace("Total time: " + (getTimer() - startTime) + "ms");

My trace:

Attempting to launch and connect to Player using URL C:\Users\      \Desktop\Lab\JSONParse\LargeJSON.swf

[SWF] C:\Users\        \Desktop\Lab\JSONParse\LargeJSON.swf - 3052 bytes after decompression

Bad Influence

Total time: 6173ms

[UnloadSWF] C:\Users\       \Desktop\Lab\junk\JSONParse\LargeJSON.swf

Debug session terminated.

Yes, it took 6 seconds (on an old laptop), but it just did 1 million complex string ops. There's no reason you should hit any wall anywhere near 2500. Or 25000. Or 150,000. And actually it's just my non-redundant nature that caused over an extra second of time because I'm appending "str" twice when I could have used the more verbose:

if (i == 0)

{

          str += '{"Title":"Bad Influence","Artist":"Pink","DiscID":"EMI8988"}';

}

else

{

          str += ',{"Title":"Bad Influence","Artist":"Pink","DiscID":"EMI8988"}';

}

To reduce to a single "str" append at 4992ms.

You get the idea..

Here's 2 million reading 1999999:

Bad Influence

Total time: 13680ms

After that things get shaky but if you're dealing with 2 million rows, you better be dealing with a database. And a good database at that.

[Guest]

Ok - first THANK YOU for that piece of code - it is VERY helpful.  And it works.

However, it's bugging me that my old txt file isn't working.  There's no trailing comma.  Under an amount of records (unsure as to exactly how many) it's fine, above, not.

I have uploaded a copy of the txt file if you are curious to take a look at http://www.billygoatkaraoke.com.au/test.txt

I can't see anything screwy with it.

[sinious]

Search for this in your text: "Ar [... text truncated]"

Not sure what's causing that but check it out. That's killing it.

edit:

I'm away for a bit today but ultimately it's just your test data that's causing an issue. A DataGrid's dataProvider will be a valid object to JSON.stringify() so you need not worry about some 'testing' parse issue. You know you have enough breathing room with the examples above. Just convert it directly to JSON and reload and parse that JSON and you should be fine. Doesn't get easier!

[Guest]

hehehe - I know that what you have coded for me will do the job, but it's bugging me why that other string is screwing up.  My app also has an import function that allows users to import a csv file into the dataGrid too, to be exported as json, and since I've not dealt with the native json in as3, so I just want to understand why something screws up so as to make it all watertight.

I'm not sure what you mean by "Ar [... text truncated]"

Thank you for your thorough reply.

[sinious]

I mean in the link you posted:

http://www.billygoatkaraoke.com.au/test.txt

Search that page for the string "Ar [... text truncated]" (without the quotes). That's actually in your own text (and invalid JSON) which is why it's failing to parse. No worries it's just a demo and you know the basic limits of JSON (or lack reasonably thereof).

You're welcome and good luck!

[Guest]

Hi guys

It doesn't seem to be the case that dataProviders are valid for stringifying.

trace(JSON.stringify(myDataProvider)); displays {"length":3}.

Is there something else I need to do to the dp before stringifying it, or should it just translate?

I can successfully load the following from a txt file:

[{"Firstname":"Billy","Lastname":"Smith"},{"Firstname":"Jill","Lastname":"Thomson"},{"Firstname":"John","Lastname":"Jiminy"}]

...then...

myData = JSON.parse(event.target.data);

listDP = new DataProvider(myData);

data_grid.dataProvider = listDP;

This successfully populates the datagrid.

But stringifying listDP, or putting it into an object and stringifying the object like var person2:Object = {holder:songListDP} displays {"length":3} and {holder:"length":3} respectively.

[sinious]

Yes by default the dataProvider only has a few properties so you need to use a DataProvider method to get the data properly, like toArray(). Then you need to encode that in JSON. Afterwards when you parse it you'll need to turn that object (the type 'parse' returns) back into an array.

Basically like this (in its simplest form):

// convert datagrid to JSON

var myDGJSON:String = JSON.stringify(myDataGrid.dataProvider.toArray());

// save that string..

// on loading, read string, then parse into object

var dataObj:Object = JSON.parse(myDGJSON);

// convert the object back into an array

var arrayObjs:Array = new Array();

for each (var curValue in dataObj)

{

          arrayObjs.push(curValue);

}

// assign this array back to the dataProvider

myDataGrid.dataProvider = new DataProvider(arrayObjs);

I did it this way because you have a chance to deal with complex data in the loop, if needed (before stepping into even more advanced "onJSON/reviver" functionality).

You can create your own DataProvider object and use the addItems method to directly add the parsed object to save some lines of code if you don't need this:

// convert datagrid to JSON

var myDGJSON:String = JSON.stringify(myDataGrid.dataProvider.toArray());

// save that string..

// ...

// ..load string

// on loading, read string, then parse into object

var dataObj:Object = JSON.parse(myDGJSON);

// new DataProvider object

var dp:DataProvider = new DataProvider();

dp.addItems(JSON.parse(myDGJSON));

// assign this array back to the dataProvider

myDataGrid.dataProvider = dp;

[Guest]

toArray was the missing piece!!!!

There isn't much info on the net about dealing with arrays and stringify - I've been pulling my hair out all arvo to the point I experimented with byteArrays as an option for a while.

Thanks you Sinious - it's almost 3am and because of toArray I will sleep!!!!!!!

[sinious]

You're welcome and good luck!

[Guest]

While on topic, do you know of any reason why a Cannot access a property or method of a null object reference gets thrown here...

open txt file, it loads into datagrid fine

open another, same thing

save one

open another - all good.

However, when the DG event listener listevent ITEM_CLICK picks up on activity, then I try to open a txt file, the error gets thrown.

I trimmed the function that gets called to the point where it's empty:

private function data_gridClicked(event:ListEvent):void

  {

  }

...but the error still gets called.

And after the error gets called, the txt file sucessfully loads into the DG.

Anything jump to mind?  I had this problem earlier in the day when attempting opening files after having clicked in the DG, couldn't solve it, so concntrated on the other problem.

[sinious]

Have you populated the DataGrid before you assigned that listener? Otherwise when you click there won't be any ListItem to click and might return a null. You should only be assigning that listener after the grid is populated. That's about all that comes to mind.

[Guest]

Yes re the listener.  Clicking the grid doesn't seem to be a problem - it's opening a new list after having clicked in the grid that is making things screwy. I've added listener removal and adding just in case.

The full error is:

TypeError: Error #1009: Cannot access a property or method of a null object reference.

at fl.controls::TextInput/setFocus()

at fl.managers::FocusManager/activateHandler()

I've added trace points:

private function openSongList(event:MouseEvent):void

  {  

   data_grid.removeEventListener(ListEvent.ITEM_CLICK, data_gridClicked);

   data_grid.addEventListener(ListEvent.ITEM_CLICK, data_gridClicked);

   var fileFilter:FileFilter = new FileFilter("Files","*.txt;");

   var listFile:File = new File(); ;//main datagrid song list file

   listFile.addEventListener(Event.SELECT, loadlist,false,0,true);

   listFile.addEventListener(Event.CANCEL, cancellist,false,0,true);

   listFile.browseForOpen("Select a .txt file to import...",[fileFilter]);

   trace('here1');

  }

private function loadlist (event:Event) {

   trace('here2');

   var myLoader = new URLLoader();
   myLoader.addEventListener(Event.COMPLETE, openListComplete);
   myLoader.load(new URLRequest(event.target.nativePath)); 
  }

It traces here1, and then the error, and doesn't trace here2.  the error occurs when I double click the new file to open (or click Open).

Anything look out of place there?

[sinious]

Nothing besides unnecessary double semicolons on "var listFile:File = new File();;" but that shouldn't really kill it (althought it's not proper). Perhaps needing a cast of event.target.nativePath to File(event.target).nativePath. Outside that I see nothing to really note. But if that were the case you'd see "here2".

Looks like something before the code you're showing me. If you're using Flash Pro, open publish settings for SWF, check off to permit debugging, then use CTRL+SHIFT+ENTER to launch in debug mode. Find the line of code that this error is occurring from.

[Guest]

Thanks Sinious.  I haven't had a chance to apply your answer - been practising OOP design today.  But thank you.

BTW, to solve the problem of overlaying the latest Flex sdk over CS5 I had the other day, I ended up buying a cloud subscription to CS6, and added the sdk in about two clicks.  Ahhhhhhhhh.  Very nice.

re your answer - I did have a problem yesterday when trying to debug before posting the question.  The debug panels come up, but the swf doesn't appear.  I set permit debugging on, but still nothing.  Is there something else in CS6 I need to set?

[sinious]

Check for errors. You'd really have to have an early-on error for the debugger not to come up. Also make sure if you update your AIR SDK that you update your Flash Player/Debugger executables. I still use Flash CS5.5 (because I only use it to create graphics and code in Flash Builder) but overlaying AIR is only one part of Flash. You also need to update Flash Player (playerglobal.swc, plugin executables, standalone projector and debugger projector). The playerglobal.swc comes in AIR but it's good to update FP as well. 

Here's all the Flash Player updates:

http://www.adobe.com/support/flashplayer/downloads.html

Be sure to install those debug versions of the Flash Player plugin (ActiveX (IE), Plugin (All other browsers)).

The locations in CS5.5 (check locations for CS6) for Flash Player/Debugger are:

Standalone:

C:\Program Files (x86)\Adobe\Adobe Flash CS5.5\Players\Release

Debugger:

C:\Program Files (x86)\Adobe\Adobe Flash CS5.5\Players\Debug

playerglobal.swc:

C:\Program Files (x86)\Adobe\Adobe Flash CS5.5\Common\Configuration\ActionScript 3.0\FP11.5

FlashPlayer11_5.xml:

C:\Program Files (x86)\Adobe\Adobe Flash CS5.5\Common\Configuration\Players

Assuming you updated the other AIR files during the AIR install (SWF version should be 18 for 11.5).

Season to taste for CS6, I'm not sure if the paths changed much. They may have. Flash Builder 4.6 Premium changed paths in eclipse when updated to 4.7 premium, so watch out.