7 Replies Latest reply on Apr 2, 2008 5:15 AM by chouervos

    Data from external text files

    chouervos
      I want to convert numerical data from external text files to values.

      I am developing a project that reads volumetric data files, which are text files containing a large ammount (e.g. 500,000) of values (float numbers).
      I read the files with FileIO but I get them as a string.
      I actualy need to sort this data and have fast access to them, which means I should convert them to values and put them in a list.

      Since the data are very large, repeat loops consume a lot of time (believe me!).

      Is there any other way inport the data from external text files as values and not as a string?
      Is there any way to convert a large string of thousants of numbers to values?

      Any idea would be very (very) helpful.

      thanks
      nick
        • 1. Re: Data from external text files
          Level 7

          "chouervos" <webforumsuser@macromedia.com> wrote in message
          news:fsdhpe$h0i$1@forums.macromedia.com...
          >I want to convert numerical data from external text files to values.
          >
          > I am developing a project that reads volumetric data files, which
          > are text
          > files containing a large ammount (e.g. 500,000) of values (float
          > numbers).
          > I read the files with FileIO but I get them as a string.
          > I actualy need to sort this data and have fast access to them, which
          > means I
          > should convert them to values and put them in a list.
          >
          > Since the data are very large, repeat loops consume a lot of time
          > (believe
          > me!).
          >
          > Is there any other way inport the data from external text files as
          > values and
          > not as a string?
          > Is there any way to convert a large string of thousants of numbers
          > to values?
          >
          > Any idea would be very (very) helpful.
          >
          > thanks
          > nick
          >

          Hi Nick,
          you could put the data into a database, and read it from there.
          There are many Xtras that can do that.

          If your data is in csv format, and you are on Windows, you could try
          ODBC to read the data.

          Richard.


          • 2. Re: Data from external text files
            Wolfgang Herold Level 2
            if you allready read it in , line by line with FileIO, you can simply convert it with the VALUE command. the delimiter for floats must be "." .

            eg.
            put ("1234.44").value
            result:
            -- 1234.4400

            As of sorting and similar functionallity is not really the strongest part of Director, i also suggest to implement this with a database.

            Wolfgang
            • 3. Re: Data from external text files
              chouervos Level 1
              Thank you for your replies

              I will try a database xtra to read my data.
              Since my data files are tab delimited text files it yould be very helpfull if you could suggest any xtra to to the job
              • 4. Re: Data from external text files
                Level 7
                Could you have a pre-processor program to read in the data? Then you could
                parse it into a list and save that list with the PropSave xtra
                (www.pimz.com). For the parsing, I suggest using the PregEx xtra
                ( http://openxtras.org/pregex/ - the PC one is inside the .hqx file).

                Then, in the main program, read in the list with PropSave. It will be much
                faster that way.

                If you aren't using any Director-specific features, you are likely to find
                that any other language will be vastly faster, e.g. Perl or VB.

                Andrew


                • 5. Re: Data from external text files
                  chouervos Level 1
                  Thank you Andrew!

                  Your solution seems much more suitable

                  I will try the PregEx xtra to parse the data

                  Actually I need these data to visualize 3D vectors and 3D surfaces in Director to achieve customized scientific visualizations. I use Director to develop educational software ( http://www.molwave.com/software/3dmolsym/3dmolsym.htm ) and now I try to use it for research purposes in the field of molecular visualization.

                  • 6. Re: Data from external text files
                    Production Monkey Level 3
                    It would be much simpler and faster to use the vList xtra to read/write your list. You probably can even read/write a list of Vectors. It's worth your time to try it out.

                    http://xtras.tabuleiro.com
                    • 7. Re: Data from external text files
                      chouervos Level 1
                      Thank you Andrew!

                      PregEx xtra is fantastic!

                      I used it to parse data strings containing up to 1,594,323 values and convert them to lingo lists very fast!
                      This could not be achieved with lingo
                      Now my project can be developed in Director with PregEx xtra

                      Thank you again

                      nick