3 Replies Latest reply on Oct 7, 2007 3:21 PM by Stressed_Simon

    Need ideas Please

    Level 7
      As you can see from my past postings I am working on a reality site and it
      needs to download files everyday and use them so people can search against
      them for homes. There are 5 total files per day. (Residential, Vacant Land,
      Condos, Photos, and MultiFamily). Monday - Saturday are updates and are
      relatively small. (2meg or less). Sunday, however, is a complete new set of
      listings and replaces the prior listings. So, data wise, Sunday is the
      beginning and the main listings and Monday - Saturday are updates. The
      Sunday residential is averaging about 54 meg and is causing the web site to
      timeout while looping and inserting into the database. Everything works fine
      Mon-Sat, but crashes and burns on Sunday. No rest for the developer.

      Originally I wanted to loop and insert due to the client entering his own
      private listings into the same tables, but since the specs have changed and
      he has is own for each of the 5 types. The people will only be able to
      search the MLS Listings (the 5 files I download).

      Any ideas on how any of this can work. I need alternate ideas.

      One idea I have is to only download the files and query against them as a
      flat database. Can I UNION the results of 5 residential files into one
      output? Wouldn't that take a query of queries? Can q of q do a UNION?

      If possible, I can download and store the 35 total files (7 days of 5 files)
      and then just set a function to delete them on Sunday prior to the new files
      downloading.

      Does that sound feasible? Is there a better way?

      --
      Wally Kolcz
      MyNextPet.org
      Founder / Developer
      586.871.4126