6 Replies Latest reply on Aug 25, 2006 6:09 PM by srushing

    search engines and dynamic pages and CF

    srushing Level 1
      when I search google for a product, say a DVD player, I see that some companies have been able to get the search engines to index their product listings. In doing some research it seems that one way they've done this is to make it look like each product has a static page by replacing the query string character "?" with "/" so the URL looks like a static page when really the server is pulling dynamic content. I saw one reference to the following line in one of the articles which referred to Coldfusion:

      Reconfigure your Cold Fusion setup to replace the "?" in a query string with a '/' and pass the value to the URL. The browser interprets that as a static URL page.

      Has anyone done this? I have a product database I'd like to get indexed so in theory if I create a single page on my server that has static links to all the products, each getting called by the mocked up "static" URL, it seems like that will work. The bigger quesiton is how does one go about changing the query string in CF and does it apply to everything, meaning all the existing pages I've built on the site that pass URL variables using "?", will those have to be rebuilt now that the query string is changed?

      Thanks
      S
        • 1. Re: search engines and dynamic pages and CF
          dempster Level 1
          Despite many posts on this issue, I am skeptical that there is a problem with search engines indexing pages with query strings. I have a number of pages with URLs like showdetail.cfm?articleID=25 and they are always indexed by Google and other search engines.

          Having said that, you can use a format like this: showdetail.cfm/25

          The CGI variable PATH_INFO should return the string that follows the page name.
          • 2. Re: search engines and dynamic pages and CF
            Level 7
            Despite many posts on this issue, I am skeptical that there is a problem
            with search engines indexing pages with query strings. I have a number
            of pages with URLs like showdetail.cfm?articleID=25 and they are always
            indexed by Google and other search engines.

            It is my understanding it largely depends on how you link to the detail
            pages. If you have direct anchors to the detail pages Google will index
            them. If you use a form or something else, Google is much less likely
            to follow.

            For example:
            <a href="showdetail.cfm?articleID=25">The best red ball ever</a>
            <!--- This will be followed by Google to be indexed--->

            <form action="showdetail.cfm" method="get">
            <select name="articleID">
            <option value="25">The best red ball ever<option>
            <option value="26">The best green ball ever<option>
            <option value="27">The best blue ball ever<option>
            </select>
            <input type="submit"/>
            </form>
            <!--- This will not be followed by Google to index the detail pages. --->

            This makes some sense, because how can a robot complete a form even if
            it could determine it is for navigation.
            • 3. Re: search engines and dynamic pages and CF
              dempster Level 1
              Ian is right - the key to indexing is the presence of a link that a robot can follow. And of course the same ranking algorithms will apply, so the more relevant pages that link specifcally to your page, the better your rank.
              • 4. Re: search engines and dynamic pages and CF
                srushing Level 1
                Thanks on the responses. So here's a followup....so if I have a file that has links to all my products, that file itself will get indexed into the search engine as well, correct? Which would mean that in a search result, it's possible that my file containing all my products will come back as a result and someone could go directly to that page. So I'm not 100% sure I have a problem with that but I'm a little uneasy about someone being able to grab that page and get links to all my stuff.

                Is it possible to detect when a spider is view a page vs visitor so I could redirect the interactive visitor to my main page instead of that listing?

                Just curious...thanks
                S
                • 5. Re: search engines and dynamic pages and CF
                  Level 7
                  Is it possible to detect when a spider is view a page vs visitor so I
                  could redirect the interactive visitor to my main page instead of that
                  listing?

                  Not 100% reliably. Also it could be considered against the search
                  engines user agreements, thus you risk being black listed by them.

                  I would just make that page as useful to a human user as possible. This
                  will probably help with the search engines as well. They like to see
                  relevant copy associated with the links.


                  • 6. Re: search engines and dynamic pages and CF
                    srushing Level 1
                    THanks Ian,

                    I see your point. I don't want to do anything to turn a way the spider or get blacklisted. But am I thinking about this correctly? Somewhere they must have static links on a page that points to product X. So either they have all them listed on the pages somewhere in their site or they have bulk files not really related to the site and aren't linked to the sites pages which just contain the static links to all the products.

                    Well, now that I'm thinking about it, the pages would have to be "linked" to the site in some way for the spider to find them...correct?

                    Well, thanks for any thoughts..I'm still trying to get a handle on the best way to handle this.
                    S