0 Replies Latest reply on Jun 15, 2008 1:48 PM by future-architect

    dont search my site

    future-architect Level 1
      i am using this site ( http://www.mcanerin.com/EN/search-engine/robots-txt.asp) so the search engines do not search particular sites. i got this code;

      # robots.txt
      User-agent: Googlebot
      Disallow: /
      User-agent: Googlebot-Image
      Disallow: /
      User-agent: MSNBot
      Disallow: /
      User-agent: Slurp
      Disallow: /
      User-agent: Teoma
      Disallow: /
      User-agent: Gigabot
      Disallow: /
      User-agent: Scrubby
      Disallow: /
      User-agent: Robozilla
      Disallow: /
      User-agent: Nutch
      Disallow: /
      User-agent: ia_archiver
      Disallow: /
      User-agent: baiduspider
      Disallow: /
      User-agent: yahoo-mmcrawler
      Disallow: /
      User-agent: psbot
      Disallow: /
      User-agent: asterias
      Disallow: /
      User-agent: yahoo-blogs/v3.9
      Disallow: /
      User-agent: *
      Disallow: /
      Disallow: /login_required.html
      Disallow: newsletter_bye.html
      Disallow: newsletter_thank_you.html
      Disallow: service_unavailable.html
      Disallow: flash_not_found.html

      and was wondering if it works, and is correct. i saved it as robots.txt, and uploaded it. if this method does not work, or there is another way, please help me. thanks!