robots.txt

Path: samples/railsapp/public/robots.txt
Last Update: Mon Apr 21 19:36:15 -0600 2008

See www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file

To ban all spiders from the entire site uncomment the next two lines: User-Agent: * Disallow: /

[Validate]