# Robots.txt file from http://www.searchengineworld.com # # All robots will spider the domain User-agent: * Sitemap: http://www.holidayex.com/sitemap.xml Disallow: /banners/ Disallow: /images/ Disallow: /includes/ Disallow: /thumbnails/ Disallow: /upload/ User-agent: googlebot Crawl-delay: 10 User-agent: Googlebot-Image Crawl-delay: 10 User-agent: msnbot Crawl-delay: 10 User-agent: Slurp Crawl-delay: 10 User-agent: fast Crawl-delay: 10 User-agent: teomaagent1 Crawl-delay: 10 User-agent: lycos Crawl-delay: 10