# robots.txt for http://www.example.com/ # temporarily disallow repo crawling User-agent: * Disallow: /r User-agent: * Disallow: /r #/cyberworld/map/ # This is an infinite virtual URL space