# Massendownloader vom CGI Verzeichnis fernhalten User-agent: wget User-agent: webzip User-agent: webmirror User-agent: webcopy Disallow: / # Alexa Crawler (used for Web archive.org) User-agent: ia_archiver #Verifikationsfile fuer Google Webmaster Tools Disallow: /google283d10edf1dbd431.html Disallow: /files/ Disallow: /cgi-bin/ Disallow: /backup/ Disallow: /error/ Disallow: /inc/ Disallow: /intern/ Disallow: /pcm/ Disallow: /wordpress/ # restliche Crawler User-agent: * #Verifikationsfile fuer Google Webmaster Tools Disallow: /google283d10edf1dbd431.html Disallow: /files/ Disallow: /cgi-bin/ Disallow: /backup/ Disallow: /error/ Disallow: /inc/ Disallow: /intern/ Disallow: /pcm/ Disallow: /wordpress/