# allow internet archive's crawler User-agent: ia_archiver Disallow: # allow googlebot User-agent: Googlebot Disallow: # allow yahoo bot User-agent: Slurp Disallow: # disallow all others User-agent: * Disallow: /