User-agent: * Disallow: /_mm/ Disallow: /_notes/ Disallow: /_baks/ Disallow: /MMWIP/ User-agent: googlebot Disallow: *.csi # Asks bots to index the site less frequently, preventing # accidental bot based DDoS. # KB Article - https://kb.wisc.edu/webhosting/page.php?id=62214 User-agent: * Crawl-delay: 10