Google Git
Sign in
gerrit / gerrit / 3a5a131bf97ee99685d45a83dd537ad9866c020e / . / webapp / robots.txt
blob: c0339176af2d1ca1b3931eb9ffb4fe492a365328 [file] [log] [blame]
Shawn O. Pearce2c2e1722009-01-15 08:19:40 -0800[diff] [blame]1# Directions for web crawlers.
2# See http://www.robotstxt.org/wc/norobots.html.
3
4User-agent: HTTrack
5User-agent: puf
6User-agent: MSIECrawler
7User-agent: Nutch
8Disallow: /