User-agent: * Crawl-delay: 2 Disallow: /admin/ Disallow: /cv/ Disallow: /cvtxt/ Disallow: /my/ Disallow: /portal/ # RT 6440 - an employer wanted their old vacancies removing Disallow: /job/37594/ Disallow: /job/35459/ Disallow: /job/19687/