---
layout: post
date: 2025-04-08
link: https://adactio.com/journal/21831
title: And no, a robots txt file doesn’t help
seo_title: "And No, a robots.txt File Does Not Stop AI Crawlers"
seo_description: "Linking Jeremy Keith: if you think AI crawlers respect robots.txt, you are several assumptions of good faith removed from reality."
type: linked
cited: Jeremy Keith
---

> If you think these crawlers respect robots.txt then you are several assumptions of good faith removed from reality. These bots crawl everything they can find, robots.txt be damned.