Quick answer
robots.txt blocks crawling, not indexing.
robots.txt vs noindex
robots.txt blocks crawling, not indexing. Use noindex meta or header to prevent indexing.
Common causes
- Expecting Disallow to remove from index.
- Confusion.
How to fix
- Disallow = do not crawl.
- noindex = do not index (meta or header).
FAQ
- Disallow removes from index?
- No; blocks crawl.
- How prevent index?
- noindex meta or header.
Fix it now
Try in validator (prefill this example)