Quick answer
* in path matches any sequence.
robots.txt Wildcard
* in path matches any sequence. Wrong use can over-block or under-block.
Common causes
- Disallow: * (blocks all).
- Overly broad pattern.
How to fix
- Use * in path, e.g. Disallow: /api/*.
- Test with robots.txt tester.
FAQ
- * in path?
- Matches any sequence.
- Disallow: *?
- Often blocks all.
Fix it now
Try in validator (prefill this example)