- To temporarily redirect users from one URL to another
- To prevent search engine crawlers from indexing a page
- To resolve duplicate content issues
- To permanently redirect users from one URL to another
When should you use a robots.txt file?
When you have multiple versions of a page to indicate the preferred version When your website receives a penalty from Google When you have pages
