Robots.txt Generator

Free SEO & Web Tools

Robots.txt Generator

Default - All Robots are:  
Sitemap: (leave blank if you don't have) 
Search Robots: Google
  Google Image
  Google Mobile
  MSN Search
  Yahoo MM
  Yahoo Blogs
  DMOZ Checker
  MSN PicSearch
Restricted Directories: The path is relative to root and must contain a trailing slash "/"

Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.

About Robots.txt Generator

Now, make a file called "robots.txt" in the root directory of your site. Simply cut and paste the preceding content into the document.

The Robots.txt Generator makes a file that is the opposite of the site map and lists the pages that will be included. The robots.txt syntax is therefore very important for any website. A robots.txt file, located at the root of the domain, instructs search engines how to crawl the site. When a file is found, the tracker will read it to find out which files and directories may be locked.