• Twitter
  • Facebook
  • Snapchat
  • Instagram
  • Medium

Robots.txt Generator


Default - All Robots are:  
    
Crawl-Delay:
    
Sitemap: (leave blank if you don't have) 
     
Search Robots: Google
  Google Image
  Google Mobile
  MSN Search
  Yahoo
  Yahoo MM
  Yahoo Blogs
  Ask/Teoma
  GigaBlast
  DMOZ Checker
  Nutch
  Alexa/Wayback
  Baidu
  Naver
  MSN PicSearch
   
Restricted Directories: The path is relative to root and must contain a trailing slash "/"
 
 
 
 
 
 
   



Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.


gajabwap Tools

Tools

  • Alexa rank
  • Css Minify
  • js Minify
  • Job exp calculator
  • Email Sender
  • zip converter
  • unzip conveter
  • password genrator
  • Vimeo thumbnail
  • whois checker
  • Word Counter
  • youtube thumbnail
  • instagram
  • youtube Tag Finder
  • password strength
  • Source code viewer
  • youtube id Finder
  • text 2 voice
  • my ip address
  • Dns Checker
  • Html Encoder
  • Html decoder
  • Url Encoder/Decoder
  • Gst calculator
  • Http header Finder
  • Htaccess Genator
  • Disclaimer page genrator
  • Privacy page genrator
  • Domain 2 Ip
  • image resize
  • Meta tag analyzer
  • Meta tag genrator
  • Image formate convert
  • Website available checker
  • Percent Calculator
  • rebots.txt genator
  • iframe genrator
  • Keyword Density

© 2020 Online Tools