Results 1 to 4 of 4

Thread: How best to refer to sitemap in robots.tx?

  1. #1
    Join Date
    Oct 2007
    Posts
    136

    Default How best to refer to sitemap in robots.tx?

    Hello all,

    I am interested to learn what is the best way to refer to the sitemap in the robots.txt file. Currently my robots.txt file contains:

    User-agent: *
    Disallow: /cgi-bin/
    Disallow: /admin/
    Disallow: /cp/
    Disallow: /add.php
    Disallow: /modify.php
    Disallow: /upgrade.php
    Disallow: /tell_friend.php
    Disallow: /send_pwd.php
    Disallow: /sendmail.php
    Disallow: /review.php
    Disallow: /trackclick.php
    Disallow: /bad_link.php
    Disallow: /captcha.php
    Disallow: /login.php
    Disallow: /rss.php

    Can somebody advice where and what line to include for the sitemap?

    Thanks & regards,
    Wessel

  2. #2
    Join Date
    May 2007
    Location
    NJ, United States
    Posts
    1,651

    Default

    This is how I have mine

    Code:
    User-agent: *
    Sitemap: http://www.your-domain.com/sitemap_index.xml.gz
    Disallow: /cgi-bin/
    Disallow: /admin/
    Disallow: /cp/
    Disallow: /add.php
    Disallow: /modify.php
    Disallow: /upgrade.php
    Disallow: /tell_friend.php
    Disallow: /send_pwd.php
    Disallow: /sendmail.php
    Disallow: /review.php
    Disallow: /trackclick.php
    Disallow: /bad_link.php
    Disallow: /captcha.php
    Disallow: /login.php
    Disallow: /rss.php
    FSGDAG | IndexU Hosting | Owner
    Website | NiceCoder Script Hosting and More! | Web4URL is For Sale!
    Follow Us On Twitter | FaceBook Profile | YouTube Videos

  3. #3
    Join Date
    Oct 2007
    Posts
    136

    Default

    thanks!

  4. #4
    Join Date
    May 2007
    Location
    NJ, United States
    Posts
    1,651

    Default

    Quote Originally Posted by weswa View Post
    thanks!
    Anytime!
    FSGDAG | IndexU Hosting | Owner
    Website | NiceCoder Script Hosting and More! | Web4URL is For Sale!
    Follow Us On Twitter | FaceBook Profile | YouTube Videos

Similar Threads

  1. Need Help - Robots.txt File in indexu5.4.0
    By bugsdirectory in forum v5.x
    Replies: 2
    Last Post: 12-07-2007, 01:00 PM
  2. Replies: 9
    Last Post: 10-29-2007, 05:49 PM
  3. Robots/spider counter
    By linksor.com in forum Blocks and Modification
    Replies: 1
    Last Post: 09-27-2006, 08:13 PM
  4. Spam from Refer a Friend
    By Dorian in forum v5.x
    Replies: 1
    Last Post: 08-16-2006, 01:08 AM
  5. Blocking Robots
    By barnkin in forum v5.x
    Replies: 31
    Last Post: 06-20-2006, 01:33 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •