Custom robot text how to use.

 Robot text like means each command explained.

User-agent: *

Disallow: /search

Disallow: /archives

Disallow: /feeds

Disallow: /comments

Disallow: /about

Disallow: /contact

Disallow: /privacy-policy

Disallow: /terms-of-service

Disallow: /disclaimer

Sitemap: http://yourblogurl.com/sitemap.xml

Here's what each line means:

User-agent: *: This line specifies that the following directives apply to all robots that crawl your site.

Disallow: /search: This line tells search engine bots not to crawl the search results pages on your blog. This is because these pages often contain duplicate content and may not be relevant to searchers.

Disallow: /archives: This line tells search engine bots not to crawl your archives pages, which can also contain duplicate content.

Disallow: /feeds: This line tells search engine bots not to crawl your RSS feed pages.

Disallow: /comments: This line tells search engine bots not to crawl your blog comments pages.

Disallow: /about: This line tells search engine bots not to crawl your About page, as it may not be relevant to searchers.

Disallow: /contact: This line tells search engine bots not to crawl your Contact page, as it may not be relevant to searchers.

Disallow: /privacy-policy: This line tells search engine bots not to crawl your Privacy Policy page, as it is usually not relevant to searchers.

Disallow: /terms-of-service: This line tells search engine bots not to crawl your Terms of Service page, as it is usually not relevant to searchers.

Disallow: /disclaimer: This line tells search engine bots not to crawl your Disclaimer page, as it is usually not relevant to searchers.

Sitemap: http://yourblogurl.com/sitemap.xml: This line specifies the location of your blog's sitemap, which is used by search engines to discover and index all of the pages on your site.

By adding this robots.txt file to your Blogger blog, you can help ensure that your site is crawled and indexed properly by search engines, which can ultimately help improve your search rankings and drive more traffic to your blog.

Lton2023