FlossieAmbriz430

Several men and women believe that by employing a robots.txt file on their web site they are defending pages and folders from thieves and hackers. In reality it is totally the opposite! robots.txt opens up an huge security hole that hackers and theives will use to effortlessly obtain access to the parts of your website that you do not want them to.

What is robots.txt?

robots.txt is a file that you develop and up...

robots.txt is possibly the most miss understood file that a web site can include.

A lot of people think that by using a robots.txt file on their site they are defending pages and folders from thieves and hackers. In fact it is absolutely the opposite! robots.txt opens up an enormous safety hole that hackers and theives will use to simply acquire access to the parts of your website that you do not want them to.

What is robots.txt?

robots.txt is a file that you create and upload to your sites root directory that is employed by search engine spiders to determine which parts of your web site they ought to index and which folders/pages that you, the website owner,

do not want listed in search engine indexes.

Why would not want pages indexed?

There are a lot of reasons why you might not want search engines to index pages on your website, such as private membership pages or exclusive instruction pages and such like.

If you are an Web Marketer selling your personal ebook or other digital item, you would not want your thank you pages indexed either!

And this is exactly where the misunderstanding comes to the fore, and robots.txt becomes your foe.

Several on-line marketers who provide ebooks or other digital items for instant download will list their download thank you pages in the robots.txt file since they certainly don't want those pages indexed in search engines.

By utilizing robots.txt this way though, you will be opening up your product to any individual who has a slight bit of expertise about how the file performs.

robots.txt is effortlessly readable by any human that opens a browser and sorts in and if you have listed your thank you pages, all they have to do is go to that url and take your product(s)!

It's that simple!

And I am living proof that this performs, as this is specifically what happened to me. I had listed my thank you pages in robots.txt and thought that they had been safe from hackers and thieves, then a single day I was checking my web internet site stats and BAM, somebody had been to every single single thank you web page, and taken anything.

The moral is, do not list any URL in robots.txt that you do not want people to have free of charge access to. Use robots.txt with great caution and secure your thank you pages using dedicated application. industrial robots types