<P><code>robots.txt</code> provides a means to request that robots limit their
activities at the site, or more often than not, to leave the site alone.</P>
-<P>When the first robots were developed, they had a bad reputation for sending hundreds/thousands of requests to each site, often resulting in the site being overloaded. Things have improved dramatically since then, thanks to <A HREF="http://info.webcrawler.com/mak/projects/robots/guidelines.html"> Guidelines for Robot Writers</A>, but even so, some robots may <A HREF="http://www.zyzzyva.com/robots/alert/">exhibit unfriendly behavior</A> which the webmaster isn't willing to tolerate, and will want to stop.</P>
+<P>When the first robots were developed, they had a bad reputation for
+sending hundreds/thousands of requests to each site, often resulting
+in the site being overloaded. Things have improved dramatically since
+then, thanks to <A
+HREF="http://info.webcrawler.com/mak/projects/robots/guidelines.html">
+Guidelines for Robot Writers</A>, but even so, some robots may exhibit
+unfriendly behavior which the webmaster isn't willing to tolerate, and
+will want to stop.</P>
<P>Another reason some webmasters want to block access to robots, is to
stop them indexing dynamic information. Many search engines will use the