seo

One of the things I?d been meaning to do for some time was to get round to creating a robots.txt file for this site. I?d looked at various sources of information including A Standard For Robot Exclusion and How to create a robots.txt file but was still unsure as to which bits should be excluded...