Question: Do all search engines require a robots.txt file?
Answer: Technically, no. A search engine will not ignore your website if you do not have a robots.txt file, however, the robots.txt should still be considered mandatory.
Even though many robots ignore the robots.txt command file it is always better to have a robots.txt file than to not have one.
Major search engines (including Google, Yahoo, MSN, Ask Jeeves) will always look for a robots.txt file on your website. And, having a robots.txt may help your site from being attacked or flooded by malicious robots.
