Using the robots.txt file websites can specify that certain files or directories, search engines not to index, but this may, however, attackers highlight interesting information. Using robots.txt as reconnaissance attacks has long been known. Still, there are plenty of websites that leave all sorts of interesting information in this file, it discovered a French researcher.
He analyzed nearly 60,000 websites, of which 35,000 appeared to contain a valid robots.txt file. The text files, he discovered many interesting things, such as a comment to photographs of stalked students not to index, with a link to an image was recorded. He also found all kinds of admin and manager directories that were kept hidden, as well as all kinds of Word documents.
The researcher concludes that the use of robots.txt is not inconsequential. "In the simplest case reveals the shielded paths and the technology that your servers. But can you find content with further research that should not stand." Administrators also should not assume that robots.txt is read only by robots and crawlers he notes.
No comments:
Post a Comment