In my server logs (Apache 2.2.3), I get requests from the useragent "Mozilla/4.0 (compatible;)". Some quick googling seems to suggest that these requests are from proxies that are trying to prefetch pages, but the information is not very conclusive.
Does anyone have experience with these? Do you know where those requests really come from? Is it safe to ignore them if they are 404s?
Sample logs:
1.1.1.1 - - [10/Jun/2010:01:55:01 -0400] "GET /popup/authentication HTTP/1.1" 404 1831 "-" "Mozilla/4.0 (compatible;)"
2.2.2.2 - - [06/Jul/2010:08:50:27 -0400] "GET /images/social/twitter.1.png HTTP/1.1" 304 - "-" "Mozilla/4.0 (compatible;)"
3.3.3.3 - - [08/Jul/2010:14:58:41 -0400] "GET /calendar/list/json/6/27/2010 HTTP/1.1" 404 46 "-" "Mozilla/4.0 (compatible;)"
4.4.4.4 - - [11/Jun/2010:13:41:27 -0400] "GET / HTTP/1.1" 200 3875 "-" "Mozilla/4.0 (compatible;)"
-
According to several resources (such as this entry), these are requests used by proxy tools that employ pre-fetching strategies (they go and download all of the pages that are linked from the page a user is visiting in order to cache them for fast retrieval). Blue Coat, Verizon, LookSmart, and others have all been found at one time or another to use that UserAgent in their tools.
Also, according to Microsoft, some versions of Internet Explorer, for historical reasons, have used variations of the Mozilla/4.0 (compatible) UserAgent. See here for more details.
fahadsadah : Actually, all versions of Internet Exploder up to IE8 identify as Mozilla/4.0 (compatible), but there's usually more within the brackets, like Trident/OS version numbers. IE9 identifies as 5.0 compatible, like all other modern browsersFrom runlevel6
0 comments:
Post a Comment