Visible to the public HTTP security headers analysis of top one million websites

TitleHTTP security headers analysis of top one million websites
Publication TypeConference Paper
Year of Publication2018
AuthorsLavrenovs, A., Melón, F. J. R.
Conference Name2018 10th International Conference on Cyber Conflict (CyCon)
Date PublishedMay 2018
PublisherIEEE
ISBN Number978-9-9499-9043-6
KeywordsAlexa top one million list, Charge coupled devices, Collaboration, content security policy, content-security-policy, Databases, HTTP headers, HTTP response headers analysis, HTTP security headers analysis, HTTP strict transport security, HTTP/2, HTTPs, HTTPS requests, hypermedia, Internet, policy-based governance, Protocols, pubcrawl, redirections, referrer-policy, security, security of data, Security Policies Analysis, server, set-cookie, strict- transport-security, Tools, top one million websites survey, transport protocols, Uniform resource locators, web security, web security policies, Web sites, websites, World Wide Web, www subdomain, X-content-type, X-frame-options, X-XSS-protection, X.509 Certificate
AbstractWe present research on the security of the most popular websites, ranked according to Alexa's top one million list, based on an HTTP response headers analysis. For each of the domains included in the list, we made four different requests: an HTTP/1.1 request to the domain itself and to its "www" subdomain and two more equivalent HTTPS requests. Redirections were always followed. A detailed discussion of the request process and main outcomes is presented, including X.509 certificate issues and comparison of results with equivalent HTTP/2 requests. The body of the responses was discarded, and the HTTP response header fields were stored in a database. We analysed the prevalence of the most important response headers related to web security aspects. In particular, we took into account Strict- Transport-Security, Content-Security-Policy, X-XSS-Protection, X-Frame-Options, Set-Cookie (for session cookies) and X-Content-Type. We also reviewed the contents of response HTTP headers that potentially could reveal unwanted information, like Server (and related headers), Date and Referrer-Policy. This research offers an up-to-date survey of current prevalence of web security policies implemented through HTTP response headers and concludes that most popular sites tend to implement it noticeably more often than less popular ones. Equally, HTTPS sites seem to be far more eager to implement those policies than HTTP only websites. A comparison with previous works show that web security policies based on HTTP response headers are continuously growing, but still far from satisfactory widespread adoption.
URLhttps://ieeexplore.ieee.org/document/8405025
DOI10.23919/CYCON.2018.8405025
Citation Keylavrenovs_http_2018