2012 ab is-your-browser-putting-you-at-risk

ANALYSIS BRIEF – September 2012

IS YOUR BROWSER PUTTING YOU AT RISK? PART 1 – GENERAL MALWARE BLOCKING

Authors -‐ Bob Walder, Francisco Artes, Stefan Frei, Ken Baylor, Jayendra Pathak, Vikram Phatak

Overview The ineffectiveness of Web browser security is one of the most common reasons for malware infection. Browsers offer a direct and unique route for infection, bypassing corporate protection layers and bringing malware deep into the corporate environment, often protecting it from detection using SSL. Browsers must provide a strong layer of defense from malware, rather than defer to operating system antimalware solutions. This series examines the effectiveness of leading browsers to block malware.

The four leading browsers were tested against three million samples of real world malicious software. Major discrepancies were noted in their ability to block malware. Data represented in this report was captured over one hundred and seventy-‐five (175) days through NSS Labs’ unique live testing harness, and provides in-‐depth insight into the built-‐in protection capabilities of modern browsers, including Chrome, Firefox, Internet Explorer, and Safari.

This series of papers will examine the ability of the four leading browsers to block each of the five main purposes of malware and malware monetization. Monetization of malware is achieved by multiple means, including click fraud, fake antivirus, account / password theft, bank/financial fraud, and gaming fraud. Collectively they account for billions of dollars worth of corporate and consumer theft per year, yet browsers vary widely in their ability to block malware, despite adverse effects on business and individual users alike.

Tested Products

• Apple Safari 5 • Google Chrome 15 -‐ 19 • Microsoft Internet Explorer 9 • Mozilla Firefox 7 – 13

Over 3,000,000 test cases were used in the data sampling captured via NSS Labs’ unique live testing harness. An initial sample set of 227,841 unique and suspicious URLs entered the system; 84,396 were found active and malicious and met the criteria for entry into the test. In total 3,038,324 test runs were performed by the four browsers against these unique 84,396 URLs – resulting in over 750,000 tests cases per browser.

NSS Labs Analysis Brief – Is Your Browser Putting You At Risk? Part 1

© 2012 NSS Labs, Inc. All rights reserved. 2

Testing was repeated every six (6) hours until the target URL was no longer active. Samples that did not pass the validation criteria were removed, including false positives and adware. Ultimately, 1,407,233 URL test cases passed the post-‐validation process and are included in the results. Each sample payload was validated internally. MD5 hashes of samples were submitted to VirusTotal and the resulting scanner reports were then used to classify malware types. Additionally, the test samples were verified by multiple independent external sources to confirm distribution accuracy and malware classification.

Figure 1 – Malware Block Rate Over Time with 10-Day Moving Average (higher % is better)

During the testing period, Internet Explorer maintained a malware block rate of 95% while Firefox and Safari’s block rate remained just under 6%. Over the same time period, Chrome’s block rate varied from 13% to just over 74%. This could be attributed to changing protection tactics over time that is indicative of the ongoing battle between antimalware developers and malicious actors.

NSS Lab Findings:

• Browsers offer the largest attack surface in most enterprise networks and are the most common vector for malware installations

• The use of SSL by browsers presents additional problems to enterprises since it offers the opportunity to bypass many layers of corporate security protection

• The leading browsers show a significant variance in their ability to block malware. • Given the increasing mobility of users and devices, blocking malware is not only extremely important, but

potentially the only means of reducing risk when outside of the corporate perimeter of protection. • Web browsing is the primary attack vector of criminals attempting to monetize malware, using a variety

of means, including click fraud, fake antivirus, account / password theft, bank/financial fraud, and gaming fraud.

0%

20%

40%

60%

80%

100%

Firefox

Chrome

Internet Explorer

Safari



• The tolerance of browsers with low malware block rates may present undue risk to an organization.

NSS Labs Recommendations:

• Users should evaluate browser security as part of their layered security strategy. • Enterprises should perform a risk analysis of the browsers in the organization and remove those with

unjustified high risk where possible. • Enterprise and individual users should use the findings in this report to assist in the selection of the

browser most appropriate to their protection needs. However, malware infection rather than exploits were the subject of this test, and readers should not draw conclusions based upon this analysis brief alone.

Analysis As the most widely used and ubiquitous means of accessing the Internet, web browsers are uniquely positioned to filter and stop malware at an early stage. This capability becomes even more important given the increasing mobility of devices, which means corporate perimeter and network protection services cannot always be relied upon.

To complement traditional defenses and to address the highly dynamic nature of current attacks and attack distribution methods, modern web browsers employ technologies to block access to malicious URLs before loading the content. Blocking access to malicious URLs is a formidable first line of defense, since it provides complete protection against malware entering the system. However, little is known or published on the effectiveness of web browser’s internal blocking technology and performance.

This analysis examines the ability of four different web browsers to protect users from malware downloads, also known as socially-‐engineered malware.1 Modern web browsers offer an added layer of protection against these threats by leveraging in-‐the-‐cloud, reputation-‐based mechanisms to warn users of potential infection. However, not all vendors have taken the same approach.

Browser protection contains two main functional components. The foundation is an “in-‐the-‐cloud” reputation-‐based system which scours the Internet for malicious web sites and categorizes content accordingly, either by adding it to a black or white list, or assigning a score (depending on the vendor’s approach.) This categorization may be performed manually, automatically, or using both methods. Some vendors will utilize feedback from user agents on their customers’ endpoints to report back to the reputation system automatically, providing information relevant to the trustworthiness, or otherwise, of applications and files downloaded from the Internet. The second functional component resides within the web browser itself, and requests reputation information from the in-‐the-‐cloud systems about specific URLs and then enforces warning and blocking functions.

1 Exploits that install malware without the user being aware (also referred to as “drive-by downloads”) are not included in this particular study.



When results are returned that a site is “bad,” the web browser redirects the user to a warning message or page informing that the URL is malicious. In the event that the URL links to a download, the web browser instructs the user that the content is likely malicious and that the download should be cancelled. Conversely, when a website is determined to be “good,” the web browser takes no action and the user is unaware that a security check was performed.

Figure 1 – Browser Warnings

Functionality unique to Chrome

NSS Labs determined that Safe Browsing API v2 includes additional functionality that has been integrated into Chrome, but not Firefox or Safari. This functionality provides reputation services for executable files, or as Google describes them “malicious downloads”.

Figure 2 -‐ Chrome Safe Browsing Warning

Internet Explorer Warning Chrome Warning

Firefox Warning Safari Warning



Malware Block Performance Each browser’s individual block performance was tracked over time and mapped by malware purpose. When aggregated an overall block rate of all collected malware by browser was developed. A browser’s overall block rate is defined as the percentage of successful blocks divided by the total number of test cases. With tests conducted every 6 hours, a URL that was online for 48 hours will be tested 8 times. A browser blocking it on 6 (out of a maximum 8) test runs will achieve a block rate of 75%. Figure 3 shows the overall block performance of the four browsers tested. As expected, since Firefox and Safari using the same technology they achieve similar block rates. However, the large difference of the average block rate between browsers is noteworthy, with results ranging from 4.7% up to 94%.

Figure 3 – Overall Malware Block Rate by Browser (higher % is better)

To assess the effectiveness of different blocking technologies, the NSS test harness also records the mechanism that blocked access to a URL.

Of the three browsers using Google’s Safe Browsing API, Chrome is the only one to also utilize Google’s malicious download technology. Figure 4 shows the block performance of the URL blocking component and the additional download block component used only by Google’s Chrome. The URL blocking performance of these three Safe Browsing browsers was consistent at around 5%. Google’s malicious download protection proved to be almost five times more effective than URL blocking alone. As seen in Figure , it increases overall blocking performance by 28% compared to URL blocking alone, and accounts for the majority of the blocking performance of Google Chrome.

The core protection technology in Internet Explorer is SmartScreen, which provides URL-‐based protection from attacks via an integrated cloud-‐based URL-‐reputation service. SmartScreen also works with Download Manager to prevent malicious downloads.

4.7%

94.0%

5.0%

27.6%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Safari

Internet Explorer

Firefox

Chrome



Figure 4 – Blocking technologies used by browsers (higher % is better)

Time to block Malicious Sites Every time a new campaign is launched by malicious actors, it is vital that it is detected as quickly as possible by security solutions deployed in the enterprise. The following response time graph shows how long it took each of the browsers to block a threat once it was introduced into the test cycle. Cumulative protection rates are calculated each day until blocked.

Figure 5 -‐ Time to Block Malicious Sites

94.0%

4.7%

5.0%

4.6% 23.0%

0%! 10%! 20%! 30%! 40%! 50%! 60%! 70%! 80%! 90%! 100%!

Safari!

Internet Explorer!

Firefox!

Chrome!

Safari! Internet Explorer! Firefox! Chrome!SmartScreen! 94.0%!SafeBrowsing! 4.7%! 5.0%! 4.6%!Malicious Download! 0.0%! 0.0%! 23.0%!

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0 5 10 15 20 25 30

Bloc

k Ra

te

Days

Internet Explorer

Chrome

Firefox

Safari



Days Firefox Chrome Internet Explorer Safari

1 4% 20% 91% 4%

2 5% 22% 92% 4%

3 5% 23% 92% 4%

4 5% 24% 92% 4%

5 5% 25% 93% 4%

6 5% 25% 93% 5%

7 5% 26% 93% 5%

10 5% 27% 93% 5%

15 5% 28% 94% 5%

20 5% 28% 94% 5%

25 5% 28% 94% 5%

30 5% 28% 94% 5%

Table 1-‐ Time to Block Malicious Sites

Ultimately, the results reveal significant variations in the abilities of the browsers to protect against malware. Chrome provides more protection than Safari or Firefox using the Safe Browsing feed, apparently due to its malicious download protection. Trends show minor differences between Firefox and Safari.

Results from these tests indicate that the four browsers vary both in their approach and effectiveness in blocking different malware categories. It was decided to further categorize the malware behind the suspicious URLs to measure the browser’s block performance for each class of malware.

The ability of the four leading browsers to block each of the five main purposes of malware: click fraud, banking/financial fraud, fake antivirus, password/account theft and game fraud was examined and will be detailed in subsequent papers in this series.

Reading List Analysis Brief: Did Google Pull a Fast One on Firefox and Safari Users?



Appendix A – Methodology

Client Host Description

All tested browser software was installed on identical virtual machines with the following specifications:

Microsoft Windows 7

2GB RAM

40GB hard drive

Browser machines were tested prior to, and during, the test to ensure proper functionality. Browsers were given full access to the Internet to enable them to visit live sites.

Tested Browsers The browsers, or products under test, were obtained independently by NSS Labs. Generally, available software releases were used in all cases. Each product was updated to the most current version available at the time testing began. The following is a current list of the web browsers that were tested:

• Google ChromeTM v15-‐19 • Microsoft® Internet Explorer® 9 • Mozilla® Firefox® v7-‐13 • Safari® v5.

Once testing began, the product version was monitored and new updates were applied in a realistic patching methodology. As a new version of a browser was made publicly available during the testing window, NSS would begin updating the test harness machines and run both versions in parallel over the course of a two-‐week phase-‐out of the prior version of the browser. This maintained the integrity of the virtual instances that were under test while allowing for fresh instances to start with the new browser version. This test relied upon Internet access for the reputation systems and access to live content. Generally, there is a configurable separation between software updates and database or signature updates, to draw analogies from anti-‐virus, intrusion prevention, and general software practices.

Network Description The browsers were tested for their ability to protect the client in “connected” use cases. Thus, the tests consider and analyze the effectiveness of browser protection in NSS Labs’ real-‐world, live Internet testing harness.

The host system had one network interface card (NIC) and was connected to the network via a 1Gb switch port. For the purposes of this test, NSS Labs utilized 384 desktop systems each running a web browser. Results were recorded into a MySQL database.

Test Duration



NSS Labs’ browser test was performed continuously (24 x 7) for 175 days. Throughout the duration of the test, new URLs were added as they were discovered.

Test Frequency

Over the course of the test, each URL was run through the test harness every six hours. Regardless of success or failure, NSS Labs continued to attempt to download a malware sample with the web browser for the duration of the test.

Sample Sets for Malware URLs

Freshness of malware sites is a key attribute of this type of test. In order to utilize the freshest, most representative URLs, NSS Labs received a broad range of samples from a number of different sources.

Sources

NSS Labs operates its own network of spam traps and honeypots. These e-‐mail accounts with high-‐volume traffic yield thousands of unique e-‐mails and URLs per day. In addition, NSS Labs maintains relationships with other independent security researchers, networks, and security companies that provide access to URLs and malicious content. Sample sets contain malicious URLs distributed via: e-‐mail, instant messaging, social networks, and malicious websites. No content is used from the tested parties.

Malicious URLs targeting users throughout the globe are identified and selected for inclusion in this test. Users are defined as individuals residing within the North America, South American, European, and Asia-‐Pacific regions, including: Argentina, Australia, Austria, Brazil, Canada, China, France, Germany, India, Italy, Japan, Indonesia, Mexico, New Zealand, Singapore, Spain, South Korea, Sweden, Thailand, the United Kingdom, the United States of America, and Vietnam. This report is comprised only of data from the United States of America samples; future papers will include the additional data. The ultimate determinant of whether or not a malicious URL is included in this test is its participation in a malware campaign targeting users. Lastly, just because a malicious URL is included in a campaign targeting an Asia-‐Pacific or a North American user does not mean that the URL is not used in other campaigns targeting users from other regions.

Collect New Suspicious Malicous Sites from Sources

Pre-Filter, Validate, Prune & Archive

Sites

Distribute to Test Clients

Test Clients Visit Site & Record Block/Allow

Results Collected & Archived



Exploits containing malware payloads (exploits plus malware), also known as “clickjacking” or “drive-‐by downloads” are excluded from the test. Every effort is made to consider submissions that reflect a real-‐world distribution of malware—categorically, geographically, and by platform.

In addition, NSS Labs maintains a collection of “clean URLs” which includes sites from Yahoo, Amazon, Microsoft, Google, NSS Labs, major banks, and others. Periodically, clean URLs are run through the system to verify that the browsers are not over-‐blocking.

Catalog URLs

New sites are added to the URL consideration set as soon as possible. The date and time each sample is introduced is noted. Most sources are automatically and immediately inserted, while some methods require manual handling and can be processed in under 30 minutes. All items in the consideration set are cataloged with a unique NSS Labs ID, regardless of their validity. This enables correct tracking of effectiveness of sample sources.

Confirm Sample Presence of URLs

Time is of the essence since the objective is to test the effectiveness against the freshest possible malware sites. Given the nature of the feeds, and the velocity of change, it is not possible to validate each site in depth before the test, since the sites could quickly disappear. Thus, each of the test items is given a cursory review to verify it is present and accessible on the live Internet.

In order to be included in the execution set, URLs must be live during the test iteration. At the beginning of each test cycle, the availability of the URL is confirmed by ensuring that the site can be reached and is active, such that a non-‐404 web page is returned.

This validation occurs within minutes of receiving the samples from NSS sources. Note: These classifications are further validated after the test, and URLs are reclassified and/or removed accordingly.

Archive active URL content

The active URL content is downloaded and saved to an archive server with a unique NSS ID number. This enables NSS Labs to preserve the URL content for control and validation purposes.

Dynamically Execute Each URL

A client automation utility requests each of the URLs deemed “present” (based upon results of the test described in Section 5.4) via each of the web browsers in the test. NSS Labs records whether or not the malware is downloaded and if the download attempt triggers a warning from the browser’s malware protection.

Scoring and Recording the results

The resulting response is recorded as either “Allowed” or “Blocked and Warned.”

Success: NSS Labs defines success based upon a web browser successfully preventing malware from being downloaded and correctly issuing a warning.

Failure: NSS Labs defines a failure based upon a web browser failing to prevent the malware from being downloaded and/or failing to issue a warning.



Pruning

Throughout the test, lab engineers review and remove non-‐conforming URLs and content from the test execution set. For example, a URL that was initially classified as malware, but that has since been replaced with a generic splash page, will be removed from the test.

If a URL sample becomes unavailable for download during the course of the test, the sample is removed from the test collection for that iteration. NSS Labs continually verifies each sample’s presence (availability for download) and adds/removes each sample from the test set accordingly. Should a malware sample be unavailable for a test iteration and then become available again for a subsequent iteration, it will be added back into the test collection. Unavailable samples are not included in calculations of success or failure by a web browser.

Post-‐Test Validation

Post-‐test validation enables NSS Labs to reclassify and even remove samples that were either not malicious or not available before the test started. NSS Labs uses two different commercial sandboxes to prune and validate the malware (Sunbelt’s CWSandbox and Norman® Analyzer). Further validation is performed using proprietary tools, system instrumentation, and code analysis as needed.



NSS Labs Test Environment and Methodology NSS Labs has created a complex “live” test environment and methodology to assess the protective capabilities of Internet browsers under the most real-‐world conditions possible, while also maintaining control and verification of the procedures.

The purpose of the study was to determine how well current web browsers protect users from the most prevalent malware threats on the Internet today. A key aspect in any test of this nature is the timing. Given the rapid rate and aggression with which criminals propagate and manipulate malicious websites, a key objective is to ensure that the “freshest” sites possible are included in the test.

NSS Labs has developed a unique proprietary “Live Testing” harness and methodology. As part of this methodology, NSS Labs continually collects web-‐based threats from multiple sources, including partners and NSS’ own servers and high-‐interaction honeynets. Potential threats are vetted algorithmically before being inserted into the test queue; threats are being inserted and vetted continually. Unique in this procedure is that NSS Labs validates the samples before and after the test. Actual testing of the threats is repeated every six hours and starts with validation of the site’s existence and conformance to the test definition.

All tests are executed in a highly controlled manner, and results are meticulously recorded and archived at each interval.

Figure 2 -‐ NSS Test Framework



© 2012 NSS Labs, Inc. All rights reserved. No part of this publication may be reproduced, photocopied, stored on a retrieval system, or transmitted without the express written consent of the authors.

Please note that access to or use of this report is conditioned on the following:

1. The information in this report is subject to change by NSS Labs without notice.

2. The information in this report is believed by NSS Labs to be accurate and reliable at the time of publication, but is not guaranteed. All use of and reliance on this report are at the reader’s sole risk. NSS Labs is not liable or responsible for any damages, losses, or expenses arising from any error or omission in this report.

3. NO WARRANTIES, EXPRESS OR IMPLIED ARE GIVEN BY NSS LABS. ALL IMPLIED WARRANTIES, INCLUDING IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-‐INFRINGEMENT ARE DISCLAIMED AND EXCLUDED BY NSS LABS. IN NO EVENT SHALL NSS LABS BE LIABLE FOR ANY CONSEQUENTIAL, INCIDENTAL OR INDIRECT DAMAGES, OR FOR ANY LOSS OF PROFIT, REVENUE, DATA, COMPUTER PROGRAMS, OR OTHER ASSETS, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.

4. This report does not constitute an endorsement, recommendation, or guarantee of any of the products (hardware or software) tested or the hardware and software used in testing the products. The testing does not guarantee that there are no errors or defects in the products or that the products will meet the reader’s expectations, requirements, needs, or specifications, or that they will operate without interruption.

5. This report does not imply any endorsement, sponsorship, affiliation, or verification by or with any organizations mentioned in this report.

6. All trademarks, service marks, and trade names used in this report are the trademarks, service marks, and trade names of their respective owners.

Contact Information NSS Labs, Inc. 6207 Bee Caves Road, Suite 350 Austin, TX 78746 USA +1 (512) 961-‐5300 [email protected] www.nsslabs.com

This analysis brief was produced as part of NSS Labs’ independent testing information services. Leading products were tested at no cost to the vendor, and NSS Labs received no vendor funding to produce this analysis brief.

Technology

2012 ab is-your-browser-putting-you-at-risk