False Alarm Test Report September 2015

AV-Comparatives releases an appendix report for its False Alarm Test done during the File-Detection Test. The False Alarm Test report contains details about the false alarms encountered by the various products, such a the affected programs, the detection names and the supposed prevalence (according to various telemetry data sources). You can download the appendix False Alarm Test report of September 2015 as PDF here.

With AV testing it is important to measure not only detection capabilities but also reliability – one of reliability aspects is certainly product’s tendency to flag clean files as infected. No product is immune from false positives (FP’s) but there are differences among them and the goal is to measure them. Nobody has all legitimate files that exist and so no “ultimate” test of FP’s can be done.What can be done and is reasonable, is to create and use a set of clean files which is independent. If on such set one product has e.g. 50 FP’s and another only 10, it is likely that the first product is more prone to FP’s than the other. It doesn’t mean the product with 10 FP’s doesn’t have more than 10 FP’s globally, but important is the relative number.In order to better evaluate the quality of the detection capabilities of anti-virus products, we provide also a false alarm test. False alarms can sometimes cause as much troubles as a real infection. Please consider the false alarm rate when looking at the detection rates, as a product which is prone to cause false alarms achieves higher scores easier.