Austrian antivirus test lab AV-Comparatives has posted harsh criticism on YouTubers that perform antivirus tests. Results of antivirus tests performed by YouTube users usually differ a lot from the results of professional test labs like AV-Comparatives or AV-Test.
Antivirus suites usually perform worse in tests by YouTubers than in tests performed by the professional test labs. AV-Comparatives explains that the reason is that YouTubers usually only test a relative small collection of malware and only have access to publicly available malware collections. Besides that, YouTubers also frequently rely on tests results from Virustotal, Google’s multi-engine antivirus service that scans files with engines of 60 different antivirus suites.
VirusTotal only scans user uploaded files with the on-demand scanning engine of suites. Other mechanisms that assist in malware detection are not used such as behavioral analysis and sand boxing. This makes it possible that malware isn’t detected by the engine used on VirusTotal, while the same engine does detect malware when the entire suite is installed on the system.
Also, the test methods of YouTubers leave much to be desired, which causes incorrect results, according to AV-Comparatives. The test lab also adds that most YouTubers are unable to properly interpret test results. Sometimes because they are not able to properly determine whether malware really infected the system. E.g. it’s easy to check whether ransomware infected a system, but infections with rootkits or key loggers are often harder to identify. YouTubers might miss those. And most YouTubers also don’t verify false-positives which means that they mistakenly think an antivirus product finds malware, while in reality it has falsely detected a clean file as infected malware.
“Most YouTube tests finish with a block percentage of the tested AV product, which may be well below the score achieved by the same product in a professional test. With overstretched conclusions based on limited test approaches, and without a representative and balanced set of malware samples, YouTube testers’ results may not reflect the real AV product’s protection performance,” AV-Comparatives writes.
AV-Comparatives does see some use in tests performed by YouTubers, “However, we would like to point out that some YouTube testers may publish their reviews with the best of intentions and contain useful insights into e.g. the GUI of the product. We encourage users to install a trial version of any AV product they are interested in before making a purchase, so that they can decide for themselves whether it fits their own personal requirements.”