The German firm AV-TEST today stood by the results of its search engine investigation that claimed Microsoft's Bing shows five times the number of malware-hosting websites than Google in its results.
On Friday, Microsoft called AV-TEST's results flawed.
"AV-TEST's study doesn't represent the true experience or risk to customers," alleged David Felstead, senior development lead for Bing, in a Friday blog.
The website that Felstead cited as an example of how Bing warns of dangerous destinations was, its owners claimed, free of malware and had never been compromised in its 14 years on the Internet.
In his blog post, Felstead reacted to a report issued April 6 by AV-TEST of Magdeburg, Germany. The report said Bing indexed and returned in its search results nearly five times as many malware-infected links as Google.
Over an 18-month stretch, AV-TEST evaluated more than 40 million websites to determine the extent of a long-held maxim by security professionals: That even with extensive efforts to scrub search results of dangerous links, engines such as Google and Bing cannot stop cyber criminals from exploiting search tools -- and users' reliance on them -- by either compromising legitimate sites or artificially promoting malformed websites to host attack code.
"Google achieved the best results in the study, followed by Bing," said AV-TEST in its conclusions (download PDF). "Attention must, however, be drawn to the fact that Bing delivered five times as many websites containing malware as Google during the study."
According to AV-TEST, of the 10.9 million tested with Bing, 1,285 were found to host malware, for a infection rate of 0.012%, or 12 sites out of every 100,000.
Of the 10.9 million websites tested with Google, 272 contained attack code, an infection rate of 0.0025%, or 2.5 sites out of every 100,000.
Those infection rates may be minuscule, but AV-TEST argued that in practicality, the number of malware-hosting sites encountered by users was significant simply because of the volume of queries run each day on the major engines.
"It is important to remember that Google alone deals with a phenomenal total of 2 to 3 billion search requests worldwide every day," AV-TEST said. "If this total is factored into the calculations, the total number of websites containing malware found by the search engine is enough to make your head spin!"
Microsoft took nearly two weeks to respond to AV-TEST's claims, but when it did, it pulled few punches. "The conclusions many have drawn from the study are wrong," Felstead said flatly.
Felstead based his argument on the warning that appears when links suspected of harboring malware appear within Bing's results and those links are clicked by the user.
"By using the API instead of the user interface, AV-TEST bypassed our warning system designed to keep customers from being harmed by malware," said Felstead. "Bing actually does prevent customers from clicking on malware infected sites."
Felstead said that users see the warning only once in every 10,000 searches, or 0.01% of the time, a number close to AV-TEST's 0.012%. "In any case, the overall scale of the problem is very small," Felstead asserted.
AV-TEST confirmed today that it relied on a Bing API (application programming interface) to collect search results from Microsoft's engine.
"No links were clicked/followed through the search engine," Andreas Marx, CEO of AV-TEST, said in a Monday email reply to questions. "We simply grabbed the URLs and downloaded them on our own systems for further analysis. We didn't want to test the warnings from the search engine but simply how many potentially malicious websites are returned by the search engine."
