A row is brewing over benchmarks used to assess the performance of smartphone semiconductors after one of the tools had to be revised following claims of discrepancies over its results.
The AnTuTu benchmark, which is used on Android smartphones and tablets, had suggested that Intel's latest Atom microprocessors just pipped rival ARM-based chips from various vendors in key areas of the benchmark.
Further reading ARM vs Intel: a battle of business models TSMC 'tapes out' high-performance 64-bit ARMs, ready for production Smartphone and tablet demand boosts ARM profits
However, following a revision, the overall benchmarking results for the Intel Atom were cut by 20 per cent - and by 50 per cent on its RAM benchmarking tests, according to Tirias Research.
The revisions follow claims that the AnTuTu benchmarking code used on the Intel architecture was not executing all instructions intended for the RAM test, artificially boosting the results.
Scores for the rival ARM microprocessors, including the Samsung Exynos 5 Octa and the Qualcomm Snapdragon 600, benchmarked using the same tool remained substantially the same after the AnTuTu revisions.
"It's clear that the ARM processors still hold a significant advantage over the Intel processors, whether you include the new AnTuTu scores or just eliminate them completely from the evaluation," suggested Tirias Research analyst Jim McGregor in an editorial for microprocessor industry newspaper EE Times.
It added: "The current and upcoming revisions to the AnTuTu benchmark will also drastically alter the scores of Intel's upcoming Bay Trail Atom processor... It's clear that Intel still has an uphill climb to catch the ARM camp, much less surpass it. It is also clear that we need better benchmarks for mobile devices that test for platform efficiency and usage models."
The benchmarks and the row over their accuracy comes at a time when the rivalry between Intel and ARM is hotting up, with ARM stepping into the server market with specially designed 32-bit and, soon, 64-bit parts, while Intel has focused on power efficiency with both its Atom and Haswell microprocessors in a bid to break into the mobile device market dominated by ARM.
