Apple released the latest major version of its Web browser, Safari 5, earlier this week. Along with several new features—one of them somewhat controversial—the update was said to pack a number of performance improvements, including DNS pre-fetching and optimizations to Safari's Nitro JavaScript engine. "Safari continues to lead the pack in performance," Apple SVP of worldwide marketing, Phil Schiller, said in a statement. We decided to put those claims to the test, pitting Safari against leading browsers on both Mac OS X and Windows 7.
Included in our tests are a number of JavaScript benchmarks, including the WebKit team's SunSpider, Google's V8 Benchmark, and Mozilla's Dromaeo. We also took a look at graphics acceleration performance using Microsoft's HTML5 "Flying Images" speed demo. The tests were run on the latest stable versions of Safari, Chrome, Firefox, and Opera on Mac OS X 10.6.3 running on a 3.0GHz, 8-core Mac Pro with 10GB RAM and two ATI Radeon 2600XT GPUs. The tests were also run on Windows 7 on a 2.67 GHz Core 2 Duo PC with 4 GB of RAM and an ATI 4830 GPU. On our Windows 7 rig we had a recent development build of Chrome (as opposed to the latest stable version), and we also ran the tests using IE8 and a developer preview of IE9.
Right off bat, our tests more or less confirmed Apple's JavaScript speed claims on Mac OS X. "Powered by the Nitro JavaScript engine, Safari 5 on the Mac runs JavaScript 30 percent faster than Safari 4, three percent faster than Chrome 5.0, and over twice as fast as Firefox 3.6," according to a statement released by the company. We found Safari to be about four percent faster than Chrome, and slightly faster than Opera. However, Firefox continues to lag significantly behind the other three browsers on raw JavaScript performance—Safari was nearly three times faster on our testbed.
However, looking at the other common JavaScript benchmarks, Safari didn't fare so well. Chrome was nearly twice as fast as Safari running the V8 Benchmark suite, which attempts to simulate the kind of code that large-scale, next-generation Web apps would use. Even Opera bested Safari on these tests by a significant margin. Unsurprisingly, Firefox trailed much farther behind the other three browsers—Safari is 6x faster, Opera is 8x faster, and Chrome is nearly 12x faster on these benchmarks.
The most surprising results, however, come from the Dromaeo JavaScript tests. Safari was more than twice as fast as Firefox, but Opera was nearly 11 times faster than Safari, and Chrome was a whopping 23 times faster. Looking closer at the individual tests, Both Chrome and Opera performed far, far better on regular expressions tests than Safari. Optimizing regular expressions was something that the WebKit team had done two years ago to push the performance of earlier versions of its Nitro (née SquirrelFish Extreme) engine, but both Opera and especially Chrome have made significant improvements since then.
Our own Ryan Paul explained that V8's approach to regular expressions spends more time on regex compilation to extract faster performance. Regexes aren't necessarily that common on all websites, but are commonly used for form input validation. Google's implementation provides the biggest boost to regexes that are iterated repeatedly over larger data sets—quite similar to those used in benchmarking. The benchmarks may weigh the results a little too heavily on regex performance, but from the looks of it, the WebKit team definitely has its work cut out for it to catch up on this particular optimization.
Another surprising result came from running one of the speed demos from Microsoft's Internet Explorer 9 Test Drive page. The Flying Images demo uses a number of browser icons arranged in three-dimensional space, and spins the arrangement around. Pointing the cursor in the middle of the browser window forces the perspective to exaggerate, taxing the graphics drawing code. Chrome only managed to animate five frames per second. Safari was twice as fast at 11fps, but Firefox was faster still at 17fps. Opera was the dark horse in this test, turning in a staggering 51fps. To Safari's credit, however, it rendered the demo far better than Opera. On the other hand, Firefox's rendering was just as good with much smoother animation. Given Apple's focus on accelerating 3D CSS Transforms and otherwise leveraging the GPU for more performance, and its push to replace Flash with HTML5 and related standards, it seems unusual that Safari didn't perform better on this test.
On Windows, Safari ran just slightly slower than the development build of Chrome 6 installed when running the SunSpider benchmarks—a sign that Chrome's fast pace of development is retuning dividends. No surprise that IE8 just plain sucked, but the preview of IE9 shows promise. The patterns on V8 Bench and Dromaeo were repeated on Windows, with Chrome having a massive lead on these tests. IE9 again shows some promise of decent performance, besting Firefox, but it's still not on par with either Safari or Opera.
The Flying Images demo results again surprised us. Apple specifically mentioned limited GPU acceleration as one of the features for the Windows version of Safari 5. However, Safari tied for dead last place on this test with Chrome—which hasn't done much if anything to leverage GPU acceleration—at a measly 3fps. Firefox was far faster at 12fps, and Opera was far faster yet again with 28fps. However, IE9 turned in a mind-blowing 64fps. Again, given Apple's promotion of newer Web standards for animation, the fact that this demo runs 6 times slower on Mac OS X on much more capable hardware has to be embarrassing.
Overall, Safari 5 is quite snappy, and does feel faster than Safari 4—especially on Windows. Though Safari's Nitro JavaScript engine may have a very slight raw performance advantage, the benchmarks we ran show that Apple and the WebKit team certainly have some areas where it can improve—both in JavaScript performance characteristics as well as optimizing performance for animation and other graphics rendering. All this fierce competition among Apple, Microsoft, Google, Mozilla, and Opera for "fastest" browser, though, ultimately benefits all users in the long run.
reader comments
158