Aug 16 2007

Inside CNET Labs: Windows virtual machine performance on the Mac

Note: This post was updated on August 17, 2007, to make a correction.

With VMware's official release of Fusion 1.0 less than two weeks ago, there are now no less than four different ways to run Windows applications on Intel-based Macs. Fusion, as well as SWsoft's Parallels Desktop 3.0 for Mac use virtualization technology to allow you to simultaneously run the Windows operating system as a virtual machine alongside the Mac OS. CrossOver Mac 6.0 from CodeWeavers uses a different virtualization approach by building on the open-source API, Wine, which allows you to run individual Windows applications in the Mac OS without needing to install or run the Windows operating system.

(Credit: VMware)

Perhaps the easiest and most common way to run Windows on a Mac today is with Apple's own Boot Camp Public Beta. While Boot Camp is currently the only way to run Windows natively on a Mac, it requires booting directly into the Windows OS, during which the Mac OS is not accessible (until you reboot the computer back into the Mac OS, leaving the Windows OS).

With all these different options for running Windows applications on Intel Macs, we wanted to see how they compare against each other in terms of performance; so CNET Labs applied its systems testing methodology to an eight-core, 2.66GHz Mac Pro running the Mac OS X 10.4.10. We used Boot Camp 1.3 Beta, CrossOver Mac 6.1.0, Parallels Desktop 3.0 for Mac (4560), and Fusion 1.0 (51348). Windows Vista Ultimate was used as the operating system for Boot Camp, Fusion, and Parallels. Fusion and Parallels were both set to 1,024MB of system memory, a 32GB hard disk, and the maximum amount of graphics memory they each supported (Fusion: 128MB, Parallels: 64MB).

Multimedia multitasking test (in seconds)
(Shorter bars indicate better performance)
Apple Mac OS X 10.4.10
248 
Apple Boot Camp 1.3 Beta
529 
VMware Fusion 1.0
874 
Parallels Desktop 3.0 for Mac
3,260 
Note: Apple QuickTime 7.2 and Apple iTunes 7.3.2.6

The Multimedia multitasking test is the most taxing of our tests. It encodes QuickTime video in the foreground, while iTunes simultaneously encodes audio in the background. While Fusion's performance on this test was significantly slower than we saw with Windows running natively under Boot Camp, Fusion was still more than 3.5-times faster than Parallels. One of the primary reasons why Fusion is faster here is that it supports up to two CPU cores, while Parallels supports only one CPU core. This test benefits greatly from multiple CPU cores, as is evidenced by the Mac OS and Boot Camp performance, each with eight cores available.

Adobe Photoshop CS3 test (in seconds)
(Shorter bars indicate better performance)
Apple Mac OS X 10.4.10
120 
Apple Boot Camp 1.3 Beta
201 
VMware Fusion 1.0
271 
Parallels Desktop 3.0 for Mac
516 

Our Photoshop CS3 test results repeat what we saw with the Multimedia multitasking test: Fusion is faster than Parallels, but neither is as speedy as Windows running in Boot Camp. While the Photoshop CS3, Multimedia multitasking, and Cinebench (see below) tests alone are not enough to make an absolute conclusion of how Mac performance compares to Windows, the performance here does at least show that Intel Macs can be very competitive with Windows systems, and at times even faster.

Cinebench R10 test
(Longer bars indicate better performance)
Rendering Multiple CPUs  
Rendering Single CPU  
Apple Mac OS X 10.4.10
16,853 
2,840 
Apple Boot Camp 1.3 Beta
15,692 
2,672 
VMware Fusion 1.0
7,184 
2,442 
CodeWeavers CrossOver Mac 6.1.0
N/A
2,567 
Parallels Desktop 3.0 for Mac
N/A
2,425 

Maxon's Cinebench R10 test is a CPU-intensive test that renders a 3D photo-realistic scene. The same scene is rendered twice--once using only a single CPU core, the second time using all available processor cores. As Fusion is the only virtualization application here with support for more than processing core, it was the only virtualization application able to complete the multiple CPU-based render. As for single-core performance, Fusion and Parallels put in nearly identical performance. Interestingly, this was the only one of our tests that CrossOver Mac could run. Its single-core performance on this test was marginally better than the other two virtualization applications.

Quake 4 test (in fps)
(Longer bars indicate better performance)
Apple Boot Camp 1.3 Beta
97.2 
Apple Mac OS X 10.4.10
95.6 
Parallels Desktop 3.0 for Mac
27.2 
Note: Tested with Quake 1.2 patch

The only virtualization software we could get to run Quake 4 was Parallels. We used somewhat conservative settings by today' standards (1,024x768, High Quality, 4x antialiasing, 8x anisotropic filtering) and saw frame rates that were barely playable. Lower quality settings should show higher, more playable frame rates, but wouldn't look as good. However, we were pleasantly surprised to find that we could get a Windows-based game to run at all in virtualization software! The developers of the virtualization apps promise improved compatibility with Windows games in the near future.

From a pure performance perspective, Fusion proves to be a faster performing platform than Parallels. Additionally, in our anecdotal hands-on testing we found Fusion to be somewhat more stable than Parallels--which is not necessarily a fair comparison, as we've spent more time using the shipping version of Parallels than we have Fusion. Usability between Fusion and Parallels also differs slightly, and personal preference will likely depend on individual tastes--I find Parallels a bit more intuitive than Fusion, but this could also be a function of my having spent more time with Parallels. CrossOver still has a lot of catching up to do, but the folks over at CodeWeavers appear to be making progress all the time. To date, mainstream gaming on the Mac still has only one significant outlet: using a native Windows install via Boot Camp. All that said, if you all you need to do is run more pedestrian Windows applications on your Mac, such as Outlook or Excel, you are not likely to notice significant performance differences between any of the virtualization options mentioned here--for less resource-intensive applications, the performance of any of these virtulization applications is probably fast enough.

Find out more about how we test systems.

System configuration:

Apple Mac Pro
2x 2.66GHz Intel Xeon 5355 CPUs (8 total cores); 2.048MB DDR2 FB-SDRAM 667MHz memory; 512MB ATI Radeon X1900 graphics card; 500GB Seagate 7,200rpm SATA/150 hard drive

 

Recent posts from Crave:

Add a Comment (Log in or register) 32 comments (Page 1 of 3)
Bootcamp isn't the easiest...
by getwired August 16, 2007 12:34 PM PDT
"Perhaps the easiest and most common way to run Windows on a Mac today is with Apple's own Boot Camp Public Beta." While Apple has done an admirable job of making BootCamp "Applish", and it is remarkably easy to install and use, I'd say using one of the virtualization products is truly easier to both begin using and to actually use day to day for the average new user. They don't require you to reboot, and don't require you to "leave the Mac". Filesharing between OS's for example can be quite painful with BootCamp, versus ridiculously easy with both emulation apps. Fusion, which I've used for months, is amazingly simple to begin using (admittedly, I'm a Windows/VMware Workstation user for quite some time) and makes interactions between OS X and Windows remarkably simple (copy and paste, dragging files in/out, etc). I do use BootCamp as well - but primarily only to play Flight Simulator X on Vista - since no emulator has robust enough DirectX 10 (or even 9) support to do much gaming while in OS X and virtualized.
Reply to this comment
Why bother? Run Windows on Windows.
by WJeansonne August 16, 2007 1:59 PM PDT
Does anyone but me get the joke here? If Mac's are so great, why bother using this sort of software?? LOL!
Reply to this comment View all 2 replies
Universal Machine
by theveggiedude August 16, 2007 2:25 PM PDT
Let's face it - giving people choice is always good. Why buy a Dell when a Mac gives you two machines in one? Mac has become a Universal Machine - the holy grail of computing. It makes UNIX cool.
Reply to this comment
How About Compatibility?
by JonWilber August 16, 2007 2:25 PM PDT
It is all well & good that one or other of the VM applications is faster but will it run the software? I have yet to find a user that will state that any of the VM products (other than Bootcamp and even then it is a "it should") will run Delorme's Street Atlas combined with a USB GPS receiver. Does anyone know for sure if Fusion, Parallels, or CrossOver will run Street Atlas?
Reply to this comment
Wrong software for test
by rockaway61 August 16, 2007 5:36 PM PDT
How about some benchmarks for software that you can't get on a Mac. I already have Photoshop on my Mac. I only use virtualization for Windows apps like QuickBooks Premier 2007, which doesn't have a mac version.
Reply to this comment
Too short, too few tests
by odysseuscm August 16, 2007 11:40 PM PDT
Quake4 ist not a DirectX Game but an OpenGL game. Most OpenGL games run quite fine under Parallels. With DirectX it's a different story. I'd have liked to see more single CPU comparisions between Fusion and Parallels (which will probably catch up with multi CPU sooner or later) to get a better picture of memory, disk and network throughput regardless of raw CPU power. That would also be more conclusive for the average user who doesn't have 8 cores... I did some synthetic benchmarks in SiSoft's Sandra and Parallels was about 3 times faster than Fusion there... that musn't apply to "real life", but it's inteesting. Also apart from the quote *claiming* that fusion has less overhead: what's really interesting is, how the performance impact is on OS X while using the VM, after all you want to use both at the same time. For my taste your comparison is too shallow.
Reply to this comment View reply
Practice Safe Computing!
by Gatesbasher August 16, 2007 11:53 PM PDT
I'll stick with Boot Camp. I want Windows COMPLETELY segregated from the rest of my hard drive. When one of the 100 or 200 Thousand viruses to which Windows is heir invades that partition, I can just wipe it and start over. Or better yet: Just say no to Windows!
Reply to this comment View all 2 replies
Linux performance
by macinchem August 17, 2007 1:09 AM PDT
Has anyone compared Linux performance, I have a couple of scientific applications I'd like to compare.
Reply to this comment
Maximum Irrelevancy?
by Walt French August 18, 2007 5:59 PM PDT
OK, what fraction of Mac users have 8-core machines? For the "rest of us," how about showing how important it is for somewhat time-consuming Win apps to have access while the one- or two-CPU Mac is handling ordinary functions (e.g., networking, listening to Sirius, ...) in the background? It looks like you chose to test on an 8-core box to maximize the differences between the two VM systems, which maximizes a VMWare feature, and very little else. How about choosing the tests to maximize the relevance to users?
Reply to this comment
ZOMG, the interweb is c00l
by fattymelt August 18, 2007 6:56 PM PDT
@WJeansonne - Your comment is poorly thought out. If Windows was so great, no one would be buying Macs. If Windows was so great, Apple wouldn't be /gaining/ market share. People aren't buying Mac so that they can run Windows. People are buying Macs because they are superior. However, they may need to run some applications that are unavailable for OSX.
Reply to this comment
1 | 2 | 3 | Next 10 Comments >>
Powered by Jive Software