Performance Testing

When we first mentioned plans for performance testing of netbooks we encountered stiff resistance from some manufacturers. They expressed concern that performance comparisons defeated the whole idea of a netbook, which is a cheap portable computer based on cheaper processors that provided competent performance for basic computing tasks like web search and writing. While we agree that it would be a mistake to evaluate netbooks on performance alone, the facts are that netbook performance is increasing rapidly, as is the definition of what represents a "basic" computing task.

If the first netbooks represent all that is needed in a netbook there would be no reason at all for any future models. In fact people are doing more and more with netbooks - and the more that is demanded of these little computers the more that is demanded in raw computing power. The caveat, of course, is that it is unfair to compare the performance of a $300 to $400 netbook to a $2000 state-of-the-art notebook. They are aimed at different users and tasks. However, with the growth of computing power in recent years it is truly remarkable how very much the tiny netbook is actually able to do today.

Determining the best method for testing netbook performance proved quite a challenge. Of the readily available test suites PCMark Vantage seemed the ideal tool for testing overall computing capabilities. The problem is that Vantage only runs on Windows Vista and none of the current netbooks available for testing even offer Vista as an OS option. In addition Vantage requires a 1280x1024 minimum screen configuration which is far beyond resolution capabilities of current netbooks, so we would need to attach an external display.

Serious consideration was given to testing netbooks with Linux or a Linux variant OS like Ubuntu. In staff discussions at AnandTech we looked at benchmark issues with testing Linux notebooks, and we also considered the fact that all of the netbooks in house for testing had arrived with the Windows XP Home OS installed.  We do plan to compare "Linux" and Microsoft OS options on netbooks in a future article.

With Windows XP Home already installed on all five netbooks that became our current OS choice for performance comparisons. Even that choice presents testing problems. PCMark05 runs fine on Windows XP, but the test suite still requires a minimum screen resolution of 1024x768 and several test modules fail at any lower resolution. The best current netbooks can do is 1024x600. After considering various options PCMark05 testing was standardized on an external monitor fed by the netbook being tested at a 1024x768 resolution. On some netbooks the screen output at 1024x768 is hardly ideal, but using a standard resolution removed a variable from the performance testing.

With the continuing growth in computing power in netbooks we fully expect to add additional tests to our netbook test suite. It is common knowledge that current netbooks can handle Flash and streaming videos present on the web without much difficulty. We confirmed this in our own internal testing. We also concur with other reports that current netbooks can handle up to 720p video playback with relative ease. However 1080p and BD playback is definitely beyond current capabilities.

In the future there is no doubt netbooks will be able to smoothly handle higher resolution video playback, Blu-ray, and perhaps even mid-level gaming. As those capabilities appear we will add relevant tests to the netbook test suite.

Battery Life Comparison Performance Comparison
Comments Locked

42 Comments

View All Comments

  • JackPack - Friday, July 17, 2009 - link

    Will AnandTech consider updating this article with the 1005HA Premium/Value?

    Honestly, this article feels a little outdated given the 1000HE is EOL.
  • Wesley Fink - Friday, July 17, 2009 - link

    We mentioned in the article that the 1005HA was replacing the 1000HE, which might make the 1005HE a bigger bargain as it is closed out. We have received a 1005HA in the lab for testing.
  • Wesley Fink - Friday, July 17, 2009 - link

    I think I created a new Asus model in the reply. The model that may see close-out pricing is the 1000HE.
  • goinginstyle - Friday, July 17, 2009 - link

    Is there any reason why wireless tests were not run or performance not even mentioned. I have to depend on a wireless connection and without a DVD drive, it is really important to know how well the wireless setup works and if there are any compatibility problems hooking into a router.
    Maybe it was on the hands on pages but my eyes hurt after looking at the background colors on the pics. It could just be me but red and pink just seems out of character around here.
    Both units have what appears to be nice webcams and once again no mention on how well they work and the quality or if still pictures are a choice. Is the wireless connection or even the machine fast enough to handle skype video?
    I know these units are not built for gaming but can you run WoW or Company of Heroes on them and how well do they run the multitude of flash games, especially the card games.
    How good is the video quality on the VGA port and do the drivers offer support for widescreen formats or just 4:3. I guess I am frustrated as I am looking for a netbook and need more information than the asus, acer, hp, or msi websites provide.
    The battery tests were really good but what happens if i buy a usb DVD drive and rip a movie to the hard drive. Will the msi have enough juice to play a couple of movies on a plane ride. Maybe I need to spend more and get a 14" notebook but i was hoping to save some money here.
  • Wesley Fink - Friday, July 17, 2009 - link

    We found no issues at all connecting any of the 5 netbooks tested by wireless for our battery tests. All found the connection to our wireless router and held onto to it for the 2 to 8.5 hours until the battery was depleted. Testing wireless was not the point of the battery test, but it is some evidence of wireless stability.

    We do intend to add wireless connectivity testing to future netbook reviews. We are currently looking at several potential test methods.
  • TotalLamer - Friday, July 17, 2009 - link

    With netbooks getting larger, more capable, and more expensive... at what point does a netbook cease being a netbook, and simply becoming a notebook?
  • nafhan - Friday, July 17, 2009 - link

    I think hardware plays a big part in what defines a netbook. That said, I wouldn't pay more than $400 for a netbook.
    It would be nice if they would release a netbook based around a dual core atom processor (the 330). I'd gladly sacrifice a couple hours of battery life on the MSI lappy for an extra core.
  • The0ne - Friday, July 17, 2009 - link

    Definitely not over $400 as you could always find laptop deals for around the same price. For the same price the lack of CPU power is a huge drawback.
  • Wesley Fink - Friday, July 17, 2009 - link

    That is a very interesting suggestion. As I mentioned in the article we used an exterior monitor for the PCMark05 testing since a minimum 1024x768 was required to even run PCMark05. I was very surprised at how different the netbooks were in their output to the external monitor.

    As you say some are very clean and some are quite ragged. We will try to find a way to include VGA port output in future netbook testing.
  • sprockkets - Friday, July 17, 2009 - link

    Would like to see the HP mini 2140 though.

Log in

Don't have an account? Sign up now