Quest Benchmark Factory

Benchmark Factory for Databases is a performance and code scalability testing tool that simulates users and transactions on a database and replays a production or synthetic workload in non-production environments. This enables organizations to validate database scalability as user loads increase, application changes are made, and platform changes are implemented. Benchmark Factory is available for Oracle, SQL Server, DB2, Sybase, MySQL, and other databases via ODBC and Native connectivity.

Benchmark Factory provides many tests you can run, and has a very nice and customizable metric reporting engine. We decided to run the AS3AP test, and the Scalable Hardware CPU, Reads, and Mixed tests. Here is what Quest's help file says about these tests:

AS3AP

The AS3AP benchmark is an American National Standards Institute (ANSI) Structured Query Language (SQL) relational database benchmark. The AS3AP benchmark provides the following features:
  • Tests database processing power
  • Built-in scalability and portability that tests a broad range of database systems
  • Minimizes effort in implementing and running benchmark tests
  • Provides a uniform metric and straightforward interpretation of benchmark results
Systems tested with the AS3AP benchmark must support common data types and provide a complete relational interface with basic integrity, consistency, and recovery mechanisms. The AS3AP tests systems ranging from a single-user microcomputer Database Management System (DBMS) to a high-performance parallel or distributed database.

Scalable Hardware

The Scalable Hardware benchmark measures relational database systems. This benchmark is a subset of the AS3AP benchmark and tests the following:
  • CPU
  • Disk
  • Network
  • Any combination of the above three entities
We run three iterations of each load point, and then average the results. We also monitor deviations to ensure they are within an acceptable range. We like to see a max deviation of +/- 3%.

Benchmarking Low Voltage Test Setup
Comments Locked

27 Comments

View All Comments

  • sc3252 - Tuesday, July 17, 2007 - link

    Even though this is for low power, it is nice to see amd win some benchmarks for a change.

    If I was building a small server it looks like the logical choice would be AMD, for now. Of course if I was building a small server it probably would be a x2 3600 or a sempron, unless you are actually running a database you are never going to be at high utilization. Running mythtv, samba and a mail client would never require the amount of power either the core 2 dou have or the amd 64 cpu's.
  • TA152H - Tuesday, July 17, 2007 - link

    I run one server on a K6-III+ running at 250 MHz, and it never gets pegged. A lot of servers are just file servers, and you can get by with an underclocked processor easily, since the main performance characteristics come from I/O devices. So, a lot of it depends on the workload.

    I was looking again at some mini-ITX stuff lately, and VIA has some really impressive processors now too. A year or so ago, I bought some 800 MHZ Eden processor that took less than 8 watts, but it proved worthless since the performance was less than a K6-III+ using the same amount of power (although the platform was more stable). So, I wasn't too impressed with it. However, now they have a 1.5 GHz version with the same power use, and an amazing 1 GHz version using only 3.5 watts. With a hard disk (notebook) and some memory, you can get power draw of considerably under 30 watts with a whole system, and the performance is fine for a lot of stuff. It looks like they are finally beating the older technology and offer interesting products for those who do not need a lot of processing power, and are very interested in saving power. The bad thing is though, with 2.5 inch drives, even your I/O performance is not going to be very good, but you can always use these in other cases too if you need I/O performance. How reliable they are is unknown to me, and of course if you have critical data, they are a bad choice because unless you use the slot, you don't have RAID either. But, in limited situations they are worth considering.

    They have released a new chipset, the CN896, that looks interesting, but so far I have not seen any motherboards based on it available. It's irritating that they release these things so far in advance of any products, but it looks to be a pretty impressive chipset. The only thing I don't like is that it doesn't support DX10, and I don't know why anyone would settle for DX9 anymore, especially with Vista becoming the standard. So, it's really just for Vista Basic, which kind of sucks. But, I guess this type of compromise is necessary in this power envelope.
  • Spoelie - Tuesday, July 17, 2007 - link

    Your last paragraph doesn't make much sense.
    Vista aero only needs a dx9 card, it does not require dx10. The only thing that requires dx10 are dx10 games, and performance in those applications has been utterly abysmal on everything but the highest end cards. Integrated dx10 graphics at this moment would be completely useless.

    So it is in fact the other way around, why would anyone pay extra for unusable features at this moment and the foreseeable future.
  • TA152H - Tuesday, July 17, 2007 - link

    Well, Microsoft only validates Vista Basic for that chipset, not Aero.

    Abysmal is a subjective term. Sure, you have the dweebs that want the highest settings on everything, and this is true, an IGP wouldn't be a great choice. But, you're obviously ignorant to the fact DirectX is not just for games, so I'm not sure what else to say to you. Learn something, OK? I can help you, here is a nice interview with the a guy from Microsoft that was quite involved with DirectX 10. Maybe you'll believe him when he says it's for more people than just gamers?

    http://www.bootdaily.com/index.php?option=com_cont...">http://www.bootdaily.com/index.php?opti...Itemid=6...

    DirectX 10 is useful in a lot of ways, not just idiot kids shooting aliens. There are now low cost cards supporting this, and only a fool would buy old technology for the same price, unless they like obsolescence. The problem with the CN896 is it doesn't support DirectX 10, or Aero, and that's just a bad thing for something being released right now. On top of this, it isn't even available in motherboards. Compare this with the 965, which is a lot older, and still has more features. I think most people buying a machine, even a low power one, would prefer to run Aero on their machine, Vista isn't quite Vista without it. But, maybe they couldn't do it with the power envelope, and decided it would have to wait until they could. So, they just support the obsolete and inferior DX9. It's understandable, but I don't really like it. Not for something totally new. I don't think anyone should be releasing DX9 only hardware anymore. The future is clear, and it's DX10 and what succeeds it.
  • Spoelie - Tuesday, July 17, 2007 - link

    The fact still remains it only is being used in games for the moment, try to find a CAD/CAM solution that utilizes dx10 features, or an announcement for one in the coming year.

    If ms doesn't certify aero, then you have a point, but dx10 is a non-issue.

    And yes, performance is non-subjectively abysmal. We're not talking about ultra high settings, we're talking about 1024x768 with no AA or AF whatsoever. That is below the standard LCD resolution... You need a high end chip to run it, the mainstream line of both ati and nvidia do not cut it, let alone any hypothetical dx10 integrated graphics.

    Any person would buy older technology if it provides a better value, and that is what most current low-end to mid-range dx9 solutions provide, better performance for a better price. Maybe in a year or so the situation will have changed, but now and in the near future dx10 is next to useless on anything but the high end.
  • ButterFlyEffect78 - Tuesday, July 17, 2007 - link

    What does AMD stand for? Advanced micro systems. If you apply this logic it should be very self-explanatory to this subject or topic.
  • Gul Westfale - Tuesday, July 17, 2007 - link

    it's "than", not "then". also, AMD stands for Advanced Micro Devices, not Systems.

    as for the article, it is certainly interesting to learn something about servers, but the actual consumption of a new system is certainly going to be less important than things like future upgradeability, performance, and reliability. in that sense, the rather small differences between AMD and intel do not matter so much, they are not the most important deciding factor when it comes to making a buying decision. also, there are so many different system configurations out there that with a few different components the numbers here would be very different.

    so, like i said, interesting read but we should not draw too many conclusions from it.

Log in

Don't have an account? Sign up now