AMD Opteron 248 vs. Intel Xeon 2.8: 2-way Web Servers go Head to Head
by Anand Lal Shimpi & Jason Clark on December 17, 2003 9:15 AM EST- Posted in
- IT Computing
The Database Server Bottleneck
There is almost always a bottleneck in hardware tests — ours was the database. The majority of web applications require a database server for data retrieval and storage. We used Microsoft SQL Server 2000 as our back-end database server for these tests. Prior web server tests that we’ve performed have been on single or dual CPU servers and have reflected accurately the performance of the hardware on which the tests were run. When we started running the tests on the servers, specifically the Opteron, we found that the tests had hit a wall and were not reflecting accurately the performance of the hardware. The issue was I/O, the largest bottleneck in any database server. We were putting so much load on the servers to get the CPU working hard that the database server I/O was maxed out. This, of course, caused the web tests to hit a performance plateau as each request run against the web application server has to wait for the database to complete its data transaction.So how do you get rid of an I/O bottleneck? Well, if you have the resources, you throw more disks at the problem. The problem with that solution is it gets expensive to have massive RAID arrays that are just used for testing purposes and have to be continually upgraded as hardware improves. To get around this expensive issue, we used the fastest and most inexpensive storage medium you can — memory. We created a RAMDISK and put the database on that new drive. RAMDISK has been around for years and has matured over the years to solutions that can be formatted as NTFS drives, which are easier to manage. Effectively, a RAMDISK partitions a portion of physical memory and allows it to be formatted as a drive letter that is available to the operating system.
Once we solved our bottleneck, the testing began and as you read on, you’ll see how the different platforms performed while under real-world web application load.
Facebook
Twitter
RSS
43 Comments
View All Comments
Zuni - Friday, December 19, 2003 - link
"That entirely depends on the application being tested, and the load on the server, and the memory on the server. If you're calling the same 10 files over and over again then they'll likely be cached. If you have a true application where someone hits a page and creates a 600 megabyte recordset"Sure I/O usage would depend on the type of web application being tested. However, most web applications feeding off a database backend would not pull 600 mb recordsets :) Pulling such record sets would be a fairly silly thing to do in most any circumstance. Our test was absolutely not hitting I/O on the webservers, the template cache is more than enough to hold the templates being tested. In fact as I'm writing this I pulled the disk time on one of Anands busiest web servers and its showing less than 1% of disk time :) If an application is well written, and the webserver has a decent amount of memory (most do) then I/O should not be a factor in a run of the mill application such as FuseTalk.
"This smacks of rhetoric. ASP and ASP.NET are also 'very good reference points' as enterprise level application environments. "
Not at all, although I can see how you could draw that conclusion. FuseTalk is currently being ported to .NET, once we have a stable working version or another app we deem worthy of a real world test we’d be glad to run tests on .NET as well. I quite like the platform. This is just the beginning of our server testing and developing real world tests. It takes time, and a lot of it to develop and run theses tests.
Cheers
Zuni - Friday, December 19, 2003 - link
Trog, again RAMDISK was used so that the application server was free to tax the cpus as much as possible instead of waiting for data requests from the database. This has nothing to do with normal or abnormal. We aren’t testing the database here we’re testing the application server, and we wanted no bottlenecks on the backend to cause any skewing on the front end. We don’t have fibre channel backend database hardware just laying around, thus a RAMDISK was used to remove the I/O bottleneck.Again, the database is not in question here or being tested so its irrelevant.
Cheers.
Falco. - Friday, December 19, 2003 - link
" 1) disk subsystem is irrelevant in a web application driven by a database backend. Most any web application server caches the templates in memory so I/O is not a factor. However, each web machine had a 36gb u320 scsi drive. "That entirely depends on the application being tested, and the load on the server, and the memory on the server. If you're calling the same 10 files over and over again then they'll likely be cached. If you have a true application where someone hits a page and creates a 600 megabyte recordset, your disk IO is gonna come in handy right quick. Saying "I/O is not a factor" is misleading.
However, I'm glad they responed the way they did, because if you hvae a testbed that starts to swap or spin out to disk, that doesn't test the CPU as much. I was expecting they'd come back and say they used a test set that was cached compiled templates (which is a good thing). My question was aimed at eliminating it as a possible variable.
" 2) Not sure I understand your friends point really... Of course ColdFusion tests performance, we used FuseTalk as the web application and ColdFusion as the web application server. "
That's not entirely what I meant. Cold Fusion is a middleware layer. It's not a "web server" as per the title of the review (Cold Fusion actually can include its own webserver if you want it to).
I found the title of the review to be somewhat misleading, since it implied they were testing a web server and not an application server.
" ColdFusion in its current version is a J2EE server which allows you to write in the ColdFusion language. After its compiled its the same compiled byte code as any J2EE app. "
Correct; it's an interpreter (CF) running on an interpreted language (Java). It's a good test in that it has multiple subsystems communicating together on top of having to handle actual web serving such as setting up sessions and opening and closing connections.
" We may include .NET in the future, but J2EE is a very good reference point as its a enterprise level application environment. "
This smacks of rhetoric. ASP and ASP.NET are also 'very good reference points' as enterprise level application environments. A whole huge great pile of systems run ASP. It struck me that it would have been worthwhile to round out the review by testing a number of common web application server configurations:
* Pure J2EE
* ColdFusion
* ASP
* ASP.NET
* PHP (although this would not test IIS, as most PHP systems drive it with Apache -- mind you, a test with Apache would be equally interesting)
" Sustained load throughout the test keeping the CPU working in the 80-100% bracket. We measure performance by how long the cpu took to run each code template from start to end. (this was all detailed in the review btw.) "
This part I understood and took for granted. Interestingly if they had sustained 80-100% CPU load, this is an added shine for the AMD parts, because past 70% load on your webserver you should be adding another box or upgrading your CPUs.
TrogdorJW - Thursday, December 18, 2003 - link
Hey Zuni... who are you? Anand? Or "just" an Anandtech employee (or reviewer or whatever)? Anyway, let me clarify:"The entire reason we used a RAMDISK was to ISOLATE the cpu not to hold it back by something else."
Exactly. Ergo, without the RAMDISK holding the database, file I/O can be a much bigger factor than CPU speed. Yes, you can get bigger and faster disk subsystems, but that's not always the way it's done. Anyway, it *sounds* to me like a more "normal" setup on the database side would end up being a major bottleneck on the overall throughput, so that either of these systems is overkill. You need a DB system with a very high I/O rate first, and then looking at these faster CPUs comes second.
On a separate thought, are there going to be some benchmarks *without* using a separate DB server (which used a RAMDISK)? I understand that your site may not run that way, but the vast majority of web servers do, I believe. If you have one server running both the database backend as well as the application services, what do the benchmarks look like? That would be something to look at, surely?
Falco. - Thursday, December 18, 2003 - link
thanks Zuni, i'll let him know and post back any replies he may haveZuni - Thursday, December 18, 2003 - link
Falco,1) disk subsystem is irrelevant in a web application driven by a database backend. Most any web application server caches the templates in memory so I/O is not a factor. However, each web machine had a 36gb u320 scsi drive.
2) Not sure I understand your friends point really... Of course ColdFusion tests performance, we used FuseTalk as the web application and ColdFusion as the web application server. ColdFusion in its current version is a J2EE server which allows you to write in the ColdFusion language. After its compiled its the same compiled byte code as any J2EE app. We may include .NET in the future, but J2EE is a very good reference point as its a enterprise level application environment.
3) Sustained load throughout the test keeping the CPU working in the 80-100% bracket. We measure performance by how long the cpu took to run each code template from start to end. (this was all detailed in the review btw.)
Falco. - Thursday, December 18, 2003 - link
a friend has some questions about this review :Did they get into details about the disk subsystems on each of the web servers? I know they tried to keep the database in ramdisk (even though SQL server will do that anyway if the database is small enough).
What sort of web application were they running? I find it curious they were using a Coldfusion application as well, since that doesn't test web response as much as the cold fusion runtime services; they should have used an ASP or ASP.NET application test, at least as an additional data point.
What did they do to test web response time? Sustained load? stressed load? burst load?
thats what hes asking in a forum i frequent...
Icewind - Thursday, December 18, 2003 - link
Man, how the hell do you learn and keep up with all this server technology, both hardware and software? Im having a hardtime just keeping desktop technology and changes....Zuni - Thursday, December 18, 2003 - link
skiboysteve, because thats what was sent with the systems :) 8GB of it.Zuni - Thursday, December 18, 2003 - link
Visual, the db server didnt need hardly any ram the database was barely 50 MB :) Again the point of the article was to eliminate the db as anything but a boundless data resorce so the web applications servers could process pages as fast as possible, keeping the cpu taxed. This was a 32 bit test, OS etc. We're trying to get ahold of a 64 JVM and we're looking at some UNIX based 64 bit tests for another article.Cheers!