Powermac G5 SPEC numbers cooked?

I'd say that like with most things on the internet, this article holds no water. The machine is not even out yet and someone is saying it has faked results. I am not saying it is wrong, just that you can produce a lot of fancy looking numbers and "prove" anything wrong these days and with a website people will believe everything written.

During the Keynote a guy from Adobe said they had been testing it and that it is faster than the PC equivalent.

I'll hold my thoughts on the speed until I can lay my hands on it myself.

Though, even then, at these speeds it is all just user preference.
 
He picks only the benchmarks that make the G5 look bad, and completely ignores the ones that show it kicking the P4's a$$.

I guess he didn't watch the keynote stream where not only did Photoshop take 2 times longer to do the same tasks as the dual G5, but the G5 blew it away in 4 or 5 other programs as well.
Mathamatica, Lightwave(? can't remember exactly what prog), some PRO music editing softward(G4 had one CPU at 25% usage and P4 Xeon had 100% CPU usage for the same tasks), and a few other programs all blew away the P4/Xeon.

Photoshop launched in less than a second on the G5(amazing). That might have something to do with the Serial ATA, but the G5s would make it faster anyways.

This was the best keynote I've seen in a VERY long time! Now I have to come up with ~$5000 Canadian to buy the dual 2.0GHz :D
 
his article is wierd, he talks about the cheats. and then makes some tables out of who knows what data (presumably from the official pdf from veritest) and procedes to declare that the g5 is not the fastest personal computer in the world.

not very convincing.
 
According to an article on macbidouille, what's fake is not the benchmark but the article the guy wrote is totally wrong
 
When interpreting these FP results, let us keep in mind that most people use Integer (not FP) most of the time. Therefore, integer results (SPECint) are much more important than floating-point results (SPECfp). In other words, most people should ignore floating-point results because they do not use floating-point anyway (or not much).

SPECfp_base2000 is a single-processor test, so in the following results, where the computer has a second processor, it is either disabled or not used.

Oops! Those lines just made the article's credibility drop to zero :p
I don't expect the average user to know the difference between floating-point and integer, but it should be elementary for a person like this, being supposed to know so much about processors and benchmarking :p
 
Well some of the stuff he said was a bit off, but the fact is that much of what he said was right on. When I was listening to the keynote, I was bemused by the fact that the benchmark numbers they produced were using gcc. An odd choice on the Wintel side since, unless you're using Linux, a user is very unlikely to be running anything that is based on gcc.

Fact is that Apple definitely didn't go through any pains to make sure that there was a "level" playing field in their benchmarking, which really makes the benchmarks themselves a bit worthless (and some would say that almost any benchmarks are worthless regardless of who runs them). If you look at Apples published numbers vs Dell published numbers, the Dell indeed comes out ahead in several of the areas that Apple claims superiority in.

His statement about floating point is generally true. QuartzX might make more use of FP than any current gui, but that is a fairly new phenomenon. FP is becoming more and more important, but for your average web browsing and email user, it really doesn't mean much.

I'd say that like with most things on the internet, this article holds no water. The machine is not even out yet and someone is saying it has faked results. I am not saying it is wrong, just that you can produce a lot of fancy looking numbers and "prove" anything wrong these days and with a website people will believe everything written.

He uses information from SPEC and Apple themselves. Not conjecture, just the numbers. And keep in mind that the argument for using pre-release hardware makes Apple look bad for crowing about pre-production superiority. He is addressing Apples claims.

He picks only the benchmarks that make the G5 look bad, and completely ignores the ones that show it kicking the P4's a$$.

You missed his point. His big rant is that the G5/970 isn't as fast as Apple claims it to be, not that it isn't faster than a P4 at all.

I guess he didn't watch the keynote stream where not only did Photoshop take 2 times longer to do the same tasks as the dual G5, but the G5 blew it away in 4 or 5 other programs as well.
Mathamatica, Lightwave(? can't remember exactly what prog), some PRO music editing softward(G4 had one CPU at 25% usage and P4 Xeon had 100% CPU usage for the same tasks), and a few other programs all blew away the P4/Xeon.

Photoshop launched in less than a second on the G5(amazing). That might have something to do with the Serial ATA, but the G5s would make it faster anyways.

Quick joke, some of you have heard this, but I think its very appropriate to the above statement specifically and this thread in general:

A software product manager is driving home one day. He's daydreaming about the big presentation he has to give the next day and misses the sharp corner, running into a tree, and dies. Next thing he knows he's at the pearly gates (notice little 'g' here) in front of St Peter. St Peter asks him his name and looks puzzled. He says to the man "I'm so sorry, I don't know how this happened, but I have no record of you." "I don't know if I'm supposed to allow you in or send you down to, well, you know". "Tell you what I'll do, I'll let you check out both places, and I'll let you decide". OK says the guy. First he checks out heaven. There's soft classical music playing, people are resting on clouds and reading the classics. Everyone talks in whispers and is very polite. "Interesting" he says to himself. Next he goes to hell. There's loud rock and roll music blaring. Guys are chasing half naked woman around with giant beers in their hands. "VERY interesting" he says to himself. He goes back before St. Peter and gives his decision. "Heaven was very nice" he said, "but hell seems like it's more my speed, so I choose hell". "OK" says St Peter and he is cast down to hell. When he gets there, instead of what he saw before there is only flames and people screaming in eternal torment. The man is very confused when he is met by Satan. "What happened to the place I saw before?" asked the man. "Oh that" said Satan, "that was just the demo" [badda bing] ::evil:: ;)

Apple's just playing the game, really nothing to see and really not much to discuss. The new machines represent a large leap forward in performance and gets the Mac much closer (if not past) PC's in performance. Exactly how much is a bit irrelevant to most people except those who seem to define their own self worth by such things ("my video card does .43 more fps in Quake than yours, I'm l33t" yadda yadda).
 
Originally posted by malexgreen
Let talk about this: http://www.haxial.com/spls-soapbox/apple-powermac-G5/
"You fear in others what you despise in yourself."

What gripes my butt in all of this is that I have been around long enough to see benchmarks abused and mutilated by the Intel/MS-DOS/Windows crowd since they were trying to upseat CP/M-80. This is a crowd that is absolutely fixated on the most minute and inconsequential speed advantage. Now that we have Macintoshes that will kick their machines' butts and take names, they are besides themselves. I am buying one of the new 2 GHz dualies. Wintel users can weep and wail and knash their teeth and scream to the top of their lungs that Apple lied. I will do my job on my new machine while they suck on warm beer seasoned by the salt from their tears.
 
As I said in the other similar post:

Nah! That guy and the majority of Wintel users are trolling BIG way! :p

I will take my chance with Jobs, Schiller, eMagic, Mathematice and Luxology guys anyday before I will listen to ANY Wintel FUD... :p ::ha::

Damn even Adobe's CEO (yeah that Adobe with the PC Preferred fiasco) said that G5 is a monster computer! Well, not exactly in those words! :rolleyes:

Boom! :D ;) :)
 
from macosrumors.com :

For one thing, benchmarks should always be taken with a grain of salt, particularly those which span platforms such as Apple's G5 vs. Pentium/Xeon comparisons. They often don't reflect real-world performance -- which is why Apple has presented application comparisons for Photoshop, QuarkXPress, Quake 3, Mathematica, and several other popular apps.

The result? The G5 puts the smack down on the competition. Hard.

Regardless of whether the SPEC numbers may have been boosted on the G5 because Apple used an optimized compiler on the PMG5s and didn't do quite as good a job of optimizing on the Windows machines, the huge performance advantage seen in real-world applications makes this all but totally irrelevant. Not to mention -- many well-known developers have pointed out that benchmarks like SPEC are focused heavily in the CPU(s), and don't take into account very important real-world factors like the G5's 800MHz-2x1GHz frontside busses, Serial ATA hard disks, built-in Gigabit Ethernet, HyperTransport interconnects, et cetera.
 
I agree with TsizKeik. Lets see what the G5 does in the real world when it comes out and people start using it. Benchmarks, especially between Apple and Intel is like comparing apples to oranges.
 
Thank you powermac...

Marketing is something that involves fibbing, and fibbing hard. I never go by benchmarks, they just don't vibe most of the time.

This guy is simply pointing out what we all know about marketing, these companies, even our beloved Apple, will hype. Always they will hype. You need to take it all with a grain of salt. If you don't, you'll end up with egg on your face, and a foot in your mouth...

(From experience ;) )
 
i also take every benchmark lightly, no matter who comes out on top.
as i understand what this guy is doing, i see that he's eating up this newfound fame (slashdotted, etc.) with all his cute little 'responses' to 'hate email' at the bottom. doing that really destroyed the point he was trying to make with me, and just showed him as immature-- making fun of peoples' spelling errors as opposed to answering questions, etc.
too bad about the fame of this article-- a lot of people tend to agree with anyone anti-mac without further proof. i'll be hearing it from my pc using buds now, i'm afraid =/.
 
Originally posted by malexgreen
Let talk about this: http://www.haxial.com/spls-soapbox/apple-powermac-G5/

I just read the veritest report and haxial's page. IMHO, Apple royally screwed up the SPEC numbers. Their SPEC numbers are invalid.

The WWDC Keynote "bakeoff" using several real world apps, however was telling.

What I'd like to see Apple do is pair the Intel Compiler/with a 3.2GHz/800MHz FSB system, with a IBM Metroworks/G5 dual 2.0GHz system and rerun the SPEC numbers again. At least then the idea that one machine was more optimized than the other would be an invalid point.

Regardless, I didn't by my Pbook because it was the fastest, I bought it because of the other nice featurs, like MacOSX.
 
Originally posted by malexgreen
I just read the veritest report and haxial's page. IMHO, Apple royally screwed up the SPEC numbers. Their SPEC numbers are invalid.
That is a strong statement are you saying that they lied about the numbers? Also you do know that many of the "facts" reported by haxial are complete bull? They actually ran the most useful test for me since I would be compiling with GCC on wither of these machines.

The WWDC Keynote "bakeoff" using several real world apps, however was telling.
That also showed off the massive bandwidth of that 1GHz bus. which is where the real win is. The Intel chips could not get the data fast enough to keep up even when they did have better SPEC scores.

What I'd like to see Apple do is pair the Intel Compiler/with a 3.2GHz/800MHz FSB system, with a IBM Metroworks/G5 dual 2.0GHz system and rerun the SPEC numbers again. At least then the idea that one machine was more optimized than the other would be an invalid point.
There are two problems with what you are suggesting. First, the tests were done with was was the best machine available. Apple did get a 40 day lead that way but who says Dell will have one of those shipping then.

The second problem is the fallacy of the magical vendor's compiler allowing us to compare equally optimized systems. That is a bunch of hooey and I have written a couple of compilers in my day. The truth of the matter is that vendor compilers are usually tailored to making good scores on these types of benchmarks. As a result they include a whole host of "optimizations" which are only really helpful on the benchmarks. In normal code they either have no benificial effect or the criteria for when they are applicable is so narrow they never get used.

On top of all that how much of the software beside the SPEC benchmarks are you ever going to encounter in real life which is compiled using the chip maker's compiler on these two platforms? Next to nil - Now Suns, AIX and that lot are a bit different as they do use the chipmaker's compiler.
 
Everyone knows that the SPEC benchmark is optimized so it makes Intel processors look as best as they can. IBM, Motorola, Sun and others do the same thing. It's just the way things work.

A better benchmark test is how well the computer works in the real world. Which processor will slice 90 minutes off a two hour rendering job. That's what I want to know and everyone else doing video, 3d, big Photoshop jobs, that sort of thing.

I could care less if Apple is the fastest or second fastest. I am more impressed with speed, stability and workflow.

Show me where Intel is twice as fast as a G5 in these areas and I might take a look.
 
Back
Top