German magazine "c't" f's up, Appleturns.com lays the smackdown

simX

simX

Unofficial Mac Genius
Well, well, the world turned upside down today. Appleturns.com actually did some REAL reporting.

It seemed like nobody mentioned it, but German magazine c't said that they benchmarked the 1 GHz Motorola PPC chip (they turned off the second processor in a dual GHz Mac somehow), and pitted it against a 1 GHz Pentium III, finding that they were comparable, and that the Pentium III even beat the socks off of the G4 in floating point operations, where the G4 is traditionally supposed to excel.

Well, they failed to mention that the SPECing software that they used IS NOT ALTIVEC OPTIMIZED.

Haha, what a joke. Of course a G4's not going to win when you cripple it by not utilizing a feature that makes the G4 the supercomputer that it is.

Further, appleturns.com notes, is the fact that the SPEC benchmarking software just spews a constant stream of operations through the processor, which biases performance to Intel processors. Why? Because the Intel's pipeline is longer but it has a faster clock speed, so that it can process operations faster. But the benchmarking software fails to take in to account "bubbling" or "branching", where the entire pipeline gets cleared of operations. This is more taxing to Intel's processors because the pipeline is much longer than the G4s (Pentium 4 = 20 pipeline stages, G4 [not Apollo] = 7 pipeline stages).

To quote directly from appleturns.com:

As you may recall from Jon Rubinstein's "Megahertz Myth" spiel, Intel's recent chips take a speed hit from the recurring need to clear and refill those extra-long pipelines due to incorrect predictive branching-- it's that whole "pipeline tax" thing. With the SPEC test, there are no data dependency bubbles, and therefore no pipeline tax, so Intel's chips perform better than they would in actual battle conditions.
Click to expand...

Sheesh. Some people (or magazines, such that it is) sure like to knock Apple.
 
the c't magazine sure doesn't want to knock apple. normally they praise apple for what they do. but everybody can make mistakes, so did they with this article about g4 and piii
 
Um, I think it'd be a pretty trivial question to ask. "Is this software Altivec optimized?"

c't magazine probably replied: "No? Let's do the test anyway."
 
Originally posted by Dradts
the c't magazine sure doesn't want to knock apple. normally they praise apple for what they do. but everybody can make mistakes, so did they with this article about g4 and piii
Click to expand...

yeah, right!:eek:
 
Funny, they don't mention that it wasn't optimized for ANY SIMD implementation (Altivec or SSE2) either. Even if it had been optimized AltiVec would probably have still lost. WHY? Because SPECfp uses double precision floating point numbers (which is required for many areas of scientific research, such as nuclear reaction simulation). Currently AltiVec only supports single precision (and thus couldn't be used), while SSE2 supports double.
 
The bottom line is, c't messed up in a big way. The holes in their testing are so big, you could sail the Titanic through them. Apple's "supercomputer" claim was based on the computing capability of the G4's Altivec unit. Doesn't anyone remember the G4 unveiling, in which Steve talked extensively about the power of the G4 "Velocity Engine?" It was the Velocity Engine that gives the G4 its power, and that's precisely what c't didn't test!

By that definition, Apple is perfectly right in its claim - the PPC 7400 at 500 Mhz could top 1 Gigaflops sustained. Note, Apple never said, "the G4's integer unit can do 1 billion integer operations per second."

Somehow, the geniuses at c't assumed that's what Apple meant when anyone seeing the MacWorld presentation could clearly understand that Apple was talking about the power of the Altivec vector processing unit.

It looks as if the Register now has a follow-up article to the "Apple speed claims demolished" article

http://www.theregister.co.uk/content/35/24378.html

I quote:


Visit Apple's Advanced Computing Group web page at http://developer.apple.com/hardware/ve/acgresearch.html and scroll to the bottom. You will find a link to (an admittedly dated as it lists a 500 Mhz G4 as the fastest available) white paper from NASA evaluating the G4 for scientific computation against Cray, Alpha, MIPS, and P3 based systems in terms of flops per dollar and concluded that the G4 offers "bang for the buck" advantages in factors of between 5 and 8 over Alpha and P3 systems.
Click to expand...

And also:

The newer PPC7450 series machines have even better vector performance than their predecessors, for a ruthless extremely optimized cross-platform demonstration, try running distributed.net's RC5 cracker on a G4 against anything else. The dual 1ghz G4 cracks more than 25 million keys a second. That is more than an order of magnitude faster than a 1.5ghz P4 from Dell does on the exact same job. As for P3, it does better than the P4, but still runs three times slower per clock than the 7400 G4. (The newer 7450 is substantially 3x faster than clock). A 1 Ghz P3 outperforms the 1.5 Ghz P4 by a sound margin. This application is a good demonstration of the problems with the P4's ridiculously deep pipes and crappy registers.
Click to expand...


Also, somehow, the slowpoke G4 can run BLAST (a popular bioinformatics tool) 5 times faster than a 2 GHz P4, albeit on a dual processor 1 Ghz G4 machine. That still makes a single 1 GHz G4 more than twice as fast as the 2 GHz P4 for BLAST. And you don't have to just take Apple's word on this - Genentech also backs up this particular claim:

http://www.apple.com/pr/library/2002/feb/07blast.html

Which goes to show you - software that uses the "supercomputer" aspects of the G4 really shows supercomputer-like performance. That is the basis of Apple's claim, and for that, they cannot be faulted. That's something c't purposefully and systematically ignored by employing SPEC benchmarks. Gee, why not also cut out the on-chip L1 cache and put the processor a 33 Mhz system bus?

It's interesting to note that when the G4 PowerMac debuted just a couple of years ago, Apple's claim to fame was topping 1 gigaflops. With the new dual 1 gig G4, it's now at 15 gigaflops. That's a 15 fold improvement in performance, at least for Altivec-enabled apps.
 
okay, c't messed up on this one. but they don't usually just 'praise' apple. neither do they just praise anything. they are highly effective in taking apart any kind of hard- and software as soon as they get their hands on it. when other magazines were talking about the form factor of the imac (the first one) they were talking about what funny things you could do with the hidden mezzanine slot and how to attach a floppy drive (the imac was the first mac without one).

so basically, c't is very, very objective. but they fucked up this one time.

good mag, though. :)
 
I wouldn't say they fucked up.
It is a fact that if you Benchmark Pentium vs. G4 that you get different results depending of what benchmarks you are using.
So it basically depends on what you are doing with your computer which one will be faster for you.

If you only use programs that use integer-arithmetic, the Pentium will probably be faster for you. If you use double precision floatingpoint, the same.
For floatingpoint the G4 will probably be way faster, but only if the program is Altivec enhanced.

I don't buy a Mac because I think it is faster. I buy it because I like it better.
 
Nope. specINT and specFP are intended to *exactly* measure processor speeds only. They have nothing to do with your personal preferences and do not pretend to test any of those things. But by doing those tests with a suite that is not AltiVec enabled c't got results that are not objective. Remove the wheels from a car and test its ability to handle curves. Hmm...

And btw. we all don't buy Macs because of their processor speed. If we would, we would be morons. We all know that Apple has to get up to speed fast, we all know that our processors are only faster in specific applications that are highly optimised for our machines but suck the rest of the time. We *stay* loyal to this brand because it gives us an overall experience.

It's still shocking. I almost believed by now that 'MHz don't matter'. I mean, we all *know* that it's not the only thing that matters, but as long as the PowerPC wasn't only a better architecture but could also very much keep up with the speeds in MHz, it was always clear the Mac had a clear advantage in speed. Ever since the 500 MHz trauma (the G4 @ 500 Mhz was the best Mac processor over a period of a YEAR without any enhancements, and afterwards was replaced by a 'faster' processor with less efficiency), the PowerPC platform was stagnating for too long. We all know that, and we are all hoping that newer G4s and - maybe in a year or two, depending on who you believe - G5 processors will not only keep up with the competition but also go ahead of it.

Benchmarks alone don't tell you what the final machine enables you to do, that's clear from the start. And c't cleary created a report that wasn't very useful to anyone (but hurt Apple quite a bit). But it's also clear that there's a gap right now that even high end design machines won't be able to fill forever. The new iMac has an advantage right now. Until the copycats are out of the bag and people 'have seen' it for about half a year. It can't really create the same effect the original iMac had. The new one won't change the people's look at computing as much as the original did. But it can extend on that, if the power of the platform starts again to extend further. The TiBook is a really great mobile computer, and although the original wasn't really that much faster than a Pismo 500 (mostly because Mac OS X wasn't out by then and because the G4 optimised wares were still missing), the newer ones can extend on that design. But they also have to be further improved. The iBook is a best-deal consumer notebook. Can't get anything better for the price. But even there, the processor speed gap will eventually start to hurt. And we know that there actually ARE G3's at 1 GHz being produced at IBM, so we need to see them in iBooks. And if nobody can make G4s at or above 1 GHz for notebook computers, Apple should seriously consider turning to G3s in notebooks for the time being. The PowerMacs are a great line right now, but they should turn to dual processing for all machines except the low end. Have a 1 GHz single processor machine as the low end and a 1.2 GHz Dual as the middle and maybe a four processor machine at the top. The machine can even be more expensive. If people have a *choice* they might buy it.

Back when the lineup was 'PowerMacintosh 7300', '8600' and '9600', the high end was so expensive and far out technology-wise and speed-wise that it made sense to say 'I don't need that horsepower'. Right now you need the top end, because all other machines tend to be a bit sluggish in OS X, which is unacceptable, I think. (I love my TiBook under OS X, don't forget that, I'm exaggerating here.)

Lineup in July? Anything below the following won't do:

- iBooks with 1 GHz G3 processors and at least 256 MB RAM (OS X is unusable below that)
- iMacs with 1 GHz G4 processors and *highend* graphics cards. This is a GAME machine for God's sake!
- TiBooks with 1 GHz G4
- PowerMacs starting at 1 GHz (better 1.2) and the option for quad-processor machines

I'll consider that as the lowest acceptable solution. I'd also like to finally see dual processor mobile machines (where one processor is shut down when on battery power). I'd also like to see high end RAM in all machines. We're far behind there, you know. And not talk-about. Deliver.

Enough raving.
 
The Spec Benchmark is a very respectable benchmark used in industry, The mention of double precision floating point numbers earlier is true, the G4 Altivec cant handle it the P4 can, but dont forget this is an old Slot A Cartridge style P3 with 100 mhz FSB and no SSE2 hence no double precision floating point number support JUST like the G4.

I did post up a link to this article minutes after it was released a few days ago but no one seemed bothered. slagging of Spec by saying its not a good comparison hahahaha please give me a break.
 
oh and by the way the pipeline used in the P3 in the test is exactly the same length as the 1Ghz G4, so there goes that argument.
 
the mention of rc5 above is pointing out an app which the p4 sucks at because it does not have hardware rotate on the core. the G4 has always been good at RC5 why dont you do a comparison with OGR also from Dnet and then see what the difference is, the performance gap is more then reversed.

id put a dual athlon xp2100 against that dual G4 1Ghz at RC5 anyday. thats what you call dishing out a beating on an away ground.
 
FrgMstr: Just the fact that the SPEC benchmark software is NOT ALTIVEC OPTIMIZED makes the benchmark totally useless, and you cannot deny that. It does not matter if we're talking about double floating point units or not, because that most likely is not used in real-life applications.

As Appleturns.com says:

What c't did is tantamount to forcing you to write with your toes and then telling you that your handwriting sucks.
Click to expand...

Sorry, but the testing that German magazine c't is flawed, and there's no argument about it. Like I said, it's a TRIVIAL question to ask if the software is Altivec optimized, and obviously c't didn't care.
 
My opinion on this is just to calmly ignore it, know that they are wrong, and go on with life...it can't hurt too much, can it?
 
You must log in or register to reply here.
Back
Top