David Patterson, UC Berkeley, on “The Parallel Revolution Has Started: Are You Part of the Solution or Part of the Problem”, at a Google TechTalk a few days ago:
http://www.youtube.com/watch?v=A2H_SrpAPZU

He even touches on i/o issues and the future of flash. It’s worth watching to the end, his answers to audience questions are illuminating, too.

So, for the stacked-memory case, things are just fine: there isn’t much performance gain to be had by going multi-core beyond 8 cores / CPU, BUT the density gain is linear, and efficiency gain is likely to be a significant positive slope as well.

As for all servers going NUMA, then that might well be (and at least it means that operating systems will be optimised for this). However, this isn’t about servers – its about multi-core chips. It’s possible to go NUMA with a multi-core chip and support multiple external memory buses with the appropriate high speed interconnects in the chip’s memory management, but it still amounts to providing more memory bandwidth to the chip. Faster links will play a part, but that will hit clocking limits so we will still end up with more paths creating packaging and cost issues. The inter-chip NUMA inteconnnects would similarly have to be scaled up adding even more to the packaging issues. Such a server architecture would also be NUMA at two layers requiring more optimisation. It’s also possible to come up with workload types that don’t scale well on NUMA.

However, I’m more optimistic that major strides can be made in increasing memory bandwidth than can be done with hard disks. The latter are inherently constrained by mechanical and geometric issues. It’s also easier to deal with bandwidth issues than latency ones. The mismatch between increased processor speed and memory latency is one of the main factors behind the drive towards hardware multi-threaded cores.

Nobody used Rambus RAM except Sony, so their initiatives are irrelevant.

All servers are going to be NUMA anyway, so it’s a little late to worry about that.

There are little tweaks (wider busses, bigger caches), but these all break down eventually and the bottlenecks become very real.

The introduction of hardware threading (really virtual CPUs) to make use of processing time otherwise lost due to memory stalls eeks out a little more. Hypethreading, SUN’s Niagara processor – this is popping up all over the place).

However, we’ve no arrived at a processor architectures which now has to be viewed as a system – the hardware threads on these machines are now virtual resources which are presented to the operating system as individual CPUs. The procesossor has to be viewed as a system with memory access like I/O to understand it properly. It’s causing massive problems of understanding of capacity planning figures now as reported CPU utilisation does not reflect the underlying processor resource usage figures with all these hardware threads. Only the SUN Niagara has any really measurements of underlying core resources, and those are not the ones seen by the OS.

Basically the unit of what a core computer is has shrunk back to the chip. Once the Hyperviser arrives there too, there will be a very different relationship with memory.

Given that the controller heads support up to 128 GB of RAM in some configurations, that’s a lot of cache.

This trend was startlingly obvious to storage manufacturers because they spent more of their time flinging bits from disk to client than they did computing on those bits. One of the reasons I, among others, was thrilled with the original Opteron work from AMD was that it added more memory controllers, a four socket Opteron with each chip having its own memory controller is an impressive beast.

The massive cache experiment that was the Itanium was also flawed in that multiple cores not only raw memory bandwidth requirements to feed the execution engine pipelines but also add coherence traffic which consumes bandwidth as well (and was the death of 8 core Opteron systems which, with a simple probe/reponse coherence protocol quickly lost the extra bandwidth benefit)

Dave Hitz commented that Flash was the new Disk and Disk is the new Tape, perhaps he got it wrong and DRAM is the new disk. When you have 16 billion transistors in a 45nm process why not just a couple of cores and 8GB of static RAM?

–Chuck

Enjoyed your post.

Granted those of us who have been around as customers, vendors, analysts, media or what ever are familiar with the decades old I/O performance, storage capacity and processor gap, its far from being known by everyone in the industry.

From what I continue to see and hear from IT customers, even vendors and others particularly those who have not been in the industry for a decade or more, there is still an amazing lack of awareness of the I/O performance gap. In fact, the industry trends white paper Data center performance bottlenecks and the server storage I/O performance gap continues to be a very popular download and point of discussion during presentations, seminars and keynote discussions.

So just how long until we hit the wall and is it a moving wall similar to the super parametric barrier wall for disk drives that we were supposed to hit several years ago?

Granted the disk drive wall has been pushed back for awhile, however will the related memory wall be pushed back before or it is supposed to occur delaying the impact, is it an isolated case for extreme corner case environments, or, a major concern for commercial computing, or, simply something fun to talk about.

Cheers
Gs

That memory bandwidth and latency are essential is not news. That the problem is now hitting mainstream computing is. This is why Intel is long overdue with the QPI switched interconnect in Nehalem processors (AMD’s Opteron has had it for years). The 8-core Mac Pro was benchmarked as no faster than the 4-core version for many workloads, so this is not a theoretical concern.

One important solution to the problem is seldom discussed – refactoring software to make it more efficient, and reduce the bloat from layers and layers of abstraction layers piled upon legacy code. Apple is adopting this approach with Snow Leopard, and other software vendors would do well to follow suit. Moore’s law stalled a number of years ago for single-thread workloads, the sloppy coding habits of many programmers have to go, and be replaced with a newfound emphasis on performance and its necessary precondition, a ruthless culling of unnecessary features that add bloat but little value (JVMs inside the database, anyone?).

Stacked memory shows a leveling off — i.e. no performance drop, but no performance gain. What’s the point of adding more cores? Back to the drawing board to solve the memory bandwidth problem.

Massively multi-core systems have at least two major barriers to overcome — memory constraints, and the difficulty of writing software to take advantage of MMC systems.

Generations of PhD theses and commercial research have not yet cracked the software problem for supercomputing, so don’t expect major breakthroughs soon.