I’ve just completed writing a white paper (which can now be downloaded for free from the download page of this site.) If you want to read the paper’s executive summary, rather than the whole paper, here it is:

• An Interruption to Moore’s Law. Starting in 1965, Moore’s Law held sway for 40 years as the power of cpus and, ultimately, all computer components roughly doubled every 18 months. In 2005, Moore’s Law changed direction. It was no longer possible to make chips go faster by raising the clock speed. It was possible only to redesign chips to be “multicore” so that they accommodated multiple processors on the same chip. In theory, this still increases the performance of the chip, but there is a problem in making effective use of multicore chips once you move beyond having a few cores. The immediate problem is that you need to ?nd applications that can make effective use of multicore. (In the longer term the problem will be to move the IT industry towards finding ways to write parallel applications.)
• The VM Solution. One pro-tem solution is to use multicore to run multiple virtual machines, using VMware or similar software. However, it is unlikely that this can make use of more than 8 cores on a chip. The problem comes from having large numbers of interrupt-driven OSes residing on a single chip. There comes a point where it is difficult to scale because a bottleneck forms. (It’s also possible that the chip vendors may not want their chips utilized in this way, because it seriously reduces potential market volume in the server market).
• The Drive to Parallelism. Ultimately the applications that will be able to make effective use of multicore are those that can run in a parallel manner.
• Database and Parallelism. Database is a specific area where parallelism can be used and, remarkably, the size of databases has been growing dramatically. In fact, while the power of cpus was increasing by a factor of 10 every 6 years or so, the size of databases has been growing by a factor of 1000 every 6 years or so!! Not only is this an area where the capabilities of multicore can be exploited, it is an area where there is a pressing need for good technology solutions. There is a need to manage and utilize multi-petabyte databases effectively.
• Parallel Database Technology. The database products that can make best use of multicore are not the traditional commercial database products, but those that have been specifically built to scale up and exploit parallel operation through a “shared-nothing architecture.” Such database can deliver performance gains of more than 100 to 1 over traditional database products and they can manage very large amounts of data.
• The Killer App. The logical conclusion in the paper is that the killer application ­ that can make the most effective use of multicore chips ­ is the querying of very, very large databases using database products that have been engineered specifically for that purpose.

Incidentally such technology gets less expensive year by year (as does all computer technology) so the likelihood is that it will even be fairly inexpensive to manage very very large databases (VVLDBs).