One of things that makes HPC (High Performance Computing, i.e. weather, aerodynamic, vehicle crash and nuclear bomb simulation) so expensive is the huge memory systems on these machines. There is an old saying in the field that latency is hard, bandwidth you just pay for. Anyway, the cost of a memory system isn't the memory itself -- Pricewatch puts a stick of 512MB of 200 MHz DDR memory at $137, or $274K for a terabyte. The cost has generally been in the plumbing necessary to connect that memory to the CPU(s)*. You need lots of very fast wires, which has historically required exotic packaging.
Things change. I just read that half of all cellphones are made on circuit boards with enough density to make an HPC architect drool. Chuck the unobtainium and use ALIVH -- this stuff is cheap! It makes me want to design a hobby HPC.
* Well, actually, the prices mostly come from market dynamics. You have consumers (big companies and government labs) with deep pockets and little ability or incentive to ensure they get real value for their money, and producers who try hard to lock their consumers into proprietary hardware and software to avoid real competition. The HPC money well is drying up as (a) defense budgets have been slashed and (b) many customers have realized that racks of commodity PCs will do the job just as well.