Entries in Server (1)


Oracle/Sun F20 Flash Card - How fast is it?

I received several questions about the performance of the Oracle/Sun F20 flash card I used in my previous post about block alignment, so I put together a quick overview of the card's performance capabilities. The following results are from testing the card in a dual socket 2.93Ghz Nehalem (x5570) system running Solaris x64. This is similar to the server platform Oracle uses in the ExaData 2 platform. The F20 card is a SAS controller with 4 x 24GB flash modules attached to it. You can find more info on the flash modules on Adam Leventhal's blog and the official Oracle product page has the F20 details. All of my tests used 100% random 4KB blocks. I focused on random operations, because in most cases it is not cost effective to use SSD for sequential operations. These tests were run with a variety of different thread counts to give an idea of how the card scales with multiple threads. The first test compared the performance of a single 24GB flash module to the performance of all 4 modules.

4KB Random Operations 4KB Random Operations At lower thread counts the 4 module test is roughly 4x the operations per second of the single module test. As the thread count rises, the single module test tops out at 35,411 ops and 4 modules can deliver 97,850 ops, or 2.76x the single module test. It would be great if the card was able to drive the 4 modules at full speed, but 97K+ ops is not too shabby. What is more impressive to me is that those 97K+ ops are delivered at roughly 1ms of latency. The next round of testing included three different workloads. The three phases were 100% read, 80% read, and 100% write and they were run against all 4 flash modules. Again, all tests used 4KB random operations. Here are the operations per second results.


And a throughput version in MB/s for anyone that is interested.


Flash and solid state disk (SSD) technologies are developing at an incredibly fast pace. They are a great answer, but I think we are still figuring out what the question is. At some point down the line, they may replace spinning disk drives, but I do not think that is going to happen in the short term. There are some applications that can leverage this low latency capacity inside the servers today, but this is not the majority of applications.

Where flash and SSD make more sense to me is as a large cache. NetApp and Sun are using flash this way today in their storage array product lines. DRAM is very expensive, but flash can provide a very large and very low latency cache. I expect we will see more vendors adopting this "flash for cache" approach moving forward. The economics just make sense. Disks are too slow and DRAM is too expensive.

It would also be great to see operating systems that were intelligent enough to use technologies like the F20 card and the Fusion IO card as extended filesystem read cache. Solaris can do it for zfs filesystems using the L2ARC. As far as I know, there are no filesystems that have this feature in the other major operating systems. What about using as a client side NFS cache? At one point, Solaris offered CacheFS for NFS caching, but I do not believe it is still being actively developed. While CacheFS had its challenges, I believe the idea was a very good one. It costs a lot more to buy a storage array capable of delivering 97K ops than it does to put more cache into the server.

Click to read more ...