Yesterday, we announced Vertica’s FlexStore for Flash. We think this is a big deal because of what it means for our customers: a cost-effective and simple way to take advantage of the speed and performance of Flash-based memory. You get the performance of having everything in Flash at a cost that’s comparable to having everything on magnetic disk. This solution is available today on industry standard x86 servers from Cisco, Dell, IBM, HP, and others.
Flash = Speed
Flash-based memory devices deliver significant increases in query and load performance compared to magnetic disks because Flash has higher bandwidth and lower latency. How fast? We’ve benchmarked loads and queries in Flash at 850% the transactions per hour compared to 15K RPM SAS drives.
Vertica’s FlexStore for Flash
FlexStore is the secret sauce that gives Vertica the flexibility to create projections and assign different columns to different kinds of storage to maximize performance. FlexStore is able to place the most commonly used columns on Flash while storing rarely used columns on slower, cheaper magnetic disks. You use only as much Flash as you need to optimize performance. Most Vertica customers are close to the 85/15 rule – 85% of the queries utilize 15% of the columns. Since columns used in predicates are typically sorted, less than 12% of the total compressed data is generally read by a query.
With Vertica’s FlexStore, a relatively small Flash device accelerates terabytes of SAS or SATA drives per node – simply, easily and cost effectively using off-the-shelf hardware. Even complex queries and analytic workloads only require a handful of tightly directed accesses to spinning disk, returning results in milliseconds where competitors take minutes, hours, or simply reject queries due to data volume.
Vertica’s True Column Store Compression Is Key for Flash
The Vertica Analytic Platform was designed and built from the ground up as a true column store to maximize I/O efficiency and minimize legacy disk impact on analytic performance.
Vertica’s FlexStore and true column store compression combined with Flash-based memory delivers optimal price/performance. FlexStore automatically distributes the most commonly used columns into Flash media, SSD’s, etc. while our true column store compression ensures each gigabyte of flash houses significantly more user data than it would in competing databases.
- Vertica’s FlexStore Delivers Near-MemoryPerformance
Row store vendors have attempted to emulate true column store architecture by slicing blocks of rows into columns – and they’ve subsequently claimed similar advantages. On the surface, some of these claims appear to have merit, but dig a little and you’ll see that these row-store performance numbers only work on high-bandwidth devices such as main memory DRAM, flash media, or SSD’s. When data must be requested from disk, the entire column-shredded block of rows is fetched – requiring an order of magnitude greater I/O bandwidth. To keep all relevant data in cache, vast amounts of unused data must also be retained. This brute force approach is far from efficient.
- Vertica’s FlexStore Provides the Most Efficient Flash Solution
Using Flash for Temporary Space
Vertica FlexStore for Flash allows you to store more than just frequently used columns in Flash media. Administrators leverage main memory as much as possible as they tune and optimize their analytics platforms. They often have to deal with analytical operations, such as large sorts and joins that require temporary space because they won’t fit in main memory. Traditional architectures often run into bottlenecks during these operations because they rely on magnetic disks that have high latency and low throughput.
Vertica FlexStore removes these bottlenecks by assigning temporary data, such as intermediate results, to faster kinds of storage like Flash Media and SSD’s. With FlexStore for Flash, you get significant performance gains on high concurrency workloads with substantial hardware savings over traditional architectures.
- Flash Out Performs Disk
There’s more on using Flash-based memory for temporary space in Curt Monash’s post.
Best of Both Worlds: Performance and Flexibility
We’re excited because combining Vertica’s FlexStore and Flash-based memory devices on industry standard x86 hardware, our customers can easily leverage Vertica’s flexible, hybrid analytics platform combining traditional storage media and Flash memory to support their business at a fraction of the cost of traditional architectures. Over time, as the relative cost of flash declines, Vertica customers can easily move more of their data onto it on the fly without taking the database down.