Block Storage Limitations and the Value of
Key-Value over Fabrics
Written by Ken Grohe
Published on March 11, 2020
Block storage is a popular data storage type that splits data into blocks and stores them as individual hard drives, each with a unique identifier. Block storage typically is used for static files because any change made to a file results in the creation of a new object, which can be hard to manage as the system grows. I wrote about this topic on my personal Medium page earlier this year here, and specifically called out some of the cons of block storage in that article:
Cons of Block Storage
Single Server Access — The volume is limited to one server at any given time, and it’s impossible to access the data from a different target server.
Limited Metadata — Block storage metadata is limited to the basic file attributes. As a result, applications that depend on detailed metadata might suffer.
Cost — Block storage is more affordable than other cloud storage models. Still, it is expensive for managed service providers since they use SAN, which is costly and requires highly skilled storage experts to control the optimization of the data arrays.
Global Maps — Block storage requires extensive global maps in order to quickly locate data within the system. As the amount of data grows, managing the global maps can be challenging.
Caching — Block data storage systems use complex caching, which does not scale consistently across parallel workloads.
Enterprise Storage & the Key-Value Store Approach
The key-value store concept has been around for years, but it’s seen a resurgence as hyperscale companies rely on it as a scalable way to manage large collections of data from millions of users. When you need to scale quickly without losing performance, Key-Value Stores are the answer.
Key-Value Stores were created to efficiently manage unstructured data, and they allow enterprises to deliver fast and reliable user experiences. This is the reason we used Key-Value Store technology in our innovative Key-Value over Fabric architecture. Stellus was designed to drive scale-through growth, where a database can scale up (i.e., move to bigger storage volumes) and scale out (i.e., add more storage servers) at the same time, independent of one another. If you’re looking for a way to manage large amounts of unstructured data where consistent, fast performance is a requirement, the Stellus Data Platform is an excellent choice for your environment.
How do Key-Value Stores Work?
A Key-Value Store is an array database of keys, where each key is associated with a single value. These values can be one piece of arbitrary data such as an integer, or they can be more complex pieces of information such as another array. Key-Value Stores use their keys as identifiers. These identifiers point to a specific place in memory and allow the data to be accessed almost instantly regardless of its position in memory.
When we’re looking at large quantities of data, and particularly unstructured data (just think how many videos are stored on your smartphone alone!), key-value stores make the difference. In fact, there are three primary benefits Key-Value Stores (and particularly Key-Value over Fabric technology) offer enterprise data management:
1 — Unmatched Performance & Speed
Traditional data structures that require indexes are taxing on the processing speed and retrieval of data. If these structures were asked to access data located in the rear of an array without knowing the exact location of that information, it would take a long time. The structures would spend more time locating data in memory because they would need to compare the index with the data they wish to access.
Key-Value Stores bypass this restriction. Instead of an index, key-value stores use keys to point directly to the required data in memory, providing exceptional performance and speed that traditional data paths cannot replicate.
This unmatched speed and performance are two of the biggest differentiators for the Stellus Data Platform. In fact, Stellus recently set a new industry record for single client data storage at 33.2 GB/s sustained writes.
2 — Personalized and Improved User Experience
With this speed, administrators can pull data quickly to identify customers along with their behaviors and preferences. This data can be used to customize the user experience in a high-performance environment. Hyperscale companies figured this out early and have been using key-value stores from the beginning.
As people surf the web, check their social media, or make purchases online, they leave a trail of information about their habits and preferences. This information is stored and used to personalize experiences and offer products that they likely are interested in purchasing. Every company knows that understanding the customer is crucial to long-term business success.
In this way, Key-Value Stores offer a better storage mechanism to allow easy retrieval of user information. As the number of users on a given site increases and the data for each user increases as well, simple SQL tables can’t handle the strain. Key-Value Stores can manage this data with ease and retrieve it just as easily. This information can be in the form of session stores (storing user info for first-time visitors and subsequently updating their info with each visit), cookie stores (identifying a user and saving site login information for future use), and profile stores (creating personalized keys that relate to each user). Each of these scenarios creates a richer user experience that can help increase user retention and purchases.
3 — Faster Application Response
Time is just as important as — or perhaps more important than — personalization. Applications that run slowly because of the amount of time spent on retrieving information generally have bad reputations with users. Key-Value Stores make data retrieval fast, easy, and almost instantaneous in most cases. For applications that use real-time statistics, the need for instantaneous data retrieval is apparent. Applications that access data from multiple platforms or web servers need the speed and reliability of Key-Value Stores to retrieve data fast across different channels. Key-Value Stores seamlessly connect multi-channel applications, making them work together as one smooth system.
Key-Value Stores for the Future
Key-value stores are changing the way enterprises manage growing amounts of corporate data. Key-Value Stores are designed to easily scale and handle large amounts of data, while maintaining one of their most essential advantages: speed. This speed is critical for any application that needs to record specific user information to improve experiences and overall satisfaction.
Key-Value Stores are central to our innovative and unique Key-Value over Fabric architecture of the Stellus Data Platform. I’m proud to be part of Stellus as we are leading the charge in undoubtedly what will be a game-changer in enterprise storage and unstructured data management.