The Future of  Enterprise Data Storage is

Key-Value over Fabrics

Written by Ken Grohe

Written by Ken Grohe

Published on March 11, 2020

The Key-Value Store concept has been around awhile, but it has enjoyed a revival as a scalable way to manage large collections of data . Big-name companies such as Facebook, LinkedIn, and Amazon have made extensive use of the paradigm, and for a good reason: when you need to scale quickly without losing performance, it works.

How Key-Value Stores Work

The basic concept is simple. A key, which is a number or string, is used to look up a value, which can be any kind of data. The software converts the key to a hash value, which maps to a storage address. This approach works efficiently, no matter how big the store gets.

A key then maps to a single value, but it could be a list, a JSON object, an XML description, or something else. The value can hold keys to other values, so a value can be as complex as necessary. When applied consistently, it yields a graph database that a query can walk through quickly.

Stores use caching to improve performance. They can cache hashes, values, or both. Hashes are small, and a cache can hold many of them, but caching an entire value provides a more significant saving.

Big data tends to be sparse, and some fields have no value. The key-value store approach handles unstructured data efficiently since there’s no need to store a null pair. Deletion is reasonably efficient.

Use in NoSQL databases

What broadly is called a “NoSQL database” includes several approaches to storage; the key-value store is one of the simplest and most popular. SQL queries degrade in performance as tables grow larger. A Key-Value Store scales better with size, making it a great fit for organizations dealing with massive amounts of unstructured data.

Key-Value Store databases can scale up (i.e., move to bigger storage volumes) or scale out (i.e., add more storage servers). With this in mind, we created the Key-Value Store over Fabric architecture that is the foundation of the Stellus Data Platform , which enables the “scale-through” concept. Stellus’s scale-through architecture allows a database to efficiently scale up and out at the same time, independent of one another. The decoupling of capacity and performance are critical when dealing with increasing amounts of unstructured data. This is especially true when data collections get into the terabyte range and higher because this scalability keeps the operation going without having to rework the architecture.

Examples of key-value services

LevelDB is an opensource C++ library from Google for implementing a key-value engine.

RocksDB is a key-value engine from Facebook, inspired by LevelDB. It’s intended for databases that are terabytes in size, and it’s optimized for fast, low-latency storage systems.

PalDB is LinkedIn’s opensource key-value engine. It was designed for “side data” or information that doesn’t change often or that specific pieces of code need.

Amazon S3 is a key-value storage service available to AWS customers.

In Summary

Key-Value Stores are powerful when dealing with large amounts of data, and particularly when dealing with unstructured data (Media & Entertainment is a prime example).

The paradigm is simple, and implementations are highly scalable. Scaling can consist of increasing a single storage device’s capacity (scaling up) or adding additional devices (scaling out) — and better yet, scaling through, where they can scale up and out at the same time independently. The Stellus Data Platform is the only solution available based on Key-Value over Fabric architecture that enables this record-setting throughput.

As companies continue to face exponential data growth, the key-value store approach is steadily gaining ground as the future of enterprise storage.

Related Post

What Makes HDR Video So Special?

What Makes HDR Video So Special?

What Makes HDR Video So Special? Most people can appreciate the art of beautifully implemented cinematography, yet one might argue that those of us who are fans of classic cinema and television are especially attuned to the miracle that high dynamic range (HDR).
Why Will NVMe and NVMeOF Dominate the Land?

Why Will NVMe and NVMeOF Dominate the Land?

Side Note: There are four V’s in data: volume, variety, veracity, and velocity. Well, really five if you add value, but that is another conversation. For this conversation we will focus on just one: velocity. Velocity is the frequency of incoming
Hollywood Has a Velocity Problem

Hollywood Has a Velocity Problem

What do Amazon Web Services and Facebook have in common? Of course, they are both fantastically successful, but they also share something else. Technologically, they are both back-ended by Key-Value Stores. So what is a Key-Value Store (KVS)?successful, but they also

Cognitive AI

Artificial intelligence management requires massive data sets and high-speed processing to achieve the degree of efficiency and accuracy necessary to train neural networks and establish actionable insights. Through innovative software and services, Stellus Data Platform empowers and inspires customers around the world to transform data into intelligence.

Read Solution

Media & Entertainment

The Stellus Data Platform (SDP) sets a new standard for media storage performance, empowering M&E companies to support more workloads, more simultaneous playback streams, and faster render times. Unlike architectures that waste resources on tasks irrelevant to modern storage, the SDP is an entirely new file system, built from the ground up for unstructured data and solid-state media.

Read Solution

Life Science

Stellus is rewriting the rules for life sciences computing. The Stellus Data Platform (SDP) file system is built from the ground up for storing and processing unstructured data. It enables organizations to dramatically accelerate application performance for genomic analysis. Researchers can now process more workloads in far less time and take concrete steps to enable personalized medicine therapies.

Read Solution

Stellus Data Platform

Stellus provides a modern, highly performant file storage platform that enables hyperscale throughput for digital enterprises. Based on key-value store technology and exceptionally suited for unstructured data.

Learn More

Solution Brief- Genomics

Unlock the Targeted Therapies of the Future

Read more

Solution Brief- M&E

Transform Media Economics with Breakthrough Capacity & Performance

Read more

Solution Brief- Cryo-EM

Break the Backlog for High-Speed Microscopy

Read more