Hollywood Has a Velocity Problem 

Written by Erik Weaver

Written by Erik Weaver

Published on April 7, 2020

Hollywood has a “velocity” problem. Let’s talk about why Key-Value Stores are critical.

What do Amazon Web Services and Facebook have in common? Of course, they are both fantastically successful, but they also share something else. Technologically, they are both back-ended by Key-Value Stores. So what is a Key-Value Store (KVS)? In technical terms, it’s a distributed hash table. It’s a way for you to store very simple objects or BLOBS (binary large objects) across a cluster of machines. The cluster provides a robust failover system and a way to store a large number of records with extremely low latency.

Benefit of Key-Value Store

The benefit of a Key-Value Store over a block store is that a KVS is much better suited for more substantial and varying data sets. Block stores were built to accommodate the much smaller 4K blocks associated with databases and ERP (enterprise resource planning) applications. Most are built for a data set with the sector size of 4K—which is what DB and ERP applications typically write.

Unstructured data, especially media file data, has relatively huge file sizes compared to the DB/ERP data sets. Key-Value Stores are designed for larger and varying sized data sets. One example, one of these is the Stellus Data Platform, which uses an atomic size of 1MB. It is significantly more efficient to write a 20MB file into 20 x 1MB chunks than hundreds of thousands of 4k chunks. This means that substantially fewer resources are needed to serve application requests with KVS.

KVS and NVMeOF

Key-value SSDs and NVMe Over Fabrics (NVMeOF) were ratified by SNIA in the 1.0 API standard last April.

There are two additional reasons why modern architectures need to move toward a Key-Value Store: scalability and speed. In terms of scalability, Key-Value Stores handle large data sets well and are good at processing a constant stream of read/write operations. This makes it highly scalable. As far as speed, Key-Value Stores use simple operational commands such as get, put, and delete. This makes them highly efficient at processing constant streams of read/write operations.

This combination of benefits brings us into a profoundly more efficient future.

The world of SAN/NAS will shrink over the coming years. Also, the future is moving towards Object and Non-Volatile Memory Express over Fabrics (NVMeOF) for Velocity and the demands of modern content.

 

Related Post

What Makes HDR Video So Special?

What Makes HDR Video So Special?

What Makes HDR Video So Special? Most people can appreciate the art of beautifully implemented cinematography, yet one might argue that those of us who are fans of classic cinema and television are especially attuned to the miracle that high dynamic range (HDR).
Why Will NVMe and NVMeOF Dominate the Land?

Why Will NVMe and NVMeOF Dominate the Land?

Side Note: There are four V’s in data: volume, variety, veracity, and velocity. Well, really five if you add value, but that is another conversation. For this conversation we will focus on just one: velocity. Velocity is the frequency of incoming
Optimizing Data Locality for Efficient Data Management

Optimizing Data Locality for Efficient Data Management

Data locality optimizing algorithms work to reduce the time it takes to retrieve data from a network. This post looks at what role they can play in your enterprise data management. Data locality is a basic computing principle that

Cognitive AI

Artificial intelligence management requires massive data sets and high-speed processing to achieve the degree of efficiency and accuracy necessary to train neural networks and establish actionable insights. Through innovative software and services, Stellus Data Platform empowers and inspires customers around the world to transform data into intelligence.

Read Solution

Media & Entertainment

The Stellus Data Platform (SDP) sets a new standard for media storage performance, empowering M&E companies to support more workloads, more simultaneous playback streams, and faster render times. Unlike architectures that waste resources on tasks irrelevant to modern storage, the SDP is an entirely new file system, built from the ground up for unstructured data and solid-state media.

Read Solution

Life Science

Stellus is rewriting the rules for life sciences computing. The Stellus Data Platform (SDP) file system is built from the ground up for storing and processing unstructured data. It enables organizations to dramatically accelerate application performance for genomic analysis. Researchers can now process more workloads in far less time and take concrete steps to enable personalized medicine therapies.

Read Solution

Stellus Data Platform

Stellus provides a modern, highly performant file storage platform that enables hyperscale throughput for digital enterprises. Based on key-value store technology and exceptionally suited for unstructured data.

Learn More

Solution Brief- Genomics

Unlock the Targeted Therapies of the Future

Read more

Solution Brief- M&E

Transform Media Economics with Breakthrough Capacity & Performance

Read more

Solution Brief- Cryo-EM

Break the Backlog for High-Speed Microscopy

Read more