[logo] a small computer

Should You Move Your Whole System to In-Memory Computing?

should you move your whole system to in-memory computing?

Bradley Kingsley
Published a year ago.
6 minute read
logo of undefined

The modern web has seen to it that few other metrics are more important than speed, scalability and high availability in delivering an application to consumers. It's lucky that computers are faster than ever and the cost of bandwidth continues to go down, enabling programmers and engineers to optimize to incredible extents. With such unprecedented progress being made in the computing world, it's important to remember that that growth hasn't been happening in a vacuum. The requirements we have for software and hardware are also growing. So much so, in fact, that in some applications, traditional data processing, even with the unbelievably powerful architecture we've come to depend on today, just doesn't cut it. Could in-memory computing be the saviour we've been waiting for to usher us to the new age of software development?

Why Use In-Memory Computing?

Like most technology, in-memory computing isn't applicable in every application you can possibly think of. In order to understand why in-memory computing came up as a problem-solving strategy, it helps to look at current approaches to understand the flaws that plague them.

Reduced Latency

As businesses grow, the amount of data they consume grows and as new technology is invented, data that was previously useless suddenly finds a myriad of applications. This usually necessitates either vertical scaling or horizontal scaling. Vertical scaling involves stacking up a single computer with more processing power, memory, and disk space to meet demands. This might include moving to newer hardware such as SSDs whenever possible. Horizontal scaling involves adding more computers to divide processing between them. It's useful to imagine servers in the form of a literal stack. More servers in the horizontal direction = horizontal scaling, we can't really place servers on top of each other, so the only realistic way to describe vertical scaling would be the addition of more resources.

adding more computers horizontally & vertically

Vertical and horizontal scaling is incredibly useful techniques to keep the engines going on a fast-growing system. Small, incremental changes to hardware alone can't make up for the slowest part of most modern tech stocks: the read-write cycles of disk-based systems. One of the main reasons the use of in-memory computing has become so widespread is for its ability to dramatically reduce the amount of latency experienced by a system. RAM is inherently faster than any solid-state drive and faster still as compared to hard drives. Storing information in-memory makes it available much faster, thus faster processing.

Real-time Processing

The biggest consequence of reduced latencies thanks to in-memory processing is real-time processing. Real-time processing, as opposed to deferred or batch processing, is analyzing data and sending it where it needs to be immediately (or so fast the end-user can barely notice a delay).

Real-time processing is an application that has made itself extremely important over the last few years, with live streaming videos, games and movies being a mainstay on social media and the internet.

In order to tap the full extent your computer has to offer and offer real-time data, you will likely have to utilize several layers of caching and at least some amount of in-memory processing. Rather than store data on the disk, as with transactional databases such as Postgres, Mysql and OracleDB, or even NoSQL databases such as Mongo, IMC loads all the data into memory and it's read, analysed, processed and returned to the client in almost no time.

To get some perspective on just how mind-mindbogglingly fast this is, let's consider one of the highest-performing databases in the world - Postgres. Typically, Postgres will typically process 1,000,000 rows per second on the lower side. With IMC, that number jumps to 10,000,000.

The importance of real-time processing should be immediately apparent in cases such as fraud detection, where the user can't be expected to sit around for minutes as the computer slogs away at petabytes of transaction history, or even self-driving cars, where life-or-death decisions need to be made in a split second.

Less Risk than You'd Expect

At the risk of sounding redundant, in-memory processing is very fast. But since all this data is stored in the RAM, an obvious concern is volatility. What happens to all those terabytes of data if there's a sudden blackout or the system crashes for whatever reason?

The developers of projects like Spark and Apache Kafka recognized the problem early on and developed the programs to be fault-tolerant. They rely on both disk and memory processing at the same time in case anything goes wrong. But this doubles up to save the day when it comes to rebooting your system, too.

In-memory processing requires that all your data should be within the RAM to be processed, but even the fastest RAM modules will take hours to fully-load terabytes of data. Rather than waiting around for this data to be available, in-memory systems rely on the beauty of fault-tolerance to process the data that was stored on disk while the rest loads into memory. It's virtually zero downtime.

Drawbacks Of In-memory Computing

In-memory processing can make certain applications so much faster it may seem like magic to anyone without the background to understand what's going on. Imagine shipping a NASA engineer from 40 years ago, relying on just 4KB(!!) of memory to get to the moon, to the present day, when 4KB is literally smaller than the average website on the internet. If that's not magic from some inter-dimensional time-traveller, then what is?

Despite all the romanticizing we like to be so pent up on, everything has its drawbacks. Yes, even the magic of in-memory computing hits the wall every once in a while. Failure to understand this may lead newbies into a dangerous spiral where they attempt to leverage it into every single application in the world, which can be a pretty costly mistake (hint hint: RAM is volatile).

As such, some crucial factors must be taken into consideration before the decision for a full migration to in-memory processing is reached.


The most apparent problem with moving all your data to an in-memory processing system is that it will cost you big time. RAM is still almost five times as expensive as hard-drive space, despite being more than ten times as cheaper than ten years ago.

Bearing that in mind, it will cost you a lot of money to operate even a single Spark cluster, and if you're not careful, with little to show for it. Granted, even traditional disk-based processing models are relatively expensive, but you typically don't need thousands of gigabytes in RAM just to run a bunch of queries.

Limits on Database Size

With that said, unless you have bottomless pits for pockets or you're literally a megacorp in the US, you likely won't be able to afford in-memory processing. To be fair, you don't have to - we have Redis, which performs amazingly if set up right, but that works better as a cache or an in-memory store rather than a fully-fledged processing powerhouse.

Risk of Data Loss

The final benchmark for whether you are ready to adopt an in-memory processing system is how much you're willing to contend with the loss of your data. For the umpteenth time, RAM is very volatile, so if there happens to be a blackout, all your data is gone forever. Sure, you can run a backup system once every other day, but that also costs money. When you're just a system crash away from losing years' worth of work, it helps to have piles of money sitting around, though.

Ultimately, no in-memory system can survive on its own. Disk persistence is crucial if any system is going to remain immune to data loss due to abrupt power outages. Apache Zookeeper, the same technology that gives Kafka and Spark their fault-tolerance - is an example of a library that can be used to achieve just that.

Copyright © 2020 The Kenyan Dev