In yesterday’s blog post, we learned about the importance of the Cloud in the Big Data story. Today, we will dive deeper into the role of operational databases in supporting Big Data.

While we often talk about Big Data architecture, it is crucial to understand that a Big Data system cannot exist in isolation. Many business needs can only be fully addressed with the help of operational databases. Simply having a system that can analyze Big Data may not solve every data problem.

Let’s consider a real-world example. Imagine you are using Facebook and you update your relationship status. Within seconds, this information is reflected in the timeline of your partner and a few immediate friends. After a while, you notice that the same information is available to your remote friends. Later, when someone searches for all relationship changes with their friends, your relationship change will also show up in the same list.

Now, here’s the question: Do you think the Big Data architecture is responsible for all of these changes? Is the immediate reflection of your relationship change with your family member also due to the technology used in Big Data? The answer is no. Facebook actually uses MySQL to handle various updates in the timeline and events on their homepage. Operational databases, like MySQL, play a crucial role in real-world businesses.

Relational Databases

Relational databases are widely used in businesses and have been around for many years. They are an essential part of the Big Data story. Relational databases, such as Oracle, SQL Server, and MySQL, are everywhere. If you are looking for an open-source and widely accepted database, I suggest trying out MySQL, which has gained popularity in recent years. PostgreSQL is also worth considering, as it has interesting licensing policies that allow modifications and distribution of the application in open or closed source form.

Nonrelational Databases (NoSQL)

NoSQL databases, which stands for Not Only SQL, are another type of operational database that supports Big Data. There are plenty of NoSQL databases available in the market, making the selection process challenging. When choosing the right NoSQL database for operational purposes, it is essential to consider properties such as data and query model persistence, eventual consistency, and scalability.

One property that stands out to me is eventual consistency. In contrast to the ACID (Atomicity, Consistency, Isolation, Durability) mechanism used by RDBMS for data consistency, NoSQL databases use BASE (Basically Available, Soft state, Eventual consistency). Eventual consistency is a widely deployed consistency model in distributed computing. It expects unexpected changes in large distributed systems, where nodes join and leave frequently. Even if one or more nodes are down, the system is expected to function normally, allowing applications to perform updates and retrieve data successfully. Eventually, all functioning nodes should return the same updated data.

As per Wikipedia, eventual consistency is a consistency model used in distributed computing that guarantees that if no new updates are made to a given data item, eventually all accesses to that item will return the last updated value. In other words, if no additional updates are made to a data item, all reads to that item will eventually return the same value.

In tomorrow’s blog post, we will discuss various other operational databases that support Big Data.