What is a database?

Key takeaways:

• A database is an organized and systematic collection of data stored in a computer system.

• There are different types of databases designed for specific purposes, including SQL, NoSQL, time-series, in-memory, NewSQL and vector databases.

• The main components of a database are its schema, data model, DBMS, and data storage layer.

• Basic functions of a database are data management, data storage, data manipulation, concurrency control, data security, data integrity and transaction management.

• Some of the business which uses databases are banking, insurance companies, healthcare centers, manufacturers, law firms, social networks, medicines companies, bioinformatics companies, commerce stores, etc.

A database is an organized and systematic collection of data stored and managed electronically in a computer system. It allows users to perform different operations on data, such as storing, retrieving, updating, and deleting it.

Note: Data can be any information like text, images, numeric numbers, media files, and so on.

Types of databases

There are many types of databases due to different applications and systems having different needs in terms of data, performance, scalability, and flexibility. Some of the common database types are as follows:

• Relational databases or SQL databases

• NoSQL databases

• Time series databases

• In-memory databases

• NewSQL databases

• Vector databases

Let’s discuss each of these databases:

SQL databases

SQL databases, also referred to as relational databases, organize data into tables with rows and columns. Each row in a table represents a record, and each column represents a field or attribute. SQL databases follow a strict schema and use a specific language called structured query language (SQL) to perform different operations on the data. These databases are preferred to handle structured data and ensure ACID (Atomicity, Consistency, Isolation, and Durability) properties. Some of the SQL databases include:

• MySQL

• PostgreSQL

• Oracle

NoSQL databases

The NoSQL (not only SQL) databases are designed to handle unstructured or semi-structured data. These databases provide more flexibility than relational databases because they don’t require a rigid schema and can be scaled horizontally across multiple servers. Some types of the NoSQL databases are:

• Key-value store

• Document databases

• Graph databases

• Columnar database

Time series databases

Many modern systems, such as IoT, monitoring systems, and stock markets, produce time-stamped data. Such data requires specialized databases known as time series databases that are optimized for managing and querying time-stamped data. Time series databases are highly efficient at storing and retrieving data indexed by time. Some use cases of the time series databases are IoT applications, financial market data, and environmental and weather data. Following are some of the examples of time series databases:

• Amazon Timestream

• InfluxDB

• TimeScaleDB

• Prometheus

In-memory databases

As the name suggests, in-memory databases store data directly in the system’s RAM instead of on the disk, which helps rapid access to data. They are designed for applications where speed is critical, such as real-time applications, caching, and high-frequency trading. Because data is stored in memory. In-memory databases provide much lower latency compared to disk-based databases, though they typically trade off durability for performance. To mitigate data loss, the in-memory databases are used in tandem with other persistent databases. Some use cases of in-memory databases are high-frequency trading, session management, and online/live gaming applications. Following are some popular in-memory databases:

• Redis

• Memcached

In-memory databases are very fast because they store data in RAM, but this comes with some trade-offs related to data persistence and recovery. To prevent losing data if the system crashes, they often use techniques like taking snapshots and keeping transaction logs. While these methods help to protect data, they can add some complexity and overhead, so it’s essential to find a balance between speed and reliable data protection.

NewSQL databases

NewSQL databases combine the best of both relational and NoSQL databases, i.e., ACID properties of SQL databases and the scalability and distributed nature of NoSQL databases. These databases are designed for high-throughput transactional workloads and require both consistency and scalability. The nature of these databases makes them ideal for modern web-scale applications such as online retail platforms, real-time analytics, and healthcare systems. Some common NewSQL databases are:

• Google Spanner

• CockroachDB

• VoltDB

Vector databases

Vector databases store and manage large-scale high-dimensional vector data, often generated by large language models (LLMs) or machine learning models. These databases are designed to expedite the process of similarity search in big data, such as finding similar text, images, videos, and audio using their vector representations. These databases are primarily used in LLMs and recommendation systems where large volume of data is handled with higher performance and scalability. Some well-known vector databases include:

• ChromaDB

• PinCone

• Milvus

• Weaviate

• ScaNN

Note: To understand more about large scale (big data), you can look at the difference between big data and data warehousing.

The main components of a database

There are many components of the database that are responsible for different operations. Some of the key components are:

• Database schema or data model: The database schema and data model define the architecture of how data is organized within the database. It includes the tables, columns, data types, relationships (such as one-to-many or many-to-many), and constraints (like primary keys and foreign keys). A relational database schema may include a set of interrelated tables, while a NoSQL database may have a flexible schema.

• Database management system (DBMS): The DBMS is the software layer that interacts with the database and manages all operations on the data including indexing and transaction management. It provides an interface to interact with the database in a secure, efficient, and reliable manner. The DBMS ensures data consistency, manages concurrent access, and enforces security via access controls. It also includes query processors to optimize SQL or NoSQL queries for efficient and fast data retrieval.

• Data storage layer: Data storage refers to how and where the actual data is physically stored, whether on disk, SSD, or memory. For example, traditional databases store data on disk using storage engines that efficiently write and read data using techniques like B-trees or hashing. In contrast, in-memory databases, like Redis, store data in RAM to ensure high-speed access. The storage layer also includes data compression, caching, and replication to optimize space usage and access times.

Key features of a database

Some of the key features and importance of a database are:

• Data management and storage: A database allows us to manage and store data physically, whether on disk, SSD, or memory.

• Data manipulation: The database allows us to perform the CRUD (Create, Read, Update, and Delete) operations on the data.

• Concurrency control: The database also controls the simultaneous access of multiple users accessing the data.

• Data security: A database protects and secures data from unauthorized access and breaches. This involves authentication, encryption, and an access control mechanism.

• Transaction management: Following the ACID properties, SQL databases ensure that all database operations are executed as a single unit and maintain data integrity.

Note: The ACID properties are typically not applied to the NoSQL database as they are designed for scalability, flexibility, and high availability. They usually follow BASE (Basically Available, Soft state, Eventual consistency) principles.

Database use cases

Databases are used in almost every field. We can store media files, images, songs, genomic and biological data, texts, and numbers. Social media companies have their own database systems to manage. The following are some of the businesses where databases are being used nowadays.

Note: You might be interested in top 20 database interview questions.