# AWS Database Services Overview

> Chapter 1 of the Courseiva AWS-DATABASE-SPECIALTY curriculum — https://courseiva.com/learn/aws-database-specialty/aws-database-services-overview

**Official objective:** 1.1 — Understand the different database services offered by AWS and their appropriate use cases

## Introduction

AWS Database Services are a collection of fully managed database engines that handle storing, retrieving, and managing data in the cloud. The entire AWS Certified Database Specialty exam (DBS-C01) tests your ability to choose the correct database service for a given workload. This understanding is the single most critical foundation for passing the exam.

## The City Transport Hub Analogy

A city's central transport hub is the place where all kinds of vehicles and travellers meet. 

This hub has different sections for different needs. There is a long-term parking garage for cars that stay for weeks, like a Relational Database (RDS) that stores structured data with strict rules. There is a quick drop-off zone for delivery vans that come and go fast, like DynamoDB which handles fast, key-value lookups. Next to it, a bus terminal handles thousands of scheduled routes with flexibility, like Amazon Aurora which is a high-performance relational database. For storing historical records, there is a document archive (like Amazon S3, though not a database itself), and a special room for real-time train arrival updates, like Amazon ElastiCache which caches data for speed. Across the street, a data warehouse stores years of passenger travel logs, like Amazon Redshift for analytics. A separate building holds video footage from security cameras, similar to a document database like Amazon DocumentDB. By choosing the right section of the hub, the city ensures every traveller gets the right service without congestion. This maps precisely to AWS database services: picking the right tool for the specific data structure and access pattern prevents performance issues and cost waste.

## Core explanation

A database is a structured collection of data, stored electronically, that can be accessed, managed, and updated. Before cloud computing, you had to physically buy servers, install database software, configure storage, manage backups, and handle security patches yourself. AWS Database Services eliminate this manual work. They provide pre-configured, scalable, and highly available database engines that you can spin up in minutes via the AWS Management Console, CLI, or API. 

AWS offers several distinct categories of database services, each built for specific data models and access patterns. The most common categories are:

- **Relational Databases**: These store data in tables with predefined schemas, rows, and columns. They use Structured Query Language (SQL) for querying. Examples include Amazon RDS (Relational Database Service) which supports engines like MySQL, PostgreSQL, MariaDB, Oracle, and Microsoft SQL Server, and Amazon Aurora which is a MySQL and PostgreSQL-compatible relational database built for the cloud with better performance and availability. Use these for transactional workloads like order processing, customer relationship management (CRM), and financial systems where data integrity and ACID (Atomicity, Consistency, Isolation, Durability) compliance are crucial.

- **Key-Value Databases**: These store data as a collection of key-value pairs, where each key maps to a value. Amazon DynamoDB is a fully managed key-value and document database. It is designed for high throughput, low latency, and automatic scaling. Use it for real-time applications, gaming leaderboards, session state management, and Internet of Things (IoT) data.

- **Document Databases**: These store data in flexible, JSON-like documents. Amazon DocumentDB (with MongoDB compatibility) is a managed document database. It is designed for content management, catalogues, and user profiles where the schema can evolve over time.

- **In-Memory Databases**: These store data primarily in memory (RAM) for extremely fast read and write performance. Amazon ElastiCache supports Redis and Memcached. Use it for caching frequently accessed data, session stores, and real-time analytics.

- **Graph Databases**: These store relationships between data points using nodes and edges. Amazon Neptune is a managed graph database. Use it for social networks, recommendation engines, and fraud detection.

- **Time-Series Databases**: These are optimised for data that arrives over time, like metrics and sensor readings. Amazon Timestream is a managed time-series database. Use it for monitoring, IoT analytics, and industrial telemetry.

- **Ledger Databases**: These provide an immutable, cryptographically verifiable log of data changes. Amazon Quantum Ledger Database (QLDB) is a fully managed ledger database. Use it for applications requiring transparent and verifiable history, such as supply chain tracking and financial audits.

- **Data Warehousing**: Amazon Redshift is a fully managed, petabyte-scale data warehouse. It is designed for analytics and business intelligence (BI) workloads, allowing you to run complex queries across huge datasets from other databases and data sources.

Each service is managed by AWS, meaning AWS handles hardware provisioning, patching, backups, and replication. You only need to configure the database instance type (size), storage, and security settings. This replaces the traditional approach where you had to manage all infrastructure components yourself.

The core reason these different services exist is that no single database engine is optimal for every use case. A relational database excels at enforcing strict data relationships but can be slower for high-volume key lookups. A key-value database is blazing fast for simple lookups but lacks the complex querying capabilities of SQL. AWS designed its portfolio to let you pick the right tool, reducing cost and improving performance.

## Real-world context

Consider a medium-sized e-commerce company migrating its platform to AWS. The IT professional, a solutions architect, must decide which AWS database services to use for each component of the application. 

First, the **order management system** needs a reliable, ACID-compliant database to handle customer orders, payments, and inventory updates. The architect chooses **Amazon Aurora** because it is MySQL-compatible, offers high availability with automatic failover, and provides faster performance than standard MySQL. The architect creates an Aurora cluster with a primary instance to handle writes and multiple read replicas to handle read traffic from the website. 

Second, the **product catalogue** contains thousands of items with varying attributes (size, colour, material). The schema changes frequently as new categories are added. The architect picks **Amazon DocumentDB (MongoDB compatible)** because its flexible document model allows storing each product with a unique set of fields without requiring schema changes. The architect configures auto-scaling so the database can handle spikes during sales events. 

Third, the **shopping cart** needs extremely fast read and write operations to store which items each customer has added. The architect uses **Amazon DynamoDB** for its key-value structure. Each session ID is the key, and the cart items are the value. DynamoDB scales automatically as the number of concurrent shoppers increases. The architect also sets up DynamoDB Accelerator (DAX) as a caching layer to achieve microsecond latency. 

Fourth, the **user session** data is stored in **Amazon ElastiCache for Redis**. Every time a user logs in, their session information is stored in the cache so the application can quickly verify permissions without querying the main database. The architect configures replication and automatic failover to ensure session data is not lost if a cache node fails. 

Fifth, the **analytics team** needs to run complex queries across all orders, customer behaviour, and inventory data to generate business reports. The architect sets up **Amazon Redshift** as the data warehouse. A daily batch job extracts data from Aurora and DocumentDB and loads it into Redshift. The team uses Redshift’s columnar storage and massively parallel processing to run queries that take minutes rather than hours. 

Finally, for **fraud detection**, the architect uses **Amazon Neptune** to build a graph of complex relationships between customers, payments, and shipping addresses, enabling real-time fraud pattern detection. 

Each step involves choosing the right service, configuring instance sizes, setting up backups, implementing security (encryption at rest and in transit), and monitoring performance with Amazon CloudWatch. The architect also considers cost, setting up reserved instances for Aurora and Redshift to save money on predictable workloads, while keeping on-demand capacity for DynamoDB to handle unpredictable spikes.

## Exam focus

The DBS-C01 exam tests your ability to select the correct AWS database service for a specific scenario. The questions are scenario-based, presenting you with a description of a workload (e.g., access patterns, data volume, latency requirements, consistency needs) and asking you to choose the best database service. 

Key concepts the exam loves to test:

- **Relational vs Non-Relational**: You must understand when to use RDS/Aurora (structured data, complex queries, ACID) vs DynamoDB (unstructured, high throughput, simple key-value lookups). A typical trap: a question describes a workload with many relationships (e.g., a social network) but high read/write throughput. The correct answer is often Neptune for the graph relationships, not a relational database. 

- **Managed vs Unmanaged**: The exam emphasises that services like RDS, DynamoDB, and ElastiCache are fully managed (AWS handles patching, backups, replication). You must not recommend actions that the customer would need to do manually, like installing patches or creating backups yourself. 

- **In-Memory vs Disk-Based**: A question about caching or session storage with extremely low latency (sub-millisecond) will always point to ElastiCache (Redis or Memcached). A common trap: a question describes a read-heavy workload with moderate latency tolerance — the wrong answer suggests ElastiCache, but the correct answer might be DynamoDB with DAX. 

- **ACID vs BASE**: Relational databases (RDS, Aurora) provide ACID compliance (Atomicity, Consistency, Isolation, Durability). NoSQL databases (DynamoDB, DocumentDB) provide BASE (Basically Available, Soft state, Eventual consistency). A question about financial transactions requiring strict consistency forces the use of a relational database. 

- **Data Warehousing vs OLTP**: If a question involves analytical queries across huge datasets (e.g., reporting, BI), the answer is Amazon Redshift. If it is about transactional processing (e.g., updating a single order), the answer is a relational database like RDS or Aurora. A trap: read-heavy OLTP workload with complex joins — the wrong answer might be Redshift because it is columnar, but the correct answer is Aurora with read replicas. 

- **Scalability Patterns**: DynamoDB scales horizontally automatically (adds more partitions). RDS scales vertically (bigger instance) and horizontally via read replicas. Aurora scales storage automatically up to 128 TB. The exam will test whether you understand which service scales in which way. 

Exam traps:

- A question that describes a workload as 'real-time' but does not mention latency requirements — candidates often jump to ElastiCache, but DynamoDB can also be real-time with lower cost. 
- A question about a 'graph' workload that does not mention relationships explicitly — candidates might choose a relational database, but the exam expects you to identify the graph pattern. 
- A question about 'document' storage — candidates might confuse Amazon S3 (object storage) with DocumentDB (document database). The former is for files, the latter for document data with query capabilities. 

Key definitions to memorise:
- ACID: properties ensuring reliable processing of database transactions. 
- BASE: a consistency model used in NoSQL databases allowing for eventual consistency. 
- Read replica: a copy of a database that can handle read queries to offload the primary. 
- Multi-AZ: deployment across multiple Availability Zones for high availability. 
- DAX: DynamoDB Accelerator, an in-memory cache for DynamoDB.

## Step by step

1. **Step 1: Understand Workload Requirements** — Identify the data's structure (structured, semi-structured, or unstructured), access patterns (read-heavy, write-heavy, balanced), latency needs (sub-millisecond, millisecond, seconds), consistency requirements (ACID or eventual), and query complexity (simple key lookups or complex joins). This determines which database category to choose.
2. **Step 2: Select the Database Category** — Based on Step 1, determine if you need a relational (RDS/Aurora), key-value (DynamoDB), document (DocumentDB), in-memory (ElastiCache), graph (Neptune), time-series (Timestream), or ledger (QLDB) database. This narrowing is critical because each category optimises for different trade-offs.
3. **Step 3: Evaluate Managed vs Unmanaged** — Decide whether your team needs to manage the database server yourself (e.g., installing on EC2) or use a fully managed service (RDS, DynamoDB, etc.). For the exam, managed services are almost always preferred because AWS handles patching, backups, and replication.
4. **Step 4: Choose the Specific Service within the Category** — Within the relational category, for example, pick RDS if you need standard MySQL/PostgreSQL features, or Aurora if you need higher performance and availability. Within NoSQL, pick DynamoDB for key-value, DocumentDB for document workloads, or Neptune for graphs.
5. **Step 5: Configure Deployment and Scaling** — Decide on instance size, storage type (SSD, magnetic), multi-AZ or single-AZ deployment, read replicas, and auto-scaling settings. For DynamoDB, set provisioned or on-demand capacity mode. For Redshift, configure node type and number of nodes.
6. **Step 6: Set Up Security, Backup, and Monitoring** — Enable encryption at rest (AWS KMS), encryption in transit (TLS), configure network access using security groups and VPCs, set automated backups with retention periods, enable point-in-time recovery (PITR), and set up Amazon CloudWatch alarms for performance metrics and cost monitoring.

## Comparisons

### Amazon RDS vs Amazon Aurora

**Amazon RDS:**
- Supports multiple engines: MySQL, PostgreSQL, Oracle, SQL Server, MariaDB.
- Maximum storage size limited to 16 TB per instance.
- Performance is about the same as running the standard database engine on EC2.

**Amazon Aurora:**
- Only supports MySQL and PostgreSQL compatibility (its own engine).
- Auto-scales storage up to 128 TB with no manual intervention.
- Provides up to 5x performance improvement over standard MySQL and 3x over PostgreSQL.

### Amazon DynamoDB vs Amazon DocumentDB

**Amazon DynamoDB:**
- Designed for key-value and document data with simple access patterns.
- Uses a hash-and-range key model for partitioning.
- Supports transactions limited to single partition or across partitions with some limitations.

**Amazon DocumentDB:**
- Designed for document data with flexible schema (MongoDB-compatible).
- Uses a hierarchical document model (JSON) with nested arrays.
- Supports complex queries, indexing, and aggregation pipelines similar to MongoDB.

### Amazon ElastiCache vs Amazon DynamoDB Accelerator (DAX)

**Amazon ElastiCache:**
- Fully managed in-memory caching service supporting Redis or Memcached.
- Can cache data from any data source (e.g., RDS, DynamoDB, external).
- Requires managing eviction policies and cluster configuration.

**Amazon DynamoDB Accelerator (DAX):**
- Specifically built as an in-memory cache only for DynamoDB.
- Provides microsecond latency by sitting in front of DynamoDB as a write-through cache.
- Automatically synchronises with DynamoDB, reducing cache management overhead.

### Amazon Redshift vs Amazon Athena

**Amazon Redshift:**
- A fully managed petabyte-scale data warehouse with columnar storage.
- Requires provisioning nodes and storing data within the warehouse.
- Best for frequent, complex analytical queries on structured data.

**Amazon Athena:**
- A serverless query service that runs SQL directly on data in Amazon S3.
- No infrastructure to manage; pay per query.
- Best for ad-hoc queries on large datasets that are stored in S3 (e.g., logs, flat files).

### Amazon RDS (OLTP) vs Amazon Redshift (OLAP)

**Amazon RDS (OLTP):**
- Optimised for high-volume, low-latency row-level transactions (INSERT/UPDATE/DELETE).
- Uses row-oriented storage for fast single-row operations.
- Supports ACID transactions and foreign key constraints.

**Amazon Redshift (OLAP):**
- Optimised for complex analytical queries scanning millions of rows (SELECT aggregates).
- Uses column-oriented storage for efficient compression and fast scans.
- Does not enforce foreign keys during normal operations (designed for bulk load).

### Amazon Neptune vs Amazon RDS (Relational)

**Amazon Neptune:**
- Designed for graph data: nodes, edges, and properties.
- Optimised for traversing relationships (e.g., friends of friends).
- Uses graph query languages like SPARQL and Gremlin.

**Amazon RDS (Relational):**
- Designed for tabular data with rows and columns.
- Optimised for complex JOINs and aggregations across tables.
- Uses SQL for querying.

## Diagram

_Decision tree for selecting the appropriate AWS database service based on data structure and workload type._

```mermaid
flowchart TD
    A[Application Data] --> B{Data Structure?}
    B -->|Structured, SQL, ACID| C[Relational]
    C --> D[Amazon RDS]
    C --> E[Amazon Aurora]
    B -->|Unstructured, Key-Value| F[NoSQL Key-Value]
    F --> G[Amazon DynamoDB]
    B -->|Documents, JSON| H[NoSQL Document]
    H --> I[Amazon DocumentDB]
    B -->|Fast Caching| J[In-Memory]
    J --> K[Amazon ElastiCache]
    B -->|Graph Relationships| L[Graph]
    L --> M[Amazon Neptune]
    B -->|Analytics, Big Queries| N[Data Warehouse]
    N --> O[Amazon Redshift]
    B -->|Time-Series Data| P[Time-Series]
    P --> Q[Amazon Timestream]
```

## Common misconceptions

- **Misconception:** All AWS database services store data on hard drives and are basically the same under the hood. **Reality:** Different AWS database services use different storage architectures. For example, DynamoDB uses solid-state drives (SSDs) and in-memory caching, while RDS can use either SSDs or magnetic storage. ElastiCache stores data primarily in memory (RAM). In-memory databases are orders of magnitude faster than disk-based ones. (This misconception arises because beginners assume 'database' means a single type of technology. Without understanding storage engines, they think all databases read/write from disk similarly.)
- **Misconception:** Amazon S3 is a database, so I can use it for anything a database does. **Reality:** Amazon S3 is an object storage service, not a database. It cannot enforce schemas, run SQL queries, or manage transactions. It is ideal for storing files, backups, and static assets, but not for transactional data requiring ACID compliance or complex queries. (S3 is so widely used and can store structured data (e.g., JSON files) that beginners mistakenly treat it as a database. The confusion is reinforced because some AWS services like Athena can query data in S3 using SQL.)
- **Misconception:** DynamoDB can handle complex SQL queries like relational databases. **Reality:** DynamoDB is a NoSQL key-value and document database. It does not support SQL. It uses its own API for querying based on primary keys and secondary indexes. Complex joins, aggregations, and subqueries are not possible natively. To run analytical queries on DynamoDB data, you must export it to other services like Amazon Athena or Redshift. (Because many NoSQL databases now offer limited SQL-like interfaces, beginners assume DynamoDB does too. However, DynamoDB's query capabilities are limited compared to relational databases.)
- **Misconception:** If I use Amazon RDS, I don't need to worry about any database administration at all. **Reality:** RDS manages the underlying infrastructure (hardware, patching, backups), but you still need to manage the database itself: schema design, indexing, query optimisation, user permissions, and monitoring. AWS manages the operating system and database engine, but the application-level administration is your responsibility. (The term 'fully managed' is often misinterpreted as 'no work required'. Beginners think managed means hands-off completely. AWS clarifies 'managed' means infrastructure management, not application management.)
- **Misconception:** Amazon Redshift is just a bigger version of Amazon RDS. **Reality:** Redshift is a data warehouse optimised for analytical queries across massive datasets using columnar storage and massively parallel processing (MPP). RDS is a transactional (OLTP) database optimised for high-volume, low-latency row-based operations. They serve completely different purposes and have different performance characteristics. (Both are relational and support SQL, so beginners lump them together. The difference lies in workload type (OLTP vs OLAP), storage architecture (row vs column), and scaling patterns.)

## Key takeaways

- AWS offers multiple database categories including relational, key-value, document, in-memory, graph, time-series, and ledger databases, each designed for a specific data model and access pattern.
- Amazon RDS provides managed relational databases (MySQL, PostgreSQL, Oracle, SQL Server, MariaDB) with automatic patching, backups, and multi-AZ replication.
- Amazon Aurora is a MySQL and PostgreSQL-compatible relational database with better performance, availability, and auto-scaling storage compared to standard RDS engines.
- Amazon DynamoDB is a fully managed NoSQL key-value and document database with single-digit millisecond latency at any scale, ideal for high-traffic web applications.
- Amazon ElastiCache for Redis or Memcached provides in-memory caching to reduce database load and achieve microsecond response times.
- Amazon Redshift is a columnar data warehouse for analytical queries across petabytes of data, not for transactional workloads.
- Selecting the wrong database service can lead to poor performance, high costs, or application failures; the DBS-C01 exam tests your ability to match workload requirements to the correct service.
- Fully managed means AWS handles infrastructure patching, backups, and replication, but you are still responsible for schema design, query optimisation, and security configuration.

## FAQ

**What is the difference between Amazon RDS and Amazon Aurora?**

RDS is a managed service for standard relational databases (MySQL, PostgreSQL, etc.). Aurora is a proprietary relational database engine that is MySQL and PostgreSQL compatible, but offers significantly better performance (up to 5x), automatic scaling storage (up to 128 TB), and higher availability.

**Can I use Amazon S3 as a database?**

No. S3 is object storage for files, not a database. It does not support SQL queries, transactions, or indexing. However, you can use S3 for storing database backups, logs, and static data, and query it with Amazon Athena (which is SQL-based, but not a database itself).

**When should I choose DynamoDB over RDS?**

Choose DynamoDB when you need high throughput for simple key-value lookups, low latency (single-digit milliseconds), and automatic scaling without planning for capacity. Choose RDS when you need complex SQL queries, joins, ACID transactions on structured data, or when your data model requires strict schema enforcement.

**What is the difference between ElastiCache for Redis and Memcached?**

Redis supports advanced data structures (lists, sets, sorted sets), persistence to disk, replication, and clustering. Memcached is simpler, multi-threaded, and easier to scale horizontally for simple caching, but lacks persistence and complex data types. For most caching workloads with durability requirements, Redis is preferred.

**Is Amazon Redshift a transactional database?**

No. Redshift is a data warehouse designed for analytical queries (OLAP) across large datasets. It uses columnar storage and parallel processing. It is not suitable for transactional workloads (OLTP) where you need fast, atomic row-level operations and ACID compliance.

**What is Multi-AZ and why is it important?**

Multi-AZ (Multi-Availability Zone) means deploying the database in two or more separate physical locations (AZs) within an AWS region. If one AZ fails, the database automatically fails over to the standby, providing high availability. It is recommended for production workloads that cannot tolerate downtime.

## Check your understanding

1. **Which AWS database service would you use to cache frequent database queries to reduce load on an RDS instance?**

   Answer: Amazon ElastiCache (Redis or Memcached) is the managed in-memory caching service designed for this purpose.

2. **What is the key difference between Amazon RDS and Amazon Aurora in terms of storage scaling?**

   Answer: Aurora auto-scales storage up to 128 TB without manual intervention, whereas RDS has a maximum storage limit per instance (up to 16 TB) and may require manual resizing.

3. **Your application needs to store user session data with extremely low latency (sub-millisecond). Which service should you use?**

   Answer: Amazon ElastiCache for Redis, because it is an in-memory data store that provides sub-millisecond latency for simple read and write operations.

4. **A company needs to run complex analytical queries on petabytes of historical sales data. Which service is the best fit?**

   Answer: Amazon Redshift, as it is a fully managed data warehouse designed for high-performance analytical queries on large datasets.

5. **What does 'fully managed' mean in the context of Amazon RDS?**

   Answer: AWS manages the underlying infrastructure, including hardware provisioning, patching of the database engine and OS, automated backups, and replication, but you are still responsible for schema design, query optimisation, and user permissions.

---

Interactive version with quiz and diagrams: https://courseiva.com/learn/aws-database-specialty/aws-database-services-overview