Non-Relational DB Types: Document, Key-Value, Graph, Column

How it works internally: The database indexes fields within documents to allow queries on any field. For example, in Azure Cosmos DB (which supports document model via SQL API), documents are stored in a collection, and the database automatically indexes all properties by default. When you query, it uses the index to locate matching documents. Documents are identified by a unique ID within a collection. Partitioning is based on a partition key, which distributes documents across physical partitions.

Key components: - Collection: A container for documents (like a table in relational). - Document: A JSON object with fields and values. - ID: Unique identifier within a collection. - Partition key: A property used to distribute data across partitions.

Configuration and verification: In Azure Cosmos DB, create a database and a container with a partition key. Use the Data Explorer to insert and query documents.

Example:

{
  "id": "1",
  "name": "John Doe",
  "address": {
    "street": "123 Main St",
    "city": "Seattle"
  },
  "orders": [
    { "orderId": "A100", "total": 50.00 }
  ]
}

Interaction with related technologies: Document databases often support SQL-like query languages (e.g., Cosmos DB SQL API). They can be used with change feed for real-time processing.

How it works internally: The database uses a hash table or B-tree to map keys to storage locations. Lookups by key are extremely fast (O(1) average). There is no support for querying by value or relationships; you must know the key to retrieve the value. In Azure, Redis Cache is a popular key-value store. It stores data in memory for low latency. Azure Cosmos DB also offers a Table API that is a key-value store (similar to Azure Table Storage).

Key components: - Key: Unique identifier (string, up to 1024 bytes in Cosmos DB Table API). - Value: Data associated with the key (max size 1 MB per entity in Azure Table Storage). - Partition key: Determines partition for scalability.

Configuration and verification: In Azure Redis Cache, use SET key value and GET key. In Cosmos DB Table API, use SDKs to insert entities with PartitionKey and RowKey.

Example:

SET user:1000 '{"name":"Alice","email":"alice@example.com"}'
GET user:1000

Interaction with related technologies: Key-value stores are often used for caching, session management, and real-time data. They integrate with web apps via Redis clients.

How it works internally: The database uses adjacency lists or index-free adjacency to store connections. Each node stores pointers to its adjacent nodes, making traversals very fast regardless of graph size. Queries use graph query languages like Gremlin or SPARQL. In Azure, Cosmos DB offers Gremlin API for graph data.

Key components: - Node (vertex): Represents an entity (e.g., person, product). - Edge (relationship): Connects two nodes with a direction and label (e.g., "knows", "purchased"). - Properties: Key-value pairs on nodes and edges. - Label/Type: Categorizes nodes or edges (e.g., "Person", "Product").

Configuration and verification: In Cosmos DB, create a graph container and use Gremlin queries:

g.addV('person').property('name', 'Alice')
g.addV('person').property('name', 'Bob')
g.V().has('name','Alice').addE('knows').to(g.V().has('name','Bob'))
g.V().has('name','Alice').out('knows').values('name')

Interaction with related technologies: Graph databases are used for social networks, recommendation engines, and fraud detection. They can be integrated with analytics tools.

How it works internally: Data is stored on disk in column-oriented format, allowing efficient compression and I/O for column scans. In Apache Cassandra (used in Azure Cosmos DB Cassandra API), data is partitioned by partition key and sorted within a partition by clustering keys. Writes are append-only to commit logs and memtables, then flushed to SSTables.

Key components: - Column family (table): Container for rows. - Row key (partition key): Determines partition. - Column: Has a name, value, and timestamp. Columns can be added dynamically. - Clustering key: Sorts columns within a partition.

Configuration and verification: In Cosmos DB Cassandra API, use CQL:

CREATE TABLE users (user_id UUID PRIMARY KEY, name text, email text);
INSERT INTO users (user_id, name, email) VALUES (uuid(), 'Alice', 'alice@example.com');
SELECT * FROM users;

Interaction with related technologies: Column-family stores are used for time-series data, IoT, and large-scale analytics. They integrate with Spark and Hadoop.

Scenario 1: E-commerce Product Catalog An online retailer needs to store product information where each product can have different attributes (e.g., electronics have 'wattage', clothing has 'size'). Using a relational database would require a complex schema with many tables or sparse columns. They choose Azure Cosmos DB with SQL API (document model). Each product is a JSON document with flexible fields. The 'category' field is used as partition key to distribute products evenly. Queries like 'find all products with price < $50' are supported by indexing. They provision 10,000 RU/s to handle peak traffic. A common misconfiguration is choosing a partition key with low cardinality (e.g., 'productType' with only 3 values), causing hot partitions and throttling. The solution is to use a high-cardinality key like 'productId' or a synthetic key.

Scenario 2: Real-Time Leaderboard A gaming company needs a low-latency leaderboard that updates frequently and supports point lookups by player ID. They use Azure Redis Cache as a key-value store. Player scores are stored as strings with key pattern 'player:{playerId}:score'. Updates are done with SET commands. The leaderboard is maintained using Redis sorted sets (ZADD). This provides sub-millisecond latency. A common mistake is using a relational database for this, which would introduce latency and scalability issues. Another mistake is not setting an expiration on session data, leading to memory exhaustion.

Scenario 3: Social Network Friend Recommendations A social media platform wants to suggest friends based on mutual connections. They use Cosmos DB with Gremlin API (graph model). Users are nodes, 'friend' relationships are edges. Traversals like 'find friends of friends' are efficient using graph queries. They partition by user ID. A misconfiguration would be using a document model and trying to store friend lists as arrays in documents, which makes traversals expensive and limits scale. The graph model naturally supports this use case.