Amazon DOP-C02: Storage, Databases and Data Management — Study Guide
Part of the AWS DevOps Engineer Professional DOP-C02 — Study Guide. Practice with verified answers in the Amazon exam hub, or take timed practice tests on ExamRoll.io.
Overview
Storage, databases, and data movement on AWS must be designed for durability, availability, cost efficiency, and automation. Mastering S3 storage classes and replication, DynamoDB capacity and global distribution, RDS/Aurora configuration controls and backup patterns, in-memory caching, shared file systems, and data migration services enables reliable, low-latency systems with predictable recovery behavior and controlled spend.
Amazon S3: storage classes, lifecycle, intelligent tiering, and replication
S3 storage classes align cost to access patterns:
- Standard: multi-AZ, low latency, no retrieval fees. Default for hot data.
- Intelligent-Tiering (S3 INT): multi-AZ with automatic tiering across Frequent and Infrequent Access tiers and optional archive tiers. Monitoring and automation charge per object; objects smaller than 128 KB aren’t auto-tiered. Archive Access and Deep Archive Access tiers are opt-in with last-access thresholds; retrieval fees apply from non-frequent tiers.
- Standard-IA and One Zone-IA: lower storage cost with retrieval fees; 30-day minimum storage charge. One Zone-IA is single-AZ for recreatable data.
- Glacier Instant Retrieval: millisecond access with archive economics; 90-day minimum.
- Glacier Flexible Retrieval: minutes to hours retrieval, bulk/standard/expedited options; 90-day minimum.
- Glacier Deep Archive: hours to 12 hours retrieval; 180-day minimum. Choose the coldest viable tier while considering minimum storage duration charges, retrieval fees, and required access times.
Lifecycle policies automate transitions and expirations using filters (prefix, tags) for fine-grained control. Key actions include transition to IA/Glacier tiers after inactivity thresholds, noncurrent-version transition/expiration in versioned buckets, expiration of delete markers, and abort incomplete multipart uploads. Lifecycle and object tagging are crucial to enforce data retention and defensible deletion alongside S3 Object Lock (governance/compliance modes) when immutability is required.
Intelligent-Tiering is ideal when access patterns are unknown or variable. It preserves performance (no retrieval delay from frequent/IA tiers), eliminates re-architecture when patterns change, and can optionally auto-archive into deep tiers based on last access, providing the best blend of agility and cost control for long-lived datasets with sporadic access.
S3 replication provides durable, asynchronous copy of objects:
- Requirements: versioning enabled on source and destination. Replication configuration defines destination bucket/account/Region, filter by prefix/tags, metadata replication (ACLs, tags, S3 Object Lock), storage class, and whether to replicate delete markers and existing objects.
- Same-Region Replication (SRR): compliance/data sovereignty, log aggregation, atomic processing across accounts.
- Cross-Region Replication (CRR): DR, latency reduction, global distribution, compliance.
- KMS-encrypted objects: the replication role must be allowed to decrypt with the source KMS key and encrypt with the destination KMS key. Specify the replica KMS key in the replication rule EncryptionConfiguration. For cross-account, update the destination bucket policy to allow the replication role to write.
- Existing objects: use S3 Batch Replication to backfill.
- Replication Time Control (RTC): adds a 15-minute SLA for replication completion, with replication metrics and notifications to monitor SLAs. Useful for compliance and strict RPOs.
- Ownership and access: when crossing accounts, enable bucket owner preferred or Object Ownership bucket owner enforced to avoid ACL complexity and ensure the destination account owns replicas.
Databases on AWS: DynamoDB, RDS, and Aurora
DynamoDB capacity modes and scaling:
- On-demand: no capacity planning; per-request pricing; ideal for unpredictable or spiky workloads and for new tables without known traffic.
- Provisioned: set RCUs/WCUs with DynamoDB Application Auto Scaling on target utilization; appropriate for steady or predictable traffic and cost control.
- Adaptive capacity: auto-redistributes partition throughput to hot keys, but extreme hot partitions still need load-leveling (e.g., write sharding). GSIs have separate capacity; model carefully to avoid throttling.
- Item size affects capacity: 1 WCU per 1 KB write; 1 RCU per 4 KB strongly consistent read or 8 KB eventually consistent read.
DynamoDB streams and DAX:
- Streams capture item-level mutations with a 24-hour retention. Choose view types to include NEW/OLD images. Common patterns: Lambda triggers for CQRS/event-driven writes, cross-table sync, and audit trails. Ordering is per partition key, and delivery is at-least-once.
- DAX is a managed, API-compatible, in-memory cache for DynamoDB that drastically reduces read latency. It supports eventually consistent reads; strongly consistent reads must bypass DAX. It offers write-through for item mutations and TTL-based invalidation. Use multi-AZ clusters for high availability and place DAX subnets close to clients.
DynamoDB global tables:
- Multi-Region, multi-master replication using streams with last-writer-wins conflict resolution based on a system timestamp. Design to avoid concurrent updates to the same attributes across Regions or implement application-side reconciliation.
- TTL attribute replicates as normal item data; TTL-driven deletions are processed per Region and are not replicated as explicit deletes.
- Backups and PITR are Region-scoped; restore to new tables per Region and (optionally) re-create as a new global table.
Amazon RDS configuration and backups:
- Parameter groups define engine parameters. Static parameters require reboot; dynamic parameters apply immediately where supported. Use DB parameter groups for instance-level engines and cluster parameter groups for Aurora.
- Option groups enable engine-native features (e.g., Oracle TDE/OEM, SQL Server native backup/restore, MySQL/MariaDB plugins). Options can require engine restarts; manage change windows carefully.
- Automated backups enable PITR within a retention window (up to 35 days). They capture daily snapshots and transaction logs to S3; restores produce new instances.
- Manual snapshots are retained until deleted, are copyable across Regions, and are shareable across accounts (respecting KMS key permissions for encrypted snapshots). Use snapshot copy across Regions for DR seeds.
Amazon Aurora specifics:
- Endpoints: the cluster (writer) endpoint always points to the primary for writes. The reader endpoint load-balances across replicas. Custom endpoints can select a subset of readers for tiered read pools or specialized workloads. Always point writes to the writer endpoint and reads to the reader/appropriate custom endpoint for minimal interruptions during failovers.
- Serverless v2: fine-grained, instant scaling of ACUs without restart. It runs within an Aurora cluster, supports mixed serverless and provisioned instances, and is well-suited to spiky workloads, dev/test, or multi-tenant apps with uneven demand. It preserves connection consistency better than v1 due to continuous scaling.
- Cloning: fast, copy-on-write clones within a Region for dev/test, data science, or blue/green change validation. Clones are space-efficient and diverge only on changed pages. You can chain clones; delete when done to reclaim storage.
Caching and shared file systems: ElastiCache and EFS
ElastiCache for Redis vs Memcached:
- Redis: advanced data structures, replication, Pub/Sub, Lua, streams, geospatial, sorted sets, and persistence via snapshots; supports Multi-AZ automatic failover and Redis Global Datastore for cross-Region read replicas. Choose Redis when you need rich data types, durability (snapshot restore), or high availability with failover.
- Memcached: simple, multithreaded, no replication or persistence; scale-out via client-side sharding; stateless and easy to horizontally scale. Choose Memcached for ephemeral, pure caching with very high throughput and when you want to control sharding at the client. Redis cluster mode and replication groups:
- Cluster mode disabled: one shard with a primary and replicas; vertical scale or limited horizontal scaling via read replicas.
- Cluster mode enabled: hash-slot sharding across multiple primary shards, each with replicas, enabling near-linear scale-out. Replication groups define primary/replica topology and Multi-AZ failover. Backups are per replication group; test failover to validate RTO.
Amazon EFS for shared POSIX files:
- Mount targets: create one in each AZ of the VPC to ensure in-AZ access paths and availability. Security groups on mount targets control NFS traffic; use the EFS mount helper for TLS-in-transit and IAM authorization if required.
- Access points: enforce a root directory and POSIX identity (UID/GID) for applications, enabling multi-tenant isolation and simple, least-privilege mounting by ECS/EKS/EC2 without coordinating OS user management.
- Lifecycle management and storage classes: EFS Standard and Standard-IA (Regional, multi-AZ) and One Zone/One Zone-IA (single-AZ). Intelligent tiering automatically moves files between standard and IA classes based on last access time; you can also set explicit transition policies. Choose One Zone variants for recreatable or non-critical data to save cost. Combine with AWS Backup for centralized policies and cross-account/Region backups.
Data migration: DMS, Snowball, and DataSync
- AWS Database Migration Service (DMS): online migration with minimal downtime using full load plus change data capture (CDC). Supports homogeneous and heterogeneous migrations through built-in schema conversion (with AWS Schema Conversion Tool for complex conversions). Use for lift-and-shift to RDS/Aurora, to DynamoDB (via JSON mapping), or ongoing replication for read offloading or phased cutovers. Size replication instances for peak change rates; ensure source logs (e.g., binlog/redo) retain enough history.
- AWS Snowball (Edge Storage/Compute Optimized): petabyte-scale, offline data transfer when networks are limited/expensive or when you need to seed massive S3/EFS datasets quickly. Chain multiple devices for multi-petabyte loads. Use for initial bulk loads, remote/edge collection, or migration out of constrained data centers. Data is encrypted end-to-end with KMS; device tracking and tamper-evident seals support chain of custody.
- AWS DataSync: online, accelerated transfer for NFS/SMB to S3/EFS/FSx and between AWS storage services/Regions. It handles incremental change detection, parallelization, compression, bandwidth control, scheduling, and integrity checks. Use to move recurring deltas, hybrid workflows, and to replace custom rsync scripts with managed automation. Deploy the DataSync agent on-prem to access local storage.
Practical Problem Scenario
Shopify needs to modernize its global product media pipeline and catalog data while improving resilience and latency for buyers worldwide. The company must: replicate product images across Regions and accounts with strict RPO, reduce DynamoDB read latency in North America and Europe, migrate on-prem NFS assets with ongoing deltas, and simplify RDS operations with reliable backups.
- Implement S3 CRR with Replication Time Control from the primary media bucket in us-east-1 (merchandising account) to a destination bucket in eu-west-1 (delivery account).
- Why: CRR satisfies cross-Region and cross-account separation for least privilege and data sovereignty. RTC provides a 15-minute replication SLA and monitoring for compliance-grade RPO. Cross-account bucket policy ensures the source replication role can write, and specifying a destination KMS key maintains encryption domains.
- Define S3 replication rules filtered by prefix and tag to segregate originals, thumbnails, and logs, and enable replication of delete markers. Use S3 Batch Replication to backfill legacy objects.
- Why: Rule scoping avoids unnecessary replication cost, and delete-marker replication keeps Regions semantically consistent. Batch Replication closes historical gaps without bespoke scripts.
- Convert product catalog and inventory to a DynamoDB global table across us-east-1 and eu-west-1; switch tables to on-demand capacity and add DAX clusters per Region for read-heavy APIs.
- Why: Global tables provide active-active writes with low-latency local reads/writes and continuous replication. On-demand removes capacity planning risk during traffic spikes. DAX reduces P99 latencies for hot reads, shielding DynamoDB from bursty access.
- Replatform the orders relational workload to Amazon Aurora MySQL with writer and reader endpoints; add a small Aurora Serverless v2 reader for spiky analytics and enable automated backups with a 14-day retention policy.
- Why: Cluster/reader endpoints decouple read/write and minimize disruption during maintenance or failover. Serverless v2 absorbs unpredictable analytic bursts cost-effectively. Automated backups deliver PITR and simplified restore processes.
- Introduce ElastiCache for Redis (cluster mode enabled) for session storage and product-availability caching with Multi-AZ and snapshot backups; set TTLs aligned to business SLAs.
- Why: Redis data structures and Multi-AZ failover ensure fast, stateful sessions and near-real-time cache invalidation. Cluster mode scales horizontally as catalog size and traffic grow.
- Create an EFS Regional file system with mount targets in each application AZ and Access Points for workloads that require shared POSIX storage (e.g., media processors). Enable EFS lifecycle transitions to IA after 30 days.
- Why: EFS provides elastic, multi-AZ shared storage; Access Points enforce per-application isolation and POSIX identities. Lifecycle management cuts costs automatically for cold assets that remain accessible.
- Migrate on-prem NFS media libraries using AWS DataSync with scheduled tasks for nightly incremental syncs into S3 and EFS.
- Why: DataSync handles change detection, parallelism, integrity verification, and bandwidth control better than ad hoc rsync, automating ongoing deltas with minimal ops.
- Move the legacy PostgreSQL catalog to Aurora using AWS DMS (full load plus CDC) and the AWS Schema Conversion Tool where needed; cut over after CDC lag drains.
- Why: DMS enables near-zero-downtime migration, with continuous replication ensuring data parity at cutover. SCT handles engine-specific conversions.
- Seed multi-petabyte historical media into S3 using Snowball Edge devices, then switch to DataSync for ongoing increments.
- Why: Snowball accelerates bulk initial transfer without saturating WAN links; DataSync sustains continuous updates post-seed with verification and scheduling.
This architecture reduces global read latencies, gives predictable replication RPOs, simplifies relational operations and backups, centralizes shared storage with access controls, and provides a pragmatic path from bulk offline migration to automated, incremental data movement.
← Event-Driven Architectures and Automation · All domains · Networking and Content Delivery →
Practice these questions → · Timed practice on ExamRoll.io →
Pass the whole exam — not just this question
You found this answer. Get every verified question and explanation in one place, and save hours of prep. Free to start.
Pass your exam →