Amazon SOA-C02: Monitoring, Logging, and Remediation — Study Guide
Part of the AWS SysOps Administrator Associate SOA-C02 — Study Guide. Practice with verified answers in the Amazon exam hub, or take timed practice tests on ExamRoll.io.
Monitoring, logging, and remediation provide the operational nervous system for AWS environments: they detect problems, give context, and drive corrective actions. This domain covers creating meaningful metrics and dashboards, collecting and retaining logs cost-effectively, building traceable application observability, and automating alerts and remediations. Good implementations balance signal-to-noise, control costs, and ensure playbooks and automations are tested and auditable.
CloudWatch metrics, dashboards, and alarms
Design metrics around business and operational SLIs (latency, error rate, queue depth, CPU/memory for infra). Use built-in metrics (EC2, RDS, ELB) plus custom metrics via PutMetricData for application-level counters (aws cloudwatch put-metric-data –namespace MyApp –metric-name OrdersPerMinute –value 42). Prefer dimensions that enable filtering (InstanceId, ServiceName) and avoid high-cardinality dimensions that explode metric costs.
Use CloudWatch dashboards to combine metrics, logs, and alarms into operational views. Create widgets in the console or via CloudFormation (AWS::CloudWatch::Dashboard) with metric math for derived metrics: compute error rate using metric math (ERRORS/SUM(REQUESTS)) and display percentiles (p50, p90, p99). For alarms, choose configuration patterns based on intent:
- Single-metric alarms for simple thresholds: aws cloudwatch put-metric-alarm –alarm-name HighCPU –metric-name CPUUtilization –namespace AWS/EC2 –statistic Average –period 300 –evaluation-periods 2 –threshold 80 –comparison-operator GreaterThanThreshold –alarm-actions arn:aws:sns:…
- Composite alarms to reduce noise by combining conditions (AND/OR) across multiple alarms.
- Anomaly detection to auto-adjust thresholds: use CloudWatch anomaly detection on a metric with expected seasonal behavior.
Decision criteria: use evaluation periods and datapoint-to-alarm settings to avoid flapping; trigger SNS, Auto Scaling, or Systems Manager Automation as alarm actions. Prefer composite alarms and anomaly detection for environments with variable baselines.
CloudWatch Logs, Logs Insights, and retention
Aggregate logs with CloudWatch Logs groups and structure them by application and environment. Create log groups via CLI: aws logs create-log-group –log-group-name /aws/myapp/frontend and enforce retention with aws logs put-retention-policy –log-group-name /aws/myapp/frontend –retention-in-days 30. Use subscription filters to stream logs to Kinesis Data Firehose (for S3/Redshift), Lambda (real-time processing), or partner tools; compress and partition in S3 to reduce storage costs.
Use CloudWatch Logs Insights for ad-hoc queries and dashboards; build saved queries that extract trace IDs and error contexts (e.g., fields @timestamp, @message | filter @message like /ERROR/ | parse “traceId=* " as traceId). Retention and ingestion cost control practices:
- Set appropriate retention per log group (7/30/90/365 days) based on compliance and troubleshooting needs.
- Export older logs to S3 via retention lifecycle or Firehose with compression and lifecycle rules to Glacier/Archive.
- Use sampling or structured logs (JSON) and Embedded Metric Format (EMF) to reduce expensive high-cardinality logs while still deriving metrics.
Decision criteria: short retention for verbose debug logs, longer retention for audit/security logs; route high-volume logs to S3 instead of indefinite CloudWatch retention.
CloudTrail, audit trails, and event history
Enable CloudTrail across all regions and accounts; create an organization trail for centralized audit logging to a secured S3 bucket, with log file validation and SSE-KMS encryption. Configure management events (Read/Write) and selectively enable data events (S3 object-level, Lambda function invoke) where detailed audit is required because data events are higher volume and cost.
Use the CloudTrail event history in the console for 90-day quick searches and CloudTrail Lake or Athena on exported S3 logs for long-term analytics and investigations. Protect the trail by:
- Enforcing multi-region trails to capture global service events.
- Integrating CloudTrail with CloudWatch Logs for near-real-time detection, or EventBridge to route specific events to Lambda/Systems Manager for automated remediation.
- Applying S3 bucket policies and S3 Object Lock (if required) to prevent tampering.
Decision criteria: enable data events only for buckets/functions where forensic visibility is necessary; use centralized trails and cross-account access patterns to simplify compliance.
Application tracing and observability (X-Ray)
Instrument applications with AWS X-Ray SDKs to emit segments and subsegments. For non-instrumented runtimes, run the X-Ray daemon/agent as a sidecar or service (ECS task, EC2 daemon or Lambda built-in tracing). Configure sampling rules to control trace volume and set service maps in ServiceLens to visualize dependencies across services. Annotate traces with business keys (userId, orderId) and record exceptions/metadata to aid triage.
Correlate traces with logs by including the X-Ray trace ID in application logs (use the trace header or SDK to get current trace ID) so CloudWatch Logs Insights queries can join logs and traces. For Lambda, enable active tracing (Console or aws lambda update-function-configuration –function-name fn –tracing-config Mode=Active) to automatically send traces to X-Ray. Use trace analytics to detect tail latencies, hotspots, and database call breakdowns.
Decision criteria: enable tracing for critical services and use adaptive sampling to limit cost; prefer structured traces (annotations/metadata) to make Logs+Traces correlation deterministic.
Automated remediation and alerting (EventBridge/Lambda)
Use EventBridge rules to match CloudWatch Alarm state changes, CloudTrail events, or custom events and route them to targets such as Lambda, Systems Manager Automation documents, Step Functions, or SNS. Create rules with input transformers to pass minimal context to the remediation action (aws events put-rule –name HighErrorRule –event-pattern ‘{“source”:[“aws.cloudwatch”],…}’). Implement Lambda functions for lightweight remediation (restart a service, revoke credentials) but use SSM Automation or Step Functions for long-running, auditable playbooks with checkpoints.
Design remediations with safety: include dry-run modes, idempotency, validation steps, IAM least-privilege, logging, and a kill-switch. Use dead-letter queues and retry policies on EventBridge/Lambda integrations and publish remediation attempts to an audit log or security trail. Test automations in a staging account and run canary tests after deployment.
Decision criteria: prefer SSM Automation or Step Functions for multi-step recoveries and human approval; use Lambda for simple, fast fixes. Always include a manual rollback or human-in-the-loop breakpoint for risky actions.
Common Pitfalls and Decision Criteria
- Relying on a single metric for health decisions: combine metrics (e.g., error rate + latency + throttles) or use composite alarms/metric math to avoid false positives.
- Not accounting for log retention and ingestion costs: set per-log-group retention, route bulk logs to S3 via Firehose with compression, and use lifecycle policies to move old data to cheaper tiers.
- Over-instrumentation with high-cardinality dimensions or trace sampling disabled: limit dimensions and enable adaptive sampling to control cost while preserving signal.
- Deploying automated remediation without testing: validate in staging, use dry-run flags, and assert idempotency and safe rollbacks before production activation.
- Alert fatigue from noisy alarms: use anomaly detection, composite alarms, suppression windows, and escalate only meaningful events to on-call.
- Missing correlation between logs, metrics, and traces: propagate trace IDs into logs and EMF metrics, and build saved Logs Insights queries and ServiceLens views to tie data together.
Practical Problem: Use-Case Scenario
Acme Payments experiences intermittent payment processing failures during peak traffic; engineers see increased latency and sporadic 5xx errors, but automated restarts have sometimes masked the root cause.
- Instrument the payment service with X-Ray SDK and EMF metrics; add trace IDs to application logs and send structured metrics OrdersFailed and OrdersProcessed via PutMetricData/EMF.
- Create CloudWatch metric math to calculate error rate (OrdersFailed / OrdersProcessed) and a composite alarm combining error rate > threshold AND p99 latency > threshold.
- Route alarm actions to an EventBridge rule that triggers a Step Functions workflow for diagnostic steps (collect recent traces/logs, run health checks) and, if safe, an automated restart via SSM Automation.
- Configure CloudTrail and CloudWatch Logs subscription to archive full logs to S3 (compressed) with lifecycle to Glacier, and set short retention in CloudWatch for verbose debug logs.
- Run end-to-end tests and canary synthetic transactions (CloudWatch Synthetics) to validate observability and remediation flow before enabling auto-remediation in production.
Rationale: correlate metrics, logs, and traces to locate the root cause rather than repeatedly treating symptoms; combine alarms to reduce noise and use auditable, tested automation (Step Functions/SSM) for safe remediation while controlling log storage costs.
All domains · High Availability →
Practice these questions → · Timed practice on ExamRoll.io →
Pass the whole exam — not just this question
You found this answer. Get every verified question and explanation in one place, and save hours of prep. Free to start.
Pass your exam →