Chapter 23 — CloudTrail — Audit Logging
📖 Story First
The school has grown to be very large. Hundreds of staff members, thousands of students, dozens of buildings.
One morning, the Principal walks in and discovers something alarming.
The school's financial records have been accessed. Some grades in the examination database have been changed. And the school's main website was taken down for 2 hours last night.
But there is one big problem: nobody knows who did it.
The Principal has no way to find out:
- Who accessed the financial records?
- Who logged into the examination system?
- When was the website taken down and by whom?
- Were they inside the school? From outside? From another country?
Without a record of every action taken in the school, the Principal cannot investigate, cannot fix the root cause, and cannot prevent it from happening again.
The school implements a Visitor and Activity Log System.
From now on:
- Every door has a digital scanner
- Every time someone enters or exits, it is recorded
- Every system login (financial, exam, website) is logged
- The log shows: WHO → WHAT → WHERE → WHEN
- Logs are stored safely for 90 days inside the school, and archived permanently in a secure offsite vault
Now when something goes wrong, the Principal can open the log, search for the exact event, see exactly who did what, and take corrective action.
This is CloudTrail in AWS.
🎯 Learning Objectives
By the end of this chapter, you will be able to:
- ✅ Explain what CloudTrail is and why it matters
- ✅ Understand the difference between Event History and Trails
- ✅ Understand Management Events vs Data Events
- ✅ View CloudTrail Event History in AWS Console
- ✅ Create a Trail that logs to S3
- ✅ Explain the difference between CloudWatch and CloudTrail
🏫 School Analogy
┌─────────────────────────────────────────────────────────┐
│ SCHOOL ←→ CLOUDTRAIL MAPPING │
├──────────────────────────┬──────────────────────────────┤
│ SCHOOL CONCEPT │ AWS CONCEPT │
├──────────────────────────┼──────────────────────────────┤
│ School visitor log │ CloudTrail Event History │
│ Digital door scanners │ CloudTrail recording API │
│ Offsite document vault │ S3 bucket (Trail storage) │
│ 90-day on-site log │ Event History (free, 90 days)│
│ CCTV recording someone │ Data Events (S3, Lambda) │
│ entering the building │ │
│ Manager checking roster │ Management Events (IAM, VPC) │
│ Log at the main gate │ Management Events (control │
│ (who enters school) │ plane) │
│ Log inside the library │ Data Events (data plane) │
│ (who borrowed what book) │ │
│ Security camera footage │ CloudTrail Logs (detailed) │
│ saved for 1 year │ (configurable retention) │
└──────────────────────────┴──────────────────────────────┘
☁️ The Actual Concept
AWS CloudTrail is an audit logging service that records every API call made in your AWS account.
Every single action you take in AWS — whether through the Management Console, the AWS CLI, SDKs, or automated tools — is an API call. CloudTrail records all of them.
Think of CloudTrail as a permanent, uneditable surveillance camera pointed at every action in your account. It answers the three most important security questions:
┌─────────────────────────────────────────────────────────┐
│ CLOUDTRAIL ANSWERS │
├─────────────────────────────────────────────────────────┤
│ │
│ 🔍 WHO made the call? │
│ → IAM user, role, or federated user │
│ │
│ 🔍 WHAT did they do? │
│ → The exact API action (e.g., RunInstances) │
│ │
│ 🔍 WHEN and WHERE? │
│ → Timestamp + Source IP address + Region │
│ │
│ 🔍 What was the result? │
│ → Success or failure, with error codes │
│ │
└─────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────┐
│ CLOUDTRAIL FACTS │
├─────────────────────────────────────────────────────────┤
│ │
│ • Event History = Free, 90 days retention │
│ • Trail = Long-term storage in S3 │
│ • Management Events = Control plane (create, delete) │
│ • Data Events = Data plane (S3 GetObject, Lambda inv.) │
│ • Enabled by default (Event History) │
│ • Trails can be organization-wide │
│ • Logs are delivered to S3 within ~15 minutes │
│ • Log files are JSON format │
│ • Logs can be analyzed with Athena or CloudWatch Logs │
│ │
└─────────────────────────────────────────────────────────┘
🎭 Event History vs Trail — What is the Difference?
This is a very common question in AWS interviews. Understand this clearly.
┌───────────────────────────────────────────────────────────────┐
│ EVENT HISTORY vs TRAIL │
├───────────────────────┬───────────────────────────────────────┤
│ EVENT HISTORY │ TRAIL │
├───────────────────────┼───────────────────────────────────────┤
│ Free │ Free to create, pay for S3 storage │
│ Available by default │ You must create it manually │
│ 90 days retention │ Unlimited (as long as in S3) │
│ Management events only│ Management + Data events (optional) │
│ Single Region only │ All Regions (or single Region) │
│ Cannot customize │ Full customization │
│ Quick view in console │ Long-term auditing & compliance │
│ │ │
│ School: The register │ School: Copies of all registers │
│ at the front gate │ sent to a secure vault, stored │
│ (kept for 90 days) │ forever │
└───────────────────────┴───────────────────────────────────────┘
🗂️ Management Events vs Data Events
┌───────────────────────────────────────────────────────────────┐
│ MANAGEMENT EVENTS vs DATA EVENTS │
├───────────────────────┬───────────────────────────────────────┤
│ MANAGEMENT EVENTS │ DATA EVENTS │
│ (Control Plane) │ (Data Plane) │
├───────────────────────┼───────────────────────────────────────┤
│ What they record: │ What they record: │
│ • CreateVPC │ • GetObject (S3) │
│ • RunInstances (EC2) │ • PutObject (S3) │
│ • DeleteDBInstance │ • Invoke (Lambda) │
│ • CreateUser (IAM) │ • GetItem (DynamoDB) │
│ • ModifySecurityGroup │ • PutRecord (Kinesis) │
│ │ │
│ Recorded by default │ NOT recorded by default │
│ in Event History │ (must be enabled in Trail) │
│ │ │
│ Who changed the │ What data was accessed │
│ infrastructure │ inside the resources │
│ │ │
│ High volume │ VERY high volume (costs more) │
│ Low cost │ Can generate GBs per hour │
│ │ │
│ School: Security at │ School: Detailed tracking of │
│ the main gate — who │ who opened which file in │
│ entered the campus │ the library │
└───────────────────────┴───────────────────────────────────────┘
📄 What Does a CloudTrail Log Look Like?
CloudTrail logs are stored as JSON files in S3. Here is a simplified example:
{
"eventVersion": "1.08",
"userIdentity": {
"type": "IAMUser",
"principalId": "AIDACKCEVSQ6C2EXAMPLE",
"arn": "arn:aws:iam::123456789012:user/rahul-dev",
"userName": "rahul-dev"
},
"eventTime": "2024-11-15T10:30:15Z",
"eventSource": "ec2.amazonaws.com",
"eventName": "RunInstances",
"awsRegion": "ap-south-1",
"sourceIPAddress": "203.0.113.45",
"userAgent": "console.amazonaws.com",
"requestParameters": {
"instanceType": "t2.micro",
"imageId": "ami-0abcdef1234567890"
},
"responseElements": {
"instancesSet": {
"items": [{ "instanceId": "i-0abcd1234efgh5678" }]
}
},
"eventType": "AwsApiCall",
"recipientAccountId": "123456789012"
}
Reading this log in plain English:
"On November 15, 2024 at 10:30:15 UTC, IAM user 'rahul-dev' (from IP address 203.0.113.45) called the RunInstances API in the Mumbai region and launched a t2.micro EC2 instance with ID i-0abcd1234efgh5678. The call was successful."
This level of detail is what makes CloudTrail invaluable for security investigations and compliance audits.
🧪 Hands-On Lab — View CloudTrail Event History
STEP 1: Go to CloudTrail Console
Search → "CloudTrail" in AWS Console
STEP 2: Click "Event History" (left menu)
You will see ALL API calls in your account
for the past 90 days
STEP 3: Filter events:
Try filtering by:
• User name → "root" (see what root did)
• Event name → "CreateVpc" or "RunInstances"
• Resource type → "AWS::EC2::Instance"
STEP 4: Click on any event to see full details:
• Who made the call
• When it happened
• From which IP address
• What the response was
• Which region
✅ You are now looking at CloudTrail Event History!
🧪 Hands-On Lab — Create a Trail
A Trail sends CloudTrail logs to an S3 bucket for long-term storage and analysis.
STEP 1: Go to CloudTrail Console → Trails → "Create trail"
STEP 2: Trail details:
Trail name: MyAuditTrail
Enable for all accounts in organization: No
(We are using a single account)
STEP 3: Trail log bucket:
Create new S3 bucket
Name: my-audit-logs-<your-unique-id>
(S3 bucket names must be globally unique)
STEP 4: Log file SSE-KMS encryption: Disabled (for learning)
(In production, enable SSE-KMS for security)
STEP 5: CloudWatch Logs (optional):
Skip this for now (we can add later)
STEP 6: Management events:
Read/Write: All
(Records both reads and writes)
STEP 7: Data events (for advanced):
Uncheck for now (data events generate high volume)
(In production, you might enable these for S3)
STEP 8: Create trail → Click "Create trail"
✅ Trail is created! All API calls from now on will
be logged to the S3 bucket for permanent storage.
🔍 Investigating with CloudTrail — Real Scenarios
Scenario 1: Who deleted an S3 bucket?
1. Go to CloudTrail → Event History
2. Filter: Event Name = "DeleteBucket"
3. Filter: Time Range = "Last 7 days"
4. Result: Shows exactly who called DeleteBucket, when,
from which IP, and whether it was successful.
Scenario 2: Who stopped that EC2 instance at 2 AM?
1. Go to CloudTrail → Event History
2. Filter: Event Name = "StopInstances"
3. Filter: Time Range = "Custom" → select the date
4. Filter: Resource Name = "i-0abcd1234efgh5678"
5. Result: Shows the IAM user who stopped it at 2:03 AM
from IP address 103.x.x.x
Scenario 3: Security incident — unauthorized access?
1. Go to CloudTrail → Event History
2. Look for patterns:
• Multiple "ConsoleLogin" failures from same IP
• "Unauthorized" operations in response elements
• API calls from unusual geographic locations
3. Cross-reference with GuardDuty findings (if enabled)
🔐 CloudTrail Security Best Practices
┌─────────────────────────────────────────────────────────┐
│ CLOUDTRAIL BEST PRACTICES │
├─────────────────────────────────────────────────────────┤
│ │
│ 1. Create a Trail in EVERY Region │
│ → Or use "Apply to all Regions" option │
│ → You do not want gaps in your audit trail │
│ │
│ 2. Enable log file validation │
│ → Ensures logs have not been tampered with │
│ → Digital signatures on every log file │
│ │
│ 3. Protect the S3 bucket with MFA Delete │
│ → Prevents deletion of audit logs │
│ → Even the Root account needs MFA to delete │
│ │
│ 4. Use CloudTrail with CloudWatch Logs │
│ → Set up metric filters for suspicious events │
│ → Trigger alarms on: Unauthorized API calls, │
│ root user activity, console login failures │
│ │
│ 5. Monitor CloudTrail itself │
│ → "StopLogging" events from CloudTrail are a │
│ red flag — set up an alarm for this! │
│ │
└─────────────────────────────────────────────────────────┘
💡 Pro Tips
💡 Tip 1: Enable a Trail the day you create your AWS account. Even before you launch any resources. The Event History only keeps 90 days, but if you create a Trail on day 1, you will have logs going back to day 1 stored in S3 forever. It is the first thing you should do after setting up billing alerts.
💡 Tip 2: Do NOT enable Data Events for all S3 buckets unless you need them. Data events on S3 can produce millions of log entries per day, costing significant money. For learning, Management Events are sufficient. Enable Data Events only for specific high-security buckets.
💡 Tip 3: Use CloudTrail Insights to automatically detect unusual API activity. CloudTrail Insights uses machine learning to identify patterns like abnormal volumes of API calls, unusual error rates, or calls originating from unexpected geographies. It highlights these as "Insights Events" so you do not have to manually hunt through millions of logs.
💡 Tip 4: Combine CloudTrail with AWS Config for full governance. CloudTrail tells you who made a change and when. AWS Config tells you what the resource looked like before and after the change. Together, they give you complete change management visibility.
❓ Quick Quiz
🎤 Interview Questions
Q: What is AWS CloudTrail and why is it important?
CloudTrail is a service that records every API call in your AWS account, including who made the call, when, from which IP address, and what the response was. It is important for security investigations, operational troubleshooting, and compliance auditing. Without CloudTrail, you would have no way to determine what happened in your account after a security incident.
Q: What is the difference between CloudTrail Event History and a Trail?
Event History is a free, 90-day rolling log of management events only. It is enabled by default. A Trail is a resource you create to deliver CloudTrail logs to an S3 bucket for long-term storage (years if needed). Trails can capture both management and data events, can apply to all regions, and can be configured with log file validation and encryption.
Q: How would you detect if someone is trying to brute-force access your AWS account?
I would create a CloudTrail Trail that delivers to CloudWatch Logs. Then I would set up a metric filter for "ConsoleLogin" events with a failure status. I would configure a CloudWatch Alarm on this metric that triggers if there are more than, say, 5 failed login attempts within 5 minutes. The alarm could send an SNS notification or even trigger a Lambda function to automatically block the source IP using a WAF or NACL.
Q: What security measures should you take with your CloudTrail logs?
First, enable log file validation so you can detect if logs were tampered with. Second, protect the S3 bucket with a policy that prevents anyone (including the root user) from deleting logs without MFA. Third, do not store the logs in the same account that is being audited — use a separate, locked-down audit account. Fourth, encrypt the logs with KMS. Finally, set up CloudWatch alarms for critical events like "StopLogging" or "DeleteTrail."
Q: How is CloudWatch different from CloudTrail?
CloudWatch monitors performance and operational health — CPU utilization, memory, application logs, and setting alarms on thresholds. CloudTrail records API activity for auditing — who created an EC2, who deleted an S3 bucket, who modified a security group. CloudWatch is about "is everything running well?" CloudTrail is about "who did what and when?"
📝 Chapter Summary
┌─────────────────────────────────────────────────────────┐
│ CHAPTER 23 SUMMARY │
├─────────────────────────────────────────────────────────┤
│ │
│ ✅ CloudTrail = Records ALL API calls in AWS │
│ ✅ Like a school security log — every entry recorded │
│ ✅ Event History = Free, 90 days, management only │
│ ✅ Trail = Long-term storage in S3, unlimited │
│ ✅ Management Events = Create/delete resources │
│ ✅ Data Events = Operations inside resources (S3, etc)│
│ ✅ Logs include: Who, What, When, Where, How, Result │
│ ✅ Logs stored as JSON in S3 │
│ ✅ Enable log file validation to prevent tampering │
│ ✅ Create a Trail the FIRST day you open your account │
│ ✅ CloudWatch = monitoring; CloudTrail = auditing │
│ │
└─────────────────────────────────────────────────────────┘