AWS EC2 Governance: Real-Time Alerts for Manual Instance (Standalone) Creation (Complete Guide)

Problem Statement & Impact

Maintaining a secure, compliant, and cost-optimized AWS environment requires real-time visibility into resource provisioning lifecycle events. In enterprise cloud architectures, compute infrastructure such as Amazon Elastic Compute Cloud (EC2) instances must adhere to strict infrastructure-as-code (IaC) deployment pipelines—using frameworks such as Terraform, AWS CloudFormation, or managed orchestration services like Auto Scaling Groups (ASGs). When administrators or developers bypass these governance pipelines and spin up standalone EC2 instances directly via the AWS Management Console, AWS CLI, or SDKs, it introduces severe operational, financial, and security risks.

The core technical challenge stems from how AWS CloudTrail logs compute allocation events. By default, both automated ASG scaling actions and manual operator requests generate the identical RunInstances API event within AWS CloudTrail. Consequently, generic alerting mechanisms configured to catch every RunInstances call create overwhelming operational noise. Security and FinOps teams are bombarded with alerts for expected, automated scaling events, causing alert fatigue and increasing the likelihood that unauthorized, rogue, or improperly configured manual instances go completely unnoticed.

The business and security consequences of lacking granular detection for manual instance creation include:

• Uncontrolled Cloud Spend: Developers frequently launch high-performance instances (e.g., c5.4xlarge or gpu instances) for ad-hoc debugging or proof-of-concept testing and forget to terminate them, resulting in unexpected budget overruns.
• Security Posture Drift: Manually created instances often bypass organizational security baselines, lacking mandated security group restrictions, centralized Identity and Access Management (IAM) role attachments, automated vulnerability scanning agents, or required resource tagging policies.
• Compliance and Auditing Failures: Regulatory frameworks (such as SOC 2, ISO 27001, and HIPAA) mandate strict change management tracking. Untracked manual infrastructure modifications violate change control policies and hinder forensic investigations during security incidents.

Root Cause Analysis & Architecture

To construct a precise filtering mechanism, we must analyze the structural payload of the CloudTrail event generated when an Amazon EC2 instance starts. Every API call made to AWS is recorded as a JSON-formatted event object. Within the context of EC2 provisioning, the RunInstances event captures parameters including the caller’s identity, source IP, request parameters, and critically, the userAgent string.

The userAgent field provides granular context regarding the client application, SDK, or internal AWS service responsible for triggering the API call. When an Auto Scaling Group provisions new instances to handle load, the internal ASG service orchestrates the call using a distinct agent signature, typically containing strings associated with the Auto Scaling internal service engine (e.g., auto-scaling.amazonaws.com). Conversely, a manual invocation executed via the AWS Management Console registers a browser-based user agent, while calls executed via the AWS CLI register specific Python/Botocore signatures devoid of orchestration engine tokens.

By leveraging Amazon EventBridge (formerly CloudWatch Events) in conjunction with AWS CloudTrail, we can intercept these JSON event payloads at the ingestion layer, evaluate them against a precise event pattern filter, and route only matching non-ASG events to a notification sink. This event-driven architecture eliminates the need for polling scripts or dedicated worker infrastructure, ensuring real-time notification delivery with sub-second latency.

Step-by-Step Implementation & Remediation Guide

Implementing this governance pipeline requires configuring three primary components: an Amazon SNS destination topic, an Amazon EventBridge filtering rule, and verification testing to ensure proper noise reduction.

Step 1: Provision and Configure the Amazon SNS Topic

First, create an SNS topic that will ingest filtered events from EventBridge and dispatch notifications to your infrastructure engineering or security operations team.

1. Navigate to the Amazon SNS console and select Topics > Create topic.
2. Select the Standard topic type (FIFO is not required for stateless notification routing).
3. Set the Name to EC2CreationAlerts and optionally define display names or encryption keys.
4. Click Create topic.
5. Within the newly created topic, click Create subscription.
6. Select Email as the protocol and input your operational distribution list email address.
7. Access your email inbox and click the Confirm subscription link provided in the AWS verification message.

Step 2: Define the Amazon EventBridge Custom Rule

Next, configure an EventBridge rule with an advanced JSON event pattern to isolate manual RunInstances calls while explicitly discarding requests originating from Auto Scaling Groups and other internal service orchestrators.

1. Navigate to the Amazon EventBridge console, select Rules, and click Create rule.
2. Name the rule EC2CreationAlerts-rule and provide a descriptive explanation (e.g., “Captures manual EC2 instance creations and filters out ASG events”).
3. Under Rule type, select Rule with an event pattern and click Next.
4. Scroll down to the Event pattern section, select Custom pattern (JSON editor), and paste the configuration block below:

{
  "source": ["aws.ec2"],
  "detail-type": ["AWS API Call via CloudTrail"],
  "detail": {
    "eventSource": ["ec2.amazonaws.com"],
    "eventName": ["RunInstances"],
    "userAgent": [{
      "anything-but": {
        "prefix": "auto-scaling.amazonaws.com"
      }
    }]
  }
}

Configuration Directive Breakdown:

• source: ["aws.ec2"]: Restricts incoming evaluations exclusively to events emitted by the EC2 service namespace.
• detail-type: ["AWS API Call via CloudTrail"]: Ensures the rule only listens to centralized CloudTrail audit logs.
• eventName: ["RunInstances"]: Targets specifically the instance initialization API action.
• userAgent.anything-but.prefix: "auto-scaling.amazonaws.com": Filters out any invocation where the user agent begins with the Auto Scaling service signature, successfully silencing automated scaling noise.

5. Under the Target(s) section, select AWS service as the target type.
6. Choose SNS topic from the dropdown menu and select your previously created EC2CreationAlerts topic.
7. Expand Additional settings if you wish to configure input transformers (e.g., to transform raw JSON into a human-readable plain text summary), then click Next.
8. Review your configuration settings and click Create rule.

Production Hardening & Edge Cases

While the basic EventBridge filter successfully captures most manual console and CLI launches, production environments require accounting for advanced edge cases and infrastructure-as-code pipelines.

Handling Infrastructure-as-Code (Terraform & CloudFormation):
When engineers deploy infrastructure via Terraform, the AWS provider executes API calls using standard Botocore/Python user agent strings (e.g., Boto3/1.X.X Python/3.X.X Linux/X.X). Because Terraform execution user agents do not contain auto-scaling.amazonaws.com, the current EventBridge filter will trigger alerts for Terraform-driven deployments. If your organization relies strictly on CI/CD pipelines for infrastructure provisioning, you must further refine your EventBridge pattern to exclude specific IAM role ARN prefixes utilized by your automation runners:

{
  "source": ["aws.ec2"],
  "detail-type": ["AWS API Call via CloudTrail"],
  "detail": {
    "eventSource": ["ec2.amazonaws.com"],
    "eventName": ["RunInstances"],
    "userIdentity": {
      "sessionContext": {
        "sessionIssuer": {
          "userName": [{
            "anything-but": ["TerraformDeploymentRole", "CloudFormationExecutionRole"]
          }]
        }
      }
    },
    "userAgent": [{
      "anything-but": {
        "prefix": "auto-scaling.amazonaws.com"
      }
    }]
  }
}

CloudTrail Latency Considerations:
CloudTrail event delivery to EventBridge typically occurs within seconds, but network partitions or regional API throttling can introduce ingestion latency ranging from 5 to 15 minutes. Security engineers must not rely on this alert mechanism as a real-time blocking inline security control; rather, treat it as a near-real-time auditing and notification guardrail.

Diagnostic Verification & Checklist

To validate that your governance pipeline operates correctly across both standalone and automated scenarios, execute the following verification matrix:

Post-Implementation Monitoring Commands:
Verify that EventBridge is successfully invoking your targets by inspecting CloudWatch metrics via the AWS CLI:

aws cloudwatch get-metric-data --metric-data-queries '[
  {
    "id": "m1",
    "metricStat": {
      "metric": {
        "namespace": "AWS/Events",
        "metricName": "Invocations",
        "dimensions": [{"name": "RuleName", "value": "EC2CreationAlerts-rule"}]
      },
      "period": 300,
      "stat": "Sum"
    }
  }
]' --start-time 2026-03-01T00:00:00Z --end-time 2026-03-01T23:59:59Z