The AWS Account and Network Architecture for Regulated AI Workloads.

Where AI compliance is actually decided

Most AI governance effort goes into policy documents. Most AI compliance outcomes are decided by account boundaries, IAM, and routing tables, because those are the things that determine where data can physically go.

This is the AWS architecture we deploy for AI workloads that have to survive a real review: HIPAA, SOC 2, FedRAMP alignment, or a client contract with hard residency terms. It assumes you are using Bedrock, a self-hosted model on EC2 or EKS, or both.

Account topology

One account is not a boundary. AWS accounts are the strongest isolation primitive available, and the blast radius of a mistake is bounded by the account it happened in.

The minimum viable split for a regulated AI workload:

The rule that makes this worth the overhead: ai-dev never holds production data. The single most common finding in AI audits is a copy of production documents in a sandbox account, imported once for testing and never deleted.

Service control policies

SCPs are organization-level guardrails that no account administrator can override, including the root user. They are how you make a policy statement into a technical fact.

Four that earn their place for AI workloads:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyOutsideApprovedRegions",
      "Effect": "Deny",
      "NotAction": [
        "iam:*",
        "organizations:*",
        "sts:*",
        "cloudfront:*",
        "route53:*",
        "support:*"
      ],
      "Resource": "*",
      "Condition": {
        "StringNotEquals": { "aws:RequestedRegion": ["us-east-1", "us-west-2"] }
      }
    },
    {
      "Sid": "DenyCloudTrailTampering",
      "Effect": "Deny",
      "Action": [
        "cloudtrail:StopLogging",
        "cloudtrail:DeleteTrail",
        "cloudtrail:UpdateTrail"
      ],
      "Resource": "*"
    },
    {
      "Sid": "DenyBedrockWithoutPrivateEndpoint",
      "Effect": "Deny",
      "Action": [
        "bedrock:InvokeModel",
        "bedrock:InvokeModelWithResponseStream"
      ],
      "Resource": "*",
      "Condition": { "Null": { "aws:sourceVpce": "true" } }
    },
    {
      "Sid": "DenyUnencryptedModelArtifacts",
      "Effect": "Deny",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::acme-model-registry/*",
      "Condition": {
        "StringNotEquals": { "s3:x-amz-server-side-encryption": "aws:kms" }
      }
    }
  ]
}

The third one is the interesting one. It makes it structurally impossible to call Bedrock over the public internet from your organization, because any invocation not arriving through a VPC endpoint is denied. Policy documents ask people to use private networking; this makes the alternative fail.

Private networking

The inference VPC has no internet gateway and no NAT gateway. Nothing in it can reach the public internet, which means nothing in it can exfiltrate to the public internet.

Everything it needs arrives through VPC endpoints:

resource "aws_vpc_endpoint" "bedrock_runtime" {
  vpc_id              = aws_vpc.inference.id
  service_name        = "com.amazonaws.${var.region}.bedrock-runtime"
  vpc_endpoint_type   = "Interface"
  subnet_ids          = aws_subnet.private[*].id
  security_group_ids  = [aws_security_group.endpoints.id]
  private_dns_enabled = true

  # Endpoint policies are a second enforcement point, independent of IAM.
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect    = "Allow"
      Principal = "*"
      Action    = ["bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream"]
      Resource  = [
        "arn:aws:bedrock:${var.region}::foundation-model/anthropic.claude-*"
      ]
      Condition = {
        StringEquals = { "aws:PrincipalOrgID" = var.org_id }
      }
    }]
  })
}

The endpoint policy restricts which models can be invoked and by whom, enforced at the network edge rather than only in IAM. Add a gateway endpoint for S3 with a policy scoped to your buckets, and interface endpoints for the handful of other services you genuinely need (KMS, Secrets Manager, CloudWatch Logs, ECR for image pulls). If a service is not on that list, workloads in this VPC cannot reach it, which is the point.

For agent workloads specifically, the egress broker lives outside this VPC and is the only path to anything else.

The cross-region setting that surprises people

Bedrock's cross-region inference profiles improve throughput and availability by routing requests across multiple regions. It is a good feature and it has a compliance consequence that is easy to miss.

Per AWS's own documentation on geographic cross-region inference: data is stored only in the source region, but input prompts and output results can move outside your source region during inference, encrypted in transit across Amazon's network. Geographic profiles keep that movement inside a stated geography (US, EU, APAC). Global profiles route to supported commercial regions worldwide.

If you have told a client that their data stays in the EU, or you are working under a contract with explicit residency language, the difference between a geographic EU profile and a global profile is the difference between honoring that clause and breaching it. This is a one-line configuration choice that most teams make for performance reasons without routing it past anyone who reads the contracts.

Decide it deliberately, write down which profile type you use and why, and enforce it: the model ID or inference profile ARN your applications are permitted to call belongs in an IAM or endpoint policy, not in application config where any engineer can change it.

Keys and separation of duties

Customer-managed KMS keys, one per data domain rather than one for everything. The key policy is a second, independent access boundary: even an over-broad IAM policy in the application account cannot decrypt data whose key policy does not name that principal.

Two practices that matter more than key count. Keep key administrators and key users as different principals, so the person who can delete a key cannot read the data and vice versa. And enable KMS key rotation plus CloudTrail on key usage, because Decrypt call volume against a document-corpus key is one of the better exfiltration signals you will get.

Logging that survives the incident

Three log streams, all landing in log-archive, which no operational role can write over:

Organization CloudTrail, all accounts, all regions, delivered to an S3 bucket with Object Lock in compliance mode. Object Lock is what makes "immutable audit log" true rather than aspirational, since even an account administrator cannot shorten a compliance-mode retention period.

Bedrock model invocation logging, enabled explicitly, because CloudTrail records that InvokeModel was called but not what was in the prompt or the completion. Only invocation logging captures content. Turning it on is what lets you answer "what did the model actually see," and it also creates a log that contains every prompt, which means the destination bucket now holds client data and inherits that classification. Encrypt it with the right CMK and scope access accordingly.

VPC flow logs for the inference VPC. In a no-egress design, flow logs are close to a pure signal: an accepted connection to an unexpected destination is either a misconfiguration or an incident.

Verifying it rather than believing it

Every control above can be asserted automatically, and untested controls decay:

1. From an inference subnet, attempt to reach a public address. Expect a timeout.
2. Attempt InvokeModel with credentials but from outside the VPC endpoint. Expect an SCP denial.
3. Attempt s3:PutObject to the model registry without KMS encryption. Expect a denial.
4. Attempt to stop CloudTrail as an account administrator. Expect an SCP denial.
5. Query the deployed inference profile ARN and assert it matches the approved geography.
6. Attempt to delete an object under Object Lock. Expect a denial.

Run them on a schedule, keep the output, and you have both a working control set and the evidence package an auditor will ask for. AWS Config rules and Security Hub controls cover the drift-detection half of this; the point of the active tests is that they prove the boundary from the attacker's side rather than from the configuration's side.

The summary

Accounts are the blast radius. SCPs turn policies into physics. A VPC with no internet gateway cannot exfiltrate to the internet. Endpoint policies restrict which models can be called at the network layer. KMS key policies are a second boundary that IAM mistakes cannot cross. Object Lock makes logs immutable. And the cross-region inference profile is a compliance decision wearing a performance costume.

Build this before the first workload rather than after the first audit finding, because retrofitting account topology under a live system is one of the more expensive projects in cloud engineering.