A New Incentive for Using AWS VPC Endpoints

If you haven’t been using VPC endpoints until now, AWS' two new condition keys should make you consider doing so.

AWS enables connectivity between resources and certain AWS services using virtual devices called VPC endpoints.

According to AWS documentation, a VPC endpoint “...enables customers to privately connect to supported AWS services and VPC endpoint services powered by AWS PrivateLink. Amazon VPC instances do not require public IP addresses to communicate with resources of the service. Traffic between an Amazon VPC and a service does not leave the Amazon network.”

Using VPC endpoints has several security benefits:

• Avoiding data communication over the public internet with AWS services, which also allows for disabling public internet connectivity for the resources that need to connect with them
• The ability to apply VPC endpoint policies to create data perimeters. (See Becky Weiss’s great re:Invent 2021 presentation “Securing your data perimeter using VPC endpoints”).

Using VPC endpoints also has performance advantages, such as lower latency.

Depending on the situation, using a VPC endpoint may not be applicable. An example would be when using an AWS service for which you can’t use a VPC endpoint. (Check out the list of services for which you can create a VPC endpoint).Another example would be when making calls to resources in another AWS region. Also, as with any architecture component, you’ll want to take cost into account. (This interesting VPC Endpoint cost analysis article details how).

If you haven’t considered using VPC endpoints until now, consider it again. AWS just introduced a huge new incentive: two new condition keys for setting an effective guardrail against Amazon Elastic Cloud Compute (EC2) instances’ most common misconfiguration that attackers leverage for credentials theft.

In a retrospective on public cloud breaches, Christophe Tafani-Dereeper, Rami McCarthy and Houston Hopkins describe the exploitation of an SSRF (server-side request forgery) vulnerability to steal application cloud credentials as one of the four main causes of cloud security incidents in 2022.

In a recent blog post on the instance metadata service (IMDS) for EC2 instances, we reviewed how an SSRF vulnerability can be used to exploit EC2 instances and several effective, mitigating steps you can perform in your environment. This blog’s key message is: EC2 instances have a local service for serving applications, such as the AWS SDK, with the temporary credentials created for sessions when the EC2 service assumes the IAM role attached to the EC2 profile. In this way, applications running on the EC2 instance can access the AWS resources that the IAM role can access. If the EC2 runs vulnerable software (e.g. vulnerable to SSRF) or is otherwise misconfigured, such calls to the IMDS can be made on its behalf by an attacker, and the attacker may be able to exfiltrate the credentials from the IMDS and use them to access the resources themselves. We explore in the article how the EC2’s configuration (mainly the available API versions of the IMDS) is a significant factor in determining how hard (or easy!) it is for a malicious actor to exfiltrate the credentials.

Now back to the AWS announcement. As mentioned, the two new condition keys allow for the creation of a data perimeter that is a perfect fit for minimizing the impact of the exfiltration of credentials from EC2 instances.

The new condition keys available for use are:

• aws:EC2InstanceSourceVPC — This single-valued condition key contains the VPC ID to which an EC2 instance is deployed
• aws:EC2InstanceSourcePrivateIPv4 — This single-valued condition key contains the primary IPv4 address of an EC2 instance

When making calls over a VPC endpoint, the value for aws:SourceVpc is set to the ID of the VPC into which the endpoint is deployed, and the value aws:VpcSourceIP is set to the IP address from which the endpoint receives the request.

With the ability to reference these values, and the availability of the new condition keys, we can form a policy that looks like this:

{
  "Version": "2012-10-17",
  "Statement": [
    {
          "Effect": "Deny",
          "Action": "*",
          "Resource": "*",
          "Condition": {
                "StringNotEquals": {
                      "aws:ec2InstanceSourceVPC": "${aws:SourceVpc}"
                },
                "Null": {
                      "ec2:SourceInstanceARN": "false"
                },
                "BoolIfExists": {
                      "aws:ViaAWSService": "false"
                }
          }
    },
    {
          "Effect": "Deny",
          "Action": "*",
          "Resource": "*",
          "Condition": {
                "StringNotEquals": {
                      "aws:ec2InstanceSourcePrivateIPv4": "${aws:VpcSourceIp}"
                },
                "Null": {
                      "ec2:SourceInstanceARN": "false"
                },
                "BoolIfExists": {
                      "aws:ViaAWSService": "false"
                }
          }
    }
  ]
}

The policy ensures that any request that uses EC2 instance role credentials must be made from the VPC where the EC2 is deployed and the IP address of the EC2 instance. (The combination of both conditions is important, as the IP address is a private one and may be reused across different environments.) This means that even if the credentials of an EC2 instance are exfiltrated, the ability to use them will be highly limited. That is, even if credentials are removed from your EC2 instance, it will be impossible to use them successfully to interact with any AWS resource because their access will be denied.

Requests not made by EC2 instance role credentials are excluded from this enforcement (by the condition verifying that the ec2:SourceInstanceARN value is NOT Null). The policy also excludes requests made via an AWS service. (To learn more about why making this exclusion is important, see this post on the topic).

If you’ve made the needed validations and are confident that all EC2 calls in an AWS account go strictly through a VPC endpoint – and getting to this level of confidence requires diligence – you can apply this policy as an AWS Organizations service control policy (SCP) in your account.

However, if you feel this kind of enforcement is too cavalier (which it probably is), you can also apply the policy as an inline AWS IAM policy on an AWS IAM role used by EC2s that you are sure are making requests to AWS services through a VPC endpoint.

You can make additional modifications to the policy so it won’t break legitimate behavior that doesn’t use a VPC endpoint. For example, if the EC2 makes an action on an AWS service that doesn’t support VPC endpoints, such as CloudFront, you can use “NotAction” with those actions (which will explicitly include all actions except those you want to exclude) instead of “Action”: “*” (which will apply to all actions). So if, for example, you have an IAM role that is used by EC2s that need to list your CloudFront distributions, you can use this inline policy for it:

{
  "Version": "2012-10-17",
  "Statement": [
    {
          "Effect": "Deny",
          "NotAction": "cloudfront:ListDistributions",
          "Resource": "*",
          "Condition": {
                "StringNotEquals": {
                      "aws:ec2InstanceSourceVPC": "${aws:SourceVpc}"
                },
                "Null": {
                      "ec2:SourceInstanceARN": "false"
                },
                "BoolIfExists": {
                      "aws:ViaAWSService": "false"
                }
          }
    },
    {
          "Effect": "Deny",
          "NotAction": "cloudfront:ListDistributions",
          "Resource": "*",
          "Condition": {
                "StringNotEquals": {
                      "aws:ec2InstanceSourcePrivateIPv4": "${aws:VpcSourceIp}"
                },
                "Null": {
                      "ec2:SourceInstanceARN": "false"
                },
                "BoolIfExists": {
                      "aws:ViaAWSService": "false"
                }
          }
    }
  ]
}

Of course, these are just a few examples. SCPs are very flexible, and you can use a wide set of condition keys to customize them based on the kind of activity that occurs in your AWS accounts (e.g. exclude/apply only to specific EC2 instances, IP ranges).

If you’re currently using a VPC endpoint for all calls to AWS services and can enforce such a policy safely, it’s highly recommended that you do. If you don’t yet employ VPC endpoints, specifically for EC2s running in private subnets, it is highly recommended that you look into the possibility of doing so. The new condition keys are another great incentive to do so.

Circling back to our recent publication about IMDS, it’s important to note that this kind of enforcement works great in addition to controls such as enforcement of the IMDSv2 - to both make the exfiltration of the credentials more difficult and mitigate the unintended usage of them from outside the EC2 instance, in accordance with the principle of defense-in-depth. That being said, performing the restriction discussed in this article can be a very effective, specific mechanism for providing an important layer of protection for legacy instances that require usage of IMDSv1 so IMDSv2 can not be enforced on them.

Lior Zatlavi
[email protected]