LogoLogo
HomeGitHub RepoBook Demo
dev
dev
  • About Quilt
  • Architecture
  • Mental Model
  • Metadata Management
  • Metadata Workflows
  • Quilt Platform (Catalog) User
    • About the Catalog
    • Bucket Browsing
    • Document Previews
    • Embeddable iFrames
    • Packaging Engine
    • Query
    • Quilt+ URIs
    • Qurator Omni
    • Search
    • Visualization & Dashboards
    • Advanced
      • Athena
      • Elasticsearch
      • Removing Stacks
  • Quilt Platform Administrator
    • Admin Settings UI
    • Catalog Configuration
    • Cross-Account Access
    • Enterprise Installs
    • quilt3.admin Python API
    • Advanced
      • Package Events
      • Private Endpoints
      • Restrict Access by Bucket Prefix
      • S3 Events via EventBridge
      • SSO Permissions Mapping
      • Tabulator
      • Troubleshooting
        • SSO Redirect Loop
    • Best Practices
      • GxP for Security & Compliance
      • Organizing S3 Buckets
  • Quilt Python SDK
    • Installation
    • Quick Start
    • Editing a Package
    • Uploading a Package
    • Installing a Package
    • Getting Data from a Package
    • Example: Git-like Operations
    • API Reference
      • quilt3
      • quilt3.Package
      • quilt3.Bucket
      • quilt3.hooks
      • Local Catalog
      • CLI, Environment
      • Known Limitations
      • Custom SSL Certificates
    • Advanced
      • Browsing Buckets
      • Filtering a Package
      • .quiltignore
      • Manipulating Manifests
      • Materialization
      • S3 Select
    • More
      • Changelog
      • Contributing
      • Frequently Asked Questions
      • Troubleshooting
  • Quilt Ecosystem Integrations
    • Benchling Packager
    • Event-Driven Packaging
    • Nextflow Plugin
Powered by GitBook
On this page
  • Data perimeters
  • Example Service Control Policy
  • Verifying your setup
  • Considerations
  • References

Was this helpful?

  1. Quilt Platform Administrator
  2. Advanced

Private Endpoints

PreviousPackage EventsNextRestrict Access by Bucket Prefix

Last updated 2 years ago

Was this helpful?

This page describes a feature that is not enabled by default. You can ask your Quilt account manager to enable it.

Data perimeters

A data perimeter ensures that only trusted principals on expected networks can access trusted resources.

For example, you may wish to ensure that only private IPs can access data in Amazon S3, Quilt's primary data store. Such a data perimeter strengthens your security by ensuring that S3 credentials alone are not sufficient to access data in Amazon S3.

In order for Quilt to function properly with expected private networks, your Quilt account manager must configure your CloudFormation stack to run its services (e.g. Lambda, API Gateway) on private IPs.

Additionally you will need to create and configure the following AWS resources, or equivalents depending on your network architecture:

  1. Create an for Amazon API Gateway.

    This interface endpoint is used by Quilt's backend services to keep network traffic private to your VPC. Enter the VPC endpoint ID in your CloudFormation template as the ApiGatewayVPCEndpointId template parameter.

    Note that, even if you do not use private endpoints for Quilt services, traffic between your VPC and AWS services .

  2. Create an Amazon S3 Gateway endpoint.

    facilitate access to S3 from the VPC that you run Quilt in.

    AWS permits one per VPC per region If you wish to connect buckets from multiple stacks to Quilt, a transit VPC or similar design is required.

  3. Provide a NAT gateway (or similar).

    Quilt's private endpoints require access to public Internet services like Amazon ECR and Amazon SNS.

    See .

  4. Test and apply policies to enforce your data perimeter.

    We recommend that you test an individual bucket policy on a clean bucket to prevent inadvertent loss of access to your data. Once Quilt and other services are able to access this experimental bucket as expected, you can graduate to a more comprehensive to implement your data perimeter at the organization level. SCPs define guardrails on any action that the account's administrator delegates to the IAM users and roles in the account.

    See for more on SCPs.

Example Service Control Policy

The following SCP establishes a data perimeter around all in-organization Amazon S3 buckets prefixed with the string "quilt" such that only principals with one or more of the following characteristics can access data in Amazon S3.

  1. The source VPC is either vpc-LOCAL or vpc-VPN.

  2. The principal on the request has the NetworkRestrictedExempt tag.

    Use this tag as a failsafe entry point when testing and debugging your SCP

  3. The request comes from a specific IP range (e.g. 192.0.2.0 - 192.0.2.255).

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PreventUnexpectedNetworksButAllowAWSServices",
            "Effect": "Deny",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::quilt*",
                "arn:aws:s3:::quilt*/*"
            ],
            "Condition": {
                "StringNotEqualsIfExists": {
                    "aws:SourceVpc": [
                        "vpc-LOCAL",
                        "vpc-VPN"
                    ]
                },
                "Null": {
                    "aws:PrincipalTag/NetworkRestrictedExempt": "true"
                },
                "NotIpAddressIfExists": {
                    "aws:SourceIp": [
                        "192.0.2.0/24",
                        "203.0.113.0/24"
                    ]
                },
                "Bool": {
                    "aws:PrincipalIsAWSService": "false",
                    "aws:ViaAWSService": "false"
                }
            }
        }
    ]
}

Verifying your setup

Considerations

  1. There can only be one S3 gateway endpoint per VPC.

  2. Your S3 buckets must be in the same region as the gateway endpoint.

  3. Routing traffic on private networks may incur Transit Gateway, inter-VPC, and Interface Endpoint charges.

  4. The DNS of any VPN clients must assign AWS global and regional S3 service names to the Interface Endpoint IP addresses.

References

The principal is an .

If you have enabled, check the catalog admin panel to ensure that they are functioning.

interface VPC endpoint
does not leave the AWS network backbone
S3 gateway endpoints
S3 gateway endpoint
Amazon's guide on NAT gateways
Service Control Policy (SCP)
"Enabling and disabling policy types"
AWS service principal
Quilt canaries
Choosing your VPC Endpoint Strategy for Amazon S3
Secure hybrid access to Amazon S3 using AWS PrivateLink
Establishing a Data Perimeter
Building a Data Perimeter Whitepaper