LogoLogo
HomeGitHub RepoBook Demo
version-3.5.x
version-3.5.x
  • Introduction
  • Installation
  • Quick start
  • Mental model
  • Architecture
  • Walkthrough
    • Editing a Package
    • Uploading a Package
    • Installing a Package
    • Getting Data from a Package
    • Working with the Catalog
    • Working with a Bucket
  • API Reference
    • quilt3
    • quilt3.Package
    • quilt3.Bucket
    • CLI, environment
    • Known limitations
  • Catalog
    • Configuration
    • Preview
    • Search & query
    • Metadata for teams
    • Admin UI
    • Embed
  • Advanced Usage
    • Filtering a Package
    • .quiltignore
    • Materialization
    • Working with Manifests
    • S3 Select
    • Workflows
    • Enterprise install
    • S3 Events, EventBridge
  • More
    • Frequently Asked Questions
    • Troubleshooting
    • Contributing
    • Changelog
Powered by GitBook
On this page
  • Catalog push
  • API
  • Metadata

Was this helpful?

  1. API Reference

Known limitations

PreviousCLI, environmentNextConfiguration

Last updated 3 years ago

Was this helpful?

By design, Quilt is backed by Amazon S3 and scales to billions of objects and petabytes of data. The underlying limitations of S3 apply.

The Quilt catalog can browse packages and S3 buckets of any size.

Catalog push

To ensure usability and quick package pushes, the Quilt web catalog imposes the following limits on pushes. These limits do not apply to the quilt3 Python API.

Dimension

Max

Package manifest size (metadata)

100 MB

Package size (data; via promotion or from an S3 directory)

10 GB

Maximum number of files per push

5,000

API

As of this writing, with sufficient client-side memory, you can comfortable scale Quilt packages to at least one million objects per package, with no practical limit on object size (save S3's 5 TB per object limit). A fast network, or better yet an AWS compute instance in the same region as your Quilt S3 buckets, is recommended.

Metadata

See .

Metadata