LogoLogo
HomeGitHub RepoBook Demo
dev
dev
  • About Quilt
  • Architecture
  • Mental Model
  • Metadata Management
  • Metadata Workflows
  • Quilt Platform (Catalog) User
    • About the Catalog
    • Bucket Browsing
    • Document Previews
    • Embeddable iFrames
    • Packaging Engine
    • Query
    • Quilt+ URIs
    • Qurator Omni
    • Search
    • Visualization & Dashboards
    • Advanced
      • Athena
      • Elasticsearch
      • Removing Stacks
  • Quilt Platform Administrator
    • Admin Settings UI
    • Catalog Configuration
    • Cross-Account Access
    • Enterprise Installs
    • quilt3.admin Python API
    • Advanced
      • Package Events
      • Private Endpoints
      • Restrict Access by Bucket Prefix
      • S3 Events via EventBridge
      • SSO Permissions Mapping
      • Tabulator
      • Troubleshooting
        • SSO Redirect Loop
    • Best Practices
      • GxP for Security & Compliance
      • Organizing S3 Buckets
  • Quilt Python SDK
    • Installation
    • Quick Start
    • Editing a Package
    • Uploading a Package
    • Installing a Package
    • Getting Data from a Package
    • Example: Git-like Operations
    • API Reference
      • quilt3
      • quilt3.Package
      • quilt3.Bucket
      • quilt3.hooks
      • Local Catalog
      • CLI, Environment
      • Known Limitations
      • Custom SSL Certificates
    • Advanced
      • Browsing Buckets
      • Filtering a Package
      • .quiltignore
      • Manipulating Manifests
      • Materialization
      • S3 Select
    • More
      • Changelog
      • Contributing
      • Frequently Asked Questions
      • Troubleshooting
  • Quilt Ecosystem Integrations
    • Benchling Packager
    • Event-Driven Packaging
    • Nextflow Plugin
Powered by GitBook
On this page
  • Searching for packages
  • Installing a package
  • Browsing a package manifest
  • Importing a package

Was this helpful?

  1. Quilt Python SDK

Installing a Package

PreviousUploading a PackageNextGetting Data from a Package

Last updated 2 years ago

Was this helpful?

Searching for packages

As explained in , packages are managed using registries. There is a one local registry on your machine, and potentially many remote registries elsewhere "in the world". Use list_packages to see the packages available on a registry:

import quilt3 # list local packages
list(quilt3.list_packages())
['aneesh/cli-push',
 'examples/hurdat',
 'aleksey/hurdat']
import quilt3 # list remote packages
list(quilt3.list_packages("s3://quilt-example"))
['aleksey/hurdat',
 'examples/hurdat',
 'quilt/altair',
 'quilt/hurdat',
 'quilt/open_fruit',
 'quilt/open_images']

Installing a package

To make a remote package and all of its data available locally, install it.

The examples in this section use the examples/hurdat :

import quilt3 # install package
quilt3.Package.install(
    "examples/hurdat",
    "s3://quilt-example",
)
Loading manifest: 100%|██████████| 5/5 [00:00<00:00, 7049.25entries/s]

Successfully installed package 'examples/hurdat', tophash=f8d1478 from s3://quilt-example

Note that unless this registry is public, you will need to be logged into a user who has read access to this registry in order to install from it:

# only need to run this once
# ie quilt3.config('https://your-catalog-homepage/')
quilt3.config('https://open.quiltdata.com/')

# follow the instructions to finish login
quilt3.login()

Data files that you download are written to a folder in your local registry by default. You can specify an alternative destination using dest:

import quilt3  # install package into a specific folder
quilt3.Package.install(
    "examples/hurdat", 
    "s3://quilt-example", 
    dest="./"
)
Loading manifest: 100%|██████████| 5/5 [00:00<00:00, 9027.77entries/s]
Copying objects: 100%|██████████| 3.62M/3.62M [00:00<00:00, 303MB/s]

Successfully installed package 'examples/hurdat', tophash=f8d1478 from s3://quilt-example

Finally, you can install a specific version of a package by specifying the corresponding top hash:

import quilt3  # install specific version of package
quilt3.Package.install(
    "examples/hurdat", 
    "s3://quilt-example", 
    top_hash="058e62c"
)
Loading manifest: 100%|██████████| 5/5 [00:00<00:00, 11491.24entries/s]
Copying objects: 100%|██████████| 35.4k/35.4k [00:02<00:00, 14.3kB/s]

Successfully installed package 'examples/hurdat', tophash=058e62c from s3://quilt-example

Browsing a package manifest

An alternative to install is browse. browse downloads a package manifest without also downloading the data in the package.

import quilt3 # load a package manifest from a remote registry
p = quilt3.Package.browse("examples/hurdat", "s3://quilt-example")

# load a package manifest from the default remote registry
quilt3.config(default_remote_registry="s3://quilt-example")
p = quilt3.Package.browse("examples/hurdat")
Loading manifest: 100%|██████████| 5/5 [00:00<00:00, 7541.00entries/s]
Loading manifest: 100%|██████████| 5/5 [00:00<00:00, 10710.68entries/s]

browse is advantageous when you don't want to download everything in a package at once. For example if you just want to look at a package's metadata.

Importing a package

You can import a local package from within Python:

from quilt3.data.examples import hurdat
Loading manifest: 100%|██████████| 5/5 [00:00<00:00, 9637.65entries/s]

This allows you to manage your data and code dependencies all in one place in your Python scripts or Jupyter notebooks.

"Uploading a Package"
demo package