LogoLogo
HomeGitHub RepoBook Demo
dev
dev
  • About Quilt
  • Architecture
  • Mental Model
  • Metadata Management
  • Metadata Workflows
  • Quilt Platform (Catalog) User
    • About the Catalog
    • Bucket Browsing
    • Document Previews
    • Embeddable iFrames
    • Packaging Engine
    • Query
    • Quilt+ URIs
    • Qurator Omni
    • Search
    • Visualization & Dashboards
    • Advanced
      • Athena
      • Elasticsearch
      • Removing Stacks
  • Quilt Platform Administrator
    • Admin Settings UI
    • Catalog Configuration
    • Cross-Account Access
    • Enterprise Installs
    • quilt3.admin Python API
    • Advanced
      • Package Events
      • Private Endpoints
      • Restrict Access by Bucket Prefix
      • S3 Events via EventBridge
      • SSO Permissions Mapping
      • Tabulator
      • Troubleshooting
        • SSO Redirect Loop
    • Best Practices
      • GxP for Security & Compliance
      • Organizing S3 Buckets
  • Quilt Python SDK
    • Installation
    • Quick Start
    • Editing a Package
    • Uploading a Package
    • Installing a Package
    • Getting Data from a Package
    • Example: Git-like Operations
    • API Reference
      • quilt3
      • quilt3.Package
      • quilt3.Bucket
      • quilt3.hooks
      • Local Catalog
      • CLI, Environment
      • Known Limitations
      • Custom SSL Certificates
    • Advanced
      • Browsing Buckets
      • Filtering a Package
      • .quiltignore
      • Manipulating Manifests
      • Materialization
      • S3 Select
    • More
      • Changelog
      • Contributing
      • Frequently Asked Questions
      • Troubleshooting
  • Quilt Ecosystem Integrations
    • Benchling Packager
    • Event-Driven Packaging
    • Nextflow Plugin
Powered by GitBook
On this page
  • Basics
  • Example: query package-level metadata
  • Example: query object-level metadata
  • Configuration

Was this helpful?

  1. Quilt Platform (Catalog) User

Query

PreviousPackaging EngineNextQuilt+ URIs

Last updated 4 months ago

Was this helpful?

is an interactive query service that makes it easy to analyze data in Amazon S3 using standard SQL. Athena is serverless, so there is no infrastructure to manage, and you pay only for the queries that you run.

The Catalog's Queries tab allows you to run Athena queries against your S3 buckets, and any other data sources your users have access to. There are prebuilt tables for packages and objects, and you can create your own tables and views. See, for example, .

NOTE: This page describes how to use Athena for precise querying of specific tables and fields. For full-text searching using Elasticsearch, see the page.

Basics

"Run query" executes the selected query and waits for the result.

Individual users will also see their past queries, and easily re-run them.

Example: query package-level metadata

Suppose we wish to find all packages produced by algorithm version 1.3 with a cell index of 5.

SELECT * FROM "YOUR-BUCKET_packages-view"
-- extract and query package-level metadata
WHERE json_extract_scalar(meta, 
  '$.user_meta.nucmembsegmentationalgorithmversion') LIKE '1.3%'
AND json_array_contains(json_extract(meta, '$.user_meta.cellindex'), '5');

Example: query object-level metadata

Suppose we wish to find all .tiff files produced by algorithm version 1.3 with a cell index of 5.

SELECT * FROM "YOUR-BUCKET_objects-view"
WHERE substr(logical_key, -5) = '.tiff'
-- extract and query object-level metadata
AND json_extract_scalar(meta, 
  '$.user_meta.nucmembsegmentationalgorithmversion') LIKE '1.3%'
AND json_array_contains(json_extract(meta, '$.user_meta.cellindex'), '5');

Configuration

Athena queries saved from the AWS Console for a given workgroup will be available in the Quilt Catalog for all users to run.

history

Administrators can hide the "Queries" tab by setting ui > nav > queries: false ().

learn more
Amazon Athena
Tabulator
Search
ui