lamindb

Data management for traceable, multimodal AI.

If you just want to read data from a LaminDB instance, use DB:

import lamindb as ln

db = ln.DB("laminlabs/cellxgene")

To write data, connect to a writable instance in a development directory:

lamin login
cd myproject
lamin connect --here account/myproject

You can create an instance at lamin.ai and invite collaborators. If you prefer to work with a local database (no login required), run:

mkdir myproject && cd myproject && lamin init

LaminDB will then auto-connect upon import and you can create & save objects like this:

import lamindb as ln
# → connected lamindb: account/instance

ln.Artifact("./my_dataset.csv", key="datasets/my_dataset.csv").save()

Lineage

Track inputs, outputs, parameters, and environments of scripts, notebooks, and workflows. For tracking agents, see the CLI command lamin track.

track([transform, project, space, branch, ...])	Track a run of a notebook or script.
finish([ignore_non_consecutive])	Finish the run of a notebook or script.
flow([uid, global_run, track_arg_aliases])	Use @flow() to track a function as a workflow.
step([uid])	Use @step() to track a function as a step.

Artifacts & collections

The central Artifact registry holds files, folders & arrays across any number of storage locations. The Collection registry holds versioned collections of artifacts and allows, e.g., to construct big sharded datasets across many parquet files or zarr stores.

Artifact()	Datasets & models stored as files, folders, or arrays.
Collection()	Versioned collections of artifacts, such as sharded datasets across many parquet files or zarr stores.

All other registries link to Artifact to provide context for finding, querying, validating, and managing artifacts. Here is an overview of the core data model:

Transforms & runs

Data transformations and their executions.

Transform()	Data transformations such as scripts, notebooks, functions, or pipelines.
Run()	Runs of transforms such as the executions of a script.

Records, labels, features & schemas

Manage flexible records, e.g., for samples or donors, and create simple labels.

Record()	Structured records with support for notes.
ULabel()	Simple labels.

Define features & schemas to validate artifacts & records.

Feature()	Measurable properties such as the columns of a DataFrame.
Schema()	Schemas to impose structure on artifacts and records.

Managing operations

Project()	Projects to label artifacts, transforms, records, and runs.
Storage()	Storage locations of artifacts such as local directories or S3 buckets.
User()	Users.
Branch()	Branches for change management with archive and trash states.
Space()	Spaces for access permissions.
Reference()	References such as internal studies, papers, documents, or URLs.

Basic utilities

Connecting, viewing database content, accessing settings & run context.

DB(instance)	Query any registry of any instance.
connect([instance])	Connect the default database.
view(*[, limit, modules, registries, df])	View metadata.
save(records[, ignore_conflicts, ...])	Bulk save objects.
UPath(*args[, protocol, chain_parser])	Path-like access to files.
settings	Global live settings (Settings).
context	Global run context (Context).

Curators and integrations

curators	Curators.
integrations	Integrations.

Examples, errors & setup

examples	Examples.
errors	Errors.
setup	Setup & configure LaminDB.

Developer API

base	Base library.
core	Core library.
models	Auxiliary models & database library.