Cli

`zenml.cli` `special`

ZenML CLI.

The ZenML CLI tool is usually downloaded and installed via PyPI and a pip install zenml command. Please see the Installation & Setup section above for more information about that process.

How to use the CLI

Our CLI behaves similarly to many other CLIs for basic features. In order to find out which version of ZenML you are running, type:

   zenml version

If you ever need more information on exactly what a certain command will do, use the --help flag attached to the end of your command string.

For example, to get a sense of all the commands available to you while using the zenml command, type:

   zenml --help

If you were instead looking to know more about a specific command, you can type something like this:

   zenml artifact-store register --help

This will give you information about how to register an artifact store. (See below for more on that).

If you want to instead understand what the concept behind a group is, you can use the explain sub-command. For example, to see more details behind what a artifact-store is, you can type:

zenml artifact-store explain

This will give you an explanation of that concept in more detail.

Beginning a Project

In order to start working on your project, initialize a ZenML repository within your current directory with ZenML's own config and resource management tools:

zenml init

This is all you need to begin using all the MLOps goodness that ZenML provides!

By default, zenml init will install its own hidden .zen folder inside the current directory from which you are running the command. You can also pass in a directory path manually using the --path option:

zenml init --path /path/to/dir

If you wish to use one of the available ZenML project templates to generate a ready-to-use project scaffold in your repository, you can do so by passing the --template option:

zenml init --template <name_of_template>

Running the above command will result in input prompts being shown to you. If you would like to rely on default values for the ZenML project template - you can add --template-with-defaults to the same command, like this:

zenml init --template <name_of_template> --template-with-defaults

In a similar fashion, if you would like to quickly explore the capabilities of ZenML through a notebook, you can also use:

zenml go

Cleaning up

If you wish to delete all data relating to your workspace from the directory, use the zenml clean command. This will:

delete all pipelines, pipeline runs and associated metadata
delete all artifacts

Using Integrations

Integrations are the different pieces of a project stack that enable custom functionality. This ranges from bigger libraries like kubeflow for orchestration down to smaller visualization tools like facets. Our CLI is an easy way to get started with these integrations.

To list all the integrations available to you, type:

zenml integration list

To see the requirements for a specific integration, use the requirements command:

zenml integration requirements INTEGRATION_NAME

If you wish to install the integration, using the requirements listed in the previous command, install allows you to do this for your local environment:

zenml integration install INTEGRATION_NAME

Note that if you don't specify a specific integration to be installed, the ZenML CLI will install all available integrations.

If you want to install all integrations apart from one or multiple integrations, use the following syntax, for example, which will install all integrations except feast and aws:

zenml integration install -i feast -i aws

Uninstalling a specific integration is as simple as typing:

zenml integration uninstall INTEGRATION_NAME

For all these zenml integration commands, you can pass the --uv flag and we will use uv as the package manager instead of pip. This will resolve and install much faster than with pip, but note that it requires uv to be installed on your machine. This is an experimental feature and may not work on all systems. In particular, note that installing onto machines with GPU acceleration may not work as expected.

If you would like to export the requirements of all ZenML integrations, you can use the command:

zenml integration export-requirements

Here, you can also select a list of integrations and write the result into and output file:

zenml integration export-requirements gcp kubeflow -o OUTPUT_FILE

Filtering when listing

Certain CLI list commands allow you to filter their output. For example, all stack components allow you to pass custom parameters to the list command that will filter the output. To learn more about the available filters, a good quick reference is to use the --help command, as in the following example:

zenml orchestrator list --help

You will see a list of all the available filters for the list command along with examples of how to use them.

The --sort_by option allows you to sort the output by a specific field and takes an asc or desc argument to specify the order. For example, to sort the output of the list command by the name field in ascending order, you would type:

zenml orchestrator list --sort_by "asc:name"

For fields marked as being of type TEXT or UUID, you can use the contains, startswith and endswith keywords along with their particular identifier. For example, for the orchestrator list command, you can use the following filter to find all orchestrators that contain the string sagemaker in their name:

zenml orchestrator list --name "contains:sagemaker"

For fields marked as being of type BOOL, you can use the 'True' or 'False' values to filter the output.

Finally, for fields marked as being of type DATETIME, you can pass in datetime values in the %Y-%m-%d %H:%M:%S format. These can be combined with the gte, lte, gt and lt keywords (greater than or equal, less than or equal, greater than and less than respectively) to specify the range of the filter. For example, if I wanted to find all orchestrators that were created after the 1st of January 2021, I would type:

zenml orchestrator list --created "gt:2021-01-01 00:00:00"

This syntax can also be combined to create more complex filters using the or and and keywords.

Artifact Stores

In ZenML, the artifact store is where all the inputs and outputs of your pipeline steps are stored. By default, ZenML initializes your repository with an artifact store with everything kept on your local machine. You can get a better understanding about the concept of artifact stores by executing:

zenml artifact-store explain

If you wish to register a new artifact store, do so with the register command:

zenml artifact-store register ARTIFACT_STORE_NAME --flavor=ARTIFACT_STORE_FLAVOR [--OPTIONS]

You can also add any labels to your stack component using the --label or -l flag:

zenml artifact-store register ARTIFACT_STORE_NAME --flavor=ARTIFACT_STORE_FLAVOR -l key1=value1 -l key2=value2

As you can see from the command above, when you register a new artifact store, you have to choose a flavor. To see the full list of available artifact store flavors, you can use the command:

zenml artifact-store flavor list

This list will show you which integration these flavors belong to and which service connectors they are adaptable with. If you would like to get additional information regarding a specific flavor, you can utilize the command:

zenml artifact-store flavor describe FLAVOR_NAME

If you wish to list the artifact stores that have already been registered within your ZenML:

zenml artifact-store list

If you want the name of the artifact store in the active stack, you can also use the get command:

zenml artifact-store get

For details about a particular artifact store, use the describe command. By default, (without a specific artifact store name passed in) it will describe the active or currently used artifact store:

zenml artifact-store describe ARTIFACT_STORE_NAME

If you wish to update/rename an artifact store, you can use the following commands respectively:

zenml artifact-store update ARTIFACT_STORE_NAME --property_to_update=new_value
zenml artifact-store rename ARTIFACT_STORE_OLD_NAME ARTIFACT_STORE_NEW_NAME

If you wish to delete a particular artifact store, pass the name of the artifact store into the CLI with the following command:

zenml artifact-store delete ARTIFACT_STORE_NAME

If you would like to connect/disconnect your artifact store to/from a service connector, you can use the following commands:

zenml artifact-store connect ARTIFACT_STORE_NAME -c CONNECTOR_NAME
zenml artifact-store disconnect

The ZenML CLI provides a few more utility functions for you to manage your artifact stores. In order to get a full list of available functions, use the command:

zenml artifact-store --help

Orchestrators

An orchestrator is a special kind of backend that manages the running of each step of the pipeline. Orchestrators administer the actual pipeline runs. By default, ZenML initializes your repository with an orchestrator that runs everything on your local machine. In order to get a more detailed explanation, you can use the command:

zenml orchestrator explain

If you wish to register a new orchestrator, do so with the register command:

zenml orchestrator register ORCHESTRATOR_NAME --flavor=ORCHESTRATOR_FLAVOR [--ORCHESTRATOR_OPTIONS]

You can also add any label to your stack component using the --label or -l flag:

zenml orchestrator register ORCHESTRATOR_NAME --flavor=ORCHESTRATOR_FLAVOR -l key1=value1 -l key2=value2

As you can see from the command above, when you register a new orchestrator, you have to choose a flavor. To see the full list of available orchestrator flavors, you can use the command:

zenml orchestrator flavor list

This list will show you which integration these flavors belong to and which service connectors they are adaptable with. If you would like to get additional information regarding a specific flavor, you can utilize the command:

zenml orchestrator flavor describe FLAVOR_NAME

If you wish to list the orchestrators that have already been registered within your ZenML workspace / repository, type:

zenml orchestrator list

If you want the name of the orchestrator in the active stack, you can also use the get command:

zenml orchestrator get

For details about a particular orchestrator, use the describe command. By default, (without a specific orchestrator name passed in) it will describe the active or currently used orchestrator:

zenml orchestrator describe [ORCHESTRATOR_NAME]

If you wish to update/rename an orchestrator, you can use the following commands respectively:

zenml orchestrator update ORCHESTRATOR_NAME --property_to_update=new_value
zenml orchestrator rename ORCHESTRATOR_OLD_NAME ORCHESTRATOR_NEW_NAME

If you wish to delete a particular orchestrator, pass the name of the orchestrator into the CLI with the following command:

zenml orchestrator delete ORCHESTRATOR_NAME

If you would like to connect/disconnect your orchestrator to/from a service connector, you can use the following commands:

zenml orchestrator connect ORCHESTRATOR_NAME -c CONNECTOR_NAME
zenml orchestrator disconnect

The ZenML CLI provides a few more utility functions for you to manage your orchestrators. In order to get a full list of available functions, use the command:

zenml orchestrators --help

Container Registries

The container registry is where all the images that are used by a container-based orchestrator are stored. To get a better understanding regarding container registries, use the command:

zenml container-registry explain

By default, a default ZenML local stack will not register a container registry. If you wish to register a new container registry, do so with the register command:

zenml container-registry register REGISTRY_NAME --flavor=REGISTRY_FLAVOR [--REGISTRY_OPTIONS]

You can also add any label to your stack component using the --label or -l flag:

zenml container-registry register REGISTRY_NAME --flavor=REGISTRY_FLAVOR -l key1=value1 -l key2=value2

As you can see from the command above, when you register a new container registry, you have to choose a flavor. To see the full list of available container registry flavors, you can use the command:

zenml container-registry flavor list

This list will show you which integration these flavors belong to and which service connectors they are adaptable with. If you would like to get additional information regarding a specific flavor, you can utilize the command:

zenml container-registry flavor describe FLAVOR_NAME

To list all container registries available and registered for use, use the list command:

zenml container-registry list

If you want the name of the container registry in the active stack, you can also use the get command:

zenml container-registry get

For details about a particular container registry, use the describe command. By default, (without a specific registry name passed in) it will describe the active or currently used container registry:

zenml container-registry describe [CONTAINER_REGISTRY_NAME]

If you wish to update/rename a container registry, you can use the following commands respectively:

zenml container-registry update CONTAINER_REGISTRY_NAME --property_to_update=new_value
zenml container-registry rename CONTAINER_REGISTRY_OLD_NAME CONTAINER_REGISTRY_NEW_NAME

To delete a container registry (and all of its contents), use the delete command:

zenml container-registry delete REGISTRY_NAME

If you would like to connect/disconnect your container registry to/from a service connector, you can use the following commands:

zenml container-registry connect CONTAINER_REGISTRY_NAME -c CONNECTOR_NAME
zenml container-registry disconnect

The ZenML CLI provides a few more utility functions for you to manage your container registries. In order to get a full list of available functions, use the command:

zenml container-registry --help

Data Validators

In ZenML, data validators help you profile and validate your data.

By default, a default ZenML local stack will not register a data validator. If you wish to register a new data validator, do so with the register command:

zenml data-validator register DATA_VALIDATOR_NAME --flavor DATA_VALIDATOR_FLAVOR [--DATA_VALIDATOR_OPTIONS]

You can also add any label to your stack component using the --label or -l flag:

zenml data-validator register DATA_VALIDATOR_NAME --flavor DATA_VALIDATOR_FLAVOR -l key1=value1 -l key2=value2

As you can see from the command above, when you register a new data validator, you have to choose a flavor. To see the full list of available data validator flavors, you can use the command:

zenml data-validator flavor list

This list will show you which integration these flavors belong to and which service connectors they are adaptable with. If you would like to get additional information regarding a specific flavor, you can utilize the command:

zenml data-validator flavor describe FLAVOR_NAME

To list all data validators available and registered for use, use the list command:

zenml data-validator list

If you want the name of the data validator in the active stack, use the get command:

zenml data-validator get

For details about a particular data validator, use the describe command. By default, (without a specific data validator name passed in) it will describe the active or currently-used data validator:

zenml data-validator describe [DATA_VALIDATOR_NAME]

If you wish to update/rename a data validator, you can use the following commands respectively:

zenml data-validator update DATA_VALIDATOR_NAME --property_to_update=new_value
zenml data-validator rename DATA_VALIDATOR_OLD_NAME DATA_VALIDATOR_NEW_NAME

To delete a data validator (and all of its contents), use the delete command:

zenml data-validator delete DATA_VALIDATOR_NAME

If you would like to connect/disconnect your data validator to/from a service connector, you can use the following commands:

zenml data-validator connect DATA_VALIDATOR_NAME -c CONNECTOR_NAME
zenml data-validator disconnect

The ZenML CLI provides a few more utility functions for you to manage your data validators. In order to get a full list of available functions, use the command:

zenml data-validator --help

Experiment Trackers

Experiment trackers let you track your ML experiments by logging the parameters and allow you to compare between different runs. To get a better understanding regarding experiment trackers, use the command:

zenml experiment-tracker explain

By default, a default ZenML local stack will not register an experiment tracker. If you want to use an experiment tracker in one of your stacks, you need to first register it:

zenml experiment-tracker register EXPERIMENT_TRACKER_NAME     --flavor=EXPERIMENT_TRACKER_FLAVOR [--EXPERIMENT_TRACKER_OPTIONS]

You can also add any label to your stack component using the --label or -l flag:

zenml experiment-tracker register EXPERIMENT_TRACKER_NAME       --flavor=EXPERIMENT_TRACKER_FLAVOR -l key1=value1 -l key2=value2

As you can see from the command above, when you register a new experiment tracker, you have to choose a flavor. To see the full list of available experiment tracker flavors, you can use the command:

zenml experiment-tracker flavor list

This list will show you which integration these flavors belong to and which service connectors they are adaptable with. If you would like to get additional information regarding a specific flavor, you can utilize the command:

zenml experiment-tracker flavor describe FLAVOR_NAME

To list all experiment trackers available and registered for use, use the list command:

zenml experiment-tracker list

If you want the name of the experiment tracker in the active stack, use the get command:

zenml experiment-tracker get

For details about a particular experiment tracker, use the describe command. By default, (without a specific experiment tracker name passed in) it will describe the active or currently-used experiment tracker:

zenml experiment-tracker describe [EXPERIMENT_TRACKER_NAME]

If you wish to update/rename an experiment tracker, you can use the following commands respectively:

zenml experiment-tracker update EXPERIMENT_TRACKER_NAME --property_to_update=new_value
zenml experiment-tracker rename EXPERIMENT_TRACKER_OLD_NAME EXPERIMENT_TRACKER_NEW_NAME

To delete an experiment tracker, use the delete command:

zenml experiment-tracker delete EXPERIMENT_TRACKER_NAME

If you would like to connect/disconnect your experiment tracker to/from a service connector, you can use the following commands:

zenml experiment-tracker connect EXPERIMENT_TRACKER_NAME -c CONNECTOR_NAME
zenml experiment-tracker disconnect

The ZenML CLI provides a few more utility functions for you to manage your experiment trackers. In order to get a full list of available functions, use the command:

zenml experiment-tracker --help

Model Deployers

Model deployers are stack components responsible for online model serving. They are responsible for deploying models to a remote server. Model deployers also act as a registry for models that are served with ZenML. To get a better understanding regarding model deployers, use the command:

zenml model-deployer explain

By default, a default ZenML local stack will not register a model deployer. If you wish to register a new model deployer, do so with the register command:

zenml model-deployer register MODEL_DEPLOYER_NAME --flavor MODEL_DEPLOYER_FLAVOR [--MODEL_DEPLOYER_OPTIONS]

You can also add any label to your stack component using the --label or -l flag:

zenml model-deployer register MODEL_DEPLOYER_NAME --flavor MODEL_DEPLOYER_FLAVOR -l key1=value1 -l key2=value2

As you can see from the command above, when you register a new model deployer, you have to choose a flavor. To see the full list of available model deployer flavors, you can use the command:

zenml model-deployer flavor list

This list will show you which integration these flavors belong to and which service connectors they are adaptable with. If you would like to get additional information regarding a specific flavor, you can utilize the command:

zenml model-deployer flavor describe FLAVOR_NAME

To list all model deployers available and registered for use, use the list command:

zenml model-deployer list

If you want the name of the model deployer in the active stack, use the get command:

zenml model-deployer get

For details about a particular model deployer, use the describe command. By default, (without a specific operator name passed in) it will describe the active or currently used model deployer:

zenml model-deployer describe [MODEL_DEPLOYER_NAME]

If you wish to update/rename a model deployer, you can use the following commands respectively:

zenml model-deployer update MODEL_DEPLOYER_NAME --property_to_update=new_value
zenml model-deployer rename MODEL_DEPLOYER_OLD_NAME MODEL_DEPLOYER_NEW_NAME

To delete a model deployer (and all of its contents), use the delete command:

zenml model-deployer delete MODEL_DEPLOYER_NAME

If you would like to connect/disconnect your model deployer to/from a service connector, you can use the following commands:

zenml model-deployer connect MODEL_DEPLOYER_NAME -c CONNECTOR_NAME
zenml model-deployer disconnect

Moreover, ZenML features a set of CLI commands specific to the model deployer interface. If you want to simply see what models have been deployed within your stack, run the following command:

zenml model-deployer models list

This should give you a list of served models containing their uuid, the name of the pipeline that produced them including the run id and the step name as well as the status. This information should help you identify the different models.

If you want further information about a specific model, simply copy the UUID and the following command.

zenml model-deployer models describe <UUID>

If you are only interested in the prediction url of the specific model you can also run:

zenml model-deployer models get-url <UUID>

Finally, you will also be able to start/stop the services using the following two commands:

zenml model-deployer models start <UUID>
zenml model-deployer models stop <UUID>

If you want to completely remove a served model you can also irreversibly delete it using:

zenml model-deployer models delete <UUID>

The ZenML CLI provides a few more utility functions for you to manage your model deployers. In order to get a full list of available functions, use the command:

zenml model-deployer --help

Step Operators

Step operators allow you to run individual steps in a custom environment different from the default one used by your active orchestrator. One example use-case is to run a training step of your pipeline in an environment with GPUs available. To get a better understanding regarding step operators, use the command:

zenml step-operator explain

By default, a default ZenML local stack will not register a step operator. If you wish to register a new step operator, do so with the register command:

zenml step-operator register STEP_OPERATOR_NAME --flavor STEP_OPERATOR_FLAVOR [--STEP_OPERATOR_OPTIONS]

You can also add any label to your stack component using the --label or -l flag:

zenml step-operator register STEP_OPERATOR_NAME --flavor STEP_OPERATOR_FLAVOR -l key1=value1 -l key2=value2

As you can see from the command above, when you register a new step operator, you have to choose a flavor. To see the full list of available step operator flavors, you can use the command:

zenml step-operator flavor list

This list will show you which integration these flavors belong to and which service connectors they are adaptable with. If you would like to get additional information regarding a specific flavor, you can utilize the command:

zenml step-operator flavor describe FLAVOR_NAME

To list all step operators available and registered for use, use the list command:

zenml step-operator list

If you want the name of the step operator in the active stack, use the get command:

zenml step-operator get

For details about a particular step operator, use the describe command. By default, (without a specific operator name passed in) it will describe the active or currently used step operator:

zenml step-operator describe [STEP_OPERATOR_NAME]

If you wish to update/rename a step operator, you can use the following commands respectively:

zenml step-operator update STEP_OPERATOR_NAME --property_to_update=new_value
zenml step-operator rename STEP_OPERATOR_OLD_NAME STEP_OPERATOR_NEW_NAME

To delete a step operator (and all of its contents), use the delete command:

zenml step-operator delete STEP_OPERATOR_NAME

If you would like to connect/disconnect your step operator to/from a service connector, you can use the following commands:

zenml step-operator connect STEP_OPERATOR_NAME -c CONNECTOR_NAME
zenml step-operator disconnect

The ZenML CLI provides a few more utility functions for you to manage your step operators. In order to get a full list of available functions, use the command:

zenml step-operator --help

Alerters

In ZenML, alerters allow you to send alerts from within your pipeline.

By default, a default ZenML local stack will not register an alerter. If you wish to register a new alerter, do so with the register command:

zenml alerter register ALERTER_NAME --flavor ALERTER_FLAVOR [--ALERTER_OPTIONS]

You can also add any label to your stack component using the --label or -l flag:

zenml alerter register ALERTER_NAME --flavor ALERTER_FLAVOR -l key1=value1 -l key2=value2

As you can see from the command above, when you register a new alerter, you have to choose a flavor. To see the full list of available alerter flavors, you can use the command:

zenml alerter flavor list

This list will show you which integration these flavors belong to and which service connectors they are adaptable with. If you would like to get additional information regarding a specific flavor, you can utilize the command:

zenml alerter flavor describe FLAVOR_NAME

To list all alerters available and registered for use, use the list command:

zenml alerter list

If you want the name of the alerter in the active stack, use the get command:

zenml alerter get

For details about a particular alerter, use the describe command. By default, (without a specific alerter name passed in) it will describe the active or currently used alerter:

zenml alerter describe [ALERTER_NAME]

If you wish to update/rename an alerter, you can use the following commands respectively:

zenml alerter update ALERTER_NAME --property_to_update=new_value
zenml alerter rename ALERTER_OLD_NAME ALERTER_NEW_NAME

To delete an alerter (and all of its contents), use the delete command:

zenml alerter delete ALERTER_NAME

If you would like to connect/disconnect your alerter to/from a service connector, you can use the following commands:

zenml alerter connect ALERTER_NAME -c CONNECTOR_NAME
zenml alerter disconnect

The ZenML CLI provides a few more utility functions for you to manage your alerters. In order to get a full list of available functions, use the command:

zenml alerter --help

Feature Stores

Feature stores allow data teams to serve data via an offline store and an online low-latency store where data is kept in sync between the two. To get a better understanding regarding feature stores, use the command:

zenml feature-store explain

By default, a default ZenML local stack will not register a feature store. If you wish to register a new feature store, do so with the register command:

zenml feature-store register FEATURE_STORE_NAME --flavor FEATURE_STORE_FLAVOR [--FEATURE_STORE_OPTIONS]

You can also add any label to your stack component using the --label or -l flag:

zenml feature-store register FEATURE_STORE_NAME --flavor FEATURE_STORE_FLAVOR -l key1=value1 -l key2=value2

As you can see from the command above, when you register a new feature store, you have to choose a flavor. To see the full list of available feature store flavors, you can use the command:

zenml feature-store flavor list

This list will show you which integration these flavors belong to and which service connectors they are adaptable with. If you would like to get additional information regarding a specific flavor, you can utilize the command:

Note: Currently, ZenML only supports connecting to a Redis-backed Feast feature store as a stack component integration.

zenml feature-store flavor describe FLAVOR_NAME

To list all feature stores available and registered for use, use the list command:

zenml feature-store list

If you want the name of the feature store in the active stack, use the get command:

zenml feature-store get

For details about a particular feature store, use the describe command. By default, (without a specific feature store name passed in) it will describe the active or currently-used feature store:

zenml feature-store describe [FEATURE_STORE_NAME]

If you wish to update/rename a feature store, you can use the following commands respectively:

zenml feature-store update FEATURE_STORE_NAME --property_to_update=new_value
zenml feature-store rename FEATURE_STORE_OLD_NAME FEATURE_STORE_NEW_NAME

To delete a feature store (and all of its contents), use the delete command:

zenml feature-store delete FEATURE_STORE_NAME

If you would like to connect/disconnect your feature store to/from a service connector, you can use the following commands:

zenml feature-store connect FEATURE_STORE_NAME -c CONNECTOR_NAME
zenml feature-store disconnect

The ZenML CLI provides a few more utility functions for you to manage your feature stores. In order to get a full list of available functions, use the command:

zenml feature-store --help

Annotators

Annotators enable the use of data annotation as part of your ZenML stack and pipelines.

By default, a default ZenML local stack will not register an annotator. If you wish to register a new annotator, do so with the register command:

zenml annotator register ANNOTATOR_NAME --flavor ANNOTATOR_FLAVOR [--ANNOTATOR_OPTIONS]

You can also add any label to your stack component using the --label or -l flag:

zenml annotator register ANNOTATOR_NAME --flavor ANNOTATOR_FLAVOR -l key1=value1 -l key2=value2

As you can see from the command above, when you register a new annotator, you have to choose a flavor. To see the full list of available annotator flavors, you can use the command:

zenml annotator flavor list

This list will show you which integration these flavors belong to and which service connectors they are adaptable with. If you would like to get additional information regarding a specific flavor, you can utilize the command:

zenml annotator flavor describe FLAVOR_NAME

To list all annotator available and registered for use, use the list command:

zenml annotator list

If you want the name of the annotator in the active stack, use the get command:

zenml annotator get

For details about a particular annotator, use the describe command. By default, (without a specific annotator name passed in) it will describe the active or currently used annotator:

zenml annotator describe [ANNOTATOR_NAME]

If you wish to update/rename an annotator, you can use the following commands respectively:

zenml annotator update ANNOTATOR_NAME --property_to_update=new_value
zenml annotator rename ANNOTATOR_OLD_NAME ANNOTATOR_NEW_NAME

To delete an annotator (and all of its contents), use the delete command:

zenml annotator delete ANNOTATOR_NAME

If you would like to connect/disconnect your annotator to/from a service connector, you can use the following commands:

zenml annotator connect ANNOTATOR_NAME -c CONNECTOR_NAME
zenml annotator disconnect

Finally, you can use the dataset command to interact with your annotation datasets:

zenml annotator dataset --help

The ZenML CLI provides a few more utility functions for you to manage your annotator. In order to get a full list of available functions, use the command:

zenml annotator --help

Image Builders

In ZenML, image builders allow you to build container images such that your machine-learning pipelines and steps can be executed in remote environments.

By default, a default ZenML local stack will not register an image builder. If you wish to register a new image builder, do so with the register command:

zenml image-builder register IMAGE_BUILDER_NAME --flavor IMAGE_BUILDER_FLAVOR [--IMAGE_BUILDER_OPTIONS]

You can also add any label to your stack component using the --label or -l flag:

zenml image-builder register IMAGE_BUILDER_NAME --flavor IMAGE_BUILDER_FLAVOR -l key1=value1 -l key2=value2

As you can see from the command above, when you register a new image builder, you have to choose a flavor. To see the full list of available image builder flavors, you can use the command:

zenml image-builder flavor list

This list will show you which integration these flavors belong to and which service connectors they are adaptable with. If you would like to get additional information regarding a specific flavor, you can utilize the command:

zenml image-builder flavor describe FLAVOR_NAME

To list all image builders available and registered for use, use the list command:

zenml image-builder list

If you want the name of the image builder in the active stack, use the get command:

zenml image-builder get

For details about a particular image builder, use the describe command. By default, (without a specific image builder name passed in) it will describe the active or currently used image builder:

zenml image-builder describe [IMAGE_BUILDER_NAME]

If you wish to update/rename an image builder, you can use the following commands respectively:

zenml image-builder update IMAGE_BUILDER_NAME --property_to_update=new_value
zenml image-builder rename IMAGE_BUILDER_OLD_NAME IMAGE_BUILDER_NEW_NAME

To delete a image builder (and all of its contents), use the delete command:

zenml image-builder delete IMAGE_BUILDER_NAME

If you would like to connect/disconnect your image builder to/from a service connector, you can use the following commands:

zenml image-builder connect IMAGE_BUILDER_NAME -c CONNECTOR_NAME
zenml image-builder disconnect

The ZenML CLI provides a few more utility functions for you to manage your image builders. In order to get a full list of available functions, use the command:

zenml image-builder --help

Model Registries

Model registries are centralized repositories that facilitate the collaboration and management of machine learning models. To get a better understanding regarding model registries as a concept, use the command:

zenml model-registry explain

By default, a default ZenML local stack will not register a model registry. If you wish to register a new model registry, do so with the register command:

zenml model-registry register MODEL_REGISTRY_NAME --flavor MODEL_REGISTRY_FLAVOR [--MODEL_REGISTRY_OPTIONS]

You can also add any label to your stack component using the --label or -l flag:

zenml model-registry register MODEL_REGISTRY_NAME --flavor MODEL_REGISTRY_FLAVOR -l key1=value1 -l key2=value2

As you can see from the command above, when you register a new model registry, you have to choose a flavor. To see the full list of available model registry flavors, you can use the command:

zenml model-registry flavor list

This list will show you which integration these flavors belong to and which service connectors they are adaptable with. If you would like to get additional information regarding a specific flavor, you can utilize the command:

zenml model-registry flavor describe FLAVOR_NAME

To list all model registries available and registered for use, use the list command:

zenml model-registry list

If you want the name of the model registry in the active stack, use the get command:

zenml model-registry get

For details about a particular model registry, use the describe command. By default, (without a specific operator name passed in) it will describe the active or currently used model registry:

zenml model-registry describe [MODEL_REGISTRY_NAME]

If you wish to update/rename a model registry, you can use the following commands respectively:

zenml model-registry update MODEL_REGISTRY_NAME --property_to_update=new_value
zenml model-registry rename MODEL_REGISTRY_OLD_NAME MODEL_REGISTRY_NEW_NAME

To delete a model registry (and all of its contents), use the delete command:

zenml model-registry delete MODEL_REGISTRY_NAME

If you would like to connect/disconnect your model registry to/from a service connector, you can use the following commands:

zenml model-registry connect MODEL_REGISTRY_NAME -c CONNECTOR_NAME
zenml model-registry disconnect

The ZenML CLI provides a few more utility functions for you to manage your model registries. In order to get a full list of available functions, use the command:

zenml model-registry --help

Managing your Stacks

The stack is a grouping of your artifact store, your orchestrator, and other optional MLOps tools like experiment trackers or model deployers. With the ZenML tool, switching from a local stack to a distributed cloud environment can be accomplished with just a few CLI commands.

To register a new stack, you must already have registered the individual components of the stack using the commands listed above.

Use the zenml stack register command to register your stack. It takes four arguments as in the following example:

zenml stack register STACK_NAME        -a ARTIFACT_STORE_NAME        -o ORCHESTRATOR_NAME

Each corresponding argument should be the name, id or even the first few letters of the id that uniquely identify the artifact store or orchestrator.

If you want to immediately set this newly created stack as your active stack, simply pass along the --set flag.

zenml stack register STACK_NAME ... --set

To list the stacks that you have registered within your current ZenML workspace, type:

zenml stack list

To delete a stack that you have previously registered, type:

zenml stack delete STACK_NAME

By default, ZenML uses a local stack whereby all pipelines run on your local computer. If you wish to set a different stack as the current active stack to be used when running your pipeline, type:

zenml stack set STACK_NAME

This changes a configuration property within your local environment.

To see which stack is currently set as the default active stack, type:

zenml stack get

If you want to copy a stack, run the following command:

zenml stack copy SOURCE_STACK_NAME TARGET_STACK_NAME

If you wish to transfer one of your stacks to another machine, you can do so by exporting the stack configuration and then importing it again.

To export a stack to YAML, run the following command:

zenml stack export STACK_NAME FILENAME.yaml

This will create a FILENAME.yaml containing the config of your stack and all of its components, which you can then import again like this:

zenml stack import STACK_NAME -f FILENAME.yaml

If you wish to update a stack that you have already registered, first make sure you have registered whatever components you want to use, then use the following command:

# assuming that you have already registered a new orchestrator
# with NEW_ORCHESTRATOR_NAME
zenml stack update STACK_NAME -o NEW_ORCHESTRATOR_NAME

You can update one or many stack components at the same time out of the ones that ZenML supports. To see the full list of options for updating a stack, use the following command:

zenml stack update --help

To remove a stack component from a stack, use the following command:

# assuming you want to remove the image builder and the feature-store
# from your stack
zenml stack remove-component -i -f

If you wish to rename your stack, use the following command:

zenml stack rename STACK_NAME NEW_STACK_NAME

If you want to copy a stack component, run the following command:

zenml STACK_COMPONENT copy SOURCE_COMPONENT_NAME TARGET_COMPONENT_NAME

If you wish to update a specific stack component, use the following command, switching out "STACK_COMPONENT" for the component you wish to update (i.e. 'orchestrator' or 'artifact-store' etc.):

zenml STACK_COMPONENT update --some_property=NEW_VALUE

Note that you are not permitted to update the stack name or UUID in this way. To change the name of your stack component, use the following command:

zenml STACK_COMPONENT rename STACK_COMPONENT_NAME NEW_STACK_COMPONENT_NAME

If you wish to remove an attribute (or multiple attributes) from a stack component, use the following command:

zenml STACK_COMPONENT remove-attribute STACK_COMPONENT_NAME ATTRIBUTE_NAME [OTHER_ATTRIBUTE_NAME]

Note that you can only remove optional attributes.

If you want to register secrets for all secret references in a stack, use the following command:

zenml stack register-secrets [<STACK_NAME>]

If you want to connect a service connector to a stack's components, you can use the connect command:

zenml stack connect STACK_NAME -c CONNECTOR_NAME

Note that this only connects the service connector to the current components of the stack and not to the stack itself, which means that you need to rerun the command after adding new components to the stack.

The ZenML CLI provides a few more utility functions for you to manage your stacks. In order to get a full list of available functions, use the command:

zenml stack --help

Managing your Models

ZenML provides several CLI commands to help you administer your models and their versions as part of the Model Control Plane.

To register a new model, you can use the following CLI command:

zenml model register --name <NAME> [--MODEL_OPTIONS]

To list all registered models, use:

zenml model list [MODEL_FILTER_OPTIONS]

To update a model, use:

zenml model update <MODEL_NAME_OR_ID> [--MODEL_OPTIONS]

If you would like to add or remove tags from the model, use:

zenml model update <MODEL_NAME_OR_ID> --tag <TAG> --tag <TAG> ..
   --remove-tag <TAG> --remove-tag <TAG> ..

To delete a model, use:

zenml model delete <MODEL_NAME_OR_ID>

The CLI interface for models also helps to navigate through artifacts linked to a specific model versions.

zenml model data_artifacts <MODEL_NAME_OR_ID> [-v <VERSION>]
zenml model deployment_artifacts <MODEL_NAME_OR_ID> [-v <VERSION>]
zenml model model_artifacts <MODEL_NAME_OR_ID> [-v <VERSION>]

You can also navigate the pipeline runs linked to a specific model versions:

zenml model runs <MODEL_NAME_OR_ID> [-v <VERSION>]

To list the model versions of a specific model, use:

zenml model version list [--model-name <MODEL_NAME> --name <MODEL_VERSION_NAME> OTHER_OPTIONS]

To delete a model version, use:

zenml model version delete <MODEL_NAME_OR_ID> <VERSION>

To update a model version, use:

zenml model version update <MODEL_NAME_OR_ID> <VERSION> [--MODEL_VERSION_OPTIONS]

These are some of the more common uses of model version updates:

stage (i.e. promotion)

zenml model version update <MODEL_NAME_OR_ID> <VERSION> --stage <STAGE>

tags

zenml model version update <MODEL_NAME_OR_ID> <VERSION> --tag <TAG> --tag <TAG> ..
   --remove-tag <TAG> --remove-tag <TAG> ..

Managing your Pipelines & Artifacts

ZenML provides several CLI commands to help you administer your pipelines and pipeline runs.

To explicitly register a pipeline you need to point to a pipeline instance in your Python code. Let's say you have a Python file called run.py and it contains the following code:

from zenml import pipeline

@pipeline
def my_pipeline(...):
   # Connect your pipeline steps here
   pass

You can register your pipeline like this:

zenml pipeline register my_pipeline

To list all registered pipelines, use:

zenml pipeline list

Since every pipeline run creates a new pipeline by default, you might occasionally want to delete a pipeline, which you can do via:

zenml pipeline delete <PIPELINE_NAME>

This will delete the pipeline and change all corresponding pipeline runs to become unlisted (not linked to any pipeline).

To list all pipeline runs that you have executed, use:

zenml pipeline runs list

To delete a pipeline run, use:

zenml pipeline runs delete <PIPELINE_RUN_NAME_OR_ID>

If you run any of your pipelines with pipeline.run(schedule=...), ZenML keeps track of the schedule and you can list all schedules via:

zenml pipeline schedule list

To delete a schedule, use:

zenml pipeline schedule delete <SCHEDULE_NAME_OR_ID>

Note, however, that this will only delete the reference saved in ZenML and does NOT stop/delete the schedule in the respective orchestrator. This still needs to be done manually. For example, using the Airflow orchestrator you would have to open the web UI to manually click to stop the schedule from executing.

Each pipeline run automatically saves its artifacts in the artifact store. To list all artifacts that have been saved, use:

zenml artifact list

Each artifact has one or several versions. To list artifact versions, use:

zenml artifact versions list

If you would like to rename an artifact or adjust the tags of an artifact or artifact version, use the corresponding update command:

zenml artifact update <NAME> -n <NEW_NAME>
zenml artifact update <NAME> -t <TAG1> -t <TAG2> -r <TAG_TO_REMOVE>
zenml artifact version update <NAME> -v <VERSION> -t <TAG1> -t <TAG2> -r <TAG_TO_REMOVE>

The metadata of artifacts or artifact versions stored by ZenML can only be deleted once they are no longer used by any pipeline runs. I.e., an artifact version can only be deleted if the run that produced it and all runs that used it as an input have been deleted. Similarly, an artifact can only be deleted if all its versions can be deleted.

To delete all artifacts and artifact versions that are no longer linked to any pipeline runs, use:

zenml artifact prune

Each pipeline run that requires Docker images also stores a build which contains the image names used for this run. To list all builds, use:

zenml pipeline builds list

To delete a specific build, use:

zenml pipeline builds delete <BUILD_ID>

Managing the local ZenML Dashboard

The ZenML dashboard is a web-based UI that allows you to visualize and navigate the stack configurations, pipelines and pipeline runs tracked by ZenML among other things. You can start the ZenML dashboard locally by running the following command:

zenml up

This will start the dashboard on your local machine where you can access it at the URL printed to the console.

If you have closed the dashboard in your browser and want to open it again, you can run:

zenml show

If you want to stop the dashboard, simply run:

zenml down

The zenml up command has a few additional options that you can use to customize how the ZenML dashboard is running.

By default, the dashboard is started as a background process. On some operating systems, this capability is not available. In this case, you can use the --blocking flag to start the dashboard in the foreground:

zenml up --blocking

This will block the terminal until you stop the dashboard with CTRL-C.

Another option you can use, if you have Docker installed on your machine, is to run the dashboard in a Docker container. This is useful if you don't want to install all the Zenml server dependencies on your machine. To do so, simply run:

zenml up --docker

The TCP port and the host address that the dashboard uses to listen for connections can also be customized. Using an IP address that is not the default localhost or 127.0.0.1 is especially useful if you're running some type of local ZenML orchestrator, such as the k3d Kubeflow orchestrator or Docker orchestrator, that cannot directly connect to the local ZenML server.

For example, to start the dashboard on port 9000 and have it listen on all locally available interfaces on your machine, run:

zenml up --port 9000 --ip-address 0.0.0.0

Note that the above 0.0.0.0 IP address also exposes your ZenML dashboard externally through your public interface. Alternatively, you can choose an explicit IP address that is configured on one of your local interfaces, such as the Docker bridge interface, which usually has the IP address 172.17.0.1:

zenml up --port 9000 --ip-address 172.17.0.1

Connecting to a ZenML Server

The ZenML client can be configured to connect to a remote database or ZenML server with the zenml connect command. If no arguments are supplied, ZenML will attempt to connect to the last ZenML server deployed from the local host using the 'zenml deploy' command:

zenml connect

To connect to a ZenML server, you can either pass the configuration as command line arguments or as a YAML file:

zenml connect --url=https://zenml.example.com:8080 --no-verify-ssl

or

zenml connect --config=/path/to/zenml_server_config.yaml

The YAML file should have the following structure when connecting to a ZenML server:

url: <The URL of the ZenML server>
verify_ssl: |
   <Either a boolean, in which case it controls whether the
   server's TLS certificate is verified, or a string, in which case it
   must be a path to a CA certificate bundle to use or the CA bundle
   value itself>

Both options can be combined, in which case the command line arguments will override the values in the YAML file. For example:

zenml connect --no-verify-ssl --config=/path/to/zenml_server_config.yaml

You can open the ZenML dashboard of your currently connected ZenML server using the following command:

zenml show

If you would like to take a look at the logs for the ZenML server:

zenml logs

Note that if you have set your AUTO_OPEN_DASHBOARD environment variable to false then this will not open the dashboard until you set it back to true. To disconnect from the current ZenML server and revert to using the local default database, use the following command:

zenml disconnect

You can inspect the current ZenML configuration at any given time using the following command:

zenml status

Example output:

 zenml status
Running without an active repository root.
Connected to a ZenML server: 'https://ac8ef63af203226194a7725ee71d85a-7635928635.us-east-1.elb.amazonaws.com'
The current user is: 'default'
The active workspace is: 'default' (global)
The active stack is: 'default' (global)
The status of the local dashboard:
              ZenML server 'local'
┏━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ URL            │ http://172.17.0.1:9000      ┃
┠────────────────┼─────────────────────────────┨
┃ STATUS         │ ✅                          ┃
┠────────────────┼─────────────────────────────┨
┃ STATUS_MESSAGE │ Docker container is running ┃
┠────────────────┼─────────────────────────────┨
┃ CONNECTED      │                             ┃
┗━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛

The zenml connect command can also be used to configure your client with more advanced options, such as connecting directly to a local or remote SQL database. In this case, the --raw-config flag must be passed to instruct the CLI to not validate or fill in the missing configuration fields. For example, to connect to a remote MySQL database, run:

zenml connect --raw-config --config=/path/to/mysql_config.yaml

with a YAML configuration file that looks like this:

type: sql
url: mysql://<username>:<password>@mysql.database.com/<database_name>
ssl_ca: |
   -----BEGIN CERTIFICATE-----
   ...
   -----END CERTIFICATE-----

ssl_cert: null
ssl_key: null
ssl_verify_server_cert: false

Keep in mind, while connecting to a ZenML server, you will be provided with the option to Trust this device. If you opt out of it a 24-hour token will be issued for the authentication service. If you opt-in, you will be issued a 30-day token instead.

If you would like to see a list of all trusted devices, you can use:

zenml authorized-device list

or if you would like to get the details regarding a specific device, you can use:

zenml authorized-device describe DEVICE_ID_OR_PREFIX

Alternatively, you can lock and unlock an authorized device by using the following commands:

zenml authorized-device lock DEVICE_ID_OR_PREFIX
zenml authorized-device unlock DEVICE_ID_OR_PREFIX

Finally, you can remove an authorized device by using the delete command:

zenml authorized-device delete DEVICE_ID_OR_PREFIX

Secrets management

ZenML offers a way to securely store secrets associated with your other stack components and infrastructure. A ZenML Secret is a collection or grouping of key-value pairs stored by the ZenML secrets store. ZenML Secrets are identified by a unique name which allows you to fetch or reference them in your pipelines and stacks.

Depending on how you set up and deployed ZenML, the secrets store keeps secrets in the local database or uses the ZenML server your client is connected to:

if you are using the default ZenML client settings, or if you connect your ZenML client to a local ZenML server started with zenml up, the secrets store is using the same local SQLite database as the rest of ZenML
if you connect your ZenML client to a remote ZenML server, the secrets are no longer managed on your local machine, but through the remote server instead. Secrets are stored in whatever secrets store back-end the remote server is configured to use. This can be a SQL database, one of the managed cloud secrets management services, or even a custom back-end.

To create a secret, use the create command and pass the key-value pairs as command-line arguments:

zenml secret create SECRET_NAME --key1=value1 --key2=value2 --key3=value3 ...

# Another option is to use the '--values' option and provide key-value pairs in either JSON or YAML format.
zenml secret create SECRET_NAME --values='{"key1":"value2","key2":"value2","key3":"value3"}'

Note that when using the previous command the keys and values will be preserved in your bash_history file, so you may prefer to use the interactive create command instead:

zenml secret create SECRET_NAME -i

As an alternative to the interactive mode, also useful for values that are long or contain newline or special characters, you can also use the special @ syntax to indicate to ZenML that the value needs to be read from a file:

zenml secret create SECRET_NAME    --aws_access_key_id=1234567890    --aws_secret_access_key=abcdefghij    --aws_session_token=@/path/to/token.txt

# Alternatively for providing key-value pairs, you can utilize the '--values' option by specifying a file path containing
# key-value pairs in either JSON or YAML format.
zenml secret create SECRET_NAME --values=@/path/to/token.txt

To list all the secrets available, use the list command:

zenml secret list

To get the key-value pairs for a particular secret, use the get command:

zenml secret get SECRET_NAME

To update a secret, use the update command:

zenml secret update SECRET_NAME --key1=value1 --key2=value2 --key3=value3 ...

# Another option is to use the '--values' option and provide key-value pairs in either JSON or YAML format.
zenml secret update SECRET_NAME --values='{"key1":"value2","key2":"value2","key3":"value3"}'

Note that when using the previous command the keys and values will be preserved in your bash_history file, so you may prefer to use the interactive update command instead:

zenml secret update SECRET_NAME -i

Finally, to delete a secret, use the delete command:

zenml secret delete SECRET_NAME

Secrets can be scoped to a workspace or a user. By default, secrets are scoped to the current workspace. To scope a secret to a user, use the --scope user argument in the register command.

Auth management

Building and maintaining an MLOps workflow can involve numerous third-party libraries and external services. In most cases, this ultimately presents a challenge in configuring uninterrupted, secure access to infrastructure resources. In ZenML, Service Connectors streamline this process by abstracting away the complexity of authentication and help you connect your stack to your resources. You can find the full docs on the ZenML service connectors here.

The ZenML CLI features a variety of commands to help you manage your service connectors. First of all, to explore all the types of service connectors available in ZenML, you can use the following commands:

# To get the complete list
zenml service-connector list-types

# To get the details regarding a single type
zenml service-connector describe-type

For each type of service connector, you will also see a list of supported resource types. These types provide a way for organizing different resources into logical classes based on the standard and/or protocol used to access them. In addition to the resource types, each type will feature a different set of authentication methods.

Once you decided which service connector to use, you can create it with the register command as follows:

zenml service-connector register SERVICE_CONNECTOR_NAME     --type TYPE [--description DESCRIPTION] [--resource-type RESOURCE_TYPE]     [--auth-method AUTH_METHOD] ...

For more details on how to create a service connector, please refer to our docs.

To check if your service connector is registered properly, you can verify it. By doing this, you can both check if it is configured correctly and also, you can fetch the list of resources it has access to:

zenml service-connector verify SERVICE_CONNECTOR_NAME_ID_OR_PREFIX

Some service connectors come equipped with the capability of configuring the clients and SDKs on your local machine with the credentials inferred from your service connector. To use this functionality, simply use the login command:

zenml service-connector login SERVICE_CONNECTOR_NAME_ID_OR_PREFIX

To list all the service connectors that you have registered, you can use:

zenml service-connector list

Moreover, if you would like to list all the resources accessible by your service connectors, you can use the following command:

zenml service-connector list-resources [--resource-type RESOURCE_TYPE] /
    [--connector-type CONNECTOR_TYPE] ...

This command can possibly take a long time depending on the number of service connectors you have registered. Consider using the right filters when you are listing resources.

If you want to see the details about a specific service connector that you have registered, you can use the describe command:

zenml service-connector describe SERVICE_CONNECTOR_NAME_ID_OR_PREFIX

You can update a registered service connector by using the update command. Keep in mind that all service connector updates are validated before being applied. If you want to disable this behaviour please use the --no-verify flag.

zenml service-connector update SERVICE_CONNECTOR_NAME_ID_OR_PREFIX ...

Finally, if you wish to remove a service connector, you can use the delete command:

zenml service-connector delete SERVICE_CONNECTOR_NAME_ID_OR_PREFIX

Managing users

When using the ZenML service, you can manage permissions by managing users using the CLI. If you want to create a new user or delete an existing one, run either

zenml user create USER_NAME
zenml user delete USER_NAME

To see a list of all users, run:

zenml user list

For detail about the particular user, use the describe command. By default, (without a specific user name passed in) it will describe the active user:

zenml user describe [USER_NAME]

If you want to update any properties of a specific user, you can use the update command. Use the --help flag to get a full list of available properties to update:

zenml user update --help

If you want to change the password of the current user account:

zenml user change-password --help

Service Accounts

ZenML supports the use of service accounts to authenticate clients to the ZenML server using API keys. This is useful for automating tasks such as running pipelines or deploying models.

To create a new service account, run:

zenml service-account create SERVICE_ACCOUNT_NAME

This command creates a service account and an API key for it. The API key is displayed as part of the command output and cannot be retrieved later. You can then use the issued API key to connect your ZenML client to the server with the CLI:

zenml connect --url https://... --api-key <API_KEY>

or by setting the ZENML_STORE_URL and ZENML_STORE_API_KEY environment variables when you set up your ZenML client for the first time:

export ZENML_STORE_URL=https://...
export ZENML_STORE_API_KEY=<API_KEY>

To see all the service accounts you've created and their API keys, use the following commands:

zenml service-account list
zenml service-account api-key <SERVICE_ACCOUNT_NAME> list

Additionally, the following command allows you to more precisely inspect one of these service accounts and an API key:

zenml service-account describe <SERVICE_ACCOUNT_NAME>
zenml service-account api-key <SERVICE_ACCOUNT_NAME> describe <API_KEY_NAME>

API keys don't have an expiration date. For increased security, we recommend that you regularly rotate the API keys to prevent unauthorized access to your ZenML server. You can do this with the ZenML CLI:

zenml service-account api-key <SERVICE_ACCOUNT_NAME> rotate <API_KEY_NAME>

Running this command will create a new API key and invalidate the old one. The new API key is displayed as part of the command output and cannot be retrieved later. You can then use the new API key to connect your ZenML client to the server just as described above.

When rotating an API key, you can also configure a retention period for the old API key. This is useful if you need to keep the old API key for a while to ensure that all your workloads have been updated to use the new API key. You can do this with the --retain flag. For example, to rotate an API key and keep the old one for 60 minutes, you can run the following command:

zenml service-account api-key <SERVICE_ACCOUNT_NAME> rotate <API_KEY_NAME>       --retain 60

For increased security, you can deactivate a service account or an API key using one of the following commands:

zenml service-account update <SERVICE_ACCOUNT_NAME> --active false
zenml service-account api-key <SERVICE_ACCOUNT_NAME> update <API_KEY_NAME>       --active false

Deactivating a service account or an API key will prevent it from being used to authenticate and has immediate effect on all workloads that use it.

To permanently delete an API key for a service account, use the following command:

zenml service-account api-key <SERVICE_ACCOUNT_NAME> delete <API_KEY_NAME>

Managing Code Repositories

Code repositories enable ZenML to keep track of the code version that you use for your pipeline runs. Additionally, running a pipeline which is tracked in a registered code repository can decrease the time it takes Docker to build images for containerized stack components.

To register a code repository, use the following CLI command:

zenml code-repository register <NAME> --type=<CODE_REPOSITORY_TYPE]    [--CODE_REPOSITORY_OPTIONS]

ZenML currently supports code repositories of type github and gitlab, but you can also use your custom code repository implementation by passing the type custom and a source of your repository class.

zenml code-repository register <NAME> --type=custom    --source=<CODE_REPOSITORY_SOURCE> [--CODE_REPOSITORY_OPTIONS]

The CODE_REPOSITORY_OPTIONS depend on the configuration necessary for the type of code repository that you're using.

If you want to list your registered code repositories, run:

zenml code-repository list

You can delete one of your registered code repositories like this:

zenml code-repository delete <REPOSITORY_NAME_OR_ID>

Building an image without Runs

To build Docker images for your pipeline without actually running the pipeline, use:

zenml pipeline build <PIPELINE_ID_OR_NAME>

To specify settings for the Docker builds, use the --config/-c option of the command. For more information about the structure of this configuration file, check out the zenml.pipelines.base_pipeline.BasePipeline.build(...) method.

zenml pipeline build <PIPELINE_ID_OR_NAME> --config=<PATH_TO_CONFIG_YAML>

If you want to build the pipeline for a stack other than your current active stack, use the --stack option.

zenml pipeline build <PIPELINE_ID_OR_NAME> --stack=<STACK_ID_OR_NAME>

To run a pipeline that was previously registered, use:

zenml pipeline run  <PIPELINE_ID_OR_NAME>

To specify settings for the pipeline, use the --config/-c option of the command. For more information about the structure of this configuration file, check out the zenml.pipelines.base_pipeline.BasePipeline.run(...) method.

zenml pipeline run <PIPELINE_ID_OR_NAME> --config=<PATH_TO_CONFIG_YAML>

If you want to run the pipeline on a stack different than your current active stack, use the --stack option.

zenml pipeline run <PIPELINE_ID_OR_NAME> --stack=<STACK_ID_OR_NAME>

Tagging your resources with ZenML

When you are using ZenML, you can use tags to organize and categorize your assets. This way, you can streamline your workflows and enhance the discoverability of your resources more easily.

Currently, you can use tags with artifacts, models and their versions:

# Tag the artifact
zenml artifact update ARTIFACT_NAME -t TAG_NAME

# Tag the artifact version
zenml artifact version update ARTIFACT_NAME ARTIFACT_VERSION -t TAG_NAME

# Tag an existing model
zenml model update MODEL_NAME --tag TAG_NAME

# Tag a specific model version
zenml model version update MODEL_NAME VERSION_NAME --tag TAG_NAME

Besides these interactions, you can also create a new tag by using the register command:

zenml tag register -n TAG_NAME [-c COLOR]

If you would like to list all the tags that you have, you can use the command:

zenml tag list

To update the properties of a specific tag, you can use the update subcommand:

zenml tag update TAG_NAME_OR_ID [-n NEW_NAME] [-c NEW_COLOR]

Finally, in order to delete a tag, you can execute:

zenml tag delete TAG_NAME_OR_ID

Managing the Global Configuration

The ZenML global configuration CLI commands cover options such as enabling or disabling the collection of anonymous usage statistics, changing the logging verbosity.

In order to help us better understand how the community uses ZenML, the library reports anonymized usage statistics. You can always opt out by using the CLI command:

zenml analytics opt-out

If you want to opt back in, use the following command:

zenml analytics opt-in

The verbosity of the ZenML client output can be configured using the zenml logging command. For example, to set the verbosity to DEBUG, run:

zenml logging set-verbosity DEBUG

Deploying ZenML to the cloud

The ZenML CLI provides a simple way to deploy ZenML to the cloud. Simply run

zenml deploy

You will be prompted to provide a name for your deployment and details like what cloud provider you want to deploy to — and that's it! It creates the database and any VPCs, permissions, and more that are needed.

In order to be able to run the deploy command, you should have your cloud provider's CLI configured locally with permissions to create resources like MySQL databases and networks.

Deploying Stack Components

Stack components can be deployed directly via the CLI. You can use the deploy subcommand for this. For example, you could deploy a GCP artifact store using the following command:

zenml artifact-store deploy -f gcp -p gcp -r us-east1 -x project_id=zenml-core basic_gcp_artifact_store

For full documentation on this functionality, please refer to the dedicated documentation on stack component deploy.

Interacting with the ZenML Hub

The ZenML Hub is a central location for discovering and sharing third-party ZenML code, such as custom integrations, components, steps, pipelines, materializers, and more. You can browse the ZenML Hub at https://hub.zenml.io.

The ZenML CLI provides various commands to interact with the ZenML Hub:

Listing all plugins available on the Hub:

zenml hub list

Installing a Hub plugin:

zenml hub install

Installed plugins can be imported via from zenml.hub.<plugin_name> import ....

Uninstalling a Hub plugin:

zenml hub uninstall

Cloning the source code of a Hub plugin (without installing it):

zenml hub clone

This is useful, e.g., for extending an existing plugin or for getting the examples of a plugin.

Submitting/contributing a plugin to the Hub (requires login, see below):

zenml hub submit

If you are unsure about which arguments you need to set, you can run the command in interactive mode:

zenml hub submit --interactive

This will ask for and validate inputs one at a time.

Logging in to the Hub:

zenml hub login

Logging out of the Hub:

zenml hub logout

Viewing the build logs of a plugin you submitted to the Hub:

zenml hub logs

Cli

zenml.cli special

How to use the CLI

Beginning a Project

Cleaning up

Using Integrations

Filtering when listing

Artifact Stores

Orchestrators

Container Registries

Data Validators

Experiment Trackers

Model Deployers

Step Operators

Alerters

Feature Stores

Annotators

Image Builders

Model Registries

Managing your Stacks

Managing your Models

Managing your Pipelines & Artifacts

Managing the local ZenML Dashboard

Connecting to a ZenML Server

Secrets management

Auth management

Managing users

Service Accounts

Managing Code Repositories

Building an image without Runs

Tagging your resources with ZenML

Managing the Global Configuration

Deploying ZenML to the cloud

Deploying Stack Components

Interacting with the ZenML Hub

`zenml.cli` `special`