Skip to content

Container Registries

Initialization for ZenML's container registries module.

A container registry is a store for (Docker) containers. A ZenML workflow involving a container registry would automatically containerize your code to be transported across stacks running remotely. As part of the deployment to the cluster, the ZenML base image would be downloaded (from a cloud container registry) and used as the basis for the deployed 'run'.

For instance, when you are running a local container-based stack, you would therefore have a local container registry which stores the container images you create that bundle up your pipeline code. You could also use a remote container registry like the Elastic Container Registry at AWS in a more production setting.

AzureContainerRegistryFlavor

Bases: BaseContainerRegistryFlavor

Class for Azure Container Registry.

Source code in src/zenml/container_registries/azure_container_registry.py 
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
class AzureContainerRegistryFlavor(BaseContainerRegistryFlavor):
    """Class for Azure Container Registry."""

    @property
    def name(self) -> str:
        """Name of the flavor.

        Returns:
            The name of the flavor.
        """
        return ContainerRegistryFlavor.AZURE.value

    @property
    def service_connector_requirements(
        self,
    ) -> Optional[ServiceConnectorRequirements]:
        """Service connector resource requirements for service connectors.

        Specifies resource requirements that are used to filter the available
        service connector types that are compatible with this flavor.

        Returns:
            Requirements for compatible service connectors, if a service
            connector is required for this flavor.
        """
        return ServiceConnectorRequirements(
            connector_type="azure",
            resource_type=DOCKER_REGISTRY_RESOURCE_TYPE,
            resource_id_attr="uri",
        )

    @property
    def docs_url(self) -> Optional[str]:
        """A url to point at docs explaining this flavor.

        Returns:
            A flavor docs url.
        """
        return self.generate_default_docs_url()

    @property
    def sdk_docs_url(self) -> Optional[str]:
        """A url to point at docs explaining this flavor.

        Returns:
            A flavor docs url.
        """
        return self.generate_default_sdk_docs_url()

    @property
    def logo_url(self) -> str:
        """A url to represent the flavor in the dashboard.

        Returns:
            The flavor logo.
        """
        return "https://public-flavor-logos.s3.eu-central-1.amazonaws.com/container_registry/azure.png"

docs_url property

A url to point at docs explaining this flavor.

Returns:

Type Description
Optional[str]

A flavor docs url.

logo_url property

A url to represent the flavor in the dashboard.

Returns:

Type Description
str

The flavor logo.

name property

Name of the flavor.

Returns:

Type Description
str

The name of the flavor.

sdk_docs_url property

A url to point at docs explaining this flavor.

Returns:

Type Description
Optional[str]

A flavor docs url.

service_connector_requirements property

Service connector resource requirements for service connectors.

Specifies resource requirements that are used to filter the available service connector types that are compatible with this flavor.

Returns:

Type Description
Optional[ServiceConnectorRequirements]

Requirements for compatible service connectors, if a service
Optional[ServiceConnectorRequirements]

connector is required for this flavor.

BaseContainerRegistry

Bases: AuthenticationMixin

Base class for all ZenML container registries.

Source code in src/zenml/container_registries/base_container_registry.py 
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
class BaseContainerRegistry(AuthenticationMixin):
    """Base class for all ZenML container registries."""

    _docker_client: Optional["DockerClient"] = None

    @property
    def config(self) -> BaseContainerRegistryConfig:
        """Returns the `BaseContainerRegistryConfig` config.

        Returns:
            The configuration.
        """
        return cast(BaseContainerRegistryConfig, self._config)

    @property
    def requires_authentication(self) -> bool:
        """Returns whether the container registry requires authentication.

        Returns:
            `True` if the container registry requires authentication,
            `False` otherwise.
        """
        return bool(self.config.authentication_secret)

    @property
    def credentials(self) -> Optional[Tuple[str, str]]:
        """Username and password to authenticate with this container registry.

        Returns:
            Tuple with username and password if this container registry
            requires authentication, `None` otherwise.
        """
        secret = self.get_typed_authentication_secret(
            expected_schema_type=BasicAuthSecretSchema
        )
        if secret:
            return secret.username, secret.password

        connector = self.get_connector()
        if connector:
            from zenml.service_connectors.docker_service_connector import (
                DockerServiceConnector,
            )

            if isinstance(connector, DockerServiceConnector):
                return (
                    connector.config.username.get_secret_value(),
                    connector.config.password.get_secret_value(),
                )

        return None

    @property
    def docker_client(self) -> "DockerClient":
        """Returns a Docker client for this container registry.

        Returns:
            The Docker client.

        Raises:
            RuntimeError: If the connector does not return a Docker client.
        """
        from docker.client import DockerClient

        # Refresh the client also if the connector has expired
        if self._docker_client and not self.connector_has_expired():
            return self._docker_client

        connector = self.get_connector()
        if connector:
            client = connector.connect()
            if not isinstance(client, DockerClient):
                raise RuntimeError(
                    f"Expected a DockerClient while trying to use the "
                    f"linked connector, but got {type(client)}."
                )
            self._docker_client = client
        else:
            self._docker_client = (
                docker_utils._try_get_docker_client_from_env()
            )

            credentials = self.credentials
            if credentials:
                username, password = credentials
                self._docker_client.login(
                    username=username,
                    password=password,
                    registry=self.config.uri,
                    reauth=True,
                )

        return self._docker_client

    def prepare_image_push(self, image_name: str) -> None:
        """Preparation before an image gets pushed.

        Subclasses can overwrite this to do any necessary checks or
        preparations before an image gets pushed.

        Args:
            image_name: Name of the docker image that will be pushed.
        """

    def push_image(self, image_name: str) -> str:
        """Pushes a docker image.

        Args:
            image_name: Name of the docker image that will be pushed.

        Returns:
            The Docker repository digest of the pushed image.

        Raises:
            ValueError: If the image name is not associated with this
                container registry.
        """
        if not image_name.startswith(self.config.uri):
            raise ValueError(
                f"Docker image `{image_name}` does not belong to container "
                f"registry `{self.config.uri}`."
            )

        self.prepare_image_push(image_name)
        return docker_utils.push_image(
            image_name, docker_client=self.docker_client
        )

config property

Returns the BaseContainerRegistryConfig config.

Returns:

Type Description
BaseContainerRegistryConfig

The configuration.

credentials property

Username and password to authenticate with this container registry.

Returns:

Type Description
Optional[Tuple[str, str]]

Tuple with username and password if this container registry
Optional[Tuple[str, str]]

requires authentication, None otherwise.

docker_client property

Returns a Docker client for this container registry.

Returns:

Type Description
DockerClient

The Docker client.

Raises:

Type Description
RuntimeError

If the connector does not return a Docker client.

requires_authentication property

Returns whether the container registry requires authentication.

Returns:

Type Description
bool

True if the container registry requires authentication,
bool

False otherwise.

prepare_image_push(image_name)

Preparation before an image gets pushed.

Subclasses can overwrite this to do any necessary checks or preparations before an image gets pushed.

Parameters:

Name Type Description Default
image_name str

Name of the docker image that will be pushed.

 required
Source code in src/zenml/container_registries/base_container_registry.py 
163
164
165
166
167
168
169
170
171
def prepare_image_push(self, image_name: str) -> None:
    """Preparation before an image gets pushed.

    Subclasses can overwrite this to do any necessary checks or
    preparations before an image gets pushed.

    Args:
        image_name: Name of the docker image that will be pushed.
    """

push_image(image_name)

Pushes a docker image.

Parameters:

Name Type Description Default
image_name str

Name of the docker image that will be pushed.

 required

Returns:

Type Description
str

The Docker repository digest of the pushed image.

Raises:

Type Description
ValueError

If the image name is not associated with this container registry.
Source code in src/zenml/container_registries/base_container_registry.py 
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
def push_image(self, image_name: str) -> str:
    """Pushes a docker image.

    Args:
        image_name: Name of the docker image that will be pushed.

    Returns:
        The Docker repository digest of the pushed image.

    Raises:
        ValueError: If the image name is not associated with this
            container registry.
    """
    if not image_name.startswith(self.config.uri):
        raise ValueError(
            f"Docker image `{image_name}` does not belong to container "
            f"registry `{self.config.uri}`."
        )

    self.prepare_image_push(image_name)
    return docker_utils.push_image(
        image_name, docker_client=self.docker_client
    )

DefaultContainerRegistryFlavor

Bases: BaseContainerRegistryFlavor

Class for default ZenML container registries.

Source code in src/zenml/container_registries/default_container_registry.py 
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
class DefaultContainerRegistryFlavor(BaseContainerRegistryFlavor):
    """Class for default ZenML container registries."""

    @property
    def name(self) -> str:
        """Name of the flavor.

        Returns:
            The name of the flavor.
        """
        return ContainerRegistryFlavor.DEFAULT.value

    @property
    def docs_url(self) -> Optional[str]:
        """A URL to point at docs explaining this flavor.

        Returns:
            A flavor docs url.
        """
        return self.generate_default_docs_url()

    @property
    def sdk_docs_url(self) -> Optional[str]:
        """A URL to point at docs explaining this flavor.

        Returns:
            A flavor docs url.
        """
        return self.generate_default_sdk_docs_url()

    @property
    def logo_url(self) -> str:
        """A URL to represent the flavor in the dashboard.

        Returns:
            The flavor logo.
        """
        return "https://public-flavor-logos.s3.eu-central-1.amazonaws.com/container_registry/local.svg"

docs_url property

A URL to point at docs explaining this flavor.

Returns:

Type Description
Optional[str]

A flavor docs url.

logo_url property

A URL to represent the flavor in the dashboard.

Returns:

Type Description
str

The flavor logo.

name property

Name of the flavor.

Returns:

Type Description
str

The name of the flavor.

sdk_docs_url property

A URL to point at docs explaining this flavor.

Returns:

Type Description
Optional[str]

A flavor docs url.

DockerHubContainerRegistryFlavor

Bases: BaseContainerRegistryFlavor

Class for DockerHub Container Registry.

Source code in src/zenml/container_registries/dockerhub_container_registry.py 
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
class DockerHubContainerRegistryFlavor(BaseContainerRegistryFlavor):
    """Class for DockerHub Container Registry."""

    @property
    def name(self) -> str:
        """Name of the flavor.

        Returns:
            The name of the flavor.
        """
        return ContainerRegistryFlavor.DOCKERHUB.value

    @property
    def service_connector_requirements(
        self,
    ) -> Optional[ServiceConnectorRequirements]:
        """Service connector resource requirements for service connectors.

        Specifies resource requirements that are used to filter the available
        service connector types that are compatible with this flavor.

        Returns:
            Requirements for compatible service connectors, if a service
            connector is required for this flavor.
        """
        return ServiceConnectorRequirements(
            connector_type="docker",
            resource_type=DOCKER_REGISTRY_RESOURCE_TYPE,
            resource_id_attr="uri",
        )

    @property
    def docs_url(self) -> Optional[str]:
        """A url to point at docs explaining this flavor.

        Returns:
            A flavor docs url.
        """
        return self.generate_default_docs_url()

    @property
    def sdk_docs_url(self) -> Optional[str]:
        """A url to point at docs explaining this flavor.

        Returns:
            A flavor docs url.
        """
        return self.generate_default_sdk_docs_url()

    @property
    def logo_url(self) -> str:
        """A url to represent the flavor in the dashboard.

        Returns:
            The flavor logo.
        """
        return "https://public-flavor-logos.s3.eu-central-1.amazonaws.com/container_registry/docker.png"

docs_url property

A url to point at docs explaining this flavor.

Returns:

Type Description
Optional[str]

A flavor docs url.

logo_url property

A url to represent the flavor in the dashboard.

Returns:

Type Description
str

The flavor logo.

name property

Name of the flavor.

Returns:

Type Description
str

The name of the flavor.

sdk_docs_url property

A url to point at docs explaining this flavor.

Returns:

Type Description
Optional[str]

A flavor docs url.

service_connector_requirements property

Service connector resource requirements for service connectors.

Specifies resource requirements that are used to filter the available service connector types that are compatible with this flavor.

Returns:

Type Description
Optional[ServiceConnectorRequirements]

Requirements for compatible service connectors, if a service
Optional[ServiceConnectorRequirements]

connector is required for this flavor.

GCPContainerRegistryFlavor

Bases: BaseContainerRegistryFlavor

Class for GCP Container Registry.

Source code in src/zenml/container_registries/gcp_container_registry.py 
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
class GCPContainerRegistryFlavor(BaseContainerRegistryFlavor):
    """Class for GCP Container Registry."""

    @property
    def name(self) -> str:
        """Name of the flavor.

        Returns:
            The name of the flavor.
        """
        return ContainerRegistryFlavor.GCP.value

    @property
    def service_connector_requirements(
        self,
    ) -> Optional[ServiceConnectorRequirements]:
        """Service connector resource requirements for service connectors.

        Specifies resource requirements that are used to filter the available
        service connector types that are compatible with this flavor.

        Returns:
            Requirements for compatible service connectors, if a service
            connector is required for this flavor.
        """
        return ServiceConnectorRequirements(
            connector_type="gcp",
            resource_type=DOCKER_REGISTRY_RESOURCE_TYPE,
            resource_id_attr="uri",
        )

    @property
    def docs_url(self) -> Optional[str]:
        """A url to point at docs explaining this flavor.

        Returns:
            A flavor docs url.
        """
        return self.generate_default_docs_url()

    @property
    def sdk_docs_url(self) -> Optional[str]:
        """A url to point at docs explaining this flavor.

        Returns:
            A flavor docs url.
        """
        return self.generate_default_sdk_docs_url()

    @property
    def logo_url(self) -> str:
        """A url to represent the flavor in the dashboard.

        Returns:
            The flavor logo.
        """
        return "https://public-flavor-logos.s3.eu-central-1.amazonaws.com/container_registry/gcp.png"

docs_url property

A url to point at docs explaining this flavor.

Returns:

Type Description
Optional[str]

A flavor docs url.

logo_url property

A url to represent the flavor in the dashboard.

Returns:

Type Description
str

The flavor logo.

name property

Name of the flavor.

Returns:

Type Description
str

The name of the flavor.

sdk_docs_url property

A url to point at docs explaining this flavor.

Returns:

Type Description
Optional[str]

A flavor docs url.

service_connector_requirements property

Service connector resource requirements for service connectors.

Specifies resource requirements that are used to filter the available service connector types that are compatible with this flavor.

Returns:

Type Description
Optional[ServiceConnectorRequirements]

Requirements for compatible service connectors, if a service
Optional[ServiceConnectorRequirements]

connector is required for this flavor.

GitHubContainerRegistryFlavor

Bases: BaseContainerRegistryFlavor

Class for GitHub Container Registry.

Source code in src/zenml/container_registries/github_container_registry.py 
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
class GitHubContainerRegistryFlavor(BaseContainerRegistryFlavor):
    """Class for GitHub Container Registry."""

    @property
    def name(self) -> str:
        """Name of the flavor.

        Returns:
            The name of the flavor.
        """
        return ContainerRegistryFlavor.GITHUB

    @property
    def docs_url(self) -> Optional[str]:
        """A url to point at docs explaining this flavor.

        Returns:
            A flavor docs url.
        """
        return self.generate_default_docs_url()

    @property
    def sdk_docs_url(self) -> Optional[str]:
        """A url to point at docs explaining this flavor.

        Returns:
            A flavor docs url.
        """
        return self.generate_default_sdk_docs_url()

    @property
    def logo_url(self) -> str:
        """A url to represent the flavor in the dashboard.

        Returns:
            The flavor logo.
        """
        return "https://public-flavor-logos.s3.eu-central-1.amazonaws.com/container_registry/github.png"

docs_url property

A url to point at docs explaining this flavor.

Returns:

Type Description
Optional[str]

A flavor docs url.

logo_url property

A url to represent the flavor in the dashboard.

Returns:

Type Description
str

The flavor logo.

name property

Name of the flavor.

Returns:

Type Description
str

The name of the flavor.

sdk_docs_url property

A url to point at docs explaining this flavor.

Returns:

Type Description
Optional[str]

A flavor docs url.