Skip to content

Code Repositories

Initialization of the ZenML code repository base abstraction.

BaseCodeRepository

Bases: ABC

Base class for code repositories.

Code repositories are used to connect to a remote code repository and store information about the repository, such as the URL, the owner, the repository name, and the host. They also provide methods to download files from the repository when a pipeline is run remotely.

Source code in src/zenml/code_repositories/base_code_repository.py 
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
class BaseCodeRepository(ABC):
    """Base class for code repositories.

    Code repositories are used to connect to a remote code repository and
    store information about the repository, such as the URL, the owner,
    the repository name, and the host. They also provide methods to
    download files from the repository when a pipeline is run remotely.
    """

    def __init__(
        self,
        id: UUID,
        name: str,
        config: Dict[str, Any],
    ) -> None:
        """Initializes a code repository.

        Args:
            id: The ID of the code repository.
            name: The name of the code repository.
            config: The config of the code repository.
        """
        self._id = id
        self._name = name
        self._config = config
        self.login()

    @property
    def config(self) -> "BaseCodeRepositoryConfig":
        """Config class for Code Repository.

        Returns:
            The config class.
        """
        return BaseCodeRepositoryConfig(**self._config)

    @classmethod
    def from_model(cls, model: CodeRepositoryResponse) -> "BaseCodeRepository":
        """Loads a code repository from a model.

        Args:
            model: The CodeRepositoryResponseModel to load from.

        Returns:
            The loaded code repository object.
        """
        class_: Type[BaseCodeRepository] = (
            source_utils.load_and_validate_class(
                source=model.source, expected_class=BaseCodeRepository
            )
        )
        return class_(id=model.id, name=model.name, config=model.config)

    @classmethod
    def validate_config(cls, config: Dict[str, Any]) -> None:
        """Validate the code repository config.

        This method should check that the config/credentials are valid and
        the configured repository exists.

        Args:
            config: The configuration.
        """
        # The initialization calls the login to verify the credentials
        code_repo = cls(id=uuid4(), name="", config=config)

        # Explicitly access the config for pydantic validation
        _ = code_repo.config

    @property
    def id(self) -> UUID:
        """ID of the code repository.

        Returns:
            The ID of the code repository.
        """
        return self._id

    @property
    def name(self) -> str:
        """Name of the code repository.

        Returns:
            The name of the code repository.
        """
        return self._name

    @property
    def requirements(self) -> Set[str]:
        """Set of PyPI requirements for the repository.

        Returns:
            A set of PyPI requirements for the repository.
        """
        from zenml.integrations.utils import get_requirements_for_module

        return set(get_requirements_for_module(self.__module__))

    @abstractmethod
    def login(self) -> None:
        """Logs into the code repository.

        This method is called when the code repository is initialized.
        It should be used to authenticate with the code repository.

        Raises:
            RuntimeError: If the login fails.
        """
        pass

    @abstractmethod
    def download_files(
        self, commit: str, directory: str, repo_sub_directory: Optional[str]
    ) -> None:
        """Downloads files from the code repository to a local directory.

        Args:
            commit: The commit hash to download files from.
            directory: The directory to download files to.
            repo_sub_directory: The subdirectory in the repository to
                download files from.

        Raises:
            RuntimeError: If the download fails.
        """
        pass

    @abstractmethod
    def get_local_context(
        self, path: str
    ) -> Optional["LocalRepositoryContext"]:
        """Gets a local repository context from a path.

        Args:
            path: The path to the local repository.

        Returns:
            The local repository context object.
        """
        pass

config property

Config class for Code Repository.

Returns:

Type Description
BaseCodeRepositoryConfig

The config class.

id property

ID of the code repository.

Returns:

Type Description
UUID

The ID of the code repository.

name property

Name of the code repository.

Returns:

Type Description
str

The name of the code repository.

requirements property

Set of PyPI requirements for the repository.

Returns:

Type Description
Set[str]

A set of PyPI requirements for the repository.

__init__(id, name, config)

Initializes a code repository.

Parameters:

Name Type Description Default
id UUID

The ID of the code repository.

 required
name str

The name of the code repository.

 required
config Dict[str, Any]

The config of the code repository.

 required
Source code in src/zenml/code_repositories/base_code_repository.py 
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
def __init__(
    self,
    id: UUID,
    name: str,
    config: Dict[str, Any],
) -> None:
    """Initializes a code repository.

    Args:
        id: The ID of the code repository.
        name: The name of the code repository.
        config: The config of the code repository.
    """
    self._id = id
    self._name = name
    self._config = config
    self.login()

download_files(commit, directory, repo_sub_directory) abstractmethod

Downloads files from the code repository to a local directory.

Parameters:

Name Type Description Default
commit str

The commit hash to download files from.

 required
directory str

The directory to download files to.

 required
repo_sub_directory Optional[str]

The subdirectory in the repository to download files from.

 required

Raises:

Type Description
RuntimeError

If the download fails.
Source code in src/zenml/code_repositories/base_code_repository.py 
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
@abstractmethod
def download_files(
    self, commit: str, directory: str, repo_sub_directory: Optional[str]
) -> None:
    """Downloads files from the code repository to a local directory.

    Args:
        commit: The commit hash to download files from.
        directory: The directory to download files to.
        repo_sub_directory: The subdirectory in the repository to
            download files from.

    Raises:
        RuntimeError: If the download fails.
    """
    pass

from_model(model) classmethod

Loads a code repository from a model.

Parameters:

Name Type Description Default
model CodeRepositoryResponse

The CodeRepositoryResponseModel to load from.

 required

Returns:

Type Description
BaseCodeRepository

The loaded code repository object.
Source code in src/zenml/code_repositories/base_code_repository.py 
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
@classmethod
def from_model(cls, model: CodeRepositoryResponse) -> "BaseCodeRepository":
    """Loads a code repository from a model.

    Args:
        model: The CodeRepositoryResponseModel to load from.

    Returns:
        The loaded code repository object.
    """
    class_: Type[BaseCodeRepository] = (
        source_utils.load_and_validate_class(
            source=model.source, expected_class=BaseCodeRepository
        )
    )
    return class_(id=model.id, name=model.name, config=model.config)

get_local_context(path) abstractmethod

Gets a local repository context from a path.

Parameters:

Name Type Description Default
path str

The path to the local repository.

 required

Returns:

Type Description
Optional[LocalRepositoryContext]

The local repository context object.
Source code in src/zenml/code_repositories/base_code_repository.py 
162
163
164
165
166
167
168
169
170
171
172
173
174
@abstractmethod
def get_local_context(
    self, path: str
) -> Optional["LocalRepositoryContext"]:
    """Gets a local repository context from a path.

    Args:
        path: The path to the local repository.

    Returns:
        The local repository context object.
    """
    pass

login() abstractmethod

Logs into the code repository.

This method is called when the code repository is initialized. It should be used to authenticate with the code repository.

Raises:

Type Description
RuntimeError

If the login fails.
Source code in src/zenml/code_repositories/base_code_repository.py 
133
134
135
136
137
138
139
140
141
142
143
@abstractmethod
def login(self) -> None:
    """Logs into the code repository.

    This method is called when the code repository is initialized.
    It should be used to authenticate with the code repository.

    Raises:
        RuntimeError: If the login fails.
    """
    pass

validate_config(config) classmethod

Validate the code repository config.

This method should check that the config/credentials are valid and the configured repository exists.

Parameters:

Name Type Description Default
config Dict[str, Any]

The configuration.

 required
Source code in src/zenml/code_repositories/base_code_repository.py 
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
@classmethod
def validate_config(cls, config: Dict[str, Any]) -> None:
    """Validate the code repository config.

    This method should check that the config/credentials are valid and
    the configured repository exists.

    Args:
        config: The configuration.
    """
    # The initialization calls the login to verify the credentials
    code_repo = cls(id=uuid4(), name="", config=config)

    # Explicitly access the config for pydantic validation
    _ = code_repo.config

LocalRepositoryContext

Bases: ABC

Base class for local repository contexts.

This class is used to represent a local repository. It is used to track the current state of the repository and to provide information about the repository, such as the root path, the current commit, and whether the repository is dirty.

Source code in src/zenml/code_repositories/local_repository_context.py 
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
class LocalRepositoryContext(ABC):
    """Base class for local repository contexts.

    This class is used to represent a local repository. It is used
    to track the current state of the repository and to provide
    information about the repository, such as the root path, the current
    commit, and whether the repository is dirty.
    """

    def __init__(self, code_repository: "BaseCodeRepository") -> None:
        """Initializes a local repository context.

        Args:
            code_repository: The code repository.
        """
        self._code_repository = code_repository

    @property
    def code_repository(self) -> "BaseCodeRepository":
        """Returns the code repository.

        Returns:
            The code repository.
        """
        return self._code_repository

    @property
    @abstractmethod
    def root(self) -> str:
        """Returns the root path of the local repository.

        Returns:
            The root path of the local repository.
        """
        pass

    @property
    @abstractmethod
    def is_dirty(self) -> bool:
        """Returns whether the local repository is dirty.

        A repository counts as dirty if it has any untracked or uncommitted
        changes.

        Returns:
            Whether the local repository is dirty.
        """
        pass

    @property
    @abstractmethod
    def has_local_changes(self) -> bool:
        """Returns whether the local repository has local changes.

        A repository has local changes if it is dirty or there are some commits
        which have not been pushed yet.

        Returns:
            Whether the local repository has local changes.
        """
        pass

    @property
    @abstractmethod
    def current_commit(self) -> str:
        """Returns the current commit of the local repository.

        Returns:
            The current commit of the local repository.
        """
        pass

code_repository property

Returns the code repository.

Returns:

Type Description
BaseCodeRepository

The code repository.

current_commit abstractmethod property

Returns the current commit of the local repository.

Returns:

Type Description
str

The current commit of the local repository.

has_local_changes abstractmethod property

Returns whether the local repository has local changes.

A repository has local changes if it is dirty or there are some commits which have not been pushed yet.

Returns:

Type Description
bool

Whether the local repository has local changes.

is_dirty abstractmethod property

Returns whether the local repository is dirty.

A repository counts as dirty if it has any untracked or uncommitted changes.

Returns:

Type Description
bool

Whether the local repository is dirty.

root abstractmethod property

Returns the root path of the local repository.

Returns:

Type Description
str

The root path of the local repository.

__init__(code_repository)

Initializes a local repository context.

Parameters:

Name Type Description Default
code_repository BaseCodeRepository

The code repository.

 required
Source code in src/zenml/code_repositories/local_repository_context.py 
36
37
38
39
40
41
42
def __init__(self, code_repository: "BaseCodeRepository") -> None:
    """Initializes a local repository context.

    Args:
        code_repository: The code repository.
    """
    self._code_repository = code_repository