Code Repositories
zenml.code_repositories
Initialization of the ZenML code repository base abstraction.
Attributes
__all__ = ['BaseCodeRepository', 'LocalRepositoryContext']
module-attribute
Classes
BaseCodeRepository(id: UUID, name: str, config: Dict[str, Any])
Bases: ABC
Base class for code repositories.
Code repositories are used to connect to a remote code repository and store information about the repository, such as the URL, the owner, the repository name, and the host. They also provide methods to download files from the repository when a pipeline is run remotely.
Initializes a code repository.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
id
|
UUID
|
The ID of the code repository. |
required |
name
|
str
|
The name of the code repository. |
required |
config
|
Dict[str, Any]
|
The config of the code repository. |
required |
Source code in src/zenml/code_repositories/base_code_repository.py
44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 |
|
Attributes
config: BaseCodeRepositoryConfig
property
Config class for Code Repository.
Returns:
Type | Description |
---|---|
BaseCodeRepositoryConfig
|
The config class. |
id: UUID
property
ID of the code repository.
Returns:
Type | Description |
---|---|
UUID
|
The ID of the code repository. |
name: str
property
Name of the code repository.
Returns:
Type | Description |
---|---|
str
|
The name of the code repository. |
requirements: Set[str]
property
Set of PyPI requirements for the repository.
Returns:
Type | Description |
---|---|
Set[str]
|
A set of PyPI requirements for the repository. |
Functions
download_files(commit: str, directory: str, repo_sub_directory: Optional[str]) -> None
abstractmethod
Downloads files from the code repository to a local directory.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
commit
|
str
|
The commit hash to download files from. |
required |
directory
|
str
|
The directory to download files to. |
required |
repo_sub_directory
|
Optional[str]
|
The subdirectory in the repository to download files from. |
required |
Raises:
Type | Description |
---|---|
RuntimeError
|
If the download fails. |
Source code in src/zenml/code_repositories/base_code_repository.py
145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 |
|
from_model(model: CodeRepositoryResponse) -> BaseCodeRepository
classmethod
Loads a code repository from a model.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
model
|
CodeRepositoryResponse
|
The CodeRepositoryResponseModel to load from. |
required |
Returns:
Type | Description |
---|---|
BaseCodeRepository
|
The loaded code repository object. |
Source code in src/zenml/code_repositories/base_code_repository.py
71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 |
|
get_local_context(path: str) -> Optional[LocalRepositoryContext]
abstractmethod
Gets a local repository context from a path.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
path
|
str
|
The path to the local repository. |
required |
Returns:
Type | Description |
---|---|
Optional[LocalRepositoryContext]
|
The local repository context object. |
Source code in src/zenml/code_repositories/base_code_repository.py
162 163 164 165 166 167 168 169 170 171 172 173 174 |
|
login() -> None
abstractmethod
Logs into the code repository.
This method is called when the code repository is initialized. It should be used to authenticate with the code repository.
Raises:
Type | Description |
---|---|
RuntimeError
|
If the login fails. |
Source code in src/zenml/code_repositories/base_code_repository.py
133 134 135 136 137 138 139 140 141 142 143 |
|
validate_config(config: Dict[str, Any]) -> None
classmethod
Validate the code repository config.
This method should check that the config/credentials are valid and the configured repository exists.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
config
|
Dict[str, Any]
|
The configuration. |
required |
Source code in src/zenml/code_repositories/base_code_repository.py
88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 |
|
LocalRepositoryContext(code_repository: BaseCodeRepository)
Bases: ABC
Base class for local repository contexts.
This class is used to represent a local repository. It is used to track the current state of the repository and to provide information about the repository, such as the root path, the current commit, and whether the repository is dirty.
Initializes a local repository context.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
code_repository
|
BaseCodeRepository
|
The code repository. |
required |
Source code in src/zenml/code_repositories/local_repository_context.py
36 37 38 39 40 41 42 |
|
Attributes
code_repository: BaseCodeRepository
property
current_commit: str
abstractmethod
property
Returns the current commit of the local repository.
Returns:
Type | Description |
---|---|
str
|
The current commit of the local repository. |
has_local_changes: bool
abstractmethod
property
Returns whether the local repository has local changes.
A repository has local changes if it is dirty or there are some commits which have not been pushed yet.
Returns:
Type | Description |
---|---|
bool
|
Whether the local repository has local changes. |
is_dirty: bool
abstractmethod
property
Returns whether the local repository is dirty.
A repository counts as dirty if it has any untracked or uncommitted changes.
Returns:
Type | Description |
---|---|
bool
|
Whether the local repository is dirty. |
root: str
abstractmethod
property
Returns the root path of the local repository.
Returns:
Type | Description |
---|---|
str
|
The root path of the local repository. |
Functions
Modules
base_code_repository
Base class for code repositories.
Classes
BaseCodeRepository(id: UUID, name: str, config: Dict[str, Any])
Bases: ABC
Base class for code repositories.
Code repositories are used to connect to a remote code repository and store information about the repository, such as the URL, the owner, the repository name, and the host. They also provide methods to download files from the repository when a pipeline is run remotely.
Initializes a code repository.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
id
|
UUID
|
The ID of the code repository. |
required |
name
|
str
|
The name of the code repository. |
required |
config
|
Dict[str, Any]
|
The config of the code repository. |
required |
Source code in src/zenml/code_repositories/base_code_repository.py
44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 |
|
config: BaseCodeRepositoryConfig
property
Config class for Code Repository.
Returns:
Type | Description |
---|---|
BaseCodeRepositoryConfig
|
The config class. |
id: UUID
property
ID of the code repository.
Returns:
Type | Description |
---|---|
UUID
|
The ID of the code repository. |
name: str
property
Name of the code repository.
Returns:
Type | Description |
---|---|
str
|
The name of the code repository. |
requirements: Set[str]
property
Set of PyPI requirements for the repository.
Returns:
Type | Description |
---|---|
Set[str]
|
A set of PyPI requirements for the repository. |
download_files(commit: str, directory: str, repo_sub_directory: Optional[str]) -> None
abstractmethod
Downloads files from the code repository to a local directory.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
commit
|
str
|
The commit hash to download files from. |
required |
directory
|
str
|
The directory to download files to. |
required |
repo_sub_directory
|
Optional[str]
|
The subdirectory in the repository to download files from. |
required |
Raises:
Type | Description |
---|---|
RuntimeError
|
If the download fails. |
Source code in src/zenml/code_repositories/base_code_repository.py
145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 |
|
from_model(model: CodeRepositoryResponse) -> BaseCodeRepository
classmethod
Loads a code repository from a model.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
model
|
CodeRepositoryResponse
|
The CodeRepositoryResponseModel to load from. |
required |
Returns:
Type | Description |
---|---|
BaseCodeRepository
|
The loaded code repository object. |
Source code in src/zenml/code_repositories/base_code_repository.py
71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 |
|
get_local_context(path: str) -> Optional[LocalRepositoryContext]
abstractmethod
Gets a local repository context from a path.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
path
|
str
|
The path to the local repository. |
required |
Returns:
Type | Description |
---|---|
Optional[LocalRepositoryContext]
|
The local repository context object. |
Source code in src/zenml/code_repositories/base_code_repository.py
162 163 164 165 166 167 168 169 170 171 172 173 174 |
|
login() -> None
abstractmethod
Logs into the code repository.
This method is called when the code repository is initialized. It should be used to authenticate with the code repository.
Raises:
Type | Description |
---|---|
RuntimeError
|
If the login fails. |
Source code in src/zenml/code_repositories/base_code_repository.py
133 134 135 136 137 138 139 140 141 142 143 |
|
validate_config(config: Dict[str, Any]) -> None
classmethod
Validate the code repository config.
This method should check that the config/credentials are valid and the configured repository exists.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
config
|
Dict[str, Any]
|
The configuration. |
required |
Source code in src/zenml/code_repositories/base_code_repository.py
88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 |
|
BaseCodeRepositoryConfig(warn_about_plain_text_secrets: bool = False, **kwargs: Any)
Bases: SecretReferenceMixin
, ABC
Base config for code repositories.
Source code in src/zenml/config/secret_reference_mixin.py
30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 |
|
Functions
Modules
git
Initialization of the local git repository context.
Classes
LocalGitRepositoryContext(code_repository: BaseCodeRepository, git_repo: Repo, remote_name: str)
Bases: LocalRepositoryContext
Local git repository context.
Initializes a local git repository context.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
code_repository
|
BaseCodeRepository
|
The code repository. |
required |
git_repo
|
Repo
|
The git repo. |
required |
remote_name
|
str
|
Name of the remote. |
required |
Source code in src/zenml/code_repositories/git/local_git_repository_context.py
40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 |
|
current_commit: str
property
The current commit.
Returns:
Type | Description |
---|---|
str
|
The current commit sha. |
git_repo: Repo
property
The git repo.
Returns:
Type | Description |
---|---|
Repo
|
The git repo object of the local git repository. |
has_local_changes: bool
property
Whether the git repo has local changes.
A repository has local changes if it is dirty or there are some commits which have not been pushed yet.
Returns:
Type | Description |
---|---|
bool
|
True if the git repo has local changes, False otherwise. |
Raises:
Type | Description |
---|---|
RuntimeError
|
If the git repo is in a detached head state. |
is_dirty: bool
property
Whether the git repo is dirty.
By default, a repository counts as dirty if it has any untracked or uncommitted changes. Users can use an environment variable to ignore untracked files.
Returns:
Type | Description |
---|---|
bool
|
True if the git repo is dirty, False otherwise. |
remote: Remote
property
The git remote.
Returns:
Type | Description |
---|---|
Remote
|
The remote of the git repo object of the local git repository. |
root: str
property
The root of the git repo.
Returns:
Type | Description |
---|---|
str
|
The root of the git repo. |
at(path: str, code_repository: BaseCodeRepository, remote_url_validation_callback: Callable[[str], bool]) -> Optional[LocalGitRepositoryContext]
classmethod
Returns a local git repository at the given path.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
path
|
str
|
The path to the local git repository. |
required |
code_repository
|
BaseCodeRepository
|
The code repository. |
required |
remote_url_validation_callback
|
Callable[[str], bool]
|
A callback that validates the remote URL of the git repository. |
required |
Returns:
Type | Description |
---|---|
Optional[LocalGitRepositoryContext]
|
A local git repository if the path is a valid git repository |
Optional[LocalGitRepositoryContext]
|
and the remote URL is valid, otherwise None. |
Source code in src/zenml/code_repositories/git/local_git_repository_context.py
57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 |
|
Modules
local_git_repository_context
Implementation of the Local git repository context.
LocalGitRepositoryContext(code_repository: BaseCodeRepository, git_repo: Repo, remote_name: str)
Bases: LocalRepositoryContext
Local git repository context.
Initializes a local git repository context.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
code_repository
|
BaseCodeRepository
|
The code repository. |
required |
git_repo
|
Repo
|
The git repo. |
required |
remote_name
|
str
|
Name of the remote. |
required |
Source code in src/zenml/code_repositories/git/local_git_repository_context.py
40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 |
|
current_commit: str
property
The current commit.
Returns:
Type | Description |
---|---|
str
|
The current commit sha. |
git_repo: Repo
property
The git repo.
Returns:
Type | Description |
---|---|
Repo
|
The git repo object of the local git repository. |
has_local_changes: bool
property
Whether the git repo has local changes.
A repository has local changes if it is dirty or there are some commits which have not been pushed yet.
Returns:
Type | Description |
---|---|
bool
|
True if the git repo has local changes, False otherwise. |
Raises:
Type | Description |
---|---|
RuntimeError
|
If the git repo is in a detached head state. |
is_dirty: bool
property
Whether the git repo is dirty.
By default, a repository counts as dirty if it has any untracked or uncommitted changes. Users can use an environment variable to ignore untracked files.
Returns:
Type | Description |
---|---|
bool
|
True if the git repo is dirty, False otherwise. |
remote: Remote
property
The git remote.
Returns:
Type | Description |
---|---|
Remote
|
The remote of the git repo object of the local git repository. |
root: str
property
The root of the git repo.
Returns:
Type | Description |
---|---|
str
|
The root of the git repo. |
at(path: str, code_repository: BaseCodeRepository, remote_url_validation_callback: Callable[[str], bool]) -> Optional[LocalGitRepositoryContext]
classmethod
Returns a local git repository at the given path.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
path
|
str
|
The path to the local git repository. |
required |
code_repository
|
BaseCodeRepository
|
The code repository. |
required |
remote_url_validation_callback
|
Callable[[str], bool]
|
A callback that validates the remote URL of the git repository. |
required |
Returns:
Type | Description |
---|---|
Optional[LocalGitRepositoryContext]
|
A local git repository if the path is a valid git repository |
Optional[LocalGitRepositoryContext]
|
and the remote URL is valid, otherwise None. |
Source code in src/zenml/code_repositories/git/local_git_repository_context.py
57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 |
|
local_repository_context
Base class for local code repository contexts.
Classes
LocalRepositoryContext(code_repository: BaseCodeRepository)
Bases: ABC
Base class for local repository contexts.
This class is used to represent a local repository. It is used to track the current state of the repository and to provide information about the repository, such as the root path, the current commit, and whether the repository is dirty.
Initializes a local repository context.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
code_repository
|
BaseCodeRepository
|
The code repository. |
required |
Source code in src/zenml/code_repositories/local_repository_context.py
36 37 38 39 40 41 42 |
|
code_repository: BaseCodeRepository
property
current_commit: str
abstractmethod
property
Returns the current commit of the local repository.
Returns:
Type | Description |
---|---|
str
|
The current commit of the local repository. |
has_local_changes: bool
abstractmethod
property
Returns whether the local repository has local changes.
A repository has local changes if it is dirty or there are some commits which have not been pushed yet.
Returns:
Type | Description |
---|---|
bool
|
Whether the local repository has local changes. |
is_dirty: bool
abstractmethod
property
Returns whether the local repository is dirty.
A repository counts as dirty if it has any untracked or uncommitted changes.
Returns:
Type | Description |
---|---|
bool
|
Whether the local repository is dirty. |
root: str
abstractmethod
property
Returns the root path of the local repository.
Returns:
Type | Description |
---|---|
str
|
The root path of the local repository. |