Bentoml
zenml.integrations.bentoml
Initialization of the BentoML integration for ZenML.
The BentoML integration allows you to use the BentoML model serving to implement continuous model deployment.
Attributes
BENTOML = 'bentoml'
module-attribute
BENTOML_MODEL_DEPLOYER_FLAVOR = 'bentoml'
module-attribute
Classes
BentoMLIntegration
Bases: Integration
Definition of BentoML integration for ZenML.
Functions
activate() -> None
classmethod
Activate the BentoML integration.
Source code in src/zenml/integrations/bentoml/__init__.py
36 37 38 39 40 41 |
|
flavors() -> List[Type[Flavor]]
classmethod
Declare the stack component flavors for BentoML.
Returns:
Type | Description |
---|---|
List[Type[Flavor]]
|
List of stack component flavors for this integration. |
Source code in src/zenml/integrations/bentoml/__init__.py
43 44 45 46 47 48 49 50 51 52 53 54 |
|
Flavor
Class for ZenML Flavors.
Attributes
config_class: Type[StackComponentConfig]
abstractmethod
property
Returns StackComponentConfig
config class.
Returns:
Type | Description |
---|---|
Type[StackComponentConfig]
|
The config class. |
config_schema: Dict[str, Any]
property
The config schema for a flavor.
Returns:
Type | Description |
---|---|
Dict[str, Any]
|
The config schema. |
docs_url: Optional[str]
property
A url to point at docs explaining this flavor.
Returns:
Type | Description |
---|---|
Optional[str]
|
A flavor docs url. |
implementation_class: Type[StackComponent]
abstractmethod
property
Implementation class for this flavor.
Returns:
Type | Description |
---|---|
Type[StackComponent]
|
The implementation class for this flavor. |
logo_url: Optional[str]
property
A url to represent the flavor in the dashboard.
Returns:
Type | Description |
---|---|
Optional[str]
|
The flavor logo. |
name: str
abstractmethod
property
The flavor name.
Returns:
Type | Description |
---|---|
str
|
The flavor name. |
sdk_docs_url: Optional[str]
property
A url to point at SDK docs explaining this flavor.
Returns:
Type | Description |
---|---|
Optional[str]
|
A flavor SDK docs url. |
service_connector_requirements: Optional[ServiceConnectorRequirements]
property
Service connector resource requirements for service connectors.
Specifies resource requirements that are used to filter the available service connector types that are compatible with this flavor.
Returns:
Type | Description |
---|---|
Optional[ServiceConnectorRequirements]
|
Requirements for compatible service connectors, if a service |
Optional[ServiceConnectorRequirements]
|
connector is required for this flavor. |
type: StackComponentType
abstractmethod
property
Functions
from_model(flavor_model: FlavorResponse) -> Flavor
classmethod
Loads a flavor from a model.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
flavor_model
|
FlavorResponse
|
The model to load from. |
required |
Raises:
Type | Description |
---|---|
CustomFlavorImportError
|
If the custom flavor can't be imported. |
ImportError
|
If the flavor can't be imported. |
Returns:
Type | Description |
---|---|
Flavor
|
The loaded flavor. |
Source code in src/zenml/stack/flavor.py
122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 |
|
generate_default_docs_url() -> str
Generate the doc urls for all inbuilt and integration flavors.
Note that this method is not going to be useful for custom flavors, which do not have any docs in the main zenml docs.
Returns:
Type | Description |
---|---|
str
|
The complete url to the zenml documentation |
Source code in src/zenml/stack/flavor.py
206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 |
|
generate_default_sdk_docs_url() -> str
Generate SDK docs url for a flavor.
Returns:
Type | Description |
---|---|
str
|
The complete url to the zenml SDK docs |
Source code in src/zenml/stack/flavor.py
232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 |
|
to_model(integration: Optional[str] = None, is_custom: bool = True) -> FlavorRequest
Converts a flavor to a model.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
integration
|
Optional[str]
|
The integration to use for the model. |
None
|
is_custom
|
bool
|
Whether the flavor is a custom flavor. |
True
|
Returns:
Type | Description |
---|---|
FlavorRequest
|
The model. |
Source code in src/zenml/stack/flavor.py
159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 |
|
Integration
Base class for integration in ZenML.
Functions
activate() -> None
classmethod
Abstract method to activate the integration.
Source code in src/zenml/integrations/integration.py
175 176 177 |
|
check_installation() -> bool
classmethod
Method to check whether the required packages are installed.
Returns:
Type | Description |
---|---|
bool
|
True if all required packages are installed, False otherwise. |
Source code in src/zenml/integrations/integration.py
65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 |
|
flavors() -> List[Type[Flavor]]
classmethod
Abstract method to declare new stack component flavors.
Returns:
Type | Description |
---|---|
List[Type[Flavor]]
|
A list of new stack component flavors. |
Source code in src/zenml/integrations/integration.py
179 180 181 182 183 184 185 186 |
|
get_requirements(target_os: Optional[str] = None, python_version: Optional[str] = None) -> List[str]
classmethod
Method to get the requirements for the integration.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
target_os
|
Optional[str]
|
The target operating system to get the requirements for. |
None
|
python_version
|
Optional[str]
|
The Python version to use for the requirements. |
None
|
Returns:
Type | Description |
---|---|
List[str]
|
A list of requirements. |
Source code in src/zenml/integrations/integration.py
135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 |
|
get_uninstall_requirements(target_os: Optional[str] = None) -> List[str]
classmethod
Method to get the uninstall requirements for the integration.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
target_os
|
Optional[str]
|
The target operating system to get the requirements for. |
None
|
Returns:
Type | Description |
---|---|
List[str]
|
A list of requirements. |
Source code in src/zenml/integrations/integration.py
152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 |
|
plugin_flavors() -> List[Type[BasePluginFlavor]]
classmethod
Abstract method to declare new plugin flavors.
Returns:
Type | Description |
---|---|
List[Type[BasePluginFlavor]]
|
A list of new plugin flavors. |
Source code in src/zenml/integrations/integration.py
188 189 190 191 192 193 194 195 |
|
Modules
constants
BentoML constants.
flavors
BentoML integration flavors.
Classes
BentoMLModelDeployerConfig(warn_about_plain_text_secrets: bool = False, **kwargs: Any)
Bases: BaseModelDeployerConfig
Configuration for the BentoMLModelDeployer.
Source code in src/zenml/stack/stack_component.py
61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 |
|
BentoMLModelDeployerFlavor
Bases: BaseModelDeployerFlavor
Flavor for the BentoML model deployer.
config_class: Type[BentoMLModelDeployerConfig]
property
Returns BentoMLModelDeployerConfig
config class.
Returns:
Type | Description |
---|---|
Type[BentoMLModelDeployerConfig]
|
The config class. |
docs_url: Optional[str]
property
A url to point at docs explaining this flavor.
Returns:
Type | Description |
---|---|
Optional[str]
|
A flavor docs url. |
implementation_class: Type[BentoMLModelDeployer]
property
Implementation class for this flavor.
Returns:
Type | Description |
---|---|
Type[BentoMLModelDeployer]
|
The implementation class. |
logo_url: str
property
A url to represent the flavor in the dashboard.
Returns:
Type | Description |
---|---|
str
|
The flavor logo. |
name: str
property
Name of the flavor.
Returns:
Type | Description |
---|---|
str
|
Name of the flavor. |
sdk_docs_url: Optional[str]
property
A url to point at SDK docs explaining this flavor.
Returns:
Type | Description |
---|---|
Optional[str]
|
A flavor SDK docs url. |
Modules
bentoml_model_deployer_flavor
BentoML model deployer flavor.
BentoMLModelDeployerConfig(warn_about_plain_text_secrets: bool = False, **kwargs: Any)
Bases: BaseModelDeployerConfig
Configuration for the BentoMLModelDeployer.
Source code in src/zenml/stack/stack_component.py
61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 |
|
BentoMLModelDeployerFlavor
Bases: BaseModelDeployerFlavor
Flavor for the BentoML model deployer.
config_class: Type[BentoMLModelDeployerConfig]
property
Returns BentoMLModelDeployerConfig
config class.
Returns:
Type | Description |
---|---|
Type[BentoMLModelDeployerConfig]
|
The config class. |
docs_url: Optional[str]
property
A url to point at docs explaining this flavor.
Returns:
Type | Description |
---|---|
Optional[str]
|
A flavor docs url. |
implementation_class: Type[BentoMLModelDeployer]
property
Implementation class for this flavor.
Returns:
Type | Description |
---|---|
Type[BentoMLModelDeployer]
|
The implementation class. |
logo_url: str
property
A url to represent the flavor in the dashboard.
Returns:
Type | Description |
---|---|
str
|
The flavor logo. |
name: str
property
Name of the flavor.
Returns:
Type | Description |
---|---|
str
|
Name of the flavor. |
sdk_docs_url: Optional[str]
property
A url to point at SDK docs explaining this flavor.
Returns:
Type | Description |
---|---|
Optional[str]
|
A flavor SDK docs url. |
materializers
Initialization of the BentoML Bento Materializer.
Classes
Modules
bentoml_bento_materializer
Materializer for BentoML Bento objects.
BentoMaterializer(uri: str, artifact_store: Optional[BaseArtifactStore] = None)
Bases: BaseMaterializer
Materializer for Bentoml Bento objects.
Source code in src/zenml/materializers/base_materializer.py
125 126 127 128 129 130 131 132 133 134 135 |
|
extract_metadata(bento: bento.Bento) -> Dict[str, MetadataType]
Extract metadata from the given Bento
object.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
bento
|
Bento
|
The |
required |
Returns:
Type | Description |
---|---|
Dict[str, MetadataType]
|
The extracted metadata as a dictionary. |
Source code in src/zenml/integrations/bentoml/materializers/bentoml_bento_materializer.py
77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 |
|
load(data_type: Type[bento.Bento]) -> bento.Bento
Read from artifact store and return a Bento object.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
data_type
|
Type[Bento]
|
An bento.Bento type. |
required |
Returns:
Type | Description |
---|---|
Bento
|
An bento.Bento object. |
Source code in src/zenml/integrations/bentoml/materializers/bentoml_bento_materializer.py
41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 |
|
save(bento: bento.Bento) -> None
Write to artifact store.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
bento
|
Bento
|
An bento.Bento object. |
required |
Source code in src/zenml/integrations/bentoml/materializers/bentoml_bento_materializer.py
66 67 68 69 70 71 72 73 74 75 |
|
model_deployers
Initialization of the BentoML Model Deployer.
Classes
BentoMLModelDeployer(name: str, id: UUID, config: StackComponentConfig, flavor: str, type: StackComponentType, user: Optional[UUID], created: datetime, updated: datetime, labels: Optional[Dict[str, Any]] = None, connector_requirements: Optional[ServiceConnectorRequirements] = None, connector: Optional[UUID] = None, connector_resource_id: Optional[str] = None, *args: Any, **kwargs: Any)
Bases: BaseModelDeployer
BentoML model deployer stack component implementation.
Source code in src/zenml/stack/stack_component.py
328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 |
|
config: BentoMLModelDeployerConfig
property
Returns the BentoMLModelDeployerConfig
config.
Returns:
Type | Description |
---|---|
BentoMLModelDeployerConfig
|
The configuration. |
local_path: str
property
Returns the path to the root directory.
This is where all configurations for BentoML deployment daemon processes are stored.
If the service path is not set in the config by the user, the path is set to a local default path according to the component ID.
Returns:
Type | Description |
---|---|
str
|
The path to the local service root directory. |
get_model_server_info(service_instance: BaseService) -> Dict[str, Optional[str]]
staticmethod
Return implementation specific information on the model server.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
service_instance
|
BaseService
|
BentoML deployment service object |
required |
Returns:
Type | Description |
---|---|
Dict[str, Optional[str]]
|
A dictionary containing the model server information. |
Raises:
Type | Description |
---|---|
ValueError
|
If the service type is not supported. |
Source code in src/zenml/integrations/bentoml/model_deployers/bentoml_model_deployer.py
110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 |
|
get_service_path(id_: UUID) -> str
staticmethod
Get the path where local BentoML service information is stored.
This includes the deployment service configuration, PID and log files are stored.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
id_
|
UUID
|
The ID of the BentoML model deployer. |
required |
Returns:
Type | Description |
---|---|
str
|
The service path. |
Source code in src/zenml/integrations/bentoml/model_deployers/bentoml_model_deployer.py
66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 |
|
perform_delete_model(service: BaseService, timeout: int = DEFAULT_SERVICE_START_STOP_TIMEOUT, force: bool = False) -> None
Method to delete all configuration of a model server.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
service
|
BaseService
|
The service to delete. |
required |
timeout
|
int
|
Timeout in seconds to wait for the service to stop. |
DEFAULT_SERVICE_START_STOP_TIMEOUT
|
force
|
bool
|
If True, force the service to stop. |
False
|
Source code in src/zenml/integrations/bentoml/model_deployers/bentoml_model_deployer.py
331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 |
|
perform_deploy_model(id: UUID, config: ServiceConfig, timeout: int = DEFAULT_SERVICE_START_STOP_TIMEOUT) -> BaseService
Create a new BentoML deployment service or update an existing one.
This should serve the supplied model and deployment configuration.
This method has two modes of operation, depending on the replace
argument value:
-
if
replace
is False, calling this method will create a new BentoML deployment server to reflect the model and other configuration parameters specified in the supplied BentoML serviceconfig
. -
if
replace
is True, this method will first attempt to find an existing BentoML deployment service that is equivalent to the supplied configuration parameters. Two or more BentoML deployment services are considered equivalent if they have the samepipeline_name
,pipeline_step_name
andmodel_name
configuration parameters. To put it differently, two BentoML deployment services are equivalent if they serve versions of the same model deployed by the same pipeline step. If an equivalent BentoML deployment is found, it will be updated in place to reflect the new configuration parameters.
Callers should set replace
to True if they want a continuous model
deployment workflow that doesn't spin up a new BentoML deployment
server for each new model version. If multiple equivalent BentoML
deployment servers are found, one is selected at random to be updated
and the others are deleted.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
id
|
UUID
|
the UUID of the BentoML model deployer. |
required |
config
|
ServiceConfig
|
the configuration of the model to be deployed with BentoML. |
required |
timeout
|
int
|
the timeout in seconds to wait for the BentoML server to be provisioned and successfully started or updated. If set to 0, the method will return immediately after the BentoML server is provisioned, without waiting for it to fully start. |
DEFAULT_SERVICE_START_STOP_TIMEOUT
|
Returns:
Type | Description |
---|---|
BaseService
|
The ZenML BentoML deployment service object that can be used to |
BaseService
|
interact with the BentoML model http server. |
Source code in src/zenml/integrations/bentoml/model_deployers/bentoml_model_deployer.py
179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 |
|
perform_start_model(service: BaseService, timeout: int = DEFAULT_SERVICE_START_STOP_TIMEOUT) -> BaseService
Method to start a model server.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
service
|
BaseService
|
The service to start. |
required |
timeout
|
int
|
Timeout in seconds to wait for the service to start. |
DEFAULT_SERVICE_START_STOP_TIMEOUT
|
Returns:
Type | Description |
---|---|
BaseService
|
The started service. |
Source code in src/zenml/integrations/bentoml/model_deployers/bentoml_model_deployer.py
314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 |
|
perform_stop_model(service: BaseService, timeout: int = DEFAULT_SERVICE_START_STOP_TIMEOUT, force: bool = False) -> BaseService
Method to stop a model server.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
service
|
BaseService
|
The service to stop. |
required |
timeout
|
int
|
Timeout in seconds to wait for the service to stop. |
DEFAULT_SERVICE_START_STOP_TIMEOUT
|
force
|
bool
|
If True, force the service to stop. |
False
|
Returns:
Type | Description |
---|---|
BaseService
|
The stopped service. |
Source code in src/zenml/integrations/bentoml/model_deployers/bentoml_model_deployer.py
295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 |
|
Modules
bentoml_model_deployer
Implementation of the BentoML Model Deployer.
BentoMLModelDeployer(name: str, id: UUID, config: StackComponentConfig, flavor: str, type: StackComponentType, user: Optional[UUID], created: datetime, updated: datetime, labels: Optional[Dict[str, Any]] = None, connector_requirements: Optional[ServiceConnectorRequirements] = None, connector: Optional[UUID] = None, connector_resource_id: Optional[str] = None, *args: Any, **kwargs: Any)
Bases: BaseModelDeployer
BentoML model deployer stack component implementation.
Source code in src/zenml/stack/stack_component.py
328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 |
|
config: BentoMLModelDeployerConfig
property
Returns the BentoMLModelDeployerConfig
config.
Returns:
Type | Description |
---|---|
BentoMLModelDeployerConfig
|
The configuration. |
local_path: str
property
Returns the path to the root directory.
This is where all configurations for BentoML deployment daemon processes are stored.
If the service path is not set in the config by the user, the path is set to a local default path according to the component ID.
Returns:
Type | Description |
---|---|
str
|
The path to the local service root directory. |
get_model_server_info(service_instance: BaseService) -> Dict[str, Optional[str]]
staticmethod
Return implementation specific information on the model server.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
service_instance
|
BaseService
|
BentoML deployment service object |
required |
Returns:
Type | Description |
---|---|
Dict[str, Optional[str]]
|
A dictionary containing the model server information. |
Raises:
Type | Description |
---|---|
ValueError
|
If the service type is not supported. |
Source code in src/zenml/integrations/bentoml/model_deployers/bentoml_model_deployer.py
110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 |
|
get_service_path(id_: UUID) -> str
staticmethod
Get the path where local BentoML service information is stored.
This includes the deployment service configuration, PID and log files are stored.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
id_
|
UUID
|
The ID of the BentoML model deployer. |
required |
Returns:
Type | Description |
---|---|
str
|
The service path. |
Source code in src/zenml/integrations/bentoml/model_deployers/bentoml_model_deployer.py
66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 |
|
perform_delete_model(service: BaseService, timeout: int = DEFAULT_SERVICE_START_STOP_TIMEOUT, force: bool = False) -> None
Method to delete all configuration of a model server.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
service
|
BaseService
|
The service to delete. |
required |
timeout
|
int
|
Timeout in seconds to wait for the service to stop. |
DEFAULT_SERVICE_START_STOP_TIMEOUT
|
force
|
bool
|
If True, force the service to stop. |
False
|
Source code in src/zenml/integrations/bentoml/model_deployers/bentoml_model_deployer.py
331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 |
|
perform_deploy_model(id: UUID, config: ServiceConfig, timeout: int = DEFAULT_SERVICE_START_STOP_TIMEOUT) -> BaseService
Create a new BentoML deployment service or update an existing one.
This should serve the supplied model and deployment configuration.
This method has two modes of operation, depending on the replace
argument value:
-
if
replace
is False, calling this method will create a new BentoML deployment server to reflect the model and other configuration parameters specified in the supplied BentoML serviceconfig
. -
if
replace
is True, this method will first attempt to find an existing BentoML deployment service that is equivalent to the supplied configuration parameters. Two or more BentoML deployment services are considered equivalent if they have the samepipeline_name
,pipeline_step_name
andmodel_name
configuration parameters. To put it differently, two BentoML deployment services are equivalent if they serve versions of the same model deployed by the same pipeline step. If an equivalent BentoML deployment is found, it will be updated in place to reflect the new configuration parameters.
Callers should set replace
to True if they want a continuous model
deployment workflow that doesn't spin up a new BentoML deployment
server for each new model version. If multiple equivalent BentoML
deployment servers are found, one is selected at random to be updated
and the others are deleted.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
id
|
UUID
|
the UUID of the BentoML model deployer. |
required |
config
|
ServiceConfig
|
the configuration of the model to be deployed with BentoML. |
required |
timeout
|
int
|
the timeout in seconds to wait for the BentoML server to be provisioned and successfully started or updated. If set to 0, the method will return immediately after the BentoML server is provisioned, without waiting for it to fully start. |
DEFAULT_SERVICE_START_STOP_TIMEOUT
|
Returns:
Type | Description |
---|---|
BaseService
|
The ZenML BentoML deployment service object that can be used to |
BaseService
|
interact with the BentoML model http server. |
Source code in src/zenml/integrations/bentoml/model_deployers/bentoml_model_deployer.py
179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 |
|
perform_start_model(service: BaseService, timeout: int = DEFAULT_SERVICE_START_STOP_TIMEOUT) -> BaseService
Method to start a model server.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
service
|
BaseService
|
The service to start. |
required |
timeout
|
int
|
Timeout in seconds to wait for the service to start. |
DEFAULT_SERVICE_START_STOP_TIMEOUT
|
Returns:
Type | Description |
---|---|
BaseService
|
The started service. |
Source code in src/zenml/integrations/bentoml/model_deployers/bentoml_model_deployer.py
314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 |
|
perform_stop_model(service: BaseService, timeout: int = DEFAULT_SERVICE_START_STOP_TIMEOUT, force: bool = False) -> BaseService
Method to stop a model server.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
service
|
BaseService
|
The service to stop. |
required |
timeout
|
int
|
Timeout in seconds to wait for the service to stop. |
DEFAULT_SERVICE_START_STOP_TIMEOUT
|
force
|
bool
|
If True, force the service to stop. |
False
|
Returns:
Type | Description |
---|---|
BaseService
|
The stopped service. |
Source code in src/zenml/integrations/bentoml/model_deployers/bentoml_model_deployer.py
295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 |
|
services
Initialization for BentoML services.
Classes
BentoMLContainerDeploymentConfig(**data: Any)
Bases: ContainerServiceConfig
BentoML container deployment configuration.
Source code in src/zenml/services/service.py
131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 |
|
BentoMLContainerDeploymentService(config: Union[BentoMLContainerDeploymentConfig, Dict[str, Any]], **attrs: Any)
Bases: ContainerService
, BaseDeploymentService
BentoML container deployment service.
Initialize the BentoML deployment service.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
config
|
Union[BentoMLContainerDeploymentConfig, Dict[str, Any]]
|
service configuration |
required |
attrs
|
Any
|
additional attributes to set on the service |
{}
|
Source code in src/zenml/integrations/bentoml/services/bentoml_container_deployment.py
122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 |
|
is_running: bool
property
Check if the service is currently running.
This method will actively poll the external service to get its status and will return the result.
Returns:
Type | Description |
---|---|
bool
|
True if the service is running and active (i.e. the endpoints are |
bool
|
responsive, if any are configured), otherwise False. |
prediction_apis_urls: Optional[List[str]]
property
Get the URI where the prediction api services is answering requests.
Returns:
Type | Description |
---|---|
Optional[List[str]]
|
The URI where the prediction service apis can be contacted to process |
Optional[List[str]]
|
HTTP/REST inference requests, or None, if the service isn't running. |
prediction_url: Optional[str]
property
Get the URI where the http server is running.
Returns:
Type | Description |
---|---|
Optional[str]
|
The URI where the http service can be accessed to get more information |
Optional[str]
|
about the service and to make predictions. |
predict(api_endpoint: str, data: Any) -> Any
Make a prediction using the service.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
data
|
Any
|
data to make a prediction on |
required |
api_endpoint
|
str
|
the api endpoint to make the prediction on |
required |
Returns:
Type | Description |
---|---|
Any
|
The prediction result. |
Raises:
Type | Description |
---|---|
Exception
|
if the service is not running |
ValueError
|
if the prediction endpoint is unknown. |
Source code in src/zenml/integrations/bentoml/services/bentoml_container_deployment.py
373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 |
|
provision() -> None
Provision the service.
Source code in src/zenml/integrations/bentoml/services/bentoml_container_deployment.py
297 298 299 300 301 302 |
|
run() -> None
Start the service.
Raises:
Type | Description |
---|---|
FileNotFoundError
|
If the bento file is not found. |
CalledProcessError
|
If the bentoml serve command fails. |
Source code in src/zenml/integrations/bentoml/services/bentoml_container_deployment.py
304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 |
|
BentoMLDeploymentType
Bases: Enum
BentoML Service Deployment Types.
BentoMLLocalDeploymentConfig(**data: Any)
Bases: LocalDaemonServiceConfig
BentoML model deployment configuration.
Attributes:
Name | Type | Description |
---|---|---|
model_name |
str
|
name of the model to deploy |
model_uri |
str
|
URI of the model to deploy |
port |
Optional[int]
|
port to expose the service on |
bento_tag |
str
|
Bento package to deploy. A bento tag is a combination of the name of the bento and its version. |
workers |
int
|
number of workers to use |
backlog |
int
|
number of requests to queue |
production |
bool
|
whether to run in production mode |
working_dir |
str
|
working directory for the service |
host |
Optional[str]
|
host to expose the service on |
ssl_parameters |
Optional[SSLBentoMLParametersConfig]
|
SSL parameters for the Bentoml deployment |
Source code in src/zenml/services/service.py
131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 |
|
BentoMLLocalDeploymentService(config: Union[BentoMLLocalDeploymentConfig, Dict[str, Any]], **attrs: Any)
Bases: LocalDaemonService
, BaseDeploymentService
BentoML deployment service used to start a local prediction server for BentoML models.
Attributes:
Name | Type | Description |
---|---|---|
SERVICE_TYPE |
a service type descriptor with information describing the BentoML deployment service class |
|
config |
BentoMLLocalDeploymentConfig
|
service configuration |
endpoint |
BentoMLDeploymentEndpoint
|
optional service endpoint |
Initialize the BentoML deployment service.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
config
|
Union[BentoMLLocalDeploymentConfig, Dict[str, Any]]
|
service configuration |
required |
attrs
|
Any
|
additional attributes to set on the service |
{}
|
Source code in src/zenml/integrations/bentoml/services/bentoml_local_deployment.py
160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 |
|
prediction_apis_urls: Optional[List[str]]
property
Get the URI where the prediction api services is answering requests.
Returns:
Type | Description |
---|---|
Optional[List[str]]
|
The URI where the prediction service apis can be contacted to process |
Optional[List[str]]
|
HTTP/REST inference requests, or None, if the service isn't running. |
prediction_url: Optional[str]
property
Get the URI where the http server is running.
Returns:
Type | Description |
---|---|
Optional[str]
|
The URI where the http service can be accessed to get more information |
Optional[str]
|
about the service and to make predictions. |
predict(api_endpoint: str, data: Any, sync: bool = True) -> Any
Make a prediction using the service.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
data
|
Any
|
data to make a prediction on |
required |
api_endpoint
|
str
|
the api endpoint to make the prediction on |
required |
sync
|
bool
|
if set to False, the prediction will be made asynchronously |
True
|
Returns:
Type | Description |
---|---|
Any
|
The prediction result. |
Raises:
Type | Description |
---|---|
Exception
|
if the service is not running |
ValueError
|
if the prediction endpoint is unknown. |
Source code in src/zenml/integrations/bentoml/services/bentoml_local_deployment.py
290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 |
|
run() -> None
Start the service.
Source code in src/zenml/integrations/bentoml/services/bentoml_local_deployment.py
196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 |
|
Modules
bentoml_container_deployment
Implementation for the BentoML container deployment service.
BentoMLContainerDeploymentConfig(**data: Any)
Bases: ContainerServiceConfig
BentoML container deployment configuration.
Source code in src/zenml/services/service.py
131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 |
|
BentoMLContainerDeploymentEndpoint(*args: Any, **kwargs: Any)
Bases: ContainerServiceEndpoint
A service endpoint exposed by the BentoML container deployment service.
Attributes:
Name | Type | Description |
---|---|---|
config |
BentoMLContainerDeploymentEndpointConfig
|
service endpoint configuration |
Source code in src/zenml/services/service_endpoint.py
111 112 113 114 115 116 117 118 119 120 121 122 123 |
|
prediction_url: Optional[str]
property
Gets the prediction URL for the endpoint.
Returns:
Type | Description |
---|---|
Optional[str]
|
the prediction URL for the endpoint |
BentoMLContainerDeploymentEndpointConfig
Bases: ContainerServiceEndpointConfig
BentoML container deployment service configuration.
Attributes:
Name | Type | Description |
---|---|---|
prediction_url_path |
str
|
URI subpath for prediction requests |
BentoMLContainerDeploymentService(config: Union[BentoMLContainerDeploymentConfig, Dict[str, Any]], **attrs: Any)
Bases: ContainerService
, BaseDeploymentService
BentoML container deployment service.
Initialize the BentoML deployment service.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
config
|
Union[BentoMLContainerDeploymentConfig, Dict[str, Any]]
|
service configuration |
required |
attrs
|
Any
|
additional attributes to set on the service |
{}
|
Source code in src/zenml/integrations/bentoml/services/bentoml_container_deployment.py
122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 |
|
is_running: bool
property
Check if the service is currently running.
This method will actively poll the external service to get its status and will return the result.
Returns:
Type | Description |
---|---|
bool
|
True if the service is running and active (i.e. the endpoints are |
bool
|
responsive, if any are configured), otherwise False. |
prediction_apis_urls: Optional[List[str]]
property
Get the URI where the prediction api services is answering requests.
Returns:
Type | Description |
---|---|
Optional[List[str]]
|
The URI where the prediction service apis can be contacted to process |
Optional[List[str]]
|
HTTP/REST inference requests, or None, if the service isn't running. |
prediction_url: Optional[str]
property
Get the URI where the http server is running.
Returns:
Type | Description |
---|---|
Optional[str]
|
The URI where the http service can be accessed to get more information |
Optional[str]
|
about the service and to make predictions. |
predict(api_endpoint: str, data: Any) -> Any
Make a prediction using the service.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
data
|
Any
|
data to make a prediction on |
required |
api_endpoint
|
str
|
the api endpoint to make the prediction on |
required |
Returns:
Type | Description |
---|---|
Any
|
The prediction result. |
Raises:
Type | Description |
---|---|
Exception
|
if the service is not running |
ValueError
|
if the prediction endpoint is unknown. |
Source code in src/zenml/integrations/bentoml/services/bentoml_container_deployment.py
373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 |
|
provision() -> None
Provision the service.
Source code in src/zenml/integrations/bentoml/services/bentoml_container_deployment.py
297 298 299 300 301 302 |
|
run() -> None
Start the service.
Raises:
Type | Description |
---|---|
FileNotFoundError
|
If the bento file is not found. |
CalledProcessError
|
If the bentoml serve command fails. |
Source code in src/zenml/integrations/bentoml/services/bentoml_container_deployment.py
304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 |
|
bentoml_local_deployment
Implementation for the BentoML local deployment service.
BentoMLDeploymentEndpoint(*args: Any, **kwargs: Any)
Bases: LocalDaemonServiceEndpoint
A service endpoint exposed by the BentoML deployment daemon.
Attributes:
Name | Type | Description |
---|---|---|
config |
BentoMLDeploymentEndpointConfig
|
service endpoint configuration |
Source code in src/zenml/services/service_endpoint.py
111 112 113 114 115 116 117 118 119 120 121 122 123 |
|
prediction_url: Optional[str]
property
Gets the prediction URL for the endpoint.
Returns:
Type | Description |
---|---|
Optional[str]
|
the prediction URL for the endpoint |
BentoMLDeploymentEndpointConfig
Bases: LocalDaemonServiceEndpointConfig
BentoML deployment service configuration.
Attributes:
Name | Type | Description |
---|---|---|
prediction_url_path |
str
|
URI subpath for prediction requests |
BentoMLLocalDeploymentConfig(**data: Any)
Bases: LocalDaemonServiceConfig
BentoML model deployment configuration.
Attributes:
Name | Type | Description |
---|---|---|
model_name |
str
|
name of the model to deploy |
model_uri |
str
|
URI of the model to deploy |
port |
Optional[int]
|
port to expose the service on |
bento_tag |
str
|
Bento package to deploy. A bento tag is a combination of the name of the bento and its version. |
workers |
int
|
number of workers to use |
backlog |
int
|
number of requests to queue |
production |
bool
|
whether to run in production mode |
working_dir |
str
|
working directory for the service |
host |
Optional[str]
|
host to expose the service on |
ssl_parameters |
Optional[SSLBentoMLParametersConfig]
|
SSL parameters for the Bentoml deployment |
Source code in src/zenml/services/service.py
131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 |
|
BentoMLLocalDeploymentService(config: Union[BentoMLLocalDeploymentConfig, Dict[str, Any]], **attrs: Any)
Bases: LocalDaemonService
, BaseDeploymentService
BentoML deployment service used to start a local prediction server for BentoML models.
Attributes:
Name | Type | Description |
---|---|---|
SERVICE_TYPE |
a service type descriptor with information describing the BentoML deployment service class |
|
config |
BentoMLLocalDeploymentConfig
|
service configuration |
endpoint |
BentoMLDeploymentEndpoint
|
optional service endpoint |
Initialize the BentoML deployment service.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
config
|
Union[BentoMLLocalDeploymentConfig, Dict[str, Any]]
|
service configuration |
required |
attrs
|
Any
|
additional attributes to set on the service |
{}
|
Source code in src/zenml/integrations/bentoml/services/bentoml_local_deployment.py
160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 |
|
prediction_apis_urls: Optional[List[str]]
property
Get the URI where the prediction api services is answering requests.
Returns:
Type | Description |
---|---|
Optional[List[str]]
|
The URI where the prediction service apis can be contacted to process |
Optional[List[str]]
|
HTTP/REST inference requests, or None, if the service isn't running. |
prediction_url: Optional[str]
property
Get the URI where the http server is running.
Returns:
Type | Description |
---|---|
Optional[str]
|
The URI where the http service can be accessed to get more information |
Optional[str]
|
about the service and to make predictions. |
predict(api_endpoint: str, data: Any, sync: bool = True) -> Any
Make a prediction using the service.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
data
|
Any
|
data to make a prediction on |
required |
api_endpoint
|
str
|
the api endpoint to make the prediction on |
required |
sync
|
bool
|
if set to False, the prediction will be made asynchronously |
True
|
Returns:
Type | Description |
---|---|
Any
|
The prediction result. |
Raises:
Type | Description |
---|---|
Exception
|
if the service is not running |
ValueError
|
if the prediction endpoint is unknown. |
Source code in src/zenml/integrations/bentoml/services/bentoml_local_deployment.py
290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 |
|
run() -> None
Start the service.
Source code in src/zenml/integrations/bentoml/services/bentoml_local_deployment.py
196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 |
|
SSLBentoMLParametersConfig
Bases: BaseModel
BentoML SSL parameters configuration.
Attributes:
Name | Type | Description |
---|---|---|
ssl_certfile |
Optional[str]
|
SSL certificate file |
ssl_keyfile |
Optional[str]
|
SSL key file |
ssl_keyfile_password |
Optional[str]
|
SSL key file password |
ssl_version |
Optional[int]
|
SSL version |
ssl_cert_reqs |
Optional[int]
|
SSL certificate requirements |
ssl_ca_certs |
Optional[str]
|
SSL CA certificates |
ssl_ciphers |
Optional[str]
|
SSL ciphers |
deployment_type
BentoML Service Deployment Types.
BentoMLDeploymentType
Bases: Enum
BentoML Service Deployment Types.
steps
Initialization of the BentoML standard interface steps.
Functions
bento_builder_step(model: UnmaterializedArtifact, model_name: str, model_type: str, service: str, version: Optional[str] = None, labels: Optional[Dict[str, str]] = None, description: Optional[str] = None, include: Optional[List[str]] = None, exclude: Optional[List[str]] = None, python: Optional[Dict[str, Any]] = None, docker: Optional[Dict[str, Any]] = None, working_dir: Optional[str] = None) -> bento.Bento
Build a BentoML Model and Bento bundle.
This steps takes a model artifact of a trained or loaded ML model in a previous step and save it with BentoML, then build a BentoML bundle.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
model
|
UnmaterializedArtifact
|
the model to be packaged. |
required |
model_name
|
str
|
the name of the model to be packaged. |
required |
model_type
|
str
|
the type of the model. |
required |
service
|
str
|
the name of the BentoML service to be deployed. |
required |
version
|
Optional[str]
|
the version of the model if given. |
None
|
labels
|
Optional[Dict[str, str]]
|
the labels of the model if given. |
None
|
description
|
Optional[str]
|
the description of the model if given. |
None
|
include
|
Optional[List[str]]
|
the files to be included in the BentoML bundle. |
None
|
exclude
|
Optional[List[str]]
|
the files to be excluded from the BentoML bundle. |
None
|
python
|
Optional[Dict[str, Any]]
|
dictionary for configuring Bento's python dependencies, |
None
|
docker
|
Optional[Dict[str, Any]]
|
dictionary for configuring Bento's docker image. |
None
|
working_dir
|
Optional[str]
|
the working directory of the BentoML bundle. |
None
|
Returns:
Type | Description |
---|---|
Bento
|
the BentoML Bento object. |
Source code in src/zenml/integrations/bentoml/steps/bento_builder.py
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 |
|
bentoml_model_deployer_step(bento: bento.Bento, model_name: str, port: int, deployment_type: BentoMLDeploymentType = BentoMLDeploymentType.LOCAL, deploy_decision: bool = True, workers: Optional[int] = 1, backlog: Optional[int] = 2048, production: bool = False, working_dir: Optional[str] = None, host: Optional[str] = None, image: Optional[str] = None, image_tag: Optional[str] = None, platform: Optional[str] = None, ssl_certfile: Optional[str] = None, ssl_keyfile: Optional[str] = None, ssl_keyfile_password: Optional[str] = None, ssl_version: Optional[str] = None, ssl_cert_reqs: Optional[str] = None, ssl_ca_certs: Optional[str] = None, ssl_ciphers: Optional[str] = None, timeout: int = 30) -> BaseService
Model deployer pipeline step for BentoML.
This step deploys a given Bento to a local BentoML http prediction server.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
bento
|
Bento
|
the bento artifact to deploy |
required |
model_name
|
str
|
the name of the model to deploy. |
required |
port
|
int
|
the port to use for the prediction service. |
required |
deployment_type
|
BentoMLDeploymentType
|
the type of deployment to use. Either "local" or "container". |
LOCAL
|
deploy_decision
|
bool
|
whether to deploy the model or not |
True
|
workers
|
Optional[int]
|
number of workers to use for the prediction service |
1
|
backlog
|
Optional[int]
|
the number of requests to queue up before rejecting requests. |
2048
|
production
|
bool
|
whether to deploy the service in production mode. |
False
|
working_dir
|
Optional[str]
|
the working directory to use for the prediction service. |
None
|
host
|
Optional[str]
|
the host to use for the prediction service. |
None
|
image
|
Optional[str]
|
the image to use for the prediction service in the container deployment. |
None
|
image_tag
|
Optional[str]
|
the image tag to use for the prediction service in the container deployment. |
None
|
platform
|
Optional[str]
|
the platform to use to build the image for the container deployment. |
None
|
ssl_certfile
|
Optional[str]
|
the path to the ssl cert file. |
None
|
ssl_keyfile
|
Optional[str]
|
the path to the ssl key file. |
None
|
ssl_keyfile_password
|
Optional[str]
|
the password for the ssl key file. |
None
|
ssl_version
|
Optional[str]
|
the ssl version to use. |
None
|
ssl_cert_reqs
|
Optional[str]
|
the ssl cert requirements. |
None
|
ssl_ca_certs
|
Optional[str]
|
the path to the ssl ca certs. |
None
|
ssl_ciphers
|
Optional[str]
|
the ssl ciphers to use. |
None
|
timeout
|
int
|
the number of seconds to wait for the service to start/stop. |
30
|
Returns:
Type | Description |
---|---|
BaseService
|
BentoML deployment service |
Source code in src/zenml/integrations/bentoml/steps/bentoml_deployer.py
45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 |
|
Modules
bento_builder
Implementation of the BentoML bento builder step.
bento_builder_step(model: UnmaterializedArtifact, model_name: str, model_type: str, service: str, version: Optional[str] = None, labels: Optional[Dict[str, str]] = None, description: Optional[str] = None, include: Optional[List[str]] = None, exclude: Optional[List[str]] = None, python: Optional[Dict[str, Any]] = None, docker: Optional[Dict[str, Any]] = None, working_dir: Optional[str] = None) -> bento.Bento
Build a BentoML Model and Bento bundle.
This steps takes a model artifact of a trained or loaded ML model in a previous step and save it with BentoML, then build a BentoML bundle.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
model
|
UnmaterializedArtifact
|
the model to be packaged. |
required |
model_name
|
str
|
the name of the model to be packaged. |
required |
model_type
|
str
|
the type of the model. |
required |
service
|
str
|
the name of the BentoML service to be deployed. |
required |
version
|
Optional[str]
|
the version of the model if given. |
None
|
labels
|
Optional[Dict[str, str]]
|
the labels of the model if given. |
None
|
description
|
Optional[str]
|
the description of the model if given. |
None
|
include
|
Optional[List[str]]
|
the files to be included in the BentoML bundle. |
None
|
exclude
|
Optional[List[str]]
|
the files to be excluded from the BentoML bundle. |
None
|
python
|
Optional[Dict[str, Any]]
|
dictionary for configuring Bento's python dependencies, |
None
|
docker
|
Optional[Dict[str, Any]]
|
dictionary for configuring Bento's docker image. |
None
|
working_dir
|
Optional[str]
|
the working directory of the BentoML bundle. |
None
|
Returns:
Type | Description |
---|---|
Bento
|
the BentoML Bento object. |
Source code in src/zenml/integrations/bentoml/steps/bento_builder.py
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 |
|
bentoml_deployer
Implementation of the BentoML model deployer pipeline step.
bentoml_model_deployer_step(bento: bento.Bento, model_name: str, port: int, deployment_type: BentoMLDeploymentType = BentoMLDeploymentType.LOCAL, deploy_decision: bool = True, workers: Optional[int] = 1, backlog: Optional[int] = 2048, production: bool = False, working_dir: Optional[str] = None, host: Optional[str] = None, image: Optional[str] = None, image_tag: Optional[str] = None, platform: Optional[str] = None, ssl_certfile: Optional[str] = None, ssl_keyfile: Optional[str] = None, ssl_keyfile_password: Optional[str] = None, ssl_version: Optional[str] = None, ssl_cert_reqs: Optional[str] = None, ssl_ca_certs: Optional[str] = None, ssl_ciphers: Optional[str] = None, timeout: int = 30) -> BaseService
Model deployer pipeline step for BentoML.
This step deploys a given Bento to a local BentoML http prediction server.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
bento
|
Bento
|
the bento artifact to deploy |
required |
model_name
|
str
|
the name of the model to deploy. |
required |
port
|
int
|
the port to use for the prediction service. |
required |
deployment_type
|
BentoMLDeploymentType
|
the type of deployment to use. Either "local" or "container". |
LOCAL
|
deploy_decision
|
bool
|
whether to deploy the model or not |
True
|
workers
|
Optional[int]
|
number of workers to use for the prediction service |
1
|
backlog
|
Optional[int]
|
the number of requests to queue up before rejecting requests. |
2048
|
production
|
bool
|
whether to deploy the service in production mode. |
False
|
working_dir
|
Optional[str]
|
the working directory to use for the prediction service. |
None
|
host
|
Optional[str]
|
the host to use for the prediction service. |
None
|
image
|
Optional[str]
|
the image to use for the prediction service in the container deployment. |
None
|
image_tag
|
Optional[str]
|
the image tag to use for the prediction service in the container deployment. |
None
|
platform
|
Optional[str]
|
the platform to use to build the image for the container deployment. |
None
|
ssl_certfile
|
Optional[str]
|
the path to the ssl cert file. |
None
|
ssl_keyfile
|
Optional[str]
|
the path to the ssl key file. |
None
|
ssl_keyfile_password
|
Optional[str]
|
the password for the ssl key file. |
None
|
ssl_version
|
Optional[str]
|
the ssl version to use. |
None
|
ssl_cert_reqs
|
Optional[str]
|
the ssl cert requirements. |
None
|
ssl_ca_certs
|
Optional[str]
|
the path to the ssl ca certs. |
None
|
ssl_ciphers
|
Optional[str]
|
the ssl ciphers to use. |
None
|
timeout
|
int
|
the number of seconds to wait for the service to start/stop. |
30
|
Returns:
Type | Description |
---|---|
BaseService
|
BentoML deployment service |
Source code in src/zenml/integrations/bentoml/steps/bentoml_deployer.py
45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 |
|