Feature Stores

zenml.feature_stores special

A feature store enables an offline and online serving of feature data.

Feature stores allow data teams to serve data via an offline store and an online low-latency store where data is kept in sync between the two. It also offers a centralized registry where features (and feature schemas) are stored for use within a team or wider organization.

As a data scientist working on training your model, your requirements for how you access your batch / 'offline' data will almost certainly be different from how you access that data as part of a real-time or online inference setting. Feast solves the problem of developing train-serve skew where those two sources of data diverge from each other.

base_feature_store

The base class for feature stores.

BaseFeatureStore (StackComponent, ABC)

Base class for all ZenML feature stores.

Source code in zenml/feature_stores/base_feature_store.py

class BaseFeatureStore(StackComponent, ABC):
    """Base class for all ZenML feature stores."""

    @property
    def config(self) -> BaseFeatureStoreConfig:
        """Returns the `BaseFeatureStoreConfig` config.

        Returns:
            The configuration.
        """
        return cast(BaseFeatureStoreConfig, self._config)

    @abstractmethod
    def get_historical_features(
        self,
        entity_df: Union[pd.DataFrame, str],
        features: List[str],
        full_feature_names: bool = False,
    ) -> pd.DataFrame:
        """Returns the historical features for training or batch scoring.

        Args:
            entity_df: The entity DataFrame or entity name.
            features: The features to retrieve.
            full_feature_names: Whether to return the full feature names.

        Returns:
            The historical features as a Pandas DataFrame.
        """

    @abstractmethod
    def get_online_features(
        self,
        entity_rows: List[Dict[str, Any]],
        features: List[str],
        full_feature_names: bool = False,
    ) -> Dict[str, Any]:
        """Returns the latest online feature data.

        Args:
            entity_rows: The entity rows to retrieve.
            features: The features to retrieve.
            full_feature_names: Whether to return the full feature names.

        Returns:
            The latest online feature data as a dictionary.
        """

config: BaseFeatureStoreConfig property readonly

Returns the BaseFeatureStoreConfig config.

Returns:

Type	Description
BaseFeatureStoreConfig	The configuration.

get_historical_features(self, entity_df, features, full_feature_names=False)

Returns the historical features for training or batch scoring.

Parameters:

Name	Type	Description	Default
entity_df	Union[pandas.core.frame.DataFrame, str]	The entity DataFrame or entity name.	required
features	List[str]	The features to retrieve.	required
full_feature_names	bool	Whether to return the full feature names.	False

Returns:

Type	Description
DataFrame	The historical features as a Pandas DataFrame.

Source code in zenml/feature_stores/base_feature_store.py

@abstractmethod
def get_historical_features(
    self,
    entity_df: Union[pd.DataFrame, str],
    features: List[str],
    full_feature_names: bool = False,
) -> pd.DataFrame:
    """Returns the historical features for training or batch scoring.

    Args:
        entity_df: The entity DataFrame or entity name.
        features: The features to retrieve.
        full_feature_names: Whether to return the full feature names.

    Returns:
        The historical features as a Pandas DataFrame.
    """

get_online_features(self, entity_rows, features, full_feature_names=False)

Returns the latest online feature data.

Parameters:

Name	Type	Description	Default
entity_rows	List[Dict[str, Any]]	The entity rows to retrieve.	required
features	List[str]	The features to retrieve.	required
full_feature_names	bool	Whether to return the full feature names.	False

Returns:

Type	Description
Dict[str, Any]	The latest online feature data as a dictionary.

Source code in zenml/feature_stores/base_feature_store.py

@abstractmethod
def get_online_features(
    self,
    entity_rows: List[Dict[str, Any]],
    features: List[str],
    full_feature_names: bool = False,
) -> Dict[str, Any]:
    """Returns the latest online feature data.

    Args:
        entity_rows: The entity rows to retrieve.
        features: The features to retrieve.
        full_feature_names: Whether to return the full feature names.

    Returns:
        The latest online feature data as a dictionary.
    """

BaseFeatureStoreConfig (StackComponentConfig) pydantic-model

Base config for feature stores.

Source code in zenml/feature_stores/base_feature_store.py

class BaseFeatureStoreConfig(StackComponentConfig):
    """Base config for feature stores."""

BaseFeatureStoreFlavor (Flavor)

Base class for all ZenML feature store flavors.

Source code in zenml/feature_stores/base_feature_store.py

class BaseFeatureStoreFlavor(Flavor):
    """Base class for all ZenML feature store flavors."""

    @property
    def type(self) -> StackComponentType:
        """Returns the flavor type.

        Returns:
            The flavor type.
        """
        return StackComponentType.FEATURE_STORE

    @property
    def config_class(self) -> Type[BaseFeatureStoreConfig]:
        """Config class for this flavor.

        Returns:
            The config class.
        """
        return BaseFeatureStoreConfig

    @property
    @abstractmethod
    def implementation_class(self) -> Type[BaseFeatureStore]:
        """Implementation class.

        Returns:
            The implementation class.
        """
        return BaseFeatureStore

config_class: Type[zenml.feature_stores.base_feature_store.BaseFeatureStoreConfig] property readonly

Config class for this flavor.

Returns:

Type	Description
Type[zenml.feature_stores.base_feature_store.BaseFeatureStoreConfig]	The config class.

implementation_class: Type[zenml.feature_stores.base_feature_store.BaseFeatureStore] property readonly

Implementation class.

Returns:

Type	Description
Type[zenml.feature_stores.base_feature_store.BaseFeatureStore]	The implementation class.

type: StackComponentType property readonly

Returns the flavor type.

Returns:

Type	Description
StackComponentType	The flavor type.