Skip to content

sparkdq.plugin

The sparkdq.plugin subpackage enables extensibility of the SparkDQ framework through user-defined plugins.

plugin

load_config_module 

load_config_module(module: str) -> None

Dynamically loads a module to trigger registration of custom check configurations.

This function is essential when integrating custom checks into the SparkDQ framework. By importing the specified module, all @register_check_config decorators within that module are executed, ensuring that the associated checks are registered and discoverable by the framework.

This step is required for custom checks to be properly recognized during validation, especially when they are defined outside the core package. If this function is not called, any custom checks in the given module will not be available for execution.

Example

load_config_module("my_project.custom_checks")

Parameters:

Name Type Description Default
module str

Fully qualified module name (e.g., "my_project.custom_checks") that contains custom check classes decorated with @register_check_config.

 required
Source code in sparkdq/plugin/check_config_registry.py 
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
def load_config_module(module: str) -> None:
    """
    Dynamically loads a module to trigger registration of custom check configurations.

    This function is essential when integrating custom checks into the SparkDQ framework.
    By importing the specified module, all ``@register_check_config`` decorators within
    that module are executed, ensuring that the associated checks are registered
    and discoverable by the framework.

    This step is required for custom checks to be properly recognized during validation,
    especially when they are defined outside the core package. If this function is not
    called, any custom checks in the given module will not be available for execution.

    Example:
        load_config_module("my_project.custom_checks")

    Args:
        module (str): Fully qualified module name (e.g., "my_project.custom_checks")
                      that contains custom check classes decorated with `@register_check_config`.
    """
    importlib.import_module(module)

register_check_config 

register_check_config(
    check_name: str,
) -> Callable[
    [Type[BaseCheckConfig]], Type[BaseCheckConfig]
]

Class decorator to register a check configuration class.

This decorator registers the given configuration class in the CheckConfigRegistry under the specified name and returns the class unchanged.

Parameters:

Name Type Description Default
check_name str

The name to register the configuration class under.

 required

Returns:

Type Description
Callable[[Type[BaseCheckConfig]], Type[BaseCheckConfig]]

Callable[[Type[BaseCheckConfig]], Type[BaseCheckConfig]]: The original configuration class.
Source code in sparkdq/plugin/check_config_registry.py 
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
def register_check_config(check_name: str) -> Callable[[Type[BaseCheckConfig]], Type[BaseCheckConfig]]:
    """
    Class decorator to register a check configuration class.

    This decorator registers the given configuration class in the CheckConfigRegistry
    under the specified name and returns the class unchanged.

    Args:
        check_name (str): The name to register the configuration class under.

    Returns:
        Callable[[Type[BaseCheckConfig]], Type[BaseCheckConfig]]: The original configuration class.
    """

    def decorator(cls: Type[BaseCheckConfig]) -> Type[BaseCheckConfig]:
        CheckConfigRegistry.register(check_name, cls)
        return cls

    return decorator