Skip to main content

databricks.labs.dqx.llm.llm_core

LLMModelConfigurator Objects​

class LLMModelConfigurator()

Configures DSPy language models.

__init__​

def __init__(model_config: LLMModelConfig)

Initialize model configurator.

Arguments:

  • model_config - Configuration for the LLM model.

create_lm​

def create_lm() -> dspy.LM

Create an LM instance with current config for per-request override.

Budget caps (max_tokens, temperature, timeout) come from LLMModelConfig and are forwarded to litellm via dspy.LM kwargs to bound cost and latency on pathological prompts (OWASP LLM04).

Returns:

A new LM instance configured with the current model config.

lm_context​

@contextmanager
def lm_context() -> Iterator[None]

Scope a block of work to a freshly created LM instance.

Each call creates a new LM so that the current credentials are picked up, rather than relying on a globally configured model. Use this to wrap any DSPy module invocation.

Yields:

None. The LM is active for the duration of the with block.

DspySchemaGuesserSignature Objects​

class DspySchemaGuesserSignature(dspy.Signature)

Guess a table schema based on business description.

DspySchemaGuesser Objects​

class DspySchemaGuesser(dspy.Module)

Guess table schema from business description.

forward​

def forward(
        business_description: str) -> dspy.primitives.prediction.Prediction

Guess schema based on business description.

Arguments:

  • business_description - Natural language description of the dataset.

Returns:

Prediction containing guessed schema and assumptions.

DspyRuleSignature Objects​

class DspyRuleSignature(dspy.Signature)

Generate data quality rules with improved output format.

DspyRuleGeneration Objects​

class DspyRuleGeneration(dspy.Module)

Generate data quality rules.

Now focused solely on rule generation, with schema inference delegated.

forward​

def forward(schema_info: str, business_description: str,
            available_functions: str) -> dspy.primitives.prediction.Prediction

Generate data quality rules.

Arguments:

  • schema_info - JSON string containing table schema.
  • business_description - Natural language description of requirements.
  • available_functions - JSON string of available check functions.

Returns:

Prediction containing quality_rules and reasoning.

DspyRuleGenerationWithSchemaInference Objects​

class DspyRuleGenerationWithSchemaInference(dspy.Module)

Combines schema inference and rule generation.

Follows Dependency Inversion Principle by depending on abstractions (protocols).

forward​

def forward(schema_info: str, business_description: str,
            available_functions: str) -> dspy.primitives.prediction.Prediction

Generate rules with optional schema inference.

Arguments:

  • schema_info - JSON string of schema (can be empty to trigger inference).
  • business_description - Natural language requirements.
  • available_functions - JSON string of available functions.

Returns:

Prediction with quality_rules, reasoning, and optional schema info.

DspyRuleUsingDataStatsSignature Objects​

class DspyRuleUsingDataStatsSignature(dspy.Signature)

Generate data quality rules using data summary statistics.

DspyRuleUsingDataStats Objects​

class DspyRuleUsingDataStats(dspy.Module)

Generate data quality rules using data summary statistics.

forward​

def forward(
    data_summary_stats: str,
    available_functions: str,
    business_description: str | None = None
) -> dspy.primitives.prediction.Prediction

Generate data quality rules.

Arguments:

  • data_summary_stats - JSON string containing summary statistics of the data.
  • available_functions - JSON string of available check functions.
  • business_description - Optional natural language description of data quality requirements.

Returns:

Prediction containing quality_rules and reasoning.

LLMRuleCompiler Objects​

class LLMRuleCompiler()

Compiles and optimizes LLM-based data quality rules.

Note: This class assumes DSPy is already configured with a language model. The configuration should be done externally before instantiating this class.

__init__​

def __init__(custom_check_functions: dict[str, Callable] | None = None,
             rule_validator: RuleValidator | None = None,
             optimizer: BootstrapFewShotOptimizer | None = None)

Initialize the rule compiler.

Note: DSPy must be configured before creating this instance.

Arguments:

  • custom_check_functions - Optional custom check functions.
  • rule_validator - Optional rule validator instance.
  • optimizer - Optional optimizer instance.

model​

@cached_property
def model() -> dspy.Module

Get the optimized DSPy model.

Returns:

Optimized DSPy module for generating data quality rules.

model_using_data_stats​

@cached_property
def model_using_data_stats() -> dspy.Module

Get the optimized DSPy model for generating rules from data summary statistics.

Returns:

Optimized DSPy module for generating data quality rules from data stats.