diff --git a/docs/reference/advanced/managed-variables/configuration-reference.md b/docs/reference/advanced/managed-variables/configuration-reference.md index f5b9fc6c7..bb1c44dee 100644 --- a/docs/reference/advanced/managed-variables/configuration-reference.md +++ b/docs/reference/advanced/managed-variables/configuration-reference.md @@ -14,7 +14,8 @@ | `json_schema` | JSON Schema for validation (optional) | | `description` | Human-readable description (optional) | | `aliases` | Alternative names that resolve to this variable (optional, for migrations) | -| `example` | JSON-serialized example value, used as template in UI (optional) | +| `example` | JSON-serialized example value, used as starting point when creating versions in the UI (optional) | +| `template_inputs_schema` | JSON Schema for template `{{placeholder}}` inputs (optional, set automatically by `logfire.template_var()`) | **LabeledValue** — A label with an inline serialized value: diff --git a/docs/reference/advanced/managed-variables/index.md b/docs/reference/advanced/managed-variables/index.md index b2fec38f7..3cb578a1c 100644 --- a/docs/reference/advanced/managed-variables/index.md +++ b/docs/reference/advanced/managed-variables/index.md @@ -14,6 +14,7 @@ Managed variables are a way to externalize runtime configuration from your code. - **Observability-integrated**: Every variable resolution creates a span, and using the context manager automatically sets baggage so downstream operations are tagged with which label and version was used - **Versions and labels**: Create immutable version snapshots of your variable's value, and assign labels (like `production`, `staging`, `canary`) that point to specific versions - **Rollouts and targeting**: Control what percentage of requests receive each labeled version, and route specific users or segments based on attributes +- **Templates and composition**: Use `{{placeholder}}` Handlebars syntax in values that get rendered with runtime inputs, and compose variables from reusable fragments via `<>` references (see [Templates and Composition](templates-and-composition.md)) ### Versions and Labels @@ -112,6 +113,36 @@ With managed variables, you can iterate safely in production: - **Instant rollback**: If a version is causing problems, move the label back to the previous version in seconds, with no deploy required - **Full history**: Every version is immutable and preserved, so you can always see exactly what was served and when +## Template Variables + +For AI applications, variables often contain prompt templates with placeholders that get filled in at runtime. **Template variables** support this natively with Handlebars `{{placeholder}}` syntax: + +```python skip="true" +from pydantic import BaseModel + +import logfire + + +class PromptInputs(BaseModel): + user_name: str + is_premium: bool = False + + +prompt = logfire.template_var( + 'system_prompt', + type=str, + default='Hello {{user_name}}!{{#if is_premium}} Welcome back, valued member.{{/if}}', + inputs_type=PromptInputs, +) + +with prompt.get(PromptInputs(user_name='Alice', is_premium=True)) as resolved: + print(resolved.value) # "Hello Alice! Welcome back, valued member." +``` + +Variables can also reference other variables using `<>` syntax, allowing you to compose values from reusable fragments that can be independently updated in the UI. + +For full details, see [Templates and Composition](templates-and-composition.md). + ## How It Works Here's the typical workflow using the `AgentConfig` example from above: @@ -231,8 +262,10 @@ This bypasses the rollout weights and directly resolves the value from the speci ### Variable Parameters -| Parameter | Description | -|-----------|-------------------------------------------------------------------------| -| `name` | Unique identifier for the variable | +| Parameter | Description | +|-----------|-------------| +| `name` | Unique identifier for the variable | | `type` | Expected type for validation; can be a primitive type or Pydantic model | -| `default` | Default value when no configuration is found (can also be a function) | +| `default` | Default value when no configuration is found (can also be a function) | + +For variables with Handlebars template rendering, use `logfire.template_var()` instead, which adds an `inputs_type` parameter. See [Templates and Composition](templates-and-composition.md). diff --git a/docs/reference/advanced/managed-variables/templates-and-composition.md b/docs/reference/advanced/managed-variables/templates-and-composition.md new file mode 100644 index 000000000..b2b16084f --- /dev/null +++ b/docs/reference/advanced/managed-variables/templates-and-composition.md @@ -0,0 +1,263 @@ +# Template Variables and Composition + +Managed variables can contain **Handlebars templates** (`{{placeholder}}`) and **composition references** (`<>`), enabling dynamic values that are assembled from multiple sources and rendered with runtime inputs. + +This is especially useful for AI applications where prompts are built from reusable fragments and personalized with request-specific data. + +## Template Variables + +A **template variable** is a variable whose value contains `{{placeholder}}` expressions that are rendered with typed inputs at resolution time. Define one with `logfire.template_var()`: + +```python +from pydantic import BaseModel + +import logfire + +logfire.configure() + + +class PromptInputs(BaseModel): + user_name: str + is_premium: bool = False + + +prompt = logfire.template_var( + 'system_prompt', + type=str, + default='Hello {{user_name}}! Welcome to our service.', + inputs_type=PromptInputs, +) +``` + +When you call `.get()`, you pass an instance of the inputs type. The SDK renders all `{{placeholder}}` expressions in the resolved value before returning: + +```python skip="true" +with prompt.get(PromptInputs(user_name='Alice')) as resolved: + print(resolved.value) # "Hello Alice! Welcome to our service." +``` + +The full resolution pipeline is: + +1. **Resolve** — fetch the serialized value from the provider (or use the code default) +2. **Compose** — expand any `<>` references (see [Composition](#variable-composition) below) +3. **Render** — render `{{placeholder}}` Handlebars templates using the provided inputs +4. **Deserialize** — validate and deserialize to the variable's type + +### Template Variables Parameters + +`logfire.template_var()` accepts the same parameters as `logfire.var()` plus: + +| Parameter | Description | +|-----------|-------------| +| `inputs_type` | A Pydantic `BaseModel` (or any type supported by `TypeAdapter`) describing the expected template inputs. This is used for type-safe `.get(inputs)` calls and generates a `template_inputs_schema` for validation. | + +### Handlebars Syntax + +Template variables use [Handlebars](https://handlebarsjs.com/) syntax, powered by the [`pydantic-handlebars`](https://github.com/pydantic/pydantic-handlebars) library. The most common patterns: + +| Syntax | Description | +|--------|-------------| +| `{{field}}` | Insert a value | +| `{{obj.nested}}` | Dot-notation access | +| `{{#if field}}...{{/if}}` | Conditional block | +| `{{#unless field}}...{{/unless}}` | Inverse conditional | +| `{{#each items}}...{{/each}}` | Iterate over a list | +| `{{#with obj}}...{{/with}}` | Change context | +| `{{! comment }}` | Comment (not rendered) | + +**Example with conditionals:** + +```python skip="true" +prompt = logfire.template_var( + 'greeting', + type=str, + default='Hello {{user_name}}!{{#if is_premium}} Thank you for being a premium member.{{/if}}', + inputs_type=PromptInputs, +) + +with prompt.get(PromptInputs(user_name='Alice', is_premium=True)) as resolved: + print(resolved.value) + # "Hello Alice! Thank you for being a premium member." + +with prompt.get(PromptInputs(user_name='Bob', is_premium=False)) as resolved: + print(resolved.value) + # "Hello Bob!" +``` + +### Structured Template Variables + +Template variables work with structured types too. Only string fields containing `{{placeholders}}` are rendered — other fields pass through unchanged: + +```python +from pydantic import BaseModel + +import logfire + +logfire.configure() + + +class UserContext(BaseModel): + user_name: str + tier: str + + +class AgentConfig(BaseModel): + instructions: str + model: str + temperature: float + + +agent_config = logfire.template_var( + 'agent_config', + type=AgentConfig, + default=AgentConfig( + instructions='You are helping {{user_name}}, a {{tier}} customer.', + model='openai:gpt-4o-mini', + temperature=0.7, + ), + inputs_type=UserContext, +) +``` + +```python skip="true" +with agent_config.get(UserContext(user_name='Alice', tier='premium')) as resolved: + print(resolved.value.instructions) # "You are helping Alice, a premium customer." + print(resolved.value.model) # "openai:gpt-4o-mini" (unchanged) +``` + +### Ad-hoc Rendering with `resolved.render()` + +For regular variables (created with `logfire.var()`) that happen to contain template syntax, you can render them after resolution using `resolved.render()`: + +```python skip="true" +from pydantic import BaseModel + +import logfire + + +class Inputs(BaseModel): + user_name: str + + +prompt = logfire.var('prompt', type=str, default='Hello {{user_name}}') + +with prompt.get() as resolved: + rendered = resolved.render(Inputs(user_name='Alice')) + print(rendered) # "Hello Alice" +``` + +This is useful when you want the flexibility to render templates on some code paths but not others. + +### Template Validation + +When a template variable is pushed to Logfire (via `logfire.variables_push()`), the `template_inputs_schema` is synced alongside the variable's JSON schema. The system validates that all `{{field}}` references in variable values (including values reachable through composition) are compatible with the declared schema. + +For example, if your `inputs_type` declares `user_name: str` and `is_premium: bool`, but a version value references `{{unknown_field}}`, the validation will flag this as an error. + +## Variable Composition {#variable-composition} + +**Composition** lets a variable's value reference other variables using `<>` syntax. When the variable is resolved, `<>` references are expanded by looking up the referenced variable and substituting its value. + +This is useful for building values from reusable fragments: + +```python skip="true" +import logfire + +logfire.configure() + +# A reusable instruction fragment +safety_rules = logfire.var( + 'safety_rules', + type=str, + default='Never share personal data. Always be respectful.', +) + +# A prompt that includes the safety rules via composition +agent_prompt = logfire.var( + 'agent_prompt', + type=str, + default='You are a helpful assistant. <>', +) + +with agent_prompt.get() as resolved: + print(resolved.value) + # "You are a helpful assistant. Never share personal data. Always be respectful." +``` + +When `safety_rules` is updated in the Logfire UI, all variables that reference `<>` automatically pick up the new value — no code changes or redeployment required. + +### Composition with Handlebars Power + +The `<<>>` syntax supports the full Handlebars feature set — not just simple variable substitution. You can use conditionals, loops, and more: + +| Syntax | Description | +|--------|-------------| +| `<>` | Insert a variable's value | +| `<>` | Access a nested field | +| `<<#if variable>>...<<#else>>...<>` | Conditional on whether a variable is set | +| `<<#each items>>...<>` | Iterate over a list variable | + +### Composition Tracking + +Every `<>` expansion is recorded in the resolution result. You can inspect which variables were composed and their values: + +```python skip="true" +with agent_prompt.get() as resolved: + for ref in resolved.composed_from: + print(f" {ref.name}: version={ref.version}, label={ref.label}") +``` + +These composition details are also recorded as span attributes, so you can see the full composition chain in your Logfire traces. + +### Combining Templates and Composition + +Template variables and composition work together. A common pattern is to compose reusable fragments via `<>` and render runtime inputs via `{{}}`: + +```python skip="true" +from pydantic import BaseModel + +import logfire + +logfire.configure() + + +class ChatInputs(BaseModel): + user_name: str + language: str + + +# Reusable fragment (no template inputs) +tone_instructions = logfire.var( + 'tone_instructions', + type=str, + default='Be friendly and concise.', +) + +# Template variable that composes the fragment and renders inputs +chat_prompt = logfire.template_var( + 'chat_prompt', + type=str, + default='You are helping {{user_name}}. Respond in {{language}}. <>', + inputs_type=ChatInputs, +) + +# Resolution: compose <> first, then render {{user_name}} and {{language}} +with chat_prompt.get(ChatInputs(user_name='Alice', language='French')) as resolved: + print(resolved.value) + # "You are helping Alice. Respond in French. Be friendly and concise." +``` + +### Cycle Detection + +The system detects circular references at write time. If variable A references `<>` and variable B references `<>`, pushing this configuration will produce an error. This prevents infinite loops during resolution. + +## Requirements + +Template rendering and composition require the [`pydantic-handlebars`](https://github.com/pydantic/pydantic-handlebars) library, which is included in the `variables` extra: + +```bash +pip install 'logfire[variables]' +``` + +!!! note "Python 3.10+" + `pydantic-handlebars` requires Python 3.10 or later. On Python 3.9, basic variable features (`logfire.var()` without templates or composition) still work, but template rendering is not available. diff --git a/examples/python/variable_composition_demo.py b/examples/python/variable_composition_demo.py new file mode 100644 index 000000000..33967f933 --- /dev/null +++ b/examples/python/variable_composition_demo.py @@ -0,0 +1,613 @@ +"""Demo: Variable Composition & Template Rendering with Logfire Managed Variables. + +This script demonstrates the full power of Logfire's variable composition +(<> references) and Handlebars template rendering ({{field}}) +using a purely local configuration — no remote server needed. + +Key features shown: + 1. Basic variable composition: <> references expand inline + 2. Nested composition: variable A references B, which references C + 3. Subfield variable references: <> accesses a field of a structured variable + 4. Template rendering with {{field}} placeholders and Pydantic input models + 5. Accessing subfields of template inputs (e.g. {{user.name}}, {{user.email}}) + 6. Handlebars conditionals: {{#if}}, {{else}}, {{/if}} + 7. Handlebars iteration: {{#each items}}...{{/each}} + 8. TemplateVariable: single-step get(inputs) with automatic rendering + 9. Variable.get() + .render(inputs): two-step manual rendering + 10. Rollout overrides with attribute-based conditions + 11. Composition-time conditionals: <<#if flag>>...<>...<> +""" + +from __future__ import annotations + +import json + +from pydantic import BaseModel + +import logfire +from logfire._internal.config import LocalVariablesOptions +from logfire.variables.config import ( + LabeledValue, + Rollout, + RolloutOverride, + ValueEquals, + VariableConfig, + VariablesConfig, +) + +# --------------------------------------------------------------------------- +# 1. Define Pydantic models for structured data & template inputs +# --------------------------------------------------------------------------- + + +class UserProfile(BaseModel): + """Nested model used as a variable value (composed into other variables).""" + + name: str + email: str + tier: str = 'free' + + +class PromptInputs(BaseModel): + """Template inputs with nested subfields.""" + + user: UserProfile + topic: str + max_tokens: int = 500 + + +class NotificationInputs(BaseModel): + """Template inputs for notification templates.""" + + user: UserProfile + action: str + details: str = '' + + +class OnboardingInputs(BaseModel): + """Template inputs demonstrating conditionals and iteration.""" + + user: UserProfile + is_new_user: bool = True + features: list[str] = [] + + +# --------------------------------------------------------------------------- +# 2. Build a local VariablesConfig with several interconnected variables +# --------------------------------------------------------------------------- + +variables_config = VariablesConfig( + variables={ + # --- Leaf variables (no references to other variables) --- + 'app_name': VariableConfig( + name='app_name', + labels={ + 'production': LabeledValue(version=1, serialized_value=json.dumps('Acme Platform')), + 'staging': LabeledValue(version=1, serialized_value=json.dumps('Acme Platform [STAGING]')), + }, + rollout=Rollout(labels={'production': 1.0}), + overrides=[ + RolloutOverride( + conditions=[ValueEquals(attribute='environment', value='staging')], + rollout=Rollout(labels={'staging': 1.0}), + ), + ], + ), + 'safety_disclaimer': VariableConfig( + name='safety_disclaimer', + labels={ + 'production': LabeledValue( + version=1, + serialized_value=json.dumps( + 'Always verify critical information independently. This AI assistant may make mistakes.' + ), + ), + }, + rollout=Rollout(labels={'production': 1.0}), + overrides=[], + ), + 'support_email': VariableConfig( + name='support_email', + labels={ + 'production': LabeledValue(version=1, serialized_value=json.dumps('help@acme.com')), + }, + rollout=Rollout(labels={'production': 1.0}), + overrides=[], + ), + # --- Structured variable (JSON object) for subfield composition --- + # Other variables can reference subfields like <> + 'brand': VariableConfig( + name='brand', + labels={ + 'production': LabeledValue( + version=1, + serialized_value=json.dumps( + { + 'tagline': 'Build faster, ship smarter', + 'color': '#4F46E5', + 'support_url': 'https://acme.dev/support', + } + ), + ), + }, + rollout=Rollout(labels={'production': 1.0}), + overrides=[], + ), + # --- Composed variable: references <> --- + 'support_footer': VariableConfig( + name='support_footer', + labels={ + 'production': LabeledValue( + version=1, + serialized_value=json.dumps('Need help? Contact us at <>.'), + ), + }, + rollout=Rollout(labels={'production': 1.0}), + overrides=[], + ), + # --- Composed + templated variable --- + # References <>, <>, <> + # Also contains {{user.name}}, {{user.tier}}, {{topic}} template placeholders + 'system_prompt': VariableConfig( + name='system_prompt', + labels={ + 'production': LabeledValue( + version=1, + serialized_value=json.dumps( + 'You are a helpful assistant for <>.\n\n' + 'The user you are speaking with is {{user.name}} ({{user.tier}} tier).\n' + 'They want help with: {{topic}}\n\n' + 'Guidelines:\n' + '- Be concise and helpful\n' + '- <>\n\n' + '<>' + ), + ), + 'concise': LabeledValue( + version=1, + serialized_value=json.dumps( + '<> assistant. User: {{user.name}} ({{user.tier}}). ' + 'Topic: {{topic}}. Be brief. <>' + ), + ), + }, + rollout=Rollout(labels={'production': 1.0}), + overrides=[], + template_inputs_schema=PromptInputs.model_json_schema(), + ), + # --- Notification template: uses {{user.name}}, {{user.email}}, {{action}} --- + # Also demonstrates {{#if details}} conditional + 'notification_template': VariableConfig( + name='notification_template', + labels={ + 'production': LabeledValue( + version=1, + serialized_value=json.dumps( + 'Hi {{user.name}},\n\n' + 'Your action "{{action}}" has been completed on <>.\n' + '{{#if details}}Details: {{details}}\n{{/if}}' + '\nA confirmation has been sent to {{user.email}}.\n\n' + '<>' + ), + ), + }, + rollout=Rollout(labels={'production': 1.0}), + overrides=[], + template_inputs_schema=NotificationInputs.model_json_schema(), + ), + # --- Onboarding template: demonstrates #if/#else and #each --- + # Also uses <> subfield composition + 'onboarding_message': VariableConfig( + name='onboarding_message', + labels={ + 'production': LabeledValue( + version=1, + serialized_value=json.dumps( + '{{#if is_new_user}}' + 'Welcome to <>, {{user.name}}! ' + '<>.\n' + '{{else}}' + 'Welcome back to <>, {{user.name}}!\n' + '{{/if}}' + '\n' + '{{#if features}}' + 'Here are your enabled features:\n' + '{{#each features}}' + ' - {{this}}\n' + '{{/each}}' + '{{else}}' + 'No features enabled yet. Visit <> to get started.\n' + '{{/if}}' + '\nQuestions? Reach out to <>.' + ), + ), + }, + rollout=Rollout(labels={'production': 1.0}), + overrides=[], + template_inputs_schema=OnboardingInputs.model_json_schema(), + ), + # --- Structured variable (JSON object) with template fields --- + # Shows that templates work inside structured types, not just strings + 'welcome_config': VariableConfig( + name='welcome_config', + labels={ + 'production': LabeledValue( + version=1, + serialized_value=json.dumps( + { + 'greeting': 'Welcome to <>, {{user.name}}!', + 'subtitle': 'Your {{user.tier}} account is ready. <>.', + 'cta_text': 'Explore {{topic}}', + 'show_banner': True, + 'max_tokens': 500, + } + ), + ), + }, + rollout=Rollout(labels={'production': 1.0}), + overrides=[], + template_inputs_schema=PromptInputs.model_json_schema(), + ), + # --- Feature flag (boolean) for composition-time conditionals --- + 'beta_enabled': VariableConfig( + name='beta_enabled', + labels={ + 'enabled': LabeledValue(version=1, serialized_value='true'), + 'disabled': LabeledValue(version=1, serialized_value='false'), + }, + rollout=Rollout(labels={'enabled': 1.0}), + overrides=[], + ), + # --- Composed variable using <<#if>> at composition time --- + # The <<#if beta_enabled>> block is evaluated during composition, NOT at + # template-render time. This means the conditional is resolved when the + # variable value is expanded, controlled by the beta_enabled flag variable. + 'banner_message': VariableConfig( + name='banner_message', + labels={ + 'production': LabeledValue( + version=1, + serialized_value=json.dumps( + '<<#if beta_enabled>>' + 'Try our new beta features! <>.' + '<>' + 'Welcome to <>.' + '<>' + ), + ), + }, + rollout=Rollout(labels={'production': 1.0}), + overrides=[], + ), + } +) + +# --------------------------------------------------------------------------- +# 3. Configure Logfire with local variables (no remote server needed) +# --------------------------------------------------------------------------- + +logfire.configure( + send_to_logfire=False, + variables=LocalVariablesOptions( + config=variables_config, + instrument=False, # Keep output clean for the demo + ), +) + +# --------------------------------------------------------------------------- +# 4. Define variables in code +# --------------------------------------------------------------------------- + +app_name_var = logfire.var('app_name', type=str, default='MyApp') +safety_var = logfire.var('safety_disclaimer', type=str, default='Be careful.') +support_email_var = logfire.var('support_email', type=str, default='support@example.com') +support_footer_var = logfire.var('support_footer', type=str, default='Contact support.') +brand_var = logfire.var( + 'brand', + type=dict, + default={'tagline': 'Default tagline', 'color': '#000', 'support_url': 'https://example.com'}, +) + +# A Variable with template_inputs — uses two-step get() + render() +system_prompt_var = logfire.var( + 'system_prompt', + type=str, + default='Hello {{user.name}}, how can I help with {{topic}}?', + template_inputs=PromptInputs, +) + +# TemplateVariables — single-step get(inputs) with automatic rendering +notification_var = logfire.template_var( + 'notification_template', + type=str, + default='Hi {{user.name}}, your {{action}} is done.', + inputs_type=NotificationInputs, +) + +onboarding_var = logfire.template_var( + 'onboarding_message', + type=str, + default='Welcome, {{user.name}}!', + inputs_type=OnboardingInputs, +) + +# Structured (dict) variable with templates inside +welcome_var = logfire.template_var( + 'welcome_config', + type=dict, + default={'greeting': 'Welcome, {{user.name}}!', 'show_banner': True, 'max_tokens': 500}, + inputs_type=PromptInputs, +) + +# Composition-time conditional variable +banner_var = logfire.var('banner_message', type=str, default='Welcome.') + + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + + +def section(title: str) -> None: + """Print a section header.""" + print(f'\n{"=" * 70}') + print(f' {title}') + print(f'{"=" * 70}\n') + + +# --------------------------------------------------------------------------- +# 5. Demo: Basic composition (no templates) +# --------------------------------------------------------------------------- + +section('1. Basic Composition: <> references expand inline') + +result = support_footer_var.get() +print(f'support_footer resolved to:\n "{result.value}"\n') +print(f'Composed from {len(result.composed_from)} reference(s):') +for ref in result.composed_from: + print(f' - <<{ref.name}>> -> "{ref.value}" (label={ref.label}, v{ref.version})') + +# --------------------------------------------------------------------------- +# 6. Demo: Nested composition (A -> B -> C) +# --------------------------------------------------------------------------- + +section('2. Nested Composition: system_prompt -> support_footer -> support_email') + +# Get the raw (unrendered) system prompt to see composition in action +raw_result = system_prompt_var.get() +print('After composition (before template rendering):') +print(f' label={raw_result.label}, version={raw_result.version}') +print() + +# Show the composed value — <> are expanded but {{fields}} remain +composed_value = raw_result.value +# Since templates haven't been rendered yet, {{...}} placeholders are literal +print('Composed value ({{placeholders}} still present):') +for line in composed_value.split('\n'): + print(f' {line}') + +print(f'\nComposed from {len(raw_result.composed_from)} top-level reference(s):') +for ref in raw_result.composed_from: + print(f' - <<{ref.name}>> -> "{ref.value}"') + # Show nested references (e.g. support_footer -> support_email) + for nested in ref.composed_from: + print(f' -> <<{nested.name}>> -> "{nested.value}"') + +# --------------------------------------------------------------------------- +# 7. Demo: Subfield references to structured variables +# --------------------------------------------------------------------------- + +section('3. Subfield Variable References: <>, <>') + +print('The "brand" variable is a JSON object:') +brand_result = brand_var.get() +for key, value in brand_result.value.items(): + print(f' {key}: {value!r}') + +print() +print('Other variables can reference individual fields via <>.') +print('For example, the onboarding_message template contains:') +print(' <> -> expands to the tagline string') +print(' <> -> expands to the support URL string') + +# --------------------------------------------------------------------------- +# 8. Demo: Handlebars conditionals (#if / #else) +# --------------------------------------------------------------------------- + +section('4. Handlebars Conditionals: {{#if}}, {{else}}, {{/if}}') + +print('The onboarding_message template uses conditionals:') +print(' {{#if is_new_user}}...welcome...{{else}}...welcome back...{{/if}}') +print(' {{#if features}}...list them...{{else}}...no features yet...{{/if}}') +print() + +# New user WITH features +new_user_inputs = OnboardingInputs( + user=UserProfile(name='Alice', email='alice@example.com'), + is_new_user=True, + features=['Dashboard Analytics', 'API Access', 'Team Management'], +) + +result_new = onboarding_var.get(new_user_inputs) +print('--- New user with features ---') +for line in result_new.value.split('\n'): + print(f' {line}') + +print() + +# Returning user WITHOUT features +returning_user_inputs = OnboardingInputs( + user=UserProfile(name='Bob', email='bob@example.com', tier='premium'), + is_new_user=False, + features=[], +) + +result_returning = onboarding_var.get(returning_user_inputs) +print('--- Returning user, no features ---') +for line in result_returning.value.split('\n'): + print(f' {line}') + +print() +print('Composed references in the onboarding message:') +for ref in result_new.composed_from: + print(f' - <<{ref.name}>> -> "{ref.value}"') + +# --------------------------------------------------------------------------- +# 9. Demo: Template rendering with subfield access (user.name, user.tier) +# --------------------------------------------------------------------------- + +section('5. Template Rendering: Subfields of inputs ({{user.name}}, {{user.tier}})') + +user = UserProfile(name='Alice Johnson', email='alice@example.com', tier='premium') +inputs = PromptInputs(user=user, topic='billing questions') + +# Two-step: get() then render() +with system_prompt_var.get() as resolved: + rendered = resolved.render(inputs) + +print('Rendered system prompt:') +for line in rendered.split('\n'): + print(f' {line}') + +# --------------------------------------------------------------------------- +# 10. Demo: TemplateVariable single-step get(inputs) +# --------------------------------------------------------------------------- + +section('6. TemplateVariable: Single-step get(inputs) with auto-rendering') + +# {{#if details}} conditional: included because details is non-empty +notif_inputs = NotificationInputs( + user=UserProfile(name='Bob Smith', email='bob@corp.com', tier='enterprise'), + action='project deployment', + details='Deployed v2.3.1 to production', +) + +result = notification_var.get(notif_inputs) +print('--- With details ({{#if details}} is truthy) ---') +for line in result.value.split('\n'): + print(f' {line}') + +print() + +# {{#if details}} conditional: omitted because details is empty +notif_no_details = NotificationInputs( + user=UserProfile(name='Carol', email='carol@startup.io'), + action='password reset', +) + +result_no_details = notification_var.get(notif_no_details) +print('--- Without details ({{#if details}} is falsy) ---') +for line in result_no_details.value.split('\n'): + print(f' {line}') + +# --------------------------------------------------------------------------- +# 11. Demo: Structured variable with templates and subfield composition +# --------------------------------------------------------------------------- + +section('7. Structured Variable: Templates + <> in dict values') + +struct_inputs = PromptInputs( + user=UserProfile(name='Carol', email='carol@startup.io'), + topic='AI integrations', +) + +struct_result = welcome_var.get(struct_inputs) +print('Rendered welcome_config (dict with templates in string fields):') +for key, value in struct_result.value.items(): + print(f' {key}: {value!r}') + +print('\nNote: string values were rendered, non-strings (bool, int) pass through unchanged.') +print('The subtitle used <> to compose in the brand tagline.') + +# --------------------------------------------------------------------------- +# 12. Demo: Rollout overrides with attributes +# --------------------------------------------------------------------------- + +section('8. Rollout Overrides: Attribute-based label selection') + +prod_result = app_name_var.get() +print(f'Default (production): "{prod_result.value}" (label={prod_result.label})') + +staging_result = app_name_var.get(attributes={'environment': 'staging'}) +print(f'With env=staging: "{staging_result.value}" (label={staging_result.label})') + +# --------------------------------------------------------------------------- +# 13. Demo: Explicit label selection +# --------------------------------------------------------------------------- + +section('9. Explicit Label Selection: Choosing a specific label') + +verbose_result = system_prompt_var.get(label='production') +concise_result = system_prompt_var.get(label='concise') + +print('Production prompt (first 80 chars):') +print(f' "{verbose_result.value[:80]}..."') +print('\nConcise prompt:') +print(f' "{concise_result.value}"') + +# Now render the concise one with template inputs +rendered_concise = concise_result.render(inputs) +print('\nConcise prompt rendered:') +print(f' "{rendered_concise}"') + +# --------------------------------------------------------------------------- +# 14. Demo: Composition-time conditionals (<<#if>> at composition time) +# --------------------------------------------------------------------------- + +section('10. Composition-Time Conditionals: <<#if>> with feature flags') + +print('The banner_message variable uses <<#if beta_enabled>> at composition time.') +print('This conditional is resolved when <<>> references are expanded, NOT at') +print('template render time. The beta_enabled variable controls which branch appears.') +print() + +# beta_enabled is true by default (the "enabled" label has weight 1.0) +banner_result = banner_var.get() +print('With beta_enabled=true:') +print(f' "{banner_result.value}"') +print() + +# Show the composed references +print(f'Composed from {len(banner_result.composed_from)} reference(s):') +for ref in banner_result.composed_from: + print(f' - <<{ref.name}>> = {ref.value!r}') + +# --------------------------------------------------------------------------- +# 15. Demo: Using context manager for baggage propagation +# --------------------------------------------------------------------------- + +section('11. Context Manager: Baggage propagation for observability') + +with system_prompt_var.get() as resolved: + print('Inside context manager:') + print(f' Variable: {resolved.name}') + print(f' Label: {resolved.label}') + print(f' Version: {resolved.version}') + print(f' Baggage key: logfire.variables.{resolved.name}') + print(f' Baggage value: {resolved.label}') + print() + print(' Any spans created in this block will carry the variable') + print(' resolution as baggage, enabling downstream correlation.') + +# --------------------------------------------------------------------------- +# 16. Summary +# --------------------------------------------------------------------------- + +section('Summary') +print('This demo showed:') +print(' - <> composition: inline expansion of variable references') +print(' - Nested composition: A -> B -> C chains expand recursively') +print(' - <> subfield refs: access fields of structured variables') +print(' - {{field}} templates with Handlebars syntax') +print(' - Subfield access in templates: {{user.name}}, {{user.email}}, {{user.tier}}') +print(' - {{#if cond}}...{{else}}...{{/if}} conditionals (template-time)') +print(' - {{#each list}}...{{/each}} iteration (template-time)') +print(' - <<#if flag>>...<>...<> conditionals (composition-time)') +print(' - Structured variables: templates render inside dict string values') +print(' - TemplateVariable: single-step get(inputs) with auto-rendering') +print(' - Variable + render(): two-step manual rendering') +print(' - Rollout overrides: attribute-based label selection') +print(' - Explicit label selection: get(label="concise")') +print(' - Context manager: baggage propagation for observability') +print() +print('All using LocalVariablesOptions — no remote server required!') diff --git a/logfire-api/logfire_api/_internal/main.pyi b/logfire-api/logfire_api/_internal/main.pyi index 740a0aac5..c3345f4e5 100644 --- a/logfire-api/logfire_api/_internal/main.pyi +++ b/logfire-api/logfire_api/_internal/main.pyi @@ -1177,9 +1177,9 @@ class Logfire: `False` if the timeout was reached before the shutdown was completed, `True` otherwise. """ @overload - def var(self, name: str, *, default: T, description: str | None = None) -> Variable[T]: ... + def var(self, name: str, *, default: T, description: str | None = None, template_inputs: type[Any] | None = None) -> Variable[T]: ... @overload - def var(self, name: str, *, type: type[T], default: T | ResolveFunction[T], description: str | None = None) -> Variable[T]: ... + def var(self, name: str, *, type: type[T], default: T | ResolveFunction[T], description: str | None = None, template_inputs: type[Any] | None = None) -> Variable[T]: ... def variables_clear(self) -> None: """Clear all registered variables from this Logfire instance. diff --git a/logfire/__init__.py b/logfire/__init__.py index 8c0d01e02..bd6f644d0 100644 --- a/logfire/__init__.py +++ b/logfire/__init__.py @@ -97,6 +97,7 @@ # Variables var = DEFAULT_LOGFIRE_INSTANCE.var +template_var = DEFAULT_LOGFIRE_INSTANCE.template_var variables_clear = DEFAULT_LOGFIRE_INSTANCE.variables_clear variables_get = DEFAULT_LOGFIRE_INSTANCE.variables_get variables_push = DEFAULT_LOGFIRE_INSTANCE.variables_push @@ -196,6 +197,7 @@ def loguru_handler() -> Any: 'LocalVariablesOptions', 'variables', 'var', + 'template_var', 'variables_clear', 'variables_get', 'variables_push', diff --git a/logfire/_internal/integrations/pytest.py b/logfire/_internal/integrations/pytest.py index 05916d398..1afaad1bb 100644 --- a/logfire/_internal/integrations/pytest.py +++ b/logfire/_internal/integrations/pytest.py @@ -455,7 +455,12 @@ def pytest_runtest_makereport( # Record exception details if call.excinfo and call.excinfo.value: # pragma: no branch # Branch coverage: excinfo.value is always present for failed tests in normal pytest execution - span.record_exception(call.excinfo.value) + try: + span.record_exception(call.excinfo.value) + except RuntimeError: + # CPython 3.11+ can raise "generator raised StopIteration" from + # traceback.extract_tb when processing certain bytecode positions. + pass elif report.skipped: # pragma: no cover # TODO: this needs improvement in processing skip reasons skip_reason = '' diff --git a/logfire/_internal/main.py b/logfire/_internal/main.py index 60d4c0e29..1130ad6b2 100644 --- a/logfire/_internal/main.py +++ b/logfire/_internal/main.py @@ -115,6 +115,7 @@ from ..integrations.wsgi import RequestHook as WSGIRequestHook, ResponseHook as WSGIResponseHook from ..variables import ( ResolveFunction, + TemplateVariable, ValidationReport, Variable, VariablesConfig, @@ -156,7 +157,7 @@ def __init__( self._sample_rate = sample_rate self._console_log = console_log self._otel_scope = otel_scope - self._variables: dict[str, Variable[Any]] = {} + self._variables: dict[str, Variable[Any] | TemplateVariable[Any, Any]] = {} @property def config(self) -> LogfireConfig: @@ -2455,6 +2456,7 @@ def var( *, default: T, description: str | None = None, + template_inputs: type[Any] | None = None, ) -> Variable[T]: ... @overload @@ -2465,6 +2467,7 @@ def var( type: type[T], default: T | ResolveFunction[T], description: str | None = None, + template_inputs: type[Any] | None = None, ) -> Variable[T]: ... def var( @@ -2474,6 +2477,7 @@ def var( type: type[T] | None = None, default: T | ResolveFunction[T], description: str | None = None, + template_inputs: type[Any] | None = None, ) -> Variable[T]: """Define a managed variable. @@ -2498,6 +2502,28 @@ def var( ... ``` + Template rendering example: + + ```py + from pydantic import BaseModel + + + class PromptInputs(BaseModel): + user_name: str + is_premium: bool = False + + + prompt = logfire.var( + 'system_prompt', + type=str, + default='Hello {{user_name}}', + template_inputs=PromptInputs, + ) + + with prompt.get() as resolved: + rendered = resolved.render(PromptInputs(user_name='Alice')) + ``` + Args: name: Unique identifier for the variable. Must match the name configured in the Logfire UI when using remote variables. @@ -2509,6 +2535,9 @@ def var( Can also be a callable with `targeting_key` and `attributes` parameters (requires `type` to be set explicitly). description: Optional human-readable description of what the variable controls. + template_inputs: Optional Pydantic model type describing the expected template inputs + for Handlebars ``{{placeholder}}`` rendering. When set, the JSON Schema of this + model is pushed to the server and used by the UI for autocomplete and preview. """ from logfire.variables.variable import Variable, is_resolve_function @@ -2536,7 +2565,90 @@ def var( f"A variable with name '{name}' has already been registered. Each variable must have a unique name." ) - variable = Variable[T](name, default=default, type=tp, logfire_instance=self, description=description) + variable = Variable[T]( + name, + default=default, + type=tp, + logfire_instance=self, + description=description, + template_inputs=template_inputs, + ) + self._variables[name] = variable + + return variable + + def template_var( + self, + name: str, + *, + type: type[T], + default: T | ResolveFunction[T], + inputs_type: type[Any], + description: str | None = None, + ) -> TemplateVariable[T, Any]: + """Define a managed template variable with integrated rendering. + + Like ``var()``, but ``get(inputs)`` automatically renders Handlebars ``{{placeholder}}`` + templates in the resolved value before returning. The pipeline is: + resolve → compose ``<>`` → render ``{{}}`` → deserialize. + + ```py + from pydantic import BaseModel + + import logfire + + logfire.configure() + + + class PromptInputs(BaseModel): + user_name: str + is_premium: bool = False + + + prompt = logfire.template_var( + 'system_prompt', + type=str, + default='Hello {{user_name}}', + inputs_type=PromptInputs, + ) + + with prompt.get(PromptInputs(user_name='Alice')) as resolved: + print(resolved.value) # "Hello Alice" + ``` + + Args: + name: Unique identifier for the variable. + type: Expected type for validation and JSON schema generation. + default: Default value used when no remote configuration is found. + Can also be a callable with ``targeting_key`` and ``attributes`` parameters. + inputs_type: The type (typically a Pydantic ``BaseModel``) describing the expected + template inputs. Used for type-safe ``get(inputs)`` calls and JSON schema generation. + description: Optional human-readable description of what the variable controls. + """ + import re + + from logfire.variables.variable import TemplateVariable + + if not re.match(r'^[a-zA-Z_][a-zA-Z0-9_]*$', name): + raise ValueError( + f"Invalid variable name '{name}'. " + 'Variable names must be valid Python identifiers (letters, digits, and underscores, ' + 'not starting with a digit).' + ) + + if name in self._variables: + raise ValueError( + f"A variable with name '{name}' has already been registered. Each variable must have a unique name." + ) + + variable = TemplateVariable[T, Any]( + name, + type=type, + default=default, + inputs_type=inputs_type, + description=description, + logfire_instance=self, + ) self._variables[name] = variable return variable @@ -2544,19 +2656,20 @@ def var( def variables_clear(self) -> None: """Clear all registered variables from this Logfire instance. - This removes all variables previously registered via [`var()`][logfire.Logfire.var], + This removes all variables previously registered via [`var()`][logfire.Logfire.var] + or [`template_var()`][logfire.Logfire.template_var], allowing them to be re-registered. This is primarily intended for use in tests to ensure a clean state between test cases. """ self._variables.clear() - def variables_get(self) -> list[Variable[Any]]: + def variables_get(self) -> list[Variable[Any] | TemplateVariable[Any, Any]]: """Get all variables registered with this Logfire instance.""" return list(self._variables.values()) def variables_push( self, - variables: list[Variable[Any]] | None = None, + variables: list[Variable[Any] | TemplateVariable[Any, Any]] | None = None, *, dry_run: bool = False, yes: bool = False, @@ -2672,7 +2785,7 @@ class UserSettings(BaseModel): def variables_validate( self, - variables: list[Variable[Any]] | None = None, + variables: list[Variable[Any] | TemplateVariable[Any, Any]] | None = None, ) -> ValidationReport: """Validate that provider-side variable label values match local type definitions. @@ -2774,7 +2887,7 @@ def variables_pull_config(self) -> VariablesConfig: # pragma: no cover def variables_build_config( self, - variables: list[Variable[Any]] | None = None, + variables: list[Variable[Any] | TemplateVariable[Any, Any]] | None = None, ) -> VariablesConfig: """Build a VariablesConfig from registered Variable instances. diff --git a/logfire/variables/__init__.py b/logfire/variables/__init__.py index 74a3534e9..509605d35 100644 --- a/logfire/variables/__init__.py +++ b/logfire/variables/__init__.py @@ -13,6 +13,11 @@ VariableNotFoundError, VariableWriteError, ) +from logfire.variables.composition import ( + ComposedReference, + VariableCompositionCycleError, + VariableCompositionError, +) if TYPE_CHECKING: # We use a TYPE_CHECKING block here because we need to do these imports lazily to prevent issues due to loading the @@ -39,6 +44,7 @@ ) from logfire.variables.variable import ( ResolveFunction, + TemplateVariable, Variable, targeting_context, ) @@ -46,6 +52,7 @@ __all__ = [ # Variable classes 'Variable', + 'TemplateVariable', 'ResolvedVariable', 'ResolveFunction', # Configuration classes @@ -112,6 +119,7 @@ def __getattr__(name: str): ) from logfire.variables.variable import ( ResolveFunction, + TemplateVariable, Variable, targeting_context, ) diff --git a/logfire/variables/abstract.py b/logfire/variables/abstract.py index 12d7d9099..1be4c49b0 100644 --- a/logfire/variables/abstract.py +++ b/logfire/variables/abstract.py @@ -6,8 +6,8 @@ from abc import ABC, abstractmethod from collections.abc import Mapping, Sequence from contextlib import ExitStack -from dataclasses import dataclass -from typing import TYPE_CHECKING, Any, Generic, Literal, TypeVar +from dataclasses import dataclass, field +from typing import TYPE_CHECKING, Any, Callable, Generic, Literal, TypeVar SyncMode = Literal['merge', 'replace'] @@ -15,8 +15,9 @@ from pydantic import TypeAdapter import logfire + from logfire.variables.composition import ComposedReference from logfire.variables.config import VariableConfig, VariablesConfig, VariableTypeConfig - from logfire.variables.variable import Variable + from logfire.variables.variable import _BaseVariable # pyright: ignore[reportPrivateUsage] # ANSI color codes for terminal output ANSI_RESET = '\033[0m' @@ -37,6 +38,7 @@ 'VariableWriteError', 'VariableNotFoundError', 'VariableAlreadyExistsError', + 'render_serialized_string', ) T = TypeVar('T') @@ -112,6 +114,23 @@ class ResolvedVariable(Generic[T_co]): """The version number of the resolved value, if any.""" exception: Exception | None = None """Any exception that occurred during resolution.""" + composed_from: list[ComposedReference] = field(default_factory=list) # pyright: ignore[reportUnknownVariableType] + """Variables that were composed into this value via <> expansion. + + Each entry is a ComposedReference for a referenced variable, including + its label, version, reason, and any nested composed_from entries. + """ + _serialized_value: str | None = None + """Internal: the post-composition, pre-deserialization JSON string. + + Used by render() to apply Handlebars template rendering on the serialized + form before deserializing to the variable's type. + """ + _deserializer: Callable[[str], Any] | None = None + """Internal: function to deserialize a JSON string to the variable's type. + + Returns the deserialized value or an Exception on failure. + """ def __post_init__(self): self._exit_stack = ExitStack() @@ -130,6 +149,147 @@ def __enter__(self): def __exit__(self, exc_type: type[BaseException] | None, exc_val: BaseException | None, exc_tb: Any) -> None: self._exit_stack.__exit__(exc_type, exc_val, exc_tb) + def render(self, inputs: Any = None) -> T_co: + """Render Handlebars templates in this variable's value. + + Operates on the serialized JSON (post-composition), renders all ``{{placeholder}}`` + expressions using the provided inputs, then deserializes to the variable's type. + + For ``str`` variables, this renders the template and returns a string. + For structured variables (e.g., Pydantic models), all string values containing + ``{{placeholders}}`` are rendered while non-string fields pass through unchanged. + + Args: + inputs: Template context values. Can be a Pydantic ``BaseModel`` (uses ``model_dump()``), + a ``dict``, or any ``Mapping``. If ``None``, renders with an empty context. + + Returns: + The rendered value, typed as the variable's type ``T_co``. + + Raises: + ValueError: If no serialized value is available for rendering. + ImportError: If ``pydantic-handlebars`` is not installed. + + Example: + ```python skip="true" + from pydantic import BaseModel + + + class Inputs(BaseModel): + user_name: str + + + prompt = logfire.var('prompt', type=str, default='Hello {{user_name}}') + with prompt.get() as resolved: + rendered = resolved.render(Inputs(user_name='Alice')) + # rendered == "Hello Alice" + ``` + """ + if self._serialized_value is None: + raise ValueError( + 'Cannot render template: no serialized value available. ' + 'This can happen if the variable resolved to a context override ' + 'or if serialization of the default value failed.' + ) + if self._deserializer is None: + raise ValueError('Cannot render template: no deserializer available.') + + rendered_json = render_serialized_string(self._serialized_value, inputs) + + # Deserialize the rendered JSON + result = self._deserializer(rendered_json) + if isinstance(result, Exception): + raise result + return result + + +def _inputs_to_context(inputs: Any) -> dict[str, Any]: + """Convert inputs (Pydantic model, dict, or Mapping) to a template context dict. + + Args: + inputs: Template context values. Can be a Pydantic ``BaseModel`` (uses ``model_dump()``), + a ``dict``, or any ``Mapping``. If ``None``, returns an empty dict. + + Returns: + A dict suitable for use as a Handlebars template context. + + Raises: + TypeError: If inputs is not a supported type. + """ + if inputs is None: + return {} + elif hasattr(inputs, 'model_dump'): + return inputs.model_dump() + elif isinstance(inputs, Mapping): + return dict(inputs) # pyright: ignore[reportUnknownArgumentType] + else: + raise TypeError(f'Expected a dict, Mapping, or Pydantic model for render inputs, got {type(inputs).__name__}') + + +def render_serialized_string(serialized_json: str, inputs: Any) -> str: + """Render Handlebars templates in a serialized JSON string. + + Decodes the JSON, renders all string values containing ``{{placeholders}}`` + using the provided inputs, then re-encodes to JSON. + + Args: + serialized_json: A JSON-encoded string potentially containing Handlebars templates. + inputs: Template context values. Can be a Pydantic ``BaseModel``, ``dict``, + ``Mapping``, or ``None``. + + Returns: + The rendered JSON string. + """ + from pydantic_handlebars import SafeString, render as hbs_render + + safe_string_cls: type[str] = SafeString + render_fn: Callable[..., str] = hbs_render + + context = _inputs_to_context(inputs) + + # Wrap all string values in SafeString to disable HTML escaping. + # For prompt/config templates (not HTML), escaping is undesirable. + context = _wrap_safe_context(context, safe_string_cls) + + # Decode the serialized JSON, render string values, then re-encode. + # We can't render the raw JSON directly because substituted values + # might contain JSON-special characters (e.g., double quotes) that + # would make the resulting JSON invalid. + decoded = json.loads(serialized_json) + rendered_value = _render_json_value(decoded, render_fn, context) + return json.dumps(rendered_value) + + +def _wrap_safe_context(context: dict[str, Any], safe_string_cls: type[str]) -> dict[str, Any]: + """Recursively wrap all string values in SafeString to disable HTML escaping.""" + return {k: _wrap_safe_value(v, safe_string_cls) for k, v in context.items()} + + +def _wrap_safe_value(value: Any, safe_string_cls: type[str]) -> Any: + """Wrap a single value: strings become SafeString, dicts/lists are recursed.""" + if isinstance(value, str): + return safe_string_cls(value) + if isinstance(value, dict): + return _wrap_safe_context(value, safe_string_cls) # pyright: ignore[reportUnknownArgumentType] + if isinstance(value, list): + return [_wrap_safe_value(item, safe_string_cls) for item in value] # pyright: ignore[reportUnknownVariableType] + return value + + +def _render_json_value(value: Any, hbs_render: Callable[..., str], context: dict[str, Any]) -> Any: + """Recursively render Handlebars templates in a decoded JSON value. + + Only string values are rendered; dicts and lists are walked recursively. + """ + if isinstance(value, str): + return hbs_render(value, context) + if isinstance(value, dict): + return {k: _render_json_value(v, hbs_render, context) for k, v in value.items()} # pyright: ignore[reportUnknownVariableType] + if isinstance(value, list): + return [_render_json_value(item, hbs_render, context) for item in value] # pyright: ignore[reportUnknownVariableType] + # Numbers, booleans, None pass through unchanged + return value + # --- Dataclasses for push/validate operations --- @@ -158,6 +318,7 @@ class VariableChange: local_description: str | None = None server_description: str | None = None description_differs: bool = False # True if descriptions differ (for warning) + template_inputs_schema: dict[str, Any] | None = None # JSON Schema for template inputs @dataclass @@ -166,6 +327,8 @@ class VariableDiff: changes: list[VariableChange] orphaned_server_variables: list[str] # Variables on server not in local code + reference_warnings: list[str] = field(default_factory=list) # pyright: ignore[reportUnknownVariableType] + """Warnings about variable references (non-existent refs, cycles, etc.).""" @property def has_changes(self) -> bool: @@ -216,6 +379,8 @@ class ValidationReport: """Names of variables that exist locally but not on the server.""" description_differences: list[DescriptionDifference] """List of variables where local and server descriptions differ.""" + reference_warnings: list[str] = field(default_factory=list) # pyright: ignore[reportUnknownVariableType] + """Warnings about variable references (non-existent refs, cycles, etc.).""" @property def has_errors(self) -> bool: @@ -279,6 +444,12 @@ def format(self, *, colors: bool = True) -> str: lines.append(f' Local: {local_desc}') lines.append(f' Server: {server_desc}') + # Show reference warnings + if self.reference_warnings: + lines.append(f'\n{yellow}=== Reference warnings ==={reset}') + for warning in self.reference_warnings: + lines.append(f' {yellow}⚠ {warning}{reset}') + # Summary line if not self.is_valid: error_count = variables_with_errors + len(self.variables_not_on_server) @@ -292,12 +463,12 @@ def format(self, *, colors: bool = True) -> str: # --- Helper functions for push/validate operations --- -def _get_json_schema(variable: Variable[object]) -> dict[str, Any]: +def _get_json_schema(variable: _BaseVariable[object]) -> dict[str, Any]: """Get the JSON schema for a variable's type.""" return variable.type_adapter.json_schema() -def _get_default_serialized(variable: Variable[object]) -> str | None: +def _get_default_serialized(variable: _BaseVariable[object]) -> str | None: """Get the serialized default value for a variable. Returns None if the default is a ResolveFunction (can't serialize a function). @@ -311,7 +482,7 @@ def _get_default_serialized(variable: Variable[object]) -> str | None: def _check_label_compatibility( - variable: Variable[object], + variable: _BaseVariable[object], label: str, serialized_value: str, ) -> LabelCompatibility: @@ -335,7 +506,7 @@ def _check_label_compatibility( def _check_all_label_compatibility( - variable: Variable[object], + variable: _BaseVariable[object], server_var: VariableConfig, ) -> list[LabelCompatibility]: """Check all labeled values and latest_version against the variable's Python type. @@ -411,8 +582,92 @@ def _check_type_label_compatibility( return incompatible +def _check_reference_warnings( + variables: Sequence[_BaseVariable[object]], + server_config: VariablesConfig, +) -> list[str]: + """Check for reference warnings: non-existent refs and cycles. + + Scans local variable defaults and server label values for <> + and validates that referenced variables exist and there are no cycles. + """ + from logfire.variables.composition import find_references + from logfire.variables.config import LabeledValue + from logfire.variables.variable import is_resolve_function + + warnings_list: list[str] = [] + + # Collect all known variable names (local + server) + all_names: set[str] = {v.name for v in variables} | set(server_config.variables.keys()) + + # Build a reference graph: variable_name -> set of referenced names + ref_graph: dict[str, set[str]] = {} + + # Scan local variable defaults for references + for variable in variables: + refs: set[str] = set() + if not is_resolve_function(variable.default): + try: + serialized_default = variable.type_adapter.dump_json(variable.default).decode('utf-8') + refs.update(find_references(serialized_default)) + except Exception: + pass + + # Also scan server label values for this variable + server_var = server_config.variables.get(variable.name) + if server_var is not None: + for _, labeled_value in server_var.labels.items(): + if isinstance(labeled_value, LabeledValue): + refs.update(find_references(labeled_value.serialized_value)) + if server_var.latest_version is not None: + refs.update(find_references(server_var.latest_version.serialized_value)) + + if refs: + ref_graph[variable.name] = refs + + # Check for non-existent references + for ref_name in refs: + if ref_name not in all_names: + warnings_list.append(f"Variable '{variable.name}' references '<<{ref_name}>>' which does not exist.") + + # Check for cycles using DFS + def _detect_cycles(graph: dict[str, set[str]]) -> list[list[str]]: + cycles: list[list[str]] = [] + visited: set[str] = set() + in_stack: set[str] = set() + path: list[str] = [] + + def dfs(node: str) -> None: + if node in in_stack: + # Found a cycle - extract it + cycle_start = path.index(node) + cycles.append(path[cycle_start:] + [node]) + return + if node in visited: + return + visited.add(node) + in_stack.add(node) + path.append(node) + for neighbor in graph.get(node, set()): + dfs(neighbor) + path.pop() + in_stack.remove(node) + + for node in graph: + if node not in visited: + dfs(node) + return cycles + + cycles = _detect_cycles(ref_graph) + for cycle in cycles: + cycle_str = ' -> '.join(cycle) + warnings_list.append(f'Reference cycle detected: {cycle_str}') + + return warnings_list + + def _compute_diff( - variables: Sequence[Variable[object]], + variables: Sequence[_BaseVariable[object]], server_config: VariablesConfig, ) -> VariableDiff: """Compute the diff between local variables and server config. @@ -432,6 +687,9 @@ def _compute_diff( local_description = variable.description server_var = server_config.variables.get(variable.name) + # Get template_inputs_schema if available + template_inputs_schema = variable.get_template_inputs_schema() + if server_var is None: # New variable - needs to be created default_serialized = _get_default_serialized(variable) @@ -442,6 +700,7 @@ def _compute_diff( local_schema=local_schema, initial_value=default_serialized, local_description=local_description, + template_inputs_schema=template_inputs_schema, ) ) else: @@ -495,7 +754,10 @@ def _compute_diff( # Find orphaned server variables (on server but not in local code) orphaned = [name for name in server_config.variables.keys() if name not in local_names] - return VariableDiff(changes=changes, orphaned_server_variables=orphaned) + # Check for reference warnings (non-existent refs, cycles) + reference_warnings = _check_reference_warnings(variables, server_config) + + return VariableDiff(changes=changes, orphaned_server_variables=orphaned, reference_warnings=reference_warnings) def _format_diff(diff: VariableDiff) -> str: @@ -558,6 +820,12 @@ def _format_diff(diff: VariableDiff) -> str: lines.append(f' Local: {local_desc}') lines.append(f' Server: {server_desc}') + # Show reference warnings + if diff.reference_warnings: + lines.append(f'\n{ANSI_YELLOW}=== Reference warnings ==={ANSI_RESET}') + for warning in diff.reference_warnings: + lines.append(f' {ANSI_YELLOW}⚠ {warning}{ANSI_RESET}') + return '\n'.join(lines) @@ -591,6 +859,7 @@ def _create_variable( overrides=[], json_schema=change.local_schema, example=change.initial_value, # Store the code default as an example for the UI + template_inputs_schema=change.template_inputs_schema, ) provider.create_variable(config) @@ -981,7 +1250,7 @@ def pull_config(self) -> VariablesConfig: # pragma: no cover def push_variables( self, - variables: Sequence[Variable[object]], + variables: Sequence[_BaseVariable[object]], *, dry_run: bool = False, yes: bool = False, @@ -1077,7 +1346,7 @@ def push_variables( def validate_variables( self, - variables: Sequence[Variable[object]], + variables: Sequence[_BaseVariable[object]], ) -> ValidationReport: """Validate that provider-side variable label values match local type definitions. @@ -1152,11 +1421,15 @@ def validate_variables( ) ) + # Check for reference warnings + reference_warnings = _check_reference_warnings(variables, server_config) + return ValidationReport( errors=errors, variables_checked=len(variables), variables_not_on_server=variables_not_on_server, description_differences=description_differences, + reference_warnings=reference_warnings, ) # --- Variable Types API --- @@ -1428,7 +1701,7 @@ def get_variable_config(self, name: str) -> VariableConfig | None: def push_variables( self, - variables: Sequence[Variable[Any]], + variables: Sequence[_BaseVariable[Any]], *, dry_run: bool = False, yes: bool = False, @@ -1444,7 +1717,7 @@ def push_variables( def validate_variables( self, - variables: Sequence[Variable[Any]], + variables: Sequence[_BaseVariable[Any]], ) -> ValidationReport: """No-op implementation that returns an empty validation report. diff --git a/logfire/variables/angle_bracket.py b/logfire/variables/angle_bracket.py new file mode 100644 index 000000000..82c25298b --- /dev/null +++ b/logfire/variables/angle_bracket.py @@ -0,0 +1,69 @@ +"""Angle-bracket Handlebars: low-level swap primitives for <<>> rendering. + +This module provides ``render_once`` which performs a single-pass render using +``<<>>`` as the delimiter instead of ``{{}}``. It is the engine behind variable +composition — it gives ``<<>>`` syntax the full power of Handlebars +(conditionals, loops, helpers, etc.) while preserving any ``{{}}`` runtime +placeholders untouched. + +Algorithm (swap + protect): + a. Protect ``{}<>`` characters in context values with HTML entities + b. Swap ``{↔<`` and ``}↔>`` in the template (so ``<<>>`` becomes ``{{}}``) + c. Run standard Handlebars + d. Reverse swap + e. Unescape the entities we introduced +""" + +from __future__ import annotations + +from typing import Any + +from pydantic_handlebars import SafeString, render as hbs_render + +# --------------------------------------------------------------------------- +# Character swap table: { ↔ < and } ↔ > +# --------------------------------------------------------------------------- +_SWAP = str.maketrans('{}<>', '<>{}') + +# --------------------------------------------------------------------------- +# Protection: escape {}<> in values to numeric entities that contain +# NO {}<> characters, so the reverse swap can't corrupt them. +# --------------------------------------------------------------------------- +_PROTECT = str.maketrans( + { + '{': '{', + '}': '}', + '<': '<', + '>': '>', + } +) + + +def _unescape_protected(s: str) -> str: + """Undo only the four entities we introduced.""" + return s.replace('{', '{').replace('}', '}').replace('<', '<').replace('>', '>') + + +def _protect_value(value: Any) -> Any: + """Recursively protect string values, preserving structure for dicts/lists.""" + if isinstance(value, str): + return SafeString(value.translate(_PROTECT)) + if isinstance(value, dict): + return {k: _protect_value(v) for k, v in value.items()} # pyright: ignore[reportUnknownVariableType] + if isinstance(value, list): + return [_protect_value(v) for v in value] # pyright: ignore[reportUnknownVariableType] + return value # bools, ints, None, etc. — pass through + + +# --------------------------------------------------------------------------- +# Core single-pass render: swap → Handlebars → unswap → unescape +# --------------------------------------------------------------------------- + + +def render_once(template: str, context: dict[str, Any]) -> str: + """Single-pass render: swap <<>>↔{{}}, run Handlebars, reverse swap, unescape.""" + swapped_template = template.translate(_SWAP) + safe_context = {k: _protect_value(v) for k, v in context.items()} + result: str = hbs_render(swapped_template, safe_context) + result = result.translate(_SWAP) + return _unescape_protected(result) diff --git a/logfire/variables/composition.py b/logfire/variables/composition.py new file mode 100644 index 000000000..1dedaa8f7 --- /dev/null +++ b/logfire/variables/composition.py @@ -0,0 +1,336 @@ +"""Variable composition: expand <> references in serialized values. + +This module provides pure functions for expanding variable references in serialized +JSON strings. References use the ``<>`` syntax and are expanded using +the Handlebars engine via character-swap, giving ``<<>>`` the full power of +Handlebars: ``<<#if>>``, ``<<#each>>``, ``<<#unless>>``, ``<<#with>>``, etc. + +Meanwhile, any ``{{runtime}}`` placeholders are preserved untouched for later +template rendering. + +The composition logic is shared between the SDK (client-side expansion) and the +backend OFREP endpoint (server-side expansion). +""" + +from __future__ import annotations + +import json +import re +from dataclasses import dataclass, field +from typing import Any, Callable, Optional, Tuple # noqa: UP035 + +from logfire.variables.angle_bracket import render_once + +__all__ = ( + 'MAX_COMPOSITION_DEPTH', + 'VariableCompositionError', + 'VariableCompositionCycleError', + 'ComposedReference', + 'expand_references', + 'find_references', + 'has_references', +) + +# Matches unescaped << (not preceded by \). +# In JSON-serialized strings, a real backslash is \\, so \\<< is an escaped ref. +_HAS_ANGLE = re.compile(r'(?> or <> +_SIMPLE_REF = re.compile(r'(?>') + +# Block helper references: <<#helper identifier ...>> — extracts the first identifier after the helper name. +_BLOCK_REF = re.compile(r'(?>)') + +# Handlebars keywords that should never be treated as variable references. +# These are valid in <> syntax but are Handlebars built-ins. +_HBS_KEYWORDS = frozenset({'else', 'this'}) + +MAX_COMPOSITION_DEPTH = 20 + + +class VariableCompositionError(Exception): + """Error during variable composition (reference expansion).""" + + +class VariableCompositionCycleError(VariableCompositionError): + """Circular reference detected during variable composition.""" + + +@dataclass +class ComposedReference: + """Metadata about a single <> that was encountered during expansion. + + This is a lightweight dataclass used to track composition results without + depending on ResolvedVariable, making it reusable from both the SDK and backend. + """ + + name: str + """Name of the referenced variable.""" + value: str | None + """Expanded raw string value, or None if unresolved.""" + label: str | None + """Label of the referenced variable's resolution.""" + version: int | None + """Version of the referenced variable's resolution.""" + reason: str + """Resolution reason (e.g., 'resolved', 'unrecognized_variable').""" + error: str | None = None + """Error message if the reference could not be expanded.""" + composed_from: list[ComposedReference] = field(default_factory=list) # pyright: ignore[reportUnknownVariableType] + """Nested references that were expanded within this reference.""" + + +# resolve_fn signature: (ref_name) -> (serialized_value, label, version, reason) +ResolveFn = Callable[[str], Tuple[Optional[str], Optional[str], Optional[int], str]] # noqa: UP006 + + +def has_references(serialized_value: str) -> bool: + """Quick check for any unescaped ``<<`` in a serialized value.""" + return _HAS_ANGLE.search(serialized_value) is not None + + +def expand_references( + serialized_value: str, + variable_name: str, + resolve_fn: ResolveFn, + *, + _visited: frozenset[str] = frozenset(), + _depth: int = 0, +) -> tuple[str, list[ComposedReference]]: + """Expand <> references in a serialized variable value. + + Uses the Handlebars engine via character-swap so that ``<<>>`` supports the + full Handlebars feature set (``<<#if>>``, ``<<#each>>``, etc.) while + preserving ``{{runtime}}`` placeholders untouched. + + Args: + serialized_value: The raw JSON-serialized variable value. + variable_name: Name of the variable being expanded (for cycle detection). + resolve_fn: Function that resolves a variable name to + (serialized_value, label, version, reason). + _visited: Internal - set of variable names in the current expansion chain. + _depth: Internal - current recursion depth. + + Returns: + Tuple of (expanded_serialized_value, list_of_composed_references). + + Raises: + VariableCompositionError: If max depth is exceeded. + VariableCompositionCycleError: If a circular reference is detected. + """ + if _depth > MAX_COMPOSITION_DEPTH: + raise VariableCompositionError( + f'Maximum composition depth ({MAX_COMPOSITION_DEPTH}) exceeded ' + f"while expanding '{variable_name}'. This likely indicates a circular reference." + ) + + if variable_name in _visited: + raise VariableCompositionCycleError(f'Circular reference detected: {" -> ".join(_visited)} -> {variable_name}') + + visited = _visited | {variable_name} + composed: list[ComposedReference] = [] + + # JSON-decode the serialized value so we can work with actual strings. + try: + decoded = json.loads(serialized_value) + except (json.JSONDecodeError, TypeError): + return serialized_value, composed + + # Collect all unique base variable names referenced anywhere in the decoded value. + all_ref_names = _collect_ref_names(decoded) + if not all_ref_names: + # No references at all — return unchanged (but still unescape \<< → <<). + expanded = _unescape_serialized(serialized_value) + return expanded, composed + + # Resolve each unique variable name and recursively expand nested references. + context: dict[str, Any] = {} + unresolved_names: set[str] = set() + + for ref_name in all_ref_names: + ref_serialized, ref_label, ref_version, ref_reason = resolve_fn(ref_name) + + if ref_serialized is None: + composed.append( + ComposedReference( + name=ref_name, + value=None, + label=ref_label, + version=ref_version, + reason=ref_reason, + ) + ) + unresolved_names.add(ref_name) + continue + + # JSON-decode the referenced value. + try: + raw_value = json.loads(ref_serialized) + except (json.JSONDecodeError, TypeError): + composed.append( + ComposedReference( + name=ref_name, + value=None, + label=ref_label, + version=ref_version, + reason=ref_reason, + error=f"Referenced variable '{ref_name}' has a non-JSON serialized value.", + ) + ) + unresolved_names.add(ref_name) + continue + + # Recursively expand references within the resolved value (if it's a string). + nested_composed: list[ComposedReference] = [] + if isinstance(raw_value, str) and has_references(json.dumps(raw_value)): + try: + expanded_serialized, nested_composed = expand_references( + json.dumps(raw_value), + ref_name, + resolve_fn, + _visited=visited, + _depth=_depth + 1, + ) + raw_value = json.loads(expanded_serialized) + except VariableCompositionError as e: + composed.append( + ComposedReference( + name=ref_name, + value=None, + label=ref_label, + version=ref_version, + reason=ref_reason, + error=str(e), + ) + ) + unresolved_names.add(ref_name) + continue + + # Build the ComposedReference for this variable. + value_str: str | None + if isinstance(raw_value, str): + value_str = raw_value + else: + value_str = json.dumps(raw_value) + + composed.append( + ComposedReference( + name=ref_name, + value=value_str, + label=ref_label, + version=ref_version, + reason=ref_reason, + composed_from=nested_composed, + ) + ) + + context[ref_name] = raw_value + + # For unresolved variable names, add a self-referential context entry so that + # Handlebars renders <> back as literal "<>". The _protect_value + # function in render_once will entity-encode the <> characters in the value, + # preventing the swap from consuming them. + for name in unresolved_names: + context[name] = f'<<{name}>>' + + # Walk the decoded value and render each string through the Handlebars swap engine. + rendered = _render_value(decoded, context) + + result_serialized = json.dumps(rendered) + return result_serialized, composed + + +def find_references(serialized_value: str) -> list[str]: + """Find all <> references in a serialized value. + + Detects both simple ``<>`` and block ``<<#helper var>>`` patterns. + For dotted references like ``<>``, only the base variable name + (first segment) is returned. This ensures correct cycle detection and + reference graph building. + + Args: + serialized_value: The raw JSON-serialized variable value to scan. + + Returns: + List of unique variable names referenced, in order of first occurrence. + """ + seen: set[str] = set() + result: list[str] = [] + + # Simple references: <> or <> + for match in _SIMPLE_REF.finditer(serialized_value): + full_ref = match.group(1) + var_name = full_ref.split('.')[0] + if var_name not in seen and var_name not in _HBS_KEYWORDS: + seen.add(var_name) + result.append(var_name) + + # Block helper references: <<#if var>>, <<#each var>>, etc. + for match in _BLOCK_REF.finditer(serialized_value): + var_name = match.group(1) + if var_name not in seen and var_name not in _HBS_KEYWORDS: + seen.add(var_name) + result.append(var_name) + + return result + + +# --------------------------------------------------------------------------- +# Internal helpers +# --------------------------------------------------------------------------- + + +def _collect_ref_names(value: Any) -> list[str]: + """Recursively walk a decoded JSON value and collect all unique base variable names.""" + seen: set[str] = set() + result: list[str] = [] + + def _walk(v: Any) -> None: + if isinstance(v, str): + for match in _SIMPLE_REF.finditer(v): + full_ref = match.group(1) + name = full_ref.split('.')[0] + if name not in seen and name not in _HBS_KEYWORDS: + seen.add(name) + result.append(name) + for match in _BLOCK_REF.finditer(v): + name = match.group(1) + if name not in seen and name not in _HBS_KEYWORDS: + seen.add(name) + result.append(name) + elif isinstance(v, dict): + for val in v.values(): # pyright: ignore[reportUnknownVariableType] + _walk(val) + elif isinstance(v, list): + for item in v: # pyright: ignore[reportUnknownVariableType] + _walk(item) + + _walk(value) + return result + + +def _render_value(value: Any, context: dict[str, Any]) -> Any: + """Recursively walk a decoded JSON value, rendering strings through Handlebars. + + Unresolved variable names should already be present in the context as their + literal ``<>`` text so that Handlebars preserves them. + """ + if isinstance(value, str): + if not has_references(value): + # Unescape \<< to << for non-reference strings. + return value.replace('\\<<', '<<') + return render_once(value, context) + if isinstance(value, dict): + return {k: _render_value(v, context) for k, v in value.items()} # pyright: ignore[reportUnknownVariableType] + if isinstance(value, list): + return [_render_value(v, context) for v in value] # pyright: ignore[reportUnknownVariableType] + return value + + +def _unescape_serialized(serialized: str) -> str: + r"""Unescape ``\<<`` to ``<<`` in a JSON-serialized string. + + In JSON encoding, a literal backslash is ``\\``, so ``\<<`` in user content + appears as ``\\<<`` in the serialized JSON. + """ + return serialized.replace('\\\\<<', '<<') diff --git a/logfire/variables/config.py b/logfire/variables/config.py index 50cdee7c1..f6650ed25 100644 --- a/logfire/variables/config.py +++ b/logfire/variables/config.py @@ -14,7 +14,7 @@ VariablesOptions as VariablesOptions, ) from logfire.variables.abstract import ResolvedVariable -from logfire.variables.variable import Variable +from logfire.variables.variable import _BaseVariable # pyright: ignore[reportPrivateUsage] try: from pydantic import Discriminator @@ -312,6 +312,12 @@ class VariableConfig(BaseModel): """Alternative names that resolve to this variable; useful for name migrations.""" example: str | None = None """JSON-serialized example value from code; used as a template when creating new values in the UI.""" + template_inputs_schema: dict[str, Any] | None = None + """JSON Schema describing the expected template inputs for Handlebars rendering. + + When set, the variable's values can contain {{placeholder}} Handlebars syntax. + The schema is derived from a Pydantic model passed as `template_inputs` to `logfire.var()`. + """ # NOTE: Context-based targeting_key can be set via targeting_context() from logfire.variables. # TODO(DavidM): Consider adding remotely-managed targeting_key_attribute for automatic attribute-based targeting. @@ -532,7 +538,7 @@ def _get_variable_config(self, name: VariableName) -> VariableConfig | None: return None - def get_validation_errors(self, variables: list[Variable[Any]]) -> dict[str, dict[str | None, Exception]]: + def get_validation_errors(self, variables: Sequence[_BaseVariable[Any]]) -> dict[str, dict[str | None, Exception]]: """Validate that all variable label values can be deserialized to their expected types. Args: @@ -565,7 +571,7 @@ def get_validation_errors(self, variables: list[Variable[Any]]) -> dict[str, dic return errors @staticmethod - def from_variables(variables: list[Variable[Any]]) -> VariablesConfig: + def from_variables(variables: Sequence[_BaseVariable[Any]]) -> VariablesConfig: """Create a VariablesConfig from a list of Variable instances. This creates a minimal config with just the name, schema, and example for each variable. @@ -589,6 +595,9 @@ def from_variables(variables: list[Variable[Any]]) -> VariablesConfig: if not is_resolve_function(variable.default): example = variable.type_adapter.dump_json(variable.default).decode('utf-8') + # Get template inputs schema if available + template_inputs_schema = variable.get_template_inputs_schema() + config = VariableConfig( name=variable.name, description=variable.description, @@ -597,6 +606,7 @@ def from_variables(variables: list[Variable[Any]]) -> VariablesConfig: overrides=[], json_schema=json_schema, example=example, + template_inputs_schema=template_inputs_schema, ) variable_configs[variable.name] = config diff --git a/logfire/variables/template_validation.py b/logfire/variables/template_validation.py new file mode 100644 index 000000000..39ecf8133 --- /dev/null +++ b/logfire/variables/template_validation.py @@ -0,0 +1,206 @@ +"""Template validation: check ``{{field}}`` references against ``template_inputs_schema``. + +This module validates that Handlebars ``{{field}}`` references in template variable +values (including composed ``<>`` dependencies) match the declared +``template_inputs_schema``. It uses ``pydantic_handlebars.check_template_compatibility`` +for full AST-based schema checking (nested paths, block scopes, helpers). + +It also provides cycle detection for composition graphs. + +Used by both the SDK and the backend for pre-write validation. +""" + +from __future__ import annotations + +import json +import re +from collections.abc import Callable +from dataclasses import dataclass, field +from typing import Any + +from pydantic_handlebars import check_template_compatibility + +from logfire.variables.composition import find_references + +__all__ = ( + 'TemplateFieldIssue', + 'TemplateValidationResult', + 'validate_template_composition', + 'detect_composition_cycles', + 'find_template_fields', +) + +# Matches {{identifier}} — simple Handlebars variable references. +# Excludes block helpers ({{#if}}), closing tags ({{/if}}), partials ({{> name}}), +# comments ({{! text}}), and triple-stache ({{{raw}}}). +TEMPLATE_FIELD_PATTERN = re.compile(r'\{\{\s*([a-zA-Z_]\w*)\s*\}\}') + + +@dataclass +class TemplateFieldIssue: + """A ``{{field}}`` reference that doesn't match the variable's ``template_inputs_schema``.""" + + field_name: str + """The template field name (e.g., ``user_name`` from ``{{user_name}}``).""" + found_in_variable: str + """Name of the variable whose value contains this field reference.""" + found_in_label: str | None + """Label of the value where the field was found, or ``None`` for the latest version.""" + reference_path: list[str] + """Composition path from the root variable to ``found_in_variable``.""" + + +@dataclass +class TemplateValidationResult: + """Result of template composition validation.""" + + issues: list[TemplateFieldIssue] = field(default_factory=list[TemplateFieldIssue]) + + +def find_template_fields(text: str) -> set[str]: + """Find all ``{{field}}`` references in a string. + + Returns: + Set of field names found in the text. + """ + return set(TEMPLATE_FIELD_PATTERN.findall(text)) + + +def _extract_template_strings(serialized_json: str) -> list[str]: + """Extract all string values from serialized JSON that contain ``{{...}}`` templates.""" + try: + decoded = json.loads(serialized_json) + except (json.JSONDecodeError, TypeError): + # If it's not valid JSON, treat the raw string as a potential template + if '{{' in serialized_json: + return [serialized_json] + return [] + return _collect_template_strings(decoded) + + +def _collect_template_strings(value: Any) -> list[str]: + """Recursively collect strings containing ``{{...}}`` from a decoded JSON value.""" + if isinstance(value, str): + return [value] if '{{' in value else [] + if isinstance(value, dict): + result: list[str] = [] + for v in value.values(): # pyright: ignore[reportUnknownVariableType] + result.extend(_collect_template_strings(v)) + return result + if isinstance(value, list): + result = [] + for item in value: # pyright: ignore[reportUnknownVariableType] + result.extend(_collect_template_strings(item)) + return result + return [] + + +def validate_template_composition( + variable_name: str, + template_inputs_schema: dict[str, Any], + get_all_serialized_values: Callable[[str], dict[str | None, str]], +) -> TemplateValidationResult: + """Validate that ``{{field}}`` references in a template variable match its schema. + + Walks the composition graph starting from *variable_name*, collecting all + template strings from the variable's values and its ``<>`` dependencies, + then uses AST-based schema checking via ``check_template_compatibility`` to + find incompatible field references. + + Args: + variable_name: Name of the template variable to validate. + template_inputs_schema: JSON Schema describing the expected template inputs. + get_all_serialized_values: Function that returns ``{label_or_none: serialized_json}`` + for any variable name. ``None`` key represents the latest version. + + Returns: + A :class:`TemplateValidationResult` with any issues found. + """ + issues: list[TemplateFieldIssue] = [] + seen_issues: set[tuple[str, str, str | None]] = set() + + def _collect(name: str, path: list[str], visited: frozenset[str]) -> None: + if name in visited: + return + visited = visited | {name} + + for label, serialized_value in get_all_serialized_values(name).items(): + templates = _extract_template_strings(serialized_value) + if not templates: + for ref in find_references(serialized_value): + _collect(ref, path + [ref], visited) + continue + + result = check_template_compatibility(templates, template_inputs_schema) + for issue in result.issues: + if issue.severity != 'error': + continue + key = (issue.field_path, name, label) + if key not in seen_issues: + seen_issues.add(key) + issues.append( + TemplateFieldIssue( + field_name=issue.field_path, + found_in_variable=name, + found_in_label=label, + reference_path=list(path), + ) + ) + + for ref in find_references(serialized_value): + _collect(ref, path + [ref], visited) + + _collect(variable_name, [], frozenset()) + + return TemplateValidationResult(issues=issues) + + +def detect_composition_cycles( + variable_name: str, + new_references: set[str], + get_all_references: Callable[[str], set[str]], +) -> list[str] | None: + """Check if adding *new_references* to *variable_name* would create a cycle. + + Args: + variable_name: The variable being updated. + new_references: Set of variable names directly referenced by the new value. + get_all_references: Function that returns all variable names referenced by + any value of the given variable name. + + Returns: + The cycle path (e.g., ``['A', 'B', 'C', 'A']``) if a cycle is detected, + or ``None`` if no cycle exists. + """ + for ref in sorted(new_references): # sort for deterministic results + path = _find_cycle(variable_name, ref, get_all_references, frozenset()) + if path is not None: + return path + return None + + +def _find_cycle( + target: str, + current: str, + get_all_references: Callable[[str], set[str]], + visited: frozenset[str], + path: list[str] | None = None, +) -> list[str] | None: + """DFS to find a path from *current* back to *target*.""" + if path is None: + path = [target, current] + + if current == target: + return path + + if current in visited: + return None + + visited = visited | {current} + + for ref in sorted(get_all_references(current)): # sort for deterministic results + result = _find_cycle(target, ref, get_all_references, visited, path + [ref]) + if result is not None: + return result + + return None diff --git a/logfire/variables/variable.py b/logfire/variables/variable.py index 5b9a5a382..318804db9 100644 --- a/logfire/variables/variable.py +++ b/logfire/variables/variable.py @@ -1,7 +1,8 @@ from __future__ import annotations as _annotations import inspect -from collections.abc import Iterator, Mapping, Sequence +import json +from collections.abc import Callable, Iterator, Mapping, Sequence from contextlib import ExitStack, contextmanager from contextvars import ContextVar from dataclasses import dataclass, field, replace @@ -10,9 +11,18 @@ from opentelemetry.trace import get_current_span from pydantic import TypeAdapter, ValidationError +from pydantic_handlebars import HandlebarsError from typing_extensions import TypeIs +from logfire.variables.composition import ( + ComposedReference, + VariableCompositionError, + expand_references, + has_references, +) + if TYPE_CHECKING: + from logfire.variables.abstract import VariableProvider from logfire.variables.config import VariableConfig if find_spec('anyio') is not None: # pragma: no branch @@ -27,11 +37,14 @@ __all__ = ( 'ResolveFunction', 'is_resolve_function', + '_BaseVariable', 'Variable', + 'TemplateVariable', 'targeting_context', ) T_co = TypeVar('T_co', covariant=True) +InputsT = TypeVar('InputsT') _VARIABLE_OVERRIDES: ContextVar[dict[str, Any] | None] = ContextVar('_VARIABLE_OVERRIDES', default=None) @@ -115,8 +128,12 @@ def is_resolve_function(f: Any) -> TypeIs[ResolveFunction[Any]]: return required_positional <= 2 and total_positional >= 2 -class Variable(Generic[T_co]): - """A managed variable that can be resolved dynamically based on configuration.""" +class _BaseVariable(Generic[T_co]): + """Base class for managed variables with shared resolution infrastructure. + + Contains all shared logic: init, deserialization, override, refresh, config, + resolution pipeline. Subclasses (Variable, TemplateVariable) add their own get() method. + """ name: str """Unique name identifying this variable.""" @@ -126,6 +143,8 @@ class Variable(Generic[T_co]): """Default value or function to compute the default.""" description: str | None """Description of the variable.""" + template_inputs_type: type[Any] | None + """The Pydantic model type for template inputs, if template rendering is enabled.""" logfire_instance: logfire.Logfire """The Logfire instance this variable is associated with.""" @@ -137,6 +156,7 @@ def __init__( type: type[T_co], default: T_co | ResolveFunction[T_co], description: str | None = None, + template_inputs: type[Any] | None = None, logfire_instance: logfire.Logfire, ): """Create a new managed variable. @@ -147,21 +167,35 @@ def __init__( default: Default value to use when no configuration is found, or a function that computes the default based on targeting_key and attributes. description: Optional human-readable description of what this variable controls. + template_inputs: Optional Pydantic model type describing the expected template inputs + for Handlebars rendering. When set, values can contain ``{{placeholder}}`` syntax. logfire_instance: The Logfire instance this variable is associated with. Used to determine config, etc. """ self.name = name self.value_type = type self.default = default self.description = description + self.template_inputs_type = template_inputs self.logfire_instance = logfire_instance.with_settings(custom_scope_suffix='variables') self.type_adapter = TypeAdapter[T_co](type) - def _deserialize(self, serialized_value: str) -> T_co | Exception: + if template_inputs is not None: + self._template_inputs_adapter: TypeAdapter[Any] | None = TypeAdapter(template_inputs) + else: + self._template_inputs_adapter = None + + def get_template_inputs_schema(self) -> dict[str, Any] | None: + """Return the JSON schema for template inputs, or None if not configured.""" + if self._template_inputs_adapter is not None: + return self._template_inputs_adapter.json_schema() + return None + + def _deserialize(self, serialized_value: str) -> T_co | ValidationError | ValueError: """Deserialize a JSON string to the variable's type, returning an Exception on failure.""" try: return self.type_adapter.validate_json(serialized_value) - except Exception as e: + except (ValidationError, ValueError) as e: return e @contextmanager @@ -187,82 +221,13 @@ def refresh_sync(self, force: bool = False): """Synchronously refresh the variable.""" self.logfire_instance.config.get_variable_provider().refresh(force=force) - def get( - self, - targeting_key: str | None = None, - attributes: Mapping[str, Any] | None = None, - *, - label: str | None = None, - ) -> ResolvedVariable[T_co]: - """Resolve the variable and return full details including label, version, and any errors. - - Args: - targeting_key: Optional key for deterministic label selection (e.g., user ID). - If not provided, falls back to contextvar targeting key (set via targeting_context), - then to the current trace ID if there is an active trace. - attributes: Optional attributes for condition-based targeting rules. - label: Optional explicit label name to select. If provided, bypasses rollout - weights and targeting, directly selecting the specified label. If the label - doesn't exist in the configuration, falls back to default resolution. - - Returns: - A ResolvedVariable object containing the resolved value, selected label, - version, and any errors that occurred. - """ - merged_attributes = self._get_merged_attributes(attributes) - - # Targeting key resolution: call-site > contextvar > trace_id - if targeting_key is None: - targeting_key = _get_contextvar_targeting_key(self.name) - - if targeting_key is None and (current_trace_id := get_current_span().get_span_context().trace_id): - # If there is no active trace, the current_trace_id will be zero - targeting_key = f'trace_id:{current_trace_id:032x}' - - # Include the variable name directly here to make the span name more useful, - # it'll still be low cardinality. This also prevents it from being scrubbed from the message. - # Don't inline the f-string to avoid f-string magic. - span_name = f'Resolve variable {self.name}' - with ExitStack() as stack: - span: logfire.LogfireSpan | None = None - if _get_variables_instrument(self.logfire_instance.config.variables): - span = stack.enter_context( - self.logfire_instance.span( - span_name, - name=self.name, - targeting_key=targeting_key, - attributes=merged_attributes, - ) - ) - result = self._resolve(targeting_key, merged_attributes, span, label) - if span is not None: - # Serialize value safely for OTel span attributes, which only support primitives. - # Try to JSON serialize the value; if that fails, fall back to string representation. - try: - serialized_value = self.type_adapter.dump_json(result.value).decode('utf-8') - except Exception: - serialized_value = repr(result.value) - span.set_attributes( - { - 'name': result.name, - 'value': serialized_value, - 'label': result.label, - 'version': result.version, - 'reason': result._reason, # pyright: ignore[reportPrivateUsage] - } - ) - if result.exception: - span.record_exception( - result.exception, - ) - return result - def _resolve( self, targeting_key: str | None, attributes: Mapping[str, Any] | None, span: logfire.LogfireSpan | None, label: str | None = None, + render_fn: Callable[[str], str] | None = None, ) -> ResolvedVariable[T_co]: serialized_result: ResolvedVariable[str | None] | None = None try: @@ -270,6 +235,10 @@ def _resolve( context_value = context_overrides[self.name] if is_resolve_function(context_value): context_value = context_value(targeting_key, attributes) + # For TemplateVariable (render_fn set), the override is a template + # that still gets rendered with inputs. + if render_fn is not None: + context_value = self._render_default(context_value, render_fn) return ResolvedVariable(name=self.name, value=context_value, _reason='context_override') provider = self.logfire_instance.config.get_variable_provider() @@ -278,21 +247,8 @@ def _resolve( if label is not None: serialized_result = provider.get_serialized_value_for_label(self.name, label) if serialized_result.value is not None: - # Successfully got the explicit label - value_or_exc = self._deserialize(serialized_result.value) - if isinstance(value_or_exc, Exception): - if span: # pragma: no branch - span.set_attribute('invalid_serialized_label', serialized_result.label) - span.set_attribute('invalid_serialized_value', serialized_result.value) - default = self._get_default(targeting_key, attributes) - reason: str = 'validation_error' if isinstance(value_or_exc, ValidationError) else 'other_error' - return ResolvedVariable(name=self.name, value=default, exception=value_or_exc, _reason=reason) - return ResolvedVariable( - name=self.name, - value=value_or_exc, - label=serialized_result.label, - version=serialized_result.version, - _reason='resolved', + return self._expand_and_deserialize( + serialized_result, provider, targeting_key, attributes, span, render_fn=render_fn ) # Label not found - fall through to default resolution @@ -300,33 +256,136 @@ def _resolve( if serialized_result.value is None: default = self._get_default(targeting_key, attributes) + if render_fn is not None: + default = self._render_default(default, render_fn) return _with_value(serialized_result, default) - # Deserialize - returns T | Exception - value_or_exc = self._deserialize(serialized_result.value) - if isinstance(value_or_exc, Exception): - if span: # pragma: no branch - span.set_attribute('invalid_serialized_label', serialized_result.label) - span.set_attribute('invalid_serialized_value', serialized_result.value) - default = self._get_default(targeting_key, attributes) - reason: str = 'validation_error' if isinstance(value_or_exc, ValidationError) else 'other_error' - return ResolvedVariable(name=self.name, value=default, exception=value_or_exc, _reason=reason) - - return ResolvedVariable( - name=self.name, - value=value_or_exc, - label=serialized_result.label, - version=serialized_result.version, - _reason='resolved', + return self._expand_and_deserialize( + serialized_result, provider, targeting_key, attributes, span, render_fn=render_fn ) - except Exception as e: + except ( # Safety net: providers and resolve functions are user-defined and may raise any of these + ValidationError, + ValueError, + TypeError, + KeyError, + AttributeError, + RuntimeError, + OSError, + HandlebarsError, + VariableCompositionError, + ) as e: if span and serialized_result is not None: # pragma: no cover span.set_attribute('invalid_serialized_label', serialized_result.label) span.set_attribute('invalid_serialized_value', serialized_result.value) default = self._get_default(targeting_key, attributes) return ResolvedVariable(name=self.name, value=default, exception=e, _reason='other_error') + def _render_default(self, default: Any, render_fn: Callable[[str], str]) -> T_co: + """Serialize the default value, apply render_fn, then deserialize back.""" + try: + serialized = self.type_adapter.dump_json(default).decode('utf-8') + rendered = render_fn(serialized) + result = self._deserialize(rendered) + if isinstance(result, (ValidationError, ValueError)): + raise result + return result + except (ValidationError, ValueError, TypeError, HandlebarsError): + # If rendering the default fails, return the original default + return default + + def _expand_and_deserialize( + self, + serialized_result: ResolvedVariable[str | None], + provider: VariableProvider, + targeting_key: str | None, + attributes: Mapping[str, Any] | None, + span: logfire.LogfireSpan | None, + render_fn: Callable[[str], str] | None = None, + ) -> ResolvedVariable[T_co]: + """Expand <> in a serialized value, optionally render templates, then deserialize. + + Handles composition between the provider fetch and Pydantic deserialization. + When render_fn is provided, it is applied after composition and before deserialization. + """ + assert serialized_result.value is not None + + serialized_value = serialized_result.value + composed: list[ComposedReference] = [] + + # Expand <> if any are present + if has_references(serialized_value): + + def resolve_ref(ref_name: str) -> tuple[str | None, str | None, int | None, str]: + ref_resolved = provider.get_serialized_value(ref_name, targeting_key, attributes) + return ( + ref_resolved.value, + ref_resolved.label, + ref_resolved.version, + ref_resolved._reason, # pyright: ignore[reportPrivateUsage] + ) + + try: + serialized_value, composed = expand_references( + serialized_value, + self.name, + resolve_ref, + ) + except VariableCompositionError as e: + default = self._get_default(targeting_key, attributes) + return ResolvedVariable( + name=self.name, + value=default, + exception=e, + _reason='other_error', + label=serialized_result.label, + version=serialized_result.version, + composed_from=composed, + ) + + # Apply render_fn (template rendering) if provided + if render_fn is not None: + try: + serialized_value = render_fn(serialized_value) + except (HandlebarsError, ValueError, TypeError) as e: + default = self._get_default(targeting_key, attributes) + return ResolvedVariable( + name=self.name, + value=default, + exception=e, + _reason='other_error', + label=serialized_result.label, + version=serialized_result.version, + composed_from=composed, + ) + + # Deserialize the (possibly expanded/rendered) value + value_or_exc = self._deserialize(serialized_value) + if isinstance(value_or_exc, Exception): + if span: # pragma: no branch + span.set_attribute('invalid_serialized_label', serialized_result.label) + span.set_attribute('invalid_serialized_value', serialized_value) + default = self._get_default(targeting_key, attributes) + reason: str = 'validation_error' if isinstance(value_or_exc, ValidationError) else 'other_error' + return ResolvedVariable( + name=self.name, + value=default, + exception=value_or_exc, + _reason=reason, + composed_from=composed, + ) + + return ResolvedVariable( + name=self.name, + value=value_or_exc, + label=serialized_result.label, + version=serialized_result.version, + _reason='resolved', + composed_from=composed, + _serialized_value=serialized_value, + _deserializer=self._deserialize, + ) + def _get_default( self, targeting_key: str | None = None, merged_attributes: Mapping[str, Any] | None = None ) -> T_co: @@ -374,6 +433,10 @@ def to_config(self) -> VariableConfig: if not is_resolve_function(self.default): example = self.type_adapter.dump_json(self.default).decode('utf-8') + template_inputs_schema: dict[str, Any] | None = None + if self._template_inputs_adapter is not None: + template_inputs_schema = self._template_inputs_adapter.json_schema() + return VariableConfig( name=self.name, description=self.description, @@ -382,8 +445,191 @@ def to_config(self) -> VariableConfig: overrides=[], json_schema=json_schema, example=example, + template_inputs_schema=template_inputs_schema, ) + def _get_result_and_record_span( + self, + targeting_key: str | None, + attributes: Mapping[str, Any] | None, + label: str | None, + render_fn: Callable[[str], str] | None = None, + ) -> ResolvedVariable[T_co]: + """Common get() logic: resolve targeting key, open span, call _resolve, record attributes.""" + merged_attributes = self._get_merged_attributes(attributes) + + # Targeting key resolution: call-site > contextvar > trace_id + if targeting_key is None: + targeting_key = _get_contextvar_targeting_key(self.name) + + if targeting_key is None and (current_trace_id := get_current_span().get_span_context().trace_id): + # If there is no active trace, the current_trace_id will be zero + targeting_key = f'trace_id:{current_trace_id:032x}' + + # Include the variable name directly here to make the span name more useful, + # it'll still be low cardinality. This also prevents it from being scrubbed from the message. + # Don't inline the f-string to avoid f-string magic. + span_name = f'Resolve variable {self.name}' + with ExitStack() as stack: + span: logfire.LogfireSpan | None = None + if _get_variables_instrument(self.logfire_instance.config.variables): + span = stack.enter_context( + self.logfire_instance.span( + span_name, + name=self.name, + targeting_key=targeting_key, + attributes=merged_attributes, + ) + ) + result = self._resolve(targeting_key, merged_attributes, span, label, render_fn=render_fn) + # Ensure rendering support is always available + if result._deserializer is None: # pyright: ignore[reportPrivateUsage] + result._deserializer = self._deserialize # pyright: ignore[reportPrivateUsage] + if result._serialized_value is None and result.value is not None: # pyright: ignore[reportPrivateUsage] + try: + result._serialized_value = self.type_adapter.dump_json(result.value).decode('utf-8') # pyright: ignore[reportPrivateUsage] + except (ValueError, TypeError, RuntimeError): + pass + if span is not None: + # Serialize value safely for OTel span attributes, which only support primitives. + # Try to JSON serialize the value; if that fails, fall back to string representation. + try: + serialized_value = self.type_adapter.dump_json(result.value).decode('utf-8') + except (ValueError, TypeError, RuntimeError): + serialized_value = repr(result.value) + attrs: dict[str, Any] = { + 'name': result.name, + 'value': serialized_value, + 'label': result.label, + 'version': result.version, + 'reason': result._reason, # pyright: ignore[reportPrivateUsage] + } + if result.composed_from: + attrs['composed_from'] = json.dumps( + [ + { + 'name': c.name, + 'version': c.version, + 'label': c.label, + 'reason': c.reason, + 'error': c.error, + } + for c in result.composed_from + ] + ) + span.set_attributes(attrs) + if result.exception: + span.record_exception( + result.exception, + ) + return result + + +class Variable(_BaseVariable[T_co]): + """A managed variable that can be resolved dynamically based on configuration.""" + + def get( + self, + targeting_key: str | None = None, + attributes: Mapping[str, Any] | None = None, + *, + label: str | None = None, + ) -> ResolvedVariable[T_co]: + """Resolve the variable and return full details including label, version, and any errors. + + Args: + targeting_key: Optional key for deterministic label selection (e.g., user ID). + If not provided, falls back to contextvar targeting key (set via targeting_context), + then to the current trace ID if there is an active trace. + attributes: Optional attributes for condition-based targeting rules. + label: Optional explicit label name to select. If provided, bypasses rollout + weights and targeting, directly selecting the specified label. If the label + doesn't exist in the configuration, falls back to default resolution. + + Returns: + A ResolvedVariable object containing the resolved value, selected label, + version, and any errors that occurred. + """ + return self._get_result_and_record_span(targeting_key, attributes, label) + + +class TemplateVariable(_BaseVariable[T_co], Generic[T_co, InputsT]): + """A managed variable with integrated template rendering. + + Like ``Variable``, but ``get()`` requires ``inputs`` and automatically renders + Handlebars ``{{placeholder}}`` templates in the resolved value before returning. + The pipeline is: resolve → compose ``<>`` → render ``{{}}`` → deserialize. + """ + + inputs_type: type[InputsT] + """The type used for template inputs.""" + + def __init__( + self, + name: str, + *, + type: type[T_co], + default: T_co | ResolveFunction[T_co], + inputs_type: type[InputsT], + description: str | None = None, + logfire_instance: logfire.Logfire, + ): + """Create a new template variable. + + Args: + name: Unique name identifying this variable. + type: The expected type of this variable's values, used for validation. + default: Default value to use when no configuration is found, or a function + that computes the default based on targeting_key and attributes. + inputs_type: The type (typically a Pydantic ``BaseModel``) describing the expected + template inputs. Used for type-safe ``get(inputs)`` calls and JSON schema generation. + description: Optional human-readable description of what this variable controls. + logfire_instance: The Logfire instance this variable is associated with. + """ + super().__init__( + name, + type=type, + default=default, + description=description, + template_inputs=inputs_type, + logfire_instance=logfire_instance, + ) + self.inputs_type = inputs_type + + def get( + self, + inputs: InputsT, + targeting_key: str | None = None, + attributes: Mapping[str, Any] | None = None, + *, + label: str | None = None, + ) -> ResolvedVariable[T_co]: + """Resolve the variable, render templates with the given inputs, and return the result. + + The resolution pipeline is: + 1. Fetch serialized value from provider (or use default) + 2. Expand ``<>`` composition references + 3. Render ``{{placeholder}}`` Handlebars templates using ``inputs`` + 4. Deserialize to the variable's type + + Args: + inputs: Template context values. Typically a Pydantic ``BaseModel`` instance + matching ``inputs_type``. All ``{{placeholder}}`` expressions in the value + are rendered using this context. + targeting_key: Optional key for deterministic label selection (e.g., user ID). + attributes: Optional attributes for condition-based targeting rules. + label: Optional explicit label name to select. + + Returns: + A ResolvedVariable with the fully rendered and deserialized value. + """ + from logfire.variables.abstract import render_serialized_string + + def _render_fn(serialized_json: str) -> str: + return render_serialized_string(serialized_json, inputs) + + return self._get_result_and_record_span(targeting_key, attributes, label, render_fn=_render_fn) + def _with_value(details: ResolvedVariable[Any], new_value: T_co) -> ResolvedVariable[T_co]: """Return a copy of the provided resolution details, just with a different value. @@ -401,7 +647,7 @@ def _with_value(details: ResolvedVariable[Any], new_value: T_co) -> ResolvedVari @contextmanager def targeting_context( targeting_key: str, - variables: Sequence[Variable[Any]] | None = None, + variables: Sequence[Variable[Any] | TemplateVariable[Any, Any]] | None = None, ) -> Iterator[None]: """Set the targeting key for variable resolution within this context. diff --git a/mkdocs.yml b/mkdocs.yml index da23654bb..690e196ae 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -208,6 +208,7 @@ nav: - Managed Variables: - Overview: reference/advanced/managed-variables/index.md - UI Guide: reference/advanced/managed-variables/ui.md + - Templates & Composition: reference/advanced/managed-variables/templates-and-composition.md - A/B Testing: reference/advanced/managed-variables/ab-testing.md - Targeting: reference/advanced/managed-variables/targeting.md - Remote Variables: reference/advanced/managed-variables/remote.md diff --git a/pyproject.toml b/pyproject.toml index 8adec8d49..4e0b8abd9 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -84,7 +84,7 @@ google-genai = ["opentelemetry-instrumentation-google-genai >= 0.4b0"] litellm = ["openinference-instrumentation-litellm >= 0"] dspy = ["openinference-instrumentation-dspy >= 0"] datasets = ["httpx>=0.27.2", "pydantic>=2", "pydantic-evals>=1.0.0; python_version >= '3.10'"] -variables = ["pydantic>=2"] +variables = ["pydantic>=2", "pydantic-handlebars>=0.1.0; python_version >= '3.10'"] [project.urls] Homepage = "https://logfire.pydantic.dev/" @@ -197,6 +197,7 @@ dev = [ "pytest-examples>=0.0.18", "pytest-timeout>=2.4.0", "pytest-asyncio>=0.24.0", + "pydantic-handlebars>=0.1.0; python_version >= '3.10'", ] docs = [ "black>=23.12.0", diff --git a/tests/test_template_validation.py b/tests/test_template_validation.py new file mode 100644 index 000000000..1486ddfb3 --- /dev/null +++ b/tests/test_template_validation.py @@ -0,0 +1,577 @@ +"""Tests for template_validation: {{field}} validation and cycle detection.""" + +# pyright: reportPrivateUsage=false, reportUnknownVariableType=false, reportUnknownArgumentType=false + +from __future__ import annotations + +from logfire.variables.template_validation import ( + TemplateFieldIssue, + TemplateValidationResult, + _extract_template_strings, + detect_composition_cycles, + find_template_fields, + validate_template_composition, +) + +# ============================================================================= +# find_template_fields +# ============================================================================= + + +class TestFindTemplateFields: + def test_simple_field(self): + assert find_template_fields('Hello {{name}}!') == {'name'} + + def test_multiple_fields(self): + result = find_template_fields('{{greeting}} {{name}}, age {{age}}') + assert result == {'greeting', 'name', 'age'} + + def test_duplicate_fields(self): + """Duplicate fields produce a single entry in the set.""" + result = find_template_fields('{{name}} and {{name}} again') + assert result == {'name'} + + def test_empty_string(self): + assert find_template_fields('') == set() + + def test_no_templates(self): + assert find_template_fields('Hello world, no templates here') == set() + + def test_ignores_block_helpers(self): + """{{#if condition}} is a single block; not matched as {{identifier}}.""" + result = find_template_fields('{{#if condition}}yes{{/if}}') + # The entire {{#if condition}} has # after {{ so it doesn't match + assert result == set() + + def test_block_helper_hash_excluded(self): + """{{#if}} has a # prefix so the identifier doesn't start with [a-zA-Z_].""" + result = find_template_fields('{{#if}}content{{/if}}') + assert 'if' not in result + assert result == set() + + def test_closing_tag_excluded(self): + """{{/if}} has a / prefix so it won't match.""" + result = find_template_fields('{{/if}}') + assert result == set() + + def test_partial_excluded(self): + """{{> partial}} has a > prefix so it won't match.""" + result = find_template_fields('{{> myPartial}}') + assert result == set() + + def test_comment_excluded(self): + """{{! comment}} has a ! prefix so it won't match.""" + result = find_template_fields('{{! this is a comment}}') + assert result == set() + + def test_triple_stache_not_matched(self): + """{{{raw}}} — the outer braces don't form a valid {{identifier}} match.""" + # {{{raw}}} is 3 opening braces + raw + 3 closing braces + # The regex looks for {{ identifier }}, so {{{ would have an extra { before the identifier + result = find_template_fields('{{{raw}}}') + # The regex matches {{raw}} inside {{{raw}}}, leaving extra braces. + # Actually {{ raw }} is a valid match embedded in {{{ raw }}} + # Let's just verify empirically. + assert 'raw' in result # {{raw}} is still matched within {{{raw}}} + + def test_field_with_spaces(self): + """Spaces inside {{ field }} are allowed by the regex.""" + result = find_template_fields('{{ name }}') + assert result == {'name'} + + def test_field_with_underscore(self): + result = find_template_fields('{{user_name}}') + assert result == {'user_name'} + + def test_field_with_digits(self): + result = find_template_fields('{{item1}}') + assert result == {'item1'} + + def test_field_starting_with_underscore(self): + result = find_template_fields('{{_private}}') + assert result == {'_private'} + + def test_mixed_valid_and_invalid(self): + """Valid {{field}} mixed with helpers and partials.""" + text = '{{name}} {{#if active}}{{role}}{{/if}} {{> footer}} {{! ignored}}' + result = find_template_fields(text) + assert 'name' in result + assert 'role' in result + # Helpers, closing tags, partials, and comments should not appear + assert '#if' not in result + assert '/if' not in result + assert '> footer' not in result + assert '! ignored' not in result + + +# ============================================================================= +# _extract_template_strings +# ============================================================================= + + +class TestExtractTemplateStrings: + def test_json_string(self): + """JSON string value like '"Hello {{name}}"'.""" + result = _extract_template_strings('"Hello {{name}}"') + assert result == ['Hello {{name}}'] + + def test_json_object(self): + """JSON object with multiple string values containing fields.""" + result = _extract_template_strings('{"key": "Hello {{name}}", "other": "{{age}}"}') + assert result == ['Hello {{name}}', '{{age}}'] + + def test_json_array(self): + """JSON array with string values containing fields.""" + result = _extract_template_strings('["{{a}}", "{{b}}"]') + assert result == ['{{a}}', '{{b}}'] + + def test_invalid_json_falls_back_to_plain_text(self): + """Invalid JSON with templates falls back to the raw string.""" + result = _extract_template_strings('not valid json {{field}}') + assert result == ['not valid json {{field}}'] + + def test_invalid_json_no_templates(self): + """Invalid JSON without templates returns empty list.""" + result = _extract_template_strings('not valid json no templates') + assert result == [] + + def test_json_number_no_fields(self): + """JSON number value has no template strings.""" + result = _extract_template_strings('42') + assert result == [] + + def test_json_boolean_no_fields(self): + """JSON boolean value has no template strings.""" + result = _extract_template_strings('true') + assert result == [] + + def test_json_null_no_fields(self): + """JSON null value has no template strings.""" + result = _extract_template_strings('null') + assert result == [] + + def test_nested_json_object(self): + """Nested JSON objects have their string values collected.""" + result = _extract_template_strings('{"outer": {"inner": "{{deep}}"}}') + assert result == ['{{deep}}'] + + def test_mixed_types_in_object(self): + """Object with mixed types: only strings with templates are collected.""" + result = _extract_template_strings('{"text": "{{name}}", "count": 42, "active": true, "nothing": null}') + assert result == ['{{name}}'] + + def test_array_with_mixed_types(self): + """Array with mixed types: only template strings collected.""" + result = _extract_template_strings('["{{a}}", 42, true, null, "{{b}}"]') + assert result == ['{{a}}', '{{b}}'] + + def test_empty_json_string(self): + result = _extract_template_strings('""') + assert result == [] + + def test_empty_json_object(self): + result = _extract_template_strings('{}') + assert result == [] + + def test_empty_json_array(self): + result = _extract_template_strings('[]') + assert result == [] + + def test_deeply_nested_structure(self): + """Deeply nested JSON structure with template strings at various levels.""" + result = _extract_template_strings('{"a": [{"b": "{{x}}"}, {"c": ["{{y}}", {"d": "{{z}}"}]}]}') + assert result == ['{{x}}', '{{y}}', '{{z}}'] + + def test_string_without_templates_excluded(self): + """Strings without {{}} are not collected.""" + result = _extract_template_strings('{"a": "no templates", "b": "has {{one}}"}') + assert result == ['has {{one}}'] + + +# ============================================================================= +# validate_template_composition +# ============================================================================= + + +def _make_get_all_serialized( + data: dict[str, dict[str | None, str]], +) -> ...: + """Helper: build get_all_serialized_values from a simple mapping.""" + + def get_all_serialized_values(name: str) -> dict[str | None, str]: + return data.get(name, {}) + + return get_all_serialized_values + + +class TestValidateTemplateComposition: + def test_all_fields_valid(self): + """All {{field}} references match schema properties — no issues.""" + schema = {'properties': {'name': {'type': 'string'}, 'age': {'type': 'integer'}}} + get_values = _make_get_all_serialized( + { + 'my_var': {None: '"Hello {{name}}, you are {{age}}"'}, + } + ) + result = validate_template_composition('my_var', schema, get_values) + assert result.issues == [] + + def test_field_not_in_schema(self): + """A {{field}} not in schema properties produces an issue.""" + schema = {'properties': {'name': {'type': 'string'}}} + get_values = _make_get_all_serialized( + { + 'my_var': {None: '"Hello {{name}} {{unknown}}"'}, + } + ) + result = validate_template_composition('my_var', schema, get_values) + assert len(result.issues) == 1 + issue = result.issues[0] + assert issue.field_name == 'unknown' + assert issue.found_in_variable == 'my_var' + assert issue.found_in_label is None + assert issue.reference_path == [] + + def test_transitive_reference_issue(self): + """var_a references <>, var_b has {{field}} not in var_a's schema.""" + schema = {'properties': {'name': {'type': 'string'}}} + get_values = _make_get_all_serialized( + { + 'var_a': {None: '"Hello {{name}} <>"'}, + 'var_b': {None: '"extra {{bad_field}}"'}, + } + ) + result = validate_template_composition('var_a', schema, get_values) + assert len(result.issues) == 1 + issue = result.issues[0] + assert issue.field_name == 'bad_field' + assert issue.found_in_variable == 'var_b' + assert issue.reference_path == ['var_b'] + + def test_multiple_labels(self): + """Issues across multiple labels (None for latest, 'prod' for labeled).""" + schema = {'properties': {'x': {'type': 'string'}}} + get_values = _make_get_all_serialized( + { + 'my_var': { + None: '"{{x}} {{bad1}}"', + 'prod': '"{{x}} {{bad2}}"', + }, + } + ) + result = validate_template_composition('my_var', schema, get_values) + assert len(result.issues) == 2 + field_names = {i.field_name for i in result.issues} + assert field_names == {'bad1', 'bad2'} + labels = {i.found_in_label for i in result.issues} + assert labels == {None, 'prod'} + + def test_cycle_does_not_infinite_loop(self): + """A cycle in composition references terminates without infinite recursion.""" + schema = {'properties': {}} + get_values = _make_get_all_serialized( + { + 'a': {None: '"<>"'}, + 'b': {None: '"<>"'}, + } + ) + # Should complete without hanging + result = validate_template_composition('a', schema, get_values) + # No fields to report as issues — the cycle just stops traversal + assert isinstance(result, TemplateValidationResult) + + def test_no_template_fields(self): + """Variable with no {{}} fields produces no issues.""" + schema = {'properties': {'name': {'type': 'string'}}} + get_values = _make_get_all_serialized( + { + 'my_var': {None: '"Hello world"'}, + } + ) + result = validate_template_composition('my_var', schema, get_values) + assert result.issues == [] + + def test_empty_schema_no_restrictions(self): + """With empty properties, any field is allowed (no declared properties to conflict with).""" + schema = {'properties': {}} + get_values = _make_get_all_serialized( + { + 'my_var': {None: '"{{a}} {{b}}"'}, + } + ) + result = validate_template_composition('my_var', schema, get_values) + assert result.issues == [] + + def test_unknown_fields_with_declared_properties(self): + """When schema declares properties, unlisted fields are issues.""" + schema = {'properties': {'x': {'type': 'string'}}} + get_values = _make_get_all_serialized( + { + 'my_var': {None: '"{{a}} {{b}}"'}, + } + ) + result = validate_template_composition('my_var', schema, get_values) + assert len(result.issues) == 2 + field_names = {i.field_name for i in result.issues} + assert field_names == {'a', 'b'} + + def test_schema_without_properties_key(self): + """Schema missing 'properties' key treats all fields as invalid.""" + schema = {'type': 'object'} + get_values = _make_get_all_serialized( + { + 'my_var': {None: '"{{field}}"'}, + } + ) + result = validate_template_composition('my_var', schema, get_values) + assert len(result.issues) == 1 + assert result.issues[0].field_name == 'field' + + def test_variable_with_no_values(self): + """Variable with no serialized values produces no issues.""" + schema = {'properties': {'name': {'type': 'string'}}} + get_values = _make_get_all_serialized( + { + 'my_var': {}, + } + ) + result = validate_template_composition('my_var', schema, get_values) + assert result.issues == [] + + def test_unknown_variable_no_values(self): + """Unknown variable (not in data) produces no issues.""" + schema = {'properties': {'name': {'type': 'string'}}} + get_values = _make_get_all_serialized({}) + result = validate_template_composition('unknown', schema, get_values) + assert result.issues == [] + + def test_transitive_chain(self): + """A -> B -> C, field in C not in A's schema.""" + schema = {'properties': {'ok': {'type': 'string'}}} + get_values = _make_get_all_serialized( + { + 'a': {None: '"{{ok}} <>"'}, + 'b': {None: '"<>"'}, + 'c': {None: '"{{deep_field}}"'}, + } + ) + result = validate_template_composition('a', schema, get_values) + assert len(result.issues) == 1 + issue = result.issues[0] + assert issue.field_name == 'deep_field' + assert issue.found_in_variable == 'c' + assert issue.reference_path == ['b', 'c'] + + def test_duplicate_issue_dedup(self): + """Same field/variable/label combination is only reported once.""" + schema = {'properties': {'allowed': {'type': 'string'}}} + # Two labels in a, both reference b which has the same field + get_values = _make_get_all_serialized( + { + 'a': {None: '"<>"', 'prod': '"<>"'}, + 'b': {None: '"{{field}}"'}, + } + ) + result = validate_template_composition('a', schema, get_values) + # field in b/None should only appear once even though a has two labels pointing to b + b_issues = [i for i in result.issues if i.found_in_variable == 'b'] + assert len(b_issues) == 1 + + def test_issue_reference_path_is_copy(self): + """reference_path in issues is an independent list, not a shared reference.""" + schema = {'properties': {'allowed': {'type': 'string'}}} + get_values = _make_get_all_serialized( + { + 'a': {None: '"<> <>"'}, + 'b': {None: '"{{field_b}}"'}, + 'c': {None: '"{{field_c}}"'}, + } + ) + result = validate_template_composition('a', schema, get_values) + assert len(result.issues) >= 2 + paths = [i.reference_path for i in result.issues] + # Each path should be independent + for p in paths: + assert isinstance(p, list) + + def test_json_object_value_fields(self): + """Fields inside JSON object string values are found.""" + schema = {'properties': {'name': {'type': 'string'}}} + get_values = _make_get_all_serialized( + { + 'my_var': {None: '{"greeting": "Hello {{name}} {{extra}}"}'}, + } + ) + result = validate_template_composition('my_var', schema, get_values) + assert len(result.issues) == 1 + assert result.issues[0].field_name == 'extra' + + +# ============================================================================= +# detect_composition_cycles +# ============================================================================= + + +def _make_get_all_references( + graph: dict[str, set[str]], +) -> ...: + """Helper: build get_all_references from a simple adjacency dict.""" + + def get_all_references(name: str) -> set[str]: + return graph.get(name, set()) + + return get_all_references + + +class TestDetectCompositionCycles: + def test_no_cycle(self): + """No cycle returns None.""" + get_refs = _make_get_all_references( + { + 'a': {'b'}, + 'b': {'c'}, + 'c': set(), + } + ) + result = detect_composition_cycles('a', {'b'}, get_refs) + assert result is None + + def test_direct_self_reference(self): + """A references itself.""" + get_refs = _make_get_all_references( + { + 'a': set(), + } + ) + result = detect_composition_cycles('a', {'a'}, get_refs) + assert result is not None + assert result[0] == 'a' + assert result[-1] == 'a' + + def test_a_b_a_cycle(self): + """A -> B -> A cycle.""" + get_refs = _make_get_all_references( + { + 'a': set(), # a's current refs don't matter; new_references is what we're adding + 'b': {'a'}, # b currently references a + } + ) + result = detect_composition_cycles('a', {'b'}, get_refs) + assert result is not None + assert result == ['a', 'b', 'a'] + + def test_a_b_c_a_cycle(self): + """A -> B -> C -> A cycle.""" + get_refs = _make_get_all_references( + { + 'b': {'c'}, + 'c': {'a'}, + } + ) + result = detect_composition_cycles('a', {'b'}, get_refs) + assert result is not None + assert result == ['a', 'b', 'c', 'a'] + + def test_diamond_no_cycle(self): + """Diamond shape (A->B, A->C, B->D, C->D) has no cycle.""" + get_refs = _make_get_all_references( + { + 'b': {'d'}, + 'c': {'d'}, + 'd': set(), + } + ) + result = detect_composition_cycles('a', {'b', 'c'}, get_refs) + assert result is None + + def test_empty_new_references(self): + """No new references means no cycle.""" + get_refs = _make_get_all_references({}) + result = detect_composition_cycles('a', set(), get_refs) + assert result is None + + def test_long_chain_no_cycle(self): + """Long chain without cycle returns None.""" + get_refs = _make_get_all_references( + { + 'b': {'c'}, + 'c': {'d'}, + 'd': {'e'}, + 'e': set(), + } + ) + result = detect_composition_cycles('a', {'b'}, get_refs) + assert result is None + + def test_cycle_path_deterministic(self): + """Cycle detection is deterministic (sorted references).""" + get_refs = _make_get_all_references( + { + 'b': {'a'}, + } + ) + result1 = detect_composition_cycles('a', {'b'}, get_refs) + result2 = detect_composition_cycles('a', {'b'}, get_refs) + assert result1 == result2 + + def test_multiple_new_refs_one_cycles(self): + """Multiple new_references, only one causes a cycle — cycle is detected.""" + get_refs = _make_get_all_references( + { + 'b': set(), + 'c': {'a'}, + } + ) + result = detect_composition_cycles('a', {'b', 'c'}, get_refs) + assert result is not None + assert result[-1] == 'a' + + def test_reference_to_unknown_variable(self): + """Referencing an unknown variable (no entries in graph) — no cycle.""" + get_refs = _make_get_all_references({}) + result = detect_composition_cycles('a', {'unknown'}, get_refs) + assert result is None + + +# ============================================================================= +# Dataclass tests +# ============================================================================= + + +class TestTemplateFieldIssue: + def test_attributes(self): + issue = TemplateFieldIssue( + field_name='user_name', + found_in_variable='prompt', + found_in_label='production', + reference_path=['snippet', 'prompt'], + ) + assert issue.field_name == 'user_name' + assert issue.found_in_variable == 'prompt' + assert issue.found_in_label == 'production' + assert issue.reference_path == ['snippet', 'prompt'] + + def test_none_label(self): + issue = TemplateFieldIssue( + field_name='x', + found_in_variable='v', + found_in_label=None, + reference_path=[], + ) + assert issue.found_in_label is None + + +class TestTemplateValidationResult: + def test_default_empty_issues(self): + result = TemplateValidationResult() + assert result.issues == [] + + def test_with_issues(self): + issue = TemplateFieldIssue( + field_name='x', + found_in_variable='v', + found_in_label=None, + reference_path=[], + ) + result = TemplateValidationResult(issues=[issue]) + assert len(result.issues) == 1 diff --git a/tests/test_variable_composition.py b/tests/test_variable_composition.py new file mode 100644 index 000000000..cccdd4d3a --- /dev/null +++ b/tests/test_variable_composition.py @@ -0,0 +1,705 @@ +"""Tests for variable composition (<> reference expansion).""" + +# pyright: reportPrivateUsage=false + +from __future__ import annotations + +import json +from typing import Any + +import pytest +from pydantic import BaseModel + +import logfire +from logfire._internal.config import LocalVariablesOptions +from logfire.testing import TestExporter +from logfire.variables.composition import ( + VariableCompositionCycleError, + VariableCompositionError, + expand_references, + find_references, +) +from logfire.variables.config import ( + LabeledValue, + LabelRef, + LatestVersion, + Rollout, + VariableConfig, + VariablesConfig, +) + +# ============================================================================= +# Tests for the pure composition functions (expand_references, find_references) +# ============================================================================= + + +def _make_resolve_fn( + variables: dict[str, str | None], +) -> Any: + """Create a resolve_fn from a simple name->serialized_value dict.""" + + def resolve_fn(ref_name: str) -> tuple[str | None, str | None, int | None, str]: + if ref_name in variables: + value = variables[ref_name] + if value is None: + return (None, None, None, 'unrecognized_variable') + return (value, 'production', 1, 'resolved') + return (None, None, None, 'unrecognized_variable') + + return resolve_fn + + +class TestExpandReferences: + def test_no_references(self): + """Values without <<>> are returned unchanged.""" + resolve_fn = _make_resolve_fn({}) + expanded, composed = expand_references('"hello world"', 'my_var', resolve_fn) + assert expanded == '"hello world"' + assert composed == [] + + def test_simple_string_reference(self): + """Simple <> expands to the referenced string value.""" + resolve_fn = _make_resolve_fn({'greeting': '"Hello"'}) + expanded, composed = expand_references('"<> World"', 'my_var', resolve_fn) + assert expanded == '"Hello World"' + assert len(composed) == 1 + assert composed[0].name == 'greeting' + assert composed[0].value == 'Hello' + assert composed[0].label == 'production' + assert composed[0].version == 1 + assert composed[0].reason == 'resolved' + assert composed[0].error is None + + def test_multiple_references(self): + """Multiple <> in one value are all expanded.""" + resolve_fn = _make_resolve_fn( + { + 'greeting': '"Hello"', + 'name': '"World"', + } + ) + expanded, composed = expand_references('"<> <>!"', 'my_var', resolve_fn) + assert expanded == '"Hello World!"' + assert len(composed) == 2 + assert composed[0].name == 'greeting' + assert composed[1].name == 'name' + + def test_same_reference_multiple_times(self): + """The same <> used multiple times expands each occurrence.""" + resolve_fn = _make_resolve_fn({'word': '"echo"'}) + expanded, composed = expand_references('"<> <>"', 'my_var', resolve_fn) + assert expanded == '"echo echo"' + # Handlebars resolves all occurrences in one pass, so only one ComposedReference + assert len(composed) == 1 + assert composed[0].name == 'word' + + def test_nested_references(self): + """References within referenced values are expanded recursively.""" + resolve_fn = _make_resolve_fn( + { + 'a': '"Hello <>"', + 'b': '"World"', + } + ) + expanded, composed = expand_references('"<>!"', 'my_var', resolve_fn) + assert expanded == '"Hello World!"' + assert len(composed) == 1 + assert composed[0].name == 'a' + assert composed[0].value == 'Hello World' + assert len(composed[0].composed_from) == 1 + assert composed[0].composed_from[0].name == 'b' + + def test_cycle_detection(self): + """Circular references are caught and the reference is left unexpanded.""" + resolve_fn = _make_resolve_fn( + { + 'a': '"<>"', + 'b': '"<>"', + } + ) + # The cycle is caught inside expand_references; b tries to expand a + # which is already in the visited set. + _, composed = expand_references('"<>"', 'my_var', resolve_fn) + # a expands, but when b tries to expand <>, it hits the cycle. + # b is successfully resolved but its nested ref to a fails (cycle). + assert len(composed) == 1 + assert composed[0].name == 'a' + # b resolved but a inside b failed with cycle error + assert len(composed[0].composed_from) == 1 + b_ref = composed[0].composed_from[0] + assert b_ref.name == 'b' + # b itself resolved, but its expansion of <> failed + assert len(b_ref.composed_from) == 1 + assert b_ref.composed_from[0].name == 'a' + assert b_ref.composed_from[0].error is not None + assert 'Circular reference' in b_ref.composed_from[0].error + + def test_self_reference_cycle(self): + """A variable referencing itself is caught.""" + resolve_fn = _make_resolve_fn({'a': '"<>"'}) + # my_var references a, a references itself + _, composed = expand_references('"<>"', 'my_var', resolve_fn) + assert len(composed) == 1 + assert composed[0].name == 'a' + # a resolved, but its self-reference <> failed with cycle + assert len(composed[0].composed_from) == 1 + assert composed[0].composed_from[0].name == 'a' + assert composed[0].composed_from[0].error is not None + assert 'Circular reference' in composed[0].composed_from[0].error + + def test_depth_limit(self): + """Chains exceeding MAX_COMPOSITION_DEPTH are caught.""" + # Build a chain: var_0 -> var_1 -> var_2 -> ... -> var_21 + variables: dict[str, str | None] = {} + for i in range(22): + if i < 21: + variables[f'var_{i}'] = f'"<>"' + else: + variables[f'var_{i}'] = '"end"' + resolve_fn = _make_resolve_fn(variables) + _, composed = expand_references('"<>"', 'my_var', resolve_fn) + # Should have error about depth limit somewhere in the chain + assert len(composed) == 1 + + # Walk down the chain to find the depth error + ref = composed[0] + depth_error_found = False + while ref.composed_from: + if ref.error and 'Maximum composition depth' in ref.error: + depth_error_found = True + break + ref = ref.composed_from[0] + if not depth_error_found and ref.error: + depth_error_found = 'Maximum composition depth' in ref.error + assert depth_error_found, 'Expected depth limit error somewhere in the chain' + + def test_unresolvable_reference(self): + """References to non-existent variables are left unexpanded.""" + resolve_fn = _make_resolve_fn({}) + expanded, composed = expand_references('"Hello <>"', 'my_var', resolve_fn) + assert expanded == '"Hello <>"' + assert len(composed) == 1 + assert composed[0].name == 'nonexistent' + assert composed[0].value is None + assert composed[0].reason == 'unrecognized_variable' + + def test_none_value_reference(self): + """References to variables with None value are left unexpanded.""" + resolve_fn = _make_resolve_fn({'missing': None}) + expanded, composed = expand_references('"Hello <>"', 'my_var', resolve_fn) + assert expanded == '"Hello <>"' + assert len(composed) == 1 + assert composed[0].value is None + + def test_non_string_reference(self): + """Non-string variables (numbers) are rendered via Handlebars toString.""" + resolve_fn = _make_resolve_fn({'number': '42'}) + expanded, composed = expand_references('"Value: <>"', 'my_var', resolve_fn) + assert expanded == '"Value: 42"' + assert len(composed) == 1 + assert composed[0].error is None + + def test_boolean_reference(self): + """Boolean variables are rendered via Handlebars toString.""" + resolve_fn = _make_resolve_fn({'flag': 'true'}) + expanded, composed = expand_references('"Flag: <>"', 'my_var', resolve_fn) + assert expanded == '"Flag: true"' + assert len(composed) == 1 + assert composed[0].error is None + + def test_object_reference(self): + """Object variables are available in the Handlebars context.""" + resolve_fn = _make_resolve_fn({'obj': '{"key": "value"}'}) + expanded, composed = expand_references('"Data: <>"', 'my_var', resolve_fn) + # Handlebars renders objects via toString — typically [object Object] or similar + result = json.loads(expanded) + assert 'Data:' in result + assert len(composed) == 1 + assert composed[0].error is None + + def test_structured_type_with_references(self): + """References inside JSON string values of structured types expand correctly.""" + resolve_fn = _make_resolve_fn({'safety': '"Be safe."'}) + serialized = json.dumps({'prompt': '<> Always.', 'model': 'gpt-4'}) + expanded, composed = expand_references(serialized, 'my_var', resolve_fn) + parsed = json.loads(expanded) + assert parsed['prompt'] == 'Be safe. Always.' + assert parsed['model'] == 'gpt-4' + assert len(composed) == 1 + assert composed[0].name == 'safety' + + def test_json_encoding_newlines(self): + """Newlines in referenced values are properly JSON-escaped.""" + resolve_fn = _make_resolve_fn({'multi': '"Line1\\nLine2"'}) + expanded, _ = expand_references('"Before <> After"', 'my_var', resolve_fn) + result = json.loads(expanded) + assert result == 'Before Line1\nLine2 After' + + def test_json_encoding_quotes(self): + """Quotes in referenced values are properly JSON-escaped.""" + resolve_fn = _make_resolve_fn({'quoted': '"She said \\"hello\\""'}) + expanded, _ = expand_references('"<>!"', 'my_var', resolve_fn) + result = json.loads(expanded) + assert result == 'She said "hello"!' + + def test_json_encoding_unicode(self): + """Unicode in referenced values works correctly.""" + resolve_fn = _make_resolve_fn({'emoji': json.dumps('Hello 🌍')}) + expanded, _ = expand_references('"<>!"', 'my_var', resolve_fn) + result = json.loads(expanded) + assert result == 'Hello 🌍!' + + def test_json_encoding_backslashes(self): + """Backslashes in referenced values are properly JSON-escaped.""" + resolve_fn = _make_resolve_fn({'path': json.dumps('C:\\Users\\test')}) + expanded, _ = expand_references('"Path: <>"', 'my_var', resolve_fn) + result = json.loads(expanded) + assert result == 'Path: C:\\Users\\test' + + def test_escape_sequence(self): + r"""Escaped \<< is converted to literal <<. + + In serialized JSON, a literal backslash before << is encoded as \\<<. + The regex lookbehind prevents matching, and post-processing converts \<< to <<. + """ + resolve_fn = _make_resolve_fn({'ref': '"expanded"'}) + # Build a JSON string that contains: not \<> but <> + # In JSON encoding, backslash must be \\, so the raw JSON is: + # "not \\<> but <>" + raw_python_str = 'not \\<> but <>' + serialized = json.dumps(raw_python_str) + # serialized is: "not \\<> but <>" + expanded, composed = expand_references(serialized, 'my_var', resolve_fn) + result = json.loads(expanded) + assert result == 'not <> but expanded' + # Only the real ref (second one) is in composed + assert len(composed) == 1 + assert composed[0].name == 'ref' + + def test_escape_only(self): + r"""Only escaped references, no real references.""" + resolve_fn = _make_resolve_fn({}) + raw_python_str = 'literal \\<>' + serialized = json.dumps(raw_python_str) + expanded, composed = expand_references(serialized, 'my_var', resolve_fn) + result = json.loads(expanded) + assert result == 'literal <>' + assert composed == [] + + def test_invalid_json_reference(self): + """References to values with invalid JSON are left unexpanded.""" + resolve_fn = _make_resolve_fn({'bad': 'not json at all'}) + expanded, composed = expand_references('"<>"', 'my_var', resolve_fn) + assert expanded == '"<>"' + assert len(composed) == 1 + assert composed[0].error is not None + assert 'non-JSON' in composed[0].error + + +class TestFindReferences: + def test_no_references(self): + assert find_references('"hello world"') == [] + + def test_single_reference(self): + assert find_references('"<>"') == ['greeting'] + + def test_multiple_unique_references(self): + assert find_references('"<> <> <>"') == ['a', 'b', 'c'] + + def test_duplicate_references(self): + """Duplicates are deduplicated, order preserved.""" + assert find_references('"<> <> <>"') == ['a', 'b'] + + def test_escaped_not_matched(self): + assert find_references(r'"\\<>"') == [] + + def test_mixed_escaped_and_real(self): + result = find_references(r'"\\<> <>"') + assert result == ['real'] + + def test_in_structured_json(self): + serialized = json.dumps({'prompt': '<>', 'other': '<>'}) + assert find_references(serialized) == ['safety', 'format'] + + def test_find_references_block_helpers(self): + """find_references detects variable names from block helper syntax.""" + serialized = json.dumps('<<#if brand>>show<>hide<>') + result = find_references(serialized) + assert 'brand' in result + + def test_find_references_block_and_simple(self): + """find_references finds both simple and block-helper references.""" + serialized = json.dumps('<> <<#if flag>>yes<>') + result = find_references(serialized) + assert 'greeting' in result + assert 'flag' in result + + +# ============================================================================= +# Tests for Handlebars-powered <<>> block helpers +# ============================================================================= + + +class TestBlockHelpers: + def test_block_if_true(self): + """<<#if flag>>yes<>no<> with truthy flag.""" + resolve_fn = _make_resolve_fn({'flag': 'true'}) + expanded, composed = expand_references('"<<#if flag>>yes<>no<>"', 'my_var', resolve_fn) + assert json.loads(expanded) == 'yes' + assert len(composed) == 1 + assert composed[0].name == 'flag' + + def test_block_if_false(self): + """<<#if flag>>yes<>no<> with falsy flag.""" + resolve_fn = _make_resolve_fn({'flag': 'false'}) + expanded, composed = expand_references('"<<#if flag>>yes<>no<>"', 'my_var', resolve_fn) + assert json.loads(expanded) == 'no' + assert len(composed) == 1 + assert composed[0].name == 'flag' + + def test_block_each(self): + """<<#each items>>- <><> iterates over a list.""" + resolve_fn = _make_resolve_fn({'items': '["a", "b", "c"]'}) + expanded, composed = expand_references('"<<#each items>><> <>"', 'my_var', resolve_fn) + result = json.loads(expanded) + assert result == 'a b c ' + assert len(composed) == 1 + assert composed[0].name == 'items' + + def test_block_unless(self): + """<<#unless flag>>shown<> with falsy flag.""" + resolve_fn = _make_resolve_fn({'flag': 'false'}) + expanded, _ = expand_references('"<<#unless flag>>shown<>"', 'my_var', resolve_fn) + assert json.loads(expanded) == 'shown' + + def test_block_unless_truthy(self): + """<<#unless flag>>shown<> with truthy flag shows nothing.""" + resolve_fn = _make_resolve_fn({'flag': 'true'}) + expanded, _ = expand_references('"<<#unless flag>>shown<>"', 'my_var', resolve_fn) + assert json.loads(expanded) == '' + + def test_block_with(self): + """<<#with config>><><> accesses nested fields.""" + resolve_fn = _make_resolve_fn({'config': '{"name": "acme"}'}) + expanded, _ = expand_references('"<<#with config>><><>"', 'my_var', resolve_fn) + assert json.loads(expanded) == 'acme' + + def test_block_if_with_composition(self): + """<<#if brand>><><> — conditional with dotted access.""" + resolve_fn = _make_resolve_fn({'brand': '{"tagline": "Build faster"}'}) + expanded, _ = expand_references('"<<#if brand>><><>"', 'my_var', resolve_fn) + assert json.loads(expanded) == 'Build faster' + + def test_mixed_angle_and_curly_preserved(self): + """<> expands, {{user.name}} is preserved for later rendering.""" + resolve_fn = _make_resolve_fn({'greeting': '"Hello"'}) + expanded, _ = expand_references('"<> {{user.name}}"', 'my_var', resolve_fn) + assert json.loads(expanded) == 'Hello {{user.name}}' + + def test_escape_angle_bracket(self): + r"""Escaped \<> becomes literal <> in output.""" + resolve_fn = _make_resolve_fn({'ref': '"expanded"'}) + raw_python_str = '\\<>' + serialized = json.dumps(raw_python_str) + expanded, _ = expand_references(serialized, 'my_var', resolve_fn) + result = json.loads(expanded) + assert result == '<>' + + def test_escape_mixed(self): + r"""Escaped \<> stays literal, real <> expands.""" + resolve_fn = _make_resolve_fn({'escaped': '"X"', 'real': '"expanded"'}) + raw_python_str = '\\<> <>' + serialized = json.dumps(raw_python_str) + expanded, _ = expand_references(serialized, 'my_var', resolve_fn) + result = json.loads(expanded) + assert result == '<> expanded' + + +# ============================================================================= +# Integration tests using LocalVariableProvider +# ============================================================================= + + +def _make_variables_config(**variables: str | None) -> VariablesConfig: + """Helper to create a VariablesConfig with simple string variables. + + Each kwarg is name=serialized_value (JSON string). + """ + configs: dict[str, VariableConfig] = {} + for name, value in variables.items(): + labels: dict[str, LabeledValue | LabelRef] = {} + latest: LatestVersion | None = None + if value is not None: + labels['production'] = LabeledValue(version=1, serialized_value=value) + latest = LatestVersion(version=1, serialized_value=value) + configs[name] = VariableConfig( + name=name, + json_schema={'type': 'string'} if value is not None and value.startswith('"') else None, + labels=labels, + rollout=Rollout(labels={'production': 1.0}) if value is not None else Rollout(labels={}), + overrides=[], + latest_version=latest, + ) + return VariablesConfig(variables=configs) + + +class TestCompositionIntegration: + def test_simple_reference(self, config_kwargs: dict[str, Any]): + """End-to-end: variable with <> is resolved with composition.""" + variables_config = _make_variables_config( + greeting='"Hello"', + main='"<> World"', + ) + config_kwargs['variables'] = LocalVariablesOptions(config=variables_config) + lf = logfire.configure(**config_kwargs) + + var = lf.var(name='main', default='fallback', type=str) + result = var.get() + assert result.value == 'Hello World' + assert len(result.composed_from) == 1 + assert result.composed_from[0].name == 'greeting' + assert result.composed_from[0].value == 'Hello' + + def test_nested_reference(self, config_kwargs: dict[str, Any]): + """A→B→C chain resolves fully.""" + variables_config = _make_variables_config( + c='"end"', + b='"<>_b"', + a='"<>_a"', + ) + config_kwargs['variables'] = LocalVariablesOptions(config=variables_config) + lf = logfire.configure(**config_kwargs) + + var = lf.var(name='a', default='fallback', type=str) + result = var.get() + assert result.value == 'end_b_a' + assert len(result.composed_from) == 1 + assert result.composed_from[0].name == 'b' + assert result.composed_from[0].composed_from[0].name == 'c' + + def test_cycle_falls_back_gracefully(self, config_kwargs: dict[str, Any]): + """Cycles in references cause graceful fallback to default.""" + variables_config = _make_variables_config( + a='"<>"', + b='"<>"', + ) + config_kwargs['variables'] = LocalVariablesOptions(config=variables_config) + lf = logfire.configure(**config_kwargs) + + var = lf.var(name='a', default='fallback', type=str) + result = var.get() + # The cycle in b trying to reference a (which is in the visited set) means + # b's expansion fails, b is left as <> inside a's value. + # So a's value becomes "<>" (the literal unexpanded ref from b's failed expansion). + # Actually the value should still deserialize as a string, just with unexpanded refs. + assert isinstance(result.value, str) + + def test_nonexistent_reference_left_unexpanded(self, config_kwargs: dict[str, Any]): + """References to non-existent variables are left as-is.""" + variables_config = _make_variables_config( + main='"Hello <>"', + ) + config_kwargs['variables'] = LocalVariablesOptions(config=variables_config) + lf = logfire.configure(**config_kwargs) + + var = lf.var(name='main', default='fallback', type=str) + result = var.get() + assert result.value == 'Hello <>' + + def test_non_string_reference_expanded(self, config_kwargs: dict[str, Any]): + """Non-string variables are now expanded via Handlebars.""" + # Create a variable config with a non-string variable + configs: dict[str, VariableConfig] = { + 'number': VariableConfig( + name='number', + json_schema={'type': 'integer'}, + labels={'production': LabeledValue(version=1, serialized_value='42')}, + rollout=Rollout(labels={'production': 1.0}), + overrides=[], + latest_version=LatestVersion(version=1, serialized_value='42'), + ), + 'main': VariableConfig( + name='main', + json_schema={'type': 'string'}, + labels={'production': LabeledValue(version=1, serialized_value='"Value: <>"')}, + rollout=Rollout(labels={'production': 1.0}), + overrides=[], + latest_version=LatestVersion(version=1, serialized_value='"Value: <>"'), + ), + } + variables_config = VariablesConfig(variables=configs) + config_kwargs['variables'] = LocalVariablesOptions(config=variables_config) + lf = logfire.configure(**config_kwargs) + + var = lf.var(name='main', default='fallback', type=str) + result = var.get() + assert result.value == 'Value: 42' + + def test_structured_type_composition(self, config_kwargs: dict[str, Any]): + """Composition works in string fields of Pydantic models.""" + + class AgentConfig(BaseModel): + prompt: str + model: str + + safety_value = json.dumps('Be safe.') + agent_value = json.dumps({'prompt': '<> Always.', 'model': 'gpt-4'}) + + configs: dict[str, VariableConfig] = { + 'safety': VariableConfig( + name='safety', + json_schema={'type': 'string'}, + labels={'production': LabeledValue(version=1, serialized_value=safety_value)}, + rollout=Rollout(labels={'production': 1.0}), + overrides=[], + latest_version=LatestVersion(version=1, serialized_value=safety_value), + ), + 'agent_config': VariableConfig( + name='agent_config', + json_schema=None, + labels={'production': LabeledValue(version=1, serialized_value=agent_value)}, + rollout=Rollout(labels={'production': 1.0}), + overrides=[], + latest_version=LatestVersion(version=1, serialized_value=agent_value), + ), + } + variables_config = VariablesConfig(variables=configs) + config_kwargs['variables'] = LocalVariablesOptions(config=variables_config) + lf = logfire.configure(**config_kwargs) + + var = lf.var(name='agent_config', default=AgentConfig(prompt='default', model='default'), type=AgentConfig) + result = var.get() + assert result.value.prompt == 'Be safe. Always.' + assert result.value.model == 'gpt-4' + assert len(result.composed_from) == 1 + assert result.composed_from[0].name == 'safety' + + def test_no_composition_for_context_override(self, config_kwargs: dict[str, Any]): + """Context overrides return typed values directly, no composition.""" + variables_config = _make_variables_config( + greeting='"Hello"', + main='"<> World"', + ) + config_kwargs['variables'] = LocalVariablesOptions(config=variables_config) + lf = logfire.configure(**config_kwargs) + + var = lf.var(name='main', default='fallback', type=str) + with var.override('override_value'): + result = var.get() + assert result.value == 'override_value' + assert result.composed_from == [] + assert result._reason == 'context_override' + + def test_composition_with_explicit_label(self, config_kwargs: dict[str, Any]): + """Composition works when using explicit label parameter.""" + variables_config = _make_variables_config( + greeting='"Hello"', + main='"<> World"', + ) + config_kwargs['variables'] = LocalVariablesOptions(config=variables_config) + lf = logfire.configure(**config_kwargs) + + var = lf.var(name='main', default='fallback', type=str) + result = var.get(label='production') + assert result.value == 'Hello World' + assert len(result.composed_from) == 1 + + def test_span_attributes_with_composition(self, config_kwargs: dict[str, Any], exporter: TestExporter): + """Span attributes include composed_from when composition occurs.""" + variables_config = _make_variables_config( + greeting='"Hello"', + main='"<> World"', + ) + config_kwargs['variables'] = LocalVariablesOptions(config=variables_config, instrument=True) + lf = logfire.configure(**config_kwargs) + + var = lf.var(name='main', default='fallback', type=str) + exporter.clear() + + result = var.get() + assert result.value == 'Hello World' + + # Find the completed span for 'main' variable resolution (last one with this name) + spans = exporter.exported_spans + resolve_spans = [s for s in spans if s.name == 'Resolve variable main'] + main_span = resolve_spans[-1] # last = completed span + attrs = dict(main_span.attributes or {}) + + # Check composed_from attribute + composed_from_json = attrs.get('composed_from') + assert isinstance(composed_from_json, str) + composed_data = json.loads(composed_from_json) + assert len(composed_data) == 1 + assert composed_data[0]['name'] == 'greeting' + assert composed_data[0]['version'] == 1 + assert composed_data[0]['label'] == 'production' + + def test_span_attributes_without_composition(self, config_kwargs: dict[str, Any], exporter: TestExporter): + """Span attributes do NOT include composed_from when no composition occurs.""" + variables_config = _make_variables_config( + main='"no refs here"', + ) + config_kwargs['variables'] = LocalVariablesOptions(config=variables_config, instrument=True) + lf = logfire.configure(**config_kwargs) + + var = lf.var(name='main', default='fallback', type=str) + exporter.clear() + + var.get() + + # Find the completed span for 'main' variable resolution (last one with this name) + spans = exporter.exported_spans + resolve_spans = [s for s in spans if s.name == 'Resolve variable main'] + main_span = resolve_spans[-1] # last = completed span + attrs = dict(main_span.attributes or {}) + assert 'composed_from' not in attrs + + def test_no_value_no_composition(self, config_kwargs: dict[str, Any]): + """When variable resolves to None (code default), no composition happens.""" + variables_config = VariablesConfig( + variables={ + 'main': VariableConfig( + name='main', + json_schema={'type': 'string'}, + labels={}, + rollout=Rollout(labels={}), + overrides=[], + ), + } + ) + config_kwargs['variables'] = LocalVariablesOptions(config=variables_config) + lf = logfire.configure(**config_kwargs) + + var = lf.var(name='main', default='<> fallback', type=str) + result = var.get() + # Default is returned as-is (no composition on defaults) + assert result.value == '<> fallback' + assert result.composed_from == [] + + +class TestCompositionExceptions: + """Test the exception hierarchy.""" + + def test_composition_error_is_exception(self): + assert issubclass(VariableCompositionError, Exception) + + def test_cycle_error_is_composition_error(self): + assert issubclass(VariableCompositionCycleError, VariableCompositionError) + + def test_direct_cycle_error(self): + with pytest.raises(VariableCompositionCycleError, match='Circular reference'): + expand_references( + '"test"', + 'a', + _make_resolve_fn({}), + _visited=frozenset({'a'}), + ) + + def test_direct_depth_error(self): + with pytest.raises(VariableCompositionError, match='Maximum composition depth'): + expand_references( + '"test"', + 'a', + _make_resolve_fn({}), + _depth=21, + ) diff --git a/tests/test_variable_templates.py b/tests/test_variable_templates.py new file mode 100644 index 000000000..807da7c65 --- /dev/null +++ b/tests/test_variable_templates.py @@ -0,0 +1,554 @@ +"""Tests for variable template rendering (Handlebars {{placeholder}} support).""" + +# pyright: reportPrivateUsage=false + +from __future__ import annotations + +import json +from typing import Any + +import pytest +from pydantic import BaseModel + +import logfire +from logfire._internal.config import LocalVariablesOptions +from logfire.variables.config import ( + LabeledValue, + Rollout, + VariableConfig, + VariablesConfig, +) + + +def _make_lf(variables_config: VariablesConfig, config_kwargs: dict[str, Any]) -> logfire.Logfire: + """Create a Logfire instance with LocalVariablesOptions for testing.""" + config_kwargs['variables'] = LocalVariablesOptions(config=variables_config) + return logfire.configure(**config_kwargs) + + +def _simple_config(name: str, serialized_value: str) -> VariablesConfig: + """Create a minimal VariablesConfig with one variable and one label.""" + return VariablesConfig( + variables={ + name: VariableConfig( + name=name, + labels={'production': LabeledValue(version=1, serialized_value=serialized_value)}, + rollout=Rollout(labels={'production': 1.0}), + overrides=[], + ), + }, + ) + + +# ============================================================================= +# ResolvedVariable.render() tests +# ============================================================================= + + +class TestRenderSimpleString: + """Test rendering string variables with Handlebars templates.""" + + def test_simple_placeholder(self, config_kwargs: dict[str, Any]): + """Simple {{placeholder}} replacement in a string variable.""" + lf = _make_lf(_simple_config('greeting', json.dumps('Hello {{name}}!')), config_kwargs) + var = lf.var('greeting', type=str, default='default') + resolved = var.get() + assert resolved.value == 'Hello {{name}}!' + rendered = resolved.render({'name': 'Alice'}) + assert rendered == 'Hello Alice!' + + def test_multiple_placeholders(self, config_kwargs: dict[str, Any]): + """Multiple {{placeholders}} in a single string.""" + lf = _make_lf( + _simple_config('prompt', json.dumps('Hello {{user_name}}, welcome to {{company}}!')), + config_kwargs, + ) + var = lf.var('prompt', type=str, default='default') + resolved = var.get() + rendered = resolved.render({'user_name': 'Bob', 'company': 'Acme'}) + assert rendered == 'Hello Bob, welcome to Acme!' + + def test_conditional_template(self, config_kwargs: dict[str, Any]): + """Handlebars #if conditional in a string variable.""" + lf = _make_lf( + _simple_config('prompt', json.dumps('Hello {{name}}.{{#if is_premium}} Premium member!{{/if}}')), + config_kwargs, + ) + var = lf.var('prompt', type=str, default='default') + resolved = var.get() + + rendered_premium = resolved.render({'name': 'Alice', 'is_premium': True}) + assert rendered_premium == 'Hello Alice. Premium member!' + + rendered_basic = resolved.render({'name': 'Bob', 'is_premium': False}) + assert rendered_basic == 'Hello Bob.' + + def test_each_helper(self, config_kwargs: dict[str, Any]): + """Handlebars #each iteration in a string variable.""" + lf = _make_lf( + _simple_config( + 'prompt', + json.dumps('Items: {{#each items}}{{this}}{{#unless @last}}, {{/unless}}{{/each}}'), + ), + config_kwargs, + ) + var = lf.var('prompt', type=str, default='default') + resolved = var.get() + rendered = resolved.render({'items': ['apple', 'banana', 'cherry']}) + assert rendered == 'Items: apple, banana, cherry' + + def test_no_html_escaping(self, config_kwargs: dict[str, Any]): + """String values should NOT be HTML-escaped (not an HTML context).""" + lf = _make_lf(_simple_config('prompt', json.dumps('Value: {{value}}')), config_kwargs) + var = lf.var('prompt', type=str, default='default') + resolved = var.get() + # These characters would normally be HTML-escaped by Handlebars + rendered = resolved.render({'value': ''}) + assert rendered == 'Value: ' + + def test_empty_context(self, config_kwargs: dict[str, Any]): + """Rendering with no inputs leaves placeholders as empty strings.""" + lf = _make_lf(_simple_config('prompt', json.dumps('Hello {{name}}!')), config_kwargs) + var = lf.var('prompt', type=str, default='default') + resolved = var.get() + rendered = resolved.render() + assert rendered == 'Hello !' + + def test_no_templates(self, config_kwargs: dict[str, Any]): + """Rendering a value with no {{placeholders}} returns the value unchanged.""" + lf = _make_lf(_simple_config('prompt', json.dumps('Hello world!')), config_kwargs) + var = lf.var('prompt', type=str, default='default') + resolved = var.get() + rendered = resolved.render({'name': 'unused'}) + assert rendered == 'Hello world!' + + +class TestRenderWithPydanticInputs: + """Test rendering with Pydantic model inputs.""" + + def test_pydantic_model_inputs(self, config_kwargs: dict[str, Any]): + """Rendering with a Pydantic model as inputs.""" + + class PromptInputs(BaseModel): + user_name: str + is_premium: bool = False + + lf = _make_lf( + _simple_config('prompt', json.dumps('Welcome {{user_name}}!{{#if is_premium}} VIP{{/if}}')), + config_kwargs, + ) + var = lf.var('prompt', type=str, default='default', template_inputs=PromptInputs) + resolved = var.get() + rendered = resolved.render(PromptInputs(user_name='Alice', is_premium=True)) + assert rendered == 'Welcome Alice! VIP' + + def test_nested_model_inputs(self, config_kwargs: dict[str, Any]): + """Rendering with nested Pydantic model fields using dot notation.""" + + class Address(BaseModel): + city: str + country: str + + class UserInfo(BaseModel): + name: str + address: Address + + lf = _make_lf( + _simple_config('prompt', json.dumps('User {{name}} from {{address.city}}, {{address.country}}')), + config_kwargs, + ) + var = lf.var('prompt', type=str, default='default', template_inputs=UserInfo) + resolved = var.get() + rendered = resolved.render(UserInfo(name='Alice', address=Address(city='London', country='UK'))) + assert rendered == 'User Alice from London, UK' + + +class TestRenderStructuredType: + """Test rendering structured types (Pydantic models) where string fields contain templates.""" + + def test_model_with_template_fields(self, config_kwargs: dict[str, Any]): + """Rendering a Pydantic model where string fields contain {{placeholders}}.""" + + class PromptConfig(BaseModel): + system_prompt: str + temperature: float + max_tokens: int + + serialized = json.dumps( + { + 'system_prompt': 'Hello {{user_name}}, how can I help?', + 'temperature': 0.7, + 'max_tokens': 100, + } + ) + + lf = _make_lf(_simple_config('config', serialized), config_kwargs) + var = lf.var( + 'config', + type=PromptConfig, + default=PromptConfig(system_prompt='default', temperature=0.5, max_tokens=50), + ) + resolved = var.get() + rendered = resolved.render({'user_name': 'Alice'}) + assert isinstance(rendered, PromptConfig) + assert rendered.system_prompt == 'Hello Alice, how can I help?' + assert rendered.temperature == 0.7 + assert rendered.max_tokens == 100 + + +class TestRenderCodeDefault: + """Test rendering when using code default values (no remote configuration).""" + + def test_render_code_default_string(self, config_kwargs: dict[str, Any]): + """Rendering a code default string that contains templates.""" + config_kwargs['variables'] = LocalVariablesOptions(config=VariablesConfig(variables={})) + lf = logfire.configure(**config_kwargs) + var = lf.var('prompt', type=str, default='Hello {{name}}!') + resolved = var.get() + # Value is the code default + assert resolved.value == 'Hello {{name}}!' + # Rendering should still work + rendered = resolved.render({'name': 'Alice'}) + assert rendered == 'Hello Alice!' + + +class TestRenderErrors: + """Test error handling in render().""" + + def test_render_invalid_inputs_type(self, config_kwargs: dict[str, Any]): + """Passing a non-dict/non-model to render() raises TypeError.""" + lf = _make_lf(_simple_config('prompt', json.dumps('Hello {{name}}')), config_kwargs) + var = lf.var('prompt', type=str, default='default') + resolved = var.get() + with pytest.raises(TypeError, match='Expected a dict, Mapping, or Pydantic model'): + resolved.render(42) + + +# ============================================================================= +# template_inputs parameter tests +# ============================================================================= + + +class TestTemplateInputsParam: + """Test the template_inputs parameter on logfire.var().""" + + def test_template_inputs_schema_in_config(self, config_kwargs: dict[str, Any]): + """template_inputs generates JSON Schema in the variable config.""" + + class MyInputs(BaseModel): + user_name: str + count: int = 5 + + lf = logfire.configure(**config_kwargs) + var = lf.var('prompt', type=str, default='Hello {{user_name}}', template_inputs=MyInputs) + config = var.to_config() + assert config.template_inputs_schema is not None + assert config.template_inputs_schema['type'] == 'object' + assert 'user_name' in config.template_inputs_schema['properties'] + assert 'count' in config.template_inputs_schema['properties'] + + def test_no_template_inputs(self, config_kwargs: dict[str, Any]): + """Without template_inputs, schema is None.""" + lf = logfire.configure(**config_kwargs) + var = lf.var('prompt', type=str, default='Hello') + config = var.to_config() + assert config.template_inputs_schema is None + + def test_template_inputs_stored_on_variable(self, config_kwargs: dict[str, Any]): + """template_inputs_type is stored on the Variable instance.""" + + class MyInputs(BaseModel): + name: str + + lf = logfire.configure(**config_kwargs) + var = lf.var('prompt', type=str, default='Hello', template_inputs=MyInputs) + assert var.template_inputs_type is MyInputs + + +# ============================================================================= +# VariableConfig.template_inputs_schema tests +# ============================================================================= + + +class TestVariableConfigTemplateInputs: + """Test template_inputs_schema on VariableConfig.""" + + def test_round_trip_serialization(self): + """template_inputs_schema survives serialization/deserialization.""" + schema = {'type': 'object', 'properties': {'name': {'type': 'string'}}, 'required': ['name']} + config = VariableConfig( + name='test_var', + labels={}, + rollout=Rollout(labels={}), + overrides=[], + template_inputs_schema=schema, + ) + data = config.model_dump() + restored = VariableConfig.model_validate(data) + assert restored.template_inputs_schema == schema + + def test_none_by_default(self): + """template_inputs_schema defaults to None.""" + config = VariableConfig( + name='test_var', + labels={}, + rollout=Rollout(labels={}), + overrides=[], + ) + assert config.template_inputs_schema is None + + +# ============================================================================= +# Composition + rendering pipeline tests +# ============================================================================= + + +class TestCompositionThenRendering: + """Test the full pipeline: resolve → compose → render.""" + + def test_composition_then_render(self, config_kwargs: dict[str, Any]): + """<> are expanded first, then {{placeholders}} are rendered.""" + variables_config = VariablesConfig( + variables={ + 'snippet': VariableConfig( + name='snippet', + labels={ + 'production': LabeledValue(version=1, serialized_value=json.dumps('Welcome to {{company}}!')), + }, + rollout=Rollout(labels={'production': 1.0}), + overrides=[], + ), + 'full_prompt': VariableConfig( + name='full_prompt', + labels={ + 'production': LabeledValue( + version=1, + serialized_value=json.dumps('Hello {{user_name}}. <>'), + ), + }, + rollout=Rollout(labels={'production': 1.0}), + overrides=[], + ), + }, + ) + lf = _make_lf(variables_config, config_kwargs) + var = lf.var('full_prompt', type=str, default='default') + resolved = var.get() + # After composition, <> is expanded but {{placeholders}} remain + assert resolved.value == 'Hello {{user_name}}. Welcome to {{company}}!' + # After rendering, all {{placeholders}} are filled + rendered = resolved.render({'user_name': 'Alice', 'company': 'Acme Corp'}) + assert rendered == 'Hello Alice. Welcome to Acme Corp!' + + +# ============================================================================= +# TemplateVariable tests +# ============================================================================= + + +class TestTemplateVariable: + """Test TemplateVariable[T, InputsT] — single-step get(inputs) rendering.""" + + def test_basic_rendering(self, config_kwargs: dict[str, Any]): + """get(inputs) returns rendered value directly.""" + + class Inputs(BaseModel): + name: str + + lf = _make_lf(_simple_config('greeting', json.dumps('Hello {{name}}!')), config_kwargs) + var = lf.template_var('greeting', type=str, default='default', inputs_type=Inputs) + resolved = var.get(Inputs(name='Alice')) + assert resolved.value == 'Hello Alice!' + + def test_composition_then_render(self, config_kwargs: dict[str, Any]): + """<> expanded first, then {{}} rendered with inputs.""" + + class Inputs(BaseModel): + user_name: str + company: str + + variables_config = VariablesConfig( + variables={ + 'snippet': VariableConfig( + name='snippet', + labels={ + 'production': LabeledValue(version=1, serialized_value=json.dumps('Welcome to {{company}}!')), + }, + rollout=Rollout(labels={'production': 1.0}), + overrides=[], + ), + 'full_prompt': VariableConfig( + name='full_prompt', + labels={ + 'production': LabeledValue( + version=1, + serialized_value=json.dumps('Hello {{user_name}}. <>'), + ), + }, + rollout=Rollout(labels={'production': 1.0}), + overrides=[], + ), + }, + ) + lf = _make_lf(variables_config, config_kwargs) + var = lf.template_var('full_prompt', type=str, default='default', inputs_type=Inputs) + resolved = var.get(Inputs(user_name='Alice', company='Acme Corp')) + # Both composition AND rendering done in one step + assert resolved.value == 'Hello Alice. Welcome to Acme Corp!' + + def test_structured_type(self, config_kwargs: dict[str, Any]): + """Pydantic model with template fields renders correctly.""" + + class PromptConfig(BaseModel): + system_prompt: str + temperature: float + max_tokens: int + + class Inputs(BaseModel): + user_name: str + + serialized = json.dumps( + { + 'system_prompt': 'Hello {{user_name}}, how can I help?', + 'temperature': 0.7, + 'max_tokens': 100, + } + ) + + lf = _make_lf(_simple_config('config', serialized), config_kwargs) + var = lf.template_var( + 'config', + type=PromptConfig, + default=PromptConfig(system_prompt='default', temperature=0.5, max_tokens=50), + inputs_type=Inputs, + ) + resolved = var.get(Inputs(user_name='Alice')) + assert isinstance(resolved.value, PromptConfig) + assert resolved.value.system_prompt == 'Hello Alice, how can I help?' + assert resolved.value.temperature == 0.7 + assert resolved.value.max_tokens == 100 + + def test_default_rendering(self, config_kwargs: dict[str, Any]): + """Code default with {{}} templates is rendered.""" + + class Inputs(BaseModel): + name: str + + config_kwargs['variables'] = LocalVariablesOptions(config=VariablesConfig(variables={})) + lf = logfire.configure(**config_kwargs) + var = lf.template_var('prompt', type=str, default='Hello {{name}}!', inputs_type=Inputs) + resolved = var.get(Inputs(name='Alice')) + # The default value should be rendered with the inputs + assert resolved.value == 'Hello Alice!' + + def test_override_renders_template(self, config_kwargs: dict[str, Any]): + """override() overrides the template, which still gets rendered with inputs.""" + + class Inputs(BaseModel): + name: str + + lf = _make_lf(_simple_config('greeting', json.dumps('Hello {{name}}!')), config_kwargs) + var = lf.template_var('greeting', type=str, default='default', inputs_type=Inputs) + with var.override('Overridden {{name}}!'): + resolved = var.get(Inputs(name='Alice')) + # Override value is treated as a template and rendered + assert resolved.value == 'Overridden Alice!' + + def test_override_literal_string(self, config_kwargs: dict[str, Any]): + """override() with a literal string (no placeholders) works as a plain override.""" + + class Inputs(BaseModel): + name: str + + lf = _make_lf(_simple_config('greeting', json.dumps('Hello {{name}}!')), config_kwargs) + var = lf.template_var('greeting', type=str, default='default', inputs_type=Inputs) + with var.override('exact override value'): + resolved = var.get(Inputs(name='Alice')) + # No placeholders, so rendering is a no-op — value returned as-is + assert resolved.value == 'exact override value' + + def test_pydantic_model_inputs(self, config_kwargs: dict[str, Any]): + """InputsT as Pydantic BaseModel works correctly.""" + + class MyInputs(BaseModel): + user_name: str + is_premium: bool = False + + lf = _make_lf( + _simple_config('prompt', json.dumps('Welcome {{user_name}}!{{#if is_premium}} VIP{{/if}}')), + config_kwargs, + ) + var = lf.template_var('prompt', type=str, default='default', inputs_type=MyInputs) + + resolved = var.get(MyInputs(user_name='Alice', is_premium=True)) + assert resolved.value == 'Welcome Alice! VIP' + + resolved2 = var.get(MyInputs(user_name='Bob')) + assert resolved2.value == 'Welcome Bob!' + + def test_registration(self, config_kwargs: dict[str, Any]): + """template_var() registers in _variables.""" + + class Inputs(BaseModel): + x: str + + lf = logfire.configure(**config_kwargs) + lf.template_var('tv1', type=str, default='x', inputs_type=Inputs) + assert 'tv1' in {v.name for v in lf.variables_get()} + + def test_duplicate_name_error(self, config_kwargs: dict[str, Any]): + """Same name as existing var raises ValueError.""" + + class Inputs(BaseModel): + x: str + + lf = logfire.configure(**config_kwargs) + lf.var('myvar', type=str, default='x') + with pytest.raises(ValueError, match="A variable with name 'myvar' has already been registered"): + lf.template_var('myvar', type=str, default='x', inputs_type=Inputs) + + def test_context_manager(self, config_kwargs: dict[str, Any]): + """with template_var.get(inputs) as resolved: sets baggage.""" + + class Inputs(BaseModel): + name: str + + lf = _make_lf(_simple_config('prompt', json.dumps('Hello {{name}}!')), config_kwargs) + var = lf.template_var('prompt', type=str, default='default', inputs_type=Inputs) + with var.get(Inputs(name='Alice')) as resolved: + assert resolved.value == 'Hello Alice!' + baggage = logfire.get_baggage() + assert baggage.get('logfire.variables.prompt') == 'production' + + def test_no_templates_passthrough(self, config_kwargs: dict[str, Any]): + """Value with no {{}} returns as-is after rendering.""" + + class Inputs(BaseModel): + name: str + + lf = _make_lf(_simple_config('greeting', json.dumps('Hello world!')), config_kwargs) + var = lf.template_var('greeting', type=str, default='default', inputs_type=Inputs) + resolved = var.get(Inputs(name='unused')) + assert resolved.value == 'Hello world!' + + def test_template_inputs_schema_in_config(self, config_kwargs: dict[str, Any]): + """template_var() generates JSON Schema in the variable config.""" + + class MyInputs(BaseModel): + user_name: str + count: int = 5 + + lf = logfire.configure(**config_kwargs) + var = lf.template_var('prompt', type=str, default='Hello {{user_name}}', inputs_type=MyInputs) + config = var.to_config() + assert config.template_inputs_schema is not None + assert config.template_inputs_schema['type'] == 'object' + assert 'user_name' in config.template_inputs_schema['properties'] + assert 'count' in config.template_inputs_schema['properties'] + + def test_dict_inputs(self, config_kwargs: dict[str, Any]): + """Passing a dict as inputs works (via Mapping path).""" + lf = _make_lf(_simple_config('greeting', json.dumps('Hello {{name}}!')), config_kwargs) + var = lf.template_var('greeting', type=str, default='default', inputs_type=dict) + resolved = var.get({'name': 'Alice'}) + assert resolved.value == 'Hello Alice!' diff --git a/uv.lock b/uv.lock index 72d75cf4c..393e06b43 100644 --- a/uv.lock +++ b/uv.lock @@ -3320,6 +3320,7 @@ system-metrics = [ ] variables = [ { name = "pydantic" }, + { name = "pydantic-handlebars", marker = "python_full_version >= '3.10'" }, ] wsgi = [ { name = "opentelemetry-instrumentation-wsgi" }, @@ -3409,6 +3410,7 @@ dev = [ { name = "pydantic-ai-slim", version = "0.8.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, { name = "pydantic-ai-slim", version = "1.65.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, { name = "pydantic-evals", marker = "python_full_version >= '3.10'" }, + { name = "pydantic-handlebars", marker = "python_full_version >= '3.10'" }, { name = "pymongo" }, { name = "pymysql" }, { name = "pyright" }, @@ -3495,6 +3497,7 @@ requires-dist = [ { name = "pydantic", marker = "extra == 'datasets'", specifier = ">=2" }, { name = "pydantic", marker = "extra == 'variables'", specifier = ">=2" }, { name = "pydantic-evals", marker = "python_full_version >= '3.10' and extra == 'datasets'", specifier = ">=1.0.0" }, + { name = "pydantic-handlebars", marker = "python_full_version >= '3.10' and extra == 'variables'", specifier = ">=0.1.0" }, { name = "rich", specifier = ">=13.4.2" }, { name = "tomli", marker = "python_full_version < '3.11'", specifier = ">=2.0.1" }, { name = "typing-extensions", specifier = ">=4.1.0" }, @@ -3570,6 +3573,7 @@ dev = [ { name = "pydantic", specifier = ">=2.11.0" }, { name = "pydantic-ai-slim", specifier = ">=0.0.39" }, { name = "pydantic-evals", marker = "python_full_version >= '3.10'", specifier = ">=1.0.0" }, + { name = "pydantic-handlebars", marker = "python_full_version >= '3.10'", specifier = ">=0.1.0" }, { name = "pymongo", specifier = ">=4.10.1" }, { name = "pymysql", specifier = ">=1.1.1" }, { name = "pyright", specifier = "!=1.1.407" }, @@ -6837,6 +6841,18 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/4b/c2/275557630cdfa84cde67ee6e55d5e7f8e4e1450a61e78eb0f5c012f44937/pydantic_graph-1.65.0-py3-none-any.whl", hash = "sha256:e1645a18fefee7ae62698b637c8a239ebcd3a5fa125cadcf8424d54907dd7122", size = 72350, upload-time = "2026-03-03T23:46:05.445Z" }, ] +[[package]] +name = "pydantic-handlebars" +version = "0.1.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "pydantic", marker = "python_full_version >= '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/90/16/d41768bd3fd77e6250c20be11a3e68fee5fff07c3356455e6708f6a60f2a/pydantic_handlebars-0.1.0.tar.gz", hash = "sha256:1931c54946add1b5e3796c9bf6a005ed7662cef0109bb05c352f0b3d031a1260", size = 159826, upload-time = "2026-03-01T20:00:17.497Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/99/5f/86b1630be61bdebf253c2f953a6c3f073ec21bb0725565ea3896802e1ca3/pydantic_handlebars-0.1.0-py3-none-any.whl", hash = "sha256:8a436fe8bc607295eb04bec58bd6e2c9498c9e069c557ff0b505e3d568c783bc", size = 40890, upload-time = "2026-03-01T20:00:16.106Z" }, +] + [[package]] name = "pydantic-settings" version = "2.13.1"