L2: Prompt Evolution
Layer 2 operates at medium frequency to refine the agent's system prompts. It analyzes conversation quality metrics and proposes prompt modifications, testing them through an A/B framework before permanent adoption.
Overview
L2 evolution addresses:
- System prompt refinement -- improve instruction clarity and task coverage
- Persona tuning -- adjust tone, verbosity, and communication style
- Tool usage instructions -- optimize how tools are described to the LLM
- A/B testing -- statistically validate prompt changes before rollout
A/B Testing Framework
When a prompt modification is proposed, L2 runs both the original and modified prompts in parallel for a configurable evaluation period:
- Split traffic -- alternate between original and candidate prompts
- Collect metrics -- track task completion, user satisfaction, tool usage efficiency
- Statistical test -- apply significance testing to determine the winner
- Promote or rollback -- adopt the winner or keep the original
Configuration
toml
[self_evolution.l2]
enabled = false
schedule = "weekly"
min_samples = 50
confidence_level = 0.95
max_concurrent_experiments = 2