Optimization

Optimization improves a program without changing its code. You give an optimizer a program, a set of Examples, and a metric, and it searches for better few-shot demonstrations and instructions, then hands back a new program with the same type.

The common shape

Every optimizer follows the same pattern: construct it with its configuration, then call compile. The result is an OptimizationReport whose bestProgram is the improved program:

def optimize[P: {OptimizableStructure, ProgramRunner}](
    metric  : Metric,
    program : P,
    trainset: Vector[Example]
)(using RuntimeContext): Either[DspyError, P] =
  val teleprompter = BootstrapFewShotWithRandomSearch[P](RandomSearchConfig(
    metric = metric,
    maxBootstrappedDemos = DemoCount(4),
    maxLabeledDemos = DemoCount(4),
    numCandidates = SearchCandidateCount(10),
    numThreads = Some(ThreadCount(4))
  ))
  teleprompter.compile(program, trainset).map(_.bestProgram)

Optimizers are generic over the program type. OptimizableStructure[P] exposes each leaf's writable OptimizableParameters, while ProgramRunner[P] executes either domain-valued programs or the record-valued DynamicModule spine. The returned program can be saved and loaded, so optimization runs once and the result ships with your application.

The optimizers

dspy4s groups them by what they tune:

Optimizer	Tunes	Page
LabeledFewShot	Demonstrations (no model calls)	Few-shot
BootstrapFewShot	Demonstrations (self-generated)	Few-shot
BootstrapFewShotWithRandomSearch	Demonstrations (searched)	Few-shot
KNNFewShot	Demonstrations (nearest-neighbor)	Few-shot
COPRO	Instructions	Instructions
MIPROv2	Instructions and demonstrations	Instructions
GEPA	Instructions (reflective)	Instructions
Ensemble	Combines several programs	below

Ensemble is the odd one out: instead of tuning one program, it combines several into one by majority vote or a custom reduce function.

Next: Few-shot demonstrations.