Saving an optimized program

This example walks the full optimize-then-reuse cycle: build a question -> answer predictor, compile it into demonstrations with an optimizer, write that state to disk, then recreate the program and load the state back. It demonstrates that a program's learned state survives a round-trip through JSON.

Build the program

def program(): DynamicPredict =
  DynamicPredict(Signature.fromString("question -> answer").layout)

The example uses a single dynamic question -> answer predictor for brevity. Optimizers and ProgramPersistence are generic over any program with OptimizableStructure evidence, including Predict and ChainOfThought programs.

Compile it

def compile(metric: Metric, student: DynamicPredict, trainset: Vector[Example])(using
    RuntimeContext
)
    : Either[DspyError, DynamicPredict] =
  new BootstrapFewShot[DynamicPredict](BootstrapFewShotConfig(
    metric = Some(metric),
    maxBootstrappedDemos = DemoCount(4),
    maxLabeledDemos = DemoCount(4),
    maxRounds = RoundCount(5)
  )).compile(student, trainset).map(_.bestProgram)

BootstrapFewShot runs the program over an LM against a trainset to collect demonstration traces, and returns the best program it found. The result is the same predictor with demos attached. This step needs OPENAI_API_KEY because it calls the LM. Bring your own Examples for the trainset.

Save the state

def save(program: DynamicPredict, path: String): Either[DspyError, Unit] =
  ProgramPersistence.save(program, path)

ProgramPersistence.save writes each leaf's OptimizableParameters: instructions, demos, and module-level config. It does not write signature field structure, module names, runtime bindings, tools, or program code.

Recreate and load

def load(fresh: DynamicPredict, path: String): Either[DspyError, DynamicPredict] =
  ProgramPersistence.load(fresh, path)

load takes a freshly built program with the same optimizable structure and leaf order, then returns a new immutable program with the saved instructions, demos, and config written into it. The fresh value keeps its signature structure, name, runtime, output schema, bound LM, and tools. For the complete contract and ordinal-ID caveat, see Saving and loading.

Running it

sbt "examples/runMain dspy4s.examples.tutorials.saving.savingMain"

The runnable savingMain performs the round-trip offline. It hand-attaches a couple of demos in place of compile, saves to a temp file, loads into a fresh program, and asserts the demo count is preserved, so no LM is required to run it.

Notes

The whole-program save form (Python's save_program=True, which serializes the program's architecture into a directory) is out of scope. There is no code or pickle serialization, hence no .pkl variant and no modules_to_serialize option. The former layout-bearing predictor-state format is unsupported; regenerate saved artifacts with the current version.

Full source: Saving.scala