Observability

dspy4s emits a stream of events as a program runs: modules, language-model calls, adapters, and tools each report when they start and end. You observe a program by installing a callback handler, or by inspecting the call history.

Callbacks

A CallbackHandler receives a sealed CallbackEvent stream. Match the case you care about. This handler logs the end of every module:

final class AgentLoggingCallback extends CallbackHandler:
  override def onEvent(event: CallbackEvent)(using RuntimeContext): Unit =
    event match
      case e: ModuleEndEvent =>
        val outcome = e.output.fold(err => s"error: ${err.message}", out => out.toString)
        println(s"== ${e.moduleName} step ended ==\n  $outcome\n")
      case _ => () // other scopes (LM / adapter / tool start+end) are ignored by this handler

Install it for a scope with RuntimeEnvironment.withCallbacks:

def runWithLogging(question: String)(using ctx: RuntimeContext): Either[DspyError, String] =
  RuntimeEnvironment.withCallbacks(ctx.callbacks :+ new AgentLoggingCallback) {
    given RuntimeContext = RuntimeEnvironment.current
    agent((question = question)).map(_.output.answer)
  }

RuntimeEnvironment.withCallbacks(Vector.empty) turns off the callbacks for a block, also the callbacks of the global context.

Because CallbackEvent is a sealed type, the compiler tells you every event kind you could handle: ModuleStartEvent / ModuleEndEvent, the language-model and adapter start/end events, and ToolStartEvent / ToolEndEvent.

Inspecting history

For a quick look at what was sent to and returned from the model, wrap the model in a ManagedLanguageModel (which records history) and read it with RuntimeEnvironment.inspectHistory:

def askThenInspect(question: String)(using ctx: RuntimeContext): Either[DspyError, (String, String)] =
  SettingsProgramRuntime.default.resolveModel.flatMap { lm =>
    Predict(Signature.fromString("question -> answer"))((question = question))(
      using ctx.copy(lm = Some(ManagedLanguageModel(lm)))
    ).map(p => (p.output.answer, RuntimeEnvironment.inspectHistory(1)))
  }

Which to use

You want	Use
To react to events as they happen	A CallbackHandler
A quick after-the-fact look at calls	RuntimeEnvironment.inspectHistory(n)

Next: Saving & loading.