Observability

dspy4s emits a stream of events as a program runs: modules, language-model calls, adapters, and tools each report when they start and end. You observe a program by installing a callback handler, or by inspecting the call history.

Callbacks

A CallbackHandler receives a sealed CallbackEvent stream. Match the case you care about. This handler logs the end of every module:

final class AgentLoggingCallback extends CallbackHandler:
  override def onEvent(event: CallbackEvent)(using RuntimeContext): Unit =
    event match
      case e: ModuleEndEvent =>
        val outcome = e.output.fold(err => s"error: ${err.message}", out => out.toString)
        println(s"== ${e.moduleName} step ended ==\n  $outcome\n")
      case _ => () // other scopes (LM / adapter / tool start+end) are ignored by this handler

Install it for a scope with RuntimeEnvironment.withCallbacks:

def runWithLogging(question: String)(using ctx: RuntimeContext): Either[DspyError, String] =
  RuntimeEnvironment.withCallbacks(ctx.callbacks :+ new AgentLoggingCallback) {
    given RuntimeContext = RuntimeEnvironment.current
    agent((question = question)).map(_.output.answer)
  }

Because CallbackEvent is a sealed type, the compiler tells you every event kind you could handle: ModuleStartEvent / ModuleEndEvent, the language-model and adapter start/end events, and ToolStartEvent / ToolEndEvent.

Inspecting history

For a quick look at what was sent to and returned from the model, wrap the model in a ManagedLanguageModel (which records history) and read it with RuntimeEnvironment.inspectHistory:

def askThenInspect(question: String)(using ctx: RuntimeContext): Either[DspyError, (String, String)] =
  ctx.lm match
    case Some(lm: LanguageModel) => RuntimeEnvironment.withSettings(ctx.copy(lm = Some(ManagedLanguageModel(lm)))) {
        given RuntimeContext = RuntimeEnvironment.current
        Predict(Signature.fromString("question -> answer"))((question = question))
          .map(p => (p.output.answer, RuntimeEnvironment.inspectHistory(1)))
      }
    case _ => Left(RuntimeError("no_lm", "no ambient LanguageModel to record history"))

Which to use

You want	Use
To react to events as they happen	A CallbackHandler
A quick after-the-fact look at calls	RuntimeEnvironment.inspectHistory(n)

Next: Saving & loading.