Async program execution¶
This example runs dspy4s programs off the calling thread and returns a scala.concurrent.Future. It shows how to await a single Predict and how to chain two predictors inside one asynchronous module.
Running a single Predict asynchronously¶
def askAsync(question: String)(using RuntimeContext, ExecutionContext): Future[Either[DspyError, String]] =
ContextPropagation.future(
Predict(Signature.fromString("question -> answer"))((question = question)).map(_.output.answer)
)
ContextPropagation.future captures the active RuntimeContext (LM, adapter, callbacks) and runs the synchronous program on a context-propagating ExecutionContext, returning a Future. A plain Future { ... } would run on a thread that has no RuntimeContext, so the program could not resolve its model. The result type stays Either[DspyError, String], with the Future carrying only the off-thread scheduling.
Chaining predictors in an async module¶
final class SimplifierModule:
private val predict1 = ChainOfThought(Signature.fromString("question -> answer"))
private val predict2 = ChainOfThought(Signature.fromString("answer -> simplified_answer"))
private val pipeline = (predict1.mapOutput(result => (answer = result.answer)) >>> predict2)
.mapOutput(_.simplified_answer)
/** Sequential-but-asynchronous, like Python's `aforward`: `>>>` builds the two-stage program, which runs inside one
* context-propagating off-thread computation and returns a `Future`.
*/
def aforward(question: String)(using RuntimeContext, ExecutionContext): Future[Either[DspyError, String]] =
ContextPropagation.future(pipeline(ProgramCall((question = question))).map(_.output))
aforward mirrors a module that runs two steps in sequence. Both ChainOfThought predicts execute in order inside a single off-thread computation. The for-comprehension threads the Either, so a failure in the first step short-circuits and the second step is skipped. The whole sequence is returned as one Future.
Running it¶
Notes¶
Async tools are out of scope. ToolFunction.invoke is synchronous and there is
no async tool path, so tools run synchronously inside the program whether or not
the program itself is run asynchronously.
Full source: Async.scala