Configuring a language model

A program needs a language model to run. dspy4s ships one provider, OpenAiLanguageModel, which speaks the OpenAI /chat/completions shape. That covers OpenAI and every OpenAI-compatible server (Azure, Ollama, vLLM, SGLang, LM Studio, OpenRouter).

Constructing a model

Pass a model name and an API key:

def openAi(model: String, apiKey: String): LanguageModel =
  OpenAiLanguageModel(model, apiKey)

For a local server that does not check credentials, use local with the server's base URL:

def ollama(model: String = "llama3.2"): LanguageModel =
  OpenAiLanguageModel.local(model, baseUrl = "http://localhost:11434/v1")

OpenAiLanguageModel.fromEnv(model) reads OPENAI_API_KEY from the environment, which is what the bundled examples use.

Installing it

A model becomes active by putting it in the runtime context. The Quickstart shows the full wiring with RuntimeEnvironment.withSettings.

Calling a model directly

Most of the time a module calls the model for you. When you need the raw call, use LanguageModel.call, which returns Either[DspyError, LmResponse]:

def callDirect(prompt: String)(using ctx: RuntimeContext): Either[DspyError, String] =
  ctx.lm match
    case Some(lm: LanguageModel) =>
      val request = LmRequest(
        model = lm.id,
        messages = Vector(Message(role = MessageRole.User, text = Some(prompt)))
      )
      lm.call(request).map(_.outputs.headOption.map(_.text).getOrElse(""))
    case _ => Left(dspy4s.core.contracts.ConfigurationError("no LanguageModel configured"))

Per-call generation settings

Generation parameters such as temperature go in a per-call config bag. rolloutId is a dedicated field used to bust the cache for an otherwise-identical call:

def askWithConfig(question: String, temperature: Double, rolloutId: Int)(using
    RuntimeContext
): Either[DspyError, String] =
  qa(ProgramCall(
    input = (question = question),
    config = DynamicValues.record("temperature" := temperature),
    rolloutId = Some(rolloutId)
  )).map(_.output.answer)

Errors

dspy4s never throws on a model failure. Every call returns an Either, and DspyError carries a stable code and message:

def askHandlingErrors(question: String)(using RuntimeContext): String =
  ask(question) match
    case Right(answer) => answer
    case Left(err)     => s"LM failed: code=${err.code}, message=${err.message}"

Next: Adapters.