Prompt API
The Prompt API lets you create well-structured prompts with Kotlin DSL, execute them against different LLM providers, and process responses in different formats.
Create a prompt
The Prompt API uses Kotlin DSL to create prompts. It supports the following types of messages:
system
: sets the context and instructions for the LLM.user
: represents user input.assistant
: represents LLM responses.
Here's an example of a simple prompt:
val prompt = prompt("prompt_name", LLMParams()) {
// Add a system message to set the context
system("You are a helpful assistant.")
// Add a user message
user("Tell me about Kotlin")
// You can also add assistant messages for few-shot examples
assistant("Kotlin is a modern programming language...")
// Add another user message
user("What are its key features?")
}
Execute a prompt
To execute a prompt with a specific LLM, you need to the following:
- Create a corresponding LLM client that handles the connection between your application and LLM providers. For example:
- Call the
execute
method with the prompt and LLM as arguments.
The following LLM clients are available:
- OpenAILLMClient
- AnthropicLLMClient
- GoogleLLMClient
- OpenRouterLLMClient
- OllamaClient
- BedrockLLMClient (JVM only)
Here's a simple example of using the Prompt API:
fun main() {
// Set up the OpenAI client with your API key
val token = System.getenv("OPENAI_API_KEY")
val client = OpenAILLMClient(token)
// Create a prompt
val prompt = prompt("prompt_name", LLMParams()) {
// Add a system message to set the context
system("You are a helpful assistant.")
// Add a user message
user("Tell me about Kotlin")
// You can also add assistant messages for few-shot examples
assistant("Kotlin is a modern programming language...")
// Add another user message
user("What are its key features?")
}
// Execute the prompt and get the response
val response = client.execute(prompt = prompt, model = OpenAIModels.Chat.GPT4o)
println(response)
}
Multimodal inputs
In addition to providing text messages within prompts, Koog also lets you send images, audio, video, and files to LLMs along with user
messages. As with standard text-only prompts, you also add media to the prompt using the DSL structure for prompt construction.
val prompt = prompt("multimodal_input") {
system("You are a helpful assistant.")
user {
+"Describe these images"
attachments {
image("https://example.com/test.png")
image(Path("/User/koog/image.png"))
}
}
}
Textual prompt content
To accommodate for the support for various attachment types and create a clear distinction between text and file inputs in a prompt, you put text messages in a dedicated content
parameter within a user prompt.
To add file inputs, provide them as a list within the attachments
parameter.
The general format of a user message that includes a text message and a list of attachments is as follows:
File attachments
To include an attachment, provide the file in the attachments
parameter, following the format below:
user(
content = "Describe this image",
attachments = listOf(
Attachment.Image(
content = AttachmentContent.URL("https://example.com/capture.png"),
format = "png",
mimeType = "image/png",
fileName = "capture.png"
)
)
)
The attachments
parameter takes a list of file inputs, where each item is an instance of one of the following classes:
Attachment.Image
: image attachments, such asjpg
orpng
files.Attachment.Audio
: audio attachments, such asmp3
orwav
files.Attachment.Video
: video attachments, such asmpg
oravi
files.Attachment.File
: file attachments, such aspdf
ortxt
files.
Each of the classes above takes the following parameters:
Name | Data type | Required | Description |
---|---|---|---|
content |
AttachmentContent | Yes | The source of the provided file content. For more information, see AttachmentContent. |
format |
String | Yes | The format of the provided file. For example, png . |
mimeType |
String | Only for Attachment.File |
The MIME Type of the provided file. For example, image/png . |
fileName |
String | No | The name of the provided file including the extension. For example, screenshot.png . |
AttachmentContent
AttachmentContent
defines the type and source of content that is provided as an input to the LLM. The following
classes are supported:
AttachmentContent.URL(val url: String)
Provides file content from the specified URL. Takes the following parameter:
Name | Data type | Required | Description |
---|---|---|---|
url |
String | Yes | The URL of the provided content. |
AttachmentContent.Binary.Bytes(val data: ByteArray)
Provides file content as a byte array. Takes the following parameter:
Name | Data type | Required | Description |
---|---|---|---|
data |
ByteArray | Yes | The file content provided as a byte array. |
AttachmentContent.Binary.Base64(val base64: String)
Provides file content encoded as a Base64 string. Takes the following parameter:
Name | Data type | Required | Description |
---|---|---|---|
base64 |
String | Yes | The Base64 string containing file data. |
AttachmentContent.PlainText(val text: String)
Applies only if the attachment type is Attachment.File
. Provides content from a plain text file (such as the text/plain
MIME type). Takes the following parameter:
Name | Data type | Required | Description |
---|---|---|---|
text |
String | Yes | The content of the file. |
Mixed attachment content
In addition to providing different types of attachments in separate prompts or messages, you can also provide multiple and mixed types of attachments in a single user
message, as shown below:
val prompt = prompt("mixed_content") {
system("You are a helpful assistant.")
user {
+"Compare the image with the document content."
attachments {
image(Path("/User/koog/page.png"))
binaryFile(Path("/User/koog/page.pdf"), "application/pdf")
}
}
}
Prompt executors
Prompt executors provide a higher-level way to work with LLMs, handling the details of client creation and management.
You can use a prompt executor to manage and run prompts. You can choose a prompt executor based on the LLM provider you plan to use or create a custom prompt executor using one of the available LLM clients.
The Koog framework provides several prompt executors:
-
Single provider executors:
simpleOpenAIExecutor
: for executing prompts with OpenAI models.simpleAnthropicExecutor
: for executing prompts with Anthropic models.simpleGoogleExecutor
: for executing prompts with Google models.simpleOpenRouterExecutor
: for executing prompts with OpenRouter.simpleOllamaExecutor
: for executing prompts with Ollama.
-
Multi-provider executor:
DefaultMultiLLMPromptExecutor
: For working with multiple LLM providers
Create a single provider executor
To create a prompt executor for a specific LLM provider, use the corresponding function.
For example, to create the OpenAI prompt executor, you need to call the simpleOpenAIExecutor
function and provide it with the API key required for authentication with the OpenAI service:
- Create a prompt executor:
- Execute the prompt with a specific LLM:
Create a multi-provider executor
To create a prompt executor that works with multiple LLM providers, do the following:
- Configure clients for the required LLM providers with the corresponding API keys. For example:
- Pass the configured clients to the
DefaultMultiLLMPromptExecutor
class constructor to create a prompt executor with multiple LLM providers: - Execute the prompt with a specific LLM: