Llama API | Batch Inference › Completion

Batch Inference

batch_inference

Methods

Chat Completion -> { completion_message_batch }

post/alpha/batch-inference/chat-completion

Completion ->

BatchCompletion

post/alpha/batch-inference/completion

Parameters

X-LlamaStack-Client-Version: string

Optional

X-LlamaStack-Provider-Data: string

Optional

Response fields

BatchCompletion

Request example

curl http://any-hosted-llama-stack.com/alpha/batch-inference/completion \
    -H 'Content-Type: application/json' \
    -d '{
      "content_batch": [
        "string"
      ],
      "model": "model"
    }'

200Example

{
  "completion_message_batch": [
    {
      "content": "string",
      "role": "assistant",
      "stop_reason": "end_of_turn",
      "tool_calls": [
        {
          "arguments": {
            "foo": "string"
          },
          "call_id": "call_id",
          "tool_name": "brave_search"
        }
      ]
    }
  ]
}