Llama API | Inference › Chat Completion

Inference

inference

Methods

Chat Completion -> { completion_message, logprobs } | { event }

post/alpha/inference/chat-completion

Parameters

X-LlamaStack-Client-Version: string

Optional

X-LlamaStack-Provider-Data: string

Optional

Response fields

ChatCompletionResponse = { completion_message, logprobs }

ChatCompletionResponseStreamChunk = { event }

Request example

curl http://any-hosted-llama-stack.com/alpha/inference/chat-completion \
    -H 'Content-Type: application/json' \
    -d '{
      "messages": [
        {
          "content": "string",
          "role": "user"
        }
      ],
      "model_id": "model_id"
    }'

Completion ->

CompletionResponse

| { delta, logprobs, stop_reason }

post/alpha/inference/completion

Embeddings ->

EmbeddingsResponse

post/alpha/inference/embeddings

Domain types

CompletionResponse = { content, stop_reason, logprobs }

EmbeddingsResponse = { embeddings }

TokenLogProbs = { logprobs_by_token }