Llama API

Inference

inference

Methods

Chat Completion -> { completion_message, logprobs } | { event }
post/alpha/inference/chat-completion
Completion ->
CompletionResponse
| { delta, logprobs, stop_reason }
post/alpha/inference/completion
Embeddings ->
EmbeddingsResponse
post/alpha/inference/embeddings

Domain types

CompletionResponse = { content, stop_reason, logprobs }
EmbeddingsResponse = { embeddings }
TokenLogProbs = { logprobs_by_token }