Skip to main content

Task type: GPT

The chat-completion task type is used to designate services that perform text generation in a dialogue mode. The main example of this type of service is ChatGPT.

System name: chat-completion

List of services

Data types​

We used the OpenAI interface specifications as the base interface. The OpenAI specification contains many features that may not be supported by all existing services. But if they are supported, then the interface for accessing them will be the same for all services.

Predict request​

  • Specification
  • Python type: ... (module mlp-python-sdk)
  • Java type: com.mlp.api.datatypes.chatgpt.ChatCompletionRequest (module mlp-datatypes)

Request example:

{
  "messages": [
    { "role": "user", "content": "Hello" }
  ]
}

Predict config​

For the chat-completion task, we also described a recommended type for predict-config. The most important and frequently encountered settings for LLM inference are detailed in it. All fields of the predict-config, as well as the parameter itself, are optional.

  • Specification
  • Python type: ... (module mlp-python-sdk)
  • Java type: com.mlp.api.datatypes.chatgpt.ChatCompletionConfig (module mlp-datatypes)

Configuration example:

{
  "model": "gpt-4o",
  "stream": true,
  "max_tokens": 2000,
  "temperature": 1
}

Predict response​

  • Specification
  • Python type: ... (module mlp-python-sdk)
  • Java type: com.mlp.api.datatypes.chatgpt.ChatCompletionResult (module mlp-datatypes)

Response example:

{
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! Glad to chat with you. How are you?"
      }
    }
  ],
  "model": "claude-3-sonnet-20240229",
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 25,
    "total_tokens": 35
  }
}

Direct access mode​

Some services that provide access to third-party services, such as OpenAI, Claude, Gemini, etc., allow sending requests in the direct access mode.

To enable direct access, when sending a request, you need to specify dataType: https://caila.io/specs/mlp-data-gpt.yml#/ChatCompletionDirectProxy. The response will return a packet with the same type.

OpenAI adapter​

All services of this type are available via the OpenAI adapter.