class LittleGhost::Providers::OpenAICompatible

OpenAICompatible brings OpenAI-style Responses or Chat Completions endpoints into LittleGhost. Agents receive the same streaming events whether the endpoint is OpenAI, a hosted model service, or an application gateway.

provider = LittleGhost::Providers::OpenAICompatible.new(
  api_key: ENV.fetch("MODEL_API_KEY"),
  model: "example-model",
  base_url: "https://models.example.test/v1/"
)

The client translates ModelRequest values to the selected wire API and translates responses back to StreamEvent objects. It also sends Embeddings::Request values to the compatible embeddings endpoint.

Retries and streaming output

Transient HTTP and stream failures retry with limited exponential backoff. A :model_retry event reports each retry and whether text had already been emitted. Partial text may repeat after a retry, so consumers that assemble streams must use that event to discard or replace superseded output.

Constants

DEFAULT_BASE_URL: The OpenAI API endpoint used when base_url is omitted.

Attributes

api [R]

Provider model identifier and selected OpenAI-compatible wire API.

model [R]

Provider model identifier and selected OpenAI-compatible wire API.

Public Class Methods

new (api_key:, model:, base_url: DEFAULT_BASE_URL, api: :responses, headers: {}, open_timeout: 10, read_timeout: 120, allow_insecure_http: false, max_response_bytes: Support::HTTPClient::DEFAULT_MAX_RESPONSE_BYTES, max_embedding_response_bytes: DEFAULT_MAX_EMBEDDING_RESPONSE_BYTES, max_retries: 2, max_retry_delay: MAX_RETRY_DELAY, transport: nil, sleeper: nil, on_retry: ->(*) {})

Source

# File lib/little_ghost/providers/openai_compatible.rb, line 90
def initialize(
  api_key:,
  model:,
  base_url: DEFAULT_BASE_URL,
  api: :responses,
  headers: {},
  open_timeout: 10,
  read_timeout: 120,
  allow_insecure_http: false,
  max_response_bytes: Support::HTTPClient::DEFAULT_MAX_RESPONSE_BYTES,
  max_embedding_response_bytes: DEFAULT_MAX_EMBEDDING_RESPONSE_BYTES,
  max_retries: 2,
  max_retry_delay: MAX_RETRY_DELAY,
  transport: nil,
  sleeper: nil,
  on_retry: ->(*) {}
)
  @api_key = api_key
  @model = model
  @api = api.to_sym
  raise ConfigurationError, "api must be :responses or :chat_completions" unless %i[responses chat_completions].include?(@api)

  @headers = headers.transform_keys(&:to_s).freeze
  @max_embedding_response_bytes = Integer(max_embedding_response_bytes)
  raise ArgumentError, "max_embedding_response_bytes must be positive" unless @max_embedding_response_bytes.positive?

  @max_retries = Integer(max_retries)
  @max_retry_delay = Integer(max_retry_delay)
  @transport = transport || Support::HTTPClient.new(
    base_url:,
    open_timeout:,
    read_timeout:,
    allow_insecure_http:,
    max_response_bytes:
  )
  @sleeper = sleeper
  @on_retry = on_retry
end

Configures an OpenAI-compatible client.

api is :responses or :chat_completions. headers adds trusted endpoint-specific headers. max_retries controls retries before the original error is raised, and on_retry receives the attempt, error, and delay. Pass a custom transport for alternate HTTP execution.

request_options ()

Source

# File lib/little_ghost/providers/openai_compatible.rb, line 31
def self.request_options
  %i[max_response_bytes max_retries max_retry_delay open_timeout read_timeout].freeze
end

Request policy supported by OpenAI-compatible HTTP clients.

Public Instance Methods

capabilities (metadata: {})

Source

# File lib/little_ghost/providers/openai_compatible.rb, line 212
def capabilities(metadata: {})
  ModelCapabilities.permissive
end

Returns the permissive capability contract expected from compatible APIs. Subclasses can override this when the endpoint advertises precise support.

embed (request)

Source

# File lib/little_ghost/providers/openai_compatible.rb, line 174
def embed(request)
  attempts = 0
  begin
    request.cancellation_token.raise_if_cancelled!
    payload = {
      model:,
      input: request.inputs,
      encoding_format: "float"
    }
    dimensions = request.settings[:dimensions]
    payload[:dimensions] = Integer(dimensions) if dimensions
    body = +""
    @transport.stream(
      path: "embeddings",
      headers: {"Authorization" => "Bearer #{@api_key}", "Content-Type" => "application/json"}.merge(@headers),
      body: JSON.generate(payload),
      cancellation_token: request.cancellation_token,
      deadline: request.deadline
    ) do |chunk|
      if body.bytesize + chunk.bytesize > @max_embedding_response_bytes
        raise ProtocolError, "#{embedding_provider_name} embedding response exceeded #{@max_embedding_response_bytes} bytes"
      end
      body << chunk
    end
    normalize_embedding_response(body, request.inputs.length, dimensions && Integer(dimensions))
  rescue HTTPError => error
    raise unless error.retryable? && attempts < @max_retries

    attempts += 1
    delay = capped_retry_delay(request, retry_delay(attempts))
    @on_retry.call(attempts, error, delay)
    wait_before_retry(request, delay)
    retry
  end
end

Embeds one or more strings with the configured compatible model.

The optional :dimensions request setting selects a supported output size. The response preserves input order and raises ProtocolError when the endpoint returns an incomplete or invalid batch.

stream (request) { |event| ... }

Source

# File lib/little_ghost/providers/openai_compatible.rb, line 134
def stream(request)
  return enum_for(__method__, request) unless block_given?

  attempts = 0

  begin
    partial_text = false
    request.cancellation_token.raise_if_cancelled!
    stream_once(request) do |event|
      partial_text ||= event.type == :text_delta && !event.data[:text].to_s.empty?
      yield event
    end
  rescue HTTPError, StreamError => error
    if context_window_overflow?(error)
      raise ContextWindowOverflowError, "The model context window was exceeded"
    end
    raise if !error.retryable? || attempts >= @max_retries

    attempts += 1
    request.cancellation_token.raise_if_cancelled!
    delay = capped_retry_delay(request, retry_delay(attempts))
    @on_retry.call(attempts, error, delay)
    yield StreamEvent.build(
      :model_retry,
      attempt: attempts,
      delay:,
      error_class: error.class.name,
      partial_text:,
      **retry_error_metadata(error)
    )
    wait_before_retry(request, delay)
    retry
  end
end

Streams LittleGhost StreamEvent objects for request.

Without a block, returns an Enumerator. Context-window errors normalize to ContextWindowOverflowError, and malformed tool calls normalize to MalformedToolCallError.