Interface SamplingParams

interface SamplingParams {
    max_completion_tokens?: number;
    response_format?: ResponseFormatJSONSchema | ResponseFormatText | ResponseFormatJSONObject;
    seed?: number;
    temperature?: number;
    tools?: ChatCompletionFunctionTool[];
    top_p?: number;
}

Properties

max_completion_tokens? response_format? seed? temperature? tools? top_p?

Properties

Optionalmax_completion_tokens

max_completion_tokens?: number

The maximum number of tokens in the generated output.

Optionalresponse_format

response_format?: ResponseFormatJSONSchema | ResponseFormatText | ResponseFormatJSONObject

An object specifying the format that the model must output.

Setting to { "type": "json_schema", "json_schema": {...} } enables Structured Outputs which ensures the model will match your supplied JSON schema. Learn more in the Structured Outputs guide.

Setting to { "type": "json_object" } enables the older JSON mode, which ensures the message the model generates is valid JSON. Using json_schema is preferred for models that support it.

Optionalseed

seed?: number

A seed value to initialize the randomness, during sampling.

Optionaltemperature

temperature?: number

A higher temperature increases randomness in the outputs.

Optionaltools

tools?: ChatCompletionFunctionTool[]

A list of tools the model may call. Currently, only functions are supported as a tool. Use this to provide a list of functions the model may generate JSON inputs for. A max of 128 functions are supported.

Optionaltop_p

top_p?: number

An alternative to temperature for nucleus sampling; 1.0 includes all tokens.

Settings

Member Visibility

On This Page

Properties