Interface GoogleAIBaseLanguageModelCallOptions

The params which can be passed to the API at request time.

interface GoogleAIBaseLanguageModelCallOptions {
    allowed_function_names?: string[];
    cachedContent?: string;
    callbacks?: Callbacks;
    configurable?: Record<string, any>;
    convertSystemMessageToHumanContent?: boolean;
    frequencyPenalty?: number;
    labels?: Record<string, string>;
    logprobs?: boolean;
    ls_structured_output_format?: {
        kwargs: {
            method: string;
        };
        schema?: JsonSchema7Type;
    };
    maxConcurrency?: number;
    maxOutputTokens?: number;
    maxReasoningTokens?: number;
    metadata?: Record<string, unknown>;
    model?: string;
    modelName?: string;
    presencePenalty?: number;
    reasoningEffort?: "low" | "medium" | "high";
    recursionLimit?: number;
    responseMimeType?: GoogleAIResponseMimeType;
    responseModalities?: string[];
    runId?: string;
    runName?: string;
    safetyHandler?: GoogleAISafetyHandler;
    safetySettings?: GoogleAISafetySetting[];
    seed?: number;
    signal?: AbortSignal;
    speechConfig?: GoogleSpeechConfig | GoogleSpeechConfigSimplified;
    stop?: string[];
    stopSequences?: string[];
    streamUsage?: boolean;
    streaming?: boolean;
    tags?: string[];
    temperature?: number;
    thinkingBudget?: number;
    timeout?: number;
    tool_choice?: ToolChoice;
    tools?: GoogleAIToolType[];
    topK?: number;
    topLogprobs?: number;
    topP?: number;
}

Hierarchy (view full)

Properties

allowed_function_names? cachedContent? callbacks? configurable? convertSystemMessageToHumanContent? frequencyPenalty? labels? logprobs? ls_structured_output_format? maxConcurrency? maxOutputTokens? maxReasoningTokens? metadata? model? modelName? presencePenalty? reasoningEffort? recursionLimit? responseMimeType? responseModalities? runId? runName? safetyHandler? safetySettings? seed? signal? speechConfig? stop? stopSequences? streamUsage? streaming? tags? temperature? thinkingBudget? timeout? tool_choice? tools? topK? topLogprobs? topP?

Properties

Optionalallowed_function_names

allowed_function_names?: string[]

Allowed functions to call when the mode is "any". If empty, any one of the provided functions are called.

OptionalcachedContent

cachedContent?: string

Used to specify a previously created context cache to use with generation. For Vertex, this should be of the form: "projects/PROJECT_NUMBER/locations/LOCATION/cachedContents/CACHE_ID",

See these guides for more information on how to use context caching: https://cloud.google.com/vertex-ai/generative-ai/docs/context-cache/context-cache-create https://cloud.google.com/vertex-ai/generative-ai/docs/context-cache/context-cache-use

Optionalcallbacks

callbacks?: Callbacks

Callbacks for this call and any sub-calls (eg. a Chain calling an LLM). Tags are passed to all callbacks, metadata is passed to handle*Start callbacks.

Optionalconfigurable

configurable?: Record<string, any>

Runtime values for attributes previously made configurable on this Runnable, or sub-Runnables.

OptionalconvertSystemMessageToHumanContent

convertSystemMessageToHumanContent?: boolean

OptionalfrequencyPenalty

frequencyPenalty?: number

Frequency penalty applied to the next token's logprobs, multiplied by the number of times each token has been seen in the respponse so far. A positive penalty will discourage the use of tokens that have already been used, proportional to the number of times the token has been used: The more a token is used, the more dificult it is for the model to use that token again increasing the vocabulary of responses. Caution: A negative penalty will encourage the model to reuse tokens proportional to the number of times the token has been used. Small negative values will reduce the vocabulary of a response. Larger negative values will cause the model to start repeating a common token until it hits the maxOutputTokens limit.

Optionallabels

labels?: Record<string, string>

Custom metadata labels to associate with the request. Only supported on Vertex AI (Google Cloud Platform). Labels are key-value pairs where both keys and values must be strings.

Example:

{
  labels: {
    "team": "research",
    "component": "frontend",
    "environment": "production"
  }
}

Optionallogprobs

logprobs?: boolean

Whether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the content of message.

Optionalls_structured_output_format

ls_structured_output_format?: {
    kwargs: {
        method: string;
    };
    schema?: JsonSchema7Type;
}

Describes the format of structured outputs. This should be provided if an output is considered to be structured

Type declaration

  • kwargs: {
        method: string;
    }

    An object containing the method used for structured output (e.g., "jsonMode").

    • method: string
  • Optionalschema?: JsonSchema7Type

    The JSON schema describing the expected output structure.

OptionalmaxConcurrency

maxConcurrency?: number

Maximum number of parallel calls to make.

OptionalmaxOutputTokens

maxOutputTokens?: number

Maximum number of tokens to generate in the completion. This may include reasoning tokens (for backwards compatibility).

OptionalmaxReasoningTokens

maxReasoningTokens?: number

The maximum number of the output tokens that will be used for the "thinking" or "reasoning" stages.

Optionalmetadata

metadata?: Record<string, unknown>

Metadata for this call and any sub-calls (eg. a Chain calling an LLM). Keys should be strings, values should be JSON-serializable.

Optionalmodel

model?: string

Model to use

OptionalmodelName

modelName?: string

Model to use Alias for model

OptionalpresencePenalty

presencePenalty?: number

Presence penalty applied to the next token's logprobs if the token has already been seen in the response. This penalty is binary on/off and not dependant on the number of times the token is used (after the first). Use frequencyPenalty for a penalty that increases with each use. A positive penalty will discourage the use of tokens that have already been used in the response, increasing the vocabulary. A negative penalty will encourage the use of tokens that have already been used in the response, decreasing the vocabulary.

OptionalreasoningEffort

reasoningEffort?: "low" | "medium" | "high"

An OpenAI compatible parameter that will map to "maxReasoningTokens"

OptionalrecursionLimit

recursionLimit?: number

Maximum number of times a call can recurse. If not provided, defaults to 25.

OptionalresponseMimeType

responseMimeType?: GoogleAIResponseMimeType

Available for gemini-1.5-pro. The output format of the generated candidate text. Supported MIME types:

  • text/plain: Text output.
  • application/json: JSON response in the candidates.

Default

"text/plain"

OptionalresponseModalities

responseModalities?: string[]

The modalities of the response.

OptionalrunId

runId?: string

Unique identifier for the tracer run for this call. If not provided, a new UUID will be generated.

OptionalrunName

runName?: string

Name for the tracer run for this call. Defaults to the name of the class.

OptionalsafetyHandler

safetyHandler?: GoogleAISafetyHandler

OptionalsafetySettings

safetySettings?: GoogleAISafetySetting[]

Optionalseed

seed?: number

Seed used in decoding. If not set, the request uses a randomly generated seed.

Optionalsignal

signal?: AbortSignal

Abort signal for this call. If provided, the call will be aborted when the signal is aborted.

See

https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal

OptionalspeechConfig

speechConfig?: GoogleSpeechConfig | GoogleSpeechConfigSimplified

Speech generation configuration. You can use either Google's definition of the speech configuration, or a simplified version we've defined (which can be as simple as the name of a pre-defined voice).

Optionalstop

stop?: string[]

Stop tokens to use for this call. If not provided, the default stop tokens for the model will be used.

OptionalstopSequences

stopSequences?: string[]

OptionalstreamUsage

streamUsage?: boolean

Whether or not to include usage data, like token counts in the streamed response chunks.

Default

true

Optionalstreaming

streaming?: boolean

Whether or not to stream.

Default

false

Optionaltags

tags?: string[]

Tags for this call and any sub-calls (eg. a Chain calling an LLM). You can use these to filter calls.

Optionaltemperature

temperature?: number

Sampling temperature to use

OptionalthinkingBudget

thinkingBudget?: number

An alias for "maxReasoningTokens"

Optionaltimeout

timeout?: number

Timeout for this call in milliseconds.

Optionaltool_choice

tool_choice?: ToolChoice

Specifies how the chat model should use tools.

Default

undefined

Possible values:
- "auto": The model may choose to use any of the provided tools, or none.
- "any": The model must use one of the provided tools.
- "none": The model must not use any tools.
- A string (not "auto", "any", or "none"): The name of a specific tool the model must use.
- An object: A custom schema specifying tool choice parameters. Specific to the provider.

Note: Not all providers support tool_choice. An error will be thrown
if used with an unsupported model.

Optionaltools

tools?: GoogleAIToolType[]

OptionaltopK

topK?: number

Top-k changes how the model selects tokens for output.

A top-k of 1 means the selected token is the most probable among all tokens in the model’s vocabulary (also called greedy decoding), while a top-k of 3 means that the next token is selected from among the 3 most probable tokens (using temperature).

OptionaltopLogprobs

topLogprobs?: number

An integer between 0 and 5 specifying the number of most likely tokens to return at each token position, each with an associated log probability. logprobs must be set to true if this parameter is used.

OptionaltopP

topP?: number

Top-p changes how the model selects tokens for output.

Tokens are selected from most probable to least until the sum of their probabilities equals the top-p value.

For example, if tokens A, B, and C have a probability of .3, .2, and .1 and the top-p value is .5, then the model will select either A or B as the next token (using temperature).

Settings

Member Visibility

On This Page

Properties