Interface VertexAIInput

Input to a Google Vertex AI LLM class.

interface VertexAIInput {
    apiConfig?: GoogleAIAPIConfig;
    apiKey?: string;
    apiName?: string;
    apiVersion?: string;
    authOptions?: GoogleAuthOptions<JSONClient>;
    cache?: boolean | BaseCache<Generation[]>;
    callbackManager?: CallbackManager;
    callbacks?: Callbacks;
    concurrency?: number;
    convertSystemMessageToHumanContent?: boolean;
    endpoint?: string;
    location?: string;
    maxConcurrency?: number;
    maxOutputTokens?: number;
    maxRetries?: number;
    metadata?: Record<string, unknown>;
    model?: string;
    modelName?: string;
    onFailedAttempt?: FailedAttemptHandler;
    platformType?: GooglePlatformType;
    responseMimeType?: GoogleAIResponseMimeType;
    safetyHandler?: GoogleAISafetyHandler;
    safetySettings?: GoogleAISafetySetting[];
    stopSequences?: string[];
    streaming?: boolean;
    tags?: string[];
    temperature?: number;
    topK?: number;
    topP?: number;
    verbose?: boolean;
}

Hierarchy

  • GoogleLLMInput
    • VertexAIInput

Properties

apiConfig? apiKey? apiName? apiVersion? authOptions? cache? callbackManager? callbacks? concurrency? convertSystemMessageToHumanContent? endpoint? location? maxConcurrency? maxOutputTokens? maxRetries? metadata? model? modelName? onFailedAttempt? platformType? responseMimeType? safetyHandler? safetySettings? stopSequences? streaming? tags? temperature? topK? topP? verbose?

Properties

OptionalapiConfig

apiConfig?: GoogleAIAPIConfig

OptionalapiKey

apiKey?: string

Some APIs allow an API key instead

OptionalapiName

apiName?: string

OptionalapiVersion

apiVersion?: string

The version of the API functions. Part of the path.

OptionalauthOptions

authOptions?: GoogleAuthOptions<JSONClient>

Optionalcache

cache?: boolean | BaseCache<Generation[]>

OptionalcallbackManager

callbackManager?: CallbackManager

Deprecated

Use callbacks instead

Optionalcallbacks

callbacks?: Callbacks

Optionalconcurrency

concurrency?: number

Deprecated

Use maxConcurrency instead

OptionalconvertSystemMessageToHumanContent

convertSystemMessageToHumanContent?: boolean

Optionalendpoint

endpoint?: string

Hostname for the API call (if this is running on GCP)

Optionallocation

location?: string

Region where the LLM is stored (if this is running on GCP)

OptionalmaxConcurrency

maxConcurrency?: number

The maximum number of concurrent calls that can be made. Defaults to Infinity, which means no limit.

OptionalmaxOutputTokens

maxOutputTokens?: number

Maximum number of tokens to generate in the completion.

OptionalmaxRetries

maxRetries?: number

The maximum number of retries that can be made for a single call, with an exponential backoff between each attempt. Defaults to 6.

Optionalmetadata

metadata?: Record<string, unknown>

Optionalmodel

model?: string

Model to use

OptionalmodelName

modelName?: string

Model to use Alias for model

OptionalonFailedAttempt

onFailedAttempt?: FailedAttemptHandler

Custom handler to handle failed attempts. Takes the originally thrown error object as input, and should itself throw an error if the input error is not retryable.

OptionalplatformType

platformType?: GooglePlatformType

What platform to run the service on. If not specified, the class should determine this from other means. Either way, the platform actually used will be in the "platform" getter.

OptionalresponseMimeType

responseMimeType?: GoogleAIResponseMimeType

Available for gemini-1.5-pro. The output format of the generated candidate text. Supported MIME types:

  • text/plain: Text output.
  • application/json: JSON response in the candidates.

Default

"text/plain"

OptionalsafetyHandler

safetyHandler?: GoogleAISafetyHandler

OptionalsafetySettings

safetySettings?: GoogleAISafetySetting[]

OptionalstopSequences

stopSequences?: string[]

Optionalstreaming

streaming?: boolean

Whether or not to stream.

Default

false

Optionaltags

tags?: string[]

Optionaltemperature

temperature?: number

Sampling temperature to use

OptionaltopK

topK?: number

Top-k changes how the model selects tokens for output.

A top-k of 1 means the selected token is the most probable among all tokens in the model’s vocabulary (also called greedy decoding), while a top-k of 3 means that the next token is selected from among the 3 most probable tokens (using temperature).

OptionaltopP

topP?: number

Top-p changes how the model selects tokens for output.

Tokens are selected from most probable to least until the sum of their probabilities equals the top-p value.

For example, if tokens A, B, and C have a probability of .3, .2, and .1 and the top-p value is .5, then the model will select either A or B as the next token (using temperature).

Optionalverbose

verbose?: boolean

Settings

Member Visibility

On This Page

Properties