Interface WatsonxCallOptionsDeployedChat

interface WatsonxCallOptionsDeployedChat {
    callbacks?: Callbacks;
    configurable?: Record<string, any>;
    context?: string;
    headers?: OutgoingHttpHeaders;
    idOrName: string;
    ls_structured_output_format?: {
        kwargs: {
            method: string;
        };
        schema?: JsonSchema7Type;
    };
    maxConcurrency?: number;
    maxRetries?: number;
    messages: DeploymentTextChatMessages[];
    metadata?: Record<string, unknown>;
    promptIndex?: number;
    recursionLimit?: number;
    runId?: string;
    runName?: string;
    signal?: AbortSignal;
    streaming?: boolean;
    tags?: string[];
    timeout?: number;
    tool_choice?: string | TextChatParameterTools;
    watsonxCallbacks?: RequestCallbacks<any>;
}

Hierarchy (view full)

Properties

callbacks? configurable? context? headers? idOrName ls_structured_output_format? maxConcurrency? maxRetries? messages metadata? promptIndex? recursionLimit? runId? runName? signal? streaming? tags? timeout? tool_choice? watsonxCallbacks?

Properties

Optionalcallbacks

callbacks?: Callbacks

Callbacks for this call and any sub-calls (eg. a Chain calling an LLM). Tags are passed to all callbacks, metadata is passed to handle*Start callbacks.

Optionalconfigurable

configurable?: Record<string, any>

Runtime values for attributes previously made configurable on this Runnable, or sub-Runnables.

Optionalcontext

context?: string

If specified, context will be inserted into messages. Depending on the model, context may be inserted into the content with system role; or into the content of the last message of user role.

In the example, context "Today is Wednesday" is inserted as such content of user becomes "Today is Wednesday. Who are you and which day is tomorrow?".

Optionalheaders

headers?: OutgoingHttpHeaders

idOrName

idOrName: string

The id_or_name can be either the deployment_id that identifies the deployment or a serving_name that allows a predefined URL to be used to post a prediction. The deployment must reference a prompt template with input_mode chat.

The WML instance that is associated with the deployment will be used for limits and billing (if a paid plan).

Optionalls_structured_output_format

ls_structured_output_format?: {
    kwargs: {
        method: string;
    };
    schema?: JsonSchema7Type;
}

Describes the format of structured outputs. This should be provided if an output is considered to be structured

Type declaration

  • kwargs: {
        method: string;
    }

    An object containing the method used for structured output (e.g., "jsonMode").

    • method: string
  • Optionalschema?: JsonSchema7Type

    The JSON schema describing the expected output structure.

OptionalmaxConcurrency

maxConcurrency?: number

Maximum number of parallel calls to make.

OptionalmaxRetries

maxRetries?: number

messages

messages: DeploymentTextChatMessages[]

The messages for this chat session. You cannot specify system role in the messages. Depending on the model, the content of system role may be from system_prompt of the prompt template, and will be automatically inserted into messages.

As an example, depending on the model, if system_prompt of a prompt template is "You are Granite Chat, an AI language model developed by IBM. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior.", a message with system role having content the same as system_prompt is inserted.

Optionalmetadata

metadata?: Record<string, unknown>

Metadata for this call and any sub-calls (eg. a Chain calling an LLM). Keys should be strings, values should be JSON-serializable.

OptionalpromptIndex

promptIndex?: number

OptionalrecursionLimit

recursionLimit?: number

Maximum number of times a call can recurse. If not provided, defaults to 25.

OptionalrunId

runId?: string

Unique identifier for the tracer run for this call. If not provided, a new UUID will be generated.

OptionalrunName

runName?: string

Name for the tracer run for this call. Defaults to the name of the class.

Optionalsignal

signal?: AbortSignal

Abort signal for this call. If provided, the call will be aborted when the signal is aborted.

See

https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal

Optionalstreaming

streaming?: boolean

Optionaltags

tags?: string[]

Tags for this call and any sub-calls (eg. a Chain calling an LLM). You can use these to filter calls.

Optionaltimeout

timeout?: number

Timeout for this call in milliseconds.

Optionaltool_choice

tool_choice?: string | TextChatParameterTools

Specifies how the chat model should use tools.

Default

undefined

Possible values:
- "auto": The model may choose to use any of the provided tools, or none.
- "any": The model must use one of the provided tools.
- "none": The model must not use any tools.
- A string (not "auto", "any", or "none"): The name of a specific tool the model must use.
- An object: A custom schema specifying tool choice parameters. Specific to the provider.

Note: Not all providers support tool_choice. An error will be thrown
if used with an unsupported model.

OptionalwatsonxCallbacks

watsonxCallbacks?: RequestCallbacks<any>

Settings

Member Visibility

On This Page

Properties