OptionalcallbacksOptionalconfigurableRuntime values for attributes previously made configurable on this Runnable, or sub-Runnables.
OptionalcontextIf specified, context will be inserted into messages. Depending on the model, context may be inserted
into the content with system role; or into the content of the last message of user role.
In the example, context "Today is Wednesday" is inserted as such
content of user becomes "Today is Wednesday. Who are you and which day is tomorrow?".
OptionalheadersThe id_or_name can be either the deployment_id that identifies the deployment or a serving_name that
allows a predefined URL to be used to post a prediction. The deployment must reference a prompt template with
input_mode chat.
The WML instance that is associated with the deployment will be used for limits and billing (if a paid plan).
OptionalmaxMaximum number of parallel calls to make.
OptionalmaxThe messages for this chat session. You cannot specify system role in the messages. Depending on the
model, the content of system role may be from system_prompt of the prompt template, and will be
automatically inserted into messages.
As an example, depending on the model, if system_prompt of a prompt template is "You are Granite Chat, an AI
language model developed by IBM. You are a cautious assistant. You carefully follow instructions. You are
helpful and harmless and you follow ethical guidelines and promote positive behavior.", a message with system
role having content the same as system_prompt is inserted.
OptionalmetadataMetadata for this call and any sub-calls (eg. a Chain calling an LLM). Keys should be strings, values should be JSON-serializable.
OptionalpromptOptionalrecursionMaximum number of times a call can recurse. If not provided, defaults to 25.
OptionalrunUnique identifier for the tracer run for this call. If not provided, a new UUID will be generated.
OptionalrunName for the tracer run for this call. Defaults to the name of the class.
OptionalsignalAbort signal for this call. If provided, the call will be aborted when the signal is aborted.
OptionalstreamingOptionaltagsTags for this call and any sub-calls (eg. a Chain calling an LLM). You can use these to filter calls.
OptionaltimeoutTimeout for this call in milliseconds.
Optionaltool_Specifies how the chat model should use tools.
undefined
Possible values:
- "auto": The model may choose to use any of the provided tools, or none.
- "any": The model must use one of the provided tools.
- "none": The model must not use any tools.
- A string (not "auto", "any", or "none"): The name of a specific tool the model must use.
- An object: A custom schema specifying tool choice parameters. Specific to the provider.
Note: Not all providers support tool_choice. An error will be thrown
if used with an unsupported model.
Optionalwatsonx
Callbacks for this call and any sub-calls (eg. a Chain calling an LLM). Tags are passed to all callbacks, metadata is passed to handle*Start callbacks.