Interface ChatCohereCallOptions

interface ChatCohereCallOptions {
    accepts?: "text/event-stream";
    callbacks?: Callbacks;
    chatHistory?: Message[];
    citationQuality?: ChatRequestCitationQuality;
    configurable?: Record<string, any>;
    connectors?: ChatConnector[];
    conversationId?: string;
    documents?: ChatDocument[];
    forceSingleStep?: boolean;
    frequencyPenalty?: number;
    k?: number;
    maxConcurrency?: number;
    maxInputTokens?: number;
    maxTokens?: number;
    metadata?: Record<string, unknown>;
    model?: string;
    p?: number;
    preamble?: string;
    presencePenalty?: number;
    promptTruncation?: ChatRequestPromptTruncation;
    rawPrompting?: boolean;
    recursionLimit?: number;
    responseFormat?: ResponseFormat;
    returnPrompt?: boolean;
    runId?: string;
    runName?: string;
    safetyMode?: ChatRequestSafetyMode;
    searchQueriesOnly?: boolean;
    seed?: number;
    signal?: AbortSignal;
    stop?: string[];
    stopSequences?: string[];
    streamUsage?: boolean;
    tags?: string[];
    temperature?: number;
    timeout?: number;
    toolResults?: ToolResult[];
    tool_choice?: ToolChoice;
    tools?: ChatCohereToolType[];
}

Hierarchy (view full)

BaseChatModelCallOptions
Partial<Omit<Cohere.ChatRequest, "message" | "tools">>
Partial<Omit<Cohere.ChatStreamRequest, "message" | "tools">>
Pick<ChatCohereInput, "streamUsage">
- ChatCohereCallOptions
  - CohereChatCallOptions

Properties

`Optional`accepts

accepts?: "text/event-stream"

Pass text/event-stream to receive the streamed response as server-sent events. The default is \n delimited events.

`Optional`callbacks

callbacks?: Callbacks

Callbacks for this call and any sub-calls (eg. a Chain calling an LLM). Tags are passed to all callbacks, metadata is passed to handle*Start callbacks.

`Optional`chatHistory

chatHistory?: Message[]

A list of previous messages between the user and the model, giving the model conversational context for responding to the user's message.

Each item represents a single message in the chat history, excluding the current user turn. It has two properties: role and message. The role identifies the sender (CHATBOT, SYSTEM, or USER), while the message contains the text content.

The chat_history parameter should not be used for SYSTEM messages in most cases. Instead, to add a SYSTEM role message at the beginning of a conversation, the preamble parameter should be used.

Compatible Deployments: Cohere Platform, Azure, AWS Sagemaker/Bedrock, Private Deployments

`Optional`citationQuality

citationQuality?: ChatRequestCitationQuality

Defaults to "accurate".

Dictates the approach taken to generating citations as part of the RAG flow by allowing the user to specify whether they want "accurate" results, "fast" results or no results.

Compatible Deployments: Cohere Platform, Azure, AWS Sagemaker/Bedrock, Private Deployments

`Optional`configurable

configurable?: Record<string, any>

Runtime values for attributes previously made configurable on this Runnable, or sub-Runnables.

`Optional`connectors

connectors?: ChatConnector[]

Accepts {"id": "web-search"}, and/or the "id" for a custom connector, if you've created one.

When specified, the model's reply will be enriched with information found by querying each of the connectors (RAG).

Compatible Deployments: Cohere Platform

`Optional`conversationId

conversationId?: string

An alternative to chat_history.

Providing a conversation_id creates or resumes a persisted conversation with the specified ID. The ID can be any non empty string.

Compatible Deployments: Cohere Platform

`Optional`documents

documents?: ChatDocument[]

A list of relevant documents that the model can cite to generate a more accurate reply. Each document is a string-string dictionary.

Example:

[
  { "title": "Tall penguins", "text": "Emperor penguins are the tallest." },
  { "title": "Penguin habitats", "text": "Emperor penguins only live in Antarctica." },
]

Keys and values from each document will be serialized to a string and passed to the model. The resulting generation will include citations that reference some of these documents.

Some suggested keys are "text", "author", and "date". For better generation quality, it is recommended to keep the total word count of the strings in the dictionary to under 300 words.

An id field (string) can be optionally supplied to identify the document in the citations. This field will not be passed to the model.

An _excludes field (array of strings) can be optionally supplied to omit some key-value pairs from being shown to the model. The omitted fields will still show up in the citation object. The "_excludes" field will not be passed to the model.

See 'Document Mode' in the guide for more information.

Compatible Deployments: Cohere Platform, Azure, AWS Sagemaker/Bedrock, Private Deployments

`Optional`forceSingleStep

forceSingleStep?: boolean

Forces the chat to be single step. Defaults to false.

`Optional`frequencyPenalty

frequencyPenalty?: number

Defaults to 0.0, min value of 0.0, max value of 1.0.

Used to reduce repetitiveness of generated tokens. The higher the value, the stronger a penalty is applied to previously present tokens, proportional to how many times they have already appeared in the prompt or prior generation.

Compatible Deployments: Cohere Platform, Azure, AWS Sagemaker/Bedrock, Private Deployments

`Optional`k

k?: number

Ensures only the top k most likely tokens are considered for generation at each step. Defaults to 0, min value of 0, max value of 500.

Compatible Deployments: Cohere Platform, Azure, AWS Sagemaker/Bedrock, Private Deployments

`Optional`maxConcurrency

maxConcurrency?: number

Maximum number of parallel calls to make.

`Optional`maxInputTokens

maxInputTokens?: number

The maximum number of input tokens to send to the model. If not specified, max_input_tokens is the model's context length limit minus a small buffer.

Input will be truncated according to the prompt_truncation parameter.

Compatible Deployments: Cohere Platform

`Optional`maxTokens

maxTokens?: number

The maximum number of tokens the model will generate as part of the response. Note: Setting a low value may result in incomplete generations.

Compatible Deployments: Cohere Platform, Azure, AWS Sagemaker/Bedrock, Private Deployments

`Optional`metadata

metadata?: Record<string, unknown>

Metadata for this call and any sub-calls (eg. a Chain calling an LLM). Keys should be strings, values should be JSON-serializable.

`Optional`model

model?: string

Defaults to command-r-plus-08-2024.

The name of a compatible Cohere model or the ID of a fine-tuned model.

Compatible Deployments: Cohere Platform, Private Deployments

`Optional`p

p?: number

Ensures that only the most likely tokens, with total probability mass of p, are considered for generation at each step. If both k and p are enabled, p acts after k. Defaults to 0.75. min value of 0.01, max value of 0.99.

Compatible Deployments: Cohere Platform, Azure, AWS Sagemaker/Bedrock, Private Deployments

`Optional`preamble

preamble?: string

When specified, the default Cohere preamble will be replaced with the provided one. Preambles are a part of the prompt used to adjust the model's overall behavior and conversation style, and use the SYSTEM role.

The SYSTEM role is also used for the contents of the optional chat_history= parameter. When used with the chat_history= parameter it adds content throughout a conversation. Conversely, when used with the preamble= parameter it adds content at the start of the conversation only.

Compatible Deployments: Cohere Platform, Azure, AWS Sagemaker/Bedrock, Private Deployments

`Optional`presencePenalty

presencePenalty?: number

Defaults to 0.0, min value of 0.0, max value of 1.0.

Used to reduce repetitiveness of generated tokens. Similar to frequency_penalty, except that this penalty is applied equally to all tokens that have already appeared, regardless of their exact frequencies.

Compatible Deployments: Cohere Platform, Azure, AWS Sagemaker/Bedrock, Private Deployments

`Optional`promptTruncation

promptTruncation?: ChatRequestPromptTruncation

Defaults to AUTO when connectors are specified and OFF in all other cases.

Dictates how the prompt will be constructed.

With prompt_truncation set to "AUTO", some elements from chat_history and documents will be dropped in an attempt to construct a prompt that fits within the model's context length limit. During this process the order of the documents and chat history will be changed and ranked by relevance.

With prompt_truncation set to "AUTO_PRESERVE_ORDER", some elements from chat_history and documents will be dropped in an attempt to construct a prompt that fits within the model's context length limit. During this process the order of the documents and chat history will be preserved as they are inputted into the API.

With prompt_truncation set to "OFF", no elements will be dropped. If the sum of the inputs exceeds the model's context length limit, a TooManyTokens error will be returned.

Compatible Deployments:

AUTO: Cohere Platform Only
AUTO_PRESERVE_ORDER: Azure, AWS Sagemaker/Bedrock, Private Deployments

`Optional`rawPrompting

rawPrompting?: boolean

When enabled, the user's prompt will be sent to the model without any pre-processing.

Compatible Deployments: Cohere Platform, Azure, AWS Sagemaker/Bedrock, Private Deployments

`Optional`recursionLimit

recursionLimit?: number

Maximum number of times a call can recurse. If not provided, defaults to 25.

`Optional`responseFormat

responseFormat?: ResponseFormat

`Optional`returnPrompt

returnPrompt?: boolean

The prompt is returned in the prompt response field when this is enabled.

`Optional`runId

runId?: string

Unique identifier for the tracer run for this call. If not provided, a new UUID will be generated.

`Optional`runName

runName?: string

Name for the tracer run for this call. Defaults to the name of the class.

`Optional`safetyMode

safetyMode?: ChatRequestSafetyMode

Used to select the safety instruction inserted into the prompt. Defaults to CONTEXTUAL. When NONE is specified, the safety instruction will be omitted.

Safety modes are not yet configurable in combination with tools, tool_results and documents parameters.

Note: This parameter is only compatible with models Command R 08-2024, Command R+ 08-2024 and newer.

Compatible Deployments: Cohere Platform, Azure, AWS Sagemaker/Bedrock, Private Deployments

`Optional`searchQueriesOnly

searchQueriesOnly?: boolean

Defaults to false.

When true, the response will only contain a list of generated search queries, but no search will take place, and no reply from the model to the user's message will be generated.

Compatible Deployments: Cohere Platform, Azure, AWS Sagemaker/Bedrock, Private Deployments

`Optional`seed

seed?: number

If specified, the backend will make a best effort to sample tokens deterministically, such that repeated requests with the same seed and parameters should return the same result. However, determinism cannot be totally guaranteed.

Compatible Deployments: Cohere Platform, Azure, AWS Sagemaker/Bedrock, Private Deployments

`Optional`signal

signal?: AbortSignal

Abort signal for this call. If provided, the call will be aborted when the signal is aborted.

See

https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal

`Optional`stop

stop?: string[]

Stop tokens to use for this call. If not provided, the default stop tokens for the model will be used.

`Optional`stopSequences

stopSequences?: string[]

A list of up to 5 strings that the model will use to stop generation. If the model generates a string that matches any of the strings in the list, it will stop generating tokens and return the generated text up to that point not including the stop sequence.

Compatible Deployments: Cohere Platform, Azure, AWS Sagemaker/Bedrock, Private Deployments

`Optional`streamUsage

streamUsage?: boolean

Whether or not to include token usage when streaming. This will include an extra chunk at the end of the stream with eventType: "stream-end" and the token usage in usage_metadata.

Default

{true}

`Optional`tags

tags?: string[]

Tags for this call and any sub-calls (eg. a Chain calling an LLM). You can use these to filter calls.

`Optional`temperature

temperature?: number

Defaults to 0.3.

A non-negative float that tunes the degree of randomness in generation. Lower temperatures mean less random generations, and higher temperatures mean more random generations.

Randomness can be further maximized by increasing the value of the p parameter.

Compatible Deployments: Cohere Platform, Azure, AWS Sagemaker/Bedrock, Private Deployments

`Optional`timeout

timeout?: number

Timeout for this call in milliseconds.

`Optional`toolResults

toolResults?: ToolResult[]

A list of results from invoking tools recommended by the model in the previous chat turn. Results are used to produce a text response and will be referenced in citations. When using tool_results, tools must be passed as well. Each tool_result contains information about how it was invoked, as well as a list of outputs in the form of dictionaries.

Note: outputs must be a list of objects. If your tool returns a single object (eg {"status": 200}), make sure to wrap it in a list.

tool_results = [
  {
    "call": {
      "name": <tool name>,
      "parameters": {
        <param name>: <param value>
      }
    },
    "outputs": [{
      <key>: <value>
    }]
  },
  ...
]

Note: Chat calls with tool_results should not be included in the Chat history to avoid duplication of the message text.

Compatible Deployments: Cohere Platform, Azure, AWS Sagemaker/Bedrock, Private Deployments

`Optional`tool_choice

tool_choice?: ToolChoice

Specifies how the chat model should use tools.

Default

undefined

Possible values:
- "auto": The model may choose to use any of the provided tools, or none.
- "any": The model must use one of the provided tools.
- "none": The model must not use any tools.
- A string (not "auto", "any", or "none"): The name of a specific tool the model must use.
- An object: A custom schema specifying tool choice parameters. Specific to the provider.

Note: Not all providers support tool_choice. An error will be thrown
if used with an unsupported model.

`Optional`tools

tools?: ChatCohereToolType[]

Interface ChatCohereCallOptions

Hierarchy (view full)

Index

Properties

Properties

Optionalaccepts

Optionalcallbacks

OptionalchatHistory

OptionalcitationQuality

Optionalconfigurable

Optionalconnectors

OptionalconversationId

Optionaldocuments

OptionalforceSingleStep

OptionalfrequencyPenalty

Optionalk

OptionalmaxConcurrency

OptionalmaxInputTokens

OptionalmaxTokens

Optionalmetadata

Optionalmodel

Optionalp

Optionalpreamble

OptionalpresencePenalty

OptionalpromptTruncation

OptionalrawPrompting

OptionalrecursionLimit

OptionalresponseFormat

OptionalreturnPrompt

OptionalrunId

OptionalrunName

OptionalsafetyMode

OptionalsearchQueriesOnly

Optionalseed

Optionalsignal

See

Optionalstop

OptionalstopSequences

OptionalstreamUsage

Default

Optionaltags

Optionaltemperature

Optionaltimeout

OptionaltoolResults

Optionaltool_choice

Default

Optionaltools

Settings

On This Page

`Optional`accepts

`Optional`callbacks

`Optional`chatHistory

`Optional`citationQuality

`Optional`configurable

`Optional`connectors

`Optional`conversationId

`Optional`documents

`Optional`forceSingleStep

`Optional`frequencyPenalty

`Optional`k

`Optional`maxConcurrency

`Optional`maxInputTokens

`Optional`maxTokens

`Optional`metadata

`Optional`model

`Optional`p

`Optional`preamble

`Optional`presencePenalty

`Optional`promptTruncation

`Optional`rawPrompting

`Optional`recursionLimit

`Optional`responseFormat

`Optional`returnPrompt

`Optional`runId

`Optional`runName

`Optional`safetyMode

`Optional`searchQueriesOnly

`Optional`seed

`Optional`signal

`Optional`stop

`Optional`stopSequences

`Optional`streamUsage

`Optional`tags

`Optional`temperature

`Optional`timeout

`Optional`toolResults

`Optional`tool_choice

`Optional`tools