Class CSVLoader

A class that extends the TextLoader class. It represents a document loader that loads documents from a CSV file. It has a constructor that takes a filePathOrBlob parameter representing the path to the CSV file or a Blob object, and an optional options parameter of type CSVLoaderOptions or a string representing the column to use as the document's pageContent.

Hierarchy

  • TextLoader
    • CSVLoader

Constructors

constructor

Properties

filePathOrBlob options

Methods

load loadAndSplit parse imports

Constructors

constructor

Properties

filePathOrBlob

filePathOrBlob: string | Blob

Protectedoptions

options: CSVLoaderOptions = {}

Methods

load

  • load(): Promise<Document<Record<string, any>>[]>

  • A method that loads the text file or blob and returns a promise that resolves to an array of Document instances. It reads the text from the file or blob using the readFile function from the node:fs/promises module or the text() method of the blob. It then parses the text using the parse() method and creates a Document instance for each parsed page. The metadata includes the source of the text (file path or blob) and, if there are multiple pages, the line number of each page.

    Returns Promise<Document<Record<string, any>>[]>

    A promise that resolves to an array of Document instances.

loadAndSplit

  • loadAndSplit(splitter?): Promise<Document<Record<string, any>>[]>

  • Parameters

    • Optionalsplitter: BaseDocumentTransformer<DocumentInterface<Record<string, any>>[], DocumentInterface<Record<string, any>>[]>

    Returns Promise<Document<Record<string, any>>[]>

    A Promise that resolves with an array of Document instances, each split according to the provided TextSplitter.

    Deprecated

    Use this.load() and splitter.splitDocuments() individually. Loads the documents and splits them using a specified text splitter.

Protectedparse

  • parse(raw): Promise<string[]>

  • A protected method that parses the raw CSV data and returns an array of strings representing the pageContent of each document. It uses the dsvFormat function from the d3-dsv module to parse the CSV data. If the column option is specified, it checks if the column exists in the CSV file and returns the values of that column as the pageContent. If the column option is not specified, it converts each row of the CSV data into key/value pairs and joins them with newline characters.

    Parameters

    • raw: string

      The raw CSV data to be parsed.

    Returns Promise<string[]>

    An array of strings representing the pageContent of each document.

Staticimports

  • imports(): Promise<{
        readFile: {
            (path: PathLike | FileHandle, options?: null | {
                encoding?: null;
                flag?: OpenMode;
            } & Abortable): Promise<Buffer>;
            (path: PathLike | FileHandle, options: BufferEncoding | {
                encoding: BufferEncoding;
                flag?: OpenMode | undefined;
            } & Abortable): Promise<string>;
            (path: PathLike | FileHandle, options?: null | BufferEncoding | ObjectEncodingOptions & Abortable & {
                flag?: OpenMode | undefined;
            }): Promise<string | Buffer>;
        };
    }>

  • A static method that imports the readFile function from the node:fs/promises module. It is used to dynamically import the function when needed. If the import fails, it throws an error indicating that the fs/promises module is not available in the current environment.

    Returns Promise<{
        readFile: {
            (path: PathLike | FileHandle, options?: null | {
                encoding?: null;
                flag?: OpenMode;
            } & Abortable): Promise<Buffer>;
            (path: PathLike | FileHandle, options: BufferEncoding | {
                encoding: BufferEncoding;
                flag?: OpenMode | undefined;
            } & Abortable): Promise<string>;
            (path: PathLike | FileHandle, options?: null | BufferEncoding | ObjectEncodingOptions & Abortable & {
                flag?: OpenMode | undefined;
            }): Promise<string | Buffer>;
        };
    }>

    A promise that resolves to an object containing the readFile function from the node:fs/promises module.

Settings

Member Visibility

On This Page

Constructors
Properties
Methods