Class BaseSageMakerContentHandler<InputType, OutputType>Abstract

A handler class to transform input from LLM to a format that SageMaker endpoint expects. Similarily, the class also handles transforming output from the SageMaker endpoint to a format that LLM class expects.

Example:

class ContentHandler implements ContentHandlerBase<string, string> {
contentType = "application/json"
accepts = "application/json"

transformInput(prompt: string, modelKwargs: Record<string, unknown>) {
const inputString = JSON.stringify({
prompt,
...modelKwargs
})
return Buffer.from(inputString)
}

transformOutput(output: Uint8Array) {
const responseJson = JSON.parse(Buffer.from(output).toString("utf-8"))
return responseJson[0].generated_text
}

}

Type Parameters

  • InputType
  • OutputType

Constructors

Properties

accepts: string = "text/plain"
contentType: string = "text/plain"

Methods

  • Transforms the prompt and model arguments into a specific format for sending to SageMaker.

    Parameters

    • prompt: InputType

      The prompt to be transformed.

    • modelKwargs: Record<string, unknown>

      Additional arguments.

    Returns Promise<Uint8Array>

    A promise that resolves to the formatted data for sending.