Skip to content

Marvin API reference

Image

render_for_transcript

Renders a JSON representation of the image for use in a Transcript object, including IMAGE and TEXT tags.

Model

A Pydantic model that can be instantiated from a natural language string, in addition to keyword arguments.

from_text_async async classmethod

Class method to create an instance of the model from a natural language string.

Parameters:

Name Type Description Default
text str

The natural language string to convert into an instance of the model.

required
instructions str

Specific instructions for the conversion. Defaults to None.

None
model_kwargs dict

Additional keyword arguments for the language model. Defaults to None.

None
client AsyncMarvinClient

The client to use for the AI function.

None

Returns:

Name Type Description
Model Model

An instance of the model.

Example
from marvin.ai.text import Model
class Location(Model):
    '''A location'''
    city: str
    state: str
    country: str

await Location.from_text_async("big apple, ny, usa")

caption

Generates a caption for an image using a language model synchronously.

Parameters:

Name Type Description Default
data Union[Image, List[Image]]

The image or images to caption.

required
instructions str

Instructions for the caption generation.

None
model_kwargs dict

Additional arguments for the language model.

None

Returns:

Name Type Description
str str

Generated caption.

caption_async async

Generates a caption for a set of images using a language model.

Parameters:

Name Type Description Default
data Union[Image, List[Image]]

The image or images to caption.

required
instructions str

Instructions for the caption generation.

None
model_kwargs dict

Additional arguments for the language model.

None

Returns:

Name Type Description
str str

Generated caption.

cast

Converts the input data into the specified type.

This function uses a language model to convert the input data into a specified type. The conversion process can be guided by specific instructions. The function also supports additional arguments for the language model.

Parameters:

Name Type Description Default
data FN_INPUT_TYPES

Union[str, Image, list[Union[str, Image]]]: the data to which the function will be applied.

required
target type

The type to convert the data into. If none is provided but instructions are provided, str is assumed.

None
instructions str

Specific instructions for the conversion. Defaults to None.

None
model_kwargs dict

Additional keyword arguments for the language model. Defaults to None.

None
client AsyncMarvinClient

The client to use for the AI function.

None

Returns:

Name Type Description
T T

The converted data of the specified type.

cast_async async

Converts the input data into the specified type.

This function uses a language model to convert the input data into a specified type. The conversion process can be guided by specific instructions. The function also supports additional arguments for the language model.

Parameters:

Name Type Description Default
data FN_INPUT_TYPES

Union[str, Image, list[Union[str, Image]]]: the data to which the function will be applied.

required
target type

The type to convert the data into. If none is provided but instructions are provided, str is assumed.

None
instructions str

Specific instructions for the conversion. Defaults to None.

None
model_kwargs dict

Additional keyword arguments for the language model. Defaults to None.

None
client AsyncMarvinClient

The client to use for the AI function.

None

Returns:

Name Type Description
T T

The converted data of the specified type.

classifier

Class decorator that modifies the behavior of an Enum class to classify a string.

This decorator modifies the call method of the Enum class to use the marvin.classify function instead of the default Enum behavior. This allows the Enum class to classify a string based on its members.

Parameters:

Name Type Description Default
cls Enum

The Enum class to be decorated.

None
instructions str

Instructions for the AI on how to perform the classification.

None
model_kwargs dict

Additional keyword arguments to pass to the model.

None

Returns:

Name Type Description
Enum

The decorated Enum class with modified call method.

Raises:

Type Description
AssertionError

If the decorated class is not a subclass of Enum.

classify

Classifies the provided data based on the provided labels.

This function uses a language model with a logit bias to classify the input data. The logit bias constrains the language model's response to a single token, making this function highly efficient for classification tasks. The function will always return one of the provided labels.

Parameters:

Name Type Description Default
data FN_INPUT_TYPES

Union[str, Image, list[Union[str, Image]]]: the data to which the function will be applied.

required
labels Union[Enum, list[T], type]

The labels to classify the data into.

required
instructions str

Specific instructions for the classification. Defaults to None.

None
return_index bool

Whether to return the index of the label instead of the label itself.

False
model_kwargs dict

Additional keyword arguments for the language model. Defaults to None.

None
client AsyncMarvinClient

The client to use for the AI function.

None

Returns:

Type Description
Union[T, int]

Union[T, int]: The label or index that the data was classified into.

classify_async async

Classifies the provided data based on the provided labels.

This function uses a language model with a logit bias to classify the input data. The logit bias constrains the language model's response to a single token, making this function highly efficient for classification tasks. The function will always return one of the provided labels.

Parameters:

Name Type Description Default
data FN_INPUT_TYPES

Union[str, Image, list[Union[str, Image]]]: the data to which the function will be applied.

required
labels Union[Enum, list[T], type]

The labels to classify the data into.

required
instructions str

Specific instructions for the classification. Defaults to None.

None
model_kwargs dict

Additional keyword arguments for the language model. Defaults to None.

None
client AsyncMarvinClient

The client to use for the AI function.

None

Returns:

Type Description
Union[T, int]

Union[T, int]: The label or index that the data was classified into.

extract

Extracts entities of a specific type from the provided data.

This function uses a language model to identify and extract entities of the specified type from the input data. The extracted entities are returned as a list.

Note that either a target type or instructions must be provided (or both). If only instructions are provided, the target type is assumed to be a string.

Parameters:

Name Type Description Default
data FN_INPUT_TYPES

Union[str, Image, list[Union[str, Image]]]: the data to which the function will be applied.

required
target type

The type of entities to extract.

None
instructions str

Specific instructions for the extraction. Defaults to None.

None
model_kwargs dict

Additional keyword arguments for the language model. Defaults to None.

None
client AsyncMarvinClient

The client to use for the AI function.

None

Returns:

Name Type Description
list list[T]

A list of extracted entities of the specified type.

extract_async async

Extracts entities of a specific type from the provided data.

This function uses a language model to identify and extract entities of the specified type from the input data. The extracted entities are returned as a list.

Note that either a target type or instructions must be provided (or both). If only instructions are provided, the target type is assumed to be a string.

Parameters:

Name Type Description Default
data FN_INPUT_TYPES

Union[str, Image, list[Union[str, Image]]]: the data to which the function will be applied.

required
target type

The type of entities to extract.

None
instructions str

Specific instructions for the extraction. Defaults to None.

None
model_kwargs dict

Additional keyword arguments for the language model. Defaults to None.

None
client MarvinClient

The client to use for the AI function.

None

Returns:

Name Type Description
list list[T]

A list of extracted entities of the specified type.

fn

Converts a Python function into an AI function using a decorator.

This decorator allows a Python function to be converted into an AI function. The AI function uses a language model to generate its output.

Parameters:

Name Type Description Default
func Callable

The function to be converted. Defaults to None.

None
model_kwargs dict

Additional keyword arguments for the language model. Defaults to None.

None
client AsyncMarvinClient

The client to use for the AI function.

None

Returns:

Name Type Description
Callable Callable

The converted AI function.

Example
@fn
def list_fruit(n:int) -> list[str]:
    '''generates a list of n fruit'''

list_fruit(3) # ['apple', 'banana', 'orange']

generate

Generates a list of 'n' items of the provided type or based on instructions.

Either a type or instructions must be provided. If instructions are provided without a type, the type is assumed to be a string. The function generates at least 'n' items.

Parameters:

Name Type Description Default
target type

The type of items to generate. Defaults to None.

None
instructions str

Instructions for the generation. Defaults to None.

None
n int

The number of items to generate. Defaults to 1.

1
use_cache bool

If True, the function will cache the last 100 responses for each (target, instructions, and temperature) and use those to avoid repetition on subsequent calls. Defaults to True.

True
temperature float

The temperature for the generation. Defaults to 1.

1
model_kwargs dict

Additional keyword arguments for the language model. Defaults to None.

None
client AsyncMarvinClient

The client to use for the AI function.

None

Returns:

Name Type Description
list list[T]

A list of generated items.

generate_async async

Generates a list of 'n' items of the provided type or based on instructions.

Either a type or instructions must be provided. If instructions are provided without a type, the type is assumed to be a string. The function generates at least 'n' items.

Parameters:

Name Type Description Default
target type

The type of items to generate. Defaults to None.

None
instructions str

Instructions for the generation. Defaults to None.

None
n int

The number of items to generate. Defaults to 1.

1
use_cache bool

If True, the function will cache the last 100 responses for each (target, instructions, and temperature) and use those to avoid repetition on subsequent calls. Defaults to True.

True
temperature float

The temperature for the generation. Defaults to 1.

1
model_kwargs dict

Additional keyword arguments for the language model. Defaults to None.

None
client AsyncMarvinClient

The client to use for the AI function.

None

Returns:

Name Type Description
list list[T]

A list of generated items.

image

A decorator that transforms a function's output into an image.

This decorator takes a function that returns a string, and uses that string as instructions to generate an image. The generated image is then returned.

The decorator can be used with or without parentheses. If used without parentheses, the decorated function's output is used as the instructions for the image. If used with parentheses, an optional literal argument can be provided. If literal is set to True, the function's output is used as the literal instructions for the image, without any modifications.

Parameters:

Name Type Description Default
fn callable

The function to decorate. If None, the decorator is being used with parentheses, and fn will be provided later.

None
literal bool

Whether to use the function's output as the literal instructions for the image. Defaults to False.

False

Returns:

Name Type Description
callable

The decorated function.

model

Class decorator for instantiating a Pydantic model from a string.

This decorator allows a Pydantic model to be instantiated from a string. It's equivalent to subclassing the Model class.

Parameters:

Name Type Description Default
type_ Union[Type[M], None]

The type of the Pydantic model. Defaults to None.

None
instructions str

Specific instructions for the conversion.

None
model_kwargs dict

Additional keyword arguments for the language model. Defaults to None.

None

Returns:

Type Description
Union[Type[M], Callable[[Type[M]], Type[M]]]

Union[Type[M], Callable[[Type[M]], Type[M]]]: The decorated Pydantic model.

paint

Generates an image based on the provided instructions and context.

This function uses the DALLE-3 API to generate an image based on the provided instructions and context. By default, the API modifies prompts to add detail and style. This behavior can be disabled by setting literal=True.

Parameters:

Name Type Description Default
instructions str

The instructions for the image generation. Defaults to None.

None
context dict

The context for the image generation. Defaults to None.

None
literal bool

Whether to disable the API's default behavior of modifying prompts. Defaults to False.

False
model_kwargs dict

Additional keyword arguments for the language model. Defaults to None.

None

Returns:

Name Type Description
ImagesResponse

The response from the DALLE-3 API, which includes the generated image.

paint_async async

Generates an image based on the provided instructions and context.

This function uses the DALLE-3 API to generate an image based on the provided instructions and context. By default, the API modifies prompts to add detail and style. This behavior can be disabled by setting literal=True.

Parameters:

Name Type Description Default
instructions str

The instructions for the image generation. Defaults to None.

None
context dict

The context for the image generation. Defaults to None.

None
literal bool

Whether to disable the API's default behavior of modifying prompts. Defaults to False.

False
model_kwargs dict

Additional keyword arguments for the language model. Defaults to None.

None

Returns:

Name Type Description
ImagesResponse

The response from the DALLE-3 API, which includes the generated image.

speak

Generates audio from text using an AI.

This function uses an AI to generate audio from the provided text. The voice used for the audio can be specified.

Parameters:

Name Type Description Default
text str

The text to generate audio from.

required
voice Literal['alloy', 'echo', 'fable', 'onyx', 'nova', 'shimmer']

The voice to use for the audio. Defaults to None.

None
stream bool

Whether to stream the audio. If False, the audio can not be saved or played until it has all been generated. If True, .save() and .play() can be called immediately.

True
model_kwargs dict

Additional keyword arguments for the language model. Defaults to None.

None

Returns:

Name Type Description
Audio Audio

The generated audio.

speak_async async

Generates audio from text using an AI.

This function uses an AI to generate audio from the provided text. The voice used for the audio can be specified.

Parameters:

Name Type Description Default
text str

The text to generate audio from.

required
voice Literal['alloy', 'echo', 'fable', 'onyx', 'nova', 'shimmer']

The voice to use for the audio. Defaults to None.

None
stream bool

Whether to stream the audio. If False, the audio can not be saved or played until it has all been generated. If True, .save() and .play() can be called immediately.

True
model_kwargs dict

Additional keyword arguments for the language model. Defaults to None.

None

Returns:

Name Type Description
Audio Audio

The generated audio.

speech

Function decorator that generates audio from the wrapped function's return value. The voice used for the audio can be specified.

Parameters:

Name Type Description Default
fn Callable

The function to wrap. Defaults to None.

None
voice str

The voice to use for the audio. Defaults to None.

None
stream bool

Whether to stream the audio. If False, the audio can not be saved or played until it has all been generated. If True, .save() and .play() can be called immediately.

True

Returns:

Name Type Description
Callable Callable

The wrapped function.

transcribe

Transcribes audio from a file.

This function converts audio from a file to text.

transcribe_async async

Transcribes audio from a file.

This function converts audio from a file to text.