Marvin API reference

Image

render_for_transcript

Renders a JSON representation of the image for use in a Transcript object, including IMAGE and TEXT tags.

Model

A Pydantic model that can be instantiated from a natural language string, in addition to keyword arguments.

from_text_async async classmethod

Class method to create an instance of the model from a natural language string.

Parameters:

Name	Type	Description	Default
text	str	The natural language string to convert into an instance of the model.	required
instructions	str	Specific instructions for the conversion. Defaults to None.	None
model_kwargs	dict	Additional keyword arguments for the language model. Defaults to None.	None
client	AsyncMarvinClient	The client to use for the AI function.	None

Returns:

Name	Type	Description
Model	Model	An instance of the model.

Example

from marvin.ai.text import Model
class Location(Model):
    '''A location'''
    city: str
    state: str
    country: str

await Location.from_text_async("big apple, ny, usa")

caption

Generates a caption for an image using a language model synchronously.

Parameters:

Name	Type	Description	Default
data	Union[Image, List[Image]]	The image or images to caption.	required
instructions	str	Instructions for the caption generation.	None
model_kwargs	dict	Additional arguments for the language model.	None

Returns:

Name	Type	Description
str	str	Generated caption.

caption_async async

Generates a caption for a set of images using a language model.

Parameters:

Name	Type	Description	Default
data	Union[Image, List[Image]]	The image or images to caption.	required
instructions	str	Instructions for the caption generation.	None
model_kwargs	dict	Additional arguments for the language model.	None

Returns:

Name	Type	Description
str	str	Generated caption.

cast

Converts the input data into the specified type.

This function uses a language model to convert the input data into a specified type. The conversion process can be guided by specific instructions. The function also supports additional arguments for the language model.

Parameters:

Name	Type	Description	Default
data	FN_INPUT_TYPES	Union[str, Image, list[Union[str, Image]]]: the data to which the function will be applied.	required
target	type	The type to convert the data into. If none is provided but instructions are provided, str is assumed.	None
instructions	str	Specific instructions for the conversion. Defaults to None.	None
model_kwargs	dict	Additional keyword arguments for the language model. Defaults to None.	None
client	AsyncMarvinClient	The client to use for the AI function.	None

Returns:

Name	Type	Description
T	T	The converted data of the specified type.

cast_async async

Converts the input data into the specified type.

This function uses a language model to convert the input data into a specified type. The conversion process can be guided by specific instructions. The function also supports additional arguments for the language model.

Parameters:

Name	Type	Description	Default
data	FN_INPUT_TYPES	Union[str, Image, list[Union[str, Image]]]: the data to which the function will be applied.	required
target	type	The type to convert the data into. If none is provided but instructions are provided, str is assumed.	None
instructions	str	Specific instructions for the conversion. Defaults to None.	None
model_kwargs	dict	Additional keyword arguments for the language model. Defaults to None.	None
client	AsyncMarvinClient	The client to use for the AI function.	None

Returns:

Name	Type	Description
T	T	The converted data of the specified type.

classifier

Class decorator that modifies the behavior of an Enum class to classify a string.

This decorator modifies the call method of the Enum class to use the marvin.classify function instead of the default Enum behavior. This allows the Enum class to classify a string based on its members.

Parameters:

Name	Type	Description	Default
cls	Enum	The Enum class to be decorated.	None
instructions	str	Instructions for the AI on how to perform the classification.	None
model_kwargs	dict	Additional keyword arguments to pass to the model.	None

Returns:

Name	Type	Description
Enum		The decorated Enum class with modified call method.

Raises:

Type	Description
AssertionError	If the decorated class is not a subclass of Enum.

classify

Classifies the provided data based on the provided labels.

This function uses a language model with a logit bias to classify the input data. The logit bias constrains the language model's response to a single token, making this function highly efficient for classification tasks. The function will always return one of the provided labels.

Parameters:

Name	Type	Description	Default
data	FN_INPUT_TYPES	Union[str, Image, list[Union[str, Image]]]: the data to which the function will be applied.	required
labels	Union[Enum, list[T], type]	The labels to classify the data into.	required
instructions	str	Specific instructions for the classification. Defaults to None.	None
return_index	bool	Whether to return the index of the label instead of the label itself.	False
model_kwargs	dict	Additional keyword arguments for the language model. Defaults to None.	None
client	AsyncMarvinClient	The client to use for the AI function.	None

Returns:

Type	Description
Union[T, int]	Union[T, int]: The label or index that the data was classified into.

classify_async async

Classifies the provided data based on the provided labels.

This function uses a language model with a logit bias to classify the input data. The logit bias constrains the language model's response to a single token, making this function highly efficient for classification tasks. The function will always return one of the provided labels.

Parameters:

Name	Type	Description	Default
data	FN_INPUT_TYPES	Union[str, Image, list[Union[str, Image]]]: the data to which the function will be applied.	required
labels	Union[Enum, list[T], type]	The labels to classify the data into.	required
instructions	str	Specific instructions for the classification. Defaults to None.	None
model_kwargs	dict	Additional keyword arguments for the language model. Defaults to None.	None
client	AsyncMarvinClient	The client to use for the AI function.	None

Returns:

Type	Description
Union[T, int]	Union[T, int]: The label or index that the data was classified into.

extract

Extracts entities of a specific type from the provided data.

This function uses a language model to identify and extract entities of the specified type from the input data. The extracted entities are returned as a list.

Note that either a target type or instructions must be provided (or both). If only instructions are provided, the target type is assumed to be a string.

Parameters:

Name	Type	Description	Default
data	FN_INPUT_TYPES	Union[str, Image, list[Union[str, Image]]]: the data to which the function will be applied.	required
target	type	The type of entities to extract.	None
instructions	str	Specific instructions for the extraction. Defaults to None.	None
model_kwargs	dict	Additional keyword arguments for the language model. Defaults to None.	None
client	AsyncMarvinClient	The client to use for the AI function.	None

Returns:

Name	Type	Description
list	list[T]	A list of extracted entities of the specified type.

extract_async async

Extracts entities of a specific type from the provided data.

This function uses a language model to identify and extract entities of the specified type from the input data. The extracted entities are returned as a list.

Note that either a target type or instructions must be provided (or both). If only instructions are provided, the target type is assumed to be a string.

Parameters:

Name	Type	Description	Default
data	FN_INPUT_TYPES	Union[str, Image, list[Union[str, Image]]]: the data to which the function will be applied.	required
target	type	The type of entities to extract.	None
instructions	str	Specific instructions for the extraction. Defaults to None.	None
model_kwargs	dict	Additional keyword arguments for the language model. Defaults to None.	None
client	MarvinClient	The client to use for the AI function.	None

Returns:

Name	Type	Description
list	list[T]	A list of extracted entities of the specified type.

fn

Converts a Python function into an AI function using a decorator.

This decorator allows a Python function to be converted into an AI function. The AI function uses a language model to generate its output.

Parameters:

Name	Type	Description	Default
func	Callable	The function to be converted. Defaults to None.	None
model_kwargs	dict	Additional keyword arguments for the language model. Defaults to None.	None
client	AsyncMarvinClient	The client to use for the AI function.	None

Returns:

Name	Type	Description
Callable	Callable	The converted AI function.

Example

@fn
def list_fruit(n:int) -> list[str]:
    '''generates a list of n fruit'''

list_fruit(3) # ['apple', 'banana', 'orange']

generate

Generates a list of 'n' items of the provided type or based on instructions.

Either a type or instructions must be provided. If instructions are provided without a type, the type is assumed to be a string. The function generates at least 'n' items.

Parameters:

Name	Type	Description	Default
target	type	The type of items to generate. Defaults to None.	None
instructions	str	Instructions for the generation. Defaults to None.	None
n	int	The number of items to generate. Defaults to 1.	1
use_cache	bool	If True, the function will cache the last 100 responses for each (target, instructions, and temperature) and use those to avoid repetition on subsequent calls. Defaults to True.	True
temperature	float	The temperature for the generation. Defaults to 1.	1
model_kwargs	dict	Additional keyword arguments for the language model. Defaults to None.	None
client	AsyncMarvinClient	The client to use for the AI function.	None

Returns:

Name	Type	Description
list	list[T]	A list of generated items.

generate_async async

Generates a list of 'n' items of the provided type or based on instructions.

Either a type or instructions must be provided. If instructions are provided without a type, the type is assumed to be a string. The function generates at least 'n' items.

Parameters:

Name	Type	Description	Default
target	type	The type of items to generate. Defaults to None.	None
instructions	str	Instructions for the generation. Defaults to None.	None
n	int	The number of items to generate. Defaults to 1.	1
use_cache	bool	If True, the function will cache the last 100 responses for each (target, instructions, and temperature) and use those to avoid repetition on subsequent calls. Defaults to True.	True
temperature	float	The temperature for the generation. Defaults to 1.	1
model_kwargs	dict	Additional keyword arguments for the language model. Defaults to None.	None
client	AsyncMarvinClient	The client to use for the AI function.	None

Returns:

Name	Type	Description
list	list[T]	A list of generated items.

image

A decorator that transforms a function's output into an image.

This decorator takes a function that returns a string, and uses that string as instructions to generate an image. The generated image is then returned.

The decorator can be used with or without parentheses. If used without parentheses, the decorated function's output is used as the instructions for the image. If used with parentheses, an optional literal argument can be provided. If literal is set to True, the function's output is used as the literal instructions for the image, without any modifications.

Parameters:

Name	Type	Description	Default
fn	callable	The function to decorate. If None, the decorator is being used with parentheses, and fn will be provided later.	None
literal	bool	Whether to use the function's output as the literal instructions for the image. Defaults to False.	False

Returns:

Name	Type	Description
callable		The decorated function.

model

Class decorator for instantiating a Pydantic model from a string.

This decorator allows a Pydantic model to be instantiated from a string. It's equivalent to subclassing the Model class.

Parameters:

Name	Type	Description	Default
type_	Union[Type[M], None]	The type of the Pydantic model. Defaults to None.	None
instructions	str	Specific instructions for the conversion.	None
model_kwargs	dict	Additional keyword arguments for the language model. Defaults to None.	None

Returns:

Type	Description
Union[Type[M], Callable[[Type[M]], Type[M]]]	Union[Type[M], Callable[[Type[M]], Type[M]]]: The decorated Pydantic model.

paint

Generates an image based on the provided instructions and context.

This function uses the DALLE-3 API to generate an image based on the provided instructions and context. By default, the API modifies prompts to add detail and style. This behavior can be disabled by setting literal=True.

Parameters:

Name	Type	Description	Default
instructions	str	The instructions for the image generation. Defaults to None.	None
context	dict	The context for the image generation. Defaults to None.	None
literal	bool	Whether to disable the API's default behavior of modifying prompts. Defaults to False.	False
model_kwargs	dict	Additional keyword arguments for the language model. Defaults to None.	None

Returns:

Name	Type	Description
ImagesResponse		The response from the DALLE-3 API, which includes the generated image.

paint_async async

Generates an image based on the provided instructions and context.

This function uses the DALLE-3 API to generate an image based on the provided instructions and context. By default, the API modifies prompts to add detail and style. This behavior can be disabled by setting literal=True.

Parameters:

Name	Type	Description	Default
instructions	str	The instructions for the image generation. Defaults to None.	None
context	dict	The context for the image generation. Defaults to None.	None
literal	bool	Whether to disable the API's default behavior of modifying prompts. Defaults to False.	False
model_kwargs	dict	Additional keyword arguments for the language model. Defaults to None.	None

Returns:

Name	Type	Description
ImagesResponse		The response from the DALLE-3 API, which includes the generated image.

speak

Generates audio from text using an AI.

This function uses an AI to generate audio from the provided text. The voice used for the audio can be specified.

Parameters:

Name	Type	Description	Default
text	str	The text to generate audio from.	required
voice	Literal['alloy', 'echo', 'fable', 'onyx', 'nova', 'shimmer']	The voice to use for the audio. Defaults to None.	None
stream	bool	Whether to stream the audio. If False, the audio can not be saved or played until it has all been generated. If True, .save() and .play() can be called immediately.	True
model_kwargs	dict	Additional keyword arguments for the language model. Defaults to None.	None

Returns:

Name	Type	Description
Audio	Audio	The generated audio.

speak_async async

Generates audio from text using an AI.

This function uses an AI to generate audio from the provided text. The voice used for the audio can be specified.

Parameters:

Name	Type	Description	Default
text	str	The text to generate audio from.	required
voice	Literal['alloy', 'echo', 'fable', 'onyx', 'nova', 'shimmer']	The voice to use for the audio. Defaults to None.	None
stream	bool	Whether to stream the audio. If False, the audio can not be saved or played until it has all been generated. If True, .save() and .play() can be called immediately.	True
model_kwargs	dict	Additional keyword arguments for the language model. Defaults to None.	None

Returns:

Name	Type	Description
Audio	Audio	The generated audio.

speech

Function decorator that generates audio from the wrapped function's return value. The voice used for the audio can be specified.

Parameters:

Name	Type	Description	Default
fn	Callable	The function to wrap. Defaults to None.	None
voice	str	The voice to use for the audio. Defaults to None.	None
stream	bool	Whether to stream the audio. If False, the audio can not be saved or played until it has all been generated. If True, .save() and .play() can be called immediately.	True

Returns:

Name	Type	Description
Callable	Callable	The wrapped function.

transcribe

Transcribes audio from a file.

This function converts audio from a file to text.

transcribe_async async

Transcribes audio from a file.

This function converts audio from a file to text.