marvin.utilities.jinja

Module for Jinja utilities.

BaseEnvironment

BaseEnvironment provides a configurable environment for rendering Jinja templates.

This class encapsulates a Jinja environment with customizable global functions and template settings, allowing for flexible template rendering.

Attributes:

Name	Type	Description
environment	Environment	The Jinja environment for template rendering.
globals	dict[str, Any]	A dictionary of global functions and variables available in templates.

Example

Basic Usage of BaseEnvironment

env = BaseEnvironment()

rendered = env.render("Hello, {{ name }}!", name="World")
print(rendered)  # Output: Hello, World!

render

Renders a given template str or BaseTemplate with provided context.

Parameters:

Name	Type	Description	Default
template	Union[str, Template]	The template to be rendered.	required
**kwargs	Any	Context variables to be passed to the template.	{}

Returns:

Type	Description
str	The rendered template as a string.

Example

Basic Usage of BaseEnvironment.render

from marvin.utilities.jinja import Environment as jinja_env

rendered = jinja_env.render("Hello, {{ name }}!", name="World")
print(rendered) # Output: Hello, World!

Transcript

A Transcript is a model that represents a conversation involving multiple
roles as a single string. It can be parsed into discrete JSON messages.

Transcripts contain special tokens that indicate how to split the transcript
into discrete messages.

The first special token type indicates the message `role`. Default roles are
`|SYSTEM|`, `|HUMAN|`, `|USER|`, and `|ASSISTANT|`. When these tokens appear
at the start of a newline, all text following the token until the next
newline or token is considered part of the message with the given role.

The second special token type indicates the message `type`. By default, messages all have the `text` type. By supplying a token like `|IMAGE|`, you can indicate that a portion of the message is an image. Use `|TEXT|` to end the image portion and return to text. An

Attributes:
    content: The content of the transcript.
    roles: The roles involved in the transcript.
    environment: The jinja environment to use for rendering the transcript.

Example:
    Basic Usage of Transcript:
    ```python
    from marvin.utilities.jinja import Transcript

    transcript = Transcript(
        content="|SYSTEM| Hello, there!

|USER| Hello, yourself!", roles={"|SYSTEM|": "system", "|USER|": "user"}, ) print(transcript.render_to_messages()) # [ # BaseMessage(content='system: Hello, there!', role='system'), # BaseMessage(content='Hello, yourself!', role='user') # ] ```

split_text_by_tokens

Splits a given text by a list of tokens.

Parameters:

Name	Type	Description	Default
text	str	The text to be split. split_tokens: The tokens to split the text	required
by.	only_on_newline	If True, only match tokens that are either	required

Returns:

Type	Description
list[tuple[str, str]]	A list of tuples containing the token and the text following it.

Example

Basic Usage of split_text_by_tokens ```python from marvin.utilities.jinja import split_text_by_tokens

text = "Hello, World!" split_tokens = ["Hello", "World"] pairs = split_text_by_tokens(text, split_tokens) print(pairs) # Output: [("Hello", ", "), ("World", "!")] ```