Files attached to a chat (spec §5.3, §9).
Like PhoenixKitAgent.Conversations, every function acts for a live
browser session: the {session_token_uuid, user_uuid} reference and, in
the options, impersonated: boolean (PhoenixKitAgent.Conversations has the
checks that come first and their refusals: an impersonated or revoked
session, a user without agent, a chat that is not the user's). The owner
is the only one who adds a file to a chat, reads its row, discards it or is
given the address of its bytes; an Owner or an Admin of the site is no
exception here.
Adding a file
add/4 takes the file the Dock got from allow_upload (its temporary
path and the name and type the browser gave), checks it, keeps it and starts
its processing, and returns the row as pending. In order:
- the chat is the session user's; the upload is a readable, non-empty
file no larger than
max_attachment_mb(the size is read from the file, not believed from the caller); - the content tells the kind (
PhoenixKitAgent.Attachments.Detect): png/jpg/webp/heic, pdf, xlsx/csv, txt/md; anything else is:unsupported_type. A picture for a user whose model is not known to take images, while the module has no vision model to describe it, is:no_vision_modelhere, before anything is stored (see the processor); - the chat has room: no more than
max_attachments_per_messagefiles that are not part of a message yet (a failed one takes no room there), and the user has no more thanunsent_files(Attachments.Bounds) files of any status waiting in all their chats that are not deleted — a failed one stays until it is discarded, and counts here. The count and the row are made under a lock on the chat, so parallel uploads cannot pass it; - the bytes are stored in the module's hidden Storage library, in the
user's folder (
PhoenixKitAgent.Library), uploaded by the user — a Manager withmedia.view_allneither sees nor reads them, and the permanent file token is refused; the file is not in Media. The same bytes of the same user are one Storage file, as in any library. Storage is given a neutral name for the file (the row keeps the user's), since Storage writes the name to its own log lines and job arguments; PhoenixKitAgent.Attachments.Processorworks on the file in a task.
add/4 writes to Storage and may take long: call it off the LiveView
process. The row's status goes pending → ready | failed; when it
does the chat's topic (PhoenixKitAgent.Conversations.topic/1) hears
{:agent_attachment_processed, conversation_uuid, attachment_uuid}, and the
listener reads the row with get/3. error of a failed row is a code (see
the processor), never text of the file.
The row add/4 returns is pending as it was when processing started, and
a small file can be processed — and its message sent — before the caller has
that return value. So a listener must not let it replace a row it already
read as ready or failed: after add/4 returns, read the row again with
get/3 (or keep whichever of the two is settled).
Sending
A ready attachment that belongs to the user and to the chat is passed to
PhoenixKitAgent.Turns.start/6 by uuid; the turn links it to the user
message it stores. Its extracted text goes with the message and stays in the
history, the attachments of one message sharing agent_history_chars
between them; a picture's pixels go only in the turn of that message (and,
for a model that sees, are not described afterwards: later turns carry a line
that names the picture and the assistant's own answer). When the provider
refuses the first model call of a turn that carries pictures, the turn takes
its files off the message again (PhoenixKitAgent.Turns.Runner) — only a
refusal of the request itself (400, 413, 415, 422), not a missing model or
a failure of the provider. So does a turn whose model is no longer known to
take images: the pixels are not sent, the model is not called, and a notice
(pictures_not_seen) says to attach the files again, to be described. After
a turn has ended, list_unsent/3 is the list to show.
The bytes
Nobody but the owner gets at them, and only through url/3: an expiring
address of the file in Storage, minted for the session's own scope.
Refusals
Those of PhoenixKitAgent.Conversations, and :file_unreadable,
:empty_file, :too_large, :unsupported_type, :unsupported_encoding (a
text, CSV or Markdown file that is not UTF-8: save it as UTF-8),
:no_vision_model, :too_many, :storage_unavailable, :already_sent.
A row that is still pending after the processing deadline and a margin (the
node was restarted, the task was lost) is settled as failed/timeout the
next time the user's rows are read, so it neither blocks a message nor holds
a place; the Dock offers to discard it like any failed one.
Summary
Types
Why a call was refused.
The file as the Dock got it from allow_upload. A content_type the browser
claims may be there; it is not looked at, the content tells what the file is.
Functions
Adds upload to the session user's chat (see the moduledoc). Returns the
pending row; the file's processing has started. Read the row again with
get/3 afterwards: it may be settled already (see "Adding a file").
Removes a file that has not been sent yet from the chat, and its bytes from
Storage unless another attachment (or anything else) still uses them; a file
that is part of a message is {:error, :already_sent}.
The session user's attachment attachment_uuid.
The files of the chat that wait for a message, oldest first, whatever their status (a failed one is there to be seen and discarded).
Whether the model is shown this attachment as pixels: a ready picture, or
a scanned PDF, that has no text.
An expiring address of the bytes of the user's attachment, minted for the
session's own scope (variant: "original" by default, or a Storage
variant such as "thumbnail" of a picture).
Types
@type refusal() :: PhoenixKitAgent.Conversations.refusal() | :file_unreadable | :empty_file | :too_large | :unsupported_type | :unsupported_encoding | :no_vision_model | :too_many | :storage_unavailable | :already_sent
Why a call was refused.
@type upload() :: %{ :path => Path.t(), :filename => String.t(), optional(:content_type) => String.t() | nil }
The file as the Dock got it from allow_upload. A content_type the browser
claims may be there; it is not looked at, the content tells what the file is.
path is trusted. It must be the temporary file of LiveView's upload
(consume_uploaded_entries/3) and nothing a client sent: whatever regular
file it names is read and stored in the user's chat. The filename is the
client's and is only kept as a label.
Functions
@spec add(PhoenixKitAgent.Conversations.session_ref(), term(), upload(), keyword()) :: {:ok, PhoenixKitAgent.Schemas.Attachment.t()} | {:error, refusal()}
Adds upload to the session user's chat (see the moduledoc). Returns the
pending row; the file's processing has started. Read the row again with
get/3 afterwards: it may be settled already (see "Adding a file").
@spec discard(PhoenixKitAgent.Conversations.session_ref(), term(), keyword()) :: :ok | {:error, refusal()}
Removes a file that has not been sent yet from the chat, and its bytes from
Storage unless another attachment (or anything else) still uses them; a file
that is part of a message is {:error, :already_sent}.
@spec get(PhoenixKitAgent.Conversations.session_ref(), term(), keyword()) :: {:ok, PhoenixKitAgent.Schemas.Attachment.t()} | {:error, refusal()}
The session user's attachment attachment_uuid.
@spec list_unsent(PhoenixKitAgent.Conversations.session_ref(), term(), keyword()) :: {:ok, [PhoenixKitAgent.Schemas.Attachment.t()]} | {:error, refusal()}
The files of the chat that wait for a message, oldest first, whatever their status (a failed one is there to be seen and discarded).
@spec pixels?(PhoenixKitAgent.Schemas.Attachment.t()) :: boolean()
Whether the model is shown this attachment as pixels: a ready picture, or
a scanned PDF, that has no text.
@spec url(PhoenixKitAgent.Conversations.session_ref(), term(), keyword()) :: {:ok, String.t()} | {:error, refusal()}
An expiring address of the bytes of the user's attachment, minted for the
session's own scope (variant: "original" by default, or a Storage
variant such as "thumbnail" of a picture).