PhoenixKitAgent.Attachments (PhoenixKitAgent v0.1.0)

Copy Markdown View Source

Files attached to a chat (spec §5.3, §9).

Like PhoenixKitAgent.Conversations, every function acts for a live browser session: the {session_token_uuid, user_uuid} reference and, in the options, impersonated: boolean (PhoenixKitAgent.Conversations has the checks that come first and their refusals: an impersonated or revoked session, a user without agent, a chat that is not the user's). The owner is the only one who adds a file to a chat, reads its row, discards it or is given the address of its bytes; an Owner or an Admin of the site is no exception here.

Adding a file

add/4 takes the file the Dock got from allow_upload (its temporary path and the name and type the browser gave), checks it, keeps it and starts its processing, and returns the row as pending. In order:

  1. the chat is the session user's; the upload is a readable, non-empty file no larger than max_attachment_mb (the size is read from the file, not believed from the caller);
  2. the content tells the kind (PhoenixKitAgent.Attachments.Detect): png/jpg/webp/heic, pdf, xlsx/csv, txt/md; anything else is :unsupported_type. A picture for a user whose model is not known to take images, while the module has no vision model to describe it, is :no_vision_model here, before anything is stored (see the processor);
  3. the chat has room: no more than max_attachments_per_message files that are not part of a message yet (a failed one takes no room there), and the user has no more than unsent_files (Attachments.Bounds) files of any status waiting in all their chats that are not deleted — a failed one stays until it is discarded, and counts here. The count and the row are made under a lock on the chat, so parallel uploads cannot pass it;
  4. the bytes are stored in the module's hidden Storage library, in the user's folder (PhoenixKitAgent.Library), uploaded by the user — a Manager with media.view_all neither sees nor reads them, and the permanent file token is refused; the file is not in Media. The same bytes of the same user are one Storage file, as in any library. Storage is given a neutral name for the file (the row keeps the user's), since Storage writes the name to its own log lines and job arguments;
  5. PhoenixKitAgent.Attachments.Processor works on the file in a task.

add/4 writes to Storage and may take long: call it off the LiveView process. The row's status goes pending → ready | failed; when it does the chat's topic (PhoenixKitAgent.Conversations.topic/1) hears {:agent_attachment_processed, conversation_uuid, attachment_uuid}, and the listener reads the row with get/3. error of a failed row is a code (see the processor), never text of the file.

The row add/4 returns is pending as it was when processing started, and a small file can be processed — and its message sent — before the caller has that return value. So a listener must not let it replace a row it already read as ready or failed: after add/4 returns, read the row again with get/3 (or keep whichever of the two is settled).

Sending

A ready attachment that belongs to the user and to the chat is passed to PhoenixKitAgent.Turns.start/6 by uuid; the turn links it to the user message it stores. Its extracted text goes with the message and stays in the history, the attachments of one message sharing agent_history_chars between them; a picture's pixels go only in the turn of that message (and, for a model that sees, are not described afterwards: later turns carry a line that names the picture and the assistant's own answer). When the provider refuses the first model call of a turn that carries pictures, the turn takes its files off the message again (PhoenixKitAgent.Turns.Runner) — only a refusal of the request itself (400, 413, 415, 422), not a missing model or a failure of the provider. So does a turn whose model is no longer known to take images: the pixels are not sent, the model is not called, and a notice (pictures_not_seen) says to attach the files again, to be described. After a turn has ended, list_unsent/3 is the list to show.

The bytes

Nobody but the owner gets at them, and only through url/3: an expiring address of the file in Storage, minted for the session's own scope.

Refusals

Those of PhoenixKitAgent.Conversations, and :file_unreadable, :empty_file, :too_large, :unsupported_type, :unsupported_encoding (a text, CSV or Markdown file that is not UTF-8: save it as UTF-8), :no_vision_model, :too_many, :storage_unavailable, :already_sent.

A row that is still pending after the processing deadline and a margin (the node was restarted, the task was lost) is settled as failed/timeout the next time the user's rows are read, so it neither blocks a message nor holds a place; the Dock offers to discard it like any failed one.

Summary

Types

Why a call was refused.

The file as the Dock got it from allow_upload. A content_type the browser claims may be there; it is not looked at, the content tells what the file is.

Functions

Adds upload to the session user's chat (see the moduledoc). Returns the pending row; the file's processing has started. Read the row again with get/3 afterwards: it may be settled already (see "Adding a file").

Removes a file that has not been sent yet from the chat, and its bytes from Storage unless another attachment (or anything else) still uses them; a file that is part of a message is {:error, :already_sent}.

The session user's attachment attachment_uuid.

The files of the chat that wait for a message, oldest first, whatever their status (a failed one is there to be seen and discarded).

Whether the model is shown this attachment as pixels: a ready picture, or a scanned PDF, that has no text.

An expiring address of the bytes of the user's attachment, minted for the session's own scope (variant: "original" by default, or a Storage variant such as "thumbnail" of a picture).

Types

refusal()

@type refusal() ::
  PhoenixKitAgent.Conversations.refusal()
  | :file_unreadable
  | :empty_file
  | :too_large
  | :unsupported_type
  | :unsupported_encoding
  | :no_vision_model
  | :too_many
  | :storage_unavailable
  | :already_sent

Why a call was refused.

upload()

@type upload() :: %{
  :path => Path.t(),
  :filename => String.t(),
  optional(:content_type) => String.t() | nil
}

The file as the Dock got it from allow_upload. A content_type the browser claims may be there; it is not looked at, the content tells what the file is.

path is trusted. It must be the temporary file of LiveView's upload (consume_uploaded_entries/3) and nothing a client sent: whatever regular file it names is read and stored in the user's chat. The filename is the client's and is only kept as a label.

Functions

add(session_ref, conversation_uuid, upload, opts)

Adds upload to the session user's chat (see the moduledoc). Returns the pending row; the file's processing has started. Read the row again with get/3 afterwards: it may be settled already (see "Adding a file").

discard(session_ref, attachment_uuid, opts)

@spec discard(PhoenixKitAgent.Conversations.session_ref(), term(), keyword()) ::
  :ok | {:error, refusal()}

Removes a file that has not been sent yet from the chat, and its bytes from Storage unless another attachment (or anything else) still uses them; a file that is part of a message is {:error, :already_sent}.

get(session_ref, attachment_uuid, opts)

The session user's attachment attachment_uuid.

list_unsent(session_ref, conversation_uuid, opts)

The files of the chat that wait for a message, oldest first, whatever their status (a failed one is there to be seen and discarded).

pixels?(attachment)

Whether the model is shown this attachment as pixels: a ready picture, or a scanned PDF, that has no text.

url(session_ref, attachment_uuid, opts)

@spec url(PhoenixKitAgent.Conversations.session_ref(), term(), keyword()) ::
  {:ok, String.t()} | {:error, refusal()}

An expiring address of the bytes of the user's attachment, minted for the session's own scope (variant: "original" by default, or a Storage variant such as "thumbnail" of a picture).