Office for Pithagoras

@pithagoras/office lets you and the agent work on the same Word (.docx) and Excel (.xlsx) documents. You edit them in the real OnlyOffice Document Server editor, embedded in the portal, which the extension runs in a container through Docker for extensions; the agent reads and edits the same documents through tools of its own, in an editor of its own beside you, so you see its changes as it makes them. An Office tab holds the library.

This extension is not made or endorsed by Ascensio System SIA. "ONLYOFFICE" names what it runs.

What it needs

• Pithagoras 0.3.0 or later. For agents to answer comments and the editor's chat, a portal whose extension API has chats.start, chats.lastAnswer, chats.owner and http.person (thecodacus/pithagoras#339); on an older one the rest of Office works, and the Office page, the editor's bar and the agents' office tools say that agents do not answer there. Such a portal cannot say whose a chat is, so there a chat's panel lists documents, and its agent has office tools, only where the chat works in an agent's home or a project's folder itself, not in a folder inside one.
• Docker for extensions (pithagoras-docker 1.3.0 or later). It ships with the portal but is not on by default; installing this extension from npm brings it along.
• A Linux host with Docker Engine 28.0 or later that the portal can reach (its socket): the portal on the machine itself, or in Docker (both of Pithagoras' Compose files). On macOS or Windows the portal has to run in Docker.
• Recommended: 1 GB of RAM. About 700 MiB were measured while one document is edited, and 770 MiB to 1.05 GB with three documents open in the agents' editors (the most there are at once: 540-975 MiB of it the agents' browser), 800-940 MiB with people's editors of two of them open as well. The settings page shows the memory all of Office uses while it runs: the Document Server's and, while it runs, the agents' browser's. That browser starts on demand, stops 10 minutes after its last editor closed, and is capped at 3 GB. The figure is what docker stats shows: the containers' memory less the file cache the kernel can drop again.
• Disk for the images (onlyoffice/documentserver, several GB, Node's node:22.23.4-alpine3.23, about 50 MB, for the relay below, and for the agents' editor chromedp/headless-shell:157.0.8103.15, Chromium's headless shell, about 540 MB), which the portal pulls on the first start, all at once, with the progress of each shown in Settings.
• Behind a reverse proxy (one that ends TLS, say), a browser that says which page a request comes from, as every current one does (Sec-Fetch-Site), or a proxy that passes X-Forwarded-Host and X-Forwarded-Proto (or keeps Host): the Document Server names the addresses it hands the editor (the opened document among them) by the address the browser used, and an http:// address on an https:// page is refused by the browser, which shows as "Download failed".

Installing

In Settings → Extensions → Discover → Install by name: npm:@pithagoras/office. Then open Settings → Office and press Start container (that switches Office on too). The image is pulled on the first start (onlyoffice/documentserver:10.0.0.1 by default, the full tag of the build this version was tested with, never :latest or a shorter tag like 10.0, which move to each new build: a new build can change the editor's callback and token behaviour, so it is yours to change on purpose, in the same page). A new Document Server container answers within seconds but then makes its fonts for about two minutes and restarts its own services; until it is done, Settings says it is starting and the editor waits for it, as an editor opened before would lose what is typed in it. The same start downloads the agents' editor beside the Document Server. Settings → Office shows Office as one thing: stopped, downloading (the progress of every download under way), starting, running, or failed, with the error naming the part that failed (the Document Server or the agents' editor). Until the agents' editor is there the agents read the saved files and cannot edit; an install that had the Document Server before gets it with the next Start container, and the page says it is missing until then. It is pinned by its digest: a tag that came to name another build is refused, not run, and said as the error.

What you get

• The library, on the portal's Office tab: create blank Word documents (text in Arial 11, set in the document's own defaults, so a built-in table style shows one font in every row; a document you add keeps its own) and spreadsheets (Calibri 11, with the workbook's styles part, which the editor needs for a table over a bold or linked cell; a workbook you add keeps its own), open one full screen in the embedded editor, sort the list by name, modified, size or type, and download a copy. Every document lives in an agent's home or a project's folder and belongs to that agent or project; there are no documents of nobody's. Every .docx and .xlsx in the agents' homes and the projects' folders (not the folders other chats work in, nor the rest of the workspace) is indexed when the extension starts, and again with Rescan, in place: the file is the document, and editing it edits that file. New document on the Office tab makes it in the home of the agent New starts your chats with (the first agent's, unless you picked another on its Agent page); the one in a chat's panel, and the agent's office_create, make it in that chat's agent home or project folder, as the portal says whose the chat is. One whose file is gone (a git checkout of another branch, say) stays listed for a week, marked, with its owner and grants, and is the same document again when the file comes back. Deleting one moves its real file into the library's trash (.trash/<time>-<id>/ in the data folder), with document.json, which says where it was, and what was kept for it; the entry is kept for 7 days and then removed. There is no button to restore it, but the file can be moved back by hand.

• A panel beside each chat with the documents that chat's agent or project sees (below). A document opened there opens in the editor right in the panel, beside the conversation, so you watch the agent's edits while you talk to it: ← All documents goes back to the list, and a small button opens it in the Office tab instead. The panel can be docked left, right or at the bottom and dragged as narrow or wide as you like; the editor fills it. A card in the chat for each call of the agent's tools that names its document opens that document in the same panel (a call that lists the documents opens the list). The same document open in the panel and in the Office tab (or in two browser tabs) is one editing session seen twice.

• The agent's tools, in the chats of agents and projects (a chat in a folder inside a project is the project's, as the portal says whose a chat is; the panel beside it lists by the same rule): office_list, office_create, office_read, office_edit, office_format, office_comments and office_screenshot. An agent sees the documents in its own folder (its home, or its project's) and the ones it was granted by a mention; another agent's, and the files in another agent's or project's folder, are not there for it, and asking for one gets the same answer as for a document that does not exist. What an agent makes goes into its folder and belongs to it, not to one of its chats. A file in its folder is read through the portal as the agent itself would read it: with the sandbox on, one the agent could not read is refused, and a link is not followed. A document in another agent's home or project folder that you opened to it by a mention is read with the portal's rights (your mention says it may), and changed only where the agent could write that folder itself: with the sandbox on, it reads and answers in comments.

• The agent edits beside you, live: each agent (each agent's or project's folder) that works in a document has an OnlyOffice editor of its own open in it, in a browser of Office's own (the agents' editor), in the same editing session as the people in the document. People see it there as " (agent)", with its cursor. office_read gives the document as it stands in that editor, with what people typed and have not saved yet; office_edit makes its edits there, so they show in your editor at once, and nothing you typed is lost or written over. An edit is made whole or not at all: every quote, sheet name and position is checked against the document as it stands first, and an edit that fails after it changed something is taken back. Where even that fails (a person changed the same sheets meanwhile), the agent hears so (error:partial) and reads the document again. In a spreadsheet, what you changed yourself after the agent's step (a cell you typed into or formatted, a row or a sheet of the call's you wrote into, frozen panes you moved) is never taken back with the agent's call: it stays as you made it, and the agent's answer (error:partial) names it. After an edit the Document Server is asked to save the session, and the agent hears the new revision; that save holds what you typed until then too. Word: replace quoted text, add paragraphs (in any paragraph style the document has) or a table after a paragraph, remove a paragraph (a document keeps one: its only paragraph removed leaves it empty). A quote is looked for in the body's text only: not in headers, footers, footnotes or a table of contents' entries (which repeat the headings), and a text found more than once is refused unless the call says which place it means (nth, from 1, in reading order). A list item's number or bullet is no part of its text: office_read gives the list in the brackets ([5] (bullet) Point). How a Word document looks is office_format's, a tool of its own (so a chat that never formats can leave it lazy in Settings → Tools, and pay nothing for it): characters (font, size, bold, italic, underline, strike, superscript, subscript, colour, highlight, links on and off, character styles, and taking a text's own formatting off), paragraphs (alignment, spacing, line spacing, indents, bulleted and numbered lists with their levels and a new start, page break before, keep with next), styles (new paragraph and character styles; an existing one changes only when the call says change: true, and the answer counts the paragraphs that use it), tables (a table style, width, column widths, header rows, borders, cell fills, alignment, row heights, merged cells), the page (size, orientation, margins, headers and footers with the page number and count, a first page of its own), sections, page breaks, footnotes and a table of contents. A table of contents follows the document: every office_edit and office_format call that changes a document holding one brings its entries and page numbers up to date as the last part of the same step (so a failed call's Undo takes that back too), and its answer says so; what was changed by hand in the entries is overwritten then, as Word's own update does. A comment on an entry is made again on the text of the new entry (the one with its text, else the one at its place), with its author, time, state and replies, so its thread and its chat go on; the answer says that too. A table of contents that lists nothing yet is found by the editor's English text for it ("No table of contents entries found.", as Office's editors and an English Word write it): one made in a Word of another language is not brought up to date until it lists something. With two tables of contents that list nothing yet in sections of different widths, the second one's new entries take their page-number tab at the first one's width. Each update lays the whole document out: in the agent's editor about 17 seconds for 340 pages of text (a file of 130 kB), so an office_edit or office_format call has more time the larger the document's file is (two minutes at most). Each op names its place by a quote, as office_edit's do. A call is made whole or not at all: everything it names is checked first (quotes, styles, tables and cells, what a header holds), and a call that fails after it changed something is taken back by the editor's own Undo, as one step. A header or footer that holds more than text (a picture, a table, a field other than the page number and count) is replaced only with replace: true; setting the header of a later section that showed the one before it gives it one of its own, and the earlier one stays ("" there gives it an empty one, so its pages show none). A link is changed or taken off whole: a quote of part of a link is refused. A style is never based on itself, also not through others. A section break's type (continuous, evenPage, oddPage) is how the section after it starts, and a page size alone keeps the section's orientation. Lengths are points, or a text with its unit ("2.5cm", "10mm", "1in"), colours "#RRGGBB". Borders are single lines (the editor draws no other kind through its API). When a call moves the pages (page setup, a style changed, a section, a page break, a table of contents), its answer says so, and office_screenshot shows them. office_read with formatting: true also says how a Word document looks: the document's defaults and the styles in use (each as it is itself; what it inherits from another is not added), each section's page, margins, header and footer (or the earlier section's it shows), the footnotes and where each one's mark is (a table's in its cell), and each paragraph's and run's own formatting (not what its style gives it; what it sets off or to zero is said too, "not bold", "indent left 0cm", as it hides the style's), a numbered item's number as the editor gives it ((numbered "3."), so a list that goes on and one that starts again are told apart), a table's style, header rows, merged cells and borders, and each cell's own fill, vertical alignment and borders with its paragraphs' and runs' own formatting (B2: fill #FFFF00, valign center; (right) "42": bold, under the table's rows; a table or a content control inside a cell is the cell's text, its paragraphs counted and read with the cell's); 50,000 characters of it at most. Not read: the saved file's formatting, when Office is stopped. Excel: write values and formulas into a range of any sheet (the editor computes the formulas; =Data!A1 and ='My data'!A1 reach other sheets), insert and delete rows and columns ({insertRows: "5:6"} makes new rows 5 and 6; formulas follow the cells they name, and those that named deleted cells turn #REF!), and make, rename, move, copy, hide, show and delete sheets, up to 20 of these in one call (Excel's rules for names: at most 31 characters, none of []:*?/\, no two alike in any case; one sheet always stays visible). A hidden sheet is moved by showing it, moving it and hiding it again. Deleting a sheet turns the formulas that name it into #REF!, as in Excel. office_read lists every sheet with its position and whether it is hidden, and reads one sheet or all of them. A change to the sheets shows in your editor once the Document Server has let it through, a moment after the call makes it. A sheet the edit deletes is hidden and renamed ~deleted … at once, and goes when the edit's other ops are made. Deleted rows and columns cannot be taken back, so a call deletes them last, after every op that can be (a call that does otherwise is refused); where the call fails after a delete was made, what it changed below or right of the deleted lines stays too (its cells moved), and the answer names those ops. A call inserts and deletes 20,000 rows and columns at most. Rows and columns are inserted and deleted while you are in the workbook too: the agent's answer then names the people in it, as their cells move. If you are in the middle of typing into a cell when the agent's insert moves it, OnlyOffice 10.0 puts what you type at the cell's old address (a row off after a row inserted above it), and your own editor may show the cell empty until you reload it: the Document Server's co-editing, reported as ONLYOFFICE/DocumentServer#3867. Your editor's view can also differ when the agent writes the very cell you are typing in. **How a spreadsheet looks** is office_format's too: cells ("A1:D9", or whole columns "C:D" and rows "5:6") take a number format ("#,##0.00", "0%", "dd.mm.yyyy"), font, size, bold, italic, underline, strike, superscript and subscript (in cells that hold text: OnlyOffice sets them on a text's characters), colour, fill, alignment, wrap, text rotation, borders (single, double, dashed or dotted lines), merging and splitting, and taking all of a cell's formatting off (clear, merges too, and the conditional formats of those cells: OnlyOffice takes them off with it, and a call taken back cannot make them again, so the answer names them; the values stay). Columns take a width (in characters, as the sheet shows it), rows a height (in points), and both can be hidden, shown and fitted to their contents; frozen panes keep the rows above and the columns left of a cell in view. A call is made whole or not at all: it is checked against the workbook first (superscript on a number, a freeze of a hidden sheet, more than 20,000 cells, whole columns and rows counting the used cells in them, are refused before anything changes), each op reads what it changes first, and where a later op of the call fails, what it changed is put back. Some things cannot be put back, because OnlyOffice's API cannot read them: the alignment and the borders, the formatting of a text whose characters differ (part of it bold, say) and the width of a hidden column. Where a later op fails, they stay as the call set them, and the answer (error:partial) names them. OnlyOffice's API takes no border off and sets no "general" alignment back: only clear takes them off, with everything else. A merge keeps the top-left cell's value (as in Excel); taking a merge back puts the others' values back. A column width or row height put back is the column's or row's own from then on (the API sets none back to the sheet's default), which looks the same. Besides the cells' looks, office_format makes **conditional formats** (a rule on cells, or a whole column: a comparison with a value, a formula, a colour scale, data bars, the top or bottom n, duplicate or unique values; with a fill, a font colour, bold, italic, underline or strike where the kind has a look), **data validation** (a list of entries or the cells that hold them, whole numbers, decimals, dates and text lengths with a condition such as ">= 0" or "between 1 and 10"; or taking it off; not an error message, which OnlyOffice's API sets in the agent's own editor only and sends nowhere), **links** in cells (http, https or mailto; a cell that holds a text keeps it, one that holds a number or a formula needs the text to show, as OnlyOffice would put the address there), **charts** of a range with its header row and label column (column, stacked column, bar, stacked bar, line, pie, doughnut, area and scatter; title, axis titles, legend, place and size), a **filter** on a range with one column's criteria (or taking the sheet's off) and a **formatted table** of a range, in the editor's default style (the API has no other). office_edit **sorts** the rows of a range by up to three of its columns (the cells' formats and formulas move with their rows; merged cells are refused) and makes, changes and deletes **named ranges**. The same rule holds for them as for formats: a call is checked first, and where a later op fails what the call made is put back: a rule the call added is deleted (one a person changed since stays, and is named), validation goes back to what the cells had (cell by cell; the messages of a person's validation, and whether it ignores blanks, shows a dropdown in the cell, an error alert and an input message, are lost with the validation the call replaces, and the answer says so; OnlyOffice 10.0's editor takes no validation message from the API, ONLYOFFICE/DocumentServer#3869), a link to the text, look and link the cell had (the link's look replaces the cell's font, so a bold cell with a link comes back plain, and the answer names its font; a link a person changed meanwhile keeps its cell as they made it), a sort to the values, formulas and readable formats the range had (the alignment and borders it moved cannot be read back, so the answer names them), a name to its old reference or none. OnlyOffice's API takes **no chart, no table away again, and a filter only whole** (filter: "none" takes the sheet's off), and a rule cleared cannot be made again: these go last in a call, and the answer names the ones made before the op that failed, which stay. Whether a table overlaps another one is not known to the API, which ignores it without a word, and a table over a cell with a link is refused (in a workbook without a styles part, xl/styles.xml, OnlyOffice 10.0 can no longer read the formats of a linked or bold cell a table covers, ONLYOFFICE/DocumentServer#3870; the workbooks Office makes have the part and do not break, but the editor's API cannot tell the two apart, so the refusal stays for all of them, and office_read leaves out a cell whose format it cannot read, and says so). A sort moves the links of its cells with them, and where a later op fails the links stay where the sort put them (the answer names them). A sort re-orders the rows of its range: while you are in the workbook the agent's answer names the people in it, as it does after inserted or deleted rows (merging cells moves none, so it does not). A text in a rule or a list of a validation cannot hold a double quote (the file would hold a broken formula), a sheet with more than 2,000 conditional formats takes no new one (each is made by reading all of them). A cell under a conditional format reads the rule's fill as its own in office_read (the API gives what is drawn); taking back a format of such a cell can then set that fill on it. office_read with formatting: true says how each sheet looks: the font most cells have, the cells that look otherwise (grouped into ranges), merged cells, the columns' widths and rows' heights that differ from the sheet's, hidden ones and frozen panes, its conditional formats, the validation of the cells it lists (of the used range, 20,000 cells at most), its links, its filter and its charts, 50,000 characters of it at most. It cannot say the alignment or the borders (OnlyOffice has no getter for them), a formatted table (its style, its total row), the data a chart shows or the workbook's named ranges (none is listed by the API; the agent deletes the ones it made by name): office_screenshot shows the first three. **Pictures** are office_edit's too: the agent puts a PNG, JPEG or GIF from its own folder into a Word document ({after?, picture, width?, height?, alt?}: inline in a paragraph of its own after the paragraph that holds after, or at the end) or a spreadsheet ({sheet?, picture, at: "B2", width?, height?, alt?}: its top left corner in that cell). picture is a path below the agent's home or the project's folder, never an address; the file is opened through the portal as the agent itself would read it (with the sandbox on, one it could not read is refused, and a link that leads out of the folder is not followed), told by its first bytes and not by its name, and checked by its header before anything goes to the editor: 5 MB a picture, 10 MB and 20 pictures a call (MB of 1,000,000 bytes; a file used twice counts twice), 40 megapixels (a small PNG can unpack to gigabytes; a GIF's frames count where they reach past its screen) and 100 frames of a GIF (the editor shows its first). The picture goes to the editor as it is, up to its end marker (bytes after it are left out), so the saved file holds the file's own bytes, **its metadata too**: a photo's EXIF (where it was taken, the camera) goes into the document with it. Its data is outside the ops' own 1 MB limit. Its size is its own at 96 dpi (a JPEG turned by its EXIF orientation as it is shown), made smaller to fit the page's text area (Word) or 16 × 12 cm (a sheet), or the width and height given (a length like "6cm", a number is points, 0.2 to 50 cm; one given keeps the proportions, and the other side may not come out past 50 cm); its alt text is the file's name unless alt is given ("" gives none). A call that fails after its pictures is taken back like the other ops: in Word by the editor's Undo, in a sheet by deleting each picture the call made, except one you moved, resized or described meanwhile, which stays as you made it and which the answer names. (OnlyOffice's builder API has no way to delete a sheet's drawing; Office does it with the plugin method RemoveOleObject, which in 10.0.0.1 deletes any drawing of a sheet by its id.) office_read lists the pictures, with or without formatting: under a Word paragraph with their size and alt text (under a table by their cell, A2: a picture …, for one put after a cell's text), per sheet after its cells (Pictures: B2 4 × 2 cm "Logo", 50 at most); several in one place carry #1, #2. **The pictures a document has**, a person's and the agent's own, are changed by {editPicture, …} ops of office_edit: deleted, moved, sized or given new alt text. No crop, rotation or wrapping.

  Naming a picture. The agent names a picture by what office_read lists it under, and the two sit side by side:

  office_read says	the op that changes that picture
  [2] Their text and under it a picture 4 × 2 cm, alt text "Their logo"	{editPicture: 2, width: "8cm"}, {editPicture: 2, delete: true}
  [4] table, under it B2: #1 a picture 2 × 1 cm, … and B2: #2 a picture 1 × 1 cm, …	{editPicture: 4, cell: "B2", n: 2, alt: "New words"}
  [3] Floating text and under it a picture 3 × 3 cm, alt text "Their floating"	{editPicture: 3, move: true, after: "Closing paragraph"} (to the end: {editPicture: 3, move: true})
  Pictures: B2 7 × 4.2 cm "Wider red"; B8 #1 1.6 × 0.8 cm "First blue"; B8 #2 2.6 × 1.3 cm "Second green".	{sheet?, editPicture: "B8", n: 2, at: "B14"}, {editPicture: "B2", height: "3cm"}, {editPicture: "B8", delete: true}

  In Word editPicture is the number in brackets that office_read gives the paragraph or the table (a content control's paragraphs are counted in reading order), cell is the table cell the picture is listed under, and n (from 1) says which of several listed there: they carry #1, #2 in the listing, and only where there are several. In a sheet editPicture is the cell the picture is anchored at (its top left corner), and n which of the pictures anchored there. The numbers are those office_read showed before the call, in Word and in a sheet: every picture the ops name is found before the first op changes anything, so a call's own inserts, deletes and moves do not renumber them (two deletes at one cell are n: 1 and n: 2, and a picture moved to a cell is not the first there). An op that names a picture an earlier op of the call has taken away (deleted or moved) is refused, as is a picture that is not listed before the call (one the same call puts in cannot be edited by it, except on a sheet the call makes itself, where the op looks at its turn). The answer repeats each changed picture with what it was (Pictures: deleted the picture [2], which was 4 × 2 cm, alt text "Their logo"), so that a number that meant another picture (a person changed the document since it was read) shows; a sheet picture a person deleted since the call began is told so, and is not swapped for another.

  What an op does. An op does one thing:

  • delete: true.
  • A move. Word: move: true and after, nth, as an insert places a picture, in an inline paragraph of its own after the paragraph that holds after, or at the end. A sheet: at, the cell its top left corner moves to. A model that wrote the destination under the other of the two, or under to, is understood, and keys given as null and delete or move as false are left out, as the model of the real-portal tests filled in every key of the schema.
  • A new width, height and alt (a length like "6cm", 0.2 to 50 cm). One side keeps the picture's proportions as it is now, and the other may not come out past 50 cm. OnlyOffice cannot take a picture's alt text away, so alt is a text.

  A deleted or moved picture takes its paragraph with it where that held nothing else. It stays where it is the only paragraph of a table cell or the document, where it ends a section, and where removing it would leave a table last in its cell or the document or join two tables (Word draws those as one table); a picture inside a paragraph with text leaves the text. A moved picture is the same file (a copy of its media: not made again as a PNG), inline from then on, with its alt text and title. A cropped, rotated or wrapped picture is changed in its size and place only, and the text inside a picture is not read (the guard does not see it either).

  Undone, and a person's change. All or nothing as for the other ops:

  • Word: by the editor's Undo, which takes back the whole call. A person's change to the same picture between the agent's read and its call is not guarded (the answer names what the picture was), and what the Undo does to a change a person made to that picture between the call's first op and its failure was not measured: Word is not covered there.
  • A sheet: a move, a size and an alt text are put back by the steps that made them, except where a person changed the picture since (theirs stays and the answer names it), or where rows or columns were deleted for good after the move (then it stays where the call put it and the answer names the op), or an alt text where the picture had none (the editor cannot put none back).
  • A deleted sheet picture cannot be taken back: OnlyOffice reads a picture's bytes back only as a PNG it draws of the picture (a JPEG or GIF would not be the file it was), so a delete goes last in a call, as deleted rows do, and where a later op of the call fails (a delete of another picture, say) the answer names what stays deleted.
  • The delete is done by the plugin method RemoveOleObject, as the inverse of a picture put in is. It reaches the sheet the editor shows, so the agent's editor shows the picture's sheet for a moment and the sheet it showed before again; a hidden sheet's picture is not deleted (the op is refused: show the sheet first, in the call before the delete or in another).

  The agent cannot write comments or tracked changes (it reads comments with office_comments). Calls on one document run one at a time. A document nobody has open is opened by the agent's editor alone, and saved when it closes. While an agent's editor is in a document its session stays open after the people left it, so it is saved as the last person leaves: what they typed is in the file then, not only once the agent's editor closes. The agent's editor of a document closes after five minutes without a call; at most three are open at once (the least recently used one closes for a fourth); the browser stops ten minutes after its last editor closed, and starts again with the next call (a few seconds). Where you stopped Office, office_read reads the saved file and says so, and office_edit is refused until it runs again: there is no other way to write. The agent sees what it made: office_screenshot gives it a picture (JPEG) of a document as its own editor shows it: a Word page (by number, or the page that holds a quote), laid out once the editor's page count holds still and drawn once the document's pictures have loaded (the editor shows a document before its pictures come, and draws nothing where one is missing: the picture waits up to 6 s for them, and where one has still not come, or could not be loaded, takes the page as it is and says so to the agent; the same for the pictures the agent has just put in); or a range of a sheet with its column letters and row numbers, at 100 % zoom, at 75 % where it does not fit, and refused beyond that with the largest range from its first cell that fits (about 23 columns by 41 rows of default cells at 100 %, 30 by 55 at 75 %); hidden rows and columns are left out. By default a sheet's used range from A1, as much as fits. 1,024 px on the longer side (large: as large as the editor draws it, up to 1,568 for a sheet range, about 1,170 for a Word page). The picture is drawn from the canvases the editor draws the page or the cells on (a sheet's with its charts and pictures), not from the one above them where it draws selections, cursors and the names of the people in the document: so it shows that document alone, with no other editor in it, and no person's selection, cursor or name (measured with a person selecting and typing in the same sheet, and with their cursor on the same page). The agent's own active cell shows only as its highlighted column letter and row number. The text the picture shows comes with it, as office_read gives it, so the guard reads it: a Word page's body text (of a table over many pages, the rows on the page), or the cells of the range that are shown; 20,000 characters at most. Only in the picture, so the guard does not read them: a page's header, footer and footnotes, text boxes, shapes and charts, text inside the document's own pictures, and a long text that runs into the range from a cell outside it. A model that cannot take pictures is refused (error:no_images) before anything is done. A picture costs the model about 1,000 to 1,600 input tokens on every later turn of the chat. Taking one moves the agent's own view (its page, zoom, sheet and selection), which you may see as its cursor moving; pictures are taken one at a time in the agents' browser, and a sheet a person hid, moved or renamed meanwhile is not shown (sheet_changed). The page is laid out as the editor lays it out: with its fonts (Carlito for Calibri, for instance), so a page break may fall elsewhere than in desktop Word. A save that would overwrite a newer revision (the file changed on disk while it was open: a script, a git pull) is not lost and does not overwrite it: it is kept beside the document as " (conflict )", and the audit log says so; the agent is told, and its editor of that document closes. A save the Document Server could not hand over while the portal was away (it keeps such a save as "forgotten") is taken back once the portal runs again. An open editor says within a second or two when the file changed on disk; its button loads the new revision in place.

• Talk to agents in comments: type @ in a comment (the comment box that opens on the text, or a cell's) and pick an agent or a project from the list; it inserts +Aegis or +project:Report (OnlyOffice puts a + before what it inserts), and typing @Aegis or @project:Report by hand does the same (in the Comments panel on the left, which has no list, too; a name with spaces is matched as a whole, @My Agent). Each comment that mentions an agent or project starts a chat of its own with it, in that agent's home or project's folder, which shows nowhere in the chat lists and needs no chat of yours with that agent, given only that comment: the document's id, the thread, and what you wrote. A reply or an edit that mentions an agent on a comment you did not make here (the file's own, another tool's) gives it your reply or what you added, never the comment's text as yours. The agent reads the comment's text and the document with its tools, does what you asked (office_edit changes the document), and its answer appears as its reply in the comment's thread. A reply you write in the thread goes to the same chat (only to the ones it mentions, where a thread has several); a reply or an edit that mentions another agent adds that one (an edit counts only the mentions it adds). Resolving and reopening keep the chat; deleting the comment removes it, two minutes later by default (an undo in that time keeps it; an answer that comes meanwhile is posted only after an undo, since a reply would bring the comment back). The agent gets access to the document with the mention. Word and spreadsheets alike. An agent that may only read the document (a file in a folder it cannot write) still answers in the thread, through an editor that may only comment. When the agent's edit replaces a larger text that a comment is on, the comment is made again on the new text, with its author, time and replies, and its thread and chat go on there.

• Talk to the document's agent in the editor's chat: what you write in the editor's own chat (the speech bubble on the left) goes to the agent or project the document belongs to (in whose folder it is), in one chat for the editing session; its answers come in the editor's chat as the agent's. Messages written while it is still answering reach it together as its next message. The chat goes a minute after the last person left the document (a reload in that time keeps it), and when the Document Server starts again. An answer that fails, or a run cut off by a restart of the portal, says so in the chat, as a comment's does in its thread. An answer that came while Office itself restarted is posted once it is back.

• Nobody is asked from these chats: the portal holds them as chats nobody attends. They have no question tool, a confirmation an extension's tool asks for is answered at once with no, and a call the guard would ask you about is refused; the agent hears each refusal and goes on. The editor's bar shows how the agents stand in the document: answering, waiting for their turn, or why one could not answer.

• What counts as your words: only what you did through the editor's own comment box, reply box and chat, on your page. A comment pasted in, brought back by undo, made by someone else, or already in the file starts nothing.

• Limits, on the Office page in Settings: comment chats running at once per agent or project (2; more wait their turn), comment chats per document (50; a comment refused for it gets its chat when a reply names the agent again once there is room), the longest answer in a comment (4,000 characters), messages per answer in the editor's chat (4 of 2,048 characters), the seconds before a deleted comment's chat goes (120), the seconds the editor's chat stays after the last person left (60), the messages kept for one chat while it answers (20; the oldest go, and it is told how many), the longest comment or chat message taken (20,000 characters), the reports of one editing session in 10 seconds (50; a document takes four times that from all its sessions), and the seconds between saves after answers (3). Fixed, as the protocol of the reports and not a limit on your use: 64 events and 256 KB per report (the page splits its reports below that, and a long thread's report keeps its latest replies), and the 50 messages (20,000 characters) of a session a chat made anew is given again.

Security

• A network of its own. The Document Server is on pithagoras-office-ds-net, which Docker for extensions makes internal and with no gateway (gateway mode isolated): it reaches neither the machine, its loopback, the local network nor the internet, and nothing outside the machine reaches it. A converter that a hostile .docx breaks into finds nothing there but the relay below. Its plugin manager is off (PLUGINS_ENABLED=false): it would try the internet at every start.
• One relay is its only way in and back. Docker for extensions gives a way onto such a network but none back, and the Document Server has to fetch each document and report each save. So Office runs one small container of its own on that network, pithagoras-office-ds-relay, on Node's alpine image, unprivileged, with none of the kernel's capabilities, a read-only root and 128 MB, as Docker for extensions runs its forwarder. Its port 3890 leads to the Document Server's port 80 (the way in). The way back runs over the way in, opened from the portal's side: the extension keeps a few connections to that port open, each opened with a key of the extension's (relay-secret), which the Document Server is not given; the relay hands each connection the Document Server makes to its port 3891 to one of them, and the extension passes it on to its listener on 127.0.0.1. Nothing of Office's shares the portal's network or is on any network but its own (the relay is on the agents' editor's too, below). The portal reaches the relay as it reaches Blender and Godot: for a portal on the host's network, through the forwarder pithagoras-office-ds-relay-forward, which publishes port 3890 on 127.0.0.1; for a portal in a container of its own, by putting that container on the network. Port 80 is never published on the machine. A restart of the portal, or of its container, breaks nothing: the portal opens its connections to the relay anew.
• Two routes are open to the Document Server, on a listener of their own. The container has no portal session, so the two things it needs are served by a small listener bound to 127.0.0.1, which the way back leads to and nothing else: a document fetch with a signed ticket for one document at one revision that expires in 10 minutes (GET /doc/<ticket>), and the save callback (POST /callback, which must carry a valid HS256 token). Besides them it serves only the agents' editor: its seat page, under a secret that keeps out everything else that reaches the listener, and each editor's plugin, under a token that gives that one editor's jobs and nothing else (below). The Document Server's own scripts run in the seat page (its api.js, its plugin SDK), so that secret is none to the Document Server, which holds every document and the JWT secret already: nothing on those paths gives more than it has. Nothing else answers there. The editor configs and callbacks are signed with a secret created once in the extension's data folder (jwt-secret, mode 0600), which the Document Server is given; the tickets with another (ticket-secret), which it is not, so a converter that a hostile file breaks into cannot forge one. A save is downloaded from the Document Server itself, from its files (/cache/files/) alone, whatever address the callback names. A document's file is read and written at its own name only: a link put in its place is refused, never followed.
• The agents' editor is a browser of its own, on a network of its own. pithagoras-office-ds-agent, from chromedp/headless-shell pinned by its digest, runs on pithagoras-office-ds-agents alone, with no port published. That network is made as the Document Server's is (internal, no gateway), and only the relay is on both: the browser reaches the Document Server through the relay's port 3890 alone, which leads to its port 80, where the Document Server's nginx keeps its statistics and cluster switches to itself (its own port 8000 beside it, which does not, is out of reach), and the extension's listener through the relay's way back; nothing else, never the portal, its pages or anyone's session (a portal in a container of its own is on the Document Server's network, not on this one). It runs as a user that is not root, with none of the kernel's capabilities, no new privileges, a read-only root, its profile in memory, 3 GB of memory and 512 processes at most. Chromium's own sandbox is off (--no-sandbox), as it needs what the default seccomp profile refuses: the container is the boundary, for a hostile file the editor renders. It opens one page, the seat page, from the extension's listener through the relay, under a secret that is new at every start of the server half and is only in its command line (/seat/<secret>/), and that page opens an editor for each document an agent works in, with a config the server half signs. Those configs are the only ones with a plugin: Office's own (people's editors keep plugins and macros off, signed). The plugin takes the agent's edits from the listener under a token of its editor's (/plugin/<token>/), and runs one of its fixed commands with them as data: the model never sends code. Anything else on those paths is answered as an unknown address. The agent's write permission is checked before each edit and when its editor opens (one it may not change opens read-only), and a file in a folder the agent could not read is never opened. Its debugging port (CDP, which controls the whole browser and with it every document an agent has open) is for office_screenshot alone. Chromium opens it on its own loopback (127.0.0.1:9223) only, and socat passes port 9222 on the agents' network to it; nothing publishes it. On that network only the relay is besides the browser, and the port's one user is a fixed Node script that the server half runs in the relay with docker exec (which needs the portal's Docker access, so nothing on the host or in a container of anybody else's reaches it). The script's only argument is data: which editor, and the page or range, which the server half checked and which reaches the editor as an argument, never as code. A page in the browser cannot use the port: Chromium refuses a page's request for its list of pages, and a WebSocket that names an origin (none is allowed). The Document Server and the island are not on that network. What this browser does not keep apart: the editors of every document the agents have open (three at most, of any agent) are in that one page, on one origin, so a document that got a script running in its editor (a bug of OnlyOffice's own; macros are off) reaches the other documents open there at that moment. And office_read reads through the editor, so a document an agent reads (one it downloaded, say) is converted by the Document Server and rendered by this browser with nobody opening it, as a person's editor does when they open it. Such a script could also hold the relay's way back, which it shares with the Document Server, until Office is restarted.
• The editor sits behind the portal's sign-in. The browser reaches the editor (api.js, the editor frame and every asset) only through /api/x/@pithagoras/office/ds/…, a proxy under the extension's routes, which the portal gates like all of them. It forwards GET requests and the POSTs the editor's socket.io channel sends, and only to the Document Server, and tells it the address the browser used, which it builds the editor's own addresses from. The Document Server's own endpoints are not passed on: its statistics, its cluster switches, its conversion services, and its command service, which the portal alone calls (to save the session after an agent's edit), over its own way in. The editor therefore runs on the portal's own origin, where a script has the signed-in person's session: a document's macros and the editor's plugins are switched off in the editor config the portal hands the page, as OnlyOffice holds macros apart only by a guard inside the page, and a document cannot switch them on. The Document Server 10.0 does not hold the page to the signed config (a page that changes it before it opens the editor gets plugins, as measured), so everything the proxy answers carries a content security policy that lets the editor's frame fetch and frame nothing but the portal's own origin: a plugin from anywhere else, which the editor's frame fetches (its config.json) and frames (its page), does not load. A script that could change the config has the person's session already. A Document Server on an origin of its own would be the full answer; it is not done here.
• Changes only from the portal's pages. Every request of the extension's that changes something (a document created, renamed or deleted, the Document Server started, stopped or removed, the settings) is refused unless it comes from the portal's own pages: the portal's cookie is SameSite=Lax, which a page on another port or another name of the same site is still sent.
• What the tools return is outside content to the guard: the text, comments and file names of documents anybody may have written. A comment's chat is given the words of the comment it was made for as yours only where you made that comment in this portal's editor (your page saw you write it); on any other comment only your reply, or what your edit added, is yours. The comment's text, the text it is on, the thread's other replies and the document are read with office_comments and office_read, as outside content. These chats are chats of their own, in Standard mode with the tools a new chat of that folder has: none of your chats' modes or tool switches carry over. A comment that reads like an order and is read that way can mark the chat; an answer of a marked chat is not posted in the document, also one that ended while Office restarted (the portal keeps the mark with the chat).
• Who can make an agent act through a comment: the portal's primary person. Every route asks the portal who is behind the request: nobody signed in is refused, and a person in another role may list the documents and download them but not open them in the editor (an editor is opened for changes), change them, nor talk to the agents through comments or the editor's chat (what they write would reach an agent as the primary person's words). Today every login to the portal is its primary person. The mention gives that agent the document.
• Every post into a document is in the audit log, with its document, thread and chat: each answer, each time it was posted again, a post whose outcome is not known (its editor closed or timed out while it ran), and a message of the editor's chat that failed after the ones before it went.
• The agent writes only through its editor, whose saves go through the same revision check as yours, and each document-changing call is recorded in the portal's audit log with the chat that made it, its ops (for pictures, each file as the agent named it and its bytes, never its data; an edit of a picture is named picture edits, apart from pictures put in) and the revision it saved; an edit the editor took without a clear outcome (not sent in time, no answer, cut off) is recorded too, as such.

Data

Everything is in the extension's own data folder, beside the portal's other data: settings.json (what you set; a default you never changed is not written down, so a later version's default applies), jwt-secret, ticket-secret, relay-secret, ds.json (the container and image of the last start), index.json (the library), .trash/, state/threads/<id>.json (each comment thread's agents, their chats and what waits to be posted) and state/sessions.json (the chats of the editor's chat, by editing session). A document stays where it is, in its agent's home or project folder; only its entry is in the index. Documents an earlier version made in docs/<id>/ of the data folder are moved once, at the first start of this one, into the folder of the agent or project that made them, else into the default agent's home (the first agent's), keeping their id, grants and revisions; a name taken there gets a number ("Report (2)"), and each move is in the portal's log. One that cannot be moved yet (no default agent listed) is tried again at each Rescan and every half minute. The containers (pithagoras-office-ds, its relay -relay and the agents' editor -agent), the networks (pithagoras-office-ds-net, and -agents for the agents' editor) and the Document Server's volumes are Docker's, shown in Settings → Docker under this extension. The volumes are one for each folder its image keeps apart: pithagoras-office-ds-cache (/var/lib/onlyoffice: the cache of open documents, with the saves it could not hand over yet, which the portal takes back once it reaches it again), -data (/var/www/onlyoffice/Data, its own keys), -logs and -fonts. They outlive the container being made again, after an update say.

Settings from the environment

The portal's environment can change each name, for a second portal on the same Docker host or for a test:

Removing

Remove the Document Server in Settings → Office first (Remove), then uninstall the extension: the portal has no uninstall hook, so containers left running keep running. Remove takes the Document Server, its relay, the agents' editor, the networks (and with them the forwarder) and the Document Server's volumes; the library and every document stay. Docker for extensions cannot remove an image, so the images stay on disk: docker image rm onlyoffice/documentserver:<tag> deletes the big one, docker image rm chromedp/headless-shell:157.0.8103.15 the agents' editor.

Limits

• Only .docx and .xlsx. .pptx is not supported.
• A comment made while the page could not report it (the tab closed at once, the server half away) starts nothing; reply to it again.
• In Word, an agent that may only comment changes the comment through the editor's comment methods, which give it a new id: its thread follows. Every person in the editor is "Portal". Such an agent's reads and posts share one editor that may only comment.
• A cell comment's answer is posted once the comment is five seconds old: a reply on a cell comment made a moment ago was now and then lost. A reply that does not reach the thread is posted again, a few times.
• The agents' browser opens for an answer as it does for the tools (about 0.3-0.55 GB while it runs), and an editor opened only to post closes after a minute.
• A file above 512 MiB is not indexed.
• One Document Server for the portal, run by this extension. A container of that name that nobody owns (one from before Docker for extensions) is taken over and made again; one of another extension's is refused.
• Not in this version: copying a document into a chat's folder from the library (the library downloads a copy), and expiry for the access a comment grants.
• What the portal knows of the editor's sessions (which one saved which revision) is kept in memory. An editor that stays open across a restart of the portal, and saved once before it, has its later saves kept as conflict copies, not written.
• The agent edits text, paragraphs, tables, cells and sheets, and formats Word documents and spreadsheets; it cannot write comments of its own or make tracked changes (it answers in the threads it was asked in). A sheet the agent copies takes the values, formulas, number formats and column widths of the old one, not its other formatting (fonts, colours, borders), charts, pictures, row heights or settings. An edit made while a person types in the same paragraph merges as the editor merges two people's typing.
• The agents' editors are one portal-wide browser: at most three documents at once, and a call that finds all three busy is refused (error:busy).
• office_format leaves out, for now: small caps, character spacing, paragraph borders and shading, tab stops, text columns, odd and even pages' headers, how a picture looks (cropping, borders, text wrapping), and inserting or deleting a table's rows and columns. A link lies inside one paragraph, and goes on text, not on table cells. office_read's formatting leaves out what a table that holds a picture looks like and the cells of a table inside a cell (office_screenshot shows them). For spreadsheets office_format leaves out, for now: how a picture looks, cell styles, a table's style and total row, a chart's series and axes beyond what it is made with, sparklines, pivot tables, a sheet's tab colour, grouped rows and columns, protection and print setup; office_read does not read a sheet's alignment and borders (office_screenshot shows them).
• office_screenshot shows one Word page or one sheet range a call, at most what the editor's grid shows at 75 %, and 100 columns and 200 rows (hidden ones counted); a larger range is taken in parts. A Word page beyond the count the editor has laid out after 5 s is refused with that count. Its text is 20,000 characters at most. A Word picture waits for all of the document's pictures, not only the page's (the editor does not say which are on a page), so a slow picture elsewhere in the document delays it, up to 6 s. A sheet's range waits for the sheet editor's pictures and charts' images the same way (the grid draws nothing where one has not come), and the answer's first line says how many were still loading or failed, as a Word page's does.

Licences

Apache-2.0 (LICENSE). The built server half bundles jszip 3.10.2 (MIT or GPL-3.0-or-later, used under the MIT licence) and the browser half icons from react-icons (MIT), drawn from Lucide (ISC); NOTICE says more, and THIRD-PARTY-LICENSES has the licence texts of what is bundled. The Document Server image is OnlyOffice's own (AGPL-3.0 for the community edition) and is pulled from Docker Hub by the portal, not shipped by this package; so is the agents' editor, chromedp/headless-shell, the chromedp project's image of Chromium's headless shell (BSD-3-Clause, with the licences of the components Chromium includes), downloaded by Start in Settings → Office.

Building and testing

Office is an npm workspace of this repository; npm ci at its root installs what it is built with (esbuild, tailwindcss, postcss, the extension API, react-icons and jszip).

npm run build -w office    # web/ and server.js, then the type check of src/ and the tests
npm test -w office         # the unit and integration tests, with Docker, the portal and the Document Server faked
./scripts/pithagoras.sh && PITHAGORAS_DIR=$PWD/.pithagoras npm run test:portal -w office   # under the portal's own extension host; the editor page, the chat's panel and Settings in Chromium

The integration tests start nothing in Docker: they run the extension's real loopback listener in the test process, with a small server of their own standing in for the Document Server's downloads; the seats' tests play the agents' browser over HTTP. The end-to-end tests run the real Document Server and the agents' editor through Docker for extensions under the portal's own extension host, open a document in Chromium in the Office page's own editor, and have the agent edit it beside the person (e2e.test.mts, and spike.test.mts, which pins what the editor's API does that the agent's commands rely on); they are opt-in, as they need Docker Engine 28 or later, both images (about 3.5 GB) and a few minutes:

./scripts/pithagoras.sh && OFFICE_E2E=1 PITHAGORAS_DIR=$PWD/.pithagoras npm run test:e2e -w office

What the editor's API does (a getter's answer, a setter's arguments) is quickest to pin with the Document Server's own docbuilder, which runs the same builder API on a file of its own in about two seconds, with no editor and no network. A script in probe/probe.docbuilder:

builder.CreateFile("xlsx");
var ws = Api.GetActiveSheet();
ws.GetRange("A1").SetValue("abc");
ws.GetRange("A1").GetCharacters(2, 1).GetFont().SetBold(true);
console.log(ws.GetRange("A1").GetCharacters(1, 3).GetFont().GetBold()); // null: the characters mix it
builder.CloseFile();

prints what it logs:

docker run --rm --network none -v "$PWD/probe:/probe" --entrypoint /var/www/onlyoffice/documentserver/server/FileConverter/bin/docbuilder \
  onlyoffice/documentserver:<version> /probe/probe.docbuilder

What lands later in the editor than in docbuilder (anything that goes through the Document Server's co-editing lock, as a freeze, an autofit or a change to the sheets) is for the end-to-end tests to pin.

real-portal.test.mts proves the same in a real portal: the portal's own server started from that folder (built with npm run build there), Docker for extensions and Office installed into it through POST /api/packages, a person in the real Office page in Chromium, and the agent's office tools called from a chat whose model is a script (scripted-model.mjs, no model server needed; one of its two models says it takes pictures, and keeps the pictures pi gives it, so office_screenshot's JPEGs are checked as a model gets them). OFFICE_PORTAL_MODE=container runs the portal in a container of its own (Node's alpine image, with this machine's Docker socket), on a network of its own:

OFFICE_E2E=1 PITHAGORAS_DIR=$PWD/.pithagoras [OFFICE_PORTAL_MODE=container] npm run test:real -w office

Its case p gives a real model a fixed task (a report with a title, headings, lists, a table, a style, a footnote, the page set up, a header, a footer and a table of contents, written and formatted with the office tools) and counts the calls it made, the ones refused and their codes, so the tool texts can be changed until a run has no refusal they could have prevented. It runs only where a model is given, any model with an OpenAI-compatible API, with the portal on the machine. Its chat has the office tools alone (every other tool switched off for it, so the model has no shell and no file tools on the machine), the proxy refuses a request that offers the model any other tool, and the model's key stays in the test's own process (the portal is started without any OFFICE_REAL_MODEL_*):

OFFICE_E2E=1 PITHAGORAS_DIR=$PWD/.pithagoras OFFICE_REAL_MODEL_URL=<its API, ending in /v1> OFFICE_REAL_MODEL=<its id> \
  [OFFICE_REAL_MODEL_KEY=<key>] [OFFICE_REAL_MODEL_IMAGES=1] [OFFICE_REAL_MODEL_REPORT=calls.json] \
  node --import tsx --test --test-timeout=1800000 --test-name-pattern='^(the portal|p: )' test/portal/real-portal.test.mts

(from office/; the pattern runs the portal's set-up and case p alone). With a report named, the last request the model got is kept beside it (calls.json.request.json): what the model wrote and the pictures office_screenshot gave it.

scripts/make-fixtures.mjs makes fixtures/sample.docx and fixtures/sample.xlsx again (node scripts/make-fixtures.mjs in office/; the zip entries carry the time of the run, so the bytes differ but not what the files contain).