These pages describe PrivacyFence 6.3.0. Download it · What changed

Grist setup¶

PrivacyFence connects to Grist at docs.getgrist.com, on a team site, or on a self-hosted server. The assistant can list your documents and tables straight away, read records after you review them, and add or change records, tables and columns only with your approval. Nothing is deleted: there is no record, column or table delete, no rename and no type change.

Two ways to connect¶

• OAuth, when your organization registered a PrivacyFence app in Grist and put it in the organization config bundle. OAuth apps are available in Grist's paid editions. You click Authenticate… (desktop) or Connect (organization server) and sign in on Grist's own page.
• A personal API key otherwise, on any Grist edition. When the bundle holds an OAuth app, API keys are not accepted: the organization chose OAuth.

Connect with an API key¶

1. In Grist, open Account settings → Developer → API Key and create the key, then copy it.
2. On a desktop install, open PrivacyFence's Settings page, Connectors → Grist → Authenticate…, enter the server address (https://docs.getgrist.com for the hosted service, or your own server's address, without /api) and the key. When the organization bundle names the server, only the key is asked for.
3. On an organization server, the connections page asks only for the key; the server comes from the bundle.

The key acts as you: it has your access to every document you can open, and Grist offers no scopes for it. PrivacyFence sends it only to the server it was entered for, stores it in a file only your account can read, and never logs it.

Register an OAuth app¶

For administrators, in Grist's paid editions.

1. In Grist, open Account settings → Developer → OAuth apps → Register app.
2. Enter a name such as PrivacyFence.
3. Add the redirect URIs, each on its own line: http://localhost:53685/callback for desktop installs and, for an organization server, https://<server>/oauth/callback/grist, with <server> your server's --server-issuer-url. One app can carry both.
4. Allow the permissions doc:read, doc:write, doc.schema:write and offline_access.
5. Save, then copy the client id and client secret.

Grist notes that doc.schema:write can reveal any data in a document through formulas. PrivacyFence requests it only to add tables and columns, and requests nothing beyond the four permissions above.

Values¶

The client id and secret go together, and the sign-in server needs them.

Build and distribute the bundle¶

scripts/build_org_bundle.py is in the PrivacyFence source repository and is attached to every stable GitHub Release. It needs only Python 3.

python3 scripts/build_org_bundle.py \
  --grist-server-url https://docs.getgrist.com \
  --grist-client-id abc123 \
  --grist-client-secret abcdef0123456789 \
  -o org_config.json --merge

--merge adds Grist to an existing org_config.json. A bundle with only --grist-server-url pins the server for API keys; organization mode needs at least that. See org-mode-setup-guide.md for signing and the server flags, and connecting-a-service.md for the shared connect flow.

What the assistant can do¶

Every Grist tool has a gate fixed in code. The tools reference lists each one.

Each card names the Grist server the data comes from or goes to.

A table larger than one call is read in pages, in record-id order: each page continues after the last record id of the one before, so rows added in between are neither repeated nor skipped. Every page is its own review card, and its Page line shows where the page starts, which record ids it holds and whether more follow. The card does not show the table's total row count. ADR 0146 records why.

Under OAuth, some servers do not let apps list documents. The assistant then asks you for the Document ID, shown in Grist under the document's Settings (the gear icon). One id form is used everywhere: the full Document ID, not the shorter id in the document's address. An auto-accept rule names that full Document ID.

Bulk import and update from a CSV file¶

grist_add_records and grist_update_records take at most 100 records inline. To load or refresh a larger table, the assistant hands PrivacyFence a CSV file instead, and you approve it once.

• Handing over the file. The assistant gives either local_path, a path on your computer, or upload_id, the id privacyfence_create_upload_slot returned after the file's bytes were sent to its upload URL. On an organization install only upload_id is available.
• Header. The first row names the columns, by column id or by label as grist_list_tables shows them. The delimiter is a comma, a semicolon or a tab. Formula columns and columns of other types than Text, Numeric, Int, Bool, Date, Choice and Any cannot be written.
• Cells. An empty cell is left out: an import leaves the column at Grist's default, and an update does not change the cell, unless clear_empty is true on grist_update_csv: then an empty cell clears the value (the card counts them as "Cells to clear"). Columns that are not in the CSV are never touched, and a row added by create_missing gets no empty values. Dates are written as YYYY-MM-DD, and numbers with a point as the decimal mark.
• Update. grist_update_csv matches each row to the record whose key column has the same value. The key column must be a Text, Choice, Int or Numeric column, in the header, unique in the file and unique in the table. It can also be id, the Grist record id, for a table that has no other unique column; create_missing is not available then. Only cells that differ change. Rows that match no record are skipped, or added when the assistant sets create_missing.
• Limits. At most 20000 rows, 100 columns and 50 MB. A file over a limit, or with a cell that does not fit its column, is refused before the card is shown, with the line and column of the first problems.
• The card. It names the server, the document and the table, and shows the counts, the first 20 rows (import) or the first 20 changed cells and unmatched rows (update), and the file's SHA-256. The approval is bound to that SHA-256: a different file needs a new approval.
• Writes. The rows go to Grist in chunks, which are not atomic. If Grist refuses a chunk, the assistant is told how many rows were written. After a failed import the rows written so far stay in the table; running grist_update_csv again with the same file finishes a failed update.

ADR 0148 records why.

Exporting to a file¶

grist_export_records writes a whole table, and grist_export_sql the result of one read-only SQL query, to a CSV or JSON file on your computer. You approve it once, and the assistant gets back only the row count, the column names and the file's SHA-256, not the rows.

• Limit. At most 50,000 rows. A larger table or result is refused; narrow it with a filter or the query.
• Organization installs. Refused: the file would land on the server, not on your computer.
• Rules. A grist.document read rule for the document covers both tools, so a scheduled run can export without a card. The PII gate still applies: an export whose content trips the scan gets a card, and an unattended session is denied.
• SQL. grist_export_sql needs full copy access to the document, because that is what Grist requires for SQL, and it cannot read the _grist tables. Cells are written as Grist stores them: dates as seconds since 1970, true/false as 1/0.
• Round trip. To write an export back with grist_update_csv, keep only id and the data columns you change; id is the key column.

ADR 0155 and ADR 0156 record why.

Auto-accept rules¶

A rule can name a document by its grist.document scope, set on Settings > Auto-accept. The approval card never offers "Always allow" for a Grist operation. Every scope is in the scope catalogue.

The rule's verbs cover these tools:

A CSV import or update rule lets the assistant load a file into that document without a card, so grant add and update only for a document you are happy to have bulk-written.

Troubleshooting¶

Grist is not authenticated. Use Authenticate… in PrivacyFence Settings. — no key or sign-in is saved yet. Use Authenticate… on the Grist row.

Grist's saved sign-in could not be read. — the saved credential file is damaged. Connect again.

Your organization connects to Grist with OAuth. — the bundle holds an OAuth app, so an API key is not accepted. Use Authenticate… (desktop) or Connect (organization server).

Grist was connected to a different server than your organization uses. — the saved credential belongs to another server than the bundle names. Connect again.

Enter the Grist server address, such as https://docs.getgrist.com., The Grist server address must start with https://, …must not contain a user name, password, query or fragment., Enter the server address without /api. — correct the server address.

Grist refused the sign-in (HTTP 401). — the key or sign-in is wrong, expired or revoked. Connect again.

Grist refused the request (HTTP 403). — your Grist account, or what you allowed PrivacyFence when signing in, does not cover the request.

Grist found no such document, table or record (HTTP 404). — check the Document ID and the table id.

This Grist server does not let PrivacyFence list your documents. — open the document in Grist, then Settings (the gear icon) → Document ID, and give the assistant that id.

Grist sign-in failed: <error> — Grist refused the sign-in; the text after the colon is Grist's own reason. Check the client id, the client secret and the redirect URI registered for the app.

Grist did not return a refresh token. — the app in Grist must allow offline_access.

Your Grist sign-in has expired or was revoked. — use Authenticate… to sign in again.

Could not reach the Grist server at <host> — check the server address and your network.

The Grist server answered with a redirect (HTTP 30x). — the server address is not the final one; enter the address Grist ends up at.

Grist's sign-in server answered with a redirect (HTTP 30x). — the sign-in server address in the organization config is not the final one; enter the address it ends up at.

Could not reach Grist's sign-in server at <host> — check the sign-in server address and your network.

Grist's sign-in settings at <host> are not usable. — check the Grist server address, and the sign-in server if set, in the organization config.