Developer setup

From a fresh clone to a running application, the tests, the layout of the project, the server mode on your own machine in three steps, and how to extend the software: a storage adapter, a sign-in method, a field type. It ends with the conventions that matter and a troubleshooting list.

Contents

• Prerequisites
• Clone, install, run the demo
• Commands
• Running the tests
• Project layout
• How the pieces fit
• Server mode locally
• Trying the API without the user interface
• Adding a storage adapter
• Adding a sign-in method
• Adding a field type
• Coding conventions that matter
• Troubleshooting

Prerequisites

No database and no identity provider are needed to run the demo or the bulk of the tests.

Clone, install, run the demo

git clone <repository URL> countersign
cd countersign
node --version          # v22.18.0 or newer
npm install
npm run dev

Open http://localhost:3000. With no COUNTERSIGN_* variables set this is the demo: the landing page at /, the application at /app, the documents at /docs. There is no database and no sign-in. Pick one of the four demo domains (fleet, property, clinic, org) and a person to act as; your changes are stored in your browser's localStorage on top of a deterministic demo history and never leave the browser.

To start over, clear the site's data in the browser, or use the reset on the application's Integrity page.

Commands

Continuous integration (.github/workflows/ci.yml) runs lint, typecheck, the tests with a PostgreSQL service, node scripts/trace.ts --check and the build. Run the same before you open a pull request.

Running the tests

Without PostgreSQL

$ npm test
…
# tests 88
# suites 18
# pass 88
# fail 0

This covers the engine, option resolution, migration, the memory, file and SQLite stores, the service, settings, sessions, OpenID Connect sign-in (against a fake identity provider that the tests start on a local port) and the HTTP handlers. It takes a couple of seconds. The PostgreSQL store's tests are skipped and say why: COUNTERSIGN_TEST_DATABASE_URL is not set.

With PostgreSQL

Point COUNTERSIGN_TEST_DATABASE_URL at a database the tests may use. They drop and recreate the countersign_entries table there, so never give them a database that holds anything.

A throwaway PostgreSQL with Docker:

docker run --rm -d --name countersign-pg -e POSTGRES_PASSWORD=postgres -e POSTGRES_DB=countersign_test \
  -p 5432:5432 postgres:16-alpine

COUNTERSIGN_TEST_DATABASE_URL=postgres://postgres:postgres@localhost:5432/countersign_test npm test

docker stop countersign-pg

Or with an existing server:

createdb -h localhost -U postgres countersign_test
COUNTERSIGN_TEST_DATABASE_URL=postgres://postgres@localhost:5432/countersign_test npm test

The store tests then run a fourth time, against PostgreSQL, including the ones that try to UPDATE, DELETE and TRUNCATE behind the application's back and the one that lets two services write at once.

Other checks

npm run shots needs Playwright (npm i -D playwright && npx playwright install chromium). It drives the demo through every feature from a fresh state, fails if a step fails, and rewrites the screenshots under public/. Its steps carry the [FR-UI-…] requirement tags, so it is the verification of the user-interface requirements. Look at the screenshots after a UI change.

Project layout

app/              Next.js routes
  page.tsx          /           the landing page
  app/              /app        the application (one client-side app)
  docs/             /docs       the documents of docs/, rendered; /docs/search-index.json
  api/              /api/health, /api/bootstrap, /api/entries, /api/act
  auth/             /auth/login, /auth/callback, /auth/logout, /auth/demo
components/       React user interface (client components)
  store.tsx         application state; the local and the remote backend
  app.tsx, pages.tsx, record.tsx, forms.tsx, tree.tsx, ui.tsx, site.tsx
  docs-client.tsx   the documentation's search box, "On this page" list and diagrams
lib/              the engine and everything DOM-free
  types.ts          the data model: configuration, records, commands, audit entry
  config.ts         checkConfig, permissions, hierarchy lookups
  engine.ts         act, validate, replay, verifyChain, the hash
  resolve.ts        option inheritance
  present.ts        what the UI shows: fields, diffs, allowed actions, inbox, CSV
  migrate.ts        the import: mapping, conversions, report
  csv.ts            CSV reader
  sha256.ts         synchronous SHA-256
  seed.ts           builds a preset's demo history by running real commands
  presets/          fleet, property, clinic: configuration plus demo data
  base.ts           base path and storage-key prefix
  docs-search.ts    matching and ranking of the documentation search
  adopt/            the migration assistant: model.ts (reads SQL and Prisma), draft.ts, survey.ts, verify.ts, journal.ts, guide.ts
  *.test.ts         engine.test.ts, migrate.test.ts, adopt/adopt.test.ts, trace.test.ts
server/           Node-only code
  settings.ts       every COUNTERSIGN_* variable, defaults, settingsProblems
  storage/          types.ts (the Store interface), memory, file, sqlite, postgres, index.ts (openStore)
  service.ts        the store's entries replayed and cached; submit with retry
  auth/session.ts   signed cookies
  auth/oidc.ts      OpenID Connect: discovery, PKCE, token verification, claims
  app.ts            the HTTP handlers, on the standard Request and Response
  docs.ts           renders the Markdown documents for /docs and builds their search index
  server.test.ts    stores, service, settings, sessions, sign-in, HTTP
db/postgres.sql   the PostgreSQL schema, read from disk at start
scripts/
  countersign.ts    the command-line tool
  api-server.ts     the HTTP API on a plain Node server
  trace.ts          the traceability matrix
  oq-report.ts      the test execution report
  guide-shots.mjs   end-to-end walkthrough and screenshots
examples/
  config/           example configurations (lab-equipment.json)
  migration/        an example export, its mapping, a README
docs/             these documents
  validation/       the validation pack: requirements.json, protocols, the generated matrix
public/           screenshots and static files
Dockerfile, docker-compose.yml, .env.example, package.json

Where to look for what:

How the pieces fit

Every change is a command that goes through one pure function, act in lib/engine.ts, which either refuses it or returns the new state and one hash-chained audit entry. The audit trail is the only thing stored; the records are what replaying it gives. In the demo the browser runs the engine itself and keeps the entries in localStorage. In server mode server/app.ts identifies the person from the session, server/service.ts brings its in-memory state up to date from the store, runs act, and appends the entry through a Store adapter; the browser fetches the entries and replays them with the same engine, so it verifies what the server serves. The command-line tool uses service.ts directly. The full description, with the hash construction, the request flow and the trust boundaries, is in Architecture.

Server mode locally

Setting COUNTERSIGN_STORAGE turns the server mode on. Put the variables in .env.local (Next.js reads it; see .env.example) or export them in the shell. The CLI reads the shell's environment only, so export them when you use both.

Step 1: SQLite and demo sign-in

No database server, no identity provider.

export COUNTERSIGN_STORAGE=sqlite
export COUNTERSIGN_DATA_FILE=./data/dev.sqlite
export COUNTERSIGN_AUTH=demo
export COUNTERSIGN_SESSION_SECRET="$(openssl rand -base64 48)"
export COUNTERSIGN_CONFIG=preset:fleet
export COUNTERSIGN_DEMO_DATA=1
npm run dev

Open http://localhost:3000/app: a sign-in page lists the fleet demo people. Sign in as Bob in one browser and as Alice in another (or a private window) and watch a change made by one appear for the other within 15 seconds. The store is now shared, and the CLI sees the same data:

$ npm run cli -- status
store          sqlite
entries        290
head hash      1780d6e781351665621ddf081638c81c089441d8fa814422e8d4dc73879ff57d
configuration  fleet "Fleet configuration", revision 1, SHA-256 547e2d8aea0405603c034b4fc1637fcc516e7ebbb59e5ad5509236947104cd8d
records        103 (104 including removed)
integrity      ok

Keep the session secret stable between restarts (put it in .env.local) or you are signed out every time. To start over, stop the server and delete ./data/dev.sqlite*.

The demo sign-in takes its people from the preset. For a configuration of your own, say who can be picked with COUNTERSIGN_DEMO_USERS, using role ids of that configuration (and leave COUNTERSIGN_DEMO_DATA out: demo data exists only for the presets):

export COUNTERSIGN_CONFIG=./examples/config/lab-equipment.json
export COUNTERSIGN_DEMO_USERS='[{"id":"mia","name":"Mia Keller","roles":["metrologist"]},{"id":"noor","name":"Noor Haddad","roles":["lab-manager"]}]'
unset COUNTERSIGN_DEMO_DATA

Step 2: PostgreSQL

Start a PostgreSQL (the Docker command above will do, with a database of your choice) and switch the store:

export COUNTERSIGN_STORAGE=postgres
export COUNTERSIGN_DATABASE_URL=postgres://postgres:postgres@localhost:5432/countersign_dev
unset COUNTERSIGN_DATA_FILE
npm run cli -- init --config preset:fleet --demo-data
npm run dev

Initialised the postgres store with 290 entries from preset:fleet. Head #290 1780d6e781351665621ddf081638c81c089441d8fa814422e8d4dc73879ff57d

The server creates the table, triggers and view itself (db/postgres.sql). Things to try:

• psql … -c "SELECT seq, at, actor_name, action, record_id FROM countersign_audit_trail ORDER BY seq DESC LIMIT 5"
• psql … -c "DELETE FROM countersign_entries WHERE seq = 1" and read the trigger's answer;
• a second server on another port (PORT=3001 npm start after npm run build, with the same variables) and two people writing at once;
• the split into an owner and an application role with COUNTERSIGN_SKIP_MIGRATE=1 (Storage).

docker compose up gives you this step ready-made: PostgreSQL 16 and the application with the fleet demo data and demo sign-in.

Step 3: OpenID Connect

Replace the demo sign-in with a real one. Either a local Keycloak or a test application in Microsoft Entra ID.

A local Keycloak.

docker run --rm -d --name countersign-kc -p 8080:8080 \
  -e KC_BOOTSTRAP_ADMIN_USERNAME=admin -e KC_BOOTSTRAP_ADMIN_PASSWORD=admin \
  quay.io/keycloak/keycloak:latest start-dev

(The names of the admin variables have changed between Keycloak versions; check the image's documentation if the admin sign-in fails.) In the admin console at http://localhost:8080:

1. Create a realm, countersign.
2. Create a client countersign: OpenID Connect, client authentication on, standard flow, valid redirect URI http://localhost:3000/auth/callback. Copy its secret.
3. Add a Group Membership mapper to the client's dedicated scope: claim name groups, full group path off, in the ID token.
4. Create groups named after the fleet roles you want to try (coordinator, qa, steward, …) and two users with passwords; put them in groups. You need two users to see an approval.

export COUNTERSIGN_AUTH=oidc
export COUNTERSIGN_OIDC_ISSUER=http://localhost:8080/realms/countersign
export COUNTERSIGN_OIDC_CLIENT_ID=countersign
export COUNTERSIGN_OIDC_CLIENT_SECRET=<the client's secret>
export COUNTERSIGN_ROLE_CLAIM=groups
npm run dev

Start from a store without demo data (init --config preset:fleet on an empty database), since the demo people do not exist in your realm. Sign in, create and confirm a record as one user, approve it as the other, and look at the entry's auth (npm run cli -- export | tail -1). Then set COUNTERSIGN_SIGN_MAX_AGE_MINUTES=1, wait a minute and approve again to see the re-authentication.

A test application in Entra ID. Follow Authentication with the redirect URI http://localhost:3000/auth/callback, then:

export COUNTERSIGN_AUTH=entra
export COUNTERSIGN_ENTRA_TENANT_ID=<tenant id>
export COUNTERSIGN_ENTRA_CLIENT_ID=<client id>
export COUNTERSIGN_ENTRA_CLIENT_SECRET=<client secret>

Neither of these two was run while writing this page (no identity provider was available); the sign-in code itself is covered by server/server.test.ts, which runs the whole flow against a fake provider.

Trying the API without the user interface

scripts/api-server.ts serves the eight API and sign-in routes on Node's own http server, without Next.js. It starts in a moment and is the quickest way to try the API with curl:

export COUNTERSIGN_STORAGE=sqlite COUNTERSIGN_DATA_FILE=./scratch/api.sqlite COUNTERSIGN_AUTH=demo
export COUNTERSIGN_SESSION_SECRET="$(openssl rand -base64 48)" COUNTERSIGN_CONFIG=preset:fleet COUNTERSIGN_DEMO_DATA=1
npm run api -- 3111

Countersign API on http://localhost:3111 (server mode)

$ curl -s http://localhost:3111/api/health
{"status":"ok","mode":"server","version":"1.0.0","storage":"sqlite","auth":"demo","entries":290,"headHash":"1780d6e781351665621ddf081638c81c089441d8fa814422e8d4dc73879ff57d",…}

The complete walk-through (sign in, read, create, confirm, approve as someone else) is in the API reference. The script is also the example of how to mount the handlers of server/app.ts in a server of your own: they take a standard Request and return a standard Response.

Adding a storage adapter

A store keeps the audit trail and nothing else. The interface (server/storage/types.ts):

export interface Head {
  seq: number;   // number of the newest entry; 0 when the store is empty
  hash: string;  // its hash; the genesis value (64 zeros) when the store is empty
}

export interface Store {
  readonly kind: string;
  /** Create tables or files if they are missing. Safe to call every time the server starts. */
  init(): Promise<void>;
  head(): Promise<Head>;
  /** Entries with seq > after, in order. */
  read(after: number, limit?: number): Promise<AuditEntry[]>;
  /** Append one entry. Returns false, and stores nothing, unless entry.seq is exactly head + 1 and entry.prevHash is the head's hash. */
  append(entry: AuditEntry): Promise<boolean>;
  close(): Promise<void>;
}

The contract is spelled out in Storage: append must be atomic and conditional, entries must come back exactly as they went in, nothing is ever updated or deleted.

Steps, with a hypothetical mysql adapter:

1. Write server/storage/mysql.ts exporting class MysqlStore implements Store. The existing adapters are short; sqlite.ts is the closest model for a database, memory.ts for anything else. Store the entry as the JSON text you were given (JSON.stringify(entry)), plus whatever columns you want for queries. Return false from append only for "someone else was first" (a unique-key violation); throw on everything else.

2. Register it in server/storage/index.ts (openStore), loading the driver with a dynamic import() so that installations that do not use it need not have it, and add the name to StorageKind and the STORAGE list in server/settings.ts, with any settings it needs and their checks in settingsProblems.

3. Add it to the conformance tests. In server/server.test.ts the stores are listed in adapters; every entry is run through the same tests:

   const MY = process.env.COUNTERSIGN_TEST_MYSQL_URL;
   
   const adapters: Adapter[] = [
     // … memory, file, sqlite, postgres …
     {
       name: "mysql",
       skip: MY ? undefined : "COUNTERSIGN_TEST_MYSQL_URL is not set",
       open: async () => {
         await dropTable(MY!);                       // each call must start from an empty location
         const store = new MysqlStore(MY!);
         return {
           store,
           again: () => new MysqlStore(MY!),         // a second connection to the same location
           raw: (sql) => runSql(MY!, sql),           // optional: lets the append-only test try UPDATE and DELETE
         };
       },
     },
   ];
   

   Test	Requires of the adapter
   FR-STO-01	Entries are appended and read back in order, unchanged; init twice is harmless; read(after, limit) works.
   FR-STO-02	An entry that does not continue the chain is refused: wrong first entry, a repeated number, a gap, a wrong previous hash.
   FR-STO-03	The trail survives closing and reopening (not for memory).
   FR-STO-04	The database itself refuses UPDATE and DELETE and a row inserted out of order. Runs for sqlite and postgres only: if your store can enforce this, extend the condition in the test to include it and supply raw.
   FR-STO-05	Two services writing eight commands at once through two connections produce entries 1 to 9, without gaps or a fork. For stores without cross-process safety the test uses one connection (as for memory and file); a real database must pass it with two, so use again().

4. Run COUNTERSIGN_TEST_MYSQL_URL=… npm test, then try it for real with the CLI (init, status, verify) and the server.

5. Requirements and documents. The storage requirements (FR-STO-…) already cover any adapter; the test names carry the tag, so the traceability matrix stays correct (node scripts/trace.ts --check). Add the adapter to Storage, to the variable table in Deployment, and to CI if it should be tested there.

Adding a sign-in method

How sign-in is structured:

• server/settings.ts: AuthKind and the AUTH list name the methods; settingsProblems says what each needs.
• server/app.ts: identify(req) answers "who is asking?" for every request. For proxy it reads headers; for everything else it reads the session cookie. login and callback run the OpenID Connect flow; demoLogin sets a session for a demo person; sessionCookie writes a session.
• server/auth/session.ts: sealing and reading the signed cookie. server/auth/oidc.ts: the OpenID Connect flow.

Which shape does your method have?

Whatever the shape:

1. Add the name to AuthKind and AUTH, its settings to readSettings, and its requirements to settingsProblems (the server refuses to serve until that list is empty).
2. Produce an Actor (id, name, roles): a stable id that is never reused; roles passed through settings.roleMap.
3. Produce AuthEvidence (method, issuer, authTime). It is written into every audit entry the person causes. Be exact about authTime: it must be when the person proved their identity, not when a token was issued.
4. Decide about signatures. The re-authentication check in act (the HTTP handler) applies to entra and oidc. If your method can force a fresh authentication, extend that condition and make login?fresh=1 do it; if it cannot, say so in the documentation, as for proxy.
5. Report the login address in bootstrap (loginUrl) so that the user interface can send people there.
6. Tests: server/server.test.ts has the patterns: a fake provider on a local port for the redirect flow, forged and expired cookies, missing secrets. New behaviour needs a requirement and tagged tests (see the conventions below).
7. Never let the request body or query say who the actor is.

Adding a field type

A field type touches the model, the validator, the engine, the presentation layer, the form and the import. Take a time type (HH:MM) as the example.

Two rules decide whether the change is safe:

• Old trails must still replay. Adding a type cannot break existing entries, because no existing configuration uses it. Changing how an existing type is validated can: an entry that was accepted then must still be accepted on replay. If you tighten a check, existing installations would fail verification at the first old entry that the new rule refuses. Do not tighten; add a new type or a new option instead.
• A configuration that uses the new type cannot be read by older versions. Say so in the changelog.

Coding conventions that matter

These are the ones that break something when ignored.

• lib/ is DOM-free and Node-free. No window, document, localStorage, fs, process, node: imports. The same files run in the browser, on the server, in the CLI and in the tests. server/ may use Node but no DOM.
• Relative imports carry the .ts extension in lib/, server/ and scripts/: import { act } from "./engine.ts". Node runs these files directly and resolves imports literally.
• No enums, no parameter properties, no namespaces, no decorators. Node strips types; it does not transform TypeScript. Anything that needs code generation fails at run time with a syntax error. Use union types and plain classes with explicit fields (see server/storage/file.ts).
• Use import type (or inline type specifiers) for types, so that stripping leaves no dangling import.
• Every state change goes through act. Never construct or modify a Db, a record or an audit entry elsewhere, not in the UI, not in a script, not in a migration. If something cannot be expressed as a command, add a command.
• act stays pure: no clock, no randomness, no I/O. The caller passes the time. Demo data is seeded deterministically.
• Refusals are return values, with a message a user can act on. Do not throw for rule violations.
• Do not change the outcome of existing commands. Replay must reproduce every entry ever written. Treat the entry format, stable, hashEntry and the validation of existing types as a compatibility surface.
• Tests are tagged with requirement ids. A test name starts with the requirement it verifies: test("[FR-WF-04] the author cannot approve their own version", …). scripts/trace.ts links tests to docs/validation/requirements.json by these tags.
• npm run trace must stay green. A behaviour change comes with a requirement, a tagged test and a regenerated matrix; node scripts/trace.ts --check (in npm test and in CI) fails if a requirement has no test, a tag is unknown, or the matrix is stale.
• Root-relative URLs go through url() from lib/base.ts, and browser storage keys start with STORAGE_PREFIX, so that the application can be mounted under a base path.
• Settings are read in one place, server/settings.ts. Do not read process.env elsewhere (the two store tuning variables in server/storage/postgres.ts are the exception).
• Messages are plain sentences for the people who use the system, with the record and version they concern.
• No secrets, no real personal data in tests, examples or screenshots.

Details on proposing changes are in CONTRIBUTING.

Troubleshooting