Supplier assessment


Document id	CS-VAL-12
Title	Supplier assessment
Version	1.0
Applies to	Countersign 1.0.0
Status	Template supplied with the software; to be completed, reviewed and approved by the regulated user

1. Purpose

Regulated organisations assess the suppliers of their computerised systems. Countersign is an open-source project, not a company: there is nobody to send a questionnaire to and nobody to audit. This document states the facts an assessor needs, honestly including what does not exist, and provides a completed questionnaire to file. The assessment decision and the compensating measures are the organisation's.

Facts about the repository were checked against the source of version 1.0.0. Where a statement depends on the standalone repository as published (issue tracker, release tags, continuous integration), verify it there yourself; the cells are marked "verify".

2. The product

Item	Fact
Name, version	Countersign 1.0.0
Kind	Open-source web application with a command-line tool; self-hosted by the user
Licence	Apache License 2.0 (`LICENSE` in the repository). It grants use, modification and redistribution, and disclaims warranty and liability (sections 7 and 8 of the licence)
Source code	Public, complete, in the project's repository; no closed components
Languages and platform	TypeScript on Node.js 22.18 or newer; Next.js and React for the web interface; PostgreSQL (recommended), SQLite, file or memory for storage
Release identification	Version number in `server/app.ts` (`VERSION`), reported by `/api/health`; release tags and commit hashes in the repository (verify)
Change history	`CHANGELOG.md`
Security contact	`SECURITY.md` describes how to report a vulnerability
Documentation	`README.md`; `docs/` (developer setup, configuration, storage, authentication, deployment, operations, architecture, API, migration, security, user guide); this validation pack
Installed base	Version 1.0.0 is the first release. No track record in regulated use can be claimed

3. Development practices present in the repository

Only what can be seen in the code is listed.

Practice	Evidence	Where
One rules engine as pure functions, shared by browser, server, command-line tool and tests	`act` takes a state, an actor, a command and a time and returns a new state and an entry or a refusal; no input or output inside	`lib/engine.ts`, `lib/config.ts`, `lib/resolve.ts`
Requirement catalogue	15 user and 88 functional requirements with risk rating and verification method	`docs/validation/requirements.json`
Requirement-tagged automated tests	Every test name starts with the id of the requirement it verifies, for example `[FR-WF-04]`	`lib/engine.test.ts`, `lib/migrate.test.ts`, `server/server.test.ts`
Traceability generated, and checked in the test run	`scripts/trace.ts` builds the matrix from the catalogue and the test names; `lib/trace.test.ts` fails if a requirement has no test or walkthrough step, if a test names an unknown requirement, or if the matrix is out of date	`scripts/trace.ts`, `lib/trace.test.ts`, `docs/validation/traceability-matrix.md`
Storage conformance tests	The same tests run against every storage adapter: append and read back, refusal of an entry that does not continue the chain, survival of a restart, refusal of UPDATE, DELETE and TRUNCATE at database level, two writers at once	`server/server.test.ts` (PostgreSQL only when `COUNTERSIGN_TEST_DATABASE_URL` is set)
Tests of sign-in against a mock identity provider	Code flow with PKCE, state and nonce; wrong signature, audience, issuer, nonce and state are refused; re-authentication for approvals; CSRF checks; proxy secret	`server/server.test.ts`
Negative and tamper tests	Altered, forged, missing, reordered and inserted entries; altered store behind a running service	`lib/engine.test.ts`, `server/server.test.ts`
End-to-end walkthrough of the interface	A browser script drives the demo through every feature and fails on a broken step; its steps carry the FR-UI ids	`scripts/guide-shots.mjs`
Dated execution report	Runs the suite and writes environment, one row per requirement with result, totals and a signature block	`scripts/oq-report.ts` (`npm run oq`)
Reviewable database schema	One SQL file, idempotent, read from disk when the server applies it	`db/postgres.sql`
Demo data built by the engine	The demo history is produced by real commands, so it is itself a replay test for three configurations	`lib/seed.ts`, `lib/presets/`
Type checking	TypeScript throughout	Repository

Not evidenced in the repository itself, so not claimed here: a documented code review process, branch protection, continuous integration, signed releases, static security analysis, a defined support period. Verify what the published repository shows and record it in section 6.

4. Dependencies and software bill of materials

Runtime dependencies declared by the application:

Package	Used for	Notes
`next`	Web framework: pages and HTTP route handlers
`react`, `react-dom`	User interface
`pg`	PostgreSQL driver	Loaded only when the PostgreSQL store is selected
`jose`	Verification of OpenID Connect ID tokens against the provider's keys	Used only in `server/auth/oidc.ts` (and by the tests' mock provider)
`marked`	Rendering of the documentation pages (Markdown to HTML)	Not involved in records or the audit trail
`mermaid`	Drawing the diagrams of the documentation pages in the browser	Loaded only by documentation pages that contain a diagram; not involved in records or the audit trail

Node.js built-in modules used instead of further dependencies:

Module	Used for
`node:crypto`	HMAC-SHA256 of the session cookie, constant-time comparison, random values and the PKCE challenge of the sign-in flow, SHA-256 of migration source files
`node:sqlite`	The SQLite store
`node:fs`, `node:path`, `node:os`, `node:util`	Files, the file store, command-line parsing
`node:test`, `node:assert`	The test suite (no test framework dependency)

One point for the assessor: the SHA-256 of the audit trail's hash chain is computed by a small implementation inside the repository (lib/sha256.ts), not by node:crypto, so that the identical code runs in the browser and on the server. The test [FR-INT-01] checks it against published SHA-256 known answers and against node:crypto, and you can check it independently on your own trail with the standalone script in Data integrity, section 6.2, which uses node:crypto and must reproduce every hash.

Producing the bill of materials for the release you install (attach to the IQ):

npm ci                                            # install exactly what the lock file names
npm ls --all --omit=dev > npm-ls-runtime.txt      # the runtime dependency tree with versions
npm ls --all > npm-ls.txt                         # including development tools
npm sbom --sbom-format cyclonedx > sbom.cdx.json  # CycloneDX; or --sbom-format spdx
npm audit --omit=dev                              # known vulnerabilities in runtime dependencies

The lock file (package-lock.json) pins every direct and transitive dependency with an integrity hash; installing with npm ci reproduces that set. Transitive dependencies of next are numerous; they are in the bill of materials, not in the table above.

5. What the project does not provide

Not provided	Consequence for the regulated user
Warranty or liability	The licence disclaims both. The risk of use is yours
A vendor to audit; a postal or on-site audit	Assess from the public evidence; compensate with your own verification
A certified quality management system (for example ISO 9001) or documented development procedures	Treat supplier assurance as low; run the complete test suite yourself, review high-risk code, execute the manual and negative tests
A support contract, service levels, guaranteed response to defects or security reports	Decide who maintains your installation: your own developers or a service provider under contract
A penetration test report or independent security assessment	Commission your own if your risk assessment requires one
A statement of compliance with 21 CFR Part 11, EU GMP Annex 11 or any other regulation, or any certification	See Part 11 and Annex 11 mapping: a description with stated gaps, not a claim
Validation of your installation	This pack is a set of templates; execution and approval are yours
Hosting, backup, monitoring, identity	Your infrastructure
A guaranteed roadmap, long-term support versions or backports	Plan upgrades under your change control; keep the ability to stay on a version you validated
Training services	User guide and the operator documentation only
Escrow	Not needed: you hold the complete source under a licence that lets you maintain it

What open source gives in return: complete transparency of the code and its history, the right to review, test, fix and keep any version indefinitely, and independence from a vendor's continued existence.

6. Questionnaire

Completed for Countersign 1.0.0 from the repository. Fill the "verified by you" column when you have checked the statement against the release you install; complete the rows marked "verify".

6.1 Supplier

No.	Question	Answer
1	Legal entity supplying the software	None. Open-source project; maintainers as listed in the repository (verify)
2	Contract or agreement with the supplier	None, other than the Apache-2.0 licence
3	Certified quality system	None
4	Can the supplier be audited	No
5	Years in business; number of customers; references in regulated industry	Not applicable; first release
6	Financial stability; continuity if the supplier ceases	Not applicable; the source remains available to you under the licence
7	Support, service levels, escalation	None committed

6.2 Development

No.	Question	Answer
8	Is there a requirements specification	Yes: `docs/validation/requirements.json`, 15 user and 88 functional requirements
9	Is there a design or functional specification	Yes: functional specification, Architecture
10	Are requirements traced to tests	Yes: generated matrix; the test run fails if a requirement is unverified
11	Is testing automated and repeatable by the customer	Yes: `npm test`; report with `npm run oq`
12	Are there negative, boundary and security tests	Yes: refusals for every rule, tamper detection, sign-in failures, CSRF, concurrency
13	Is the source under version control with history	Yes: Git (verify the published history)
14	Is every change reviewed by a second person before release	Not evidenced in the repository (verify)
15	Is there continuous integration running the tests on every change	Verify in the published repository
16	Coding standards, static analysis	TypeScript type checking; no further evidence (verify)
17	Are developers' qualifications documented	No
18	Is third-party code inventoried	Yes: `package.json`, `package-lock.json`; SBOM by `npm sbom`
19	Are cryptographic functions standard library implementations	Partly. HMAC, random numbers and token verification use `node:crypto` and `jose`; the SHA-256 of the audit chain is an in-repository implementation, tested against known answers and `node:crypto` (section 4)

6.3 Release and maintenance

No.	Question	Answer
20	How are releases identified	Version number; tag and commit hash (verify)
21	Are release notes provided	`CHANGELOG.md`
22	Are releases signed or checksummed	Verify; otherwise record the commit hash and your own checksum in the IQ
23	How are defects reported and tracked	The project's issue tracker (verify)
24	How are security vulnerabilities reported and announced	`SECURITY.md`
25	Are customers notified of critical defects	No notification service; watch the repository's releases and advisories
26	How long is a version supported	No commitment
27	Is backward compatibility of stored data assured across versions	No formal commitment. The audit trail is replayed by the installed version, so the user must run the new version's `countersign verify` on a copy of production before upgrading (SOP templates, SOP 10)
28	Is there a documented way to verify data without the software	Yes: Data integrity, section 6

6.4 Product characteristics relevant to assessment

No.	Question	Answer
29	Does the software send data to the supplier or third parties	No telemetry in the application's code. Outbound connections at run time: to your identity provider (discovery, token, keys) and your database. The Next.js framework collects anonymous usage data during builds unless `NEXT_TELEMETRY_DISABLED=1` is set
30	Does it need internet access at run time	Only to reach the identity provider, if that is a cloud service
31	Can the audit trail be disabled or configured off	No. It is the data store
32	Can records be deleted	No. Retire and remove are audited state changes; nothing is physically deleted by the application
33	Are there built-in or default accounts or passwords	No accounts and no passwords in the application. The actor `system` writes the first configuration entry only. Demo sign-in exists only with `COUNTERSIGN_AUTH=demo`
34	Are there back doors or administrative overrides of the workflow	None in the engine. The command-line tool can import records as approved and change the configuration in one step, under stated identity, with the corresponding permission, recorded in the trail. A database owner can write to the table directly; that is detected as described in Data integrity, section 5
35	Known limitations	Listed in Part 11 and Annex 11 mapping, section 2, and user requirements, section 6

7. Assessment and decision (complete)

Item	Entry
Software category confirmed (see README, section 4)
Supplier assurance rating (for example: none, low, medium, high) and reason
Compensating measures decided	[ ] Full automated suite executed by us (OQ part A) [ ] Manual and negative tests (OQ parts B and C) [ ] Code review of: ______ [ ] Independent verification script run [ ] Penetration test [ ] Support arrangement with: ______ [ ] Internal fork under our code control [ ] Other: ______
Who maintains the installation and applies security updates
How new releases and advisories are monitored
Decision	[ ] Accepted [ ] Accepted with conditions: ______ [ ] Rejected

Role	Name	Signature	Date
Assessed by
System owner
Quality assurance