Operational qualification protocol
| Document id | CS-VAL-07 |
| Title | Operational qualification (OQ) protocol |
| Version | 1.0 |
| Applies to | Countersign 1.0.0 |
| Status | Template supplied with the software; to be completed, reviewed and approved by the regulated user |
Approval of the completed document:
| Role | Name | Signature | Date |
|---|---|---|---|
| Prepared by | |||
| Reviewed by | |||
| Approved by (quality assurance) |
1. Purpose and structure
This protocol documents that Countersign operates as specified in the functional specification in your qualified test environment.
| Part | Content | Covers |
|---|---|---|
| A | The project's automated test suite, run by you, with a generated execution report | All 80 functional requirements verified by automated test, including the storage adapter you use |
| B | Manual test cases through the user interface and the HTTP interface | The highest-risk requirements again, end to end with your identity provider, and the user-interface requirements |
| C | Negative tests at database level on a non-production copy | Append-only enforcement and detection of alteration |
Expected results quote the software's messages. Record the actual message; a different wording is a deviation.
| Item | Entry |
|---|---|
| Environment (must have passed IQ) | |
| Application version and commit | |
| Configuration id and SHA-256 | |
| Storage adapter and database version | |
| Sign-in method and identity provider | |
| Start and end date |
2. Prerequisites
| No. | Prerequisite | P/F | Initials, date |
|---|---|---|---|
| 2.1 | IQ of this environment is approved | ||
| 2.2 | This protocol is approved before execution | ||
| 2.3 | Testers are trained in the application, this protocol and deviation handling | ||
| 2.4 | The test accounts of section 4.1 exist in the identity provider with the roles stated | ||
| 2.5 | For part A: a separate, empty PostgreSQL database exists that the tests may use freely | ||
| 2.6 | For part C: a scratch copy of the database exists, with owner access for the tester, clearly separated from production |
3. Part A: automated tests
3.1 What runs
| File | Verifies | Requirement groups |
|---|---|---|
lib/engine.test.ts |
The rules engine | Configuration, workflow, versions, validation, options, retire and remove, audit trail, integrity, access control, import, export |
lib/migrate.test.ts |
Migration and CSV reading | Migration |
lib/adopt/adopt.test.ts |
The migration assistant: reading a source model, drafting, parity survey, verification, journal | Migration |
server/server.test.ts |
Storage adapters (memory, file, SQLite, PostgreSQL), service, settings, sessions, the HTTP handlers against a mock identity provider | Storage, operation, sign-in and signatures |
lib/trace.test.ts |
That every functional requirement has a test or walkthrough step and that the traceability matrix is current | Traceability |
Every test name begins with the requirement it verifies, for example [FR-WF-04]. The PostgreSQL storage tests run
only when COUNTERSIGN_TEST_DATABASE_URL is set; without it they are reported as skipped.
Warning: the PostgreSQL tests drop and recreate the table countersign_entries and the view
countersign_audit_trail in the database that COUNTERSIGN_TEST_DATABASE_URL names. Point it at a database created
for this purpose only. Never point it at a database that holds records.
3.2 Steps
| No. | Instruction | Expected result | Actual result | P/F | Initials, date |
|---|---|---|---|---|---|
| A-1 | On the test application server, in the application directory of the release under test, confirm the commit: git rev-parse HEAD and git status --porcelain |
The approved commit; no local modification | |||
| A-2 | Install dependencies exactly as locked: npm ci |
Completes without error | |||
| A-3 | Set COUNTERSIGN_TEST_DATABASE_URL to the dedicated test database (PostgreSQL of the same major version as production) |
Set; record host and database name, not the password | |||
| A-4 | Run the suite and produce the report: npm run oq or node scripts/oq-report.ts --out oq-report-<date>.md |
The command exits with code 0 and writes the report | |||
| A-5 | Read the report's environment section | It shows the date of this run and the Node.js version of IQ step 4.1; the row "PostgreSQL store tests" reads "run against the database given in COUNTERSIGN_TEST_DATABASE_URL", not "NOT RUN" | |||
| A-6 | Read the report's totals | Failed: 0. Skipped: 0 for the storage adapter you use (with PostgreSQL in production nothing may be skipped) | |||
| A-7 | Check the report's requirement rows against requirements.json | One row per requirement id that has an automated test; every row passed; the 8 FR-UI rows are verified by the walkthrough, not by this run | |||
| A-8 | Regenerate the traceability matrix and compare: npm run trace, then git diff --stat docs/validation/traceability-matrix.md |
No difference; the matrix states "Unverified: 0" | |||
| A-9 | Sign the report's signature block and attach it to this protocol as attachment 1 | Attached |
3.3 Acceptance criteria for part A
- Every automated test passed.
- No test of the storage adapter chosen for production was skipped. (Tests of adapters you do not use may be skipped
only if the report shows that as a skip with its reason; with
COUNTERSIGN_TEST_DATABASE_URLset nothing is skipped.) - The traceability check passed and the matrix shows no unverified requirement.
- The run was made on the release identified in the IQ, without local modification.
4. Part B: manual test cases
4.1 Test accounts and data
Use your approved configuration, or for a first evaluation the fleet preset. Fill in the accounts; each must be a
real, personal test account in the identity provider.
| Placeholder | Needs | Account | Role(s) |
|---|---|---|---|
| A1 | Author permission for record type T | ||
| A2 | Author permission for record type T (a second person with the same role) | ||
| B1 | Approve permission for T; not A1 or A2 | ||
| C1 | Author permission for another record type only, not for T | ||
| R1 | Signed in, no role known to the configuration | ||
| T | A record type of your configuration with a required field, a unique field and, if possible, a reference to a parent type | ||
| O | An option definition, and three levels at which a value can be set (for example parent, group, record), if you use options |
Each case lists: requirements covered; preconditions; steps; expected result. Write the record ids you create into the actual-result cell. "Shown as refused" means: the action is not offered or is disabled and the stated reason is displayed (FR-UI-03); if the interface offers the action anyway, perform it and record the message returned.
B-01 Four eyes: the author cannot approve
| Requirements | FR-WF-01, FR-WF-03, FR-WF-04, FR-UI-03 |
| Preconditions | A1 signed in |
| Steps | 1. As A1 create a record of type T with valid data (button "Create (v1, Created)"). 2. Note its id and state. 3. As A1 confirm it ("Confirm v1"). 4. As A1 look for "Sign and approve" |
| Expected | After step 1 the record exists as version 1, state Created, with the next id of its type. After step 3 the state is Confirmed. Step 4: "Sign and approve" is not offered; the panel shows "Four eyes: you wrote <id> v1, so someone else has to approve it." (if A1's role has no approve permission the message is "As <role name> you can't approve a <type>."; then repeat with an account that has both permissions). The state stays Confirmed |
| Actual result | |
| Pass / Fail | |
| Tester, date |
B-02 Approval by a second person, with meaning
| Requirements | FR-WF-05, FR-WF-06, FR-UI-01, FR-UI-02, FR-AUD-01 |
| Preconditions | The Confirmed record of B-01; B1 signed in within the maximum sign-in age |
| Steps | 1. As B1 open the record (it is also listed in the inbox under "Waiting for your approval"). 2. Read the signature panel "Approve change". 3. Untick the meaning statement. 4. Tick it again and choose "Sign and approve". 5. Open the record's "Version history" and its audit trail |
| Expected | Step 2: the panel shows the change, who wrote it, B1's name and role and the meaning of the signature before signing. The meaning reads "I have reviewed this change and approve it for use." and the signature line reads "Signature: <B1's name> (State what your signature means."). Step 4: the panel shows "In effect" with the version. Step 5: version 1 shows author A1, confirmed by A1 with time, approved by B1 with time and the meaning, labelled "Approved · effective"; the audit trail shows three entries for the record (create, confirm, approve) with consecutive numbers, B1's name and roles on the approval |
| Actual result | |
| Pass / Fail | |
| Tester, date |
B-03 Drafts belong to their author; a Confirmed version is locked
| Requirements | FR-WF-02, FR-WF-03, FR-WF-07 |
| Preconditions | A1 and A2 signed in (two browsers) |
| Steps | 1. As A1 create a record of type T. 2. As A2 try to edit it. 3. As A2 try to confirm it. 4. As A1 edit it and check the version number. 5. As A1 confirm it. 6. As A1 try to edit it |
| Expected | Step 2: shown as refused with "Only the author of v1 can edit it while it is Created." (the server, if the command is sent with changed data, answers "Only <A1's name>, who wrote <id> v1, can edit it while it is Created.") Step 3: refused with "Only the author of <id> v1 can confirm it." Step 4: accepted; still version 1. Step 6: shown as refused with "v1 is Confirmed and locked. Step it back to Created to change it." (from the server: "<id> v1 is Confirmed and locked. Step it back to Created to change it.") |
| Actual result | |
| Pass / Fail | |
| Tester, date |
B-04 Changing an approved record opens a new version; the approved one stays in effect
| Requirements | FR-VER-01, FR-VER-02, FR-WF-09, FR-UI-01, FR-UI-04 |
| Preconditions | The approved record of B-02 |
| Steps | 1. As A2 choose "Edit (new version)", change a field and choose "Save as new version" without a reason. 2. Enter a reason and save. 3. View the record as R1. 4. As A2 try to approve. 5. As B1 approve. 6. Open the history |
| Expected | Step 1: not saved; the form asks "Give a reason for the change" (from the server: "Changing an approved record needs a reason."). Step 2: the record shows "v2 Confirmed, awaiting approval", author A2; the badge "Approved v1" remains and version 1 is still in effect; the pending change is visible beside it ("pending v2"). Step 3: R1 sees version 1's data as in effect and version 2 as pending. Step 4: refused with the four-eyes message for v2 (or the permission message if A2's role cannot approve). Step 5: version 2 is Approved and in effect; version 1 is shown as Superseded. Step 6: both versions with their authors, approvers, times, reason and meaning |
| Actual result | |
| Pass / Fail | |
| Tester, date |
B-05 Step back
| Requirements | FR-WF-08, FR-WF-09, FR-VER-03 |
| Preconditions | The record of B-04 (v2 in effect); a new Confirmed record by A1 |
| Steps | 1. As B1 step the Confirmed record back ("Step back to Created…") with a reason. 2. As B1 look for a further step back. 3. As B1 revoke the approval of v2 of the B-04 record ("Revoke approval…") with a reason. 4. View the record. 5. As R1 try to step back any record |
| Expected | Step 1: the record shows "v1 Created, draft"; the confirmation is cleared; A1 can edit again. Step 2: not offered; from the server "<id> v1 is already Created, the first stage." Step 3: v2 is Confirmed again; v1 is shown as Approved and in effect again. Step 5: shown as refused ("Only the author or an approver can step <id> v<n> back." or, for an approved version, "As someone without a role you can't revoke an approval of a <type>."). Each accepted step back is one audit entry with the reason |
| Actual result | |
| Pass / Fail | |
| Tester, date |
B-06 A reason is mandatory
| Requirements | FR-WF-09, FR-UI-08 |
| Preconditions | An approved record; a Confirmed record |
| Steps | For each of "Step back to Created…", "Revoke approval…", "Retire…" and "Remove…" (or "Remove draft…"): 1. Start the action. 2. Try to submit the dialog with an empty reason. 3. Try with two characters. 4. Cancel |
| Expected | The dialog cannot be submitted empty or with fewer than three characters: it stays open and shows "A reason is required. It is written into the audit trail." (the server's own refusals, if such a command is sent, are "Stepping back needs a reason.", "To retire a record you need to give a reason.", "To remove a record you need to give a reason."). No audit entry is written by any refused attempt (compare the head number before and after) |
| Actual result | |
| Pass / Fail | |
| Tester, date |
B-07 Retire
| Requirements | FR-LIFE-01, FR-LIFE-04, FR-VAL-05 |
| Preconditions | An approved record P of a parent type with at least one live record referring to it; an approved record L of type T that nothing refers to; accounts with the matching permissions |
| Steps | 1. Try to retire P with a reason. 2. As A1 retire L with a reason. 3. View L. 4. As A1 try to approve the retirement. 5. As B1 approve it. 6. As A1 try to edit L. 7. Try to create a record that refers to a retired record, if T can be referred to |
| Expected | Step 1: refused with "<name of P> still has <n> active record(s) that refer(s) to it (<ids>). Retire or move them before you retire it." Step 2: a new version, Confirmed, marked as a retirement; L is still in effect. Step 4: four-eyes refusal. Step 5: L is shown as retired; its history is complete. Step 6: shown as refused with "<id> is being retired or already is." (from the server: "<id> is being retired or retired and can't be edited.") Step 7: refused with "<Type> <id> is retired." |
| Actual result | |
| Pass / Fail | |
| Tester, date |
B-08 Remove
| Requirements | FR-LIFE-02, FR-LIFE-03 |
| Preconditions | A1 has a draft D (never approved); an approved record M that nothing refers to |
| Steps | 1. As A1 try to retire D. 2. As A1 remove D with a reason. 3. Look for D in the lists and in the audit trail. 4. As A1 remove M with a reason. 5. As B1 approve the removal. 6. Look for M in the lists and in the audit trail. 7. Try any action on M (for example through a direct link) |
| Expected | Step 1: refused with "<id> was never approved, so there is nothing to retire. Remove it instead." Step 2 ("Remove draft…"): D is removed at once, without approval; the page shows "Removed." Step 3: D is no longer in lists; its entries (create, remove with reason) are in the audit trail. Step 4: a new Confirmed version marked as removal; M still in effect. Step 5: M disappears from lists. Step 6: all entries of M remain in the audit trail. Step 7: refused with "<id> was removed; it can only be looked at in the audit trail." |
| Actual result | |
| Pass / Fail | |
| Tester, date |
B-09 Role restrictions
| Requirements | FR-ACC-01, FR-ACC-02, FR-WF-05, FR-UI-03 |
| Preconditions | C1 and R1 signed in; a Confirmed record of type T written by A1 |
| Steps | 1. As C1 try to create a record of type T. 2. As C1 try to approve the Confirmed record. 3. As R1 try to create a record of any type. 4. As R1 try to approve. 5. As R1 open the records and the audit trail |
| Expected | Step 1: shown as refused with "As <C1's role name> you can't create a <type>." Step 2: "As <C1's role name> you can't approve a <type>." Steps 3 and 4: the same messages beginning "As someone without a role …" Step 5: R1 can read every record and the whole audit trail (reading is not restricted by role; this is the specified behaviour) |
| Actual result | |
| Pass / Fail | |
| Tester, date |
B-10 Data validation
| Requirements | FR-VAL-01, FR-VAL-02, FR-VAL-03, FR-VAL-04, FR-VAL-05, FR-VAL-07 |
| Preconditions | A1 signed in; an existing record of type T with a value in a unique field |
| Steps | As A1 try to create records of type T with: 1. a required field empty; 2. a number outside its range or a text that breaks its pattern (if T has such a field); 3. the unique value of the existing record; 4. finally with valid data |
| Expected | Step 1: "Enter <label>." Step 2: "<Label>: the smallest allowed value is <min>.", "<Label>: the largest allowed value is <max>." or the configured pattern hint. Step 3: "<Label> <value> is already used by <id>." Step 4: accepted. No record and no audit entry results from steps 1 to 3 |
| Actual result | |
| Pass / Fail | |
| Tester, date |
B-11 Option inheritance and its source
| Requirements | FR-OPT-03, FR-OPT-04, FR-OPT-05, FR-UI-05 |
| Preconditions | Option definition O approved with a default; a record X with a parent Y (and a group G if configured); no value set for O on X, Y, G; accounts with the value permissions for those levels. Skip with justification if options are not used |
| Steps | 1. View the options of X. 2. Set and have approved a value for O at Y. 3. View X. 4. Create and confirm, but do not approve, a value for O at X. 5. View X. 6. Have the value at X approved. 7. View X. 8. Try to set a second value for O at X |
| Expected | Step 1: O shows the definition's default, source "definition default". Step 3: the value of Y, source Y's level and record. Step 5: still the value of Y in effect; the pending value at X is shown beside it as pending. Step 7: the value of X in effect, source X; the value of Y is shown as overridden. Step 8: refused with "<option name> is already set here (<id>). Edit that value instead." |
| Actual result | |
| Pass / Fail | |
| Tester, date |
B-12 Audit trail completeness
| Requirements | FR-AUD-01, FR-AUD-02, FR-AUD-04, FR-UI-04, FR-UI-06 |
| Preconditions | Note the head number H from the integrity page or /api/health before starting |
| Steps | 1. Perform, with times noted from a reference clock: create (A1), edit of the draft (A1), confirm (A1), one refused action (A1 tries to approve), approve (B1). 2. Open the audit trail. 3. Filter by the record, by action, by person. 4. Open the newest entry's details |
| Expected | Exactly four new entries, numbered H+1 to H+4 without gap; the refused action left none. Each entry shows time (within your tolerance of the reference clock), person's name, action, record id and type, version, what changed from what to what, and the reason where one was given. The details show the roles held, the sign-in method (your approved method), the authentication time, the previous hash and the hash. Filters return the matching entries only |
| Actual result | |
| Pass / Fail | |
| Tester, date |
B-13 Export
| Requirements | FR-EXP-01, FR-UI-06 |
| Preconditions | The entries of B-12 |
| Steps | 1. On the audit trail page, with no filter set, choose "Export CSV" and "Export JSON (all)". (With a filter set the CSV holds only the entries shown and the button says how many; the JSON always holds the whole trail.) 2. On the server run countersign export --what audit --format jsonl --out audit.jsonl. 3. Count the entries in each. 4. Compare the entries H+1 to H+4 across the three files and with the screen. 5. Open the CSV in a text editor. 6. Run countersign export --what records --out records.json and countersign export --what config --out config.json |
| Expected | Step 3: the same number of entries in all three, equal to the head number. Step 4: the same sequence numbers, times, actors, actions, records and hashes. Step 5: the header line is seq,at_utc,actor_id,actor_name,actor_roles,action,record_id,record_type,record_name,version,change,reason,signature_meaning,auth_method,auth_time,prev_hash,hash; one line per entry; the approval of B-12 has its meaning in signature_meaning; no cell begins with =, +, - or @ (such values are prefixed with an apostrophe). Step 6: the records file lists every record with all its versions; the configuration file equals the approved document (same SHA-256 by countersign config validate config.json) |
| Actual result | |
| Pass / Fail | |
| Tester, date |
B-14 Integrity page and head hash
| Requirements | FR-UI-07, FR-INT-01, FR-OPS-06 |
| Preconditions | No one else is writing during the test |
| Steps | 1. Open the integrity page and run the verification. 2. Note the number of entries and the head hash. 3. Request /api/health. 4. Run countersign status and countersign verify. 5. Run the standalone verification script of data integrity, section 6.2, on audit.jsonl exported now |
| Expected | Step 1: the page reports "verify prints hash chain intact and replay all <n> entries reproduce |
| Actual result | |
| Pass / Fail | |
| Tester, date |
B-15 A signature needs a recent sign-in (OpenID Connect and Entra ID only)
| Requirements | FR-SIG-05, FR-AUD-04 |
| Preconditions | COUNTERSIGN_SIGN_MAX_AGE_MINUTES = M (record the value); a Confirmed record B1 may approve; B1 signs in freshly and notes the time |
| Steps | 1. Wait until more than M minutes have passed since B1's sign-in. 2. As B1 confirm that reading and a non-signature action still work (for example open records). 3. As B1 approve the record. 4. Follow the prompt to sign in again. 5. Observe the identity provider. 6. Approve again. 7. Open the approval's audit entry |
| Expected | Step 3: refused; the interface reports "A signature needs a sign-in from the last <M> minutes. Sign in again, then approve." (HTTP 401, code reauth_required) and shows "A signature needs a fresh sign-in. Sign in again, then come back and sign."; no entry is written. Step 5: the provider asks B1 to authenticate again (credentials, not a silent return). If the provider returns silently with an old authentication time, the application refuses the sign-in with "The identity provider did not ask you to sign in again (its authentication time is old)."; if it returns silently and the sign-in is accepted (a provider that sends no auth_time), record a deviation. Step 6: accepted. Step 7: the entry's authentication time is the time of the second sign-in, the method is your approved method, the issuer is your provider |
| Actual result | |
| Pass / Fail | |
| Tester, date |
B-16 Sign-out and access without a session
| Requirements | FR-SIG-06, FR-SIG-01 |
| Preconditions | A1 signed in |
| Steps | 1. Copy the address of a record page. 2. Choose "Sign out". 3. Open the copied address. 4. In the same browser request /api/entries. 5. From a command line without cookies: curl -s -w '\n%{http_code}\n' <public URL>/api/entries |
| Expected | Step 3: no record data is shown; the application asks to sign in. Steps 4 and 5: HTTP 401 with {"error":"Sign in first.","code":"signin_required"} |
| Actual result | |
| Pass / Fail | |
| Tester, date |
B-17 The web interface accepts writes only from its own pages and cannot import or configure
| Requirements | FR-SIG-07, FR-SIG-08 |
| Preconditions | A valid session cookie of a test account, copied from the browser into a file for curl (delete it afterwards); the id of a draft record of that account |
| Steps | 1. curl -s -w '\n%{http_code}\n' -X POST <public URL>/api/act -b cookie.txt -H 'Content-Type: application/json' -d '{"command":{"kind":"confirm","id":"<id>"}}' (no custom header). 2. The same with -H 'X-Countersign-Request: 1' -H 'Origin: https://evil.example'. 3. With -H 'X-Countersign-Request: 1' and the body {"command":{"kind":"configure","config":{},"reason":"test"}}. 4. With the header and the body {"command":{"kind":"import","type":"x","data":{},"origin":{"system":"s","id":"1","batch":"b"}}}. 5. With the header and the body {"actor":{"id":"someone-else","name":"Someone Else","roles":["<an approver role>"]},"command":{"kind":"confirm","id":"<id>"}} |
| Expected | Step 1: 403, "The request is missing the X-Countersign-Request header." Step 2: 403, "The request comes from another site." Step 3: 403, ""configure" is only available from the command-line tool (see docs/operations.md)." Step 4: 403, the same for "import". Step 5: 200; the entry returned names the session's person as actor, not "someone-else" |
| Actual result | |
| Pass / Fail | |
| Tester, date |
B-18 Roles follow the identity provider; an altered session is rejected
| Requirements | FR-SIG-04, FR-SIG-01, FR-ACC-02 |
| Preconditions | A test account U with an author role; an identity administrator available |
| Steps | 1. As U sign in and create a draft. 2. Have U's role removed in the identity provider. 3. As U sign out and sign in again. 4. As U try to create a record. 5. In the browser's developer tools change one character of the countersign_session cookie value and reload |
| Expected | Step 4: refused with "As someone without a role you can't create a <type>." Step 5: the application treats the browser as signed out. Note for the record: between steps 2 and 3 the running session keeps the old role until it ends (at most COUNTERSIGN_SESSION_HOURS); this is the specified behaviour and is covered by procedure |
| Actual result | |
| Pass / Fail | |
| Tester, date |
B-19 Two people at once
| Requirements | FR-STO-05, FR-OPS-07, FR-AUD-03 |
| Preconditions | A Confirmed record; B1 and a second approver B2 signed in, each with the record open |
| Steps | 1. B1 approves. 2. Without reloading, B2 approves. 3. Both open the audit trail |
| Expected | Step 1: accepted. Step 2: refused with "<id> v<n> is approved; only a Confirmed version can be approved." Step 3: one approval entry; sequence numbers without gap; times not decreasing |
| Actual result | |
| Pass / Fail | |
| Tester, date |
B-20 Configuration change through the command-line tool
| Requirements | FR-CFG-01, FR-CFG-03, FR-CFG-04, FR-CFG-06 |
| Preconditions | Test environment. A copy of the configuration with one harmless change (for example a field hint), and a second copy that changes the type of a field of a record type that has records. --actor-roles names a role with the configure permission |
| Steps | 1. countersign config validate changed.json. 2. countersign config apply changed.json --actor-id <you> --actor-name "<Your Name>" --actor-roles <role> --reason "OQ B-20" (no reviewer). 3. The same with --reviewer-id <you> --reviewer-name "<Your Name>". 4. The same with --actor-roles naming a role without the permission and a real reviewer. 5. Apply the second copy with a correct role and reviewer. 6. Apply changed.json with a correct role, a reviewer other than you and the reason. 7. countersign status, then countersign verify. 8. Open the audit trail in the application |
| Expected | Step 1: "changed.json is valid: <n> record types, <m> roles. SHA-256 <hash>". Step 2: "countersign: A configuration change needs a reviewer: a second person." Step 3: "countersign: Four eyes: the reviewer must be someone other than you." Step 4: "countersign: As <role name> you can't change the configuration. A role needs the "configure" permission." Step 5: "countersign: Field "<key>" of "<type>" has records, so its type can't change from <old> to <new>." Step 6: "Configuration revision <r> is in effect (entry #<n>, SHA-256 <hash of step 1>)." Step 7: status shows the new revision and hash; verify reports all entries reproduced. Step 8: one entry with action configure, your name, method cli, the reason |
| Actual result | |
| Pass / Fail | |
| Tester, date |
5. Part C: negative tests at database level
Run these on a scratch copy of the database only: a restore of a backup, or the test database after parts A and B are
evaluated. Never run them on production. They need the table owner's credentials, which the application does not have.
In the commands, owner is the owner role and app is the application role of the scratch copy; countersign runs
with COUNTERSIGN_DATABASE_URL pointing at the scratch copy and COUNTERSIGN_SKIP_MIGRATE=1.
| Item | Entry |
|---|---|
| Scratch database (host, name) | |
| How it was created (backup of, date) | |
| Confirmation that it is not production (second person) |
C-01 The database refuses changes to entries
| Requirements | FR-STO-04, FR-STO-02 |
| Preconditions | At least five entries in the scratch copy; triggers enabled (IQ query 6.2) |
| Steps | As app: 1. UPDATE countersign_entries SET actor = 'mallory' WHERE seq = 2; 2. DELETE FROM countersign_entries WHERE seq = 2; 3. TRUNCATE countersign_entries; As owner: 4. the same UPDATE; 5. the same DELETE; 6. the same TRUNCATE; 7. INSERT INTO countersign_entries (seq, at, actor, action, record_id, record_type, prev_hash, hash, entry) VALUES (999999, now(), 'm', 'create', 'X', 'x', repeat('0', 64), repeat('1', 64), '{}'); 8. SELECT count(*), max(seq) FROM countersign_entries; before and after |
| Expected | Steps 1 to 3: ERROR: permission denied for table countersign_entries. Step 4: ERROR: countersign: the audit trail is append-only (UPDATE refused). Step 5: … (DELETE refused). Step 6: … (TRUNCATE refused). Step 7: ERROR: countersign: entry 999999 does not continue the chain. Step 8: count and maximum unchanged |
| Actual result | |
| Pass / Fail | |
| Tester, date |
C-02 An altered entry is detected
| Requirements | FR-INT-02, FR-OPS-02, FR-OPS-06 |
| Preconditions | countersign verify on the scratch copy reports intact; choose an entry number N between 2 and the head; the test application server is stopped |
| Steps | As owner: 1. ALTER TABLE countersign_entries DISABLE TRIGGER USER; 2. UPDATE countersign_entries SET entry = regexp_replace(entry, '"actorName":"[^"]*"', '"actorName":"Somebody Else"') WHERE seq = N; 3. ALTER TABLE countersign_entries ENABLE TRIGGER USER; 4. countersign verify; note the exit code. 5. countersign status. 6. Start the application server against the scratch copy and request /api/health. 7. Sign in and try any change. 8. Open the audit trail |
| Expected | Step 4: hash chain BROKEN at #N: contents changed after writing and replay FAILED at #N: its contents were changed after it was written (the hash no longer matches); exit code 1. Step 5: integrity FAULT at #N: its contents were changed after it was written (the hash no longer matches); exit code 1. Step 6: HTTP 503, "status":"integrity-fault", "fault":{"seq":N,"message":"its contents were changed after it was written (the hash no longer matches)"}, "entries":N-1. Step 7: refused, HTTP 503, "The audit trail failed its integrity check at entry #N (its contents were changed after it was written (the hash no longer matches)). Changes are refused until it has been investigated." Step 8: reading still works and shows the entries before N |
| Actual result | |
| Pass / Fail | |
| Tester, date |
C-03 Removal of the newest entry: what is and is not detected
This case documents a limit as well as a control: a trail shortened at its end is still a valid chain, so only a running server that had already read the entry, or a comparison with an anchored head hash, reveals it.
| Requirements | FR-OPS-03, FR-INT-04; anchoring procedure of data integrity, section 7 |
| Preconditions | A fresh scratch copy that verifies. Record entries (E) and head hash (H) from countersign status as the anchor. Start the test application server against it and request /api/health once (status ok) |
| Steps | As owner: 1. ALTER TABLE countersign_entries DISABLE TRIGGER USER; DELETE FROM countersign_entries WHERE seq = E; ALTER TABLE countersign_entries ENABLE TRIGGER USER; 2. Request /api/health from the running server. 3. Stop the server. 4. countersign verify. 5. countersign verify --expect-head H, then countersign verify --expect-entry E:H; note the exit codes. 6. Start the server and request /api/health |
| Expected | Step 2: HTTP 503, integrity-fault, message "the store holds <E-1> entries, fewer than the <E> already read: entries were removed". Step 4: hash chain intact, replay all <E-1> entries reproduce, exit code 0 (the shortened trail is consistent in itself). Step 5: additionally expected head DOES NOT MATCH <H>, respectively anchor #<E> DOES NOT MATCH (entry #<E> does not exist); exit code 1 each. Step 6: HTTP 200, status ok, entries E-1 (a restarted server cannot know an entry is missing). Conclusion to record: detection after a restart depends on the anchor |
| Actual result | |
| Pass / Fail | |
| Tester, date |
C-04 An entry inserted past the application that breaks the rules
Optional; execute if the risk assessment rates direct database access above low.
| Requirements | FR-INT-03, FR-INT-04, FR-STO-02 |
| Preconditions | A fresh scratch copy that verifies, with head number E and head hash H |
| Steps | As owner, with triggers enabled: 1. Insert a row with seq = E+1, prev_hash = H, any 64-character hash not used before, and entry = '{}'. 2. countersign verify. 3. Start the server and request /api/health |
| Expected | Step 1: accepted by the database (the row continues the chain as far as the trigger can tell). Step 2: hash chain BROKEN at #<E+1>: the entry is not readable and replay FAILED at #<E+1>: the entry is not readable; exit code 1. Step 3: HTTP 503, integrity-fault at E+1 |
| Actual result | |
| Pass / Fail | |
| Tester, date |
After part C destroy the scratch copy or mark it unmistakably as tampered. Record: ______________________.
6. Deviations
| No. | Case | Description | Classification | Resolution | Closed (initials, date) |
|---|---|---|---|---|---|
7. Summary
| Part | Cases executed | Passed | Failed | Deviations |
|---|---|---|---|---|
| A | ||||
| B | ||||
| C |
Attachments: 1. the signed execution report of part A; 2. the regenerated traceability matrix; 3. screenshots and exports of part B; 4. SQL and command output of part C.
| Statement | Yes / No |
|---|---|
| All acceptance criteria of part A are met | |
| All cases of parts B and C passed or deviations are closed | |
| The system is released for performance qualification |
| Role | Name | Signature | Date |
|---|---|---|---|
| Executed by | |||
| Reviewed by | |||
| Approved by (QA) |