# SAKSHI (साक्षी, witness): product spec and methods note

SAKSHI is the oversight layer under every Trinetra product. It answers three questions for an
oversight officer, a data protection officer or a court: who did what, under which declared
purpose, and can it be shown that the record was not altered.

- Console: `/dashboard/sakshi/` (overview, chain, policy, approvals, retention, PII scan, cards, forecast check, methods)
- Public: `/transparency/` (transparency report) and `/products/sakshi.html`
- Code: `pipeline/trinetra/sakshi/` (Python), `dashboard/sakshi/*.js` (browser)
- Data: `dashboard/data/sakshi/` (see "Exports")
- Tests: `tests/test_sakshi.py` (offline; runs the browser code in Node for parity)

## 1. Users and decisions

| User | Decision SAKSHI supports |
|---|---|
| Oversight and audit officers | Was the record altered? Should this request have been allowed? |
| Data protection officers | Is data deleted on schedule? Does any public file carry personal data? |
| Courts and inquiries | Can the integrity of an evidence trail be shown without trusting the operator? |
| Commanders | Second key on exports of tracks, person-level searches and sharing outside |

## 2. Audit chain

### Entry

```json
{"seq": 12, "ts": "2026-10-21T09:30:00.123Z", "actor": "op:a07", "action": "export.download",
 "purpose": "disaster_response", "resource": "/dashboard/data/varuna/stations.json",
 "detail": {"rows": 18}, "prev_hash": "<64 hex>", "hash": "<64 hex>"}
```

- `hash = SHA-256(UTF-8(JCS(entry without "hash")))`, JCS = RFC 8785 canonical JSON.
- `prev_hash` of entry n is the hash of entry n-1; entry 0 is `chain.genesis` with 64 zeros. The
  genesis detail records the scheme (`sakshi-chain/1`), the checkpoint interval, the chain id and,
  for a new segment, where the previous segment ended (`continues`).
- `ts` is UTC with milliseconds. Writers should sync clocks to NIC or NPL NTP (CERT-In direction (i),
  28 April 2022).
- `detail` is a JSON object of at most 8 KB. Entries hold references, counts and hashes, never record
  contents. The API refuses entries carrying a mobile number, an Aadhaar number (grouped 4-4-4 or next
  to an Aadhaar word) or an individual PAN.

Canonical JSON rules (both runtimes): keys sorted by UTF-16 code units; strings escaped as ECMAScript
`JSON.stringify` does; numbers in ECMAScript `Number::toString` form (Python reproduces it from the
shortest round-trip digits, checked against Node on 400,000 doubles); refuse NaN, Infinity, integers
not exactly representable as doubles, lone surrogates, non-string keys, nesting deeper than 64.

### Checkpoints and anchors

Every `checkpoint_every` entries (16 browser and API, 8 platform ledger) the writer appends
`chain.checkpoint` with `{"tree_size": n, "root": MTH(hashes of entries 0..n-1)}`:

```
leaf = SHA-256(0x00 || entry_hash_bytes)     node = SHA-256(0x01 || left || right)
MTH(D[n]) = node(MTH(D[0:k]), MTH(D[k:n])), k = largest power of two < n,  MTH({}) = SHA-256("")
```

Inclusion proofs follow RFC 9162 §2.1.3.2; consistency proofs §2.1.4.2. The Merkle code reproduces
the Certificate Transparency reference roots for trees of 0 to 8 leaves.

Anchors are checkpoint roots kept where the chain's writer cannot change them: `anchors.json`, the
transparency report, git history, or a file an auditor saved. `verify(anchors=…)` recomputes each
anchored prefix.

### Verification

Per line: `bad_json` (NaN and Infinity included: they are not JSON), `bad_schema`, `hash_mismatch`,
`not_canonical` (a value with no canonical form, such as a lone surrogate, so the stored hash cannot
vouch for the line), `link_broken`, `seq_gap`, `seq_backwards` (both
judged against the previous line, so one insertion or deletion is one issue), `genesis_invalid`,
`checkpoint_mismatch`, `checkpoint_missing`, warning `ts_regress`. Per anchor: `anchor_truncated`,
`anchor_mismatch`, `anchor_invalid`. Diagnosis from the sequence numbers and codes: `edit`,
`deletion`, `insertion`, `reorder`, `truncation`, `rewrite`, `corruption`.

| Change | Without anchors | With anchors |
|---|---|---|
| Edit, hash not recomputed | detected (`hash_mismatch`) | detected |
| Edit, that entry re-hashed | detected (`link_broken` on the next line, checkpoint) | detected |
| Delete, insert, reorder | detected | detected |
| Remove the newest entries | not detectable | detected if an anchor covers them |
| Edit and recompute everything after | not detectable | detected (`rewrite`) |

### Browser chain

`dashboard/sakshi/chain.js` (`window.SK`) keeps the chain in `localStorage["trinetra.sakshi.chain"]`,
one entry per line, hashing with WebCrypto (a pure-JS SHA-256 is the fallback and a self-test).
Appends hold a Web Lock so two tabs cannot fork the chain, and run in batches so checkpoints cost
O(log n). `SK.chain.importPlainAudit()` seals entries the shell's `T.audit` wrote to
`localStorage["trinetra.audit"]`, masking phone, Aadhaar-like, PAN and email patterns first and
keeping the original time as `detail.original_at`; a watermark (`trinetra.sakshi.sealed`) prevents
double sealing. Each plain entry is made sealable first (strings well formed and clipped to 1,000
characters, a detail still over 8 KB replaced by its size and keys), so one odd entry cannot block
later imports. Every write, the import included, runs in one queue (a Web Lock plus an in-page
promise queue), so concurrent writers neither fork the chain nor seal an entry twice.

The shell's `T.audit` calls `SK.audit(action, detail)` whenever `chain.js` is on the page (Sakshi and
Gati pages). That call seals the entry from the plain log, with its page and original time, so the
page imports that follow find nothing new. Module code passes options and gets one sealed entry:
`SK.audit(action, detail, { purpose, resource })`.
`SK.chain.rollover()` starts a new segment whose genesis records the old chain id, head hash, entry
count and root; the page downloads the old segment first.

To seal `T.audit` entries on every other console page, the shell loads one script after `app.js`
(`dashboard/sakshi/hook.js`): it loads `chain.js` when needed, wraps `T.audit`, and calls
`importPlainAudit()` after each entry (debounced; imports are idempotent). The integrator change is
two lines in `renderShell()`:

```js
const h = document.createElement("script"); h.src = "/dashboard/sakshi/hook.js"; document.head.appendChild(h);
```

For other modules:

```html
<script src="/dashboard/sakshi/chain.js"></script>
<script>
  await SK.audit("gati.assign", { alert: "A-17", team: "QRT-2" },
                 { purpose: "disaster_response", resource: "alert:A-17" });
</script>
```

## 3. Purpose-binding policy

One file, `dashboard/data/sakshi/policy.json`, read by `policy.py` and `policy.js`.

- Roles: analyst, operator, commander, oversight_officer, researcher, trainee.
- Purposes: disaster_response, interdiction_ops, search_and_rescue, training, oversight_audit,
  public_research. Operational purposes need a ticket (reference plus expiry, at most 7 to 90 days).
- Data classes: public_osint, aggregated_stats, sensor_tracks, simulated, environmental,
  personal_redacted, personal_identifiable (always denied).
- Actions: view, query, export, share_external.

Evaluation order (every step is in the trace): `V-1` vocabulary; doctrine rules `RL-1` (identifiable
personal data), `RL-2` (redacted records never shared outside), `RL-3` (research and training stay
non-operational), `TP-1` and `TP-2` (two-person rule for person-level search or export, and for track
copies); `RL-4` protected-attribute firewall over filters and query text (whole tokens, plurals);
`B-1` ticket; `B-2` role-purpose; `B-3` role-action; `B-4` purpose-class; `B-5` action under that
binding. Combining is deny-overrides (XACML 3.0): any deny, else any approval, else allow.
Obligations `OB-1` to `OB-7` attach to non-deny decisions.

`policy_vectors.json` holds 47 hand-written cases with expected decision, deciding checks and
obligations. Tests compare Python and JavaScript on all 1,008 combinations, trace text included.

Known gap: the firewall checks attribute words, not community names.

## 4. Two-person rule

Request only when the policy says `approval_required` (a deny is logged as `approval.refused`).
Justification of at least 12 characters; expiry 24 hours; approver must be another actor with an
approver role for the purpose (commander or oversight officer; oversight officer only for
`oversight_audit`; a commander only for purposes the commander holds); rejection needs a note; the
decision carries the request's hash; single use by the requester before expiry.

State is a fold over the chain (`approvals.fold`, `SK.approvals.fold`). The fold re-checks every step
and records violations as blocked: decisions by the requester, by an ineligible role, after expiry,
or bound to another version of the request.

## 5. Retention

| Class | Keep | Then | Basis |
|---|---|---|---|
| Public OSINT | 2 years | delete | Trinetra policy; re-fetchable |
| Aggregated statistics | 10 years | archive | no record-level data |
| Sensor tracks | 90 days | delete | operational need; legal hold if attached to an incident |
| Simulated | 30 days | delete | regenerable from seed |
| Environmental | 10 years | archive | climatology baselines |
| Personal, redacted | 1 year | delete | DPDP Act 2023 s.8(7); DPDP Rules 2025 Rule 6 one-year logs |
| Personal, identifiable | never stored | | Red line 3 |
| Audit chain | 3 years | seal and archive segment | CERT-In direction (iv) 180 days; DPDP Rules one year |

Purge: `expires_at = created_at + days`; expired and not held is deleted; receipt holds counts by
class, bytes freed and an RFC 6962 root over sorted purged ids. The demo store is SIMULATED (360
records, seed 20261021); the browser and pipeline receipts match for the same date.

## 6. PII scanner

Scope: every `.json`, `.geojson`, `.jsonl`, `.csv` under `dashboard/data/` and the public site
folders, except SAKSHI's own PII outputs. Every string is checked with its JSON path; numbers in URLs,
decimals and longer digit runs are ignored.

| Rule | Severity | Pattern |
|---|---|---|
| mobile | high | 10 digits starting 6-9, optional +91 / 0091 / 0, groupings 5-5 and 3-3-4 |
| aadhaar | high | 12 digits, first 2-9, not a palindrome, Verhoeff valid, grouped 4-4-4 or next to an Aadhaar word |
| aadhaar_like | medium | as above, ungrouped and without context; a YYYYMMDDhhmm stamp is skipped |
| pan / pan_entity | high / medium | `AAA` + holder type + `A9999A`; P (individual) is high |
| email / email_official | medium / info | official: gov.in, nic.in, ac.in, edu, res.in, int, Trinetra |
| person_unmarked | medium | `kind: person` without `official` or `redacted` marking or a role label |
| person_scored | high | person object with `risk_score` or similar (red line 1) |
| person_in_text | medium | name of an unmarked person entity inside another text field |
| kinship | medium | S/o, D/o, W/o, son/daughter/wife of + capitalised name |

Output `pii_report.json` (masked values only: last four digits, PAN holder type, email initial and
domain, name initials) and `pii_history.json` (one row per run). Tests generate every number at run
time in a temporary directory, so no value that could belong to a real person is committed.

## 7. Model and data cards

`cards.json`: one card per registered module plus CONSOLE, joining the module's manifest (sources,
licences, retrieval times, files, simulated flag, seed, methods, notes), SAKSHI-authored users,
intended use, prohibited use (module-specific plus five doctrine prohibitions), limits and
computation, and ten checks: manifest published, UTC generated time, sources with links, licence per
source, retrieval time per web source, listed files exist, JSON under 1.5 MB and PNG under 600 KB,
seed recorded for simulated data, PII scan clean, updated within 7 days. Without a console manifest,
the CONSOLE card is derived from `stats.json` and `source_health.json` and says so.

## 8. Integrity seal and ledger

`seal.json` lists SHA-256 and size of every file under `dashboard/data/` and every public data file,
with an RFC 6962 root over leaves `JCS({bytes, path, sha256})` sorted by path, and the diff from the
previous seal. Inside a git checkout the seal hashes each file as committed in `HEAD` (the version a push
publishes, read with `git ls-tree` and `git cat-file`); SAKSHI's own outputs are hashed from disk because
the run writes them and they are committed with it; files that differ on disk are listed under
`basis.pending` (modified, untracked, deleted) and sealed by a later run. Outside a checkout it hashes the
files on disk. `ledger.jsonl` is the platform's real chain (append-only across runs, checkpoint every
8): `policy.publish` when the policy hash changes, `pii.scan`, `cards.publish`, `forecast.verify`
(or `forecast.stale` when the last check was kept), `data.seal`. The pipeline verifies the ledger
before extending it and stops if it fails.

In the browser (`SK.sealRoot`, `SK.checkSeal`, `SK.ledgerReceipts`), the verifier page and the
public transparency report let anyone: verify the ledger against the published anchors; confirm that
the served `pii_report.json`, `cards.json`, `forecast_check.json` and `seal.json` are byte for byte the
files the ledger recorded (`report_sha256`, `cards_sha256`, `check_sha256`, `seal_sha256`); and
re-hash every sealed public file. A
file re-exported by its product after the last SAKSHI run shows as changed until the next run, and a
removed file shows as missing (the browser logs its 404).

## 9. Forecast check

The heat, cold, road and patrol advisories (Raksha, Marga, Drishti, Saathi and others) rest on
Open-Meteo best-match forecasts. `forecast_check.py` scores them against routine airport reports near
the frontier over the last 28 complete UTC days, at leads of 1 to 7 days, and writes
`forecast_check.json`; the overview, the cards of the products that use these forecasts and the
public report carry the result.

- Forecast: Open-Meteo Previous Runs API, `<variable>_previous_dayN` = value predicted N x 24 h before
  the valid hour, at each airport's coordinates and elevation (temperature, dew point, 10 m wind in kn).
- Observed: routine METARs (no SPECI) from the NOAA Aviation Weather Center data API, at the minute each
  airport mostly reports; at half past, the hourly forecast is interpolated. 12 airports report; 7
  frontier airfields publish nothing and are listed as gaps.
- Metrics: bias = mean(f - o), MAE, RMSE, share within 2 °C (5 kn for wind); skill = 1 - MAE_f /
  MAE_persistence on the same hours (persistence = the value N x 24 h earlier; "better" needs skill
  above 0.05); afternoon peak error over 06:00 to 11:59 UTC on days with at least 4 matched hours.
- Plains and coast are reported apart from the mountain airports (Leh, Pakyong).
- Network: about 61 requests on a first run and about 13 a day after, cached under
  `pipeline/cache/sakshi/`; windows that ended more than 36 hours ago are never refetched. `--offline`
  rebuilds from the cache; a failed run keeps the last check, marks it stale and records
  `forecast.stale` in the ledger.
- Limits: airports stand in for places up to 100 km away; whole-degree reports add about 0.3 °C of
  error; one season at a time; rain, fog, sea state and river flow are not scored. Advisory only.

## 10. Exports (`dashboard/data/sakshi/`)

| File | Content | Simulated |
|---|---|---|
| policy.json, policy_vectors.json, vectors.json | policy and shared test vectors | no |
| pii_report.json, pii_history.json | PII scan | no |
| cards.json | model and data cards | no |
| forecast_check.json | forecast check against airport reports | no |
| seal.json, ledger.jsonl, anchors.json | integrity seal, ledger, anchors | no |
| transparency.json | summary for the public report | no |
| demo_chain.jsonl, approvals.json, retention.json | demo session, approvals, record store | yes |
| manifest.json | provenance | |

## 11. API

`pipeline/trinetra/sakshi/api.py`, `create_router()`, prefix `/api/sakshi`: `POST /append`,
`GET /verify`, `GET /entries`, `GET /head`, `GET /checkpoints`, `GET /proof/{seq}`,
`GET /consistency`, `POST /policy/evaluate` (logs the decision; query text only as SHA-256; refuses
filters, ticket references or actors carrying personal data). Chain
file `pipeline/data/sakshi/audit.jsonl` (gitignored) or `SAKSHI_CHAIN_PATH`. Mount in `serve.py`:

```python
from ..sakshi.api import create_router as sakshi_router
app.include_router(sakshi_router())
```

## 12. Commands

```
cd pipeline
python3 -m trinetra.sakshi run [--offline] [--no-forecast]
                                                    # all exports; only the forecast check uses the
                                                    # network (--offline: from the cache only)
python3 -m trinetra.sakshi scan                     # PII scan only
python3 -m trinetra.sakshi verify <log.jsonl> [--anchors anchors.json] [--json]
python3 -m trinetra.sakshi canonical <file.json>
cd .. && python3 -m pytest tests/test_sakshi.py -q
```

Run SAKSHI last in a deploy, after every module's exports are committed, so the seal matches the deployed files.

## 13. Limits and failure modes

- A chain can be rewritten end to end by its holder; only anchors kept elsewhere catch it.
- The browser chain is in localStorage: clearing site data deletes it; quotas are a few MB.
- Until the shell loads the chain on every page, pages other than Sakshi and Gati write a plain,
  editable log first; entries older than its last 1,000 are dropped before they can be sealed.
- Declared purposes are trusted at evaluation time; misuse shows up in review, not in the decision.
- PII detection is pattern-based; free-text names, images and PDFs are out of reach.
- Approval identities on the demo page are simulated.
- Oversight findings are advisory; decisions about people rest with accountable officers.

## 14. References

- NIST FIPS 180-4 (SHA-256): https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.180-4.pdf
- RFC 8785 (JCS): https://www.rfc-editor.org/rfc/rfc8785.html
- RFC 6962 (Certificate Transparency): https://www.rfc-editor.org/rfc/rfc6962.html
- RFC 9162 (CT 2.0): https://www.rfc-editor.org/rfc/rfc9162.html
- OASIS XACML 3.0 core: https://docs.oasis-open.org/xacml/3.0/xacml-3.0-core-spec-os-en.html
- W3C Web Cryptography API: https://www.w3.org/TR/WebCryptoAPI/
- DPDP Act 2023, s.8 (unofficial consolidated text): https://www.dpdpa.com/dpdpa2023/chapter-2/section8.html
- PIB backgrounder, DPDP Rules 2025 notified (17 Nov 2025): https://static.pib.gov.in/WriteReadData/specificdocs/documents/2025/nov/doc20251117695301.pdf
- DPDP Rules 2025, Rule 6 (unofficial): https://dpdpa.com/dpdparules/rule6.html
- CERT-In Directions No. 20(3)/2022, 28 April 2022: https://www.cert-in.org.in/PDF/CERT-In_Directions_70B_28.04.2022.pdf
- python-stdnum Aadhaar and PAN: https://github.com/arthurdejong/python-stdnum/tree/master/stdnum/in_
- Verhoeff algorithm: https://en.wikipedia.org/wiki/Verhoeff_algorithm
- Mitchell et al., Model Cards for Model Reporting: https://arxiv.org/abs/1810.03993
- Gebru et al., Datasheets for Datasets: https://arxiv.org/abs/1803.09010
- NOAA Aviation Weather Center Data API: https://aviationweather.gov/data/api/
- Open-Meteo Previous Runs API: https://open-meteo.com/en/docs/previous-runs-api
- JWGFVR, Forecast verification: methods, issues and FAQ: https://jwgfvr.github.io/forecastverification
- Research notes: `docs/research/2026-10-oversight-and-law.md`