Data Governance & Privacy Operations

This guide describes the operational controls shipped with IMCore Ultimate: a reviewed data inventory, access/export and erasure request workflows, legal holds, configurable retention, audit evidence, and a fail-closed execution mode.

It is a capability and operations guide, not a certification or legal opinion. A self-hosted operator normally determines the purposes and means of processing in its deployment and remains responsible for notices, lawful bases, request authentication, processor contracts, cross-border transfers, retention choices, incident response, and local legal review.

Capability map

The admin console shows the same code-reviewed catalogue at System → Data Governance. Reconcile it with deployment-specific systems that IMCore cannot discover, including object-storage buckets, CDN caches, database replicas, backups, log platforms, analytics stores, support tools and downstream processors.

Safe deployment

1. Apply 20260710_create_data_governance.sql and 20260818_complete_data_governance.sql with the migration runner:

   make migrate
   bin/migrate -status
   bin/migrate

   The completion migration creates scoped message-retention policies, archive manifests, erasure markers, legal-hold scopes, and immutable audit-chain state/triggers. Apply the matching file from migrations/mysql/ or migrations/postgres/.

   The migration backfills sender indexes for historical DM and both room-history copies. On large message tables, back up first, estimate locking/runtime and use a maintenance window; pass bin/migrate -timeout 1h (or a reviewed value) when the default 10-minute migration timeout is insufficient.

2. Grant governance:read only to reviewers and governance:write only to authorised privacy/security operators. Built-in superadmins have both; an operations role does not receive them automatically.

3. Keep destructive execution disabled while policies and the runbook are reviewed:

   governance:
     execution_enabled: false
     pseudonym_salt: ""
     requests:
       default_due_days: 30
     retention:
       scheduler_enabled: false
       interval_hours: 24
       batch_size: 1000

4. Store the stable pseudonym key outside source control, for example IMCORE_GOVERNANCE_PSEUDONYM_SALT. Losing or changing it makes pseudonyms inconsistent; exposing it weakens pseudonym separation. Back it up as a production secret and rotate only through a reviewed migration plan.

5. Run retention previews, sample the eligible records, test restoration and downstream deletion, and obtain policy/legal approval.

6. Only then set IMCORE_GOVERNANCE_EXECUTION_ENABLED=true. Enable IMCORE_GOVERNANCE_RETENTION_SCHEDULER_ENABLED=true separately after manual runs are accepted.

Both erasure and retention are fail-closed. Erasure requires the execution flag and pseudonym salt. Scheduled deletion requires both the scheduler and execution flags. The scheduler does not run immediately at startup.

Subject-request workflow

1. Intake and identity verification: authenticate the requester outside IMCore; register an access or erasure case with its source, reason and due date. Do not place identity documents in the free-text reason field.
2. Review: confirm scope, identity, legal basis/approval authority, exceptions and active disputes. Approval requires a legal-basis/authority note; rejection requires a review note.
3. Legal hold check: create a subject legal hold before review if litigation, investigation, fraud, security or another preservation duty may apply. An active, unexpired hold blocks erasure and excludes subject-linked rows from retention.
4. Execute: first deactivate the source business account and revoke its client tokens so data cannot be recreated, then type the exact subject user ID. Access builds a fresh structured JSON export. Erasure force-logs-out current IMCore sessions, deletes directly owned records and replaces retained message authorship/conversation identifiers with a deterministic pseudonym.
5. Deliver and close: transfer exports through an authenticated, time-limited channel outside IMCore. Record delivery, decision correspondence and any residual-system work in the organisation’s case system.

Exports set Cache-Control: no-store, redact push credentials, and include an attachment manifest (object_key, thumbnail) without presigned URLs. The export endpoint is still sensitive: reverse proxies, browser downloads and operator workstations must enforce suitable controls.

Erasure boundaries

Built-in erasure covers the database tables listed in the inventory, records a durable data_erasure_markers row for every discovered attachment, deletes objects through the configured object store, and makes a best-effort Redis cache purge. Failed object deletions remain replayable with governance -command replay-erasure. It deliberately does not perform substring replacement inside free-form message text because doing so could corrupt other participants’ records and produce false matches.

Before closing a case, separately account for:

• attachments, thumbnails and recordings in S3/OSS/MinIO/local object storage;
• database replicas, search indexes, data warehouses, analytics and log/SIEM systems;
• broker queues, dead-letter queues and offline exports;
• backups and snapshots, including the point at which data ages out and how restoration re-applies tombstones;
• downstream processors, integrations and customer business systems;
• external identity/account systems and any credential that could reconnect and recreate the subject;
• free-form content in which another person mentioned the subject;
• statutory or contractual records retained under a documented exception.

A practical backup policy is to prevent normal access to erased data, expire it on the documented backup schedule, restrict restores to authorised incidents, and re-run approved erasure tombstones immediately after a restore.

Retention policies

Message retention, archive and restore

Message policy scope is version, session, or group; scene can narrow it to DM, room, group, community, or all messages. The active version is selected by governance.message_retention.version (default default). A matching session/group policy overrides the version policy. Rows older than hot_days are serialized to governance/message-archives/ in the configured object store with SHA-256 manifests before hot rows are deleted. Set governance.archive.enabled=true for execution and use archive.storage_class for the provider’s cold-tier class. Related group audiences, community mentions and message receipts are included in v2 archive envelopes.

Use governance -command retention for JSON evidence and governance -command restore-archive -archive-id <id> for a controlled restore.

The example defaults are deployment templates, not legal recommendations:

Set a policy to 0 to disable it. Every manual or scheduled run records its policy snapshot, mode, status, summary, actor and timestamps in data_retention_runs. Always preview after policy or schema changes. Legal holds protect only policies with a configured subject column; infrastructure-wide audit evidence may require separate case tagging and preservation procedures.

API and audit surface

The admin JWT and RBAC-protected endpoints live under /admin/api/governance:

• GET /overview
• GET|POST /requests, GET /requests/{id}
• POST /requests/{id}/review, POST /requests/{id}/execute, GET /requests/{id}/export
• GET|POST /legal-holds, POST /legal-holds/{id}/release
• POST /retention/run
• GET|PUT /message-retention-policies, DELETE /message-retention-policies/{id}
• GET /message-archives, POST /message-archives/{id}/restore

Audit integrity is available at GET /admin/api/audit-logs/integrity and via governance -command verify-audit. It reports signed, legacy and failed-chain evidence; database triggers reject audit UPDATE/DELETE.

The complete request/response contract is embedded in controllers/openapi/service.openapi.yaml and served by the service at /openapi.yaml and /openapi.json. Mutations and subject exports pass through admin audit capture. Keep audit access independent from request execution where staffing permits.

Regulatory capability mapping

This mapping helps a reviewer locate product controls; it does not determine applicability or prove compliance.

• China’s Personal Information Protection Law provides rights and processing obligations around access/copy, correction, deletion and convenient request mechanisms in Articles 45–50. IMCore’s access/export, reviewed erasure, due dates and evidence workflow support the access and deletion portions; correction still uses the existing profile/business-system APIs under the operator’s case process. See the official CAC text.
• GDPR Article 5 includes data minimisation, storage limitation and accountability; the inventory, retention previews and run evidence support those operational principles. See official Article 5.
• GDPR Article 15 addresses access and copies, which the structured export can support. See official Article 15.
• GDPR Article 17 defines erasure rights and exceptions; the approval, legal-hold and pseudonymisation workflow supports implementation without deciding whether a request must be granted. See official Article 17.

Release and incident checklist

• Review the inventory whenever a table, message field, storage provider or processor is added.
• Run make secret-check; keep real keys out of tracked files and build artifacts.
• Rotate any credential that was ever committed. Removing it from the current tree does not revoke it or remove it from Git history; coordinated history rewriting is a separate operation.
• Test access, hold, erasure and retention paths in a production-like environment with synthetic users.
• Verify permissions, audit records, TLS, proxy caching, download handling and operator endpoint security.
• Document backup expiry, restore-time tombstone replay, processor notifications and incident ownership.
• Have counsel/privacy reviewers approve the final policy values, notices and exception handling for each deployment.