Traceability and audit trail

The Nortinia AI Assistant is not a black box. Every mutation a user triggers through chat lands in the audit log just like manual UI actions — with a few extra fields that chat-originated actions deserve.

What we record on every mutation

1. Actor: ai_assistant:<userId> — the user who wrote the prompt. Never "the bot".
2. Original prompt: the chat message that triggered the tool call.
3. Tool name: e.g. refund.initiate, user.changeRole.
4. Tool args: parameters passed to the tool (PII masked where required).
5. Resulting diff: before/after key fields of the changed entity.
6. Tenant ID: which customer account it happened in.
7. Timestamp: second-precision UTC.
8. Success flag: did the tool run, or did it fail at guard / validation.
9. MCP server: which server (admin-mcp, finance-mcp, etc.) served it.
10. Conversation ID: the chat session id so the whole dialogue is reachable.

Retention

Chat-originated audit events live for 365 days in the audit_events Postgres table, then move to an S3 archive bucket (another 6 years for GDPR + bookkeeping). The recent 365 days are queryable in the admin UI; archive queries go through a support ticket (average turnaround: 4 hours).

Filtering in the admin audit view

The admin /audit-events page got four chat-specific filters:

• "Chat-originated only" — actor LIKE 'ai_assistant:%' toggle
• User — who wrote the prompt
• Tool name — which MCP tool ran
• Conversation — every mutation from one dialogue in one view

Common use: "show me what Anna did last week with the assistant in finance". One filter, one list, CSV export.

What we deliberately do NOT store

The model's "inner thought" tokens — the LLM's internal reasoning stream that does not get sent back to the user. The volume would be huge (an average prompt has 2-8k extra tokens), it is noisy, and it is not interpretable as audit data. Even when something goes wrong: the end result (tool call with args) plus the visible reply is enough to reproduce.

Likewise we do not store:

• The retrieval-augmented chunks (the KB document fragments) — only the citation IDs (document + page range).
• The model stream's heartbeat events.
• Voice session internal timing signals (only the final transcript).

What the customer sees

Every tenant admin sees the assistant audit inside their own audit view, in the same place as manual actions. There is no separate "AI audit" tab. This is deliberate: the assistant is an interaction channel between the user and the system — it is not a separate identity. What the user did (by hand or through chat) shows up in one view.

Compliance integration

The chat audit log was reviewed in our SOC 2 Type II and ISO 27001 audit in March 2026. The two main requirements:

1. Reproducibility — we can return the cause of any mutation (the user prompt). Pass, thanks to the Original prompt field.
2. Identity separation — the bot does not appear as its own identity. Pass, thanks to the ai_assistant:<userId> shape.

The most common lookup use-case

Danger-zone mutations (refund, role-change, delete) get next-day sanity check. The finance lead runs a filter: "show me every chat-originated refund in the last 24 hours over 50k HUF". A list. A glance. If something looks wrong: click the conversation ID, see the dialogue that produced the mutation. If it really is wrong: revert the refund and call the operator.

This flow typically takes 2-3 minutes a day and outperforms the old approach (asking on Slack "who did this?").

Where it is going

The next iteration adds two things: real-time audit streaming (Kafka topic so it can feed SIEM systems) and an "explain this action" button on every audit row that returns a natural-language summary of what the tool did. Particularly useful for non-technical leaders who get nothing out of a tool_args JSON blob.