Conversion metric — the nuanced truth about AI Chat

The worst question to ask a chat-system seller: "What is the conversion rate?". The worst answer you can get: a single percentage. Because the word conversion here hides more than a number can tell.

The KPI that is not a KPI

The metric most often quoted about chat systems: "X% of visitors who opened the chat made a purchase". That would be meaningful if the chat's only goal were to convert opening visitors into buyers. But that is not the chat's goal. (Article 1 in this series covers what the chat actually does for the visitor — worth re-reading.)

Problems with this number:

1. It does not measure navigational use. The 38% "I just asked where something is" group counts as zero here, even though the chat was equally valuable to them.
2. It does not measure deflected support. Those 30-50 daily questions a human did not have to answer are not conversions. But they are money saved (in operator hours, measured in euros).
3. It does not measure assisted purchases. A visitor asks about the Pro vs Enterprise difference, leaves, comes back two days later, buys. Attribution typically credits the last touch, not the chat. Yet the chat was the deciding moment.
4. It is too easy to game. If an A/B test shows 12% conversion after chat-open vs 9% no-chat, marketing will post "+33% conversion!". But the +3 pp can contain selection bias (those who open chat were already more motivated), contamination (after chat-open, sessions average 4 minutes longer), and ignore the deflected users.

The three metrics that matter

Nortinia AI Chat's dashboard shows three base metrics, each decomposable:

1. Assisted-purchase rate

Definition: orders where there was at least one chat interaction in the 24 hours before checkout, divided by all orders. Plus: the chat interaction must be at least 2 user messages (otherwise spurious).

Fleet median: 17%. Range: 8% (low-touch B2C storefront) – 31% (high-touch consulting catalog). This is the real "did the chat contribute to the purchase" indicator.

2. Deflected support-ticket count

Definition: chat sessions where the visitor got an answer to a question that would otherwise have become a support ticket (FAQ-style, billing, warranty, or shipping question). Handed-off cases do NOT count here.

Fleet median: 38 deflected tickets / tenant / day. Assuming €4.50 / ticket average operator cost, that is €5,130 / month savings per tenant. One of the chat's most concrete financial values.

3. Qualified-lead count

Definition: an email or phone contact the visitor offered voluntarily (per article 2's "on-async-request" pattern), AND that sales later validated (real contact + business interest exists). Spam and never-replying addresses do not appear here.

Fleet median: 12 qualified leads / tenant / week. The direct feed of the SDR funnel. (Raw lead count is 3-4× higher, but only the qualified count matters here.)

The honest dashboard

A dashboard line:

Last 30 days:
  Sessions:                  12,348
  Sessions with purchase:       287 (2.3%)
  Sessions with navigation:   4,692 (38.0%)
  Assisted-purchase rate:     17.4%
  Deflected support tickets: 1,140 (38/day)
  Qualified leads:               52 (12/week)
  Handoff to human:             132 (1.1%)
  Handoff median reply:       4.1 min

No single "conversion" number. No marketing euphemism. Four useful numbers and context next to them.

What not to do

Do not try to reduce the chat system's ROI to a single percentage for leadership. The two most common mistakes:

1. "How much did conversion go up?" — answer: that is not the question. The question is: how many support hours did we redeem, how many leads did we qualify, how many purchases did we assist.
2. "Let's optimize chat-open rate!" — that is popup-marketing logic. Nobody wants to open chat panels. The value is that those who open it get help.

Lesson

A chat system's ROI is measured on three or four numbers, not one. Anyone giving you a single percentage is either selling you something or not measuring seriously. A good dashboard is not pretty — it is true.