Skip to content

Test the assistant and find out what it costs

In System → AI Centre → Assistant tests a set of questions is run through the real assistant, at the basic level and with your company’s data. When it finishes it says which questions it answered well, which ones it gets wrong and why, and what a month of use would cost compared with the credits a plan includes. It is a technical support screen.

What questions does the set contain?

From four families, about thirty in total:

FamilyWhat it checks
Self-knowledgeHelp with the program itself (‘where are favourites set?’) and questions about the assistant (‘which model are you?’)
Tasks in sequenceRequests with several chained steps (‘create the customer and then make them a quote’)
Model escalationThat it offers the professional analysis when the question needs judgement, and doesn’t offer it for a figure
Complex queriesData questions that require combining several lookups (‘how much have I invoiced, how much am I owed and by whom’)

Each question opens a new conversation. Some have several messages in a row, to check that the assistant follows the context (‘and how much do they owe me now?’).

Can it change anything in my company?

No. The assistant prepares its proposed actions as usual, and the test run cancels them all when each question ends: it never confirms anything. The only thing it uses is credits, just as if a person had asked the questions.

While it runs you can go to other screens: the test run carries on by itself and saves each question as soon as it ends. What you shouldn’t do is use the assistant panel in the same tab, because the test run uses that same assistant; when it finishes, the panel starts a new conversation. If the tab is closed, the test run stops, but what was done up to then is saved as an incomplete run.

How is it decided whether an answer is right?

In two ways, and both can be seen in the Case by case table:

Each question comes out Right, Wrong or Could not be assessed. The last one is for what does not depend on the assistant: if your company’s professional analysis is used up this month, the questions that should offer it cannot do so, and they don’t count as a failure.

What do the cost columns mean?

All of them start from what each question really cost —what was deducted from the credits— and take it to a month of use with the Queries per month figure (200 to start with: about ten per working day).

ColumnWhat it says
Credits per queryWhat a query like this costs at the basic level
Monthly costWhat all the month’s queries would cost if they were like this one
ShortfallWhat is missing over the credits included in the plan chosen in Compare with the plan; ‘covered’ if they are enough
Advanced monthly costThe same, adding the professional analysis every time it was offered
Advanced shortfallWhat is missing in that case

The professional analysis is not accepted unless you tick Accept the professional analysis when it is offered: without that box, the advanced cost uses what the offer itself estimated.

Are previous runs kept?

Yes. Each run is saved with its date, and the Run drop-down lets you go back to any of the earlier ones. That way you can compare how the assistant answers before and after a change.

If something goes wrong

What you seeWhat has happenedWhat to do
The escalation questions come out ‘Could not be assessed’The professional analysis for the month is used upIncrease the month’s budget and run the set again
A question comes out ‘The turn failed’The assistant didn’t answer in time or gave an errorRun it again; if it happens again, look at the incident in the AI Centre
The run shows as ‘incomplete’The tab was closed, Stop was pressed or the server restarted halfwayWhat you see is valid; for the rest, run the set again
The cost of a question looks too highSomeone in the company used the assistant at the same timeRun the set at a quiet time

See also: See what the assistant uses · Ask for a professional analysis · See what level it answered you at

More about Assistant