Quizzes and assessments¶

A question set is a list of questions with their answers. Your colleagues can take someone through one, a question at a time, in the conversation they are already having — marking each answer, explaining it afterwards, and keeping the score.

The same set of tools covers three quite different things. What changes is what is at stake, and you choose that for each set:

Stakes	What it is	What is kept
Compliance	An assessment that counts. It has a pass mark.	A record of who passed.
Practice	A quiz to help people learn.	A score, nothing kept for long.
Play	A game.	Nobody's words are kept.

Whatever the stakes, AIR keeps whether each answer was right, not what the person typed. The only exception is an answer that is waiting for a person to mark it — and never for a game.

The answers are private

A set's answers are kept private. The colleague running a quiz can't see them, so it can't let an answer slip, and an answer is only explained to someone after they have given their own.

Where to find them¶

Question sets belong either to the whole organisation or to one team.

Organisation assessments — from Teams, choose Organisation assessments. Every team can run what is published here.
A team's assessments — open the team, and in The wall section choose assessments →. These can only be run in that team's conversations.

Each page lists its sets in three groups: waiting for a person, drafts and published. Only a published set can be put in front of anyone.

You need to be a manager to see these pages.

Adding a set¶

There are three ways to add a set, all under add a set at the top of the page. Whichever you use, you name the set and choose what is at stake. For a compliance set you also give a pass mark.

Upload one you already have¶

Choose Upload one you have and drop in the file.

A spreadsheet (.xlsx or .csv) is read exactly as written, so nothing needs reviewing. Use one row per question and a header row. The columns are:

Column	What goes in it
`type`	`choice`, `short` or `narrative`
`prompt`	The question
`option_a`, `option_b`, …	The options, for a choice question
`answer`	The right answer (see below)
`must_mention`	For a narrative question, the points a good answer makes
`must_not_say`	For a narrative question, anything a good answer must not say (optional)
`pass_points`	For a narrative question, how many of the points are needed (optional — all of them if left empty)
`explanation`	Why that is the answer. Shown after someone has answered (optional)

Where a cell holds more than one thing, separate them with a vertical bar (|):

Choice: answer is the letter of the right option — b — or letters such as a|c if more than one is right.
Short answer: answer lists every accepted answer, such as the fire marshal|the fire warden.
Narrative: must_mention and must_not_say list one point each between the bars.

A column AIR doesn't recognise is refused rather than ignored, so a misspelled heading tells you straight away instead of quietly dropping an answer.

A document (Word, PDF or plain text) is read for you by a colleague who has been taught to write sets — you choose which one. Every question it reads in is flagged for you to check before the set can be published. The colleague copies the answers that are in the document; it never supplies one of its own. If the document asks a question and doesn't give the answer, that question is left out and listed for you, so you can add the answer and upload again.

The file itself is not kept — only the questions read from it.

Ask a colleague to write one¶

Choose Ask a colleague to write one. Pick up to five items from the wall for it to write from, and say how many questions you want. You can also say what people should be able to do afterwards, and who the set is for.

The colleague writes questions whose answers are in the material you chose. A question it can't show in the material is dropped, not guessed. The set appears under waiting for a person a minute or two later, with every question flagged for you to check.

Tick Seal the answers from everyone if you want to quiz yourself on your own notes. Nobody — you included — can then read the answers; reviewers check that each question is clear and fair instead.

Only some colleagues can write sets. See who can write assessments.

Write one yourself¶

Choose Write one yourself and add questions one at a time:

Choice — put each option on its own line, with * in front of each right one.
Short answer — list the accepted answers, one per line. People can give the answer inside a sentence; it still counts.
Narrative — list the points the answer must mention, anything it must not say, and how many points are needed to pass.

For any question you can add Why that is the answer, which is shown after someone has answered.

Choose Save as a draft when you're done. Nothing you type is flagged — you wrote the answers yourself — so you can open the set and publish it straight away.

Adding a picture¶

Choose + A picture on a question to attach an image (under 10 MB). You must then describe the picture. That description is part of the question: it is what someone who can't see the picture is told, and it is always asked in words, so the question still makes sense wherever it is read.

A picture is checked before anyone is shown it. Until then the question is asked without it — see a picture on a question isn't showing.

Reviewing and publishing¶

Open a set to see it question by question. A flagged question says why:

A colleague wrote or read this answer. Check it, then choose Approve.
It isn't suitable for the people the set is for. This can't be approved — choose Retire to take it out.

When nothing is left to look at, choose Publish. Every question is checked against the audience you gave the set on the way. If any aren't right for them, the set stays a draft and those questions come back flagged with the reason.

Retire takes a question out of every future run, including after a set is published. The last question in a set can't be retired.

If a set won't publish, see a question set won't publish.

Running a quiz¶

A colleague can only run quizzes once it has been taught to. Open the colleague, go to its Work tab and choose Teach (its name). Choose Choose from here, find Run a quiz in the catalogue and choose bring in. To teach a whole team at once, use Train the team on the team's jobs page instead.

After that, anyone talking to the colleague can ask for one — "quiz me", "test my knowledge", or the name of the set. The colleague offers the published sets that conversation is allowed: the organisation's, and the team's own. If there is more than one, it asks which.

It then asks each question exactly as written, marks the answer, tells the person whether they were right and why, and moves on. They can say stop at any time, and what they have answered so far is kept. An answer the colleague can't mark with confidence is passed to a person rather than guessed at.

If someone isn't being offered a quiz you expect, see a quiz isn't offered to someone.

Giving a set to people¶

You can also ask particular people to take a set. Open a published set and choose Give it to people.

Tick who should take it.
Choose which colleague runs it. Only colleagues taught Run a quiz are listed.
Optionally, set a due by date.
Choose Give it to them.

Each person gets the first question straight away, in their own conversation with that colleague. Giving someone a set they already have open doesn't create a second one — it counts as a nudge.

Due dates, the reminder, and lapsing¶

With a due date, each person is reminded once, in their own conversation, about a day before it's due (sooner, for a short deadline). The reminder only goes out during their hours, and because the quiz is still waiting for them, their next message is simply their next answer.
Past the due date, anything unfinished is marked lapsed.
With no due date, it never lapses. It stays outstanding until they finish.

Withdrawing¶

Sent a set to the wrong person? In the set's results, choose Withdraw next to them. You can only withdraw an ask that hasn't been finished. Anything they have already answered is kept, and you can give them the set again later.

Giving a set to a colleague¶

The same dialog has Colleagues who should sit it. A colleague takes the set with everything it normally has, and is marked as a person would be. You also choose which specialist examines them — a colleague taught to write that kind of set.

A colleague you give a set to appears in the results beside the people, can be withdrawn, and lapses if it never takes it. It isn't sent a reminder.

Results¶

Once a set is published and given to someone, opening it shows:

how many it was given to, how many have finished, how many are outstanding and how many lapsed
for a compliance set, how many passed
each person, with their score and due date
the questions most often got wrong

Above the list, Airica gives her read of it: how it is going for the people, and whether the questions people keep getting wrong suggest the material behind them needs a look.

For a weekly summary across every set, switch on a colleague's Assessment progress report, one of the standard reports every colleague has. It names nobody — names are only on the Assessments page.

Answers waiting for you to mark¶

Some answers can't be marked automatically — a longer written answer that doesn't clearly cover the points, for instance. They are never counted as right on a guess; they wait for a person, and until they are marked that person's result isn't final.

Open the set and they're listed at the top under Waiting for you to mark, each with the question, what the person wrote, and what a good answer covers. Choose Right, Partly or Wrong. Once the last one for a person is marked, their result is settled, and what they wrote is no longer kept.

For a game, nobody's words are kept at all, so there is nothing to read — you mark it on the question alone. A set whose answers are sealed from everyone shows no answer to mark against, for the same reason.

Examining a colleague¶

You can check whether a colleague has actually learned its material by sitting it through a set. Open the set — a draft works too — and choose Examine a colleague.

Choose the colleague, then how it sits the set:

With everything it normally has
Without its wall, memory or retrieval
Both — and test the material too

It answers every question the way it would answer anyone, and each answer is marked as a person's would be. Results appear under Examinations on the set, with Airica's read of them. The colleague pays for its own answers.

Examining with and without its material tells you something about the set as well as the colleague. A question it gets right without the material is testing general knowledge, not your material. A question it gets wrong with the material points at the material — something missing, unclear or contradicted. Each is flagged on the question for you to decide on; nothing is retired for you.

When the material changes¶

If a colleague wrote a set from items on the wall, and one of those items is placed on the wall, locked, unlocked or removed, AIR re-checks any colleague already examined on that set with its material. Moving an item between the public and internal sides doesn't count — the colleague can read both.

The re-check shows in the set's examinations — for example, re-checked after "Fire procedure" was locked (was 9/10). If the score drops, whoever ran the original examination is told, with the change that caused it.

Offering a set as training¶

A published set can become training a team takes on as a job. Open the set and choose Offer as training.

It then appears in your organisation's training catalogue under the set's name. A team brings it in from Train the team → Choose from here, like any other training, and it lands on the team's jobs wall in review. From there you place it — offer it at the team's front door, or keep it internal — as you would any job. A colleague can take it on too.

A team's own set can only be taken on by that team.
Changes to the questions — approving or retiring one — reach everyone who has taken it on straight away.
Renaming the set makes a new version. Teams are offered the update; nobody is moved to it automatically.
Withdraw from catalogue takes it out of the catalogue. Teams that already have it keep it.

Limiting quizzes a day¶

When a quiz is offered at a team's front door, its job on the wall has a Quizzes a day control: how many quizzes each person may start for themselves in a day, from Unlimited up to 100. When someone reaches it, the colleague tells them kindly they've done today's quizzes and can do another tomorrow.

A set someone was given never counts towards the limit — that was your ask, not theirs.

Who can write assessments¶

Writing a set is training too, and there are two kinds:

Training	What it lets a colleague write	Who can be taught it
Write quizzes and games	Practice and play sets	Any colleague
Write assessments	Compliance sets too	Learning or HR colleagues only

Teach it from the colleague's Work tab, the same way as Run a quiz. A colleague taught to write a kind of set can also examine other colleagues with it.

Airica doesn't write sets herself — she trains the colleague who does. If nobody can, AIR tells you to teach a colleague or recruit one. See nobody can write an assessment.

Jobs — placing a job at the front door or keeping it internal.
Teams and front doors — what a team's wall and front door are.
Partner API — launching a quiz from your own application.