Three Students Click Through (the test we haven't run) - Figma AI Test

SEO Keyword Tags

usability test script, AI predicted misreads, Claude usability prediction, UX testing consent script, model prediction vs real users, first-year student UX test, design misread prediction, Claude Code test script, usability moderator checklist, AI UX research limits, Figma education experiment, predicted vs actual usability, model agreement not evidence, AI usability testing prep, UX research methodology

Transcript

A lightly formatted transcript of the video, with timestamps so you can jump to any section on YouTube.

[0:00] Hello, this is Liam in for Bear. Three students clicked through the tutor's cards. Not yet, nobody has. This film is the part an agent can do before they arrive and the part it can't. Three terms: a usability test, one person tries the design while a moderator watches, writes, and gives no hints. A misread: a reading of a card that differs from what its designer meant. And a model prediction: a model's guess at what will happen, written down before it happens so it can be checked. A guess is not a result. In film 12, an agent designed four tutor cards in Figma. Confident with a source, partly

[0:32] Sure, I don't know with the next step, and needs your approval. The plan was to put them in front of three first-year students. The students haven't come yet, and tonight no one pretends to be one. So, tonight an agent wrote the test itself. 10 minutes per student, one at a time, one moderator, and no hints. The moderator reads one line at a time. What is the tutor telling you? Where did the answer come from? What happens if you press the button on the left? The script is in the record. Before the clock starts, a consent line read aloud in the same words every time. It's a test of the design, not of you.

[1:02] Your words may be quoted without your name. Nothing here affects a grade. You can stop, and you can strike anything you said. The moderator writes down yes or no. Then the checklist for each card, the four or five things the designer meant as yes or no lines. Did they read I don't know as honest or as broken? Did they say what the tutor wants to do before touching anything? A reading the designer didn't intend is a misread, and the student's own words go beside it. That's the half an agent can do. For preview of the other half, three separate runs of Claude in separate sessions with different prompts in two

[1:33] Different models. Each saw the four cards and was asked what a first-year would misread on each. Every answer is labeled a prediction. 60 predictions, 20 a run for about 48 cents at list price. Tallying them, six misreads were predicted by all three runs. Next question taken as the tutor about to quiz you. Open section 4.2 with no idea what will open. Partly sure taken to mean half right. The half-filled circle taken for a loading spinner. I don't know taken is broken and ask my instructor taken to mean it sends it once. 10 more came from two runs like the green check taken is graded correct and

[2:07] The draft taken for an email already sent. 22 came from one run only and the approval card, the one where a real email goes out, got no three-way agreement at all. Agreement between models isn't evidence. It's a short list, the six things to watch for first. The thing none of them predicted maybe the first thing a student says. So the verdict on what students misread is pending. Not weak, not strong, pending because the deciding half needs three students 10 minutes each with their consent. When they've clicked through the 60 predictions get scored, seen not seen or can't tell. The students decide. Your turn, paste this

[2:42] Into Claude. Here are four screens from my design then paste them. Before I test them with real people, predict the three most likely misreads on each screen. Name the exact word or element that causes each one and mark every item as a prediction. Then write me a 10-minute test script with a consent line and a yes or no checklist I can use to check your predictions. Then check two things yourself. Does every prediction name a word or element that is actually on the screen? And after your first real participant, do their words sit beside a line on the checklist? Three students click through the test we

[3:13] Haven't run at Nick Bear Brown.