Muse Can See

Muse Can See

Muse Spark reads images and answers in text: this film walks through transcribing a screenshot, building a web page from a reference image, and what to trust and not trust when a model looks at a picture.

You come away knowing how to hand a model an image, how to judge a transcription and a generated page like a practitioner, how to iterate instead of single-shotting, what the 0 to 1000 grid for locating things is, and where the limits and cautions are, including numbers read off charts.

Honest about how it was made: this film was drafted in Muse and then cleaned up and rebuilt with Claude. The film follows Meta's pages over the course where they differ: the news site is NHK's NEWS WEB EASY, an unsupported "early-2000s" claim was cut (PHP-Nuke's first release was August 1998), and the course's "100% accurate" is said as its own check of one screenshot. One walkthrough detail could not be re-verified against a full transcript and is voiced as the course's account.

A note that opens the film Muse drafted this, Claude cleaned it up. This film began as a rough draft made with Muse. Muse runs on a simple model called Spark, and Spark makes a lot of mistakes, so Claude cleaned the draft up. The method is the point: draft in Muse, then finish in Claude or Codex. One kind of mistake we looked for: a draft Muse diagram of two big boxes and a check mark under a question about whether a non-technical person could use it, with both boxes empty. A diagram that shows nothing teaches nothing, so Claude replaced diagrams like this one.

Narrated by Liam, in for Bear. Narration is an AI voice; the film was made with AI tools.

Chapters 0:00 Muse note 0:28 The idea and the key terms 0:55 Read It 1:56 Build It 3:07 Work It 3:43 Trust It 4:45 Recap 5:04 Your Turn

Your turn Attach a screenshot to Claude, or to any model that can see, then paste this: Here is a screenshot of [say what it is]. Transcribe all the text in it exactly, line by line. Then list every number you read and say how you read it. If a number is not readable, say so instead of guessing. Then check two things yourself. Is every line of the transcript really on the image? And did it mark any number it could not read, instead of guessing?

Sources shown on screen Meta's public docs and cookbook pages at dev.meta.ai (image understanding, perception grounding, browser-verified web design, chart analysis, models), the PHP-Nuke article on Wikipedia, and MDN's flexbox guide, captured 2026-10-07. Narration follows a Muse course taught by Andrew Brown, named on screen. Nothing here is a product endorsement.

Part of the playlist Muse for Educational AI.

@NikBearBrown

What this video covers

• 0:00 — Muse note
• 0:28 — The idea and the key terms
• 0:55 — Read It
• 1:56 — Build It
• 3:07 — Work It
• 3:43 — Trust It
• 4:45 — Recap
• 5:04 — Your Turn

Transcript

A lightly formatted transcript of the video, with timestamps so you can jump to any section on YouTube.

Muse note

[0:00] A note before we start, this film began as a rough draft made with Muse. Muse runs on a simple model called Spark and Spark makes a lot of mistakes, so Claude cleaned this draft up. The method is the point, draft in Muse then finish in Claude or Codex. Here's the kind of mistake we look for. A draft diagram from Muse, two big boxes and a check mark under a question about whether a non-technical person could use it. Both boxes are empty. A diagram that shows nothing teaches nothing, so Claude replaced diagrams like this one.

The idea and the key terms

[0:28] Hello, this is Liam and forbear. You might think a chat model only reads text. Muse Spark also looks at pictures. This lecture tests that, reading a screenshot, building a webpage from a reference image, and how far to trust what it sees. Four terms, vision, the model reading an image you give it, transcription, turning an image of text into typed text, a reference image, a picture you hand over and say build from this, and a system prompt, the standing orders the model gets before it starts.

Read It

[0:55] Meta's own docs put it plainly, send images alongside a text prompt and Muse Spark reads the visuals and returns text. It can describe a scene, answer questions about it, pull out text and data, and say where things are. Handing it a picture is simple. Give it a web link, the image bytes inline, or a file you uploaded earlier. In the request, the picture is one more block inside your message sitting beside your words. Now a test from Andrew Brown who teaches a Muse course. NHK publishes a news site, News Web Easy, written in simple Japanese for foreign residents and children. He screenshots an article and gives Muse Spark one prompt, transcribe. He checks the result against the original. The course reports 100% accurate across all three Japanese scripts, kanji, the characters borrowed from Chinese, hiragana, the rounded phonetic script, and katakana, the angular one. One footnote, pasted into an editor, the text looked garbled. The walk-through traced that to the editor, not the model. So when output looks wrong, check which step broke before you blame the model.

Build It

[1:56] For the second test, he needs something to copy. PHP-Nuke is an old web publishing system first released in 1998. He uses a screenshot of an old PHP-Nuke template as his reference image. The template has three columns with boxes of plugins down the left and right. He gives Musepark the picture as a reference image and asks for a concept for a tech community site in that spirit. But first, the standing orders. His system prompt says, "You are a front-end web developer. One single HTML file. No CSS framework. Use flexbox. Keep the markup lean. Make it mobile-friendly." Flexbox is the layout method behind that rule. Boxes in a row share the width and can shrink to fit it. And with wrapping on, they drop onto new lines. That is how three columns can fit a phone. And it builds one file, three columns, a retro look. In the courses run, the side rails held modules such a system load, who's online, and a shoutbox, while the center column held the feed and a top 10 list. Then he judges it like a practitioner. The look is right, though he wants more spacing. The plugins are not. They lean toward site performance, and he wanted community engagement. So, he asks for one more pass with that specific feedback.

Work It

[3:07] Notice what he did not do, single-shot it and walk away. In Andrew's experience, a single shot shows the model's floor, and specific feedback is how you reach its ceiling. The draft was the start. The judging was the work. He also asked for mobile-friendly up front, and it paid off. In the walk-through on a narrow screen, the three columns collapse into one. A requirement stated before the build beats a fix begged for after. Meta's own cookbook adds a twist. Let the model do the looking. Give a coding agent a browser tool, and it screenshots the page it wrote and reads the pixels. It caught mis-spaced characters and fixed the CSS by itself.

Trust It

[3:43] Musepark can also say where things are. It answers on a grid from 0 to 1,000, whatever the image size. A box is four numbers, left, top, right, bottom. The same numbers work at any resolution. Meta's cookbook puts that to work on a fridge photo. One request returns a page with a colored dot on every food item scored for the diet in the prompt. The model knew where each item sat. Know the limits. In line, each image can be 50 megabytes uploaded through the files API up to a gigabyte. A request takes up to 50 images in JPEG, PNG, GIF, or WebP. Meta's own chart recipe adds a caution. The model reliably gets the chart's trend, but numbers vary between identical runs, and it sometimes misreads values printed on the chart. Meta's advice, treat every extracted number as a claim to check. Everything here came from Meta's own pages and one Wikipedia article captured on October 7th, 2026. Pages change, so check them before you build. The two tests are Andrew Brown's own, and they were not rerun for this lecture.

Recap

[4:46] Let's recap with Claude. Muse Spark reads a picture and answers in text, and the course reports 100% on a Japanese news screenshot. A reference image and a clear system prompt build a whole page in one file. Judge the draft, give specific feedback, and state your requirements up front, and it can point at things on a grid, but you check every number it reads. Your turn. Attach a

Your Turn

[5:05] Screenshot to Claude or to any model that can see, then paste this. Here is a screenshot of say what it is. Transcribe all the text in it exactly line by line, then list every number you read and say how you read it. If a number is not readable, say so instead of guessing. Then check two things yourself. Is every line of the transcript really on the image, and did it mark any number it could not read instead of guessing? Muse can see @nickbearbrown.