A pelican-on-a-bicycle SVG is a meaningful benchmark of overall AI reasoning

What's this about?

People disagree about whether making an SVG of a pelican riding a bike can test an AI’s full thinking skill.

This task checks some real skills, but it cannot test every kind of thinking.

What supporters say

• The AI must understand that the pelican rides the bike, not just stand near it.
• It must join words, shapes, places, and actions into one clear scene.
• SVG asks the AI to write picture code that makes a real image.
• We can check both the code and the final picture for mistakes.

What critics say

• One silly picture task cannot show how well an AI thinks about all kinds of problems.
• Full thinking includes many skills, such as math, plans, facts, and hard choices.
• An AI might make one good picture from luck or from patterns it saw before.
• A good score needs many tasks, not just one animal riding one bike.

The bottom line

A pelican-on-a-bike SVG can test a small set of useful skills.

It can help in a larger test set, but it cannot measure overall AI reasoning by itself.

A request to create an SVG image of a pelican riding a bicycle can reveal some useful things about an AI system. But the evidence suggests it is not enough on its own to measure the much broader ability known as overall reasoning.

The case for

The prompt does test a real mix of skills. To succeed, an AI must understand that there should be both a pelican and a bicycle, then show the bird in the specific relationship of riding it. That requires combining objects, attributes and spatial or action-based relationships rather than simply naming familiar things. Research on text-to-image evaluation treats these as separate abilities that can be checked (see Figure 2). 1

The SVG requirement makes the task more demanding than asking for a written description or a loose image. SVG is structured visual code: the system must turn a language instruction into a file that can be rendered as an image. A good result can be inspected for valid syntax, object structure, placement, layering and whether the final scene actually looks as requested. 2

That means a successful pelican-on-a-bicycle SVG is evidence of selected cross-modal skills. It combines language understanding, visual composition and code generation. If the question is narrowly framed—can a system bind objects and relationships in a structured vector graphic?—the prompt can be a meaningful test.

Its value would grow further if it were part of a carefully designed set of tasks. Novel animal-and-vehicle pairings, paraphrased instructions, clear spatial requirements, repeated attempts and requests to edit an existing SVG could all help distinguish genuine capability from a lucky one-off response. Scoring should examine both the rendered image and the underlying code.

The case against

The main problem is that one whimsical illustration cannot stand in for overall reasoning. Broad reasoning covers many different abilities, contexts and kinds of problems. Major benchmarks and software-engineering evaluations usually test models across multiple task types, often using functional tests and richer real-world contexts, precisely because no single task can capture a general capability. 3

A controlled visual reasoning task can test rule learning and generalization more directly than an open-ended drawing. In such tasks, systems must infer new rules from limited examples, rather than produce one of many plausible illustrations (see Figure 3). A pelican riding a bicycle may be creative and technically competent, but it does not by itself show that a system can reason reliably in unrelated domains.

Scoring is another weakness. There are many visually plausible ways to draw the scene, and a file that parses correctly is not necessarily one that fulfills the prompt. An image may look appealing while failing the requested riding relationship; conversely, structurally sound code may render an awkward image. Research on SVG evaluation indicates that visual quality, code structure and semantic accuracy need explicit criteria, not an informal pass-or-fail judgment. 4

The prompt could also reward familiarity rather than reasoning. A model may have encountered similar prompts, common SVG templates or standard ways of assembling cartoon animals and bicycles. Fixed, familiar tasks risk measuring memorized patterns or implementation habits. Changing tasks over time is one way benchmarks try to reduce that problem. 5

Most importantly, there is no direct evidence that high scores on a rigorously scored pelican-on-a-bicycle task predict strong performance on a wide range of independent reasoning tests. That missing connection—known as external validation—is the central limit on any broader claim.

The bottom line

A pelican-on-a-bicycle SVG is a meaningful probe of specific abilities, including compositional visual understanding, object-and-relation binding, and structured SVG generation. Success shows more than fluent text production because the output must become inspectable, renderable visual code.

But it is not, by itself, a meaningful benchmark of overall AI reasoning. The evidence strongly supports a narrow conclusion and rejects the broad one: this task can measure selected multimodal and coding skills, but cannot establish general reasoning without validation against varied, independent reasoning measures.