Spatial Kernel · a game

Real or printed?

A robot was told to put a mug on the table. A vision model looked at the photo and judged whether it worked. Sometimes the "mug" is a sheet of paper. Gemini Robotics-ER passed 10 of 18 printed decoys as real objects. You get the same single photo it got. Then flip to a second photo taken 10 cm to the side.

1 · One photoIs there a real object on the table? Same question, same picture the model saw.

2 · The modelSee what Gemini Robotics-ER answered, in its own words.

3 · Second viewBoth photos are lined up on the two black dots. The table holds still; a real object jumps, a print doesn't.

What's going on. One photo doesn't contain height. A good print of a mug and a mug can produce nearly the same pixels, so any judge that looks at one frame has to guess. A second photo from a slightly different spot does contain height: things above the table shift more than the table does. The Spatial Kernel measures that shift with plain computer vision: no training, same answer every time.

The fine print. These are 27 scenes the model was asked about: one model version, one neutral prompt, one run. On the full set of 113 photo pairs the geometry check is right on 99 of 109 it could track. It struggles with clear glass and with very low objects, and a print taped onto a box fools it (the print really is raised). All of that is in the known issues. Photos by the author, CC BY 4.0.