Amazon Go’s computer vision needed labelled images of shoppers reaching into shelves, more and stranger than real stores could supply. The synthetic data framework, built in Blender through its Python API, rebuilt a virtual Go store (floor, shelves, fridges, the store’s real camera configurations and lighting) and filled it with virtual customers: textured 3D body scans in SMPL topology, rigged and animated by Body Labs’ Golem package, with skeletal motion retargeted from real in-store events.
Crowd “drivers” composed each scene: replaying a real event, scattering a random crowd without collisions, augmenting a real event with extra shoppers for occlusion, or composing hard cases on purpose (crossed arms, vertical picks, a child in a parent’s arms). Hand poses were randomized so networks wouldn’t overfit to SMPL’s flat hands.
Every frame came with its ground truth, rendered from every camera: 2D and 3D body joints (with visibility and inside-shelf flags for the palms), instance and body-part segmentation, head-tracking masks, depth, and the event-to-customer association vectors.
Golem generated the rendered data itself: bodies placed in scenes across combinations of shape, pose, background and lighting, each image written out with its render passes (beauty, depth, diffuse albedo, normals, segmentation).
Motion came in through Moshcap, packaged as a Docker image: pointed at CMU motion-capture files, it fit them to the body model and wrote animations Golem could load.
