# Freeform Bodies

Venue: IEEE ICRA 2024

Authors: Muhan Li, David Matthews, Sam Kriegman

A design policy builds 3D robots with arbitrary internal and external structure directly from low-level material building blocks.

- [Paper](https://arxiv.org/abs/2310.05670): Reinforcement Learning for Freeform Robot Design
- [Code](https://github.com/phenoweave/RL4design)
- [Interactive page](https://www.beyond-infinity.top/research/rl4design/)

## Steps

### 1. AN EMPTY SPACE

Here a reinforcement learning agent does not learn to control a robot; it learns to build one, starting from empty space.

- GRID: 20 × 20 × 8
- STEP: 00 / 40

![An empty wireframe box with ticks along its edges, the space the agent builds in.](https://www.beyond-infinity.top/agent/shots/rl4design_step1.jpg)

### 2. ONE CHOICE, ONE BLOCK

Each choice places one block of passive tissue or muscle; the hills show where the agent is most likely to build.

- STEP: 28 / 40
- CHOICE: MUSCLE B

![Three hills of stacked rings rise from three faces of the box, and dashed lines from their peaks meet at one bright new block beside the dim body built so far.](https://www.beyond-infinity.top/agent/shots/rl4design_step2.jpg)

### 3. A BODY TAKES SHAPE

Forty choices later a body has grown; loose pieces are dropped, and only the connected part becomes the robot.

- STEP: 40 / 40
- CHOICE: MUSCLE B
- BLOCKS: 112
- BODY: 102

![Blocks appear one after another inside the box until a body has grown, and the loose pieces fade away, leaving one connected body.](https://www.beyond-infinity.top/agent/shots/rl4design_step3.jpg)

### 4. DISTANCE IS THE REWARD

These little creatures are simulated and ranked by the distance they travel; the agent learns from the ranking, and better bodies emerge.

- DISTANCE: 8.7 VOXELS
- REWARD: 13.1 VOXELS
- PLAYBACK: ×0.5

![The body crawls across a floor of hexagons while a ruler on the ground marks how far it has moved.](https://www.beyond-infinity.top/agent/shots/rl4design_step4.jpg)