Moodpack / Director

Moodpack / Director is an experimental environment for connecting AI agents to Unreal Engine — not to generate pictures of 3D scenes, but to operate the editor that builds them.

Language models are fluent about 3D work and largely unable to do it. They can describe a scene, name the right tools and explain a workflow, then produce nothing that exists inside the software.

The gap is not knowledge. It is state: an editor is a live environment with a selection, a hierarchy, a coordinate system and consequences, and none of that is present in a text prompt.

Give the agent real handles on the editor and a way to read back what happened. Actions become tools, the scene becomes observable state, and the loop closes.

The system is built as a control layer between an agent and the editor. Commands are explicit and inspectable rather than free-form, so a failed action is a readable failure instead of silence.

Much of the work turns out to be spatial rather than linguistic — placement, collision, orientation, whether an object ended up where it was asked to go.

Each experiment stays small and repeatable: request one operation, observe the scene, compare against intent.

Unreal Engine as the environment, with an agent-facing command surface on one side and scene inspection on the other. Object and sound generation exist as separate generators feeding the same environment.

The recurring test is spatial reasoning. An agent can produce a perfectly plausible sentence about where a chair should go and still place it inside a wall.

Collision and placement work is therefore the honest measure of progress — more so than any transcript of the agent explaining itself.

In development, and an experiment rather than a product. It is not an autonomous system and is not presented as one.

The bottleneck is not the model’s vocabulary but its grip on state. Progress came from making the environment more legible to the agent, not from asking the agent more elaborately.