Agentic Exploration in Games

Real-Time 3D Occupancy Grids for Game AI

Introduction

This project started with a simple curiosity: what if the same ideas used to help self-driving cars understand roads in real time could also help AI agents navigate complex 3D worlds in video games? Our team has been fascinated by voxel-based games like Minecraft, where environments can be enormous and ever-changing. We built a pipeline that blends real-time 3D occupancy grids (inspired by Tesla FSD concepts) with in-game data to teach AI agents to move, explore, and react more like actual players. We wanted our bots to dynamically update and change as the world shifts around them.

What We Did

We used Minecraft as our main testbed. First, we collected six-degrees-of-freedom screenshots along with block-level ground truth data (basically telling us exactly which blocks are where). This helped us build highly detailed occupancy grids—think of them like fancy 3D maps that show exactly what space is filled or empty in the game. We wrote scripts (using Minescript and Anvil) to extract, format, and label these blocks automatically. We then fed that into machine learning models that let in-game agents “learn” the environment on the fly.

Observations

We found that the occupancy-grid approach helped AI agents maintain a more consistent view of the world, particularly as chunks (sections of the game) loaded or unloaded. Agents navigated more confidently across diverse terrain and could adapt to sudden changes—like a new building appearing where there was once open space. This method also reduced weird or impossible movements, since the grids alerted the agent whenever an area was filled.

Looking Ahead

One of our favorite parts of this research is envisioning its applications in other games—or even entirely different AI domains. In environments that dynamically evolve or where new objects appear in real time, 3D occupancy grids can ensure the AI stays in sync with ever-changing surroundings. The result isn’t just better navigation but also more coherent, immersive experiences for players.

Acknowledgements

Many thanks to Cornell Tech, Bowers CIS, and open-source communities for Minescript, Anvil, OccRWKV, and Diamond Diffusion. Your support and feedback have been instrumental in pushing our AI-driven gameplay to the next level.