Collaborative Construction and Communication in Minecraft

The goal of this project is to develop systems that can collaborate and communicate with each other to solve tasks in a 3D environment.

The particular task we consider is a two-player game in which one player (the Architect) is shown a 3D target structure constructed out of building blocks, and has to instruct the other player (the Builder) to construct the same structure. The Builder needs to be able to execute these instructions, and both players communicate via chat.

We have released the Minecraft Dialogue corpus, a corpus of human-human dialogues and game logs:

We have also created models for the Architect and Builder roles.

Our work is intended as a stepping stone towards human-robot collaboration and communication, but for ease of implementation and experimentation, we use the popular Minecraft gaming platform.

Publications

For more details, visit the pages for our papers.

• Anjali Narayan-Chen*, Prashant Jayannavar*, Julia Hockenmaier, Collaborative Dialogue in Minecraft, ACL 2019
• Prashant Jayannavar, Anjali Narayan-Chen, Julia Hockenmaier, Learning to execute instructions in a Minecraft dialogue, ACL 2020

*equal contribution

Acknowledgements

This work was supported by Contract W911NF-15-1-0461 with the US Defense Advanced Research Projects Agency (DARPA) Communicating with Computers Program and the Army Research Office (ARO). Approved for Public Release, Distribution Unlimited. The views expressed are those of the authors and do not reflect the official policy or position of the Department of Defense or the U.S. Government.