UnrealZoo: Enriching Photo-realistic Virtual Worlds for Embodied AI

Fangwei Zhong*♯† , Kui Wu*§, Churan Wang‡, Hao Chen≬, Hai Ci◊, Zhoujun Li§, Yizhou Wang‡
♯Beijing Normal University, §Beihang University, ‡Peking University,
†BIGAI, ≬City University of Macau, ◊National University of Singapore

* Core Contributors

arXiv Video Code UE Environments Documentation

Towards a Harmonious Human-AI Society

Overview Image

Abstract

We introduce UnrealZoo, a rich collection of photo-realistic 3D virtual worlds built on Unreal Engine, designed to reflect the complexity and variability of the open worlds. Additionally, we offer a variety of playable entities for embodied AI。

  • Photo-realistic Rendering: Empowered by Unreal Engine(4.27/5.4).
  • Large-scale Maps: 100+ Maps, the largest one covers 16km².
  • Diverse Scenes: Landscape🏜️🏞️🏝️, historic architecture⛩️🛕🏰, settlement🏘️🏙️, industrial zone🏭🏗️, facilities🤽🏻‍♂️🚉🏪...
  • Diverse Bodies: Human🚶🏻‍♂️️🚶🏻, Robot🤖, Vehicles🚗🏍️🛩️, Animals🐘🐕🐎🐧🐢🐖🐏...
  • Diverse Actions: Running🏃🏻‍♂️, climbing🧗🏻, sitting🧘🏻, jumping, squatting, picking up...
  • Easy-to-use: Pre-build binaries with UnrealCV are to be used without any prior knowledge of UE.
  • Flexible Python APIs: Provide Python APIs and Gym Interfaces for various potential applications.
  • Multi-agent Interactions: Real-time interaction of 10+ vision-based agents👩‍👩‍👧‍👧 in one scene.

Video

Enriching Scenes

The scenes are sourced from the Unreal Engine Marketplace Please refer to the Scene Gallery for more details.

Enriching Bodies

Please refer to the document of playable entities for more details.

Enriching Interactions

Gym Interface for Agents

Embodied Visual Navigation

Embodied Visual Tracking

Heterogeneous Multi-agent Cooperation

BibTeX

@article{zhong2024unrealzoo,
  author    = {Zhong, Fangwei and Wu, Kui and Wang, Churan and Chen, Hao and Ci, Hai and Li, Zhoujun and Wang, Yizhou},
  title     = {UnrealZoo: Enriching Photo-realistic Virtual Worlds for Embodied AI},
  journal   = {arXiv preprint arXiv:2412.20977},
  year      = {2024}
}

Contact

If you have any suggestion or interested in using or contributing to UnrealZoo, please contact fangweizhong@bnu.edu.cn

Acknowledgments

We would like to thank the Unreal Engine community for providing the engine and the Unreal Engine Marketplace and its contributors for the sharing assets. We also thank Weichao Qiu and other active developers for providing valuable supports in the improvements of UnrealCV plugin.