FlightGoggles Overview

FlightGoggles is envisioned to be a development environment for design, implementation, testing and validation of autonomous super-vehicles. FlightGoggles currently provides exteroceptive sensor simulation based on the Unity3D engine, as well as vehicle dynamics and inertial sensor simulation capabilities. Photorealistic camera simulation is achieved using high-quality assets that were generated using photogrammetry.

Python and ROS Interfaces: FlightGoggles provides flexible Robot Operating System (ROS) and Python interfaces, specifically aimed at research in robotics and machine learning. Users are able to generate simulation scenarios including multiple vehicles and custom sensor configurations.

Virtual-Reality Development Environment: Additionally, FlightGoggles enables virtual reality (VR) robotics development where real-world dynamics are combined with simulated camera imagery. This is achieved by flying a vehicle under motion capture and simulating exteroceptive sensors, e.g., cameras, in real time based on the motion capture pose. What results is a physics-in-the-loop simulation that has the real unsteady aerodynamics, vehicle vibrations, inertial measurements etc., but also provides the safety, flexibility, and repeatability of simulation.

Publications & Videos: Visit our research page for more technical videos and our research papers. If you find this work useful and use it for your research, please cite the following paper. A pre-print is available on arXiv.

CODE

@inproceedings{guerra2019flightgoggles,
  doi = {10.1109/iros40897.2019.8968116},
  url = {https://doi.org/10.1109/iros40897.2019.8968116},
  year = {2019},
  month = nov,
  publisher = {{IEEE}},
  author = {Guerra, Winter and Tal, Ezra and Murali, Varun and Ryou, Gilhyun and Karaman, Sertac},
  title = {{FlightGoggles}: Photorealistic Sensor Simulation for Perception-driven Robotics using Photogrammetry and Virtual Reality},
  booktitle = {2019 {IEEE}/{RSJ} International Conference on Intelligent Robots and Systems ({IROS})}
}