ThreeDWorld (TDW) is a platform for interactive multi-modal physical simulation. With TDW, users can simulate high-fidelity sensory data and physical interactions between mobile agents and objects in a wide variety of rich 3D environments.
Getting Started (Read this first!)
| Document | Description |
|---|---|
| Command API | API for every command a controller can send to the build. |
| Command API Guide | Overview of how to send commands to the build. |
| Output Data | API for all output data a controller can receive from the build. |
| API | Description |
|---|---|
| Image dataset | Generate 1.3M photorealistic images. |
| tdw_physics | Generate physics datasets. |
| Magnebot | High-level Magnebot robotics API. |
| Transport Challenge | A higher-level API that uses the Magnebot API. |
| Document | Description |
|---|---|
tdw module |
Overview of the Python tdw module. |
| Controller | Base class for all controllers. |
| TDWUtils | Utility class. |
| AssetBundleCreator | Covert 3D models into TDW-compatible asset bundles. |
| PyImpact | Generate impact sounds at runtime. |
| DebugController | Child class of Controller that has useful debug features. |
| KeyboardController | Child class of Controller that can listen for keyboard input. |
| FloorplanController | Child class of Controller that creates an interior environment and populates it with objects. |
| Librarian | "Librarians" hold asset bundle metadata records. |
| FluidTypes | Access different NVIDIA Flex fluid types. |
| Object Init Data | Wrapper classes for storing object initialization data. |
| Document | Description |
|---|---|
| Build | Helper functions for downloading the build. |
| PyPi | Helper functions for checking the version of the tdw module on PyPi. |
| Document | Description |
|---|---|
| Impact Sounds | Generate impact sounds at runtime using PyImpact. |
| PyImpact | PyImpact API. |
| Audio/Video Recording | Record audio, video, or audio+video. |
| Remote rendering | How to render using xpra. |
| Document | Description |
|---|---|
| Observation Data | Different means of determining what an agent is observing. |
| Depth Maps | How to use depth maps. |
| Avatar Movement | Different techniques for moving agents. |
| Magnebot | High-level Magnebot API. |
| Document | Description |
|---|---|
| Benchmarks | Performance benchmarks. |
| Performance Optimizations | Increase simulation speed. |
| Document | Description |
|---|---|
| Example Controllers | Index of all example controllers in the repo. |
| Use Cases (see below) | The "use cases" section showcases "advanced" usage of TDW. |
| Document | Description |
|---|---|
| VR | VR in TDW. |
| Humanoids | Add "humanoids" and play animations. |
| Document | Description |
|---|---|
| Physics | Common physics problems and solutions. |
| NVIDIA Flex | Add soft bodies, cloth, and fluids to TDW. |
| FluidTypes | Access different NVIDIA Flex fluid types. |
| Physics Determinism | Benchmark of PhysX physics determinism. |
| tdw_physics | Generate a physics dataset. |
| Rube Goldberg (demo) | Demo of complex physical interactions between objects, with PyImpact generation of impact sounds, set in a photorealistic scene. |
| Document | Description |
|---|---|
| C# code | Access to C# backend source code |
| Releases | Release versioning in TDW. |
| Freezing your code | "Freeze" your controller into a compiled executable. |
| v1.6 to v1.7 | How to upgrade from TDW v1.6 to TDW v1.7 |
| Document | Description |
|---|---|
| Docker | Create a Docker container for TDW. |
| Remote rendering | How to render using xpra. |
| BinaryManager | Manage multiple instances of TDW builds on a remote server. |
| bash scripts | Useful bash scripts for Linux. |
| Document | Description |
|---|---|
| Asset Bundle Librarians | Overview of what asset bundles are, how to add objects, scenes, materials, HDRI skyboxes, and humanoids, and how to access each asset bundle's metadata. |
| Model Screenshotter | Generate images of every model in TDW. |
| Material Screenshotter | Generate images of every material in TDW. |
| Materials, textures, and colors | Defines materials, textures, and colors. |
| Depth of Field | Prevent blurry images and increase realism. |
| Depth Maps | How to use depth maps. |
| Remote rendering | How to render using xpra. |
| Observation Data | Different means of determining what an agent is observing. |
| Document | Description |
|---|---|
| Robots | Overview of robotics and the Magnebot in TDW. |
| Robot Librarian | API for accessing robot metadata. |
| Magnebot | High-level Magnebot API. |
| Document | Description |
|---|---|
| Scene Setup | Overview of how to set up a scene. |
| Asset Bundle Librarians | Overview of what asset bundles are, how to add objects, scenes, materials, HDRI skyboxes, and humanoids, and how to access each asset bundle's metadata. |
| Model Librarian | Overview of how to add objects and access metadata. |
| Rotation | Different means of rotating objects and agents in a scene. |
| Scene Reset | How to reset a scene. |
| FloorplanController | Child class of Controller that creates an interior environment and populates it with objects. |
| Document | Description |
|---|---|
| Model Librarian | Overview of how to add objects and access metadata. |
| Model Screenshotter | Generate images of every model in TDW. |
| Non-free models | Access the TDW "full model library". |
| Local 3D models | Add your own objects to TDW. |
| ShapeNet models | Convert ShapeNET models into TDW objects. |
| Composite Objects Creating Composite Objects |
Use and create "composite objects". |
| AssetBundleCreator | API for the AssetBundleCreator class (used to convert 3D models into TDW-compatible asset bundles). |
| Rotation | Different means of rotating objects and agents in a scene. |
| Document | Description |
|---|---|
| Debug TDW | Several strategies for debugging errors in your code. |
| DebugController | API for the DebugController class. |
| Depth of Field | Prevent blurry images and increase realism. |
| Performance Optimizations | Increase simulation speed. |
| OS X | Common OS X problems and solutions. |
| Physics | Common physics problems and solutions. |
| Use Case | Description |
|---|---|
| Image dataset | Generate 1.3M photorealistic images. |
| IntPhys (demo) | Demo of how to simulate IntPhys in TDW. |
| Humanoid videos | Generate a dataset of humanoid animations. |
| tdw_sound20k | Generate an audio dataset. |
| tdw_physics | Generate a physics dataset. |
| Rube Goldberg (demo) | Demo of complex physical interactions between objects, with PyImpact generation of impact sounds, set in a photorealistic scene. |
| Magnebot | High-level Magnebot API. |