Drake
Drake C++ Documentation
Supported file formats

Drake offers a number of primitive types for representing objects' extents in space.

They include:

The primitives are good for coarse representations of real world shapes and, in some cases, may be preferred over more accurate representations for performance reasons (e.g., for proximity queries). However, there are cases where a higher fidelity representation is preferred. To this end, Drake also provides two more shapes:

Mesh vs Convex

Both of those Shape types reference on-disk geometry files. When a role is assigned to a geometry with the Convex or Mesh shape, the file will be parsed and some portion of the data in the file will be used.

The Convex shape is a declaration that the on-disk geometry file should only be used to compute a convex polytope (the convex hull of the vertices defined in the geometry file). The Convex shape is fully supported across all roles (illustration, perception, and proximity). For visual roles, whatever materials, normals, textures, etc. which may be specified in the referenced geometry file will not affect the visualized convex hull. It will always be a faceted, single-color polytope. The color is defined by assigning the ("phong", "diffuse") geometry property to the geometry. When it comes to file formats' compatibility with the Convex shape, the only question is if Drake has implemented the convex hull computation for the given file format. In the discussion below, each file format indicates whether Drake can compute its convex hull and, by implication, whether the file format can be referenced by Convex.

The Mesh shape is a declaration that the on-disk geometry file should be used to the fullest extent that the role and geometry consumer can. For visual roles, that includes materials, textures, normals, etc. The degree to which the possible mesh data is used is outlined below.

Things are slightly more complex with the proximity role. Performing contact and signed distance queries on general non-convex meshes is complicated. As such, Drake's implementation of Mesh support is incomplete. Where full support does not exist, the Mesh object is represented by its convex hull. Generally, full support is limited to a few cases:

In these remaining cases, the Mesh's convex hull is used:

It is our intent that with time, more queries will move from the lower list to the fully-supported list.

Supported file types

To make good representation choices, it is important to understand what kind of files are supported, in what roles, and to what degree. This page aims to make the answers to those questions clear and provide best practices. As geometry file format support is in flux the contents of this page will change frequently.

Drake is working towards a robust and consistent treatment of supported mesh types. During the transitional period, support will not be consistent. For each supported mesh type, we'll enumerate the nature of Drake's support for it, generally and with respect to each geometry role.

You should assume that if a file format is not listed, it is not supported.

Drake doesn't care about the capitalization of the geometry file extensions.

This documentation will focus on how the geometry is supported in meshcat.

Wavefront (obj) file

Wavefront obj is a decades-old geometry format. The file is encoded in ASCII text with a simple syntax; simple objects can be created in a text editor. Most modern modeling packages can import and export .obj files. Its age has made it a universal format.

The .obj specification includes an additional material library file (.mtl). The .mtl file defines materials that are applied to zero or more faces in the .obj file. The standard practice is for an .obj file to reference one or more .mtl files.

Drake can compute the convex hull for a .obj file, so it is fully compatible with Convex.

A Mesh file which references an .obj file can be used with any role (with limitations).

glTFᵀᴹ (.gltf)

A glTF file is an "open interoperable 3D asset 'transmission' format". Like an .obj file it can include the definition of meshes and materials. glTF is designed to include data for an entire scene, including such diverse elements as nodes, animations, transform hierarchies, cameras, animations, morph targets, etc. Drake does not necessarily use all of that data.

It's worth noting that glTF is designed, from the start, to support Physically Based Rendering (PBR). While the Wavefront .obj specification has informal extensions to achieve the same (replacing Phong with PBR), glTF uses it as its native shading model. This can produce images with higher fidelity, in both rasterization and path-tracing rendering algorithms.

Drake can compute the convex hull for a .gltf file, so it is fully compatible with Convex.

Generally, Drake provides broad support of glTF features. But there are some notable exceptions:

A Mesh file can reference a .gltf file but only for visual roles.

As a rule of thumb, for Drake's purposes .gltf files with external assets (i.e., separate .bin and .png files) will load faster than .gltf files with assets embedded as base64-encoded data URIs.

For best performance, we recommend that meshes destined for the Illustration role use the KHR_texture_basisu extension, where textures are stored using the .ktx2 file format. Loading .ktx2 textures in a web browser is substantially faster than .png textures. In the common case where the same mesh will also be used by the Perception role, be aware that many render engines do not support the KHR_texture_basisu extension so it should be listed in "extensionsUsed" but not "extensionsRequired" in the .gltf file for widest compatibility. (The mesh will have both .png and .ktx2 textures available.) Refer to the Drake models repository for examples. To convert .png files to .ktx2 files, refer to the KTX-Software repository.

VTK file (.vtk)

Drake uses tetrahedral meshes a couple of different ways: as compliant hydroelastic geometries (see here for more details) and deformable bodies (see, e.g., multibody::DeformableModel). For some of the Shape types (the mathematical primitives), Drake will provide a tetrahedralization of the shape at run time. However, Drake cannot yet do so for arbitrary surface meshes. Drake allows you to create your own tetrahedral mesh offline and request that Drake use it. We use the .vtk file format.

Depending on the application (hydroelastics or deformable bodies), not every tetrahedralization is equally valid. There are best practices for each domain which must be followed to get best results. More details on these best practices are coming.

Drake can compute the convex hull for a tetrahedral .vtk file, so it is fully compatible with Convex.

In addition to specifying the tetrahedral mesh for deformable or compliant hydroelastic geometries, the .vtk file can be used more generally:

Drake materials and the specification heuristic

Drake has a very rudimentary internal definition of visual materials. Don't mistake a "visual" material from a "dynamics" material. The latter consists of parameters such as elasticity, friction, etc. The former models how light interacts with the object including parameters such as diffuse color, specularity, etc. Drakes visual material model is based on the Phong shading model, but only makes use of a subset of that model's parameters. Currently, only the diffuse color property is parameterized. It can be either a single Rgba value, or a texture map.

Even if a mesh file contains material definitions of its own, for some geometry operations, Drake will apply its own heuristic to define a Drake visual material. This may result in a visualization that appears different from the model itself. Drake is actively working on closing this gap.

The heuristic defines a Phong material with the diffuse property defined with the following prioritized algorithm. The algorithm evaluates the following steps in sequence, but stops at the first step that provides a viable material definition.

  1. If the mesh file specifies materials, then the diffuse color or texture specified in that material will be applied. Note: cases where an .obj references a material name, but the material is not defined in the .mtl file, or the .mtl file is missing, will be treated as if no material is specified and we proceed to step 2.
    • If diffuse properties are also defined in the mesh's GeometryProperties, a warning will be written to the console.
    • If the material specifies a texture, but the mesh doesn't have texture coordinates, the texture is dismissed and only the material's diffuse color is used.
    • If the material specifies a texture, but the texture isn't accessible, the texture is dismissed and only the material's diffuse color is used.
    • If the mesh file contains multiple materials, then all materials are discarded as if they weren't present and the next step of the algorithm is evaluated.
  2. If the geometry in SceneGraph has had diffuse color or diffuse texture defined in its GeometryProperties, then that diffuse specification is used. The final diffuse color of the object is always the channel-wise product of the texture and the diffuse color. If the diffuse color is white the final diffuse color is defined solely by the texture. Otherwise, the final diffuse color can be a darkened texture (via a grey diffuse color) or tinted (via a diffuse color with a distinct hue). If the geometry properties define a texture but not a color, then the material is assigned a white diffuse color.
    • If a diffuse texture map is defined, but it isn't accessible, then only the white diffuse color is used.
  3. For a mesh file named my_mesh.file, if the file my_mesh.png is located in the same directory (and readable by Drake), it will be applied as an unmodulated diffuse texture.
  4. Default diffuse color defined by geometry consumer (visualizer or render engine) will be applied.

¹ The distinction between Mesh and Convex is only relevant for proximity queries; a Convex shape is treated differently from a Mesh shape. For illustration and perception roles, there is no practical distinction.