Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions geometry/meshcat.h
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,37 @@ in the visualizer.
Delete(). You are welcome to use absolute paths to organize your data, but the
burden on tracking and cleaning them up lie on you.

@section meshcat_url_parameters Parameters for the hosted Meshcat page

%Meshcat has an *experimental* AR/VR option (using WebXR). It can be enabled
through url parameters. For example, for a meshcat url `http://localhost:7000`,
the following will enable the VR mode:

http://localhost:7000?webxr=vr

To to use augmented reality (where the meshcat background is replaced with your
device's camera image), use:

http://localhost:7000?webxr=ar

If augmented reality is not available, it will fallback to VR mode.

Some notes on using the AR/VR modes:

- Before starting the WebXR session, position the interactive camera to be
approximately where you want the origin of the head set's origin to be.
- The meshcat scene controls are disabled while the WebXR session is active.
- WebXR sessions can only be run with *perspective* cameras.
- The controllers can be *visualized* but currently can't interact with the
Drake simulation physically. To visualize the controllers append the
additional url parameter `controller=on` as in
`http://localhost:7000?webxr=vr&controller=on`.

If you do not have AR/VR hardware, you can use an emulator in your browser to
experiment with the mode. Use an browser plugin like WebXR API Emulator (i.e.,
for [Chrome](https://chrome.google.com/webstore/detail/webxr-api-emulator/mjddjgeghkdijejnciaefnkjmkafnnje)
or [Firefox](https://addons.mozilla.org/en-US/firefox/addon/webxr-api-emulator/)).

@section network_access Network access

See MeshcatParams for options to control the hostname and port to bind to.
Expand Down
17 changes: 17 additions & 0 deletions geometry/meshcat.html
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,23 @@
make_connection(url, reconnect_ms);
<!-- CONNECTION BLOCK END -->

const webxrMode = urlParams.get('webxr');
if (webxrMode) {
viewer.handle_command({
type: "enable_webxr",
mode: webxrMode
});
}

const controllerMode = urlParams.get('controller');
if (controllerMode === "on") {
viewer.handle_command({
type: "visualize_vr_controller"
});
}



</script>

<style>
Expand Down
46 changes: 46 additions & 0 deletions geometry/test/meshcat_manual_test.cc
Original file line number Diff line number Diff line change
Expand Up @@ -508,6 +508,52 @@ Open up your browser to the URL above.
}
}

std::cout << "\n";
std::cout << "Now we'll test the WebXR functionality.\n";
std::cout << "In a new browser window, open the URL:\n "
<< meshcat->web_url() << "?webxr=vr&controller=on\n";
std::cout << "If you don't have VR hardware installed on your machine, "
"you'll have to install the WebXR API emulator appropriate to "
"your browser. For Google Chrome "
"see:\n https://chrome.google.com/webstore/detail/"
"webxr-api-emulator/mjddjgeghkdijejnciaefnkjmkafnnje\n"
"If you are using Firefox see:\n "
"https://addons.mozilla.org/de/firefox/addon/"
"webxr-api-emulator/";
std::cout << "\nIf the emulator is installed properly, you should see a "
"button at the bottom that says \"Enter VR\".\n";
std::cout << "Open the developer tools of your browser (F12). At the top "
"of the developer tools windows click on the double arrows icon "
"and a new tab should be available saying \" WebXR \". "
"Make sure to select a device with controllers from the top "
"drop-down menu (e.g., Oculus Quest). Click the "
"\"Enter VR\" button. You should see the following:\n"
<< " - The rendering screen is now split into two images.\n"
<< " - The meshcat controls are gone (there is a message in the "
"console informing you of this).\n"
<< " - You should be able to manipulate the view in the WebXR "
"emulator to affect what you see."
<< " - Clicking on the headset/controller "
"mesh for the first time in "
"the emulator window will bring up colored arrows which you can "
"use to move the mesh. Click a second time on the mesh to "
"switch to rotation mode."
<< "When you're done, close the browser window.\n\n";

MaybePauseForUser();

std::cout << "\nNow we'll try it again with *augmented* reality.\n"
<< "In yet another browser window, open:\n"
<< meshcat->web_url() << "?webxr=ar&controller=on\n"
<< "This should be the same as before but with two differences:\n"
<< " - The button reads \"Enter XR\"\n"
<< " - When you click the button, the background becomes white. "
"If you have an actual AR device, you should see the camera's "
"image as the background.\n"
<< "When you're done, close the browser window.\n\n";

MaybePauseForUser();

std::cout << "Exiting..." << std::endl;
return 0;
}
Expand Down