Overview Videos
SPLANNING side-by-side real world
Abstract
Neural Radiance Fields and Gaussian Splatting have recently transformed the field of computer vision. By estimating a scene’s radiance field, these techniques enable photo-realistic representation of complex scenes. Despite this success, they have seen only limited use in real-world robotics tasks such as motion planning. Two key factors have contributed to this limited success. First, it is challenging to reason about collisions in radiance models. Second, it is difficult to perform inference of radiance models fast enough for real-time motion planning. This paper addresses these challenges by proposing SPLANNING, a risk-aware trajectory planner that operates in a Gaussian Splatting model. This paper first derives a method for rigorously upper-bounding the probability of collision between the robot and a radiance field. Second, this paper introduces a normalized reformulation of Gaussian Splatting that allows for the efficient computation of the collision bound in a Gaussian Splat. Third, a method is presented to optimize trajectories while avoiding collisions with a scene represented by a Gaussian Splat. Experiments demonstrate that SPLANNING outperforms state-of-the-art methods in generating collision-free trajectories in highly cluttered environments. The proposed system is also tested on a real-world robot manipulator.
Approach
This paper proposed SPLANNING, a real-time, receding-horizon motion planning algorithm. SPLANNING combines a simple and tight sphere-based geometric representation of the reachability of a robot with radiance field representations of an environment scene to enable risk-aware planning through a rendering-based probability bound. This differs from existing approaches such as CATNIPS and SplatNav in that planning can be performed with probabilistic bounds and directly in a scene with a normalized splatting representation without additional preprocessing passes, opening the door for real-time splatting and risk-aware planning.
Prior to planning, a normalized Gaussian Splatting representation of the scene is constructed. At every planning iteration, the robot is given \(t_{p} \leq 0.5\) seconds to find a feasible trajectory by solving
\[\newcommand{\qlim}{q_{j,\mathrm{lim}}} \newcommand{\dqlim}{\dot{q}_{j,\mathrm{lim}}} \newcommand{\bound}{\mathcal{H}} \begin{align} \underset{k \in K}{\min}& \quad \mathrm{\texttt{cost}}(k) \\ % \text{s.t.}& \quad q_j(T_i; k) \subseteq [\qlim^-, \qlim^+] \quad\quad\quad \forall (i,j) \in N_t \times N_q \label{eq:pz_optpos} \\ % & \quad \dot{q}_j(T_i; k) \subseteq [\dqlim^-, \dqlim^+] \quad\quad\quad \forall (i,j) \in N_t \times N_q \label{eq:pz_optvel}\\ \text{s.t.}&\quad \texttt{Prob}(\texttt{collision}) < \beta \end{align}\]The paper, coming soon, details the derivation of the bound on the probability of collision.
Our key contributions are:
- A derivation of a bound on the probability that a 3D body collides with a scene represented by a radiance field, starting directly from the rendering equation;
- A method to efficiently compute this collision bound in a Gaussian Splatting model;
- A re-formulation of Gaussian Splatting that normalizes the 3D Gaussians to ensure the correctness of the collision probabilities;
- A novel risk-aware trajectory planner for robot manipulators, with experiments that show the resulting risk-aware planner solves challenging tasks in simulation and on hardware in real-time.
Results
Simulation Results
The following videos demonstrate the performance of SPLANNING in simulated worlds with increasing amounts of clutter.
10 obstacles
20 obstacles
40 obstacles
Hardware Results
We also demonstrate success with other hardware configurations.
Citation
This project was developed in Robotics and Optimization for Analysis of Human Motion (ROAHM) Lab at the University of Michigan - Ann Arbor.
@article{michauxisaacson2024splanning,
title={Let's Make a Splan: Risk-Aware Trajectory Optimization in a Normalized Gaussian Splat},
author={Jonathan Michaux and Seth Isaacson and Challen Enninful Adu and Adam Li and Rahul Kashyap Swayampakula and Parker Ewen and Sean Rice and Katherine A. Skinner and Ram Vasudevan},
year={2024},
eprint={2409.16915},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2409.16915},
}