On-Road Object Importance Estimation

Example Video


Are All Objects Equal? Deep Spatio-Temporal Importance Prediction in Driving Videos
E. Ohn-Bar and M. Trivedi
Pattern Recognition, 2017

What Makes an On-road Object Important? (Oral, Best paper award finalist)
E. Ohn-Bar and M. Trivedi
IEEE International Conference on Pattern Recognition, 2016


Human drivers continuously attend to important scene elements in order to safely and smoothly navigate in intricate environments and under uncertainty. This paper develops a human-centric framework for object recognition by analyzing a notion of object importance, as measured in a spatio-temporal context of driving a vehicle. Given a video, a main research question in this paper is - which of the surrounding agents are most important? The answer inherently requires complex reasoning over the current driving task, object properties, scene context, intent, and possible future actions. Therefore, we find that various spatio-temporal cues are relevant for the importance classification task. Furthermore, we demonstrate the usefulness of the importance annotations in evaluating vision algorithms (specifically, for the task of object detection) in an application where trust in automation is imperative and errors are costly. Finally, we show that importance-guided training of object detection models results in improved detection performance of surrounding objects of higher importance. Hence, such models may be better suited for use in representing safety-critical situations, predicting surrounding agents' intentions, and in human-robot interactivity.

Code and Data