Three-dimensional tracking of rigid objects in motion using 2D optical flows | |
| Author | Marikhu, Ramesh |
| Call Number | AIT Diss. no.CS-24-01 |
| Subject(s) | Three-dimensional imaging Optical data processing Image processing |
| Note | A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Engineering in Computer Science |
| Publisher | Asian Institute of Technology |
| Abstract | Detecting and tracking objects in image sequences is paramount for any video analytic. While object detectors have become increasingly robust, object trajectories based solely on bounding box tracks (BBTs) are prone to noise due to variations in bounding box position and size, more so when the object’s motion involves rotation. The main al ternative, object tracking based on feature tracking and common motion constraints, especially optical flow tracks (OFTs), is riddled with challenging scenarios such as the presence of noisy feature matches and similarly moving neighboring objects, resulting in high susceptibility to both over-segmentation and under-segmentation of objects in a scene. We present an algorithm that uses a hybrid of these two approaches for monoc ular estimation of the 3D trajectories of rigid objects that combines an object detector and minimal camera calibration with 2D optical flows observed on the image plane. Our combining of bounding box tracks and optical flow tracks resolves the difficulties that arise when BBTs and OFTsare considered in isolation. The algorithm performs 2D Delaunay triangulation over features within a detection bounding box to identify spa tially proximal regions that evince motion common to the same object. The algorithm f irst validates the motion of each individual triangle in flow, after which each triangle in flow is compared with its immediate neighbors, enabling grouping of triangles with geometrically constrained common motion. Nonlinear least squares optimization (we use Levenberg Marquardt) is applied on the resulting triangular mesh in flow to esti mate the motion parameters (3D feature points and 3D transformations) of the rigid object, up to scale. To resolve scale ambiguity, we consider plumb line projection of a particular 2D feature point onto the object detection bounding box, which we use to estimate the height of the feature. We focus on rigid objects moving parallel to a planar surface and evaluate the algorithm on the special case of image sequences of vehicles on roads. We present the result of our method on a large-scale BrnoCompSpeed dataset and find that both BBT and OFT produce competitive results for vehicle speed estima tion. In order to analyze the effectiveness of OFTs, we prepared synthetic datasets to analyze the effect of noise and the number of frames on the accuracy of motion parame ters estimation arising from LM optimization. We use the Iteratively Re-weighted Least Squares (IRLS) loss function based on empirical findings that IRLS reduces the influ ence of outliers. We prepared a real-world dataset with manually annotated ground truth trajectories to allow comparison of bounding box track (BBT) trajectories and optical f lowtrack (OFT) trajectories. We find that OFT trajectories are significantly more accu rate than BBT trajectories on our dataset. Our method for estimating trajectories from OFTs can be applied in a variety of video analytic applications that require accurate 3D trajectories, such as vehicle speed estimation and lane change detection. |
| Year | 2024 |
| Type | Dissertation |
| School | School of Engineering and Technology |
| Department | Department of Information and Communications Technologies (DICT) |
| Academic Program/FoS | Computer Science (CS) |
| Chairperson(s) | Dailey, Mathew N. |
| Examination Committee(s) | Huynh, Trung Luong;Mongkol Ekpanyapong |
| Scholarship Donor(s) | AIT Fellowship |
| Degree | Thesis (Ph.D.) - Asian Institute of Technology, 2024 |