BAFS: Bundle adjustment with feature scale constraints for enhanced estimation accuracy

We propose to incorporate within bundle adjustment (BA) a new type of constraint that uses feature scale information, leveraging the scale invariance property of typical image feature detectors (e.g., SIFT). While feature scales play an important role in image matching, they have not been utilized thus far for estimation purposes in a BA framework. Our approach exploits the already-available feature scale information and uses it to enhance the accuracy of BA, especially along the optical axis of the camera in a monocular setup. Importantly, the mentioned feature scale constraints can be formulated on a frame to frame basis and do not require loop closures. We study our approach in synthetic environments and the real-imagery KITTI dataset, demonstrating significant improvement in positioning error.