IIT Madras And Rice University Researchers Develop Algorithms For Lensless, Miniature Cameras

Indian Institute of Technology Madras and Rice University, US, have developed algorithms for lensless, miniature cameras. Such lensless cameras have numerous vision applications in areas such as Augmented Reality (AR)/ Virtual Reality (VR), security, smart wearables and robotics where cost, form-factor, and weight are major constraints (see Figure 1).

Lensless cameras do not have a lens which, in a conventional camera, acts as a focusing element allowing the sensor to capture a sharp photograph of the scene. Due to the absence of this focusing element, the lensless camera captures a multiplexed or globally blurred measurement of the scene. IIT Madras and Rice University researchers have developed a deep learning algorithm for producing photo-realistic images from the blurred lensless capture.

Taking out a lens can lead to the miniaturization of a camera. Researchers globally are trying to find substitutes for lenses.

In 2016, Prof. Ashok Veeraraghavan’s lab at Rice University, U.S., registered success in making a lensless camera. They were able to develop a low-cost and low-weight ultra-thin lensless camera. The function of lenses is to focus the incoming light. In these newly developed lensless cameras, a thin optical mask was placed just in front of the sensor at a distance of approximately 1 mm. However, because of the absence of focusing elements, the lensless camera captures blurred images restricting their commercial use.

IIT Madras and Rice researchers have now developed a computational solution to this problem. The team developed a de-blurring algorithm, which can correct the blurred images taken from a lensless camera (see Figure 2).

The findings were presented as a paper in the prestigious IEEE International Conference on Computer Vision and an extended version appeared in IEEE Transactions on Pattern Analysis and Machine Intelligence.

This research was led at IIT Madras by Dr. Kaushik Mitra, Assistant Professor, Department of Electrical Engineering. The research team included Mr. Salman Siddique Khan, Mr. Varun Sundar, and Mr. Adarsh VR from IIT Madras. Prof. Ashok Veeraghavan led the Rice University team which included Dr. Vivek Boominathan and Mr. Jasper Tan.

Elaborating on the importance of this research, Dr. Kaushik Mitra, Head of Computational Imaging Laboratory, IIT Madras and Assistant Professor, Department of Electrical Engineering, said, “Existing algorithms to deblur images based on traditional optimization schemes yield low-resolution ‘noisy images.’ Our Research team used Deep Learning to develop a reconstruction algorithm called ‘FlatNet’ for lensless cameras which resulted in significant improvement over traditional optimization-based algorithms (see Figure 2). FlatNet was tested on various real and challenging scenarios and was found to be effective in de-blurring images captured by the lensless camera.”

Further, Dr. Kaushik Mitra said, “Lensless imaging is a new technology and its true potential in solving imaging/vision problems has not been exploited completely. Therefore, we are working on designing newer and better lensless cameras using data-driven techniques, devising efficient algorithms for doing inference on lensless captures and looking into interesting and important applications like endoscopy and smart surveillance, among other areas, where one can fully realize the benefits of lensless imaging.”

This research was funded by National Science Foundation (NSF) CAREER and NSF EXPEDITIONS, U.S., Neural Engineering System Design (NESD) - Defense Advanced Research Projects Agency (DARPA), U.S., National Institutes of Health (NIH) Grant, U.S., and Qualcomm Innovation Fellowship India 2020.