Image and Video Processing
Participants: Sanjay Patel, Minh Do, Tom Huang, Daniel Kubacki
Executive Summary
The speed up of computationally expensive image processing tasks, specifically in the area of computational photography. Examples include 2D Motion Deblurring from Coded Exposure and Human in the Loop Parameter Estimation for Blind Deconvolution.
Goals - Extended Description
Our strategy with UPCRC applications work is find compelling applications that have a clear need for speed on the client, and to use these as drivers for the language, tools, and correctness efforts going on within the Center. We first bring to bear whatever languages, tools, and know-how that are most natural for the apps people to gain leverage in their own research. Today that means parallelizing their codes using GPUs and CUDA. This can also mean parallelizing for Larrabee. The reason this is most natural is that most of our applications exhibit fine grained parallelism (lots of short parallel tasks).
Computational Photography is an area rich with intensive computational image processing tasks that provide novel or improved photographic output at the expense of processing. Current realizable tasks include the stitching and merging of multiple images, point spread function shaping with subsequent deconvolution, and methods of computational illumination such as "flash - no flash". The goal of applying parallelism to these tasks is to speed them up to allow real time use and application to video. Along side the goal of improving current techniques, another goal is to extend the reach of computational photography. By increasing the available processing power through parallism, what novel computationally expensive innovations can now be performed? Likely, this will lead into new ideas in the video processing domain, which has been typically intractable due to processing constraints. The goals of the Vivid group and this group are closely related.
We have worked on speeding up a simple deblurring (deconvolution) problem to the point where a human user can be utilized to solve for an unknown parameter. For example, consider an image that has been blurred. When an image is blurred, each pixel becomes a weighted average of its neighboring pixels. Sometimes this weighted average can be modeled as a gaussian function with a certain variance, sigma. In general, at the time of deconvolution, sigma is often unknown. Current implementations solve the decovolution problem multiple times for various sigma in order to determine a rough optimal choice. By speeding up the deconvolution process, the estimation of sigma can be left to the human user, who can decide when the image is sufficiently recovered.
Recently, we are looking into the application of Image Relighting using a Depth Based Camera. The added dimension of depth to a classical photo enables a rough reconstruction of the 3-D structure of a scene. Given this structure, a new light source can be added to the image in a realistic manner conformable to the 3-D structure perceivable in a 2-D image.
Results
- GPU - accelerated Depth Based Image Rendering technology
- Real time cubic interpolation of video demo
- Human in the Loop Parameter Estimation for Blind Deconvolution
Additional Resources
| Graphics | Computational Imaging and Video | Natural Language Processing | Secure Web Browsing | Tele-Immersion | Video Vision Library |