Exploring Spatial Resolution in Image Processing

[PDF]

Abstract

Motivated by the human visual system’s instinct to explore details, image processing algorithms designed to facilitate the viewer’s interpretation of details in an image are ubiquitous. Such algorithms seek to extract the highest spatial frequency information that an original image has to offer, and to render that information clearly to the viewer in the form of an image with often an increased number of pixels. This thesis focuses on methods for extracting the highest possible spatial frequency information from digital imagery. Classical sampling theory provides a full understanding of the highest possible spatial frequency information that can be represented by sampled images that have been spatially band-limited to the Nyquist rate. However, natural digital images are rarely band-limited and often carry substantial energy (and information) at frequencies well beyond the Nyquist rate. My research investigates approaches for extracting information from this out-of-band (beyond the Nyquist frequency limit) energy and proposes algorithms to use that information to generate images with higher spatial resolution. This thesis pursues three approaches to extracting high spatial frequency information from digital imagery, based on frequency, spatial, and cross-channel perspectives to the problem. a) Coefficients representing out-of-band high-frequency contents are closely related to co-located coefficients representing in-band, low-frequency contents. The frequency perspective seeks to exploit those relationships to estimate both the uncorrupted out-of-band and in-band coefficients representing an image with higher spatial resolution; b) Spatial patches (blocks of pixels) of an image are known to be similar to other spatial patches elsewhere in the image. Thus, a patch with high-resolution details that has an insufficient number of samples to accurately represent its details could benefit from its similarity to other spatial patches. Although each individual patch may still be insufficiently sampled to retain its details, the ensemble of samples from the collection of similar patches provides a richer sampling pattern that I seek to exploit in the spatial perspective to the problem; c) In some imaging settings, multiple electro-magnetic channels of images are available from the same scene, with different imaging modalities offering different sensor information, each with its own spatial resolution. The cross-channel perspective seeks to exploit cross-channel proximity to produce high-resolution versions of multiple channels.