The Principles Behind Digital Image Sharpening

 

General Background

Sharpening as an Operation in the Frequency Domain

Sharpening as an Operation in the Spatial Domain

Interpreting the Inverse of the Blurring Matrix

Sharpening as Executed by an Image Editing Program

Local Contrast Enhancement and Sharpening by Unsharp Mask

Laplacian Edge Sharpening is Just More of the Same

Home

 

General Background

Optical imaging, recording, and reproduction systems are imperfect, and as a result, they
degrade and blur images. To the extent that degradation in image resolution is known and
is reproducible over the area of an image, often it can be reduced or reversed. 
Generically, the inversion process is called sharpening, and purpose of the following
material is to explain how sharpening works.

 

Blurring and its inversion by sharpening can be analyzed in several ways. One basic
approach considers the spreading of a single sharp point of light. This form of analysis is
analogous to analyzing electronic circuits by their response to a sharp pulse input. The
second basic way to analyze blurring and sharpening considers images to consist of a sum
of signals that oscillate with varying position in the image. Low frequency components
describe properties that vary across large portions of an image and high frequency
components describe small details like the edges of objects.

 

Sharpening as an Operation in the Frequency Domain

A perfect lens faithfully images all details of an object. That is, it perfectly transmits all
spatial frequencies without loss. A real lens may transmit low spatial frequencies with an
efficiency near one, but it attenuates image information carried at the highest frequencies.
For a nice discussion of the phenomenon, see
http://www.normankoren.com/Tutorials/MTF.html.

 

   

 

 

The loss of high frequency components produces blurring. If the high frequency
components could be restored, then images would possess their normal contrast for the
smallest objects, that is, they would be sharpened. In reality it is quite easy to boost some
of the higher spatial frequencies and thereby improve the sharpness of an image. The
steps are the following:

 

1.     Blur a copy of the image. This destroys fine detail, that is, it eliminates the
high spatial frequencies while it leaves gross features that are described by
low spatial frequencies largely unaltered.

2.     Subtract a fraction, say 0.2, of the blurred image from the original. This
subtracts low frequency components from the original image while leaving the
high frequency components unaltered.

3.     Finally, boost the amplitude of all frequencies (contrast enhancement). This
produces an end result in which the high frequencies have been boosted closer
to what a perfect lens would deliver, and the low frequencies remain as
before.

 

      

 

The procedure described above is the unsharp mask sharpening method,
http://www.unm.edu/~keithw/astroPhotography/imageSharpening.html. Clearly, the
radius used in the blurring step needs to be carefully chosen for optimum results.

 

Sharpening as an Operation in the Spatial Domain

To simplify the following discussion, images will be considered as one dimensional. The
same principles and conclusions apply to two dimensional images. A one dimensional
image can be represented as a set of pixel intensities along a line. Represent these
intensities as a column vector where x1 provides the intensity of the first pixel of the line,
x2 the intensity of the second, and so on.

 

 

A point object P that is positioned so that its image should fall on pixel 4 is represented as 

 

.

 

An imaging system somewhat blurs the point object P, and the blurred image output B is
spread somewhat into adjacent line segments, perhaps as follows

 

.

 

Represent the effect of the imperfect lens and sensor that produced the above blurred
image as a matrix operator M acting on the point object vector P to produce the blurred
output B,

,

or, writing out the matrices,

 

 

If other points along the original image line are similarly blurred, for example, the next
point above the original point, then another column must be added to the left of the center
of M. Hence, it is clear that the blurring matrix for a complete “image” is the following.

 

 

So as to maintain total image intensity and to make subsequent steps easier,  the image
intensities that run off the edges of the matrix are piled up along the edges. Of course, for
a real image thousands of pixels across, edge effects are negligible. The examples shown
here have been chosen to be just large enough to display the general properties.

 

In some cases, and this is one of them, the inverse of exists such that   
where I is the identity matrix with ones along the main diagonal and zeroes everywhere
else. ( times any vector equals the vector.) The inverse of in this case can be found
with an Excel™ spreadsheet and is

 

.

 

In this case it is possible to reverse the blurring because the blurred image,  of a
general object  is  and hence, . That
is, is the original image . Let us test this by applying our sharpening or
unblurring matrix to our blurred image. Multiplying our blurred point vector

 

 

 

by  yields

 

 

 

and except for round off errors, we recover the original image.

 

Areas of an image that possess uniform intensity ought not to be changed by the
application of the sharpening matrix. This can be seen to be the case because the sum of
the elements in each row is close to one. The sum of the elements in the central row is
1.0000 when the more precise values in the spreadsheet are summed.

 

 

 

Interpreting the Inverse of the Blurring Matrix

The inverse of the blurring matrix that was determined in the previous section contains an
interesting prescription as to how to increase sharpness. Consider how the intensity of the
central pixel, pixel 4, in the blurred image contributes to the final unblurred image. For
pixel 1, examination of the unblurring matrix shows that the contribution of blurred pixel
4 to the final sharpened intensity of pixel 1 is 0.03 time the intensity of blurred pixel 4.
Similarly, the contribution of blurred pixel 4 to the sharpened intensity of pixel 2 is 0.14
times the intensity of blurred pixel 4. As expected, the strongest contribution of pixel 4 to
the sharpened image is to adjacent pixels 3 and 5, where it contributes –1.1 times the
intensity of pixel 4. This last contribution is nothing other than unsharp masking! That is,
the prescription for unsharp masking is to blur the image and subtract a portion from the
original. That is the same as what is accomplished by the matrix elements of value –1.1.
The unblurring or sharpening matrix contains the combined effects of subtracting a
portion of the blurred original and then multiplying the amplitude so as to restore normal
contrast.

 

Photographers were not the first to invent the unsharp mask procedure for sharpening
images. The human retina is hard wired to perform the unsharp masking operation.
Psychologists call the phenomenon lateral inhibition,
http://www-psych.stanford.edu/~lera/psych115s/notes/lecture3/ because exciting one
sensor in the eye slightly desensitizes nearby sensors. This is the same as subtracting a
portion of the blurred image from the original. The halos which surround objects
following aggressive sharpening of digital images also occur in human vision where they
produce an optical illusion called Mach bands, http://dragon.uml.edu/psych/mach.html.

 

An interesting property of the unblurring matrix is the fact that it contains instructions for
improving upon unsharp masking. That is, it also instructs us to ADD a portion of the
intensity of pixel 4 to pixels 2 and 6 and a smaller portion to pixels 1 and 7. One might
wonder whether the presence of both subtractive and additive terms in the unblurring
matrix might be a special property dependent upon the precise shape of the blurring curve
or blurring matrix. Experimenting with a wide collection of peak shapes for the blurring
function shows that this is unlikely to be the case. Hence, this raises the question, can the
unsharp mask sharpening algorithm usefully be improved upon by blurring a real image
and subtracting a portion from the original, and then blurring with about twice the
blurring radius and ADDING a smaller portion of this to the original?

 

Another question that the above analysis raises is whether customizing a sharpening
algorithm to a particular picture significantly improves the results. In principle, if one
knew that a particular feature in an image was the blurred image of a point source, then
one could measure the intensities of nearby pixels and determine the elements in the
blurring matrix. Inverting this should give the best possible method of sharpening that
particular image.

 

 

Sharpening as Executed by an Image Editing Program

The sharpening matrix can be deduced by applying it to a suitable image because, as we
saw above, 

 

 

will extract the values from M-1 giving the contribution of the central pixel to surrounding
pixels. Since image editing programs do not record or use negative intensities or
intensities above 1.0, a more useful “image” to process with a sharpening procedure is

 

 

Picture Window Pro™. http://www.dl-c.com/ is an image editing program
designed for the serious photographer. Below are shown images prepared
with Picture Window Pro. The first is an enlarged view of an 11 pixel x 11 pixel “image”
in which the background has a luminosity of 0.2 and the central pixel has a luminosity of
0.6.

      

 

Sharpening not only lightens the central pixel, but it also darkens surrounding pixels as
shown below. The exact values can be obtained with the program's readout tool.

 

 

 

Applying an unsharp mask also clearly lightens the central pixel, but because more of the
surrounding pixels are darkened, the darkening effect is spread more thinly and is less
conspicuous.

 

 

 

Local Contrast Enhancement and Sharpening by Unsharp Mask

Images frequently can be brightened by increasing contrast. Usually this cannot be done
globally because the process converts dark areas to pure black and light areas to pure
white. Instead, techniques are used that enhance the contrast of light and dark areas
differently. One method is to generate contrast masks so as to allow separate
manipulation of contrast in different parts of an image. While this method can be
laborious, it can also yield beautiful results,
http://www.normankoren.com/PWP_contrast_masking.html. There is, however, a rapid
global method for enhancing contrast that frequently gives good results.

 

Before describing how one uses an image editing program to enhance contrast, it is
helpful to see how contrast can be increased without simply turning up contrast on the
entire image. Consider an object in an image with features whose intensities vary
between 9 and 10 units. The contrast in this area is then about one in ten. If in this area of
the image we subtract five units of intensity and then multiply the intensities by two,
points that previously had an intensity of nine, now have an intensity of eight, and points
that previously had an intensity of ten still end with an intensity of ten. By the subtract
and multiply operations, contrast has been increased from one in ten to two in ten, that is,
it has been doubled. Similarly, if there are areas in the image with average intensity of
100 and detail in this area has intensities between 90 and 100, then subtracting 50 and
multiplying by two also doubles the contrast in these areas as well.

 

Blurring the original image discussed above with a radius on the order of the major light
and dark objects in the image generates a new image with an intensity of about 10 in the
one area of the image and an intensity about 100 in the other area of the image. That is,
the blurred image is what we need to subtract from the original. If half the blurred image
is subtracted, and then the overall contrast is increased by a factor of two, the local
contrast over most of the image is doubled.

 

The local contrast enhancement method just described above boosts the amplitudes of the
higher frequency spatial components of an image by the same method that unsharp mask
boosts the amplitudes of high frequency spatial components. In using the unsharp mask
procedure to enhance local contrast, the radius for the blurring operation, however, needs
to be chosen considerably larger than when the only objective is the sharpening of edges.
The net result is that the amplitudes of some of the spatial frequencies are increased well
above their natural values. When not done to excess, the process adds snap to an image.
Typically a blur radius of a couple hundred pixels and an amplitude of about 20% is used.
Note that the process also boosts the amplitudes of the high frequency components that
define edges. Thus, less edge sharpening is required after local contrast enhancement,
LCE.

 

 

 

 

 

Laplacian Edge Sharpening is Just More of the Same

Sometimes sharpening is described in terms of the Laplacian differential operator,
, or in one dimension . At a slightly blurred edge, the
first and second derivatives of intensity would look roughly as shown, and when a
portion of the second derivative is subtracted from the image, the end result is a slight
halo and a sharpened edge.

 

 

The first derivative of intensity at the pixel i, xi, is the difference between the intensity of
xi+1 and xi, that is xi+1xi  divided by the distance between i and i + 1 which we shall take
to be one. Similarly, the first derivative of the intensity at pixel i-1 is xi - xi‑1. The second
derivative of intensity at the origin is the difference of the first derivatives, and thus is
xi+1 – 2xi + xi-1. If a portion of this second derivative is to be subtracted from the original
intensity, the operation is - xi+1 + 2xi - xi-1 which is the by now familiar recipe for
subtracting a fraction of a pixel’s intensity from the intensities of adjacent pixels.

 

 

 

August 2, 2005