The Principles Behind Digital Image Sharpening

Sharpening as an Operation in the Frequency Domain

Sharpening
as an Operation in the Spatial Domain

Interpreting
the Inverse of the Blurring Matrix

Sharpening
as Executed by an Image Editing Program

Local
Contrast Enhancement and Sharpening by Unsharp Mask

Laplacian Edge Sharpening is Just More of the Same

Optical imaging, recording, and reproduction systems are
imperfect, and as a result, they

degrade and blur images. To the extent that degradation in image resolution is
known and

is reproducible over the area of an image, often it can be reduced or
reversed.

Generically, the inversion process is called sharpening, and purpose of the
following

material is to explain how sharpening works.

Blurring and its inversion by sharpening can be analyzed in
several ways. One basic

approach considers the spreading of a single sharp point of light. This form of
analysis is

analogous to analyzing electronic circuits by their response to a sharp pulse
input. The

second basic way to analyze blurring and sharpening considers images to consist
of a sum

of signals that oscillate with varying position in the image. Low frequency
components

describe properties that vary across large portions of an image and high
frequency

components describe small details like the edges of objects.

A perfect lens faithfully images all details of an object.
That is, it perfectly transmits all

spatial frequencies without loss. A real lens may transmit low spatial
frequencies with an

efficiency near one, but it attenuates image information carried at the highest
frequencies.

For a nice discussion of the phenomenon, see

http://www.normankoren.com/Tutorials/MTF.html.

The loss of high frequency components produces blurring. If
the high frequency

components could be restored, then images would possess their normal contrast
for the

smallest objects, that is, they would be sharpened. In reality it is quite easy
to boost some

of the higher spatial frequencies and thereby improve the sharpness of an
image. The

steps are the following:

1.
Blur a copy of the image. This destroys fine detail, that is,
it eliminates the

high spatial frequencies while it leaves gross features that are described by

low spatial frequencies largely unaltered.

2.
Subtract a fraction, say 0.2, of the blurred image from the
original. This

subtracts low frequency components from the original image while leaving the

high frequency components unaltered.

3.
Finally, boost the amplitude of all frequencies (contrast
enhancement). This

produces an end result in which the high frequencies have been boosted closer

to what a perfect lens would deliver, and the low frequencies remain as

before.

The procedure described above is the unsharp mask sharpening
method,

http://www.unm.edu/~keithw/astroPhotography/imageSharpening.html.
Clearly, the

radius used in the blurring step needs to be carefully chosen for optimum
results.

**Sharpening
as an Operation in the Spatial Domain**

To simplify the following discussion, images will be
considered as one dimensional. The

same principles and conclusions apply to two dimensional images. A one
dimensional

image can be represented as a set of pixel intensities along a line. Represent
these

intensities as a column vector where *x*_{1} provides the
intensity of the first pixel of the line,

*x*_{2} the intensity of the second, and so on.

_{}_{}

A point object P that is positioned so that its image should fall on pixel 4 is represented as

_{}.

An imaging system somewhat blurs the point object P, and the
blurred image output B is

spread somewhat into adjacent line segments, perhaps as follows

_{}.

Represent the effect of the imperfect lens and sensor that
produced the above blurred

image as a matrix operator *M* acting on the point object vector *P*
to produce the blurred

output *B*,

_{},

or, writing out the matrices,

_{}_{}

If other points along the original image line are similarly
blurred, for example, the next

point above the original point, then another column must be added to the left
of the center

of *M*. Hence, it is clear that the blurring matrix for a complete “image”
is the following.

_{}

So as to maintain total image intensity and to make
subsequent steps easier, the image

intensities that run off the edges of the matrix are piled up along the edges.
Of course, for

a real image thousands of pixels across, edge effects are negligible. The examples
shown

here have been chosen to be just large enough to display the general
properties.

In some cases, and this is one of them, the inverse of _{}exists such that _{}

where I is the identity matrix with ones along the main diagonal and zeroes
everywhere

else. (_{} times any vector equals the vector.) The inverse of _{}in this case can be found

with an Excel™ spreadsheet and is

_{}.

In this case it is possible to reverse the blurring because
the blurred image, _{} of a

general object _{} is _{} and hence, _{}. That

is, is the original image _{}. Let us test this by applying our sharpening or

unblurring matrix _{}to our blurred image. Multiplying our blurred point vector

_{}

by _{} yields

_{}

and except for round off errors, we recover the original image.

Areas of an image that possess uniform intensity ought not
to be changed by the

application of the sharpening matrix. This can be seen to be the case because
the sum of

the elements in each row is close to one. The sum of the elements in the
central row is

1.0000 when the more precise values in the spreadsheet are summed.

** **

**Interpreting the Inverse of the
Blurring Matrix**

The inverse of the blurring matrix that was determined in
the previous section contains an

interesting prescription as to how to increase sharpness. Consider how the
intensity of the

central pixel, pixel 4, in the blurred image contributes to the final unblurred
image. For

pixel 1, examination of the unblurring matrix shows that the contribution of
blurred pixel

4 to the final sharpened intensity of pixel 1 is 0.03 time the intensity of
blurred pixel 4.

Similarly, the contribution of blurred pixel 4 to the sharpened intensity of
pixel 2 is 0.14

times the intensity of blurred pixel 4. As expected, the strongest contribution
of pixel 4 to

the sharpened image is to adjacent pixels 3 and 5, where it contributes –1.1
times the

intensity of pixel 4. This last contribution is nothing other than unsharp
masking! That is,

the prescription for unsharp masking is to blur the image and subtract a
portion from the

original. That is the same as what is accomplished by the matrix elements of
value –1.1.

The unblurring or sharpening matrix _{}contains the combined effects of subtracting a

portion of the blurred original and then multiplying the amplitude so as to
restore normal

contrast.

Photographers were not the first to invent the unsharp mask
procedure for sharpening

images. The human retina is hard wired to perform the unsharp masking
operation.

Psychologists call the phenomenon lateral inhibition,

http://www-psych.stanford.edu/~lera/psych115s/notes/lecture3/
because exciting one

sensor in the eye slightly desensitizes nearby sensors. This is the same as
subtracting a

portion of the blurred image from the original. The halos which surround
objects

following aggressive sharpening of digital images also occur in human vision
where they

produce an optical illusion called Mach bands, http://dragon.uml.edu/psych/mach.html.

An interesting property of the unblurring matrix is the fact
that it contains instructions for

improving upon unsharp masking. That is, it also instructs us to ADD a portion
of the

intensity of pixel 4 to pixels 2 and 6 and a smaller portion to pixels 1 and 7.
One might

wonder whether the presence of both subtractive and additive terms in the
unblurring

matrix might be a special property dependent upon the precise shape of the
blurring curve

or blurring matrix. Experimenting with a wide collection of peak shapes for the
blurring

function shows that this is unlikely to be the case. Hence, this raises the
question, can the

unsharp mask sharpening algorithm usefully be improved upon by blurring a real
image

and subtracting a portion from the original, and then blurring with about twice
the

blurring radius and ADDING a smaller portion of this to the original?

Another question that the above analysis raises is whether
customizing a sharpening

algorithm to a particular picture significantly improves the results. In
principle, if one

knew that a particular feature in an image was the blurred image of a point
source, then

one could measure the intensities of nearby pixels and determine the elements
in the

blurring matrix. Inverting this should give the best possible method of
sharpening that

particular image.

**Sharpening
as Executed by an Image Editing Program**

The sharpening matrix can be deduced by applying it to a
suitable image because, as we

saw above,

** **

_{}

will extract the values from M^{-1} giving the
contribution of the central pixel to surrounding

pixels. Since image editing programs do not record or use negative intensities
or

intensities above 1.0, a more useful “image” to process with a sharpening
procedure is

_{}

Picture Window Pro™. http://www.dl-c.com/
is an image editing program

designed for the serious photographer. Below are shown images prepared

with Picture Window Pro. The first is an enlarged view of an 11 pixel x 11 pixel
“image”

in which the background has a luminosity of 0.2 and the central pixel has a
luminosity of

0.6.

Sharpening not only lightens the central pixel, but it also
darkens surrounding pixels as

shown below. The exact values can be obtained with the program's readout tool.

Applying an unsharp mask also clearly lightens the central
pixel, but because more of the

surrounding pixels are darkened, the darkening effect is spread more thinly and
is less

conspicuous.

** **

**Local
Contrast Enhancement and Sharpening by Unsharp Mask**

Images frequently can be brightened by increasing contrast.
Usually this cannot be done

globally because the process converts dark areas to pure black and light areas
to pure

white. Instead, techniques are used that enhance the contrast of light and dark
areas

differently. One method is to generate contrast masks so as to allow separate

manipulation of contrast in different parts of an image. While this method can
be

laborious, it can also yield beautiful results,

http://www.normankoren.com/PWP_contrast_masking.html.
There is, however, a rapid

global method for enhancing contrast that frequently gives good results.

Before describing how one uses an image editing program to
enhance contrast, it is

helpful to see how contrast can be increased without simply turning up contrast
on the

entire image. Consider an object in an image with features whose intensities
vary

between 9 and 10 units. The contrast in this area is then about one in ten. If
in this area of

the image we subtract five units of intensity and then multiply the intensities
by two,

points that previously had an intensity of nine, now have an intensity of
eight, and points

that previously had an intensity of ten still end with an intensity of ten. By
the subtract

and multiply operations, contrast has been increased from one in ten to two in
ten, that is,

it has been doubled. Similarly, if there are areas in the image with average
intensity of

100 and detail in this area has intensities between 90 and 100, then
subtracting 50 and

multiplying by two also doubles the contrast in these areas as well.

Blurring the original image discussed above with a radius on
the order of the major light

and dark objects in the image generates a new image with an intensity of about
10 in the

one area of the image and an intensity about 100 in the other area of the
image. That is,

the blurred image is what we need to subtract from the original. If half the
blurred image

is subtracted, and then the overall contrast is increased by a factor of two,
the local

contrast over most of the image is doubled.

The local contrast enhancement method just described above
boosts the amplitudes of the

higher frequency spatial components of an image by the same method that unsharp
mask

boosts the amplitudes of high frequency spatial components. In using the
unsharp mask

procedure to enhance local contrast, the radius for the blurring operation,
however, needs

to be chosen considerably larger than when the only objective is the sharpening
of edges.

The net result is that the amplitudes of some of the spatial frequencies are
increased well

above their natural values. When not done to excess, the process adds snap to
an image.

Typically a blur radius of a couple hundred pixels and an amplitude of about
20% is used.

Note that the process also boosts the amplitudes of the high frequency
components that

define edges. Thus, less edge sharpening is required after local contrast
enhancement,

LCE.

** **

**Laplacian
Edge Sharpening is Just More of the Same**

Sometimes sharpening is described in terms of the Laplacian
differential operator,

_{}, or in one dimension _{}. At a slightly blurred edge, the

first and second derivatives of intensity would look roughly as shown, and when
a

portion of the second derivative is subtracted from the image, the end result
is a slight

halo and a sharpened edge.

The first derivative of intensity at the pixel i, *x*_{i},
is the difference between the intensity of

*x*_{i+1} and *x*_{i}, that is *x*_{i+1}
– *x*_{i } divided by the
distance between i and i + 1 which we shall take

to be one. Similarly, the first derivative of the intensity at pixel i-1 is *x*_{i
}- *x*_{i‑1. }The second

derivative of intensity at the origin is the difference of the first
derivatives, and thus is

*x*_{i+1} – 2*x*_{i} + *x*_{i-1}.
If a portion of this second derivative is to be subtracted from the original

intensity, the operation is - *x*_{i+1} + 2*x*_{i}
- *x*_{i-1} which is the by now familiar recipe for

subtracting a fraction of a pixel’s intensity from the intensities of adjacent
pixels.

August 2, 2005