Lab 4.1 Basic image representations

Introduction

Humans are visual animals with unique skills for interpreting images. A picture is truly worth 1000 words. The ease with which we humans process images creates unique opportunities and problems with regard to using images in data analysis. On the positive side:

- When done carefully, telling a story using images can quickly convey complex ideas to a broad audience.

On the negative side:

- The methods by which images are usually collected and represented is intimately related to properties of the human visual system rather than properties of the physical world.

It is critical to understand how images are collected, represented, and displayed digitally in order to perform analysis on acquired images and to create images that are easy to see and interpret.

Digital images

What are images? For our purposes, images are 2-dimensional or 3-dimensional arrays of data. These arrays can be generated by computer (for example, a graph of a mathematical function) or can be acquired by a camera, microscope, or other imaging device.

"Intensity" images

The simplest type of image is what I will call an intensity image (my term). The value of each pixel corresponds to the intensity of the signal at that location, whether it is the value of a mathematical function or the intensity of some signal falling on a detector such as a CCD camera.

Let's read in and display a simple intensity image using the functions imread and image. Please download the 2 images attached at the end of this page.

myimage = imread('black-and-white-cow-2-3.tif');

image(myimage);

You'll see that this intensity image is being plotted with Matlab's default color map that varies from cold (bluish colors) to warm (red). But we can also plot the image using luminance to indicate intensity (in my opinion, this is more natural).

colormap(gray(256))

Let's look at the size of this image.

size(myimage)

You'll notice that this is an N x M matrix; that is, a 2-dimensional matrix.

Now let's explore the relationship between what you see on the screen and intensity values. Let's create a very simple image manually:

A = [ 20 40 60 80; 100 120 140 160; 180 200 220 240];

A, % look at A to see it in matrix form

figure;

image(A);

colormap(gray(256));

We can also examine the intensity data in our example image above:

myimage(200,1:10)

You can see that the image is just comprised of numbers. We can plot these values for an individual row (as done here) or column:

figure;

plot(myimage(200,:),'k-');

Compare the line plot with the original picture; can you see the bright and dark spots of the row you plotted? Can you see the relationship? In particular, the big white stripe at X=260 through 270ish?

We can also go in the other direction and edit the image with our own numbers.

myimage(200,:) = 255;

figure;

image(myimage);

colormap(gray(256));

Now you should be able to see that the 200th row of the image has been set to all white.

Creating simple images from mathematical functions

Now let's create an image using a simple mathematical function.

X = [0 : 1 : 20 ]; % a 21 column vector with 1 row

X, % look at it

X = [X; X; X; X; X;]; % copy this row 5 times

X, % look at it

figure;

sinewaveimage = 127.5 + 127.5 * sin(X);

image(sinewaveimage);

colormap(gray(256));

Now you can see the sine wave plotted as an image.

There is a mapping between intensity values and the color that is displayed (color maps aka color tables)

This tiny example has already raised an important issue. The pixel values determine how intensity images are displayed on the screen. Suppose we had plotted:

sinewaveimage2 = sin(X);

image(sinewaveimage2);

Because the function sin(X) varies between -1 and 1, we can't really see anything. When the function is less than 0, we see black; where the function is between 0 and 1, we see 1/256th of the maximum brightness that the screen can produce. Now try this:

sinewaveimage3 = 255 * sin(X);

image(sinewaveimage3);

Now you can see that everywhere sin(X) is 0 or negative, the pixels are drawn in black (that is, the lowest color map entry) because these values are below the lower limit of the color map (which has entries ranging from 1 to 256), and the function maxes out at the brightest luminance the monitor can produce (the 256th entry).

It is important to tailor our image brightness values so they line up with the display mode we are using (in this case, a color map). We can have Matlab do this for us with the function imagesc (image scale). First, read the help for imagesc

help imagesc

Now try using it

figure;

imagesc(sinewaveimage2);

colormap(gray(256));

Now the image is scaled so that its maximum value corresponds to the highest entry in the colormap (entry 256) and the minimum value to the lowest value in the color table (entry 1).

The color map (more commonly called a color table or a color look-up table (CLUT) ) is a set of instructions to the computer of how to display each value in the image. Let's examine a small color table that is built into Matlab:

gray(4) % this asks for a gray color table with 4 entries

jet(4) % this asks for a blue-to-red (cold-to-warm) color table with 4 entries

Color maps for computer monitors are 3 x N (where N is the number of entries) because they indicate how strongly each of the 3 color signals (red, green, and blue) on the monitor should be activated. Why are there only 3 colors? This is a simplification (lucky for engineers) that we will get to in the homework...humans have only 3 types of cone photoreceptors, so all we need to reproduce any image is some means of activating these 3 types independently, hence the 3 colors.

Let's return to our simple manually-created image above:

A = [ 20 40 60 80; 100 120 140 160; 180 200 220 240];

A, % look at A to see it in matrix form

figure;

image(A);

colormap(gray(256));

Let's modify the color table to change some colors:

map = colormap; % returns the current color map

map, % look at it, it's a big matrix 3 x 256

map(20,:) = [ 1 0 0]; % let's make entry 20 (image value 20) "red"

map(40,:) = [ 0 1 0]; % let's make entry 40 (image value 40) "green"

map(60,:) = [ 0 0 1]; % let's make entry 60 (image value 60) "blue"

colormap(map); % reinstall our color map

"Intensity images" are really part of a larger class of images called index-based images

So we have seen that images for human consumption can be represented with 2 pieces of information: 1) a set of values that refer to entries in 2) a color table. These types of images are called index-based images or index images; the image is a set of index values for the computer to look up in the specified color table.

Mathematically, you may notice that what I have called "intensity images" are just a special case of indexed-based images; in an intensity image, I have taken the value of the image at a given point to have a physical meaning, but, as far as the computer is concerned, the value simply describes which value of the color map to use to display that pixel. When you use software such as Photoshop, GraphicConverter, or even Matlab, there is no special class of image called "intensity images", these are just indexed-based images.

To drive this home, let's "animate" the color map, where we progressively change the definition of which color should be used to represent each value in the index-based image.

map = jet(256);

for i=1:256,

newmap = map([i:256 1:i-1],:); % grab the color map at different places

pause(0.1); % pause for 1/10 second

colormap(newmap);

end;

Rotating the color map like this changes the color to which each image value corresponds.

Full color images - the RGB format

Now we're going to talk about another common image format that is different from indexed-images: the RGB format, where the image is represented in 3 channels with red, green, and blue information.

In the old days, index-based images were quite useful because they are more compact memory-wise than storing full information about color. Consider a 512 x 512 image with 256 colors in R, G, B format. If you used a single 8-bit number (equal to 1 byte, or a value between 0 and 255) for each pixel and a color table of 256 entries, then the image would require 512 x 512 bytes = 262144 bytes. Each 3-color entry in the color table would require 3 bytes, so the entire color table would require 3 x 256 bytes = 768 byes. Thus, the total image would require 262144 + 768 = 262912 bytes (or 256.75 kB). On the other hand, if you used one byte to represent each of the R, G, and B values, then you would require 512 x 512 x 3 = 786,432 bytes (768 kB) of information. So indexed-based images require less memory than images that specify full color information. Further, if you had to update the image that was on the screen, it was a lot faster to draw 256.75 kB of data rather than 768 kB of information. In the old days, this was fine.

But consider a modern computer operating system; if you just look at the screen, there are many full color windows and icons. If you had to represent each one with an image and color table, then the video card would have to process multiple color tables. There's no reason this couldn't be done, but engineers have decided instead, now that chips are faster and memory is easier to come by, to simplify the situation and to represent each pixel with 3 bytes of color information. Matlab allows you to read images that are specified this way.

myrgbimage = imread('Griffins_toys.jpg');

figure;

image(myrgbimage);

Let's examine the size of this image:

size(myrgbimage)

You can see that this image is size NxMx3. Each NxM matrix represents how intense the image should be displayed on the red, green, and blue channels, respectively. Let's look at the 3 channels.

figure;

subplot(2,2,1);

image(myrgbimage);

title('full image');

subplot(2,2,2);

image(myrgbimage(:,:,1));

colormap(gray(256));

title('red channel');

subplot(2,2,3);

image(myrgbimage(:,:,2));

title('green channel');

subplot(2,2,4);

image(myrgbimage(:,:,3));

title('blue channel');

Now you can see image broken down into its 3 channels. Let's do one experiment that shows the usefulness of 3-channel images for creating the final output. Suppose you wanted to show each channel in its native color. You could do this by plotting the 3 2-dimensional intensity channels, and assigning each one a different colormap. For example, we could create a red linear color map with the function linspace (see help linspace):

redmap = [ linspace( 0, 1, 256)' zeros(256,1) zeros(256,1)]

Notice that this matrix is 256x3, and is 0 except for its first column, where values steadily increase. We can also define analogous green and blue color maps:

greenmap = [ zeros(256,1) linspace( 0, 1, 256)' zeros(256,1) ]

bluemap = [ zeros(256,2) linspace( 0, 1, 256)' ]

Now let's try our example:

figure;

subplot(2,2,1);

image(myrgbimage);

title('full image');

subplot(2,2,2);

image(myrgbimage(:,:,1));

colormap(redmap);

title('red channel');

subplot(2,2,3);

image(myrgbimage(:,:,2));

title('green channel');

colormap(greenmap);

subplot(2,2,4);

image(myrgbimage(:,:,3));

title('blue channel');

colormap(bluemap);

What happened? You've discovered a limitation in Matlab's figure function...each figure can have only 1 color map. So each time you try to use a different indexed-based image, you end up overwriting the previous color table. What's the right way to do this?

One way is to convert the 2-dimensional intensity images to their own rgb images, like so:

sz = size(myrgbimage)

redimage = myrgbimage(:,:,1);

redimage(:,:,2) = zeros(sz(1),sz(2));

redimage(:,:,3) = zeros(sz(1),sz(2));

You can see that this image is NxMx3:

size(redimage)

Let's try using this image:

figure;

image(redimage);

subplot(2,2,1);

image(myrgbimage);

title('full image');

subplot(2,2,2);

image(redimage);

Now the image shows in full color, and you can use any number of sub-axes. There is a Matlab function ind2rgb (see help ind2rgb) that allows you to perform this conversion for shorter code:

figure;

subplot(2,2,1);

image(myrgbimage);

title('full image');

subplot(2,2,2);

image(ind2rgb(myrgbimage(:,:,1),redmap));

title('red channel');

subplot(2,2,3);

image(ind2rgb(myrgbimage(:,:,2),greenmap));

subplot(2,2,4);

image(ind2rgb(myrgbimage(:,:,3),bluemap));

One final note: the printed page and the CMYK color space

We would be remiss if we didn't discuss the printed page while we are discussing different image formats. The color space that can be represented using light-emitting devices is very different from the space that can be represented using reflected light and common printer inks. This is why many images appear different on your screen as compared to the printed page. The printer ink space is more limited than the light space -- that is, there are colors you can produce on the computer that you cannot produce on paper with the standard printer inks that your printer and journal use. If you are preparing a document for publication, and it is important to you that it look the same on a computer screen as it does on paper, use a program like Photoshop or Illustrator to convert the image to the CMYK color space; you can read about it here.

References

black-and-white-cow-2-3.tif is from public_domain_photos.com and Griffins_toys.jpg is courtesy my friend Elizabeth N. Johnson.