Forums

Simple Autoexposure algorithm for an Image sensor

Started by Bryan October 6, 2004
Hi,

I'm involved in a project where we are building a still image capture
device. We have a CMOS image sensor (Pixart PAS106B) connected via I2C
to a philips lpc2106 (Arm 7) microcontroller.

I'm able to capture images without trouble, however, I notice -
depending on the ambient light - my image is either too light or too
dark. To correct this problem, I want to write a small (hopefully
simple) and fast autoexposure routine to adjust the exposure setting
and hopefully get a more consistent image when the light varies.

I was hoping someone might be able to point me to some resources
(books, news groups, source code, people, etc) that might be able to
help me understand the methods and best practices for accomplishing
this.

I was hoping that I could just get a "luminance" reading from the
image sensor, but I guess I can't. So, I'm pretty sure that I need to
evaluate (somehow) the light/darkness of the image, change the image
sensor's exposre settings, grab another frame, test it, and so on.

If anyone has any experience or knows where I might go to get a grip
on the best way to do this, I'd be grateful.

Thanks
 
Bryan
Bryan wrote:

> Hi, > > I'm involved in a project where we are building a still image capture > device. We have a CMOS image sensor (Pixart PAS106B) connected via I2C > to a philips lpc2106 (Arm 7) microcontroller.
<snip>
> If anyone has any experience or knows where I might go to get a grip > on the best way to do this, I'd be grateful. >
I had this same problem to work on for a commercial product. I couldn't find any reference material directly on the subject so I just implemented a PID controller. I did an averaging of selected pixels to get a measure of the tone of the scene. This approach (reflective metering) is of course limited by assuming that the scene is of average reflectance. It won't work too well for highly reflective or unreflective scenes. Look for any good photography book for a more detailed explanation. I also suggest looking for control theory books (or on the web) for details on closed loop control. Felix
On 6 Oct, in article
     <a01fd7ec.0410061138.581728eb@posting.google.com>
     clapper@bluewin.ch "Bryan" wrote:
>Hi, > >I'm involved in a project where we are building a still image capture >device. We have a CMOS image sensor (Pixart PAS106B) connected via I2C >to a philips lpc2106 (Arm 7) microcontroller. > >I'm able to capture images without trouble, however, I notice - >depending on the ambient light - my image is either too light or too >dark. To correct this problem, I want to write a small (hopefully >simple) and fast autoexposure routine to adjust the exposure setting >and hopefully get a more consistent image when the light varies.
Can you take some images BEFORE the capture you want or continuously to calibrate the image levels?
>I was hoping someone might be able to point me to some resources >(books, news groups, source code, people, etc) that might be able to >help me understand the methods and best practices for accomplishing >this.
Suggest looking at either additional light sensor, or see how video camera AGC circuits work on averaging the light as they go.
>I was hoping that I could just get a "luminance" reading from the >image sensor, but I guess I can't. So, I'm pretty sure that I need to >evaluate (somehow) the light/darkness of the image, change the image >sensor's exposre settings, grab another frame, test it, and so on.
This will only tell you how much luminance has been reflected or directly transmitted to the sensor, a specific light sensor will give more general light levels.
>If anyone has any experience or knows where I might go to get a grip >on the best way to do this, I'd be grateful.
Look up Video AGC (Automatic Gain Control) implementations. -- Paul Carpenter | paul@pcserviceselectronics.co.uk <http://www.pcserviceselectronics.co.uk/> PC Services <http://www.gnuh8.org.uk/> GNU H8 & mailing list info <http://www.badweb.org.uk/> For those web sites you hate
Bryan <clapper@bluewin.ch> wrote:

> I'm able to capture images without trouble, however, I notice - > depending on the ambient light - my image is either too light or too > dark.
Here's a nasty question: how do you know that? Who/what defines when an image is "too" light or dark?
> To correct this problem, I want to write a small (hopefully > simple) and fast autoexposure routine to adjust the exposure setting > and hopefully get a more consistent image when the light varies.
Ouch. You may not be aware of it, but you've just introduced another criterion which actually contradicts the previous ones to some extent. Anyway, what do you plan on being consistent *with*, then? About the only auto-exposure algorithm you could use that uses no additional sensors at all, and no knowledge about the sensitivity curve of your image sensor either, is: repeat { l1 := brightnees of <n1>th lightest pixel in the entire image l2 := brightness of <n2>th darkest pixel in the entire image modify exposure by <factor>*(l1+l2 - (black + white)) } until (exposure correction < <tolerance>) if ((l1 - l2) < <min_contrast>) complain The problem is that the two first steps grow quite complex if <n1> and <n2> are not very small numbers, and that you need at least one complete "dummy" shot plus analysis time just to calibrate. The repeat loop is, of course, nothing else but a crude renditon of a closed-loop control algorithm (only the 'P' part of a PID controller), with (l1+l2)/2 as the controlled term, which you want to be equal to 50% gray. -- Hans-Bernhard Broeker (broeker@physik.rwth-aachen.de) Even if all the snow were burnt, ashes would remain.
On 6 Oct 2004 12:38:06 -0700, clapper@bluewin.ch (Bryan) wrote:

>I'm involved in a project where we are building a still image capture >device. We have a CMOS image sensor (Pixart PAS106B) connected via I2C >to a philips lpc2106 (Arm 7) microcontroller.
How many frames/scan you transfer over the I2C link ? Does the microcontroller have enough RAM to store the image to 8-16 bits/pixel? Shooting multiple frames and integrating each frame pixel by pixel into the frame store will increase the signal to noise ratio as more and more frames are added. Stop the imagining, when the first pixel hits the maximum value. The frame update rate should be such that the brightest pixel does not saturate the sensor or ADC, but on the other hand, there should be enough (1 LSB) dark current noise to act as dither. I have seen some astronomical imaging, in which in the beginning only a few bright stars were visibly, but after a few seconds more and more details became visible, when more frames were added. Paul
Paul Keinanen wrote:
> On 6 Oct 2004 12:38:06 -0700, clapper@bluewin.ch (Bryan) wrote: > >> I'm involved in a project where we are building a still image capture >> device. We have a CMOS image sensor (Pixart PAS106B) connected via I2C >> to a philips lpc2106 (Arm 7) microcontroller. > > How many frames/scan you transfer over the I2C link ? Does the > microcontroller have enough RAM to store the image to 8-16 bits/pixel? > > Shooting multiple frames and integrating each frame pixel by pixel > into the frame store will increase the signal to noise ratio as more > and more frames are added. Stop the imagining, when the first pixel > hits the maximum value. The frame update rate should be such that the > brightest pixel does not saturate the sensor or ADC, but on the other > hand, there should be enough (1 LSB) dark current noise to act as > dither. > > I have seen some astronomical imaging, in which in the beginning only > a few bright stars were visibly, but after a few seconds more and more > details became visible, when more frames were added.
This is much like the old CAT (computer of average transients) that I built 40 years ago. On a sync signal (internal or external) it inputs a time based set of input values, and sums these over some total number of scans. You have to limit the number to avoid overflow. The result is the noise averages out, and after a few (for appropriate values of few) the signal appears out of the noise. With the appropriate display you can watch the signal appearing out of the noise during the processing. The CAT was useful for such things as picking out brainwaves resulting from external stimulae, such as a sharp noise. It is much like a sampling scope. -- Chuck F (cbfalconer@yahoo.com) (cbfalconer@worldnet.att.net) Available for consulting/temporary embedded and systems. <http://cbfalconer.home.att.net> USE worldnet address!
clapper@bluewin.ch (Bryan) wrote in message news:<a01fd7ec.0410061138.581728eb@posting.google.com>...
> Hi, > > I'm involved in a project where we are building a still image capture > device. We have a CMOS image sensor (Pixart PAS106B) connected via I2C > to a philips lpc2106 (Arm 7) microcontroller. > > I'm able to capture images without trouble, however, I notice - > depending on the ambient light - my image is either too light or too > dark. To correct this problem, I want to write a small (hopefully > simple) and fast autoexposure routine to adjust the exposure setting > and hopefully get a more consistent image when the light varies. > > I was hoping someone might be able to point me to some resources > (books, news groups, source code, people, etc) that might be able to > help me understand the methods and best practices for accomplishing > this. > > I was hoping that I could just get a "luminance" reading from the > image sensor, but I guess I can't. So, I'm pretty sure that I need to > evaluate (somehow) the light/darkness of the image, change the image > sensor's exposre settings, grab another frame, test it, and so on. > > If anyone has any experience or knows where I might go to get a grip > on the best way to do this, I'd be grateful.
The first thing to keep in mind is that "auto exposure" from behind the lens is not actually generally possible, unless you some idea of what you're looking at. The problem is that unless you know how much light is *supposed* to be reflected, there's no way of telling how brightly the object is lit, and therefore you can't determine the correct exposure (which is properly based on the level of illumination, *not* the level of reflection). You'll notice that serious photographers are always running around with incident light meters, which measure the intensity of light falling on the object(s) in question, and they compute their proper exposure from there. In short, a certain intensity of light falling on a black object should get the same (film/sensor) exposure as the same amount of light falling on a white object, even though there will be much more light reflected towards the camera in the latter case. So the best you can do is fake it. The simplest scheme, used for many decades in cameras with built in exposure meters, is to assume that the scene has a certain reflectivity, and just measure the intensity of all the light coming through the lens, or perhaps just the center portion. The standard in photography is to balance against an 18% gray card (basically just a gray card of the correct density to reflect 18% of the incident light - available at any decent photo supply place). The "18% gray" card is supposed to be equivalent in density to a "typical" scene (which explains why it's used). For an application like this, you'd calibrate your sensor against such a card, and then adjust the exposure of the image by adjusting the exposure so that the entire brightness, averaged over the entire sampling region, came out to the same value as you calibrated for the gray card. Trivially, convert all the pixel values to a linear scale, add them all up, divide by the number of pixels, and adjust the exposure by the ratio of that number to the value you'd see for the reference card. One simple enhancement is to use a center-weighted metering scheme, where you bias the exposure towards the center of the frame. High end cameras perform quite complex schemes to try and "understand" a frame enough to measure it accurately, often breaking up the scene into many individual segments, and applying some balancing algorithm to those. Some even go so far as to attempt image analysis to try and recognize objects and apply reflectivity values to them. I'd start with something simple, maybe a basic center-weighted scheme, where the inner ninth (the middle third in both dimensions, or something more-or-less in that range) counts for two thirds of the exposure weight, and the remainder of the frame for the other third, and play from there. Given that responses are not real linear, this may cycle occur over several frame before it stabilizes (and you probably want to add some sort of damping to this function). I don't know if it's an issue for your application, but one thing to watch out for is that if you're far enough off scale to lose all detail (either via extreme under or overexposure), you'll want to have a fast step function to try and get the exposure back into a "reasonable" range.