Forums

C3088 CMOS Imaging Sensor Questions

Started by Unknown July 13, 2006
Hi all,

I'm looking to purchase the C3088 CMOS imaging sensor.  I plan to
interface it with a PIC microcontroller in order to do some image
processing on-board my robot.  I was wondering if someone could take a
moment and answer a couple of questions for me.

(1) Would this be reasonable code for capturing a frame of data from
the C3088

row=0;
col=0;

// generate interrupt if VSYNC goes high
#INT vsync_isr()
{
	while(VSYNC);	// wait until VSYNC goes low
	while(!HREF);	// wait until HREF goes high
	while(HREF)	// while HREF remains high
	{
		while(!PCLK); // wait until PCLK goes high
		image[row][col] = Y7*128+Y6*64+Y5*32+Y4*16+Y3*8+Y2*4+Y1*2+Y0;
		while(PCLK);  // wait until PCLK goes low

		if(row<356) row++;
		else row=0;
	}

	if(col<292) col++;
	else col=0;
}



(2) I know the clock on the C3088 (I believe it's called the pixel
clock or PCLK) has to be synced up with the clock on my PIC
microcontroller.  What's the best way of going about this?  If I use a
20 MHz external clock for my PIC, can I also feed that same clock line
into the CMOS sensor?

(3) Any other recommendations for a digital CMOS imaging sensor besides
the C3088?

Thanks in advance,
-weg

On 13 Jul, in article
     <1152823068.907808.161790@p79g2000cwp.googlegroups.com>
     weg22@drexel.edu wrote:

>Hi all, > >I'm looking to purchase the C3088 CMOS imaging sensor. I plan to >interface it with a PIC microcontroller in order to do some image >processing on-board my robot. I was wondering if someone could take a >moment and answer a couple of questions for me. > >(1) Would this be reasonable code for capturing a frame of data from >the C3088
Which uses the Omnivision OV6620 sensor.
>row=0; >col=0; > >// generate interrupt if VSYNC goes high >#INT vsync_isr() >{ > while(VSYNC); // wait until VSYNC goes low > while(!HREF); // wait until HREF goes high > while(HREF) // while HREF remains high > { > while(!PCLK); // wait until PCLK goes high > image[row][col] = Y7*128+Y6*64+Y5*32+Y4*16+Y3*8+Y2*4+Y1*2+Y0; > while(PCLK); // wait until PCLK goes low > > if(row<356) row++; > else row=0; > } > > if(col<292) col++; > else col=0; >}
Assuming the PIC can keep up with normally 8.68MHz * 2 bytes transfers during an image caputure, what else do you want it to do? Because it will be almost permanently locked in that interupt routine doing NOTHING else. Are you sure your PIC has enough RAM for your program variables and at least a buffer 356 x 292 x 2 bytes EACH! Your PIC does have 256kB of RAM at least I take it. Normally you need at least two buffers for image processing on whole images so double that memory requirement to 0.5MB! You will NOT be able to process on the fly.
>(2) I know the clock on the C3088 (I believe it's called the pixel >clock or PCLK) has to be synced up with the clock on my PIC >microcontroller. What's the best way of going about this? If I use a >20 MHz external clock for my PIC, can I also feed that same clock line >into the CMOS sensor?
So you end up with 10MHz burst data to keep up with, which if done by software means like above, you might get one pixel a line! Just because you can clock the PIC at 20Mhz does not mean it could do even one of your high level instructions before another pixel was LOST! Why a PIC? Work out what image capture size and update rate you need first. Work out what algorithms you need to run and how much memory and time they are likely to use, then when they need to complete by. When you have worked all that out, THEN and ONLY THEN think about how you are going to capture it, into what memory, THEN see what is the best micro match to perform the function.
>(3) Any other recommendations for a digital CMOS imaging sensor besides >the C3088?
Nothing wrong with the sensor, PIC cannot do that sort of image processing without complicated external memory and/or excessively small windowing of the data, you are making a rod for your own back by trying to squeeze the Atlantic Ocean into a pint pot!
>Thanks in advance, >-weg > >
-- Paul Carpenter | paul@pcserviceselectronics.co.uk <http://www.pcserviceselectronics.co.uk/> PC Services <http://www.gnuh8.org.uk/> GNU H8 & mailing list info <http://www.badweb.org.uk/> For those web sites you hate
weg22@drexel.edu wrote:

> Hi all, > > I'm looking to purchase the C3088 CMOS imaging sensor. I plan to > interface it with a PIC microcontroller in order to do some image > processing on-board my robot. I was wondering if someone could take a > moment and answer a couple of questions for me. > > (1) Would this be reasonable code for capturing a frame of data from > the C3088 > > row=0; > col=0; > > // generate interrupt if VSYNC goes high > #INT vsync_isr() > { > while(VSYNC); // wait until VSYNC goes low > while(!HREF); // wait until HREF goes high
No. You should run through an ISR as fast as possible. While loops like these should never occur in an ISR. Ian
weg22@drexel.edu wrote:
> Hi all, > > I'm looking to purchase the C3088 CMOS imaging sensor. I plan to > interface it with a PIC microcontroller in order to do some image > processing on-board my robot. I was wondering if someone could take a > moment and answer a couple of questions for me. > > (1) Would this be reasonable code for capturing a frame of data from > the C3088
<snip> Well, if you're interested in switching over to Atmel's line of AVR microcontrollers, this is certainly feasible. Check out: http://www.jrobot.net/Projects/AVRcam.html This open-source system was developed for testing out various embedded image processing ideas. Paul C's comments are somewhat on target though. It would probably make sense to sit down and figure out what your frame-rate requirements are, what you want to do with the data, and how you plan on actually getting access to the data. There are schematics, source code, and a Forums setup to discuss this topic at the above website. Stop by... Regards, John O www.jrobot.net
> Assuming the PIC can keep up with normally 8.68MHz * 2 bytes transfers > during an image caputure, what else do you want it to do? Because it will be > almost permanently locked in that interupt routine doing NOTHING else.
I'd like to (a) take spatial (x & y), and temporal derivatives and (b) incorporate thos derivatives into another equation.
> Are you sure your PIC has enough RAM for your program variables and at least > a buffer 356 x 292 x 2 bytes EACH! Your PIC does have 256kB of RAM at least > I take it. Normally you need at least two buffers for image processing on > whole images so double that memory requirement to 0.5MB!
Can you please explain why you need two buffers when doing image processing (I'm new at this)? My PIC has 3.94 KB of RAM and I probably won't be processing the entire image...more like a 100 x 292 image.
> You will NOT be able to process on the fly.
I didn't think I'd be able to get an algorithm working at 30 fps, but I was hoping for at least 5. Do you still not think this is feasible?
> Why a PIC?
I'm use to working with them...besides is the AVR that much different than the PIC I'm working with (PIC18F8722)?
> Work out what image capture size and update rate you need first. > > Work out what algorithms you need to run and how much memory and time they > are likely to use, then when they need to complete by. > > When you have worked all that out, THEN and ONLY THEN think about how > you are going to capture it, into what memory, THEN see what is the best > micro match to perform the function.
Great suggestions...thanks!
<weg22@drexel.edu> wrote in message
news:1152878792.710362.210770@i42g2000cwa.googlegroups.com...
> Can you please explain why you need two buffers when doing image > processing (I'm new at this)? My PIC has 3.94 KB of RAM and I probably > won't be processing the entire image...more like a 100 x 292 image.
Ehhm, 100 x 292 is 29kbytes. How is that going to fit into 3.94kbytes? Regarding the sampling: I have used a Motorola DSP (DSP56303) to sample a 5MHz pixel rate, using an interrupt handler written in assembly that only read one line at a time. This DSP ran at 80MHz. I very much doubt that a PIC would be able to do the same, especially when programmed in C.... Meindert
On 14 Jul, in article
     <1152878792.710362.210770@i42g2000cwa.googlegroups.com>
     weg22@drexel.edu wrote:

>> Assuming the PIC can keep up with normally 8.68MHz * 2 bytes transfers >> during an image caputure, what else do you want it to do? Because it will be >> almost permanently locked in that interupt routine doing NOTHING else. > >I'd like to (a) take spatial (x & y), and temporal derivatives and (b) >incorporate thos derivatives into another equation.
Which require LOTS of computation time and memory to process. Especially as spatial (x & y) in image processing can mean a lot of things depending on how many points to be found and how those points are to be recognised.
>> Are you sure your PIC has enough RAM for your program variables and at least >> a buffer 356 x 292 x 2 bytes EACH! Your PIC does have 256kB of RAM at least >> I take it. Normally you need at least two buffers for image processing on >> whole images so double that memory requirement to 0.5MB! > >Can you please explain why you need two buffers when doing image >processing (I'm new at this)?
To maintain an reasonable speed on frame update or just for processing. Often applications have dedicated hardware to grab one frame, whilst working on a copy of a previous frame. Many algorithms need two buffers for part or result of process, having an input buffer and an output (or temporary buffer), the input buffer may well be used more than once depending on output buffer contents.
> My PIC has 3.94 KB of RAM and I probably >won't be processing the entire image...more like a 100 x 292 image.
The basic problem is you will NOT be able to use the PIC directly to probably get more than two pixels per video line. The video rate is too fast for the PIC and the many instructions required to do the transfer to keep up. Even in machine code! Some people use algorithms that ASSUME that if they take several pictures and grab at different times they can build up an image over many frames. This method can SOMETIMES work to a limited extent as lighting variations let alone movement in the image over time greatly reduces the accuracy and resolution of data.
>> You will NOT be able to process on the fly. > >I didn't think I'd be able to get an algorithm working at 30 fps, but I >was hoping for at least 5. Do you still not think this is feasible?
You will be lucky to get ONE per second. The PIC is the wrong beast to do this, image processing digitally needs lots of dataspace and processors that can do 8/16/32 bit operations for quite a few of the operations. Multibyte arithemtic will not cut it for any form of control algorithm as the overhead of doing one 16bit add many thousands of times slows the process to almost glacial speed.
>> Why a PIC? > >I'm use to working with them...besides is the AVR that much different >than the PIC I'm working with (PIC18F8722)?
Having larger memory areas and other attributes for the processing and I believe things like DMA make it a lot easier. Video is not like serial data there is a lot of it, in short spaces of time and repeatedly. Miss one pixel and everything afterwards is meaning less. I have done some highly compute intensive applications on H8 which was grabbing one frame then doing full frame processing, with speeds by lookup tables to produce enhancement maps, that involved all sorts of complicated maths. This device had PLD controlled frame grab (monochrome) into dual ported RAM, and running on 20MHz clock took about 2 seconds to run! H8 was using its 16 and 32bit integer operations and avoiding floats and other nasties. In that application the 2 second wait was faster than what the operator was doing at the time. Finding spatial information and object recognition to find the spatial points is a similar amount of computations if not more on a worse architecture, is asking for trouble.
>> Work out what image capture size and update rate you need first. >> >> Work out what algorithms you need to run and how much memory and time they >> are likely to use, then when they need to complete by. >> >> When you have worked all that out, THEN and ONLY THEN think about how >> you are going to capture it, into what memory, THEN see what is the best >> micro match to perform the function. > >Great suggestions...thanks!
You will need a high end 16bit or more probably 32bit processor like ARM with fast memory access to get anywhere near your required update rate of 5 fps. Even then a dedicated piece of hardware to grab the image into a dual-ported buffer will speed things up immensely. -- Paul Carpenter | paul@pcserviceselectronics.co.uk <http://www.pcserviceselectronics.co.uk/> PC Services <http://www.gnuh8.org.uk/> GNU H8 & mailing list info <http://www.badweb.org.uk/> For those web sites you hate
Paul Carpenter wrote:
"[..]
Are you sure your PIC has enough RAM for your program variables and at least
a buffer 356 x 292 x 2 bytes EACH! Your PIC does have 256kB of RAM at least
I take it. Normally you need at least two buffers for image processing on
whole images so double that memory requirement to 0.5MB!
[..]"


On Fri, 14 Jul 2006 weg22@drexel.edu wrote:

"[..]

Can you please explain why you need two buffers when doing image
processing (I'm new at this)?"

Well, you could read about the technique of double buffering in animation 
for DOS games and OpenGL applications and other graphical applications, in 
which an older image is displayed while a newer image is being computed. 
After the newer image has been completed, the two buffers can be swapped 
very quickly so that drawing the new buffer on the screen is performed 
more smoothly.

I doubt you want to do this, but perhaps Paul Carpenter was thinking of 
something similar to this (e.g. your robot using the older buffer as its 
interpretation of the enviornment while making a newer image in the newer 
buffer).

"  My PIC has 3.94 KB of RAM and I probably
won't be processing the entire image...more like a 100 x 292 image.

[..]"

The product of 100 multiplied by 292 is 29200. I am not familiar with the 
sensor you mentioned, but if it has just 8 bit grayscale resolution you 
will need 29200 bytes (which is over 28KB) for an image (excluding data 
structure overhead).
"Meindert Sprang" <ms@NOJUNKcustomORSPAMware.nl> wrote in message
news:12bf6u8bm26j88e@corp.supernews.com...
> <weg22@drexel.edu> wrote in message > news:1152878792.710362.210770@i42g2000cwa.googlegroups.com... > > Can you please explain why you need two buffers when doing image > > processing (I'm new at this)? My PIC has 3.94 KB of RAM and I probably > > won't be processing the entire image...more like a 100 x 292 image. > > Ehhm, 100 x 292 is 29kbytes. How is that going to fit into 3.94kbytes? > > Regarding the sampling: I have used a Motorola DSP (DSP56303) to sample a > 5MHz pixel rate, using an interrupt handler written in assembly that only > read one line at a time. This DSP ran at 80MHz. I very much doubt that a
PIC
> would be able to do the same, especially when programmed in C.... > > Meindert
I don't know if it will easily interface to your PIC, but I used a IDT 7205 FIFO to capture sub-windowed images from a camera. Once a sub-frame is in the FIFO, you can read it out non-destructively for multiple passes at the data. Gruesome, but better than nothing. I was using an AtMega64 at the time. If you can afford the princely sum of, of, $10 or $20, you could get a real processor with DMA and an address space that could eat the whole image. You still need some serious horsepower to do anything with it at 5 fps. BTW: can't a PIC read an entire port as one byte? Why did you think you needed code like this?
> image[row][col] = Y7*128+Y6*64+Y5*32+Y4*16+Y3*8+Y2*4+Y1*2+Y0;
instead of imagePtr++ = PORTY; Bob
On Friday, in article
     <20060714161612.A1141@docenti.ing.unipi.it>
     Colin_Paul_Gloster@ACM.org "Colin Paul Gloster" wrote:

>Paul Carpenter wrote: >"[..] >Are you sure your PIC has enough RAM for your program variables and at least >a buffer 356 x 292 x 2 bytes EACH! Your PIC does have 256kB of RAM at least >I take it. Normally you need at least two buffers for image processing on >whole images so double that memory requirement to 0.5MB! >[..]" > > >On Fri, 14 Jul 2006 weg22@drexel.edu wrote: > >"[..] > >Can you please explain why you need two buffers when doing image >processing (I'm new at this)?" > >Well, you could read about the technique of double buffering in animation >for DOS games and OpenGL applications and other graphical applications, in >which an older image is displayed while a newer image is being computed. >After the newer image has been completed, the two buffers can be swapped >very quickly so that drawing the new buffer on the screen is performed >more smoothly. > >I doubt you want to do this, but perhaps Paul Carpenter was thinking of >something similar to this (e.g. your robot using the older buffer as its >interpretation of the enviornment while making a newer image in the newer >buffer).
That is the main reason as well as quite a few algorithms have input and output buffer (especially for object recognition). Dual buffering with separate acquire and processing buffers will speed up the frame processing rate.
>" My PIC has 3.94 KB of RAM and I probably >won't be processing the entire image...more like a 100 x 292 image. > >[..]" > >The product of 100 multiplied by 292 is 29200. I am not familiar with the >sensor you mentioned, but if it has just 8 bit grayscale resolution you >will need 29200 bytes (which is over 28KB) for an image (excluding data >structure overhead).
Actually it is a colour sensor producing YUV data normally as 16bit 4:2:2 so as 16bit data it will need twice the information, and if converted to RGB THREE times the space. In reality most robotic vision only need monochrome details unless they actually need to see colour information of objects (e.g. print colour checks, fruit picking, colour area of right colour means bottle/can has label on correctly). -- Paul Carpenter | paul@pcserviceselectronics.co.uk <http://www.pcserviceselectronics.co.uk/> PC Services <http://www.gnuh8.org.uk/> GNU H8 & mailing list info <http://www.badweb.org.uk/> For those web sites you hate