ISTR playing with de-encapsulated DRAMs as image sensors back in school (DRAM being relatively new technology, then). But, most cameras seem to have (bit- or word-) serial interfaces nowadays. Are there any (mainstream/high volume) devices that "look" like a chunk of memory, in their native form?
Camera interfaces
Started by ●December 29, 2022
Reply by ●December 29, 20222022-12-29
On 12/29/2022 15:16, Don Y wrote:> ISTR playing with de-encapsulated DRAMs as image sensors > back in school (DRAM being relatively new technology, then). > > But, most cameras seem to have (bit- or word-) serial interfaces > nowadays. Are there any (mainstream/high volume) devices that > "look" like a chunk of memory, in their native form? >Hah, Don, consider yourself lucky if you find a camera you have enough documentation to use at all, serial or whatever. The MIPI standards are only for politburo members (last time I looked you need to make several millions annually to be able to *apply* for membership, which of course costs thousands, annually again). Not use about USB, perhaps USB cameras are covered in the standard (yet to deal with that one).
Reply by ●December 29, 20222022-12-29
On 12/29/22 8:16 AM, Don Y wrote:> ISTR playing with de-encapsulated DRAMs as image sensors > back in school (DRAM being relatively new technology, then). > > But, most cameras seem to have (bit- or word-) serial interfaces > nowadays. Are there any (mainstream/high volume) devices that > "look" like a chunk of memory, in their native form? >Using a DRAM in that manner would only give you a single bit value for each pixel (maybe some more modern memories store multiple bits in a cell so you get a few grey levels). There are some CMOS sensors that let you address pixels individually and in a random order (like you got with the DRAM) but by its nature, such a readout method tends to be slow, and space inefficient, so these interfaces tend to be only available on smaller camera arrays. That is why most sensors read out via row/column shift registers to a pixel serial (maybe multiple pixels per clock) output, and if the camera includes its own A/D conversion, might serialize the results to minimize interconnect.
Reply by ●December 29, 20222022-12-29
On 12/29/22 8:33 AM, Dimiter_Popoff wrote:> On 12/29/2022 15:16, Don Y wrote: >> ISTR playing with de-encapsulated DRAMs as image sensors >> back in school (DRAM being relatively new technology, then). >> >> But, most cameras seem to have (bit- or word-) serial interfaces >> nowadays. Are there any (mainstream/high volume) devices that >> "look" like a chunk of memory, in their native form? >> > > Hah, Don, consider yourself lucky if you find a camera you have > enough documentation to use at all, serial or whatever. > > The MIPI standards are only for politburo members (last time I looked > you need to make several millions annually to be able to *apply* > for membership, which of course costs thousands, annually again).If you are looking for the very latest standards, yes. Enough data is out there to handle a lot of basic MIPI operations. Since the small player isn't going to be trying to implement the low level interface themselves (or at least shouldn't be trying to), unless you are trying to work with a bleeding edge camera (which you probably can't actually buy if you are a small player) you can tend to find enough information to use the camera. My experiance is if you can actually buy the camera normally, there will be the data available to use it. The big problem is "Grey Market" cameras, via unauthorized distributors you are at the mercy of the distributor to get you the needed data.> > Not use about USB, perhaps USB cameras are covered in the standard > (yet to deal with that one).There is a USB video standard, and many USB cameras can just be plugged in and used.
Reply by ●December 29, 20222022-12-29
On 12/29/2022 19:21, Richard Damon wrote:> On 12/29/22 8:33 AM, Dimiter_Popoff wrote: >> On 12/29/2022 15:16, Don Y wrote: >>> ISTR playing with de-encapsulated DRAMs as image sensors >>> back in school (DRAM being relatively new technology, then). >>> >>> But, most cameras seem to have (bit- or word-) serial interfaces >>> nowadays. Are there any (mainstream/high volume) devices that >>> "look" like a chunk of memory, in their native form? >>> >> >> Hah, Don, consider yourself lucky if you find a camera you have >> enough documentation to use at all, serial or whatever. >> >> The MIPI standards are only for politburo members (last time I looked >> you need to make several millions annually to be able to *apply* >> for membership, which of course costs thousands, annually again). > > If you are looking for the very latest standards, yes. Enough data is > out there to handle a lot of basic MIPI operations. Since the small > player isn't going to be trying to implement the low level interface > themselves (or at least shouldn't be trying to),So how does one use a MIPI camera without using the low level interface?> unless you are trying > to work with a bleeding edge camera (which you probably can't actually > buy if you are a small player) you can tend to find enough information > to use the camera.That is fair enough, as long as we are talking about some internal sensor specifics of the "bleeding edge" cameras.> > My experiance is if you can actually buy the camera normally, there will > be the data available to use it.That's really reassuring. I am more interested in talking to MIPI display modules than to cameras (at least the sequence is this) but still.> The big problem is "Grey Market" > cameras, via unauthorized distributors you are at the mercy of the > distributor to get you the needed data.Don't they conform to the MIPI standard? (which I have no access to).> >> >> Not use about USB, perhaps USB cameras are covered in the standard >> (yet to deal with that one). > > There is a USB video standard, and many USB cameras can just be plugged > in and used.OK, I thought I had seen that some years ago. Might be an escape (though cameras found in phones and tablets etc. are probably all MIPI).
Reply by ●December 29, 20222022-12-29
On Thursday, December 29, 2022 at 12:06:40 PM UTC-5, Richard Damon wrote:> On 12/29/22 8:16 AM, Don Y wrote: > > ISTR playing with de-encapsulated DRAMs as image sensors > > back in school (DRAM being relatively new technology, then). > > > > But, most cameras seem to have (bit- or word-) serial interfaces > > nowadays. Are there any (mainstream/high volume) devices that > > "look" like a chunk of memory, in their native form? > > > Using a DRAM in that manner would only give you a single bit value for > each pixel (maybe some more modern memories store multiple bits in a > cell so you get a few grey levels).You could probably modulate the timing of the scans to get a range of grey scale, even if small. Let the chip integrate for 1 unit, 2 units, 4 units, etc. of time. I'm assuming light responsiveness of the human eye is logarithmic, rather than linear. If not, then 1, 2, 3, 4 units of time. Even 16 levels of grey is much better than black and white. It would be a bit of processing to translate the thermometer codes into pixel values, but just time consuming, not hard. -- Rick C. - Get 1,000 miles of free Supercharging - Tesla referral code - https://ts.la/richard11209
Reply by ●December 29, 20222022-12-29
On 12/29/22 12:45 PM, Dimiter_Popoff wrote:> On 12/29/2022 19:21, Richard Damon wrote: >> On 12/29/22 8:33 AM, Dimiter_Popoff wrote: >>> On 12/29/2022 15:16, Don Y wrote: >>>> ISTR playing with de-encapsulated DRAMs as image sensors >>>> back in school (DRAM being relatively new technology, then). >>>> >>>> But, most cameras seem to have (bit- or word-) serial interfaces >>>> nowadays. Are there any (mainstream/high volume) devices that >>>> "look" like a chunk of memory, in their native form? >>>> >>> >>> Hah, Don, consider yourself lucky if you find a camera you have >>> enough documentation to use at all, serial or whatever. >>> >>> The MIPI standards are only for politburo members (last time I looked >>> you need to make several millions annually to be able to *apply* >>> for membership, which of course costs thousands, annually again). >> >> If you are looking for the very latest standards, yes. Enough data is >> out there to handle a lot of basic MIPI operations. Since the small >> player isn't going to be trying to implement the low level interface >> themselves (or at least shouldn't be trying to), > > So how does one use a MIPI camera without using the low level interface?You use a chip that has a mipi interface, either a CPU or FPGA with a built in MIPI interface or a MIPI converter chip that converts the MIPI interface into something you can deal with.> >> unless you are trying to work with a bleeding edge camera (which you >> probably can't actually buy if you are a small player) you can tend to >> find enough information to use the camera. > > That is fair enough, as long as we are talking about some internal > sensor specifics of the "bleeding edge" cameras.Bleeding Edge cameras/displays may need newer versions of MIPI than may be easy to find in the consumer market. They may need bleeding edge processors. As I mention below, more important are the configuration registers, which might be harder to get for bleeding edge parts. This is often proprietary, as knowing what is adjustable is often part of the secret sauce for those cameras.> >> >> My experiance is if you can actually buy the camera normally, there >> will be the data available to use it. > > That's really reassuring. I am more interested in talking to MIPI > display modules than to cameras (at least the sequence is this) but > still.So you want a chip with MIPI DSI capability built in, or a convert chip.> >> The big problem is "Grey Market" cameras, via unauthorized >> distributors you are at the mercy of the distributor to get you the >> needed data. > > Don't they conform to the MIPI standard? (which I have no access to).Yes, but MIPI doesn't define the more important configuration registers you need to setup for the device. MIPI is a video data protocol, it has limited configuration capability. Generally there will be something like an I2C bus to the camera that is used to configure it. (There might be a way to tunnel the configuration over the MIPI lines, but I doubt MIPI defines a configuration protocol)> >> >>> >>> Not use about USB, perhaps USB cameras are covered in the standard >>> (yet to deal with that one). >> >> There is a USB video standard, and many USB cameras can just be >> plugged in and used. > > OK, I thought I had seen that some years ago. Might be an escape (though > cameras found in phones and tablets etc. are probably all MIPI). >Yes, the cameras in phones will almost always be MIPI. There is no need for them to use a USB Video connection, that is just way too much overhead. In fact, a lot of stand alone USB Cameras might have a MIPI based camera core internally, with a MIPI to USB inteface to send the data to the Host.
Reply by ●December 29, 20222022-12-29
On 12/29/22 1:20 PM, Rick C wrote:> On Thursday, December 29, 2022 at 12:06:40 PM UTC-5, Richard Damon wrote: >> On 12/29/22 8:16 AM, Don Y wrote: >>> ISTR playing with de-encapsulated DRAMs as image sensors >>> back in school (DRAM being relatively new technology, then). >>> >>> But, most cameras seem to have (bit- or word-) serial interfaces >>> nowadays. Are there any (mainstream/high volume) devices that >>> "look" like a chunk of memory, in their native form? >>> >> Using a DRAM in that manner would only give you a single bit value for >> each pixel (maybe some more modern memories store multiple bits in a >> cell so you get a few grey levels). > > You could probably modulate the timing of the scans to get a range of grey scale, even if small. Let the chip integrate for 1 unit, 2 units, 4 units, etc. of time. I'm assuming light responsiveness of the human eye is logarithmic, rather than linear. If not, then 1, 2, 3, 4 units of time. Even 16 levels of grey is much better than black and white. > > It would be a bit of processing to translate the thermometer codes into pixel values, but just time consuming, not hard. >Yes, at a drastic reduction in frame rate. Power of two spacing will show a lot of banding in the image, but 16 levels spaces at about 1.4x exposure steps might be acceptable. (2**16 dynamic range is extream and well beyond even what "HDR" video can deal with)
Reply by ●December 29, 20222022-12-29
On 12/29/2022 10:06 AM, Richard Damon wrote:> On 12/29/22 8:16 AM, Don Y wrote: >> ISTR playing with de-encapsulated DRAMs as image sensors >> back in school (DRAM being relatively new technology, then). >> >> But, most cameras seem to have (bit- or word-) serial interfaces >> nowadays. Are there any (mainstream/high volume) devices that >> "look" like a chunk of memory, in their native form? > > Using a DRAM in that manner would only give you a single bit value for each > pixel (maybe some more modern memories store multiple bits in a cell so you get > a few grey levels).I mentioned the DRAM reference only as an exemplar of how a "true" parallel, random access interface could exist.> There are some CMOS sensors that let you address pixels individually and in a > random order (like you got with the DRAM) but by its nature, such a readout > method tends to be slow, and space inefficient, so these interfaces tend to be > only available on smaller camera arrays.But, if you are processing the image, such an approach can lead to higher throughput than having to transfer a serial data stream into memory (thus consuming memory bandwidth).> That is why most sensors read out via row/column shift registers to a pixel > serial (maybe multiple pixels per clock) output, and if the camera includes its > own A/D conversion, might serialize the results to minimize interconnect.Yes, but then you have to store it in memory in order to examine it. I.e., if your goal isn't just to pass the image out to a display, then having to unpack the serial stream into RAM is an added cost.
Reply by ●December 29, 20222022-12-29
On 12/29/2022 6:33 AM, Dimiter_Popoff wrote:> On 12/29/2022 15:16, Don Y wrote: >> ISTR playing with de-encapsulated DRAMs as image sensors >> back in school (DRAM being relatively new technology, then). >> >> But, most cameras seem to have (bit- or word-) serial interfaces >> nowadays. Are there any (mainstream/high volume) devices that >> "look" like a chunk of memory, in their native form? >> > > Hah, Don, consider yourself lucky if you find a camera you have > enough documentation to use at all, serial or whatever. > > The MIPI standards are only for politburo members (last time I looked > you need to make several millions annually to be able to *apply* > for membership, which of course costs thousands, annually again). > > Not use about USB, perhaps USB cameras are covered in the standard > (yet to deal with that one).I built my prototypes (proof-of-principle) using COTS USB cameras. But, getting the data out of the serial data stream and into RAM so it can be analyzed consumes memory bandwidth. I'm currently trying to sort out an approximate cost factor "per camera" (per video stream) and looking for ways that I can cut costs (memory bandwidth requirements) to allow greater numbers of cameras or higher frame rates.
