Hi Lewin,

On 6/18/2011 4:03 AM, larwe wrote:
> On Jun 17, 2:17 pm, Don Y<nowh...@here.com>  wrote:
>
>>> overlay window to perform YUV->RGB conversion. Exposure and white
>>> balance are typically adjusted on the fly by host-side software but
>>> this is certainly not universally true, especially for the USB
>>
>> But these aren't (as) important when processing monochromatic
>> video (e.g., barcode symbols).
>
> These sensors don't have a heck of a lot of dynamic range, so it's
> easy to wash out even a barcode. I think you really would be well

Dunno.  I've never been particularly interested in the camera
aspects of these devices before.  I noticed that the cameras
were "slow to start" and would typically take three "repaints"
to get the brightness (?) sorted out in low light conditions
(I would usually play with these in the evening hours so light
wasn't overly abundant).  I should take one outdoors and turn it
on to see how that fares...

> served by buying a couple - for example go to www.dealextreme.com or
> www.merimobiles.com and look at some of their shanzai phones and
> tablets - and just experiment with them using the apps available in
> their app stores. None of them that I'm aware of support realtime
> decode - you acquire a picture, and then they analyze it.

I have several (older) smartphones that I've accumulated over
the years.  I used an HTC wizard for a few months while traveling
as a "portable email (wifi) client".  And, I know I have an
assotment of Treos (kept because they have "keyboards" -- as does
the wizard).

But, since I don't *use* cell phones, I confess I've never played
with anything other than the built-in applications...

> The augmented reality apps that work on a printed orientation glyph do
> almost exactly what you're talking about, but they don't need to read
> much information out of the shape, and they also incorporate other
> sensor input to make inferences about how the shape is moving around
> in the FOV.

I have the luxury of being able to bastardize the code for my own
purposes -- though being able to read "real" QR codes might have
some advantages (and DISadvantages).

>>> These sensors also typically require fairly long exposure times, so
>>
>> So, "motion video" is, of necessity, pretty poor quality?
>
> Varies :) For the low end, absolutely. At the extreme low end, even
> the resolution is very constrained.
>
>> have to deal with already.  E.g., I don't have to worry about
>> compression because I'm not saving images.  How the video is
>
> You don't care about it, but the APIs available to you may implicitly
> turn on those features. Getting fancy behind the scenes by poking at
> the power management features of the system will likely get the
> drivers/OS very confused.

We'll either work on bare iron or contract the development of
our own drivers to give us the features we want/need.  We're not
trying to be a "typical phone/tablet" so doubt solutions intended
for those sorts of markets would fit well.

>> need to be done *eventually*.  Annoying to have a successful
>> prototype and then have to "start over" to come up with a
>> *real* product -- about which the prototype will have taught you
>
> (choosing words with some care here) With a similar though not
> identical problem to solve, a large corporation with which I'm
> familiar has decided to approach this problem entirely at the app
> layer, and qualify individual devices.

We want to be sure the devices are *not* usable for any purpose
other than our own.  And, we will have other I/O's that you
just can't expect to get in a phone/tablet.  So, rather than
design yet another set of "peripherals" and figure out how to
marry them to <whatever>, it makes more sense to roll all the
development into one set of devices.  This should also greatly
simplify the maintenance of the system as well as the software.

On Jun 18, 2:46=A0pm, Don Y <nowh...@here.com> wrote:
> On 6/17/2011 11:23 AM, linnix wrote:
>
> >>>>> - does the use of the camera (in movie mode) eat batteries?
>
> >>> Yes, you have to keep charging it.
>
> >> Every 3 minutes? =A02 hours? etc.? =A0You have to "keep charging" a
> >> cell phone *regardless*... =A0the point is how much of an impact
> >> it makes on battery life, relatively speaking. =A0(e.g., does
> >> phone w/ radio off, camera on consume as much/more/less than
> >> phone w/ radio on, camera off, etc.?
>
> > I took 10 pictures and the battery drop from 66% to 65%. =A0So, i guess
> > you can take 1000 pictures on a full charge.
>
> <grin> =A0I'm not sure if that extrapolation necesssarily
> follows from the empirical evidence. =A0:> =A0If that were
> the case, you could take 60 seconds of 15FPS video (?)

The pictures were taken over 20 to 30 seconds.  So, you can probably
run the phone for about an hour (pictures or video).  The processors
(600MHz CPU + 400MHz DSP) are the main battery killer anyway.

>
> > larwe: USB sensors are impractical beyond VGA. =A0Need lots of memory
> > and bandwidth on chip to do so. =A0The 3M pixel sensor of VS740 has
> > parallel data bus into the system DSP. =A0Pictures are first captured i=
n
> > SDRAM, then write to SD card. =A0However, in Linux, you can probably
> > memory map the SDRAM for processing, but you need rooted Android. =A0Th=
e
> > camera software library is available in C++ (unsupported), but wrapper
> > in Java (supported) available.
>
> Pointer to a URL?

google "android camera programming" will give you plenty to read.

On 6/17/2011 11:23 AM, linnix wrote:
>>>>> - does the use of the camera (in movie mode) eat batteries?
>>
>>> Yes, you have to keep charging it.
>>
>> Every 3 minutes?  2 hours? etc.?  You have to "keep charging" a
>> cell phone *regardless*...  the point is how much of an impact
>> it makes on battery life, relatively speaking.  (e.g., does
>> phone w/ radio off, camera on consume as much/more/less than
>> phone w/ radio on, camera off, etc.?
>
> I took 10 pictures and the battery drop from 66% to 65%.  So, i guess
> you can take 1000 pictures on a full charge.

<grin>  I'm not sure if that extrapolation necesssarily
follows from the empirical evidence.  :>  If that were
the case, you could take 60 seconds of 15FPS video (?)

> larwe: USB sensors are impractical beyond VGA.  Need lots of memory
> and bandwidth on chip to do so.  The 3M pixel sensor of VS740 has
> parallel data bus into the system DSP.  Pictures are first captured in
> SDRAM, then write to SD card.  However, in Linux, you can probably
> memory map the SDRAM for processing, but you need rooted Android.  The
> camera software library is available in C++ (unsupported), but wrapper
> in Java (supported) available.

Pointer to a URL?

On Jun 17, 2:17=A0pm, Don Y <nowh...@here.com> wrote:

> > overlay window to perform YUV->RGB conversion. Exposure and white
> > balance are typically adjusted on the fly by host-side software but
> > this is certainly not universally true, especially for the USB
>
> But these aren't (as) important when processing monochromatic
> video (e.g., barcode symbols).

These sensors don't have a heck of a lot of dynamic range, so it's
easy to wash out even a barcode. I think you really would be well
served by buying a couple - for example go to www.dealextreme.com or
www.merimobiles.com and look at some of their shanzai phones and
tablets - and just experiment with them using the apps available in
their app stores. None of them that I'm aware of support realtime
decode - you acquire a picture, and then they analyze it.

The augmented reality apps that work on a printed orientation glyph do
almost exactly what you're talking about, but they don't need to read
much information out of the shape, and they also incorporate other
sensor input to make inferences about how the shape is moving around
in the FOV.

> > These sensors also typically require fairly long exposure times, so
>
> So, "motion video" is, of necessity, pretty poor quality?

Varies :) For the low end, absolutely. At the extreme low end, even
the resolution is very constrained.

> have to deal with already. =A0E.g., I don't have to worry about
> compression because I'm not saving images. =A0How the video is

You don't care about it, but the APIs available to you may implicitly
turn on those features. Getting fancy behind the scenes by poking at
the power management features of the system will likely get the
drivers/OS very confused.

> need to be done *eventually*. =A0Annoying to have a successful
> prototype and then have to "start over" to come up with a
> *real* product -- about which the prototype will have taught you

(choosing words with some care here) With a similar though not
identical problem to solve, a large corporation with which I'm
familiar has decided to approach this problem entirely at the app
layer, and qualify individual devices.

Hi Hans-Bernhard,

On 6/17/2011 1:03 PM, Hans-Bernhard Br�ker wrote:
> On 17.06.2011 20:08, Don Y wrote:
>
>> I'm not claiming there is anything *wrong* with it! :> Rather,
>> I am wondering how much of a "fortunate circumstance" this is for
>> the phone implementor? I.e., knowing he only has to *display*
>> ~1/4 of the data that he's taking in?
>
> None --- people do expect those images to look good shown on other
> displays, including ones considerably better than the phone's own one.

My point was concerned with the fact that the camera (software)
need not have to deal with *processing* all that data for *display*
in real-time.  Presumably, this reduces the demands on that
process ("fortunate circumstance" for the phone implementor)

> And the image display has zooming capability, too, so you don't get away
> with all that much nonsense even before the image is uploaded somewhere.
>
> This mismatch between sensor resolution and on-camera display resolution
> has been with us since just about day 1 of digital photography. There is
> a reason most digicams still have viewfinders of some design.

On 17.06.2011 20:08, Don Y wrote:

> I'm not claiming there is anything *wrong* with it! :> Rather,
> I am wondering how much of a "fortunate circumstance" this is for
> the phone implementor? I.e., knowing he only has to *display*
> ~1/4 of the data that he's taking in?

None --- people do expect those images to look good shown on other 
displays, including ones considerably better than the phone's own one. 
And the image display has zooming capability, too, so you don't get away 
with all that much nonsense even before the image is uploaded somewhere.

This mismatch between sensor resolution and on-camera display resolution 
has been with us since just about day 1 of digital photography.  There 
is a reason most digicams still have viewfinders of some design.

On Jun 17, 11:03=A0am, Don Y <nowh...@here.com> wrote:
> On 6/16/2011 11:11 AM, linnix wrote:
>
>
>
>
>
>
>
>
>
> > On Jun 16, 10:11 am, "Boudewijn Dijkstra"
> > <sp4mtr4p.boudew...@indes.com> =A0wrote:
> >> Op Thu, 16 Jun 2011 16:54:47 +0200 schreef Don Y<nowh...@here.com>:
>
> > My comments are mostly for Android camera phones (i.e. LG VS740).
>
> >>> Hi,
>
> >>> I currently have a kiosk application that I would like
> >>> to port to a wireless, *portable* implementation. =A0So,
> >>> small tablets or (ack!) smart-phones (with the phone
> >>> functionality permanently and irreparably disabled)
> >>> seem a good choice. =A0Phones being better suited in
> >>> terms of size (want to tuck this into a shirt-pocket
> >>> while not in use).
>
> >>> But, I rely extensively on barcodes to provide data to
> >>> the application (item identification, location identification,
> >>> etc.).
>
> > Phones cameras are more geared for 2D barcodes, but i guess old
> > fashion barcodes are workable.
>
> Yes -- hence my mention of QR (below). =A0It's conceivable that
> you could decode linear ("classic") barcodes but they would
> probably need to be printed at exaggerated scales to get the
> proper discrimination between wide/narrow bars/spaces.
>
> >>> Before I get myself "pregnant" on a solution path, I need
> >>> to get a handle on what's "realistic" in terms of expectations.
>
> >>> [I don't *use* cell phones so my ignorance here requires
> >>> patience =A0:> =A0]
>
> >>> I know (some) cameras have motion video capabilities. =A0From
> >>> that, I can (?) deduce that the optics and electronics in
> >>> the camera are "fast enough" that images can be captured
> >>> at "several" frames per second.
>
> > Motion video are captured and processed in SDRAM (256M in my case)
>
> So, if there is enough horsepower (or, clever enough recognizer),
> the image could be examined -- and discarded -- relatively quickly
> (i.e., no need to compress and/or store it)
>
> > first. =A0Single hi-res (upto 2000x1500) takes couple of seconds. =A0Th=
e
> > jpeg file can be several megabytes compressed.
> >>> - does the use of the camera (in movie mode) eat batteries?
>
> > Yes, you have to keep charging it.
>
> Every 3 minutes? =A02 hours? etc.? =A0You have to "keep charging" a
> cell phone *regardless*... =A0the point is how much of an impact
> it makes on battery life, relatively speaking. =A0(e.g., does
> phone w/ radio off, camera on consume as much/more/less than
> phone w/ radio on, camera off, etc.?
>

I took 10 pictures and the battery drop from 66% to 65%.  So, i guess
you can take 1000 pictures on a full charge.

larwe: USB sensors are impractical beyond VGA.  Need lots of memory
and bandwidth on chip to do so.  The 3M pixel sensor of VS740 has
parallel data bus into the system DSP.  Pictures are first captured in
SDRAM, then write to SD card.  However, in Linux, you can probably
memory map the SDRAM for processing, but you need rooted Android.  The
camera software library is available in C++ (unsupported), but wrapper
in Java (supported) available.

Hi Lewin,

On 6/17/2011 6:42 AM, larwe wrote:
> On Jun 16, 10:54 am, Don Y<nowh...@here.com>  wrote:
>
>> I know (some) cameras have motion video capabilities.  From
>> that, I can (?) deduce that the optics and electronics in
>> the camera are "fast enough" that images can be captured
>> at "several" frames per second.
>
> This is highly variable. To run off a few possibilities - some phones/
> tablets have the camera on an internal USB interface. Some have the
> image sensor connected directly to a dedicated port on the SoC (this
> would almost always be the case for cameras). To acquire images, you
> have to power up the sensor, enable a DMA channel (for the dedicated
> camera port type solution), maybe instruct the GPU to set up an
> overlay window to perform YUV->RGB conversion. Exposure and white
> balance are typically adjusted on the fly by host-side software but
> this is certainly not universally true, especially for the USB

But these aren't (as) important when processing monochromatic
video (e.g., barcode symbols).

> solutions. The frame rate will typically be some divisor of 60Hz, for
> some resolution which might be anything from 160x120 up to full HD
> resolution depending on the camera. Some cameras, especially higher
> resolution cameras, have mechanical autofocus.
>
> Notice how I'm using words like "variable" "typical" etc. These
> systems can range from a single-core 400MHz ARM9 to a dual-core 1.2GHz
> Cortex with a separate DSP and/or other coprocessor(s), so there is
> absolutely no one size fits all answer.

Understood.

> These sensors also typically require fairly long exposure times, so

So, "motion video" is, of necessity, pretty poor quality?

Is the issue related to luminance, too (instead of just chrominance)?

> simply waving the camera about won't acquire a good image - you need
> to select a subject and hold the sensor still while acquiring the
> image. Usually, the live-view mode is significantly lower resolution
> than the sensor's actual max resolution.
>
>> - does the use of the camera (in movie mode) eat batteries?
>
> Sure, because you have powered up the sensor, moved the CPU core into
> a higher power mode, possibly activated a separate DSP core to handle
> MPEG compression, possibly powered up additional segments of the GPU
> (if any) to handle the overlay window, increased RAM bandwidth usage,
> forced the screen to remain on with 100% duty cycle, ... ... ...

As expected.  But, many of these things I can live without or
have to deal with already.  E.g., I don't have to worry about
compression because I'm not saving images.  How the video is
displayed is entirely up to me (if it is displayed at all!).
CPU will already tend to be running at a higher duty cycle
(imagine being *on* the phone all the time it is powered up).
Etc.

>> - does the *camera* drive the interface or does the application?
>
> And for this you should start by looking at the APIs offered by the OS
> in the device you intend to use, because life's too short for hacking
> below the OS level in this case - especially if you want to insulate
> yourself against supply chain changes.

Supply chain won't be a problem in the quantities we're looking
at.  In the short term (prototyping), I suspect trying to
fit some "middleware" above the OS to make it behave the way
the application would *want* it to behave would be more work
than gutting the device completely -- especially as it will
need to be done *eventually*.  Annoying to have a successful
prototype and then have to "start over" to come up with a
*real* product -- about which the prototype will have taught you
very little!  :-/

Hi Boudewijn,

On 6/17/2011 1:26 AM, Boudewijn Dijkstra wrote:

[attributions elided]

>> You mentioned narrow and wide/fixed. Then, went on to
>> describe wide and fixed. Are wide/fixed synonyms? In
>> which case, what about "narrow" (is this what would be
>> called "macro" on a digital camera?)
>
> No, wide and fixed are not synonyms. Wide focus means the image is sharp
> over a wide range (like in a disposable camera), fixed focus means the
> image is only sharp at a fixed distance. Analogously, narrow focus means
> the image is sharp only in a narrow (but not fixed) distance range. IIRC
> "macro" is for short distances only, and involves dynamic mechanical
> zoom (which most phone cameras don't provide).

Ah, OK. "depth of field".

> But, I was just trying to illustrate that some lenses need time (and
> energy) to focus on the object of interest.
>
>>>> I know (some) cameras have motion video capabilities. From
>>>> that, I can (?) deduce that the optics and electronics in
>>>> the camera are "fast enough" that images can be captured
>>>> at "several" frames per second.
>>>
>>> All phone camera's I've seen are fast enough to show captured images on
>>> the main LCD at several frames per second. Compressing and storing the
>>> data somewhere is a different matter.
>>
>> But, the display in most (?) phones seems to be lower quality
>> than the stated camera capabilities. Is this real-time display
>> deliberately discarding data?
>
> It has no choice but to create a scaled version of the image. What's
> wrong with that?

I'm not claiming there is anything *wrong* with it!  :>  Rather,
I am wondering how much of a "fortunate circumstance" this is for
the phone implementor?  I.e., knowing he only has to *display*
~1/4 of the data that he's taking in?

>> What role does the application
>> software take in doing that? Does the application *know* when the
>> scene is "in focus" or do you rely on the user delaying "snapping"
>> the picture until the displayed image "looks good"?
>
> My phone camera has a "dual action" button like in regular cameras.
> Press it softly and it will try to focus (which may fail), press it all
> the way to capture the image. If you press it all the way directly, it
> will first try to focus. The user is notified when the camera thinks
> that the image is focused. But, it will only know after trying to focus.
> So it is an interactive process.

But, there is nothing that prevents a piece of code from making
that decision (albeit possibly "less well")?

>>>> - does the *camera* drive the interface or does the application?
>>>> By that, I mean, do you say, "go into movie mode" and then sit
>>>> back and wait for the camera to deliver frames at whatever rate
>>>> *it* wants? Or, do you say, "give me a frame, *now* (note that
>>>> in this latter case, you could effectively run the video at
>>>> 2 FPS, 3 FPS, 12 FPS, etc. -- as appropriate to your application)
>>>
>>> Since at the very least, light sensitivity adjustment will need to be
>>> made at a certain rate, frames will probably be grabbed at a fixed rate
>>
>> This suggests that the *camera* does this adjustment? Or, is it in
>> the "camera application"? I.e., do I have any control over it?
>
> Dunno. But why would you want to mess with it?

Just rying to understand what's involved, what's "canned", what's
likely to be "proprietary", what's clonable, etc.

On 6/16/2011 11:11 AM, linnix wrote:
> On Jun 16, 10:11 am, "Boudewijn Dijkstra"
> <sp4mtr4p.boudew...@indes.com>  wrote:
>> Op Thu, 16 Jun 2011 16:54:47 +0200 schreef Don Y<nowh...@here.com>:
>
> My comments are mostly for Android camera phones (i.e. LG VS740).
>
>>> Hi,
>>
>>> I currently have a kiosk application that I would like
>>> to port to a wireless, *portable* implementation.  So,
>>> small tablets or (ack!) smart-phones (with the phone
>>> functionality permanently and irreparably disabled)
>>> seem a good choice.  Phones being better suited in
>>> terms of size (want to tuck this into a shirt-pocket
>>> while not in use).
>>
>>> But, I rely extensively on barcodes to provide data to
>>> the application (item identification, location identification,
>>> etc.).
>
> Phones cameras are more geared for 2D barcodes, but i guess old
> fashion barcodes are workable.

Yes -- hence my mention of QR (below).  It's conceivable that
you could decode linear ("classic") barcodes but they would
probably need to be printed at exaggerated scales to get the
proper discrimination between wide/narrow bars/spaces.

>>> Before I get myself "pregnant" on a solution path, I need
>>> to get a handle on what's "realistic" in terms of expectations.
>>
>>> [I don't *use* cell phones so my ignorance here requires
>>> patience  :>  ]
>>
>>> I know (some) cameras have motion video capabilities.  From
>>> that, I can (?) deduce that the optics and electronics in
>>> the camera are "fast enough" that images can be captured
>>> at "several" frames per second.
>
> Motion video are captured and processed in SDRAM (256M in my case)

So, if there is enough horsepower (or, clever enough recognizer),
the image could be examined -- and discarded -- relatively quickly
(i.e., no need to compress and/or store it)

> first.  Single hi-res (upto 2000x1500) takes couple of seconds.  The
> jpeg file can be several megabytes compressed.

>>> - does the use of the camera (in movie mode) eat batteries?
>
> Yes, you have to keep charging it.

Every 3 minutes?  2 hours? etc.?  You have to "keep charging" a
cell phone *regardless*...  the point is how much of an impact
it makes on battery life, relatively speaking.  (e.g., does
phone w/ radio off, camera on consume as much/more/less than
phone w/ radio on, camera off, etc.?

>> Since at the very least, light sensitivity adjustment will need to be made
>> at a certain rate, frames will probably be grabbed at a fixed rate when
>> the camera is active.  I guess it is the choice of the application to skip
>> as many frames as it needs to.
>>
>>> What I would *like* to do is leave the camera on continuously and
>>> have my barcode recognizer/decoder processing images as fast as
>>> feasible.  So, instead of the user having to explicitly enable
>>> the camera, point it at the targeted barcode label and *then*
>>> take a "capture barcode" snapshot, the application can hunt
>>> for suitable labels itself and use them as appropriate.
>
> You will have to write your custom apps.

Yes, I plan on writing most everything "from scratch" as I suspect
it will be *more* work to try to coerce a device made for one
type of use into a very different type of usage (see comments
re: software reuse thread  :> )

>>> Are there any other issues that could eat my lunch that I
>>> haven't considered?
>>
>>> Finally, any pointers for *hackable* camera tablets/phones?
>>> (of course, they need to have some form of WiFi capability)
>
> I would go for a rooted Android.  Unfortunately, theVS740 is hard to
> root.

Thanks, I'll look and see what's around with my basic needs
(incl tablets).