PDA as "X terminal"

On Mon, 11 Oct 2010 22:48:00 -0700, D Yuniskis wrote:

> Hi Ulf,
> 
> Ulf Samuelsson wrote:
>> 2010-10-11 22:20, D Yuniskis skrev:
>>>
>>> I want to use sme WiFi-enabled PDAs as "wireless terminals". I.e., in
>>> much the same way that X terminals operate (though obviously not using
>>> XDMCP, etc. -- OTOH, that would make life a lot easier! :> )
>>>
>>> Does anything like this *already* exist or do I have to write a simple
>>> protocol to do it? Maybe port something like Xnest?
>> 
>> Have a look at the "www.openembedded.org" project. This will run Linux
>> and X-Windows on a lot of different targets, including some Compaq
>> PDAs.
> 
> Hmmm... I didn't see any devices that I recognized. I *think* I could go
> the NetBSD route if I wanted to wipe the PDAs.  I was hoping to find
> something that would coexist, intially, with the native OS so I don't
> have to do major surgery just to evaluate the technology...

Try <http://www.handhelds.org> as well.

Which devices would you want to use ?

Robert Swindells

Hi Dimiter,

Didi wrote:
> On Oct 12, 5:18 am, D Yuniskis <not.going.to...@seen.com> wrote:
>>> On Mon, 11 Oct 2010 13:20:34 -0700, D Yuniskis wrote:
>> Yes.  Put 40 "clients" on that box (i.e., 40 different users/sessions)
>> and the "server" sees a markedly bigger load.  Meanwhile, all that
>> processing power in the PDA is sitting there waiting for "keystrokes"...
> 
> 40 clients served simultaneously over RFB (VNC) would be a significant
> load indeed.

I think the problem is that RDP and VNC were intended (basically)
as a low number of clients (e.g., 1 or 2) viewing a low number
of sessions.  So, much of the work is done "server side" (<frown>
again, the C-S names tend to be hard to keep straight in pedantic
vs. functional terms :-/ ).

By contrast, X treats the remote device as an intelligent,
abstract device.  So, it expects the display to "do much of the
work".

E.g., I can hang 100 X terminals on a small server and get
adequate performance.  The same server would probably struggle
with 3 or 4 RDP/VNC sessions (owing to the increased resource
requirements server side)

> My netMCA product line relies practically only on VNC for
> user access and maintaining a live 800x600, 16 bpp frame takes about
> 10% of the power of a 400 MHz mpc5200b (not counting networking
> overhead, just frame change detection and compression). Then it

Ouch!  I assume VNC sits *past* your "presentation layer"
so your application can't provide hints to it to expedite
the change detection?

> runs under DPS, which is particularly friendly to it.
> I have limited the number of connected users to 2 (just at the
> highest level, there are constantly up to 2 VNC server tasks running).
> 
> But there is another significant advantage of using VNC.
> The (freely!) available clients (and most notably realVNC, the best
> in my experience) will listen for an incoming RFB connection.
> This can be the solution to a load of security issues, e.g. with
> the netmca you can set it up so it will persistently "call" some
> host and put it behind a NAT router (uhm, that makes my limit
> being actually 3 users :-), I did not count that one above).
> Works great, will come back after all sorts of network failures
> etc. (I have made it really persistent :-) ).

My remote devices will auto-authenticate (stored credentials...
possession of the device *is* "The Credential") so powering
them on is all that will be needed to "connect".  I *want* to
be able to detect when they have powered off, left the coverage
area, etc.  The application needs to make note of these things
and adapt accordingly (location-aware computing).

E.g., your laptop wants to know when it is no longer *docked*
as how the application(s) running on it will perform when
untethered may well be different from when it is tethered
(i.e., the "hard connections" are no longer available...)

> All that said, VNC is a good choice only if the server has
> its display framebuffer anyway. Which does not have to be very
> large, of course, so it won't necessarily take a 400 MHz power
> beast (which has no problems to maintain an SXGA 16 bpp frame).

The server is busy with other things.  Wasting resources
(CPU time and wired down memory) for "display services"
when there is a ~200-600MHz processor SITTING BONE IDLE
(waiting for "keystrokes") is a foolish misuse of resources!
(IMHO  :> )

Since adding another "terminal" (handheld/PDA) *gives* you
another BONE IDLE processor, it seems that the more you
can offload to *that* processor, the more gracefully the
application will *scale*!

Remember the Sun Rays?  :-/

On Mon, 11 Oct 2010, D Yuniskis wrote:

> Hi,
>
> I want to use sme WiFi-enabled PDAs as "wireless terminals".
> I.e., in much the same way that X terminals operate (though
> obviously not using XDMCP, etc. -- OTOH, that would make life
> a lot easier!  :> )
>
> A browser interface is out because the browser has functionality
> that the application can't disable (e.g., history, font selection,
> etc.).
>
> I've never used "terminal services" (there is a TS client built
> into the PDAs) but suspect it is MS's version of X -- *tied* to
> a MS implementation (I want an open source solution).
>
> Does anything like this *already* exist or do I have to write
> a simple protocol to do it?  Maybe port something like Xnest?
>
> Thx,
> --don
>

It depends upon what you mean by PDA's.  If you mean inexpensive, obsolete 
IPAQ or Palm pda's, you'll find that support for a [L|U]nix is uneven at 
best.  You *COULD* write to PalmOS or WinCE, but you're still on older, 
and unsupported, or soon to be unsupported hardware.

OTOH, arm based tablet PC's have begun to flood in from the East, 
and an Android based tablet will cost you about $120-$200 USD brand new.

 	http://www.MeriMobiles.com/Tablet_PC_s_s/205.htm

Writing to Android will open the door to using smart-phones, or dedicated 
tablets as your device, and give you a certain degree of vendor 
independence ... perhaps this approach might have some short term and 
longer term advantages??

Cheers,
Rob Sciuk.

Hi Robert,

Robert Swindells wrote:
> On Mon, 11 Oct 2010 22:48:00 -0700, D Yuniskis wrote:
> 
>> Ulf Samuelsson wrote:
>>> Have a look at the "www.openembedded.org" project. This will run Linux
>>> and X-Windows on a lot of different targets, including some Compaq
>>> PDAs.
>> Hmmm... I didn't see any devices that I recognized. I *think* I could go
>> the NetBSD route if I wanted to wipe the PDAs.  I was hoping to find
>> something that would coexist, intially, with the native OS so I don't
>> have to do major surgery just to evaluate the technology...
> 
> Try <http://www.handhelds.org> as well.

Thanks, I'll poke around there...

> Which devices would you want to use ?

I've thought of using iPhones *if* I could permanently disable
(i.e., destroy) the portion of the radio that makes it usable
as a phone (irrecoverably).  The screen is a bit on the small side
but it has the advantage of being "current".

There are a few Compaq/HP PDA's that have more reasonably sized
displays which would probably be the better approach -- they
also have the advantage of NOT having a phone within...

Presently, I want to see what changes will be necessary to move
the application onto a handheld (small, portable, etc.) device
and what new challenges it will present.  Of course, I would like
to do that without a *huge* investment before deciding if it is
worth pursuing.

On Oct 12, 6:42=A0pm, D Yuniskis <not.going.to...@seen.com> wrote:
> I've thought of using iPhones *if* I could permanently disable
> (i.e., destroy) the portion of the radio that makes it usable
> as a phone (irrecoverably).

Sounds like an iPod touch.

Looking at the iTunes store, there's an iPhone/iPod touch app called
iX11 that's an X11 server.  There are also a number of VNC apps.

     Steve

Hi Robert,

Spam@ControlQ.com wrote:
> 
> On Mon, 11 Oct 2010, D Yuniskis wrote:
> 
>> Hi,
>>
>> I want to use sme WiFi-enabled PDAs as "wireless terminals".
>> I.e., in much the same way that X terminals operate (though
>> obviously not using XDMCP, etc. -- OTOH, that would make life
>> a lot easier!  :> )
>>
>> A browser interface is out because the browser has functionality
>> that the application can't disable (e.g., history, font selection,
>> etc.).
>>
>> I've never used "terminal services" (there is a TS client built
>> into the PDAs) but suspect it is MS's version of X -- *tied* to
>> a MS implementation (I want an open source solution).
>>
>> Does anything like this *already* exist or do I have to write
>> a simple protocol to do it?  Maybe port something like Xnest?
>>
>> Thx,
>> --don
>>
> 
> It depends upon what you mean by PDA's.  If you mean inexpensive, 
> obsolete IPAQ or Palm pda's, you'll find that support for a [L|U]nix is 
> uneven at best.  You *COULD* write to PalmOS or WinCE, but you're still 
> on older, and unsupported, or soon to be unsupported hardware.
> 
> OTOH, arm based tablet PC's have begun to flood in from the East, and an 
> Android based tablet will cost you about $120-$200 USD brand new.
>     http://www.MeriMobiles.com/Tablet_PC_s_s/205.htm

Most tablets are a bit large.  I'm trying to hit a "sweet spot"
that's a bit bigger than most phones but not as big as a tablet.
I.e., you have to carry this around with you *while* you are working
(including activities that *don't* require you to interact with
the device)

This is a possible contender in terms of size (though pricey):
http://www.merimobiles.com/BenQ_S6_4_8_3G_WIFI_Windows_XP_8GB_512MB_RAM_p/meri0521.htm

> Writing to Android will open the door to using smart-phones, or 
> dedicated tablets as your device, and give you a certain degree of 
> vendor independence ... perhaps this approach might have some short term 
> and longer term advantages??

Phone is a definite DISadvantage.  As is the potential for
reusing the device as anything *other* than this purpose.
I want to just remote a display/user interface.  Like
taking the controls off your microwave oven so you can
interact with it *without* standing directly in front
of it (silly example).  It's still PART OF THE MICROWAVE
OVEN (in much the same way that the remote control for
your television is part of your TV!)

E.g., imagine if your employer had a box full of iPhones
that you casually selected from in order to do your
"work" each day (i.e., imagine the PC in your cubicle has
been replaced by an iPhone... *intentionally* small and
portable).  At the end of the day, you drop the phone
off in the box for recharging, updates, service, etc.

How many of those, do you think, would "grow legs" and
find new use as someone's personal phone??

Now, imagine you are NOT an engineer but, rather, a
"laborer", salesman, etc. and not knowledgeable of
what makes *these* iPhones "unusable" as regular phones
(i.e., they have been crippled, intentionally).  They
sure *look* like iPhones!

How many of these would grow legs?  (even if the
"borrowers" eventually discovereed that they *can't*
"get service" on them)

Imagine if the headsets used to listen to movies on
long airline flights were iPhones (gives you access to
movies, music, your own intelligent interface to
those services, etc. and eliminates all the wiring in
the plane).  How many would "disappear" on each flight?
Even if folks "discovered" after they had deplaned that
the "iphone" didn't work as such!

On Oct 12, 11:23=A0pm, D Yuniskis <not.going.to...@seen.com> wrote:
> Most tablets are a bit large. =A0I'm trying to hit a "sweet spot"
> that's a bit bigger than most phones but not as big as a tablet.
> I.e., you have to carry this around with you *while* you are working
> (including activities that *don't* require you to interact with
> the device)


Archos (obvious URL) has a variety of tablet sizes running Android.
Their 43 Internet Tablet, for example, has a 4.3 inch screen, WiFi and
Bluetooth, along with most of the other stuff you'd expect.  No phone
(although any device of this class is ultimately capable of running IP
phone software).  The have larger and smaller devices (2.8, 3.2, 5.0
and 7.0 inch sizes would seem to cover the range you're looking for).
Some of their tablets are unlocked (the 5, for example), and other
Linux distributions are available for them.

While the Android tablet market is just getting rolling (heck, the
whole tablet market is just getting rolling), I think you're going to
see many devices in a variety of sizes.

On Oct 12, 9:08=A0pm, D Yuniskis <not.going.to...@seen.com> wrote:
> ......
> > My netMCA product line relies practically only on VNC for
> > user access and maintaining a live 800x600, 16 bpp frame takes about
> > 10% of the power of a 400 MHz mpc5200b (not counting networking
> > overhead, just frame change detection and compression). Then it
>
> Ouch! =A0I assume VNC sits *past* your "presentation layer"
> so your application can't provide hints to it to expedite
> the change detection?

VNC is an implementation of RFB (remote framebuffer protocol),
so basically this is where it belongs. It transfers pixels.
DPS will tell you when a window has been (or not) modified so
the VNC server will know that; but somehow signalling which pixels
got modified is obviously impractical, not to speak on how
this will be done with all the overlapping live windows etc.
So the sane solution is to maintain a display frame in
system memory and if some of the visible windows has changed, to
locate the changed pixels, compress and send. At least I know of
no better way of doing it :-) .

Dimiter

------------------------------------------------------
Dimiter Popoff               Transgalactic Instruments

http://www.tgi-sci.com
------------------------------------------------------
http://www.flickr.com/photos/didi_tgi/sets/72157600228621276/

Hi Dimiter,

Didi wrote:
> On Oct 12, 9:08 pm, D Yuniskis <not.going.to...@seen.com> wrote:
>> ......
>>> My netMCA product line relies practically only on VNC for
>>> user access and maintaining a live 800x600, 16 bpp frame takes about
>>> 10% of the power of a 400 MHz mpc5200b (not counting networking
>>> overhead, just frame change detection and compression). Then it
>> Ouch!  I assume VNC sits *past* your "presentation layer"
>> so your application can't provide hints to it to expedite
>> the change detection?
> 
> VNC is an implementation of RFB (remote framebuffer protocol),
> so basically this is where it belongs. It transfers pixels.
> DPS will tell you when a window has been (or not) modified so
> the VNC server will know that; but somehow signalling which pixels
> got modified is obviously impractical, not to speak on how
> this will be done with all the overlapping live windows etc.

For example, my curses implementation watches all accesses
to the "pad" -- equivalent of frame buffer except it's just
a 2D array of (character, attribute) tuples -- and notes where
the changes are.  Then, when told to "update the physical display",
the code examines these "change flags" and decides what the optimal
(e.g., least number of characters sent over the serial link)
method of getting the physical display to coincide with the
*desired* "virtual image".

A similar thing can be done by noting where accesses to
the frame buffer are made.  Of course, the cost of doing
this is considerably more than, for example, a 2000 - 8000
character "text display".  But, you have to look at the
costs of transport vs. computation (e.g., with a serial
port, you can save *lots* of transport time by cutting all
"redundant" character data out of the transmission.

(you would have to look *into* the VNC implementation in
order to let it notice the begin/end change information
that you have been logging with each FB access)

Of course, if you have graphics accelerator involved, then
all bets are off (since you can't hack *into* that).  But,
in your case, you're just presenting a virtual frame buffer
(??) driven entirely in software

> So the sane solution is to maintain a display frame in
> system memory and if some of the visible windows has changed, to
> locate the changed pixels, compress and send. At least I know of
> no better way of doing it :-) .

Imagine your "draw_window" routine *marking* what parts of the
display it has altered.  Then, your VNC implementation *knows*
which parts of the FB have been modified and which haven't.
If all accesses to your FB go through *your* API, then you
can hook each of them to maintain this "change information"
incrementally instead of defering the entire task to a single
"check_for_changes()" routine.

On Oct 13, 11:53=A0am, D Yuniskis <not.going.to...@seen.com> wrote:
> Hi Dimiter,
>
>
>
> Didi wrote:
> > On Oct 12, 9:08 pm, D Yuniskis <not.going.to...@seen.com> wrote:
> >> ......
> >>> My netMCA product line relies practically only on VNC for
> >>> user access and maintaining a live 800x600, 16 bpp frame takes about
> >>> 10% of the power of a 400 MHz mpc5200b (not counting networking
> >>> overhead, just frame change detection and compression). Then it
> >> Ouch! =A0I assume VNC sits *past* your "presentation layer"
> >> so your application can't provide hints to it to expedite
> >> the change detection?
>
> > VNC is an implementation of RFB (remote framebuffer protocol),
> > so basically this is where it belongs. It transfers pixels.
> > DPS will tell you when a window has been (or not) modified so
> > the VNC server will know that; but somehow signalling which pixels
> > got modified is obviously impractical, not to speak on how
> > this will be done with all the overlapping live windows etc.
>
> For example, my curses implementation watches all accesses
> to the "pad" -- equivalent of frame buffer except it's just
> a 2D array of (character, attribute) tuples -- and notes where
> the changes are. =A0Then, when told to "update the physical display",
> the code examines these "change flags" and decides what the optimal
> (e.g., least number of characters sent over the serial link)
> method of getting the physical display to coincide with the
> *desired* "virtual image".
>
> A similar thing can be done by noting where accesses to
> the frame buffer are made. =A0Of course, the cost of doing
> this is considerably more than, for example, a 2000 - 8000
> character "text display". =A0But, you have to look at the
> costs of transport vs. computation (e.g., with a serial
> port, you can save *lots* of transport time by cutting all
> "redundant" character data out of the transmission.

Hi Don,

transport time can be saved and is saved, this is exactly
what my vnc (rfb) server does compression for. Over a slow
link, say, a 500 kbpS, redrawing a blank "shell" screen
goes instantly; it takes pretty long if the image is some
landscape photo. Over a 10 MbpS link you can work normally,
although you will notice a fullscreen redraw; over a
100 MbpS link, you just don't feel this is a remote host
(at 800x600 pixels, 16bpp; same on a 1280x1024, although
I rarely use that over VNC).

But keeping track on which pixels have changed in a multi-
window multitask environment while drawing these is
impractical. It will take a lot more resources to do so
than to detect changes in the final framebuffer, not to
speak about the mess it will add to the entire code.
As it is, applications draw what they want to (spectra
displays, oscilloscope displays etc. live things), do
the bulk of the work (which is signal processing), and
a fraction of the resources goes on VNC, 10% is not
that much at all for what you get.

> (you would have to look *into* the VNC implementation in
> order to let it notice the begin/end change information
> that you have been logging with each FB access)

This is why the vnc server does these checks, the code is
highly optimized, very efficient. Not much room for improvement
there. Of course it takes decisions which parts to send,
whether compression of a particular changed region is worth it
(e.g. if you have a series of one changed and one unchanged
pixel compression will be wasting rather than saving bandwidth).

> Imagine your "draw_window" routine *marking* what parts of the
> display it has altered. =A0Then, your VNC implementation *knows*
> which parts of the FB have been modified and which haven't.

Well, the end result is indeed that. But someone has to keep
track on which window has been modified, where on the "screen"
this windows origin currently is, how much of it is visible
(i.e. not covered by another window or outside of the "screen"
frame); doing this all the time - while drawing each pixel
or each line (only to add the overhead of clipping and rendering
to the vnc server as well....) by the vnc is insane, to put it
mildly :-).
This why an OS - like DPS - does this for the applications, the
VNC server being one of them - and I am sure you cannot
point me to a more efficient working implementation which has
all that functionality. In fact, I doubt you can locate
one which has all that - applications included for the netMCA-2 -
which
will fit within a 2M flash - with some free space.

Dimiter

------------------------------------------------------
Dimiter Popoff               Transgalactic Instruments

http://www.tgi-sci.com
------------------------------------------------------
http://www.flickr.com/photos/didi_tgi/sets/72157600228621276/