x86 real mode

On 10/17/2014 3:25 PM, glen herrmannsfeldt wrote:
> Don Y <this@is.not.me.com> wrote:
>
>>> Well, the 8086 was an upgrade from the 8080 or 8085.
>
>> The 8085 was essentially an 8080 with an internal clock driver
>> (and a couple of other little fixups -- RIM/SIM, INT5.5/6.5/7.5,
>> etc.)
>
> And a 5V only power supply.

Note that most practical memory still required two (even *three*!)
power supplies.

> As well as I know it, the 8085 and Z80
> are but successors to the 8080, but different groups from intel.
> (One group left, one stayed.)  Being 5V only made it a lot easier
> to use in small systems.

Different approaches to the "problem" (that being, the design of
an 8 bit MPU).  Likewise, the Motogorilla/NatSemi/Signetics/TI/etc.
folks each took a trip in a different direction.  We seem to have
been left with the worst of the bunch  :-/

(Well, perhaps not *worst* -- there were some really ghastly designs
in the early/mid 70's.  But, it beat the hell out of coding for the
i4004!)

The Z80 was a much more powerful architecture than the 8080/85.
Especially if you were trying to deal with quick response times
for ISRs, structured coding, etc.   Far more suited to HLL constructs
(though still very clumsy).

Most of the "16b" machines were dog slow, by comparison.  And, ate
more memory on top of it!

>>> They knew by then that people were running out of address space.
>
>> Yes, an Our Hero, Bill figured 640K was more than anyone would
>> EVER need!  :-/
>
>> [How much nicer things would have been had the 68K won that design-in]
>
> Might have been a 68008, though. They seem to like the 8 bit bus.

But, there was a SEEMLESS path upward from the 68000.  Intel gave us all these
"backward compatibility issues" to carry forward (into the 31st century!).
It was just a memory bandwidth tradeoff (08, 010, 020, etc.) moving forward.

>> IIRC, the original "ISA" bus wasn't even formally characterized
>> until after the fact.
>
> IBM always was, and still is, good at documented what they did,
> though not always why they did it. There are manuals that give lots
> of detail on every line of the bus, and BIOS listings with comments.
>
> But then there were mistakes along the way. Edge triggered
> interrupts make it hard to share INT lines.

I think documentation exists *now* but not "back then".  I can recall
having one helluva time trying to design "ISA bus" cards with any
sort of *guarantee* that they would work in ANY pc (even genuine big blue).
You ended up designing empirically and to "typical" numbers as there were
no hard and fast numbers that you could rely upon from a PC vendor.

Tauno Voipio wrote:
> On 17.10.14 17:23, Don Y wrote:
>> Hi,
>>
>> [What a screwed up architecture!]
>>
>> Thx,
>> --don
>
> Actually, Intel did not intend to use the 8086 real mode
> as such, only as bootstrap mode for 80286. The success
> of 8088 and original PC came as a surprise, and it influenced
> the design of 80386 with the virtual 8086 mode.
>
> There were no PC's at the design time of 8086 and 80286,
> and the aim was for well protected real-time multitasking
> applications.

For an 8086? I'm pretty sure it was all 100% "real mode".

> So it was natural that 80286 does not voluntarily
> return back to real mode from protected mode. This led to
> a massive kludge in PC/AT, where the keyboard controller
> can reset the main processor and the RTC CMOS RAM will
> contain a code explaining why the reset was this time.
>
-- 
Les Cargill

On Fri, 17 Oct 2014 22:00:01 +0000 (UTC), glen herrmannsfeldt
<gah@ugcs.caltech.edu> wrote:

>George Neuner <gneuner2@comcast.net> wrote:
> 
>> Arithmetic on "far" pointers affects only the 16-bit offset and wraps
>> at the segment boundary.  This means that two far pointers having
>> different segments can't reasonably be compared.  If you are using
>> multiple data segments and you need to compare pointers, you have to
>> use normalized "huge" pointers.  Do be aware that huge pointer
>> arithmetic can be quite a bit slower due to (re)normalization of the
>> results.
>
>You don't say what language you use, but C only allows comparison
>of pointers to different objects as equal or not equal. With large
>model, objects (arrays and structs) can't get larger than 64K.
>(For C, they can't even equal 64K.)  A C compiler should generate
>properly comparable (for equal or not equal) pointers in large
>model.

No, actually it won't - at least not in the "large" model with Intel
or Micro$oft compilers.

Far pointers are constructed by left shifting the segment value by 4
bits and adding a 16 bit offset - i.e. they are a 20 bit address
expressed as 4:16.  In real mode there are no reserved segment values
so there are 4096 possible far pointers for *every* location in memory
(with wraparound arithmetic there is a possible segment base every
16-bytes within -32K..+32K of the location).

Huge pointers are 16:4, the 20 bit address normalized to a 16 bit
segment and a 4 bit offset.  This ensures that every distinct huge
pointer value references a unique location in the 1MB space.

The problem is that normalization is costly (a few extra instructions)
and compilers don't do it unless you explicitly ask for it by using
the huge model or by using huge pointers in other models.

 
>In many cases, I have used 2D arrays, actual C arrays of pointers
>to arrays.  As long as the array of pointers and pointed-to arrays
>fit in 64K, you can get plenty big enough. 
>
>I believe in the OS/2 1.x days, I had 16M on my system before
>converting to OS/2 2.x and appropriate compilers.

OS/2 1.x operated in 80286 protected mode, which has other issues that
are not relevant to Don's question.

That said, you are absolutely correct that there are ways to get
around the limitations of 16-bit segments.


>There is medium and compact, and I forget which is which.

Yeah, I don't recall which is which just now, but one is 64K code +
1MB data and the other is the reverse.  

The "large" model allows 1MB code+data using unnormalized pointers.
The "huge" mode is the same but using normalized pointers.

George

On Fri, 17 Oct 2014 22:00:01 +0000 (UTC), glen herrmannsfeldt
<gah@ugcs.caltech.edu> wrote:

>George Neuner <gneuner2@comcast.net> wrote:
>
>> In the "huge" model arrays and structs can straddle segments. However,
>> you should take care that no individual element of an array or struct
>> lies across a segment boundary ... I have found that many compilers
>> don't renormalize intermediate pointers when indexing from a huge
>> pointer, so you can get into trouble when the offset is close to or
>> exceeds 64K.  I've been bitten by that more times than I care to
>> admit.
>
>Seems like a compiler bug to me.  If you assign to a far pointer,
>then it is your fault.

Definitely a compiler bug which was fixed eventually - but I can't
remember which compilers and versions had the bug.  I do know they all
were Micro$quish compilers ... I don't recall every having the problem
with an Intel compiler.

The problem occurred when using a huge pointer with an index that took
it very close to a 64K step: the intermediate pointer would be correct
so far as the location it addressed, but would be a 4:16 unnormalized
pointer rather than a normalized 16:4 pointer.  If the value in memory
straddled the 64K boundary, the fetch or store would wrap around to
the beginning of the segment and the result would be incorrect (and
often very hard to find).

George

Hi Don,


On Fri, 17 Oct 2014 15:39:20 -0700, Don Y <this@is.not.me.com> wrote:

>On 10/17/2014 3:25 PM, glen herrmannsfeldt wrote:
>> Don Y <this@is.not.me.com> wrote:
>>
>>> Yes, an Our Hero, Bill figured 640K was more than anyone would
>>> EVER need!  :-/

Billy boy had nothing to do with it - he wasn't even in the picture
when the PC was designed.  IBM wanted CPM-86 for the PC and only
turned to Microsoft when DR made a mistake in dealing with them.


>>> [How much nicer things would have been had the 68K won that design-in]
>>
>> Might have been a 68008, though. They seem to like the 8 bit bus.
>
>But, there was a SEEMLESS path upward from the 68000.  Intel gave us all these
>"backward compatibility issues" to carry forward (into the 31st century!).
>It was just a memory bandwidth tradeoff (08, 010, 020, etc.) moving forward.

IBM wanted to keep the PC "cheap" and that meant an 8-bit bus.  At the
time the 8088 was available but the 68008 was still in pre-production
and Motorola couldn't guarantee delivery in quantity by IBM's
deadline.  So the 8088 won by de fault of Motorola not having an 8-bit
part.


>>> IIRC, the original "ISA" bus wasn't even formally characterized
>>> until after the fact.

True.

George

George Neuner <gneuner2@comcast.net> wrote:

(snip, someone wrote)

>>> different segments can't reasonably be compared.  If you are using
>>> multiple data segments and you need to compare pointers, you have to
>>> use normalized "huge" pointers.  Do be aware that huge pointer
>>> arithmetic can be quite a bit slower due to (re)normalization of the
>>> results.

(then I wrote)
>>You don't say what language you use, but C only allows comparison
>>of pointers to different objects as equal or not equal. With large
>>model, objects (arrays and structs) can't get larger than 64K.
>>(For C, they can't even equal 64K.)  A C compiler should generate
>>properly comparable (for equal or not equal) pointers in large
>>model.
 
> No, actually it won't - at least not in the "large" model with Intel
> or Micro$oft compilers.
 
> Far pointers are constructed by left shifting the segment value by 4
> bits and adding a 16 bit offset - i.e. they are a 20 bit address
> expressed as 4:16.  In real mode there are no reserved segment values
> so there are 4096 possible far pointers for *every* location in memory
> (with wraparound arithmetic there is a possible segment base every
> 16-bytes within -32K..+32K of the location).

Yes, but the compiler should make sure that only one of those
is ever used. That is, unless you cheat and change them yourself.
 
> Huge pointers are 16:4, the 20 bit address normalized to a 16 bit
> segment and a 4 bit offset.  This ensures that every distinct huge
> pointer value references a unique location in the 1MB space.

I tried to avoid huge, but yes. 

Actually, no. I am pretty sure that at least MSC 6.x, which is what
I used at the time, generated segment selectors 4K apart and
16 bit offsets.  The other way doesn't work right with OS/2. 
You don't want to allocates a segment selector for every 16 bytes.

The compilers ran under either DOS or OS/2, and generated the
same code in either case. I even used to generate bound executables
which run under either DOS or OS/2.

But in any case, it is the responsibility of the compiler to
make it work, if you follow the C standard. Normalizing them
yourself isn't standard C.  Casting a huge pointer to a far
pointer isn't, either. 

I don't think I ever tried arrays of struct of unusual size. 
It might have padded them out.  I did use large model pretty much
all the time, and never had any problems. 
 
> The problem is that normalization is costly (a few extra instructions)
> and compilers don't do it unless you explicitly ask for it by using
> the huge model or by using huge pointers in other models.

And I pretty much never used it. But a 4K entry array of pointers
to 8K doubles allows for 32MB which was big for a 286.
 
>>In many cases, I have used 2D arrays, actual C arrays of pointers
>>to arrays.  As long as the array of pointers and pointed-to arrays
>>fit in 64K, you can get plenty big enough. 

>>I believe in the OS/2 1.x days, I had 16M on my system before
>>converting to OS/2 2.x and appropriate compilers.
 
> OS/2 1.x operated in 80286 protected mode, which has other issues that
> are not relevant to Don's question.
 
> That said, you are absolutely correct that there are ways to get
> around the limitations of 16-bit segments.
 
>>There is medium and compact, and I forget which is which.
 
> Yeah, I don't recall which is which just now, but one is 64K code +
> 1MB data and the other is the reverse.  
 
> The "large" model allows 1MB code+data using unnormalized pointers.
> The "huge" mode is the same but using normalized pointers.

Actually, large allows for a lot more than 1MB. And OS/2 will swap
whole segments to/from disk, so you can do really large arrays.

OS/2 1.x allowed for 8K segment selectors for a user process,
each could be up to 64K, so 512MB.  With DOS, only 640K, or if
you use the right display board, you can go up a litle more.

I started OS/2 1.0 in 1990, and tried not to use DOS after that.

-- glen

On Fri, 17 Oct 2014 21:45:21 -0400, George Neuner
<gneuner2@comcast.net> wrote:

>On Fri, 17 Oct 2014 22:00:01 +0000 (UTC), glen herrmannsfeldt
><gah@ugcs.caltech.edu> wrote:
>
>>George Neuner <gneuner2@comcast.net> wrote:
>> 
>>> Arithmetic on "far" pointers affects only the 16-bit offset and wraps
>>> at the segment boundary.  This means that two far pointers having
>>> different segments can't reasonably be compared.  If you are using
>>> multiple data segments and you need to compare pointers, you have to
>>> use normalized "huge" pointers.  Do be aware that huge pointer
>>> arithmetic can be quite a bit slower due to (re)normalization of the
>>> results.
>>
>>You don't say what language you use, but C only allows comparison
>>of pointers to different objects as equal or not equal. With large
>>model, objects (arrays and structs) can't get larger than 64K.
>>(For C, they can't even equal 64K.)  A C compiler should generate
>>properly comparable (for equal or not equal) pointers in large
>>model.
>
>No, actually it won't - at least not in the "large" model with Intel
>or Micro$oft compilers.
>
>Far pointers are constructed by left shifting the segment value by 4
>bits and adding a 16 bit offset - i.e. they are a 20 bit address
>expressed as 4:16.  In real mode there are no reserved segment values
>so there are 4096 possible far pointers for *every* location in memory
>(with wraparound arithmetic there is a possible segment base every
>16-bytes within -32K..+32K of the location).
>
>Huge pointers are 16:4, the 20 bit address normalized to a 16 bit
>segment and a 4 bit offset.  This ensures that every distinct huge
>pointer value references a unique location in the 1MB space.
>
>The problem is that normalization is costly (a few extra instructions)
>and compilers don't do it unless you explicitly ask for it by using
>the huge model or by using huge pointers in other models.
>
> 
>>In many cases, I have used 2D arrays, actual C arrays of pointers
>>to arrays.  As long as the array of pointers and pointed-to arrays
>>fit in 64K, you can get plenty big enough. 
>>
>>I believe in the OS/2 1.x days, I had 16M on my system before
>>converting to OS/2 2.x and appropriate compilers.
>
>OS/2 1.x operated in 80286 protected mode, which has other issues that
>are not relevant to Don's question.
>
>That said, you are absolutely correct that there are ways to get
>around the limitations of 16-bit segments.
>
>
>>There is medium and compact, and I forget which is which.
>
>Yeah, I don't recall which is which just now, but one is 64K code +
>1MB data and the other is the reverse.  


As the memory models persisted in 16-bit protected mode, it's more
correct to say that medium model supported one code segment and
multiple data segments, and compact the other way around.  And
theoretically you could have implemented segmentation and memory
models in 32-bit protected mode as well, but that was quite rare (I
think WATCOM's C compiler actually supported that), and almost all
32-bit (and 64-bit) protected mode programs effectively run in tiny
model.

Hi George,

On 10/17/2014 7:47 PM, George Neuner wrote:
> On Fri, 17 Oct 2014 15:39:20 -0700, Don Y <this@is.not.me.com> wrote:
>
>> On 10/17/2014 3:25 PM, glen herrmannsfeldt wrote:
>>> Don Y <this@is.not.me.com> wrote:
>>>
>>>> Yes, an Our Hero, Bill figured 640K was more than anyone would
>>>> EVER need!  :-/
>
> Billy boy had nothing to do with it - he wasn't even in the picture
> when the PC was designed.  IBM wanted CPM-86 for the PC and only
> turned to Microsoft when DR made a mistake in dealing with them.

Didn't mean to imply that he had a hand in the design of the hardware.
Only that he had "concluded" that 640K was effectively "infinite"
and, as such, cast the foundations of MS-DOS in mud instead of
"thinking forward".

Sort of like picking Jan 1 1970 as the epoch ("Obviously, time will
come to an end before <original_unix_developers> die...")

Always amusing to see lack of vision, in practice!  :>

On Sat, 18 Oct 2014 03:26:45 +0000 (UTC), glen herrmannsfeldt
<gah@ugcs.caltech.edu> wrote:

>George Neuner <gneuner2@comcast.net> wrote:
>
>(snip, someone wrote)
>
>>>> different segments can't reasonably be compared.  If you are using
>>>> multiple data segments and you need to compare pointers, you have to
>>>> use normalized "huge" pointers.  Do be aware that huge pointer
>>>> arithmetic can be quite a bit slower due to (re)normalization of the
>>>> results.
>
>(then I wrote)
>>>You don't say what language you use, but C only allows comparison
>>>of pointers to different objects as equal or not equal. With large
>>>model, objects (arrays and structs) can't get larger than 64K.
>>>(For C, they can't even equal 64K.)  A C compiler should generate
>>>properly comparable (for equal or not equal) pointers in large
>>>model.
> 
>> No, actually it won't - at least not in the "large" model with Intel
>> or Micro$oft compilers.
> 
>> Far pointers are constructed by left shifting the segment value by 4
>> bits and adding a 16 bit offset - i.e. they are a 20 bit address
>> expressed as 4:16.  In real mode there are no reserved segment values
>> so there are 4096 possible far pointers for *every* location in memory
>> (with wraparound arithmetic there is a possible segment base every
>> 16-bytes within -32K..+32K of the location).
>
>Yes, but the compiler should make sure that only one of those
>is ever used. That is, unless you cheat and change them yourself.
> 
>> Huge pointers are 16:4, the 20 bit address normalized to a 16 bit
>> segment and a 4 bit offset.  This ensures that every distinct huge
>> pointer value references a unique location in the 1MB space.
>
>I tried to avoid huge, but yes. 
>
>Actually, no. I am pretty sure that at least MSC 6.x, which is what
>I used at the time, generated segment selectors 4K apart and
>16 bit offsets.  The other way doesn't work right with OS/2. 
>You don't want to allocates a segment selector for every 16 bytes.


IIRC, the usual scheme for huge mode support was to always use 64KB
segments, and then tweak the segment arithmetic to match the tiling.
So for real mode, consecutive segments for a huge object were 4096
apart.  In PM, the OS allocated consecutive selectors, which were
numerically 8 apart.  So an 80KB huge object might be in segments
0x1234 and 0x2234 in real mode, and in selectors 0x4567 and 0x456F in
protected mode.  The environment provided some run-time readable
variables to define the increment (4096 vs. 8) and corresponding shift
counts (12 and 3).

Robert Wessel <robertwessel2@yahoo.com> wrote:

(snip, somewone wrote)
>>> No, actually it won't - at least not in the "large" model with Intel
>>> or Micro$oft compilers.
 
>>> Far pointers are constructed by left shifting the segment value by 4
>>> bits and adding a 16 bit offset - i.e. they are a 20 bit address
>>> expressed as 4:16.  In real mode there are no reserved segment values
>>> so there are 4096 possible far pointers for *every* location in memory
>>> (with wraparound arithmetic there is a possible segment base every
>>> 16-bytes within -32K..+32K of the location).

(then I wrote)
>>Yes, but the compiler should make sure that only one of those
>>is ever used. That is, unless you cheat and change them yourself.
 
>>> Huge pointers are 16:4, the 20 bit address normalized to a 16 bit
>>> segment and a 4 bit offset.  This ensures that every distinct huge
>>> pointer value references a unique location in the 1MB space.

>>I tried to avoid huge, but yes. 

>>Actually, no. I am pretty sure that at least MSC 6.x, which is what
>>I used at the time, generated segment selectors 4K apart and
>>16 bit offsets.  The other way doesn't work right with OS/2. 
>>You don't want to allocates a segment selector for every 16 bytes.
 
> IIRC, the usual scheme for huge mode support was to always use 64KB
> segments, and then tweak the segment arithmetic to match the tiling.
> So for real mode, consecutive segments for a huge object were 4096
> apart.  In PM, the OS allocated consecutive selectors, which were
> numerically 8 apart.  So an 80KB huge object might be in segments
> 0x1234 and 0x2234 in real mode, and in selectors 0x4567 and 0x456F in
> protected mode.  The environment provided some run-time readable
> variables to define the increment (4096 vs. 8) and corresponding shift
> counts (12 and 3).

MS used the same compiler for DOS and OS/2. You could even generate
bound executables that would run on either, so it isn't so easy to
change the addressing system for different targets.  So, there was
at least one that would generate selectors 4K apart. That does
have the disadvantage that you can only go to 16 segments.

But they might have done both at different times. I mostly remember
the MSC 6.0 compiler.

-- glen