How Physical Memory Mapping / Allocation works

Hi guys,

I am new on software programming and I am trying to understand some basic/ important things.I am interested in Bare-metal software programming, so please give me answers related on this. My question is: who is responsible and manage the physical memory mapping / allocation? Meaning, If I have 1GB physical memory on my system, the 1) CPU or 2) the Bootloader (like GRUB, U-boot)  is responsible for the memory addresses that I can use for my program? 
Is hardware specific architecture? Each platform manufacture has their own memory address mapping scheme and layout?
Furthermore, is there any standart way that you have to partition the physical memory? meaning, give some memory for I/Oports and so on?

Sorry for the mess, but I dont understand when you create/ build your software, how you know what range of physical addresses you can use for start/entry point,  stack
and so on. 

Please help me to clear all this in my mind. If you can give me examples will help me more
P.S. Programming in C 

Thanks

On 02/26/2018 05:07 PM, Ilias Abrams wrote:
> Hi guys,
> 
> I am new on software programming and I am trying to understand some basic/ important things.I am interested in Bare-metal software programming, so please give me answers related on this. My question is: who is responsible and manage the physical memory mapping / allocation? Meaning, If I have 1GB physical memory on my system, the 1) CPU or 2) the Bootloader (like GRUB, U-boot)  is responsible for the memory addresses that I can use for my program?
> Is hardware specific architecture? Each platform manufacture has their own memory address mapping scheme and layout?
> Furthermore, is there any standart way that you have to partition the physical memory? meaning, give some memory for I/Oports and so on?
> 
> Sorry for the mess, but I dont understand when you create/ build your software, how you know what range of physical addresses you can use for start/entry point,  stack
> and so on.
> 
> Please help me to clear all this in my mind. If you can give me examples will help me more
> P.S. Programming in C
> 
> Thanks
> 

If you have a gig of RAM on your system, it's pretty unlikely that 
you're programming "on the metal".  Especially when you're talking about 
GRUB.

In any case the compiler looks after the userspace memory layout.  (In 
that class of hardware, you'd expect to have a built-in memory 
management unit to translate from logical to physical addresses.)

Cheers

Phil Hobbs

-- 
Dr Philip C D Hobbs
Principal Consultant
ElectroOptical Innovations LLC / Hobbs ElectroOptics
Optics, Electro-optics, Photonics, Analog Electronics
Briarcliff Manor NY 10510

http://electrooptical.net
http://hobbs-eo.com

On 26/02/18 23:07, Ilias Abrams wrote:
> Hi guys,
> 
> I am new on software programming and I am trying to understand some
> basic/ important things.I am interested in Bare-metal software
> programming, so please give me answers related on this. My question
> is: who is responsible and manage the physical memory mapping /
> allocation? Meaning, If I have 1GB physical memory on my system, the
> 1) CPU or 2) the Bootloader (like GRUB, U-boot)  is responsible for
> the memory addresses that I can use for my program? Is hardware
> specific architecture? Each platform manufacture has their own memory
> address mapping scheme and layout? Furthermore, is there any standart
> way that you have to partition the physical memory? meaning, give
> some memory for I/Oports and so on?
> 
> Sorry for the mess, but I dont understand when you create/ build your
> software, how you know what range of physical addresses you can use
> for start/entry point,  stack and so on.
> 
> Please help me to clear all this in my mind. If you can give me
> examples will help me more P.S. Programming in C
> 
> Thanks
> 

What sort of systems are you talking about here?  Very roughly, you can
divide processors into two categories.

There are "big" ones designed for running multiple programs from
different sources.  These use a "memory management unit" which maps the
logical addresses (seen by the program code) into physical addresses
(used by the memory itself).  The OS is responsible for setting up this
mapping, and making sure each process can only access the memory it is
allowed to use.

And there are "small" ones designed for running a single program, either
"bare metal" or with a small "real-time operating system", with all the
code compiled and linked at one time.  Then the logical addresses and
physical addresses are the same, and you often don't have any kind of
memory protection at all.


"Bare metal" programming means there is no OS or significant software
layers between the main code and the hardware - and that usually means
small devices with no memory mappings.  GB memory sizes usually means
big devices with a full OS (like Linux).  But there are some cross-over
areas - bootloaders like Grub and OS kernels are "bare metal" on the big
devices.


Memory allocation is a completely different issue.  It is always handled
by software, but may be done in different ways in different levels.

On 27.2.18 00:07, Ilias Abrams wrote:
> Hi guys,
> 
> I am new on software programming and I am trying to understand some basic/ important things.I am interested in Bare-metal software programming, so please give me answers related on this. My question is: who is responsible and manage the physical memory mapping / allocation? Meaning, If I have 1GB physical memory on my system, the 1) CPU or 2) the Bootloader (like GRUB, U-boot)  is responsible for the memory addresses that I can use for my program?
> Is hardware specific architecture? Each platform manufacture has their own memory address mapping scheme and layout?
> Furthermore, is there any standart way that you have to partition the physical memory? meaning, give some memory for I/Oports and so on?
> 
> Sorry for the mess, but I dont understand when you create/ build your software, how you know what range of physical addresses you can use for start/entry point,  stack
> and so on.
> 
> Please help me to clear all this in my mind. If you can give me examples will help me more
> P.S. Programming in C
> 
> Thanks


You did not say it, but it seems that you're interested how Linux
does it, in a pretty customary way.

The subject is quite large, and there are good books, e.g.
'Understanding the Linux Kernel' by Daniel P. Bovet et al.
I found a PDF of it at 
<http://www.johnchukwuma.com/training/UnderstandingTheLinuxKernel3rdEdition.pdf>

In principle, a mapped memory system needs two allocation
systems: for logical addresses and for physical memory.

-- 

-TV

Τη Τρίτη, 27 Φεβρουαρίου 2018 - 7:51:33 π.μ. UTC, ο χρήστης David Brown έγραψε:
> On 26/02/18 23:07, Ilias Abrams wrote:
> > Hi guys,
> > 
> > I am new on software programming and I am trying to understand some
> > basic/ important things.I am interested in Bare-metal software
> > programming, so please give me answers related on this. My question
> > is: who is responsible and manage the physical memory mapping /
> > allocation? Meaning, If I have 1GB physical memory on my system, the
> > 1) CPU or 2) the Bootloader (like GRUB, U-boot)  is responsible for
> > the memory addresses that I can use for my program? Is hardware
> > specific architecture? Each platform manufacture has their own memory
> > address mapping scheme and layout? Furthermore, is there any standart
> > way that you have to partition the physical memory? meaning, give
> > some memory for I/Oports and so on?
> > 
> > Sorry for the mess, but I dont understand when you create/ build your
> > software, how you know what range of physical addresses you can use
> > for start/entry point,  stack and so on.
> > 
> > Please help me to clear all this in my mind. If you can give me
> > examples will help me more P.S. Programming in C
> > 
> > Thanks
> > 
> 
> What sort of systems are you talking about here?  Very roughly, you can
> divide processors into two categories.
> 

> 
> And there are "small" ones designed for running a single program, either
> "bare metal" or with a small "real-time operating system", with all the
> code compiled and linked at one time.  Then the logical addresses and
> physical addresses are the same, and you often don't have any kind of
> memory protection at all.
> 
> 
Thanks David, I would like to focus on this more. The 1GB memory that I said is only for example. Can you please give more details? Microcontrollers work on that way? Basically I want to write a program with no OS. Only my software run on the hardware and have 1-1 mapping meaning my physical address space translated to logical addresses. I dont know so much yet about programming but I think that C make use of the logical addresses and not the physical but with 1-1 mapping maybe can achieve this.

Τη Τρίτη, 27 Φεβρουαρίου 2018 - 7:51:33 π.μ. UTC, ο χρήστης David Brown έγραψε:
> On 26/02/18 23:07, Ilias Abrams wrote:
> > Hi guys,
> > 
> > I am new on software programming and I am trying to understand some
> > basic/ important things.I am interested in Bare-metal software
> > programming, so please give me answers related on this. My question
> > is: who is responsible and manage the physical memory mapping /
> > allocation? Meaning, If I have 1GB physical memory on my system, the
> > 1) CPU or 2) the Bootloader (like GRUB, U-boot)  is responsible for
> > the memory addresses that I can use for my program? Is hardware
> > specific architecture? Each platform manufacture has their own memory
> > address mapping scheme and layout? Furthermore, is there any standart
> > way that you have to partition the physical memory? meaning, give
> > some memory for I/Oports and so on?
> > 
> > Sorry for the mess, but I dont understand when you create/ build your
> > software, how you know what range of physical addresses you can use
> > for start/entry point,  stack and so on.
> > 
> > Please help me to clear all this in my mind. If you can give me
> > examples will help me more P.S. Programming in C
> > 
> > Thanks
> > 
> 
> What sort of systems are you talking about here?  Very roughly, you can
> divide processors into two categories.
> 
> There are "big" ones designed for running multiple programs from
> different sources.  These use a "memory management unit" which maps the
> logical addresses (seen by the program code) into physical addresses
> (used by the memory itself).  The OS is responsible for setting up this
> mapping, and making sure each process can only access the memory it is
> allowed to use.
> 
> And there are "small" ones designed for running a single program, either
> "bare metal" or with a small "real-time operating system", with all the
> code compiled and linked at one time.  Then the logical addresses and
> physical addresses are the same, and you often don't have any kind of
> memory protection at all.
> 
> 
> "Bare metal" programming means there is no OS or significant software
> layers between the main code and the hardware - and that usually means
> small devices with no memory mappings.  GB memory sizes usually means
> big devices with a full OS (like Linux).  But there are some cross-over
> areas - bootloaders like Grub and OS kernels are "bare metal" on the big
> devices.
> 
> 
> Memory allocation is a completely different issue.  It is always handled
> by software, but may be done in different ways in different levels.

Thanks David, I would like to focus on this more. The 1GB memory that I said is only for example. Can you please give more details? Microcontrollers work on that way? Basically I want to write a program with no OS. Only my software run on the hardware and have 1-1 mapping meaning my physical address space translated to logical addresses. I dont know so much yet about programming but I think that C make use of the logical addresses and not the physical but with 1-1 mapping maybe can achieve this. 

On 27/02/18 11:34, Ilias Abrams wrote:
> Τη Τρίτη, 27 Φεβρουαρίου 2018 - 7:51:33 π.μ. UTC, ο χρήστης David Brown έγραψε:
>>
>> What sort of systems are you talking about here?  Very roughly, you can
>> divide processors into two categories.
>>
> 
>>
>> And there are "small" ones designed for running a single program, either
>> "bare metal" or with a small "real-time operating system", with all the
>> code compiled and linked at one time.  Then the logical addresses and
>> physical addresses are the same, and you often don't have any kind of
>> memory protection at all.
>>
>>
> Thanks David, I would like to focus on this more. The 1GB memory that
> I said is only for example. Can you please give more details?
> Microcontrollers work on that way? Basically I want to write a
> program with no OS. Only my software run on the hardware and have 1-1
> mapping meaning my physical address space translated to logical
> addresses. I dont know so much yet about programming but I think that
> C make use of the logical addresses and not the physical but with 1-1
> mapping maybe can achieve this.
> 

You are not making much sense here.  It is like saying you'd like to go
on a car holiday, and would prefer a manual gear car to an automatic.
No one could give you advice because they don't know where you want to
go, what you want to do, if you want to buy a care or hire it, or
anything else.

Or it is like going to an author's conference and saying "I'd like to
write a book.  Perhaps it will be hardback, but that's just an example.
 Do the numbers on the bottom of the pages match the physical sheets of
paper in book printing?"

Saying "I want to write a program" is equally vague.  We have no idea if
you are trying to program on a PC, or have bought a microcontroller
based embedded card, or even if you have a target at all.  We have no
idea what your program would be doing, or what you know or don't know
about the matter.

We don't know if you are a student trying to get help with some
homework, a professional who has been given a task way outside your
experience, or a hobby developer just trying to learn something new.

You have to take a step back and think about what you know, and where
you want to go, and try to give a better description of it.  And be
ready for when the advice you get is "don't start with C" or "don't
worry about physical memory mappings - you'll get there in a couple of
years".

On Tue, 27 Feb 2018 02:36:48 -0800, Ilias Abrams wrote:
[ ... ]
> Thanks David, I would like to focus on this more. The 1GB memory that I
> said is only for example. Can you please give more details?
> Microcontrollers work on that way? Basically I want to write a program
> with no OS. Only my software run on the hardware and have 1-1 mapping
> meaning my physical address space translated to logical addresses. I
> dont know so much yet about programming but I think that C make use of
> the logical addresses and not the physical but with 1-1 mapping maybe
> can achieve this.

The quickest way to get hands-on experience is to check out the Arduino, 
and its comprehensive web site <https://www.arduino.cc/>
After that, a more general environment, AVRFreaks, for example.  With the 
more general software you can dig in to the .h files and linker control 
files to see what's being done for you.
Also the datasheets and programmer's documentation for whatever line of 
processors you've chosen.
Read, read, read.

Τη Τρίτη, 27 Φεβρουαρίου 2018 - 1:34:17 μ.μ. UTC, ο χρήστης David Brown έγραψε:
> On 27/02/18 11:34, Ilias Abrams wrote:
> > Τη Τρίτη, 27 Φεβρουαρίου 2018 - 7:51:33 π.μ. UTC, ο χρήστης David Brown έγραψε:
> >>
> >> What sort of systems are you talking about here?  Very roughly, you can
> >> divide processors into two categories.
> >>
> > 
> >>
> >> And there are "small" ones designed for running a single program, either
> >> "bare metal" or with a small "real-time operating system", with all the
> >> code compiled and linked at one time.  Then the logical addresses and
> >> physical addresses are the same, and you often don't have any kind of
> >> memory protection at all.
> >>
> >>
> > Thanks David, I would like to focus on this more. The 1GB memory that
> > I said is only for example. Can you please give more details?
> > Microcontrollers work on that way? Basically I want to write a
> > program with no OS. Only my software run on the hardware and have 1-1
> > mapping meaning my physical address space translated to logical
> > addresses. I dont know so much yet about programming but I think that
> > C make use of the logical addresses and not the physical but with 1-1
> > mapping maybe can achieve this.
> > 
> 
> You are not making much sense here.  It is like saying you'd like to go
> on a car holiday, and would prefer a manual gear car to an automatic.
> No one could give you advice because they don't know where you want to
> go, what you want to do, if you want to buy a care or hire it, or
> anything else.
> 
> Or it is like going to an author's conference and saying "I'd like to
> write a book.  Perhaps it will be hardback, but that's just an example.
>  Do the numbers on the bottom of the pages match the physical sheets of
> paper in book printing?"
> 
> Saying "I want to write a program" is equally vague.  We have no idea if
> you are trying to program on a PC, or have bought a microcontroller
> based embedded card, or even if you have a target at all.  We have no
> idea what your program would be doing, or what you know or don't know
> about the matter.
> 
> We don't know if you are a student trying to get help with some
> homework, a professional who has been given a task way outside your
> experience, or a hobby developer just trying to learn something new.
> 
> You have to take a step back and think about what you know, and where
> you want to go, and try to give a better description of it.  And be
> ready for when the advice you get is "don't start with C" or "don't
> worry about physical memory mappings - you'll get there in a couple of
> years".

You are absolutely right. Lets start again, I am a student and I have a project. I want to work on ARM architecture and not on x86. I am working some C examples on RPI2 and 3 and also I have a microcontroller Tiva C TM4C123G Launchpad. 
I want to right a baremetal software that can save registers values into the memory address like stack lets say and print those values from memory to screen. But I want to use C and assembly (maybe inline assembly) and I want to use physical addresses (If its not possible to use directly the physical addresses maybe I can achieve a 1-1 mapping logical to physical) 
What are your thoughts about that? Can you guide me what I have to do? 
Thanks!

Il giorno martedì 27 febbraio 2018 19:11:09 UTC+1, Ilias Abrams ha scritto:
> Τη Τρίτη, 27 Φεβρουαρίου 2018 - 1:34:17 μ.μ. UTC, ο χρήστης David Brown έγραψε:
> > On 27/02/18 11:34, Ilias Abrams wrote:
> > > Τη Τρίτη, 27 Φεβρουαρίου 2018 - 7:51:33 π.μ. UTC, ο χρήστης David Brown έγραψε:
> > >>
> > >> What sort of systems are you talking about here?  Very roughly, you can
> > >> divide processors into two categories.
> > >>
> > > 
> > >>
> > >> And there are "small" ones designed for running a single program, either
> > >> "bare metal" or with a small "real-time operating system", with all the
> > >> code compiled and linked at one time.  Then the logical addresses and
> > >> physical addresses are the same, and you often don't have any kind of
> > >> memory protection at all.
> > >>
> > >>
> > > Thanks David, I would like to focus on this more. The 1GB memory that
> > > I said is only for example. Can you please give more details?
> > > Microcontrollers work on that way? Basically I want to write a
> > > program with no OS. Only my software run on the hardware and have 1-1
> > > mapping meaning my physical address space translated to logical
> > > addresses. I dont know so much yet about programming but I think that
> > > C make use of the logical addresses and not the physical but with 1-1
> > > mapping maybe can achieve this.
> > > 
> > 
> > You are not making much sense here.  It is like saying you'd like to go
> > on a car holiday, and would prefer a manual gear car to an automatic.
> > No one could give you advice because they don't know where you want to
> > go, what you want to do, if you want to buy a care or hire it, or
> > anything else.
> > 
> > Or it is like going to an author's conference and saying "I'd like to
> > write a book.  Perhaps it will be hardback, but that's just an example.
> >  Do the numbers on the bottom of the pages match the physical sheets of
> > paper in book printing?"
> > 
> > Saying "I want to write a program" is equally vague.  We have no idea if
> > you are trying to program on a PC, or have bought a microcontroller
> > based embedded card, or even if you have a target at all.  We have no
> > idea what your program would be doing, or what you know or don't know
> > about the matter.
> > 
> > We don't know if you are a student trying to get help with some
> > homework, a professional who has been given a task way outside your
> > experience, or a hobby developer just trying to learn something new.
> > 
> > You have to take a step back and think about what you know, and where
> > you want to go, and try to give a better description of it.  And be
> > ready for when the advice you get is "don't start with C" or "don't
> > worry about physical memory mappings - you'll get there in a couple of
> > years".
> 
> You are absolutely right. Lets start again, I am a student and I have a project. I want to work on ARM architecture and not on x86. I am working some C examples on RPI2 and 3 and also I have a microcontroller Tiva C TM4C123G Launchpad. 
> I want to right a baremetal software that can save registers values into the memory address like stack lets say and print those values from memory to screen. But I want to use C and assembly (maybe inline assembly) and I want to use physical addresses (If its not possible to use directly the physical addresses maybe I can achieve a 1-1 mapping logical to physical) 
> What are your thoughts about that? Can you guide me what I have to do? 
> Thanks!

Guide you is the job of your professor, he is paid for it, so use him.

Said that, forget the RPi, you can't use it bare metal, too complicated.

Start with an Arduino (but it's not ARM) or some board from mbed.org (they have ARM).
Start with something simple, like an ARM of the Cortex M family (there is a good book about the Cortex M: "The Definitive Guide to ARM Cortex M", you'll find it on Amazon).

If you want to start with something smaller Atmel AVR are good (used in Arduino), or NXP (Freescale) HCS08 (Book: "HCS08 Unleashed", very good, with examples).

Bye Jack