How Physical Memory Mapping / Allocation works

Started by Ilias Abrams February 26, 2018
Hi guys,

I am new on software programming and I am trying to understand some basic/ important
things.I am interested in Bare-metal software programming, so please give me answers
related on this. My question is: who is responsible and manage the physical memory
mapping / allocation? Meaning, If I have 1GB physical memory on my system, the 1)
CPU or 2) the Bootloader (like GRUB, U-boot)  is responsible for the memory
addresses that I can use for my program? 
Is hardware specific architecture? Each platform manufacture has their own memory
address mapping scheme and layout?
Furthermore, is there any standart way that you have to partition the physical
memory? meaning, give some memory for I/Oports and so on?

Sorry for the mess, but I dont understand when you create/ build your software, how
you know what range of physical addresses you can use for start/entry point,  stack
and so on. 

Please help me to clear all this in my mind. If you can give me examples will help
me more
P.S. Programming in C 

Thanks
On 02/26/2018 05:07 PM, Ilias Abrams wrote:
> Hi guys, > > I am new on software programming and I am trying to understand some basic/
important things.I am interested in Bare-metal software programming, so please give me answers related on this. My question is: who is responsible and manage the physical memory mapping / allocation? Meaning, If I have 1GB physical memory on my system, the 1) CPU or 2) the Bootloader (like GRUB, U-boot) is responsible for the memory addresses that I can use for my program?
> Is hardware specific architecture? Each platform manufacture has their own memory
address mapping scheme and layout?
> Furthermore, is there any standart way that you have to partition the physical
memory? meaning, give some memory for I/Oports and so on?
> > Sorry for the mess, but I dont understand when you create/ build your software,
how you know what range of physical addresses you can use for start/entry point, stack
> and so on. > > Please help me to clear all this in my mind. If you can give me examples will help
me more
> P.S. Programming in C > > Thanks >
If you have a gig of RAM on your system, it's pretty unlikely that you're programming "on the metal". Especially when you're talking about GRUB. In any case the compiler looks after the userspace memory layout. (In that class of hardware, you'd expect to have a built-in memory management unit to translate from logical to physical addresses.) Cheers Phil Hobbs -- Dr Philip C D Hobbs Principal Consultant ElectroOptical Innovations LLC / Hobbs ElectroOptics Optics, Electro-optics, Photonics, Analog Electronics Briarcliff Manor NY 10510 http://electrooptical.net http://hobbs-eo.com
On 26/02/18 23:07, Ilias Abrams wrote:
> Hi guys, > > I am new on software programming and I am trying to understand some > basic/ important things.I am interested in Bare-metal software > programming, so please give me answers related on this. My question > is: who is responsible and manage the physical memory mapping / > allocation? Meaning, If I have 1GB physical memory on my system, the > 1) CPU or 2) the Bootloader (like GRUB, U-boot) is responsible for > the memory addresses that I can use for my program? Is hardware > specific architecture? Each platform manufacture has their own memory > address mapping scheme and layout? Furthermore, is there any standart > way that you have to partition the physical memory? meaning, give > some memory for I/Oports and so on? > > Sorry for the mess, but I dont understand when you create/ build your > software, how you know what range of physical addresses you can use > for start/entry point, stack and so on. > > Please help me to clear all this in my mind. If you can give me > examples will help me more P.S. Programming in C > > Thanks >
What sort of systems are you talking about here? Very roughly, you can divide processors into two categories. There are "big" ones designed for running multiple programs from different sources. These use a "memory management unit" which maps the logical addresses (seen by the program code) into physical addresses (used by the memory itself). The OS is responsible for setting up this mapping, and making sure each process can only access the memory it is allowed to use. And there are "small" ones designed for running a single program, either "bare metal" or with a small "real-time operating system", with all the code compiled and linked at one time. Then the logical addresses and physical addresses are the same, and you often don't have any kind of memory protection at all. "Bare metal" programming means there is no OS or significant software layers between the main code and the hardware - and that usually means small devices with no memory mappings. GB memory sizes usually means big devices with a full OS (like Linux). But there are some cross-over areas - bootloaders like Grub and OS kernels are "bare metal" on the big devices. Memory allocation is a completely different issue. It is always handled by software, but may be done in different ways in different levels.
On 27.2.18 00:07, Ilias Abrams wrote:
> Hi guys, > > I am new on software programming and I am trying to understand some basic/
important things.I am interested in Bare-metal software programming, so please give me answers related on this. My question is: who is responsible and manage the physical memory mapping / allocation? Meaning, If I have 1GB physical memory on my system, the 1) CPU or 2) the Bootloader (like GRUB, U-boot) is responsible for the memory addresses that I can use for my program?
> Is hardware specific architecture? Each platform manufacture has their own memory
address mapping scheme and layout?
> Furthermore, is there any standart way that you have to partition the physical
memory? meaning, give some memory for I/Oports and so on?
> > Sorry for the mess, but I dont understand when you create/ build your software,
how you know what range of physical addresses you can use for start/entry point, stack
> and so on. > > Please help me to clear all this in my mind. If you can give me examples will help
me more
> P.S. Programming in C > > Thanks
You did not say it, but it seems that you're interested how Linux does it, in a pretty customary way. The subject is quite large, and there are good books, e.g. 'Understanding the Linux Kernel' by Daniel P. Bovet et al. I found a PDF of it at <http://www.johnchukwuma.com/training/UnderstandingTheLinuxKernel3rdEdition.pdf> In principle, a mapped memory system needs two allocation systems: for logical addresses and for physical memory. -- -TV
&Tau;&eta; &Tau;&rho;&#943;&tau;&eta;, 27
&Phi;&epsilon;&beta;&rho;&omicron;&upsilon;&alpha;&rho;&#943;&omicron;&upsilon; 2018
- 7:51:33 &pi;.&mu;. UTC, &omicron; &chi;&rho;&#942;&sigma;&tau;&eta;&sigmaf; David
Brown &#941;&gamma;&rho;&alpha;&psi;&epsilon;:
> On 26/02/18 23:07, Ilias Abrams wrote: > > Hi guys, > > > > I am new on software programming and I am trying to understand some > > basic/ important things.I am interested in Bare-metal software > > programming, so please give me answers related on this. My question > > is: who is responsible and manage the physical memory mapping / > > allocation? Meaning, If I have 1GB physical memory on my system, the > > 1) CPU or 2) the Bootloader (like GRUB, U-boot) is responsible for > > the memory addresses that I can use for my program? Is hardware > > specific architecture? Each platform manufacture has their own memory > > address mapping scheme and layout? Furthermore, is there any standart > > way that you have to partition the physical memory? meaning, give > > some memory for I/Oports and so on? > > > > Sorry for the mess, but I dont understand when you create/ build your > > software, how you know what range of physical addresses you can use > > for start/entry point, stack and so on. > > > > Please help me to clear all this in my mind. If you can give me > > examples will help me more P.S. Programming in C > > > > Thanks > > > > What sort of systems are you talking about here? Very roughly, you can > divide processors into two categories. >
> > And there are "small" ones designed for running a single program, either > "bare metal" or with a small "real-time operating system", with all the > code compiled and linked at one time. Then the logical addresses and > physical addresses are the same, and you often don't have any kind of > memory protection at all. > >
Thanks David, I would like to focus on this more. The 1GB memory that I said is only for example. Can you please give more details? Microcontrollers work on that way? Basically I want to write a program with no OS. Only my software run on the hardware and have 1-1 mapping meaning my physical address space translated to logical addresses. I dont know so much yet about programming but I think that C make use of the logical addresses and not the physical but with 1-1 mapping maybe can achieve this.
&Tau;&eta; &Tau;&rho;&#943;&tau;&eta;, 27
&Phi;&epsilon;&beta;&rho;&omicron;&upsilon;&alpha;&rho;&#943;&omicron;&upsilon; 2018
- 7:51:33 &pi;.&mu;. UTC, &omicron; &chi;&rho;&#942;&sigma;&tau;&eta;&sigmaf; David
Brown &#941;&gamma;&rho;&alpha;&psi;&epsilon;:
> On 26/02/18 23:07, Ilias Abrams wrote: > > Hi guys, > > > > I am new on software programming and I am trying to understand some > > basic/ important things.I am interested in Bare-metal software > > programming, so please give me answers related on this. My question > > is: who is responsible and manage the physical memory mapping / > > allocation? Meaning, If I have 1GB physical memory on my system, the > > 1) CPU or 2) the Bootloader (like GRUB, U-boot) is responsible for > > the memory addresses that I can use for my program? Is hardware > > specific architecture? Each platform manufacture has their own memory > > address mapping scheme and layout? Furthermore, is there any standart > > way that you have to partition the physical memory? meaning, give > > some memory for I/Oports and so on? > > > > Sorry for the mess, but I dont understand when you create/ build your > > software, how you know what range of physical addresses you can use > > for start/entry point, stack and so on. > > > > Please help me to clear all this in my mind. If you can give me > > examples will help me more P.S. Programming in C > > > > Thanks > > > > What sort of systems are you talking about here? Very roughly, you can > divide processors into two categories. > > There are "big" ones designed for running multiple programs from > different sources. These use a "memory management unit" which maps the > logical addresses (seen by the program code) into physical addresses > (used by the memory itself). The OS is responsible for setting up this > mapping, and making sure each process can only access the memory it is > allowed to use. > > And there are "small" ones designed for running a single program, either > "bare metal" or with a small "real-time operating system", with all the > code compiled and linked at one time. Then the logical addresses and > physical addresses are the same, and you often don't have any kind of > memory protection at all. > > > "Bare metal" programming means there is no OS or significant software > layers between the main code and the hardware - and that usually means > small devices with no memory mappings. GB memory sizes usually means > big devices with a full OS (like Linux). But there are some cross-over > areas - bootloaders like Grub and OS kernels are "bare metal" on the big > devices. > > > Memory allocation is a completely different issue. It is always handled > by software, but may be done in different ways in different levels.
Thanks David, I would like to focus on this more. The 1GB memory that I said is only for example. Can you please give more details? Microcontrollers work on that way? Basically I want to write a program with no OS. Only my software run on the hardware and have 1-1 mapping meaning my physical address space translated to logical addresses. I dont know so much yet about programming but I think that C make use of the logical addresses and not the physical but with 1-1 mapping maybe can achieve this.
On 27/02/18 11:34, Ilias Abrams wrote:
> &Tau;&eta; &Tau;&rho;&#943;&tau;&eta;, 27
&Phi;&epsilon;&beta;&rho;&omicron;&upsilon;&alpha;&rho;&#943;&omicron;&upsilon; 2018 - 7:51:33 &pi;.&mu;. UTC, &omicron; &chi;&rho;&#942;&sigma;&tau;&eta;&sigmaf; David Brown &#941;&gamma;&rho;&alpha;&psi;&epsilon;:
>> >> What sort of systems are you talking about here? Very roughly, you can >> divide processors into two categories. >> > >> >> And there are "small" ones designed for running a single program, either >> "bare metal" or with a small "real-time operating system", with all the >> code compiled and linked at one time. Then the logical addresses and >> physical addresses are the same, and you often don't have any kind of >> memory protection at all. >> >> > Thanks David, I would like to focus on this more. The 1GB memory that > I said is only for example. Can you please give more details? > Microcontrollers work on that way? Basically I want to write a > program with no OS. Only my software run on the hardware and have 1-1 > mapping meaning my physical address space translated to logical > addresses. I dont know so much yet about programming but I think that > C make use of the logical addresses and not the physical but with 1-1 > mapping maybe can achieve this. >
You are not making much sense here. It is like saying you'd like to go on a car holiday, and would prefer a manual gear car to an automatic. No one could give you advice because they don't know where you want to go, what you want to do, if you want to buy a care or hire it, or anything else. Or it is like going to an author's conference and saying "I'd like to write a book. Perhaps it will be hardback, but that's just an example. Do the numbers on the bottom of the pages match the physical sheets of paper in book printing?" Saying "I want to write a program" is equally vague. We have no idea if you are trying to program on a PC, or have bought a microcontroller based embedded card, or even if you have a target at all. We have no idea what your program would be doing, or what you know or don't know about the matter. We don't know if you are a student trying to get help with some homework, a professional who has been given a task way outside your experience, or a hobby developer just trying to learn something new. You have to take a step back and think about what you know, and where you want to go, and try to give a better description of it. And be ready for when the advice you get is "don't start with C" or "don't worry about physical memory mappings - you'll get there in a couple of years".
On Tue, 27 Feb 2018 02:36:48 -0800, Ilias Abrams wrote:
[ ... ]
> Thanks David, I would like to focus on this more. The 1GB memory that I > said is only for example. Can you please give more details? > Microcontrollers work on that way? Basically I want to write a program > with no OS. Only my software run on the hardware and have 1-1 mapping > meaning my physical address space translated to logical addresses. I > dont know so much yet about programming but I think that C make use of > the logical addresses and not the physical but with 1-1 mapping maybe > can achieve this.
The quickest way to get hands-on experience is to check out the Arduino, and its comprehensive web site <https://www.arduino.cc/> After that, a more general environment, AVRFreaks, for example. With the more general software you can dig in to the .h files and linker control files to see what's being done for you. Also the datasheets and programmer's documentation for whatever line of processors you've chosen. Read, read, read.
&Tau;&eta; &Tau;&rho;&#943;&tau;&eta;, 27
&Phi;&epsilon;&beta;&rho;&omicron;&upsilon;&alpha;&rho;&#943;&omicron;&upsilon; 2018
- 1:34:17 &mu;.&mu;. UTC, &omicron; &chi;&rho;&#942;&sigma;&tau;&eta;&sigmaf; David
Brown &#941;&gamma;&rho;&alpha;&psi;&epsilon;:
> On 27/02/18 11:34, Ilias Abrams wrote: > > &Tau;&eta; &Tau;&rho;&#943;&tau;&eta;, 27
&Phi;&epsilon;&beta;&rho;&omicron;&upsilon;&alpha;&rho;&#943;&omicron;&upsilon; 2018 - 7:51:33 &pi;.&mu;. UTC, &omicron; &chi;&rho;&#942;&sigma;&tau;&eta;&sigmaf; David Brown &#941;&gamma;&rho;&alpha;&psi;&epsilon;:
> >> > >> What sort of systems are you talking about here? Very roughly, you can > >> divide processors into two categories. > >> > > > >> > >> And there are "small" ones designed for running a single program, either > >> "bare metal" or with a small "real-time operating system", with all the > >> code compiled and linked at one time. Then the logical addresses and > >> physical addresses are the same, and you often don't have any kind of > >> memory protection at all. > >> > >> > > Thanks David, I would like to focus on this more. The 1GB memory that > > I said is only for example. Can you please give more details? > > Microcontrollers work on that way? Basically I want to write a > > program with no OS. Only my software run on the hardware and have 1-1 > > mapping meaning my physical address space translated to logical > > addresses. I dont know so much yet about programming but I think that > > C make use of the logical addresses and not the physical but with 1-1 > > mapping maybe can achieve this. > > > > You are not making much sense here. It is like saying you'd like to go > on a car holiday, and would prefer a manual gear car to an automatic. > No one could give you advice because they don't know where you want to > go, what you want to do, if you want to buy a care or hire it, or > anything else. > > Or it is like going to an author's conference and saying "I'd like to > write a book. Perhaps it will be hardback, but that's just an example. > Do the numbers on the bottom of the pages match the physical sheets of > paper in book printing?" > > Saying "I want to write a program" is equally vague. We have no idea if > you are trying to program on a PC, or have bought a microcontroller > based embedded card, or even if you have a target at all. We have no > idea what your program would be doing, or what you know or don't know > about the matter. > > We don't know if you are a student trying to get help with some > homework, a professional who has been given a task way outside your > experience, or a hobby developer just trying to learn something new. > > You have to take a step back and think about what you know, and where > you want to go, and try to give a better description of it. And be > ready for when the advice you get is "don't start with C" or "don't > worry about physical memory mappings - you'll get there in a couple of > years".
You are absolutely right. Lets start again, I am a student and I have a project. I want to work on ARM architecture and not on x86. I am working some C examples on RPI2 and 3 and also I have a microcontroller Tiva C TM4C123G Launchpad. I want to right a baremetal software that can save registers values into the memory address like stack lets say and print those values from memory to screen. But I want to use C and assembly (maybe inline assembly) and I want to use physical addresses (If its not possible to use directly the physical addresses maybe I can achieve a 1-1 mapping logical to physical) What are your thoughts about that? Can you guide me what I have to do? Thanks!
Il giorno marted&igrave; 27 febbraio 2018 19:11:09 UTC+1, Ilias Abrams ha scritto:
> &Tau;&eta; &Tau;&rho;&#943;&tau;&eta;, 27
&Phi;&epsilon;&beta;&rho;&omicron;&upsilon;&alpha;&rho;&#943;&omicron;&upsilon; 2018 - 1:34:17 &mu;.&mu;. UTC, &omicron; &chi;&rho;&#942;&sigma;&tau;&eta;&sigmaf; David Brown &#941;&gamma;&rho;&alpha;&psi;&epsilon;:
> > On 27/02/18 11:34, Ilias Abrams wrote: > > > &Tau;&eta; &Tau;&rho;&#943;&tau;&eta;, 27
&Phi;&epsilon;&beta;&rho;&omicron;&upsilon;&alpha;&rho;&#943;&omicron;&upsilon; 2018 - 7:51:33 &pi;.&mu;. UTC, &omicron; &chi;&rho;&#942;&sigma;&tau;&eta;&sigmaf; David Brown &#941;&gamma;&rho;&alpha;&psi;&epsilon;:
> > >> > > >> What sort of systems are you talking about here? Very roughly, you can > > >> divide processors into two categories. > > >> > > > > > >> > > >> And there are "small" ones designed for running a single program, either > > >> "bare metal" or with a small "real-time operating system", with all the > > >> code compiled and linked at one time. Then the logical addresses and > > >> physical addresses are the same, and you often don't have any kind of > > >> memory protection at all. > > >> > > >> > > > Thanks David, I would like to focus on this more. The 1GB memory that > > > I said is only for example. Can you please give more details? > > > Microcontrollers work on that way? Basically I want to write a > > > program with no OS. Only my software run on the hardware and have 1-1 > > > mapping meaning my physical address space translated to logical > > > addresses. I dont know so much yet about programming but I think that > > > C make use of the logical addresses and not the physical but with 1-1 > > > mapping maybe can achieve this. > > > > > > > You are not making much sense here. It is like saying you'd like to go > > on a car holiday, and would prefer a manual gear car to an automatic. > > No one could give you advice because they don't know where you want to > > go, what you want to do, if you want to buy a care or hire it, or > > anything else. > > > > Or it is like going to an author's conference and saying "I'd like to > > write a book. Perhaps it will be hardback, but that's just an example. > > Do the numbers on the bottom of the pages match the physical sheets of > > paper in book printing?" > > > > Saying "I want to write a program" is equally vague. We have no idea if > > you are trying to program on a PC, or have bought a microcontroller > > based embedded card, or even if you have a target at all. We have no > > idea what your program would be doing, or what you know or don't know > > about the matter. > > > > We don't know if you are a student trying to get help with some > > homework, a professional who has been given a task way outside your > > experience, or a hobby developer just trying to learn something new. > > > > You have to take a step back and think about what you know, and where > > you want to go, and try to give a better description of it. And be > > ready for when the advice you get is "don't start with C" or "don't > > worry about physical memory mappings - you'll get there in a couple of > > years". > > You are absolutely right. Lets start again, I am a student and I have a project. I
want to work on ARM architecture and not on x86. I am working some C examples on RPI2 and 3 and also I have a microcontroller Tiva C TM4C123G Launchpad.
> I want to right a baremetal software that can save registers values into the
memory address like stack lets say and print those values from memory to screen. But I want to use C and assembly (maybe inline assembly) and I want to use physical addresses (If its not possible to use directly the physical addresses maybe I can achieve a 1-1 mapping logical to physical)
> What are your thoughts about that? Can you guide me what I have to do? > Thanks!
Guide you is the job of your professor, he is paid for it, so use him. Said that, forget the RPi, you can't use it bare metal, too complicated. Start with an Arduino (but it's not ARM) or some board from mbed.org (they have ARM). Start with something simple, like an ARM of the Cortex M family (there is a good book about the Cortex M: "The Definitive Guide to ARM Cortex M", you'll find it on Amazon). If you want to start with something smaller Atmel AVR are good (used in Arduino), or NXP (Freescale) HCS08 (Book: "HCS08 Unleashed", very good, with examples). Bye Jack