>
> Need to know the size of Memory that this Deflate algorithm will eat
> while residing and while execution . Are you aware of it ?
>
> Normally the compression / decompression algorithm results will be very
> poor if we compress Executable files . But how does this 'Deflate' work
> here ???
Hi,
Fine, Thankyou for your ideas and feedbacks which are really very
helpful.
Warm Regards,
Karthik Balaguru
Paul Keinanen wrote:
> On 28 Sep 2006 00:30:19 -0700, "KBG"
> <karthik.balaguru@lntinfotech.com> wrote:
>
> >Normally the compression / decompression algorithm results will be very
> >poor if we compress Executable files .
>
> An experienced person reading disassembled code can quite easily
> detect which compiler has been used to generate the code and quite
> reliably guess what the high level language source statements were.
> This clearly proves that a compiler generates various idiomatic
> sequences, which are easily detectable by a trained eye and are often
> repeated multiple times in a program and should thus be easily be
> compressed to one or two bytes. The same applies to frequent macro
> expansions in assemblers as well as any text strings (menus etc).
>
> Since you apparently only need to go from 1.4 MiB (not 2 MiB as
> originally claimed) down to 1 MiB, this should not be too hard, even
> considering the decompression code size.
>
> Paul
Reply by Paul Keinanen●September 28, 20062006-09-28
On 28 Sep 2006 00:30:19 -0700, "KBG"
<karthik.balaguru@lntinfotech.com> wrote:
>Normally the compression / decompression algorithm results will be very
>poor if we compress Executable files .
An experienced person reading disassembled code can quite easily
detect which compiler has been used to generate the code and quite
reliably guess what the high level language source statements were.
This clearly proves that a compiler generates various idiomatic
sequences, which are easily detectable by a trained eye and are often
repeated multiple times in a program and should thus be easily be
compressed to one or two bytes. The same applies to frequent macro
expansions in assemblers as well as any text strings (menus etc).
Since you apparently only need to go from 1.4 MiB (not 2 MiB as
originally claimed) down to 1 MiB, this should not be too hard, even
considering the decompression code size.
Paul
Reply by Clifford Heath●September 28, 20062006-09-28
KBG wrote:
>>>>>>>> Yes, i too believe that this is a good idea and need to work about it.
So stop asking us to do your work for you and GO AND LOOK AT THE
THINGS WE'VE ALREADY TOLD YOU.
> Need to know the size of Memory that this Deflate algorithm will eat
> while residing and while execution . Are you aware of it ?
Yes. No, I won't tell you - it's time you go and *find out*.
To get you started, here's some facts from my Linux box:
# ls -l /usr/src/linux/arch/i386/boot/compressed/
[..snip..]
-rwxr-xr-x 1 root root 1041787 2006-07-22 17:13 vmlinux*
-rwxr-xr-x 1 root root 2025956 2006-07-22 17:13 vmlinux.bin*
-rw-r--r-- 1 root root 1027095 2006-07-22 17:13 vmlinux.bin.gz
The .bin file is uncompressed, the .bin.gz is compressed, and
the other file has the decompressor at the start. That's for
an Intel architechure machine; you'll have to "gzip" your own
binary image file to find out how it works on your architecture.
Now run along and do some experiments and googling by yourself.
It's p^%$ easy for any half-competent programmer to find the
facts you want, so there's no reason to continue asking here.
Reply by KBG●September 28, 20062006-09-28
Hi,
Ok, so embed a decompression algorithm into your bootloader and
you're good to go. Linux uses deflate (the zip/gzip algorithm),
of which versions are freely available. Look at how Linux does
it.
>>>>>>> Yes, i too believe that this is a good idea and need to work about it.
Need to know the size of Memory that this Deflate algorithm will eat
while residing and while execution . Are you aware of it ?
Normally the compression / decompression algorithm results will be very
poor if we compress Executable files . But how does this 'Deflate' work
here ???
Is it successful in compressing executables / binaries with a good
compression ratio ?
Here i Need to compress/decompress the ELF files, BIN files .
Kindly share your views / ideas.
Tonnes of Thx in advans,
Karthik Balaguru
Reply by Clifford Heath●September 27, 20062006-09-27
KBG wrote:
> Bootloader execution is done as 'executing in-place' in
> Flash. But the application gets executed by getting copied to RAM.
> This copying is done by Bootloader as in any other normal embedded systems.
Ok, so embed a decompression algorithm into your bootloader and
you're good to go. Linux uses deflate (the zip/gzip algorithm),
of which versions are freely available. Look at how Linux does
it.
Reply by KBG●September 27, 20062006-09-27
Hi,
Are you bootloading from FLASH into RAM and executing from there?
>>>>>>>> Bootloader execution is done as 'executing in-place' in
Flash. But the application gets executed by getting copied to RAM.
This
copying is done by Bootloader as in any other normal embedded systems.
If so, you need to compress the program into FLASH and uncompress
it into RAM, in a similar way to how the Linux kernel bootloader
works.
>>>>>>>>> Do you mean to say that i need to incorporate some compression algorithms and dump into Flash and later the bootloader code should uncompress it while copying to the RAM area for execution ? I too beleive that this is one of the good ideas . But, the compression algorithms - Need to think about it . Is there any compression algorithm available in open-source for this kind of task oriented with embedded application or any tool for this kind of activitiy ?
I think, Splitting and putting-together or vice-versa is going to be a
bit easy but this also needs synchronisation and buffer management .
But, the absence of compression algorithms will ease the task here.
If there is some easy way oriented with this splitting &
putting-together idea , kindly share with me.
Which of the above method to adopt in this situation so that it would
be time-effective ?
Kindly share your ideas / views regarding this.
Regards,
Karthik Balaguru
Reply by Arlet●September 27, 20062006-09-27
KBG wrote:
> .XMAP file of the application has the following info w.r.t data & code
> for your reference :
> .text = 0x12C188 - 0xB0000 = 0x7C188
> .rodata = 0x134CD5 - 0x12C188 = 0x8B4D
> .data = 0x13620A - 0x134CE0 = 0x152A
> .bss = 0x22AB66 - 0x136210 = 0xF4956
According to this, you only have about 0.5 MB of text, rodata and data
combined. Is there additional stuff that goes into flash ? Or are you
trying to store the bss section in flash as well ? If so, check your
scripts that generate the flash image, and make sure the bss section is
removed. Instead, just clear the bss area at startup.
Reply by KBG●September 27, 20062006-09-27
Hi ,
1) Have you got optimisation set to optimise for size in the compiler?
>>>>>>>>> Yes size optimisation has been enabled.
2) If you are using ARM then you can compile in Thumb mode to gain a
reduction in size at a cost of a performance decrease.
>>>>>>>> But , i am using Coldfire MCF5272 in this embedded application.
3) If you can't add an extra flash device can you change your current
one for a bigger one that will fit in the same footprint?
>>>>>>>> Mass production issues are there and handling via software will be cost-effective.
4) Can you compress anything? In my application I have a number of big
bitmaps that must be stored as constants and I use Run Length Encoding
to dramatically reduce their size.
>>>>>>>> There are no bitmaps, Lot of tasks are running .
However, what you are doing is firefighting and the only long term
solution is redesign the code to be more efficient or make space in
the
hardware to fit your new code. Look again at was has changed to bloat
your code to double its size and think about what is necessary and
what
is not. If you are using other people's modules then this will eb a
lot
more difficult but their documentation may help.
>>>>>>>> I am trying via this method also but i beleive there should be some other
software tricks to manage this situation , so trying to design and
incorporate it.
Regards,
Karthik Balaguru
Reply by KBG●September 27, 20062006-09-27
Hi ,
I'm having trouble understanding how the application grew to 2MB when
your development platform only has 1 MB of flash??? Did you write an
extra 1MB of code before testing any of it???
>>>>>>>>>>>>>>>>>> It has exeeded in size so that it can not be placed
in Flash.
I do RAM execution, debugging and development . I did not try flashing
the application though many modules were getting integrated . Now the
code has got almost freezed as many modules have got finished . I did
try flashing now and there comes this issue.
It has not exactly doubled in size. It has crossed the 1.4 M mark.
You don't say whether or not you are executing in-place from the
flash,
or executing a copy out of RAM.
>>>>>>>>>>> Bootloader execution is done as 'executing in-place' in
Flash.
But the application gets executed by getting copied to RAM. This
copying is
done by Bootloader as in any other normal embedded systems. So, this
area also needs some tricks so that the image in bits/pieces can be put
together by some means and executed !!
How much (spare) RAM do you have? What percentage of your application
is
code vs data?
>>>>>>>>>>> RAM is not an issue. 32M is there and it is working
perfectly from
RAM . Only Flashing is the issue and i need tricks in that regard with
how to flash the big size image in bits / pieces as size has crossed
the Flash memory size and execute the bits / pieces as such or by
putting together in some groups with some tricks .
.XMAP file of the application has the following info w.r.t data & code
for your reference :
.text = 0x12C188 - 0xB0000 = 0x7C188
.rodata = 0x134CD5 - 0x12C188 = 0x8B4D
.data = 0x13620A - 0x134CE0 = 0x152A
.bss = 0x22AB66 - 0x136210 = 0xF4956
Regards,
Karthik Balaguru