nmm1@cus.cam.ac.uk (Nick Maclaren) writes:> In article <1128335888.327425@haldjas.folklore.ee>, > Sander Vesik <sander@haldjas.folklore.ee> writes: > |> In comp.arch Nick Maclaren <nmm1@cus.cam.ac.uk> wrote: > |> > > |> > No. Replacing malloc is not supported in C, and often breaks > |> > the nastier vendor and third-party libraries. The reason is that > |> > the suite of malloc-related functions is not independent, and they > |> > often use non-standard interfaces. > |> > |> There are still systems where you can reasonably do so - shipping > |> more than one possible malloc to use is a good hint for example. > > It's a hint that the VENDOR can do it - but not that the application > programmer can. Yes, an experienced hacker can, which was the basis > of my remark that I can, but I can't advise users to.Ahhh, now I know why I've never had any problems replacing malloc on Linux, Solaris or the Amiga's OS. I'm a VENDOR! (or maybe not?) -- David Gay dgay@acm.org
Regarding calculation of free memory
Started by ●September 28, 2005
Reply by ●October 3, 20052005-10-03
Reply by ●October 3, 20052005-10-03
In article <s71ek7237zp.fsf@beryl.CS.Berkeley.EDU>, David Gay <dgay@beryl.CS.Berkeley.EDU> writes: |> |> > |> > No. Replacing malloc is not supported in C, and often breaks |> > |> > the nastier vendor and third-party libraries. The reason is that |> > |> > the suite of malloc-related functions is not independent, and they |> > |> > often use non-standard interfaces. |> > |> |> > |> There are still systems where you can reasonably do so - shipping |> > |> more than one possible malloc to use is a good hint for example. |> > |> > It's a hint that the VENDOR can do it - but not that the application |> > programmer can. Yes, an experienced hacker can, which was the basis |> > of my remark that I can, but I can't advise users to. |> |> Ahhh, now I know why I've never had any problems replacing malloc on Linux, |> Solaris or the Amiga's OS. I'm a VENDOR! (or maybe not?) Or perhaps you just have a slightly narrower range of experience than I do? I have seen failures on half a dozen Unices, including Solaris and Linux. Of course, as I said, it also depends on what other software you are using in your application. Regards, Nick Maclaren.
Reply by ●October 3, 20052005-10-03
nmm1@cus.cam.ac.uk (Nick Maclaren) writes:> In article <s71ek7237zp.fsf@beryl.CS.Berkeley.EDU>, > David Gay <dgay@beryl.CS.Berkeley.EDU> writes: > |> > |> > |> > No. Replacing malloc is not supported in C, and often breaks > |> > |> > the nastier vendor and third-party libraries. The reason is that > |> > |> > the suite of malloc-related functions is not independent, and they > |> > |> > often use non-standard interfaces. > |> > |> > |> > |> There are still systems where you can reasonably do so - shipping > |> > |> more than one possible malloc to use is a good hint for example. > |> > > |> > It's a hint that the VENDOR can do it - but not that the application > |> > programmer can. Yes, an experienced hacker can, which was the basis > |> > of my remark that I can, but I can't advise users to. > |> > |> Ahhh, now I know why I've never had any problems replacing malloc on Linux, > |> Solaris or the Amiga's OS. I'm a VENDOR! (or maybe not?) > > Or perhaps you just have a slightly narrower range of experience > than I do? > > I have seen failures on half a dozen Unices, including Solaris > and Linux. Of course, as I said, it also depends on what other > software you are using in your application.Well, frankly, for the Linux case, you can go and look at the source code to check if there's any extra magic entry points or other dependencies. For the others, it definitely pays to check linker output files to make sure you didn't, e.g., accidentally end up with two copies of malloc. Yes, you'll have to repeat such checks if you change libc version - that's a price you have to pay. And if you're going to say "but I've run into problems", I'd like to get a specific description, rather than some nebulous "It happened to me somewhere, sometime, somehow". -- David Gay dgay@acm.org
Reply by ●October 3, 20052005-10-03
In article <s71wtkuilia.fsf@beryl.CS.Berkeley.EDU>, David Gay <dgay@beryl.CS.Berkeley.EDU> wrote:> >> |> > >> |> > It's a hint that the VENDOR can do it - but not that the application >> |> > programmer can. Yes, an experienced hacker can, which was the basis >> |> > of my remark that I can, but I can't advise users to. >> |> >> |> Ahhh, now I know why I've never had any problems replacing malloc on Linux, >> |> Solaris or the Amiga's OS. I'm a VENDOR! (or maybe not?) >> >> Or perhaps you just have a slightly narrower range of experience >> than I do? >> >> I have seen failures on half a dozen Unices, including Solaris >> and Linux. Of course, as I said, it also depends on what other >> software you are using in your application. > >Well, frankly, for the Linux case, you can go and look at the source code >to check if there's any extra magic entry points or other dependencies. For >the others, it definitely pays to check linker output files to make sure >you didn't, e.g., accidentally end up with two copies of malloc. Yes, >you'll have to repeat such checks if you change libc version - that's a >price you have to pay. And if you're going to say "but I've run into >problems", I'd like to get a specific description, rather than some >nebulous "It happened to me somewhere, sometime, somehow".As I said, this is (usually) within the ability of an experienced hacker. Do you SERIOUSLY think that it is reasonable for me to tell a Fortran programmer to find out what version of malloc a system is running (when he did not install it) and to inspect both the source code of that malloc and its replacement? As far as Solaris etc. goes, you first have to find out which version (or versions) of malloc the application is using, INCLUDING those called by dynamic libraries. That is unusually hard under Solaris, because ldd doesn't do what you apparently thinks that it does, and a fair amount of reverse engineering of the behaviour of the linker and loader is needed. Then you have to extract the relevant symbols, see which library is using what, and decide if there is a clash. Oh, and for non-standard symbols, you may have to reverse engineer the binary to see what they do. Regards, Nick Maclaren.
Reply by ●October 3, 20052005-10-03
"Nick Maclaren" <nmm1@cus.cam.ac.uk> wrote in message news:dhqq8e$hn8$1@gemini.csx.cam.ac.uk...> In article <fmY%e.6314$oc.4151@newsread2.news.pas.earthlink.net>, > Hank Oredson <horedson@earthlink.net> wrote: >>> >>> Problem one: in C, there is no memory management beyond stack scoped >>> except by using malloc. Any algorithm that requires more than that >>> can't be done if you don't use malloc. >> >>sbrk()? > > Not in C - in fact, I can't think of any standard that it IS in, > because it makes such extremely horrible assumptions about the > implementation.How strange. I use it. In C.>>> Problem two: there are many library facilities that use malloc either >>> explicitly or implicitly - including almost all I/O - you would have >>> to avoid them, too. >> >>Provide your own malloc() so the I/O that needs it will get it >>from your own memory pool. Since you are doing the I/O, >>you can provide optimal buffering. > > No. Replacing malloc is not supported in C, and often breaks > the nastier vendor and third-party libraries. The reason is that > the suite of malloc-related functions is not independent, and they > often use non-standard interfaces.How strange. I use my own malloc(). In C.> Yes, I have done it, over many years - but it isn't something that > I recommend to users. Or can :-(Why not? Gives you complete control. You can tune your malloc() to your application. -- ... Hank http://home.earthlink.net/~horedson http://home.earthlink.net/~w0rli
Reply by ●October 3, 20052005-10-03
"Nick Maclaren" <nmm1@cus.cam.ac.uk> wrote in message news:dhrnh7$n65$1@gemini.csx.cam.ac.uk...> > In article <s71ek7237zp.fsf@beryl.CS.Berkeley.EDU>, > David Gay <dgay@beryl.CS.Berkeley.EDU> writes: > |> > |> > |> > No. Replacing malloc is not supported in C, and often breaks > |> > |> > the nastier vendor and third-party libraries. The reason is > that > |> > |> > the suite of malloc-related functions is not independent, and > they > |> > |> > often use non-standard interfaces. > |> > |> > |> > |> There are still systems where you can reasonably do so - shipping > |> > |> more than one possible malloc to use is a good hint for example. > |> > > |> > It's a hint that the VENDOR can do it - but not that the application > |> > programmer can. Yes, an experienced hacker can, which was the basis > |> > of my remark that I can, but I can't advise users to. > |> > |> Ahhh, now I know why I've never had any problems replacing malloc on > Linux, > |> Solaris or the Amiga's OS. I'm a VENDOR! (or maybe not?) > > Or perhaps you just have a slightly narrower range of experience > than I do? > > I have seen failures on half a dozen Unices, including Solaris > and Linux. Of course, as I said, it also depends on what other > software you are using in your application.My original comments re sbrk() and malloc() were not about Unix. And even with various Unices, they still apply. If your application can be smart about it's use of memory, that can be a huge win. On any OS. Including Unix flavored ones. -- ... Hank http://home.earthlink.net/~horedson http://home.earthlink.net/~w0rli
Reply by ●October 3, 20052005-10-03
"Nick Maclaren" <nmm1@cus.cam.ac.uk> wrote in message news:dhrs58$3a5$1@gemini.csx.cam.ac.uk...> In article <s71wtkuilia.fsf@beryl.CS.Berkeley.EDU>, > David Gay <dgay@beryl.CS.Berkeley.EDU> wrote: >> >>> |> > >>> |> > It's a hint that the VENDOR can do it - but not that the >>> application >>> |> > programmer can. Yes, an experienced hacker can, which was the >>> basis >>> |> > of my remark that I can, but I can't advise users to. >>> |> >>> |> Ahhh, now I know why I've never had any problems replacing malloc on >>> Linux, >>> |> Solaris or the Amiga's OS. I'm a VENDOR! (or maybe not?) >>> >>> Or perhaps you just have a slightly narrower range of experience >>> than I do? >>> >>> I have seen failures on half a dozen Unices, including Solaris >>> and Linux. Of course, as I said, it also depends on what other >>> software you are using in your application. >> >>Well, frankly, for the Linux case, you can go and look at the source code >>to check if there's any extra magic entry points or other dependencies. >>For >>the others, it definitely pays to check linker output files to make sure >>you didn't, e.g., accidentally end up with two copies of malloc. Yes, >>you'll have to repeat such checks if you change libc version - that's a >>price you have to pay. And if you're going to say "but I've run into >>problems", I'd like to get a specific description, rather than some >>nebulous "It happened to me somewhere, sometime, somehow". > > As I said, this is (usually) within the ability of an experienced hacker. > Do you SERIOUSLY think that it is reasonable for me to tell a Fortran > programmer to find out what version of malloc a system is running > (when he did not install it) and to inspect both the source code of > that malloc and its replacement?Yes. Any programmer should know her tools well. That includes the details of how any libriaries work, and knowledge of other libraries available to provide similar support functions. NTRAN comes to mind ...> As far as Solaris etc. goes, you first have to find out which version > (or versions) of malloc the application is using, INCLUDING those > called by dynamic libraries. That is unusually hard under Solaris, > because ldd doesn't do what you apparently thinks that it does, and > a fair amount of reverse engineering of the behaviour of the linker > and loader is needed. Then you have to extract the relevant symbols, > see which library is using what, and decide if there is a clash.Nonesense.> Oh, and for non-standard symbols, you may have to reverse engineer > the binary to see what they do.-- ... Hank http://home.earthlink.net/~horedson http://home.earthlink.net/~w0rli
Reply by ●October 3, 20052005-10-03
In article <dhqq8e$hn8$1@gemini.csx.cam.ac.uk>, Nick Maclaren <nmm1@cus.cam.ac.uk> wrote:> >No. Replacing malloc is not supported in C, and often breaks >the nastier vendor and third-party libraries. The reason is that >the suite of malloc-related functions is not independent, and they >often use non-standard interfaces.That is only if people insist on trying to re-use free'd memory in unsupported ways or otherwise abuse a quite simple interface. The C malloc() interface is quite simple: You get memory if the overlying system lords decide to grant it, and you can do whatever you want with it within reason (so long as you don't make assumptions about its underlying byte/word length/order, etc.) but you can't make any assumptions whatsoever about being able to get it back if your free() it or about having it be visible in any predictable way from anyplace outside the code segment that allocated it. By the way, the system lords may lie to you, just as your bank is likely to do if you try to withdraw a large amount of cash all at once, because they already loaned it out to someone else. Replacing malloc() is quite supported in C, just as is implementing your own version of sqrt(). That's obviously the point of any reasonable programming language. It's a way to convince a stupid CPU to do work you want done. You just can't make any underlying assumptions about the implementation of the vendor function or your ability to interoperate with it. The bottom line is that the computer does what you tell it to do, not necessarily what you want. Shouting at the deaf is seldom effective. This endless bickering about malloc() never ceases to amaze me. Programmers simply won't accept that you can't really shove fifty pounds of shit into a twenty pound sack. They also can't accept that someone else may have stolen the booty before they got there, or even in between the time that they measured the bag and started shoveling. It's Chinatown, Jake.
Reply by ●October 3, 20052005-10-03
In article <dhrs58$3a5$1@gemini.csx.cam.ac.uk>, Nick Maclaren <nmm1@cus.cam.ac.uk> wrote:> >As I said, this is (usually) within the ability of an experienced hacker. >Do you SERIOUSLY think that it is reasonable for me to tell a Fortran >programmer to find out what version of malloc a system is running >(when he did not install it) and to inspect both the source code of >that malloc and its replacement?No self respecting Fortran programmer is going to believe seductive lies from an operating system or a program library.
Reply by ●October 3, 20052005-10-03
Hank Oredson wrote:> "Nick Maclaren" <nmm1@cus.cam.ac.uk> wrote in message > news:dhqq8e$hn8$1@gemini.csx.cam.ac.uk... > >>In article <fmY%e.6314$oc.4151@newsread2.news.pas.earthlink.net>, >>Hank Oredson <horedson@earthlink.net> wrote: >> >>>>Problem one: in C, there is no memory management beyond stack scoped >>>>except by using malloc. Any algorithm that requires more than that >>>>can't be done if you don't use malloc. >>> >>>sbrk()? >> >>Not in C - in fact, I can't think of any standard that it IS in, >>because it makes such extremely horrible assumptions about the >>implementation. > > How strange. I use it. In C.Nick's point is that it is not a part of the C standard. My Linux manpage for sbrk() says to #include <unistd.h> and under the "CONFORMING TO" heading says this: brk and sbrk are not defined in the C Standard and are deliberately excluded from the POSIX.1 standard (see paragraphs B.1.1.1.3 and B.8.3.3). Of course you *can* call it from C, but you can't expect it to be there on all systems, especially "bare metal" embedded systems. -- Michael N. Moran (h) 770 516 7918 5009 Old Field Ct. (c) 678 521 5460 Kennesaw, GA, USA 30144 http://mnmoran.org "So often times it happens, that we live our lives in chains and we never even know we have the key." The Eagles, "Already Gone" The Beatles were wrong: 1 & 1 & 1 is 1