• Open Menu Close Menu
  • Apple
  • Shopping Bag
  • Apple
  • Mac
  • iPad
  • iPhone
  • Watch
  • TV
  • Music
  • Support
  • Search apple.com
  • Shopping Bag

Lists

Open Menu Close Menu
  • Terms and Conditions
  • Lists hosted on this site
  • Email the Postmaster
  • Tips for posting to public mailing lists
Re: malloc was optimized out
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: malloc was optimized out

  • Subject: Re: malloc was optimized out
  • From: Chris Cleeland <email@hidden>
  • Date: Mon, 04 Jul 2016 22:27:34 -0500
Whether you agree or disagree with what clang does, it doesn't matter.  clang is doing what it is doing; *YOU* have to deal with it in your code.

Your contrived example is not at all like the example you provided where the pointers in a struct-returned-by-value are assigned to the return value of malloc().  In the contrived example clang is well within its rights to optimize out the call to malloc as has been explained well by others (yes, I know you don't agree but that honestly doesn't matter because, frankly, you're wrong in that regard).

I think that if you want to report a bug, you need to model things after your more complicated example because based solely on what you show it seems less likely that the optimizer has enough information to optimize out a malloc.

If you really want to know how much space is available for heap allocation, you're going to have to try to touch every page in your requested set.  Even then there would be cases you can't accommodate such as when memory gets freed up you wouldn't know about it.

To really do this correctly, you are going to have to dive down and get intimate with the VM subsystem of each OS on which you run, and hopefully create an abstraction over it that permits you to do what you feel you need. 

But, at this point, based on the evidence, clang is not your problem.


On Mon, Jul 4, 2016 at 9:59 PM, Dmitry Markman <email@hidden> wrote:
yes I tried it was 0


few people said here that malloc never returns NULL and therefore code like ((data = "" == NULL) can be optimized out

however I create very simple program where malloc does return NULL even in release build

so clang is at very least inconsistent


consider 2 cpp files and one header

1. memory_use.hpp

struct S {
    unsigned char *mem;
};

S *allocate_structure(size_t sz);

void dellocate_structure(S *& s);


2. memory_use.cpp

#include "memory_use.hpp"

#include <iostream>

S* allocate_structure(size_t sz) {
    S *s = (S *)malloc(sizeof(S));
    if((s->mem = (unsigned char *)malloc(sz)) == nullptr) {
        std::cout << "allocate_structure mem allocation failure" << std::endl;
        free(s);
        return nullptr;
    }
    return s;
}

void dellocate_structure(S *& s) {
    if(s == nullptr) return;
    free(s->mem);
    free(s);
    s = nullptr;
}


3. main.cpp


int main(int argc, const char * argv[]) {
    size_t need_size = 0x1000000000000;
//      size_t need_size = 0x400000000000;

    std::cout << "need_size = " << need_size << std::endl;

    S *s = allocate_structure(need_size);
    if(s == nullptr) {
        std::cout << "allocation failure" << std::endl;
        return 1;
    } else {
        if(s->mem) std::cout << "success" << std::endl;
        dellocate_structure(s);
    }

    return 0;
}


if I ask need_size = 0x1000000000000 (~250TB)

I’m getting

need_size = 281474976710656
new_malloc_problem(35609,0x100080000) malloc: *** mach_vm_map(size=281474976710656) failed (error code=3)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
allocate_structure mem allocation failure
allocation failure
Program ended with exit code: 1


if I ask need_size = 0x400000000000 (~60TB)

I’m getting

need_size = 70368744177664
success
Program ended with exit code: 0


so as you can see Jens was right and for 0x1000000000000 malloc did return NULL

in the code above optimization didn’t occurred and malloc did return NULL

since

1. compiler hardly can predict runtime hardware behavior at the compile time and

2. resulting code can run on different hardware configuration

3. code checks result of the malloc

compiler shouldn’t optimize malloc out

or runtime code should never return NULL for malloc.

otherwise behavior is not consistent and unpredictable

if malloc never returns NULL (except some edge cases as Jens explained) then we have

very bad situation, because we can’t reliably detect memory allocation failure

before actual memory use  :(((


thanks all

dm
















> On Jul 4, 2016, at 10:19 PM, Carl Hoefs <email@hidden> wrote:
>
> RETURN VALUES
>     If successful, calloc(), malloc(), realloc(), reallocf(), and valloc() functions return
>     a pointer to allocated memory.  If there is an error, they return a NULL pointer and set
>     errno to ENOMEM.
>
>>> An implementation of malloc that never returns NULL is still compliant to the standard.
>
> Have you tried checking errno after the malloc?
> -Carl
>
>
>
>> On Jul 4, 2016, at 7:13 PM, Dmitry Markman <email@hidden> wrote:
>>
>> thanks
>>
>> unfortunately you started to play with words
>>
>> I asked for ~250TB of data from my MacBook Pro with 1TB SSD and 16GB of RAM
>>
>> clearly that amount can’t be given (in any circumstances)
>>
>> so if compiler returns something that isn’t NULL it’s very troubling
>>
>> thanks again
>>
>> dm
>>
>>
>>
>>> On Jul 4, 2016, at 9:20 PM, Clark Cox <email@hidden> wrote:
>>>
>>>
>>>> On Jul 4, 2016, at 15:53, Dmitry Markman <email@hidden> wrote:
>>>>
>>>> Hmm
>>>>
>>>> behavior you described is good receipt to un-robust software
>>>>
>>>> IMHO, all talks about “effective", “typically" and so on and on is matter of interpretation
>>>>
>>>> and are much less important than standard
>>>
>>> An implementation of malloc that never returns NULL is still compliant to the standard.
>>>
>>>> first of all there is a standard and nothing more. In some cases standard says that behavior is undefined
>>>>
>>>> but in case of “malloc" and "operator new” everything is well defined
>>>
>>> The standard says that malloc returns NULL in case of failure. It however, says nothing about what constitutes a failure, or *any* situations in which NULL must be returned..
>>>
>>>
>>>
>>> --
>>> Clark Smith Cox III
>>> email@hidden
>>>
>>
>> Dmitry Markman
>>
>>
>> _______________________________________________
>> Do not post admin requests to the list. They will be ignored.
>> Xcode-users mailing list      (email@hidden)
>> Help/Unsubscribe/Update your Subscription:
>>
>> This email sent to email@hidden
>

Dmitry Markman


 _______________________________________________
Do not post admin requests to the list. They will be ignored.
Xcode-users mailing list      (email@hidden)
Help/Unsubscribe/Update your Subscription:

This email sent to email@hidden

 _______________________________________________
Do not post admin requests to the list. They will be ignored.
Xcode-users mailing list      (email@hidden)
Help/Unsubscribe/Update your Subscription:

This email sent to email@hidden
  • Follow-Ups:
    • Re: malloc was optimized out
      • From: Dmitry Markman <email@hidden>
References: 
 >malloc was optimized out (From: Dmitry Markman <email@hidden>)
 >Re: malloc was optimized out (From: Clark Cox <email@hidden>)
 >Re: malloc was optimized out (From: Dmitry Markman <email@hidden>)
 >Re: malloc was optimized out (From: Clark Cox <email@hidden>)
 >Re: malloc was optimized out (From: Dmitry Markman <email@hidden>)
 >Re: malloc was optimized out (From: Clark Cox <email@hidden>)
 >Re: malloc was optimized out (From: Dmitry Markman <email@hidden>)
 >Re: malloc was optimized out (From: Carl Hoefs <email@hidden>)
 >Re: malloc was optimized out (From: Dmitry Markman <email@hidden>)

  • Prev by Date: Re: malloc was optimized out
  • Next by Date: Re: malloc was optimized out
  • Previous by thread: Re: malloc was optimized out
  • Next by thread: Re: malloc was optimized out
  • Index(es):
    • Date
    • Thread