• Open Menu Close Menu
  • Apple
  • Shopping Bag
  • Apple
  • Mac
  • iPad
  • iPhone
  • Watch
  • TV
  • Music
  • Support
  • Search apple.com
  • Shopping Bag

Lists

Open Menu Close Menu
  • Terms and Conditions
  • Lists hosted on this site
  • Email the Postmaster
  • Tips for posting to public mailing lists
Re: malloc was optimized out
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: malloc was optimized out


  • Subject: Re: malloc was optimized out
  • From: David Duncan <email@hidden>
  • Date: Mon, 04 Jul 2016 23:41:11 -0700


On Jul 4, 2016, at 11:25 PM, Dmitry Markman <email@hidden> wrote:

you’re right

our problem isn’t clang malloc optimization 

(after list’s discussion I checked carefully release build generated code (assembler) and _malloc call was there).

it looks like it’s more related to memory overcommitment

unfortunately we didn’t see any dialog you described, application just crashed

Which version of macOS?

Did you get a crash report?

There isn’t a hard crash on overcommit – the system just pages in VM until it runs out of disk space. In particular, I would expect you to see your laptop run out of disk space if you were in an overcommit situation.


what was confusing that debug build isn’t failing, only release build of our library crashes application

This highly implies that some of your code is not behaving as you expect in release. In particular, optimizations often take advantage of undefined behavior to reduce and remove code.

An example is that code like this:

char *allocation = malloc(size);
if (*allocation && allocation) { … }

Can cause the if check to be removed an the associated block to always execute after optimization, because *allocation is undefined, and therefore the compiler can assume that it must succeed in order for the program to well defined, and thus the NULL check will be elided due to the dereference (that is, *allocation is only defined if allocation != NULL, therefore allocation cannot be NULL). Undefined behavior like this can easily propagate to large swaths of code as well. And keep in mind that reversing the order (that is, if (allocation && *allocation)) would not necessarily have the same behavior.


linux/windows aren’t failing for either build (on the comparable hardware)

no complain, though just FYI 

we’re using  clang from Xcode 7.2.1 toolchain, g++ 4.9.2 and VS2013 for Mac OS X, linux and windows respectively

thanks



On Jul 4, 2016, at 11:21 PM, David Duncan <email@hidden> wrote:


On Jul 4, 2016, at 7:11 PM, Dmitry Markman <email@hidden> wrote:

Hi Jens

I think your explanation makes sense, thank you very much

I’m asking for 0x1000000000000 or 281474976710656B which is ~250TB on my MacBook Pro with 1TB SSD and 16GB RAM and for release build it doesn’t return NULL

I can’t reproduce this. If I ask for 256TB I immediately get back NULL and a malloc error. I tried your formulation on macOS Sierra and got back NULL.

You probably need to further specify your particular case (that is what tools you are building with).

in our case, customer is trying to build huge simulink model and we’d like to error out at the very early stage (allocation)

and notify customer that it’s not possible to create such a model.

and in our real case we trying to allocate about 10 chunks of memory 200GB each, so it’s far less than in my example

malloc() should only fail on macOS if you run out of VM space, and a simple test app will show that this occurs at around 128TB. At this point malloc() returns NULL and reports error 3.

Now actually trying to use that memory may cause the process attempting to do so to be halted if there isn’t enough system memory + swap to sustain that requirement. If the OS cannot provide more physical memory, then it will halt some set of processes and present the user with a dialog to allow them to recover. This is at least the behavior on recent systems (at least as of 10.10).

Given your statement that you are basically just doing allocations without showing a corresponding free(), it seems unlikely that the compiler would actually optimize this away. In particular it is exceedingly unlikely that the compiler would optimize a malloc/free pair across a function boundary.

My suspicion is that you actually have an issue in the intermediate layer that you mentioned, but as you haven’t presented the precise case it is hard to speculate further.

--
David Duncan


Dmitry Markman


--
David Duncan

 _______________________________________________
Do not post admin requests to the list. They will be ignored.
Xcode-users mailing list      (email@hidden)
Help/Unsubscribe/Update your Subscription:

This email sent to email@hidden

References: 
 >Re: malloc was optimized out (From: Dmitry Markman <email@hidden>)
 >Re: malloc was optimized out (From: Roland King <email@hidden>)
 >Re: malloc was optimized out (From: Dmitry Markman <email@hidden>)
 >Re: malloc was optimized out (From: Roland King <email@hidden>)
 >Re: malloc was optimized out (From: Jens Alfke <email@hidden>)
 >Re: malloc was optimized out (From: Dmitry Markman <email@hidden>)
 >Re: malloc was optimized out (From: David Duncan <email@hidden>)
 >Re: malloc was optimized out (From: Dmitry Markman <email@hidden>)

  • Prev by Date: Re: malloc was optimized out
  • Next by Date: Re: malloc was optimized out
  • Previous by thread: Re: malloc was optimized out
  • Next by thread: Re: malloc was optimized out
  • Index(es):
    • Date
    • Thread