May 2010 - Darwin-dev - lists.apple.com

Re: jemalloc to replace default allocator
by Jean-Daniel Dupas 07 May '10

07 May '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Le 7 mai 2010 à 11:59, Alastair Houghton a écrit : > On 6 May 2010, at 22:51, Joel Reymont wrote: > >> I would like to use jemalloc to replace the system memory allocator in Firefox. > > Why do that? My understanding is that jemalloc's primary advantage is massive scalability for heavily threaded code. Firefox isn't *that* heavily threaded, right? > > In any event, the system allocator on Mac OS X has always been pretty good, and it looks like the Snow Leopard version is already designed with per-processor "magazines" for small allocations. > > Personally, unless you have a really good justification for using jemalloc instead (and I'd argue that, for that, you need measurements showing that it's significantly faster than the system implementation), I'd stick with the system malloc. > Not only does it mean that Firefox will get faster with every optimisation Apple makes to the system malloc, but it also means that all of the malloc debugging tools and so on will work with Firefox. > > Kind regards, > Good advices, but how do you perform measurements without a way to replace the system allocator ;-) Joel, I think you can override malloc default implementation by using DYLD_INSERT_LIBRARIES and DYLD_FORCE_FLAT_NAMESPACE to inject your malloc library at launch runtime. But I agree with Alastair, I doubt you get any performance improvement using a custom allocator. The system allocator is pretty good for general purpose applications (like Firefox) -- Jean-Daniel _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: jemalloc to replace default allocator
by Joel Reymont 07 May '10

07 May '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:subject:mime-version :content-type:from:in-reply-to:date:cc:content-transfer-encoding :message-id:references:to:x-mailer; bh=vLP4OeRoYUP5c5GS/QHxZXyTXzQ2/PeBvJZjsyEEp6E=; b=gRVh/OuoLPe1PRIU2KqrsiIUMaW8jGr0bWhb2lGdbHgO5QGf80AJuS7wIvKKO6D25J xhMlCvZsvyb9NdLkMn7WaEuS0clK1PjgSCZ1wDr8R6GHalTtOG7o1omx2rNYcN8kuJ/y wJTK6rdepOoPFP753kux7Izgxe7Wc2FrU6Atw= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; b=qTBqCETdiBxbH4f24LIVuM8gAzPD8ql2Kk16/gS1VGIjxtAWpOAr8J2l++9lj/kfPA BPBawuIuWBQKYgrm7XG9mp0EX7UNGygai0ze64SgOSPyK/iAkPlffp4UIAKGmdcD64YY owToajkwHFd4yyIfFkntcl2GsEU/JBP/JPu/E= Alastair, On May 7, 2010, at 10:59 AM, Alastair Houghton wrote: > Personally, unless you have a really good justification for using jemalloc instead (and I'd argue that, for that, you need measurements showing that it's significantly faster than the system implementation), I'd stick with the system malloc. I get your point but let's just say this is something I've been tasked with. Apparently, the Javascript team discovered that too much time is being spent in free, or something like that. My goal is to plug in jemalloc and then let the Javascript team profile again. Thanks, Joel --- http://twitter.com/wagerlabs _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: jemalloc to replace default allocator
by Alastair Houghton 07 May '10

07 May '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha1; c=relaxed; d=coriolis-systems.com; h= subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; s=aug07; bh=VOtPCrAWx9KqXVM2cmdkAJoRQc0=; b=TJmN/nkGNoFSNep5Amo5rfgs7HXs HSrgpD0Rkfzva/OtV4L/nM/D5nK/uirEAH+QA0pk6opTGNdLUYtARO5RaLEWAaJt Uw/MDvgkaJBmArwuUFvZRWOS8SUmyVxvotZo7De+xxgjhcJr3P6rvcx+F1h380ix flnE5sd2bcf9/Mw= On 6 May 2010, at 22:51, Joel Reymont wrote: > I would like to use jemalloc to replace the system memory allocator in Firefox. Why do that? My understanding is that jemalloc's primary advantage is massive scalability for heavily threaded code. Firefox isn't *that* heavily threaded, right? In any event, the system allocator on Mac OS X has always been pretty good, and it looks like the Snow Leopard version is already designed with per-processor "magazines" for small allocations. Personally, unless you have a really good justification for using jemalloc instead (and I'd argue that, for that, you need measurements showing that it's significantly faster than the system implementation), I'd stick with the system malloc. Not only does it mean that Firefox will get faster with every optimisation Apple makes to the system malloc, but it also means that all of the malloc debugging tools and so on will work with Firefox. Kind regards, Alastair. -- http://alastairs-place.net _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

jemalloc to replace default allocator
by Joel Reymont 06 May '10

06 May '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:content-type :content-transfer-encoding:subject:date:message-id:to:mime-version :x-mailer; bh=meNDX2sz/gLQJ0+g/lusDimubPX2oxMgZJgZojyuYSo=; b=tct5EM35Jn0IJ3M5LB0VcArbOMczfBuHvrGoUV4UkDIjMzwkPGsmCcfv5r1RqZ+G8W TERQ8fqiAASxzBMA48bBY5BJOwSCJ0e+Vj3SQsVvRrXA1yaNsEfBdVdpf9WOjI43WZX0 ds7NzbP+9gWDoQebDLERoQD1iFZVaZMUAt174= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:content-type:content-transfer-encoding:subject:date:message-id :to:mime-version:x-mailer; b=pClm8D/xnZTKed3tR5n8DNXyCjAY3Ezlm6J09/fY89iWCwizV2myfJkrINSEOWPUIw q1+mUg9s3qAM2lbivcTtgspKWIlLi07122EKJYS+aQWEsTfR3VxLHaioHsNMXdlmL1gW 9gWn0aO4QcT062JdTHMyawebocIZKXPs2BgdE= I would like to use jemalloc to replace the system memory allocator in Firefox. My understanding is that it needs to hook into the zone system and there's existing code in Firefox that sort of does it [1]. According to Mozilla, "it is difficult to get our code to load fast enough to replace the default zone before other allocations happen" so a linker trick is used [2] to get it to happen early. What is the proper way to make it work (fast!) in 10.5 and 10.6? I'm not familiar with the zone system but I'm a quick learner, in case someone has pointers for me. Thanks in advance, Joel [1] http://mxr.mozilla.org/mozilla-central/source/memory/jemalloc/jemalloc.c#63… [2] http://mxr.mozilla.org/mozilla-central/source/memory/jemalloc/Makefile.in#1… --- http://twitter.com/wagerlabs _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: need help on porting from 32bit to 64bit.
by Alastair Houghton 03 May '10

03 May '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On 3 May 2010, at 20:09, Nathan Herring wrote: > GNU also has some compiler intrinsics to perform the same, so you can avoid the cost of the function call to OS services. IMO it's better to use the OSByteOrder/NSByteOrder/CFByteOrder APIs instead. They will expand to compiler intrinsics or assembly language as appropriate anyway. Which you use is, as Jean-Daniel remarked, mainly a matter of taste. Kind regards, Alastair. -- http://alastairs-place.net _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

RE: need help on porting from 32bit to 64bit.
by Nathan Herring 03 May '10

03 May '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Thread-index: AQHK4UsWDE8FDlQJZUuI5GvGFWzrkJItTW+AgAAK8gCAEsyAIA== Thread-topic: need help on porting from 32bit to 64bit. GNU also has some compiler intrinsics to perform the same, so you can avoid the cost of the function call to OS services. http://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html see __builtin_bswap32 __builtin_bswap64 -----Original Message----- From: darwin-dev-bounces+nathanh=microsoft.com(a)lists.apple.com [mailto:darwin-dev-bounces+nathanh=microsoft.com@lists.apple.com] On Behalf Of Jean-Daniel Dupas Sent: Wednesday, April 21, 2010 6:02 AM To: Arnab Ganguly Cc: darwin-dev(a)lists.apple.com Dev Subject: Re: need help on porting from 32bit to 64bit. Swapping byte on OS X is a matter of taste. OSSwap API (libkern) NSSwap API (Foundation) CFSwap API (CoreFoundation) Core Endian API (CoreServices) and probably other hidden somewhere. Le 21 avr. 2010 à 14:22, Nico Schmidt a écrit : > What about OSSwapBigToHostInt64 and OSSwapHostToBigInt64 both from #include <libkern/OSByteOrder.h> ? > > Nico > > On 21.04.2010, at 14:05, Arnab Ganguly wrote: > >> Hi All, >> I am porting my application from 32bit to 64bit.I see functions like ntohl and htonl is being used. What should I use it equivalent in 64bit code base. >> Thanks, >> -A -- Jean-Daniel _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: __ prefix in system call
by Terry Lambert 02 May '10

02 May '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: How can I tell if another process is 32 or 64 bit
by Terry Lambert 02 May '10

02 May '10

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com -- Terry On May 1, 2010, at 12:32 PM, Alexander von Below <below(a)mac.com> wrote: sysctl will probably give you that info. Alex Am 01.05.2010 um 21:16 schrieb Matthew Mashyna: Thanks, Matt _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/below%40mac.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/tlambert%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… But if you use this method instead of libproc, expect to break in the next release because data interfaces are definitionally fragile. Also libproc is not a supported interface at this time (see the header file). What reasonable reason do you have for this? You are aware that 32 bit will be deprecated at some point for performance reasons, right? Check the call for CTL_KERN, KERN_PROC, and the returned struct extern_proc While I do not know which flag to check exactly, this piece of code may help you: http://www.opensource.apple.com/source/xnu/xnu-792.12.6/bsd/kern/kern_sysct… I have an app that acts as a monitor for some other critical apps and one thing the spec calls for is the need to look at other running apps and figure out if they are running in 32 or 64 bit mode. The Activity monitor knows this. How can I find a process's bit-ness? This email sent to below(a)mac.com This email sent to tlambert(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0

Re: __ prefix in system call
by Kevin Van Vechten 01 May '10

01 May '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com The extra __ prefix has no special meaning, but is a convention indicating that applications should not make use of that symbol directly — it is an internal implementation detail of the OS. These syscall stubs are generated by the Libsyscall subproject of xnu. Kevin On May 1, 2010, at 1:32 PM, Ariel Feinerman wrote: > 2010/4/23 Jean-Daniel Dupas <devlists(a)shadowlab.org> > >> Le 22 avr. 2010 à 23:16, Ariel Feinerman a écrit : >> >>> I look at libc`s posix_spawn() function, one calls __posix_spawn() internally, I think __posix_spawn() is system call, but I learn xnu source by grep carefully, threre is only one function: posix_spawn(). What is __ ? >> >> nm libSystem.dylib >> 0000000000048020 t ___posix_spawn >> >> Answer: a libSystem internal symbol (aka implementation detail). What are you trying to do ? > > Nothing, I want to look at __posix_spawn() source when I learn libc implementation. > So does it means __posix_spawn() prototype in libc is posix_spawn() prototype in xnu? > For example if I want to call function from kernel or implement wrapper I need to insert __ ? _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: __ prefix in system call
by Ariel Feinerman 01 May '10

01 May '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type; bh=iJ341d6PGV5lsdu5pk+N7+QdTrfrZU1mAcil4edM/AM=; b=SqMY2w+WKSdogELEk7rLqrkyjaDHELceFXRIVnU4R8yLFdJht3XwQKOJqjHebzODx5 bowPZkAQkfMQNyWuffRXiiddF8xuVSy+bDEBqItRT62Etmw7k2WtzfifIkofrWM7GBEw rx9ibQg2Z+gVYYaYdSASD4GibU6omxSij3Xo4= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=EROe/g/BdCGG5E71eAHuw1wZuvhcfCnxz1HjNoq4kgXIe0KV9ZEXoiOlBajITvplW+ udzWDbT/LlCAqeFNdrE8B30Hz3Dq98lPDzWjLqrJHlVL96yWNdR0fxrdoJcRBL30dEop Lq82l1xOeHMlneEIuphMa5wTJFz914hFMhEfc= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0