July 2007 - Darwin-dev - lists.apple.com

Re: malloc causing vm_allocate errors
by Omar Awile 04 Jul '07

04 Jul '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=Vc2WT52M1g8HLu7ypWPmbW9Eb/mKHtAYc3aJEzmdcFCZ5slsZyaWMaruhlvyX9obiejYiMuP1iWetfYI8dm2RehseqsWW1F2/DgCdXW6WAyCj5zDx7xaTL8oPLWYll4V4o45xmfBSpL+mmuk6dInecGxs8o4DzxOyb5Tur/IZm4= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=UnoxJB0DQtL212e+6IxDMd7EbO0ozgNWFMlRLQ8wWXUD4wPm+oYoLwwWj6KoTHCurBewh6I6HDOHJyAl8Nbi//WZU2u+3CNBBSx1Ar8NklK8Tj4eCGIuh0qKjSKGpWO3oxVOmPBMliESlyQUEYIJ48yI+H5eXeVFgeGmPIIvRHk= Jonas, thanks for the quick reply On 04 Jul 2007, at 17:23, Omar Awile wrote: > I have a program which uses depending on the input between 1 and 2 gb > (I have only ssh access to the computer so I cannot > access any gui tools and don't know how to get system information in > console). sw_vers system_profiler Ah, now I know :-) In general: http://bhami.com/rosetta.html > With the dataset in question this code will allocate about 1.1gb (in > linux, i386 & amd64) without complaining a bit. > > Running the same code in darwin results in a ton of identical > messages: > renyi(8200) malloc: *** error: can't allocate region > renyi(8200) malloc: *** set a breakpoint in szone_error to debug > renyi(8200) malloc: *** vm_allocate(size=8421376) failed (error > code=3) > > I don't understand why this error happens and what I have to do to get > rid of that error. It presumably happens because you've exhausted the VM space. This is pretty strange, since there should be at least about 2.5GB available (on PPC anyway, but I assome on i386 as well). You can try running vmmap on your process, and enable malloc debugging (man malloc, search for "debug"). Here a few more system infos: sw_vers: ProductName: Mac OS X ProductVersion: 10.4.10 BuildVersion: 8R2232 system_profiler: Hardware Overview: Model Name: Mac Pro Model Identifier: MacPro1,1 Processor Name: Dual-Core Intel Xeon Processor Speed: 3 GHz Number Of Processors: 2 Total Number Of Cores: 4 L2 Cache (per processor): 4 MB Memory: 8 GB Bus Speed: 1.33 GHz Boot ROM Version: MP11.005C.B04 SMC Version: 1.7f8 ... 'top' is telling me about VM: VM: 3.90G + 7.19M 141826(0) pageins, 0(0) pageouts (my program is not running) So this means there should be by far enough memory around. I was not yet able to check with vmmap (or malloc_history) because my program spits out these error messages after a very short time and crashes right after. Perhaps I need to emphasize that the program runs properly in darwin for some datasets, for others I get the above discribed error. cheers, omar Jonas _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/omar.awile%40gmail.com This email sent to omar.awile(a)gmail.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On 7/4/07, Jonas Maebe <jonas.maebe(a)elis.ugent.be> wrote: This email sent to site_archiver(a)lists.apple.com

1 0

Re: malloc causing vm_allocate errors
by Jonas Maebe 04 Jul '07

04 Jul '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On 04 Jul 2007, at 17:23, Omar Awile wrote: I have a program which uses depending on the input between 1 and 2 gb (I have only ssh access to the computer so I cannot access any gui tools and don't know how to get system information in console). sw_vers system_profiler In general: http://bhami.com/rosetta.html With the dataset in question this code will allocate about 1.1gb (in linux, i386 & amd64) without complaining a bit. I don't understand why this error happens and what I have to do to get rid of that error. Jonas _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Running the same code in darwin results in a ton of identical messages: renyi(8200) malloc: *** error: can't allocate region renyi(8200) malloc: *** set a breakpoint in szone_error to debug renyi(8200) malloc: *** vm_allocate(size=8421376) failed (error code=3) It presumably happens because you've exhausted the VM space. This is pretty strange, since there should be at least about 2.5GB available (on PPC anyway, but I assome on i386 as well). You can try running vmmap on your process, and enable malloc debugging (man malloc, search for "debug"). This email sent to site_archiver(a)lists.apple.com

1 0

malloc causing vm_allocate errors
by Omar Awile 04 Jul '07

04 Jul '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=OBMbhVO5QRuBPt5yR6EsCqBxcdp/lHCI6ZJLcpEPXhyQW5a8XTTShA/aHz5naO9mKxVgZRIGEJ/OvKdp/WsJkaAzb7xGkyK4QzrfbbTDmYNFaHYbwDAhJ0NlqjCAXu0tHXCg3xQ/SiMEOzpVnPGvFsy+a6bwk31liqPdsn4aMyQ= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=VtSIGVw+TBiOL9u05zVqdR2oZ0UU057KozUnR4xM1g8xJbjnQ+niEpSWVXvf094M/38+U2I/6E+s/lxqV/4Qlj6NfjgWcIK4HVRZrAZr0uMlfQLRUFhUKzcpvnoZzX5GieetKLarzwhxHxBHAnqnIv/fxIYJ8dfg6mohIhUrxQo= Dear Darwin users/developers I am very new to Darwin so please excuse my question if the answer is obvious... I first try to explain the situation and what I am trying to do, then I'll explain what exactly the problem is: I have a program which uses depending on the input between 1 and 2 gb of memory. The program is tested on linux for the same dataset and works without any problem there. The target system where the problem occures is a Mac pro with 2 core duo 2 processors (i think) and 4 gb of physical main memory. The OS is a recent mac os x (I have only ssh access to the computer so I cannot access any gui tools and don't know how to get system information in console). I have also checked ulimit where I have not found any memory constraints set. My program contains somewhere following code grid = (double***)malloc(Nx*sizeof(double**)); for (i = 0; i < Nx; i++) { grid[i] = (double**)malloc(Ny*sizeof(double*)); for (j = 0; j < Ny; j++) grid[i][j] = (double*)malloc(Nz*sizeof(double)); } With the dataset in question this code will allocate about 1.1gb (in linux, i386 & amd64) without complaining a bit. Running the same code in darwin results in a ton of identical messages: renyi(8200) malloc: *** error: can't allocate region renyi(8200) malloc: *** set a breakpoint in szone_error to debug renyi(8200) malloc: *** vm_allocate(size=8421376) failed (error code=3) I don't understand why this error happens and what I have to do to get rid of that error. I hope someone can give me a solution, or at least a few hints. cheers, Omar _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Code Obfuscation
by Dave Keck 03 Jul '07

03 Jul '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:cc:message-id:from:to:in-reply-to:content-type:content-transfer-encoding:mime-version:subject:date:references:x-mailer; b=kfViHXT0KQuHz7t4yOBLwoCaR0DTPOQA6Oy1NSYocoAqon16QReolOmi0FxwGYRkcGDZNkVdi4/lguE4QmK9hBoiqz9E5bSechz4/yb+QPzE4j2gmYcwbkZMuQlEy5N9d3fygwVNjRBHsWwMGpMSaIlYabyjkk4ISZCcBKQZsbM= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:cc:message-id:from:to:in-reply-to:content-type:content-transfer-encoding:mime-version:subject:date:references:x-mailer; b=ougnvU4OVAlYO33dS1TcikGyMh3m5RdLiF0QXqL113bo97hF1g8Ov6pDWa8axvdnrJE59lAtPNi4NG+9YVUAmBxqCeKl5BN8mVZbnkulGrUwMq7nvbcfBJf9Rml8reIHomD2wxMNYBevXUmyrOn8Zbm6hgST2qHpg4mhW01bEjw= Hey, A very nice article on the subject can be found here: http://www.osxbook.com/book/bonus/chapter7/binaryprotection/ Perhaps you could implement something like this? Dave On Jul 3, 2007, at 2:26 PM, Thomas Inskip wrote: _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I'm fairly sure this patent is referring to Apple's method of encrypting certain binaries (Finder, Dock, etc.) which can "only" (I think it's been cracked now?) be decrypted with the "Dont Steal Mac OS X.kext" kernel extension (check out your /System/Library/Extensions folder). This may be slightly off topic, but does anyone know of any object code obfuscators which will work with mach-o binaries (PPC and x86)? I've looked around but haven't had much luck. The only product even remotely related I've been able to find is a C/C++ source code obfuscator from stunnix (http://stunnix.com/prod/cxxo/overview.shtml ), which I assume won't much good in terms obfuscating the generated object code. Apparently Apple applied for a patent a couple years ago on some object code obfuscation technology (http://www.macsimumnews.com/index.php/archive/apple_files_patent_system_and… ). But if they've done anything with it, they haven't made it public. Other than the somewhat irrelevant product from stunnix above, I haven't been able to find any other third party commercial or open source technologies. This email sent to site_archiver(a)lists.apple.com

1 0

How can H.263 ( instead of H.263+ ) be streamed on RTP from Darwin
by Vipin Palawat (vpalawat) 03 Jul '07

03 Jul '07

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com Dkim-signature: v=0.5; a=rsa-sha256; q=dns/txt; l=3472; t=1183488410; x=1184352410; c=relaxed/simple; s=sjdkim1004; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=cisco.com; i=vpalawat(a)cisco.com; z=From:=20=22Vipin=20Palawat=20\(vpalawat\)=22=20<vpalawat(a)cisco.com> |Subject:=20How=20can=20H.263=20(=20instead=20of=20H.263+=20)=20be=20stre amed=20on=20RTP=20from=20Darwin |Sender:=20; bh=ZTN8ROERZbWhnMlcP5YwuhBqzAuzBZ/0L6WDaTMAaLs=; b=gfgwq4OzU80DwQ9XCFdjKTEwmVYOzS+9vc8Lb0xV7rvk9NSq0gM1Q+Y0jmgzMRlheF6l64RO FZcmBHTQFYKCZMvRZdRkaN7y9L2iZpJ1JYGARZlzQgAyzW7KfkHm8z3DxHZy72Jj2VDjJylMso VXOjb447FCI7fvKJqmLL/iWs4=; Thread-index: Ace9ooCQ6V3P7BZlRPymSa02Mukkxg== Thread-topic: How can H.263 ( instead of H.263+ ) be streamed on RTP from Darwin _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Code Obfuscation
by Thomas Inskip 03 Jul '07

03 Jul '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:mime-version:content-transfer-encoding:message-id:content-type:to:from:subject:date:x-mailer; b=udlOqQhdv/Luvk/pD893jdehHMMIKI5gU4ekU9K8Z93Hg6EqKj/MceV8kZoZ1cBKPRn1ctwYOXQQN9xEQ0HVYe9HIpaJkINUeQ//ieXAxGbe48CE7DxlEadx9NUWlsLf7cfcpg4kfuzCmuNgMA+hzCPIB88Ko4Os3VS45Yitcs8= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:mime-version:content-transfer-encoding:message-id:content-type:to:from:subject:date:x-mailer; b=Vyy2SO9loA992e6t65Z/tf4hiNIgGg8VBhkNDrCrh67M+f+jJUBgU8+6KxXHqyD0RJGTrpo1weERCs6Gz9gBJt2jpdXmVfkUFc6qfKGYjpmhs0XNal5l8PDAURTU5EID6aTB2lb0f2zihyth54i2qpSfwC4nrPvuVGVhett4emc= This may be slightly off topic, but does anyone know of any object code obfuscators which will work with mach-o binaries (PPC and x86)? I've looked around but haven't had much luck. The only product even remotely related I've been able to find is a C/C++ source code obfuscator from stunnix (http://stunnix.com/prod/cxxo/ overview.shtml), which I assume won't much good in terms obfuscating the generated object code. Apparently Apple applied for a patent a couple years ago on some object code obfuscation technology (http://www.macsimumnews.com/ index.php/archive/ apple_files_patent_system_and_method_for_creating_tamper_resistant_code/ ). But if they've done anything with it, they haven't made it public. Other than the somewhat irrelevant product from stunnix above, I haven't been able to find any other third party commercial or open source technologies. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Problem with ifnet_output_raw on ethernet card
by Matteo Merli 03 Jul '07

03 Jul '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Hi, ifnet_output_raw( msg->interface, PF_INET, pkt_buf ); ============================================ Thank you, Matteo Merli _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I'm having problems when re-injecting packet from my interface filter. The strange fact is the re-injection is working _fine_ through Airport network card but it fails when using ethernet card (en0). The mbuf I'm re-injecting is the mbuf I receive from a control socket callback and the code is this: ============================================ mbuf_settype( pkt_buf, MBUF_TYPE_HEADER ); mbuf_adj( pkt_buf, sizeof(SocketMessage) ); // To delete my custom header struct size_t hdr_len = ifnet_hdrlen( msg->interface ); mbuf_pkthdr_setheader( pkt_buf, msg->packet ); mbuf_pkthdr_setrcvif( pkt_buf, msg->interface ); setTag( &pkt_buf, Outbound ); mbuf_outbound_finalize( pkt_buf, PF_INET, hdr_len ); mbuf_inbound_modified( pkt_buf ); mbuf_clear_csum_requested( pkt_buf ); I guess the problems rely on some mbuf flag or checksum calculation issue I'm using badly... What are the steps needed to inject a "self-builded" mbuf packet? This email sent to site_archiver(a)lists.apple.com

1 0

Re: Fork, exec, and child inheritance oh my
by Brian Mastenbrook 01 Jul '07

01 Jul '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com User-agent: Thunderbird 2.0.0.4 (Windows/20070604) Hello, I'm attempting to create a program in which a child process will load certain components into the parent's address space, so that the parent is able to access the symbols that the child loaded. Like a plugin-loader, but with one caveat: due to the architecture of our program, the child must be a separate process. Right now I've been walking the dark alleys of fork(), execv(), and mmap(), but I'm not seeing much light. It seems what we need is a version of execv() that wouldn't completely overwrite the parent process, so that the parent and child would share the same address space and both able to access the symbols loaded by the child process. -- Brian Mastenbrook brian(a)mastenbrook.net http://brian.mastenbrook.net/ _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Dave Keck wrote: Is this possible? I'm not sure I understand how it's possible to have a requirement for a separate process and yet also have a requirement to share the virtual memory map of the process like threads do. What is the reason for using separate processes as opposed to threads? You can share specific segments of memory using System V shared memory. See the man page for shmget(2). This email sent to site_archiver(a)lists.apple.com

1 0

Re: Fork, exec, and child inheritance oh my
by Paul Nelson 01 Jul '07

01 Jul '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Thread-index: Ace8Lx0JW8MdmCgiEdyG7QAWy4i6sA== Thread-topic: Fork, exec, and child inheritance oh my User-agent: Microsoft-Entourage/11.3.3.061214 on 7/1/07 5:07 PM, Dave Keck at davekeck(a)gmail.com wrote: > Right now I've been walking the dark alleys of fork(), execv(), and > mmap(), but I'm not seeing much light. It seems what we need is a > version of execv() that wouldn't completely overwrite the parent > process, so that the parent and child would share the same address > space and both able to access the symbols loaded by the child process. The problem is that parent and child processes never "share" the same address space. When the child process changes memory, a copy-on-write page fault occurs, giving the child it's own copy of the changed memory. The parent process memory is unchanged. Perhaps you could explain why you don't want the parent process to simply load the code itself? Paul Nelson Thursby Software Systems, Inc. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Fork, exec, and child inheritance oh my
by Dave Keck 01 Jul '07

01 Jul '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=OzV9UZu6YPi7QFj/a1bQjHr9nvKQDvCS71+5aBV6mjejIMX5oqQb+Dl1R7u2Nl39vLNzUvsUqRuL6XcJwTgCki5ci4P7H7cTdf5MUZ7uxmr96jw9cqz8YfKCqyzhkmyi2ee0sRC4YdM4pP7zsEhp/TZ6v4gIhBcQ73JO2dzShu4= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=F76r9IFj7tOJoCpOujmuMtYvw80EuLWUnY2vXWpDOktMXBsu2wO6rMAN7QFviXAb3kyBnnS9Mc6sQ0Wluo9ui6AdBCAdsK+umA/p2ZzdJJ7RHBnn/tAMala59nxxjJGlQazcfd/IGOslkUeVVuv2uYoXGY1jcB7Xmp+xf/rNiYY= Hello, I'm attempting to create a program in which a child process will load certain components into the parent's address space, so that the parent is able to access the symbols that the child loaded. Like a plugin-loader, but with one caveat: due to the architecture of our program, the child must be a separate process. Right now I've been walking the dark alleys of fork(), execv(), and mmap(), but I'm not seeing much light. It seems what we need is a version of execv() that wouldn't completely overwrite the parent process, so that the parent and child would share the same address space and both able to access the symbols loaded by the child process. Is this possible? Thanks! Dave _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0