April 2008 - Darwin-kernel - lists.apple.com

Re: Network stack/ethernet driver issues
by Michael Cashwell 10 Apr '08

10 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Apr 10, 2008, at 11:44 AM, Justin C. Walker wrote: On Apr 10, 2008, at 06:45 , Michael Cashwell wrote: Did I imply it does? I didn't intend to. Fragmentation is fun. :-) -Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… The point in rimas' case is that just because the data are UDP does not give the Ethernet driver more permission to drop them than if they were TCP [corrected] nor does it mean they are less important. You are correct that an ethernet driver does not have permission to drop one kind of packet over another, I don't think you are correct in your implication that it cares about network-layer packet types : these are all one to the driver (it's possible for the driver to obey QOS indicators of some type, but it can't know about the IP frame type: in particular, it may not know that a particular packet is in fact (part of) a UDP frame). After all, it's ethernet. Even if the packet gets to the media layer, there's no guarantee that (a) it will make it out the door; or (b) even if it gets out the door, it may not survive the trip across the local link :-}). I don't think TCP/UDP or QoS is involved in rimas' problem. That really does look like a driver issue. My point was only to underscore that there are different dimensions to the question of what delivery characteristics are most important. If you're transferring live video having data arrive late is no different from it not arrive at all. In fact, if "late" requires retries and resends (and this consumed even more bandwidth and resources) only to just be dropped anyway then late is actually worse than not at all. Assuming that someone's data is less important just because they used UDP is flawed. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Version of dynamic library
by mm w 10 Apr '08

10 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=Ahlc3Om4O8Fjq7Uof4SySQffXpsW13bVDfFL7ehd2D8=; b=HYtidRcPB33FNmP2fyq/uOz7M8b9LI0Fz3Bg5idtrC5T/9+Bsx9BfyU6CfMPUZCrUErpurXEoyjrQ1Avma8E3g9V8LvDof0eApz4jixvX2hJTV7GVJH78PNejcdodNqkBDzex0xJX6pcfhiCuSHSM7q+U+VgZwcbAYMjh8LJvfk= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=jyoY2zci3AOJmEMMOGP1NUmfBOdIuhJeyBn26feSM8cBO64vXraBbAAMsJVSeW/nYq42z0w4ARyTKtJv6L0XMxfz3UdzkfzVXlySFD1rYC6V3qxX4WQ3ee4xOcpQv5W1onKSmoMmGqWUaQ811LM6glMp0teaZbymG98Nd5jHxvE= Hi if you want a good example you can get it from lipo.c a part of odc tools (a long time ago maintened by Shantonu) I think your problem come from that you don't test if you have a fat binary or not and which arch is stored (if 1) if you wont be able to get the src (odc) on the Internet let me know Cheers! On Thu, Apr 10, 2008 at 3:36 AM, anukriti_shrimal <anukriti_shrimal(a)persistent.co.in> wrote: > > > > > Hi All, > > > > I am currently working on Mac Os x 10.4.7 > > I am trying to retrieve the version of a dynamic library (.dylib) from its > full path. > > > > While investigating source code of dyld, I found that the mach_header or > mach_header_64 contains the version information. > > So, I tried to get the mach_header from the file contents. > > But I was unable to do so. > > > > Can anyone help me in this regard, > > Is there some other way to accomplish the same task, > > > > Thanks in advance, > > Princess > > > > > > > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) > Help/Unsubscribe/Update your Subscription: > > http://lists.apple.com/mailman/options/darwin-kernel/openspecies%40gmail.com > > This email sent to openspecies(a)gmail.com > -- -mmw _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Network stack/ethernet driver issues
by Justin C. Walker 10 Apr '08

10 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Apr 9, 2008, at 8:58 PM, William Kucharski wrote: rimas(a)cnmat.berkeley.edu wrote: Justin -- Justin C. Walker, Curmudgeon-At-Large Institute for the Absorption of Federal Funds -------- Men are from Earth. Women are from Earth. Deal with it. -------- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Apr 10, 2008, at 06:45 , Michael Cashwell wrote: If frequently dropping packets with only 40% of the available bandwidth being utilized and no load on the machine other than an application listening for and peeking at the contents of said packets is considered OK then I will stick to Linux for my low latency, high bandwidth network applications (and encourage others to do likewise). UDP can drop as many packets as it wants to, that's why it's UDP. Incorrect. You are conflating non-retry with low Quality-of-Service. They are independent things. Very often UDP is used for data that are time-sensitive where retrying (even if successful) would result in the data arriving late enough as to be useless. Using UDP does not mean the data are less important than TCP data. It just means that retries are not useful. In fact, UDP data (consider live video conferencing) is often given a HIGHER Quality of Service level specifically because retrying them is not an option. If a router along the way is congested it might preferentially drop a low-QoS TCP packet (since it's not real-time and can be more easily recovered) than a high-QoS UDP one. There may be mechanisms within a "stack" (from app to driver/device) to facilitate QoS, but the fact remains that UDP is not "guaranteed". In fact your example shows that it is not worthwhile to have the UDP (or lower) engines keep beating on the system to send a specific packet. The QoS at a particular level is best handled at that level (perhaps in terms provided by lower levels, of course). The point in rimas' case is that just because the data are UDP does not give the Ethernet driver more permission to drop them than if they were UDP nor does it mean they are less important. You are correct that an ethernet driver does not have permission to drop one kind of packet over another, I don't think you are correct in your implication that it cares about network-layer packet types: these are all one to the driver (it's possible for the driver to obey QOS indicators of some type, but it can't know about the IP frame type: in particular, it may not know that a particular packet is in fact (part of) a UDP frame). After all, it's ethernet. Even if the packet gets to the media layer, there's no guarantee that (a) it will make it out the door; or (b) even if it gets out the door, it may not survive the trip across the local link :-}). This email sent to site_archiver(a)lists.apple.com

1 0

Re: Network stack/ethernet driver issues
by Bakul Shah 10 Apr '08

10 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Thu, 10 Apr 2008 09:45:14 EDT Michael Cashwell <mboards(a)prograde.net> wrote: > On Apr 9, 2008, at 8:58 PM, William Kucharski wrote: > > > rimas(a)cnmat.berkeley.edu wrote: > > > >> If frequently dropping packets with only 40% of the available > >> bandwidth being utilized and no load on the machine other than an > >> application listening for and peeking at the contents of said > >> packets is considered OK then I will stick to Linux for my low > >> latency, high bandwidth network applications (and encourage others > >> to do likewise). > > > > UDP can drop as many packets as it wants to, that's why it's UDP. > > Incorrect. You are conflating non-retry with low Quality-of-Service. > They are independent things. > > Very often UDP is used for data that are time-sensitive where retrying > (even if successful) would result in the data arriving late enough as > to be useless. Using UDP does not mean the data are less important > than TCP data. It just means that retries are not useful. > > In fact, UDP data (consider live video conferencing) is often given a > HIGHER Quality of Service level specifically because retrying them is > not an option. If a router along the way is congested it might > preferentially drop a low-QoS TCP packet (since it's not real-time and > can be more easily recovered) than a high-QoS UDP one. > > The point in rimas' case is that just because the data are UDP does > not give the Ethernet driver more permission to drop them than if they > were UDP nor does it mean they are less important. It is a quality of implementation issue (not service) but in general if you expect that *no* packet will be dropped, _expect_ to be disappointed! Even if a single ethernet switch is in the middle, a burst of traffic from another sender will force the *switch* to drop your packets. Similarly if the machine you are sending to runs out of resources, it will drop packets. Even simpler, a packet will be dropped if it has a bad ethernet CRC or bad IP checksum and this can happen for all sorts of reasons. As RFC 768 (User Datagram Protocol) says, Applications requiring ordered reliable delivery of streams of data should use the Transmission Control Protocol (TCP) Assuming the other person wants a working solution now, the easiest solution may be to stick a fast enough but hopefully inexpensive FreeBSD/linux box with two GbE ports in the middle to convert his UDP stream to a TCP stream and feed it to a box that can sustain a 50MBps writing speed to its disk -- which will be the next thing he will discover his MacBook Pro can't do :-) _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: kernel panic on readBytes
by Timur Alperovich 10 Apr '08

10 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=yxodzYbF4J5A3TzU4Kh8F+e5kUR5Z4xqvsI+qrQa5GM=; b=dZhvsKTIV1+1yuaN1diHpuqBFrn3Qz1D7Fn8Q/O3XpKrsj4TmugUgI7gLXsT5uEd/EOu2Q50EPb/iW8D3kDHdptCEoInAAeYLZwYD7s8uC/6UA42msh7Kgk1p7JeHzwWDu1lbST2yuk/W71ugvxOIU49f+CRGjSonO8uKBa2uUI= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=XU9Dxj/wpLGJIAao/SiifHW3UJiDuinSUQsCRsEhvS8fMih3UjlTh98mPX/UeKZKZYgxOOA2zhgKcf8C51wRkZ4ryEqZDmVSq6L/V9l9EKCc8iIwH6hDLdtKMjDHQtJxLQ8XQuf4qbVJhM0q6GgcSPacz6GHbfNPK5rOLmPz0GE= > Hey Timur, > > Where does the physical address in inStruct->offset come from? > getPhysicalSegment()? The address is passed in from userspace. I wrote a little daemon that simply goes from 0x0 to the highest possible address (I figure that out by getting the maximum amount of ram on the machine through a sysctl call) and pass it to the kernel extension. The kernel then returns a 32-bit hash value for each page. I thought that should work ok, but is it a problem? Timur > Dan > > > > > > The problem is that kMemoryTypePhysical != kIOMemoryTypePhysical64. The > > > former is a 32-bit I/O physical address. The later is a 64-bit CPU > physical > > > address. > > > > > > > > Right, makes sense. I tried doing what you described and ended up with: > > Unresolved kernel trap(cpu 1): 0x300 - Data access > > DAR=0x6572436D00001000 PC=0x00000000000B3E40 > > Latest crash info for cpu 1: > > Exception state (sv=0x26c76500) > > PC=0x000B3E40; MSR=0x00009030; DAR=0x00001000; DSISR=0x00200000; > > LR=0x000B26A8; R1=0x268F2A20; XCP=0x0000000C (0x300 - Data access) > > Backtrace: > > 0x000B2618 0x003622DC 0x26C95310 0x26C949AC 0x0037AA38 0x00377CB8 > > 0x00093E0C 0x0002C1B4 0x00024C88 0x000B2C4C 0x00000000 > > backtrace terminated - frame not mapped or invalid: 0xBFFFE260 > > > > I hate to be the guy who posts code and asks for help, but I'm pretty > > stuck at this point, as the documentation I could find on this was > > scarce at best. Anyway, here is what I tried running: > > > > IOOptionBits opt = kIOMemoryTypePhysical64 | kIODirectionOut; > > struct IOPhysicalRange* range = > > (IOPhysicalRange*)IOMalloc(sizeof(struct IOPhysicalRange)); > > range->address = inStruct->offset; > > range->length = MEM_READ_CHUNK; > > task_t task; > > IOLog("calling withoptions\n"); > > IOSleep(2000); > > IOMemoryDescriptor* d = IOMemoryDescriptor::withOptions(range, 1, 0, task, > opt); > > IOLog("withoptions ran\n"); > > IOSleep(5000); > > > > IOFree(range, sizeof(IOPhysicalRange)); > > if (d == NULL) > > { > > IOLog("IO-Init failed on the page %u\n", inStruct->offset); > > goto End; > > } > > > > ret = d->prepare(); > > if (ret != kIOReturnSuccess) > > { > > IOLog("IO-Prepare failed on the page %u\n", inStruct->offset); > > goto End; > > } > > char* buf = (char*) IOMalloc(MEM_READ_CHUNK); > > int copied; > > IOLog("printing: %d %d %d\n", (int)buf, MEM_READ_CHUNK, inStruct->offset); > > copied = d->readBytes(0, buf, MEM_READ_CHUNK); > > > > It prints out "calling with options" in the system log but not > > "withoptions ran". Did I screw up that call as well? > > > > Thank you, > > Timur _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: kernel panic on readBytes
by Dan Markarian 10 Apr '08

10 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Hey Timur, Dan On 9 Apr 2008, at 6:43 PM, Timur Alperovich wrote: I hate to be the guy who posts code and asks for help, but I'm pretty stuck at this point, as the documentation I could find on this was scarce at best. Anyway, here is what I tried running: IOFree(range, sizeof(IOPhysicalRange)); if (d == NULL) { IOLog("IO-Init failed on the page %u\n", inStruct->offset); goto End; } It prints out "calling with options" in the system log but not "withoptions ran". Did I screw up that call as well? Thank you, Timur _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/markarian%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… Where does the physical address in inStruct->offset come from? getPhysicalSegment()? The problem is that kMemoryTypePhysical != kIOMemoryTypePhysical64. The former is a 32-bit I/O physical address. The later is a 64-bit CPU physical address. Right, makes sense. I tried doing what you described and ended up with: Unresolved kernel trap(cpu 1): 0x300 - Data access DAR=0x6572436D00001000 PC=0x00000000000B3E40 Latest crash info for cpu 1: Exception state (sv=0x26c76500) PC=0x000B3E40; MSR=0x00009030; DAR=0x00001000; DSISR=0x00200000; LR=0x000B26A8; R1=0x268F2A20; XCP=0x0000000C (0x300 - Data access) Backtrace: 0x000B2618 0x003622DC 0x26C95310 0x26C949AC 0x0037AA38 0x00377CB8 0x00093E0C 0x0002C1B4 0x00024C88 0x000B2C4C 0x00000000 backtrace terminated - frame not mapped or invalid: 0xBFFFE260 IOOptionBits opt = kIOMemoryTypePhysical64 | kIODirectionOut; struct IOPhysicalRange* range = (IOPhysicalRange*)IOMalloc(sizeof(struct IOPhysicalRange)); range->address = inStruct->offset; range->length = MEM_READ_CHUNK; task_t task; IOLog("calling withoptions\n"); IOSleep(2000); IOMemoryDescriptor* d = IOMemoryDescriptor::withOptions(range, 1, 0, task, opt); IOLog("withoptions ran\n"); IOSleep(5000); ret = d->prepare(); if (ret != kIOReturnSuccess) { IOLog("IO-Prepare failed on the page %u\n", inStruct->offset); goto End; } char* buf = (char*) IOMalloc(MEM_READ_CHUNK); int copied; IOLog("printing: %d %d %d\n", (int)buf, MEM_READ_CHUNK, inStruct- >offset); copied = d->readBytes(0, buf, MEM_READ_CHUNK); This email sent to markarian(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0

Re: Network stack/ethernet driver issues
by Michael Cashwell 10 Apr '08

10 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Apr 9, 2008, at 8:58 PM, William Kucharski wrote: rimas(a)cnmat.berkeley.edu wrote: -Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… If frequently dropping packets with only 40% of the available bandwidth being utilized and no load on the machine other than an application listening for and peeking at the contents of said packets is considered OK then I will stick to Linux for my low latency, high bandwidth network applications (and encourage others to do likewise). UDP can drop as many packets as it wants to, that's why it's UDP. Incorrect. You are conflating non-retry with low Quality-of-Service. They are independent things. Very often UDP is used for data that are time-sensitive where retrying (even if successful) would result in the data arriving late enough as to be useless. Using UDP does not mean the data are less important than TCP data. It just means that retries are not useful. In fact, UDP data (consider live video conferencing) is often given a HIGHER Quality of Service level specifically because retrying them is not an option. If a router along the way is congested it might preferentially drop a low-QoS TCP packet (since it's not real-time and can be more easily recovered) than a high-QoS UDP one. The point in rimas' case is that just because the data are UDP does not give the Ethernet driver more permission to drop them than if they were UDP nor does it mean they are less important. This email sent to site_archiver(a)lists.apple.com

1 0

Pros/Cons of using IOKit for Kernel/User space communication
by Nick Brandaleone 10 Apr '08

10 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Hello, Thank you, Nick Brandaleone _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This is an open-ended question, instead of code specific. However, let me give a little background. I am working on a primitive anti-virus security (non-BSD) kernel extension. I am therefore touching "kauth" quite a bit. I also want to pass information about files in my kauth scope over to a user-space daemon for logging and processing (ie, kicking off other programs to scan for viruses, etc...). After investigating the numerous methods to open a communication channel between kernel/user boundary (mach messages/RPC, IOKit, BSD calls, SYS V calls, memory mapping, notify, etc...), it seems that the most flexible (and well documented) is using IOKit (IOUserClient, IODataQueue) which maps memory between the two processes, sets up a mach-based notification port, and then allows data to be queued up for transport. I may need other methods if I need to do two way information passing, but for now this seems sufficient. Now for my question. As I did not originally see that I would be putting my code into the IOKit section of the kernel (ie device interface), is there any downside to doing this, besides putting my code at the whim of the IOKit driver life cycle? Also, is there a method I am overlooking for flexible, arbitrary communication across the K/U boundary? It seems that Mach RPC is probably the next most flexible manner, although it is obviously more a standard functional paradigm, instead of one for flexible data passing/communication. I know that there is probably no one right answer to this question, but I am curious what other, more experienced developers, might have to say. This email sent to site_archiver(a)lists.apple.com

1 0

Version of dynamic library
by anukriti_shrimal 10 Apr '08

10 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Thread-index: Acia9rAA9zXjGp5XR/+W93cUfffdog== _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Version of a dynamic library
by anukriti_shrimal 10 Apr '08

10 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Thread-index: Acia9omTnThMDbL4Tv6J4l2Ijr9Lxg== _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0