June 2006 - Darwin-kernel - lists.apple.com

Re: udp port sharing across processes on OS X?
by Josh Graessley 01 Jun '06

01 Jun '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com -josh On Jun 1, 2006, at 11:38 AM, Josh Graessley wrote: -josh On Jun 1, 2006, at 11:09 AM, Terry Lambert wrote: _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com I missed one other point. If the UDP socket is connected, the stack will use both the source and destination addresses and port to locate the socket. Actually, each socket has it's own receive buffer. The stack will only deliver unicast packets to the first socket that matches based on destination address and port. The definition of the "first socket that matches" is a little vague. On Mac OS X, this is the first socket bound to the port. This is very implementation specific. For multicast packets, the stack goes out of its way to deliver a copy of the packet to every socket. I see from your email address that you work for Asperasoft, makers of FASP, which uses UDP instead of TCP for large file transfers over high latency/packet loss links to avoid the bandwidth delay product issue, having to support huge TCP windows, or having to implement rate based ACK or TCP rate halving (e.g. CMU's experimental replacement for Nagle et. al.) in the native networking stack of the OS you are running on. With that in mind, I have to question the validity of doing this as multiple users. I can understand wanting to do this on "this is the machine" basis, but on a per-user basis, the only valid reason for this would be transmission; there's no guarantee that any given packet would end up going to the right user process on receive, since only the address of the socket differentiates inbound packets to the UDP stack: there'd only be one socket receive buffer associated with it, and whoever read first would get the data, which is probably not what you want, if you're trying to use different user identities to do privilege separation to avoid disclosure of one user's packet contents to another user. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/jgraessley% 40apple.com This email sent to jgraessley(a)apple.com smime.p7s

1 0

Re: udp port sharing across processes on OS X?
by Josh Graessley 01 Jun '06

01 Jun '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com -josh On Jun 1, 2006, at 11:09 AM, Terry Lambert wrote: _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com Actually, each socket has it's own receive buffer. The stack will only deliver unicast packets to the first socket that matches based on destination address and port. The definition of the "first socket that matches" is a little vague. On Mac OS X, this is the first socket bound to the port. This is very implementation specific. For multicast packets, the stack goes out of its way to deliver a copy of the packet to every socket. I see from your email address that you work for Asperasoft, makers of FASP, which uses UDP instead of TCP for large file transfers over high latency/packet loss links to avoid the bandwidth delay product issue, having to support huge TCP windows, or having to implement rate based ACK or TCP rate halving (e.g. CMU's experimental replacement for Nagle et. al.) in the native networking stack of the OS you are running on. With that in mind, I have to question the validity of doing this as multiple users. I can understand wanting to do this on "this is the machine" basis, but on a per-user basis, the only valid reason for this would be transmission; there's no guarantee that any given packet would end up going to the right user process on receive, since only the address of the socket differentiates inbound packets to the UDP stack: there'd only be one socket receive buffer associated with it, and whoever read first would get the data, which is probably not what you want, if you're trying to use different user identities to do privilege separation to avoid disclosure of one user's packet contents to another user. smime.p7s

1 0

Re: udp port sharing across processes on OS X?
by Terry Lambert 01 Jun '06

01 Jun '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com By setting the socket options SO_REUSEADDR and SO_REUSEPORT, I've been able to successfully get two instances of the same process running as the same user to bind and share the same UDP port on OS X. However, if the two instances of the process are running as different user ids, the bind attempt by the second instance of the process fails with "address already in use". Does anyone know if there is way to get two instances of the same process running as different user ids to reuse the same UDP port on Mac OS X? - - - -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On May 31, 2006, at 9:44 PM, Michelle Munson wrote: I see from your email address that you work for Asperasoft, makers of FASP, which uses UDP instead of TCP for large file transfers over high latency/packet loss links to avoid the bandwidth delay product issue, having to support huge TCP windows, or having to implement rate based ACK or TCP rate halving (e.g. CMU's experimental replacement for Nagle et. al.) in the native networking stack of the OS you are running on. With that in mind, I have to question the validity of doing this as multiple users. I can understand wanting to do this on "this is the machine" basis, but on a per-user basis, the only valid reason for this would be transmission; there's no guarantee that any given packet would end up going to the right user process on receive, since only the address of the socket differentiates inbound packets to the UDP stack: there'd only be one socket receive buffer associated with it, and whoever read first would get the data, which is probably not what you want, if you're trying to use different user identities to do privilege separation to avoid disclosure of one user's packet contents to another user. If this is specifically for transmission only, then the way to do it is to open the descriptor from a privileged process, and have your other processes request a copy of the descriptor, and pass this to the requesting process from the privileged process over a UNIX domain socket using standard descriptor passing techniques. This would allow multiple users to have access to the socket, even though their UIDs differed. I would strongly encourage you to use a single multithreaded program rather than multiple programs t keep the transmission data rate up (by interleaving I/O, either by multithreading or by using aio), rather than sharing the socket between programs. If the intent is receipt, and you are guaranteed that inbound packets will have in-band stream identifying data, then what it sounds like is that you are actually attempting to implement the moral equivalent of a streams MUX on MacOS X. If this is the case, then there are a couple of approaches: (1) Instead of trying to share the port, have a single user space process act as the MUX, and push the data to other user space processes through an IPC mechanism. If necessary, you could make that mechanism something less efficient, like individual UNIX domain sockets, or a loopback socket, to make it look more like a socket to a MUX getting the right data from the MUX. Otherwise, my recommendation would be to use shared memory to buffer the data in user space, and deal with it that way (to avoid copying the data back into the kernel and then back into user space, just to make it look like/be socket data). (2) It's somewhat possible to implement a real MUX using a KEXT and the socket filter API; this would let you intercept inbound packets and redispatch them to the appropriate user space process. Writing this beast would be rather complicated, but it would be doable. An alternate implementation might be to take the "tun" device driver from one of the BSDs, and modify it; there are several publically available ports forr the device to MacOS X, which are usually associated with OpenSource VPN software for MacOS X. In any case, if this doesn't answer your questions, you may want to tell us more about what you are actually trying to accomplish, and the people on this list will probably be able to come up with a better way of solving the actual problem than trying to share UDP sockets between users. This email sent to site_archiver(a)lists.apple.com

1 0

Re: cross-bounday memory communication b/w user app and the Kernel.How?
by Michael Smith 01 Jun '06

01 Jun '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Jun 1, 2006, at 9:51 AM, rohit dhamija wrote: 1) Shared structure Below is the common structure to be shared by user and kernel application for communication across user-kernel boundary: ////// typedef struct _CS { int cmd; // my command number void* cmd_payload; // command payload void* response; // response } COMMAND_STRUCTURE, *PCOMMAND_STRUCTURE; ////// Note: The maximum size of command payload was of 4272 bytes , so i want to send the command with the required bytes (and not the whole structure, as void* com_payload) 2) User-Application The user application sends the structure to the kernel space, and recieve the output in the same structure. So it will be a "Scalar Input and Scalar output" ///////// kernReturn = IOConnectMethodStructureIStructureO(*dataPort, kGnsGetVersion, //cmd inBuffSize, //insize &outBuffSize, //outSize &structIn, &structOut); ///////// 3) Kernel Level. At KEXT level, Now, the main point of discussion is ,"how does the kernel allocate/map the user-level structure in kernel and return back the data from kernel memory back to user space"? After studying the apple docs i have come to following conculsion and wrote the steps for the same. Following steps needs your review/comments ( i am not sure about these steps, so please correct me incase i am wrong) For Sending the data into kernel space from the user-application. step a) Use IOMemoryDescriptor::withAddress to create and initializes an IOMemoryDescriptor /////////// IOMemoryDescriptor *descriptor = NULL; You need to create the memory descriptor using the client's task, not the kernel task. b) Now copy data from the memory descriptor's buffer to the buffer allocated in the kernel space. Why do you do this? Your I/O method uses an IOMemoryDescriptor, and you have just created these using the user buffers. Copying is inefficient and wastes memory. /////////////// ret= descriptor ->prepare( kIODirectionOut ); IOByteCount bytecont = descriptor->readBytes( 0, //offset myBuff, Ientocopy); descriptor->complete(); //////////////// d) now we have got the data say,in outBuff, and this needs to be sent back to user space. Now at this point, do we need to map output data recieved from the device data in-order to send to user space ? If yes, then how ? Please see above. I also wanted to mention one more thing: In case of my fixed structure , mentioned below ////////// typedef struct packet{ unsigned long pid; unsigned long cmdId; } packet, *ppacket; //////// I have not mapped the data into the kernel space, still it works fine i.e the data is successfully sent to communication module and the output is successfully sent back to user without any issues. Mapping is only required if you need to dereference the contents of the descriptor. In your case I expect that you do not, so there is no need to map anything. PCOMMAND_STRUCTURE pComm; outDescriptor = IOMemoryDescriptor::withAddress( (vm_address_t)pComm->command_payload, sizeof(command_structure), kIODirectionOut, fClientTask); inDescriptor = IOMemoryDescriptor::withAddress( (vm_address_t)pComm->response, sizeof(response_structure), kIODirectionIn, fClientTask); outDescriptor->prepare(); inDescriptor->prepare(); result = do_output(outDescriptor); if (result == kIOReturnSuccess) result = do_input(inDescriptor); outDescriptor->complete(); outDescriptor->release(); inDescriptor->complete(); inDescriptor->release(); return(result); = Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… That would be "Structure Input, Scalar output" since your response buffer is already in user space. You don't change the structure in the kernel so there is no need to copy it back out. descriptor = IOMemoryDescriptor::= withAddress( address_of_CommandStructure, sizeof(command_structure), kIODirectionIn, // I am not sure about this , why do we need this ? kernel_task // since we are passing data into kernel ); ////////////// The direction is the default for I/O setup operations (prepare and complete) against this descriptor. You should create two; one with kIODirectionOut for your outbound buffer, and one with kIODirectionIn for the response buffer. c) Now use this myBuff to communicate with the device.(The module works fine) Again, a waste. Create the IOMemoryDescriptors directly against the user-space addresses and do your I/O with those. Is the method correct ? Or there is some alternate methods available ? Is the mapping is required when the data allocation is dynamic ? This email sent to site_archiver(a)lists.apple.com

1 0

Re: udp port sharing across processes on OS X?
by Josh Graessley 01 Jun '06

01 Jun '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com -josh On Jun 1, 2006, at 12:42 AM, Michael Smith wrote: On May 31, 2006, at 9:44 PM, Michelle Munson wrote: By setting the socket options SO_REUSEADDR and SO_REUSEPORT, I've been able to successfully get two instances of the same process running as the same user to bind and share the same UDP port on OS X. However, if the two instances of the process are running as different user ids, the bind attempt by the second instance of the process fails with "address already in use". Does anyone know if there is way to get two instances of the same process running as different user ids to reuse the same UDP port on Mac OS X? = Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com This is not odd at all. It is very common for multiple clients to listen for multicast traffic. What's odd are the weird socket options, SO_REUSEADDR and SO_REUSEPORT that cause endless confusion and bugs. This is a very odd thing to want to have happen. What behaviour would you expect from such a shared port with regards to inbound traffic? _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/jgraessley% 40apple.com This email sent to jgraessley(a)apple.com smime.p7s

1 0

Re: udp port sharing across processes on OS X?
by Josh Graessley 01 Jun '06

01 Jun '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Stop setting both. -josh On May 31, 2006, at 9:44 PM, Michelle Munson wrote: By setting the socket options SO_REUSEADDR and SO_REUSEPORT, I've been able to successfully get two instances of the same process running as the same user to bind and share the same UDP port on OS X. However, if the two instances of the process are running as different user ids, the bind attempt by the second instance of the process fails with "address already in use". Does anyone know if there is way to get two instances of the same process running as different user ids to reuse the same UDP port on Mac OS X? Thanks, Michelle _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com Set only SO_REUSEPORT. SO_REUSEPORT is for UDP. SO_REUSEADDR is for TCP listening sockets. SO_REUSEADDR makes it possible to kill a server process and start it again without having to wait for a timeout on the old TCP sockets. __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/jgraessley% 40apple.com This email sent to jgraessley(a)apple.com smime.p7s

1 0

Re: File System Engineer Needed
by Shawn Erickson 01 Jun '06

01 Jun '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=k9cSdsCk0j31EUcQuo2VcdxXDc90CrGOBCbwo6jmaYHgy6DUVKKbe59gWqK8UMF2Co0Sa9XMxDe0Rrs1CWvA7nf560gzpTr632KRJ8Xh5+DmiyhZUEXmax19tt5k2QNagkdj702aOqpuJrk1N6kHeuAC1Li9+BE9nDzpJfHvuTc= I don't think it says- "Do not post admin requests to the list unless you keep it brief." "admin requests" = "please unsubscribe me", "please switch me to digest deliver", etc. -Shawn _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On 6/1/06, mike scammon <mscammon(a)gmail.com> wrote: This email sent to site_archiver(a)lists.apple.com

1 0

Re: File System Engineer Needed
by mike scammon 01 Jun '06

01 Jun '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; b=gbHMu4o2JdQkqRHmPnBVyd7gV58dQtUUL7boNNL7NIwRwcMBn975ckZe2lg964UpwtKQxOfbTltLpKmSaM7DzjGj+GGQ31oBehldrUgq1LKy/iUEKkKsGgA8ldwYQLUYGZaBPRRwOX57bKRXLvPOmIYRU38/W1oQwHwMzbI6e7o= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

cross-bounday memory communication b/w user app and the Kernel.How?
by rohit dhamija 01 Jun '06

01 Jun '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=jzwR3BOw0XVV0izzFkQh9p0VhXpBPPivIbssyNAf2MrrwK8JhNAc4lDpRqnPgPI85tV4wzSPHWLA9XPwlOoKeAPVvPJJtjO/vdCt7VR+4PyfL23vEew0Rd8gYKzeF0pZvqhonurKWDFgZZUhhwvFqZ9bMXJpJi30SPU3/foEVl0= Hi, Currently developing a KEXT for USB device on Mac OS 10.4 Intel and powerpc and require your expert inputs on the following: Query: "How is the user memory mapped with the kernel memory during communication process"? I have a dynamic structure for communication b/w user land app and kernel space. Details: 1) Shared structure Below is the common structure to be shared by user and kernel application for communication across user-kernel boundary: ////// typedef struct _CS { int cmd; // my command number void* cmd_payload; // command payload void* response; // response } COMMAND_STRUCTURE, *PCOMMAND_STRUCTURE; ////// Note: The maximum size of command payload was of 4272 bytes , so i want to send the command with the required bytes (and not the whole structure, as void* com_payload) 2) User-Application The user application sends the structure to the kernel space, and recieve the output in the same structure. So it will be a "Scalar Input and Scalar output" ///////// kernReturn = IOConnectMethodStructureIStructureO(*dataPort, kGnsGetVersion, //cmd inBuffSize, //insize &outBuffSize, //outSize &structIn, &structOut); ///////// 3) Kernel Level. At KEXT level, Now, the main point of discussion is ,"how does the kernel allocate/map the user-level structure in kernel and return back the data from kernel memory back to user space"? After studying the apple docs i have come to following conculsion and wrote the steps for the same. Following steps needs your review/comments ( i am not sure about these steps, so please correct me incase i am wrong) For Sending the data into kernel space from the user-application. step a) Use IOMemoryDescriptor::withAddress to create and initializes an IOMemoryDescriptor /////////// IOMemoryDescriptor *descriptor = NULL; descriptor = IOMemoryDescriptor::= withAddress( address_of_CommandStructure, sizeof(command_structure), kIODirectionIn, // I am not sure about this , why do we need this ? kernel_task // since we are passing data into kernel ); ////////////// b) Now copy data from the memory descriptor's buffer to the buffer allocated in the kernel space. /////////////// ret= descriptor ->prepare( kIODirectionOut ); IOByteCount bytecont = descriptor->readBytes( 0, //offset myBuff, Ientocopy); descriptor->complete(); //////////////// c) Now use this myBuff to communicate with the device.(The module works fine) d) now we have got the data say,in outBuff, and this needs to be sent back to user space. Now at this point, do we need to map output data recieved from the device data in-order to send to user space ? If yes, then how ? Is the method correct ? Or there is some alternate methods available ? I also wanted to mention one more thing: In case of my fixed structure , mentioned below ////////// typedef struct packet{ unsigned long pid; unsigned long cmdId; } packet, *ppacket; //////// I have not mapped the data into the kernel space, still it works fine i.e the data is successfully sent to communication module and the output is successfully sent back to user without any issues. Is the mapping is required when the data allocation is dynamic ? Please correct me if I am wrong. Please send your useful comments/suggestions for the same. also any study material, docs , links are most welcome!!! Thanks and regards, Rohit Dhamija _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: udp port sharing across processes on OS X?
by Michael Smith 01 Jun '06

01 Jun '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On May 31, 2006, at 9:44 PM, Michelle Munson wrote: By setting the socket options SO_REUSEADDR and SO_REUSEPORT, I've been able to successfully get two instances of the same process running as the same user to bind and share the same UDP port on OS X. However, if the two instances of the process are running as different user ids, the bind attempt by the second instance of the process fails with "address already in use". Does anyone know if there is way to get two instances of the same process running as different user ids to reuse the same UDP port on Mac OS X? = Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This is a very odd thing to want to have happen. What behaviour would you expect from such a shared port with regards to inbound traffic? This email sent to site_archiver(a)lists.apple.com

1 0