October 2014 - Darwin-dev - lists.apple.com

Re: kqueue/EVFILT_WRITE behavior
by Martin Karlgren 30 Oct '14

30 Oct '14

Vincent, Damien, Alright, that makes sense. Thanks for your clarifications. Martin > On 29 Oct 2014, at 17:44 , Vincent Lubet <vlubet(a)apple.com> wrote: > > Martin, > > Damien is correct, your code should always gracefully handle EWOULDBLOCK because the conditions may have changed between the time EVFILT_WRITE was posted and when calling write(2). > > For TCP sockets EWOULDBLOCK is usually returned when the send socket buffer is full, the TCP window is closed, or when the outgoing interface is flow controlled. > > Other sockets may have contributed to filling up the outgoing interface after EVFILT_WRITE was posted and that could be the reason why a subsequent write(2) returns EWOULDBOCK. > > Vincent > >> On Oct 29, 2014, at 7:31 AM, Damien Sorresso <dsorresso(a)apple.com> wrote: >> >> Martin, >> >> EWOULDBLOCK is possible condition in this case, as far as I'm aware. For example, if you have multiple threads in your process all with kqueue(3) notifications set up on that socket, one thread may get the notification and fill the kernel's buffers with data before the other, and that second thread would get back EWOULDBLOCK. (Ditto for multiple processes with the same socket.) >> >> If neither of these conditions are true in your case, it's certainly weird (and perhaps a networking person could figure out what's happening), but in general I would say that you should gracefully handle EWOULDBLOCK in these cases since it is a non-blocking socket. A more exotic case might be that another subsystem in your project has a stale file descriptor that it thinks points to something else (to which it has been writing), but actually points to your socket. In this case, you have a potential data corruption bug. >> -damien >> >> On 29 Oct, 2014, at 06:13, Martin Karlgren <marty(a)roxen.com> wrote: >>> Hi all, >>> >>> I’m wondering if anyone could clarify the indented behavior of write(2) to a nonblocking (TCP) fd after an EVFILT_WRITE event? It works as expected most of the time, i.e. write(2) writes approximately the same number of bytes as EVFILT_WRITE reported being available in the send buffer, but sometimes I’m observing that write(2) returns -1 with errno being set to EWOULDBLOCK after such an event. If I understand correctly, the (e)poll API guarantees that write(2) is able to write at least one byte without blocking after a POLLOUT event – isn’t that the case with kqueue/EVFILT_WRITE? (The man page doesn’t specify afaics.) >>> >>> What’s actually happening inside the kernel in this case – does it ever shrink the send buffer? If yes, I guess it’s shrunk between the EVFILT_WRITE event and my write(2) call, meaning that there isn’t any buffer space available anymore at that point. (Yes, I do understand that I can simply try again in this case, but I’d like to understand the semantics...) >>> >>> I’m using Darwin 14.0.0 (OS X Yosemite.) >>> >>> Best regards, >>> Marty >>> -- > _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.ap… This email sent to site_archiver(a)lists.apple.com

1 0

Re: kqueue/EVFILT_WRITE behavior
by Vincent Lubet 29 Oct '14

29 Oct '14

Martin, Damien is correct, your code should always gracefully handle EWOULDBLOCK because the conditions may have changed between the time EVFILT_WRITE was posted and when calling write(2). For TCP sockets EWOULDBLOCK is usually returned when the send socket buffer is full, the TCP window is closed, or when the outgoing interface is flow controlled. Other sockets may have contributed to filling up the outgoing interface after EVFILT_WRITE was posted and that could be the reason why a subsequent write(2) returns EWOULDBOCK. Vincent > On Oct 29, 2014, at 7:31 AM, Damien Sorresso <dsorresso(a)apple.com> wrote: > > Martin, > > EWOULDBLOCK is possible condition in this case, as far as I'm aware. For example, if you have multiple threads in your process all with kqueue(3) notifications set up on that socket, one thread may get the notification and fill the kernel's buffers with data before the other, and that second thread would get back EWOULDBLOCK. (Ditto for multiple processes with the same socket.) > > If neither of these conditions are true in your case, it's certainly weird (and perhaps a networking person could figure out what's happening), but in general I would say that you should gracefully handle EWOULDBLOCK in these cases since it is a non-blocking socket. A more exotic case might be that another subsystem in your project has a stale file descriptor that it thinks points to something else (to which it has been writing), but actually points to your socket. In this case, you have a potential data corruption bug. > -damien > > On 29 Oct, 2014, at 06:13, Martin Karlgren <marty(a)roxen.com> wrote: >> Hi all, >> >> I’m wondering if anyone could clarify the indented behavior of write(2) to a nonblocking (TCP) fd after an EVFILT_WRITE event? It works as expected most of the time, i.e. write(2) writes approximately the same number of bytes as EVFILT_WRITE reported being available in the send buffer, but sometimes I’m observing that write(2) returns -1 with errno being set to EWOULDBLOCK after such an event. If I understand correctly, the (e)poll API guarantees that write(2) is able to write at least one byte without blocking after a POLLOUT event – isn’t that the case with kqueue/EVFILT_WRITE? (The man page doesn’t specify afaics.) >> >> What’s actually happening inside the kernel in this case – does it ever shrink the send buffer? If yes, I guess it’s shrunk between the EVFILT_WRITE event and my write(2) call, meaning that there isn’t any buffer space available anymore at that point. (Yes, I do understand that I can simply try again in this case, but I’d like to understand the semantics...) >> >> I’m using Darwin 14.0.0 (OS X Yosemite.) >> >> Best regards, >> Marty >> -- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.ap… This email sent to site_archiver(a)lists.apple.com

1 0

Re: kqueue/EVFILT_WRITE behavior
by Damien Sorresso 29 Oct '14

29 Oct '14

Martin, EWOULDBLOCK is possible condition in this case, as far as I'm aware. For example, if you have multiple threads in your process all with kqueue(3) notifications set up on that socket, one thread may get the notification and fill the kernel's buffers with data before the other, and that second thread would get back EWOULDBLOCK. (Ditto for multiple processes with the same socket.) If neither of these conditions are true in your case, it's certainly weird (and perhaps a networking person could figure out what's happening), but in general I would say that you should gracefully handle EWOULDBLOCK in these cases since it is a non-blocking socket. A more exotic case might be that another subsystem in your project has a stale file descriptor that it thinks points to something else (to which it has been writing), but actually points to your socket. In this case, you have a potential data corruption bug. -damien On 29 Oct, 2014, at 06:13, Martin Karlgren <marty(a)roxen.com> wrote: > Hi all, > > I’m wondering if anyone could clarify the indented behavior of write(2) to a nonblocking (TCP) fd after an EVFILT_WRITE event? It works as expected most of the time, i.e. write(2) writes approximately the same number of bytes as EVFILT_WRITE reported being available in the send buffer, but sometimes I’m observing that write(2) returns -1 with errno being set to EWOULDBLOCK after such an event. If I understand correctly, the (e)poll API guarantees that write(2) is able to write at least one byte without blocking after a POLLOUT event – isn’t that the case with kqueue/EVFILT_WRITE? (The man page doesn’t specify afaics.) > > What’s actually happening inside the kernel in this case – does it ever shrink the send buffer? If yes, I guess it’s shrunk between the EVFILT_WRITE event and my write(2) call, meaning that there isn’t any buffer space available anymore at that point. (Yes, I do understand that I can simply try again in this case, but I’d like to understand the semantics...) > > I’m using Darwin 14.0.0 (OS X Yosemite.) > > Best regards, > Marty > -- > Roxen Internet Software > Martin Karlgren, Developer/Consultant > Box 449, SE-581 05 Linköping, Sweden > T: +46 13 376800 > marty(a)roxen.com > > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) > Help/Unsubscribe/Update your Subscription: > https://lists.apple.com/mailman/options/darwin-dev/dsorresso%40apple.com > > This email sent to dsorresso(a)apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.ap… This email sent to site_archiver(a)lists.apple.com

1 0

kqueue/EVFILT_WRITE behavior
by Martin Karlgren 29 Oct '14

29 Oct '14

_______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.ap… This email sent to site_archiver(a)lists.apple.com

1 0

test again
by Damien Sorresso 17 Oct '14

17 Oct '14

ignore again _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.ap… This email sent to site_archiver(a)lists.apple.com

1 0

test
by Damien Sorresso 17 Oct '14

17 Oct '14

ignore _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.ap… This email sent to site_archiver(a)lists.apple.com

1 0

dsymutil source?
by Daniel Peebles 11 Oct '14

11 Oct '14

Hi all, I searched around and haven't found anything about this in the past few years. Back in 2007 there was an email saying that dsymutil and other DWARF tools weren't currently public, but I was wondering if there were plans to ever make it public? Having the source of this tool (ideally in a buildable form) would be quite useful to me and others, and from my naive point of view seems unlikely to contain anything particularly proprietary. Am I mistaken there? Thank you, Dan _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.ap… This email sent to site_archiver(a)lists.apple.com

1 0