June 2008 - Darwin-kernel - lists.apple.com

Re: multiple threads mutex, unlocking order
by Michael Smith 16 Jun '08

16 Jun '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Jun 16, 2008, at 8:09 AM, Matt Connolly wrote: They should, yes. No. HTH. = Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… Do the IOLock lock and unlock routines have to be called from the same thread? If multiple threads are waiting on the IOLock, are they executed in the order that they began waiting on the Lock? I'm looking for a way for a kernel task to wait for a user space process to respond to a message. This is generally a bad way to do things; you want the user process to drive the messaging, not the kernel. The underlying issue is that user processes are transient and unreliable, so you need to behave correctly in the case where the process fails to complete the work, or terminates while the work is in progress. IFF we assume that you have arranged for the user process to re-start, you need to maintain queues of work items (in progress, waiting) inside the kernel so that you can re-issue work to your user process if it checks in after crashing. (And your work items need to be devised such that they can safely be re-started). Basically, I want to serialise things, so that if there are multiple kernel threads sending to the user-space process, that they will have their information processed, and be woken from their locks in the correct order. A better model is for the work items to be queued inside the kernel, and for the user process to come along and pick up these work items as it has resources available. As for "the correct order", since kernel threads run concurrently with user threads, it is entirely your responsibility to ensure ordering. You can use a global sequence number, or sort the queue, or... there are many options depending solely on what sort of ordering you're actually talking about. This email sent to site_archiver(a)lists.apple.com

1 0

multiple threads mutex, unlocking order
by Matt Connolly 16 Jun '08

16 Jun '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=default; d=soundevolution.com.au; h=Received:Message-Id:From:To:Content-Type:Mime-Version:Subject:Date:X-Mailer:X-Identified-User:DomainKey-Status; b=ZDGz8fFrqKB0X8x5yZDXStYNhtLV/XIbueCu0EznXcnEALEmS5WOCxIZLgG6gKDZJEcWnXTXc5/NSzjM9S8yl2qpc9gD4Xf8hFnFwfcgFAusahSRzp0D9gxvnW7ERFsq; Domainkey-status: no signature Kernel thread #1 goes: 1. enqueue data to user-space process 2. call lock on mutex (already locked, so blocks). Kernel thread #2 1. gets message from IOUserClient subclass, 2. calls IOLockUnlock on the mutex, to wake up thread #1. -Matt _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com Do the IOLock lock and unlock routines have to be called from the same thread? If multiple threads are waiting on the IOLock, are they executed in the order that they began waiting on the Lock? I'm looking for a way for a kernel task to wait for a user space process to respond to a message. user-space process goes: 1. receive notification via data queue 2. notify kernel that we've processed the message - calling via IOUserClient connection Basically, I want to serialise things, so that if there are multiple kernel threads sending to the user-space process, that they will have their information processed, and be woken from their locks in the correct order. smime.p7s

1 0

Re: UDP kernel sockets
by Michael Tüxen 10 Jun '08

10 Jun '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Adi, Done. It is Bug ID# 5997233. Best reagards Michael On Jun 10, 2008, at 4:08 PM, Adi Masputra wrote: Michael, Please file a radar against this; thanks. Adi On Jun 10, 2008, at 5:47 AM, Michael Tüxen wrote: Dear all, I have done some further testing... I think it is a bug in the kernel: sock_receive_internal() in kpi_socket.c does: soreceive() starts with soreceive(struct socket *so, struct sockaddr **psa, struct uio *uio, struct mbuf **mp0, struct mbuf **controlp, int *flagsp) { ... if (controlp) *controlp = 0; Is this analysis correct and is it a bug in Mac OS X? As a fix I think one can just remove Best reagards Michael On Jun 9, 2008, at 6:48 PM, Michael Tüxen wrote: Dear all, I'm using a UDP kernel socket created by (error handling not shown): [mbp15:~] tuexen% netstat -m 16933/17080 mbufs in use: 473 mbufs allocated to data 16459 mbufs allocated to ancillary data 1 mbufs allocated to Appletalk data blocks 147 mbufs allocated to caches 2854/3664 mbuf 2KB clusters in use 0/16 mbuf 4KB clusters in use 0/0 mbuf 16KB clusters in use 7392 KB allocated to network (69.6% in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to drain routines Is the code above not correct or is it a bug in the kernel? Thank you very much for your help! Best regards Michael _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/michael.tuexen%40lurch… This email sent to michael.tuexen(a)lurchi.franken.de _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/adi%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… if (msg && msg->msg_control) { if ((size_t)msg->msg_controllen < sizeof(struct cmsghdr)) return EINVAL; if ((size_t)msg->msg_controllen > MLEN) return EINVAL; control = m_get(M_NOWAIT, MT_CONTROL); if (control == NULL) return ENOMEM; memcpy(mtod(control, caddr_t), msg->msg_control, msg- >msg_controllen); control->m_len = msg->msg_controllen; } /* let pru_soreceive handle the socket locking */ error = sock->so_proto->pr_usrreqs->pru_soreceive(sock, &fromsa, auio, data, control ? &control : NULL, &flags); Now assume that msg != NULL and msg->msg_control != NULL. So an mbuf is allocated and a pointer to it is stored in control. Now pru_soreceive is called. For UDP this is just soreceive() from uipc_socket.c. So the pointer to the mbuf allocated in sock_receive_internal is lost and the mbuf leaked. Lateron *controlp is set to point to an mbuf containing the control information, which is why the code works and that mbuf is freed. control = m_get(M_NOWAIT, MT_CONTROL); if (control == NULL) return ENOMEM; memcpy(mtod(control, caddr_t), msg->msg_control, msg- >msg_controllen); control->m_len = msg->msg_controllen; or is there any information passed in pru_soreceive via controlp? If that is the case, the soreceive should free the mbuf and not only throw away a pointer to it... error = sock_socket(PF_INET, SOCK_DGRAM, IPPROTO_UDP, sctp_over_udp_ipv4_cb, NULL, &sctp_over_udp_ipv4_so); error = sock_setsockopt(sctp_over_udp_ipv4_so, IPPROTO_IP, IP_RECVDSTADDR, (const void *)&on, (int)sizeof(int)); memset((void *)&addr_ipv4, 0, sizeof(struct sockaddr_in)); addr_ipv4.sin_len = sizeof(struct sockaddr_in); addr_ipv4.sin_family = AF_INET; addr_ipv4.sin_port = htons(sctp_udp_tunneling_port); addr_ipv4.sin_addr.s_addr = htonl(INADDR_ANY); error = sock_bind(sctp_over_udp_ipv4_so, (const struct sockaddr *)&addr_ipv4); and the sctp_over_udp_ipv4_cb looks like (also error handling suppressed): void sctp_over_udp_ipv4_cb(socket_t udp_sock, void *cookie, int watif) { errno_t error; size_t length; mbuf_t packet; struct msghdr msg; struct sockaddr_in src, dst; char cmsgbuf[CMSG_SPACE(sizeof (struct in_addr))]; struct cmsghdr *cmsg; struct ip *ip; struct mbuf *ip_m; bzero((void *)&msg, sizeof(struct msghdr)); bzero((void *)&src, sizeof(struct sockaddr_in)); bzero((void *)&dst, sizeof(struct sockaddr_in)); bzero((void *)cmsgbuf, CMSG_SPACE(sizeof (struct in_addr))); msg.msg_name = (void *)&src; msg.msg_namelen = sizeof(struct sockaddr_in); msg.msg_iov = NULL; msg.msg_iovlen = 0; msg.msg_control = (void *)cmsgbuf; msg.msg_controllen = CMSG_LEN(sizeof (struct in_addr)); msg.msg_flags = 0; length = (1<<16); error = sock_receivembuf(udp_sock, &msg, &packet, 0, &length); for (cmsg = CMSG_FIRSTHDR(&msg); cmsg; cmsg = CMSG_NXTHDR(&msg, cmsg)) { if ((cmsg->cmsg_level == IPPROTO_IP) && (cmsg->cmsg_type == IP_RECVDSTADDR)) { dst.sin_family = AF_INET; dst.sin_len = sizeof(struct sockaddr_in); dst.sin_port = htons(sctp_udp_tunneling_port); memcpy((void *)&dst.sin_addr, (const void *)CMSG_DATA(cmsg), sizeof(struct in_addr)); } } ip_m = sctp_get_mbuf_for_msg(sizeof(struct ip), 1, M_DONTWAIT, 1, MT_DATA); ip_m->m_pkthdr.rcvif = packet->m_pkthdr.rcvif; ip = mtod(ip_m, struct ip *); bzero((void *)ip, sizeof(struct ip)); ip->ip_v = IPVERSION; ip->ip_len = length; ip->ip_src = src.sin_addr; ip->ip_dst = dst.sin_addr; SCTP_HEADER_LEN(ip_m) = sizeof(struct ip) + length; SCTP_BUF_LEN(ip_m) = sizeof(struct ip); SCTP_BUF_NEXT(ip_m) = packet; sctp_input_with_port(ip_m, sizeof(struct ip), src.sin_port); return; } The code works. However, for each received UDP packet one mbuf for ancillary data is leaked. So after a short transfer I see: This email sent to adi(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0

Re: UDP kernel sockets
by Adi Masputra 10 Jun '08

10 Jun '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Michael, Please file a radar against this; thanks. Adi On Jun 10, 2008, at 5:47 AM, Michael Tüxen wrote: Dear all, I have done some further testing... I think it is a bug in the kernel: sock_receive_internal() in kpi_socket.c does: soreceive() starts with soreceive(struct socket *so, struct sockaddr **psa, struct uio *uio, struct mbuf **mp0, struct mbuf **controlp, int *flagsp) { ... if (controlp) *controlp = 0; Is this analysis correct and is it a bug in Mac OS X? As a fix I think one can just remove Best reagards Michael On Jun 9, 2008, at 6:48 PM, Michael Tüxen wrote: Dear all, I'm using a UDP kernel socket created by (error handling not shown): [mbp15:~] tuexen% netstat -m 16933/17080 mbufs in use: 473 mbufs allocated to data 16459 mbufs allocated to ancillary data 1 mbufs allocated to Appletalk data blocks 147 mbufs allocated to caches 2854/3664 mbuf 2KB clusters in use 0/16 mbuf 4KB clusters in use 0/0 mbuf 16KB clusters in use 7392 KB allocated to network (69.6% in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to drain routines Is the code above not correct or is it a bug in the kernel? Thank you very much for your help! Best regards Michael _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/michael.tuexen%40lurch… This email sent to michael.tuexen(a)lurchi.franken.de _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/adi%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com if (msg && msg->msg_control) { if ((size_t)msg->msg_controllen < sizeof(struct cmsghdr)) return EINVAL; if ((size_t)msg->msg_controllen > MLEN) return EINVAL; control = m_get(M_NOWAIT, MT_CONTROL); if (control == NULL) return ENOMEM; memcpy(mtod(control, caddr_t), msg->msg_control, msg- >msg_controllen); control->m_len = msg->msg_controllen; } /* let pru_soreceive handle the socket locking */ error = sock->so_proto->pr_usrreqs->pru_soreceive(sock, &fromsa, auio, data, control ? &control : NULL, &flags); Now assume that msg != NULL and msg->msg_control != NULL. So an mbuf is allocated and a pointer to it is stored in control. Now pru_soreceive is called. For UDP this is just soreceive() from uipc_socket.c. So the pointer to the mbuf allocated in sock_receive_internal is lost and the mbuf leaked. Lateron *controlp is set to point to an mbuf containing the control information, which is why the code works and that mbuf is freed. control = m_get(M_NOWAIT, MT_CONTROL); if (control == NULL) return ENOMEM; memcpy(mtod(control, caddr_t), msg->msg_control, msg- >msg_controllen); control->m_len = msg->msg_controllen; or is there any information passed in pru_soreceive via controlp? If that is the case, the soreceive should free the mbuf and not only throw away a pointer to it... error = sock_socket(PF_INET, SOCK_DGRAM, IPPROTO_UDP, sctp_over_udp_ipv4_cb, NULL, &sctp_over_udp_ipv4_so); error = sock_setsockopt(sctp_over_udp_ipv4_so, IPPROTO_IP, IP_RECVDSTADDR, (const void *)&on, (int)sizeof(int)); memset((void *)&addr_ipv4, 0, sizeof(struct sockaddr_in)); addr_ipv4.sin_len = sizeof(struct sockaddr_in); addr_ipv4.sin_family = AF_INET; addr_ipv4.sin_port = htons(sctp_udp_tunneling_port); addr_ipv4.sin_addr.s_addr = htonl(INADDR_ANY); error = sock_bind(sctp_over_udp_ipv4_so, (const struct sockaddr *)&addr_ipv4); and the sctp_over_udp_ipv4_cb looks like (also error handling suppressed): void sctp_over_udp_ipv4_cb(socket_t udp_sock, void *cookie, int watif) { errno_t error; size_t length; mbuf_t packet; struct msghdr msg; struct sockaddr_in src, dst; char cmsgbuf[CMSG_SPACE(sizeof (struct in_addr))]; struct cmsghdr *cmsg; struct ip *ip; struct mbuf *ip_m; bzero((void *)&msg, sizeof(struct msghdr)); bzero((void *)&src, sizeof(struct sockaddr_in)); bzero((void *)&dst, sizeof(struct sockaddr_in)); bzero((void *)cmsgbuf, CMSG_SPACE(sizeof (struct in_addr))); msg.msg_name = (void *)&src; msg.msg_namelen = sizeof(struct sockaddr_in); msg.msg_iov = NULL; msg.msg_iovlen = 0; msg.msg_control = (void *)cmsgbuf; msg.msg_controllen = CMSG_LEN(sizeof (struct in_addr)); msg.msg_flags = 0; length = (1<<16); error = sock_receivembuf(udp_sock, &msg, &packet, 0, &length); for (cmsg = CMSG_FIRSTHDR(&msg); cmsg; cmsg = CMSG_NXTHDR(&msg, cmsg)) { if ((cmsg->cmsg_level == IPPROTO_IP) && (cmsg->cmsg_type == IP_RECVDSTADDR)) { dst.sin_family = AF_INET; dst.sin_len = sizeof(struct sockaddr_in); dst.sin_port = htons(sctp_udp_tunneling_port); memcpy((void *)&dst.sin_addr, (const void *)CMSG_DATA(cmsg), sizeof(struct in_addr)); } } ip_m = sctp_get_mbuf_for_msg(sizeof(struct ip), 1, M_DONTWAIT, 1, MT_DATA); ip_m->m_pkthdr.rcvif = packet->m_pkthdr.rcvif; ip = mtod(ip_m, struct ip *); bzero((void *)ip, sizeof(struct ip)); ip->ip_v = IPVERSION; ip->ip_len = length; ip->ip_src = src.sin_addr; ip->ip_dst = dst.sin_addr; SCTP_HEADER_LEN(ip_m) = sizeof(struct ip) + length; SCTP_BUF_LEN(ip_m) = sizeof(struct ip); SCTP_BUF_NEXT(ip_m) = packet; sctp_input_with_port(ip_m, sizeof(struct ip), src.sin_port); return; } The code works. However, for each received UDP packet one mbuf for ancillary data is leaked. So after a short transfer I see: This email sent to adi(a)apple.com smime.p7s

1 0

Re: UDP kernel sockets
by Michael Tüxen 10 Jun '08

10 Jun '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dear all, I have done some further testing... I think it is a bug in the kernel: sock_receive_internal() in kpi_socket.c does: /* let pru_soreceive handle the socket locking */ error = sock->so_proto->pr_usrreqs->pru_soreceive(sock, &fromsa, auio, data, control ? &control : NULL, &flags); soreceive() starts with soreceive(struct socket *so, struct sockaddr **psa, struct uio *uio, struct mbuf **mp0, struct mbuf **controlp, int *flagsp) { ... if (controlp) *controlp = 0; Is this analysis correct and is it a bug in Mac OS X? As a fix I think one can just remove control = m_get(M_NOWAIT, MT_CONTROL); if (control == NULL) return ENOMEM; memcpy(mtod(control, caddr_t), msg->msg_control, msg->msg_controllen); control->m_len = msg->msg_controllen; Best reagards Michael On Jun 9, 2008, at 6:48 PM, Michael Tüxen wrote: Dear all, I'm using a UDP kernel socket created by (error handling not shown): [mbp15:~] tuexen% netstat -m 16933/17080 mbufs in use: 473 mbufs allocated to data 16459 mbufs allocated to ancillary data 1 mbufs allocated to Appletalk data blocks 147 mbufs allocated to caches 2854/3664 mbuf 2KB clusters in use 0/16 mbuf 4KB clusters in use 0/0 mbuf 16KB clusters in use 7392 KB allocated to network (69.6% in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to drain routines Is the code above not correct or is it a bug in the kernel? Thank you very much for your help! Best regards Michael _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/michael.tuexen%40lurch… This email sent to michael.tuexen(a)lurchi.franken.de _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… if (msg && msg->msg_control) { if ((size_t)msg->msg_controllen < sizeof(struct cmsghdr)) return EINVAL; if ((size_t)msg->msg_controllen > MLEN) return EINVAL; control = m_get(M_NOWAIT, MT_CONTROL); if (control == NULL) return ENOMEM; memcpy(mtod(control, caddr_t), msg->msg_control, msg->msg_controllen); control->m_len = msg->msg_controllen; } Now assume that msg != NULL and msg->msg_control != NULL. So an mbuf is allocated and a pointer to it is stored in control. Now pru_soreceive is called. For UDP this is just soreceive() from uipc_socket.c. So the pointer to the mbuf allocated in sock_receive_internal is lost and the mbuf leaked. Lateron *controlp is set to point to an mbuf containing the control information, which is why the code works and that mbuf is freed. or is there any information passed in pru_soreceive via controlp? If that is the case, the soreceive should free the mbuf and not only throw away a pointer to it... error = sock_socket(PF_INET, SOCK_DGRAM, IPPROTO_UDP, sctp_over_udp_ipv4_cb, NULL, &sctp_over_udp_ipv4_so); error = sock_setsockopt(sctp_over_udp_ipv4_so, IPPROTO_IP, IP_RECVDSTADDR, (const void *)&on, (int)sizeof(int)); memset((void *)&addr_ipv4, 0, sizeof(struct sockaddr_in)); addr_ipv4.sin_len = sizeof(struct sockaddr_in); addr_ipv4.sin_family = AF_INET; addr_ipv4.sin_port = htons(sctp_udp_tunneling_port); addr_ipv4.sin_addr.s_addr = htonl(INADDR_ANY); error = sock_bind(sctp_over_udp_ipv4_so, (const struct sockaddr *)&addr_ipv4); and the sctp_over_udp_ipv4_cb looks like (also error handling suppressed): void sctp_over_udp_ipv4_cb(socket_t udp_sock, void *cookie, int watif) { errno_t error; size_t length; mbuf_t packet; struct msghdr msg; struct sockaddr_in src, dst; char cmsgbuf[CMSG_SPACE(sizeof (struct in_addr))]; struct cmsghdr *cmsg; struct ip *ip; struct mbuf *ip_m; bzero((void *)&msg, sizeof(struct msghdr)); bzero((void *)&src, sizeof(struct sockaddr_in)); bzero((void *)&dst, sizeof(struct sockaddr_in)); bzero((void *)cmsgbuf, CMSG_SPACE(sizeof (struct in_addr))); msg.msg_name = (void *)&src; msg.msg_namelen = sizeof(struct sockaddr_in); msg.msg_iov = NULL; msg.msg_iovlen = 0; msg.msg_control = (void *)cmsgbuf; msg.msg_controllen = CMSG_LEN(sizeof (struct in_addr)); msg.msg_flags = 0; length = (1<<16); error = sock_receivembuf(udp_sock, &msg, &packet, 0, &length); for (cmsg = CMSG_FIRSTHDR(&msg); cmsg; cmsg = CMSG_NXTHDR(&msg, cmsg)) { if ((cmsg->cmsg_level == IPPROTO_IP) && (cmsg->cmsg_type == IP_RECVDSTADDR)) { dst.sin_family = AF_INET; dst.sin_len = sizeof(struct sockaddr_in); dst.sin_port = htons(sctp_udp_tunneling_port); memcpy((void *)&dst.sin_addr, (const void *)CMSG_DATA(cmsg), sizeof(struct in_addr)); } } ip_m = sctp_get_mbuf_for_msg(sizeof(struct ip), 1, M_DONTWAIT, 1, MT_DATA); ip_m->m_pkthdr.rcvif = packet->m_pkthdr.rcvif; ip = mtod(ip_m, struct ip *); bzero((void *)ip, sizeof(struct ip)); ip->ip_v = IPVERSION; ip->ip_len = length; ip->ip_src = src.sin_addr; ip->ip_dst = dst.sin_addr; SCTP_HEADER_LEN(ip_m) = sizeof(struct ip) + length; SCTP_BUF_LEN(ip_m) = sizeof(struct ip); SCTP_BUF_NEXT(ip_m) = packet; sctp_input_with_port(ip_m, sizeof(struct ip), src.sin_port); return; } The code works. However, for each received UDP packet one mbuf for ancillary data is leaked. So after a short transfer I see: This email sent to site_archiver(a)lists.apple.com

1 0

UDP kernel sockets
by Michael Tüxen 09 Jun '08

09 Jun '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dear all, I'm using a UDP kernel socket created by (error handling not shown): [mbp15:~] tuexen% netstat -m 16933/17080 mbufs in use: 473 mbufs allocated to data 16459 mbufs allocated to ancillary data 1 mbufs allocated to Appletalk data blocks 147 mbufs allocated to caches 2854/3664 mbuf 2KB clusters in use 0/16 mbuf 4KB clusters in use 0/0 mbuf 16KB clusters in use 7392 KB allocated to network (69.6% in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to drain routines Is the code above not correct or is it a bug in the kernel? Thank you very much for your help! Best regards Michael _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… error = sock_socket(PF_INET, SOCK_DGRAM, IPPROTO_UDP, sctp_over_udp_ipv4_cb, NULL, &sctp_over_udp_ipv4_so); error = sock_setsockopt(sctp_over_udp_ipv4_so, IPPROTO_IP, IP_RECVDSTADDR, (const void *)&on, (int)sizeof(int)); memset((void *)&addr_ipv4, 0, sizeof(struct sockaddr_in)); addr_ipv4.sin_len = sizeof(struct sockaddr_in); addr_ipv4.sin_family = AF_INET; addr_ipv4.sin_port = htons(sctp_udp_tunneling_port); addr_ipv4.sin_addr.s_addr = htonl(INADDR_ANY); error = sock_bind(sctp_over_udp_ipv4_so, (const struct sockaddr *)&addr_ipv4); and the sctp_over_udp_ipv4_cb looks like (also error handling suppressed): void sctp_over_udp_ipv4_cb(socket_t udp_sock, void *cookie, int watif) { errno_t error; size_t length; mbuf_t packet; struct msghdr msg; struct sockaddr_in src, dst; char cmsgbuf[CMSG_SPACE(sizeof (struct in_addr))]; struct cmsghdr *cmsg; struct ip *ip; struct mbuf *ip_m; bzero((void *)&msg, sizeof(struct msghdr)); bzero((void *)&src, sizeof(struct sockaddr_in)); bzero((void *)&dst, sizeof(struct sockaddr_in)); bzero((void *)cmsgbuf, CMSG_SPACE(sizeof (struct in_addr))); msg.msg_name = (void *)&src; msg.msg_namelen = sizeof(struct sockaddr_in); msg.msg_iov = NULL; msg.msg_iovlen = 0; msg.msg_control = (void *)cmsgbuf; msg.msg_controllen = CMSG_LEN(sizeof (struct in_addr)); msg.msg_flags = 0; length = (1<<16); error = sock_receivembuf(udp_sock, &msg, &packet, 0, &length); for (cmsg = CMSG_FIRSTHDR(&msg); cmsg; cmsg = CMSG_NXTHDR(&msg, cmsg)) { if ((cmsg->cmsg_level == IPPROTO_IP) && (cmsg->cmsg_type == IP_RECVDSTADDR)) { dst.sin_family = AF_INET; dst.sin_len = sizeof(struct sockaddr_in); dst.sin_port = htons(sctp_udp_tunneling_port); memcpy((void *)&dst.sin_addr, (const void *)CMSG_DATA(cmsg), sizeof(struct in_addr)); } } ip_m = sctp_get_mbuf_for_msg(sizeof(struct ip), 1, M_DONTWAIT, 1, MT_DATA); ip_m->m_pkthdr.rcvif = packet->m_pkthdr.rcvif; ip = mtod(ip_m, struct ip *); bzero((void *)ip, sizeof(struct ip)); ip->ip_v = IPVERSION; ip->ip_len = length; ip->ip_src = src.sin_addr; ip->ip_dst = dst.sin_addr; SCTP_HEADER_LEN(ip_m) = sizeof(struct ip) + length; SCTP_BUF_LEN(ip_m) = sizeof(struct ip); SCTP_BUF_NEXT(ip_m) = packet; sctp_input_with_port(ip_m, sizeof(struct ip), src.sin_port); return; } The code works. However, for each received UDP packet one mbuf for ancillary data is leaked. So after a short transfer I see: This email sent to site_archiver(a)lists.apple.com

1 0

Re: Using KPIs, was Accessing ifnet
by Michael Tüxen 06 Jun '08

06 Jun '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Hi Rick, you are describing exactly my situation... However, there are people out there who even use our SCTP stack in their production environment (and they are updating it for every release)... But they know about the situation... Best regards Michael On Jun 6, 2008, at 7:05 PM, Rick Macklem wrote: Just thought I'd throw in this perspective, rick _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/michael.tuexen%40lurch… This email sent to michael.tuexen(a)lurchi.franken.de _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… I'm not sure, but I have a hunch that Michael might be in a similar situation as I am. In any case, let me hypothetically assume that he is. --> Developing open source for BSD kernels, ported to several major variants (Free, Open, Mac OS X, ...) non-commercially. When you are in this situation, there is a constant tradeoff between using the Apple KPIs vs maintaining a common code base for all the BSD variants you are ported to. Personally, I've grown rather fond of the Apple KPIs and would be happy to see other BSD variants adopt them, but that just ain't gonna happen. I've ended up with a compromise, where I use the KPIs when it is easy to do so (and easy to replace the calls with generic BSD code like vp- >v_type, if the core team for that variant deems it necessary) and going beyond/around the KPIs when I can create a code base that works on all the variants I'm ported to, using macros and other tricks. And, yes, it is only appropriate for folks in the open source, non- commercial camp, like me. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Accessing ifnet
by Michael Tüxen 06 Jun '08

06 Jun '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Hi John, comments in-line. Best regards Michael On Jun 6, 2008, at 5:27 PM, Josh Graessley wrote: On Jun 6, 2008, at 10:02 AM, Rick Macklem wrote: On Fri, 6 Jun 2008, Michael TÃ¼xen wrote: Dear all, in an NKE I'm using code like TAILQ_FOREACH(ifn, &ifnet, if_list) { /* Do something with ifn */ } and need to protect it by ifnet_head_lock_shared(); TAILQ_FOREACH(ifn, &ifnet, if_list) { /* Do something with ifn */ } ifnet_head_done(); but the kextload reports kld(): Undefined symbols: _ifnet_head_done _ifnet_head_lock_shared As OSBundleLibraries I use com.apple.kernel. I do know that I'm using unsupported API... but is there a way to use the mutex routines? -josh _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… To the best of my knowledge, you can't link to those routines, because they aren't in the mach_kernel's symbol table. However, so long as you abide by the terms of the APSL, you can crib them from the sources and put them in your Kext. (I'm assuming you already understand the risks associated with doing this, such as "Apple could change them for any release and you code will then be broken".) If you download xnu-1228 sources, look in bsd/net/dlil.c for them. You can also bypass the safety on your food processor and get it to run while the lid is off and reach your hand in but I wouldn't recommend it. 1) In order to get your kext to link, you'll have to declare a dependency on the exact version of the kernel, which means your kext will only work with one specific version of Mac OS X. Every software update will break your product. I know! 2) There's a better way. There are a whole set of KPIs to do whatever you are likely to be trying to do. They will take and drop any appropriate locks for you. I'm using KPIs were possible. However, implementing a transport stack is currently not possible with using only supported KPIs... That is why I have build the NKE for each version of the OS (more precisely: change the Info.plist file). Major changes to the code where only necessary from Panther -> Tiger -> Leopard... This email sent to site_archiver(a)lists.apple.com

1 0

Re: Accessing ifnet
by Michael Tüxen 06 Jun '08

06 Jun '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Hi Vincent, many thanks! That works, I'll use that code. Best regards Michael On Jun 6, 2008, at 5:03 PM, Vincent Lubet wrote: For example: #include <net/kpi_interface.h> .... Vincent On Jun 6, 2008, at 7:21 AM, Adi Masputra wrote: On Jun 6, 2008, at 1:17 AM, Michael Tüxen wrote: Adi _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/vlubet%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… errno_t printf_ifnets(void) { errno_t error = 0; ifnet_t *ifnetlist = 0, ifp; u_int32_t n, count; error = ifnet_list_get(IFNET_FAMILY_ANY, &ifnetlist, &count); if (error != 0) { printf("ifnet_list_get failed %d\n", error); goto out; } for (n = 0; n < count; n++) { ifp = ifnetlist[n]; printf("ifp %p index: %d name: %s unit: %d family: %d type: %d flags: %08x\n", ifp, ifnet_index(ifp), ifnet_name(ifp), ifnet_unit(ifp), ifnet_family(ifp), ifnet_type(ifp), ifnet_flags(ifp) ); } out: if (ifnetlist != 0) ifnet_list_free(ifnetlist); return error; } Loop through all interfaces and addresses and add or delete them to our own list of addresses. I guess that I could also use a kernel socket and an ioctl to get the information, but the code is shared between the FreeBSD kernel, other platforms and Mac OS X. I'm trying to keep the platform specific code minimal... Have you looked at ifnet_list_get/ifnet_list_free KPI in <net/ kpi_interface.h>? SIOCGIFCONF is not for a kernel ioctl client. This email sent to vlubet(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0

Re: Accessing ifnet
by Michael Tüxen 06 Jun '08

06 Jun '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Hi Rick, comments in-line. Best regards Michael On Jun 6, 2008, at 5:02 PM, Rick Macklem wrote: On Fri, 6 Jun 2008, Michael TÃ¼xen wrote: Dear all, in an NKE I'm using code like TAILQ_FOREACH(ifn, &ifnet, if_list) { /* Do something with ifn */ } and need to protect it by ifnet_head_lock_shared(); TAILQ_FOREACH(ifn, &ifnet, if_list) { /* Do something with ifn */ } ifnet_head_done(); but the kextload reports kld(): Undefined symbols: _ifnet_head_done _ifnet_head_lock_shared As OSBundleLibraries I use com.apple.kernel. I do know that I'm using unsupported API... but is there a way to use the mutex routines? To the best of my knowledge, you can't link to those routines, because they aren't in the mach_kernel's symbol table. However, so long as you abide by the terms of the APSL, you can crib them from the sources and put them in your Kext. (I'm assuming you already understand the risks associated with doing this, such as "Apple could change them for any release and you code will then be broken".) _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… If you download xnu-1228 sources, look in bsd/net/dlil.c for them. Yes, I tried this, but it did not work. That's why I'm asking. rick This email sent to site_archiver(a)lists.apple.com

1 0