February 2005 - Darwin-dev - lists.apple.com

Re: bad fsync? (A.M.)
by Jonas Maebe 22 Feb '05

22 Feb '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On 22 feb 2005, at 10:08, Mike Smith wrote: Jonas _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… If you were to use those platforms' functionality (where avaialble) to force a full push to disk, you would see similar performance regressions. Doesn't this behaviour also mean that the "safety" provided by journaling is basically not there? Or does the journaling code use something similar to F_FULLFSYNC to ensure that everything gets written do disk in the right order? This email sent to site_archiver(a)lists.apple.com

1 0

Re: bad fsync? (A.M.)
by Mike Smith 22 Feb '05

22 Feb '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Thanks for a great explanation. A similar thread has come up on the sqlite list, claiming that using F_FULLFSYNC leads to dramatic performance decreases on Darwin over other platforms. Any thoughts on these issues? I noticed this as well, so I profiled my call and found sync was taking forever. I removed the following fcntl call, rc = fcntl(fd, F_FULLFSYNC, 0);. Performance was back to normal. ... but the security provided by F_FULLSYNC is lost. = Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… If you were to use those platforms' functionality (where avaialble) to force a full push to disk, you would see similar performance regressions. It's a simple question of performance vs. reliability; in order to give the best guarantee that data has made it to stable storage, the system has to spend a (relatively) long time waiting for the disk. If you want performance, that wait is unacceptable. If you want reliability, the uncertainty that the disk (and sometimes controller) cache(s) introduce is unacceptable. At least Darwin gives you the choice; it's up to you to make the tradeoff that suits your application's needs. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Various tools update request
by Felix Kronlage 22 Feb '05

22 Feb '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 22, 2005, at 09:38, Sam Hart wrote: - bison - sed - flex - help2man - cvs - -fkr - -- 076E 1E87 3E05 1C7F B1A0 8A48 0D31 9BD3 D9AC 74D0 http://fkr.hazardous.org/ | http://opendarwin.org/~fkr/ | FKR-RIPE -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin) iD8DBQFCGvOxDTGb09msdNARAgCzAJ4vmposFBsrzFbM51RUvb8hmc0uCQCeN1dy 0LKbfYvXgxKYgtM4Vml69bU= =IKBw -----END PGP SIGNATURE----- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Has anyone considered just doing the port and getting the port file committed to DarwinPorts[1] ? actually all except help2man are already available, in the requested version, through DarwinPorts. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Various tools update request
by Sam Hart 22 Feb '05

22 Feb '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Adriaan van Os wrote : - bison An old version gets installed. Bison 2.0 has been released <ftp://ftp.gnu.org/gnu/bison/bison-2.0.tar.gz> - sed FreeBSD sed gets installed, but it is crippled (doesn't recognize regular expressions) GNU sed is preferred <ftp://ftp.gnu.org/gnu/sed/> - flex An old version of flex gets installed. Please note that flex is now on sourceforge (previously on gnu ftp servers) <http://sourceforge.net/project/showfiles.php?group_id=72099> - help2man Unfortunately, this tool is not installed with Xcode or OS X/Darwin (it can be configured with --enable-nls=no to prevent build problems) <ftp://ftp.gnu.org/gnu/help2man/help2man-1.35.1.tar.gz> - configure problems There is a bug in Mac OS X that causes problems when configuring many software packages (see e.g.<http://lists.gnu.org/archive/html/bug-coreutils/2004-08/ msg00039.html>). As a result of this bug, the coreutils command "uname -p" reports "unknown". This confuses config.guess. Alex v.Below wrote : - cvs An old version gets installed, and overwrites updated versions cvs 1.11.17 stable has been released S a m [1] <darwinports.opendarwin.org> ------------------------------------------------------- Drawing on my fine command of language, I said nothing. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Has anyone considered just doing the port and getting the port file committed to DarwinPorts[1] ? This email sent to site_archiver(a)lists.apple.com

1 0

Re: bad fsync? (A.M.)
by Jon Moog 22 Feb '05

22 Feb '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com If you haven't already, try: PRAGMA synchronous = OFF; FYI -Jon On Feb 21, 2005, at 11:58 AM, James Berry wrote: Dominic, From: asko.kauppi(a)sci.fi Subject: [sqlite] sluggish operation on os x? Date: February 21, 2005 12:44:03 AM PST To: sqlite-users(a)sqlite.org Reply-To: sqlite-users(a)sqlite.org _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… In my experience OS X take a much more significant hit for fsync calls than other systems. As far as I can tell there isn't a whole lot of benefit in calling fsync alone since it has no effect on the actual drive cache. Sqlite works quite a bit better with synchronous (fsync) disabled and there isn't any increase in data exposure. I'm sure you have seen this regarding the benefit of transactions: http://www.sqlite.org/speed.html Thanks for a great explanation. A similar thread has come up on the sqlite list, claiming that using F_FULLFSYNC leads to dramatic performance decreases on Darwin over other platforms. Any thoughts on these issues? Finally getting my SQLite3 code working, I'm experiencing awfully slow performance when writing individual data on OS X. Linux (Ubuntu) is lightning fast, Win32 is.. tolerable, but OS X really crawls. I've tried both with built-in sources, and the new SQLite3 fink module. Same behaviour. This email sent to site_archiver(a)lists.apple.com

1 0

Re: bad fsync? (A.M.)
by Dominic Giampaolo 21 Feb '05

21 Feb '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com I know that it hurts performance but correctness is always at odds with the desire for speed. That is correct. However one does not always need to call F_FULLFSYNC -- it should only be called for the critical pieces of data that need it. That is, most writes can just be issued normally (no fsync even). Only when you need to write the header for your transaction to be complete would you need to issue the F_FULLFSYNC fcntl. That is a very good way to put it. The unfortunate thing is that most of the time it's ok. It's only when it matters that you'll find out that you can lose data. --dominic _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I know this is a comparison of Apples and Oranges. I guess I was just looking for a reality check on this seemingly large difference in performance. I suppose that boils down to the fact that it really does take a long time to get the data onto the platter. So, in other words, we've gotten very used to the speed of the cached write, without really thinking too hard about the data safety issues. This email sent to site_archiver(a)lists.apple.com

1 0

Re: bad fsync? (A.M.)
by James Berry 21 Feb '05

21 Feb '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Hey Kevin, -jdb On Feb 21, 2005, at 1:07 PM, Kevin Van Vechten wrote: _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I know this is a comparison of Apples and Oranges. I guess I was just looking for a reality check on this seemingly large difference in performance. I suppose that boils down to the fact that it really does take a long time to get the data onto the platter. If the write just writes to the track cache, it never has to pay the penalty of seek time and rotational delay, which can (I presume one would assert) be a significant multiple of the cached write. So, in other words, we've gotten very used to the speed of the cached write, without really thinking too hard about the data safety issues. It sounds like you're comparing apples and oranges (pardon the pun) . As Dominic pointed out, F_FULLFSYNC offers a level of synchronization that's not available from fsync on Mac OS X or other platforms. If you were to only rely on fsync, I imagine the performance between systems would be comparable. One way to mitigate this issue for large operations is to wrap your statements in a single transaction. I believe this allows sqlite to sync only after the entire operation is complete, instead of between individual statements. - Kevin On Feb 21, 2005, at 9:58 AM, James Berry wrote: Dominic, Thanks for a great explanation. A similar thread has come up on the sqlite list, claiming that using F_FULLFSYNC leads to dramatic performance decreases on Darwin over other platforms. Any thoughts on these issues? From: asko.kauppi(a)sci.fi Subject: [sqlite] sluggish operation on os x? Date: February 21, 2005 12:44:03 AM PST To: sqlite-users(a)sqlite.org Reply-To: sqlite-users(a)sqlite.org Finally getting my SQLite3 code working, I'm experiencing awfully slow performance when writing individual data on OS X. Linux (Ubuntu) is lightning fast, Win32 is.. tolerable, but OS X really crawls. I've tried both with built-in sources, and the new SQLite3 fink module. Same behaviour. .... From: cking(a)mumbo.ca Subject: Re: [sqlite] sluggish operation on os x? Date: February 21, 2005 9:40:27 AM PST To: sqlite-users(a)sqlite.org Reply-To: sqlite-users(a)sqlite.org I noticed this as well, so I profiled my call and found sync was taking forever. I removed the following fcntl call, rc = fcntl(fd, F_FULLFSYNC, 0);. Performance was back to normal. ck On Feb 19, 2005, at 5:59 PM, Dominic Giampaolo wrote: MySQL makes the following claim at: http://dev.mysql.com/doc/mysql/en/news-4-1-9.html "InnoDB: Use the fcntl() file flush method on Mac OS X versions 10.3 and up. Apple had disabled fsync() in Mac OS X for internal disk drives, which caused corruption at power outages." First of all, is this accurate? A pointer to some docs or a tech note on this would be helpful. The comments about fsync() are wrong... On MacOS X, fsync() always has and always will flush all file data from host memory to the drive on which the file resides. The behavior of fsync() on MacOS X is the same as it is on every other version of Unix since the dawn of time (well, since the introduction of fsync anyway :-). I believe that what the above comment refers to is the fact that fsync() is not sufficient to guarantee that your data is on stable storage and on MacOS X we provide a fcntl(), called F_FULLFSYNC, to ask the drive to flush all buffered data to stable storage. Let me explain in more detail. With fsync() even though the OS writes the data through to the disk and the disk says "yes I wrote the data", the data is not actually on permanent storage. Unless you explicitly disable it, all disks have a write buffer which holds data you've written. The disk buffers the data you wrote until it decides to flush it to the platters (and the writes may not be in the order you wrote them). If you lose power or the system crashes before the data is written, you can wind up in a situation where only some of your data is actually on disk. What is worse is that even if you write blocks A, B and C, call fsync() and then write block D you may find after rebooting that blocks A and D are on disk but B and C are not (in fact any ordering of A, B, C, and D is possible). While this may seem like a rare case it is not. In fact if you sit down and pull the plug on a system you can make it happen in one or two plug pulls. I have even gone so far as to watch this behavior with a logic analyzer on the ATA bus: I saw the data for two writes come across the ATA cable, the drive replied and said the writes were successful and then when we rebooted the data from the second write was correct on disk but the data from the first write was not. To deal with this we introduced the F_FULLFSYNC fcntl which will ask the drive to flush all of its buffered data to disk. When an app needs to guarantee that data is on disk it should use F_FULLFSYNC. In most cases you do not need such a heavy handed operation and fsync() is good enough. But in an app like a database, it is essential if you want transactional integrity. Now, a little bit more detail: on ATA drives we implement F_FULLFSYNC with the FLUSH_TRACK_CACHE command. All drives sold by Apple will honor this command. Unfortunately quite a few firewire drive vendors disable this command and do not pass it to the drive. This means that most external firewire drives are not reliable if you lose power or the system crashes. We can't work-around that unless we ask the drive to disable the write cache completely (which hurts performance quite badly -- and even that may not be enough as some drives will ignore that request too). So in summary, I believe that the comments in the MySQL news posting are slightly confused. On MacOS X fsync() behaves the same as it does on all Unices. That's not good enough if you really care about data integrity and so we also provide the F_FULLFSYNC fcntl. As far as I know, MacOS X is the only OS to provide this feature for apps that need to truly guarantee their data is on disk. Hope this clears things up. --dominic _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/james%40jberry.us This email sent to james(a)jberry.us _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/kevin% 40opendarwin.org This email sent to kevin(a)opendarwin.org This email sent to site_archiver(a)lists.apple.com

1 0

Re: bad fsync? (A.M.)
by Kevin Van Vechten 21 Feb '05

21 Feb '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com - Kevin On Feb 21, 2005, at 9:58 AM, James Berry wrote: Dominic, From: asko.kauppi(a)sci.fi Subject: [sqlite] sluggish operation on os x? Date: February 21, 2005 12:44:03 AM PST To: sqlite-users(a)sqlite.org Reply-To: sqlite-users(a)sqlite.org .... From: cking(a)mumbo.ca Subject: Re: [sqlite] sluggish operation on os x? Date: February 21, 2005 9:40:27 AM PST To: sqlite-users(a)sqlite.org Reply-To: sqlite-users(a)sqlite.org ck On Feb 19, 2005, at 5:59 PM, Dominic Giampaolo wrote: MySQL makes the following claim at: http://dev.mysql.com/doc/mysql/en/news-4-1-9.html "InnoDB: Use the fcntl() file flush method on Mac OS X versions 10.3 and up. Apple had disabled fsync() in Mac OS X for internal disk drives, which caused corruption at power outages." First of all, is this accurate? A pointer to some docs or a tech note on this would be helpful. The comments about fsync() are wrong... On MacOS X, fsync() always has and always will flush all file data from host memory to the drive on which the file resides. The behavior of fsync() on MacOS X is the same as it is on every other version of Unix since the dawn of time (well, since the introduction of fsync anyway :-). I believe that what the above comment refers to is the fact that fsync() is not sufficient to guarantee that your data is on stable storage and on MacOS X we provide a fcntl(), called F_FULLFSYNC, to ask the drive to flush all buffered data to stable storage. Let me explain in more detail. With fsync() even though the OS writes the data through to the disk and the disk says "yes I wrote the data", the data is not actually on permanent storage. Unless you explicitly disable it, all disks have a write buffer which holds data you've written. The disk buffers the data you wrote until it decides to flush it to the platters (and the writes may not be in the order you wrote them). If you lose power or the system crashes before the data is written, you can wind up in a situation where only some of your data is actually on disk. What is worse is that even if you write blocks A, B and C, call fsync() and then write block D you may find after rebooting that blocks A and D are on disk but B and C are not (in fact any ordering of A, B, C, and D is possible). While this may seem like a rare case it is not. In fact if you sit down and pull the plug on a system you can make it happen in one or two plug pulls. I have even gone so far as to watch this behavior with a logic analyzer on the ATA bus: I saw the data for two writes come across the ATA cable, the drive replied and said the writes were successful and then when we rebooted the data from the second write was correct on disk but the data from the first write was not. To deal with this we introduced the F_FULLFSYNC fcntl which will ask the drive to flush all of its buffered data to disk. When an app needs to guarantee that data is on disk it should use F_FULLFSYNC. In most cases you do not need such a heavy handed operation and fsync() is good enough. But in an app like a database, it is essential if you want transactional integrity. Now, a little bit more detail: on ATA drives we implement F_FULLFSYNC with the FLUSH_TRACK_CACHE command. All drives sold by Apple will honor this command. Unfortunately quite a few firewire drive vendors disable this command and do not pass it to the drive. This means that most external firewire drives are not reliable if you lose power or the system crashes. We can't work-around that unless we ask the drive to disable the write cache completely (which hurts performance quite badly -- and even that may not be enough as some drives will ignore that request too). So in summary, I believe that the comments in the MySQL news posting are slightly confused. On MacOS X fsync() behaves the same as it does on all Unices. That's not good enough if you really care about data integrity and so we also provide the F_FULLFSYNC fcntl. As far as I know, MacOS X is the only OS to provide this feature for apps that need to truly guarantee their data is on disk. Hope this clears things up. --dominic _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/james%40jberry.us This email sent to james(a)jberry.us _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… It sounds like you're comparing apples and oranges (pardon the pun) . As Dominic pointed out, F_FULLFSYNC offers a level of synchronization that's not available from fsync on Mac OS X or other platforms. If you were to only rely on fsync, I imagine the performance between systems would be comparable. One way to mitigate this issue for large operations is to wrap your statements in a single transaction. I believe this allows sqlite to sync only after the entire operation is complete, instead of between individual statements. Thanks for a great explanation. A similar thread has come up on the sqlite list, claiming that using F_FULLFSYNC leads to dramatic performance decreases on Darwin over other platforms. Any thoughts on these issues? Finally getting my SQLite3 code working, I'm experiencing awfully slow performance when writing individual data on OS X. Linux (Ubuntu) is lightning fast, Win32 is.. tolerable, but OS X really crawls. I've tried both with built-in sources, and the new SQLite3 fink module. Same behaviour. I noticed this as well, so I profiled my call and found sync was taking forever. I removed the following fcntl call, rc = fcntl(fd, F_FULLFSYNC, 0);. Performance was back to normal. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/ kevin%40opendarwin.org This email sent to kevin(a)opendarwin.org This email sent to site_archiver(a)lists.apple.com

1 0

Re: Various tools update request
by Felix Kronlage 21 Feb '05

21 Feb '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 21, 2005, at 13:00, Alex v.Below wrote: - - -fkr - - -- 076E 1E87 3E05 1C7F B1A0 8A48 0D31 9BD3 D9AC 74D0 http://fkr.hazardous.org/ | http://opendarwin.org/~fkr/ | FKR-RIPE - -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin) iD8DBQFCGjtgDTGb09msdNARAoneAJ9DaEyHKjjIpF5+v2ZVBOEDB2677wCfSCQ/ VLuOIJ06pYak9cZu7dogjO0= =eRqb - -----END PGP SIGNATURE----- -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin) iD8DBQFCGku0DTGb09msdNARAjSwAJ9NMnOOeURrhz7bN1akxahdQZxfsgCeI6tQ pSXK+EwurO9eJ2+2yilPnWk= =zGDk -----END PGP SIGNATURE----- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… As for cvs I can say: Been there, done that (Bug ID 3720902, Status "closed"). Apple does not consider shipping cvs 1.10 a problem, so I will gladly hop on the bandwaggon here. pull src/cvs from OpenDarwin's repository. build it, install it, be happy. Another route is to use DarwinPorts, which even places it in a place that won't be override by an update. I know, it does not cure the cause of the problem, but it helps at least a bit. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Various tools update request
by Jordan Hubbard 21 Feb '05

21 Feb '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Feb 21, 2005, at 10:48 AM, Jordan Hubbard wrote: - Jordan _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… We also experimented with upgrading to GNU make 3.80, but it broke a number of things and the bang-for-buck in a single point-release update (3.79 to 3.80) didn't seem to make it worth the effort, at least at the present time, to figure out why those things broke and just who's "fault" it was (not blaming GNU make, sometimes people misuse the tools, but it still requires investigation). Just to follow up to myself.. Actually, it does appear that we've worked through those issues after all. We have so many components under our umbrella that I can't even keep track of them all sometimes. :-) This email sent to site_archiver(a)lists.apple.com

1 0