August 2006 - Darwin-dev - lists.apple.com

Re: LWP::UserAgent Missing from Darwin Perl/5.8.6?
by Martin Costabel 29 Aug '06

29 Aug '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com User-agent: Thunderbird 1.5.0.5 (Macintosh/20060719) Stewart.Dickson(a)noaa.gov wrote: Can't locate LWP/UserAgent.pm in @INC (@INC contains: /sw/lib/perl5 /sw/lib/perl5/darwin /System/Library/Perl/5.8.6/darwin-thread-multi-2level /System/Library/Perl/5.8.6 /Library/Perl/5.8.6/darwin-thread-multi-2level /Library/Perl/5.8.6 /Library/Perl /Network/Library/Perl/5.8.6/darwin-thread-multi-2level /Network/Library/Perl/5.8.6 /Network/Library/Perl /System/Library/Perl/Extras/5.8.6/darwin-thread-multi-2level /System/Library/Perl/Extras/5.8.6 /Library/Perl/5.8.1 .) [] Since you have Fink installed, you can say fink install libwww-pm586 -- Martin _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… And how does one remedy this? This email sent to site_archiver(a)lists.apple.com

1 0

LWP::UserAgent Missing from Darwin Perl/5.8.6?
by Stewart.Dickson＠noaa.gov 29 Aug '06

29 Aug '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Priority: normal Hello, On Darwin Kernel Version 8.7.0 ppc970 (PowerPC 970) #! /usr/bin/perl use LWP::UserAgent; Returns: Can't locate LWP/UserAgent.pm in @INC (@INC contains: /sw/lib/perl5 /sw/lib/perl5/darwin /System/Library/Perl/5.8.6/darwin-thread-multi-2level /System/Library/Perl/5.8.6 /Library/Perl/5.8.6/darwin-thread-multi-2level /Library/Perl/5.8.6 /Library/Perl /Network/Library/Perl/5.8.6/darwin-thread-multi-2level /Network/Library/Perl/5.8.6 /Network/Library/Perl /System/Library/Perl/Extras/5.8.6/darwin-thread-multi-2level /System/Library/Perl/Extras/5.8.6 /Library/Perl/5.8.1 .) On-Line documentation Perl Diver 2.33 seems to indicate that LWP/UserAgent.pm is standard equipment for perl5/5.6.0/i386-linux Why isn't it standard for Darwin(FreeBSD's) Perl5.8.6? And how does one remedy this? Thanks, -Stewart Dickson, Principal Scientific Subject Matter Expert in Visualization, NOAA National Climatic Data Center _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: SQLite optimization
by Aaron Burghardt 29 Aug '06

29 Aug '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Aug 29, 2006, at 9:02 AM, Ruslan Zasukhin wrote: On 8/29/06 3:50 PM, "Aaron Burghardt" <aburgh(a)mac.com> wrote: You don't say whether you are querying the database or inserting data. SQLite does not open the db file for each SQL statement, but when inserting data it creates a separate journal file for each transaction, so there is a fair amount of overhead per insert transaction. Actually any dbms with transactions do this. Regards, Aaron _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… DB need to have for each transaction separate *logical* file. It can be inside of some "virtual volume" but this not change main point. The main point is performance and what SQLite is capable of with regards to the OP's needs, is it not? SQLite performance improves substantially when multiple inserts are performed within a single transaction because of the overhead of setting up the transaction. Another db may be necessary to achieve what he needs, but frankly I don't think enough detail is known at this point to make that judgement. This email sent to site_archiver(a)lists.apple.com

1 0

Re: SQLite optimization
by Ruslan Zasukhin 29 Aug '06

29 Aug '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Thread-index: AcbLa3O3soMVGjdeEdu3SQANkzDgag== Thread-topic: SQLite optimization User-agent: Microsoft-Entourage/11.2.0.050811 On 8/29/06 3:50 PM, "Aaron Burghardt" <aburgh(a)mac.com> wrote: > You don't say whether you are querying the database or inserting > data. SQLite does not open the db file for each SQL statement, but > when inserting data it creates a separate journal file for each > transaction, so there is a fair amount of overhead per insert > transaction. Actually any dbms with transactions do this. DB need to have for each transaction separate *logical* file. It can be inside of some "virtual volume" but this not change main point. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: SQLite optimization
by Aaron Burghardt 29 Aug '06

29 Aug '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Aug 29, 2006, at 1:16 AM, Ruslan Zasukhin wrote: On 8/28/06 1:48 PM, "Mark Gilbert" <mark(a)gallery.co.uk> wrote: Hi Mark, Hi There. My app accesses the same SQlite database from several threads, and in order to accommodate this I currently open and close the database on each access, and the various threads will wait when the database is busy on another thread. This all works OK, but I am finding performance issues, which I think may be related to the opening and closing. My idea is to limit access to the database to a single intermediate thread, with each other thread passing their requests through the 'owner' thread for the SQLite database. My hope is that this thread can leave the database permanently open with a large cache size and consequently speed things up. Would anyone with experience in SQLite performance care to comment if this is a good strategy to improve performance ? ---- Aaron Burghardt aburgh(a)mac.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… In fact SQL Lite self do open/close of db file for each transaction. This was idea of its author. Many solutions in SQL Lite design rely on this idea, and this is why it is underlined that "SQLLite is single user engine". You don't say whether you are querying the database or inserting data. SQLite does not open the db file for each SQL statement, but when inserting data it creates a separate journal file for each transaction, so there is a fair amount of overhead per insert transaction. As another reply stated, you can keep several open connections (and they can be from separate threads) to the db and let the SQLite take care of synchronization. I suggest you try keeping your connections open and batching inserts in a transaction, if you are doing a lot of inserts. This email sent to site_archiver(a)lists.apple.com

1 0

Re: SQLite optimization
by Ruslan Zasukhin 29 Aug '06

29 Aug '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Thread-index: AcbLKkOcgjPJ9DcdEduibwANkzDgag== Thread-topic: SQLite optimization User-agent: Microsoft-Entourage/11.2.0.050811 On 8/28/06 1:48 PM, "Mark Gilbert" <mark(a)gallery.co.uk> wrote: Hi Mark, > Hi There. > > My app accesses the same SQlite database from several threads, and in > order to accommodate this I currently open and close the database on > each access, and the various threads will wait when the database is > busy on another thread. > > This all works OK, but I am finding performance issues, which I think > may be related to the opening and closing. > > My idea is to limit access to the database to a single intermediate > thread, with each other thread passing their requests through the > 'owner' thread for the SQLite database. My hope is that this thread > can leave the database permanently open with a large cache size and > consequently speed things up. In fact SQL Lite self do open/close of db file for each transaction. This was idea of its author. Many solutions in SQL Lite design rely on this idea, and this is why it is underlined that "SQLLite is single user engine". This idea in the same time means that you cannot have normal db cache, because each time close forget everything. At least this is my understanding of how it works. > Would anyone with experience in SQLite performance care to comment if > this is a good strategy to improve performance ? > Currently an open/access/close cycle is taking around 250mSecs, > even though its a tiny database and fairly simple SQLite requests. I > may need to do several database accesses per second, so currently its > all stacking up. > > Any comments ? If you really want to get much better performance, I will recommend to try our Valentina database. http://www.paradigmasoft.com I predict that you will get speedup at least 10-30, sometimes 100+ times, and you will be able work with dbs in even millions records easy. Read e.g. http://www.paradigmasoft.com/en/testimonials Or just few days ago comment from our old C/C++ user: ---------------------------------------------------------------- On 8/25/06 1:23 AM, "Rosyna" <rosyna(a)unsanity.com> wrote: >>>> Is there a universal version of the Valentina C SDK? Either the 1.x >>>> version (preferred) or the 2.x versions. Specifically I am looking >>>> for a universal version of libVCSDK_Macho.dylib. >>> >>> C SDK is not ported yet to 2.x product line. >> >> It is more easy port C SDK 1.x to universal I think. > > If that is possible... The code that uses it is all C. It was written > to use both Valentina 1.x C SDK and SQLite to test the speed of each. > Of course, Valentina was 10-30x faster for what I was using it for. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: SQLite optimization
by Kevin Van Vechten 28 Aug '06

28 Aug '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com <http://www.sqlite.org/faq.html#q8> says: (8) Is SQLite threadsafe? - Kevin _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Aug 28, 2006, at 3:48 AM, Mark Gilbert wrote: My app accesses the same SQlite database from several threads, and in order to accommodate this I currently open and close the database on each access, and the various threads will wait when the database is busy on another thread. First, I'd recommend asking SQLite-specific questions on the relevant SQLite mailing list(s) at <http://www.sqlite.org/> -- that's where you'll find the experts. Yes. Sometimes. In order to be thread-safe, SQLite must be compiled with the THREADSAFE preprocessor macro set to 1. In the default distribution, the windows binaries are compiled to be threadsafe but the linux binaries are not. If you want to change this, you'll have to recompile. "Threadsafe" in the previous paragraph means that two or more threads can run SQLite at the same time on different "sqlite3" structures returned from separate calls to sqlite3_open(). It is never safe to use the same sqlite3 structure pointer in two or more threads. So the relevant question for this list is, "Was SQLite compiled threadsafe?" Taking a look at <http://www.opensource.apple.com/ darwinsource/10.4.7.ppc/SQLite-28/Makefile> shows "--enable- threadsafe" is used in the configuration options -- so I believe it is. Therefore, you don't need to repeatedly call open and close -- you just need to have one open per thread and SQLite will take care of the rest of the synchronization. This email sent to site_archiver(a)lists.apple.com

1 0

Re: MACFORGE request for a project
by Jordan K. Hubbard 28 Aug '06

28 Aug '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: MACFORGE request for a project
by Kevin Van Vechten 28 Aug '06

28 Aug '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com i would like to open a " K&R removal " project ( Source Clean up ) aka Normalisé - Kevin _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Aug 28, 2006, at 5:34 AM, almisr wrote: where/who ? Project requests should be directed to admin(a)macosforge.org. (Nitpick: it's Mac OS Forge; MacForge is something entirely different.) I realize there's presently no clear "mission statement" for Mac OS Forge. We hope to post something soon, but it's something we're still working out ourselves. However, we do feel strongly that all projects hosted there must "live there." Forking an open source project is not something that should be done lightly. Unless Mac OS Forge is the authoritative source for a project, it's likely not going to be a good fit. In this spirit of not forking open source projects, Apple is actively trying to eliminate the divergences in our sources from the "upstream" sources (in this case, FreeBSD). If you'd like to see "K&R removal" or other source normalization in Mac OS X, I strongly recommend contributing those changes to the FreeBSD project directly. Apple will then likely pick up those changes in a future synch. The alternative of a Mac OS Forge project for this purpose would add another layer of indirection in getting the patches accepted, and increase code divergence in the meantime. This email sent to site_archiver(a)lists.apple.com

1 0

Re: MACFORGE request for a project
by almisr 28 Aug '06

28 Aug '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Cheers, M sorry but the base BSD is not secret ... it's not the subject of my project request and i 'm very happy we don't work together ... I just used it to try to stay polished second : there is no secret in dev , just work I am afraid you are incorrect. yep i know , and i know well xnu we could find a serious way it's like APCI , folks cry ... and cry 1 - the standard is "open" 2 - FreeBSD project has implemented this standard 3 - Linux project has implemented this standard then it seems there is something wrong ... it's not that i ask : it's make a "parallel" work and i understand well the Apple view it 's a problem of confidence and quality " Rome ne s'est pas faite en un jour " _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I apologize if I angered you in some way but I don't really understand what you said- do you use some web translation tool? Perhaps you didn't understand what I said either. To be clear, Apple claims that the Darwin project is open source project when, in fact, the processes behind the development of the project are absolute secrets. Therefore, to make any contribution to the Darwin project, you are one of the following: 1) an Apple employee 2) contributing a small, self-contained patch 2) someone that can anticipate Apple's next move before they make it, thereby saving Apple development time and money BSD and Darwin are not the same thing - I am sure you know that. If you're talking about contributing to any of the BSDs, that is fine. However, there is no easy or traditional way for normal people to contribute changes to Darwin (and thus Mac OS X). If you are proposing to create such a way, I wish you the best of luck. The OpenDarwin project tried to do just that and failed after six years, even when numerous people from Apple itself were involved. i'm sorry but opendarwin just tried with students and a few few real devs and it was the problem before building a port system try to improve the base system , drivers , sound layer ... with a serious opensource foundation , It is a true statement to say that things related to Darwin itself and open source projects at Apple are secret. This does not allow others to contribute changes, because they do not have access to the details of the development in a collaborative way. Best Regards {|} This email sent to site_archiver(a)lists.apple.com

1 0