• Open Menu Close Menu
  • Apple
  • Shopping Bag
  • Apple
  • Mac
  • iPad
  • iPhone
  • Watch
  • TV
  • Music
  • Support
  • Search apple.com
  • Shopping Bag

Lists

Open Menu Close Menu
  • Terms and Conditions
  • Lists hosted on this site
  • Email the Postmaster
  • Tips for posting to public mailing lists
Re: I'm killing my database
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: I'm killing my database


  • Subject: Re: I'm killing my database
  • From: Chuck Hill <email@hidden>
  • Date: Thu, 26 Apr 2007 10:41:37 -0700


On Apr 26, 2007, at 8:19 AM, Scott Winn wrote:

When I have put a lot of data through the Importer previously I had the application crash with a SIGBUS error.

I have only seen that happen in three situations (there are probably more, but this will give you a good idea):
1. Bug in JVM
2. Faulty hardware
3. Crash in native code (QT Java in my case)


So, it is probably not a bug in your code.


On a separate occasion, the app didn't crash, but when I went to log out I got "Internal Server Error has occurred". Looking at the run log showed three null pointer exceptions encountered; trying to dispose the editing context from the look of them (I should have saved those exception messages). Both of these problams happened in previous tests before the DB crash.

That does look like a bug in your code. I would suspect your cleanup of the WOLongResponePage.



I'm sure both of those incidents should raise warning flags, but I'm not sure what those flags mean. Do they point to the same problem? I found basically no information on SIGBUS errors. I can run everything through from the top and see if I have the same issues, but I think I'll try to get performance boosted a bit first. I will look into batch faulting and disposing the editing context at regular intervals.

I should mention that all of this is happening on a WOLongResponse page, so a user can watch the import progress and fix any errors in the source files.

That should be fine, but you have to think twice about when you clean up stuff when you start spinning off extra threads.


Chuck



On Apr 25, 2007, at 5:44 PM, Chuck Hill wrote:

On Apr 25, 2007, at 12:03 PM, Scott Winn wrote:

Thanks Chuck.  I will give it a look.

I still don't know what the problem was, but I have updated FrontBase and its plugin (I was a few point releases behind). The release notes for some of those points included delete fixes. I also added EC.undoManager().removeAllActions(). I haven't disposed of the editing context yet, but I may once I have checked the performance results from my most recent run.

I put the same data back through the importer and no DB crash this time,

That is from the FB update.


although things still seemed very slow. More than 1/2 an hour to process about 1 MB of data seem like a bit much. I checked free memory just prior to doing the delete and it never dips below 74%.

I have no doubt that it can be optimized more than that. Make sure you have batch faulting configured in the model.


Chuck


On Apr 24, 2007, at 10:32 PM, Guido Neitzer wrote:

On 24.04.2007, at 20:59, Chuck Hill wrote:

The first thing I would do is to add this line:
recordContext .undoManager().removeAllActions();

The next thing I would do (voodoo or not) is to periodically dispose of recordContext and create a replacement. In the past that was vital.

It still is. It helps. Not necessarily for performance but normally for memory usage. I normally do bulk import in batches of up to 500 objects per batch, then save and dispose the editingContext, set it to null and create in new one in the next batch. Also for bulk updating of EOs.


That works well for my importers.

If you need to resolve relationships during import, pre- fetching the relationships can make a huge difference.

As as you are deleting, I have a DeletePrefetcher class that may be of use. I originally wrote this as a bug workaround, but it might also make the fetches more efficient. It is part of our GVCEOFExtensions framework that you can get from http:// sourceforge.net/projects/gvcsitemaker


To save you the download, I have temporarily stuck a copy here:

http://www.gvcsitemaker.com/chill/files/deleteprefetcher.java

Chuck

--

Practical WebObjects - for developers who want to increase their overall knowledge of WebObjects or who are trying to solve specific problems.
http://www.global-village.net/products/practical_webobjects

_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
40global-village.net


This email sent to email@hidden

--

Practical WebObjects - for developers who want to increase their overall knowledge of WebObjects or who are trying to solve specific problems.
http://www.global-village.net/products/practical_webobjects










--

Practical WebObjects - for developers who want to increase their overall knowledge of WebObjects or who are trying to solve specific problems.
http://www.global-village.net/products/practical_webobjects






_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list      (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden


  • Follow-Ups:
    • Re: I'm killing my database
      • From: Scott Winn <email@hidden>
    • Re: I'm killing my database
      • From: Pascal Robert <email@hidden>
References: 
 >Re: I'm killing my database (From: Scott Winn <email@hidden>)
 >Re: I'm killing my database (From: Chuck Hill <email@hidden>)
 >Re: I'm killing my database (From: Scott Winn <email@hidden>)

  • Prev by Date: Re: Un-Worder-ized Ajax framework
  • Next by Date: Re: problem with delete rule (relationship)
  • Previous by thread: Re: I'm killing my database
  • Next by thread: Re: I'm killing my database
  • Index(es):
    • Date
    • Thread