Apple

Hi List,

well, I need to thank Mark for the advice.

I've used the ERXFetchSpecificationBatchIterator in this way:

@Override
	public Object performAction() {
     // TODO Auto-generated method stub
		
     EOEditingContext ec = ERXEC.newEditingContext();
		
     EOFetchSpecification fs = new EOFetchSpecification(MyEntity.ENTITY_NAME, new TrueQualifier(), null);
		
     ERXFetchSpecificationBatchIterator iterator = new ERXFetchSpecificationBatchIterator(fs, ec, 300);
		
     totalToImport = iterator.count();
     currentImported = 0;
		
     while(iterator.hasNext()){
          NSArray<MyEntity> entities = (NSArray<MyEntity>) iterator.nextBatch();
          for (MyEntity entity : entities) {
               _ImportEntity(entity, ec);
               currentImported++;
          }
          entities = null;
          ec.dispose();
          ec = null;
          ec = ERXEC.newEditingContext();
          iterator.setEditingContext(ec);
			
          System.gc();
      }
		
		
      return _errors;
	}

This seems to work, because this procedure (plus other two that manage other 10000 objects) without ERXFetchSpecificationBatchIterator needs about 4 hours to complete.

After implementing the ERXFetchSpecificationBatchIterator I was able to to everything in less than an hour.

Thank you very much for the help!!!

Daniele

Il giorno gio 10 mar 2016 alle ore 17:27 Daniele Corti <email@hidden> ha scritto:

Hi,
I've seen the class!

Now, I'm trying to use it, when I finish the migration, I'll post the differences!

Thank You!!!!
Il giorno gio 10 mar 2016 alle ore 17:13 Mark Wardle <email@hidden> ha scritto:
Yes. I have just checked and found ERXFetchSpecificationBatchIterator in er.extensions.

Mark
On 10 Mar 2016, at 10:51, Daniele Corti <email@hidden> wrote:

Hi,
I have a question, about performance while working with LongRequest and many EO.

Here's the situation: I've prepared a class that extends ERXLongResponseTask.DefaultImplementation.
In this class, I operate a database migration, over 10000 record.
Each of them has, at least, 50/60 objects releated. From them, I create others objects, for a total of 60 new objects for each of the starting 10000 rows.

The procedure works fine, but, I experience a difference in speed during the procedure. At first, 300-400 objects are processed in few seconds, then the procedure slow down until, from 5000, it processes about 7/8 objects in 5 seconds.

The procedure is very simple:
1. Fetch all objects in a NSArray
2. Manipulate each object in a for loop, fetching related objects and creating other objects in every iteration.

I use just one EOEditingContext, so, I think the slowness begin when it became really full of elements.

I would like to know, if there is a way to "clean up" the manipulated objects from the EC, or if a different approach is preferred.

One last thing: it is good to use a NSArray of 1000 object in the for loop to do this? I ask this because, I fetch all the objects and save them in memory using EOObject.fetchAllObjects(EC) method from the class.

Thanks in advance!
Daniele C.
 _______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list      (email@hidden)
Help/Unsubscribe/Update your Subscription:

This email sent to email@hidden
References:

>Performance in many objects manipulation (From: Daniele Corti <email@hidden>)

>Re: Performance in many objects manipulation (From: Mark Wardle <email@hidden>)

>Re: Performance in many objects manipulation (From: Daniele Corti <email@hidden>)

Prev by Date: Re: (col1,col2) in ((a,b), (c,d), ...)?

Next by Date: FrontBase SQL: date formatting, „local variable“?

Previous by thread: Re: Performance in many objects manipulation

Next by thread: (col1,col2) in ((a,b), (c,d), ...)?

Index(es):

Date

Thread