Re: fetches too often (was: weird locks) in faultWithPrimaryKeyValue
Re: fetches too often (was: weird locks) in faultWithPrimaryKeyValue
- Subject: Re: fetches too often (was: weird locks) in faultWithPrimaryKeyValue
- From: René Bock via Webobjects-dev <email@hidden>
- Date: Mon, 19 Aug 2024 08:41:12 +0000
- Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=salient-doremus.de; dmarc=pass action=none header.from=salient-doremus.de; dkim=pass header.d=salient-doremus.de; arc=none
- Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OJk1hFB7grHq4IPMOij3+SR9TR7sXMyB3M37Gu1UUas=; b=G8kmQJ5YcZQJvpkCs6lkK4duMCuayOWpH00SPSPe9od+LvXdM8HnEanVb5YbUAIkzPDJdEMBJT1V50qpMJ+JYqra47PbNXqw4ZXFx4+aG5spN4rGeQiVoS0tNCFIXFaO7Ja2XiW6Ftu7iZqXFtk7kCrSqRz9vXRlIL+b/zzolwrUXA1vX5PnH154veDeXZv3WpckZ16iBsrwyP5huPF1KuVQZ7hx9qZ4QPwj7gTXn1kJSW8ayd4Kfid2Bewqa+ZHWdmPaAKXDzeG5XiEF7+9ni/eD4CQKSoTdnvCBJHK8yaAKBfC/iSteZ3gDfoljRYNJJdvFJ2i5FSXdXakFtuxzA==
- Arc-seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=bkVZcqe0a7faT7LwjKFpTBsuc88/UupzIE7C4C/tzEVnpfxz/g0qCyBwpvabNNs59Qs+4YfmbCOAB1io0Qm2A/xqnCR88or83xTOVW8jMsAFsR1bVwYuBPkuV2uH58ftcWJZBHwt+1SD7QFtlvkv9oUEu7zoxEYCbWm+5ImMy5AYKLhqLK66zFSBA/Y/hjO/VXWdPJUdRwZGIuZHiwYjkSJrfjvuTU0GfaVDGf6NTYDLkCBP1Q+j+1QjqdtVI48gQY16pnaInIdgHAH9Rgqn21ryHO50kjz13ExQtPhkBJobxTLcr2o/mnyPfTJOkyPL+9gCs7tFF1w1Q11lT/49YQ==
- Thread-topic: fetches too often (was: weird locks) in faultWithPrimaryKeyValue
Hi,
Using the same OSC for both the RR-WorkerThreads and long-response background
threads may result in deadlocks in a heavier load scenario (which is bad for
your karma)
Using en extra OSC for every background task may consume too much memory.
Fortunately there exists the combination of ERXFutureTask and ERXTask
Instances. Each ERXTask object will be connected to one OSC of a OSC pool.
The number of simultaneous ERXTask is limited to the size of OSC-pool, but
the background tasks doesn't interfere with RR-Threads.
I was very much inspired by the following screencast:
https://www.wocommunity.org/podcasts/wowodc/2011/BackgroundTasks.mov
Regards
René
Am 18.08.2024 um 15:22 schrieb Samuel Pelletier via Webobjects-dev
<email@hidden>:
OC,
I do not know your app and deployment setup but usually, I know that keeping
things simple is usually a good thing and a single instance app can be very
efficient with it's database by assuming it's cache is valid. Creating a
complete EOF stack for a single request seems very overkill for me.
EOF maintain a cache of snapshots and propagate changes, it is its main
features and differences with other ORM out there. If you can use the snapshot
cache, getting an EO from it's GID is very fast but you need this GID... EOF
shine for interactive manipulation of large data structures when you manipulate
100s of objects in a single page like managing a production schedule for
example. It is overkill for simple data manipulation like most rest like API
where basic ORM with no caching will have better performance overall.
If your app is like most app and most queries are very small and fast, do not
bother to create OSC except for known long requests, use the exceptional tool
for exceptional needs, not for the usual one. I benchmarked a ERRest app the
save and issue simple etc with a H2 local database on a Mac Book Pro few years
ago and was able to achieve more than 200 requests per second on a single
instance app.
If you have time sensitive parts distincts from more report like pages with
unpredictable fetch complexity, you may split them in different app or use a
distinct OSC for each kind of usage to maximize the snapshot cache benefits...
If you want dedicated connection for those simple requests to isolate them from
EOF, you will be better served with another ORM or by issuing direct SQL to a
pooled DB connection. Do not forget that Insert and Update in the database will
serialize access to the table if you want the latest values from your queries
even if you use multiple connections.
If your problem is creating GID and you have it's value, try this code instead
of the utility method that usually issue a fetch:
EOKeyGlobalID revisionGid =
ERXEOGlobalIDUtilities.createGlobalID(ProduitFabriqueRevision.ENTITY_NAME, new
Object[] {this.revisionId});
ProduitFabriqueRevision revision = (ProduitFabriqueRevision)
ec.faultForGlobalID(revisionGid, ec);
Regards,
Samuel
Le 18 août 2024 à 07:10, email@hidden a écrit :
Samuel,
On 17. 8. 2024, at 15:59, Samuel Pelletier
<email@hidden<mailto:email@hidden>> wrote:
If you create a new OSC, you will have no snapshot cache (I may be wrong on
this), so it is the expected behaviour.
I might be missing something, but I understand each OSC has its own set of
snapshots. Independent on other OSCs, of course, but should work normally
inside an OSC. Was I wrong?
I never create new OSC in my apps, why are you creating OSC like this ?
To create an independent DB channnel, so that my fetches in this OSC do not
need to wait for (potentially long) fetches in other OSCs.
If your fetches are long, check your database indexes.
They are fast, the problem is that they happen at all. I log EC, OSC, and SQL,
and it looks like this (they run in the same thread just by a chance; each
time, it's a new R/R loop, they get assigned the same thread just since I do
nothing else in the app at the moment):
===
18 12:58:34.815|WorkerThread2/TEMPLOG created ec EC:5e8666eb/OSC:69463522
12:58:34.816 DEBUG "DBAuction"@453059304 expression took 1 ms: SELECT ... FROM
"T_AUCTION" t0 WHERE t0."C_UID" = 1000147
//log:er.extensions.ERXAdaptorChannelDelegate.sqlLogging [WorkerThread2]
18 12:58:35.852|WorkerThread2 /TEMPLOG created ec EC:267a80b0/OSC:69463522
12:58:35.853 DEBUG "DBAuction"@453059304 expression took 1 ms: SELECT ... FROM
"T_AUCTION" t0 WHERE t0."C_UID" = 1000147
//log:er.extensions.ERXAdaptorChannelDelegate.sqlLogging [WorkerThread2]
18 12:58:36.863|WorkerThread2 /TEMPLOG created ec EC:7bcddf5b/OSC:69463522
12:58:36.880 DEBUG "DBAuction"@453059304 expression took 1 ms: SELECT ... FROM
"T_AUCTION" t0 WHERE t0."C_UID" = 1000147
//log:er.extensions.ERXAdaptorChannelDelegate.sqlLogging [WorkerThread2]
===
Note that each time a new EC is created (that's all right), but they all belong
to the same OSC (69463522). Yet, each time there's a fetch of the same object
(1000147). I must be missing something of importance; I believe there should be
only the 1st one, while instead of the latter ones the fault should get fired
quickly through snapshots of the OSC without a DB roundtrip.
Thanks,
OC
Le 17 août 2024 à 08:48, ocs--- via Webobjects-dev
<email@hidden<mailto:email@hidden>> a écrit
:
Hi there,
(I've solved the locks — were caused by a stray background thread which I've
completely forgot of; and Samuel — yes, it was related to logging. D'oh.)
Now it works all right and reliably, but I observe very strange behaviour: with
the separate OSC, the direct action is, in average, considerably slower; and
the culprit seems are DB roundtrips. I hope I am missing something completely
obvious again...
When I create my local EC for my direct action this way:
===
if (!sharedosc) sharedosc=new EOObjectStoreCoordinator()
localec=ERXEC.newEditingContext(sharedosc)
===
almost each time the fault created through this EC is accessed, there's a DB
roundtrip. On the other hand, if I create it this way (without any other change)
===
if (!sharedosc) sharedosc=EOEditingContext.defaultParentObjectStore()
localec=ERXEC.newEditingContext(sharedosc)
===
there are no roundtrips at all.
I would understand the first fetch in the separate OSC, but subsequently, it
should just use snapshots like the default root store does, should it not? What
am I missing?
Thanks,
OC
On 16. 8. 2024, at 18:14, ocs--- via Webobjects-dev
<email@hidden<mailto:email@hidden>> wrote:
Hi there,
I've got a direct action, which sometimes needs to get an object with a known
PK, for which I use faultWithPrimaryKeyValue. Works well for years, but lately
the fetches went longer, so I decided to allow it to use a separate OSC not to
clash with the normal database requests.
The result is weird:
- sometimes, faultWithPrimaryKeyValue in the dedicated OSC is lightning fast,
as presumed
- sometimes though, it never ends?!? :-O
It does not lock once and then stay locked, the cases are intermittent. Also,
it never locks when I test myself at my development machine; happens on the
deployment site only, sigh. I have also reasons to believe that the DA does not
deadlock with another thread (essentially since at the moment of the first
lock, nothing at all ran in parallel).
The code looks like this:
===
static sharedosc
WOActionResults someAction() {
try {
boolean oscpolicy=ERXProperties.booleanForKey('ActionSpecificOSC')
def localec, osc
... ...
for (... a couple of times ...) {
...
if (some-condition-which-says-I-need-to-fetch) {
if (!localec) {
if (oscpolicy && !sharedosc) sharedosc=new
ERXObjectStoreCoordinator(true)
(localec=ERXEC.newEditingContext(sharedosc?:EOEditingContext.defaultParentObjectStore())).lock()
// 1
}
log "/TEMP will fetch in $localec..." // 2
eo=EOUtilities.faultWithPrimaryKeyValue(localec ,'DBAuction',
Integer.valueOf(map.eoprimarykey))
log "/TEMP ... did fetch in $localec"
}
...
}
... ...
if (localec) localec.dispose()
} catch (exc) {
some-log-which-never-happens-thus-I-know-the-above-never-threw
}
}
===
When ActionSpecificOSC is off, it never ever locks. When it is on though,
occasionally the “will fetch” log marked // 2 is the very last thing which the
appropriate worker thread ever does. In other (intermittent) cases it all works
well.
Aside of moving the localec.dispose to finally, which would be safer, but in
this case irrelevant for no exception ever happens, can you perhaps see a
possible culprit?
Side question: originally, my // 1 line looked like
(localec=ERXEC.newEditingContext(osc)).lock(). Far as I can say, should work
precisely same way as the above, but did not: when the osc was null, I've got
an invalid EC with a null rootObjectStore. What the H.?!?
Thanks and all the best,
OC
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list
(email@hidden<mailto:email@hidden>)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden<mailto:email@hidden>
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list
(email@hidden<mailto:email@hidden>)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden