Re: ERIndexing (was: How to achieve a fuzzy match searcher)
Re: ERIndexing (was: How to achieve a fuzzy match searcher)
- Subject: Re: ERIndexing (was: How to achieve a fuzzy match searcher)
- From: Johnny Miller <email@hidden>
- Date: Tue, 20 Oct 2009 09:58:42 -1000
Hi Anjo,
I should qualify that I'm not using ERIndexing but my own homegrown
version of a Lucene index. In my solution I have EOs that update
their index when their state transition changes i.e.
awakeFromInsertion, didDelete etc...
After reading your post I became nervous that I might have overlooked
something. Since Lucene doesn't "update" objects in the index (you
can only remove and add them) if users are working on different data
sets how could they make the index of out sync with the database?
I can see a problem if they are working on the same data. Would you
recommend a different place to implement the index updates then in the
EO state transitions?
best,
Johnny
On Oct 19, 2009, at 6:58 PM, Anjo Krank wrote:
The changes go into two queues, so there is no telling which order
they get in two different instances.
Cheers, Anjo
Am 19.10.2009 um 22:46 schrieb Johnny Miller:
Hi Anjo,
I kind of understand what you are saying but I have a couple of
questions.
Wouldn't p1 and p2 have to be editing the same EO for the index to
become out of sync with the database?
If you have two users editing the same EO wouldn't the normal WO
mechanisms resolve the issue?
Thanks for your help,
Johnny
On Oct 19, 2009, at 3:23 AM, Anjo Krank wrote:
It's not a matter of thread safety, it's a matter of data in
lucene being the same as in your DB. When you run with multiple
instances and have heavy edits, you can easily construct a case
like: p1 changes data, p2 changes data, p2 updates index, p1
updates index.
When you only have one edit app, that's obviously OK (and I'm
using a queue for the single app mode anyway).
Cheers, Anjo
Am 19.10.2009 um 15:06 schrieb David LeBer:
On 2009-10-19, at 8:26 AM, David LeBer wrote:
On 2009-10-19, at 8:13 AM, Mike Schrag wrote:
it's not like you wouldn't have the exact same problems in
lucene-proper, though ...
We are using ERIndexing for a multiple instance single server
deployment. However, the app is readonly for the indexed EOs,
there is an admin app that writes, but that is only a single
instance.
And based on the Lucene docs, it's writers and readers are thread
and process safe, which means that multiple writers can access
the same index file.
Doug Cutting has posted on the topic of thread safety a couple
of times. Indexing and searching are not only thread safe, but
process safe. What this means is that:
• Multiple index searchers can read the lucene index files at
the same time.
• An index writer or reader can edit the lucene index files
while searches are ongoing
• Multiple index writers or readers can try to edit the lucene
index files at the same time (it's important for the index
writer/reader to be closed so it will release the file lock).
Not sure how well this works in practice and/or how file system
dependent it is for the file system locks to function correctly.
On Oct 19, 2009, at 7:50 AM, Gustavo Pizano wrote:
NICE!, now my hopes are gone!.
so I guess I must make use de facto lucene framework. and
follow the examples in LIA?
ok.. what can one do... :(
thx
G.
On Mon, Oct 19, 2009 at 1:45 PM, Anjo Krank <email@hidden>
wrote:
Be aware that ERIndexing is only an experiment (and was write-
only code, I don't use it yet). In particular it has several
severe drawbacks:
- it doesn't really handle multiple instances (possibly) or
servers (definitely). That means, for the cases where you
actually *do* need the speed of lucene, ie. high-traffic, high-
volume which means many servers, you can't use it as is. At
least the auto-indexing won't work without some central
notification point that actually does the indexing and then
redistribute the indexes.
If you don't account for that, your indexes won't really match
your DB, which means that you will find the wrong stuff super-
fast...
I don't have a good solution to this, maybe someone who
actually uses it might.
- The DB store for the indexes was an experiment to fix at
least the redistribution problem, but this was truly write
only, so use at your own risk.
- it duplicates your DB indexes and depending on your DB type
and query, your query to resolve the faults probably won't be
that much faster than a normal query would have been.
- it should really be an EO adaptor instead, which would mean
that you could use it in a simple displayGroup. But then
again, one of the main points in Lucene is that you don't
really need a strict schema to work with it - although you'll
probably have one.
Cheers, Anjo
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (Webobjects-
email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (Webobjects-
email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden
;david
--
David LeBer
Codeferous Software
'co-def-er-ous' adj. Literally 'code-bearing'
site: http://codeferous.com
blog: http://davidleber.net
profile: http://www.linkedin.com/in/davidleber
twitter: http://twitter.com/rebeld
--
Toronto Area Cocoa / WebObjects developers group:
http://tacow.org
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden
;david
--
David LeBer
Codeferous Software
'co-def-er-ous' adj. Literally 'code-bearing'
site: http://codeferous.com
blog: http://davidleber.net
profile: http://www.linkedin.com/in/davidleber
twitter: http://twitter.com/rebeld
--
Toronto Area Cocoa / WebObjects developers group:
http://tacow.org
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden
Johnny Miller
Kahalawai Media Corp
w: www.kahalawai.com
e: email@hidden
p: 808.661.7962
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden
Johnny Miller
Kahalawai Media Corp
w: www.kahalawai.com
e: email@hidden
p: 808.661.7962
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden