• Open Menu Close Menu
  • Apple
  • Shopping Bag
  • Apple
  • Mac
  • iPad
  • iPhone
  • Watch
  • TV
  • Music
  • Support
  • Search apple.com
  • Shopping Bag

Lists

Open Menu Close Menu
  • Terms and Conditions
  • Lists hosted on this site
  • Email the Postmaster
  • Tips for posting to public mailing lists
Re: ERIndexing (was: How to achieve a fuzzy match searcher)
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: ERIndexing (was: How to achieve a fuzzy match searcher)


  • Subject: Re: ERIndexing (was: How to achieve a fuzzy match searcher)
  • From: Asa Hardcastle <email@hidden>
  • Date: Mon, 19 Oct 2009 11:34:43 -0400

We do what David is describing, but how about this:

- one instance makes the changes and updates to the lucene index

- many instances may make changes to the databases, but use change notification to propagate updates, by doing so the lucene instance makes the appropriate updates to the lucene index.

later,

Asa


On Oct 19, 2009, at 9:44 AM, David LeBer wrote:


On 2009-10-19, at 9:23 AM, Anjo Krank wrote:

It's not a matter of thread safety, it's a matter of data in lucene being the same as in your DB. When you run with multiple instances and have heavy edits, you can easily construct a case like: p1 changes data, p2 changes data, p2 updates index, p1 updates index.

Thanks for the clarification, yeah, that wouldn't be good.

So far our usage cases have mostly involved providing searches of read-only data. In those instances Lucene has proven to be highly effective. Being able to tune the index AND tune the query parser behaviour has proven very successful. If we have the need for searching in a more mutable environment, I guess that will require some additional investigation.

When you only have one edit app, that's obviously OK (and I'm using a queue for the single app mode anyway).

Cheers, Anjo



Am 19.10.2009 um 15:06 schrieb David LeBer:


On 2009-10-19, at 8:26 AM, David LeBer wrote:


On 2009-10-19, at 8:13 AM, Mike Schrag wrote:

it's not like you wouldn't have the exact same problems in lucene-proper, though ...

We are using ERIndexing for a multiple instance single server deployment. However, the app is readonly for the indexed EOs, there is an admin app that writes, but that is only a single instance.

And based on the Lucene docs, it's writers and readers are thread and process safe, which means that multiple writers can access the same index file.


Doug Cutting has posted on the topic of thread safety a couple of times. Indexing and searching are not only thread safe, but process safe. What this means is that:

• Multiple index searchers can read the lucene index files at the same time.
• An index writer or reader can edit the lucene index files while searches are ongoing
• Multiple index writers or readers can try to edit the lucene index files at the same time (it's important for the index writer/ reader to be closed so it will release the file lock).

Not sure how well this works in practice and/or how file system dependent it is for the file system locks to function correctly.




On Oct 19, 2009, at 7:50 AM, Gustavo Pizano wrote:

NICE!, now my hopes are gone!.
so I guess I must make use de facto lucene framework. and follow the examples in LIA?


ok.. what can one do... :(

thx
G.


On Mon, Oct 19, 2009 at 1:45 PM, Anjo Krank <email@hidden> wrote:
Be aware that ERIndexing is only an experiment (and was write- only code, I don't use it yet). In particular it has several severe drawbacks:


- it doesn't really handle multiple instances (possibly) or servers (definitely). That means, for the cases where you actually *do* need the speed of lucene, ie. high-traffic, high- volume which means many servers, you can't use it as is. At least the auto-indexing won't work without some central notification point that actually does the indexing and then redistribute the indexes.

If you don't account for that, your indexes won't really match your DB, which means that you will find the wrong stuff super- fast...

I don't have a good solution to this, maybe someone who actually uses it might.

- The DB store for the indexes was an experiment to fix at least the redistribution problem, but this was truly write only, so use at your own risk.

- it duplicates your DB indexes and depending on your DB type and query, your query to resolve the faults probably won't be that much faster than a normal query would have been.

- it should really be an EO adaptor instead, which would mean that you could use it in a simple displayGroup. But then again, one of the main points in Lucene is that you don't really need a strict schema to work with it - although you'll probably have one.

Cheers, Anjo

_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list      (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden

_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list      (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden

_______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (email@hidden) Help/Unsubscribe/Update your Subscription: This email sent to email@hidden

;david

--
David LeBer
Codeferous Software
'co-def-er-ous' adj. Literally 'code-bearing'
site: 	http://codeferous.com
blog: 	http://davidleber.net
profile:	http://www.linkedin.com/in/davidleber
twitter:	http://twitter.com/rebeld
--
Toronto Area Cocoa / WebObjects developers group:
http://tacow.org




_______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (email@hidden) Help/Unsubscribe/Update your Subscription: This email sent to email@hidden

;david

--
David LeBer
Codeferous Software
'co-def-er-ous' adj. Literally 'code-bearing'
site: 	http://codeferous.com
blog: 	http://davidleber.net
profile:	http://www.linkedin.com/in/davidleber
twitter:	http://twitter.com/rebeld
--
Toronto Area Cocoa / WebObjects developers group:
http://tacow.org




_______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (email@hidden) Help/Unsubscribe/Update your Subscription: This email sent to email@hidden

_______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (email@hidden) Help/Unsubscribe/Update your Subscription: This email sent to email@hidden

;david

--
David LeBer
Codeferous Software
'co-def-er-ous' adj. Literally 'code-bearing'
site: 	http://codeferous.com
blog: 	http://davidleber.net
profile:	http://www.linkedin.com/in/davidleber
twitter:	http://twitter.com/rebeld
--
Toronto Area Cocoa / WebObjects developers group:
http://tacow.org




_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
@zenn.net


This email sent to email@hidden

_______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (email@hidden) Help/Unsubscribe/Update your Subscription: This email sent to email@hidden
References: 
 >How to achieve a fuzzy match searcher (From: Gustavo Pizano <email@hidden>)
 >Re: How to achieve a fuzzy match searcher (From: Asa Hardcastle <email@hidden>)
 >Re: How to achieve a fuzzy match searcher (From: Gustavo Pizano <email@hidden>)
 >Re: How to achieve a fuzzy match searcher (From: Asa Hardcastle <email@hidden>)
 >Re: How to achieve a fuzzy match searcher (From: David LeBer <email@hidden>)
 >Re: How to achieve a fuzzy match searcher (From: Gustavo Pizano <email@hidden>)
 >Re: How to achieve a fuzzy match searcher (From: David LeBer <email@hidden>)
 >Re: How to achieve a fuzzy match searcher (From: Gustavo Pizano <email@hidden>)
 >Re: How to achieve a fuzzy match searcher (From: Asa Hardcastle <email@hidden>)
 >ERIndexing (was: How to achieve a fuzzy match searcher) (From: Anjo Krank <email@hidden>)
 >Re: ERIndexing (was: How to achieve a fuzzy match searcher) (From: Gustavo Pizano <email@hidden>)
 >Re: ERIndexing (was: How to achieve a fuzzy match searcher) (From: Mike Schrag <email@hidden>)
 >Re: ERIndexing (was: How to achieve a fuzzy match searcher) (From: David LeBer <email@hidden>)
 >Re: ERIndexing (was: How to achieve a fuzzy match searcher) (From: David LeBer <email@hidden>)
 >Re: ERIndexing (was: How to achieve a fuzzy match searcher) (From: Anjo Krank <email@hidden>)
 >Re: ERIndexing (was: How to achieve a fuzzy match searcher) (From: David LeBer <email@hidden>)

  • Prev by Date: Ternary op + autoboxing + null value = NPE?
  • Next by Date: Re: Ternary op + autoboxing + null value = NPE?
  • Previous by thread: Re: ERIndexing (was: How to achieve a fuzzy match searcher)
  • Next by thread: Re: ERIndexing (was: How to achieve a fuzzy match searcher)
  • Index(es):
    • Date
    • Thread