1

I have got python code running on multiple analyzer machines each picking documents from solr(select operations) and modifying data in solr by re-submitting the documents with the updated fields from DB(in case of update/Insert). But since different solr instances on different machines have their own updated documents,this is leading to data inconsistency across the machines.

Is there any way i can keep a central solr document repository which will be queried and updated by different machines,thereby ensuring data consistency?

2 Answers 2

3

Solr forums would provide multiple threads on Concurrent Solr add/updates which would give you a clear picture.

You can maintain a single instance of Solr and have multiple clients commit into it.
Solr is not transactional like an RDBMS, but it does handle concurrency.
Whenever is commit is made a a lock is maintained so that others can't commit and are queued.
A commit can commit all pending commits as well.

1
  • "Single document atomicity" is the term of the day :-) If I recall right, Mongo also implements it this way. Mar 1, 2012 at 6:52
0

You are doing this the wrong way.

SOLR is perfectly capable of running with a single master server which gets all updates, and many replica servers which serve all search queries. That way all servers are identical as long as you don't have too many replicas or the network bandwidth is not constrained for any of the replicas.

You would still have your update processes but they would only update the core(s) on the master server. The replica servers will get their updates automatically via SOLR's replica capability.

Start off by reading the SOLR wiki page on replication.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.