0

Replicas DOWN, all on the same SolrCloud node. It is a two node cluster (but there are three zookeeper instances). The problem happens during indexing operation. The down replicas are all on the same solrcloud node, host2. I can not find informations about the error raised. THer error raised is the following:

ERROR - 2015-08-13 02:57:34.261; org.apache.solr.common.SolrException; 
    forwarding update to http://host1:8080/solr/collection_V_2_0_shard1_replica1/ failed - 
    retrying ... retries: 1 add{,id=RFQ||304707||188627||1} params:update.distrib=TOLEADER&distrib.from=http%3A%2F%2Fhost2%3A8080%2Fsolr%2Fak_collection_V_2_0_shard1_replica2%2F 
    rsp:503:org.apache.solr.common.SolrException: Service Unavailable

After the error, the collection_V_2_0_shard1_replica2 was down. Can someone help me to find the cause of this error?

Regards Giova

3
  • can you please explain little more about your number of shards and replica? What do mean by all down on the same SolrCloud? Aug 14, 2015 at 13:54
  • @abhishek bafna: we have a two node cluster. Each collection has two shard. Each shard is mirrored on each node. Some shard are down and are down on only one node. In other words: some shard goes down on only one node. On the other node all is ok! Thank you!
    – Giova
    Aug 14, 2015 at 14:22
  • What is the state of zookeeper instances? Only partial shards are available in your case. Aug 14, 2015 at 14:29

1 Answer 1

0

I guess the SOLR node which hosted collection_V_2_0_shard1_replica2 has gone down for some reason and the error in the log is not about why node went down, its about the replica trying to route a doc to a shard1 leader and seems leader is down already.

Some debugging tips

  1. Log on to Node which hosted shard1
  2. Scan the log files and see why the node went down, time to look for is before the routing time stamp(Say Secs/Mins before 2015-08-13 02:57:34.261)
  3. Did you override Java Heap..? Defalut is just 512KB, you can change to some thing like 4g in solr.in.sh/cmd and restart the solr
  4. Was ZooKeepers up that time..? Cluster will sustain even if the zooKeeper is down as SOLR itself has the cluster state json. But it will not be able to take the writes.
2
  • thank you. Solr node which hosted collection_V_2_0_shard1_replica2 has not gone down as some replicas was up and running.
    – Giova
    Aug 18, 2015 at 9:21
  • Are you not able to get any pointers from the log file which hosted shard1 replica2...? Aug 18, 2015 at 10:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.