We have faced timed out exception while re-indexing a large index ( size more than 1 GB) in Sitecore 8.1. We have used “AutoFacIntegration” to register the Solr cores . However this issue exist with all the IoC library (Castle.Windsor,StructureMap,Unity,Autofac etc)
we got below Exception in Indexing popup :
ManagedPoolThread #95 14:23:36 ERROR Exception Exception: System.Reflection.TargetInvocationException Message: Exception has been thrown by the target of an invocation. Source: mscorlib at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor) at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments) at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) at Sitecore.Reflection.ReflectionUtil.InvokeMethod(MethodInfo method, Object[] parameters, Object obj) at Sitecore.Jobs.JobRunner.RunMethod(JobArgs args) at (Object , Object[] ) at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args) at Sitecore.Jobs.Job.ThreadEntry(Object state) Nested Exception Exception: SolrNet.Exceptions.SolrConnectionException Message: The operation has timed out Source: SolrNet at SolrNet.Impl.SolrConnection.PostStream(String relativeUrl, String contentType, Stream content, IEnumerable`1 parameters) at SolrNet.Impl.SolrConnection.Post(String relativeUrl, String s) at SolrNet.Commands.OptimizeCommand.Execute(ISolrConnection connection) at SolrNet.Impl.SolrBasicServer`1.SendAndParseHeader(ISolrCommand cmd) at Sitecore.ContentSearch.SolrProvider.SolrSearchIndex.PerformRebuild(Boolean resetIndex, Boolean optimizeOnComplete, IndexingOptions indexingOptions, CancellationToken cancellationToken) at Sitecore.ContentSearch.SolrProvider.SolrSearchIndex.Rebuild(Boolean resetIndex, Boolean optimizeOnComplete) Nested Exception Exception: System.Net.WebException Message: The operation has timed out Source: System at System.Net.HttpWebRequest.GetResponse() at HttpWebAdapters.Adapters.HttpWebRequestAdapter.GetResponse() at SolrNet.Impl.SolrConnection.GetResponse(IHttpWebRequest request) at SolrNet.Impl.SolrConnection.PostStream(String relativeUrl, String contentType, Stream content, IEnumerable`1 parameters)
Root Cause:
Generally this issue happen when Sitecore triggers the “OptimizeCommand” to Optimize the indexes in Solr. Sitecore waits for the solr response till the Solrconnection timeout which is by default 10 seconds and if solr does not respond to sitecore with Optimize success response within the timeout limit then sitecore throws the timeout exception. However index rebuild would be completed and it gets reflected in solr also.
Now here the next question comes that why Solr does not responded to sitecore within the default timeout limit. Before answering this question i would like to brief about the Index optimization in solr.
Solr optimization reorganize the index into segments (increasing search speed) and remove any deleted (replaced) documents. Solr is a read only data store so every time you index a document it will mark the old document as deleted and then create a brand new document to replace the deleted one. Optimize will remove these deleted documents and also removes the empty space inside the document. During the optimization solr loads whole index in JVM and perform the optimization and Optimizing the index is very expensive which may take more time based on the size of index.
Sitecore Internally triggers the Index optimization command after the successful execution of Index Rebuild.
Now the answer of above question, Solr might take longer time(more than default solrconnection timeout) to optimize the large index and in this situation sitecore throws the exception as above. we didn’t get this issue earlier because during initial phase of our project, the size of indexes were small and it got gradually increased after sometime.
How to Resolve this issue:
To resolve this issue we need to increase the Solr connection timeout from default to some higher limit like 300000 milliseconds or based on the requirement. Ideally connection timeout should be configurable. We did some analysis and found that it’s a Sitecore bug. We raised the helpdesk ticket to sitecore support and they came back with Sitecore patch to solve this issue. Sitecore has already fixed this bug in the Sitecore Experience Platform 8.2 (rev. 160729).
So if you are using any lower version of sitecore than 8.2 (rev. 160729) and any IOC to support Solr then please reach out to Sitecore for Patch.
