Uploaded image for project: 'CMS'
  1. CMS
  2. CMS-10265

Make the Solr socket timeout parameterizable

XMLWordPrintable

    • Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Critical Critical
    • 4.3.0
    • 4.3.0
    • Plugin : Search
    • None
    • 4.3 RC1

      Some projects have a lot of data and Solr indexation can be very long (on optimize for example).

      By default, the socket timeout is set to 2 minutes (maybe it depends on the system configuration).

      If we have the following error if the timeout is exceeded:

      2020-02-18 00:31:58,335 ERROR [org.ametys.core.schedule.AmetysJob$org.ametys.cms.indexing.GlobalWorkspaceIndexerSchedulable] (AmetysRuntimeScheduler_Worker-4;) An exception occured during the execution of the Schedulable 'org.ametys.cms.indexing.GlobalWorkspaceIndexerSchedulable'
org.ametys.cms.indexing.IndexingException: Error indexing the workspace 'default'.
	at org.ametys.cms.indexing.solr.SolrWorkspaceIndexer.doIndex(SolrWorkspaceIndexer.java:226)
	at org.ametys.cms.indexing.solr.SolrWorkspaceIndexer._forceWorkspaceAndDoIndex(SolrWorkspaceIndexer.java:185)
	at org.ametys.cms.indexing.solr.SolrWorkspaceIndexer._index(SolrWorkspaceIndexer.java:155)
	at org.ametys.cms.indexing.solr.SolrWorkspaceIndexer.indexAllWorkspaces(SolrWorkspaceIndexer.java:111)
	at org.ametys.cms.indexing.GlobalWorkspaceIndexerSchedulable.execute(GlobalWorkspaceIndexerSchedulable.java:43)
	at org.ametys.core.schedule.AmetysJob.execute(AmetysJob.java:151)
	at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
	at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occurred while waiting response from server at: http://localhost:8983/solr
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:676)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:265)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:497)
	at org.ametys.cms.search.solr.NoAutoCommitUpdateClient.request(NoAutoCommitUpdateClient.java:52)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
	at org.apache.solr.client.solrj.SolrClient.optimize(SolrClient.java:675)
	at org.ametys.cms.search.solr.AbstractAmetysConcurrentUpdateClient.optimize(AbstractAmetysConcurrentUpdateClient.java:325)
	at org.apache.solr.client.solrj.SolrClient.optimize(SolrClient.java:595)
	at org.ametys.cms.search.solr.AbstractAmetysConcurrentUpdateClient.optimize(AbstractAmetysConcurrentUpdateClient.java:297)
	at org.ametys.cms.content.indexing.solr.SolrIndexer.optimize(SolrIndexer.java:2010)
	at org.ametys.cms.indexing.solr.SolrWorkspaceIndexer.doIndex(SolrWorkspaceIndexer.java:221)
	... 7 more
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.base/java.net.SocketInputStream.socketRead0(Native Method)
	at java.base/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
	at java.base/java.net.SocketInputStream.read(SocketInputStream.java:168)
	at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
	at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
	at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
	at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
	at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
	at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:564)
	... 18 more

      The solution could be to add a configuration parameter to set the socket timeout duration.

      For that we have to modify two calls of SolrClient:

      • DefaultSolrClientProvider#initialize()
        _solrReadClient = new HttpSolrClient.Builder(url).build(); become _solrReadClient = new HttpSolrClient.Builder(url).withSocketTimeout(timeout).build();
      • AbstractAmetysConcurrentUpdateClient#constructor
        Add .withSocketTimeout(timeout) to the Builder while calling the super constructor.
        But for this solution, it can't work for now because there is the following issue: SOLR-12550.

            sprieul Simon Prieul (Inactive)
            bmaurel Bérénice Maurel
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: