Uploaded image for project: 'CMS'
  1. CMS
  2. CMS-8346

String indexation : there is a limit of characters in Solr side

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • 4.0RC6
    • 4.0RC5
    • Plugin : Search
    • None

      Handle it, by logging a warning, and truncate the sent string

      2017-06-08 09:36:23.834 ERROR (qtp1348949648-17) [   x:ametys-default] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Exception writing document id content://c71302b9-8a78-43e8-a50a-78caa5978b87 to the index; possible analysis error: Document contains at least one immense term in field="title" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped.  Please correct the analyzer to not produce such terms.  The prefix of the first immense term is: '[76, 86, 49, 32, 97, 112, 112, 114, 111, 102, 111, 110, 100, 105, 101, 32, 40, 49, 114, 101, 32, 101, 116, 32, 84, 108, 101, 32, 76, 41]...', original message: bytes can be at most 32766 in length; got 47260. Perhaps the document has an indexed string field (solr.StrField) which is too large
      	at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:174)
      	at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:68)
      	at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
      	at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:936)
      	at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1091)
      	at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:714)
      	at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:103)
      	at org.apache.solr.handler.loader.JavabinLoader$1.update(JavabinLoader.java:97)
      	at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:179)
      	at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readIterator(JavaBinUpdateRequestCodec.java:135)
      	at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:260)
      	at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readNamedList(JavaBinUpdateRequestCodec.java:121)
      	at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:225)
      	at org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:145)
      	at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.unmarshal(JavaBinUpdateRequestCodec.java:186)
      	at org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:107)
      	at org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:54)
      	at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:94)
      	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:69)
      	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155)
      	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2033)
      	at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:652)
      	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:460)
      	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:229)
      	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:184)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
      	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
      	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
      	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
      	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
      	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
      	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
      	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
      	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
      	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
      	at org.eclipse.jetty.server.Server.handle(Server.java:518)
      	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
      	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
      	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
      	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
      	at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
      	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
      	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
      	at java.lang.Thread.run(Thread.java:748)
      Caused by: java.lang.IllegalArgumentException: Document contains at least one immense term in field="title" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped.  Please correct the analyzer to not produce such terms.  The prefix of the first immense term is: '[76, 86, 49, 32, 97, 112, 112, 114, 111, 102, 111, 110, 100, 105, 101, 32, 40, 49, 114, 101, 32, 101, 116, 32, 84, 108, 101, 32, 76, 41]...', original message: bytes can be at most 32766 in length; got 47260
      	at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:772)
      	at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:417)
      	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:373)
      	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:232)
      	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:449)
      	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1492)
      	at org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:282)
      	at org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:214)
      	at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:169)
      	... 48 more
      Caused by: org.apache.lucene.util.BytesRefHash$MaxBytesLengthExceededException: bytes can be at most 32766 in length; got 47260
      	at org.apache.lucene.util.BytesRefHash.add(BytesRefHash.java:280)
      	at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:150)
      	at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:762)
      	... 56 more
      

            sprieul Simon Prieul (Inactive)
            sprieul Simon Prieul (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: