Quantcast

Memory problem with Searchable

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Memory problem with Searchable

msmolyak
I am getting the error below when trying to perform a search against a Lucene index whose size is around 1 GB created by Searchable plugin. The program runs fine with smaller index.

Is this an unusually large index size for Searchable? What can be done to resolve this problem? How much memory do I need to allocate to the Grails app for such operations?

Thank you,

Michael

java.lang.OutOfMemoryError: GC overhead limit exceeded
        at org.apache.lucene.index.FieldsReader.addField(FieldsReader.java:357)
        at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:197)
        at org.apache.lucene.index.SegmentReader.document(SegmentReader.java:729)
        at org.apache.lucene.index.MultiSegmentReader.document(MultiSegmentReader.java:260)
        at org.apache.lucene.index.IndexReader.document(IndexReader.java:637)
        at org.compass.core.lucene.util.LuceneUtils.hitsToResourceArray(LuceneUtils.java:64)
        at org.compass.core.lucene.engine.transaction.readcommitted.ReadCommittedTransaction.get(ReadCommittedTransaction.java:255)
        at org.compass.core.lucene.engine.LuceneSearchEngine.get(LuceneSearchEngine.java:324)
        at org.compass.core.impl.DefaultCompassSession.getResourceByIdResource(DefaultCompassSession.java:188)
        at org.compass.core.impl.DefaultCompassSession.getResource(DefaultCompassSession.java:178)
        at org.compass.core.impl.DefaultCompassSession.get(DefaultCompassSession.java:229)
        at org.compass.core.converter.mapping.osem.ReferenceMappingConverter.doUnmarshall(ReferenceMappingConverter.java:98)
        at org.compass.core.converter.mapping.osem.AbstractRefAliasMappingConverter.unmarshall(AbstractRefAliasMappingConverter.java:103)
        at org.compass.core.converter.mapping.osem.AbstractCollectionMappingConverter.unmarshall(AbstractCollectionMappingConverter.java:144)
        at org.compass.core.converter.mapping.osem.ClassMappingConverter.unmarshall(ClassMappingConverter.java:303)
        at org.compass.core.converter.mapping.osem.ComponentMappingConverter.doUnmarshall(ComponentMappingConverter.java:37)
        at org.compass.core.converter.mapping.osem.AbstractRefAliasMappingConverter.unmarshall(AbstractRefAliasMappingConverter.java:103)
        at org.compass.core.converter.mapping.osem.ClassMappingConverter.unmarshall(ClassMappingConverter.java:303)
        at org.compass.core.marshall.DefaultMarshallingStrategy.unmarshall(DefaultMarshallingStrategy.java:167)
        at org.compass.core.marshall.DefaultMarshallingStrategy.unmarshall(DefaultMarshallingStrategy.java:159)
        at org.compass.core.impl.DefaultCompassSession.getByResource(DefaultCompassSession.java:250)
        at org.compass.core.impl.DefaultCompassSession.getByResource(DefaultCompassSession.java:238)
        at org.compass.core.impl.DefaultCompassHits.data(DefaultCompassHits.java:66)
        at org.codehaus.groovy.grails.plugins.searchable.compass.search.DefaultSearchableSubsetHitCollector.collect(DefaultSearchableSubsetHitCollector.java:52)
        at org.codehaus.groovy.grails.plugins.searchable.compass.search.AbstractSearchableHitCollector.collect(AbstractSearchableHitCollector.java:37)
        at org.codehaus.groovy.grails.plugins.searchable.compass.search.DefaultSearchMethod$SearchCompassCallback.doInCompass(DefaultSearchMethod.java:125)
        at org.compass.core.CompassTemplate.execute(CompassTemplate.java:130)
        at org.compass.core.CompassTemplate.execute(CompassTemplate.java:112)
        at org.codehaus.groovy.grails.plugins.searchable.compass.support.AbstractSearchableMethod.doInCompass(AbstractSearchableMethod.java:57)
        at org.codehaus.groovy.grails.plugins.searchable.compass.search.DefaultSearchMethod.invoke(DefaultSearchMethod.java:63)
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Memory problem with Searchable

webmediauk
I'm having the same issues with searchable. Has anyone managed to resolve this? or have any suggestion how to improve my indexes? As far as I can tell it does appear to be related to the index size. If I only index small datasets it's fast and fine.
Many Thanks
Baz
Loading...