Class DocTermOrds


  • public class DocTermOrds
    extends java.lang.Object
    This class enables fast access to multiple term ords for a specified field across all docIDs. Like FieldCache, it uninverts the index and holds a packed data structure in RAM to enable fast access. Unlike FieldCache, it can handle multi-valued fields, and, it does not hold the term bytes in RAM. Rather, you must obtain a TermsEnum from the getOrdTermsEnum(org.apache.lucene.index.AtomicReader) method, and then seek-by-ord to get the term's bytes. While normally term ords are type long, in this API they are int as the internal representation here cannot address more than MAX_INT unique terms. Also, typically this class is used on fields with relatively few unique terms vs the number of documents. In addition, there is an internal limit (16 MB) on how many bytes each chunk of documents may consume. If you trip this limit you'll hit an IllegalStateException. Deleted documents are skipped during uninversion, and if you look them up you'll get 0 ords. The returned per-document ords do not retain their original order in the document. Instead they are returned in sorted (by ord, ie term's BytesRef comparator) order. They are also de-dup'd (ie if doc has same term more than once in this field, you'll only get that ord back once). This class tests whether the provided reader is able to retrieve terms by ord (ie, it's single segment, and it uses an ord-capable terms index). If not, this class will create its own term index internally, allowing to create a wrapped TermsEnum that can handle ord. The getOrdTermsEnum(org.apache.lucene.index.AtomicReader) method then provides this wrapped enum, if necessary. The RAM consumption of this class can be high!
    • Field Summary

      Fields 
      Modifier and Type Field Description
      static int DEFAULT_INDEX_INTERVAL_BITS
      Every 128th term is indexed, by default.
    • Constructor Summary

      Constructors 
      Constructor Description
      DocTermOrds​(AtomicReader reader, Bits liveDocs, java.lang.String field)
      Inverts all terms
      DocTermOrds​(AtomicReader reader, Bits liveDocs, java.lang.String field, BytesRef termPrefix)
      Inverts only terms starting w/ prefix
      DocTermOrds​(AtomicReader reader, Bits liveDocs, java.lang.String field, BytesRef termPrefix, int maxTermDocFreq)
      Inverts only terms starting w/ prefix, and only terms whose docFreq (not taking deletions into account) is <= maxTermDocFreq
      DocTermOrds​(AtomicReader reader, Bits liveDocs, java.lang.String field, BytesRef termPrefix, int maxTermDocFreq, int indexIntervalBits)
      Inverts only terms starting w/ prefix, and only terms whose docFreq (not taking deletions into account) is <= maxTermDocFreq, with a custom indexing interval (default is every 128nd term).
    • Field Detail

      • DEFAULT_INDEX_INTERVAL_BITS

        public static final int DEFAULT_INDEX_INTERVAL_BITS
        Every 128th term is indexed, by default.
        See Also:
        Constant Field Values
    • Constructor Detail

      • DocTermOrds

        public DocTermOrds​(AtomicReader reader,
                           Bits liveDocs,
                           java.lang.String field)
                    throws java.io.IOException
        Inverts all terms
        Throws:
        java.io.IOException
      • DocTermOrds

        public DocTermOrds​(AtomicReader reader,
                           Bits liveDocs,
                           java.lang.String field,
                           BytesRef termPrefix)
                    throws java.io.IOException
        Inverts only terms starting w/ prefix
        Throws:
        java.io.IOException
      • DocTermOrds

        public DocTermOrds​(AtomicReader reader,
                           Bits liveDocs,
                           java.lang.String field,
                           BytesRef termPrefix,
                           int maxTermDocFreq)
                    throws java.io.IOException
        Inverts only terms starting w/ prefix, and only terms whose docFreq (not taking deletions into account) is <= maxTermDocFreq
        Throws:
        java.io.IOException
      • DocTermOrds

        public DocTermOrds​(AtomicReader reader,
                           Bits liveDocs,
                           java.lang.String field,
                           BytesRef termPrefix,
                           int maxTermDocFreq,
                           int indexIntervalBits)
                    throws java.io.IOException
        Inverts only terms starting w/ prefix, and only terms whose docFreq (not taking deletions into account) is <= maxTermDocFreq, with a custom indexing interval (default is every 128nd term).
        Throws:
        java.io.IOException
    • Method Detail

      • ramUsedInBytes

        public long ramUsedInBytes()
        Returns total bytes used.
      • getOrdTermsEnum

        public TermsEnum getOrdTermsEnum​(AtomicReader reader)
                                  throws java.io.IOException
        Returns a TermsEnum that implements ord. If the provided reader supports ord, we just return its TermsEnum; if it does not, we build a "private" terms index internally (WARNING: consumes RAM) and use that index to implement ord. This also enables ord on top of a composite reader. The returned TermsEnum is unpositioned. This returns null if there are no terms.

        NOTE: you must pass the same reader that was used when creating this class

        Throws:
        java.io.IOException
      • numTerms

        public int numTerms()
        Returns the number of terms in this field
      • isEmpty

        public boolean isEmpty()
        Returns true if no terms were indexed.
      • lookupTerm

        public BytesRef lookupTerm​(TermsEnum termsEnum,
                                   int ord)
                            throws java.io.IOException
        Returns the term (BytesRef) corresponding to the provided ordinal.
        Throws:
        java.io.IOException
      • iterator

        public SortedSetDocValues iterator​(AtomicReader reader)
                                    throws java.io.IOException
        Returns a SortedSetDocValues view of this instance
        Throws:
        java.io.IOException