public class DirectoryTaxonomyWriter extends Object implements TaxonomyWriter
TaxonomyWriterwhich uses a
Directoryto store the taxonomy information on disk, and keeps an additional in-memory cache of some or all categories.
In addition to the permanently-stored information in the
Directory, efficiency dictates that we also keep an in-memory cache of recently seen or all categories, so that we do not need to go back to disk for every category addition to see which ordinal this category already has, if any. A
TaxonomyWriterCacheobject determines the specific caching algorithm used.
This class offers some hooks for extending classes to control the
IndexWriterinstance that is used. See
Nested Class Summary
Nested Classes Modifier and Type Class Description
DirectoryTaxonomyWriter.OrdinalMapmaintained on file system
DirectoryTaxonomyWriter.OrdinalMapmaintained in memory
DirectoryTaxonomyWriter.OrdinalMapMapping from old ordinal to new ordinals, used when merging indexes wit separate taxonomies.
Constructors Constructor Description
DirectoryTaxonomyWriter(Directory d)Create this with
DirectoryTaxonomyWriter(Directory directory, IndexWriterConfig.OpenMode openMode)Creates a new instance with a default cache as defined by
DirectoryTaxonomyWriter(Directory directory, IndexWriterConfig.OpenMode openMode, TaxonomyWriterCache cache)Construct a Taxonomy writer.
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description
addCategory(FacetLabel categoryPath)addCategory() adds a category with a given path name to the taxonomy, and returns its ordinal.
addTaxonomy(Directory taxoDir, DirectoryTaxonomyWriter.OrdinalMap map)Takes the categories from the given taxonomy directory, and adds the missing ones to this taxonomy.
commit()The second phase of a 2-phase commit.
defaultTaxonomyWriterCache()Defines the default
TaxonomyWriterCacheto use in constructors which do not specify one.
getCommitData()Returns the commit user data map that was set on
Directoryof this taxonomy writer.
getParent(int ordinal)getParent() returns the ordinal of the parent category of the category with the given ordinal.
getSize()getSize() returns the number of categories in the taxonomy.
getTaxonomyEpoch()Expert: returns current index epoch, if this is a near-real-time reader.
prepareCommit()prepare most of the work needed for a two-phase commit.
replaceTaxonomy(Directory taxoDir)Replaces the current taxonomy with the given one.
rollback()Rollback changes to the taxonomy writer and closes the instance.
setCacheMissesUntilFill(int i)Set the number of cache misses before an attempt is made to read the entire taxonomy into the in-memory cache.
setCommitData(Map<String,String> commitUserData)Sets the commit user data map.
unlock(Directory directory)Forcibly unlocks the taxonomy in the named directory.
public static final String INDEX_EPOCHProperty name of user commit data that contains the index epoch. The epoch changes whenever the taxonomy is recreated (i.e. opened with
Applications should not use this property in their commit data because it will be overridden by this taxonomy writer.
- See Also:
- Constant Field Values
public DirectoryTaxonomyWriter(Directory directory, IndexWriterConfig.OpenMode openMode, TaxonomyWriterCache cache) throws IOExceptionConstruct a Taxonomy writer.
Directoryin which to store the taxonomy. Note that the taxonomy is written directly to that directory (not to a subdirectory of it).
openMode- Specifies how to open a taxonomy for writing:
APPENDmeans open an existing index for append (failing if the index does not yet exist).
CREATEmeans create a new index (first deleting the old one if it already existed).
APPEND_OR_CREATEappends to an existing index if there is one, otherwise it creates a new index.
TaxonomyWriterCacheimplementation which determines the in-memory caching policy. See for example
Cl2oTaxonomyWriterCache. If null or missing,
CorruptIndexException- if the taxonomy is corrupted.
LockObtainFailedException- if the taxonomy is locked by another writer. If it is known that no other concurrent writer is active, the lock might have been left around by an old dead process, and should be removed using
IOException- if another error occurred.
public DirectoryTaxonomyWriter(Directory directory, IndexWriterConfig.OpenMode openMode) throws IOExceptionCreates a new instance with a default cache as defined by
public static void unlock(Directory directory) throws IOExceptionForcibly unlocks the taxonomy in the named directory.
Caution: this should only be used by failure recovery code, when it is known that no other process nor thread is in fact currently accessing this taxonomy.
This method is unnecessary if your
NativeFSLockFactoryinstead of the default
SimpleFSLockFactory. When the "native" lock is used, a lock does not stay behind forever when the process using it dies.
public static TaxonomyWriterCache defaultTaxonomyWriterCache()
public void close() throws IOExceptionFrees used resources as well as closes the underlying
IndexWriter, which commits whatever changes made to it to the underlying
public int addCategory(FacetLabel categoryPath) throws IOExceptionaddCategory() adds a category with a given path name to the taxonomy, and returns its ordinal. If the category was already present in the taxonomy, its existing ordinal is returned.
Before adding a category, addCategory() makes sure that all its ancestor categories exist in the taxonomy as well. As result, the ordinal of a category is guaranteed to be smaller then the ordinal of any of its descendants.
public void commit() throws IOExceptionDescription copied from interface:
TwoPhaseCommitThe second phase of a 2-phase commit. Implementations should ideally do very little work in this method (following
TwoPhaseCommit.prepareCommit(), and after it returns, the caller can assume that the changes were successfully committed to the underlying storage.
setCommitDataSets the commit user data map. That method is considered a transaction and will be
committedeven if no other changes were made to the writer instance.
NOTE: the map is cloned internally, therefore altering the map's contents after calling this method has no effect.
getCommitDataReturns the commit user data map that was set on
public void prepareCommit() throws IOExceptionprepare most of the work needed for a two-phase commit. See
public int getSize()getSize() returns the number of categories in the taxonomy.
Because categories are numbered consecutively starting with 0, it means the taxonomy contains ordinals 0 through getSize()-1.
Note that the number returned by getSize() is often slightly higher than the number of categories inserted into the taxonomy; This is because when a category is added to the taxonomy, its ancestors are also added automatically (including the root, which always get ordinal 0).
public void setCacheMissesUntilFill(int i)Set the number of cache misses before an attempt is made to read the entire taxonomy into the in-memory cache.
This taxonomy writer holds an in-memory cache of recently seen categories to speed up operation. On each cache-miss, the on-disk index needs to be consulted. When an existing taxonomy is opened, a lot of slow disk reads like that are needed until the cache is filled, so it is more efficient to read the entire taxonomy into memory at once. We do this complete read after a certain number (defined by this method) of cache misses.
If the number is set to
0, the entire taxonomy is read into the cache on first use, without fetching individual categories first.
NOTE: it is assumed that this method is called immediately after the taxonomy writer has been created.
public int getParent(int ordinal) throws IOExceptiongetParent() returns the ordinal of the parent category of the category with the given ordinal.
When a category is specified as a path name, finding the path of its parent is as trivial as dropping the last component of the path. getParent() is functionally equivalent to calling getPath() on the given ordinal, dropping the last component of the path, and then calling getOrdinal() to get an ordinal back.
If the given ordinal is the ROOT_ORDINAL, an INVALID_ORDINAL is returned. If the given ordinal is a top-level category, the ROOT_ORDINAL is returned. If an invalid ordinal is given (negative or beyond the last available ordinal), an ArrayIndexOutOfBoundsException is thrown. However, it is expected that getParent will only be called for ordinals which are already known to be in the taxonomy. TODO (Facet): instead of a getParent(ordinal) method, consider having a
getCategory(categorypath, prefixlen) which is similar to addCategory except it doesn't add new categories; This method can be used to get the ordinals of all prefixes of the given category, and it can use exactly the same code and cache used by addCategory() so it means less code.
public void addTaxonomy(Directory taxoDir, DirectoryTaxonomyWriter.OrdinalMap map) throws IOExceptionTakes the categories from the given taxonomy directory, and adds the missing ones to this taxonomy. Additionally, it fills the given
DirectoryTaxonomyWriter.OrdinalMapwith a mapping from the original ordinal to the new ordinal.
public void rollback() throws IOExceptionRollback changes to the taxonomy writer and closes the instance. Following this method the instance becomes unusable (calling any of its API methods will yield an
public void replaceTaxonomy(Directory taxoDir) throws IOExceptionReplaces the current taxonomy with the given one. This method should generally be called in conjunction with
IndexWriter.addIndexes(Directory...)to replace both the taxonomy as well as the search index content.
public final long getTaxonomyEpoch()Expert: returns current index epoch, if this is a near-real-time reader. Used by
DirectoryTaxonomyReaderto support NRT.