io.bdrc.lucene.bo

Class TibetanAnalyzer

java.lang.Object

org.apache.lucene.analysis.Analyzer

io.bdrc.lucene.bo.TibetanAnalyzer

All Implemented Interfaces:

Closeable, AutoCloseable

public final class TibetanAnalyzer
extends org.apache.lucene.analysis.Analyzer

An Analyzer that uses TibSyllableTokenizer and filters with StopFilter Derived from Lucene 6.4.1 analysis.core.WhitespaceAnalyzer.java

Author:

Chris Tomlinson, Hélios Hildt

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.analysis.Analyzer

org.apache.lucene.analysis.Analyzer.ReuseStrategy, org.apache.lucene.analysis.Analyzer.TokenStreamComponents

Field Summary

Fields

Modifier and Type	Field and Description
static String	INPUT_METHOD_ALALC
static String	INPUT_METHOD_DEFAULT
static String	INPUT_METHOD_DTS
static String	INPUT_METHOD_EWTS
static String	INPUT_METHOD_UNICODE

Fields inherited from class org.apache.lucene.analysis.Analyzer

GLOBAL_REUSE_STRATEGY, PER_FIELD_REUSE_STRATEGY

Constructor Summary

Constructors

Constructor and Description
TibetanAnalyzer() Creates a new TibetanAnalyzer with the default values
TibetanAnalyzer(boolean segmentInWords, boolean lemmatize, boolean filterChars, String inputMethod, String stopFilename) Creates a new TibetanAnalyzer

Method Summary

Modifier and Type	Method and Description
protected org.apache.lucene.analysis.Analyzer.TokenStreamComponents	createComponents(String fieldName)
static ArrayList<String>	getWordList(InputStream inputStream, String comment)
protected Reader	initReader(String fieldName, Reader reader)

Methods inherited from class org.apache.lucene.analysis.Analyzer

attributeFactory, close, getOffsetGap, getPositionIncrementGap, getReuseStrategy, getVersion, initReaderForNormalization, normalize, normalize, setVersion, tokenStream, tokenStream

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

INPUT_METHOD_UNICODE

public static final String INPUT_METHOD_UNICODE

See Also:

Constant Field Values

INPUT_METHOD_DTS

public static final String INPUT_METHOD_DTS

See Also:

Constant Field Values

INPUT_METHOD_EWTS

public static final String INPUT_METHOD_EWTS

See Also:

Constant Field Values

INPUT_METHOD_ALALC

public static final String INPUT_METHOD_ALALC

See Also:

Constant Field Values

INPUT_METHOD_DEFAULT

public static final String INPUT_METHOD_DEFAULT

See Also:

Constant Field Values

Constructor Detail

TibetanAnalyzer

public TibetanAnalyzer(boolean segmentInWords,
                       boolean lemmatize,
                       boolean filterChars,
                       String inputMethod,
                       String stopFilename)
                throws IOException

Creates a new TibetanAnalyzer

Parameters:

segmentInWords - if the segmentation is on words instead of syllables

lemmatize - if the analyzer should remove affixed particles, and normalize words in words mode

filterChars - if the text should be converted to NFD (necessary for texts containing NFC strings)

inputMethod - if the text should be converted from EWTS to Unicode

stopFilename - a file name with a stop word list

Throws:

IOException - if the file containing stopwords can't be opened

TibetanAnalyzer

public TibetanAnalyzer()
                throws IOException

Creates a new TibetanAnalyzer with the default values

Throws:

IOException - if the file containing stopwords can't be opened

Method Detail

getWordList

public static ArrayList<String> getWordList(InputStream inputStream,
                                            String comment)
                                     throws IOException

Parameters:

inputStream - stream to the list of stopwords

comment - The string representing a comment

Returns:

result the ArrayList to fill with the reader's words

Throws:

IOException - if the file containing stopwords can't be opened

initReader

protected Reader initReader(String fieldName,
                            Reader reader)

Overrides:

initReader in class org.apache.lucene.analysis.Analyzer

createComponents

protected org.apache.lucene.analysis.Analyzer.TokenStreamComponents createComponents(String fieldName)

Specified by:

createComponents in class org.apache.lucene.analysis.Analyzer