StandardAnalyser

Namespace

Rowles.LeanLucene.Analysis.Analysers

Assembly

Rowles.LeanLucene.dll

Default analyser combining tokenisation, lowercase normalisation, and stop-word removal into a single pipeline. Uses original input offsets for lowercasing to avoid double string allocation. The returned token list is reused across calls — callers must not hold references to it beyond the current invocation.

Thread-safety: This class maintains instance-level buffers (_tokensBuf, _lowerBuf, _internCache) for performance. Each instance should be used by a single thread, or callers should create separate instances per thread (as IndexWriter does in AddDocumentsConcurrent).

public sealed class StandardAnalyser : IAnalyser

StandardAnalyser

Implements

IAnalyser

Constructors

StandardAnalyser(int, IEnumerable<string>?)

Initialises a new StandardAnalyser with the specified intern cache size and stop words.

Methods

Analyse(ReadOnlySpan<char>)

Analyses the input text and returns a list of tokens.