WhitespaceAnalyser

Namespace

Rowles.LeanCorpus.Analysis.Analysers

Assembly

Rowles.LeanCorpus.dll

Analyser that splits text only on whitespace and applies no token filters. The returned token list is reused across calls; callers must not hold references to it beyond the current invocation.

Thread-safety: This class maintains instance-level buffers for performance. Each instance should be used by a single thread, or callers should create separate instances per thread.

public sealed class WhitespaceAnalyser : IAnalyser

WhitespaceAnalyser

Implements

IAnalyser

Constructors

WhitespaceAnalyser(int)

Initialises a new WhitespaceAnalyser.

Methods

Analyse(ReadOnlySpan<char>)

Analyses the input text and returns a list of tokens.