UniqueTokenFilter

Namespace

Rowles.LeanCorpus.Analysis.Filters

Assembly

Rowles.LeanCorpus.dll

Removes duplicate tokens while preserving the first occurrence.

public sealed class UniqueTokenFilter : ITokenFilter

UniqueTokenFilter

Implements

ITokenFilter

UniqueTokenFilter(bool)

Initialises a new UniqueTokenFilter.

Apply(List<Token>)

Applies the filter to the token list, modifying it in place.