Skip to content
This repository has been archived by the owner on Sep 9, 2020. It is now read-only.
/ tok5-tag-pos Public archive

Added into @coalpha/brown-bear

Notifications You must be signed in to change notification settings

coarchive/tok5-tag-pos

Repository files navigation

tok5-tag-pos

Tokenize and Tag Parts of Speech.

This would have been so much easier if wink-tokenizer defined Token as

interface Token {
   token_type: TokenType,
   value: string,
};

// instead of 

interface Token {
   tag: Tag,
   value: string,
};

I could even patch the source but I'm too lazy to go that far. Whenever you read .tag, think .token_type.