New dictionary format

[mh-northlander]

rebased #200.

todo:

• fix errors
• add tests
• remove legacy code and organize
• add way to convert from legacy (user) dictionary
  • fix DictionaryPrinter #234 should solve this.
  • add migration guide
  • add script to convert legacy lexicon
• add feature: make split-C work.
• add feature: allow word reference with pos-id in the lexicon csv.

maybe:

• Be able to lookup normalized/dictionary entries, not just their string forms #202
  • new format does not allow easy access for the normalized/dictionary form
• Separate POS list as a def file (loaded when build like matrix.def)
• compatibility check of system/user dict
  • we can check if the user dict is built on the system dict on load. we may raise error/warning.

[sonarcloud]

Quality Gate passed

Issues
23 New issues
0 Accepted issues

Measures
0 Security Hotspots
85.6% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[mh-northlander]

This PR separates POS list as a file and #199 will be (partially) solved.

[mh-northlander]

#241 is a problem in this RP too.
I will fix it and merge to this branch.

[mh-northlander]

Sonar issue can be fixed by #242.

[sonarcloud]

Quality Gate passed

Issues
22 New issues
0 Accepted issues

Measures
0 Security Hotspots
86.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud