Add a directory of test cases

[karbassi]

What makes the todo.txt format amazing is the open-source and open-standard of it. There are so many tools written around the format ranging from shell to python to node and everything in between.

The todotxt-cli has a directory of tests that it runs to make sure all cases work. I propose that we create a directory of tests with *.in and *.out files. This allows tool developers to use whatever test framework they want as long as when a *.in file runs through their tool, the *.out matches their output.

Example

0000-add.in

add notice the daisies

0000-add.out

1 notice the daises
TODO: 1 added.

Thoughts and comments welcome.

[inkarkat]

Good idea; this would help ensure compatibility with the CLI.

I think those *.in / *.out files could be auto-generated (via make testdata) from a test run. This would avoid duplication. Though the existing tests could be transformed to also use the new files, I would prefer to keep the current format; it's flexible and has the right granularity.

[karbassi]

@inkarkat is this something you could take on as a task?

[inkarkat]

Yes, I could do this. Let's first wait for more feedback, though.

Do other implementations actually strictly adhere to the same output format / default highlighting / user messages? Do they support the same command-line switches (such as -p, -f)? If not, the generated output probably wouldn't be useful, or only a subset could be used.

[inkarkat]

I think we also need to store the original contents of todo.txt; your add example starts with an empty one, but usually the tests start out with some initial data. I would store that in an additional 0000-add.txt file.

[bram85]

I see limited use for my tool - topydo.

My tool was only inspired by the todo.txt CLI, and I borrowed quite some subcommands, but never intended to mimic the exact behavior. There is highlighting involved, different sorting orders, filters (e.g. items with a threshold). In short, a tool can do anything with the output. Moreover, topydo also does some tricks with the inputs, i.e. converting relative dates to absolute ones.

Of course I could make a fixture with a configuration that approaches the original CLI as close as possible. But I doubt it will always print the desired output (so NOK every time), and the test coverage would be quite low because topydo adds so much on top of input / output processing.

[karbassi]

@bram85 This brings up a good case.

What could we provide in the spec that would help tool developers cover the base cases? Such as if the "add" command is called with notice the daisies, it should at bare minimum save notice the daisies to the todo.txt file.