Allow designation of output filenames

[nathanielnrn]

It would be nice if within a runt.toml there was an option to designate an output filename different than the basename of the input file.

As an example
Something like this

[[tests]]
name = "[core] parsing"
# Round-tripping from the compiler should not change anything.
paths = ["./tests/parsing/file1.futil"]
cmd = """
./target/debug/calyx {} -m file -p none -l .
"""
output = "output.futil"

Would output to a file called output.futil instead of file1.expect. The thought is that this might be able to assist in testing multiple steps in a compilation cycle generated from a single source file, as opposed to needing duplicate source files for each step.

For example we could have multiple tests starting from file1.futil.
The first test would check the first step against output.futil
The second test would run the first step then check the output of the second step against output2.futil and so on.

[rachitnigam]

This alone is not sufficient because a test suite will generally have a bunch of different files being tested. What about generalizing the expect_dir stuff to define a template for the name:

expect_template = "dir0/{}-out.futil"

Where {} is replaced with the name of the test. If you just have "dir/out.futil", all files will be tested against that one file.

[ethanuppal]

What is meant by "name" precisely? Is the name the path? Is it the name in the TOML file?

[nathanielnrn]

Usually {} refers to whatever is in paths, possibly expanded with wildcards. So paths = ["/dir/*.futil] will run cmd and replace every occurrence of {}with something like dir/file1.futil then dir/file2.futil and so on

[ethanuppal]

So expect_template = "dir0/{}-out.futil"would expand to "dir0/dir/file1.futil-out.futil"?

[ethanuppal]

Just want to clarify this before I implement

[nathanielnrn]

That sounds about right. I recommend you test thi with actual runt invocations. Passing in the -n will output the commands to be run by any tests selected with the -i and -x flags

[ethanuppal]

Sorry, I just got the notification for this. I'll get on it likely Wednesday.