check out #270, which adds "./bin/pre-commit" to run pre-commit regardless of whether you are using a devcontainer or singularity

we should ignore everything in docs/ruby. those are all generated files

Can pre-commit check for Unicode characters? A common problem is when a cut/paste from, e.g., the ISA manual, brings in Unicode to the YAML. That sometimes messes with the tools.

Can pre-commit check for Unicode characters? A common problem is when a cut/paste from, e.g., the ISA manual, brings in Unicode to the YAML. That sometimes messes with the tools.

The default check-yaml https://github.com/pre-commit/pre-commit-hooks/blob/main/pre_commit_hooks/check_yaml.py#L11 does a load of the yaml file. Would this detect that problem?

It might, but I'm not sure. The problem arises when we try to load a YAML file with non-ASCII characters using the standard Ruby YAML library (which itself is just a wrapper around the C libyaml library). It will fail, but unfortunately without a helpful error message.

I see that the library used by pre-commit -- ruamel.yaml -- has some pretty cool features like key order preservation and comment preservation. It could be a part of a solution to the "scripting changes" problem I've been concerned about.

It's easy enough to make a little script to check for Unicode. Probably easier to do that then search around for something turn-key ready. How about this: I'll make the script and you figure out how to run it from pre-commit?

It might, but I'm not sure. The problem arises when we try to load a YAML file with non-ASCII characters using the standard Ruby YAML library (which itself is just a wrapper around the C libyaml library). It will fail, but unfortunately without a helpful error message.

I think we might need to discuss this more. My concern is YAML and AsciiDoc are both fully capable of supporting unicode and given these are the "docs" putting the ascii only restriction on the content seems problematic, especially if we want to support tooling that could be used to generate docs in other languages.

Good point. Maybe I'll just figure out why it's failing ;) It should be able to handle UTF-8, after all...

@dhower-qc : this PR should be good to go as we discussed. Broke up into 2 commit: one for hooks added, one for autofix changes from those hooks

When I try this, pre-commit doesn't seem to do anything:

hu-dhower-lv:riscv-unified-db-3 (65) ./bin/pre-commit 
[INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
check for broken symlinks............................(no files to check)Skipped
fix end of files.....................................(no files to check)Skipped
trim trailing whitespace.............................(no files to check)Skipped

Same with pre-commit run --all-files. Is this the expected output?

Same with pre-commit run --all-files. Is this the expected output?

No after you pulled in 2f735f8 you should have seen this:

❯ pre-commit run --all-file
check for broken symlinks............................(no files to check)Skipped
fix end of files.........................................................Failed
- hook id: end-of-file-fixer
- exit code: 1
- files were modified by this hook

Fixing arch/csr/mhartid.yaml
Fixing arch/csr/mip.yaml
Fixing arch/isa/fp.idl

trim trailing whitespace.................................................Failed
- hook id: trailing-whitespace
- exit code: 1
- files were modified by this hook

Fixing schemas/csr_schema.json
Fixing lib/test/test_yaml_loader.rb
Fixing arch/csr/mip.yaml
Fixing arch/csr/mie.yaml

But I'm not running pre-commit from the container.