Extension-based logic for fds add

[pwoolvett]

Its quite common for our use case that files which should go into dvc have the same extension (and that extension should not go into git according to our guidelines). For example: HDF5, pickle, etc.

Is that something you'd like to incorporate? eg via

• a value in fds.domain.constants
• a configuration file in $REPO_ROOT, or ~/.config/fds
• env var
• a cli flag

(or any combination of them, maybe overriding defaults given a preference order)

If it is, I'd like to take a shot at it, starting from here https://github.com/DAGsHub/fds/search?q=MAX_THRESHOLD_SIZE

While we're at it, is a configuration file in your horizon somewhere?

[guysmoilov]

@pwoolvett Thank you for the interesting idea!
This seems to make sense, we have to make sure to interactively ask the user if this was their intention and maybe explain why the small files are suggested to be added to DVC and also instruct them how to change this default behavior in the prompt's help option.
Otherwise it might get annoying fast.

About confs - it's definitely something to consider. Everything should be overridable using env variables without editing conf files, but using dotenv seems like a decent common choice to combine the approaches - conveniently storing env variables in a file if that's what you want.