Failed parsing of log files should be ignored

[OussamaSaoudi-db]

Describe the bug

When listing files in the delta log, the kernel parses each file path into a ParsedLogPath. This is fallible, but errors must not be propagated to the caller.

Details:

This issue was first described here.

The ParsedLogPath represents the file types that kernel understands such as commits, checkpoints, and compacted commits[1]. When the ParsedLogPath encounters an unknown file type, it fails, returning an error. This error is propagated to the caller.

The delta specification requires that unrecognized protocol fields be ignored.

Clients must ignore such unrecognized fields, and should not produce an error when reading a table that contains unrecognized fields.
Some of these delta table features like checkpointv2 and log compaction generate log files. Thus by extension, unrecognized files must be ignored as well.

[1]: See crate::path::LogPathFileType for more details.

To Reproduce

A _delta_log with the following files must not fail to list log files:

# hex instead of decimal
00000000deadbeef.commit.json  

# bogus part numbering
00000000000000000000.checkpoint.0000000010.0000000000.parquet 

# v2 checkpoint, as seen by a client that doesn't understand that feature
00000000000000000010.checkpoint.80a083e8-7026-4e79-81be-64bd76c43a11.json

# compacted log file, as seen by a client that doesn't understand that feature
00000000000000000004.00000000000000000006.compacted.json

# CRC files 
00000000000000000001.crc

Credit to @scovich for the examples.

Expected behavior

The unrecognized errors above must be ignored.

Additional context

Once #495 merges, the change will likely be in list_log_files in kernel/src/log_segment.rs