Add StreamVerifier

[robjtede]

Context

In web applications, it is desirable to avoid storing the request payload. It is currently impossible to verify a signature based on a request payload that is receieved in a chunk stream without allocating additional memory equal to the size of the payload.

Solution

A stateful type that holds the public key, candidate signature, and hash state that can be updated incrementally.

References

• Migrated from Add StreamVerifier ed25519-dalek#196
• As an example of the difference this feature would make: https://github.com/robjtede/actix-web-lab/pull/34/files

[elichai]

Why not a function that takes impl Read instead? (and then you can use std::io::copy to copy into the hasher or use an intermediate buffer with a better size)
@tarcieri Maybe it's worth adding an MAX_OPTIMAL_BUFFER_SIZE (bikeshedding aside) constant to the Update trait for those reasons? see for example the docs blake3 has:

Note that the degree of SIMD parallelism that update can use is limited by the size of this input buffer. The 8 KiB buffer currently used by std::io::copy is enough to leverage AVX2, for example, but not enough to leverage AVX-512. A 16 KiB buffer is large enough to leverage all currently supported SIMD instruction sets.

Source: https://docs.rs/blake3/latest/blake3/struct.Hasher.html#method.update

[tarcieri]

Why not a function that takes impl Read instead?

That requires std

Maybe it's worth adding an MAX_OPTIMAL_BUFFER_SIZE (bikeshedding aside) constant to the Update trait for those reasons?

I thought we had something like that already but now I can't find it, so maybe we need to add something.

[rozbb]

Maybe silly question: isn't this what prehashed signatues are for? Or is this in a setting where you don't have control over what the sender does?

[robjtede]

you don't have control over what the sender does

Yea exactly, my particular use case is Discord Webhooks.

There's some relevant discussion in the old PR.

[robjtede]

Will rebase after CI fixes here: #543

[rozbb]

Ok then I like this! One thing to avoid code duplication: do you think you could move the current VerifyingKey::{verify, verify_strict} functions to the stream verifier, and redefine the originals to use the stream version? I suspect it won't have any performance cost

[robjtede]

Awesome 👍🏻

redefine the originals to use the stream version

I've had a go but having trouble doing in a way that actually reduces code. From my limited knowledge of recent refactors, problem seems to be that VerifyingKey::recompute_R takes a message but we have a Scalar in the streaming case, otherwise it could be changed for the raw_verify fns. I'm hesitant to do too much refactoring in this PR.

[rozbb]

Ah np. I'll take a crack at it

[mkj]

I've pushed #556 which builds on StreamVerifier and unifies the stream/normal verifiers (as well as adding by-digest methods for signing/verify).

[pinkforest]

impl Read needs std

fwiw - embedded folk seems to be fan of a trait that works in no_std: https://crates.io/crates/embedded-io

but yeah that would be fragmented impl and further complicate things with endless permutations

[robjtede]

superseded by probably #583 or #556