Consider adding a persistence layer

[Logerfo]

Having a persistence layer would allow new features, such as those described below.

Deleting transcription messages created by an aborted operation (timeout, unhandled exception or whatnot)

After creating a message, the system should create a database record with its id and creation time. After the message is completely handled, the system should mark the record as "done". "GC-esque" calls could be made every lambda_timeout seconds to delete messages older than lambda_timeout seconds and not marked as done.

Deleting transcription messages when the original audio message is deleted

After creating a message, the system should create a database record with its id and the original audio message's id. The system should subscribe to a message deletion event and delete the associated transcription messages when the original audio message is deleted. Unfortunately, Telegram Bot API doesn't support this kind of event at the moment.

Response caching

After creating a message, the system should create a database record with its text and the original file hash. Subsequent requests for the same file hash should return the same text, skipping both the ffmpeg call and the Wit.AI transcription request. I think that the file hash is present in Telegram's webhook payload. If I'm correct, even the file download can be skipped.

Warning

As a lightweight application, the persistence layer should be opt-in, since the core feature of the bot works without it and the layer will probably inflict performance and monetary costs.

Possible tools

• Amazon DynamoDB
• Amazon EFS (with SQLite?)