Dynamodb table schema

[Shir-Avr]

Hi,

Looking at the docs of the DynamoDbCache object (link to DynamoDbCache docs),
it says the partition key for the dynamodb table is "namespace".
In the constructor's params, the namespace is described as "Name of DynamoDB hash map"

Going over at the dynamodb table itself - all records have the same partition key "responses" (see attached image)
My question is why all records have the same partition key - "responses"?

I feel like it missed the whole point of using dynamodb (which is a fully managed database)
Since by using it this way, the table uses only one partition and all records are written to the same partition.

Why doesn't it use the cache key as the partition key?
thanks!

@JWCook

[JWCook]

Thanks for bringing this up. The short answer is that you're correct; the cache key is currently used as the range key, but would be better as the partition key.

The longer answer is that there is a second namespace, redirects, which optimizes redirect chains by mapping the cache key of the initial request to the cache key of the final request in the chain. Other backends use a separate table to store this redirect info, but the DynamoDB backend uses a single table with two different "namespaces" (partition key values). That was done in the initial implementation here before I came on board as maintainer, and I did not want to break backwards-compatibility.

However, since the 1.0 release is coming soon, this would be as good a time as any to make some backwards-incompatible improvements. There are a couple different options, but probably the easiest would be to use two separate tables, remove namespace, and make key the partition key.

What do you think? And is that something you would be interested in working on?

[JWCook]

@Shir-Avr These changes have been made in #795.

One compromise is that the DynamoDB backend no longer does persistent redirect optimization, and now just stores this info in-memory. Unlike the other backends, with DynamoDB I think the situational performance boost from this is not worth the extra read throughput needed, plus using a separate table or index to store redirects.