Extend metadata with descriptors of CF Ref handling #1836
Conversation
Does the PR have any schema changes?Looking good! No breaking changes found. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1836 +/- ##
==========================================
- Coverage 49.96% 47.13% -2.84%
==========================================
Files 43 50 +7
Lines 6636 7050 +414
==========================================
+ Hits 3316 3323 +7
- Misses 3077 3483 +406
- Partials 243 244 +1 ☔ View full report in Codecov by Sentry. 🚨 Try these New Features:
|
t0yv0
changed the title
t0yv0
force-pushed
the
t0yv0/categorize-refs
branch
from
7b66ded
to
f91f587
Compare
| ) | ||
|
|
||
| // Instead of humans playing the game, let the computer assign labels based on heuristics that seem to be fairly robust. | ||
| // This should add a label in the data indicating that it was assigned by a heuristic, in case this data point ends up |
| continue | ||
| } | ||
|
|
||
| switch { |
There was a problem hiding this comment.
Can you add a comment to each one of these cases with a simple example? I think it would help in quickly understanding each one.
There was a problem hiding this comment.
It's a bit ad-hoc but I've added a quick comment.
| var categorizationRules = []categorizationRule{ | ||
| { | ||
| Category: RefReturnsArn, | ||
| Pattern: regexp.MustCompile(`^The Amazon Resource Name ..ARN.. of the certificate authority.?.?$`), |
There was a problem hiding this comment.
Just curious, did you create these by hand? Do they map more than one resource? I'm curious how scalable this is.
Also can you add a comment explaining the above so we know the process for updating this?
There was a problem hiding this comment.
This is not supposed to be scalable, this is throw-away data cleaning code. I'll add a quick README. Once we complete curating a clean data set in ref-db.json we can delete this.
provider/tools/ref-parser/cat.go
Outdated
| Pattern: regexp.MustCompile("When you pass the logical ID of this resource to the intrinsic .Ref.?.?function, .Ref.?.?returns a .* ID"), | ||
| }, | ||
| // { | ||
| // Name: "RefReturnsArn", |
|
|
||
| What Ref returns varies from resource to resource. The only available information on this is free-form text in CloudFormation documentation and anything that can be extracted experimentally. To facilitate building tooling such as `pulumi/pulumi-cdk` that emulates CF behavior, `ref-db.json` collects schematized information describing the behavior of `Ref`. | ||
|
|
||
| `ref-db.json` can be edited directly to correct errors or add entries for resources not yet covered. |
There was a problem hiding this comment.
Can you add an example here on what it looks like to update this?
There was a problem hiding this comment.
You send a PR and edit it? I'm not sure I understand. It just has to conform to the schema.
CloudFormation Ref intrinsic behavior varies from resource to resource. This change is an attempt to schematize behavior and expose is in the provider metadata. One intended use case is
pulumi/pulumi-cdkCF emulator.This change implies the
ref-db.jsonbecomes the source of truth for this metadata and we maintain it as well as possible.About 1/2 of CF resources are currently covered in the file, including 81 of the top used resources (pulumi/pulumi-cdk#237). Data cleaning and data label attachment helper code is included but is not intended to be used in production but only to maintain
ref-db.json.