Bundle() incorrectly dereferences some literal $ref properties

[pimterry]

Not all $ref properties are actual refs that should be dereferenced. In example values specifically, $ref can be used for a literal field name, and should never be dereferenced (I think? OAI/OpenAPI-Specification#1986 (comment) seems to agree).

This matters unfortunately because GitHub's API spec does exactly that: github/rest-api-description#188. In their components.examples they have:

"scim-enterprise-group-list": {
        "value": {
          "schemas": [
            "urn:ietf:params:scim:api:messages:2.0:ListResponse"
          ],
          "totalResults": 2,
          "itemsPerPage": 2,
          "startIndex": 1,
          "Resources": [
            {
              "schemas": [
                "urn:ietf:params:scim:schemas:core:2.0:Group"
              ],
              "id": "abcd27f8-a9aa-11ea-8221-f59b2be9cccc",
              "externalId": null,
              "displayName": "octo-org",
              "members": [
                {
                  "value": "92b58aaa-a1d6-11ea-8227-b9ce9e023ccc",
                  "$ref": "https://api.github.com/scim/v2/enterprises/octo-corp/Users/92b58aaa-a1d6-11ea-8227-b9ce9e023ccc",
                  "display": "[email protected]"
                },
                {
                  "value": "aaaa8c34-a6b2-11ea-9d70-bbbbbd1c8fd5",
                  "$ref": "https://api.github.com/scim/v2/enterprises/octo-corp/Users/aaaa8c34-a6b2-11ea-9d70-bbbbbd1c8fd5",
                  "display": "[email protected]"
                }
              ],
              "meta": {
                "resourceType": "Group",
                "created": "2020-06-09T03:10:17.000+10:00",
                "lastModified": "2020-06-09T03:10:17.000+10:00",
                "location": "https://api.github.com/scim/v2/enterprises/octo-corp/Groups/abcd27f8-a9aa-11ea-8221-f59b2be9cccc"
              }
            },
            {
              "schemas": [
                "urn:ietf:params:scim:schemas:core:2.0:Group"
              ],
              "id": "5e75bbbb-aa1a-11ea-8644-75ff655cdddd",
              "externalId": null,
              "displayName": "octo-docs-org",
              "members": [
                {
                  "value": "92b58aaa-a1d6-11ea-8227-b9ce9e023ccc",
                  "$ref": "https://api.github.com/scim/v2/enterprises/octo-corp/Users/92b58aaa-a1d6-11ea-8227-b9ce9e023ccc",
                  "display": "[email protected]"
                }
              ],
              "meta": {
                "resourceType": "Group",
                "created": "2020-06-09T16:28:01.000+10:00",
                "lastModified": "2020-06-09T16:28:01.000+10:00",
                "location": "https://api.github.com/scim/v2/enterprises/octo-corp/Groups/5e75bbbb-aa1a-11ea-8644-75ff655cdddd"
              }
            }
          ]
        }
      },

The $ref properties within this are literal values, as they would be returned by the API. Bundling this spec fails, like so:

> const parser = require('@apidevtools/swagger-parser')
> parser.parse('./api.github.com.json').then(s => parser.bundle(s))
ResolverError: Error downloading https://api.github.com/scim/v2/enterprises/octo-corp/Users/aaaa8c34-a6b2-11ea-9d70-bbbbbd1c8fd5 
HTTP ERROR 403

Not sure if this applies to cases other than examples, but I wouldn't be that surprised.

[philsturgeon]

Hey @pimterry, did you get a chance to dig into this at all, or is it still an issue? There's a loooooot of issues here and I'm trying to help, but maintenance of this repo isn't my job and I know you're smart! :D

[pimterry]

Hey @philsturgeon! I haven't dug into this any further, but I've just tested with the same code from above, and I can confirm it's still an issue: bundling the official GitHub OpenAPI spec fails with the same error, because it tries to dereference a $ref that's part of an example response, not a real usable reference.

[pimterry]

The prerequisite for implementing this (APIDevTools/json-schema-ref-parser#249) just shipped!

We could now fix this here, we just need to pass an excludedPathMatcher option that can recognize valid examples paths in an OpenAPI document (i.e. match any path to an Example Object) so we treat those literally and skip dereferencing them.

[pimterry]

Just to update this: this is still an issue, and the current behaviour is certainly incorrect, but it fell off my todo list because GitHub changed their spec instead so it no longer applies to my use case.

If anybody else is interested in this, feel free to pick it up! The new json-schema-ref-parser feature from APIDevTools/json-schema-ref-parser#249 should be sufficient to do this, you just need to use that to define which parts of an OpenAPI spec should not be dereferenced automatically (i.e. which parts are literal examples).