Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

3122 valid license url characters #3449

Merged
merged 4 commits into from
Nov 19, 2024
Merged

3122 valid license url characters #3449

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
b1acb4d
chore: strip unwanted characters from license URL
spiffcs Nov 17, 2024
0a9b783
test: update tests to check for new url logic
spiffcs Nov 17, 2024
062504b
chore: update comments
spiffcs Nov 19, 2024
02bdb7d
chore: remove comments
spiffcs Nov 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 24 additions & 2 deletions syft/pkg/license.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@ package pkg

import (
"fmt"
"net/url"
"sort"
"strings"

"github.com/scylladb/go-set/strset"

Expand Down Expand Up @@ -112,7 +114,12 @@ func NewLicenseFromURLs(value string, urls ...string) License {
s := strset.New()
for _, url := range urls {
if url != "" {
s.Add(url)
sanitizedURL, err := stripUnwantedCharacters(url)
if err != nil {
log.Tracef("unable to sanitize url=%q: %s", url, err)
continue
}
s.Add(sanitizedURL)
}
}

Expand All @@ -122,13 +129,28 @@ func NewLicenseFromURLs(value string, urls ...string) License {
return l
}

func stripUnwantedCharacters(rawURL string) (string, error) {
cleanedURL := strings.TrimSpace(rawURL)
_, err := url.ParseRequestURI(cleanedURL)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we leave these in the core model but then filter them when encoding sbom specs that have specific requirements? That is, should we move this to the cyclonedx encoder? We could check for any similar requirement for SPDX and do the same.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's best to clean it as early as possible in the process and get our own model's URL inline with the RFC 3987 IRI-reference

if err != nil {
return "", fmt.Errorf("invalid URL: %w", err)
}

return cleanedURL, nil
}

func NewLicenseFromFields(value, url string, location *file.Location) License {
l := NewLicense(value)
if location != nil {
l.Locations.Add(*location)
}
if url != "" {
l.URLs = append(l.URLs, url)
sanitizedURL, err := stripUnwantedCharacters(url)
if err != nil {
log.Tracef("unable to sanitize url=%q: %s", url, err)
} else {
l.URLs = append(l.URLs, sanitizedURL)
}
}

return l
Expand Down
36 changes: 36 additions & 0 deletions syft/pkg/license_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -226,3 +226,39 @@ func TestLicense_Merge(t *testing.T) {
})
}
}

func TestLicenseConstructors(t *testing.T) {
type input struct {
value string
urls []string
}
tests := []struct {
name string
input input
expected License
}{
{
name: "License URLs are stripped of newlines and tabs",
input: input{
value: "New BSD License",
urls: []string{
`
http://user-agent-utils.googlecode.com/svn/trunk/UserAgentUtils/LICENSE.txt

`},
},
expected: License{
Value: "New BSD License",
Type: license.Declared,
URLs: []string{"http://user-agent-utils.googlecode.com/svn/trunk/UserAgentUtils/LICENSE.txt"},
},
},
}

for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
got := NewLicenseFromURLs(test.input.value, test.input.urls...)
assert.Equal(t, test.expected, got)
})
}
}
Loading