[DO NOT MERGE][WORK IN PROGRESS]kie-issues#1613: Add `CI :: License headers / check` GitHub Actions workflow #2746

jomarko · 2024-11-15T13:15:52Z

This is partial fix of the issue apache/incubator-kie-issues#1613 as we open one PR per repository.

jomarko · 2024-11-15T13:19:15Z

.rat-excludes

+**/META-INF/beans.xml
+**/licenses/**
+
+### kie-tools specific excludes


this is probably most complicated repository we have, what I need from reviewers is to help with finalization of the exclude patterns, once done and we are sure all what is needed is excluded from 'apache-rat-check', I will fix those files == add license there.

tiagobento · 2024-11-15T15:40:41Z

.github/workflows/pr-rat-check.yml

+
+      - name: Download Apache RAT
+        run: |
+          curl -LO https://repository.apache.org/content/repositories/snapshots/org/apache/rat/apache-rat/0.17-SNAPSHOT/apache-rat-0.17-20241115.065104-374.jar


I don't think we should be using a SNAPSHOT version.

I agree, however the stable version 16 does not support glob pattern in the '.rat-excludes' file, that is why I used 17-SNAPSHOT.

My idea was to use this specific snapshot version until there is stable/final 17, we could track this in kie-issues ticket?

If you say we need to use stable/final 16, we need to list every single excluded file in .rat-excludes

Hmm.... understand... I'm not a big fan os glob exclusions, to be honest, as I think they make it too easy to add stuff to the excluded list. Idk, first I think we need an assessment of which files we should be excluding from "CI :: Check license headers", then I think we can discuss the best approach to do it... WDYT please?

Ok, I will prepare a spreadsheet with files, we need to discuss, FYI, even excluding:

**/target/** **/node_modules/** **/dist/** **/dist-dev/** **/dist-tests/** **/dist-tests-e2e/** **/python-venv/**

we need to discuss ~995 files

shared privately

It shouldn't be necessary to include anything that is in the .gitignore...

Hi,

I'm not a big fan os glob exclusions, to be honest, as I think they make it too easy to add stuff to the excluded list

apache-rat 0.16 uses filename with regex e.g. .*\.csv. I think "easy to add" is not so different between apache-rat 0.16 (regex) and 0.17 (glob pattern). Rather, apache-rat 0.17 glob pattern can express file paths, while apache-rat 0.16 cannot express things like **/optaplanner-examples/data/**/import/**. I'll ask about the preference in dev ML.

I'm not against the assessment of excluding files at all. Thanks!

tiagobento

Thank you for starting this effort @jomarko ! I made some suggestions for us to focus on the "what" (Checking license headers), instead of the "how" (Apache RAT).

.github/workflows/pr-rat-check.yml

tiagobento · 2024-11-15T15:44:11Z

.github/workflows/pr-rat-check.yml

Please name this ci_check_license_headers.yaml

tiagobento · 2024-11-15T15:44:41Z

.github/workflows/pr-rat-check.yml

+on:
+  pull_request:
+    types: [opened, synchronize, reopened, ready_for_review]


I think making it run on merges to main would be good too... We can use the same triggers we have for ci_build.yaml

.github/workflows/pr-rat-check.yml

Co-authored-by: Tiago Bento <[email protected]>

tkobayas · 2024-11-21T07:55:44Z

.rat-excludes

+packages/stunner-editors/kie-wb-common-stunner/kie-wb-common-stunner-sets/kie-wb-common-stunner-bpmn/kie-wb-common-stunner-bpmn-emf/src/main/java/org/eclipse/dd/di/impl/StyleImpl.java
+packages/stunner-editors/kie-wb-common-stunner/kie-wb-common-stunner-sets/kie-wb-common-stunner-bpmn/kie-wb-common-stunner-bpmn-emf/src/main/java/org/eclipse/dd/di/util/DiAdapterFactory.java
+packages/stunner-editors/kie-wb-common-stunner/kie-wb-common-stunner-sets/kie-wb-common-stunner-bpmn/kie-wb-common-stunner-bpmn-emf/src/main/java/org/eclipse/dd/di/util/DiSwitch.java
+packages/stunner-editors/kie-wb-common-stunner/kie-wb-common-stunner-sets/kie-wb-common-stunner-bpmn/kie-wb-common-stunner-bpmn-emf/src/main/java/org/eclipse/dd/di/util/DiValidator.java


listing every excluded single file is nice. However, this format (file path) doesn't work with apache-rat 0.16.1. Please confirm.

Per my testing so far, apache-rat 0.16.1 supports file name without its file path (aaa.txt) or directory name (ccc/*). So far, my idea is only listing file names to exclude (It means listing like DiValidator.java, so it would affect all DiValidator.java in any paths). If you find another way to express file paths, it would be great.

Another idea is to fall back to apache-rat 0.17 and listing every excluded single file as this PR. The problem of "snapshot is pruned" could be resolved by hosting the file somewhere (e.g. incubator-kie-kogito-pipelines)... it may not sound good though.

tkobayas · 2024-11-21T09:51:20Z

.rat-excludes

+packages/serverless-workflow-diagram-editor/uberfire-extensions/uberfire-commons-editor/uberfire-commons-editor-client/src/main/resources/org/uberfire/ext/editor/commons/client/file/exports/js/FileSaver.min.js.back
+packages/serverless-workflow-diagram-editor/uberfire-extensions/uberfire-commons-editor/uberfire-commons-editor-client/src/main/resources/org/uberfire/ext/editor/commons/client/file/exports/js/canvas2svg.js.back
+packages/stunner-editors/errai-common/src/main/java/org/jboss/errai/common/client/util/TimeUnit.java
+packages/stunner-editors/errai-common/src/main/java/org/jboss/errai/common/compat/javax/annotation/processing/Generated.java


@porcelli Per https://lists.apache.org/thread/nrq50szwx37doopb23wyq33v0t3bbccg , I think Generated.java needs to be listed in LICENSE as Category B or X. But if it's one time listing for 10.0.0, it may not be needed in main. Thoughts?

tkobayas · 2024-11-21T10:04:10Z

.rat-excludes

+packages/stunner-editors/kie-wb-common-dmn/kie-wb-common-dmn-webapp-kogito-marshaller/src/main/resources/DI.xsd
+packages/stunner-editors/kie-wb-common-dmn/kie-wb-common-dmn-webapp-kogito-marshaller/src/main/resources/DMN12.xsd
+packages/stunner-editors/kie-wb-common-dmn/kie-wb-common-dmn-webapp-kogito-marshaller/src/main/resources/DMNDI12.xsd
+packages/stunner-editors/kie-wb-common-stunner/kie-wb-common-stunner-sets/kie-wb-common-stunner-bpmn/kie-wb-common-stunner-bpmn-emf/src/main/java/org/eclipse/bpmn2/Activity.java


@porcelli @jomarko Ohh, these files with org/eclipse package classes has EPL license which is Category B. They need to be in LICENSE, right?

(I haven't fully checked the source codes. Just reported what I happened to find)

tiagobento · 2024-11-22T00:31:47Z

@jomarko Sorry for being picky, but to to make this absolutely perfect (😄), can we rename the workflow to CI :: License headers and the name of the job from check-license-headers to simply check?

This way, instead of

we'll have

Thank you!

tkobayas · 2024-11-25T06:22:49Z

Thank you for the huge efforts on the PR updates @jomarko !

Let's revisit the points which Justin mentioned in the ML thread. (If you are checking the source codes with your own order, feel free to ignore this comment for now and check this at the end)

https://lists.apache.org/thread/nrq50szwx37doopb23wyq33v0t3bbccg

The LICENSE is misisng information about bundled 3rd party files e.g [2][3][4][5][6][10][11] and many other 3rd party files.
Files like [3][6[9] seem to  incorrectly have an ASF header on them when the copyright belongs to someone else. How are these file licensed?
Minified code should, if possible, be kept out of a source release. [8][9]
This file is GPL or CDDL licensed and shouldn't be included in an ASF release. [11]

2. ./incubator-kie-tools/packages/serverless-workflow-diagram-editor/third_party/errai/src/main/java/org/jboss/errai/common/client/util/Base64Util.java


5. ./incubator-kie-tools/packages/serverless-workflow-diagram-editor/lienzo-core/src/main/java/com/ait/lienzo/client/core/event/OrientationChangeHandler.java (and many other files)

7. ./incubator-kie-tools/packages/stunner-editors/lienzo-webapp/src/main/java/org/kie/lienzo/client/LienzoExamples.native.js
8. ./incubator-kie-tools/packages/serverless-workflow-diagram-editor/uberfire-extensions/uberfire-commons-editor/uberfire-commons-editor-client/src/main/resources/org/uberfire/ext/editor/commons/client/file/exports/js/jspdf.min.js.back

10 ./incubator-kie-tools/packages/serverless-workflow-diagram-editor/uberfire-extensions/uberfire-commons-editor/uberfire-commons-editor-client/src/main/resources/org/uberfire/ext/editor/commons/client/file/exports/js/canvas2svg.js.back
11. ./incubator-kie-tools/packages/stunner-editors/errai-common/src/main/java/org/jboss/errai/common/compat/javax/annotation/processing/Generated.java

./incubator-kie-tools/packages/serverless-workflow-diagram-editor/third_party/errai/src/main/java/org/jboss/errai/common/client/util/Base64Util.java

Same issue -> ./packages/stunner-editors/errai-common/src/main/java/org/jboss/errai/common/client/util/Base64Util.java

-> Please add them to LICENSE with the notes:

 This code is a modified version of Base64Encoder from http://code.google.com/p/gwt-crypto/source/browse/trunk/src/main/java/com/googlecode/gwt/crypto/bouncycastle/util/encoders/Base64.java, used under the terms of the ASL 2.0 license.

./incubator-kie-tools/packages/serverless-workflow-diagram-editor/lienzo-core/src/main/java/com/ait/lienzo/client/core/event/OrientationChangeHandler.java (and many other files)

-> Already done

./incubator-kie-tools/packages/stunner-editors/lienzo-webapp/src/main/java/org/kie/lienzo/client/LienzoExamples.native.js

-> Already done

./incubator-kie-tools/packages/serverless-workflow-diagram-editor/uberfire-extensions/uberfire-commons-editor/uberfire-commons-editor-client/src/main/resources/org/uberfire/ext/editor/commons/client/file/exports/js/jspdf.min.js.back

-> Please add it to LICENSE with the all copyright comments and MIT license

10 ./incubator-kie-tools/packages/serverless-workflow-diagram-editor/uberfire-extensions/uberfire-commons-editor/uberfire-commons-editor-client/src/main/resources/org/uberfire/ext/editor/commons/client/file/exports/js/canvas2svg.js.back

-> Please add it to LICENSE with the copyright and MIT license

./incubator-kie-tools/packages/stunner-editors/errai-common/src/main/java/org/jboss/errai/common/compat/javax/annotation/processing/Generated.java

-> Already done

I don't know what we can do with "Minified code", but I hope it's not a blocker of 10.0.0.

Thanks!

kie-issues#1613: Add apache-rat check github action

e63e164

This is partial fix of the issue apache/incubator-kie-issues#1613 as we open one PR per repository.

jomarko requested a review from tiagobento as a code owner November 15, 2024 13:15

jomarko added the area:monorepo label Nov 15, 2024

jomarko commented Nov 15, 2024

View reviewed changes

remove checkstyle.header.template

571eab2

tiagobento reviewed Nov 15, 2024

View reviewed changes

tiagobento changed the title ~~kie-issues#1613: Add apache-rat check github action~~ kie-issues#1613: Add Apache RAT check GitHub Actions workflow Nov 15, 2024

tiagobento requested changes Nov 15, 2024

View reviewed changes

tiagobento changed the title ~~kie-issues#1613: Add Apache RAT check GitHub Actions workflow~~ kie-issues#1613: Add CI :: Check license headers GitHub Actions workflow Nov 15, 2024

tiagobento added the area:cicd label Nov 15, 2024

jomarko and others added 3 commits November 18, 2024 08:46

Update .github/workflows/pr-rat-check.yml

e83541f

Co-authored-by: Tiago Bento <[email protected]>

Update .github/workflows/pr-rat-check.yml

bed1c37

Co-authored-by: Tiago Bento <[email protected]>

update triggers

9f863fa

jomarko mentioned this pull request Nov 18, 2024

kie-issues#1613: Add CI :: License headers GitHub Actions workflow apache/incubator-kie-kogito-runtimes#3714

Open

8 tasks

jomarko requested a review from tiagobento November 19, 2024 12:59

tkobayas requested a review from porcelli November 20, 2024 03:36

jomarko added 2 commits November 20, 2024 11:40

List excluded files explicitely

7681a7e

add third-party excludes

eb1037b

tkobayas reviewed Nov 21, 2024

View reviewed changes

jomarko mentioned this pull request Nov 21, 2024

[10.0.x] Backport NOTICE and LISENSE refactoring and rat-excludes to 10.0.x apache/incubator-kie-drools#6167

Merged

tkobayas reviewed Nov 21, 2024

View reviewed changes

yesamer mentioned this pull request Nov 21, 2024

Add .rat-excludes file to each repository apache/incubator-kie-issues#1613

Open

6 tasks

jomarko added 3 commits November 21, 2024 13:47

List explicitely all files in the .rat-excludes

767a6e2

Use apache-rat v0.16.1

0d481a3

Everthing -> Everything

17003e0

jomarko added pr: DO NOT MERGE Draft PR, not ready for merging pr: wip PR is still under development labels Nov 21, 2024

jomarko changed the title ~~kie-issues#1613: Add CI :: Check license headers GitHub Actions workflow~~ [DO NOT MERGE][WORK IN PROGRESS]kie-issues#1613: Add CI :: Check license headers GitHub Actions workflow Nov 21, 2024

jomarko added 18 commits November 22, 2024 06:27

CI :: License headers

5b92220

Merge branch 'main' into kie-issues#1613

c0f1875

NOTICE -> LICENSE

4c94cde

BPMN and DMN xsd LICENSE

a548b7d

Makefile licenseheader

478818a

Fix *.dmn license headers

deea9b6

Fix README.md license headers

f8650c6

fix docker-compose

d2aff84

fix svg icons

fdb5987

application.properties

5781ee8

fix hiring.bpmn

a20a419

.less licenses

697df72

fix css and scss

c10c787

fix Chart.yaml

d8e03f9

fix java files

4e8d8ad

DEV_DEPLOYMENTS

7c48f93

mdx

bb783a8

types.ts

16ae495

jomarko added 5 commits November 25, 2024 12:30

revert run-script-if and image-builder README

1104eae

Base64Util.java

00288c9

licenses delimeter

51c0fad

Add js.back files

c612cfc

Merge branch 'main' into kie-issues#1613

61180fb

tiagobento changed the title ~~[DO NOT MERGE][WORK IN PROGRESS]kie-issues#1613: Add CI :: Check license headers GitHub Actions workflow~~ [DO NOT MERGE][WORK IN PROGRESS]kie-issues#1613: Add CI :: License headers / check GitHub Actions workflow Nov 25, 2024

sonataflow-operator yaml files

982d0d3

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DO NOT MERGE][WORK IN PROGRESS]kie-issues#1613: Add `CI :: License headers / check` GitHub Actions workflow #2746

[DO NOT MERGE][WORK IN PROGRESS]kie-issues#1613: Add `CI :: License headers / check` GitHub Actions workflow #2746

jomarko commented Nov 15, 2024

jomarko Nov 15, 2024

tiagobento Nov 15, 2024

jomarko Nov 18, 2024

tiagobento Nov 19, 2024

jomarko Nov 19, 2024 •

edited

Loading

jomarko Nov 19, 2024

tiagobento Nov 20, 2024

tkobayas Nov 20, 2024 •

edited

Loading

tiagobento left a comment

tiagobento Nov 15, 2024

jomarko Nov 18, 2024

tiagobento Nov 15, 2024

jomarko Nov 18, 2024

tkobayas Nov 21, 2024

tkobayas Nov 21, 2024 •

edited

Loading

tkobayas Nov 21, 2024

tiagobento commented Nov 22, 2024

tkobayas commented Nov 25, 2024 •

edited

Loading

[DO NOT MERGE][WORK IN PROGRESS]kie-issues#1613: Add CI :: License headers / check GitHub Actions workflow #2746

Are you sure you want to change the base?

[DO NOT MERGE][WORK IN PROGRESS]kie-issues#1613: Add CI :: License headers / check GitHub Actions workflow #2746

Conversation

jomarko commented Nov 15, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jomarko Nov 19, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tkobayas Nov 20, 2024 • edited Loading

Choose a reason for hiding this comment

tiagobento left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tkobayas Nov 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tiagobento commented Nov 22, 2024

tkobayas commented Nov 25, 2024 • edited Loading

[DO NOT MERGE][WORK IN PROGRESS]kie-issues#1613: Add `CI :: License headers / check` GitHub Actions workflow #2746

[DO NOT MERGE][WORK IN PROGRESS]kie-issues#1613: Add `CI :: License headers / check` GitHub Actions workflow #2746

jomarko Nov 19, 2024 •

edited

Loading

tkobayas Nov 20, 2024 •

edited

Loading

tkobayas Nov 21, 2024 •

edited

Loading

tkobayas commented Nov 25, 2024 •

edited

Loading