kie-issues#2659: [SonataFlow] Migrate DBMigration Image from SonataFlow operator repository to kie-tools

.gitignore

@@ -346,3 +346,7 @@ turbo.json
 # IntelliJ projects should be on _intellij-projects folder, never on specific packages.
 packages/*/.idea
 examples/*/.idea
+
+# kogito-db-migrator-tool
+packages/kogito-db-migrator-tool/src/main/resources/postgresql
+packages/kogito-db-migrator-tool/src/main/resources/ansi

packages/kogito-db-migrator-tool-image/README.md

@@ -0,0 +1,116 @@
+<!--
+   Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+     http://www.apache.org/licenses/LICENSE-2.0
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+-->
+
+# Kogito Postgres DB Migrator Tool Image
+
+This package contains the `Containerfile/Dockerfile` and scripts to build a container image for Kogito Postgres DB Migrator Tool. Details about the Kogito Postgres DB Migrator Tool can be found [here](../kogito-db-migrator-tool/README.md)
+
+## Additional requirements
+
+- docker
+
+## Build
+
+_NOTE_: Before performing this step, be sure that the Kogito Postgres DB Migrator Tool jar has been built and available for inclusion in the image.
+
+- Enable the image to be built:
+
+  ```bash
+  export KIE_TOOLS_BUILD__buildContainerImages=true
+  ```
+
+Run the following in the root folder of the repository to build the package:
+
+```bash
+pnpm -F @kie-tools/kogito-db-migrator-tool-image... build:prod
+```
+
+- Then check if the image is correctly stored:
+
+  ```bash
+  docker images
+  ```
+
+## Run
+
+- Start up a clean container with:
+
+  ```bash
+    docker run docker.io/apache/incubator-kie-kogito-service-db-migration-postgresql:latest
+  ```
+
+## Customization
+
+1. Run a container with custom environment variables:
+
+| NAME                                    | DESCRIPTION                                                                      | DEFAULT                                   |
+| --------------------------------------- | -------------------------------------------------------------------------------- | ----------------------------------------- |
+| MIGRATE_DB_DATAINDEX                    | Set to true if you want to migrate data index database, set to false otherwise   | false                                     |
+| QUARKUS_DATASOURCE_DATAINDEX_JDBC_URL   | Data index database url e.g. jdbc:postgresql://host.docker.internal:5432/di      | jdbc:postgresql://localhost:5432/postgres |
+| QUARKUS_DATASOURCE_DATAINDEX_USERNAME   | Data index database username                                                     | postgres                                  |
+| QUARKUS_DATASOURCE_DATAINDEX_PASSWORD   | Data index database password                                                     | postgres                                  |
+| QUARKUS_FLYWAY_DATAINDEX_SCHEMAS        | Data index database schema                                                       | dataindex                                 |
+| MIGRATE_DB_JOBSSERVICE                  | Set to true if you want to migrate jobs service database, set to false otherwise | false                                     |
+| QUARKUS_DATASOURCE_JOBSSERVICE_JDBC_URL | Jobs service database url e.g. jdbc:postgresql://host.docker.internal:5432/js    | jdbc:postgresql://localhost:5432/postgres |
+| QUARKUS_DATASOURCE_JOBSSERVICE_USERNAME | Jobs service database username                                                   | postgres                                  |
+| QUARKUS_DATASOURCE_JOBSSERVICE_PASSWORD | Jobs service database password                                                   | postgres                                  |
+| QUARKUS_FLYWAY_JOBSSERVICE_SCHEMAS      | Jobs service database schema                                                     | jobsservice                               |
+
+### Example
+
+An example to use diverse environment variables
+
+```bash
+   docker run \
+   --env MIGRATE_DB_DATAINDEX=true \
+   --env QUARKUS_DATASOURCE_DATAINDEX_JDBC_URL=<data-index-db-url e.g. jdbc:postgresql://host.docker.internal:5432/di> \
+   --env QUARKUS_DATASOURCE_DATAINDEX_USERNAME=<data-index-db-user> \
+   --env QUARKUS_DATASOURCE_DATAINDEX_PASSWORD=<data-index-db-password> \
+   --env QUARKUS_FLYWAY_DATAINDEX_SCHEMAS=dataindex \
+   --env MIGRATE_DB_JOBSSERVICE=true \
+   --env QUARKUS_DATASOURCE_JOBSSERVICE_JDBC_URL=<jobs-service-db-url e.g. jdbc:postgresql://host.docker.internal:5432/js> \
+   --env QUARKUS_DATASOURCE_JOBSSERVICE_USERNAME=<jobs-service-db-user> \
+   --env QUARKUS_DATASOURCE_JOBSSERVICE_PASSWORD=<jobs-service-db-password> \
+   --env QUARKUS_FLYWAY_JOBSSERVICE_SCHEMAS=jobsservice \
+   docker.io/apache/incubator-kie-kogito-service-db-migration-postgresql:999-SNAPSHOT
+```
+
+---
+
+Apache KIE (incubating) is an effort undergoing incubation at The Apache Software
+Foundation (ASF), sponsored by the name of Apache Incubator. Incubation is
+required of all newly accepted projects until a further review indicates that
+the infrastructure, communications, and decision making process have stabilized
+in a manner consistent with other successful ASF projects. While incubation
+status is not necessarily a reflection of the completeness or stability of the
+code, it does indicate that the project has yet to be fully endorsed by the ASF.
+
+Some of the incubating project’s releases may not be fully compliant with ASF
+policy. For example, releases may have incomplete or un-reviewed licensing
+conditions. What follows is a list of known issues the project is currently
+aware of (note that this list, by definition, is likely to be incomplete):
+
+- Hibernate, an LGPL project, is being used. Hibernate is in the process of
+  relicensing to ASL v2
+- Some files, particularly test files, and those not supporting comments, may
+  be missing the ASF Licensing Header
+
+If you are planning to incorporate this work into your product/project, please
+be aware that you will need to conduct a thorough licensing review to determine
+the overall implications of including this work. For the current status of this
+project through the Apache Incubator visit:
+https://incubator.apache.org/projects/kie.html

packages/kogito-db-migrator-tool-image/env/index.js

@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+const { varsWithName, composeEnv, getOrDefault } = require("@kie-tools-scripts/build-env");
+
+const rootEnv = require("@kie-tools/root-env/env");
+
+module.exports = composeEnv([rootEnv], {
+  vars: varsWithName({
+    KOGITO_DB_MIGRATOR_TOOL_IMAGE__registry: {
+      default: "docker.io",
+      description: "The image registry.",
+    },
+    KOGITO_DB_MIGRATOR_TOOL_IMAGE__account: {
+      default: "apache",
+      description: "The image registry account.",
+    },
+    KOGITO_DB_MIGRATOR_TOOL_IMAGE__name: {
+      default: "incubator-kie-kogito-db-migrator-tool",
+      description: "The image name.",
+    },
+    KOGITO_DB_MIGRATOR_TOOL_IMAGE__buildTag: {
+      default: rootEnv.env.root.streamName,
+      description: "The image tag.",
+    },
+  }),
+  get env() {
+    return {
+      kogitoDbMigratorToolImage: {
+        registry: getOrDefault(this.vars.KOGITO_DB_MIGRATOR_TOOL_IMAGE__registry),
+        account: getOrDefault(this.vars.KOGITO_DB_MIGRATOR_TOOL_IMAGE__account),
+        name: getOrDefault(this.vars.KOGITO_DB_MIGRATOR_TOOL_IMAGE__name),
+        buildTag: getOrDefault(this.vars.KOGITO_DB_MIGRATOR_TOOL_IMAGE__buildTag),
+      },
+    };
+  },
+});

packages/kogito-db-migrator-tool-image/install.js

@@ -0,0 +1,64 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+const { execSync } = require("child_process");
+const fs = require("fs");
+
+const { env } = require("./env");
+const path = require("path");
+const pythonVenvDir = path.dirname(require.resolve("@kie-tools/python-venv/package.json"));
+const sonataflowImageCommonDir = path.dirname(require.resolve("@kie-tools/sonataflow-image-common/package.json"));
+const replaceInFile = require("replace-in-file");
+
+const activateCmd =
+  process.platform === "win32"
+    ? `${pythonVenvDir}\\venv\\Scripts\\Activate.bat`
+    : `. ${pythonVenvDir}/venv/bin/activate`;
+
+execSync(
+  `${activateCmd} && \
+  python3 ${sonataflowImageCommonDir}/resources/scripts/versions_manager.py --bump-to ${env.kogitoDbMigratorToolImage.buildTag} --source-folder ./resources`,
+  { stdio: "inherit" }
+);
+
+// Find and read the -image.yaml file
+const resourcesPath = path.resolve(__dirname, "./resources");
+const files = fs.readdirSync(resourcesPath);
+const imageYamlFiles = files.filter((fileName) => fileName.endsWith("image.yaml"));
+if (imageYamlFiles.length !== 1) {
+  throw new Error("There should only be one image.yaml file on ./resources!");
+}
+const originalYamlPath = path.join(resourcesPath, imageYamlFiles[0]);
+let imageYaml = fs.readFileSync(originalYamlPath, "utf8");
+
+const imageUrl = `${env.kogitoDbMigratorToolImage.registry}/${env.kogitoDbMigratorToolImage.account}/${env.kogitoDbMigratorToolImage.name}`;
+
+// Replace the whole string between quotes ("") with the image name
+imageYaml = imageYaml.replace(/(?<=")(.*incubator-kie-kogito-service-db-migration-postgresql.*)(?=")/gm, imageUrl);
+
+// Write file and then rename it to match the image name
+fs.writeFileSync(originalYamlPath, imageYaml);
+fs.renameSync(originalYamlPath, path.join(resourcesPath, `${env.kogitoDbMigratorToolImage.name}-image.yaml`));
+
+// Replace image URL in .feature files
+replaceInFile.sync({
+  files: ["**/*.feature"],
+  from: /@docker.io\/apache\/.*/g,
+  to: `@${imageUrl}`,
+});

packages/kogito-db-migrator-tool-image/package.json

@@ -0,0 +1,40 @@
+{
+  "private": true,
+  "name": "@kie-tools/kogito-db-migrator-tool-image",
+  "version": "0.0.0",
+  "description": "",
+  "license": "Apache-2.0",
+  "homepage": "https://github.com/apache/incubator-kie-tools",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/apache/incubator-kie-tools.git"
+  },
+  "bugs": {
+    "url": "https://github.com/apache/incubator-kie-tools/issues"
+  },
+  "scripts": {
+    "build": "pnpm copy-assets && pnpm image:build && rimraf build && rimraf target && rimraf dist-tests-e2e && rimraf resources/modules/kogito-postgres-db-migration-deps/quarkus-app",
+    "build:dev": "pnpm build",
+    "build:prod": "pnpm build",
+    "copy-assets": "run-script-os",
+    "copy-assets:linux:darwin": "rimraf build && cp -R ./node_modules/@kie-tools/sonataflow-image-common/resources build && mkdir -p resources/modules/kogito-postgres-db-migration-deps/quarkus-app && cp -R ./node_modules/@kie-tools/kogito-db-migrator-tool/target/quarkus-app/* resources/modules/kogito-postgres-db-migration-deps/quarkus-app && cp -R resources/* build ",
+    "format": "prettier --write . --ignore-path=../../.prettierignore --ignore-path=../../.gitignore",
+    "image:build": "run-script-os",
+    "image:build:darwin:linux": "pnpm setup:env && cekit --descriptor build/incubator-kie-kogito-db-migrator-tool-image.yaml -v build docker",
+    "image:build:win32": "echo \"Build skipped on Windows\"",
+    "install": "node install.js && pnpm format",
+    "setup:env": ". ./node_modules/@kie-tools/python-venv/venv/bin/activate && cross-env KOGITO_IMAGE_REGISTRY=$(build-env kogitoDbMigratorToolImage.registry) KOGITO_IMAGE_REGISTRY_ACCOUNT=$(build-env kogitoDbMigratorToolImage.account) KOGITO_IMAGE_NAME=$(build-env kogitoDbMigratorToolImage.name) KOGITO_IMAGE_TAG=$(build-env kogitoDbMigratorToolImage.buildTag) QUARKUS_PLATFORM_VERSION=$(build-env versions.quarkus) KOGITO_VERSION=$(build-env versions.kogito)"
+  },
+  "dependencies": {
+    "@kie-tools/kogito-db-migrator-tool": "workspace:*",
+    "@kie-tools/python-venv": "workspace:*",
+    "@kie-tools/root-env": "workspace:*",
+    "@kie-tools/sonataflow-image-common": "workspace:*"
+  },
+  "devDependencies": {
+    "cross-env": "^7.0.3",
+    "replace-in-file": "^7.1.0",
+    "rimraf": "^3.0.2",
+    "run-script-os": "^1.1.6"
+  }
+}

packages/kogito-db-migrator-tool-image/resources/incubator-kie-kogito-db-migrator-tool-image.yaml

@@ -0,0 +1,49 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+name: "docker.io/apache/incubator-kie-kogito-db-migrator-tool"
+version: "main"
+from: registry.access.redhat.com/ubi8/openjdk-17-runtime:1.20
+description: Flyway image for Data Index and Jobs Service database migration
+
+labels:
+  - name: "org.kie.kogito.version"
+    value: "### SET ME DURING BUILD PROCESS ###"
+  - name: "maintainer"
+    value: "Apache KIE <[email protected]>"
+  - name: "io.k8s.description"
+    value: "Kogito DB Migration creates schemas and tables for Data Index and Jobs Service for PostgreSQL database"
+  - name: "io.k8s.display-name"
+    value: "Kogito DB Migration for Data Index and Jobs Service - PostgreSQL"
+  - name: "io.openshift.tags"
+    value: "kogito,db-migration"
+
+modules:
+  repositories:
+    - path: modules
+  install:
+    - name: org.kie.kogito.system.user
+    - name: org.kie.kogito.postgresql.db-migration-deps
+
+run:
+  workdir: "/home/kogito/bin"
+  entrypoint:
+    - "java"
+  cmd:
+    - "-jar"
+    - "./quarkus-run.jar"

packages/kogito-db-migrator-tool-image/resources/modules/kogito-postgres-db-migration-deps/module.yaml

@@ -0,0 +1,25 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+schema_version: 1
+name: org.kie.kogito.postgresql.db-migration-deps
+version: "main"
+artifacts:
+  - name: sonataflow-db-migrator-quarkus-app
+    path: ./quarkus-app
+    dest: /home/kogito/bin

packages/kogito-db-migrator-tool/README.md

@@ -0,0 +1,31 @@
+# Kogito Postgres DB Migrator Tool
+
+This is a Java, Quarkus-based PostgreSQL database migrator application for Data-Index and Jobs Service applications for use by SonataFlow Operator.
+
+_NOTE_: This postgres database migrator application and its corresponding images are only envisaged to be made use of by SonataFlow Operator, Data Index and Jobs Service internally. Conversely this application is of no use outside the usecases involved with SonataFlow Operator, Data Index and Jobs Service.
+
+---
+
+Apache KIE (incubating) is an effort undergoing incubation at The Apache Software
+Foundation (ASF), sponsored by the name of Apache Incubator. Incubation is
+required of all newly accepted projects until a further review indicates that
+the infrastructure, communications, and decision making process have stabilized
+in a manner consistent with other successful ASF projects. While incubation
+status is not necessarily a reflection of the completeness or stability of the
+code, it does indicate that the project has yet to be fully endorsed by the ASF.
+
+Some of the incubating project’s releases may not be fully compliant with ASF
+policy. For example, releases may have incomplete or un-reviewed licensing
+conditions. What follows is a list of known issues the project is currently
+aware of (note that this list, by definition, is likely to be incomplete):
+
+- Hibernate, an LGPL project, is being used. Hibernate is in the process of
+  relicensing to ASL v2
+- Some files, particularly test files, and those not supporting comments, may
+  be missing the ASF Licensing Header
+
+If you are planning to incorporate this work into your product/project, please
+be aware that you will need to conduct a thorough licensing review to determine
+the overall implications of including this work. For the current status of this
+project through the Apache Incubator visit:
+https://incubator.apache.org/projects/kie.html