inheritYamlMergeStrategy setting throws error when using podTemplate

[khosro-copper]

Describe the bug

Trying to upgrade from version 5.1.6 to 5.7.8 with helm, I see this error in jenkins controller:

2024-10-25 20:24:17.703+0000 [id=44]    WARNING    i.j.p.casc.BaseConfigurator#createAttribute: Can't handle class org.csanchez.jenkins.plugins.kubernetes.PodTemplate#listener: type is abstract but not Describable.
2024-10-25 20:24:17.728+0000 [id=44]    WARNING    i.j.p.casc.BaseConfigurator#createAttribute: Can't handle class org.csanchez.jenkins.plugins.kubernetes.PodTemplate#listener: type is abstract but not Describable.
2024-10-25 20:24:17.734+0000 [id=44]    SEVERE    jenkins.InitReactorRunner$1#onTaskFailed: Failed ConfigurationAsCode.init
io.jenkins.plugins.casc.UnknownAttributesException: : Invalid configuration elements for type: class org.csanchez.jenkins.plugins.kubernetes.PodTemplate : inheritYamlMergeStrategy.
Available attributes : activeDeadlineSeconds, activeDeadlineSecondsStr, annotations, containers, envVars, hostNetwork, id, idleMinutes, idleMinutesStr, imagePullSecrets, inheritFrom, instanceCap, instanceCapStr, label, name, namespace, no
deProperties, nodeSelector, nodeUsageMode, podRetention, runAsGroup, runAsUser, schedulerName, serviceAccount, showRawYaml, slaveConnectTimeout, slaveConnectTimeoutStr, supplementalGroups, terminationGracePeriodSeconds, volumes, workspace
Volume, yaml, yamlMergeStrategy, yamls
    at PluginClassLoader for configuration-as-code//io.jenkins.plugins.casc.BaseConfigurator.handleUnknown(BaseConfigurator.java:389)
    at PluginClassLoader for configuration-as-code//io.jenkins.plugins.casc.BaseConfigurator.configure(BaseConfigurator.java:374)
    at PluginClassLoader for configuration-as-code//io.jenkins.plugins.casc.BaseConfigurator.configure(BaseConfigurator.java:277)
    at PluginClassLoader for configuration-as-code//io.jenkins.plugins.casc.impl.configurators.DataBoundConfigurator.configure(DataBoundConfigurator.java:81)
    at PluginClassLoader for configuration-as-code//io.jenkins.plugins.casc.BaseConfigurator.configure(BaseConfigurator.java:355)
    at PluginClassLoader for configuration-as-code//io.jenkins.plugins.casc.BaseConfigurator.configure(BaseConfigurator.java:283)
    at PluginClassLoader for configuration-as-code//io.jenkins.plugins.casc.impl.configurators.DataBoundConfigurator.configure(DataBoundConfigurator.java:81)
    at PluginClassLoader for configuration-as-code//io.jenkins.plugins.casc.impl.configurators.HeteroDescribableConfigurator.lambda$doConfigure$16668e2$1(HeteroDescribableConfigurator.java:311)
    at PluginClassLoader for configuration-as-code//io.vavr.CheckedFunction0.lambda$unchecked$52349c75$1(CheckedFunction0.java:247)
    at PluginClassLoader for configuration-as-code//io.jenkins.plugins.casc.impl.configurators.HeteroDescribableConfigurator.doConfigure(HeteroDescribableConfigurator.java:311)
    at PluginClassLoader for configuration-as-code//io.jenkins.plugins.casc.impl.configurators.HeteroDescribableConfigurator.lambda$configure$2(HeteroDescribableConfigurator.java:88)
    at PluginClassLoader for configuration-as-code//io.vavr.control.Option.map(Option.java:392)
    at PluginClassLoader for configuration-as-code//io.jenkins.plugins.casc.impl.configurators.HeteroDescribableConfigurator.lambda$configure$3(HeteroDescribableConfigurator.java:88)
    at PluginClassLoader for configuration-as-code//io.vavr.Tuple2.apply(Tuple2.java:238)
    at PluginClassLoader for configuration-as-code//io.jenkins.plugins.casc.impl.configurators.HeteroDescribableConfigurator.configure(HeteroDescribableConfigurator.java:86)
    at PluginClassLoader for configuration-as-code//io.jenkins.plugins.casc.impl.configurators.HeteroDescribableConfigurator.check(HeteroDescribableConfigurator.java:94)
    at PluginClassLoader for configuration-as-code//io.jenkins.plugins.casc.impl.configurators.HeteroDescribableConfigurator.check(HeteroDescribableConfigurator.java:57)
    at PluginClassLoader for configuration-as-code//io.jenkins.plugins.casc.BaseConfigurator.configure(BaseConfigurator.java:355)
    at PluginClassLoader for configuration-as-code//io.jenkins.plugins.casc.BaseConfigurator.check(BaseConfigurator.java:293)
    at PluginClassLoader for configuration-as-code//io.jenkins.plugins.casc.ConfigurationAsCode.lambda$checkWith$9(ConfigurationAsCode.java:830)
    at PluginClassLoader for configuration-as-code//io.jenkins.plugins.casc.ConfigurationAsCode.invokeWith(ConfigurationAsCode.java:773)
    at PluginClassLoader for configuration-as-code//io.jenkins.plugins.casc.ConfigurationAsCode.checkWith(ConfigurationAsCode.java:830)
    at PluginClassLoader for configuration-as-code//io.jenkins.plugins.casc.ConfigurationAsCode.configureWith(ConfigurationAsCode.java:816)
    at PluginClassLoader for configuration-as-code//io.jenkins.plugins.casc.ConfigurationAsCode.configureWith(ConfigurationAsCode.java:695)
    at PluginClassLoader for configuration-as-code//io.jenkins.plugins.casc.ConfigurationAsCode.configure(ConfigurationAsCode.java:352)
    at PluginClassLoader for configuration-as-code//io.jenkins.plugins.casc.ConfigurationAsCode.init(ConfigurationAsCode.java:341)
Caused: io.jenkins.plugins.casc.ConfigurationAsCodeBootFailure
    at PluginClassLoader for configuration-as-code//io.jenkins.plugins.casc.ConfigurationAsCode.init(ConfigurationAsCode.java:343)
Caused: java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.base/java.lang.reflect.Method.invoke(Unknown Source)
    at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:109)
Caused: java.lang.Error
    at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:115)
    at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:185)
    at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:305)
    at jenkins.model.Jenkins$5.runTask(Jenkins.java:1175)
    at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:221)
    at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:120)
    at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.base/java.lang.Thread.run(Unknown Source)
2024-10-25 20:24:17.736+0000 [id=26]    SEVERE    hudson.util.BootFailure#publish: Failed to initialize Jenkins
io.jenkins.plugins.casc.UnknownAttributesException: : Invalid configuration elements for type: class org.csanchez.jenkins.plugins.kubernetes.PodTemplate : inheritYamlMergeStrategy.

And I am basically trying to override tolerations definition in the agent pods with overwriting my own definition, something like this:

yamlTemplate: |-
      apiVersion: v1
      kind: Pod
      spec:
        tolerations:
....
        affinity:
....

I am also using Additional agents as well.

Version of Helm and Kubernetes

• Helm: 3.12.1
• Kubernetes: 1.29

Chart version

jenkins-5.7.8

What happened?

...

What you expected to happen?

No response

How to reproduce it

No response

Anything else we need to know?

(inheritYamlMergeStrategy): The error states that inheritYamlMergeStrategy is an invalid attribute in the PodTemplate configuration. In the Kubernetes plugin’s current version, this attribute does not exist as a valid parameter.

Chart also is trying to upgrade kubernetes plugins (which is default behaviour)

• kubernetes:4203.v1dd44f5b_1cf9 
  

• kubernetes:4295.v7fa_01b_309c95
  

And from configuration here: https://plugins.jenkins.io/kubernetes/#plugin-content-configuration-reference I don't see any inheritYamlMergeStrategy.

[khosro-copper]

Any update on this case? based on error message, it seems this is an application bug.

[timja]

cc @Vlatombe any idea?

[khosro-copper]

I see that this feature was add here: a360087
by @james1miller93 on [jenkins-5.2.0]

[khosro-copper]

I see this is added here: https://github.com/jenkinsci/helm-charts/blob/main/charts/jenkins/templates/_helpers.tpl#L538
so, a quick fix would be to make this optional?
However, the root cause is here I guess:

 Invalid configuration elements for type: class org.csanchez.jenkins.plugins.kubernetes.PodTemplate : inheritYamlMergeStrategy.
Available attributes : activeDeadlineSeconds, activeDeadlineSecondsStr, annotations, containers, envVars, hostNetwork, id, idleMinutes, idleMinutesStr, imagePullSecrets, inheritFrom, instanceCap, instanceCapStr, label, name, namespace, no
deProperties, nodeSelector, nodeUsageMode, podRetention, runAsGroup, runAsUser, schedulerName, serviceAccount, showRawYaml, slaveConnectTimeout, slaveConnectTimeoutStr, supplementalGroups, terminationGracePeriodSeconds, volumes, workspace
Volume, yaml, yamlMergeStrategy, yamls

[timja]

are you able to send a PR to revert it?

[khosro-copper]

yes, but I don't want to break others jenkins. To me, it's a Kubernetes plugins issue. I could be wrong.

[james1miller93]

The value was added to support the configuration of taints and tolerations in the top-level pod template. If this is removed from the chart our cluster wont be able to schedule our agent pods.

It is a supported option in the kubernetes-plugin, it's just not mentioned in the documentation.

As @khosro-copper mentioned, it seems like this is a regression in the plugin itself, not the config.

[khosro-copper]

cc @carlossg @Vlatombe for more insight/fix the issue.
Can we also fix documentation in order to avoid confusion? Thanks!

[khosro-copper]

This is basically what helm diff command is trying to do, which then cause the above error:

jenkins, jenkins-jenkins-jcasc-config, ConfigMap (v1) has changed:

  # Source: jenkins/templates/jcasc-config.yaml
  apiVersion: v1
  kind: ConfigMap
  metadata:
    name: "jenkins-jenkins-jcasc-config"
....
                  yaml: |-
                  apiVersion: v1
                  kind: Pod
                  spec:
                    tolerations:
                      - key: key1
                        operator: Equal
                        value: XXX
                        effect: NoSchedule
                    affinity:
                      nodeAffinity:
                        requiredDuringSchedulingIgnoredDuringExecution:
                          nodeSelectorTerms:
                          - matchExpressions:
                            - key: key1
                              operator: In
                              values:
                              - XXX
                yamlMergeStrategy: override
+               inheritYamlMergeStrategy: false

[khosro-copper]

Any update on this bug??

[khosro-copper]

Any update on this issue?
@carlossg @Vlatombe

[khosro-copper]

@timja Who else can help here to fix this issue please?

[timja]

Can you report it directly to the kubernetes plugins issue tracker please?

[Vlatombe]

The attribute was added in jenkinsci/kubernetes-plugin#1513 (so available in 4199.va_1647c280eb_2 and later). To me it looks like the casc configuration is being applied before upgrading the plugin, so it results into an error because the attribute doesn't exist in the version where it is being applied.

The helm chart should make this attribute optional, and before applying it, one should use a recent enough version of the plugin.

[Vlatombe]

Can we also fix documentation in order to avoid confusion? Thanks!

I agree the documentation should have been updated. Can you file a pull request to make it correct?

[timja]

I see I did have a quick look and didn't see when it was added.

So a quick fix would be to update the kubernetes plugin version yourself first.

[khosro-copper]

@Vlatombe Our current Helm chart version is 5.1.6. So, would it be an issue if I upgrade the Kubernetes version first, even though our current Helm chart doesn’t support or recognize inheritYamlMergeStrategy yet?

[timja]

If you upgrade on 5.1.6 then upgrade the helm chart it should work fine

[khosro-copper]

If you upgrade on 5.1.6 then upgrade the helm chart it should work fine

no, the error shown above is coming from upgrading 5.1.6 to 5.7.8

[khosro-copper]

The attribute was added in jenkinsci/kubernetes-plugin#1513 (so available in 4199.va_1647c280eb_2 and later). To me it looks like the casc configuration is being applied before upgrading the plugin, so it results into an error because the attribute doesn't exist in the version where it is being applied.

The helm chart should make this attribute optional, and before applying it, one should use a recent enough version of the plugin.

@james1miller93 As suggested above, can you make this optional with default to True so it wouldn't break others Jenkins.
Though, for this comment:

and before applying it, one should use a recent enough version of the plugin.

I am not sure how this can be achieved in helm chart.

[khosro-copper]

An update here:
Suggested solution worked. Thanks! :
The attribute was added in https://github.com/jenkinsci/kubernetes-plugin/pull/1513 (so available in [4199.va_1647c280eb_2](https://github.com/jenkinsci/kubernetes-plugin/releases/tag/4199.va_1647c280eb_2) and later). To me it looks like the casc configuration is being applied before upgrading the plugin, so it results into an error because the attribute doesn't exist in the version where it is being applied.