Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pywinrm node executor times out at 8 hour mark #55

Open
drodriguez-rundeck opened this issue Jun 26, 2020 · 6 comments
Open

Pywinrm node executor times out at 8 hour mark #55

drodriguez-rundeck opened this issue Jun 26, 2020 · 6 comments

Comments

@drodriguez-rundeck
Copy link

When using "WinRM Node Executor Python" and executing long running job against a remote node, the job fails exactly at the 8 hour mark. It looks like pywinrm has some sort of timeout (8 hours).
The workaround was to use the default ssh as node executor and the pywinrm as file copier

Steps to reproduce
Create a project, set pywinrm as node executor and default file copier, and create a job that runs a long running batch script in a windows remote node.

Inline script on Rundeck server (RPM Installation- 3.1.2)

long_running_job

Batch script to execute on remote Windows node

script_to_execute

Sample execution screenshots for reference

pywinrm1

pywinrm2

@oj43085
Copy link

oj43085 commented Jun 26, 2020

Have you set your timeout properly in winrs at the target server?

winrm get winrm/config

look at the winrs sub-section

@drodriguez-rundeck
Copy link
Author

Hi @oj43085 , thanks for the inputs. Yes, I checked winrm config which I am sharing in this post

winrm_config

If I set WinRM as default node executor, then the job runs successfully for more than 8 hours (next attached image), but if I set Pywinrm as default executor then the job fails exactly after 8 hours. I am not sure but when using Pywinrm plugin, looks like there is some sort of hardcoded timeout

winrm_node_executor

@nvalenzuela20
Copy link
Contributor

I can't reproduce the issue in my local environments using Rundeck in a Linux server connecting to windows 10 machine.
here is the job running more than 8 hours

image

the project configuration

project.plugin.FileCopier.WinRMcpPython.authtype=basic
project.plugin.FileCopier.WinRMcpPython.interpreter=python3
project.plugin.FileCopier.WinRMcpPython.kinit=kinit
project.plugin.FileCopier.WinRMcpPython.krb5config=/etc/krb5.conf
project.plugin.FileCopier.WinRMcpPython.nossl=false
project.plugin.FileCopier.WinRMcpPython.password_storage_path=keys/windows-server/winpass
project.plugin.FileCopier.WinRMcpPython.username=rundeck-windows
project.plugin.FileCopier.WinRMcpPython.winrmport=5985
project.plugin.FileCopier.WinRMcpPython.winrmtransport=http
project.plugin.NodeExecutor.WinRMPython.authtype=basic
project.plugin.NodeExecutor.WinRMPython.interpreter=python3
project.plugin.NodeExecutor.WinRMPython.kinit=kinit
project.plugin.NodeExecutor.WinRMPython.krb5config=/etc/krb5.conf
project.plugin.NodeExecutor.WinRMPython.nossl=false
project.plugin.NodeExecutor.WinRMPython.password_storage_path=keys/windows-server/winpass
project.plugin.NodeExecutor.WinRMPython.shell=powershell
project.plugin.NodeExecutor.WinRMPython.username=rundeck-windows
project.plugin.NodeExecutor.WinRMPython.winrmport=5985
project.plugin.NodeExecutor.WinRMPython.winrmtransport=http

this is the job definition

- defaultTab: nodes
  description: ''
  executionEnabled: true
  id: e66377f8-9d5e-4eeb-a595-3c22178102f5
  loglevel: DEBUG
  name: job-node-test
  nodeFilterEditable: false
  nodefilters:
    dispatch:
      excludePrecedence: true
      keepgoing: false
      rankOrder: ascending`
      successOnEmptyNodeFilter: false
      threadcount: '1'
    filter: 'name: Windows'
  nodesSelectedByDefault: true
  plugins:
    ExecutionLifecycle: {}
  scheduleEnabled: true
  schedules: []
  sequence:
    commands:
    - script: |-
        cd /d "c:\Users\rundeck-windows"
        longloop.cmd
    keepgoing: false
    strategy: node-first
  uuid: e66377f8-9d5e-4eeb-a595-3c22178102f5

and, this is my PowerShell configuration

image (3)

@jtobard
Copy link

jtobard commented Oct 26, 2021

ref: diyan/pywinrm#325 same issue on base pywinrm project

@jgarces-pd
Copy link

Reproduced on 3.3.9

image
Execution Log
Job Definition
winrm config

@BLZB0B
Copy link

BLZB0B commented Oct 29, 2021

I don't think it's pywinrm that's timing out... whatever you've set the max shell runtime to is ignored since WinRm 2.0

https://docs.microsoft.com/en-us/windows/win32/winrm/installation-and-configuration-for-windows-remote-management#maxshellruntime

"WinRM 2.0: The MaxShellRunTime setting is set to read-only. Changing the value for MaxShellRunTime will have no effect on the remote shells."

I suspect if you had winrm 1,0 installed and changed the setting, when upgrading to v2 or v3, this hasn't reset the default so is using the modified value - whereas newer installs will default to 8hrs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants