LIU-427: Add private key support for remote submission.

[myxie]

JIRA Ticket

Type

• Feature (addition)

Problem/Issue

When we use Paramiko to make the connection to Slurm in the SlurmClient class, we do not pass a private-key to our remote utility functions. This means that Paramiko will attempt to find a private key file in the ~/.ssh directory, and specifically named id_rsa (or similar).

This is fine if using passwordless-SSH on Linux, but can cause issues if using an alternative SSH server/configuration, and on a different OS (e.g. macOS).

Solution

I have added an option to pass in an ssh_key via the command line and the SlurmClient class. This has the added bonus of adding the path of any private key file through a DALiuGE component (e.g. #298).

Checklist

• [ ] Unittests added
  • This is command-line functionality that does not have a test harness set up, and also requires setting up something to SSH to as part of the unit tests. That's not impossible, but way outside the scope of this work.
• [ ] Documentation added
  • I will add documentation to the new Slurm client docs currently being reviewed in LIU-420: Protoype config.INI and Slurm script templates #297.

Summary by Sourcery

New Features:

• Add support for specifying an SSH private key for remote submissions in the Slurm client.

Summary by Sourcery

New Features:

• Add support for specifying an SSH private key for remote submissions in the Slurm client.

[sourcery-ai]

Reviewer's Guide by Sourcery

This PR adds support for specifying a custom SSH private key when making remote submissions through the Slurm client. The implementation modifies the SlurmClient class and remote utility functions to accept and use a private key path parameter, providing more flexibility for different SSH configurations and operating systems.

Sequence diagram for Remote Submission with SSH Key

sequenceDiagram
    participant User
    participant SlurmClient
    participant Remote
    User->>SlurmClient: Initialize with ssh_key
    SlurmClient->>Remote: execRemote(host, command, username, pkeyPath=ssh_key)
    Remote-->>SlurmClient: Execute command
    SlurmClient->>Remote: copyTo(host, tjob, job_file_name, username, pkeyPath=ssh_key)
    Remote-->>SlurmClient: Copy file
    SlurmClient->>Remote: execRemote(host, sbatch command, username, pkeyPath=ssh_key)
    Remote-->>SlurmClient: Submit job

Loading

Class diagram for SlurmClient with SSH Key Support

classDiagram
    class SlurmClient {
        - String host
        - String username
        - String ssh_key
        + mk_session_dir(String dlg_root)
        + submit_job()
    }
    SlurmClient : +__init__(String facility, String host, Boolean remote, String pip_name, String username, String ssh_key="")
    note for SlurmClient "Added ssh_key attribute and updated methods to use it"

Loading

File-Level Changes

Change	Details	Files
Added SSH private key support to the SlurmClient class	Added ssh_key parameter to class constructor with empty string default Added ssh_key as an instance variable Updated remote execution and file copy operations to use the provided SSH key	daliuge-engine/dlg/deploy/slurm_client.py
Added command-line support for specifying SSH private key	Added --ssh_key command line option with help text Updated main function to pass ssh_key parameter to SlurmClient constructor	daliuge-engine/dlg/deploy/create_dlg_job.py

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time. You can also use
  this command to specify where the summary should be inserted.

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[coveralls]

coverage: 79.795%. remained the same
when pulling 98c0eb4 on LIU-427
into 20b29ec on master.

[sourcery-ai]

Hey @myxie - I've reviewed your changes - here's some feedback:

Overall Comments:

• Consider enhancing the --ssh_key help text to include details about expected key format and common file locations (e.g., ~/.ssh/id_rsa, ~/.ssh/id_ed25519)

Here's what I looked at during the review

• 🟢 General issues: all looks good
• 🟢 Security: all looks good
• 🟢 Testing: all looks good
• 🟢 Complexity: all looks good
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}