Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: TES struct #36

Closed
wants to merge 94 commits into from
Closed

Feat: TES struct #36

wants to merge 94 commits into from

Conversation

aaravm
Copy link
Collaborator

@aaravm aaravm commented Aug 12, 2024

This PR contains the TES struct.
Please ignore the files:

  • lib/src/configuration.rs
  • lib/src/transport.rs
  • lib/src/serviceinfo/mod.rs

as these files will be merged from #26 and #30, and these files are only present to have the appropriate definitions here as well and analyze only the file lib/src/tes/mod.rs

Summary by Sourcery

Introduce TES struct and related models for managing task execution services, implement Transport struct for HTTP requests, add configuration management, and set up CI/CD workflows and tests.

New Features:

  • Introduce TES struct for managing task execution services, including task creation, status checking, cancellation, and listing tasks.
  • Add support for serializing and deserializing TES-related models such as TesTask, TesState, TesExecutor, and TesResources.
  • Implement a Transport struct for handling HTTP requests with methods for GET, POST, PUT, and DELETE operations.
  • Add a Configuration struct to manage API request configurations, including base path, user agent, and authentication details.

Enhancements:

  • Add utility functions for URL encoding and JSON serialization.
  • Implement error handling and logging for HTTP requests and responses in the Transport struct.

Build:

  • Add build-models.sh script to generate models using OpenAPI specifications.
  • Add rustfmt.toml configuration file for code formatting.

CI:

  • Add GitHub Actions workflows for CI/CD, including local.yml for local testing and ci.yml for GitHub CI/CD.
  • Implement caching for Rust dependencies, build output, Maven dependencies, Go modules, and Funnel dependencies in CI workflows.

Documentation:

  • Add README.md with instructions for building the project, running tests, and testing the CI/CD workflow locally.

Tests:

  • Add unit tests for TES struct methods including task creation, status checking, cancellation, and listing tasks.
  • Add unit tests for Transport struct methods to ensure correct HTTP request handling.
  • Add test_utils module for setting up test environment and ensuring Funnel server is running.

Copy link
Contributor

sourcery-ai bot commented Aug 12, 2024

Reviewer's Guide by Sourcery

This pull request introduces a new TES (Task Execution Service) struct along with its associated models and functionality. The changes include the implementation of the TES struct, configuration settings, transport layer, and various models required for the TES API. Additionally, it sets up the project structure, adds build scripts, test utilities, and CI/CD workflows.

File-Level Changes

Files Changes
lib/src/tes/mod.rs Implemented the TES struct with methods for creating tasks, getting task status, canceling tasks, and listing tasks
lib/src/configuration.rs Added configuration struct for managing API settings
lib/src/transport.rs Implemented transport layer for handling HTTP requests
lib/src/tes/models/mod.rs
lib/src/tes/models/tes_task.rs
lib/src/tes/models/tes_state.rs
lib/src/tes/models/tes_executor.rs
lib/src/tes/models/tes_resources.rs
lib/src/tes/models/tes_input.rs
lib/src/tes/models/tes_output.rs
lib/src/tes/models/tes_task_log.rs
lib/src/tes/models/tes_executor_log.rs
lib/src/tes/models/tes_output_file_log.rs
lib/src/tes/models/tes_file_type.rs
lib/src/tes/models/tes_list_tasks_response.rs
lib/src/tes/models/tes_create_task_response.rs
lib/src/tes/models/tes_service_info.rs
lib/src/tes/models/tes_service_type.rs
Generated models for TES API using OpenAPI specifications
lib/src/serviceinfo/mod.rs Added ServiceInfo implementation for retrieving service information
lib/src/lib.rs
README.md
Set up project structure and dependencies
build-models.sh Added build script for generating models from OpenAPI specifications
lib/src/test_utils.rs
run-tests.sh
nextest.toml
Implemented test utilities and setup for running tests with Funnel
.github/workflows/ci.yml
.github/workflows/local.yml
Added CI/CD workflows for GitHub Actions

Tips
  • Trigger a new Sourcery review by commenting @sourcery-ai review on the pull request.
  • Continue your discussion with Sourcery by replying directly to review comments.
  • You can change your review settings at any time by accessing your dashboard:
    • Enable or disable the Sourcery-generated pull request summary or reviewer's guide;
    • Change the review language;
  • You can always contact us if you have any questions or feedback.

Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @aaravm - I've reviewed your changes and they look great!

Here's what I looked at during the review
  • 🟡 General issues: 4 issues found
  • 🟢 Security: all looks good
  • 🟢 Testing: all looks good
  • 🟢 Complexity: all looks good
  • 🟡 Documentation: 2 issues found

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment to tell me if it was helpful.

Comment on lines +17 to +18
fn serialize_to_json<T: Serialize>(item: T) -> Value {
serde_json::to_value(&item).unwrap()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Consider handling serialization errors more gracefully

Instead of using unwrap(), return a Result to allow proper error handling by the caller. This prevents potential panics and improves the robustness of the library.

Suggested change
fn serialize_to_json<T: Serialize>(item: T) -> Value {
serde_json::to_value(&item).unwrap()
fn serialize_to_json<T: Serialize>(item: T) -> Result<Value, serde_json::Error> {
serde_json::to_value(&item)
}

Comment on lines +21 to +22
pub fn urlencode<T: AsRef<str>>(s: T) -> String {
::url::form_urlencoded::byte_serialize(s.as_ref().as_bytes()).collect()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Update URL encoding method to use a more modern approach

The current method is deprecated. Consider using a more up-to-date approach for URL encoding, such as the percent_encode function from the percent-encoding crate.

pub fn urlencode<T: AsRef<str>>(s: T) -> String {
    use percent_encoding::{utf8_percent_encode, NON_ALPHANUMERIC};
    utf8_percent_encode(s.as_ref(), NON_ALPHANUMERIC).to_string()
}

/// # Returns
///
/// A new instance of Configuration.
pub fn new(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (bug_risk): Add validation for the base_path URL in the Configuration::new method

Adding validation for the base_path URL in the new method would prevent issues down the line. Consider checking if the URL is valid and returning a Result instead of assuming it's always correct.

    pub fn new(base_path: Url) -> Result<Self, url::ParseError> {
        if !base_path.has_host() {
            return Err(url::ParseError::EmptyHost);
        }
        Ok(Self { base_path })
    }

/// The `Configuration` struct is responsible for specifying details of the Endpoint where the requests are made.
/// It provides methods for making constructing new configuration, changing the base url, and specifying a default configuration.
#[derive(Debug, Clone)]
pub struct Configuration {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Consider using more specific types for some Configuration fields

Some fields in the Configuration struct are using Option<String>. Consider using more specific types for some of these, like Option<Url> for base_path. This would provide better type safety and make the intended use of these fields clearer.

pub struct Configuration {
    /// The base path for API requests.
    pub base_path: Url,
    pub user_agent: Option<String>,
    pub client: reqwest::Client,
    pub basic_auth: Option<BasicAuth>,
    pub oauth_access_token: Option<String>,
    pub bearer_access_token: Option<String>,
}


## Running the tests

Before running the tests, you need to install Funnel, a task execution system that is compatible with the GA4GH TES API. Follow the instructions in the [Funnel Developer's Guide](https://ohsu-comp-bio.github.io/funnel/docs/development/developers/) to install Funnel.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (documentation): Consider adding a brief description of Funnel.

Adding a brief description of Funnel can help users quickly understand what it is without needing to follow the link.

Suggested change
Before running the tests, you need to install Funnel, a task execution system that is compatible with the GA4GH TES API. Follow the instructions in the [Funnel Developer's Guide](https://ohsu-comp-bio.github.io/funnel/docs/development/developers/) to install Funnel.
Before running the tests, you need to install Funnel. Funnel is a lightweight, open-source task execution system that implements the GA4GH Task Execution Service (TES) API. It allows you to run scientific workflows and batch jobs across various compute environments. Follow the instructions in the [Funnel Developer's Guide](https://ohsu-comp-bio.github.io/funnel/docs/development/developers/) to install Funnel.

```
cargo nextest run
```
For checking the unit coverage, you can run:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (documentation): Consider changing 'unit coverage' to 'unit test coverage'.

Changing 'unit coverage' to 'unit test coverage' can improve clarity.

Suggested change
For checking the unit coverage, you can run:
For checking the unit test coverage, you can run:

@aaravm aaravm closed this Aug 12, 2024
@aaravm
Copy link
Collaborator Author

aaravm commented Aug 12, 2024

Lots of unnecessary files got added

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants