Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[10/n][pipeline-gen] Add helper method to convert test step into Buildkite step #50

Merged
merged 5 commits into from
Oct 29, 2024

Conversation

khluu
Copy link
Collaborator

@khluu khluu commented Oct 24, 2024

  • Add a method to get plugin config upon whether GPU type is provided and which type of GPU is requested. Since A100 GPU is on EKS, all tests with A100 GPU would be using K8s plugin and all other tests run with Docker plugin.
  • Add a method to convert TestStep into BuildkiteStep based on whether the test is a regular one or a multi node one (since multi-node test uses a custom script to run)
  • The helper method will be used in PipelineGenerator where the test steps are processed into BuildkiteStep and BuildkiteBlockStep. The next PR will add the method since it contains some logic that is better to put in a separate PR. The logic is here: https://github.com/vllm-project/buildkite-ci/pull/36/files#diff-82edd9d64d3e791b9faade2e3aff441908121926000ca52bd54a8f17c976f4d0R120

p
Signed-off-by: kevin <[email protected]>
p
Signed-off-by: kevin <[email protected]>
p
Signed-off-by: kevin <[email protected]>
p
Signed-off-by: kevin <[email protected]>
@khluu khluu requested a review from aslonnie October 24, 2024 21:27
@@ -54,6 +54,7 @@ class DockerPluginConfig(BaseModel):
command: List[str] = Field(default_factory=list)
environment: List[str] = DEFAULT_DOCKER_ENVIRONMENT_VARIBLES
volumes: List[str] = DEFAULT_DOCKER_VOLUMES
shell: List[str] = ["/bin/bash", "-c"]
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file contains changes from https://github.com/vllm-project/buildkite-ci/pull/49/files which would be rebased after that PR is merged to main

@khluu khluu merged commit 49ee880 into main Oct 29, 2024
2 checks passed
@khluu khluu deleted the khluu/pipeline_gen_convert_test_steps branch October 29, 2024 18:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants