Enable Non-Kubernetes Deployments for MLStacks on AWS and GCP

[strickvl]

MLStacks currently supports only Kubernetes-based deployments for stacks, as indicated by the Stack.deployment_method attribute. This approach necessitates the deployment of a Kubernetes cluster for any stack, utilizing Helm or similar tools for installation. To broaden deployment options and cater to a wider range of user needs, this ticket proposes enabling VM-based deployments for MLStacks, specifically targeting AWS and GCP as initial platforms.

Task Description

The task involves creating new Terraform recipes within MLStacks to support deployments on raw VMs or containers backed by persistent storage, without relying on Kubernetes. This addition will require the development of separate recipe folders in src/mlstacks/terraform with tailored definitions for each stack component on AWS and GCP. Leveraging existing AWS modules from the Terraform Registry is recommended to streamline the implementation process.

Expected Outcome

• New deployment options are available within MLStacks for deploying stacks directly on VMs or containerized environments, bypassing Kubernetes for AWS and GCP.
• Creation of new Terraform recipe folders in src/mlstacks/terraform for VM-based deployment definitions on AWS and GCP.
• Successful integration and testing of VM-based deployments for various stack components on AWS and GCP, ensuring functionality and reliability.
• Documentation updates to guide users on leveraging new non-Kubernetes deployment options, including setup, configuration, and usage tips.

Steps to Implement

1. Assess and outline the required stack components for VM-based deployments on AWS and GCP, considering the specifics of each cloud provider.
2. Develop new Terraform recipes for AWS and GCP within src/mlstacks/terraform, focusing on VM-based deployment mechanisms.
3. Leverage existing AWS modules from the Terraform Registry to facilitate the creation of AWS deployments.
4. Ensure that the new deployment methods are fully compatible with MLStacks, including proper integration and configuration of stack components.
5. Conduct comprehensive testing on both AWS and GCP to validate the functionality and reliability of the new deployment options.
6. Update MLStacks documentation to include detailed information on the new non-Kubernetes deployments, offering users clear guidance on how to utilize these options.

Additional Context

By enabling non-Kubernetes deployment options, MLStacks will offer greater flexibility and choice to users in how they deploy and manage their MLOps infrastructure. This enhancement aligns with the goal of making MLStacks a versatile and user-friendly platform for MLOps across diverse cloud environments.

Code of Conduct

• I agree to follow this project's Code of Conduct