feat(hydroflow): Kube integration #1436
Conversation
nickjiang2378
changed the title
nickjiang2378
changed the title
Cargo.toml
Outdated
| @@ -47,9 +47,3 @@ inherits = "release" | |||
| debug = 2 | |||
| lto = "off" | |||
| strip = "none" | |||
|
|
|||
| [profile.dev.package.website_playground] | |||
hydro_deploy/core/Cargo.toml
Outdated
| memo-map = "0.3.0" | ||
| nameof = "1.0.0" | ||
| nanoid = "0.4.0" | ||
| nix = { version = "0.29.0", features = [ "signal" ] } | ||
| once_cell = "1.17" |
There was a problem hiding this comment.
We should check again if these are necessary
hydro_deploy/core/src/deployment.rs
Outdated
| @@ -8,8 +8,8 @@ use tokio::sync::RwLock; | |||
|
|
|||
| use super::gcp::GcpNetwork; | |||
| use super::{ | |||
| progress, CustomService, GcpComputeEngineHost, Host, LocalhostHost, ResourcePool, | |||
| ResourceResult, Service, | |||
| progress, CustomService, GCPComputeEngineHost, Host, LocalhostHost, ResourcePool, | |||
There was a problem hiding this comment.
Looks like this was copied over from before @MingweiSamuel's refactor
| @@ -91,6 +91,11 @@ pub async fn build_crate_memoized(params: BuildParams) -> Result<&'static BuildO | |||
| tokio::task::spawn_blocking(move || { | |||
| let mut command = Command::new("cargo"); | |||
| command.args(["build"]); | |||
| // command.args([ | |||
| // "zigbuild".to_string(), | |||
There was a problem hiding this comment.
Yeah, we want to avoid use of zigbuild. Using the musl target with rust-lld for linking should get us sufficiently far.
hydro_deploy/core/src/gcp.rs
Outdated
| @@ -213,7 +213,7 @@ impl GcpComputeEngineHost { | |||
| #[async_trait] | |||
| impl Host for GcpComputeEngineHost { | |||
| fn target_type(&self) -> HostTargetType { | |||
| HostTargetType::Linux | |||
| HostTargetType::Linux(crate::LinuxArchitecture::AARCH64) | |||
There was a problem hiding this comment.
Hmm, we should probably default to x86_64, and also have a way for users to pass this in when constructing the host (until later when we can auto-infer based on the machine type).
| #[cfg(unix)] | ||
| use std::sync::Arc; | ||
|
|
||
| use async_channel::{Receiver, Sender}; |
There was a problem hiding this comment.
I think @MingweiSamuel replaced this with a tokio builtin if I remember correctly?? Will need to check against the SSH implementation.
| async fn provision(&mut self, _resource_result: &Arc<ResourceResult>) -> Arc<dyn LaunchedHost> { | ||
| if self.launched.is_none() { | ||
| let client = Client::try_default().await.unwrap(); | ||
| let alphabet = [ |
There was a problem hiding this comment.
We should move this to a top-level constant in this file.
| // params.profile.unwrap_or("release".to_string()), | ||
| // ]); | ||
| command.args([ | ||
| "zigbuild".to_string(), |
There was a problem hiding this comment.
This should no longer be needed now that musl is fixed?
| header.set_path(file_name).unwrap(); | ||
| header.set_size(binary_data.len() as u64); | ||
| // header.set_metadata(&metadata); | ||
| header.set_mode(0o755); // give the binary executable permissions |
There was a problem hiding this comment.
I wonder if we could try doing this via chmod after tar is done, if that would help with the I/O errors.
…o kube_integration_rebased
This PR sets up kubernetes within Hydro Deploy. Users can create Pod hosts and deploy their programs on the machines just as they would with localhost and cloud providers. Note that we're assuming here that users will already have a Kubernetes cluster set up somewhere.
We integrate with kube.rs (which integrates with kubectl). When setting up a pod, we first create the pod and loop until it is ready to execute commands. Then, we take the compiled hydroflow binary and copy it into the pod over stdin. We use
lsofto track when files have finished being written to and finally execute the binary on the pod.We're intending this PR to be the start of auto-scaling in Hydro! Kubernetes integration will let users set procedures for vertical / horizontal auto-scaling.