Hi,

I know, it's a goal of the project to get rid of using command line interfaces.

After reading issue https://github.com/Qovery/engine/issues/25, I did some research, and I thought it would be useful to start this issue to collect relevant information, and to track progress.

For kubectl u/garypen proposed kube.rs and k8s-openapi.

For the other tools: Bindings most likely also would be useful to the ecosystems of other programming languages as well. Maybe a solution that's based on gRPC, WASM, or similar would make it easier to secure support from the maintainers of these tools?

Basically, there would need to be a definition of the API (i.e. in JSON) and a gRPC server (or similar). New clients then could automatically generate the native API from the protocol definition (which would be almost trivial in Rust with serde).

Go has experimental WASM support.

Terraform already has a gRPC server for plugins, but I'm not sure if it could be used to implement a native API for what the command line tool does.

Another approach would be to use different tools, that have better APIs.

Here is a list of alternatives to Terraform, for example:

https://en.wikipedia.org/wiki/Infrastructure_as_code#Tools

Pulumi looks promising. They have already an SDK for TypeScript, JavaScript, Python, and Go, which they seem to have implemented via a gRPC server.

Currently, no support for Rust is planned, but community contributed support would be welcome.

I found the following resources that show how to implement an SDK for a new language:

• "How can I add support for my favorite language?"
• New Language Bring Up

What do you guys think?

I'd love to use Rust for my Infrastructure as Code needs, but using command line interfaces internally is a dealbreaker for me (and most likely for many other people, as well).

After reading the code, there is a lot of files that are doing command wrapping by doing shell exec behind the scene. While doing shell exec is OK as reimplementing everything of what binaries are doing is

• Tedious and error prone
• Does not bring much value
• Risk of breaking with different version

A big chunk of the code is riddled with system exec with its own had hoc way of parsing output, calling convention and doing the same kind of operation.

To centralize, standardize and isolate those system-exec, consider moving from raw Command::new to a simple interpreter. It will have the benefit of :

• Output is a standardized json
• Command only need to return a result per line and will be automatically turned into an array of string
• Every system exec will happen behind a standard code interface
• System exec will appear only once in the code, inside exec_shell
• Avoid Command::new to creep more into the code
• Ease the test of those function, only need to check script output

Please find below an example of working interpreter. It can still be improved (timeout can be added, be safer) but provide enough flexibility IMHO.

// Operations available for our shell support
pub enum Shell<'a> {
    ExecCmd(&'a str, &'a str),
    ExecCmdWithRetry(&'a str, &'a str, u32),
    EnsureCmd(&'a str),
    SetEnv(&'a str, &'a str),
    SetEnvs(&'a Vec<(&'a str, &'a str)>),

}

// Standardized output for all our shell command, a script output is just a Vec<ShellOutput>
#[derive(Deserialize, Debug)]
pub struct ShellOutput {
    name: String,
    command: String,
    error: String,
    values: Vec<String>
}

// Snippet of shell script that exec a command and format it's output to a json file "ret.json"
pub fn cmd_output_to_json(name: &str, cmd: &str) -> String {
    format!(r#"
##################
# {0}
##################
{1} > ret.stdout 2> ret.stderr
cmd_exit_code=$?
echo '{{ "name": "{0}",' >> ret.json
echo '   "command": "{1}",' >> ret.json
echo '   "error": "'$(cat ret.stderr | tr "\"" "'")'",' >> ret.json
echo '   "values": ['$(cat ret.stdout | xargs -n1 -I R echo '"R"' | paste -sd ',')']' >> ret.json
echo '}}' >> ret.json
echo ',' >> ret.json

[[ $cmd_exit_code -eq 0 ]] || exit 1
"#, name, cmd)
}

// Exec a command with nb max retry
pub fn retry(name: &str, cmd: &str, max_retry: u32) -> String {
    format!(r#"
##################
# RETRY {2} - {0}
##################
COUNTER=0
cmd_exit_code=0
while [  $COUNTER -lt {2} ]; do
    let COUNTER=COUNTER+1
    {1} > ret.stdout 2> ret.stderr
    cmd_exit_code=$?
    [[ $cmd_exit_code -eq 0 ]] && break
    sleep 2
done

echo '{{ "name": "{0}",' >> ret.json
echo '   "command": "{1}",' >> ret.json
echo '   "error": "'$(cat ret.stderr | tr "\"" "'")'",' >> ret.json
echo '   "values": ['$(cat ret.stdout | xargs -n1 -I R echo '"R"' | paste -sd ',')']' >> ret.json
echo '}}' >> ret.json
echo ',' >> ret.json

[[ $cmd_exit_code -eq 0 ]] || exit 1
"#, name, cmd, max_retry)
}

// Transform operation/opcode of our shell into real sh script
fn eval_opcode(action: &Shell, builder: &mut String) {
    match action {
        Shell::ExecCmd(name, args)=> {
            builder.push_str(cmd_output_to_json(name, args).as_str());
        },
        Shell::SetEnv(name, value) => {
            builder.push_str(format!("export {}=\"{}\"\n", name, value).as_str());
        }
        Shell::SetEnvs(envs ) => {
            for (name, value) in envs.iter() {
                builder.push_str(format!("export {}=\"{}\"\n", name, value).as_str());
            }
        }
        Shell::ExecCmdWithRetry(name, cmd, max_retry) => {
            builder.push_str(retry(name, cmd, *max_retry).as_str());
        }
        Shell::EnsureCmd(cmd) => {
            eval_opcode(&Shell::ExecCmd(format!("ensure command {}", cmd).as_str(),
                                        format!("command -v {}", cmd).as_str()),
                        builder)
        }
    };
}

// Generate a complete/executable shell script from our Shell Opcode
pub fn generate_shell(actions: &Vec<Shell>) -> String {

    let mut builder = String::with_capacity(1024);
    builder.push_str("#!/bin/sh
echo '[' >> ret.json
trap \"rm ret.stdout; rm ret.stderr; sed -i '$ d' ret.json ; echo ']' >> ret.json\" EXIT
    ");

    for cmd in vec![EnsureCmd("cat"), EnsureCmd("sed"), EnsureCmd("paste")] {
        eval_opcode(&cmd, &mut builder);
    }
    for cmd in actions {
        eval_opcode(&cmd, &mut builder);
    }

    builder
}

// Execute our shell script
// 1. Generate the shell script and write it into a temporary folder
// 2. Execute the script with a Command/system exec
// 3. Use serde to parse standard json and extract its value
pub fn exec_shell<P: FromStr>(actions: &Vec<Shell>) -> Result<P, SimpleError> 
{
    use std::fs::File;
    use tempfile::tempdir;

    // Generate and write our shell script into a tmp dir (cleaned when variable is droped)
    let dir = tempdir()?;
    let mut script_path = dir.path().join("run.sh");
    File::create(&script_path)?
        .write_all(generate_shell(&actions).into_bytes().as_slice());

    // Exec our script
    let exit = Command::new("sh")
        .arg(&script_path)
        .current_dir(dir.path())
        .output()?;

    // Parse ret.json to extract value of the commands
    let return_file = File::open(dir.path().join("ret.json"))?;
    let results: Vec<ShellOutput> = serde_json::from_reader(BufReader::new(return_file))?;

  // Check run success when empty json
  // Can be turned into None/Option to simplify
    if exit.status.success() && results.is_empty() {
        return P::from_str("").or_else(|err| {
            Err(SimpleError{
                kind: SimpleErrorKind::Other,
                message: Some(format!("Cannot parse string to correct format"))
            })
        });
    }

    if !exit.status.success() && results.is_empty() {
        return Err(SimpleError{
            kind: SimpleErrorKind::Command(exit.status),
            message: Some(format!("Error executing command: {:?}", exit.stderr))
        });
    }

    // Standard case
    for result in results.iter().rev() {
        if !result.error.is_empty() {
            return Err(SimpleError{
                kind: SimpleErrorKind::Command(exit.status),
                message: Some(format!("Error executing command: {:?}", result))
            });
        }
    }

    P::from_str(results.last().unwrap().values.join("").as_str()).or_else( |err|
        Err(SimpleError{
            kind: SimpleErrorKind::Other,
            message: Some(format!("Cannot parse string to correct format"))
        })
    )

}

#[cfg(test)]
mod tests {
    use crate::cmd::kubectl::Shell::{ExecCmd, ExecCmdWithRetry, EnsureCmd, SetEnv};
    use crate::cmd::kubectl::{generate_shell, exec_shell};

    #[test]
    fn test_generate_shell() {
        let actions = vec![
            EnsureCmd("kubectl"), EnsureCmd("docker"),
            SetEnv("KUBECONFIG", "/home/erebe/.kube/config"),
            ExecCmd("get pod number of restart", "kubectl get pods -o=custom-columns=:.status.containerStatuses..restartCount --no-headers=true"),
            ExecCmdWithRetry("list lol", "kubectl get lol", 3),
        ];

        //let script = generate_shell(&actions);
        //println!("{}", script);

        let ret = exec_shell::<String>(&actions);
        println!("{:?}", ret);
    }

}

It generates script like below

#!/bin/sh
echo '[' >> ret.json
trap "rm ret.stdout; rm ret.stderr; sed -i '$ d' ret.json ; echo ']' >> ret.json" EXIT

##################
# ensure command cat
##################
command -v cat > ret.stdout 2> ret.stderr
cmd_exit_code=$?
echo '{ "name": "ensure command cat",' >> ret.json
echo '   "command": "command -v cat",' >> ret.json
echo '   "error": "'$(cat ret.stderr | tr "\"" "'")'",' >> ret.json
echo '   "values": ['$(cat ret.stdout | xargs -n1 -I R echo '"R"' | paste -sd ',')']' >> ret.json
echo '}' >> ret.json
echo ',' >> ret.json

[[ $cmd_exit_code -eq 0 ]] || exit 1

...

With ret.json file looking like

[
{ "name": "ensure command cat",
   "command": "command -v cat",
   "error": "",
   "values": ["/usr/bin/cat"]
}
,
{ "name": "ensure command sed",
   "command": "command -v sed",
   "error": "",
   "values": ["/usr/bin/sed"]
}
,
{ "name": "ensure command paste",
   "command": "command -v paste",
   "error": "",
   "values": ["/usr/bin/paste"]
}
,
{ "name": "ensure command kubectl",
   "command": "command -v kubectl",
   "error": "",
   "values": ["/usr/bin/kubectl"]
}
,
{ "name": "ensure command docker",
   "command": "command -v docker",
   "error": "",
   "values": ["/usr/bin/docker"]
}
,
{ "name": "get pod number of restart",
   "command": "kubectl get pods -o=custom-columns=:.status.containerStatuses..restartCount --no-headers=true",
   "error": "",
   "values": ["0","0","0","0","0","1","0","0"]
}
,
{ "name": "list lol",
   "command": "kubectl get lol",
   "error": "error: the server doesn't have a resource type 'lol'",
   "values": []
}
]

As an example, 2 kubectl commands rewrote with this simple interpreter

pub fn kubectl_exec_get_number_of_restart(
    kubernetes_config: &str,
    namespace: &str,
    podname: &str,
    envs: &Vec<(&str, &str)>,
) -> Result<String, SimpleError> {
    let cmd_str = format!("get pods {} -n {} -o=custom-columns=:.status.containerStatuses..restartCount --no-headers=true", podname, namespace);
    let cmd = vec![
        EnsureCmd("kubectl"),
        SetEnv(KUBECONFIG, kubernetes_config),
        SetEnvs(&envs),
        ExecCmd("pod nb of restart", cmd_str.as_str())
    ];

   exec_shell(&cmd)
}


pub fn kubectl_exec_get_external_ingress_hostnamel(
    kubernetes_config: &str,
    namespace: &str,
    selector: &str,
    envs: &Vec<(&str, &str)>,
) -> Result<String, SimpleError>
{
    let cmd_str = format!("get svc -o json -n {} -l {}", namespace, selector);
    let cmd = vec![
        EnsureCmd("kubectl"),
        SetEnv(KUBECONFIG, kubernetes_config),
        SetEnvs(&envs),
        ExecCmd("get external ingress hostname", cmd_str.as_str())
    ];

    exec_shell(&cmd)
}

// Do the same for terraform, docker, etc...

• In order to implement restart, we want to force rollout even if nothing changes in the deployment
• CREATE action should be renamed DEPLOY to be more insightfull

This PR enables infra pause for AWS:

• Pause will destroy nodes group
• Restore will re-run deploy to get everything deployed
• Updated pod policy to critical elements to ensure they will be correctly restored and prioritized
• Added prometheus adapter to support custom metrics and report the to k8s API
• Checking no jobs are running on the engine before starting pause using custom metrics api
• Refactored terraform force upgrade hack to speed up re-deployments

Bumps thread_local from 1.0.1 to 1.1.4.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps smallvec from 0.6.13 to 0.6.14.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

I wanted to create a database in Qovery. I grabbed the db_url from the UI.

DB_URL = "postgresql://postgres:_Ij=de-6S8B$avgXtFN?@[email protected]:5432/postgres" (Don't worry, this database does not exist in Qovery anymore.)

I'm getting an error.

File "/home/harshitsinghai/.pyenv/versions/3.9.5/lib/python3.9/urllib/parse.py", line 178, in port
    raise ValueError(message) from None
ValueError: Port could not be cast to integer value as '_Ij=de-6S8B$avgXtFN'

I deleted the database, created a new one, and tried again with the new db_url. But I'm getting the same error.

I don't think using postgres as the default DB username is a good idea. It will be confused by many URL parsers.

postgresql://postgres:U8T%2T#YjO#-ep-MjGBcJ_ this string might be creating confusion for lots of parsers.

Either give the user the option to change the default username or use something more generic.

Code Snippet

QOVERY_URL = "postgresql://postgres:U8T%2T#YjO#-ep-MjGBcJ_@!fgczi!%[email protected]:5432/postgres"
database = databases.Database(QOVERY_URL)
metadata = sqlalchemy.MetaData()

engine = sqlalchemy.create_engine(QOVERY_URL)

By enabling this option the controller is going to use a service to know the ingress endpoint/address to use. In our case, it is going to pick the LB hostname thus it is going to avoid the need to add on every ingress target annotation for external DNS with the LB hostname. This reduce the toil to update LB endpoint when it changes, as it is done automatically now

• Change to support mono-repo, aka being able to build image from other directory than the root of the repository.

• Fix: Don't try to use buildkit if the specified Dockerfile is not present in the repository

In my experience Huawei cloud is pretty similar to S3 (though occasionally some things are just ever so slightly different). It would be nice if huawei cloud could be tested and added to the readme if it works. Just because it's probably the most popular cloud provider in china

Bumps regex from 1.5.4 to 1.5.5.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps lock_api from 0.3.4 to 0.4.7.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

In the readme the vote link for the Azure cloud (https://github.com/Qovery/engine/blob/dev/README.md?plain=1#L45) is down. It leads to https://hub.qovery.com/docs/using-qovery/configuration/cloud-service-provider/azure/ which throws a 404

The readme says that you support AWS, Digital Ocean, Azure, GCP and Scaleway. But as I can see it only AWS and Digital Ocean are supported currently? Why are the others also listed under that heading? Same with Build Platforms, Container registries and Monitoring services. This is really confusing when you try to understand if this piece of software is something I'd be interested in using.

It seems that Qovery is heavily inspired by heroku among other things.

I was wondering if Qovery already has a feature similar to heroku-button? And if not, that it would be considered as a new feature.

Issues information

• OS: Linux
• databases: No
• Programming language and version: Python 3.X
• Link to your project on Github/Gitlab: https://github.com/diagrams-web/diagrams-web

Your issue

The list of repositories fetched are not correct with multiple organizations.

Describe here your issue

To prevent duplication and maybe misunderstood please refer to the following thread:

https://community.qovery.com/t/select-a-repo-dont-list-the-repositorie-i-want-to-use/140

I'm not sure if it's the right place to open this issue so feel free to close it or move it to another repository.

Thanks again for the fast reply and support!