Now that HyperQueue supports disabling hyperthreading (see https://github.com/It4innovations/hyperqueue/pull/260), we should add support for starting allocations that use this feature, and update the install instructions once the next release is out.

This comment justifies why only time-request was used. https://github.com/aiidateam/aiida-hyperqueue/blob/e33376c04b456d4e7f440fff0356f66fbd7c60da/aiida_hyperqueue/scheduler.py#L125-L130

However, I think both should be used, and set to the same value. It's expected that schedulers kill the job if this takes too long. Actually, this is even more important when sharing a node: I just had a case in which, for some reasons, all jobs in a node remained stuck and stopped producing output, even if they were still using 100% of the CPU. They blocked the worker until the end of its wall time. This means that if e.g. the worker has 24h of wall time, 24h are wasted even if the job should finish within 10 minutes. It's better to kill it and let other jobs go in.

Added the backlog and workers option to the CLI when requesting allocations, and update the internal syntax to follow the changed commands in HQv.0.11.0

See e.g. docs here: https://it4innovations.github.io/hyperqueue/latest/cli/output-mode/#json

and PR with comments here: https://github.com/It4innovations/hyperqueue/pull/270

I think it should be used at least for hq jobs and hq submit (and possibly hq delete if it is also supported there?)

We should limit the number of different JobResource subclasses used by different scheduler plugins, I think, because these make different schedulers behave differently and so it's harder for the user to know which resources we pass.

For this scheduler, we clearly need to specify the total number of cores.

Memory can probably be removed as discussed in #7

Do we need a different class, and in particular both num_mpiprocs and num_cores? Or can we just reuse e.g. this below (ParEnvJobResource), simply specifying the tot_num_mpiprocs? (and a parallel_env, which is a string - I imagine this would be matched in the future to the name of the alloc on which you want to run - e.g. GPU vs CPU etc.).

https://github.com/aiidateam/aiida-core/blob/ff1318b485a8b803e115b78946cc4593fc661153/aiida/schedulers/datastructures.py#L177

Instead of defining a new memory_Mb in the resources (very specific to this scheduler), we should reuse concepts already into AiiDA and independent of the scheduler.

E.g. we have metadata.options.max_memory_kb in CalcJobs:

https://github.com/aiidateam/aiida-core/blob/ff1318b485a8b803e115b78946cc4593fc661153/aiida/engine/processes/calcjobs/calcjob.py#L249

and this is passed to the scheduler in the JobTemplate:

https://github.com/aiidateam/aiida-core/blob/ff1318b485a8b803e115b78946cc4593fc661153/aiida/schedulers/datastructures.py#L284

See e.g. how it's used in SLURM:

https://github.com/aiidateam/aiida-core/blob/develop/aiida/schedulers/plugins/slurm.py#L383-L396

There are a few points where we change the defaults: number of backlog, and use of hyper threading. I suggest we stick to the same defaults of HQ.

• HT used by default; have a --no-ht flag instead
• don't set the backlog by default, and specify it only if the user passes an option

(and any other change we are actually doing to the defaults). Otherwise a user will get confused when using directly HQ vs using it via AiiDA