Official code implementation for "Personalized Federated Learning using Hypernetworks"

Dear authors,

It is unclear to me where in your code, you update v_i as in line 11 of Algorithm 1 in your paper (Personalized Federated Learning using Hypernetworks).

In your code (main/experiments/pfedhn/trainer.py), you compute hnet_grads (line 172 in trainer.py), which is \delta_phi \theta^T \delta \theta, used to update \phi in line 10 of Alg 1. Then you assign values of hnet_grads to update hnet weights (line 177 in trainer.py), which include v_i update. In this case, it seems to me that, you update v_i as follows:

v_i = v_i - alpha \delta_phi \theta^T \delta \theta.

I do not see how you have the components of \delta_vi \phi^T as in line 11 of Alg 1 in the paper.

Could you please explain the v_i update?

Thanks a lot!

Dear authors,

When I look closer to Alg 1 and the code, I concern about updating v_i.

As I tested, if I use num_nodes =50, then the emb size is [50,13] and if I use num_nodes =75, then the emb size is [75,19]. It means that if at each iteration, you use one client to update the model, so only one row of the emb will be updated. I also checked the gradient, it is correct with what I understand (only 1 row of the grad has values not 0, and all other rows are 0).

However, in the optimizer, you use SGD with momentum=0.9. In that case, it does update other rows of the emb, not only at the selected client. This is different from the update rule that you have in Alg 1. If I use SGD without momentum, it only updates 1 row as expected (as the same as Alg 1).

Do you have any reason or insights in this case? Why you use 1 client, but update all embedding of all clients in your experiments, which is different from Alg.1?

Thank you for sharing the source code, it is extremely useful and straightforward. This paper will be one of the main references for my future work.

I have one question regarding the algorithm: is there any specific reason for choosing only one user per iteration? I guess one reason is to reduce the computational cost, I would appreciate if you could share your thoughts on my question.

Thank you very much for your time. Please stay safe during this time!

In your paper 5.2 and table 2, clients with different hardware capacity can use different model architectures. I wonder how you implement it using pFedHN. Nice job, thanks!

Hi, I wanna test the results of model size under 75 nodes setting in CIFAR-100, but it

  assert (classes_per_user * num_users) % num_classes == 0, "equal classes appearance is needed"
AssertionError: equal classes appearance is needed

If I ignore this assert, it would appear another error:

    class_partitions['prob'].append([class_dict[i]['prob'].pop() for i in c])
IndexError: pop from empty list

Anything is okay in other node number or in CIFAR-10. Could u plz tell me how to fix it, thx a lot.

Hi,

In Section 5.3, you mentioned the generalization to novel clients. However, I didn't find the implementation of generalization in this git repo. Could you please provide the code or details for reproducing the result in Figure 2?

Thanks!