Current way loss of feddyn is calculated (in utils_general):

loss_algo = alpha_coef * torch.sum(local_par_list * (-avg_mdl_param + local_grad_vector))

I think what you wanted to do (unless I'm missing something) is:

loss_algo = alpha_coef * torch.sum(local_par_list * (0.5*local_par_list - avg_mdl_param + local_grad_vector))

Maybe is much clearer (match paper equation) to write something like:

loss_algo = alpha_coef * torch.sum(local_par_list *  local_grad_vector) + 0.5*alpha_coef*(local_par_list - avg_mdl_param)**2

Feel free to correct me if I'm wrong.

Thanks for your awesome works. I'm trying to re-implement your algorithm. But when I read the source code, I cannot find where is the server state $h^t$.

If I understand correctly, in this line https://github.com/alpemreacar/FedDyn/blob/48a19fac440ef079ce563da8e0c2896f8256fef9/utils_methods.py#L389, the local_param_list_curr is the local grad $\nabla L_k (\theta_k^t)$, cld_mdl_param_tensor is the global model parameter $\theta^{t-1}$, In this line https://github.com/alpemreacar/FedDyn/blob/48a19fac440ef079ce563da8e0c2896f8256fef9/utils_methods.py#L397, the cld_mdl_param is the new global model parameter $\theta^t$, and it seems that the np.mean(local_param_list, axis=0) is the $-\frac{1}{\alpha} h^t$.

Thus, the code means that $h^t = -\frac{\alpha}{m} \sum_{k \in \left[ m \right] } \nabla L_k (\theta_k) $, in which here I ignore the $t$ because the summation is conducted on all clients, we cannot know the timestamp of $\nabla L_k (\theta_k^t)$ because of randomly client selection.

So here the actual $h^t$ is not strictly calculated as $h^t = h^{t-1} - \alpha \frac{1}{m} (\sum_{k\in {P}_t} \theta_k^t - \theta^{t-1} )$.

Current way loss of fedprox is calculated (in utils_general):

loss_algo = mu/2 * torch.sum(local_par_list * local_par_list)
loss_algo = -mu * torch.sum(local_par_list * avg_model_param_)

I think what you wanted to do (unless I'm missing something) is:

loss_algo = mu/2 * torch.sum(local_par_list * local_par_list)
loss_algo = loss_algo - mu * torch.sum(local_par_list * avg_model_param_)

Cheers, F. Varno