🐛 Bug
Using the quadratic fit example, I thought it would be reasonable to update the data in just aux_vars and re-solve, but it seems like there's a dependence on the global data_x.
Steps to Reproduce
I included a MWE below that outputs the incorrect solution in the middle here by just changing aux_inputs:
optimal a: tensor([[1.0076]], grad_fn=<AddBackward0>)
== Only changing aux_vars["x"] (this should not be the same solution)
optimal a: tensor([[1.0076]], grad_fn=<AddBackward0>)
== Globally updating data_x (this is the correct solution)
optimal a: tensor([[0.0524]], grad_fn=<AddBackward0>)
Expected behavior
I was pretty confused at first when my code using wasn't working and didn't realize it was because of this. We should make updating aux_inputs sufficient to re-solve the problem, or if this is challenging we should consider 1) raising a warning/adding a check with aux_inputs doesn't match or 2) remove duplicated passing of aux_inputs when it doesn't do anything.
Code
#!/usr/bin/env python3
import torch
import theseus as th
import theseus.optimizer.nonlinear as thnl
import numpy as np
import numdifftools as nd
def generate_data(num_points=10, a=1., b=0.5, noise_factor=0.01):
data_x = torch.rand((1, num_points))
noise = torch.randn((1, num_points)) * noise_factor
data_y = a * data_x.square() + b + noise
return data_x, data_y
num_points = 10
data_x, data_y = generate_data(num_points)
x = th.Variable(data_x.requires_grad_(), name="x")
y = th.Variable(data_y.requires_grad_(), name="y")
a = th.Vector(1, name="a")
b = th.Vector(1, name="b")
def quad_error_fn(optim_vars, aux_vars):
a, b = optim_vars
x, y = aux_vars
est = a.data * x.data.square() + b.data
err = y.data - est
return err
optim_vars = a, b
aux_vars = x, y
cost_function = th.AutoDiffCostFunction(
optim_vars, quad_error_fn, num_points, aux_vars=aux_vars, name="quadratic_cost_fn"
)
objective = th.Objective()
objective.add(cost_function)
optimizer = th.GaussNewton(
objective,
max_iterations=15,
step_size=0.5,
)
theseus_inputs = {
"a": 2 * torch.ones((1, 1)).requires_grad_(),
"b": torch.ones((1, 1)).requires_grad_()
}
aux_vars = {
"x": data_x,
"y": data_y,
}
theseus_optim = th.TheseusLayer(optimizer)
updated_inputs, info = theseus_optim.forward(
theseus_inputs, aux_vars=aux_vars,
track_best_solution=True, verbose=False,
backward_mode=thnl.BackwardMode.FULL,
)
print('optimal a: ', updated_inputs['a'])
aux_vars = {
"x": data_x+10.,
"y": data_y,
}
updated_inputs, info = theseus_optim.forward(
theseus_inputs, aux_vars=aux_vars,
track_best_solution=True, verbose=False,
backward_mode=thnl.BackwardMode.FULL,
)
print('== Only changing aux_vars["x"] (this should not be the same solution)')
print('optimal a: ', updated_inputs['a'])
data_x.data += 10.
aux_vars = {
"x": data_x,
"y": data_y,
}
updated_inputs, info = theseus_optim.forward(
theseus_inputs, aux_vars=aux_vars,
track_best_solution=True, verbose=False,
backward_mode=thnl.BackwardMode.FULL,
)
print('== Globally updating data_x (this is the correct solution)')
print('optimal a: ', updated_inputs['a'])
documentation question refactor
239 Jan 2, 2023
130 Dec 2, 2022
8 Jul 13, 2022
60 Nov 14, 2022
428 Dec 24, 2022
3.7k Jan 3, 2023
6 Dec 14, 2022
68 Dec 29, 2022
24 Apr 15, 2021
7.6k Jan 4, 2023
80 Dec 21, 2022
44 Dec 10, 2022
118 Dec 6, 2022
21 Oct 5, 2022
54 Jun 22, 2022
76 Jan 2, 2023
328 Dec 17, 2022