# 线性回归的简洁实现

24 Likes

Hi @dongming, 一般我们先call `.backward()` 再call `.grad`。详见https://zh-v2.d2l.ai/chapter_preliminaries/autograd.html

loss function reduction= ‘mean’ , lr=0.03:
epoch 1, loss 0.000361
epoch 2, loss 0.000095
epoch 3, loss 0.000095
w的估计误差： tensor([-0.0002, -0.0002])
b的估计误差： tensor([8.6784e-05])

loss function reduction = ‘sum’, lr =0.03/batch_size=0.003
epoch 1, loss 0.176310
epoch 2, loss 0.091707
epoch 3, loss 0.092358
w的估计误差： tensor([0.0007, 0.0012])
b的估计误差： tensor([0.0005])

2 Likes

mean意味着所有样本损失的平均值，即loss会除以样本数，sum没有除这个样本数，所以会放大1000倍（这里样本数为1000）

1 Like

1.我刚才试验了几遍，由于按batch_size=10抽样本进行训练并计算损失，所以sum方式时应该用lr/10来弥补，即在optim中除10来替代loss fuction中除10，这个思路应该是没有问题的。
2.按mini batch方式训练，样本数并不为1000, 若用lr除1000得到的结果更离谱

2 Likes

num_epochs = 3
for epoch in range(num_epochs):
for X, y in data_iter:
l = loss(net(X) ,y)
l.backward()
trainer.step()
l = loss(net(features), labels)
print(f’epoch {epoch + 1}, loss {l:f}’)

net.parameter() or net[0].weight.requires_grad is set to True by default; while in sgd function, it declares: with torch.no_grad():.

loss function is executed out of the loop for gradient update backwardly. I don’t think it needs come with “torch.no_grad()”.

1 Like
``````num_epochs = 3
for epoch in range(num_epochs):
for X, y in data_iter:
l = loss(net(X) ,y)
l.backward()
trainer.step()
l = loss(net(features), labels)
print(f'epoch {epoch + 1}, loss {l:f}')
``````

l.backward()

``````@torch.no_grad()
def step(self, closure=None):
"""Performs a single optimization step.

Arguments:
closure (callable, optional): A closure that reevaluates the model
and returns the loss.
"""
loss = None
if closure is not None: