实战 Kaggle 比赛:狗的品种识别(ImageNet Dogs)

https://zh-v2.d2l.ai/chapter_computer-vision/kaggle-dog.html

代码运行报错 :grimacing:
TypeError: can’t convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.evaluate_loss

是否应该将
l_sum=l.sum()
修改为
l_sum = l.sum().cpu().detach().numpy()

如下代码:
def evaluate_loss(data_iter, net, devices):
l_sum, n = 0.0, 0
for features, labels in data_iter:
features, labels = features.to(devices[0]), labels.to(devices[0])
outputs = net(features)
l = loss(outputs, labels)
l_sum = l.sum().cpu().detach().numpy()
n += labels.numel()
return l_sum / n

我的话是在train函数中,将animator.add(epoch + 1, (None, valid_loss.detach()))改成animator.add(epoch + 1, (None, valid_loss.cpu().detach()))。原理就是将cuda的tensor先转到cpu上再转成numpy。就能正常运行了。

在代码output = torch.nn.functional.softmax(net(data.to(devices[0])), dim=0)中,dim应该错了,正确的dim应该是1,dim=1,如果你dim=0,submission.csv每一行的概率和都不会等于1

1 Like

采用原始数据集是,运行reorg_dog_data(data_dir, valid_ratio)出错。

OSError Traceback (most recent call last)
/tmp/ipykernel_43/2925177068.py in
6 batch_size = 32 if demo else 128
7 valid_ratio = 0.1
----> 8 reorg_dog_data(data_dir, valid_ratio)

/tmp/ipykernel_43/2925177068.py in reorg_dog_data(data_dir, valid_ratio)
1 def reorg_dog_data(data_dir, valid_ratio):
2 labels = d2l.read_csv_labels(os.path.join(data_dir, ‘labels.csv’))
----> 3 d2l.reorg_train_valid(data_dir, labels, valid_ratio)
4 d2l.reorg_test(data_dir)
5

/opt/conda/lib/python3.7/site-packages/d2l/torch.py in reorg_train_valid(data_dir, labels, valid_ratio)
2062 copyfile(
2063 fname,
-> 2064 os.path.join(data_dir, ‘train_valid_test’, ‘train_valid’, label))
2065 if label not in label_count or label_count[label] < n_valid_per_label:
2066 copyfile(

/opt/conda/lib/python3.7/site-packages/d2l/torch.py in copyfile(filename, target_dir)
2045 def copyfile(filename, target_dir):
2046 “”“Copy a file into a target directory.”""
-> 2047 os.makedirs(target_dir, exist_ok=True)
2048 shutil.copy(filename, target_dir)
2049

/opt/conda/lib/python3.7/os.py in makedirs(name, mode, exist_ok)
211 if head and tail and not path.exists(head):
212 try:
–> 213 makedirs(head, exist_ok=exist_ok)
214 except FileExistsError:
215 # Defeats race condition when another thread created the path

/opt/conda/lib/python3.7/os.py in makedirs(name, mode, exist_ok)
211 if head and tail and not path.exists(head):
212 try:
–> 213 makedirs(head, exist_ok=exist_ok)
214 except FileExistsError:
215 # Defeats race condition when another thread created the path

/opt/conda/lib/python3.7/os.py in makedirs(name, mode, exist_ok)
221 return
222 try:
–> 223 mkdir(name, mode)
224 except OSError:
225 # Cannot rely on checking for EEXIST, since the operating system

OSError: [Errno 30] Read-only file system: ‘…/input/dog-breed-identification/train_valid_test’

  • …/data/dog-breed-identification/labels.csv
  • …/data/dog-breed-identification/sample_submission.csv
  • …/数据/种身份识别/火车
  • …/数据/种身份识别/测试

“火车”是否应该改为“训练”呢?

Kaggle input那个位置的文件只读,代码有个copy的操作,所以没权限,要把文件拷到/kaggle/working这个目录下

1 Like

请问关于“我们选择预训练的ResNet-34模型,我们们只需重复使用此模型的输出层(即提取的特征)的输入”这句话,是否要表达的是把ResNet-34模型的输出作为后面训练模型的输入。如果是这样的话,是否修改为“我们选择预训练的ResNet-34模型,我们只需重复使用此模型的输出层(即提取的特征)作为输入”更合适。
同时,“在计算损失之前,我们首先获取预训练模型的输出层的输入,即提取的特征。”,是否改为“在计算损失之前,我们首先获取预训练模型的输出层作为输入,即提取的特征。”更为合适?

根据原文,原翻译的意思应该是准确的。即重复用到的部分是ResNet-34输出层的输入。

Here, we choose a pretrained ResNet-34 model, where we simply reuse the input of this model’s output layer (i.e., the extracted features).

而且后文提到,

然后,我们可以用一个可以训练的小型自定义输出网络替换原始输出层。

可以看出并不是用ResNet-34模型的输出作为后面训练模型的输入。

如何输出train accuracy以及本地部署