For (2) When you try to automatically find the gradient for the
Conv2D class we created, what kind of error message do you see?
I get an error message to the effect of “Inplace operations are not supported using autograd” (exact message below). How do I perform a convolution without pre-allocating the tensor and setting one element at a time?
MXNetError: [05:22:28] src/imperative/imperative.cc:261: Check failed: AGInfo::IsNone(*output): Inplace operations (+=, -=, x[:]=, etc) are not supported when recording with autograd.
Hi @ganeshk, great question! As you may find that In-place-operations may raise an error when the storage of modified inputs is referenced by any other variables. Check here to see how to implement it from scratch!
Thanks @goldpiggy. Did you mean to post the link to the LeNet page in D2L as an example of how to implement in-place operations from scratch? I don’t see such an implementation on that page. Maybe I’m missing it?
Hey @ganeshk. No, LeNet is an example of CNN implementation. If you need want to see the in-place operations from scratch, please go check MXNet/Torch/TF official operators. (We cannot teach from how to build a laptop from scratch )