Convolutions for Images

[mli]

http://d2l.ai/chapter_convolutional-neural-networks/conv-layer.html

[rezahabibi96]

Help me please, I tried using our defined Conv2D class in place of tf.keras.layers.Conv2D. Whenever I specify kernel_size (1, 2) by calling build method on instance of our defined Conv2D class, it works. But then when I call conv2d(X) with X.shape is (1, 6, 8, 1), the weight kernel_size is re initialized to have the shape as X. Is build method called again inside call method? Help me please, thank you. @mli

[jugalraj]

Y_hat computation on section 6.2.4 Learning a Kernel has been repeated. Y_hat computation just above for loop can be omitted.

[qyqstc]

In the Learning a Kernel section, what’s the purpose to multiply a factor 3e-2 to the gradient in the update step.

[StevenJokess]

@qyqstc
I think, 3e-2 is the “learning rate”, which controls the velocity to update.

[goldpiggy]

Hi @qyqstc, great catch! Just fixed in https://github.com/d2l-ai/d2l-en/pull/1706.

[qyqstc]

Thanks for the timely reply!!!

[StevenJokess]

the velocity to optim…

[David_Ang]

Hey,

When I use different X, especially when X is large such as a 120*120 matrix, the kernel learning algorithm(section 6.2.4) explodes and makes a large numbers for loss functions and weights. Can you explain why it happens?

Thanks,

[gopalakrishna-r]

Hi,

You can remove the build method of Conv2d as we can move the ‘build’ method implementation in to the ‘init’ section and have ‘kernel_size’ as a initialization parameter like shown below

class Conv2d(keras.layers.Layer):

def __init__(self, kernel_size):

    super().__init__()

    self.kernel_size = kernel_size

    initializer = tf.random_normal_initializer()

    self.weight = self.add_weight(name = 'w', shape = self.kernel_size, initializer = initializer)

    self.bias = self.add_weight(name = 'b', shape = (1,), initializer = initializer)

   

def call(self , inputs):

    return corr2d(inputs, self.weight) + self.bias

[gopalakrishna-r]

While implementing autodiff for the Conv2d, g.gradient fails to extract gradients for the weight

and spews the error message " ValueError : Attempt to convert a value (None) with an unsupported type (<class ‘NoneType’>) to a Tensor."

~\d2l\lib\site-packages\tensorflow\python\framework\constant_op.py in convert_to_eager_tensor (value, ctx, dtype) 96 dtype = dtypes . as_dtype ( dtype ) . as_datatype_enum 97 ctx . ensure_initialized ( ) —> 98 return ops . EagerTensor ( value , ctx . device_name , dtype ) 99 100 ValueError : Attempt to convert a value (None) with an unsupported type (<class ‘NoneType’>) to a Tensor.

Is the exercise supposed to demonstrate this?