Padding and Stride

English Version pytorch
1

http://d2l.ai/chapter_convolutional-neural-networks/padding-and-strides.html

2

Q:
1.
4 * 4:

image
image825×723 68.7 KB

2 * 2:
image
image992×818 109 KB

2. TODO:
3. stride each 2*sampling rate?
4. Look big picture. To avoid overfitting and accerlate the training.

3

Hi all.
This is self-paced course for me to understand more about the ML.
If it is NOT allowed to add comments/questions except for the officially registered students, I will not write anything here further.

@StevenJokes
3. stride each 2*sampling rate
You mean down sampling by 2?

@Everybody
4. Cannot understand the question clearly.
Just saying that it can reduce the calculation of hidden layer seems to be too simple & obvious answer. Expecting other answers?

4

@Nicholas_Kim
I guess it is. I’m not sure about it.

5

Hi @Nicholas_Kim,

You are almost right! Strides help remove unnecessary computational burdensome and help to reduce the feature dimension more quickly.

6

Hi all,
In the paragraph after equation 6.3.1 it says:
Assuming that 𝑘ℎ is odd here, we will pad 𝑝ℎ/2 rows on both sides of the height. If 𝑘ℎ is even, one possibility is to pad ⌈𝑝ℎ/2⌉ rows on the top of the input and ⌊𝑝ℎ/2⌋ rows on the bottom. We will pad both sides of the width in the same way.
Is even and odd switched here? should it be
Assuming that 𝑘ℎ is even here, we will pad 𝑝ℎ/2 rows on both sides of the height. If 𝑘ℎ is odd, one possibility is to pad ⌈𝑝ℎ/2⌉ rows on the top of the input and ⌊𝑝ℎ/2⌋ rows on the bottom. We will pad both sides of the width in the same way.

Ignore- It shouldn’t be cause we are preserving the input dimension.

7

Exercises

  1. For the last example in this section, use mathematics to calculate the output shape to see if
    it is consistent with the experimental result.
  • it is consistent, |(8 -3 + 0 + 3)/3| , |(8-5+1+4)/4|
  1. Try other padding and stride combinations on the experiments in this section.
  • hmm tried
  1. For audio signals, what does a stride of 2 correspond to?
  • it might be two time peridod long
  1. What are the computational benefits of a stride larger than 1
  • efficiency in calculation,downsampling
8

I think you need to change the equ. 6.3.2. If we use padding there should 2ph and 2pw as we pad 0 to top bottom and left right respectively, So the output shape should be something like this:
((nh - kh + 2ph + sh) // sh) x ((nw-kw+2pw+sw) // sw).

9

My solutions to the exs: 7.3

2 Likes