Convolutional Neural Networks (LeNet)

English Version pytorch
1

http://d2l.ai/chapter_convolutional-neural-networks/lenet.html

3

in Exercises 4 : we can see in the deepth of the network, the higher feature is sparse.

image
image1261Ă—732 134 KB

1 Like
4

@min_xu
Hey i am having issues understanding how to tackle exercise number 4. Could you share the code you used to display this, or explain how to do it?
eg. i dont understand what data references in the image you posted
THank you

5

hi!
we want to do

  1. Display the activations of the first and second layer of LeNet for different inputs (e.g., sweaters and coats).

so we can check the network

image
image866Ă—697 35.2 KB

so the output of the first activations of net is net[0:3]
and the output of the second activations of net is net[0:6]

and the code is in this map,

image
image1467Ă—854 241 KB

1 Like
6

Hi Can anyone help with this . If we replace the init code

def init_weights(m):
        if type(m) == nn.Linear or type(m) == nn.Conv2d:
            nn.init.xavier_uniform_(m.weight)

    net.apply(init_weights)

with this 

for layer in model:
  if type(layer) == nn.Linear or type(layer) == nn.Conv2d:
    nn.init.xavier_uniform_(layer.weight)

Why does not it work? @goldpiggy or @anirudh any idea?

1 Like
7

What do you mean by “it does not work”? In what way does it not work?

If you can show what you expected to happen vs. what actually happened, the experts can try to help with it, and others (like me) can learn from it. Just saying “it didn’t work”, doesn’t work!

8

Exercises and my weird answers

  1. Replace the average pooling with maximum pooling. What happens?

    • has the almost the same .8177 and .8148

  2. Try to construct a more complex network based on LeNet to improve its accuracy.

    1. Adjust the convolution window size.

    2. Adjust the number of output channels.

    3. Adjust the activation function (e.g., ReLU).

    4. Adjust the number of convolution layers.

    5. Adjust the number of fully connected layers.

    6. Adjust the learning rates and other training details (e.g., initialization and number of epochs.)

    • my changed network.
    
lenet_improved = nn.Sequential(nn.Conv2d(1, 6, kernel_size=5, padding=2), nn.ReLU(),

                nn.AvgPool2d(kernel_size=2, stride=2),

                nn.Conv2d(6, 16, kernel_size=5), nn.ReLU(),

                nn.AvgPool2d(kernel_size=2, stride=2),

                nn.Conv2d(16, 32, kernel_size=4), nn.ReLU(),

                nn.AvgPool2d(kernel_size=2, stride=2), nn.Flatten(),

                nn.Linear(32, 240), nn.ReLU(),

                nn.Linear(240,120), nn.ReLU(),

                nn.Linear(120, 84), nn.ReLU(), nn.Linear(84, 10))

  3. Try out the improved network on the original MNIST dataset.

    • did try.

    Screenshot (516)
    Screenshot (516)789Ă—542 99.6 KB

  4. Display the activations of the first and second layer of LeNet for different inputs (e.g., sweaters and coats).

    • was this what was meant?
    
activation_from_first = nn.Sequential(net[0], net[1])

activation_from_second = nn.Sequential(net[0],net[1], net[2], net[3])

activation_from_first(X[0].unsqueeze(0)).shape, activation_from_second(X[2].unsqueeze(0)).shape
9

This is nice how did you convert any arbitrary size into image?

10

Good approach by in the line nn.init.xavier_uniform_(layer.weight)is it really being applied to your model which is net or aactivation layer which is different from your model.

11

Sorry for the late reply. What I meant was that the model fit would be bad. i.e the accuracy will be very low.

12

Suppose I want to tune the value of the gain argument in this function:

def init_cnn(module):  #@save
    """Initialize weights for CNNs."""
    if type(module) == nn.Linear or type(module) == nn.Conv2d:
        nn.init.xavier_uniform_(module.weight, gain = 1.0)

How do I call the init_cnn with the gain argument in it?:

model.apply_init([next(iter(data.get_dataloader(True)))[0]], init_cnn)
13

My solutions to the exs: 7.6

1 Like