Multiple Input and Output Channels

English Version mxnet
1

http://d2l.ai/chapter_convolutional-neural-networks/channels.html

2

I am a beginner. What I am wondering is that for a Multiple Output Channels layer, it seems each channel has same initialized parameters, but they can get different kernel at last.

3

Hi @sunchunqiang, yes you are right with the same initialization. However, the initialization itself will randomly sample through the given distribution. Which means even if we use the same initialization to sample, we will get different result at every time.

4

Quick question about Figure 6.4.1, multi-channel 2D convolutions:

In practice, for a standard conv2D layer, do we usually use the same 2D kernel for each input channel, or do we use a 3khkw kernel, which would mean that there are different parameters for each channel? The diagram shows an example of the latter, but it seems like actual implementations use the first.

5

Ex. 6.4.6 (last question of chapter)
Is it ok for Y1 != Y2 when the kernel size goes from 1x1 to a 2x2? Or should the 2x2 channel convolution match what the cross correlation iterative function does?

This is what I have so far:

# Exercise 6
def corr2d_multi_in_out_2x2(X, K):
    c_i, h, w = X.shape
    c_o, c_ii, kh, kw = K.shape
    assert c_ii == c_i, "Kernel channel dimensions don't match input"
    X = X.reshape((c_i, h * w))
    K = K.reshape((c_o, kh * kw, c_i))
    Y = np.dot(K, X)  # Matrix multiplication in the fully-connected layer
    Y = Y.sum(axis=2) # input channel dimension
    return Y.reshape((c_o, kh, kw))


K = np.random.normal(0, 1, (2, 3, 2, 2))
Y1 = corr2d_multi_in_out_2x2(X, K)
print(Y1.shape)
Y2 = corr2d_multi_in_out(X, K)
print(Y2.shape)
1 Like
6

i think this is no luck for doing matrix product in case of any dimensions for kernel. here conv2d will give a light to the details.

1 Like
8

Hi there,

Nothing is said in this chapter about the activation function. It may come later. Is it operating on the aggregation (sum) of channel-specific convolutional layers (multichannel input, Fig 6.4.1)? Or before the sum. Does it matters?

Omar