Custom Layers

So here right after I call dense.initialize() the weights are initialized. This is in contrast with deferred initialization. It makes sense because we specified the in_unit.

Is MyDense() as efficient as nn.Dense()?

I’m afraid not. At least because mxnet.gluon.nn.Dense is hybridized (see 12.1.2) and its backend is implemented in C/C++.