Can I unpack an `nn.ModuleList` inside `nn.Sequential`?

Question

I am parametrizing the number of hidden layers of a simple ANN using nn.ModuleList. I am wondering if passing this list into a nn.Sequential module as follows would lead to any adverse effects with the execution graph.

nn.Sequential is not necessary, however it seems cleaner to me to have the entire architecture explicit in the constructor.

class ANN(nn.Module):

    def __init__(
        self,
        in_feats=3,
        in_hidden=5,
        n_hidden_layers=3,

    ):
        super(ANN, self).__init__()

        # ====== dynamically register hidden layers ======
        self.hidden_linears = nn.ModuleList()
        for i in range(n_hidden_layers):
            self.hidden_linears.append(nn.Linear(in_hidden, in_hidden))
            self.hidden_linears.append(nn.ReLU())
        
        # ====== sequence of layers ======
        self.layers = nn.Sequential(
            nn.Linear(in_feats, in_hidden),
            nn.ReLU(),
            *self.hidden_linears,
            nn.Linear(in_hidden, 1),
            nn.Sigmoid(),
        )

    def forward(self, X):
        return self.layers(X)

Also open to suggestions of neater ways to put this together.

Answer 1

You could use nn.Sequential.modules() to get every single part of the Sequential and then pack them into a list.