That would work, but it would likely re-introduce the catastrophic forgetting problem, because now the GLN is dependent on an intermediate representation determined by conv layers.