fix(networks): invalid decoder type silently ignored (+3 more)

[andrewwhitecdw]

Small fixes in networks.py:

fix: invalid decoder type silently ignored

Fix: Replace:

        else:
            ('unkonw decoder type')

with:

        else:
            raise ValueError('unknown decoder type')

fix: invalid resblock type silently ignored

Fix: Replace:

        else:
            ('unkown block type')

with:

        else:
            raise ValueError('unknown block type')

fix: LayerNorm fp16 normalizes across batch

Fix: Replace:

        if x.type() == 'torch.cuda.HalfTensor': # For Safety
            mean = x.view(-1).float().mean().view(*shape)
            std = x.view(-1).float().std().view(*shape)
            mean = mean.half()
            std = std.half()

with:

        if x.type() == 'torch.cuda.HalfTensor': # For Safety
            mean = x.view(x.size(0), -1).float().mean(1).view(*shape).half()
            std = x.view(x.size(0), -1).float().std(1).view(*shape).half()

fix: WGAN drift penalty uses sum not mean

Fix: Replace:

            # progressive gan
            loss += Drift*( torch.sum(outs0**2) + torch.sum(outs1**2))

with:

            # progressive gan
            loss += Drift*( torch.mean(outs0**2) + torch.mean(outs1**2))

Files changed

• networks.py