fix(networks): VAEGen.forward must unpack encode tuple (+3 more)

[andrewwhitecdw]

Small fixes in networks.py:

fix: VAEGen.forward must unpack encode tuple

Fix: Replace:

        hiddens = self.encode(images)
        if self.training == True:
            noise = Variable(torch.randn(hiddens.size()).cuda(hiddens.data.get_device()))

with:

        hiddens, _ = self.encode(images)
        if self.training == True:
            noise = Variable(torch.randn(hiddens.size()).cuda(hiddens.data.get_device()))

fix: LayerNorm fp16 branch normalizes across batch

Fix: Replace:

        if x.type() == 'torch.cuda.HalfTensor': # For Safety
            mean = x.view(-1).float().mean().view(*shape)
            std = x.view(-1).float().std().view(*shape)
            mean = mean.half()
            std = std.half()

with:

        if x.type() == 'torch.cuda.HalfTensor': # For Safety
            mean = x.view(x.size(0), -1).float().mean(1).view(*shape).half()
            std = x.view(x.size(0), -1).float().std(1).view(*shape).half()

fix: replace removed F.sigmoid with torch.sigmoid

Fix: Replace:

                loss += torch.mean(F.binary_cross_entropy(F.sigmoid(out0), all0) +
                                   F.binary_cross_entropy(F.sigmoid(out1), all1))
                reg += LAMBDA* self.compute_grad2(F.sigmoid(out1), input_real).mean()

with:

                loss += torch.mean(F.binary_cross_entropy(torch.sigmoid(out0), all0) +
                                   F.binary_cross_entropy(torch.sigmoid(out1), all1))
                reg += LAMBDA* self.compute_grad2(torch.sigmoid(out1), input_real).mean()

fix: WGAN squeeze removes batch dim when batch=1

Fix: Replace:

        else:
             outputs = self.cnn(x)
             outputs = torch.squeeze(outputs)
        return outputs

with:

        else:
             outputs = self.cnn(x)
             outputs = outputs.view(outputs.size(0), -1).squeeze(-1)
        return outputs

Files changed

• networks.py