Quantcast
Channel: Recent Questions - Stack Overflow
Viewing all articles
Browse latest Browse all 12231

Why might a discriminator (for a modified WGAN architecture) with a non-singlular output perform better than traditional discriminator?

$
0
0

The following are the Generator and Discriminator model architectures for a (modified) WGAN model that I have come up with for downsampling rainfall data by a factor of 4 (only as a test). The input shape of the dataset is (8030, 14, 21) and the output shape I want is (8030, 28, 42).

def build_generator(input_shape):   inputs = Input(shape=input_shape, name='generator_input')   # Downsampling   downsample_3 = conv_block(inputs, 128, (3, 3), 'downsample_3')   downsample_2 = conv_block(downsample_3, 64, (3, 3), 'downsample_2')   downsample_1 = conv_block(downsample_2, 32, (3, 3), 'downsample_1')   # Bottleneck   bottleneck_0 = conv_block(downsample_1, 16, (3, 3), 'bottleneck_0')   bottleneck_00 = conv_block(bottleneck_0, 16, (3, 3), 'bottleneck_00')   # Upsampling   upsample_1 = deconv_block(bottleneck_00, 32, (3, 3), 'upsample_1')   upsample_2 = deconv_block(upsample_1, 64, (3, 3), 'upsample_2')   upsample_3 = deconv_block(upsample_2, 128, (3, 3), 'upsample_3')   # Final Upsample to required shape   outputs = Conv2DTranspose(filters=1, kernel_size=(3, 3), kernel_initializer=he_normal(), padding='same',                           strides=(2, 2), activation='relu', name='final_upsample_conv')(upsample_3)   model = Model(inputs=inputs, outputs=outputs, name='generator')   return modeldef build_discriminator(input_shape):   input_layer = Input(shape=input_shape, name='discriminator_input')   x = conv_block(input_layer, filters=16, kernel_size=(3, 3), name='conv1')#, use_batch_norm=False)   x = conv_block(x, filters=32, kernel_size=(3, 3), name='conv2')# use_batch_norm=False)   x = conv_block(x, filters=128, kernel_size=(3, 3), name='conv3')# use_batch_norm=False)   x = MaxPooling2D()(x)   x = Dropout(0.25)(x)   output_layer = Dense(1, activation='linear')(x)   model = Model(inputs=input_layer, outputs=output_layer, name='discriminator')   return model'''

Going by WGAN implementations in the literature I use the RMSprop optimizer instead of ADAM with a learning rate of 5e-5 and momentum of 0.5. Additionally I clip the discriminator weights to fall between -1e-2 and 1e-2. Traditionally (to the best of my knowledge), discriminator models output a single value loss conveying whether it is able to distinguish real output from fake outputs. In my case the discriminator output has the shape (None, 14, 21, 1) and it seems to work better but I don't understand why. Does anyone have an idea why this might happen? Thanks in advance!

Edit: These are the architectures of conv_block and deconv_block

def conv_block(x, filters, kernel_size, name, use_batch_norm=True):   x = Conv2D(filters=filters, kernel_size=kernel_size, padding='same', kernel_initializer=he_normal(), name=name+'_conv')(x)   x = LeakyReLU(alpha=0.2, name=name+'_lrelu')(x)   if use_batch_norm:       x = BatchNormalization(name=name+'_bn')(x)   x = Dropout(0.25, name=name+'_dropout')(x)   return xdef deconv_block(x, filters, kernel_size, name, use_batch_norm=True):   x = Conv2DTranspose(filters=filters, kernel_size=kernel_size, padding='same', kernel_initializer=he_normal(), name=name+'_deconv')(x)   x = LeakyReLU(alpha=0.2, name=name+'_lrelu')(x)   if use_batch_norm:       x = BatchNormalization(name=name+'_bn')(x)   return x

Viewing all articles
Browse latest Browse all 12231

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>