Page 31 - Kỷ yếu hội thảo khoa học lần thứ 12 - Công nghệ thông tin và Ứng dụng trong các lĩnh vực (CITA 2023)

Bao Ngoc Vi, Cao Truong Tran, Chi Cong Nguyen 15

Furthermore, the mask generation is replaced with the mask of missing data, and un-
conditional data generation is replaced with conditional generation based on the miss-
ing data.

2.3 Evolutionary Generative Adversarial Network

Although GANs already produce visually appealing samples in various applications,
they are often difficult to train. If the data distribution and the generated distribution do
not substantially overlap (usually at the beginning of training), the generator gradients
can point to more or less random directions or even result in the vanishing gradient
issue. GANs also suffer from mode collapse, i.e., the generator assigns all its probability
mass to a small region in the space [3]. In addition, appropriate hyper-parameters (e.g.,
learning rate and updating steps) and network architectures are critical configurations

reasonable results. Many recent efforts on GANs have focused on overcoming these
training difficulties by developing various adversarial training objectives. Typically,
assuming the optimal discriminator for the given generator is learned, different objec-
tive functions of the generator aim to measure the distance between the data distribution
and the generated distribution under different metrics. The original GAN uses Jensen-
Shannon divergence as the metric. A number of metrics have been introduced to im-
t-squares [9], absolute deviation [25], Kullback-
Leibler divergence [11, 12], and Wasserstein distance [2]. However, according to both
theoretical analyses and experimental results, minimizing each distance has its own
pros and cons. For example, although measuring Kullback-Leibler divergence largely
eliminates the vanishing gradient issue, it easily results in mode collapse [1, 12]. Like-
wise, Wasserstein distance greatly improves training stability but can have non-conver-
gent limit cycles near equilibrium [10].
To exploit the advantages and suppress the weaknesses of different metrics (i.e.,
GAN objectives), Yao and co-authors has proposed the E-GAN (evolutionary genera-
tive adversarial network) which utilizes different metrics to jointly optimize the gener-
ator. In doing so, they improve both the training stability and generative performance.
E-GAN treats the adversarial training procedure as an evolutionary problem. Specifi-
cally, a discriminator acts as the environment (i.e., provides adaptive loss functions)
and a population of generators evolve in response to the environment. During each ad-
versarial (or evolutionary) iteration, the discriminator is still trained to recognize real
and fake samples. However, in this method, acting as parents, generators undergo dif-
ferent mutations to produce offspring to adapt to the environment. Different adversarial
objective functions aim to minimize different distances between the generated distribu-
tion and the data distribution, leading to different mutations. Meanwhile, given the cur-
rent optimal discriminator, the quality and diversity of samples generated by the up-

-performing offspring are removed and the remaining well-performing
offspring (i.e., generators) are preserved and used for further training.
Based on the evolutionary paradigm to optimize GANs, E-GAN overcomes the in-
herent limitations in the individual adversarial training objectives and always preserves

ISBN: 978-604-80-8083-9 CITA 2023