MHXX and Anime GANS

Generative Adversarial Networks Improving the creation Pipeline

• Concept design can take time. The goal of this project is to see if we can make the creative process more efficient.

Compute Requirements

• GPU Nvidia Requirements
  • Nvidia GPU 11 gb and above
  • tensorflow-gpu 1.1.5
  • cuda 10.0
  • cuDNN 7.5
  • tensorRT 5.6.5

Data Augmentation

• Libraries
  • numpy
  • os
  • cv2
  • glob
  • pandas
  • PIL
  • scipy
  • imageio
  • keras
  • augmentor
  • fastai

• Anime faces and MHXX armor images were collected.

  • The anime face data set was collected on kaggle and had 85000 images
  • MHXX was collected from a fan website and had 1083 images

• To increase the size of the MHXX armor data set we had to do the following

  1. Split the double images and put back the single images
  2. Get rid of only top half images
  3. Mirrored the Images
  4. Did random sampling of brightness and dimness of 0.7 to 1.2
  5. The data increased to 19437 and then used photoshop to inrease resolution
  6. Used augmentor to turn them to 512 x 512 dimensions
  7. Use fastai to check if it can pass through Nvidia’s dataset_tool.py
  8. Changed to tfrecords.

Feature Map Exploration

• Libraries
  • keras.applications.vgg16
  • keras.preprocessing.image
  • matplotlib
  • numpy

• We used a model in keras vgg16 which is a convolutional neural network.

• Wanted to see the activation map of one of our images.

• Further from input the less detail

Deep Convolutional Generative Adversarial Networks

• Libraries
  • numpy
  • os
  • glob
  • imageio
  • time
  • PIL
  • keras
  • matplotlib

• DCGAN was a quick way to visualize our image datasets.

• Uses convolutional layers. Conv2DTanspose for Generator, Conv2D for Discriminator

• Used original learning rate from paper.

• Ran it for 2000 steps

• 20000 Training Steps

• Generator Loss, Discriminator Fake Loss, Discriminator Real Loss

Frechet Inception Distance

• Libraries
  • numpy
  • scipy
  • keras
  • skimage.transform
  • imblearn
• Looks at the normal distribution of real images vs fake images. The lower number the better.

Style GAN

• Libraries
  • numpy
  • tensorflow
  • dnnlib
  • config
  • train
  • training
  • copy
  • metrics

• First trained the anime face model using the preset hyperparameters.

• Stopped at 7 days of training.

• Tried to train from scratch to see if I could get similar results with the armor images

• Mode collapse from mainly b/c too small of a dataset

• Used Transfer Learning with pickled model with anime faces

• Halved learning rate and decreased mini batch repeat to 1

• End Result using interpolation

Image Generator and MP4 creation

• Libraries
  • glob
  • numpy
  • moviepy
  • os
  • PIL
  • dnnlib
  • pickle

• Full write up, blog post and github link can be found below.

MHXX GAN Blog post

MHXX GAN Write Up

MHXX GAN Github