Mini-Project 1: Google Deep Dream Vacation
Due: Thurs, Sept 17 (11:59 PM)
This project involves simply installing and producing results from Google Deep Dream for your own input photographs, such as vacation photographs. The main goals are to experiment with neural networks and see how Google Deep Dream produces different outputs depending on which layer of the neural network is chosen.
Deep Dream Overview
Google Deep Dream on the instructor's vacation photos
Please see the Deep Dream blog post and GitHub source code to get an overview of what this technology/art-form does.
The convolutional neural network for Deep Dream has been pre-trained to classify any input image as one of many categories such as dog, person, insect, car, and so forth. Ordinarily, one would run the neural network classifier on an input image to produce a classification score for each category, which could be interpreted as the probability that the image is the given category.
Unusually, the deep dream technology takes an input image and instead tries to modify it by maximizing all classification scores simultaneously, while imposing a smoothness prior to prevent the image from being too "unnatural" (although you can be the judge of how natural the images look). If one imposes this goal at the output layer of the neural network, it tends to produce surreal images of all the objects the network has been trained to recognize, which are super-imposed on the input image. However, if one imposes the objective at an intermediate layer of neural network, it will tend to visualize the kinds of features that are being recognized at that layer of the network (e.g. edges).
Your mission for this project is to install Google Deep Dream, and then apply it on a few of your own photos. You should visualize both the final output layer as well as one or more intermediate layers.
You can install Deep Dream by cloning the GitHub project and then installing the required dependencies. I recommend Anaconda Python which has several of the scientific libraries installed already. You can run at the command-line ipython notebook to open an IPython Notebook, and navigate to the Deep Dream notebook dream.ipynb. In the Notebook you can then press Shift+Enter to run each block of Python codes in order. This should give you a demonstration of Deep Dream, which you can then modify slightly to work on your own input photographs.
The biggest challenge here is probably how to build the neural network library Caffe for your system. You can follow the directions on the Caffe wiki. Alternatively, you can search Google for "how to install caffe your-operating-system," and find many helpful tutorials and/or pre-built distributions in some cases.
Policies
Feel free to collaborate on solving the problem but write your code individually.
Submission
Submit your assignment in a zip file named yourname_project1.zip. Please include the results of Deep Dream after being run on your own photographs, both for the final output layer and some intermediate layers.
Finally submit your zip to UVA Collab.