This post is editable by all of you! Please edit it to add any useful information for this weeks class, including links brought up during class, other helpful readings, useful code/shell snippets etc. Also, please help organize this wiki post by putting things in sections, adding/editing prose, etc.
- Lesson 9 discussion
- Lesson 9 video timeline
- Updated Powerpoint
- The notebook for style transfer and SR is same as last week
- DeVISE notebook
- Lesson 9 assignments
- Perceptual Losses for Real-Time Style Transfer and Super-Resolution
- Supplementary material
- DeVISE - Deep Visual Semantic Embedding model
- Github python files
- Imagenet dataset via torrent, for those that don’t have access to a .edu address (or who prefer torrent format)
A note on real-time style transfer
Sometimes we calculate the error of a network not by comparing its output to labels immediately, but by first putting its output through a function, and comparing that new output to something we consider to be ideal. That function could be another neural network. For example, in real-time style transfer (Johnson et al.), the network we train takes an image and transforms it into another image; we then take that generated image and analyze it with another neural network, comparing the new output with something we consider to be ideal. The point of the second neural network is to assess the error in the generated image in a deeper way than just calculating errors pixel by pixel with respect to an image we consider to be ideal. The authors of the real-time style transfer paper call this higher-level error “perceptual loss”, as opposed to “per-pixel loss”.