Deep Dreaming – Layers

bild2I haven’t found that much useful information regarding the different layers in the Caffe deep learning model provided by Google. Some layers seem to emphasize dogs, others interesting patterns, or contours. Inspired by this amazing page I set out to create my own guide to the layers. So, here goes. For each layer listed, I have iterated the image five times, to bring out the characteristics (everything beyond five iterations seems not to change things too much, only deepen them)

I haven’t had time to render all possible layers yet, and actually ran into a crash which I haven’t yet sorted out, so expect this page to grow further over time.

The starting image was this lovely selfie to the right (apologies for pretentious pose)

Filters and categories

Contour and texture

conv17x7_s2.jpgpool13x3_s2.jpg conv1/7x7_s2

pool1/3x3_s2

Reminds me of the classic Photoshop contour filter
pool1norm1.jpgconv23x3_reduce.jpg pool1/norm1

conv2/3x3_reduce

Interesting “line art” variations
conv23x3.jpg conv2/3×3 Metal scales? I kind of like it, could definitely be used on interesting images.
conv2norm2.jpginception_3a1x1.jpg conv2/norm2

inception_3a/1×1 

inception_3a/5x5_reduce

I like these. They add various amount of tiles/reptile scales to the structure, sometimes making me look like that guy from the Marvel universe. (Not the Hulk!)
inception_3a5x5.jpg inception_3a/5×5 I look like a mummy. With bad taste in bandaging. Very “banded”.
inception_3apool_proj.jpg inception_3a/pool_proj Mosaic tiles!
inception_3b5x5.jpginception_3b5x5_reduce.jpg inception_3b/5×5
inception_3b/5x5_reduce,
Cubist impressions! Very useful for certain things, adds a mosaic tiling that flows with the image. My hair looks great with the second one. Max Headroom anyone?

Swirls and fur

inception_4c1x1.jpg inception_4c/1×1 A few traces of dog popping out, but overall an interesting addition of “fur” or “hair” to the image.
inception_3b3x3.jpg inception_3b/3×3 Leopard spots and similar
inception_3aoutput.jpginception_3bpool_proj.jpg inception_3a/output
inception_3b/1×1
inception_3b/3x3_reduce
inception_3b/pool_proj
inception_3b/pool
inception_4a/1×1
inception_4a/5x5_reduce
inception_4a/pool_proj
All kinds of swirliness, to any degree. The occasional eye pops out in the higher numbered ones, but discreetly.

Cellular structures

inception_3a3x3_reduce.jpg inception_3a/3x3_reduce Tobacco-like cell structure. Or something.

Deeper dreams

Everything above “inception_4b” takes us into high-level dream territory. This is where dogs, buildings, birds, insects etc. are found most prominent. I may add a more detailed section to this eventually, but note that it will be more dependent on the training data and thus more likely to suggest different things based on the types of images the net “knows”. A few interesting things can be noted with the current dataset (bvlc_googlenet) as seen below.

inception_4bpool.jpginception_4b/pool sums up layer 4b pretty much. Dogs and related animals (such as “dogslugs” and the occasional bird)

inception_4dpool.jpgInception_4c and 4d adds more animals and structures start to appear.

inception_4epool.jpg

When we reach inception_4e it’s a free-for all, and every layer above adds complexity. It is interesting to note that each different layer still brings out different aspects of the image. I may add thumbnails to all variantions later.

Original images (C) Mikael Lundgren 2015

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: