4

Suppose I want to compare two images with deep convolutional NN. How can I implement two different pathways with the same kernels in keras?

Like this:

enter image description here

I need convolutional layers 1,2 and 3 use and train the same kernels.

Is it possible?

I was also thinking to concatenate images like below

enter image description here

but question is about how to implement tolopology on first picture.

0

2 Answers 2

8

You can use the same layer twice in the model, creating nodes:

from keras.models import Model    
from keras.layers import *

#create the shared layers
layer1 = Conv2D(filters, kernel_size.....)
layer2 = Conv2D(...)    
layer3 = ....

#create one input tensor for each side
input1 = Input((imageX, imageY, channels))
input2 = Input((imageX, imageY, channels))   

#use the layers in side 1
out1 = layer1(input1)   
out1 = layer2(out1)   
out1 = layer3(out1)

#use the layers in side 2
out2 = layer1(input2)   
out2 = layer2(out2)   
out2 = layer3(out2)

#concatenate and add the fully connected layers
out = Concatenate()([out1,out2])
out = Flatten()(out)
out = Dense(...)(out)   
out = Dense(...)(out)   

#create the model taking 2 inputs with one output
model = Model([input1,input2],out)

You could also use the same model twice, making it a submodel of a bigger one:

#have a previously prepared model 
convModel = some model previously prepared

#define two different inputs
input1 = Input((imageX, imageY, channels))
input2 = Input((imageX, imageY, channels))   

#use the model to get two different outputs:
out1 = convModel(input1)
out2 = convModel(input2)

#concatenate the outputs and add the final part of your model: 
out = Concatenate()([out1,out2])
out = Flatten()(out)
out = Dense(...)(out)   
out = Dense(...)(out)   

#create the model taking 2 inputs with one output
model = Model([input1,input2],out)
4
  • Are you sure it will train accordingly? Will it recognize gradients properly?
    – Dims
    Jun 28, 2017 at 20:41
  • Yes, Keras has the notion of "nodes", which are created only when you pass an input tensor to a layer. When you do it more than once, the layer gets many nodes. Then, if you want the output of that layer, instead of doing layer[i].output you will start using layer[i].get_output_at(node_index), for instance. Jun 28, 2017 at 21:04
  • I just tested a little MNIST model taking two images at the same time to classificate both together using this code. It worked great :) Jun 28, 2017 at 21:07
  • I am trying to do the same thing. the model is built and trained nicely. but then it fails to load with an obscure exception ValueError: Input 0 is incompatible with layer conv9x9: expected axis -1 of input shape to have value 16 but got shape (None, 62, 62, 4). Anyone else seen that issue? Sep 16, 2019 at 9:29
3

Indeed using the same (instance of) layer twice ensures that the weights will be shared.

Just look at the siamese example, I just put here an excerpt from the model to show an example:

# because we re-use the same instance `base_network`,
# the weights of the network
# will be shared across the two branches
processed_a = base_network(input_a)
processed_b = base_network(input_b)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.