1

I need to implement an existing Caffe model with DeepLearning4j. However i am new to DL4J so dont know how to implement. Searching through docs and examples had little help, the terminolgy of those two are very different. How would you write the below caffe prototxt in dl4j ?

Layer1:

layers {
  name: "myLayer1"
  type: CONVOLUTION
  bottom: "data"
  top: "myLayer1"
  blobs_lr: 1
  blobs_lr: 2
  convolution_param {
    num_output: 20
    kernel_w: 2
    kernel_h: 2
    stride_w: 1
    stride_h: 1
    weight_filler {
    type: "msra"
    variance_norm: AVERAGE
    }
    bias_filler {
       type: "constant"
    }
 }
}

Layer 2

 layers {
   name: "myLayer1Relu"
   type: RELU
   relu_param {
   negative_slope: 0.3
 }
 bottom: "myLayer1"
 top: "myLayer1"
 }

Layer 3

  layers {
   name: "myLayer1_dropout"
   type: DROPOUT
   bottom: "myLayer1"
   top: "myLayer1"
   dropout_param {
     dropout_ratio: 0.2
   }
 }

Layer 4

layers {
  name: "final_class"
  type: INNER_PRODUCT
  bottom: "myLayer4"
  top: "final_class"
  blobs_lr: 1
  blobs_lr: 2
  weight_decay: 1
  weight_decay: 0
  inner_product_param {
    num_output: 10
    weight_filler {
      type: "xavier"
      variance_norm: AVERAGE
    }
    bias_filler {
      type: "constant"
      value: 0
    }
  }
}

2 Answers 2

2

This Github repo contains comparisons on the same model between DL4J, Caffe, Tensorflow, Torch.

  • 1st layer is DL4J ConvolutionLayer and you can pass in attributes regarding nOut, kernel, stride and weightInit. From quick search it appears msra is equivalent to WeightInit.RELU and variance_norm is not a feature the model supports yet.
  • 2nd layer is party of the ConvolutionLayer which is the activation attribute; thus, set the attribute for the layer to "relu". Negative slope is not a feature that the model supports yet.
  • 3rd layer is also an attribute on ConvolutionLayer which is dropOut and you would pass in 0.2. There is work in progress to create a specific DropOutLayer but its not merged yet.
  • 4th layer would be a DenseLayer if there was another layer after it but since its the last layer it is an OutputLayer
  • blobs_lr applies multiplier to weight lr and bias lr respectively. You can
  • change the learning rate on the layer by setting attributes on that layer for learningRate and biasLearningRate
  • weight_decay is setting the l1 or l2 on the layer which you can set for each layer with the attributes l1 or l2. DL4J defaults to not applying l1 or l2 to bias thus the second weight_decay set to 0 in Caffe.
  • bias filler is already default to constant and defaults to 0.

Below is a quick example of how your code would translate. More information can be found in DL4J examples:

    int learningRate = 0.1;
    int l2 = 0.005;
    int intputHeight = 28;
    int inputWidth = 28;
    int channels = 1;

    MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder()
        .seed(seed)
        .iterations(iterations)
        .regularization(false).l2(l2)
        .learningRate(learningRate)
        .list()
        .layer(0, new ConvolutionLayer.Builder(new int[]{2,2}, new int[] {1,1})
            .name("myLayer1")
            .activation("relu").dropOut(0.2).nOut(20)
            .biasLearningRate(2*learningRate).weightInit(WeightInit.RELU)
            .build())
        .layer(1, new OutputLayer.Builder()
            .name("myLayer4").nOut(10)
            .activation("softmax").l2(1 * l2).biasLearningRate(2*learningRate)
            .weightInit(WeightInit.XAVIER).build())
        .setInputType(InputType.convolutionalFlat(inputHeight,inputWidth,channels))
        .build();
0
1

there's no automated way to do this but mapping the builder DSL for only a few laayers shouldn't be hard. A bare minimum example is here: https://github.com/deeplearning4j/dl4j-examples/blob/master/dl4j-examples/src/main/java/org/deeplearning4j/examples/convolution/LenetMnistExample.java

You can see the same primitives, eg: stride,padding, xavier, biasInit all in there.

Our upcoming keras import might be a way for you to bridge caffe -> keras -> dl4j though.

Edit: I'm not going to build it for you. (I'm not sure if that's what you're looking for here)

Dl4j has the right primitives already though. It doesn't have an input layer for variance_norm: you use zero mean and unit variance normalization on the input before passing it in.

We have bias Init as part of the config if you just read the javadoc: http://deeplearning4j.org/doc

6
  • Sidenote: If you want a lot more help we always welcome folks in our gitter channel: gitter.im/deeplearning4j/deeplearning4j Nov 7, 2016 at 12:46
  • Thx Adam but only the ones you mentioned are straightforward and easy to adapt. When it comes to layers (above ) there are parameters which are not easy to match-up to DL4J which I ask help for. (i.e. bias_filler, variance_norm ... for the 1st layer)
    – math_law
    Nov 7, 2016 at 13:25
  • Read through our java doc deeplearning4j.org/doc in the neural net configuration. We have things in there like covering the biasInit Nov 7, 2016 at 22:30
  • I'm not going to retype what's already out there on the internet. If you want to attempt this yourself please do - it looks like you're trying to just get me to do it for you which frankly isn't fair. If you don't do your own research not only do I have to explain every term you could possibly want to port but I would also have to show you every example manually. Please try to actually put in some effort. Dec 7, 2016 at 14:26
  • Someone will do it eventually Adam, if it is needed. No one would want to look at javadocs or google it to get the same thing in different environments. Consider this as converting a program between 2 different languages ( Java to C# i.e. ) there are tools out there which do this task (with certain limits). Many people out there experimenting both Caffee and DL4J. So please be positive and let other possible people comment.
    – math_law
    Dec 8, 2016 at 11:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.