0

I am quite new to machine learning and I am trying to implement my custom layer in keras. I found a couple of tutorials and it seems comparatively straight forward. What I do not understand, though, is how to implement my new custom layer in Sequential(). See for example this classification problem that I took from the tensorflow website(https://www.tensorflow.org/tutorials/keras/basic_text_classification), posted here for your convenience:

from __future__ import absolute_import, division, print_function

import tensorflow as tf
from tensorflow import keras
import numpy as np


imdb = keras.datasets.imdb
(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=10000)

# A dictionary mapping words to an integer index
word_index = imdb.get_word_index()
# The first indices are reserved
word_index = {k:(v+3) for k,v in word_index.items()}
word_index["<PAD>"] = 0
word_index["<START>"] = 1
word_index["<UNK>"] = 2  # unknown
word_index["<UNUSED>"] = 3

reverse_word_index = dict([(value, key) for (key, value) in word_index.items()])

def decode_review(text):
    return ' '.join([reverse_word_index.get(i, '?') for i in text])

train_data = keras.preprocessing.sequence.pad_sequences(train_data,
                                                        value=word_index["<PAD>"],
                                                        padding='post',
                                                        maxlen=256)

test_data = keras.preprocessing.sequence.pad_sequences(test_data,
                                                       value=word_index["<PAD>"],
                                                       padding='post',
                                                       maxlen=256)

# input shape is the vocabulary count used for the movie reviews (10,000 words)
vocab_size = 10000

model = keras.Sequential()
model.add(keras.layers.Embedding(vocab_size, 16))
model.add(keras.layers.GlobalAveragePooling1D())
model.add(keras.layers.Dense(16, activation=tf.nn.relu))
model.add(keras.layers.Dense(1, activation=tf.nn.sigmoid))

model.summary()

model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['acc'])

x_val = train_data[:10000]
partial_x_train = train_data[10000:]

y_val = train_labels[:10000]
partial_y_train = train_labels[10000:]

history = model.fit(partial_x_train,
                    partial_y_train,
                    epochs=40,
                    batch_size=512,
                    validation_data=(x_val, y_val),
                    verbose=1)

results = model.evaluate(test_data, test_labels)

print(results)

Do I have to change the source code for keras.Sequential() or is there an easy way?

Furthermore, looking at the source code for the class Sequential() made me wonder: I can't figure out how functions like 'summary()','compile()', 'fit()' and 'evaluate()' can be called if those are not even provided in the source code in this class. Here is the source code for Sequential():

https://github.com/keras-team/keras/blob/a1397169ddf8595736c01fcea084c8e34e1a3884/keras/engine/sequential.py

2
  • Hey, what do You really mean by the custom layer ? Do You want to create the layer from scratch as a totally new class? Or do You just want to use one of existing layers (e.g. Dense) but You are not sure how to create the instance of the class ? Commented Apr 26, 2019 at 21:56
  • I was referring to the code that Clarence Leung posted below. I don't know how to make Sequential() use this custom-made layer.
    – xabdax
    Commented Apr 26, 2019 at 22:23

1 Answer 1

0

Sequential is a Model, and not a layer.

The functions you mentioned (summary, compile, fit, evaluate) are implemented in the Model class linked here, as Sequential is a subclass of Model.

If you're writing a custom layer, you should be subclassing Layer instead, and not Model or Sequential.

You would need to implement build, call, and compute_output_shape to create your own layer.

There's a few examples on the Keras documentation:

from keras import backend as K
from keras.layers import Layer

class MyLayer(Layer):

    def __init__(self, output_dim, **kwargs):
        self.output_dim = output_dim
        super(MyLayer, self).__init__(**kwargs)

    def build(self, input_shape):
        # Create a trainable weight variable for this layer.
        self.kernel = self.add_weight(name='kernel', 
                                  shape=(input_shape[1], self.output_dim),
                                  initializer='uniform',
                                  trainable=True)
        super(MyLayer, self).build(input_shape)  # Be sure to call this at the end

    def call(self, x):
        return K.dot(x, self.kernel)

    def compute_output_shape(self, input_shape):
        return (input_shape[0], self.output_dim)

To use it, import the MyLayer class from whichever file you put it in, and then add it like the default Keras layers:

from custom.layers import MyLayer

model = keras.Sequential()
model.add(MyLayer())
4
  • Ok nice. I didn't realize that Sequential is a subclass of Model. If I wanted to use the MyLayer class that you just posted instead of keras.layers.Dense() in my classification problem above, what would I have to do?
    – xabdax
    Commented Apr 26, 2019 at 22:23
  • @xabdax I added the info on that as an edit on my original answer. Commented Apr 26, 2019 at 22:29
  • This doesn't seem to work. For model.add(MyLayer(16)) I get the following error: "Failed to convert object of type <class 'tuple'> to Tensor". Furthermore, how do I implement parameters like "activation=tf.nn.relu"? Those do not appear in MyLayer at all.
    – xabdax
    Commented Apr 26, 2019 at 23:19
  • @xabdax You might need to post your full code. You have to handle activation on your own - add it as a parameter in the constructor, for instance: github.com/keras-team/keras/blob/… Commented Apr 27, 2019 at 2:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.