0

I want to calculate and adapt the gradient by myself in tensorflow. I want to calculate the gradient for one of the multiple outputs by myself, and adapt 0 for the rest of the outputs.

I have tested building the model with the following simple program and trying to adapt it by setting one gradient to a constant value and the other gradient to 0, but I am not getting the desired output.

What should I give to apply_gradients?

I don't know much about tensorflow, so I would appreciate if you could kindly tell me about it

def cmodel():
    in_ = Input(shape=(1,))
    x=in_
    x=Dense(16)(x)
    x=Activation('relu')(x)
    x=Dense(2)(x)
    x=Activation('linear')(x)
    
    model = Model(inputs=in_, outputs=[x])
    return model

model=cmodel()
model.compile(optimizer=Nadam(learning_rate=0.005))

mylist2=[]
for i in range(100):
    g=tf.constant([-0.3,0])

    model.optimizer.apply_gradients(zip(g, model.trainable_variables))
    print('num:',model.predict(np.array([1])))
    mylist2.append(model.predict(np.array([1]))[0])

1 Answer 1

0

I'm not sure I understand what you want to do. In fact, what you are doing is applying the values -3 to the trainable variable of the first layer, i.e. to the kernel of the first dense layer and 0 to the bias. For example, if you want to modify the first layer only, you can define g=tf.constant([-0.3, 3., 0., 0.]). You could also get the training variables of the first layer: model.layers[1].trainable_variables and apply the gradient.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.