1

I create a keras sequential model with three dense layer and I just create a custom loss function as below:

def lossFunctionality(y_t, y_p):
    # 'y_t' shape is (bach_size, 500)
    # 'y_p' shape is (bach_size, 256) (output of last layer)
    # 'v' is np array of shape (500, 256)

    p = K.exp(K.dot(y_p, K.transpose(v)))
    sp = K.sum(p,axis=1)
    sp = K.expand_dims(sp, axis=1)
    sp = K.tile(sp,(1, len(v)))

    soft = p/sp
    soft=K.clip(soft, 0.0000001, 0.9999999)
    obj = K.categorical_crossentropy(soft, y_t)
    return obj

with this loss function everything goes fine.

but now suppose I have specefic 'v' array for each record in 'y_p' and I want to do line 5 of the code to have dot product of 'y_p' with record specefic 'v' array. in other words I have the number of bach_size of 'v' vectors that I want to product each record in 'y_p' to that specefic 'v' array. what I want is showing bellow:

def lossFunctionality(y_t, y_p):
    tempArray=[]
    for record in y_p:
        tempArray.append(K.exp(K.dot(record, K.transpose(v))))
    p=np.array(tempArray)
    sp = K.sum(p,axis=1)
    sp = K.expand_dims(sp, axis=1)
    sp = K.tile(sp,(1, len(v)))

    soft = p/sp
    soft=K.clip(soft, 0.0000001, 0.9999999)
    obj = K.categorical_crossentropy(soft, y_t)
    return obj

but I got error ''Tensor' object is not iterable.' how can I implement my loss function in a right way I realy appreciate your help

3
  • i think the error is in 3rd line of your code.. "for record in y_p" Jun 14, 2017 at 12:36
  • you are right and I know the error is from that line. I want to know how can I implement this logic in a right way Jun 14, 2017 at 12:41
  • I mean how can I iterate through Tensor object Jun 14, 2017 at 12:42

1 Answer 1

1

You shouldn't really iterate a tensor or use lists, numpy arrays or so in a loss function.

Ideally, you must do everything with backend functions, which work properly with tensors.

If I undesrtood it correctly, what you need is a batch_dot function, perhaps in association with permute_dimensions if you really need to transpose "v".

I'm assuming you have a list of "v" vectors, right? I'll answer it as if "v" in my code were a numpy array shaped as (batch_size, 500, 256).

import keras.backend as K

--------------

#transform v in a tensor
vTens = K.variable(v) #maybe having vTens already instanced and transposed outside the loss function would result in better performance (if it's constant)   

#transposing v without changing the batch dimension
vTrans = K.permute_dimensions(vTens,(0,2,1)) 

#doing a batch dot
dotResult = K.batch_dot(y_p,vTrans,axes=[1,2])

In the batch_dot, axes 1 and 2 were considered, while axis 0 is the batch size and will be kept untouched.

Important remark on your function:

You won't be able to have y_p and y_t with different dimensions. That's simply unacceptable. Your model predictions must have the exact same shape as your true values.

An error will certainly appear in the fit method (or any other training method) telling you that your dimensions mismatch.

You do need to have a layer to transform your output before reaching the loss function, so y_p and y_t have the same dimensions.

For that, use a LambdaLayer

model.add(LambdaLayer(transformOutput, output_shape=(500,))

Where transformOutput is the function you described before, also working with tensors.

def transformOutput(x):
    #transform v in a tensor
    vTens = K.variable(v) #maybe having vTens already instanced and transposed outside the loss function would result in better performance (if it's constant)   

    #transposing v without changing the batch dimension
    vTrans = K.permute_dimensions(vTens,(0,2,1)) 

    #doing a batch dot
    return K.batch_dot(x,vTrans,axes=[1,2])
2
  • I ran your solution. Suppose I have bach_size=2 then I got this error: 'Shape must be rank 2 but is rank 3 for 'MatMul' (op: 'MatMul') with input shapes: [?,256], [2,256,500].' Jun 15, 2017 at 7:24
  • I just make it work by adding this line: x = tf.expand_dims(x, axis=K.ndim(x) if 1 == K.ndim(x) - 1 else K.ndim(x)) and remove the transpose line. Jun 15, 2017 at 12:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.