3

I'm new to TensorFlow (version 1.2), but not to Python or Numpy. I am building a model to predict the shape of a protein molecule. I need to wrap TensorFlow's standard tf.losses.cosine_distance function in some extra code, because I need to stop the propagation of some NaN values into the loss calculation.

I know exactly which cells will be NaN. Whatever my machine learning system predicts for those cells does not count. I plan to turn the NaN part of the output of tf.losses.cosine_distance into zeros before summing up the loss function.

Here's a snippet of working code, using tf.scatter_nd_update for the element assignment:

def custom_distance(predict, actual):
    with tf.name_scope("CustomDistance"):
        loss = tf.losses.cosine_distance(predict, actual, -1, 
               reduction=tf.losses.Reduction.NONE)
        loss = tf.Variable(loss) # individual elements can be modified
        indices = tf.constant([[0,0,0],[29,1,0],[29,2,0]])
        updates = tf.constant([0., 0., 0.])
        loss = tf.scatter_nd_update(loss, indices, updates)
        return loss

But, that only works on the one protein that I have that is 30 amino acids long. What if I have a protein of a different length? I will have many. In Numpy, I would just use Python's negative indexing, and substitute -1's for the two 29's on the indices line. Tensorflow will not accept that. If I make that substitution, I get a long traceback, but I think that the most important part of it is this:

File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status
    pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Invalid indices: [0,1] = [-1, 1, 0] is not in [0, 30)

(I could also modify the predict Tensor so that the cells in question exactly match the actual Tensor before calculating the loss, but in each case the challenge is the same: to assign the values of individual elements in a TensorFlow object.)

Should I just forget about negative indexing in TensorFlow? I am poring through the TensorFlow docs to understand the correct approach to this problem. I assume that I can retrieve the length of my input Tensors long the primary axis and use that. But after seeing the strong parallels between TensorFlow and Numpy, I have to wonder whether that's clunky.

Thanks for your suggestions.

1
  • It looks like scatter_nd_update doesn't support negative indices yet. There's indeed a push to make TF match numpy semantics, but some ops haven't caught up Aug 1, 2017 at 15:42

1 Answer 1

0

It can be used with tensorflow's bindings to python slicing operators. So for example, loss[-1] is a valid slicing of loss.

In your case, if you have only three slices, you could assign them individually:

update_op0 = indices[0,0,0].assign(updates[0])
update_op1 = indices[-1,1,0].assign(updates[1])
update_op2 = indices[-1,2,0].assign(updates[2])

If you have more slices than that, or a variable number of slices, the previous approach is not practical. You can rather write a small helper function like this to convert "positive or negative indices" to "positive only indices":

def to_pos_idx(idx, x):
  # todo: shape & bound checking
  idx = tf.convert_to_tensor(idx)
  s = tf.shape(x)[:tf.size(idx)]
  idx = tf.where(idx < 0, s + idx, idx)
  return idx

and modify your code like this :

indices = tf.constant([[0,0,0],[-1,1,0],[-1,2,0]])
indices = tf.map_fn(lambda i: to_pos_idx(i, loss), indices) # transform indices here
updates = tf.constant([0., 0., 0.])
loss = tf.scatter_nd_update(loss, indices, updates)
3
  • Thanks, I will try one or both of your approaches. I don't quite understand the first one. I was looking at tf.assign(), but it appears to replace a whole Tensor and not just individual elements. I'm not sure how the assign method of a tf.constant differs. Your second example is longer, but its purpose is clear. Aug 2, 2017 at 10:07
  • I caught a bug in your second example, otherwise it works. On the line that reads "idx = tf.where(idx < 0, s - idx, idx)", the minus sign should be a plus sign. Thanks! Aug 2, 2017 at 10:25
  • Second followup: In the first example, did you mean to type "loss" on the three lines where you typed "indices"? If so, that example would also now make sense to me. Please let me know, thanks. Aug 12, 2017 at 8:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.