38

What is the difference between

   tf.add(x, y)

and

   x + y

in TensorFlow? What would be different in your computation graph when you construct your graph with + instead of tf.add()?

More generally, are + or other operations overloaded for tensors?

2 Answers 2

75

If at least one of x or y is a tf.Tensor object, the expressions tf.add(x, y) and x + y are equivalent. The main reason you might use tf.add() is to specify an explicit name keyword argument for the created op, which is not possible with the overloaded operator version.

Note that if neither x nor y is a tf.Tensor—for example if they are NumPy arrays—then x + y will not create a TensorFlow op. tf.add() always creates a TensorFlow op and converts its arguments to tf.Tensor objects. Therefore, if you are writing a library function that might accept both tensors and NumPy arrays, you might prefer to use tf.add().

The following operators are overloaded in the TensorFlow Python API:

  • __neg__ (unary -)
  • __abs__ (abs())
  • __invert__ (unary ~)
  • __add__ (binary +)
  • __sub__ (binary -)
  • __mul__ (binary elementwise *)
  • __div__ (binary / in Python 2)
  • __floordiv__ (binary // in Python 3)
  • __truediv__ (binary / in Python 3)
  • __mod__ (binary %)
  • __pow__ (binary **)
  • __and__ (binary &)
  • __or__ (binary |)
  • __xor__ (binary ^)
  • __lt__ (binary <)
  • __le__ (binary <=)
  • __gt__ (binary >)
  • __ge__ (binary >=)

Please note, __eq__ ( binary == ) is not overloaded. x == y will simply return a Python boolean whether x and y refer to the same tensor. You need to use tf.equal() explicitly to check for element-wise equality. Same goes for not equal, __ne__ ( binary != ).

4
  • what if we want == operator on tensor scalar comparation?
    – K.Wanter
    Feb 20, 2018 at 3:51
  • 1
    @K.Wanter you need to use tf.equal(). == and != are not overloaded. (Or ( x <= y ) & ( y <= x ) if you feel rebellious. :) ) May 15, 2018 at 12:14
  • 3
    One thing to add is that you can also overload tf.matmul() with @ like in numpy. Oct 22, 2019 at 12:32
  • it doesn't appear to be this simple re ==. For example, tf.constant([0]) == tf.constant([0]) is a tf.Tensor containing [True], at least in tf2. Appears eager execution has an effect
    – joel
    Jan 16, 2020 at 17:17
1

Mrry nicely explained that there is no real difference. I will just add when using tf.add is beneficial.

tf.add has one important parameter which is name. It allows you to name the operation in a graph which will be visible in tensorboard. So my rule of thumb, if it will be beneficial to name an operation in tensorboard, I use tf. equivalent, otherwise I go for brevity and use overloaded version.

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.