222

This is a logistic sigmoid function:

enter image description here

I know x. How can I calculate F(x) in Python now?

Let's say x = 0.458.

F(x) = ?

17 Answers 17

313

This should do it:

import math

def sigmoid(x):
  return 1 / (1 + math.exp(-x))

And now you can test it by calling:

>>> sigmoid(0.458)
0.61253961344091512

Update: Note that the above was mainly intended as a straight one-to-one translation of the given expression into Python code. It is not tested or known to be a numerically sound implementation. If you know you need a very robust implementation, I'm sure there are others where people have actually given this problem some thought.

5
  • 11
    Just because I need it so often to try little things: sigmoid = lambda x: 1 / (1 + math.exp(-x)) Jul 31, 2014 at 18:41
  • 5
    This does not work for extreme negative values of x. I was using this unfortunate implementation until I noticed it was creating NaNs.
    – Neil G
    Apr 25, 2015 at 10:22
  • 3
    If you replace math.exp with np.exp you won't get NaNs, although you will get runtime warnings. Jul 20, 2016 at 17:18
  • 2
    Using math.exp with numpy array can yield some errors, like: TypeError: only length-1 arrays can be converted to Python scalars. To avoid it you should use numpy.exp. Nov 23, 2017 at 13:41
  • 2
    Can numerical instability be mitigated simply by adding x = max(-709,x) before the expression? Feb 6, 2019 at 10:03
250

It is also available in scipy: http://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.logistic.html

In [1]: from scipy.stats import logistic

In [2]: logistic.cdf(0.458)
Out[2]: 0.61253961344091512

which is only a costly wrapper (because it allows you to scale and translate the logistic function) of another scipy function:

In [3]: from scipy.special import expit

In [4]: expit(0.458)
Out[4]: 0.61253961344091512

If you are concerned about performances continue reading, otherwise just use expit.

Some benchmarking:

In [5]: def sigmoid(x):
  ....:     return 1 / (1 + math.exp(-x))
  ....: 

In [6]: %timeit -r 1 sigmoid(0.458)
1000000 loops, best of 1: 371 ns per loop


In [7]: %timeit -r 1 logistic.cdf(0.458)
10000 loops, best of 1: 72.2 µs per loop

In [8]: %timeit -r 1 expit(0.458)
100000 loops, best of 1: 2.98 µs per loop

As expected logistic.cdf is (much) slower than expit. expit is still slower than the python sigmoid function when called with a single value because it is a universal function written in C ( http://docs.scipy.org/doc/numpy/reference/ufuncs.html ) and thus has a call overhead. This overhead is bigger than the computation speedup of expit given by its compiled nature when called with a single value. But it becomes negligible when it comes to big arrays:

In [9]: import numpy as np

In [10]: x = np.random.random(1000000)

In [11]: def sigmoid_array(x):                                        
   ....:    return 1 / (1 + np.exp(-x))
   ....: 

(You'll notice the tiny change from math.exp to np.exp (the first one does not support arrays, but is much faster if you have only one value to compute))

In [12]: %timeit -r 1 -n 100 sigmoid_array(x)
100 loops, best of 1: 34.3 ms per loop

In [13]: %timeit -r 1 -n 100 expit(x)
100 loops, best of 1: 31 ms per loop

But when you really need performance, a common practice is to have a precomputed table of the the sigmoid function that hold in RAM, and trade some precision and memory for some speed (for example: http://radimrehurek.com/2013/09/word2vec-in-python-part-two-optimizing/ )

Also, note that expit implementation is numerically stable since version 0.14.0: https://github.com/scipy/scipy/issues/3385

6
  • 4
    By using floats (1.) instead of ints (1) in your sigmoid function you would reduce running time by ~10%
    – kd88
    Mar 11, 2016 at 15:25
  • I'm not sure I understand what you mean (floats are used in the examples), but in any case one rarely computes a sigmoid on intergers.
    – Théo T
    Mar 11, 2016 at 17:27
  • 2
    What kd88 meant to say was that the numeric literals you used in your function (1) are parsed as integers, and have to be cast at runtime to floats. You would get better performance using floating point literals (1.0).
    – krs013
    Apr 3, 2017 at 22:54
  • 1
    You can always vectorize the function so it would support arrays.
    – user3103059
    May 14, 2018 at 11:11
  • you want to talk about an expensive wrapper? %timeit -r 1 expit(0.458) %timeit -r 1 1/(1+np.exp(0.458)) Oct 18, 2019 at 20:42
61

Here's how you would implement the logistic sigmoid in a numerically stable way (as described here):

def sigmoid(x):
    "Numerically-stable sigmoid function."
    if x >= 0:
        z = exp(-x)
        return 1 / (1 + z)
    else:
        z = exp(x)
        return z / (1 + z)

Or perhaps this is more accurate:

import numpy as np

def sigmoid(x):  
    return np.exp(-np.logaddexp(0, -x))

Internally, it implements the same condition as above, but then uses log1p.

In general, the multinomial logistic sigmoid is:

def nat_to_exp(q):
    max_q = max(0.0, np.max(q))
    rebased_q = q - max_q
    return np.exp(rebased_q - np.logaddexp(-max_q, np.logaddexp.reduce(rebased_q)))

(However, logaddexp.reduce could be more accurate.)

10
  • referring to the multinomial logistic sigmoid (softmax), if I also wanted a temperature parameter for Reinforcement learning, does it suffice to divide max_q and rebased_q by tau ? because I tried that and I don't get probabilities that sum to 1 Jan 27, 2017 at 18:48
  • @CiprianTomoiaga If you want to have a temperature, just divide your evidence (q) by your temperature. rebased_q can be anything: it doesn't change the answer; it improves the numerical stability.
    – Neil G
    Jan 27, 2017 at 18:50
  • are you sure nat_to_exp is equivalent to softmax (as you mentioned in your other answer) ? Copy-paste of it returns probabilities that don't sum to 1 Jan 27, 2017 at 19:04
  • 1
    Didn't you mean np.exp(-np.logaddexp(0, -x))? (note np instead of math) Aug 30, 2017 at 8:19
  • 1
    Got it. My concern was that math does not work on arrays Aug 30, 2017 at 12:40
11

Another way by transforming the tanh function:

sigmoid = lambda x: .5 * (math.tanh(.5 * x) + 1)
1
  • @NeilG Mathematically, sigmoid(x) == (1 + tanh(x/2))/2. So this is a valid solution, though the numerically stabilised methods are superior.
    – scottclowe
    Oct 26, 2018 at 19:01
9

I feel many might be interested in free parameters to alter the shape of the sigmoid function. Second for many applications you want to use a mirrored sigmoid function. Third you might want to do a simple normalization for example the output values are between 0 and 1.

Try:

def normalized_sigmoid_fkt(a, b, x):
   '''
   Returns array of a horizontal mirrored normalized sigmoid function
   output between 0 and 1
   Function parameters a = center; b = width
   '''
   s= 1/(1+np.exp(b*(x-a)))
   return 1*(s-min(s))/(max(s)-min(s)) # normalize function to 0-1

And to draw and compare:

def draw_function_on_2x2_grid(x): 
    fig, ((ax1, ax2), (ax3, ax4)) = plt.subplots(2, 2)
    plt.subplots_adjust(wspace=.5)
    plt.subplots_adjust(hspace=.5)

    ax1.plot(x, normalized_sigmoid_fkt( .5, 18, x))
    ax1.set_title('1')

    ax2.plot(x, normalized_sigmoid_fkt(0.518, 10.549, x))
    ax2.set_title('2')

    ax3.plot(x, normalized_sigmoid_fkt( .7, 11, x))
    ax3.set_title('3')

    ax4.plot(x, normalized_sigmoid_fkt( .2, 14, x))
    ax4.set_title('4')
    plt.suptitle('Different normalized (sigmoid) function',size=10 )

    return fig

Finally:

x = np.linspace(0,1,100)
Travel_function = draw_function_on_2x2_grid(x)

Sigmoid functions graph

0
8

Use the numpy package to allow your sigmoid function to parse vectors.

In conformity with Deeplearning, I use the following code:

import numpy as np
def sigmoid(x):
    s = 1/(1+np.exp(-x))
    return s
7

another way

>>> def sigmoid(x):
...     return 1 /(1+(math.e**-x))
...
>>> sigmoid(0.458)
4
  • 1
    What is the difference between this and unwind's function? Is math.e**-x better than math.exp(-x)? Oct 21, 2010 at 9:07
  • There is no difference in terms of output result. If you want to know the difference in terms of speed, you can use timeit to time their execution. But that's really not important.
    – ghostdog74
    Oct 21, 2010 at 9:14
  • 10
    pow is often implemented in terms of exp and log, so using exp directly is almost certainly better.
    – japreiss
    Aug 1, 2013 at 18:10
  • 3
    This suffers from overflows when x is very negative.
    – Neil G
    Apr 25, 2015 at 11:09
3

Good answer from @unwind. It however can't handle extreme negative number (throwing OverflowError).

My improvement:

def sigmoid(x):
    try:
        res = 1 / (1 + math.exp(-x))
    except OverflowError:
        res = 0.0
    return res
1
  • This is better, but you are still suffering from numerical percussion issues with negative values.
    – Neil G
    Dec 25, 2015 at 20:09
3

Tensorflow includes also a sigmoid function: https://www.tensorflow.org/versions/r1.2/api_docs/python/tf/sigmoid

import tensorflow as tf

sess = tf.InteractiveSession()
x = 0.458
y = tf.sigmoid(x)

u = y.eval()
print(u)
# 0.6125396
1
3

A numerically stable version of the logistic sigmoid function.

    def sigmoid(x):
        pos_mask = (x >= 0)
        neg_mask = (x < 0)
        z = np.zeros_like(x,dtype=float)
        z[pos_mask] = np.exp(-x[pos_mask])
        z[neg_mask] = np.exp(x[neg_mask])
        top = np.ones_like(x,dtype=float)
        top[neg_mask] = z[neg_mask]
        return top / (1 + z)
1
  • 1
    if x is positive we are simply using 1 / (1 + np.exp(-x)) but when x is negative we are using the function np.exp(x) / (1 + np.exp(x)) instead of using 1 / (1 + np.exp(-x)) because when x is negative -x will be positive so np.exp(-x) can explode due to large value of -x.
    – Yash Khare
    May 15, 2018 at 12:22
2

A one liner...

In[1]: import numpy as np

In[2]: sigmoid=lambda x: 1 / (1 + np.exp(-x))

In[3]: sigmoid(3)
Out[3]: 0.9525741268224334
1

Vectorized method when using pandas DataFrame/Series or numpy array:

The top answers are optimized methods for single point calculation, but when you want to apply these methods to a pandas series or numpy array, it requires apply, which is basically for loop in the background and will iterate over every row and apply the method. This is quite inefficient.

To speed up our code, we can make use of vectorization and numpy broadcasting:

x = np.arange(-5,5)
np.divide(1, 1+np.exp(-x))

0    0.006693
1    0.017986
2    0.047426
3    0.119203
4    0.268941
5    0.500000
6    0.731059
7    0.880797
8    0.952574
9    0.982014
dtype: float64

Or with a pandas Series:

x = pd.Series(np.arange(-5,5))
np.divide(1, 1+np.exp(-x))
1
  • Or just pass the numpy array into expit.
    – Nic F
    May 22, 2023 at 19:03
1

you can calculate it as :

import math
def sigmoid(x):
  return 1 / (1 + math.exp(-x))

or conceptual, deeper and without any imports:

def sigmoid(x):
  return 1 / (1 + 2.718281828 ** -x)

or you can use numpy for matrices:

import numpy as np #make sure numpy is already installed
def sigmoid(x):
  return 1 / (1 + np.exp(-x))
1

You can simply declare 1 / np.exp(x) if putting - before x confuse you.

def sigmoid(x):
     return 1 /(1 + 1 / np.exp(x))

sigmoid(0.458)
0
import numpy as np

def sigmoid(x):
    s = 1 / (1 + np.exp(-x))
    return s

result = sigmoid(0.467)
print(result)

The above code is the logistic sigmoid function in python. If I know that x = 0.467 , The sigmoid function, F(x) = 0.385. You can try to substitute any value of x you know in the above code, and you will get a different value of F(x).

0

Below is the python function to do the same.

def sigmoid(x) :
    return 1.0/(1+np.exp(-x))
0

OP: Let's say x = 0.458

from scipy.special import expit
import math

def sigmoid(x):
  return 1 / (1 + math.exp(-x))

print(sigmoid(0.458))
print(expit(0.458))

# 0.6125396134409151
# 0.6125396134409151

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.