198

I have installed the GPU version of tensorflow on an Ubuntu 14.04.

I am on a GPU server where tensorflow can access the available GPUs.

I want to run tensorflow on the CPUs.

Normally I can use env CUDA_VISIBLE_DEVICES=0 to run on GPU no. 0.

How can I pick between the CPUs instead?

I am not intersted in rewritting my code with with tf.device("/cpu:0"):

13 Answers 13

211

You can also set the environment variable to

CUDA_VISIBLE_DEVICES=""

without having to modify the source code.

12
  • 3
    Someone said running neural nets on CPUs after the training phase is as performant as running them on GPUs -- i.e., only the training phrase really needs the GPU. Do you know if this is true? Thanks!
    – Crashalot
    Nov 7, 2016 at 19:03
  • 13
    @Crashalot: This is not true. Look for various benchmarks for interference, CPUs are an order of magnitude slower there too.
    – Thomas
    Nov 17, 2016 at 13:58
  • 1
    @Thomas thanks. suggestions on which benchmarks to consider? probably also varies on workload and nature of the neural nets, right? apparently the google translate app runs some neural nets directly on smartphones, presumably on the cpu and not gpu?
    – Crashalot
    Nov 17, 2016 at 19:44
  • @fabrizioM, a toy example will be more useful. Jun 8, 2017 at 10:54
  • 8
    This did not work for me. :/ set the environment variable but tensorflow still uses the GPU, I'm using conda virtual env, does this make a diference? Aug 6, 2017 at 15:56
164

If the above answers don't work, try:

os.environ['CUDA_VISIBLE_DEVICES'] = '-1'
3
  • 3
    Just for the record, the first option doesn't seem to work anymore.
    – user3103059
    Apr 5, 2019 at 9:08
  • 1
    Works also for tf 2.X when using tf.keras.Sequential models.
    – Nicolas M.
    Jun 29, 2020 at 16:26
  • 1
    Is there a way to do this without tensorflow invoking the error message "CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected" ?
    – Nermin
    Mar 22, 2021 at 14:58
139

You can apply device_count parameter per tf.Session:

config = tf.ConfigProto(
        device_count = {'GPU': 0}
    )
sess = tf.Session(config=config)

See also protobuf config file:

tensorflow/core/framework/config.proto

9
  • 3
    Someone said running neural nets on CPUs after the training phase is as efficient as running them on GPUs -- i.e., only the training phrase really needs the GPU. Do you know if this is true? Thanks!
    – Crashalot
    Nov 7, 2016 at 19:03
  • 4
    That doesn't work for me (tf1.1). The solution of fabrizioM does.
    – P-Gn
    May 29, 2017 at 14:42
  • 3
    Isn't it better to use CUDA_VISIBLE_DEVICES environment variable instead of changing the config in the code?
    – Nandeesh
    Jun 30, 2017 at 15:22
  • 3
    @Nandeesh I guess it depends on your needs. So far there are at least 53 people who feel more into environment variables and 35 who prefer to set number of devices in code. The advantage of first is simplicity and of another is more explicit control over (multiple) sessions from within the python program itself (that zero is not necessary to be hardcoded, it can be a variable). Jun 30, 2017 at 16:58
  • 4
    Do you know how to adapt this to tensorflow 2.0, as there is no more session or configproto? Jul 30, 2019 at 13:25
59

The environment variable solution doesn't work for me running tensorflow 2.3.1. I assume by the comments in the github thread that the below solution works for versions >=2.1.0.

From tensorflow github:

import tensorflow as tf

# Hide GPU from visible devices
tf.config.set_visible_devices([], 'GPU')

Make sure to do this right after the import with fresh tensorflow instance (if you're running jupyter notebook, restart the kernel).

And to check that you're indeed running on the CPU:

# To find out which devices your operations and tensors are assigned to
tf.debugging.set_log_device_placement(True)

# Create some tensors and perform an operation
a = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])
c = tf.matmul(a, b)

print(c)

Expected output:

2.3.1
Executing op MatMul in device /job:localhost/replica:0/task:0/device:CPU:0
tf.Tensor(
[[22. 28.]
 [49. 64.]], shape=(2, 2), dtype=float32)
7
  • 3
    Working with tensorflow and pytorch in one script, this approach help me to disable cuda on tensorflow but still make the pytorch use cuda. I believe this answer deserved more votes. Apr 26, 2021 at 8:56
  • 2
    A potential advantage of this solution is that it doesn't rely on a variable that explicitly mentions CUDA and which might as such be reserved to specific devices. For example, it works on my Apple Silicon Mac. Jul 1, 2021 at 11:02
  • Best solution, thx (because I'm on a Silicon too :D)
    – ker2x
    Dec 8, 2021 at 3:36
  • works for me like a charm. In Jupyter notebook, just follow these steps (based on above comment) - Restart the Kernel --> put this line just beneath the tensorflow: import tf.config.set_visible_devices([], 'GPU') --> run your script
    – madhur
    Feb 19, 2022 at 0:56
  • this finally worked for me on tensorflow 2.7.0, thanks!
    – Olympia
    May 6, 2022 at 15:31
39

For me, only setting CUDA_VISIBLE_DEVICES to precisely -1 works:

Works:

import os
import tensorflow as tf

os.environ['CUDA_VISIBLE_DEVICES'] = '-1'

if tf.test.gpu_device_name():
    print('GPU found')
else:
    print("No GPU found")

# No GPU found

Does not work:

import os
import tensorflow as tf

os.environ['CUDA_VISIBLE_DEVICES'] = ''    

if tf.test.gpu_device_name():
    print('GPU found')
else:
    print("No GPU found")

# GPU found
3
  • 2
    hi, does not work for me...I'm using tensorflow-gpu 2.4.1
    – JamesAng
    Mar 17, 2021 at 2:50
  • Also failed for me on Windows with TF 2.7.
    – brethvoice
    Jun 28, 2022 at 15:35
  • "GPU found" even of tf2.0
    – MosQuan
    Mar 29 at 0:44
14

As recommended by the Tensorflow GPU guide.

# Place tensors on the CPU
with tf.device('/CPU:0'):
  a = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
  b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])
  # Any additional tf code placed in this block will be executed on the CPU

8

In my case, for tensorflow 2.4.0, none of the previous answer works unless you install tensorflow-cpu

pip install tensorflow-cpu
4
  • This works for tensorflow 2.5. But, I think my GPU for tensorflow 2.5 is no longer usable in the current environment after running the command. **(I tried the recommended way above and it doesn't work)
    – jona
    May 10, 2021 at 17:14
  • How to use this tensorflow-cpu with tf.io.decode_image?
    – agata
    Jan 20, 2022 at 9:04
  • Thank you, this also worked for TF 2.7 on Windows (Python 3.8)
    – brethvoice
    Jun 28, 2022 at 15:46
  • Still works and seems up-to-date to this day, thanks.
    – Andrew
    Jul 14 at 20:09
7

Just using the code below.

import os
os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
os.environ['CUDA_VISIBLE_DEVICES'] = '-1'
1
  • Did not work for me in Windows using TF 2.7
    – brethvoice
    Jun 28, 2022 at 15:37
4

Another possible solution on installation level would be to look for the CPU only variant

In my case, this gives right now:

pip3 install https://storage.googleapis.com/tensorflow/windows/cpu/tensorflow_cpu-2.2.0-cp38-cp38-win_amd64.whl

Just select the correct version (in this case, cp38 hints python 3.8 - moreover, Tensorflow 2.2.0 is used, the current version as of Jul 12 '20).


Bonus points for using a venv like explained eg in this answer.

3

1. Fabrizio's answer worked for me:

import os
os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"]="-1"

I had to make the change before importing tensorflow. I am using tensorflow 2.4.0

2. Another (sub par) solution could be to rename the cusolver64_10.dll file that is required for gpu computing. Since tensorflow can't find the dll, it will automatically use the CPU.

It should be in a place like: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\bin

2

You could use tf.config.set_visible_devices. One possible function that allows you to set if and which GPUs to use is:

import tensorflow as tf

def set_gpu(gpu_ids_list):
    gpus = tf.config.list_physical_devices('GPU')
    if gpus:
        try:
            gpus_used = [gpus[i] for i in gpu_ids_list]
            tf.config.set_visible_devices(gpus_used, 'GPU')
            logical_gpus = tf.config.experimental.list_logical_devices('GPU')
            print(len(gpus), "Physical GPUs,", len(logical_gpus), "Logical GPU")
        except RuntimeError as e:
            # Visible devices must be set before GPUs have been initialized
            print(e)

Suppose you are on a system with 4 GPUs and you want to use only two GPUs, the one with id = 0 and the one with id = 2, then the first command of your code, immediately after importing the libraries, would be:

set_gpu([0, 2])

In your case, to use only the CPU, you can invoke the function with an empty list:

set_gpu([])

For completeness, if you want to avoid that the runtime initialization will allocate all memory on the device, you can use tf.config.experimental.set_memory_growth. Finally, the function to manage which devices to use, occupying the GPUs memory dynamically, becomes:

import tensorflow as tf

def set_gpu(gpu_ids_list):
    gpus = tf.config.list_physical_devices('GPU')
    if gpus:
        try:
            gpus_used = [gpus[i] for i in gpu_ids_list]
            tf.config.set_visible_devices(gpus_used, 'GPU')
            for gpu in gpus_used:
                tf.config.experimental.set_memory_growth(gpu, True)
            logical_gpus = tf.config.experimental.list_logical_devices('GPU')
            print(len(gpus), "Physical GPUs,", len(logical_gpus), "Logical GPU")
        except RuntimeError as e:
            # Visible devices must be set before GPUs have been initialized
            print(e)
1
# this works on tensorflow 2.8, windows 10, jupyterlab Version 3.3.2
# this is the very FIRST lines of code

import tensorflow as tf

tf.config.set_visible_devices([], 'GPU')

# if tf.test.gpu_device_name(): # this lies and tells you about all devices
if tf.config.experimental.list_logical_devices('GPU'):
    print('GPU found')
else:
    print("No GPU found")

I spent way too many hours trying to figure this out. MOST attempts left the process running partially on CPU and still loading into GPU memory?? Strange ...

Running the above code FIRST, before anything else worked.

I was able to increase my hidden later from 6k to 12k. It is running now, only using the CPU. Each epoch is taking about 10x as long as on the GPU. From about 3 minutes per epoch to a bit over 35 minutes per epoch. This is an acceptable trade-off. Training Time vs Model Size.

1
  • @user2489629 you might want to try listing physical devices instead of logical.
    – brethvoice
    Jun 28, 2022 at 16:25
-1

In some systems one have to specify:

import os
os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"]=""  # or even "-1"

BEFORE importing tensorflow.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.