160

So I'm trying to git clone and cd into that directory using Google collab - but I cant cd into it. What am I doing wrong?

!rm -rf SwitchFrequencyAnalysis && git clone https://github.com/ACECentre/SwitchFrequencyAnalysis.git

!cd SwitchFrequencyAnalysis

!ls datalab/ SwitchFrequencyAnalysis/

You would expect it to output the directory contents of SwitchFrequencyAnalysis - but instead its the root. I'm feeling I'm missing something obvious - Is it something to do with being within the python interpreter? (where is the documentation??)

Demo here.

1
  • 3
    Bob Smith's answer below is the right way to go, but if you're curious why the approach above didn't work: the issue is that each of the !-prefixed commands runs in its own subshell -- so the !cd starts a new shell, switches directories, and then kills that shell. The !ls then starts anew in the current directory. Doing !cd SwitchFrequencyAnalysis && ls would have worked, but using python's os.chdir is the cleaner approach here. Commented Jan 18, 2018 at 7:43

6 Answers 6

225

use

%cd SwitchFrequencyAnalysis

to change the current working directory for the notebook environment (and not just the subshell that runs your ! command).

you can confirm it worked with the pwd command like this:

!pwd

further information about jupyter / ipython magics: http://ipython.readthedocs.io/en/stable/interactive/magics.html#magic-cd

2
  • 15
    Why cd using % while pwd using ! as its prefix Commented May 26, 2020 at 10:49
  • 6
    @cloudscomputes As mentioned by other comments, there is a difference between ! and % such that let's say commands initiating with ! are global and run for the whole system, while commands with % are specific to the current environment (main environment vs a virtual environment for current project). Thus, the cd command should be started by % to tell the compiler that we are changing the directory for the current temporary subshell.
    – armiro
    Commented Mar 13, 2022 at 23:02
141

As others have pointed out, the cd command needs to start with a percentage sign:

%cd SwitchFrequencyAnalysis

Difference between % and !

Google Colab seems to inherit these syntaxes from Jupyter (which inherits them from IPython). Jake VanderPlas explains this IPython behaviour here. You can see the excerpt below.

If you play with IPython's shell commands for a while, you might notice that you cannot use !cd to navigate the filesystem:

In [11]: !pwd 
/home/jake/projects/myproject

In [12]: !cd ..

In [13]: !pwd 
/home/jake/projects/myproject 

The reason is that shell commands in the notebook are executed in a temporary subshell. If you'd like to change the working directory in a more enduring way, you can use the %cd magic command:

In [14]: %cd ..
/home/jake/projects

Another way to look at this: you need % because changing directory is relevant to the environment of the current notebook but not to the entire server runtime.

In general, use ! if the command is one that's okay to run in a separate shell. Use % if the command needs to be run on the specific notebook.

2
  • 2
    You can also just use "cd", without !cd or %cd. What does this do :)? Thanks!
    – Erik
    Commented Feb 21, 2022 at 10:50
  • 1
    @Erik As I can see thru google colab's documentation pop-up, cd is a pre-defined function that the %cd magic command is bound to it. When you type the plain cd it brings the function with specific shortcut arguments. Declaring no arguments means going back to the /root directory.
    – armiro
    Commented Mar 13, 2022 at 23:07
48

Use os.chdir. Here's a full example: https://colab.research.google.com/notebook#fileId=1CSPBdmY0TxU038aKscL8YJ3ELgCiGGju

Compactly:

!mkdir abc
!echo "file" > abc/123.txt

import os
os.chdir('abc')

# Now the directory 'abc' is the current working directory.
# and will show 123.txt.
!ls
1
  • 1
    great! so, does /content and /root points to the same folder location. I mean I have a hard time understanding how the commands in googleColab related to the underlying Linux-VM filesystem. Since either you execute %cd /content | ls or !ls -la | pwd you will end up seeing /root/.kaggle which was created using the command set !pip install -U -q kaggle followed by !mkdir -p ~/.kaggle, that means ~/ which is home directory is equivalent to /root & /content, are those
    – Anu
    Commented Feb 1, 2019 at 13:14
36

If you want to use the cd or ls functions , you need proper identifiers before the function names ( % and ! respectively) use %cd and !ls to navigate

.

!pwd    # to find the directory you're in ,
%cd ./samplefolder  #if you wanna go into a folder (say samplefolder)

or if you wanna go out of the current folder

%cd ../      

and then navigate to the required folder/file accordingly

3
  • 3
    could you please point to the source where I can find the above usage? also, I am looking for using pipeline operator | as we do use in bash shell scripting, any suggestions in that regards? Also, what the distinction between using % or !
    – Anu
    Commented Feb 1, 2019 at 12:54
  • !ls doesn't give the directory you're in but the list of subfolders
    – Timothee W
    Commented Oct 17, 2023 at 13:44
  • 1
    @TimotheeW thanks for pointing this out. Since i wrote this answer almost 5 years ago, i don't really remember whether !ls gave some sort of absolute path at that time. But !pwd seems to do the trick now, editing it accordingly. Thanks.
    – zsfVishnu
    Commented Oct 18, 2023 at 15:09
31
!pwd
import os
os.chdir('/content/drive/My Drive/Colab Notebooks/Data')
!pwd

view this answer for detailed explaination https://stackoverflow.com/a/61636734/11535267

2
  • 2
    Will also add that you have to first mount Google Drive in Colab and grant Google Drive File Stream authorization.
    – Princy
    Commented Aug 17, 2020 at 11:18
  • You saved my life. Commented Sep 16, 2021 at 22:08
3

I believe you'd have to mount the Google Drive first before you do anything else.

from google.colab import drive
drive.mount('/content/drive')

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.