425

The below code will not join, when debugged the command does not store the whole path but just the last entry.

os.path.join('/home/build/test/sandboxes/', todaystr, '/new_sandbox/')

When I test this it only stores the /new_sandbox/ part of the code.

0

16 Answers 16

563

The latter strings shouldn't start with a slash. If they start with a slash, then they're considered an "absolute path" and everything before them is discarded.

Quoting the Python docs for os.path.join:

If a component is an absolute path, all previous components are thrown away and joining continues from the absolute path component.

Note on Windows, the behaviour in relation to drive letters, which seems to have changed compared to earlier Python versions:

On Windows, the drive letter is not reset when an absolute path component (e.g., r'\foo') is encountered. If a component contains a drive letter, all previous components are thrown away and the drive letter is reset. Note that since there is a current directory for each drive, os.path.join("c:", "foo") represents a path relative to the current directory on drive C: (c:foo), not c:\foo.

15
  • 108
    -1: No string should include a "/". One whole point of os.path.join is to prevent putting any slashes in the path.
    – S.Lott
    Commented Dec 22, 2009 at 12:29
  • 11
    The problem with str.join() is, of course, that it won't eliminate double slashes. I think this is the primary purpose for folks using os.path.join. e.g. '/'.join(['/etc/', '/conf']) results in three slashes: '/etc///conf' Commented Jul 31, 2012 at 14:03
  • 20
    @DustinRasener You can use os.path.normpath to achieve that aim. Commented Oct 28, 2012 at 17:48
  • 8
    no clue why people are frustrated over os.path.join behavior. In other languages, the equivalent path-join library/method behaves the exact same. It's safer, and makes more sense. Commented Dec 4, 2014 at 17:41
  • 33
    This is frustrating because it's implicit magic, contrary to the cardinal heuristic of "Explicit is better than implicit." And it is. Language designers may believe they know better, but there exist obvious and demonstrably safe reasons to occasionally want to do this. Now we can't. This is why we can't have good things. Commented Aug 18, 2015 at 5:54
171

The idea of os.path.join() is to make your program cross-platform (linux/windows/etc).

Even one slash ruins it.

So it only makes sense when being used with some kind of a reference point like os.environ['HOME'] or os.path.dirname(__file__).

2
  • 1
    The prepended slash in the relative path is what was screwing it up for me... removing it fixed it.. thanks!
    – joeyagreco
    Commented Nov 13, 2021 at 21:28
  • 1
    Upvoted for "Even one slash ruins it."
    – wim
    Commented May 31, 2023 at 21:01
96

os.path.join() can be used in conjunction with os.path.sep to create an absolute rather than relative path.

os.path.join(os.path.sep, 'home','build','test','sandboxes',todaystr,'new_sandbox')
4
  • 11
    The use of os.path.sep as a first element to build an absolute path is better than any other answer here! The whole point of using os.path rather than basic str methods is to avoid writing /. Putting every subdirectory as a new argument and removing all slashes is also great. It would probably be a good idea to make sure with a check that todaystr does not start with a slash! ;)
    – snooze92
    Commented Jan 23, 2014 at 8:59
  • 5
    This works on windows as well (python 2.7.6). It did not intefere with 'C:\' and joined the subdirectories. Commented Feb 2, 2015 at 21:22
  • 2
    @snooze92 The only problem is that it's more verbose and much less readable.
    – yugr
    Commented Jan 14, 2022 at 3:00
  • 1
    Indeed. Trade-offs I guess ;)
    – snooze92
    Commented Jan 14, 2022 at 14:26
36

Do not use forward slashes at the beginning of path components, except when refering to the root directory:

os.path.join('/home/build/test/sandboxes', todaystr, 'new_sandbox')

see also: http://docs.python.org/library/os.path.html#os.path.join

0
33
+400

To help understand why this surprising behavior isn't entirely terrible, consider an application which accepts a config file name as an argument:

config_root = "/etc/myapp.conf/"
file_name = os.path.join(config_root, sys.argv[1])

If the application is executed with:

$ myapp foo.conf

The config file /etc/myapp.conf/foo.conf will be used.

But consider what happens if the application is called with:

$ myapp /some/path/bar.conf

Then myapp should use the config file at /some/path/bar.conf (and not /etc/myapp.conf/some/path/bar.conf or similar).

It may not be great, but I believe this is the motivation for the absolute path behaviour.

8
  • 2
    Thanks! I had always hated this behavior until reading your answer! It's documented in docs.python.org/3.5/library/os.path.html#os.path.join, but not the motivation for it.
    – Eli_B
    Commented Sep 7, 2017 at 11:59
  • 2
    This moment when you need exactly the solution many people deem as terrible.
    – ashrasmun
    Commented Sep 4, 2019 at 20:33
  • 7
    gotta disagree, it is entirely terrible. In this case, you should not be using a naive sys.argv input to determine whether to prepend the config_root or not. All os.path.join should be concerned with is joining file path elements. Commented Sep 1, 2020 at 20:53
  • 2
    the problem is that the name is badly chosen. what os.path.join(p1, p2) does is not really joining p1 and p2, it's taking p2 relatively to p1 (or absolutely if p2 is an absolute path)
    – tbrugere
    Commented May 17, 2021 at 8:12
  • 1
    So a specific use case should determine how a function behaves for all other use cases? Makes no sense.
    – Gulzar
    Commented Nov 8, 2021 at 8:39
16

It's because your '/new_sandbox/' begins with a / and thus is assumed to be relative to the root directory. Remove the leading /.

16

Try combo of split("/") and * for strings with existing joins.

import os

home = '/home/build/test/sandboxes/'
todaystr = '042118'
new = '/new_sandbox/'

os.path.join(*home.split("/"), todaystr, *new.split("/"))


How it works...

split("/") turns existing path into list: ['', 'home', 'build', 'test', 'sandboxes', '']

* in front of the list breaks out each item of list its own parameter

2
  • 2
    this turns it into a relative path and not an absolute path Commented Sep 1, 2020 at 21:00
  • Perfect for working with endpoints with leading slashes. Thanks!
    – Wyrmwood
    Commented Apr 13, 2022 at 3:02
10

To make your function more portable, use it as such:

os.path.join(os.sep, 'home', 'build', 'test', 'sandboxes', todaystr, 'new_sandbox')

or

os.path.join(os.environ.get("HOME"), 'test', 'sandboxes', todaystr, 'new_sandbox')
5

Try with new_sandbox only

os.path.join('/home/build/test/sandboxes/', todaystr, 'new_sandbox')
4

do it like this, without too the extra slashes

root="/home"
os.path.join(root,"build","test","sandboxes",todaystr,"new_sandbox")
1
  • os.sep instead of / in the beggining
    – Kots
    Commented Jul 1, 2021 at 15:24
1
os.path.join("a", *"/b".split(os.sep))
'a/b'

a fuller version:

import os

def join (p, f, sep = os.sep):
    f = os.path.normpath(f)
    if p == "":
        return (f);
    else:
        p = os.path.normpath(p)
        return (os.path.join(p, *f.split(os.sep)))

def test (p, f, sep = os.sep):
    print("os.path.join({}, {}) => {}".format(p, f, os.path.join(p, f)))
    print("        join({}, {}) => {}".format(p, f, join(p, f, sep)))

if __name__ == "__main__":
    # /a/b/c for all
    test("\\a\\b", "\\c", "\\") # optionally pass in the sep you are using locally
    test("/a/b", "/c", "/")
    test("/a/b", "c")
    test("/a/b/", "c")
    test("", "/c")
    test("", "c")
3
  • What if os.sep is actually "\"? Then your first example becomes os.path.join("a", *"/b".split("\\")), which yields "/b"... I doubt that's the intended result.
    – NichtJens
    Commented May 16, 2020 at 22:13
  • 1
    Updated - I suppose you have to give a hint as the path sep you are using locally independent of that of the os you are running on
    – Goblinhack
    Commented May 17, 2020 at 1:54
  • 1
    Yes. Alternatively one could split on both commonly used options ... but then some other OS could come up with a third.
    – NichtJens
    Commented May 17, 2020 at 9:30
1

Please refer following code snippet for understanding os.path.join(a, b)

a = '/home/user.name/foo/'
b = '/bar/file_name.extension'

print(os.path.join(a, b))
>>> /bar/file_name.extension

OR

a = '/home/user.name/foo'
b = '/bar/file_name.extension'
print(os.path.join(a, b))
>>> /bar/file_name.extension

But, when

a = '/home/user.name/foo/'
b = 'bar/file_name.extension'

print(os.path.join(a, b))
>>> /bar/file_name.extension

OR

a = '/home/user.name/foo'
b = 'bar/file_name.extension'
print(os.path.join(a, b))
>>> /home/user.name/foo/bar/file_name.extension
0

Note that a similar issue can bite you if you use os.path.join() to include an extension that already includes a dot, which is what happens automatically when you use os.path.splitext(). In this example:

components = os.path.splitext(filename)
prefix = components[0]
extension = components[1]
return os.path.join("avatars", instance.username, prefix, extension)

Even though extension might be .jpg you end up with a folder named "foobar" rather than a file called "foobar.jpg". To prevent this you need to append the extension separately:

return os.path.join("avatars", instance.username, prefix) + extension
0

you can strip the '/':

>>> os.path.join('/home/build/test/sandboxes/', todaystr, '/new_sandbox/'.strip('/'))
'/home/build/test/sandboxes/04122019/new_sandbox'
0

The problem is your laptop maybe running Window. And Window annoyingly use back lash instead of forward slash'/'.
To make your program cross-platform (linux/windows/etc). You shouldn't provide any slashes (forward or backward) in your path if you want os.path.join to handle them properly. you should using:

os.path.join(os.environ.get("HOME"), 'test', 'sandboxes', todaystr, 'new_sandbox')

Or throw some Path(__file__).resolve().parent (path to parent of current file) or anything so that you don't use any slash inside os.path.join

-1

I'd recommend to strip from the second and the following strings the string os.path.sep, preventing them to be interpreted as absolute paths:

first_path_str = '/home/build/test/sandboxes/'
original_other_path_to_append_ls = [todaystr, '/new_sandbox/']
other_path_to_append_ls = [
    i_path.strip(os.path.sep) for i_path in original_other_path_to_append_ls
]
output_path = os.path.join(first_path_str, *other_path_to_append_ls)

Not the answer you're looking for? Browse other questions tagged or ask your own question.