4

i manage to connect and access a smb share server using pysmb. what i meant is to read/write/delete/create files/folders to/from the server.

majority of the time i need to read file ( be it jpg or csv and etc) from the server base on the smb device and service name (pysmb terms).

basically i have no idea what is the filename and directory name in the smb devices. meaning the naming is dynamics.

i am wondering is it a good idea to get the filtered directory tree first before processing read files. the numbers of files and directories is not known with about 3 months data about 60TB.

listShares(timeout=30)[source]
listPath(service_name, path, search=55, pattern='*', timeout=30)

the above methods about to get only 1 specific level of the hierarchy. what i want is the similar output from os.walk.path().

anybody have experience in idea? can i get suggestions? thank you very much.

3 Answers 3

9
def smbwalk(conn, shareddevice, top = u'/'):
    dirs , nondirs = [], []

    if not isinstance(conn, SMBConnection):
        raise TypeError("SMBConnection required")


    names = conn.listPath(shareddevice, top)

    for name in names:
        if name.isDirectory:
            if name.filename not in [u'.', u'..']:
                dirs.append(name.filename)
        else:
            nondirs.append(name.filename)

    yield top, dirs, nondirs

    for name in dirs:
        new_path = os.path.join(top, name)
        for x in smbwalk(conn, shareddevice, new_path):
            yield x


conn = SMBConnection(*con_str, domain='workgroup')
assert conn.connect('10.10.10.10')
ans = smbwalk(conn, 'SHARE_FOLDER',top= '/')

this is what i want, but i found out that if the network shares is too big, it is taking forever to return.

2
  • 1
    thanks for the help, i will accept your ans. thank you very much.
    – pinky
    Commented Oct 28, 2015 at 5:45
  • Thank you for this, very much !
    – Claude
    Commented Feb 5, 2022 at 11:55
5

Not sure if this is what you want. but i'm working on similar kind of stuff so here you go.

I use Impacket which actually use some base classes from pysmb. https://github.com/CoreSecurity/impacket

I hope your listPath method is returning output in text format and not SharedFile instance.

What i mean is, store below values while listing them.

get_longname is_directory get_filesize

I have tree method which traverse through share/path and checks if SharedFile instance is directory, & does recursive call to itself.

def tree(self, path):    
   for x in range(0, path.count('\\')):
            print '|  ',
    print '%s' % os.path.basename(path.replace('\\', '/'))

    self.do_ls('%s\\*' % path, pretty=False) #Stores files data in listdata[]

    for file, is_directory, size in self.listdata:
            if file in ('.', '..'):
                continue
            if is_directory > 0:
                self.tree(ntpath.join(path, file))
            else:
                for x in range(0, path.count('\\')):
                    print '|  ',
                print '|-- %s (%d bytes)' % (file, size)


>>>d.tree('test')
.snapshot
|   hourly.0
|   |   dir0
|   |   |   Test051-89
|   |   |   Test051_perf3100-test_43
|   |   |   |   Test051_perf3100-test_52
|   |-- a.txt (8 bytes)
|   |-- dir0 - Shortcut.lnk (1834 bytes)
|   |-- Thumbs.db (46080 bytes)
|   |   20743
|   |   |-- file.txt (82 bytes)
|   |   |-- link.txt (82 bytes)
|   |   |   targetdir
|   |   |   |-- file2.txt (39 bytes)
|   |-- target.txt (6394368 bytes)
|   |   linkdir
|   |   |-- file2.txt (39 bytes)
1
  • 1
    thank you very much. your code is nice. i have share my working also. please have a look if you have time.
    – pinky
    Commented Oct 26, 2015 at 3:58
1

Did you consider using threads ? quick idea is to get all the top level directories then use threads for all of them and use your smbwalk function. While tree walk, it does lookup on objects so it will take time. but you will see performance improvement using threads.

3
  • thanks for the idea. very good suggestion to me. i wonder what is the maximum threads in python. is it hardware and os dependent? for ex. i7 4 core max threads is 8. i believe i am not too correct about it.
    – pinky
    Commented Oct 29, 2015 at 20:27
  • my situation: 3 smb servers. 90 daily folders in each. 200+ folders in each daily folder. 20+ folders in each of the 200+ folder. 10+ folders in each of the 20+. many zips in the 10+ folders. each zip have atleast 5 level hierarchy. i need to return the absolute path of the zip base on the content in a file inside the zip. so user can download the zipfile based on a serial # or receipts #.
    – pinky
    Commented Oct 29, 2015 at 20:35
  • Its both actually, OS as well H/W dependent. 1>200+>20+>10+>~>read>return So you have to read the contents of the file and and then return the path based on "dont know what you look in particular". Normal dir listing i.e \\server\share* calls SMB2_FIND_BOTH_DIRECTORY_INFO. Which in turn returns, Time stamps, ea size, attributes, name, eof, allocation size, index, next offset & name length. You may want to check by reducing all those info to just index, next offset & name. Or if there are any read only optimizations available for your smbserver ? you can tweak them. All NAS servers have them
    – Rohit.M
    Commented Jan 28, 2016 at 18:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.