14

I'm testing the existence of a file in a remote share (on a Windows server). The underlying function used for testing is WinAPI's GetFileAttributes, and what happens is that function can take an inordinate amount of time (dozens of seconds) in various situations, like when the target server being offline, when there are rights or DNS issues, etc.

However, in my particular case, it's always a LAN access, so if the file can't be accessed in less than 1 second, then it typically won't be accessible by waiting dozens of seconds more...

Is there an alternative to GetFileAttributes that wouldn't stall? (apart from calling it in a thread and killing the thread after a timeout, which seems to bring its own bag of issues)

2
  • cant think of any thing except asynchronous model. Jul 17, 2009 at 8:49
  • In other circumstances, I've solved this issue by deploying a minimalist-web server to serve the shared files, as an HTTP request can easily be canceled/timeouted. But in this case, this is no solution for a variety of reasons (deployment headaches, security issues, etc.). Jul 17, 2009 at 12:34

3 Answers 3

8

The problem isn't GetFileAttributes really. It typically uses just one call to the underlying file system driver. It's that IO which is stalling.

Still, the solution is probably easy. Call CancelSynchronousIo() after one second (this obviously requires a second thread as your first is stuck inside GetFileAttributes).

4
  • 1
    Note that CancelSynchronousIo is not available in Windows XP. Nov 11, 2013 at 20:54
  • @MSalters : I am getting access is denied (5) error while using GetFileAttributes method. I have Window 2003 server with low hardware configuration. I tried the same call with permission disabled on other systems which worked perfectly. Can slow IO cause "access is denied" error.
    – Rahul KP
    Jul 11, 2016 at 9:09
  • @RahulKP: Quite unlikely.
    – MSalters
    Jul 11, 2016 at 22:46
  • 1
    FWIW calling CancelSynchronousIo() from a multimedia timer (timeSetEvent) seems to work fine, so the code only needs to setup the timer, call GetFileAttributes, and cancel the timer in a "finally" block. Dec 8, 2016 at 8:50
4

One cool thing about delegates is you can always BeginInvoke and EndInvoke them. Just make sure the called method doesn't throw an exception out since [I believe] it will cause a crash (unhandled exception).

AttributeType attributes = default(AttributeType);

Action<string> helper =
    (path) =>
    {
        try
        {
            // GetFileAttributes
            attributes = result;
        }
        catch
        {
        }
    };
IAsyncResult asyncResult = helper.BeginInvoke();
// whatever
helper.EndInvoke();
// at this point, the attributes local variable has a valid value.
2
  • 1
    So basically, no hope outside wrapping the API call in a thread? I was hoping for a solution outside threads, because killing a thread at a timeout isn't "clean" (by experience, bad things can happen), and ignoring stalled threads could potentially lead to a whole lot of stalled threads... Jul 17, 2009 at 12:30
  • Sorry, it also seems I mistakingly assumed you were working in .NET (answered a few of those before this). If the API doesn't have an asynchronous and/or timeout version available, then the threading solution may be the only reliable solution. Jul 17, 2009 at 16:38
0

I think your best solution is to use a thread-pool thread to perform the work.

  • assign a unit of work to query the attributes of a file
  • let GetFileAttributes run to completion
  • post the results back to your form
  • when your thread function completes, the thread automatically returns back to the pool (no need to kill it)

By using the thread pool you save the costs of creating new threads.
And you save the misery of trying to get rid of them.

Then you have your handy helper method that runs an object's method procedure on a thread-pool thread using QueueUserWorkItem:

RunInThreadPoolThread(
      GetFileAttributesThreadMethod, 
      TGetFileAttributesData.Create('D:\temp\foo.xml', Self.Handle), 
      WT_EXECUTEDEFAULT);

You create the object to hold the thread data information:

TGetFileAttributesData = class(TObject)
public
    Filename: string;
    WndParent: HWND;
    Attributes: DWORD;
    constructor Create(Filename: string; WndParent: HWND);
end;

and you create your thread callback method:

procedure TForm1.GetFileAttributesThreadMethod(Data: Pointer);
var
    fi: TGetFileAttributesData;
begin
    fi := TObject(Data) as TGetFileAttributesData;
    if fi = nil then
        Exit;

    fi.attributes := GetFileAttributes(PWideChar(fi.Filename));

    PostMessage(fi.WndParent, WM_GetFileAttributesComplete, NativeUInt(Data), 0);
end;

then you just handle the message:

procedure WMGetFileAttributesComplete(var Msg: TMessage); message WM_GetFileAttributesComplete;

procedure TfrmMain.WMGetFileAttributesComplete(var Msg: TMessage);
var
    fi: TGetFileAttributesData;
begin
    fi := TObject(Pointer(Msg.WParam)) as TGetFileAttributesData;
    try
        ShowMessage(Format('Attributes of "%s": %.8x', [fi.Filename, fi.attributes]));
    finally
        fi.Free;
    end;
end;

The magical RunInThreadPoolThread is just a bit of fluff that lets you execute an instance method in a thread:

Which is just a wrapper that lets you call method on an instance variable:

TThreadMethod = procedure (Data: Pointer) of object;

TThreadPoolCallbackContext = class(TObject)
public
    ThreadMethod: TThreadMethod;
    Context: Pointer;
end;

function ThreadPoolCallbackFunction(Parameter: Pointer): Integer; stdcall;
var
    tpContext: TThreadPoolCallbackContext;
begin
    try
        tpContext := TObject(Parameter) as TThreadPoolCallbackContext;
    except
        Result := -1;
        Exit;
    end;
    try
        tpContext.ThreadMethod(tpContext.Context);
    finally
        try
            tpContext.Free;
        except
        end;
    end;
    Result := 0;
end;

function RunInThreadPoolThread(const ThreadMethod: TThreadMethod; const Data: Pointer; Flags: ULONG): BOOL;
var
    tpContext: TThreadPoolCallbackContext;
begin
    {
        Unless you know differently, the flag you want to use is 0 (WT_EXECUTEDEFAULT).

        If your callback might run for a while you can pass the WT_ExecuteLongFunction flag.
                Sure, I'm supposed to pass WT_EXECUTELONGFUNCTION if my function takes a long time, but how long is long?
                http://blogs.msdn.com/b/oldnewthing/archive/2011/12/09/10245808.aspx

        WT_EXECUTEDEFAULT (0):
                By default, the callback function is queued to a non-I/O worker thread.
                The callback function is queued to a thread that uses I/O completion ports, which means they cannot perform
                an alertable wait. Therefore, if I/O completes and generates an APC, the APC might wait indefinitely because
                there is no guarantee that the thread will enter an alertable wait state after the callback completes.
        WT_EXECUTELONGFUNCTION (0x00000010):
                The callback function can perform a long wait. This flag helps the system to decide if it should create a new thread.
        WT_EXECUTEINPERSISTENTTHREAD (0x00000080)
                The callback function is queued to a thread that never terminates.
                It does not guarantee that the same thread is used each time. This flag should be used only for short tasks
                or it could affect other timer operations.
                This flag must be set if the thread calls functions that use APCs.
                For more information, see Asynchronous Procedure Calls.
                Note that currently no worker thread is truly persistent, although worker threads do not terminate if there
                are any pending I/O requests.
    }

    tpContext := TThreadPoolCallbackContext.Create;
    tpContext.ThreadMethod := ThreadMethod;
    tpContext.Context := Data;

    Result := QueueUserWorkItem(ThreadPoolCallbackFunction, tpContext, Flags);
end;

Exercise for the reader: Create a Cancelled flag inside the GetFileAttributesData object that tells the thread that it must free the data object and not post a message to the parent.


It's all a long way of saying that you're creating:

DWORD WINAPI GetFileAttributes(
  _In_      LPCTSTR                         lpFileName,
  _Inout_   LPOVERLAPPED                    lpOverlapped,
  _In_      LPOVERLAPPED_COMPLETION_ROUTINE lpCompletionRoutine
);
4
  • Using pooled threads does not solve the problem that the threads can be stalled for a very long time as GetFileAttributes is waiting for some networking timeouts. Also one needs to ignore irrelevant threads. For instance if you query the same file twice with 10 sec, 1st call can get stuck for 30 sec, while the 2nd may succeed immediately (and you need to ignore the results of the first call). Also if you monitor multiple files with a frequency of a few seconds, it is "easy" to end up with dozens or even hundreds of stalled threads... not practical at all :/ Dec 6, 2016 at 8:21
  • Ignoring irrelevant threads is solved the same way as if Windows already natively provided an overlapped (i.e. asynchronous) GetFileAttributesEx version - you have to cancel the existing call. That is solved by the exercise. Your concern is what to do when you have dozens or hundreds of stalled threads. I would submit this is not a concern, as the user work item queue will queue items until older items are flushed from the queue. Although, anything you can do to help it along will return the thread to the pool quicker.
    – Ian Boyd
    Dec 6, 2016 at 18:47
  • It's also useful to not that QueueUserWorkItem will queue your items; and not create hundreds of threads. One of the purposes of QueueUserWorkItem is to let you queue the work items - the thread pool decides when they will execute.
    – Ian Boyd
    Dec 6, 2016 at 19:34
  • If the queue has a limited size, then stalled calls will prevent (or delay) other calls (which could be fast otherwise), so this creates a bottleneck. Calling CancelSynchronousIo() in a multimedia timer seems to work AFAICT, and the timer can be canceled in a "finally" block after the GetFileAttributes. Dec 8, 2016 at 8:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.