6

I have a series of unit tests that connect to an Azure Storage emulator. In Setup my code checks if there is something listening on the emulator's port, and if not sets a flag StorageNotAvailable.

In each of my tests I have some code...

if ( StorageNotAvailable )
    Assert.Inconclusive( "Storage emulator is not available" )

// where storage emulator is available, continue as normal

As expected, when the test returns void this reports correctly in the Test Explorer as "Inconclusive".

When the test is exercising some async methods, and the [TestMethod] signature returns Task then the test is reported in the TestExplorer as "Failed" instead of "Inconclusive".

How can I get an async method to report as Inconclusive?

EDIT

Some additional detail may be in order. Here are some sample tests I rigged up to demonstrate the problem I am seeing.

[TestMethod]
public void MyTestMethod()
{
    Assert.Inconclusive( "I am inconclusive" );
}

[TestMethod]
public async Task MyTestMethodAsync()
{
    Assert.Inconclusive( "I am an error" );
}

Image of the offending code in action

Test Explorer Error

Test Explorer Inconclusive

Some environment details may be in order as well:

  • Windows 10 x64 1703 Build 15063.608
  • Visual Studio Enterprise 2017 15.3.5
  • .NET 4.7.02046
  • VS Extensions that might be relevant
    • Microsoft Visual Studio Test Platform
    • MSTest V2 Create Unit Test Extension
    • MSTest V2 IntelliTest Extension
    • MSTest V2 Templates
  • Project references that might be relevant
    • Microsoft.VisualStudio.TestPlatform.TestFramework v14.0.0.0
    • Microsoft.VisualStudio.TestPlatform.TestFramework.Extensions v14.0.0.0
  • Project nuGet packages that might be relevant
    • MSTest.TestAdapter v1.1.18
    • MSTest.TestFramework v1.1.18
  • Project build target is .NET Framework v4.7
9
  • I took the exact code from your examples and again as expected it passed. So I am at a loss as to why you are have that experience with the assertion. Literally copy, paste, run. Only difference being that I am not using VS enterprise but community
    – Nkosi
    Sep 26, 2017 at 14:32
  • Same versions & all? Could this be a VS bug? Sep 26, 2017 at 14:39
  • community 15.3.26730.16
    – Nkosi
    Sep 26, 2017 at 14:40
  • went back to community 2015 and also 2013 and works. all using mstest
    – Nkosi
    Sep 26, 2017 at 14:41
  • Well i could suggest repairing the current VS installation and trying the test(s) again.
    – Nkosi
    Sep 26, 2017 at 14:42

2 Answers 2

5
+50

Assert.Inconclusive raises a special kind of exception, which will cause the task to catch that exception. Since the Task library and async don't know about, we can't blame them for complaining. The Task framework will wrap the exception in an AggregateException, which I suspect is getting reported. This was a nice assumption, but it turned out that the code looking for AssetInconclusiveException was comparing the raised instance against MstestV1's implementation and not MsTestV2.

But I suppose this should be considered a bug in the MsTest v2 runner, which should inspect all Tasks that failed and look at the exception that caused their failure.

The behaviour is a known behaviour at the moment and I've just submitted a PR to fix this. Pull Request Merged, now just wait for the next Nuget build to trigger.


This fix was merged and released in the latest 1.2.0 package.

1
  • Thanks for confirming this - I went looking for this bug but my Google-fu (or Bing-fu?) wasn't up to the job apparently. Glad I'm not going mad. Sep 27, 2017 at 23:40
0

This is caused by a confirmed bug in MsTest.Framework code -- issue #249 on GitHub is tracking the problem and eventual solution:

Async tests which use Assert.Inconclusive report as Error

https://github.com/Microsoft/testfx/issues/249

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.