11

I'm currently running tests through visual studio. Before all the tests are run, I automatically create a set number of users with know credentials, and at the end of the run I delete those users. However, sometimes I need to cancel my tests midway. In these cases the test never gets the chance to clean up, this means that there is left over fake user info from the test run and may causes the next test run to crash (when it attempts to add user info into the DB). Is there anyway to force visual studio/mstest to run a clean up method even if the test is canceled?

I know one option is to have the test check and make sure that the user info doesn't already exist, and if it does remove it before creating the new users. But this still wouldn't solve the issue of the canceled test run leaving unwanted test data.

Update:

Sorry for the miscommunication, however cleaning up the data at the start of the test is not an option. I'm giving a very simplistic view of the issue, but put simply, I have no easy way of making sure that no test data exists at the start of the test. All clean up must occur at the end of the test.

5
  • 1
    I would subscribe to an eventhandler that triggers when you cancel or when it finishes naturally, and in that event do the cleanup
    – bill
    Jun 11, 2015 at 16:52
  • 1
    What if you perform the cleanup before the tests run?
    – Matthew
    Jun 11, 2015 at 17:02
  • Why can't you delete the data before the test? Its your only option as cancelling the test from the debugger stops all execution
    – Sam Holder
    Jun 11, 2015 at 19:02
  • @SamHolder I have no way of getting the data once the test is stopped. I can remove it manually, but with the tools I have I can't really remove it through my code at this point, which is why I would like to remove it after the test is canceled if possible.
    – Mo H.
    Jun 11, 2015 at 19:22
  • I'm not sure why you are distinguishing between "the start of the tests" and "the end of the tests". Between any two runs these are essentially the same thing. You say you can remove the test data manually - what does this involve? Nov 20, 2017 at 16:50

10 Answers 10

13
+200

That is impossible. You better find an alternative solution like using separate database for testing and clean all data before each test run, using fixed set of test users or mark test data with some flag. Check Isolating database data in integration tests article by Jimmy Bogard.

There is no built-in way to change MSTest default behavior. In theory you can write MSTest extension that utilizes TestExecution.OnTestStopping event, but that is not an easy process and it requires registry change. Moreover, a lot of people complain that it is not working.

There also MSTest V2, a new version of MSTest with new extensibility points. But it looks like you can't alter cancel behavior with this points, only write attribute decorators. See Extending MSTest V2.

You can't use AppDomain.CurrentDomain.ProcessExit and Process.GetCurrentProcess().Exited events because cancel seems to kill test run process.

NUnit also doesn't support this at the moment. See related NUnit test adapter Run TearDowns on VS Cancel Test Run issue.

1
  • Thanks everyone for all the new answers. It is obvious that some would work well in specific circumstances and not under others. This answer is the closest to what I needed right now for my current situation. Nov 23, 2017 at 2:08
7

Instead of calling the cleanup function at the end of the test, I call mine at the beginning of each test in order to address this exact problem.

5

Perform the clean up before creating the data as well, this will ensure that you have no leftover data whatever happens. Of course this is only possible if you can identify any leftover data before running the setup.

4

The idea is that a transaction is initialized before the test begins. In order for the data to be saved in the database, the transaction must be commited, but it is not commited never. It works in case when test a stop, in the case of a successful or unsuccessful completion of the test.

In integration tests we use somthing like this (with NUnit)(It real production code)

public class RollbackAttribute : TestAttribute, ITestAction
{
    private TransactionScope _transaction;

    public void BeforeTest(ITest test)
    {
        _transaction = new TransactionScope();
    }

    public void AfterTest(ITest test)
    {
        _transaction.Dispose();
    }

    public ActionTargets Targets => ActionTargets.Test;
}

[TestFixture]
public class SomeTestClass
{
    [Rollback] //No need [Test] because Rollback is inherit it.
    public void SomeTestMethod()
    {
    }
}

On MsTest you can make somthing similar, but in this case you should inherit from base class, I hope it works. For example:

public class RollbackTestBase
{
    private TransactionScope _transaction;

    [TestInitialize]
    public virtual void Setup()
    {
        _transaction = new TransactionScope();
    }


    [TestCleanup]
    public virtual void TearDown()
    {
        _transaction.Dispose();
    }
}

[TestClass]
public class IntegrationTest : RollbackTestBase
{
    [TestMethod]
    public void TestDataBase()
    {
        Assert.IsTrue(true);
    }

    [TestInitialize]
    public virtual void Init()
    {
    }

    [TestCleanup]
    public virtual void CleanUp()
    {
    }
}
3

There are 2 cases which we need to consider while allocating resources in ATPs (Resources might be Creating users, Connection with database). They are

  • Creation and deletion of resources after each test.
  • Creation and deletion of resources after set of tests.

Creation and deletion of resources after each test:

If we want to create instance of particular object before execution of a test and want to clean up memory allocated to that object after execution of that test, then we use Test SetUp and Test TearDown attributes of NUnit. In your case object is creation of number of Users.

[SetUp] : Function which is decorated with Test SetUp attribute contains piece of code that executes before the execution of any test.

[TearDown] : Function which is decorated with Test TearDown attributes contains piece of code that executes after execution of any test

Implementation:

[TestClass]
    public class UnitTest1
    {
        [SetUp]
        public void SetUP()
        {
            // Creating Users with proper credentials
        }

        [TestMethod]
        public void TestMethod1()
        {
            //Write your ATP
        }

        [TearDown]
        public void TearDown()
        {
            //Clean up
        }
    }

Creation and deletion of resources after set of tests:

Now If we want to create instance of an object for set of tests and want to clean up memory after execution of all tests then [TestFixtureSetUp] and [TestFixureTearDown] to initialize an object and to clean up memory respectively. Again In your case object can be creation of set of users.

[TestFixtureSetUp] : Function decorated with TestFixtureSetUp will executes once before the execution of group of tests.

[TestFixtureTearDown] : Function decorated with TestFixtureTearDown will executes once after the execution of group of tests.

Implementation

[TestFixture]
public class Tests
{
        [TestFixtureSetUp]
        public void Setup()
        {
           //Create users with credentials
        }

        [Test]
        public void _Test1()
        {
          //Test_1
        }
        [Test]
        public void _Test2()
        {
         //Test2
         }

        [TestFixtureTearDown]
        public void CleanUp()
        {
            //Cleanup; Here you need to add code to Delete all users
        }
}

Note: I will suggest you, if you are trying to create and delete users for particular ATP then go with SetUp and TearDown. If you are trying same for bunch of ATPs, I would recommend you to go with TestFixtureSetUp and TestFixtureTearDown.

"If your test get pass or fail, SetUp and TearDown functions will execute"

References:

3

I think you should open a transaction before you test, create the data and finish test test. But do not commit the transaction. That will ensure that the test can't affect your DB at all.

Update: The easier approach is use docker container. You can run a container from your image and remove that container after the test is done. This should definitely reduce the complexity of your test.

3
  • This does not provide an answer to the question. Once you have sufficient reputation you will be able to comment on any post; instead, provide answers that don't require clarification from the asker. - From Review
    – manish
    Nov 18, 2017 at 16:06
  • Still don't know why the question get down vote without any comment explain why. And my solution for the question is wrap everything in the test inside a transaction, which can prevent the test save data to DB. Still don't know why it doesn't provide an answer.
    – Sy Le
    Nov 18, 2017 at 16:28
  • You case that OP describes you can't rollback transaction in clean fashion. In my experience connections from connection pool can still hang for some time after application process stopped. Will transactions still be running and hold locks in this case? Nov 21, 2017 at 17:58
2

Visual studio uses NUNIT therefore, you can use TearDownAttribute. It should run after the test, even if the test is canceled. You can write a function to clean your data.

Please read the reference documentation here: http://nunit.org/docs/2.2/teardown.html

2
  • Please don't abbreviate "you" with "U". StackOverflow aims to be as professional and international as possible. While it may be easily understood for you, a non-native speaker from Indonesia might have problems.
    – MechMK1
    Nov 20, 2017 at 12:03
  • Thank you MechMK1.
    – Shuvra
    Nov 20, 2017 at 13:46
2

Just to clear more about the NUNIT standrad. Please follow the steps in the Test class:

[TestFixture]
public class _TestClass
{
        [TestFixtureSetUp]
        public void Setup()
        {
           //Clearup can be here before start of the tests. But not Recommended
        }

        [Test]
        public void _Test1()
        {
        }
        [Test]
        public void _Test2()
        {
        }

        [TestFixtureTearDown]
        public void CleanUp()
        {
            //I will recommend to clean up after all the tests complete
        }
}

Reference: http://nunit.org/docs/2.5/fixtureTeardown.html

2

A better solution to the problem to use what is called "database mocking". In this case you would have your tests run with a different database (or a fake, virtual one).

This article explains how to implement it in C#
https://msdn.microsoft.com/en-us/library/ff650441.aspx

0

You should begin a transaction and not commit your records to the DB. Thus, all your changes will be automatically rollbacked when the session is over.

1
  • As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.
    – Community Bot
    Nov 1, 2022 at 12:03

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.