128

I was wondering if anyone that has experience in both this stuff can shed some light on the significant difference between the two if any?

Any specific strength of each that makes it suitable for any specific case?

3
  • I'd go for PHPUnit it is much better Jun 28, 2013 at 9:12
  • 10
    ... and why? Try to explain your answer, provide resources/links/examples.o
    – Manuel
    Jun 28, 2013 at 9:33
  • <snark>Why not both?</snark> Jun 6, 2016 at 16:15

12 Answers 12

90

This question is quite dated but as it is still getting traffic and answers I though I state my point here again even so I already did it on some other (newer) questions.

I'm really really baffled that SimpleTest still is considered an alternative to phpunit. Maybe i'm just misinformed but as far as I've seen:

  • PHPUnit is the standard; most frameworks use it (like Zend Framework (1&2), Cake, Agavi, even Symfony is dropping their own Framework in Symfony 2 for phpunit).
  • PHPUnit is integrated in every PHP IDE (Eclipse, Netbeans, Zend Stuide, PHPStorm) and works nicely.
  • Simpletest has an eclipse extension for PHP 5.1 (a.k.a. old) and nothing else.
  • PHPUnit works fine with every continuous integration server since it outputs all standard log files for code coverage and test reports.
  • Simpletest does not. While this is not a big problem to start with it will bite you big time once you stop "just testing" and start developing software (Yes that statement is provocative :) Don't take it too seriously).
  • PHPUnit is actively maintained, stable and works great for every codebase, every scenario and every way you want to write your tests.
  • (Subjective) PHPUnit provides much nicer code coverage reports than Simpletest
  • With PHPUnit you also get these reports inside your IDE (Netbeans, Eclipse, ...)
  • Also there are a couple of suggestings for a web interface to phpunit tests.

I've yet to see any argument in favor of SimpleTest. It's not even simpler to install since PHPUnit is available via pear:

pear channel-discover pear.phpunit.de
pear install phpunit/PHPUnit

and the "first test" looks pretty much the same.

As of PHPUnit 3.7 it's even easier to install it by just using the PHAR Archive

wget http://pear.phpunit.de/get/phpunit.phar
chmod +x phpunit-3.7.6.phar

or for windows just downloading the phar and running:

php phpunit-.phar

or when using the supported composer install ways like

"require-dev": {
    "phpunit/phpunit": "3.7.*"
}

to your composer.json.


For everything you want to test PHPUnit will have a solution and you will be able to find help pretty much anywhere (SO, #phpunit irc channel on freenode, pretty much every php developer ;) )

Please correct me if I've stated something wrong or forgot something :)

Overview of PHP Testing tools

Video: http://conference.phpnw.org.uk/phpnw11/schedule/sebastian-bergmann/

Slides: http://www.slideshare.net/sebastian_bergmann/the-php-testers-toolbox-osi-days-2011

It mentions stuff like Atoum which calls its self: "A simple, modern and intuitive unit testing framework for PHP!"


Full disclosure

I've originally written this answer Jan. 2011 where I had no affiliation with any PHP Testing project. Since then I became a contributor to PHPUnit.

8
  • 5
    I have discovered that simpletest's web-testing was much easier to set up and get running than phpunit's web-testing. Since that's pretty much a necessity with php, this is a win for small to medium projects, where you just want to toss in a testing framework in an instant and the important thing is simple levels of tests. For example, excluding web-tests, I tend to only use the assertTrue() method in simpletest. Web-tests, of course, get more complicated, but they don't have to be excessively so.
    – Kzqai
    May 10, 2012 at 17:09
  • 1
    "> Since that's pretty much a necessity with php" - I strongly disagree on this. I've yet to see any php project (or framework) that runs its tests through a browser. It creates a whole lot of problems for no actually benefit. The times where php code was developed on a shared hosting system using ftp sync are dead and gone (fingers crossed) and no, even small, project requires a running web server (at most the one php ships out of the box). UNIT Testing through a browser and not the cli or an IDE is just such a rare use case that I don't think it matters at all.
    – edorian
    May 10, 2012 at 17:50
  • 13
    A strong reason to consider SimpleTest over PHPUnit is that the latter is a command line tool that must be installed on the server running the tests which at times is a lot more difficult than just includeing the SimpleTest .PHP files (as other's have explained.) For professional programmers building out their own test environment PHPUnit sounds hard to beat but if integrating a testing system into a PHP-based open-source app where "developers" for the app may be at best (what I call) "occupational programmers" then the simplicity of SimpleTest has a real appeal, to me anyway. FWIW. Jul 4, 2012 at 3:00
  • 2
    @MikeSchinkel I couldn't agree more. We used SimpleTest as an integrated browser based testing solution for our bespoke application framework. It was always available, and required no installation configuration beyond an svn pull as it was included in the source of the framework. We moved to PHPUnit for CIS integration. Because we are a Windows shop, PHPUnit can be a nightmare to install and configure, and many of our junior developers simply break the environment or lose the will to test.
    – Gavin
    Mar 4, 2013 at 22:01
  • 1
    The link to the slides is broken. Nov 9, 2013 at 13:25
35

I prefer PHPUnit now, but when I started out I used SimpleTest as I didn't always have access to the command line. SimpleTest is nice, but the only thing it really has over PHPUnit, in my opinion, is the web runner.

The reasons I like PHPUnit are that it integrates with other PHP developer tools such as phing (as does SimpleTest), phpUnderControl, and Xinc. As of version 3.0 it has mocking support, is being actively developed, and the documentation is excellent.

Really the only way to answer this question for yourself is to try both out for a time, and see which fits your style better.

EDIT: Phing now integrates with SimpleTest as well.

1
  • 4
    Just a note, phing's website says it supports SimpleTest
    – philfreo
    Dec 9, 2010 at 19:38
18
  • I could NOT understand how to download and install PHPUnit.
  • I could, however, easily understand how to install SimpleTest.

    (As far as i can remember the instructions for PHPUnit said something along the lines of "install it via PEAR and we won't give any instructions on how to do it any other way") see:

  • http://www.phpunit.de/manual/current/en/installation.html

For SimpleTest, just download it and point to it from your code.

So Simpletest won for me.

9
  • 4
    I suppose the documentation has changed since you posted your answer then. Today, you just type 3 commands and it will be installed, you dont even have to download anything yourself. Sep 18, 2010 at 16:17
  • 5
    You can install PHPUnit via PEAR, manually download it or pull the latest code from github. There is a clear link to the install instructions on the homepage. SimpleTest only gives me one option, pointing to sourceforge and it's myriad of commercials...
    – Potherca
    Oct 1, 2010 at 7:26
  • 1
    Under Windows, PHPUnit is a pain to install. I had to install it on 2 workstation. The first one took 3 hours, with the help of the PEAR IRC chanel. For the second one (same OS) I never succeeded. That said, on Linux it's a charm.
    – FMaz008
    Sep 20, 2011 at 17:26
  • 1
    SimpleTest +1 Easy to install Easy to use. PHPUnit is powerful than, however,simpletest to satisfy my needs.
    – Cheung
    Jan 4, 2012 at 1:51
  • Things may have changed since '09, but PHPUnit installed in seconds with just a few commands on Windows.
    – Cypher
    Apr 25, 2013 at 22:25
15

Half of the mentioned points in the accepted answer are simply not true:

SimpleTest has

  • the easier setup (extract to folder, include and run)
  • simply check the folder into version control (try to do that with phpunit nowadays :))
  • less dependencies and lots of extensions (webtester, formtester, auth)
  • a good code coverage reporter, which is easy to extend (dots, function names, colors)
  • a code coverage summary (finally landed in PHPUnit 4.x)
  • a decent web runner and an ajax web runner, with groups and single file executions
  • still better diff tool (with no whitespace or newline problems)
  • an adapter/wrapper to run SimpleTests by phpUnit and vice versa
  • compatibility PHP5.4+

The downside:

  • not industry standard (PHPUnit)
  • not actively maintained
5
  • 1
    Still useful to me and my boss php4 server ^^" Jan 22, 2016 at 15:40
  • Thanks. I've upgraded the code base of SimpleTest to work with newer PHP versions and some more tweaks are incoming during the year. -- PHP4 is EOL since ages and doesn't get any security patches. I wouldn't use it on external services anymore. You should convince your boss to allow an update run to at least v5.5 or better v5.6: php.net/supported-versions.php#php-5.6 Jan 22, 2016 at 15:55
  • 1
    I know Jens, thanks for the advice. Very nice testing solution yours Jan 22, 2016 at 16:10
  • 1
    =) Thank you for showing me the Github repo. I had begun to worry about the "not maintained bit". Only superficially, mind you, since it still does everything I've ever wanted from a low fuss test framework with the modern code I work with.
    – jeteon
    May 12, 2016 at 3:34
  • SimpleTest has way more flexibility in how you setup your test suites as you can do it all in php code. So you have a full programming language at your disposal, not just xml config files and terminal parameters. PHPUnit may be more supported by tools, but says more about how the PHP community keeps choosing to back inferior things. Oct 14, 2016 at 16:43
13

Baphled has a nice article on SimpleTest vs PHPUnit3.

10

I found SimpleTest was even easier than PHPUnit to set up. Just extract it and you are good to go. A benefit of this is if you are working at more than one machine, since you can store the whole testing framework the same way as your source code, and thereby know that you are using the same framework code. Especially if you modify it in any way.

So, I would say that a strength of SimpleTest is that it is very light weight and portable.

SimpleTest also ships with a very simple HTML GUI, which is quite easy to extend if you want to. As far as I know, PHPUnit does not include a HTML GUI, but there are GUI:s available to download, such as Cool.

1
  • Ah, man I downvoted this totally by accident and didn't even realize it until the next day. Now my vote is locked in... Sorry. Oct 18, 2016 at 23:08
8

Well I made a phpUnit web based UI test case runner and made it available on sourceforge. Uses ajax and has quite cool interface as well if you want to give it a shot check it at sourceforge. The project name is phpunitwebui and the website is http://phpunitwebui.sourceforge.net/

4

As it has been pointed, it's mostly a preference choice, as both will run the tests you write for it and report back the results.

The Simpletest web UI is very useful, but it can also sometimes get cumbersome. In my current project, I would have had to put more work into a system to make my application (an API) work with the web interface (set up apache correctly, copy files to the public_html root, etc.) than it would have been to simply run phpunit from the eclipse workspace. Therefore I choose PHPUnit. Also, the use of PEAR was a big plus since you don't need to manually track updates. Simply run pear upgrade once in a while and PHPUnit will be kept up-to-date.

2
  • I don't get this analysis. Are you saying you test browser-type functionality via the eclipse plugin for phpunit, instead of a local apache setup?
    – Kzqai
    May 10, 2012 at 17:14
  • No, I'm saying that in my project, as there is no browser-type functionality, I am using phpunit through eclipse.
    – T0xicCode
    Jul 1, 2012 at 11:35
2

This is from the point of view of a very casual PHP developer:

It took me two days to grasp PHPUnit, mostly trying to debug under Eclipse that I finally gave up.

It took me two hours to setup Simpletest including debug under Eclipse.

Maybe I will find the shortfalls of Simpletest in the future but so far it does well what I need: TestClasses, Mock objects, test-code debugging, and web interface for a quick snapshot of the situation.

Again: This from the point of view of a very casual PHP user (not even developer :-)

1

I haven't checked Simple Test for a while, last time it had an eclipse plugin, which is a major factor for me, but it hasn't been updated for a long time. Sebastian Bergmann is still very actively working on PHPUnit, but it still lacks a good plugin for eclipse - but it is included for the new Zend Studio.

1

This question is old, but I want to add my experience: PHPUnit seems to be the standard now, but if you work with a legacy system that uses lots and lots of global variables, you may get stuck from the get go. It seems like there is no good way to do tests with global vars in PHPUnit, you seem to have to set your variables via $GLOBALS which is NO GOOD if you have tons of files setting global variables everywhere. OK some may say that the problem is in the legacy system but that doesn't mean we cannot do tests on such system. With SimpleTest such thing is simple. I suppose if PHPUnit allows us to include a file globally, not within any class/function scope then it wouldn't be too much of an issue as well.

Another promising solution is http://www.enhance-php.com, looks nice :)

3
  • I've always just used <phpunit backupGlobals="false"> in the config XML and never had any problems with globals in tests...
    – Potherca
    Jun 28, 2014 at 14:12
  • @mr1031011 your link doesn't go somewhere useful. Perhaps you meant?: github.com/Enhance-PHP/Enhance-PHP
    – user12345
    Sep 16, 2014 at 22:42
  • @Potherca tell me about it - I initially had a lot of problems with globals in a legacy code project, and I think I tried all the annotations @backupGlobals @backupStaticAttributes @runInSeparateProcess @preserveGlobalState, global $xxx and all its combinations, before trying --no-globals-backup and suddenly seeing all working as expected. :D
    – conny
    Dec 9, 2014 at 5:18
-2

when there are thousands functions to test at one go, phpunit is way to go, simple test is falling short as it web based.

I am still using simple web to for small scale test .

But both are good

1
  • SimpleTest is not purely web based. I've used it exclusively from the command line up until now.
    – jeteon
    May 12, 2016 at 3:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.