30

How widespread, supported, developed is testing in the PHP world? On par with Java? Up there with Ruby/Rails? I Googled and found that testing frameworks exist but I'm wondering if they're widely used.

Do the major PHP IDE's have built-in test runners the way Eclipse's Java tools do or NetBeans's Ruby/Rails tools do? Is testing built into PHP's MVC frameworks as with Rails?

I ask because a group where I work wants to hire someone to develop a PHP app for them. I'm concerned about quality and maintenance as I might be called on to support this thing.

9 Answers 9

44

There are at least two mature, stand-alone, JUnit style test suites available, named PHPUnit and SimpleTest, respectively.

As far the MVC Frameworks go, Symfony has its own testing framework named lime, Code Igniter has a unit_test library and CakePHP relies on the aforementioned SimpleTest.

I know that Zend Studio has built in support for PHPUnit tests, and both PHPUnit and SimpleTest have command-line runners so integration into any workflow is possible.

The tools are there in the PHP world if a developer wants to take advantage of them, and smart shops do take advantage of them.

The caveats are your par for the course PHP complaints. There are two PHP communities; PHP as a platform for building software, and PHP as a way to interact with a web server, web browser, and database to produce application-like things on the web. It's less a black and white thing and more a continuum; Among those more on the software developer side unit testing and TDD is supported and used as much as it is on any other platform. Among the "cobble together a bunch of stuff I don't understand but still get results people", it's unheard of.

There's a lot of non-framework/custom-framework legacy PHP code around that's difficult to get a useful test harness around. PHP also lends itself easily to patterns that rely on the existence of a browser environment to run. I don't have any evidence to back this up other than my own observations, but a lot of PHP shops that care about testing end up relying on acceptance testing (i.e. Selenium) as a substitute for actual Unit Testing, test-first, etc. development.

In your specific situation, interview the hell out of the developer your group is going to hire.

  1. Ask them what unit testing framework they use

  2. Ask them to describe, in general terms, a real world example of a time they developed a new feature and its supporting tests

  3. Ask them to describe, in general terms, a real world example of a time their tests failed and what they did to resolve the situation

You're less interested in the specific situation they're going to describe and more interested in how comfortable they are discussing their knowledge of code testing in general.

0
11

Whenever I TDD a project with XUnit style tools, I have difficulty getting my head in the right spot. I find that using tools designed for Behaviour Driven Development or "Specification by example" makes it easier for me to do TDD right -- i.e. focus on design, exposing intent and describing behaviour in specific contexts. Not testing.

That said, I would like to introduce pecs into the conversation. From the readme on the project site.

pecs is a tiny behavior-driven development library for PHP 5.3, a la RSpec or JSpec.

If you've used JSpec or better yet, Jasmine-BDD (for JavaScript) the pecs style of describing behaviour should be really familiar. I find this style great for component level specs. If you are looking for a PHP tool for feature level specifications (stories or user acceptance tests) consider Behat.

Going back to pecs, here's an example culled from the pecs project site:

describe("Bowling", function() {
  it("should score 0 for a gutter game", function() {
    $bowling = new Bowling();
    for ($i=0; $i < 20; $i++) {
      $bowling->hit(0);
    }
    expect($bowling->score)->to_equal(0);
  });
});

Yes that is a PHP spec. Looking through the pecs source, it looks like the author is able to pull this off by leveraging the new hotness in PHP 5.3+, Lambdas and closures. So I guess this means that you cannot use pecs in any project based on PHP < 5.3 (just FYI).

Also, pecs is not as mature as PHPUnit or SimpleTest. However, I think the proponents of BDD in the PHP community should support the growth of tools like pecs that encourage "Specification by example" or BDD without the confusion brought on by having to use legacy XUnit testing tools.

These days I work more in Python than PHP. However, the next time I pick up a PHP project, I'll be extremely happy if I have a mature, community supported tool like pecs to craft the specifications for the software.

5

I have had an amazing experience with Behat / Mink http://behat.org

I agree with others php as a unit testing platform is not a fun or experience BDD is the best way to go if you are using any php framework

Wrapping my head around composer as a repo build tool was the biggest stumbling block but we were able to use Behat Mink Selenium Webdriver standalone server jar as an amazing design and regression testing tool. We used to run our regression suite against our CakePHP application on a Jenkins server but it proved to be not so very "fail fast" enough

Now our workflow goes like this: Create story in gherkin refine story write feature and stub out any new step defs begin coding php solution to test Then at the end we have a working feature or bug fix with a bdd test covering it

We setup an Ubuntu VM with a working Behat setup and copied it to every workstation. We baked it into our process. We just pull down changes run tests then begin coding new stuff.

We wrote a shell script to automatically run mysql dumps and load them before each feature which has made refactoring code a breeze.

The Mink WebAssert class gives you all the assertions you need to validate behavior The regular session / CommonContext classes are great for using css or xpath.

I have used Capybara / WebDriver with Java and Rails projects before and found the setup overhead / learning curve is too high compared to Behat.

4

In addition to the libraries/frameworks that Alan has already mentioned, you can make use of mod_perl's Apache::Test, which I what I use as a harness. It allows me to very simply integrate tests into my release process. The harness uses TAP output (Test Anything Protocol) to determine whether or not the tests pass or fail, using libraries like Test::Simple or Test::More (Perl and PHP).

Out of the box Apache::Test supports writing tests in both Perl and PHP. In my own projects, it took a little bit of trickery and a lot of reading to really get it working, but an implementation of Test::More in PHP is built-in to the harness. Running all tests written in both PHP and Perl is done through a single command and any failure along the way is captured Apache::Test, noting as best it can what went wrong.

The awesome part about all this is that you can even utilize PHPUnit, or Simple-Test alongside the previous two testing frameworks. By running tests in each respective library, you can use the PHP implementation of Test::More (or even Perl by testing stdout) and spit back out TAP for your harness to interpret.

Be sure to read the Apache::Test documentation and the mod_perl guide to running Apache::Test. Additionally, I found the article here a great help.

As a quick example, you could setup a test in Perl in very few lines of code that will run through all the pages on your site (that have links) and verify all result in '200 OK' responses and don't have any parsing errors:

#!perl

use strict;
use warnings;

use Apache::Test qw(:withtestmore);
use Apache::TestRequest;
use Test::More;
use Test::WWW::Mechanize;
use WWW::CheckSite::Validator;
use WWW::CheckSite::Spider;

plan 'no_plan';

my $config = Apache::Test::config();
my $host = "http://". Apache::TestRequest::hostport($config) || '';

my $s = WWW::CheckSite::Spider->new(
    uri => $host,
    ua_class => 'Test::WWW::Mechanize',
);
my $m = $s->current_agent;

while (my $page = $s->get_page) {
    is($m->status(), "200", $m->uri() ." retrieved successfully.");
    $m->content_lacks("Parse Error", $m->uri() ." does not contain syntax errors.");
}
3

On a past project, I've used the PHPUnit, and it has left me wanting. PHPUnit + Command line running of the tests, made it so that too much time was spent coding the tests, wasn't fast enough, and really seemed to constrain the style of the code in a way that I didn't like (Objects were easier to test, so it seemed to kinda favor objects).

Selenium was a solution that we talked about but never got around to getting into play, and I think we would really have benefited from that kind of output-level testing.

On this latest project, the lead programmer has taken a more functional programming approach as we've been revising software. When I mentioned that I'd like to code via TDD, he whipped up a custom solution in a day or less that I consider to have been as effective for me to use as PHPUnit. In addition, he really opened my eyes about the question of Object Oriented vs. Functional Programming.

First project, started from scratch, in at the ground floor, Object Oriented coding, large Unit Testing Framework, it became monolithic and bogged down quickly. Second project, well established CMS software with a 5 year history and old code, yet a functional programming paradigm and a simple testing framework (we actually often made use of php assert) made it get simpler instead of grow in complexity.

The second project, too, never got to the point of implementing Selenium (and I still think it would be beneficial), but the functional programming approach made it easier to deal with in-code testing.

1
  • I've used selenium and agree that it is useful in a number of cases, however it also gets bogged down very quickly, especially on a highly 'liquid' browser type project. For example changing the ID of a box, moving URL or even DOM structure around can break a huge number of tests. What's nice about selenium is that in many cases you can actually record the test manually via the Selenium IDE - export it as PHPUnit - then tweak according to your needs to add looping and variability. It's quite robust in that respect. (In fact you can export it into a number of different languages)
    – Alex C
    Nov 29, 2010 at 19:02
3

I just found this question and, while I'm still at the "researching stage" in figuring out what's going on. I just discovered something for Ruby on Rails called "Cucumber" http://cukes.info/

It is essentially 'story driven development' for Ruby and quite possibly a gold standard in the realm of functional testing, at least as far as I've seen in my travels. (I put this up there publicly, so experts can correct me if I'm wrong)

As an example of the language in Cucumber, you've got something that very closely resembles SQL. BUT seems to be even more human readable. From the cukes front page their language looks like this:

 Scenario: Add two numbers
      Given I have entered 50 in the calculator
      And I have entered 70 in the calculator
      When I press add
      Then the result should be 120 on the screen

The above will compile and run as a test.

Now that's all preamble to the point of answering your question about PHP - BDD & TDD.

In echoing the comments above, PHPUnit will allow unit testing and according to this blog post: http://sebastian-bergmann.de/archives/738-Support-for-BDD-and-Stories-in-PHPUnit-3.3.html also supports "story style" BDD testing.

To expand on the above answer with respect to "SIMPLETEST" mentioned above, the ST system has a built in browser object class for browser automation, while PHPUnit has an extension for the SELENIUM browser automation http://seleniumhq.com (the advantage of Selenium vs. SimpleTest is that Selinium will run any on-page javascript while SimpleTest will not).

I hope you find this information helpful as it's the result of a number of months personal research and hands-on trial and error with the above technologies. If there are experts out there who can clarify and improve my understanding of the above, I welcome the feedback.

  • Alex.
1
  • 4
    I would recommend you to look at [Behat][1]. It is a Cucumber implementation in PHP (it's not a port). It works really well and now also supports integration with tools like selenium. [1]: behat.org Apr 22, 2011 at 6:18
2

Michael Booth's comparison of BDD testing features in both languages:

http://mechanicalrobotfish.com/posts/117-ruby-vs-php-bdd-beauty-contest-no-contest

concludes that PHP BDD tools and culture is underdeveloped at this point.

Certainly there is nothing comparable with what's available to a Ruby programmer, either in terms of knowledge (books, videos, articles, blog posts) or tools (Rspec, Shoulda, Factory Girl, Mocha, Cucumber).

0

You might want to check out PHPStorm. I like the test runners that use PHPUnit from within the IDE.

0

Now I'm developing "Spectrum" framework for BDD test: https://github.com/m-haritonov/spectrum

Not the answer you're looking for? Browse other questions tagged or ask your own question.