First, I find that I often need step debugging when writing unit tests, as that's when I'm really scrutinizing logic.  It turns out it's pretty easy to do.  All you need is to set the XDEBUG_CONFIG environment variable like so:

XDEBUG_CONFIG="idekey=session_name" php bin/phpunit.php tests/FooTest.php

When debugging from VIM, you'll want to have two shells open. One running the debugger, and another executing the tests. Also, note that in a shared environment, you can still pass in xdebug ini settings like so:

XDEBUG_CONFIG="idekey=session_name" \
php -d xdebug.remote_host=10.0.0.1 bin/phpunit.php tests/FooTest.php

Second, for step debugging in VIM, I've settled on this plugin (there are many versions out there).  It seems to solve problems with tab handling that I was seeing with older versions.

Cheers,

Bill

When developing an application, one of the keys to maintaining high productivity and quality is to make sure that you not only have the right tool set, but that you have easy access to it.   With web applications, the usual needs are things like running Selenium tests, running unit tests and checking code coverage, profiling, and having links to documentation.  Though I've talked previously about the cost of maintaining 100% code coverage, I'm convinced that it's worth it if you not only get proficient at the tools, but make them very easy to use.  So when I started as the first developer here at Empower Campaigns a couple of months ago, I wanted to make sure these needs were addressed early on, before we had a large development team.

Inspired by the work that had been done at my previous job by Ian Eure and Matt Erkkila, I decided to start with the following:

• Use the tried and true GNU Make for building environments and executing tools
• Have a "build" virtual host in every development environment for accessing the tools' web resources

Using make, getting our project "enterprise" (we use space shuttle themed project names) up and running is as simple as this:

# clone into a directory named with your vhost name:
git clone review:enterprise.git enterprise.bshupp.dev.em.power
cd enterprise.bshupp.dev.em.power
# install into apache
make clean install

This not only provides a working application (or "shuttle" as we call them) at enterprise.bshupp.dev.em.power, but also a "build" virtual host for access to web resources at build.enterprise.bshupp.dev.em.power. Here's a screen shot of the initial build "dashboard", which provides navigation on the left, and some environment details and instructions on the main page.

Since our web tier is in PHP, the most common tool we need to run is PHPUnit, which you run via make test.  You'll see the standard output in your terminal like so (with some nice colors indicating the status - green for success and high coverage, yellow for success and medium coverage, and red for failing tests).

In addition, to view coverage details, you go to the "code coverage" link your build dashboard.

make test also generates TestDox output to see which tests are failing.

We keep a Selenium Remote Control (RC) server running on our development system for easy access by developers.  Want to see if you've broken the selenium tests?  Just run make selenium-smoke or make selenium-full, and Selenium RC will talk to your development environment.

And since developers need to run make test selenium-smoke phpcs to validate before any push to avoid breaking a build, I've wrapped them up into make validate for convenience.  We actually use these shortcuts with Hudson, which allows us to direct Selenium RC to different staging environments, or modify build steps without re-configuring Hudson.

Want to read the inline documentation in a web browser instead of your text editor?  Just run make phpdoc, then click on the "PHPDocs" link in the build dashboard.

One thing that's particularly useful is easy profiling with Facebook's XHProf.  Just enable profiling in your environment via make enable-profiling, perform the action in the application you want to profile, then open up the "XHProf Profiler" link in your build dashboard.  There you'll see a list of "runs" which you can drill down into and see exactly how each is performing.  When you're done, just run make disable-profiling.

To manage XHProf profile runs, I save them by environment, and wrote a helper that allows you to browse runs by date, and then "clean" them out when you're done.  This works easily in a shared xhprof.output_dir by namespacing the stored runs according to environment name.  Here is the example snippet for starting profiling and saving as runs:

And you can checkout the full code over on github.

If you want to get statistics on or content from APC, that's available too through the inspector that ships with the extension:

Lastly, I've also included links in the dashboard to all the tools we use: our Confluence wiki, Gerrit, gitweb, and Hudson.

This system makes it really convenient to deploy your development environment, access documentation, profile your code, run test tools and analyze results.  It's my hope that as our development team grows, it'll be relatively painless to maintain high quality, well tested code.

Cheers,

Bill Shupp, Lead Developer (twitter: http://twitter.com/shupp)

Below are some things I keep in mind when coding and testing. I thought I'd share them here in case anyone else finds them useful.

Think about testability when you write code, not when you write the test

This may not be new to you, especially if you're familiar with dependency injection or test driven development.  Your unit tests should be able to run in a vacuum:  No database access, no memcache access, no headers can be sent, etc.  (Sure those things can be tested separately, but should be a small subset of your code, not the overall application logic.)  To do this, make sure you abstract those dependencies.  For example, instead of instantiating a cache driver in the middle of your method, make a getter for it which you can mock:

If you do this abstraction when you code, there's less refactoring to be done when writing your tests.  Also, if your dependency doesn't make a network call upon instantiation, you can still test the getter using assertType().  Memcached's addServer() is an example of such a dependency - no connection is made until you do an operation.  If this is not the case, and you can't properly test the getter, you probably want wrap the getter in @codeCoverageIgnore annotations like so:

It doesn't make sense for code you never intend to test to count against your coverage percentage.

Keep documentation handy

PHPUnit is a big framework.  There are a lot of things to remember, including all the assertion methods available, data providers, and mocking objects, among other things.  Rather than always copying some other example, or bugging your co-worker, spend an hour or two, and RTFM.  You'll be better off for it.  A co-worker of mine kept a printout of the PHPUnit_Framework_TestCase::getMock() documentation (probably the hardest to remember) taped to his monitor.  I prefer delicious bookmarking, but to each his own. Though I've been using PHPUnit for a couple of years now, I still refer to the documentation regularly.

Don't worry about test code re-usability

As good OOP programmers, we are often pre-occupied with writing elegant, re-usable code, even in tests. I have recently given up on this in my unit tests, mostly due to scope issues in PHPUnit which I'll describe later. Rather than try consolidate re-usable code, I'm more likely to copy and paste chunks throughout my test and know that they will work. I think the goal of tests is to write useful assertions, not necessarily elegant, re-usable components. Why do I care how many lines are in my test code? You might be thinking "Doesn't that make your tests harder to maintain?" Maybe so, but in the big picture, I don't think it adds much time, and once you've fixed something in one test, then you can quickly find it elsewhere. It might not be pretty, but it is faster, and more pragmatic, at least until the tools are better.

Mocking scope issues

As I mentioned above, I've found (and reported) some strange behavior I've seen with scope and mocked objects.  One pattern I see people use is a getMocks() helper in a test, which returns an array of commonly used objects, pre-mocked with the most used defaults already set.  Sounds like a reasonable approach to testing a large class with lots of dependency needs, right?  Wrong.  What I've found is that many times (and my test results have been inconsistent, unfortunately), you cannot modify the return value of a mocked method after its expectation has already been set.  No warning or error is thrown when you do this, but the original mocked value is returned instead of the second value you assigned.  Re-mocking return values does seem to work when it's used for throwing exceptions, though.  It appears to be a scope issue, but again, my tests have not always been consistent.  The usual result is that you spent an hour pulling your hear out trying to figure out why your test is failing.  I eventually said screw it, and just never set default behaviour for my mocked methods.

I now use a pattern where I have getters in my test for each mocked dependency, which mock each method I may need, but don't set any initial expectations yet.  I'll then pass them all into the getter for the main class I'm trying to test, which attaches them for me.  This way, on a per test basis, I set the expectations of each dependency as needed. This seems to work much more reliably:

Meaningful assertions

Another complaint I hear is that, especially when enforcing code coverage rules in your development shop, developers focus more on the required coverage than on the quality of the tests.  It's important to think about what it is you're testing.  If you just call a method to make sure you get your code coverage, you're doing it wrong.  There are lots of assertions you can do:  assert the type of the response, assert the content of the response, assert the message of the exception  thrown, the exception code, the type of exception, how many times a mocked method was called, etc.  But you want to be smart about it as well.  We had one case where the original test for a Form framework was testing the exact template output.  Since there was markup and JS in the ouput, every time a designer or JS developer modified it, the test would break.  After a few breaks, I modified the test to use DomDocument parsing, and make sure the relevant form ids and attributes were present, which is really what we wanted to assert.  If you don't write meaningful tests, don't bother writing them at all.

Useful tools

Make sure it's easy to write and run your tests.  PHPUnit has options for generating skeletons, for example, to get you started quickly.  You can use --skeleton-class to generate a class from your test (if you're practicing TDD), or the reverse, --skeleton-test, to generate a test from your class.

Running tests should be easy too.  One thing to keep in mind is that your include path must be correct.  For example, when I'm developing a PEAR package, I may have a previous version already installed, but am developing in my repository checkout.  I need to make sure my local files are not only in my include path, but that they are at the beginning.  A handy wrapper like this should allow you to always run your development code easily:

If I put this in the root of my repository, I can then call it like so:

php runTests.php --coverage-html coverage OpenID_AllTests

(If this is a PEAR package, you'll want to ignore runTests.php when generating your package.xml file.)

At Digg, our development environment tools are based on make, and it's super easy to run not only your PHPUnit tests and coverage, but also chain together running PHP_CodeSniffer for coding standards compliance:

make coverage phpcs

Facebook has a good post about hiring in general, which includes a section on writing good tools for your team.  This idea certainly applies to your testing tools. The easier you make it for your team to write good unit tests, the better.

Conclusion

Unit testing in PHP is not that easy, in my opinion (compared to python, anyway).  Until you get good at it, that is, and learn to work around quirks in the tools. Then it's really not that hard. But everyone  values unit testing differently.  If you choose to invest in testing your code, perhaps you'll find some of the above useful. I used to not think much about unit testing, until I had a few experiences where unit tests caught problems in my code before I had run into them myself. At this point, I think unit testing my PHP code 100% is worth it, especially now that I can do it quickly and efficiently.