VulcanJ, a Firebird regression test suite

We have developed a test suite which we run nightly to verify Vulcan functionality. This test suite, called VulcanJ, has over 800 tests that are a combination of ISQL tests and Java (JDBC) tests. There have been several cases where nightly runs of the test ware uncovered problems that would have been missed otherwise. We also run the tests before committing changes to our internal source code management system. I have now run the test ware against FB2.0, and would like to make it available to the community at large.

The test ware is based on Junit and Ant. VulcanJ will generate a HTML report which immediately shows where a problem is occurring. In the case of the ISQL tests, the HTML report also shows the comparison between the faulty run and the original benchmark run. Running the tests against FB2.0 generated 2 problems, one of which Pavel has entered a bug track on, and the other, which I posted here this week as a Vulcan problem, also shown up in FB2.0. VulcanJ also includes Java tests that are based on the National Institute of Standards and Technology (NIST), and customized for firebird-specific queries.

I have sent advance copies of the test ware to Roman and Pavel, and they have encouraged me to push my code to a new CVS module. I hope to push this test ware to a sourceforge next week (at firebird/VulcanJ), but if you want advance review I can send you a zip immediately. Detailed doc and usage instructions are included in the kit. There is a little bit of work left to do, to make sure I remove company-specific references in the source, but then I should be ready to push.

-b

Bill Oliver
Product Specialist, Base SAS