The idea is to use the information created by the JUnit tests.
Every time you run an " ant tests", a reports directory is created containing a test report for each test.
each JUnit test produces an XML format report containing response times, errors and failures.
I wrote some code that gathers this information and stores it in an SQL database.
It takes two parameters :
the name of the directory (i.e. "/home/metskem/workspace/JSPWiki/build/tests/reports")
a tag that identifies this test run (i.e. "3.0.0-svn-17")
If we gather this information for each JSPWiki level that has significant changes, we could analyze if, and where performance differences occur, and also keep track of a lot of other things.
DROP TABLE IF EXISTS `testcaseresult`;
SET @saved_cs_client = @@character_set_client;
SET character_set_client = utf8;
CREATE TABLE `testcaseresult` (
`tag` varchar(32) NOT NULL,
`hostname` varchar(32) NOT NULL,
`timestamp` timestamp NOT NULL,
`classname` varchar(128) NOT NULL,
`name` varchar(128) NOT NULL,
`time` double(8,3) NOT NULL default '0.000',
PRIMARY KEY (`hostname`,`classname`,`name`,`timestamp`)
) TYPE=InnoDB DEFAULT CHARSET=utf8;
SET character_set_client = @saved_cs_client;
Each run will insert about 100 records in the testsuiteresult table, and about a 1000 rows in the testcaseresult table.
The following is an example output of a run:
21:08:14,974 WARN MainProcessor:122 - processing 106 files from directory /home/metskem/workspace/JSPWiki/build/tests/reports, tag : 3.0.0-svn-72
21:08:16,241 ERROR MainProcessor:135 - parsing completed in 1 second(s)
21:08:16,348 INFO DBUtil:100 - class com.mysql.jdbc.Driver succesfully loaded
21:08:17,993 INFO DBUtil:125 - transaction support present ? : true
21:08:27,327 WARN DBUtil:169 - committing changes....
21:08:27,330 WARN DBUtil:171 - commit completed
21:08:27,334 ERROR MainProcessor:97 - inserted 1161 rows in 11 seconds
JSPWiki JUnit test performance management#
Table of Contents
The idea is to use the information created by the JUnit tests.
containing response times, errors and failures.
Every time you run an " ant tests", a reports directory is created containing a test report for each test. each JUnit test produces an XML format report
I wrote some code that gathers this information and stores it in an SQL database.
It takes two parameters :
If we gather this information for each JSPWiki level that has significant changes, we could analyze if, and where performance differences occur, and also keep track of a lot of other things.
DDL testcaseresult table#
Sample content testcaseresult#
DDL testsuiteresult table#
Sample content testsuiteresult#
Running the collector tool#
Each run will insert about 100 records in the testsuiteresult table, and about a 1000 rows in the testcaseresult table.
The following is an example output of a run:
Some useful queries#
Here are a couple of examples of queries we could run.
Overall statistics#
Number of testsuites having errors or failures, grouped by tag#
There is also a daily refreshed version
Errors / failures by class / tag#
Number of testsuites/testcases per tag (buildlevel)#
All testsuites per tag having errors or failures#
10 most time taking testsuites#
Classes with most testcases#
15 most time taking testcase methods#
Testsuites that have increased their test time the most between releases#