JSPWiki JUnit test performance management#
Table of Contents
- JSPWiki JUnit test performance management
- DDL testcaseresult table
- Sample content testcaseresult
- DDL testsuiteresult table
- Sample content testsuiteresult
- Running the collector tool
- Some useful queries
- Overall statistics
- Number of testsuites having errors or failures, grouped by tag
- Errors / failures by class / tag
- Number of testsuites/testcases per tag (buildlevel)
- All testsuites per tag having errors or failures
- 10 most time taking testsuites
- Classes with most testcases
- 15 most time taking testcase methods
- Testsuites that have increased their test time the most between releases
The idea is to use the information created by the JUnit tests.
Every time you run an " ant tests", a reports directory is created containing a test report for each test.
each JUnit test produces an XML format report
containing response times, errors and failures.
I wrote some code that gathers this information and stores it in an SQL database.
It takes two parameters :
- the name of the directory (i.e. "/home/metskem/workspace/JSPWiki/build/tests/reports")
- a tag that identifies this test run (i.e. "3.0.0-svn-17")
If we gather this information for each JSPWiki level that has significant changes, we could analyze if, and where performance differences occur, and also keep track of a lot of other things.
DDL testcaseresult table#
DROP TABLE IF EXISTS `testcaseresult`; SET @saved_cs_client = @@character_set_client; SET character_set_client = utf8; CREATE TABLE `testcaseresult` ( `tag` varchar(32) NOT NULL, `hostname` varchar(32) NOT NULL, `timestamp` timestamp NOT NULL, `classname` varchar(128) NOT NULL, `name` varchar(128) NOT NULL, `time` double(8,3) NOT NULL default '0.000', PRIMARY KEY (`hostname`,`classname`,`name`,`timestamp`) ) TYPE=InnoDB DEFAULT CHARSET=utf8; SET character_set_client = @saved_cs_client;
Sample content testcaseresult#
mysql> select * from testcaseresult limit 10; +--------------+----------+---------------------+---------------------------------------------+-------------------------------------+-------+ | tag | hostname | timestamp | classname | name | time | +--------------+----------+---------------------+---------------------------------------------+-------------------------------------+-------+ | 3.0.0-svn-71 | bismarck | 2009-02-19 19:15:55 | org.apache.wiki.action.GroupActionBeanTest | testDeleteGroup | 1.990 | | 3.0.0-svn-71 | bismarck | 2009-02-19 19:15:55 | org.apache.wiki.action.GroupActionBeanTest | testSaveExistingGroup | 1.968 | | 3.0.0-svn-71 | bismarck | 2009-02-19 19:15:55 | org.apache.wiki.action.GroupActionBeanTest | testSaveNewGroup | 2.011 | | 3.0.0-svn-71 | bismarck | 2009-02-19 19:15:55 | org.apache.wiki.action.GroupActionBeanTest | testView | 2.398 | | 3.0.0-svn-71 | bismarck | 2009-02-19 19:15:55 | org.apache.wiki.action.GroupActionBeanTest | testViewNonExistent | 1.972 | | 3.0.0-svn-71 | bismarck | 2009-02-19 19:15:55 | org.apache.wiki.action.GroupActionBeanTest | testViewNullGroup | 1.967 | | 3.0.0-svn-71 | bismarck | 2009-02-19 19:16:18 | org.apache.wiki.action.RenameActionBeanTest | testRename | 1.975 | | 3.0.0-svn-71 | bismarck | 2009-02-19 19:16:18 | org.apache.wiki.action.RenameActionBeanTest | testRenameReferences | 1.972 | | 3.0.0-svn-71 | bismarck | 2009-02-19 19:16:18 | org.apache.wiki.action.RenameActionBeanTest | testRenameReferencesChangeRefsFalse | 1.969 | | 3.0.0-svn-71 | bismarck | 2009-02-19 19:16:18 | org.apache.wiki.action.RenameActionBeanTest | testRenameReferencesChangeRefsTrue | 1.975 | +--------------+----------+---------------------+---------------------------------------------+-------------------------------------+-------+ 10 rows in set (0.01 sec)
DDL testsuiteresult table#
DROP TABLE IF EXISTS `testsuiteresult`; SET @saved_cs_client = @@character_set_client; SET character_set_client = utf8; CREATE TABLE `testsuiteresult` ( `tag` varchar(32) NOT NULL, `hostname` varchar(32) NOT NULL, `timestamp` timestamp NOT NULL, `name` varchar(128) NOT NULL, `errors` int(11) NOT NULL default '0', `tests` int(11) NOT NULL default '0', `failures` int(11) NOT NULL default '0', `time` double(8,3) NOT NULL default '0.000', PRIMARY KEY (`hostname`,`name`,`timestamp`) ) TYPE=InnoDB DEFAULT CHARSET=utf8; SET character_set_client = @saved_cs_client;
Sample content testsuiteresult#
mysql> select * from testsuiteresult; +--------------+----------+---------------------+--------------------------------------------------------------+--------+-------+----------+--------+ | tag | hostname | timestamp | name | errors | tests | failures | time | +--------------+----------+---------------------+--------------------------------------------------------------+--------+-------+----------+--------+ | 3.0.0-svn-71 | bismarck | 2009-02-19 19:15:55 | org.apache.wiki.action.GroupActionBeanTest | 0 | 6 | 0 | 12.310 | | 3.0.0-svn-71 | bismarck | 2009-02-19 19:16:18 | org.apache.wiki.action.RenameActionBeanTest | 0 | 6 | 0 | 11.867 | | 3.0.0-svn-71 | bismarck | 2009-02-19 19:16:54 | org.apache.wiki.action.WikiContextFactoryTest | 0 | 5 | 0 | 6.423 | | 3.0.0-svn-71 | bismarck | 2009-02-19 19:17:21 | org.apache.wiki.auth.acl.DefaultAclManagerTest | 0 | 3 | 0 | 1.245 |
Running the collector tool#
Each run will insert about 100 records in the testsuiteresult table, and about a 1000 rows in the testcaseresult table.
The following is an example output of a run:
21:08:14,974 WARN MainProcessor:122 - processing 106 files from directory /home/metskem/workspace/JSPWiki/build/tests/reports, tag : 3.0.0-svn-72 21:08:16,241 ERROR MainProcessor:135 - parsing completed in 1 second(s) 21:08:16,348 INFO DBUtil:100 - class com.mysql.jdbc.Driver succesfully loaded 21:08:17,993 INFO DBUtil:125 - transaction support present ? : true 21:08:27,327 WARN DBUtil:169 - committing changes.... 21:08:27,330 WARN DBUtil:171 - commit completed 21:08:27,334 ERROR MainProcessor:97 - inserted 1161 rows in 11 seconds
Some useful queries#
Here are a couple of examples of queries we could run.
Overall statistics#
mysql> select tag,sum(tests),avg(time),sum(time) from testsuiteresult group by tag order by tag; +--------------+------------+-----------+-----------+ | tag | sum(tests) | avg(time) | sum(time) | +--------------+------------+-----------+-----------+ | 2.8.2-svn-13 | 948 | 1.9197143 | 174.694 | | 3.0.0-svn-17 | 944 | 5.5338652 | 492.514 | | 3.0.0-svn-71 | 1050 | 4.4532762 | 467.594 | | 3.0.0-svn-72 | 1055 | 4.5185849 | 478.970 | | 3.0.0-svn-73 | 1057 | 4.6632617 | 498.969 | | 3.0.0-svn-76 | 1033 | 4.5757476 | 471.302 | | 3.0.0-svn-80 | 950 | 4.3344433 | 420.441 | | 3.0.0-svn-85 | 949 | 3.0273196 | 293.650 | | 3.0.0-svn-93 | 948 | 2.7866327 | 273.090 | | 3.0.0-svn-96 | 948 | 2.8197959 | 276.340 | +--------------+------------+-----------+-----------+ 10 rows in set (0.06 sec)
Number of testsuites having errors or failures, grouped by tag#
There is also a daily refreshed version
mysql> select tag as version,max(timestamp) as date ,sum(tests) as tests ,sum(errors) as errors ,sum(failures)/sum(tests)*100 as 'error %',sum(failures) as failures, sum(failures)/sum(tests)*100 as 'failure %' from testsuiteresult group by tag order by timestamp; +---------------+---------------------+-------+--------+---------+----------+-----------+ | version | date | tests | errors | error % | failures | failure % | +---------------+---------------------+-------+--------+---------+----------+-----------+ | 3.0.0-svn-71 | 2009-02-19 19:21:30 | 1050 | 5 | 0.0952 | 1 | 0.0952 | | 2.8.2-svn-13 | 2009-02-19 20:14:21 | 948 | 0 | 0.2110 | 2 | 0.2110 | | 3.0.0-svn-17 | 2009-02-22 13:00:45 | 944 | 0 | 0.1059 | 1 | 0.1059 | | 3.0.0-svn-72 | 2009-02-24 19:37:49 | 1055 | 5 | 0.1896 | 2 | 0.1896 | | 3.0.0-svn-73 | 2009-02-25 20:21:18 | 1057 | 1 | 0.1892 | 2 | 0.1892 | | 3.0.0-svn-76 | 2009-03-01 19:04:10 | 1033 | 1 | 0.0968 | 1 | 0.0968 | | 3.0.0-svn-80 | 2009-03-06 20:41:24 | 950 | 45 | 7.6842 | 73 | 7.6842 | | 3.0.0-svn-85 | 2009-03-18 18:07:45 | 949 | 46 | 8.6407 | 82 | 8.6407 | | 3.0.0-svn-93 | 2009-04-05 17:26:52 | 948 | 53 | 12.1308 | 115 | 12.1308 | | 3.0.0-svn-96 | 2009-04-06 17:09:47 | 948 | 70 | 10.8650 | 103 | 10.8650 | | 3.0.0-svn-102 | 2009-04-16 18:28:20 | 952 | 50 | 11.6597 | 111 | 11.6597 | | 3.0.0-svn-159 | 2009-09-30 19:07:11 | 998 | 37 | 3.2064 | 32 | 3.2064 | | 3.0.0-svn-161 | 2009-10-08 17:55:33 | 1018 | 37 | 4.7151 | 48 | 4.7151 | +---------------+---------------------+-------+--------+---------+----------+-----------+ 13 rows in set (0.07 sec)
Errors / failures by class / tag#
mysql> select tag,name,errors,failures,time from testsuiteresult where failures>0 or errors > 0 order by name,tag; +--------------+------------------------------------------------------------+--------+----------+---------+ | tag | name | errors | failures | time | +--------------+------------------------------------------------------------+--------+----------+---------+ | 3.0.0-svn-80 | org.apache.wiki.action.DeleteActionBeanTest | 3 | 0 | 1.092 | | 3.0.0-svn-85 | org.apache.wiki.action.DeleteActionBeanTest | 0 | 3 | 1.123 | | 3.0.0-svn-93 | org.apache.wiki.action.DeleteActionBeanTest | 3 | 0 | 1.058 | | 3.0.0-svn-80 | org.apache.wiki.action.RenameActionBeanTest | 0 | 6 | 13.108 | | 3.0.0-svn-85 | org.apache.wiki.action.RenameActionBeanTest | 0 | 4 | 11.771 | | 3.0.0-svn-93 | org.apache.wiki.action.RenameActionBeanTest | 0 | 4 | 11.249 | | 3.0.0-svn-80 | org.apache.wiki.action.ViewActionBeanTest | 0 | 2 | 5.597 | | 3.0.0-svn-80 | org.apache.wiki.action.WikiContextFactoryTest | 0 | 1 | 6.216 | | 3.0.0-svn-80 | org.apache.wiki.auth.acl.DefaultAclManagerTest | 1 | 0 | 1.032 | | 3.0.0-svn-85 | org.apache.wiki.auth.acl.DefaultAclManagerTest | 0 | 1 | 1.097 | | 3.0.0-svn-93 | org.apache.wiki.auth.acl.DefaultAclManagerTest | 0 | 1 | 1.666 | | 3.0.0-svn-80 | org.apache.wiki.auth.AuthorizationManagerTest | 1 | 0 | 5.174 | | 3.0.0-svn-93 | org.apache.wiki.auth.AuthorizationManagerTest | 0 | 2 | 4.960 | | 3.0.0-svn-80 | org.apache.wiki.auth.user.XMLUserDatabaseTest | 0 | 1 | 3.343 | | 3.0.0-svn-85 | org.apache.wiki.auth.user.XMLUserDatabaseTest | 0 | 1 | 3.470 | | 3.0.0-svn-93 | org.apache.wiki.auth.user.XMLUserDatabaseTest | 0 | 1 | 3.194 | | 3.0.0-svn-80 | org.apache.wiki.auth.UserManagerTest | 0 | 1 | 2.225 | | 3.0.0-svn-85 | org.apache.wiki.auth.UserManagerTest | 0 | 1 | 2.316 | | 3.0.0-svn-93 | org.apache.wiki.auth.UserManagerTest | 0 | 1 | 2.111 | | 3.0.0-svn-71 | org.apache.wiki.content.ContentManagerTest | 4 | 0 | 1.236 | | 3.0.0-svn-72 | org.apache.wiki.content.ContentManagerTest | 4 | 0 | 1.243 | | 3.0.0-svn-85 | org.apache.wiki.content.ContentManagerTest | 1 | 0 | 1.261 | | 3.0.0-svn-93 | org.apache.wiki.content.ContentManagerTest | 1 | 0 | 1.388 | | 3.0.0-svn-93 | org.apache.wiki.content.EnglishPluralsPageNameResolverTest | 0 | 1 | 0.310 | | 3.0.0-svn-80 | org.apache.wiki.content.PageRenamerTest | 17 | 3 | 7.174 | | 3.0.0-svn-85 | org.apache.wiki.content.PageRenamerTest | 18 | 3 | 7.533 | | 3.0.0-svn-93 | org.apache.wiki.content.PageRenamerTest | 18 | 3 | 7.391 | | 2.8.2-svn-13 | org.apache.wiki.dav.AttachmentDavProviderTest | 0 | 1 | 0.103 | | 3.0.0-svn-72 | org.apache.wiki.dav.AttachmentDavProviderTest | 0 | 1 | 0.989 | | 3.0.0-svn-73 | org.apache.wiki.dav.AttachmentDavProviderTest | 0 | 1 | 3.377 | | 3.0.0-svn-80 | org.apache.wiki.parser.JSPWikiMarkupParserTest | 0 | 27 | 68.912 | | 3.0.0-svn-85 | org.apache.wiki.parser.JSPWikiMarkupParserTest | 1 | 28 | 67.451 | | 3.0.0-svn-93 | org.apache.wiki.parser.JSPWikiMarkupParserTest | 2 | 52 | 61.415 | | 3.0.0-svn-85 | org.apache.wiki.plugin.CounterPluginTest | 3 | 0 | 1.925 | | 3.0.0-svn-93 | org.apache.wiki.plugin.CounterPluginTest | 3 | 0 | 1.708 | | 3.0.0-svn-93 | org.apache.wiki.plugin.IndexPluginTest | 0 | 1 | 0.413 | | 3.0.0-svn-80 | org.apache.wiki.plugin.InsertPageTest | 0 | 2 | 2.137 | | 3.0.0-svn-93 | org.apache.wiki.plugin.PluginManagerTest | 15 | 0 | 6.208 | | 3.0.0-svn-80 | org.apache.wiki.plugin.RecentChangesPluginTest | 2 | 1 | 1.448 | | 3.0.0-svn-85 | org.apache.wiki.plugin.RecentChangesPluginTest | 0 | 3 | 3.612 | | 3.0.0-svn-93 | org.apache.wiki.plugin.RecentChangesPluginTest | 0 | 3 | 1.243 | | 3.0.0-svn-80 | org.apache.wiki.plugin.ReferredPagesPluginTest | 0 | 2 | 1.311 | | 3.0.0-svn-85 | org.apache.wiki.plugin.ReferredPagesPluginTest | 0 | 3 | 1.326 | | 3.0.0-svn-93 | org.apache.wiki.plugin.ReferredPagesPluginTest | 0 | 3 | 1.323 | | 3.0.0-svn-80 | org.apache.wiki.plugin.ReferringPagesPluginTest | 1 | 3 | 3.605 | | 3.0.0-svn-85 | org.apache.wiki.plugin.ReferringPagesPluginTest | 0 | 6 | 3.553 | | 3.0.0-svn-93 | org.apache.wiki.plugin.ReferringPagesPluginTest | 0 | 7 | 3.520 | | 3.0.0-svn-80 | org.apache.wiki.plugin.TableOfContentsTest | 0 | 5 | 2.926 | | 3.0.0-svn-85 | org.apache.wiki.plugin.TableOfContentsTest | 0 | 5 | 3.001 | | 3.0.0-svn-93 | org.apache.wiki.plugin.TableOfContentsTest | 0 | 5 | 2.874 | | 3.0.0-svn-80 | org.apache.wiki.plugin.UndefinedPagesPluginTest | 0 | 2 | 0.837 | | 3.0.0-svn-85 | org.apache.wiki.plugin.UndefinedPagesPluginTest | 0 | 2 | 0.836 | | 3.0.0-svn-93 | org.apache.wiki.plugin.UndefinedPagesPluginTest | 0 | 2 | 0.893 | | 3.0.0-svn-80 | org.apache.wiki.ReferenceManagerTest | 1 | 1 | 19.646 | | 3.0.0-svn-85 | org.apache.wiki.ReferenceManagerTest | 14 | 2 | 15.744 | | 3.0.0-svn-93 | org.apache.wiki.ReferenceManagerTest | 2 | 13 | 15.591 | | 3.0.0-svn-85 | org.apache.wiki.render.RenderingManagerTest | 1 | 0 | 0.359 | | 3.0.0-svn-80 | org.apache.wiki.render.WysiwygEditingRendererTest | 0 | 1 | 0.737 | | 3.0.0-svn-85 | org.apache.wiki.render.WysiwygEditingRendererTest | 0 | 1 | 0.670 | | 3.0.0-svn-93 | org.apache.wiki.render.WysiwygEditingRendererTest | 0 | 2 | 0.656 | | 3.0.0-svn-80 | org.apache.wiki.rss.RSSGeneratorTest | 2 | 0 | 0.948 | | 3.0.0-svn-80 | org.apache.wiki.search.SearchManagerTest | 4 | 1 | 8.328 | | 3.0.0-svn-85 | org.apache.wiki.search.SearchManagerTest | 4 | 1 | 8.068 | | 3.0.0-svn-93 | org.apache.wiki.search.SearchManagerTest | 4 | 0 | 7.556 | | 3.0.0-svn-71 | org.apache.wiki.ui.migrator.BundleMigratorTest | 1 | 0 | 0.092 | | 3.0.0-svn-72 | org.apache.wiki.ui.migrator.BundleMigratorTest | 1 | 0 | 0.093 | | 3.0.0-svn-73 | org.apache.wiki.ui.migrator.BundleMigratorTest | 1 | 0 | 0.479 | | 3.0.0-svn-76 | org.apache.wiki.ui.migrator.BundleMigratorTest | 1 | 0 | 0.102 | | 3.0.0-svn-80 | org.apache.wiki.ui.migrator.BundleMigratorTest | 1 | 0 | 0.507 | | 3.0.0-svn-85 | org.apache.wiki.ui.migrator.JspParserTest | 0 | 1 | 0.060 | | 3.0.0-svn-93 | org.apache.wiki.ui.migrator.JspParserTest | 0 | 1 | 0.072 | | 3.0.0-svn-80 | org.apache.wiki.ui.stripes.HandlerInfoTest | 0 | 1 | 4.437 | | 3.0.0-svn-85 | org.apache.wiki.ui.stripes.HandlerInfoTest | 0 | 1 | 4.384 | | 3.0.0-svn-93 | org.apache.wiki.ui.stripes.HandlerInfoTest | 0 | 1 | 4.135 | | 2.8.2-svn-13 | org.apache.wiki.WikiEngineTest | 0 | 1 | 20.025 | | 3.0.0-svn-17 | org.apache.wiki.WikiEngineTest | 0 | 1 | 37.644 | | 3.0.0-svn-71 | org.apache.wiki.WikiEngineTest | 0 | 1 | 35.495 | | 3.0.0-svn-72 | org.apache.wiki.WikiEngineTest | 0 | 1 | 35.517 | | 3.0.0-svn-73 | org.apache.wiki.WikiEngineTest | 0 | 1 | 35.863 | | 3.0.0-svn-76 | org.apache.wiki.WikiEngineTest | 0 | 1 | 35.308 | | 3.0.0-svn-80 | org.apache.wiki.WikiEngineTest | 5 | 13 | 15.016 | | 3.0.0-svn-85 | org.apache.wiki.WikiEngineTest | 3 | 12 | 15.350 | | 3.0.0-svn-93 | org.apache.wiki.WikiEngineTest | 4 | 10 | 14.265 | | 3.0.0-svn-80 | org.apache.wiki.workflow.ApprovalWorkflowTest | 2 | 0 | 1.621 | | 3.0.0-svn-85 | org.apache.wiki.workflow.ApprovalWorkflowTest | 0 | 2 | 1.587 | | 3.0.0-svn-93 | org.apache.wiki.workflow.ApprovalWorkflowTest | 0 | 2 | 1.409 | | 3.0.0-svn-80 | org.apache.wiki.xmlrpc.RPCHandlerTest | 4 | 0 | 1.447 | | 3.0.0-svn-85 | org.apache.wiki.xmlrpc.RPCHandlerTest | 0 | 2 | 1.525 | | 3.0.0-svn-93 | org.apache.wiki.xmlrpc.RPCHandlerTest | 1 | 0 | 1.193 | | 3.0.0-svn-80 | stress.MassiveRepositoryTest | 1 | 0 | 124.826 | | 3.0.0-svn-85 | stress.MassiveRepositoryTest | 1 | 0 | 0.391 | +--------------+------------------------------------------------------------+--------+----------+---------+ 91 rows in set (0.02 sec)
Number of testsuites/testcases per tag (buildlevel)#
mysql> select tag,count(*) from testcaseresult group by tag order by tag; +--------------+----------+ | tag | count(*) | +--------------+----------+ | 2.8.2-svn-13 | 948 | | 3.0.0-svn-17 | 944 | | 3.0.0-svn-71 | 1050 | | 3.0.0-svn-72 | 1055 | | 3.0.0-svn-73 | 1057 | | 3.0.0-svn-76 | 1033 | | 3.0.0-svn-80 | 950 | +--------------+----------+ 7 rows in set (0.22 sec)
All testsuites per tag having errors or failures#
mysql> select tag,name,tests,errors,failures from testsuiteresult where errors>0 or failures > 0 group by tag,name order by name,tag; +--------------+---------------------------------------------------+-------+--------+----------+ | tag | name | tests | errors | failures | +--------------+---------------------------------------------------+-------+--------+----------+ | 3.0.0-svn-80 | org.apache.wiki.action.DeleteActionBeanTest | 3 | 3 | 0 | | 3.0.0-svn-80 | org.apache.wiki.action.RenameActionBeanTest | 7 | 0 | 6 | | 3.0.0-svn-80 | org.apache.wiki.action.ViewActionBeanTest | 3 | 0 | 2 | | 3.0.0-svn-80 | org.apache.wiki.action.WikiContextFactoryTest | 5 | 0 | 1 | | 3.0.0-svn-80 | org.apache.wiki.auth.acl.DefaultAclManagerTest | 3 | 1 | 0 | | 3.0.0-svn-80 | org.apache.wiki.auth.AuthorizationManagerTest | 15 | 1 | 0 | | 3.0.0-svn-80 | org.apache.wiki.auth.user.XMLUserDatabaseTest | 11 | 0 | 1 | | 3.0.0-svn-80 | org.apache.wiki.auth.UserManagerTest | 5 | 0 | 1 | | 3.0.0-svn-71 | org.apache.wiki.content.ContentManagerTest | 4 | 4 | 0 | | 3.0.0-svn-72 | org.apache.wiki.content.ContentManagerTest | 4 | 4 | 0 | | 3.0.0-svn-80 | org.apache.wiki.content.PageRenamerTest | 21 | 17 | 3 | | 2.8.2-svn-13 | org.apache.wiki.dav.AttachmentDavProviderTest | 3 | 0 | 1 | | 3.0.0-svn-72 | org.apache.wiki.dav.AttachmentDavProviderTest | 3 | 0 | 1 | | 3.0.0-svn-73 | org.apache.wiki.dav.AttachmentDavProviderTest | 3 | 0 | 1 | | 3.0.0-svn-80 | org.apache.wiki.parser.JSPWikiMarkupParserTest | 200 | 0 | 27 | | 3.0.0-svn-80 | org.apache.wiki.plugin.InsertPageTest | 6 | 0 | 2 | | 3.0.0-svn-80 | org.apache.wiki.plugin.RecentChangesPluginTest | 3 | 2 | 1 | | 3.0.0-svn-80 | org.apache.wiki.plugin.ReferredPagesPluginTest | 3 | 0 | 2 | | 3.0.0-svn-80 | org.apache.wiki.plugin.ReferringPagesPluginTest | 8 | 1 | 3 | | 3.0.0-svn-80 | org.apache.wiki.plugin.TableOfContentsTest | 8 | 0 | 5 | | 3.0.0-svn-80 | org.apache.wiki.plugin.UndefinedPagesPluginTest | 2 | 0 | 2 | | 3.0.0-svn-80 | org.apache.wiki.ReferenceManagerTest | 18 | 1 | 1 | | 3.0.0-svn-80 | org.apache.wiki.render.WysiwygEditingRendererTest | 2 | 0 | 1 | | 3.0.0-svn-80 | org.apache.wiki.rss.RSSGeneratorTest | 2 | 2 | 0 | | 3.0.0-svn-80 | org.apache.wiki.search.SearchManagerTest | 6 | 4 | 1 | | 3.0.0-svn-71 | org.apache.wiki.ui.migrator.BundleMigratorTest | 8 | 1 | 0 | | 3.0.0-svn-72 | org.apache.wiki.ui.migrator.BundleMigratorTest | 8 | 1 | 0 | | 3.0.0-svn-73 | org.apache.wiki.ui.migrator.BundleMigratorTest | 8 | 1 | 0 | | 3.0.0-svn-76 | org.apache.wiki.ui.migrator.BundleMigratorTest | 8 | 1 | 0 | | 3.0.0-svn-80 | org.apache.wiki.ui.migrator.BundleMigratorTest | 8 | 1 | 0 | | 3.0.0-svn-80 | org.apache.wiki.ui.stripes.HandlerInfoTest | 4 | 0 | 1 | | 2.8.2-svn-13 | org.apache.wiki.WikiEngineTest | 44 | 0 | 1 | | 3.0.0-svn-17 | org.apache.wiki.WikiEngineTest | 44 | 0 | 1 | | 3.0.0-svn-71 | org.apache.wiki.WikiEngineTest | 44 | 0 | 1 | | 3.0.0-svn-72 | org.apache.wiki.WikiEngineTest | 44 | 0 | 1 | | 3.0.0-svn-73 | org.apache.wiki.WikiEngineTest | 44 | 0 | 1 | | 3.0.0-svn-76 | org.apache.wiki.WikiEngineTest | 44 | 0 | 1 | | 3.0.0-svn-80 | org.apache.wiki.WikiEngineTest | 34 | 5 | 13 | | 3.0.0-svn-80 | org.apache.wiki.workflow.ApprovalWorkflowTest | 5 | 2 | 0 | | 3.0.0-svn-80 | org.apache.wiki.xmlrpc.RPCHandlerTest | 4 | 4 | 0 | | 3.0.0-svn-80 | stress.MassiveRepositoryTest | 1 | 1 | 0 | +--------------+---------------------------------------------------+-------+--------+----------+ 41 rows in set (0.02 sec)
10 most time taking testsuites#
mysql> select tag,name,tests,time from testsuiteresult order by time desc,name,tag limit 10; +--------------+------------------------------------------------+-------+--------+ | tag | name | tests | time | +--------------+------------------------------------------------+-------+--------+ | 3.0.0-svn-17 | stress.MassiveRepositoryTest | 1 | 84.526 | | 3.0.0-svn-73 | stress.MassiveRepositoryTest | 1 | 81.273 | | 3.0.0-svn-71 | stress.MassiveRepositoryTest | 1 | 78.442 | | 3.0.0-svn-72 | stress.MassiveRepositoryTest | 1 | 76.723 | | 3.0.0-svn-76 | stress.MassiveRepositoryTest | 1 | 75.437 | | 3.0.0-svn-17 | org.apache.wiki.parser.JSPWikiMarkupParserTest | 205 | 70.827 | | 2.8.2-svn-13 | stress.MassiveRepositoryTest | 1 | 69.571 | | 3.0.0-svn-73 | org.apache.wiki.parser.JSPWikiMarkupParserTest | 205 | 68.615 | | 3.0.0-svn-72 | org.apache.wiki.parser.JSPWikiMarkupParserTest | 205 | 67.173 | | 3.0.0-svn-76 | org.apache.wiki.parser.JSPWikiMarkupParserTest | 205 | 66.869 | +--------------+------------------------------------------------+-------+--------+ 10 rows in set (0.03 sec)
Classes with most testcases#
mysql> select distinct name,tests from testsuiteresult order by tests desc limit 15; +-----------------------------------------------------------+-------+ | name | tests | +-----------------------------------------------------------+-------+ | org.apache.wiki.parser.JSPWikiMarkupParserTest | 205 | | org.apache.wiki.parser.CreoleToJSPWikiTranslatorTest | 83 | | org.apache.wiki.WikiEngineTest | 44 | | org.apache.wiki.TextUtilTest | 35 | | org.apache.wiki.TextUtilTest | 34 | | org.apache.wiki.workflow.WorkflowTest | 24 | | org.apache.wiki.content.PageRenamerTest | 21 | | org.apache.wiki.ReferenceManagerTest | 18 | | org.apache.wiki.auth.AuthorizationManagerTest | 17 | | org.apache.wiki.htmltowiki.HtmlStringToWikiTranslatorTest | 17 | | org.apache.wiki.ReleaseTest | 17 | | org.apache.wiki.plugin.PluginManagerTest | 16 | | org.apache.wiki.VariableManagerTest | 16 | | org.apache.wiki.workflow.SimpleDecisionTest | 16 | | org.apache.wiki.dav.DavPathTest | 15 | +-----------------------------------------------------------+-------+ 15 rows in set (0.05 sec)
15 most time taking testcase methods#
mysql> select tag,classname,name,time from testcaseresult order by time desc,name,tag limit 15; +--------------+------------------------------------------+------------------------+--------+ | tag | classname | name | time | +--------------+------------------------------------------+------------------------+--------+ | 3.0.0-svn-17 | stress.MassiveRepositoryTest | testMassiveRepository1 | 84.308 | | 3.0.0-svn-73 | stress.MassiveRepositoryTest | testMassiveRepository1 | 81.271 | | 3.0.0-svn-71 | stress.MassiveRepositoryTest | testMassiveRepository1 | 78.440 | | 3.0.0-svn-72 | stress.MassiveRepositoryTest | testMassiveRepository1 | 76.721 | | 3.0.0-svn-76 | stress.MassiveRepositoryTest | testMassiveRepository1 | 75.435 | | 2.8.2-svn-13 | stress.MassiveRepositoryTest | testMassiveRepository1 | 69.569 | | 3.0.0-svn-73 | org.apache.wiki.search.SearchManagerTest | testSimpleSearch | 10.724 | | 3.0.0-svn-17 | org.apache.wiki.search.SearchManagerTest | testSimpleSearch | 10.530 | | 3.0.0-svn-17 | org.apache.wiki.search.SearchManagerTest | testSimpleSearch2 | 10.474 | | 3.0.0-svn-72 | org.apache.wiki.search.SearchManagerTest | testSimpleSearch | 10.433 | | 3.0.0-svn-17 | org.apache.wiki.search.SearchManagerTest | testSimpleSearch3 | 10.396 | | 3.0.0-svn-71 | org.apache.wiki.search.SearchManagerTest | testSimpleSearch | 10.360 | | 3.0.0-svn-73 | org.apache.wiki.search.SearchManagerTest | testSimpleSearch2 | 10.349 | | 3.0.0-svn-76 | org.apache.wiki.search.SearchManagerTest | testSimpleSearch | 10.343 | | 3.0.0-svn-72 | org.apache.wiki.search.SearchManagerTest | testSimpleSearch2 | 10.340 | +--------------+------------------------------------------+------------------------+--------+ 15 rows in set (0.30 sec)
Testsuites that have increased their test time the most between releases#
mysql> select name,(max(time)-min(time))/avg(time)*100 as percdelta,min(time),max(time),avg(time) from testsuiteresult where tag='3.0.0-svn-71' or tag='3.0.0-svn-72' group by name order by 2 desc limit 25; +-----------------------------------------------------------+----------------+-----------+-----------+-----------+ | name | percdelta | min(time) | max(time) | avg(time) | +-----------------------------------------------------------+----------------+-----------+-----------+-----------+ | org.apache.wiki.auth.permissions.PagePermissionTest | 76.92307692308 | 0.004 | 0.009 | 0.0065000 | | org.apache.wiki.dav.DavPathTest | 73.68421052632 | 0.006 | 0.013 | 0.0095000 | | org.apache.wiki.auth.permissions.WikiPermissionTest | 66.66666666667 | 0.004 | 0.008 | 0.0060000 | | org.apache.wiki.util.ClassUtilTest | 66.66666666667 | 0.003 | 0.006 | 0.0045000 | | org.apache.wiki.workflow.SimpleDecisionTest | 60.00000000000 | 0.007 | 0.013 | 0.0100000 | | org.apache.wiki.parser.CreoleToJSPWikiTranslatorTest | 49.81412639405 | 0.101 | 0.168 | 0.1345000 | | org.apache.wiki.PageManagerTest | 47.19887955182 | 4.364 | 7.060 | 5.7120000 | | org.apache.wiki.util.CommentedPropertiesTest | 46.40000000000 | 0.048 | 0.077 | 0.0625000 | | org.apache.wiki.ui.migrator.JspDocumentTest | 40.00000000000 | 0.002 | 0.003 | 0.0025000 | | org.apache.wiki.workflow.TaskTest | 40.00000000000 | 0.006 | 0.009 | 0.0075000 | | org.apache.wiki.ui.migrator.JspParserTest | 38.46153846154 | 0.042 | 0.062 | 0.0520000 | | org.apache.wiki.auth.permissions.GroupPermissionTest | 35.29411764706 | 0.007 | 0.010 | 0.0085000 | | org.apache.wiki.auth.authorize.WebContainerAuthorizerTest | 34.02489626556 | 1.400 | 1.974 | 1.6870000 | | org.apache.wiki.auth.authorize.GroupTest | 32.66381297333 | 2.541 | 3.533 | 3.0370000 | | org.apache.wiki.util.PriorityListTest | 28.57142857143 | 0.003 | 0.004 | 0.0035000 | | org.apache.wiki.parser.MarkupParserTest | 28.57142857143 | 0.003 | 0.004 | 0.0035000 | | org.apache.wiki.content.WikiNameTest | 28.57142857143 | 0.003 | 0.004 | 0.0035000 | | org.apache.wiki.PropertyReaderTest | 25.00000000000 | 0.014 | 0.018 | 0.0160000 | | org.apache.wiki.attachment.AttachmentManagerTest | 24.69245065935 | 4.952 | 6.347 | 5.6495000 | | org.apache.wiki.auth.acl.DefaultAclManagerTest | 24.02159244265 | 0.978 | 1.245 | 1.1115000 | | org.apache.wiki.FileUtilTest | 23.85964912281 | 0.251 | 0.319 | 0.2850000 | | org.apache.wiki.util.UtilJ2eeCompatTest | 22.22222222222 | 0.004 | 0.005 | 0.0045000 | | org.apache.wiki.auth.authorize.JDBCGroupDatabaseTest | 21.84782608696 | 1.639 | 2.041 | 1.8400000 | | org.apache.wiki.util.CryptoUtilTest | 18.18181818182 | 0.005 | 0.006 | 0.0055000 | | org.apache.wiki.ReleaseTest | 18.18181818182 | 0.015 | 0.018 | 0.0165000 | +-----------------------------------------------------------+----------------+-----------+-----------+-----------+ 25 rows in set (0.03 sec)