!! JSPWiki JUnit test performance management
[{TableOfContents }]
\\
The idea is to use the information created by the JUnit tests.
\\Every time you run an " ant tests", a ''reports'' directory is created containing a test report for each test.
each JUnit test produces an [XML format report|TEST-org.apache.wiki.TextUtilTest.xml] containing response times, errors and failures.
I wrote some code that gathers this information and stores it in an SQL database.
\\It takes two parameters :
* the name of the directory %%small (i.e. "/home/metskem/workspace/JSPWiki/build/tests/reports") %%
* a tag that identifies this test run %%small (i.e. "3.0.0-svn-17") %%
If we gather this information for each JSPWiki level that has significant changes, we could analyze if, and where performance differences occur, and also keep track of a lot of other things.
! DDL testcaseresult table
{{{
DROP TABLE IF EXISTS `testcaseresult`;
SET @saved_cs_client = @@character_set_client;
SET character_set_client = utf8;
CREATE TABLE `testcaseresult` (
`tag` varchar(32) NOT NULL,
`hostname` varchar(32) NOT NULL,
`timestamp` timestamp NOT NULL,
`classname` varchar(128) NOT NULL,
`name` varchar(128) NOT NULL,
`time` double(8,3) NOT NULL default '0.000',
PRIMARY KEY (`hostname`,`classname`,`name`,`timestamp`)
) TYPE=InnoDB DEFAULT CHARSET=utf8;
SET character_set_client = @saved_cs_client;
}}}
! Sample content testcaseresult
{{{
mysql> select * from testcaseresult limit 10;
+--------------+----------+---------------------+---------------------------------------------+-------------------------------------+-------+
| tag | hostname | timestamp | classname | name | time |
+--------------+----------+---------------------+---------------------------------------------+-------------------------------------+-------+
| 3.0.0-svn-71 | bismarck | 2009-02-19 19:15:55 | org.apache.wiki.action.GroupActionBeanTest | testDeleteGroup | 1.990 |
| 3.0.0-svn-71 | bismarck | 2009-02-19 19:15:55 | org.apache.wiki.action.GroupActionBeanTest | testSaveExistingGroup | 1.968 |
| 3.0.0-svn-71 | bismarck | 2009-02-19 19:15:55 | org.apache.wiki.action.GroupActionBeanTest | testSaveNewGroup | 2.011 |
| 3.0.0-svn-71 | bismarck | 2009-02-19 19:15:55 | org.apache.wiki.action.GroupActionBeanTest | testView | 2.398 |
| 3.0.0-svn-71 | bismarck | 2009-02-19 19:15:55 | org.apache.wiki.action.GroupActionBeanTest | testViewNonExistent | 1.972 |
| 3.0.0-svn-71 | bismarck | 2009-02-19 19:15:55 | org.apache.wiki.action.GroupActionBeanTest | testViewNullGroup | 1.967 |
| 3.0.0-svn-71 | bismarck | 2009-02-19 19:16:18 | org.apache.wiki.action.RenameActionBeanTest | testRename | 1.975 |
| 3.0.0-svn-71 | bismarck | 2009-02-19 19:16:18 | org.apache.wiki.action.RenameActionBeanTest | testRenameReferences | 1.972 |
| 3.0.0-svn-71 | bismarck | 2009-02-19 19:16:18 | org.apache.wiki.action.RenameActionBeanTest | testRenameReferencesChangeRefsFalse | 1.969 |
| 3.0.0-svn-71 | bismarck | 2009-02-19 19:16:18 | org.apache.wiki.action.RenameActionBeanTest | testRenameReferencesChangeRefsTrue | 1.975 |
+--------------+----------+---------------------+---------------------------------------------+-------------------------------------+-------+
10 rows in set (0.01 sec)
}}}
! DDL testsuiteresult table
{{{
DROP TABLE IF EXISTS `testsuiteresult`;
SET @saved_cs_client = @@character_set_client;
SET character_set_client = utf8;
CREATE TABLE `testsuiteresult` (
`tag` varchar(32) NOT NULL,
`hostname` varchar(32) NOT NULL,
`timestamp` timestamp NOT NULL,
`name` varchar(128) NOT NULL,
`errors` int(11) NOT NULL default '0',
`tests` int(11) NOT NULL default '0',
`failures` int(11) NOT NULL default '0',
`time` double(8,3) NOT NULL default '0.000',
PRIMARY KEY (`hostname`,`name`,`timestamp`)
) TYPE=InnoDB DEFAULT CHARSET=utf8;
SET character_set_client = @saved_cs_client;
}}}
! Sample content testsuiteresult
{{{
mysql> select * from testsuiteresult;
+--------------+----------+---------------------+--------------------------------------------------------------+--------+-------+----------+--------+
| tag | hostname | timestamp | name | errors | tests | failures | time |
+--------------+----------+---------------------+--------------------------------------------------------------+--------+-------+----------+--------+
| 3.0.0-svn-71 | bismarck | 2009-02-19 19:15:55 | org.apache.wiki.action.GroupActionBeanTest | 0 | 6 | 0 | 12.310 |
| 3.0.0-svn-71 | bismarck | 2009-02-19 19:16:18 | org.apache.wiki.action.RenameActionBeanTest | 0 | 6 | 0 | 11.867 |
| 3.0.0-svn-71 | bismarck | 2009-02-19 19:16:54 | org.apache.wiki.action.WikiContextFactoryTest | 0 | 5 | 0 | 6.423 |
| 3.0.0-svn-71 | bismarck | 2009-02-19 19:17:21 | org.apache.wiki.auth.acl.DefaultAclManagerTest | 0 | 3 | 0 | 1.245 |
}}}
!! Running the collector tool
Each run will insert about 100 records in the testsuiteresult table, and about a 1000 rows in the testcaseresult table.
\\The following is an example output of a run:
{{{
21:08:14,974 WARN MainProcessor:122 - processing 106 files from directory /home/metskem/workspace/JSPWiki/build/tests/reports, tag : 3.0.0-svn-72
21:08:16,241 ERROR MainProcessor:135 - parsing completed in 1 second(s)
21:08:16,348 INFO DBUtil:100 - class com.mysql.jdbc.Driver succesfully loaded
21:08:17,993 INFO DBUtil:125 - transaction support present ? : true
21:08:27,327 WARN DBUtil:169 - committing changes....
21:08:27,330 WARN DBUtil:171 - commit completed
21:08:27,334 ERROR MainProcessor:97 - inserted 1161 rows in 11 seconds
}}}
!! Some useful queries
Here are a couple of examples of queries we could run.
! Overall statistics
{{{
mysql> select tag,sum(tests),avg(time),sum(time) from testsuiteresult group by tag order by tag;
+--------------+------------+-----------+-----------+
| tag | sum(tests) | avg(time) | sum(time) |
+--------------+------------+-----------+-----------+
| 2.8.2-svn-13 | 948 | 1.9197143 | 174.694 |
| 3.0.0-svn-17 | 944 | 5.5338652 | 492.514 |
| 3.0.0-svn-71 | 1050 | 4.4532762 | 467.594 |
| 3.0.0-svn-72 | 1055 | 4.5185849 | 478.970 |
| 3.0.0-svn-73 | 1057 | 4.6632617 | 498.969 |
| 3.0.0-svn-76 | 1033 | 4.5757476 | 471.302 |
| 3.0.0-svn-80 | 950 | 4.3344433 | 420.441 |
| 3.0.0-svn-85 | 949 | 3.0273196 | 293.650 |
| 3.0.0-svn-93 | 948 | 2.7866327 | 273.090 |
| 3.0.0-svn-96 | 948 | 2.8197959 | 276.340 |
+--------------+------------+-----------+-----------+
10 rows in set (0.06 sec)
}}}
! Number of testsuites having errors or failures, grouped by tag
There is also a [daily refreshed version|http://www.computerhok.nl/tmp/jspwiki-testresult.html]
{{{
mysql> select tag as version,max(timestamp) as date ,sum(tests) as tests ,sum(errors) as errors ,sum(failures)/sum(tests)*100 as 'error %',sum(failures) as failures, sum(failures)/sum(tests)*100 as 'failure %' from testsuiteresult group by tag order by timestamp;
+---------------+---------------------+-------+--------+---------+----------+-----------+
| version | date | tests | errors | error % | failures | failure % |
+---------------+---------------------+-------+--------+---------+----------+-----------+
| 3.0.0-svn-71 | 2009-02-19 19:21:30 | 1050 | 5 | 0.0952 | 1 | 0.0952 |
| 2.8.2-svn-13 | 2009-02-19 20:14:21 | 948 | 0 | 0.2110 | 2 | 0.2110 |
| 3.0.0-svn-17 | 2009-02-22 13:00:45 | 944 | 0 | 0.1059 | 1 | 0.1059 |
| 3.0.0-svn-72 | 2009-02-24 19:37:49 | 1055 | 5 | 0.1896 | 2 | 0.1896 |
| 3.0.0-svn-73 | 2009-02-25 20:21:18 | 1057 | 1 | 0.1892 | 2 | 0.1892 |
| 3.0.0-svn-76 | 2009-03-01 19:04:10 | 1033 | 1 | 0.0968 | 1 | 0.0968 |
| 3.0.0-svn-80 | 2009-03-06 20:41:24 | 950 | 45 | 7.6842 | 73 | 7.6842 |
| 3.0.0-svn-85 | 2009-03-18 18:07:45 | 949 | 46 | 8.6407 | 82 | 8.6407 |
| 3.0.0-svn-93 | 2009-04-05 17:26:52 | 948 | 53 | 12.1308 | 115 | 12.1308 |
| 3.0.0-svn-96 | 2009-04-06 17:09:47 | 948 | 70 | 10.8650 | 103 | 10.8650 |
| 3.0.0-svn-102 | 2009-04-16 18:28:20 | 952 | 50 | 11.6597 | 111 | 11.6597 |
| 3.0.0-svn-159 | 2009-09-30 19:07:11 | 998 | 37 | 3.2064 | 32 | 3.2064 |
| 3.0.0-svn-161 | 2009-10-08 17:55:33 | 1018 | 37 | 4.7151 | 48 | 4.7151 |
+---------------+---------------------+-------+--------+---------+----------+-----------+
13 rows in set (0.07 sec)
}}}
! Errors / failures by class / tag
{{{
mysql> select tag,name,errors,failures,time from testsuiteresult where failures>0 or errors > 0 order by name,tag;
+--------------+------------------------------------------------------------+--------+----------+---------+
| tag | name | errors | failures | time |
+--------------+------------------------------------------------------------+--------+----------+---------+
| 3.0.0-svn-80 | org.apache.wiki.action.DeleteActionBeanTest | 3 | 0 | 1.092 |
| 3.0.0-svn-85 | org.apache.wiki.action.DeleteActionBeanTest | 0 | 3 | 1.123 |
| 3.0.0-svn-93 | org.apache.wiki.action.DeleteActionBeanTest | 3 | 0 | 1.058 |
| 3.0.0-svn-80 | org.apache.wiki.action.RenameActionBeanTest | 0 | 6 | 13.108 |
| 3.0.0-svn-85 | org.apache.wiki.action.RenameActionBeanTest | 0 | 4 | 11.771 |
| 3.0.0-svn-93 | org.apache.wiki.action.RenameActionBeanTest | 0 | 4 | 11.249 |
| 3.0.0-svn-80 | org.apache.wiki.action.ViewActionBeanTest | 0 | 2 | 5.597 |
| 3.0.0-svn-80 | org.apache.wiki.action.WikiContextFactoryTest | 0 | 1 | 6.216 |
| 3.0.0-svn-80 | org.apache.wiki.auth.acl.DefaultAclManagerTest | 1 | 0 | 1.032 |
| 3.0.0-svn-85 | org.apache.wiki.auth.acl.DefaultAclManagerTest | 0 | 1 | 1.097 |
| 3.0.0-svn-93 | org.apache.wiki.auth.acl.DefaultAclManagerTest | 0 | 1 | 1.666 |
| 3.0.0-svn-80 | org.apache.wiki.auth.AuthorizationManagerTest | 1 | 0 | 5.174 |
| 3.0.0-svn-93 | org.apache.wiki.auth.AuthorizationManagerTest | 0 | 2 | 4.960 |
| 3.0.0-svn-80 | org.apache.wiki.auth.user.XMLUserDatabaseTest | 0 | 1 | 3.343 |
| 3.0.0-svn-85 | org.apache.wiki.auth.user.XMLUserDatabaseTest | 0 | 1 | 3.470 |
| 3.0.0-svn-93 | org.apache.wiki.auth.user.XMLUserDatabaseTest | 0 | 1 | 3.194 |
| 3.0.0-svn-80 | org.apache.wiki.auth.UserManagerTest | 0 | 1 | 2.225 |
| 3.0.0-svn-85 | org.apache.wiki.auth.UserManagerTest | 0 | 1 | 2.316 |
| 3.0.0-svn-93 | org.apache.wiki.auth.UserManagerTest | 0 | 1 | 2.111 |
| 3.0.0-svn-71 | org.apache.wiki.content.ContentManagerTest | 4 | 0 | 1.236 |
| 3.0.0-svn-72 | org.apache.wiki.content.ContentManagerTest | 4 | 0 | 1.243 |
| 3.0.0-svn-85 | org.apache.wiki.content.ContentManagerTest | 1 | 0 | 1.261 |
| 3.0.0-svn-93 | org.apache.wiki.content.ContentManagerTest | 1 | 0 | 1.388 |
| 3.0.0-svn-93 | org.apache.wiki.content.EnglishPluralsPageNameResolverTest | 0 | 1 | 0.310 |
| 3.0.0-svn-80 | org.apache.wiki.content.PageRenamerTest | 17 | 3 | 7.174 |
| 3.0.0-svn-85 | org.apache.wiki.content.PageRenamerTest | 18 | 3 | 7.533 |
| 3.0.0-svn-93 | org.apache.wiki.content.PageRenamerTest | 18 | 3 | 7.391 |
| 2.8.2-svn-13 | org.apache.wiki.dav.AttachmentDavProviderTest | 0 | 1 | 0.103 |
| 3.0.0-svn-72 | org.apache.wiki.dav.AttachmentDavProviderTest | 0 | 1 | 0.989 |
| 3.0.0-svn-73 | org.apache.wiki.dav.AttachmentDavProviderTest | 0 | 1 | 3.377 |
| 3.0.0-svn-80 | org.apache.wiki.parser.JSPWikiMarkupParserTest | 0 | 27 | 68.912 |
| 3.0.0-svn-85 | org.apache.wiki.parser.JSPWikiMarkupParserTest | 1 | 28 | 67.451 |
| 3.0.0-svn-93 | org.apache.wiki.parser.JSPWikiMarkupParserTest | 2 | 52 | 61.415 |
| 3.0.0-svn-85 | org.apache.wiki.plugin.CounterPluginTest | 3 | 0 | 1.925 |
| 3.0.0-svn-93 | org.apache.wiki.plugin.CounterPluginTest | 3 | 0 | 1.708 |
| 3.0.0-svn-93 | org.apache.wiki.plugin.IndexPluginTest | 0 | 1 | 0.413 |
| 3.0.0-svn-80 | org.apache.wiki.plugin.InsertPageTest | 0 | 2 | 2.137 |
| 3.0.0-svn-93 | org.apache.wiki.plugin.PluginManagerTest | 15 | 0 | 6.208 |
| 3.0.0-svn-80 | org.apache.wiki.plugin.RecentChangesPluginTest | 2 | 1 | 1.448 |
| 3.0.0-svn-85 | org.apache.wiki.plugin.RecentChangesPluginTest | 0 | 3 | 3.612 |
| 3.0.0-svn-93 | org.apache.wiki.plugin.RecentChangesPluginTest | 0 | 3 | 1.243 |
| 3.0.0-svn-80 | org.apache.wiki.plugin.ReferredPagesPluginTest | 0 | 2 | 1.311 |
| 3.0.0-svn-85 | org.apache.wiki.plugin.ReferredPagesPluginTest | 0 | 3 | 1.326 |
| 3.0.0-svn-93 | org.apache.wiki.plugin.ReferredPagesPluginTest | 0 | 3 | 1.323 |
| 3.0.0-svn-80 | org.apache.wiki.plugin.ReferringPagesPluginTest | 1 | 3 | 3.605 |
| 3.0.0-svn-85 | org.apache.wiki.plugin.ReferringPagesPluginTest | 0 | 6 | 3.553 |
| 3.0.0-svn-93 | org.apache.wiki.plugin.ReferringPagesPluginTest | 0 | 7 | 3.520 |
| 3.0.0-svn-80 | org.apache.wiki.plugin.TableOfContentsTest | 0 | 5 | 2.926 |
| 3.0.0-svn-85 | org.apache.wiki.plugin.TableOfContentsTest | 0 | 5 | 3.001 |
| 3.0.0-svn-93 | org.apache.wiki.plugin.TableOfContentsTest | 0 | 5 | 2.874 |
| 3.0.0-svn-80 | org.apache.wiki.plugin.UndefinedPagesPluginTest | 0 | 2 | 0.837 |
| 3.0.0-svn-85 | org.apache.wiki.plugin.UndefinedPagesPluginTest | 0 | 2 | 0.836 |
| 3.0.0-svn-93 | org.apache.wiki.plugin.UndefinedPagesPluginTest | 0 | 2 | 0.893 |
| 3.0.0-svn-80 | org.apache.wiki.ReferenceManagerTest | 1 | 1 | 19.646 |
| 3.0.0-svn-85 | org.apache.wiki.ReferenceManagerTest | 14 | 2 | 15.744 |
| 3.0.0-svn-93 | org.apache.wiki.ReferenceManagerTest | 2 | 13 | 15.591 |
| 3.0.0-svn-85 | org.apache.wiki.render.RenderingManagerTest | 1 | 0 | 0.359 |
| 3.0.0-svn-80 | org.apache.wiki.render.WysiwygEditingRendererTest | 0 | 1 | 0.737 |
| 3.0.0-svn-85 | org.apache.wiki.render.WysiwygEditingRendererTest | 0 | 1 | 0.670 |
| 3.0.0-svn-93 | org.apache.wiki.render.WysiwygEditingRendererTest | 0 | 2 | 0.656 |
| 3.0.0-svn-80 | org.apache.wiki.rss.RSSGeneratorTest | 2 | 0 | 0.948 |
| 3.0.0-svn-80 | org.apache.wiki.search.SearchManagerTest | 4 | 1 | 8.328 |
| 3.0.0-svn-85 | org.apache.wiki.search.SearchManagerTest | 4 | 1 | 8.068 |
| 3.0.0-svn-93 | org.apache.wiki.search.SearchManagerTest | 4 | 0 | 7.556 |
| 3.0.0-svn-71 | org.apache.wiki.ui.migrator.BundleMigratorTest | 1 | 0 | 0.092 |
| 3.0.0-svn-72 | org.apache.wiki.ui.migrator.BundleMigratorTest | 1 | 0 | 0.093 |
| 3.0.0-svn-73 | org.apache.wiki.ui.migrator.BundleMigratorTest | 1 | 0 | 0.479 |
| 3.0.0-svn-76 | org.apache.wiki.ui.migrator.BundleMigratorTest | 1 | 0 | 0.102 |
| 3.0.0-svn-80 | org.apache.wiki.ui.migrator.BundleMigratorTest | 1 | 0 | 0.507 |
| 3.0.0-svn-85 | org.apache.wiki.ui.migrator.JspParserTest | 0 | 1 | 0.060 |
| 3.0.0-svn-93 | org.apache.wiki.ui.migrator.JspParserTest | 0 | 1 | 0.072 |
| 3.0.0-svn-80 | org.apache.wiki.ui.stripes.HandlerInfoTest | 0 | 1 | 4.437 |
| 3.0.0-svn-85 | org.apache.wiki.ui.stripes.HandlerInfoTest | 0 | 1 | 4.384 |
| 3.0.0-svn-93 | org.apache.wiki.ui.stripes.HandlerInfoTest | 0 | 1 | 4.135 |
| 2.8.2-svn-13 | org.apache.wiki.WikiEngineTest | 0 | 1 | 20.025 |
| 3.0.0-svn-17 | org.apache.wiki.WikiEngineTest | 0 | 1 | 37.644 |
| 3.0.0-svn-71 | org.apache.wiki.WikiEngineTest | 0 | 1 | 35.495 |
| 3.0.0-svn-72 | org.apache.wiki.WikiEngineTest | 0 | 1 | 35.517 |
| 3.0.0-svn-73 | org.apache.wiki.WikiEngineTest | 0 | 1 | 35.863 |
| 3.0.0-svn-76 | org.apache.wiki.WikiEngineTest | 0 | 1 | 35.308 |
| 3.0.0-svn-80 | org.apache.wiki.WikiEngineTest | 5 | 13 | 15.016 |
| 3.0.0-svn-85 | org.apache.wiki.WikiEngineTest | 3 | 12 | 15.350 |
| 3.0.0-svn-93 | org.apache.wiki.WikiEngineTest | 4 | 10 | 14.265 |
| 3.0.0-svn-80 | org.apache.wiki.workflow.ApprovalWorkflowTest | 2 | 0 | 1.621 |
| 3.0.0-svn-85 | org.apache.wiki.workflow.ApprovalWorkflowTest | 0 | 2 | 1.587 |
| 3.0.0-svn-93 | org.apache.wiki.workflow.ApprovalWorkflowTest | 0 | 2 | 1.409 |
| 3.0.0-svn-80 | org.apache.wiki.xmlrpc.RPCHandlerTest | 4 | 0 | 1.447 |
| 3.0.0-svn-85 | org.apache.wiki.xmlrpc.RPCHandlerTest | 0 | 2 | 1.525 |
| 3.0.0-svn-93 | org.apache.wiki.xmlrpc.RPCHandlerTest | 1 | 0 | 1.193 |
| 3.0.0-svn-80 | stress.MassiveRepositoryTest | 1 | 0 | 124.826 |
| 3.0.0-svn-85 | stress.MassiveRepositoryTest | 1 | 0 | 0.391 |
+--------------+------------------------------------------------------------+--------+----------+---------+
91 rows in set (0.02 sec)
}}}
! Number of testsuites/testcases per tag (buildlevel)
{{{
mysql> select tag,count(*) from testcaseresult group by tag order by tag;
+--------------+----------+
| tag | count(*) |
+--------------+----------+
| 2.8.2-svn-13 | 948 |
| 3.0.0-svn-17 | 944 |
| 3.0.0-svn-71 | 1050 |
| 3.0.0-svn-72 | 1055 |
| 3.0.0-svn-73 | 1057 |
| 3.0.0-svn-76 | 1033 |
| 3.0.0-svn-80 | 950 |
+--------------+----------+
7 rows in set (0.22 sec)
}}}
! All testsuites per tag having errors or failures
{{{
mysql> select tag,name,tests,errors,failures from testsuiteresult where errors>0 or failures > 0 group by tag,name order by name,tag;
+--------------+---------------------------------------------------+-------+--------+----------+
| tag | name | tests | errors | failures |
+--------------+---------------------------------------------------+-------+--------+----------+
| 3.0.0-svn-80 | org.apache.wiki.action.DeleteActionBeanTest | 3 | 3 | 0 |
| 3.0.0-svn-80 | org.apache.wiki.action.RenameActionBeanTest | 7 | 0 | 6 |
| 3.0.0-svn-80 | org.apache.wiki.action.ViewActionBeanTest | 3 | 0 | 2 |
| 3.0.0-svn-80 | org.apache.wiki.action.WikiContextFactoryTest | 5 | 0 | 1 |
| 3.0.0-svn-80 | org.apache.wiki.auth.acl.DefaultAclManagerTest | 3 | 1 | 0 |
| 3.0.0-svn-80 | org.apache.wiki.auth.AuthorizationManagerTest | 15 | 1 | 0 |
| 3.0.0-svn-80 | org.apache.wiki.auth.user.XMLUserDatabaseTest | 11 | 0 | 1 |
| 3.0.0-svn-80 | org.apache.wiki.auth.UserManagerTest | 5 | 0 | 1 |
| 3.0.0-svn-71 | org.apache.wiki.content.ContentManagerTest | 4 | 4 | 0 |
| 3.0.0-svn-72 | org.apache.wiki.content.ContentManagerTest | 4 | 4 | 0 |
| 3.0.0-svn-80 | org.apache.wiki.content.PageRenamerTest | 21 | 17 | 3 |
| 2.8.2-svn-13 | org.apache.wiki.dav.AttachmentDavProviderTest | 3 | 0 | 1 |
| 3.0.0-svn-72 | org.apache.wiki.dav.AttachmentDavProviderTest | 3 | 0 | 1 |
| 3.0.0-svn-73 | org.apache.wiki.dav.AttachmentDavProviderTest | 3 | 0 | 1 |
| 3.0.0-svn-80 | org.apache.wiki.parser.JSPWikiMarkupParserTest | 200 | 0 | 27 |
| 3.0.0-svn-80 | org.apache.wiki.plugin.InsertPageTest | 6 | 0 | 2 |
| 3.0.0-svn-80 | org.apache.wiki.plugin.RecentChangesPluginTest | 3 | 2 | 1 |
| 3.0.0-svn-80 | org.apache.wiki.plugin.ReferredPagesPluginTest | 3 | 0 | 2 |
| 3.0.0-svn-80 | org.apache.wiki.plugin.ReferringPagesPluginTest | 8 | 1 | 3 |
| 3.0.0-svn-80 | org.apache.wiki.plugin.TableOfContentsTest | 8 | 0 | 5 |
| 3.0.0-svn-80 | org.apache.wiki.plugin.UndefinedPagesPluginTest | 2 | 0 | 2 |
| 3.0.0-svn-80 | org.apache.wiki.ReferenceManagerTest | 18 | 1 | 1 |
| 3.0.0-svn-80 | org.apache.wiki.render.WysiwygEditingRendererTest | 2 | 0 | 1 |
| 3.0.0-svn-80 | org.apache.wiki.rss.RSSGeneratorTest | 2 | 2 | 0 |
| 3.0.0-svn-80 | org.apache.wiki.search.SearchManagerTest | 6 | 4 | 1 |
| 3.0.0-svn-71 | org.apache.wiki.ui.migrator.BundleMigratorTest | 8 | 1 | 0 |
| 3.0.0-svn-72 | org.apache.wiki.ui.migrator.BundleMigratorTest | 8 | 1 | 0 |
| 3.0.0-svn-73 | org.apache.wiki.ui.migrator.BundleMigratorTest | 8 | 1 | 0 |
| 3.0.0-svn-76 | org.apache.wiki.ui.migrator.BundleMigratorTest | 8 | 1 | 0 |
| 3.0.0-svn-80 | org.apache.wiki.ui.migrator.BundleMigratorTest | 8 | 1 | 0 |
| 3.0.0-svn-80 | org.apache.wiki.ui.stripes.HandlerInfoTest | 4 | 0 | 1 |
| 2.8.2-svn-13 | org.apache.wiki.WikiEngineTest | 44 | 0 | 1 |
| 3.0.0-svn-17 | org.apache.wiki.WikiEngineTest | 44 | 0 | 1 |
| 3.0.0-svn-71 | org.apache.wiki.WikiEngineTest | 44 | 0 | 1 |
| 3.0.0-svn-72 | org.apache.wiki.WikiEngineTest | 44 | 0 | 1 |
| 3.0.0-svn-73 | org.apache.wiki.WikiEngineTest | 44 | 0 | 1 |
| 3.0.0-svn-76 | org.apache.wiki.WikiEngineTest | 44 | 0 | 1 |
| 3.0.0-svn-80 | org.apache.wiki.WikiEngineTest | 34 | 5 | 13 |
| 3.0.0-svn-80 | org.apache.wiki.workflow.ApprovalWorkflowTest | 5 | 2 | 0 |
| 3.0.0-svn-80 | org.apache.wiki.xmlrpc.RPCHandlerTest | 4 | 4 | 0 |
| 3.0.0-svn-80 | stress.MassiveRepositoryTest | 1 | 1 | 0 |
+--------------+---------------------------------------------------+-------+--------+----------+
41 rows in set (0.02 sec)
}}}
! 10 most time taking testsuites
{{{
mysql> select tag,name,tests,time from testsuiteresult order by time desc,name,tag limit 10;
+--------------+------------------------------------------------+-------+--------+
| tag | name | tests | time |
+--------------+------------------------------------------------+-------+--------+
| 3.0.0-svn-17 | stress.MassiveRepositoryTest | 1 | 84.526 |
| 3.0.0-svn-73 | stress.MassiveRepositoryTest | 1 | 81.273 |
| 3.0.0-svn-71 | stress.MassiveRepositoryTest | 1 | 78.442 |
| 3.0.0-svn-72 | stress.MassiveRepositoryTest | 1 | 76.723 |
| 3.0.0-svn-76 | stress.MassiveRepositoryTest | 1 | 75.437 |
| 3.0.0-svn-17 | org.apache.wiki.parser.JSPWikiMarkupParserTest | 205 | 70.827 |
| 2.8.2-svn-13 | stress.MassiveRepositoryTest | 1 | 69.571 |
| 3.0.0-svn-73 | org.apache.wiki.parser.JSPWikiMarkupParserTest | 205 | 68.615 |
| 3.0.0-svn-72 | org.apache.wiki.parser.JSPWikiMarkupParserTest | 205 | 67.173 |
| 3.0.0-svn-76 | org.apache.wiki.parser.JSPWikiMarkupParserTest | 205 | 66.869 |
+--------------+------------------------------------------------+-------+--------+
10 rows in set (0.03 sec)
}}}
! Classes with most testcases
{{{
mysql> select distinct name,tests from testsuiteresult order by tests desc limit 15;
+-----------------------------------------------------------+-------+
| name | tests |
+-----------------------------------------------------------+-------+
| org.apache.wiki.parser.JSPWikiMarkupParserTest | 205 |
| org.apache.wiki.parser.CreoleToJSPWikiTranslatorTest | 83 |
| org.apache.wiki.WikiEngineTest | 44 |
| org.apache.wiki.TextUtilTest | 35 |
| org.apache.wiki.TextUtilTest | 34 |
| org.apache.wiki.workflow.WorkflowTest | 24 |
| org.apache.wiki.content.PageRenamerTest | 21 |
| org.apache.wiki.ReferenceManagerTest | 18 |
| org.apache.wiki.auth.AuthorizationManagerTest | 17 |
| org.apache.wiki.htmltowiki.HtmlStringToWikiTranslatorTest | 17 |
| org.apache.wiki.ReleaseTest | 17 |
| org.apache.wiki.plugin.PluginManagerTest | 16 |
| org.apache.wiki.VariableManagerTest | 16 |
| org.apache.wiki.workflow.SimpleDecisionTest | 16 |
| org.apache.wiki.dav.DavPathTest | 15 |
+-----------------------------------------------------------+-------+
15 rows in set (0.05 sec)
}}}
! 15 most time taking testcase methods
{{{
mysql> select tag,classname,name,time from testcaseresult order by time desc,name,tag limit 15;
+--------------+------------------------------------------+------------------------+--------+
| tag | classname | name | time |
+--------------+------------------------------------------+------------------------+--------+
| 3.0.0-svn-17 | stress.MassiveRepositoryTest | testMassiveRepository1 | 84.308 |
| 3.0.0-svn-73 | stress.MassiveRepositoryTest | testMassiveRepository1 | 81.271 |
| 3.0.0-svn-71 | stress.MassiveRepositoryTest | testMassiveRepository1 | 78.440 |
| 3.0.0-svn-72 | stress.MassiveRepositoryTest | testMassiveRepository1 | 76.721 |
| 3.0.0-svn-76 | stress.MassiveRepositoryTest | testMassiveRepository1 | 75.435 |
| 2.8.2-svn-13 | stress.MassiveRepositoryTest | testMassiveRepository1 | 69.569 |
| 3.0.0-svn-73 | org.apache.wiki.search.SearchManagerTest | testSimpleSearch | 10.724 |
| 3.0.0-svn-17 | org.apache.wiki.search.SearchManagerTest | testSimpleSearch | 10.530 |
| 3.0.0-svn-17 | org.apache.wiki.search.SearchManagerTest | testSimpleSearch2 | 10.474 |
| 3.0.0-svn-72 | org.apache.wiki.search.SearchManagerTest | testSimpleSearch | 10.433 |
| 3.0.0-svn-17 | org.apache.wiki.search.SearchManagerTest | testSimpleSearch3 | 10.396 |
| 3.0.0-svn-71 | org.apache.wiki.search.SearchManagerTest | testSimpleSearch | 10.360 |
| 3.0.0-svn-73 | org.apache.wiki.search.SearchManagerTest | testSimpleSearch2 | 10.349 |
| 3.0.0-svn-76 | org.apache.wiki.search.SearchManagerTest | testSimpleSearch | 10.343 |
| 3.0.0-svn-72 | org.apache.wiki.search.SearchManagerTest | testSimpleSearch2 | 10.340 |
+--------------+------------------------------------------+------------------------+--------+
15 rows in set (0.30 sec)
}}}
! Testsuites that have increased their test time the most between releases
{{{
mysql> select name,(max(time)-min(time))/avg(time)*100 as percdelta,min(time),max(time),avg(time) from testsuiteresult where tag='3.0.0-svn-71' or tag='3.0.0-svn-72' group by name order by 2 desc limit 25;
+-----------------------------------------------------------+----------------+-----------+-----------+-----------+
| name | percdelta | min(time) | max(time) | avg(time) |
+-----------------------------------------------------------+----------------+-----------+-----------+-----------+
| org.apache.wiki.auth.permissions.PagePermissionTest | 76.92307692308 | 0.004 | 0.009 | 0.0065000 |
| org.apache.wiki.dav.DavPathTest | 73.68421052632 | 0.006 | 0.013 | 0.0095000 |
| org.apache.wiki.auth.permissions.WikiPermissionTest | 66.66666666667 | 0.004 | 0.008 | 0.0060000 |
| org.apache.wiki.util.ClassUtilTest | 66.66666666667 | 0.003 | 0.006 | 0.0045000 |
| org.apache.wiki.workflow.SimpleDecisionTest | 60.00000000000 | 0.007 | 0.013 | 0.0100000 |
| org.apache.wiki.parser.CreoleToJSPWikiTranslatorTest | 49.81412639405 | 0.101 | 0.168 | 0.1345000 |
| org.apache.wiki.PageManagerTest | 47.19887955182 | 4.364 | 7.060 | 5.7120000 |
| org.apache.wiki.util.CommentedPropertiesTest | 46.40000000000 | 0.048 | 0.077 | 0.0625000 |
| org.apache.wiki.ui.migrator.JspDocumentTest | 40.00000000000 | 0.002 | 0.003 | 0.0025000 |
| org.apache.wiki.workflow.TaskTest | 40.00000000000 | 0.006 | 0.009 | 0.0075000 |
| org.apache.wiki.ui.migrator.JspParserTest | 38.46153846154 | 0.042 | 0.062 | 0.0520000 |
| org.apache.wiki.auth.permissions.GroupPermissionTest | 35.29411764706 | 0.007 | 0.010 | 0.0085000 |
| org.apache.wiki.auth.authorize.WebContainerAuthorizerTest | 34.02489626556 | 1.400 | 1.974 | 1.6870000 |
| org.apache.wiki.auth.authorize.GroupTest | 32.66381297333 | 2.541 | 3.533 | 3.0370000 |
| org.apache.wiki.util.PriorityListTest | 28.57142857143 | 0.003 | 0.004 | 0.0035000 |
| org.apache.wiki.parser.MarkupParserTest | 28.57142857143 | 0.003 | 0.004 | 0.0035000 |
| org.apache.wiki.content.WikiNameTest | 28.57142857143 | 0.003 | 0.004 | 0.0035000 |
| org.apache.wiki.PropertyReaderTest | 25.00000000000 | 0.014 | 0.018 | 0.0160000 |
| org.apache.wiki.attachment.AttachmentManagerTest | 24.69245065935 | 4.952 | 6.347 | 5.6495000 |
| org.apache.wiki.auth.acl.DefaultAclManagerTest | 24.02159244265 | 0.978 | 1.245 | 1.1115000 |
| org.apache.wiki.FileUtilTest | 23.85964912281 | 0.251 | 0.319 | 0.2850000 |
| org.apache.wiki.util.UtilJ2eeCompatTest | 22.22222222222 | 0.004 | 0.005 | 0.0045000 |
| org.apache.wiki.auth.authorize.JDBCGroupDatabaseTest | 21.84782608696 | 1.639 | 2.041 | 1.8400000 |
| org.apache.wiki.util.CryptoUtilTest | 18.18181818182 | 0.005 | 0.006 | 0.0055000 |
| org.apache.wiki.ReleaseTest | 18.18181818182 | 0.015 | 0.018 | 0.0165000 |
+-----------------------------------------------------------+----------------+-----------+-----------+-----------+
25 rows in set (0.03 sec)
}}}