[Dev] eBIG User Group Meeting and TM5 comments

[Frank Cohen]

I gave the presentation at last night's eBIG user group meeting in  
Oakland, California. This is a Java user group and I felt very much  
at home. The group was very small (about 10 people) and they made up  
for size with the quality of their questions and attention. This was  
the first public demonstration of PushToTest TestMaker Version 5.  
TestMaker 5 got a very good reception from the group.

- They understood the power of being able to build a test on their  
development machines and giving it to a QA person to stage a load test.

- They liked the Data Production Library (DPL) system.

- Two of the attendees said they tried TestMaker 4 and gave up on it  
because of its complexity. Both said they will download TestMaker 5  
and try again.

- They liked the idea of dynamic scripting support but were really  
wanting to implement their tests in Java.

And I took feedback and suggestions on how to further improve the  
feature set.

- Enhanced root-cause analysis using Glassbox, a utility that uses  
JMX to gather information from a running application to do root cause  
analysis. They strongly recommended integrating Glassbox into the  
Monitor system and results charts.

- Logging to a database, they want us to log directly into a database  
table and for us to publish the schema so they can write custom  
reports (using Jasper and others). They also suggested that we  
include the Derby DB in TestMaker. They also pointed me to DB4O.com.

- Better results analysis, two of the attendees were from large  
companies that have budgets for testing tools, so they compared  
TestMaker to Mercury LoadRunner. I heard comments like "The only  
thing I like about LoadRunner is its results analysis, you can slice  
and dice the data in so many ways." I told them about the Performance  
Comparison Utility and they said that would be very helpful.

- Agent-less monitoring, they asked why we would do CPU/Net/Memory  
monitoring with an agent versus just using an SNMP trap. They don't  
want to install agents for monitoring.

- TestNode option to run on JDK 1.5, they have run into problems  
using JAX-WS on JDK 1.6, specifically the XML parsing has  
incompatibilities. They would like an option to run a TestNode with  
JDK 1.5 and only operate Java or Jython tests in this mode. So no  
ScriptEngine support in exchange for running their existing unit  
tests written in Java on a TestNode.

I'll put the above items into http://bugs.pushtotest.com as  
enhancements.

-Frank

--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA governance and test automation tool






--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool