From fcohen at pushtotest.com  Mon Feb  5 13:43:37 2007
From: fcohen at pushtotest.com (Frank Cohen)
Date: Tue Feb  6 16:30:05 2007
Subject: [Dev] Feedback on TestMaker 5 document?
Message-ID: <B635FBA3-3CAA-446A-ACB2-B205C781D884@pushtotest.com>

Is there any additional feedback on the TestMaker 5 completion  
specification document? We had the kickoff meeting today and are  
working towards an April release. -Frank

--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool



From fcohen at pushtotest.com  Sun Feb 11 22:46:13 2007
From: fcohen at pushtotest.com (Frank Cohen)
Date: Sun Feb 11 23:55:57 2007
Subject: [Dev] TestMaker 5 repository
Message-ID: <C59110CD-0735-428D-A275-CC6607DF3EF9@pushtotest.com>

The TestMaker 5 project is underway. We're tracking the schedule and  
milestones at:

http://lists.pushtotest.com/mailman/listinfo/dev

We will update this schedule approximately once a week.

We also set-up a new Web page to summarize the progress at:

http://www.pushtotest.com/Docs/testmakerdev

Also, we will be using this email list for developer-to-developer  
conversations about the TestMaker 5 project.

Please jump in!

Thanks.

-Frank


--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool



From fcohen at pushtotest.com  Mon Feb 12 10:28:51 2007
From: fcohen at pushtotest.com (Frank Cohen)
Date: Mon Feb 12 11:38:53 2007
Subject: [Dev] Introducing William and Avantica
Message-ID: <D0B43C6E-DE40-418A-8630-44D3F8B5BCE7@pushtotest.com>

Hi TestMaker developers: Let me introduce you to William Martinez  
Pomares of Group Avantica. PushToTest hired Avantica to code  
TestMaker 5. This will be Avantica's 8th project with PushToTest and  
I very much recommend them as an excellent engineering group.

William and I will be using this email list to communicate on  
TestMaker 5 development issues. Feel free to jump in with your ideas,  
feedback and criticisms. You are welcome to contribute code, patches,  
and tests. Thanks.

-Frank

--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool



From fcohen at pushtotest.com  Mon Feb 12 10:31:44 2007
From: fcohen at pushtotest.com (Frank Cohen)
Date: Mon Feb 12 11:41:37 2007
Subject: [Dev] <tally> and logging
Message-ID: <7EEF68AF-A029-4AFD-9B36-863978969435@pushtotest.com>

Hi William: I'd like to make certain that the logging in TM5 is well  
defined. Would you please confirm the following:

Users create functional, scalability/performance, and reliability  
(service monitor) tests using TestMaker. All types of tests begin as  
a JUnit TestCase with setup, runtest, and teardown methods. Users  
write a TestCase in Java or Jython. (In releases later than TM 5.0  
users will also create a TestCase in JSR 223 scripting compliant  
languages, including Ruby, Groovy, PHP, etc.) The TestCases are  
wrapped in an XSTest wrapper (defined in a TestScenario.xml file,) a  
ServiceMonitor wrapper (coming in TestMaker 5.5,) or are called from  
a Jython script.

1) Users run a TestScenario.xml defined scalability test from:
    a) The com.pushtotest.testmaker.gui.guiTestMaker GUI interface
    b) The command line using the -commandline switch in the  
TestMaker.sh and .bat start-up scripts.

When the TestScenario runs there are three things to log:
i) system.out messages that happen while the test runs, including  
stack traces and exceptions

ii) Transactional data as the test runs, including each transaction  
entry and the system monitor entries. The amount of information in  
the transaction entries depends on the log level setting from the  
TestScenario.xml file and is defined as:

Log level 0 = No logging
1 = Information about major operations, including time to perform  
transaction (testusecase)
2 = Details about each operation, for instance, GET /examples, POST / 
examples
3 = Debugging information, including the request/response message  
content
4 = Ludicrous level, meaning everything and the kitchen sink
They are additive, so logging at level 3 gives debugging, details,  
and information.

iii) Results analysis, including the comma-separated-value tallied  
TPS values, aka the Scalability Index

2) Users run a Jython script from the TestMaker script editor (J  
editor, org.armedbear.j.Editor) using  
com.pushtotest.testmaker.gui.ScriptEditor.TMFrame.

Jython scripts log their output to the Output panel in the TestMaker  
script editor window as they do in TM 4.x

Please send me your comments or confirm that this is your understanding.

Thanks.

-Frank

--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool



From wmartinez at AVANTICATEC.NET  Mon Feb 12 11:13:02 2007
From: wmartinez at AVANTICATEC.NET (=?US-ASCII?Q?William_Martinez_Pomares?=)
Date: Mon Feb 12 12:22:05 2007
Subject: [Dev] <tally> and logging
In-Reply-To: <7EEF68AF-A029-4AFD-9B36-863978969435@pushtotest.com>
Message-ID: <85A61B0FDDD4834EA295605C349BABAA0AFD14@dc1avantica.AVANTICA.AVANTICATEC.NET>

Hi.
First, we need to understand who logs and when, and actually the what.
Right now, there are five kinds of logs:
1. The system.out log, that has all text that is output to console.
2. The result log, old transactional log format for XSTest from versions
4 and lower.
3. The TXLog, New transactional log from 4.4 and on. This is an XML
format that includes tags for each scenario and each individual
transaction. The content is based on timestamp data, duration of the
transaction and result (true/false and any text that comes with it).
4. The SDLog, New System Data log that contains information for CPU,
Memory and Networkd performance during the test.
5. The Summary, New log that contains the tallied information from the
TXLog, consolidating all nodes and scenarios, plus system data.

TestScenario.xml is the script to run a test suite. There is a class
that will take that XML and upon reading the scenarios definitions, will
start to run the actual unit tests. This class is the one in a very good
position to record the TXLog, since it is starting and ending the
transactions (in fact,the is another class that should do that,
atomically, but under the control of the runner class).

The System Monitor is a service that runs in the background, taking
system readings and saving them into XML. This is in charge of SDLog

The Tally class is the one in charge of reading both logs and
consolidate them into Summary.

Now, for legacy XSTest systems where the control program is written in
jython (master.py), or any other case where the TestScenario is not
used, the jython code should contain lines to save those log records.
The TOOL contains an xml package where JaxB classes were created to help
generate or consume those logs.

Now, the new requirement:
To expand TXLog so support optional children tags for transaction, that
include details about the operation, Debuggin info or any other info
further registered. 

Note that data for level 2-4 is not known byt the runner classes, but
for the actual unit agents. That means, there should be a way for the
unit agents, in any scrip language, to pass that information to the
runner so it can log it into the transaction tag. This to be able to
modify TXLog format without requiring to reimplement the agents.

Please let me know if this makes sense.

William.

-----Original Message-----
From: dev-bounces@lists.pushtotest.com
[mailto:dev-bounces@lists.pushtotest.com]On Behalf Of Frank Cohen
Sent: Monday, February 12, 2007 12:32 PM
To: Load and TestMaker Developers List
Subject: [Dev] <tally> and logging


Hi William: I'd like to make certain that the logging in TM5 is well  
defined. Would you please confirm the following:

Users create functional, scalability/performance, and reliability  
(service monitor) tests using TestMaker. All types of tests begin as  
a JUnit TestCase with setup, runtest, and teardown methods. Users  
write a TestCase in Java or Jython. (In releases later than TM 5.0  
users will also create a TestCase in JSR 223 scripting compliant  
languages, including Ruby, Groovy, PHP, etc.) The TestCases are  
wrapped in an XSTest wrapper (defined in a TestScenario.xml file,) a  
ServiceMonitor wrapper (coming in TestMaker 5.5,) or are called from  
a Jython script.

1) Users run a TestScenario.xml defined scalability test from:
    a) The com.pushtotest.testmaker.gui.guiTestMaker GUI interface
    b) The command line using the -commandline switch in the  
TestMaker.sh and .bat start-up scripts.

When the TestScenario runs there are three things to log:
i) system.out messages that happen while the test runs, including  
stack traces and exceptions

ii) Transactional data as the test runs, including each transaction  
entry and the system monitor entries. The amount of information in  
the transaction entries depends on the log level setting from the  
TestScenario.xml file and is defined as:

Log level 0 = No logging
1 = Information about major operations, including time to perform  
transaction (testusecase)
2 = Details about each operation, for instance, GET /examples, POST / 
examples
3 = Debugging information, including the request/response message  
content
4 = Ludicrous level, meaning everything and the kitchen sink
They are additive, so logging at level 3 gives debugging, details,  
and information.

iii) Results analysis, including the comma-separated-value tallied  
TPS values, aka the Scalability Index

2) Users run a Jython script from the TestMaker script editor (J  
editor, org.armedbear.j.Editor) using  
com.pushtotest.testmaker.gui.ScriptEditor.TMFrame.

Jython scripts log their output to the Output panel in the TestMaker  
script editor window as they do in TM 4.x

Please send me your comments or confirm that this is your understanding.

Thanks.

-Frank

--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool



_______________________________________________
Dev mailing list
Dev@lists.pushtotest.com
http://lists.pushtotest.com/mailman/listinfo/dev
From fcohen at pushtotest.com  Mon Feb 12 11:37:13 2007
From: fcohen at pushtotest.com (Frank Cohen)
Date: Mon Feb 12 12:47:11 2007
Subject: [Dev] <tally> and logging
In-Reply-To: <85A61B0FDDD4834EA295605C349BABAA0AFD14@dc1avantica.AVANTICA.AVANTICATEC.NET>
References: <85A61B0FDDD4834EA295605C349BABAA0AFD14@dc1avantica.AVANTICA.AVANTICATEC.NET>
Message-ID: <2E6428E8-4ABD-48E0-AD5F-DFADF741D59E@pushtotest.com>

Thanks, see below. -Frank


> Right now, there are five kinds of logs:
> 1. The system.out log, that has all text that is output to console.
> 2. The result log, old transactional log format for XSTest from  
> versions
> 4 and lower.
> 3. The TXLog, New transactional log from 4.4 and on. This is an XML
> format that includes tags for each scenario and each individual
> transaction. The content is based on timestamp data, duration of the
> transaction and result (true/false and any text that comes with it).
> 4. The SDLog, New System Data log that contains information for CPU,
> Memory and Networkd performance during the test.
> 5. The Summary, New log that contains the tallied information from the
> TXLog, consolidating all nodes and scenarios, plus system data.

That's right. Thanks for the summary.


> TestScenario.xml is the script to run a test suite. There is a class
> that will take that XML and upon reading the scenarios definitions,  
> will
> start to run the actual unit tests. This class is the one in a very  
> good
> position to record the TXLog, since it is starting and ending the
> transactions (in fact,the is another class that should do that,
> atomically, but under the control of the runner class).
>
> The System Monitor is a service that runs in the background, taking
> system readings and saving them into XML. This is in charge of SDLog
>
> The Tally class is the one in charge of reading both logs and
> consolidate them into Summary.
>
> Now, for legacy XSTest systems where the control program is written in
> jython (master.py), or any other case where the TestScenario is not
> used, the jython code should contain lines to save those log records.
> The TOOL contains an xml package where JaxB classes were created to  
> help
> generate or consume those logs.
>
> Now, the new requirement:
> To expand TXLog so support optional children tags for transaction,  
> that
> include details about the operation, Debuggin info or any other info
> further registered.
>
> Note that data for level 2-4 is not known byt the runner classes, but
> for the actual unit agents. That means, there should be a way for the
> unit agents, in any scrip language, to pass that information to the
> runner so it can log it into the transaction tag. This to be able to
> modify TXLog format without requiring to reimplement the agents.
>

In TM 4.4 the master.py script instantiates threads to run the  
TestCase over and over again for the duration of the test case.  
Master.py passes in parameters to the TestCase constructor. So, for  
TM 5 pass the log level from the TestScenario.xml file into the  
TestCase. The TestCase uses a base class (formerly agentbase.py but  
now a Java class like agentbase.java) to do the GET/POST commands and  
it automatically logs. The TM5 runner passes in the log level  
information which gets passed to the base class and the runner calls  
the base class to do the logging for each runtest call. That keeps  
from having to reimplement the agents and lets the user change the  
logging level in the TestScenario.xml file.

What do you think?

-Frank





> Please let me know if this makes sense.
>
> William.
>
> -----Original Message-----
> From: dev-bounces@lists.pushtotest.com
> [mailto:dev-bounces@lists.pushtotest.com]On Behalf Of Frank Cohen
> Sent: Monday, February 12, 2007 12:32 PM
> To: Load and TestMaker Developers List
> Subject: [Dev] <tally> and logging
>
>
> Hi William: I'd like to make certain that the logging in TM5 is well
> defined. Would you please confirm the following:
>
> Users create functional, scalability/performance, and reliability
> (service monitor) tests using TestMaker. All types of tests begin as
> a JUnit TestCase with setup, runtest, and teardown methods. Users
> write a TestCase in Java or Jython. (In releases later than TM 5.0
> users will also create a TestCase in JSR 223 scripting compliant
> languages, including Ruby, Groovy, PHP, etc.) The TestCases are
> wrapped in an XSTest wrapper (defined in a TestScenario.xml file,) a
> ServiceMonitor wrapper (coming in TestMaker 5.5,) or are called from
> a Jython script.
>
> 1) Users run a TestScenario.xml defined scalability test from:
>     a) The com.pushtotest.testmaker.gui.guiTestMaker GUI interface
>     b) The command line using the -commandline switch in the
> TestMaker.sh and .bat start-up scripts.
>
> When the TestScenario runs there are three things to log:
> i) system.out messages that happen while the test runs, including
> stack traces and exceptions
>
> ii) Transactional data as the test runs, including each transaction
> entry and the system monitor entries. The amount of information in
> the transaction entries depends on the log level setting from the
> TestScenario.xml file and is defined as:
>
> Log level 0 = No logging
> 1 = Information about major operations, including time to perform
> transaction (testusecase)
> 2 = Details about each operation, for instance, GET /examples, POST /
> examples
> 3 = Debugging information, including the request/response message
> content
> 4 = Ludicrous level, meaning everything and the kitchen sink
> They are additive, so logging at level 3 gives debugging, details,
> and information.
>
> iii) Results analysis, including the comma-separated-value tallied
> TPS values, aka the Scalability Index
>
> 2) Users run a Jython script from the TestMaker script editor (J
> editor, org.armedbear.j.Editor) using
> com.pushtotest.testmaker.gui.ScriptEditor.TMFrame.
>
> Jython scripts log their output to the Output panel in the TestMaker
> script editor window as they do in TM 4.x
>
> Please send me your comments or confirm that this is your  
> understanding.
>
> Thanks.
>
> -Frank
>
> --
> Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
> TestMaker: The open-source SOA test automation tool
>
>
>
> _______________________________________________
> Dev mailing list
> Dev@lists.pushtotest.com
> http://lists.pushtotest.com/mailman/listinfo/dev
> _______________________________________________
> Dev mailing list
> Dev@lists.pushtotest.com
> http://lists.pushtotest.com/mailman/listinfo/dev
>



--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool



From wmartinez at AVANTICATEC.NET  Mon Feb 12 14:25:50 2007
From: wmartinez at AVANTICATEC.NET (=?US-ASCII?Q?William_Martinez_Pomares?=)
Date: Mon Feb 12 15:34:52 2007
Subject: [Dev] JSR223 and Jython Version
In-Reply-To: <2E6428E8-4ABD-48E0-AD5F-DFADF741D59E@pushtotest.com>
Message-ID: <85A61B0FDDD4834EA295605C349BABAA0AFE62@dc1avantica.AVANTICA.AVANTICATEC.NET>

Hello.
Playing aruond with JSR 223 implementation, the Jython library seems to
be built using Java 1.6. 
That brings the question of wich is the JVM version for TM5. Would it be
1.5 or 6?
If the target is 1.5, then we would need to rebuild the Jython library,
thus we will need the source.  

William Martinez Pomares
Architect
Avantica Technologies
Phone(US)  :+1 (650) 353-4522  Ext 131
Fax(US)        :+1 (877) 372-1955  
Phone(CR)  :+506 283-9100 Ext 131 
Fax(CR)        :+506 253-7451 
http://www.avantica.net



From fcohen at pushtotest.com  Mon Feb 12 14:32:14 2007
From: fcohen at pushtotest.com (Frank Cohen)
Date: Mon Feb 12 15:42:15 2007
Subject: [Dev] JSR223 and Jython Version
In-Reply-To: <85A61B0FDDD4834EA295605C349BABAA0AFE62@dc1avantica.AVANTICA.AVANTICATEC.NET>
References: <85A61B0FDDD4834EA295605C349BABAA0AFE62@dc1avantica.AVANTICA.AVANTICATEC.NET>
Message-ID: <40654133-E168-42A9-B0DD-1E8026E2BFDE@pushtotest.com>

I've been wanting to bundle the JRE with TestMaker to solve problems  
people encounter with installation. If we take that step then the  
download site would have to offer Windows, Linux, and Mac OS X  
specific downloads. How much of an effort is that to create these 3  
installers?

If the effort is small then I would say lets use Java 1.6 and call  
Jython through JSR 223. Otherwise, lets put off the 223 until  
TestMaker 5.5.

-Frank



On Feb 12, 2007, at 2:25 PM, William Martinez Pomares wrote:

> Hello.
> Playing aruond with JSR 223 implementation, the Jython library  
> seems to
> be built using Java 1.6.
> That brings the question of wich is the JVM version for TM5. Would  
> it be
> 1.5 or 6?
> If the target is 1.5, then we would need to rebuild the Jython  
> library,
> thus we will need the source.
>
> William Martinez Pomares
> Architect
> Avantica Technologies
> Phone(US)  :+1 (650) 353-4522  Ext 131
> Fax(US)        :+1 (877) 372-1955
> Phone(CR)  :+506 283-9100 Ext 131
> Fax(CR)        :+506 253-7451
> http://www.avantica.net
>
>
>
> _______________________________________________
> Dev mailing list
> Dev@lists.pushtotest.com
> http://lists.pushtotest.com/mailman/listinfo/dev
>



--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool



From fcohen at pushtotest.com  Thu Feb 15 15:08:18 2007
From: fcohen at pushtotest.com (Frank Cohen)
Date: Thu Feb 15 16:18:58 2007
Subject: [Dev] TestScenario  Run Levels
References: <880185DF-6666-4188-9D0B-F9F06AC6F2CB@pushtotest.com>
Message-ID: <C6806D05-1753-4449-910A-D911941C0350@pushtotest.com>

> From: Frank Cohen <frank@pushtotest.com>
> Date: February 15, 2007 3:06:50 PM PST
> To: William Martinez Pomares <wmartinez@AVANTICATEC.NET>
> Subject: Re: TestScenario  Run Levels
>
> By console, I hope you mean com.pushtotest.testmaker.controller.  
> There you will find guiTestRunPanel.java which provides the GUI for  
> a running test. The sliderStateChanged method uses the  
> fileTestScenario getTestCaseName() method to identify the test  
> case. Moving the slider is supposed to let you move through ANY of  
> the permutations of the test scenario. Luis Carlos has it right in  
> the heirarchy of the levels, so sliding left to right you got  
> through the 1st TestUseCase + 1st CR Level + 1st MessageSize + 1st  
> UseCase, slide to the next position to the right and you get the  
> 1st TestUseCase + 1st CR Level + 1st MessageSize + 2nd Use Case.  
> And so on.
>
> This might be crazy in the long run - too much granularity - but we  
> can reduce the slider granularity later after some user testing.
>
> -Frank
>
>
>
>
>
>
> On Feb 15, 2007, at 11:32 AM, William Martinez Pomares wrote:
>
>> Hi Frank.
>> This is a question from Luis Carlos,
>> Basically, from the console we can control, using a slider, which  
>> test
>> is running in the nodes. Then, which will be the level in the slider?
>> Should we control complete TestUseCases, or the group of  
>> usercases, or
>> individual usecases, or even individual cycles like crlevel/payload?
>>
>> William
>>
>> -----Original Message-----
>> From: Luis Carlos Lara Lopez
>> Sent: Thursday, February 15, 2007 12:52 PM
>> To: William Martinez Pomares
>> Subject: TestScenario
>>
>>
>> I have made a simplified schema of the running of a test scenario. It
>> appears that we have 6 levels of nesting.
>>
>> First I want to ask you if the schema is correct.
>>
>> Second, on which level of nesting the user have the control
>> First level - User just run a test scenario.
>> Second level - User can choose one test use case for a test
>> Third level  - User can choose one concurrent request level for a  
>> test
>> Fourth level - User can choose one message size level for a test
>> Fifth level - User can choose one use case for a level
>> Sixth level - The user can choose on sequence for a test
>> 	
>> 	Of course if you are one level you are in all the levels before
>>
>> 	Get the TestScenario
>> 	For each TestUseCase in the TestScenario
>> 		Get the Crlevels
>> 		Get the Messagesizes
>> 		Get the Usecases
>> 		For each ConcurrentRequestLevel in the Crlevels
>> 			For each MessageSize in the Messagesizes
>> 				For each Usecase in the Usecases
>> 					Run SetUp if exists
>> 					Start Timer
>> 					For each Sequence in the Usecase
>> 						Start the threads
>> 						For each Run on the
>> Sequence
>> 							Call the method
>>
>> 						Stop Timer
>> 					Run TearDown if exists
>
>
>
> --
> Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
> TestMaker: The open-source SOA test automation tool
>
>
>
>



--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool



From fcohen at pushtotest.com  Thu Feb 15 15:20:54 2007
From: fcohen at pushtotest.com (Frank Cohen)
Date: Thu Feb 15 16:31:40 2007
Subject: [Dev] Re: TestMaker Accessories specification
In-Reply-To: <85A61B0FDDD4834EA295605C349BABAA0B093F@dc1avantica.AVANTICA.AVANTICATEC.NET>
References: <85A61B0FDDD4834EA295605C349BABAA0B093F@dc1avantica.AVANTICA.AVANTICATEC.NET>
Message-ID: <135EE3CE-5781-453A-8890-396AF0AF0BF3@pushtotest.com>

Hi William: Good questions. See below. -Frank

On Feb 15, 2007, at 12:07 PM, William Martinez Pomares wrote:

> Hello Frank.
>
> We are reviewing the specs. I have a couple of questions:
> 1. For the XML Schemas drivers, should we understand as a driver  
> the set
> of libraries you mention (DPL,DVL and the context check)? Should it be
> like a factory, where I write base libraries and then simply add the
> specific classes that the factory will instantiate (STAR, CDISC)?

Yes. The DPL is straight forward from our past projects for GM, BEA,  
Sun, and others. The DVL is a validation check so the API should have  
a few methods to validate a document. The context check is a little  
more complex in that it needs to provide methods to create notes,  
walk through an XML document in the schema's context, and insert/ 
update/delete. This is kind of an XML specific version of JDOM, but  
much more focused, and consequently smaller.

>
> 2. In the databaset and applications drivers: The normal methods for
> databases are easy to define, but applications are not. Which would be
> the things to test on an application server? Could they be different
> from one app server to the other?

Let's try for the most common app functions first, then we will add  
more functions that are specific to the application. For instance, I  
think that all of the applications will have:

log-in
log-out
create record
delete record
update record
search for records
select one record

> 3. The Industry Specific Test Kits spec is not clear. It seems we  
> should
> not do anyting in here, since it only asks for the test kit to be
> possible using the drivers. Cuold you expand the idea?

The test kits are a combination of drivers, plus the business logic  
defined in a use case. For example, if I was going to test a CDISC- 
based system for tracking clinical trials then I would write a Jython  
script that creates a patient record, inserts notice of a visit to a  
clinic, updates the patient's phone numbers, and then exists the  
application. The script would be very light weight with calls to the  
CDISC driver to interact with the clinical trial db. You should break  
down the project into two phases: 1) drivers, and 2) creating the  
kits. The experience of doing 1 will let me define the spec for 2.

>
> 4. The project contains then the installer and management console for
> the drivers, right?


Yes. The installer and console are for TestMaker. It's assumed that  
the customer already has the applications, dbs, etc.

>
> 5. Since it seems quite a few number of drivers, could we talk of an
> ongoing project where the drivers are created on demand? That is, to
> create the few more requested ones to make a first delivery and be  
> able
> to put those available, and then start working on the others and so.
>

Yes, exactly! This is a formalization of the one-by-one process we've  
been using for GM, BEA, Tibco, etc.


> Thanks.
>
> William.
>
> -----Original Message-----
> From: Frank Cohen [mailto:frank@pushtotest.com]
> Sent: Wednesday, February 14, 2007 5:56 PM
> To: William Martinez Pomares
> Cc: Mario Chavez
> Subject: TestMaker Accessories specification
>
>
> Hi William:
>
> Attached is a specification for the TestMaker Accessories.
> Accessories are the drivers for XML schemas, applications, and
> databases. I'm sending this to you so you will be able to come up
> with a good and general design for the DPL in TM5.
>
> Please forward this to the Avantica management as a request for
> proposal.
>
> Thanks.
>
> -Frank
>
> --
> Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
> TestMaker: The open-source SOA test automation tool
>
>
>



--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool



From fcohen at pushtotest.com  Thu Feb 15 15:30:15 2007
From: fcohen at pushtotest.com (Frank Cohen)
Date: Thu Feb 15 16:40:54 2007
Subject: [Dev] XML change petition
References: <60A9FD56-C27F-4626-9C74-E5BF5A9A612D@pushtotest.com>
Message-ID: <44877CA2-88F9-47F6-86E1-CDE44F153282@pushtotest.com>

> I'm ok with making <sequence> mandatory.
>
> TestMaker community, now is your time to speak up to this change  
> request.
>
> -Frank
>
>
> On Feb 15, 2007, at 1:50 PM, William Martinez Pomares wrote:
>
>> What do you think of this reduction?  "Can we make the sequence tag
>> mandatory?"
>>
>> William.
>> -----Original Message-----
>> From: Luis Carlos Lara Lopez
>> Sent: Thursday, February 15, 2007 3:24 PM
>> To: William Martinez Pomares
>> Subject: XML change petition
>>
>>
>>
>> XML change petition
>>
>> In some xml you use test between sequence tags
>>      <usecase name="sequencetest">
>>          <setup name="geo_insert" testclass="com.geo"  
>> method="geosetup"
>> langtype="java">
>>              <argument name="geoval" value="1738"/>
>>          </setup>
>>          <sequence name="inserts" proportion="80">
>>              <test name="test1">
>>                  <setup name="geo_insert" testclass="com.geo"
>> method="geosetup" langtype="java">
>>           <argument name="geoval" value="1738"/>
>>                  </setup>
>>                  <run name="geo_insert" testclass="" method=""
>> langtype="java">
>>           <argument name="geoval" value="1738"/>
>>                  </run>
>>                  <teardown name="geo_insert" testclass="com.geo"
>> method="geoteardown" langtype="java"/>
>>              </test>
>>          </sequence>
>>          <sequence name="queries" proportion="20">
>>              <test>
>>                  <setup name="geo_insert" testclass="com.geo"
>> method="geosetup" langtype="java">
>>           <argument name="geoval" value="1738"/>
>>                  </setup>
>>                  <run name="geo_insert" testclass="" method=""
>> langtype="java">
>>           <argument name="geoval" value="1738"/>
>>                  </run>
>>                  <teardown name="geo_insert" testclass="com.geo"
>> method="geoteardown" langtype="java"/>
>>              </test>
>>          </sequence>
>>          <teardown name="geo_insert" testclass="com.geo"
>> method="geoteardown" langtype="java"/>
>>      </usecase>
>>
>> But In some xml you use test without sequence tags
>>      <usecase name="simpletest">
>>          <setup name="geo_insert" testclass="com.geo"  
>> method="geosetup"
>> langtype="java"/>
>>          <test>
>>              <run name="geo_insert" testclass="frankclass"
>> method="frankmethod" langtype="java"/>
>>          </test>
>>          <teardown name="geo_insert" testclass="com.geo"
>> method="geoteardown" langtype="java"/>
>>      </usecase>
>>
>> Can we make the sequence tag mandatory; it will simplify design and
>> code.
>> Then the example below will change into
>>
>>      <usecase name="simpletest">
>>         <sequence>
>>            <setup name="geo_insert" testclass="com.geo"
>> method="geosetup" langtype="java"/>
>>            <test>
>>                <run name="geo_insert" testclass="frankclass"
>> method="frankmethod" langtype="java"/>
>>            </test>
>>            <teardown name="geo_insert" testclass="com.geo"
>> method="geoteardown" langtype="java"/>
>>         <sequence>
>>      </usecase>
>
>
>
> --
> Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
> TestMaker: The open-source SOA test automation tool
>
>
>
>



--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool



From FCOHEN at PUSHTOTEST.COM  Thu Feb 15 15:42:34 2007
From: FCOHEN at PUSHTOTEST.COM (Frank Cohen)
Date: Thu Feb 15 16:53:17 2007
Subject: [Dev] Re: Coding and building?
In-Reply-To: <85A61B0FDDD4834EA295605C349BABAA0B0880@dc1avantica.AVANTICA.AVANTICATEC.NET>
References: <85A61B0FDDD4834EA295605C349BABAA0B0880@dc1avantica.AVANTICA.AVANTICATEC.NET>
Message-ID: <ACB7C23E-24CB-4C3B-AE7F-0B98664B718A@PUSHTOTEST.COM>

Hi William: See my answers below. -Frank



On Feb 15, 2007, at 9:40 AM, William Martinez Pomares wrote:

> Basically, we made it build, but the TNMaster, TNTally and all the  
> other
> migrated jython files to Java presented several problems. From missing
> imports to definition of methods that were returning complex object
> (i.e. hashtables) that are now returning ints (thus, it is not
> compiling). Some parts we commented out to make it build and start
> working on the TestScenario, and we will revisit that part to  
> completely
> fix it and make that migration work.

Yes, it is a testament for how bad of a coder I am! I have some  
atoning to do! :-o


> We also notices that the frame to
> display was replaced by the controller gui class. Is that class  
> going to
> maintain its name as test?

Yes, the new gui for a running TestScenario is in  
com.pushtotest.testmaker.controller.guiTestRunPanel

> By Now, we have an XSD that represents the TestScenario format, and a
> class to parse it and return the actual lists we need, we are now
> creating the class that "runs" the TestScenario.xml.

Great! I'm looking forward to it.

> Here I have a
> couple of questions:
>
> 1. You mention that the console would be "sending" commands to the  
> nodes
> to execute things. I was thinking of letting the Node to run the  
> XML by
> itself, with syncronization from the console. That way, we avoid
> complexity in the protocol by eliminating the need to send the command
> and all the parameters for the node to run an specific test case. We
> just send the XML and then the node has all info it needs. What do you
> think?

Yes, that's good.

> 2. As I mentioned before, to run 223 the script jars are using JVM  
> 6. We
> can either try to get the code and build it using JVM 5 or migrate TM5
> to use JVM 6. We should then, by your suggestion, to embbed JRE 6  
> in the
> TM5 distribution. An installer?

How many days will it take to write a TestMaker installer that works  
on Windows XP, Linux (Redhat and Ubuntu,) and Mac OS X 10.4? If this  
is a week long project then I'm inclined to wait for the TM 5.1 release.

-Frank


>
> William.
>
> -----Original Message-----
> From: Frank Cohen [mailto:FCOHEN@PUSHTOTEST.COM]
> Sent: Wednesday, February 14, 2007 12:42 PM
> To: William Martinez Pomares
> Subject: Coding and building?
>
>
> How is the coding going? When will TestMaker build again? -Frank
>
>
> --
> Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
> TestMaker: The open-source SOA test automation tool
>
>
>



--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool



From wmartinez at AVANTICATEC.NET  Tue Feb 20 08:03:19 2007
From: wmartinez at AVANTICATEC.NET (=?US-ASCII?Q?William_Martinez_Pomares?=)
Date: Tue Feb 20 09:14:13 2007
Subject: [Dev] Questions on the Design and XML
Message-ID: <85A61B0FDDD4834EA295605C349BABAA0C79B1@dc1avantica.AVANTICA.AVANTICATEC.NET>


Hi Frank.

Yes, sorry about that :) Spanish : Proyecto. I still need to proof
reading english with Luis.
Luis is working and tuning up the design, but he is posting it to CVS
still rough, so you may be reading interim versions. 
We do have some questions in a discussion held yesterday. Basically:

1. Global Arguments. In the TestScenario we have global arguments. These
tend to be very confusing to the user, since the Setup, Run and TearDown
elements may receive arguments. Thus, we are not sure which ones will
receive the parameters and which ones won't. One idea would be to
declare not global arguments but properties (like in version 3 of your
XML: ant like) and the ones to require one of those values simply refer
to them.

2. MessageSizes. These are the Payload Sizes, which are actually indexes
to a table that is not defined anywhere. It is assumed that the
user-written test methods do handle that index. So, we will need to
require the methods to accept that index, although not specified as an
argument anywhere. Or, to require the user to specify the argument
explicitly, having the argument an attribute to indicate it is not a
value but the payload size.

3. TargetHost. In TM 4.4, properties file, we can specify the target
hosts, they URLs and connection strings. Here we do not. That means that
to test against two different servers, I have to create two different
XMLs and each one will have to use different jars and those jars are the
ones that "know" where the host is?  Or should we add an element with
the host information and pass it as a names argument (just like the
payload above)? 

4. Multiple TestScenarios. Luis is designing thinking that there would
be multiple TestUseCases, each one containing several usecases. Is that
the case, or do you think (as I do) that the testScenario XML holds only
one TestUseCase with multiple usecases?

5. UseCases. Actually, we can have several use cases. In the questions
Luis sent, he explained the hierarchy based on the actual disposition in
the XML. That is, the outer cycle would be CRLevels, then Payload sizes
and then UseCases. You told him it was ok. But that doesn't make sense.
The actual order in TM4.4 is outer cycle UseCases, the CRLevels and then
PayloadSizes. That is because you want to see the behavior of the server
doing updates in several crlevel/payload combinations, not the behavior
of the server with X users in several combinations of payloads and
usecases. Right?

6. Instances. Lastly, in 223 you can create an interpreter and send the
code in a string to execute. But that I'm not sure if supports data from
one call to be accesible on the other one. Thus, calling a Setup may
update some variables and create data, but the call to Run may not have
access to it. Rememeber it is not we create an instance of the class and
call several methods from it, not sure if it actually creates an
instance of the class in every call (the 223). What do you think?

William Martinez Pomares
Architect
Avantica Technologies
Phone(US)  :+1 (650) 353-4522  Ext 131
Fax(US)        :+1 (877) 372-1955  
Phone(CR)  :+506 283-9100 Ext 131 
Fax(CR)        :+506 253-7451 
http://www.avantica.net



-----Original Message-----
From: Frank Cohen [mailto:fcohen@pushtotest.com]
Sent: Monday, February 19, 2007 10:38 PM
To: William Martinez Pomares
Subject: Project Design doc


Hi William: I read through your comments in the ProyectDesign.doc.  
I'll give it a closer read in the morning.

By the way, there is a typo in the name of the document. The 'y'  
should be a 'j' in ProyectDesign.doc.

-Frank

--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool


From fcohen at pushtotest.com  Tue Feb 20 08:48:42 2007
From: fcohen at pushtotest.com (Frank Cohen)
Date: Tue Feb 20 10:00:38 2007
Subject: [Dev] Questions on the Design and XML
In-Reply-To: <85A61B0FDDD4834EA295605C349BABAA0C79B1@dc1avantica.AVANTICA.AVANTICATEC.NET>
References: <85A61B0FDDD4834EA295605C349BABAA0C79B1@dc1avantica.AVANTICA.AVANTICATEC.NET>
Message-ID: <BD9136FF-2CB7-4944-B844-7C9AA664F0F4@pushtotest.com>

Hi William: See answers below. Thanks. -Frank


> 1. Global Arguments. In the TestScenario we have global arguments.  
> These
> tend to be very confusing to the user, since the Setup, Run and  
> TearDown
> elements may receive arguments. Thus, we are not sure which ones will
> receive the parameters and which ones won't. One idea would be to
> declare not global arguments but properties (like in version 3 of your
> XML: ant like) and the ones to require one of those values simply  
> refer
> to them.

I'm not 100% certain how the TestScenario.xml will work in the  
finished TM 5 project. I expect to do some usability testing with  
real users to see what is apparent to them and what is confusing. For  
now, we need to have both global and local arguments. (And not  
properties files.)

Global arguments are defined in the <arguments> element outside of  
<testusecase>. Each <run> <setup> and <teardown> may also have an  
<arguments> element. The <run> <setup> and <teardown> elements  
receive all of the arguments (global + local.)

The idea of a namespace appears to be missing to handle conflicts  
between global and local arguments. Here's my thought: By default  
there is only one namespace for global and local arguments:  
testmaker. If there is a conflict between global and local where only  
the default namespace is used then the global argument wins.

The namespace then looks like this:

    <arguments namespace="tm_globals">
         <argument name="targethost" value="10.0.0.1/gis"/>
         <argument name="id" value="admin"/>
         <argument name="password" value="admin"/>
    </arguments>

or

    <arguments>
         <argument name="targethost" namespace="tm_globals"  
value="10.0.0.1/gis"/>
         <argument name="id" value="admin"/>
         <argument name="password" value="admin"/>
     </arguments>

or

     <run name="geo_insert" testclass="" method="" langtype="java">
       <argument name="geoval" value="1738"/>
     </run>

or

     <run name="geo_insert" testclass="" method="" langtype="java">
       <argument name="geoval" namespace="mylocals" value="1738"/>
     </run>

The API to lookup arguments from within a running TestCase then must  
have alternative methods to use the namespace.


> 2. MessageSizes. These are the Payload Sizes, which are actually  
> indexes
> to a table that is not defined anywhere. It is assumed that the
> user-written test methods do handle that index. So, we will need to
> require the methods to accept that index, although not specified as an
> argument anywhere. Or, to require the user to specify the argument
> explicitly, having the argument an attribute to indicate it is not a
> value but the payload size.

<messagesizes> is an optional value in a TestScenario.xml file. I am  
expecting this will be mostly used when using one of the Accessory  
Project kits or drivers. If <messagesizes> is included in a  
TestScenario then the user needs to write code in their TestCase to  
know what to do with it, perhaps just to pass it along to a DPL. So  
perhaps the best way to handle this is to have the TestRunner insert  
the <messagesize> value into the Hashmap of arguments at runtime?

<messagesizes> also impacts the default behavior of the Tally in that  
it identifies the range of results to report.


> 3. TargetHost. In TM 4.4, properties file, we can specify the target
> hosts, they URLs and connection strings. Here we do not. That means  
> that
> to test against two different servers, I have to create two different
> XMLs and each one will have to use different jars and those jars  
> are the
> ones that "know" where the host is?  Or should we add an element with
> the host information and pass it as a names argument (just like the
> payload above)?

It's already defined in the <arguments> element...

     <arguments>
         <argument name="targethost" value="10.0.0.1/gis"/>
         <argument name="id" value="admin"/>
         <argument name="password" value="admin"/>
     </arguments>

however, I'm fine with adding another element <targethost> as a child  
of <basics> as an optional way to define the target host argument.


> 4. Multiple TestScenarios. Luis is designing thinking that there would
> be multiple TestUseCases, each one containing several usecases. Is  
> that
> the case, or do you think (as I do) that the testScenario XML holds  
> only
> one TestUseCase with multiple usecases?


Eventually the TestScenario.xml will be a full test scripting  
language of its own. For TM 5.0 there is only one TestUseCase with  
multiple use cases.



>
> 5. UseCases. Actually, we can have several use cases. In the questions
> Luis sent, he explained the hierarchy based on the actual  
> disposition in
> the XML. That is, the outer cycle would be CRLevels, then Payload  
> sizes
> and then UseCases. You told him it was ok. But that doesn't make  
> sense.
> The actual order in TM4.4 is outer cycle UseCases, the CRLevels and  
> then
> PayloadSizes. That is because you want to see the behavior of the  
> server
> doing updates in several crlevel/payload combinations, not the  
> behavior
> of the server with X users in several combinations of payloads and
> usecases. Right?


I know this seems strange. When you grab the horizontal slider and  
move it left or right it seems more "normal" to see the slider going  
through the use cases and then cr levels and payload sizes.

This will require some usability testing too before it's final. But  
for now, it's ok to code it the way I told Luis to do so.




> 6. Instances. Lastly, in 223 you can create an interpreter and send  
> the
> code in a string to execute. But that I'm not sure if supports data  
> from
> one call to be accesible on the other one. Thus, calling a Setup may
> update some variables and create data, but the call to Run may not  
> have
> access to it. Rememeber it is not we create an instance of the  
> class and
> call several methods from it, not sure if it actually creates an
> instance of the class in every call (the 223). What do you think?

There are going to be all sorts of strange 223 things to consider  
like this. For now, it's fine to expect that calls to <run> <setup>  
and <teardown> may not be able to modify arguments. We'll just have  
to see what this looks like in a running script.



Excellent questions!

-Frank




>
> William Martinez Pomares
> Architect
> Avantica Technologies
> Phone(US)  :+1 (650) 353-4522  Ext 131
> Fax(US)        :+1 (877) 372-1955
> Phone(CR)  :+506 283-9100 Ext 131
> Fax(CR)        :+506 253-7451
> http://www.avantica.net
>
>
>
> -----Original Message-----
> From: Frank Cohen [mailto:fcohen@pushtotest.com]
> Sent: Monday, February 19, 2007 10:38 PM
> To: William Martinez Pomares
> Subject: Project Design doc
>
>
> Hi William: I read through your comments in the ProyectDesign.doc.
> I'll give it a closer read in the morning.
>
> By the way, there is a typo in the name of the document. The 'y'
> should be a 'j' in ProyectDesign.doc.
>
> -Frank
>
> --
> Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
> TestMaker: The open-source SOA test automation tool
>
>
> _______________________________________________
> Dev mailing list
> Dev@lists.pushtotest.com
> http://lists.pushtotest.com/mailman/listinfo/dev
>



--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool



From fcohen at pushtotest.com  Wed Feb 21 09:35:06 2007
From: fcohen at pushtotest.com (Frank Cohen)
Date: Wed Feb 21 10:47:14 2007
Subject: [Dev] Recorder HTTPS support in TM5
References: <F15F15F5052B664392CDE9BA6BF9454401416DE4@earth.london.gamesys.local>
Message-ID: <DFD2C1A4-7CA7-4A97-83D5-34C1A0E55389@pushtotest.com>

The plan for TM5 is to add HTTPS support to the current MaxQ-based  
Recorder. I'm concerned about the number of steps it takes to set  
this up. I'm going to do some investigation on browser plug-ins to  
create TestMaker scripts.

Does anyone have experience writing plug-ins for FireFox or the other  
browsers?

-Frank



Begin forwarded message:

> From: "Geoff Meakin" <geoffm@gamesys.co.uk>
> Date: February 21, 2007 8:39:57 AM PST
> To: "Friedman, Seth" <sfriedman@videoegg.com>, "Frank Cohen"  
> <frank@pushtotest.com>
> Subject: RE: [ptt-users] Setting up the proxy server
>
> Hi Frank,
>
> I came to the same conclusion myself after writing it that it would  
> be great to have a firefox plugin to do everything you need to do.  
> I'm not sure if there's a way of overriding SSL problems with  
> firefox plugins- for example see the "selenium" project firefox  
> plugin where they discuss in detail the fact they cant get SSL to  
> behave properly without hacks and tweaks.
>
> I dont really know that much about firefox plugins so would be  
> interested to hear thoughts of somebody who does :-)
>
>
> -----Original Message-----
> From: Friedman, Seth [mailto:sfriedman@videoegg.com]
> Sent: Wed 21/02/2007 16:31
> To: Frank Cohen; Geoff Meakin
> Subject: RE: [ptt-users] Setting up the proxy server
>
>         Really cool, Frank.  I'll try both of these when I can get  
> some spare cycles.  I need cross platform capability & maxQ looks  
> like it might be the more portable of the two, so maybe I'll start  
> there.
>
>         Thanks again--
>         seth
>
>
> ________________________________
>
> From: Frank Cohen [mailto:frank@pushtotest.com]
> Sent: Tue 2/20/2007 4:00 PM
> To: Geoff Meakin
> Cc: Friedman, Seth
> Subject: Re: [ptt-users] Setting up the proxy server
>
>
>
> Hi Geoff and Seth: This is awesome stuff. Thank you for sending it. I
> forwarded it to the dev@lists.pushtotest.com list and to Avantica,
> the group doing the TestMaker 5 completion coding.
>
> The only thing that concerns me about the technique is the number of
> steps involved. I'm wondering if a better approach would be to write
> a plug-in to FireFox that outputs TestMaker scripts and embed FireFox
> into TestMaker? What do you think?
>
> -Frank
>
>
>
> On Feb 17, 2007, at 2:02 AM, Geoff Meakin wrote:
>
> > Hi guys, sorry for the delay its been a busy week at gamesys towers!
> >
> > Here's the code- no real instructions for it yet, so here you go..
> >
> > *Requirements*
> > ==============
> > In order to run this you will need on Windows:
> >
> > CYGWIN
> > Python for CYGWIN
> > Tlslite python library (download off web)
> >
> > on Unix:
> >
> > Python
> > Tlslite
> >
> > Installing those is an exercise left to the reader :-)
> >
> > *Overview*
> > ==========
> > The recorder works by becoming both a browser and a webserver in
> > one, accepting requests (both HTTP and HTTPS), decoding them,
> > outputting relevant python code, and forwarding the requests on to
> > the real server.
> >
> > *Setup*
> > =======
> > For the webserver part, first of all you need to generate your own
> > selfsigned certificates using OPENSSL. This is a (free) 5 minute
> > exercise and there are many how-tos on the web how to do it. I
> > would include my actual certificates, but they are tied to our
> > servers, and thats kind of sensitive so I dont want to release
> > them. Just tie your ssl certificates to whichever URL you are
> > wanting to record- you're trying to fool browsers into believing
> > that the recorder IS the webserver at that URL. Simple enough.
> > Note, as long as you have a working certificate it will work with
> > any URL providing you dont mind SSL warning popups when browsing.
> > There are ways to get rid of these including getting the
> > certificate 100% correct, importing the CA into your browser etc.,
> > but all these can be found on the web and I want to explain the
> > other stuff..
> >
> > I seem to remember you need to specify -dec or -dec2 on one of the
> > flags to openssl to set the encryption type of the final
> > certificate. The python tls libs dont recognise all encryption
> > types, but i found this one worked... Generate yourself files
> > called server.csr, server.crt and server.key and put them in the
> > same directory as your unpacked tarfile (attached).
> >
> > Then you just need to configure where you're pointing this recorder
> > to-
> > Thats all defined in SharedData.py- you need to specify the 4
> > forward settings at the top of this file/class with the name of the
> > server you want to record.
> >
> > The recorder works by you sending your requests for your URL to it,
> > rather than to the URL (so not a true proxy, although could be
> > adapted into one). You do this by for example on windows, popping a:
> >
> > 127.0.0.1    my.testserver.local
> >
> > into your hosts file (my.testserver.local is obviously the hostname
> > of your URL you are recording)
> >
> > This will override DNS lookups and send it to the recorder instead.
> >
> > So finally the recorder needs to know where to send the real
> > requests and what URL they were bound for- those are those 4
> > forwarding properties I talked about above, so pop the relevant
> > info in there.
> >
> > *Running*
> > =========
> >
> > To run just type
> > python Recorder.py
> >
> > you should see server started on 443, server started on 80
> >
> > If your hosts file is pointing in the right place then when you
> > browse your site you should instantly see things come out of STDOUT
> > from the recorder representing your requests.
> >
> > Each request to the webserver has its own requestid (a comment),
> > and both the HTTP(s) request and HTTP(s) response is logged IN FULL
> > in the log subdirectory of the recorder for informational purposes
> > so you can see the innards of whats going on
> >
> > To stop just type CTRL+C and reset your hosts file to normal.
> >
> > *3 final notes*
> > ===============
> >
> > I usually have 2 simple icons on my desktop/taskbar etc., one for
> > setting my hosts file in record mode, one for setting my hosts file
> > in normal mode- just takes the pain out of remembering where your
> > hosts file is ;-)
> >
> > The output the recorder produces is currently for Testmaker as we
> > use it- very slightly different to the official Testmaker release
> > as we've wrapped some of Testmaker 4's agentbase functions in our
> > own functions (so you'll see self.getPage instead of self.get). All
> > the output is controlled by Logger.py, so its very simple to adapt
> > to the latest version of Testmaker if you want, or even write the
> > output in a different language if you want.
> >
> > The tlslite library by default logs to STDOUT nasty debug messages
> > on every request which you dont want to see (kind of like apache
> > access logs), as they will be interspersed with your recorder
> > output. To get rid of these, edit /usr/lib/python2.4/
> > BaseHTTPServer.py and comment out the line:
> >
> >             # print (self.protocol_version, code, message)
> >
> > in the send_response function
> >
> > This should get rid of them.
> >
> >
> > Let me know what you think/ whether its useful/ if the instructions
> > dont make sense, I can always do a step-by-step guide if you want,
> > or give you our TWiki wiki page on it
> >
> > Cheers
> > -Geoff
> >
> >
> >
> >
> >
> > Excellent Seth. Thank you for the offer. I'd love your evaluation. I
> > will forward this to Geoff's email. Thanks! -Frank
> >
> >
> >
> > On Feb 12, 2007, at 10:35 AM, Friedman, Seth wrote:
> >
> > > I didn't see Geoff's email address to ask him directly, but if  
> he's
> > > comfortable sharing it - I'd be happy to give it a spin, e.g.
> > > briefly testing it out & giving feedback to you/him.
> > >
> > > best,
> > > Seth
> > > From: users-bounces@lists.pushtotest.com on behalf of Frank Cohen
> > > Sent: Sat 2/10/2007 10:50 PM
> > > To: TestMaker users list
> > > Subject: Re: [ptt-users] Setting up the proxy server
> > >
> > > Hi Geoff: I'd love to see it. Please send it to me at
> > > fcohen@pushtotest.com. Thanks. -Frank
> > >
> > >
> > >
> > > On Feb 10, 2007, at 10:38 AM, Geoff Meakin wrote:
> > >
> > > > Hi Frank,
> > > >
> > > > If you would be interested, I wrote a HTTP and HTTPS recorder in
> > > > python which outputs testmaker code. It just runs commandline  
> and
> > > > is based on tlslite.
> > > >
> > > > It has limitations, but works perfectly for our scenario.
> > > >
> > > > Cheers
> > > > -Geoff
> > > >
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: users-bounces@lists.pushtotest.com on behalf of Friedman,
> > Seth
> > > > Sent: Thu 08/02/2007 18:16
> > > > To: TestMaker users list; TestMaker users list
> > > > Subject: RE: [ptt-users] Setting up the proxy server
> > > >
> > > > One idea I've been intending to explore e.g. figure out if it's
> > > > already possible, is a standalone recorder packaging (ie:  
> sniffer)
> > > > that could be passed around independently of TestMaker.    I've
> > > > been in the position to help a developer fix a bug, and he was
> > > > uncomfortable installing the full testmaker suite, but probably
> > > > could be convinced to run something smaller.
> > > >
> > > > It seems like a niche that's not particularly well filled -
> > there's
> > > > all the powerful network-focused tools (ethereal, network  
> general,
> > > > etc) which are not the most effective for application level  
> work,
> > > > and then there's the lightweight HTTP specific ones (Effetech)
> > that
> > > > have some, for us, major limitations around post.
> > > >
> > > > seth
> > > >
> > > > ________________________________
> > > >
> > > > From: users-bounces@lists.pushtotest.com on behalf of Frank  
> Cohen
> > > > Sent: Thu 2/8/2007 8:26 AM
> > > > To: TestMaker users list
> > > > Subject: Re: [ptt-users] Setting up the proxy server
> > > >
> > > >
> > > >
> > > > Glad to hear it. Thanks.
> > > >
> > > > By the way, we are making substantial changes to the TestMaker
> > > > Recorder for TestMaker 5 and the team would appreciate any
> > feedback
> > > > or experiences you have on using the current Recorder so we can
> > make
> > > > it better.
> > > >
> > > > -Frank
> > > >
> > > >
> > > > On Feb 8, 2007, at 1:48 AM, Dave Talkowski wrote:
> > > >
> > > > > It's OK - I checked with the IT boys and dicovered that  
> some of
> > > the
> > > > > external traffic was being blocked by the firewall. Works fine
> > > now!
> > > > >
> > > > > Thanks.
> > > > >
> > > > > Dave T.
> > > > >
> > > > > -----Original Message-----
> > > > > From: users-bounces@lists.pushtotest.com
> > > > > [mailto:users-bounces@lists.pushtotest.com] On Behalf Of Frank
> > > Cohen
> > > > > Sent: Tuesday, February 06, 2007 6:19 PM
> > > > > To: TestMaker users list
> > > > > Subject: Re: [ptt-users] Setting up the proxy server
> > > > >
> > > > > Hi Dave: Sorry for the delayed response. I'm looking into why
> > your
> > > > > message wasn't posted immediate to the list.
> > > > >
> > > > > Is the page you are trying to record available on the public
> > > > Internet?
> > > > > If so, what is the URL. Is the site using HTTPS?
> > > > >
> > > > > -Frank
> > > > >
> > > > >
> > > > >
> > > > > On Jan 24, 2007, at 2:17 AM, Dave Talkowski wrote:
> > > > >
> > > > >>
> > > > >> Hi,
> > > > >>
> > > > >> I'm having a bit of trouble using the Agent Recorder. I've
> > > tried a
> > > > >> variety of settings in my IE6 browser, but when I try to  
> run a
> > > test
> > > > >> agent, I get one of the following:
> > > > >>
> > > > >>      - no web pages displayed and the test reported as  
> done (!)
> > > > >>      - browser reports 'Page cannot be displayed'
> > > > >>      - the following errors in the text output:
> > > > >>                 java.lang.NullPointerException
> > > > >>                         at
> > > > >> com.bitmechanic.maxq.SwingMain.reportOutsideException
> > > > >> (SwingMain.java:305)
> > > > >>                         at
> > > com.bitmechanic.maxq.RequestHandler.run
> > > > >> (RequestHandler.java:243)
> > > > >>                         at java.lang.Thread.run(Thread.java:
> > 534)
> > > > >>
> > > > >> I guess it's a security setting here or something, but I'm
> > > stumped!
> > > > >>
> > > > >> Anyone got any ideas for a fix?
> > > > >>
> > > > >> DaveTalkowski
> > > > >> Senior Analyst
> > > > >> * dave.talkowski@assureweb.co.uk
> > > > >>
> > > > >> _______________________________________________
> > > > >> Users mailing list
> > > > >> Users@lists.pushtotest.com
> > > > >> http://lists.pushtotest.com/mailman/listinfo/users
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Frank Cohen, PushToTest, http://www.PushToTest.com <http:// 
> www.pushtotest.com/>  <http://
> > > > www.pushtotest.com/> , phone 408 374 7426
> > > > > TestMaker: The open-source SOA test automation tool
> > > > >
> > > > >
> > > > >
> > > > > _______________________________________________
> > > > > Users mailing list
> > > > > Users@lists.pushtotest.com
> > > > > http://lists.pushtotest.com/mailman/listinfo/users
> > > > > _______________________________________________
> > > > > Users mailing list
> > > > > Users@lists.pushtotest.com
> > > > > http://lists.pushtotest.com/mailman/listinfo/users
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Frank Cohen, PushToTest, http://www.PushToTest.com <http:// 
> www.pushtotest.com/>  <http://
> > > > www.pushtotest.com/> , phone 408 374 7426
> > > > TestMaker: The open-source SOA test automation tool
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > Users mailing list
> > > > Users@lists.pushtotest.com
> > > > http://lists.pushtotest.com/mailman/listinfo/users
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > Users mailing list
> > > > Users@lists.pushtotest.com
> > > > http://lists.pushtotest.com/mailman/listinfo/users
> > >
> > >
> > >
> > > --
> > > Frank Cohen, PushToTest, http://www.PushToTest.com <http:// 
> www.pushtotest.com/> , phone 408 374
> > 7426
> > > TestMaker: The open-source SOA test automation tool
> > >
> > >
> > >
> > > _______________________________________________
> > > Users mailing list
> > > Users@lists.pushtotest.com
> > > http://lists.pushtotest.com/mailman/listinfo/users
> > >
> >
> >
> >
> >
> > <recorder.tgz>
>
>
>
> --
> Frank Cohen, PushToTest, http://www.PushToTest.com <http:// 
> www.pushtotest.com/> , phone 408 374 7426
> TestMaker: The open-source SOA test automation tool
>
>
>
>
>
>
>
>
>



--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool



From fcohen at pushtotest.com  Wed Feb 21 14:24:32 2007
From: fcohen at pushtotest.com (Frank Cohen)
Date: Wed Feb 21 15:36:43 2007
Subject: [Dev] Grinder, TestMaker, Jython, and JSR 223
References: <45DC69F8.9050808@bea.com>
Message-ID: <0F9D6F0E-98D3-4E03-BA64-46919F259424@pushtotest.com>

Hi William: Would you please reply with a brief explanation of TM5's  
use of the JSR 223 APIs. Thanks. -Frank




Begin forwarded message:

> From: Philip Aston <paston@bea.com>
> Date: February 21, 2007 7:49:12 AM PST
> To: grinder-use <grinder-use@lists.sourceforge.net>
> Subject: Re: [Grinder-use] Jython 2.2?
> Reply-To: grinder-use <grinder-use@lists.sourceforge.net>
>
> Frank Cohen wrote:
>> Excellent. Thanks!
>>
>> BTW, I'm going to be leading the scripting session at JavaOne again
>> this year. Let me know if you're going to the conference and I'll
>> bring up Grinder's use of Jython as a successful example.
>>
>> Does Grinder use the JSR 223 API for using Jython?
>>
> No, and its unlikely to in the future.
>
> The Grinder is only loosely coupled to Jython, and doesn't need the
> generic API of JSR 223 (although very early on I considered and ruled
> out the BSF). The only signification link is the instrumentation of
> script objects. This is done in an AOP "around"ish sort of way through
> some fiddling with Jython internals. The downside of this is that  
> I'm at
> the mercy of changes to those internals.
>
> If I replace this, I will do so using a true AOP framework at the byte
> code level. This will open up the possibility of scripting in any
> language that runs on the JVM.
>
> - Phil
> ______________________________________________________________________ 
> _
> Notice:  This email message, together with any attachments, may  
> contain
> information  of  BEA Systems,  Inc.,  its subsidiaries  and   
> affiliated
> entities,  that may be confidential,  proprietary,  copyrighted   
> and/or
> legally privileged, and is intended solely for the use of the  
> individual
> or entity named in this message. If you are not the intended  
> recipient,
> and have received this message in error, please immediately return  
> this
> by email and then delete it.
>
> ---------------------------------------------------------------------- 
> ---
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to  
> share your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php? 
> page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> grinder-use mailing list
> grinder-use@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/grinder-use
>



--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool



From fcohen at pushtotest.com  Wed Feb 21 17:48:55 2007
From: fcohen at pushtotest.com (Frank Cohen)
Date: Wed Feb 21 19:01:18 2007
Subject: [Dev] Selinium and TM5
Message-ID: <5E32C8A7-E653-4C62-B8E9-50E00F38F201@pushtotest.com>

I was talking with Ilya Sterin of Nextrials today and he says they  
may be using Selinium in an upcoming project. From what I can tell  
Selinium looks like a good little plug-in to FireFox to create JUnit  
TestCases. Ilya tells me you can also do playback of JavaScript  
enabled Web pages outside of FireFox. Pretty cool stuff. It seems to  
me - at first look - that TM5 should contribute a Selinium output  
option to create TM5 TestCases and that we should embed FireFox and  
Selinium into TM5 as its recorder, rather than try to upgrade the  
MaxQ branch. Any opinions?

-Frank

--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool



From wmartinez at AVANTICATEC.NET  Thu Feb 22 13:45:44 2007
From: wmartinez at AVANTICATEC.NET (=?US-ASCII?Q?William_Martinez_Pomares?=)
Date: Thu Feb 22 14:57:21 2007
Subject: [Dev] Grinder, TestMaker, Jython, and JSR 223
In-Reply-To: <0F9D6F0E-98D3-4E03-BA64-46919F259424@pushtotest.com>
Message-ID: <85A61B0FDDD4834EA295605C349BABAA0DE79C@dc1avantica.AVANTICA.AVANTICATEC.NET>

Hi Frank.
Sorry for the delay.

TM5 is aiming to define TestScenarios using its new TestScenario XML
schema. The XML will contain instructions about concurrency, message
sizes, and indications of what processes should the TM execute to
actually run the tests (among other things). 

Those indications come in the form:
<runTest name="name_of_test" testclass="name_of_class"
method="name_of_method" langtype="script\language"/>

For instance
<runTest name="insert_test" testclass="mytest" method="runTest"
langtype="jython"/>

will indicate TM that it has to run the method runTest in the class
"mytest" and that the code is written in "jython".
To provide TM witht he ability to run scripts other than plain Java, JSR
223 APIs are going to be used.
The langtype attribute may then be any of the supported JSR 223 engines.
Also, since scripts will come from files and the invocation will be to
specific methods, the supported ScriptEngines should implement the
invocable interface. 

Note that the control of the test is defined in the XML, and not in the
methods. This allows the methods of a unit test to be used to execute a
load test just by configuring the scenario.

JSR 223 is generic to JVM 6, so TM5 will probably come with an embedded
JRE 6.  New engines can be added too.

William.


-----Original Message-----
From: dev-bounces@lists.pushtotest.com
[mailto:dev-bounces@lists.pushtotest.com]On Behalf Of Frank Cohen
Sent: Wednesday, February 21, 2007 4:25 PM
To: TestMaker Developers List
Subject: [Dev] Grinder, TestMaker, Jython, and JSR 223


Hi William: Would you please reply with a brief explanation of TM5's  
use of the JSR 223 APIs. Thanks. -Frank




Begin forwarded message:

> From: Philip Aston <paston@bea.com>
> Date: February 21, 2007 7:49:12 AM PST
> To: grinder-use <grinder-use@lists.sourceforge.net>
> Subject: Re: [Grinder-use] Jython 2.2?
> Reply-To: grinder-use <grinder-use@lists.sourceforge.net>
>
> Frank Cohen wrote:
>> Excellent. Thanks!
>>
>> BTW, I'm going to be leading the scripting session at JavaOne again
>> this year. Let me know if you're going to the conference and I'll
>> bring up Grinder's use of Jython as a successful example.
>>
>> Does Grinder use the JSR 223 API for using Jython?
>>
> No, and its unlikely to in the future.
>
> The Grinder is only loosely coupled to Jython, and doesn't need the
> generic API of JSR 223 (although very early on I considered and ruled
> out the BSF). The only signification link is the instrumentation of
> script objects. This is done in an AOP "around"ish sort of way through
> some fiddling with Jython internals. The downside of this is that  
> I'm at
> the mercy of changes to those internals.
>
> If I replace this, I will do so using a true AOP framework at the byte
> code level. This will open up the possibility of scripting in any
> language that runs on the JVM.
>
> - Phil
> ______________________________________________________________________

> _
> Notice:  This email message, together with any attachments, may  
> contain
> information  of  BEA Systems,  Inc.,  its subsidiaries  and   
> affiliated
> entities,  that may be confidential,  proprietary,  copyrighted   
> and/or
> legally privileged, and is intended solely for the use of the  
> individual
> or entity named in this message. If you are not the intended  
> recipient,
> and have received this message in error, please immediately return  
> this
> by email and then delete it.
>
> ----------------------------------------------------------------------

> ---
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to  
> share your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php? 
> page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> grinder-use mailing list
> grinder-use@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/grinder-use
>



--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool



_______________________________________________
Dev mailing list
Dev@lists.pushtotest.com
http://lists.pushtotest.com/mailman/listinfo/dev
From fcohen at pushtotest.com  Thu Feb 22 16:32:39 2007
From: fcohen at pushtotest.com (Frank Cohen)
Date: Thu Feb 22 19:25:47 2007
Subject: [Dev] Transaction Definition
Message-ID: <AA1A54FF-5679-4566-ABBA-1914088DB605@pushtotest.com>

Hi William: Great question.

Page 8 of the TM5 Specification document envisions three types of  
<testusecase>:

Case #1

<testusecase name="simpletest">
   <test>
     <run name="geo_insert" testclass="com.pushtotest.example.test"  
method="runTest" langtype="java"/>
   </test>
</testusecase>

This is the most simple example. TestMaker operates the <test> by  
executing the <run> method, sleeps, and repeats until the duration of  
the test is over. In this case, each operation of the <test> is a  
transaction and creates a <transaction> entry in the log.


Case #2

<testusecase name="simpletest">
   <setup name="geo_insert" testclass="com.pushtotest.example.test"  
method="setUp" langtype="java"/>
   <test>
     <run name="geo_insert" testclass="com.pushtotest.example.test"  
method="runTest" langtype="java"/>
   </test>
   <teardown name="geo_insert"  
testclass="com.pushtotest.example.test" method="tearDown"
            langtype="java"/>
</testusecase>

This is a slightly more complex case. TestMaker operates the <setup>  
once. It logs a <setup> transaction to the log. TestMaker operates  
the <test> by executing the <run> method, sleeps, and repeats. Each  
operation of the <test> is a transaction and creates a <transaction>  
entry in the log. Once the duration of the test is complete then  
TestMaker operates the <teardown> once and logs a <teardown>  
transaction to the log. The Tally script reports the <setup> and  
<teardown> times and then it gives the TPS levels for the  
<transactions>.


Case #3

<testusecase name="simpletest">
   <setup name="geo_insert" testclass="com.pushtotest.example.test"  
method="setUp" langtype="java"/>
   <test>
     <run name="geo_insert" testclass="com.pushtotest.example.test"  
method="insert" langtype="java"/>
     <run name="geo_insert" testclass="com.pushtotest.example.test"  
method="query" langtype="java"/>
     <run name="geo_insert" testclass="com.pushtotest.example.test"  
method="update" langtype="java"/>
     <run name="geo_insert" testclass="com.pushtotest.example.test"  
method="delete" langtype="java"/>
   </test>
   <teardown name="geo_insert"  
testclass="com.pushtotest.example.test" method="tearDown"
            langtype="java"/>
</testusecase>

This is similar to Case 2 in that TestMaker operates and records the  
<setup> and <teardown> times separately from the <transaction>s. In  
this case there are multiple <run> elements. TestMaker records a  
single <transaction> for a iteration of the <test>. TestMaker views  
the total time for all of the <run> methods as part of a since  
<transaction>.


Case #4

     <testusecase name="sequencetest">
         <setup name="geo_insert"  
testclass="com.pushtotest.example.test" method="setUp" langtype="java">
             <argument name="geoval" value="1738"/>
         </setup>
         <sequence name="inserts" proportion="80">
             <test name="test1">
                 <setup name="geo_insert" testclass="  
com.pushtotest.example.test " method="seqSetup"
                     langtype="java">
                     <argument name="geoval" value="1738"/>
                 </setup>
                 <run name="geo_insert" testclass="" method=""  
langtype="java">
                     <argument name="geoval" value="1738"/>
                 </run>
                 <teardown name="geo_insert"  
testclass="com.pushtotest.example.test"
                             method="geoteardown" langtype="java"/>
             </test>
         </sequence>
         <sequence name="queries" proportion="20">
             <test>
                 <setup name="geo_insert"  
testclass="com.pushtotest.example.test" method="seqSetup"
                     langtype="java">
                     <argument name="geoval" value="1738"/>
                 </setup>
                 <run name="geo_insert" testclass="" method=""  
langtype="java">
                     <argument name="geoval" value="1738"/>
                 </run>
                 <teardown name="geo_insert"  
testclass="com.pushtotest.example.test"
                             method="geoteardown" langtype="java"/>
             </test>
         </sequence>
         <teardown name="geo_insert"  
testclass="com.pushtotest.example.test"
                    method="geoteardown" langtype="java"/>
     </testusecase>

At each CR level, TestMaker instantiates the corresponding number of  
concurrently running threads. The threads instantiate one of the two  
defined <sequence>s. TestMaker sets 80% of the threads to operate the  
inserts sequence and the remaining 20% of the threads to operate the  
queries sequence.

TestMaker operates the <setup> named geo_insert at the beginning of  
each CR level and  the <teardown> named geo_insert one time at the  
end of each CR level.

One <transaction> log entry is made for each iteration through the  
<test>, including the <setup> and <teardown> inside of the <test>.

Does this make sense?

-Frank

---

Frank, in the XML you can identify several levels of definition.
You have a TestCase, that may contain several UseCases.
These in turn may contain several Sequences, and each sequence may
contain several Tests.

You can run each UseCase using all combinations between CRLevels and
MessageSizes.
For each run, all sequences will run in parallel.
For each sequence, a number of threads is spawn (based on crlevel and
proportion).
Each thread runs the defined Test.

There are calls, each one is atomic. One for Setup, one for Run, and the
other one for TearDown.
You can have Setup/Teardown surrounding the UseCases. Those are meant to
be called once.
You can have Setup/Teardown surrounding the sequences. Those are meant
to be called per crlevel/MessageSize iteration.
You can have Setup/Teardown surrounding the Test. Those are meant to be
called once per thread.
You can have Setup/Teardown surrounding the Run, inside the Test. Here
is where the transaction definition is needed.

The threads are supposed to run calling a setup, then calling several
times the Test and then finish by calling the teardown.
The mentioned Setup/TearDown tuple is the one surrounding the test.
That leads to logically think the Test as the Transaction, that is
called several times.
For each transaction executed, an entry in the log is created indicating
the timestamp, the result and the time it took to complete.
That would also mean the Setup/TearDown tuple that is "inside" the Test,
is also part of the transaction, so the completion time includes the
Test's Setup/TearDown times.

Is that idea correct? Test = Transaction? I know there are times when
you want to define the transaction as a complete Setup/Run/TearDown
cycle, thus this definition works. Note also that the Setup/Teardown
tuples are optional.

Please let me know what you think.



--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool



From fcohen at pushtotest.com  Thu Feb 22 22:36:23 2007
From: fcohen at pushtotest.com (Frank Cohen)
Date: Thu Feb 22 23:48:54 2007
Subject: [Dev] Status update please
Message-ID: <9907B93D-045F-4836-812D-0E7338F56690@pushtotest.com>

Hi William: Please provide a status update on TM5. The schedule says  
we should be done writing the new classes and from 2/23 to 2/26 be  
work on changing the existing classes. Is that work on track? Thanks,  
in advance. -Frank



--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool



From fcohen at pushtotest.com  Fri Feb 23 10:21:58 2007
From: fcohen at pushtotest.com (Frank Cohen)
Date: Fri Feb 23 11:34:30 2007
Subject: [Dev] Updated the TM5 splash screen
Message-ID: <0C495B43-D9CD-4061-80F7-29BB43F703B9@pushtotest.com>

FYI: I checked-in the new splash screen for TestMaker 5 into cvs. -Frank


--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool



From wmartinez at AVANTICATEC.NET  Fri Feb 23 10:38:59 2007
From: wmartinez at AVANTICATEC.NET (=?us-ascii?Q?William_Martinez_Pomares?=)
Date: Fri Feb 23 11:50:49 2007
Subject: [Dev] Status update please
In-Reply-To: <9907B93D-045F-4836-812D-0E7338F56690@pushtotest.com>
Message-ID: <85A61B0FDDD4834EA295605C349BABAA0DEB61@dc1avantica.AVANTICA.AVANTICATEC.NET>

Hi.

Sorry for the lack of reports. We will be providing a detailed report
each week.

Up to date, we have finished the initial draft of the design. 
Much of the scheduled class development had undergoing changes, mainly
because we are trying to clean up the actual architecture and design in
the process.
Thus, some of the expected classes in the schedule plan are no longer to
be done, and are now replaced by other classes. Here is a summary of
what is meant:
1. We created an schema to define the TestScenario XML documents. 
2. Using a JAXB Reader class, we created a runner package that will
process the TestScenario XML documents compliant to that schema, and
execute the test indicated in the document.
3. From TestScenario, the following features are implemented:
-- Loading Dynamic Jars
-- Executing Sequences, using CRLevels and Payloads
-- Executing Tests. It is actually running tests written in Java, we are
still to include JSR 223 support since it is a JVM 6 feature.
4. TNMaster and other former Jython control agents are being refactored.
TNMaster in particular contains code paint UI, control the run remotely
and communication. The new set will contain different layers for that.
The same for TNTally, which now will work to create the summaries and
will delegate the work of creating the charts to a Graphing class.

Botton line, the work is on track, but we need to update the Schedule to
reflect the classes and tasks at hand.

William Martinez Pomares
Architect
Avantica Technologies
Phone(US)  :+1 (650) 353-4522  Ext 131
Fax(US)        :+1 (877) 372-1955  
Phone(CR)  :+506 283-9100 Ext 131 
Fax(CR)        :+506 253-7451 
http://www.avantica.net



-----Original Message-----
From: dev-bounces@lists.pushtotest.com
[mailto:dev-bounces@lists.pushtotest.com]On Behalf Of Frank Cohen
Sent: Friday, February 23, 2007 12:36 AM
To: TestMaker Developers List
Subject: [Dev] Status update please


Hi William: Please provide a status update on TM5. The schedule says  
we should be done writing the new classes and from 2/23 to 2/26 be  
work on changing the existing classes. Is that work on track? Thanks,  
in advance. -Frank



--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool



_______________________________________________
Dev mailing list
Dev@lists.pushtotest.com
http://lists.pushtotest.com/mailman/listinfo/dev
From fcohen at pushtotest.com  Fri Feb 23 10:58:05 2007
From: fcohen at pushtotest.com (Frank Cohen)
Date: Fri Feb 23 12:11:00 2007
Subject: [Dev] Status update please
In-Reply-To: <85A61B0FDDD4834EA295605C349BABAA0DEB61@dc1avantica.AVANTICA.AVANTICATEC.NET>
References: <85A61B0FDDD4834EA295605C349BABAA0DEB61@dc1avantica.AVANTICA.AVANTICATEC.NET>
Message-ID: <667C1779-763A-4E0D-AD2C-E8FAEE1AA8C7@pushtotest.com>

Excellent update. Thanks. I'm glad to see the Reader class is now  
functional. In my mind the next major milestone is to be able to run  
a test on a node. About the refactoring, if it is faster to rewrite/ 
replace one of the TN... classes then go for it. Don't worry about  
backward compatibility on the TestNetwork functions. -Frank



On Feb 23, 2007, at 10:38 AM, William Martinez Pomares wrote:

> Hi.
>
> Sorry for the lack of reports. We will be providing a detailed report
> each week.
>
> Up to date, we have finished the initial draft of the design.
> Much of the scheduled class development had undergoing changes, mainly
> because we are trying to clean up the actual architecture and  
> design in
> the process.
> Thus, some of the expected classes in the schedule plan are no  
> longer to
> be done, and are now replaced by other classes. Here is a summary of
> what is meant:
> 1. We created an schema to define the TestScenario XML documents.
> 2. Using a JAXB Reader class, we created a runner package that will
> process the TestScenario XML documents compliant to that schema, and
> execute the test indicated in the document.
> 3. From TestScenario, the following features are implemented:
> -- Loading Dynamic Jars
> -- Executing Sequences, using CRLevels and Payloads
> -- Executing Tests. It is actually running tests written in Java,  
> we are
> still to include JSR 223 support since it is a JVM 6 feature.
> 4. TNMaster and other former Jython control agents are being  
> refactored.
> TNMaster in particular contains code paint UI, control the run  
> remotely
> and communication. The new set will contain different layers for that.
> The same for TNTally, which now will work to create the summaries and
> will delegate the work of creating the charts to a Graphing class.
>
> Botton line, the work is on track, but we need to update the  
> Schedule to
> reflect the classes and tasks at hand.
>
> William Martinez Pomares
> Architect
> Avantica Technologies
> Phone(US)  :+1 (650) 353-4522  Ext 131
> Fax(US)        :+1 (877) 372-1955
> Phone(CR)  :+506 283-9100 Ext 131
> Fax(CR)        :+506 253-7451
> http://www.avantica.net
>
>
>
> -----Original Message-----
> From: dev-bounces@lists.pushtotest.com
> [mailto:dev-bounces@lists.pushtotest.com]On Behalf Of Frank Cohen
> Sent: Friday, February 23, 2007 12:36 AM
> To: TestMaker Developers List
> Subject: [Dev] Status update please
>
>
> Hi William: Please provide a status update on TM5. The schedule says
> we should be done writing the new classes and from 2/23 to 2/26 be
> work on changing the existing classes. Is that work on track? Thanks,
> in advance. -Frank
>
>
>
> --
> Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
> TestMaker: The open-source SOA test automation tool
>
>
>
> _______________________________________________
> Dev mailing list
> Dev@lists.pushtotest.com
> http://lists.pushtotest.com/mailman/listinfo/dev
> _______________________________________________
> Dev mailing list
> Dev@lists.pushtotest.com
> http://lists.pushtotest.com/mailman/listinfo/dev
>



--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool



From wmartinez at AVANTICATEC.NET  Fri Feb 23 11:16:19 2007
From: wmartinez at AVANTICATEC.NET (=?us-ascii?Q?William_Martinez_Pomares?=)
Date: Fri Feb 23 12:28:10 2007
Subject: [Dev] Function Buttons
In-Reply-To: <667C1779-763A-4E0D-AD2C-E8FAEE1AA8C7@pushtotest.com>
Message-ID: <85A61B0FDDD4834EA295605C349BABAA0DEB8D@dc1avantica.AVANTICA.AVANTICATEC.NET>

Hi Frank.

In the guiTestScenariopanel we found several buttons. Here are a couple
of questions:
1. We have a Pause button and a Stop button. I can think of a remote
running test, where a test may take several minutes to complete and I
left a test scenario running (with several usecases and crlevels). Now,
I may want to stop the test for something, but I would like the current
cycle to finish not to lose the actual testing done so far. I may want,
in the other hand, to stop the test inmediately, since I have an
emergency. These can be the actual examples for these two cases. Now,
for pause to actually pause an ongoing cycle? It does not make sense
since the test would become invalid. And if you need to replay the test
after it, then it is the same as pressing the Stop button and replay.
Right? So we are assuming the Pause button will not pause the current
running threads but rather wait for actual cycle to finish and then wait
for a play.
If so, the pause will not leave the Console. Remember that now each
cycle is signaled by the console to the TestNode!

2. Local/Remote.  We can run the TestScenario locally (TM will spawn the
threads and run the methods) or remotely (TM will send the TestScenario
to the TestNode to be run by remote control). But we only have a play
button. In TM 4.4 we have two play buttons, one for remote play. What do
you think?

William Martinez Pomares
Architect
Avantica Technologies
Phone(US)  :+1 (650) 353-4522  Ext 131
Fax(US)        :+1 (877) 372-1955  
Phone(CR)  :+506 283-9100 Ext 131 
Fax(CR)        :+506 253-7451 
http://www.avantica.net

  
From fcohen at pushtotest.com  Fri Feb 23 12:15:11 2007
From: fcohen at pushtotest.com (Frank Cohen)
Date: Fri Feb 23 13:27:46 2007
Subject: [Dev] Function Buttons
In-Reply-To: <85A61B0FDDD4834EA295605C349BABAA0DEB8D@dc1avantica.AVANTICA.AVANTICATEC.NET>
References: <85A61B0FDDD4834EA295605C349BABAA0DEB8D@dc1avantica.AVANTICA.AVANTICATEC.NET>
Message-ID: <7C62F172-751C-48BE-A176-82E84EE44D4E@pushtotest.com>

Good questions. See below.

> 1. We have a Pause button and a Stop button. I can think of a remote
> running test, where a test may take several minutes to complete and I
> left a test scenario running (with several usecases and crlevels).  
> Now,
> I may want to stop the test for something, but I would like the  
> current
> cycle to finish not to lose the actual testing done so far. I may  
> want,
> in the other hand, to stop the test inmediately, since I have an
> emergency. These can be the actual examples for these two cases. Now,
> for pause to actually pause an ongoing cycle? It does not make sense
> since the test would become invalid. And if you need to replay the  
> test
> after it, then it is the same as pressing the Stop button and replay.
> Right? So we are assuming the Pause button will not pause the current
> running threads but rather wait for actual cycle to finish and then  
> wait
> for a play.
> If so, the pause will not leave the Console. Remember that now each
> cycle is signaled by the console to the TestNode!


Pause is to temporarily wait on running the next test use case. Stop  
is to abort the test at the next test use case. Both use the snapshot  
mechanism.

I figure if it is really an emergency then the user can stop the  
TestNodes manually using some console like HP OpenView or the Windows  
process manager.


>
> 2. Local/Remote.  We can run the TestScenario locally (TM will  
> spawn the
> threads and run the methods) or remotely (TM will send the  
> TestScenario
> to the TestNode to be run by remote control). But we only have a play
> button. In TM 4.4 we have two play buttons, one for remote play.  
> What do
> you think?


The TestScenario.xml by default runs the test on the local node. It's  
up to the user to define the other nodes they want to run the test  
upon. That means we only need one play button.

By the way, there will also be a Run Script command in the Run drop  
down menu to run a Jython script. This is for developers working with  
the Recorder to test the function of a recorded or written TestCase  
before running it on a node.

-Frank





>
> William Martinez Pomares
> Architect
> Avantica Technologies
> Phone(US)  :+1 (650) 353-4522  Ext 131
> Fax(US)        :+1 (877) 372-1955
> Phone(CR)  :+506 283-9100 Ext 131
> Fax(CR)        :+506 253-7451
> http://www.avantica.net
>
>
> _______________________________________________
> Dev mailing list
> Dev@lists.pushtotest.com
> http://lists.pushtotest.com/mailman/listinfo/dev
>



--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool



From wmartinez at AVANTICATEC.NET  Fri Feb 23 14:02:57 2007
From: wmartinez at AVANTICATEC.NET (=?us-ascii?Q?William_Martinez_Pomares?=)
Date: Fri Feb 23 15:14:45 2007
Subject: [Dev] Function Buttons
In-Reply-To: <7C62F172-751C-48BE-A176-82E84EE44D4E@pushtotest.com>
Message-ID: <85A61B0FDDD4834EA295605C349BABAA0DECC3@dc1avantica.AVANTICA.AVANTICATEC.NET>

Ok.
I don't understand the part about running local. Per TestScenario nodes
description, I can define the remote nodes where to run. 
That means TM will connect to testnetwork testnodes and run the tests
from there. I assume the jars to load and the actual TestScenario.xml is
sent to the testnodes and it will run there. 

When I say running local, I meant to test the TestScenario locally:
uploading the jars and executing the xml. This will not require to send
it to the testnodes, and thus we can run the scenario from testmaker
rather than remotely.

What do you think.
William.

-----Original Message-----
From: dev-bounces@lists.pushtotest.com
[mailto:dev-bounces@lists.pushtotest.com]On Behalf Of Frank Cohen
Sent: Friday, February 23, 2007 2:15 PM
To: TestMaker Developers List
Subject: Re: [Dev] Function Buttons


Good questions. See below.

> 1. We have a Pause button and a Stop button. I can think of a remote
> running test, where a test may take several minutes to complete and I
> left a test scenario running (with several usecases and crlevels).  
> Now,
> I may want to stop the test for something, but I would like the  
> current
> cycle to finish not to lose the actual testing done so far. I may  
> want,
> in the other hand, to stop the test inmediately, since I have an
> emergency. These can be the actual examples for these two cases. Now,
> for pause to actually pause an ongoing cycle? It does not make sense
> since the test would become invalid. And if you need to replay the  
> test
> after it, then it is the same as pressing the Stop button and replay.
> Right? So we are assuming the Pause button will not pause the current
> running threads but rather wait for actual cycle to finish and then  
> wait
> for a play.
> If so, the pause will not leave the Console. Remember that now each
> cycle is signaled by the console to the TestNode!


Pause is to temporarily wait on running the next test use case. Stop  
is to abort the test at the next test use case. Both use the snapshot  
mechanism.

I figure if it is really an emergency then the user can stop the  
TestNodes manually using some console like HP OpenView or the Windows  
process manager.


>
> 2. Local/Remote.  We can run the TestScenario locally (TM will  
> spawn the
> threads and run the methods) or remotely (TM will send the  
> TestScenario
> to the TestNode to be run by remote control). But we only have a play
> button. In TM 4.4 we have two play buttons, one for remote play.  
> What do
> you think?


The TestScenario.xml by default runs the test on the local node. It's  
up to the user to define the other nodes they want to run the test  
upon. That means we only need one play button.

By the way, there will also be a Run Script command in the Run drop  
down menu to run a Jython script. This is for developers working with  
the Recorder to test the function of a recorded or written TestCase  
before running it on a node.

-Frank





>
> William Martinez Pomares
> Architect
> Avantica Technologies
> Phone(US)  :+1 (650) 353-4522  Ext 131
> Fax(US)        :+1 (877) 372-1955
> Phone(CR)  :+506 283-9100 Ext 131
> Fax(CR)        :+506 253-7451
> http://www.avantica.net
>
>
> _______________________________________________
> Dev mailing list
> Dev@lists.pushtotest.com
> http://lists.pushtotest.com/mailman/listinfo/dev
>



--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool



_______________________________________________
Dev mailing list
Dev@lists.pushtotest.com
http://lists.pushtotest.com/mailman/listinfo/dev
From fcohen at pushtotest.com  Fri Feb 23 14:27:58 2007
From: fcohen at pushtotest.com (Frank Cohen)
Date: Fri Feb 23 15:40:33 2007
Subject: [Dev] Function Buttons
In-Reply-To: <85A61B0FDDD4834EA295605C349BABAA0DECC3@dc1avantica.AVANTICA.AVANTICATEC.NET>
References: <85A61B0FDDD4834EA295605C349BABAA0DECC3@dc1avantica.AVANTICA.AVANTICATEC.NET>
Message-ID: <AF62668F-BF64-4D95-BB86-42E3D9CC98C0@pushtotest.com>

It's pretty well buried on page 19 of the spec. My design is to have  
a TestNode running on the local TestMaker machine. You go through a  
TestNode to run TestScenario scripts, even if it is only the local  
node (default.)

I thought of doing this so we can have less code to maintain.

You can argue me out of this design.

-Frank




On Feb 23, 2007, at 2:02 PM, William Martinez Pomares wrote:

> Ok.
> I don't understand the part about running local. Per TestScenario  
> nodes
> description, I can define the remote nodes where to run.
> That means TM will connect to testnetwork testnodes and run the tests
> from there. I assume the jars to load and the actual  
> TestScenario.xml is
> sent to the testnodes and it will run there.
>
> When I say running local, I meant to test the TestScenario locally:
> uploading the jars and executing the xml. This will not require to  
> send
> it to the testnodes, and thus we can run the scenario from testmaker
> rather than remotely.
>
> What do you think.
> William.
>
> -----Original Message-----
> From: dev-bounces@lists.pushtotest.com
> [mailto:dev-bounces@lists.pushtotest.com]On Behalf Of Frank Cohen
> Sent: Friday, February 23, 2007 2:15 PM
> To: TestMaker Developers List
> Subject: Re: [Dev] Function Buttons
>
>
> Good questions. See below.
>
>> 1. We have a Pause button and a Stop button. I can think of a remote
>> running test, where a test may take several minutes to complete and I
>> left a test scenario running (with several usecases and crlevels).
>> Now,
>> I may want to stop the test for something, but I would like the
>> current
>> cycle to finish not to lose the actual testing done so far. I may
>> want,
>> in the other hand, to stop the test inmediately, since I have an
>> emergency. These can be the actual examples for these two cases. Now,
>> for pause to actually pause an ongoing cycle? It does not make sense
>> since the test would become invalid. And if you need to replay the
>> test
>> after it, then it is the same as pressing the Stop button and replay.
>> Right? So we are assuming the Pause button will not pause the current
>> running threads but rather wait for actual cycle to finish and then
>> wait
>> for a play.
>> If so, the pause will not leave the Console. Remember that now each
>> cycle is signaled by the console to the TestNode!
>
>
> Pause is to temporarily wait on running the next test use case. Stop
> is to abort the test at the next test use case. Both use the snapshot
> mechanism.
>
> I figure if it is really an emergency then the user can stop the
> TestNodes manually using some console like HP OpenView or the Windows
> process manager.
>
>
>>
>> 2. Local/Remote.  We can run the TestScenario locally (TM will
>> spawn the
>> threads and run the methods) or remotely (TM will send the
>> TestScenario
>> to the TestNode to be run by remote control). But we only have a play
>> button. In TM 4.4 we have two play buttons, one for remote play.
>> What do
>> you think?
>
>
> The TestScenario.xml by default runs the test on the local node. It's
> up to the user to define the other nodes they want to run the test
> upon. That means we only need one play button.
>
> By the way, there will also be a Run Script command in the Run drop
> down menu to run a Jython script. This is for developers working with
> the Recorder to test the function of a recorded or written TestCase
> before running it on a node.
>
> -Frank
>
>
>
>
>
>>
>> William Martinez Pomares
>> Architect
>> Avantica Technologies
>> Phone(US)  :+1 (650) 353-4522  Ext 131
>> Fax(US)        :+1 (877) 372-1955
>> Phone(CR)  :+506 283-9100 Ext 131
>> Fax(CR)        :+506 253-7451
>> http://www.avantica.net
>>
>>
>> _______________________________________________
>> Dev mailing list
>> Dev@lists.pushtotest.com
>> http://lists.pushtotest.com/mailman/listinfo/dev
>>
>
>
>
> --
> Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
> TestMaker: The open-source SOA test automation tool
>
>
>
> _______________________________________________
> Dev mailing list
> Dev@lists.pushtotest.com
> http://lists.pushtotest.com/mailman/listinfo/dev
> _______________________________________________
> Dev mailing list
> Dev@lists.pushtotest.com
> http://lists.pushtotest.com/mailman/listinfo/dev
>



--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool



From fcohen at pushtotest.com  Sat Feb 24 13:20:23 2007
From: fcohen at pushtotest.com (Frank Cohen)
Date: Sat Feb 24 14:33:08 2007
Subject: [Dev] Web page recorder ideas
Message-ID: <76A72C51-12C1-4D45-8266-0169CF7A788E@pushtotest.com>

I'm investigating Selinium and other browser-plug-in-based tools to  
record Web applications for functional testing via playback. Selinium  
looks very polished. I'm finding there are others like TestGen4Web  
from SpikeSource.

Here's what I found so far:

http://developer.spikesource.com/wiki/index.php/Projects:TestGen4Web

http://www.openqa.org/selenium-ide/

If you know of others then please tell me about them.

Also, I am seeking your experiences with any of these.

Thanks.

-Frank

--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool


From fcohen at pushtotest.com  Sun Feb 25 00:11:04 2007
From: fcohen at pushtotest.com (Frank Cohen)
Date: Sun Feb 25 01:23:58 2007
Subject: [Dev] Web page recorder ideas
In-Reply-To: <76A72C51-12C1-4D45-8266-0169CF7A788E@pushtotest.com>
References: <76A72C51-12C1-4D45-8266-0169CF7A788E@pushtotest.com>
Message-ID: <3518E400-2171-44C0-80C3-E1A9CBD71751@pushtotest.com>

TestGen4Web has an intriguing utility: a translator that goes from a  
recorded script's XML form into Java code that uses HTTPUnit. I've  
long been impressed with HTTPUnit's API, but haven't take the steps  
needed to integrate it into TestMaker. To do so would require adding  
function to HTTPUnit to use the TOOL protocol handlers.

I'm looking now at TestGen4Web and am wondering if we should  
integrate HTTPUnit into TestMaker to make it easy to support  
TestGen4Web?

Anyone want to offer an opinion, or to vote?

-Frank



On Feb 24, 2007, at 1:20 PM, Frank Cohen wrote:

> I'm investigating Selinium and other browser-plug-in-based tools to  
> record Web applications for functional testing via playback.  
> Selinium looks very polished. I'm finding there are others like  
> TestGen4Web from SpikeSource.
>
> Here's what I found so far:
>
> http://developer.spikesource.com/wiki/index.php/Projects:TestGen4Web
>
> http://www.openqa.org/selenium-ide/
>
> If you know of others then please tell me about them.
>
> Also, I am seeking your experiences with any of these.
>
> Thanks.
>
> -Frank
>
> --
> Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
> TestMaker: The open-source SOA test automation tool
>
>
> _______________________________________________
> Dev mailing list
> Dev@lists.pushtotest.com
> http://lists.pushtotest.com/mailman/listinfo/dev
>



--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool



From fcohen at pushtotest.com  Tue Feb 27 15:35:10 2007
From: fcohen at pushtotest.com (Frank Cohen)
Date: Tue Feb 27 16:48:44 2007
Subject: [Dev] TestMaker 5 User Interaction design document
Message-ID: <F0DE8D79-1A7B-4145-A869-93162CC643E1@pushtotest.com>

Hi William: Please take a look at the TM5 UI design document at:

http://downloads.pushtotest.com/tm5/TM5_UI_Design.pdf

This defines the user interaction for the project. Please reply with  
your notes, comments, questions, feedback.

Thanks.

-Frank

--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool