[Dev] Transaction Definition

[Frank Cohen]

Hi William: Great question.

Page 8 of the TM5 Specification document envisions three types of  
<testusecase>:

Case #1

<testusecase name="simpletest">
   <test>
     <run name="geo_insert" testclass="com.pushtotest.example.test"  
method="runTest" langtype="java"/>
   </test>
</testusecase>

This is the most simple example. TestMaker operates the <test> by  
executing the <run> method, sleeps, and repeats until the duration of  
the test is over. In this case, each operation of the <test> is a  
transaction and creates a <transaction> entry in the log.


Case #2

<testusecase name="simpletest">
   <setup name="geo_insert" testclass="com.pushtotest.example.test"  
method="setUp" langtype="java"/>
   <test>
     <run name="geo_insert" testclass="com.pushtotest.example.test"  
method="runTest" langtype="java"/>
   </test>
   <teardown name="geo_insert"  
testclass="com.pushtotest.example.test" method="tearDown"
            langtype="java"/>
</testusecase>

This is a slightly more complex case. TestMaker operates the <setup>  
once. It logs a <setup> transaction to the log. TestMaker operates  
the <test> by executing the <run> method, sleeps, and repeats. Each  
operation of the <test> is a transaction and creates a <transaction>  
entry in the log. Once the duration of the test is complete then  
TestMaker operates the <teardown> once and logs a <teardown>  
transaction to the log. The Tally script reports the <setup> and  
<teardown> times and then it gives the TPS levels for the  
<transactions>.


Case #3

<testusecase name="simpletest">
   <setup name="geo_insert" testclass="com.pushtotest.example.test"  
method="setUp" langtype="java"/>
   <test>
     <run name="geo_insert" testclass="com.pushtotest.example.test"  
method="insert" langtype="java"/>
     <run name="geo_insert" testclass="com.pushtotest.example.test"  
method="query" langtype="java"/>
     <run name="geo_insert" testclass="com.pushtotest.example.test"  
method="update" langtype="java"/>
     <run name="geo_insert" testclass="com.pushtotest.example.test"  
method="delete" langtype="java"/>
   </test>
   <teardown name="geo_insert"  
testclass="com.pushtotest.example.test" method="tearDown"
            langtype="java"/>
</testusecase>

This is similar to Case 2 in that TestMaker operates and records the  
<setup> and <teardown> times separately from the <transaction>s. In  
this case there are multiple <run> elements. TestMaker records a  
single <transaction> for a iteration of the <test>. TestMaker views  
the total time for all of the <run> methods as part of a since  
<transaction>.


Case #4

     <testusecase name="sequencetest">
         <setup name="geo_insert"  
testclass="com.pushtotest.example.test" method="setUp" langtype="java">
             <argument name="geoval" value="1738"/>
         </setup>
         <sequence name="inserts" proportion="80">
             <test name="test1">
                 <setup name="geo_insert" testclass="  
com.pushtotest.example.test " method="seqSetup"
                     langtype="java">
                     <argument name="geoval" value="1738"/>
                 </setup>
                 <run name="geo_insert" testclass="" method=""  
langtype="java">
                     <argument name="geoval" value="1738"/>
                 </run>
                 <teardown name="geo_insert"  
testclass="com.pushtotest.example.test"
                             method="geoteardown" langtype="java"/>
             </test>
         </sequence>
         <sequence name="queries" proportion="20">
             <test>
                 <setup name="geo_insert"  
testclass="com.pushtotest.example.test" method="seqSetup"
                     langtype="java">
                     <argument name="geoval" value="1738"/>
                 </setup>
                 <run name="geo_insert" testclass="" method=""  
langtype="java">
                     <argument name="geoval" value="1738"/>
                 </run>
                 <teardown name="geo_insert"  
testclass="com.pushtotest.example.test"
                             method="geoteardown" langtype="java"/>
             </test>
         </sequence>
         <teardown name="geo_insert"  
testclass="com.pushtotest.example.test"
                    method="geoteardown" langtype="java"/>
     </testusecase>

At each CR level, TestMaker instantiates the corresponding number of  
concurrently running threads. The threads instantiate one of the two  
defined <sequence>s. TestMaker sets 80% of the threads to operate the  
inserts sequence and the remaining 20% of the threads to operate the  
queries sequence.

TestMaker operates the <setup> named geo_insert at the beginning of  
each CR level and  the <teardown> named geo_insert one time at the  
end of each CR level.

One <transaction> log entry is made for each iteration through the  
<test>, including the <setup> and <teardown> inside of the <test>.

Does this make sense?

-Frank

---

Frank, in the XML you can identify several levels of definition.
You have a TestCase, that may contain several UseCases.
These in turn may contain several Sequences, and each sequence may
contain several Tests.

You can run each UseCase using all combinations between CRLevels and
MessageSizes.
For each run, all sequences will run in parallel.
For each sequence, a number of threads is spawn (based on crlevel and
proportion).
Each thread runs the defined Test.

There are calls, each one is atomic. One for Setup, one for Run, and the
other one for TearDown.
You can have Setup/Teardown surrounding the UseCases. Those are meant to
be called once.
You can have Setup/Teardown surrounding the sequences. Those are meant
to be called per crlevel/MessageSize iteration.
You can have Setup/Teardown surrounding the Test. Those are meant to be
called once per thread.
You can have Setup/Teardown surrounding the Run, inside the Test. Here
is where the transaction definition is needed.

The threads are supposed to run calling a setup, then calling several
times the Test and then finish by calling the teardown.
The mentioned Setup/TearDown tuple is the one surrounding the test.
That leads to logically think the Test as the Transaction, that is
called several times.
For each transaction executed, an entry in the log is created indicating
the timestamp, the result and the time it took to complete.
That would also mean the Setup/TearDown tuple that is "inside" the Test,
is also part of the transaction, so the completion time includes the
Test's Setup/TearDown times.

Is that idea correct? Test = Transaction? I know there are times when
you want to define the transaction as a complete Setup/Run/TearDown
cycle, thus this definition works. Note also that the Setup/Teardown
tuples are optional.

Please let me know what you think.



--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool