[ptt-users] Re: TestMaker memory problems

[Frank Cohen]

Hi Carl: Excellent thoughts. See below. -Frank

On Jan 3, 2007, at 10:38 AM, Carl Herder wrote:

> My thoughts:
>
> 1. The initial URL is appended to the list, whether or not we  
> indicate to follow redirects, every time connect() is called. We  
> should use this list only if we are following redirects.

I'm a little hesitant from reducing default functionality. I haven't  
anything from other users how they are using the URLlist. So for now  
I'd like to leave it as the default behavior, and maybe later have it  
only used when connect(1) is used to automatically follow redirects.

>  2. It might be best to separate the debugging function from simply  
> following redirects. Debugging is generally for short runs. We  
> might want to run a workload for a long period that includes  
> redirection.

The change I proposed will clear the URL list each time the TestCase  
runs its runTest method.


>
> 3. This may be an unfounded concern, but the way clerURLlist() is  
> coded MAY cause a memory leak, by leaving a lot of Vectors in the  
> heap for GC to take care of. I suggest something like this:
>
>     public void clearURLlist()
>     {
>         if (URLlist != null) {
>            URLlist.removeAllElements();
>      } else {
>            URLlist = new Vector();
>      }
>     }

GC in Java 1.4 and later has gotten really very good, so I'm not that  
concerned. I'm happy to change the code as you described above and  
will do so without anyone's objection.

>
> 4. Suggestion: Could the URLlist be processed and reinitialized as  
> soon as we leave the redirection while loop in connect()?

I'm not sure I understand the suggestion. However, you can add the  
call to clearURLlist to a test script pretty much anywhere you want.

>
> Thanks,
> Carl
>
>
> Frank Cohen <fcohen at pushtotest.com> wrote:
> Hmmm... the URLlist was originally intended to help users debug Web
> services that use HTTP redirect commands. The idea is getURLlist()
> would show a list of the URLs that were redirected after an
> HTTPProtocol connect() method.
>
> I think the best solution to avoid the Out Of Memory problems in
> XSTest is to modify agentbase.py to call the HTTPProtocol clearURLlist
> () method in the config() method. That way the list is reset each
> time the test's setUp() method is called.
>
> This is an easy change so I'll add it to the TestMaker 4.4.1
> maintenance release. Please let me know your comments, feedback,  
> ideas.
>
> Thanks Carl for pointing out this problem.
>
> -Frank
>
>
>
>
>
>
>
> On Dec 28, 2006, at 10:52 AM, Carl Herder wrote:
>
> > Long running xstest loads run out of memory. It is not technically
> > a memory leak. In com.pushtotest.tool.protocolhandler.HTTPprotocol
> > there is a Vector URLlist that grows constantly during the run. The
> > connect() method appends the URL for every request to this Vector.
> > An instance is created for each CVU. As far as I can see, this list
> > is not used for anything (HTTPSprotocol uses it for redirected
> > URLs, which are printed out at the end).
> >
> > I commented out
> >
> > URLlist.addElement( con.getURL() );
> >
> > and the memory problem goes away. I have been running now for 18
> > hours, with 200 CVUs, and heap memory is stable.
> >
> > Plug for the Wily Leakhunter tool (disclosure--I work for the
> > company), which pointed me immediately to the offending List.
> >
> > Carl
> > __________________________________________________
> > Do You Yahoo!?
> > Tired of spam? Yahoo! Mail has the best spam protection around
> > http://mail.yahoo.com
>
> _______________________________________________
> Users mailing list
> Users at lists.pushtotest.com
> http://lists.pushtotest.com/mailman/listinfo/users
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam? Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
>
> _______________________________________________
> Users mailing list
> Users at lists.pushtotest.com
> http://lists.pushtotest.com/mailman/listinfo/users