[ptt-users] Setting up the proxy server
Frank Cohen
fcohen at pushtotest.com
Wed Feb 21 09:36:14 PST 2007
Thanks for the pointer to Selenium. I'll check it out. I posted a
request for help to the dev at lists.pushtotest.com list. We'll see. -Frank
On Feb 21, 2007, at 8:39 AM, Geoff Meakin wrote:
> Hi Frank,
>
> I came to the same conclusion myself after writing it that it would
> be great to have a firefox plugin to do everything you need to do.
> I'm not sure if there's a way of overriding SSL problems with
> firefox plugins- for example see the "selenium" project firefox
> plugin where they discuss in detail the fact they cant get SSL to
> behave properly without hacks and tweaks.
>
> I dont really know that much about firefox plugins so would be
> interested to hear thoughts of somebody who does :-)
>
>
> -----Original Message-----
> From: Friedman, Seth [mailto:sfriedman at videoegg.com]
> Sent: Wed 21/02/2007 16:31
> To: Frank Cohen; Geoff Meakin
> Subject: RE: [ptt-users] Setting up the proxy server
>
> Really cool, Frank. I'll try both of these when I can get
> some spare cycles. I need cross platform capability & maxQ looks
> like it might be the more portable of the two, so maybe I'll start
> there.
>
> Thanks again--
> seth
>
>
> ________________________________
>
> From: Frank Cohen [mailto:frank at pushtotest.com]
> Sent: Tue 2/20/2007 4:00 PM
> To: Geoff Meakin
> Cc: Friedman, Seth
> Subject: Re: [ptt-users] Setting up the proxy server
>
>
>
> Hi Geoff and Seth: This is awesome stuff. Thank you for sending it. I
> forwarded it to the dev at lists.pushtotest.com list and to Avantica,
> the group doing the TestMaker 5 completion coding.
>
> The only thing that concerns me about the technique is the number of
> steps involved. I'm wondering if a better approach would be to write
> a plug-in to FireFox that outputs TestMaker scripts and embed FireFox
> into TestMaker? What do you think?
>
> -Frank
>
>
>
> On Feb 17, 2007, at 2:02 AM, Geoff Meakin wrote:
>
> > Hi guys, sorry for the delay its been a busy week at gamesys towers!
> >
> > Here's the code- no real instructions for it yet, so here you go..
> >
> > *Requirements*
> > ==============
> > In order to run this you will need on Windows:
> >
> > CYGWIN
> > Python for CYGWIN
> > Tlslite python library (download off web)
> >
> > on Unix:
> >
> > Python
> > Tlslite
> >
> > Installing those is an exercise left to the reader :-)
> >
> > *Overview*
> > ==========
> > The recorder works by becoming both a browser and a webserver in
> > one, accepting requests (both HTTP and HTTPS), decoding them,
> > outputting relevant python code, and forwarding the requests on to
> > the real server.
> >
> > *Setup*
> > =======
> > For the webserver part, first of all you need to generate your own
> > selfsigned certificates using OPENSSL. This is a (free) 5 minute
> > exercise and there are many how-tos on the web how to do it. I
> > would include my actual certificates, but they are tied to our
> > servers, and thats kind of sensitive so I dont want to release
> > them. Just tie your ssl certificates to whichever URL you are
> > wanting to record- you're trying to fool browsers into believing
> > that the recorder IS the webserver at that URL. Simple enough.
> > Note, as long as you have a working certificate it will work with
> > any URL providing you dont mind SSL warning popups when browsing.
> > There are ways to get rid of these including getting the
> > certificate 100% correct, importing the CA into your browser etc.,
> > but all these can be found on the web and I want to explain the
> > other stuff..
> >
> > I seem to remember you need to specify -dec or -dec2 on one of the
> > flags to openssl to set the encryption type of the final
> > certificate. The python tls libs dont recognise all encryption
> > types, but i found this one worked... Generate yourself files
> > called server.csr, server.crt and server.key and put them in the
> > same directory as your unpacked tarfile (attached).
> >
> > Then you just need to configure where you're pointing this recorder
> > to-
> > Thats all defined in SharedData.py- you need to specify the 4
> > forward settings at the top of this file/class with the name of the
> > server you want to record.
> >
> > The recorder works by you sending your requests for your URL to it,
> > rather than to the URL (so not a true proxy, although could be
> > adapted into one). You do this by for example on windows, popping a:
> >
> > 127.0.0.1 my.testserver.local
> >
> > into your hosts file (my.testserver.local is obviously the hostname
> > of your URL you are recording)
> >
> > This will override DNS lookups and send it to the recorder instead.
> >
> > So finally the recorder needs to know where to send the real
> > requests and what URL they were bound for- those are those 4
> > forwarding properties I talked about above, so pop the relevant
> > info in there.
> >
> > *Running*
> > =========
> >
> > To run just type
> > python Recorder.py
> >
> > you should see server started on 443, server started on 80
> >
> > If your hosts file is pointing in the right place then when you
> > browse your site you should instantly see things come out of STDOUT
> > from the recorder representing your requests.
> >
> > Each request to the webserver has its own requestid (a comment),
> > and both the HTTP(s) request and HTTP(s) response is logged IN FULL
> > in the log subdirectory of the recorder for informational purposes
> > so you can see the innards of whats going on
> >
> > To stop just type CTRL+C and reset your hosts file to normal.
> >
> > *3 final notes*
> > ===============
> >
> > I usually have 2 simple icons on my desktop/taskbar etc., one for
> > setting my hosts file in record mode, one for setting my hosts file
> > in normal mode- just takes the pain out of remembering where your
> > hosts file is ;-)
> >
> > The output the recorder produces is currently for Testmaker as we
> > use it- very slightly different to the official Testmaker release
> > as we've wrapped some of Testmaker 4's agentbase functions in our
> > own functions (so you'll see self.getPage instead of self.get). All
> > the output is controlled by Logger.py, so its very simple to adapt
> > to the latest version of Testmaker if you want, or even write the
> > output in a different language if you want.
> >
> > The tlslite library by default logs to STDOUT nasty debug messages
> > on every request which you dont want to see (kind of like apache
> > access logs), as they will be interspersed with your recorder
> > output. To get rid of these, edit /usr/lib/python2.4/
> > BaseHTTPServer.py and comment out the line:
> >
> > # print (self.protocol_version, code, message)
> >
> > in the send_response function
> >
> > This should get rid of them.
> >
> >
> > Let me know what you think/ whether its useful/ if the instructions
> > dont make sense, I can always do a step-by-step guide if you want,
> > or give you our TWiki wiki page on it
> >
> > Cheers
> > -Geoff
> >
> >
> >
> >
> >
> > Excellent Seth. Thank you for the offer. I'd love your evaluation. I
> > will forward this to Geoff's email. Thanks! -Frank
> >
> >
> >
> > On Feb 12, 2007, at 10:35 AM, Friedman, Seth wrote:
> >
> > > I didn't see Geoff's email address to ask him directly, but if
> he's
> > > comfortable sharing it - I'd be happy to give it a spin, e.g.
> > > briefly testing it out & giving feedback to you/him.
> > >
> > > best,
> > > Seth
> > > From: users-bounces at lists.pushtotest.com on behalf of Frank Cohen
> > > Sent: Sat 2/10/2007 10:50 PM
> > > To: TestMaker users list
> > > Subject: Re: [ptt-users] Setting up the proxy server
> > >
> > > Hi Geoff: I'd love to see it. Please send it to me at
> > > fcohen at pushtotest.com. Thanks. -Frank
> > >
> > >
> > >
> > > On Feb 10, 2007, at 10:38 AM, Geoff Meakin wrote:
> > >
> > > > Hi Frank,
> > > >
> > > > If you would be interested, I wrote a HTTP and HTTPS recorder in
> > > > python which outputs testmaker code. It just runs commandline
> and
> > > > is based on tlslite.
> > > >
> > > > It has limitations, but works perfectly for our scenario.
> > > >
> > > > Cheers
> > > > -Geoff
> > > >
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: users-bounces at lists.pushtotest.com on behalf of Friedman,
> > Seth
> > > > Sent: Thu 08/02/2007 18:16
> > > > To: TestMaker users list; TestMaker users list
> > > > Subject: RE: [ptt-users] Setting up the proxy server
> > > >
> > > > One idea I've been intending to explore e.g. figure out if it's
> > > > already possible, is a standalone recorder packaging (ie:
> sniffer)
> > > > that could be passed around independently of TestMaker. I've
> > > > been in the position to help a developer fix a bug, and he was
> > > > uncomfortable installing the full testmaker suite, but probably
> > > > could be convinced to run something smaller.
> > > >
> > > > It seems like a niche that's not particularly well filled -
> > there's
> > > > all the powerful network-focused tools (ethereal, network
> general,
> > > > etc) which are not the most effective for application level
> work,
> > > > and then there's the lightweight HTTP specific ones (Effetech)
> > that
> > > > have some, for us, major limitations around post.
> > > >
> > > > seth
> > > >
> > > > ________________________________
> > > >
> > > > From: users-bounces at lists.pushtotest.com on behalf of Frank
> Cohen
> > > > Sent: Thu 2/8/2007 8:26 AM
> > > > To: TestMaker users list
> > > > Subject: Re: [ptt-users] Setting up the proxy server
> > > >
> > > >
> > > >
> > > > Glad to hear it. Thanks.
> > > >
> > > > By the way, we are making substantial changes to the TestMaker
> > > > Recorder for TestMaker 5 and the team would appreciate any
> > feedback
> > > > or experiences you have on using the current Recorder so we can
> > make
> > > > it better.
> > > >
> > > > -Frank
> > > >
> > > >
> > > > On Feb 8, 2007, at 1:48 AM, Dave Talkowski wrote:
> > > >
> > > > > It's OK - I checked with the IT boys and dicovered that
> some of
> > > the
> > > > > external traffic was being blocked by the firewall. Works fine
> > > now!
> > > > >
> > > > > Thanks.
> > > > >
> > > > > Dave T.
> > > > >
> > > > > -----Original Message-----
> > > > > From: users-bounces at lists.pushtotest.com
> > > > > [mailto:users-bounces at lists.pushtotest.com] On Behalf Of Frank
> > > Cohen
> > > > > Sent: Tuesday, February 06, 2007 6:19 PM
> > > > > To: TestMaker users list
> > > > > Subject: Re: [ptt-users] Setting up the proxy server
> > > > >
> > > > > Hi Dave: Sorry for the delayed response. I'm looking into why
> > your
> > > > > message wasn't posted immediate to the list.
> > > > >
> > > > > Is the page you are trying to record available on the public
> > > > Internet?
> > > > > If so, what is the URL. Is the site using HTTPS?
> > > > >
> > > > > -Frank
> > > > >
> > > > >
> > > > >
> > > > > On Jan 24, 2007, at 2:17 AM, Dave Talkowski wrote:
> > > > >
> > > > >>
> > > > >> Hi,
> > > > >>
> > > > >> I'm having a bit of trouble using the Agent Recorder. I've
> > > tried a
> > > > >> variety of settings in my IE6 browser, but when I try to
> run a
> > > test
> > > > >> agent, I get one of the following:
> > > > >>
> > > > >> - no web pages displayed and the test reported as
> done (!)
> > > > >> - browser reports 'Page cannot be displayed'
> > > > >> - the following errors in the text output:
> > > > >> java.lang.NullPointerException
> > > > >> at
> > > > >> com.bitmechanic.maxq.SwingMain.reportOutsideException
> > > > >> (SwingMain.java:305)
> > > > >> at
> > > com.bitmechanic.maxq.RequestHandler.run
> > > > >> (RequestHandler.java:243)
> > > > >> at java.lang.Thread.run(Thread.java:
> > 534)
> > > > >>
> > > > >> I guess it's a security setting here or something, but I'm
> > > stumped!
> > > > >>
> > > > >> Anyone got any ideas for a fix?
> > > > >>
> > > > >> DaveTalkowski
> > > > >> Senior Analyst
> > > > >> * dave.talkowski at assureweb.co.uk
> > > > >>
> > > > >> _______________________________________________
> > > > >> Users mailing list
> > > > >> Users at lists.pushtotest.com
> > > > >> http://lists.pushtotest.com/mailman/listinfo/users
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Frank Cohen, PushToTest, http://www.PushToTest.com <http://
> www.pushtotest.com/> <http://
> > > > www.pushtotest.com/> , phone 408 374 7426
> > > > > TestMaker: The open-source SOA test automation tool
> > > > >
> > > > >
> > > > >
> > > > > _______________________________________________
> > > > > Users mailing list
> > > > > Users at lists.pushtotest.com
> > > > > http://lists.pushtotest.com/mailman/listinfo/users
> > > > > _______________________________________________
> > > > > Users mailing list
> > > > > Users at lists.pushtotest.com
> > > > > http://lists.pushtotest.com/mailman/listinfo/users
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Frank Cohen, PushToTest, http://www.PushToTest.com <http://
> www.pushtotest.com/> <http://
> > > > www.pushtotest.com/> , phone 408 374 7426
> > > > TestMaker: The open-source SOA test automation tool
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > Users mailing list
> > > > Users at lists.pushtotest.com
> > > > http://lists.pushtotest.com/mailman/listinfo/users
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > Users mailing list
> > > > Users at lists.pushtotest.com
> > > > http://lists.pushtotest.com/mailman/listinfo/users
> > >
> > >
> > >
> > > --
> > > Frank Cohen, PushToTest, http://www.PushToTest.com <http://
> www.pushtotest.com/> , phone 408 374
> > 7426
> > > TestMaker: The open-source SOA test automation tool
> > >
> > >
> > >
> > > _______________________________________________
> > > Users mailing list
> > > Users at lists.pushtotest.com
> > > http://lists.pushtotest.com/mailman/listinfo/users
> > >
> >
> >
> >
> >
> > <recorder.tgz>
>
>
>
> --
> Frank Cohen, PushToTest, http://www.PushToTest.com <http://
> www.pushtotest.com/> , phone 408 374 7426
> TestMaker: The open-source SOA test automation tool
>
>
>
>
>
>
>
>
>
--
Frank Cohen, PushToTest, http://www.PushToTest.com, phone 408 374 7426
TestMaker: The open-source SOA test automation tool
More information about the Users
mailing list