Saturday, May 18, 2013

perform performance test using multi-mechanize

gave multi-mechanize a try recently, when in-need of behavior analysis for an application on massive concurrent requests being sent to it

What? multi-mechanize is an OpenSource load testing framework written and configurable in Python. So either make calls to any web service or simple import and utilize any python-accessible API/service.
It's successor of an old Python load testing framework called Pylot.

Developed and maintained @GitHub

Install available through 'pip' too as
   $ pip install multi-mechanize
It also requires matplotlib library if you wanna view Graphs in the generated HTML report (or otherwise), BUT it's not mandatory as the framework would perform all tests fine with an import error for matplotlib.

This also follows a bit convention oriented run structure. We'll see how.

Before starting any performance check user is supposed to create a project using
$ multimech-newproject project_name

This creates a dir 'project_name' at place of execution with following dir-structure
./project_name
├── config.cfg
└── test_scripts
    └── v_user.py

here './project_name/config.cfg' mainly contains global configuration around the duration to run,


[global]
run_time = 30                # second duration for test to run, required field
rampup = 0                   # second duration for users to ramp-up, required field
results_ts_interval = 10  # second time series interval for result analysis, required field
progress_bar = on         # console progress bar on/off, default=on
console_logging = off    # console logging to standard ouput, default=off
xml_report = off            # xml/jtl report generation, default=off
# results_database = sqlite:///results.db ## optional component to push results to DB
# post_run_script = do_whatever.py ## to run any script to do anything on test completion 
[user_group-1]
threads = 3                  # number of threads to run the following script in
script = v_user.py        # the actual script that will be run to test, give any 
## similar more users with same/different threads and script values can be added with names like user_group-ANYTHING.

Now, to run this prepared project
multimech-run ./project_name

As I already mentioned it picks certain implementations by convention, most important to notice is the way user-group-script is to be prepared.
Basically it's to be a python script that can utilize all Python magic you know or install. But it need to have a 'Transaction' class with 'run' method.
When you run your project using multi-mechanize, it instantiates 'Transaction' class and keeps calling 'run' method in a loop...

Example user-script to test load at your 'python -m SimpleHTTPServer'
# v_user_server.py
import httplib


class Transaction(object):
    def run(self):
        conn = httplib.HTTPConnection('127.0.0.1:8000')
        conn.request('GET', '/')
        resp = conn.getresponse()
        conn.close() 
        assert ((resp.status / 400) == 0), 'BadResponse: HTTP %s' % resp.status
# finito file


Find the detailed scripting guide with plenty good examples herehttp://testutils.org/multi-mechanize/scripts.html

Thursday, May 16, 2013

pycallgraph : usage example

'pycallgraph' is a fine python utility enabling developers to prepare call graph for any python code piece... it also notifies the number of times a call being made and total time spent in it

Homepage: http://pycallgraph.slowchop.com/pycallgraph/wiki
Codebase: https://github.com/gak/pycallgraph

it require 'graphviz' (http://www.graphviz.org/) to be present on the same machine for image generation from the analyzed calls data...

install : $ pip install pycallgraph

Usage#1 Selective graphing

wrap the code to be graphed as follows...
import pycallgraph
pycallgraph.start_trace()
fetch() # logic to be graphed
pycallgraph.stop_trace()
just_log() # logic NOT to be graphed
pycallgraph.start_trace()
process() # logic to be graphed
pycallgraph.stop_trace()
pycallgraph.make_dot_graph('path_to_graph.png')

Usage#2 Decorator

Import the decorator method and place the decorator over any method where call graph is required...
import this file wherever you require the @callgraph decorator and use it

Decorator Python code. Sample code with decorator usage and selective trace usage are at end of this blog... from this gist: https://gist.github.com/abhishekkr/5592520



pycallgraph.start_trace() method lets you pass a filter function to not graph some modules on purpose, say in nova I don't wanna graph all the dependency libraries magical calls...


STATUTORY WARNING: Graph will be huge for a very high level call and probably unreadable due to overworked image generator. Use it wisely on burnt areas.


Call Graph images for code sample in Gists

for call with decorator



for call with selective trace