Network Sensitivity Tests.

Network Sensitivity Tests

Network sensitivity tests are variations on Load Tests and Performance Tests that focus on the Wide Area Network (WAN) limitations and network activity (eg. traffic, latency, error rates...). Network sensitivity tests can be used to predict the impact of a given WAN segment or traffic profile on various applications that are bandwidth dependant. Network issues often arise at low levels of concurrency over low bandwidth WAN segments. Very 'chatty' applications can appear to be more prone to response time degradation under certain conditions than other applications that actually use more bandwidth. For example, some applications may degrade to unacceptable levels of response time when a certain pattern of network traffic uses 50% of available bandwidth, while other applications are virtually un-changed in response time even with 85% of available bandwidth consumed elsewhere.

This is a particularly important test for deployment of a time critical application over a WAN.

Also, some front end systems such as web servers, need to work much harder with 'dirty' communications compared with the clean communications encountered on a high speed LAN in an isolated load and performance testing environment.

Why execute Network Sensitivity Tests

The three principle reasons for executing Network Sensitivity tests are as follows:

	Determine the impact on response time of a WAN link. (Variation of a Performance Test)
	Determine the capacity of a system based on a given WAN link. (Variation of a Load Test)
	Determine the impact on the system under test that is under 'dirty' communications load. (Variation of a Load Test)

Execution of performance and load tests for analysis of network sensitivity require test system configuration to emulate a WAN. Once a WAN link has been configured, performance and load tests conducted will become Network Sensitivity Tests.

There are two ways of configuring such tests.

Use a simulated WAN and inject appropriate background traffic.

This can be achieved by putting back to back routers between a load generator and the system under test. The routers can be configured to allow the required level of bandwidth, and instead of connecting to a real WAN, they connect directly through to each other.

When back to back routers are configured to be part of a test, they will basically limit the bandwidth. If the test is to be more realistic, then additional traffic will need to be applied to the routers.

This can be achieved by a web server at one end of the link serving pages and another load generator generating requests. It is important that the mix of traffic is realistic. For example, a few continuous file transfers may impact response time in a different way to a large number of small transmissions.

By forcing extra more traffic over the simulated WAN link, the latency will increase and some packet loss may even occur. While this is much more realistic than testing over a high speed LAN, it does not take into account many features of a congested WAN such as out of sequence packets.

Use the WAN emulation facility within LoadRunner.

The WAN emulation facility within LoadRunner supports a variety of WAN scenarios. Each load generator can be assigned a number of WAN emulation parameters, such as error rates and latency. WAN parameters can be set individually, or WAN link types can be selected from a list of pre-set configurations. For detailed information on WAN emulation within LoadRunner follow this link - mercuryinteractive.com/products/LoadRunner/wan_emulation.html.

It is important to ensure that measured response times incorporate the impact of WAN effects both at an individual session, as part of a performance test, and under load as part of a load test, because a system under WAN affected load may work much harder than a system doing the same actions over a clean communications link.

Where is the WAN?

Another key consideration in network sensitivity tests is the logical location of a WAN segment. A WAN segment is often between a client application and it's server. Some application configurations may have a WAN segment to a remote service that is accessed by an application server. To execute a load test that determines the impact of such a WAN segment, or the point at which the WAN link saturates and becomes a bottleneck, one must test with a real WAN link, or a back to back router setup - as described above. As the link becomes saturated, response time for transactions that utilize the WAN link will degrade.

Response Time Calculation Example.

A simplified formula for predicting response time is as follows:

Response Time = Transmission Time + Delays + Client Processing Time + Server Processing Time.

Where:

Transmission Time = Data to be transferred divided by Bandwidth.

Delays = Number of Turns multiplied by 'Round Trip' response time.

Client Processing Time = Time taken on users software to fulfil request.

Server Processing Time = Time taken on server computer to fulfil request.

Try entering in values and clicking on various buttons below to see how various parameters affect response time. Note that this is a simplified model to demonstrate impact of various parameters. Other parameters such as error rates, lost pack rates .... are not included.

Simple Response Time Calculator / Model

Data transfer for transaction (KB)		Press any buttons to update values.
Number of Turns (or resources on web page eg gifs )
Effective Bandwidth (Kbps)
Round Trip Time (ms)
Server Processing Time (ms)
Client Processing Time (ms)		/
		/
		/

Estimated Response Time (in seconds)

If you run ping from your command line, first with a small number of bytes and then with a moderate number of bytes, and enter the results here, the actual values of bandwidth and latency from where you are to the site you pinged will be incorporated into the above table.

	Number of bytes	Time (mS)
Average roundtrip time for small number (eg 48)of bytes. (eg ping -l 48 www.merc-int.com.au)	bytes	mS
Average roundtrip time for moderate number (eg 2048)of bytes. (eg ping -l 2048 www.merc-int.com.au)	bytes	mS

A final word on bandwidth congestion.

Care should be taken when considering the congestion of an existing network link, when attempting to replicate that link in a test environment. Take the example of a site that has four staff. If one of those staff members spent all day downloading stuff from the web, using up all of the bandwidth, then analysis would show a link with high utilization. If however, three staff spent all day downloading files, the line utilization would be much the same, but the available bandwidth of the remaining staff member would be greatly reduced when compared with the first scenario of only one person downloading files.

Determining the effective available bandwidth takes into account this effect of excessive bandwidth demand and should be used in preference to the 'stated' bandwidth.