Performance Testing 104: Workload Modelling Designing & Process

What is Workload?

Performance Test Workload refers to the distribution of load across the identified scenario. Performance tester prepares a workload to simulate the real-world situation in the performance test environment. In performance testing cycle, different workloads are created to study the behavior of the system under various loads and conditions. Sometimes the workload is also known as a scenario.

What is Workload Modelling?

The process of creating a performance testing scenario with the help of non-functional requirements and performance test scripts is known as Performance Test Scenario creation or Workload Modelling.

Purpose of Workload Modelling

Workload Modelling helps to generate the expected production-like conditions in the performance test environment to check the true performance of the application.
Another purpose is that when there are some production performance issues detected in the live environment then workload modelling helps to replicate the same condition to reproduce the issue in the performance test environment.

Importance of WLM (Workload Modeling)

Your performance test results will be more accurate if you manage to properly simulate your production system parameters in your test. It’s the planning phase where the performance analyst makes sure that the information of all the parameters of AUT has been acquired in order to accurately simulate them in the performance test. Identifying AUT workload model is one of the most important parts of the planning activity. Workload model provides the information of what type of user actions will be tested under load, what will be the business scenarios for all the users and what will be users’ distribution on every scenario. This information helps the performance testing teams in many ways such as:

• Performance Scenarios Identification: The fundamental activity of the Workload model is to understand the application and identify its performance scenarios.
• Performance Test SLAs: Performance testing teams translate AUT non-functional requirements into performance test SLAs through workload model.
• Makes Communication Easier: Workload model makes it easy for the performance testing teams to communicate the AUT performance scenarios and users’ distribution on them with all the application stakeholders.
• Test Data Preparation: Workload model helps in identifying the type and amount of test data which is always required before the working on the tool is started.
• Required Number of Load Generator: You always require a lot of infrastructures to successfully conduct the performance testing activity. Incorrect results are produced if the application is tested with inadequate infrastructure. Normally users load is simulated from multiple machines (i.e. load injectors) for accurate testing which is also identified from the Workload model

Approach

The designing of workload comprises of various calculations but the ultimate aim is to achieve the desired goal in various terms. These terms could be the number of users, transactions per seconds, hits per second, throughput etc. Workload model helps to design a predictable (stress test), repeatable (regression load test), scalable (stress/breakpoint test) and sustainable (endurance test) scenario. Before designing a workload model, it is important to collect all the relevant information which helps to create an effective workload model. Such information is available in the Performance Test Plan.

The first thing to check is how many types of test are mentioned in the performance test plan? Each type will have a separate workload model. For example, the user count in the load test is lesser than the user load in the stress test. Hence both will have separate workload model or scenario.

Now, collect the respective metrics for all the identified scenarios. List of workload metrics is:

• Number of Users
• Iteration per hour
• End to End Response Time (Iteration Response Time)
• Number of Transactions
• Pacing
• Think Time
• Hits (Requests) per second
• Throughput
• Individual page response time

Workload Modelling Process

Performance testing is a complex activity as it consists of various phases and each phase has several activities in it. Workload modeling is one of the most important parts of the performance testing activity and it’s not simple by any means. Some of the activities necessary for identifying the performance test workload model are listed in below diagram.

Steps to design an efficient workload model in Performance Testing

1. First step is to have proper NFR gathered and defined for performance project (banking loan application). Performance tester uses these NFRs to create a workload model for each scenario. For a quick reference, let’s take example that we have NFR of one web application and NFR & transaction & user load simulation for this application is like below.
   

   Concurrent no of users & TPH	500 Users & 6500 TPH
   Transaction Load Simulation/User Simulation/distribution	Admin Flow – 4 User / 500 TPH
   	Loan Flow – 50 / 1000 TPH
   	Account Flow – 400 / 3000 TPH
   	Payment Flow – 46 / 2000 TPH

   
   
2. The next step is to get the iterations/transaction per second value.
   

   Script/Flow Name	Total User Count	Iteration Per Hour	Iteration Per Second = Transaction Per Hour/3600
   Admin Flow	4	500	0.138
   Loan Flow	50	1000	0.277
   Account Flow	400	3000	0.833
   Payment Flow	46	2000	0.556
   Total	500	6500	1.804

   
   
3. Now count the total no of transaction present in script for each flow. Let’s take an example that each script has 5 transactions. 
   
   
4. The next step is to find out the end-to-end response time. Here, end-to-end response time refers the time to complete one iteration. Initially, the response time metric is unknown or targeted; rather than actual. Now, performance tester needs to execute each script with one user and get the response time. It could be a chance that response time captured by replaying the script is different than the response time observed during the test. If so then this may impact the expected number of iteration metric. To overcome such a situation, a performance tester needs to plan 1 sanity test to capture the actual response time. If he runs the script without actual response time then the chances of over/under hitting to the server are more.
   

   Script Name	End to End Response Time (in Seconds)
   Admin Flow	10
   Loan Flow	40
   Account Flow	25
   Payment Flow	20

   
   
5. The main purpose of the sanity test is just to get the end-to-end response time. This test could be run without think time or pacing.
   Let’s take example values that performance tester got from sanity is below.
   
   Next step is to calculate the think time. If sanity test was executed with think time then this step is not required. If sanity test executed without think time then we need to calculate the total think time of each script. As a standard practice we usually keep 3 seconds as fixed think time between each transaction. So calculate the think time, follow below formula.
   
   Total think time = Think time * (Total number of transaction – 1)
   As earlier mentioned each script has total 5 transactions. So in this case total think time for each script will be 3*(5-1) = 12 seconds
   
   
6. Now final step is to calculate the pacing for each script. It’s very easy to calculate the pacing by using below formula which is simplified form of famous little’s law.
   Pacing = ((No. of Users * 3600 )/TPH) – (End to End Response Time + Total Think Time)
   

   Script Name	No. of Users	Iterations per second	End to End Response Time	Total Think Time	Pacing
   Admin Flow	4	0.138	10	12	0.94
   Loan Flow	50	0.277	40	12	35.56
   Account Flow	400	0.833	25	12	369.17
   Payment Flow	46	0.556	20	12	28.22