I have been able to capture AJAX calls using the http proxy server and the recording controller within Jmeter. You might have to go through the XML to update specific values such as UID's but it works great.
Here are some steps:
1. Create new test plan
2. Under the Test Plan, add Config Element > Http Request Defaults
- Enter server name and port number
3. Under Test Plan, add Config Element > HTTP Cookie Manager
4. Add Thread Group under Test Plan
5. Under the Thread Group, Add > Logic Controller > Recording Controller
6. Test Plan > Add > Listener > Aggregate Report
7. Under Workbench, Add > Non-Test Element > HTTP Proxy Server
- Port 9090
- Target Controller: Thread Group > Recording Controller
- Patterns to include: Click Add then enter “.*”
8. Under HTTP Proxy Server, Add > Timer > Gaussian Random Timer
- Set Constant Delay Offset (in milliseconds): ${T}
9. Bring up the FireFox browser and under Tools/options/advanced tab/network tab/settings button/”Manual proxy configuration”
- Set 127.0.0.1 port 9090
- Enable checkbox: “Use this proxy for all protocols”
10. Bring up HTTP Proxy Server, then click Start.
11. Now simply bring up FF and perform operations you wish to be recorded into the recording controller.
12. When complete, click the “Stop” button on the HTTP Proxy Server.
13. Verify that operations were recorded on the “Recording Controller”.
Tuesday, 29 April 2014
Key differences between QA, QC and testing
Quality Assurance (QA) - A set of activities designed to ensure that the development and/or maintenance process is adequate to ensure a system will meet its objectives.
Quality Control: A set of activities designed to evaluate a developed work product. These activities can always include the audits conducted by the QA team to assess the cost of correcting defects, documentation etc.
Testing: Finding defects by executing a system/software. (Note that the "process of executing a system" includes test planning prior to the execution of the test cases.)
Quality Control: A set of activities designed to evaluate a developed work product. These activities can always include the audits conducted by the QA team to assess the cost of correcting defects, documentation etc.
Testing: Finding defects by executing a system/software. (Note that the "process of executing a system" includes test planning prior to the execution of the test cases.)
Mechanics of writing test cases
Mechanics of writing test cases
Analyzing requirements
To write a good test case, a tester needs to understand the requirement. In what context the requirement is described and what needs to be tested and how. What must be the expected result etc?
Writing test cases (test designing)
A test case is developed based on the high level scenarios, which are in turn developed from the requirement. So, every requirement must have at least one test case. This test case needs to be wholly concentrated on the requirement.
For ex: Lets take yahoomail.com, in this website, the requirement says that username can accept alphanumeric characters. So, the test case must be written to check for different combinations like testing with only alphabets, only numeric and alphanumeric characters. And the test data what you give for each test case is different for each combination. Like this, we can write any number of test cases, but optimization of these test cases is important. Optimize exactly what all test cases we need and what not.
Executing test cases (test execution)
Once all the test cases are written, they need to be executed. Execution starts only after the testing team receives the build from the development. The build is nothing but the new code, which has been developed as per the project requirements. This build is tested thoroughly by executing all combination of these test cases. Please don’t be in a myth that we write test cases after the development is ready with the build. Development and testing has to go parallel. Remember, test designing is done purely on the available valid documentation. While executing test cases, there will always a possibility that the expected result can vary from the actual result while testing. In this case, it is a defect/bug. A defect needs to be raised against the development team, and this defect needs to be resolved as soon as possible based on the schedule of the project.
A test case is identified by ID number and prioritized. Each test case has the following criteria:
- Purpose - Reason for the test case
- Steps - A logical sequence of steps the tester must follow to execute the test case
- Expected Results - The expected result of the test case
- Actual Result - What actually happened when the test case was executed
- Status - Identifies whether the test case was passed, failed, blocked or skipped.
- Pass - Actual result matched expected result
- Failed - Bug discovered that represents a failure of the feature
- Blocked - Tester could not execute the test case because of bug
- Skipped - Test case was not executed this round
- Bug ID - If the test case was failed, identify the bug number of the resulting bug.
How to test SRS
Do you know “Most of the bugs in software are due to incomplete or inaccurate functional requirements?” The software code, doesn’t matter how well it’s written, can’t do anything if there are ambiguities in requirements.
It’s better to catch the requirement ambiguities and fix them in early development life cycle. Cost of fixing the bug after completion of development or product release is too high. So it’s important to have requirement analysis and catch these incorrect requirements before design specifications and project implementation phases of SDLC.
How to measure functional software requirement specification (SRS) documents?
Well, we need to define some standard tests to measure the requirements. Once each requirement is passed through these tests you can evaluate and freeze the functional requirements.
Let’s take an example. You are working on a web based application. Requirement is as follows:
“Web application should be able to serve the user queries as early as possible”
How will you freeze the requirement in this case?
What will be your requirement satisfaction criteria? To get the answer, ask this question to stakeholders: How much response time is ok for you?
If they say, we will accept the response if it’s within 2 seconds, then this is your requirement measure. Freeze this requirement and carry the same procedure for next requirement.
We just learned how to measure the requirements and freeze those in design, implementation and testing phases.
Now let’s take other example. I was working on a web based project. Client (stakeholders) specified the project requirements for initial phase of the project development. My manager circulated all the requirements in the team for review. When we started discussion on these requirements, we were just shocked! Everyone was having his or her own conception about the requirements. We found lot of ambiguities in the ‘terms’ specified in requirement documents, which later on sent to client for review/clarification.
Client used many ambiguous terms, which were having many different meanings, making it difficult to analyze the exact meaning. The next version of the requirement doc from client was clear enough to freeze for design phase.
From this example we learned “Requirements should be clear and consistent”
Next criteria for testing the requirements specification is “Discover missing requirements”
Many times project designers don’t get clear idea about specific modules and they simply assume some requirements while design phase. Any requirement should not be based on assumptions. Requirements should be complete, covering each and every aspect of the system under development.
Specifications should state both type of requirements i.e. what system should do and what should not.
Generally I use my own method to uncover the unspecified requirements. When I read the software requirements specification document (SRS), I note down my own understanding of the requirements that are specified, plus other requirements SRS document should supposed to cover. This helps me to ask the questions about unspecified requirements making it clearer.
For checking the requirements completeness, divide requirements in three sections, ‘Must implement’ requirements, requirements those are not specified but are ‘assumed’ and third type is ‘imagination’ type of requirements. Check if all type of requirements is addressed before software design phase.
Check if the requirements are related to the project goal.
Some times stakeholders have their own expertise, which they expect to come in system under development. They don’t think if that requirement is relevant to project in hand. Make sure to identify such requirements. Try to avoid the irrelevant requirements in first phase of the project development cycle. If not possible ask the questions to stakeholders: why you want to implement this specific requirement? This will describe the particular requirement in detail making it easier for designing the system considering the future scope.
But how to decide the requirements are relevant or not?
Simple answer: Set the project goal and ask this question: If not implementing this requirement will cause any problem achieving our specified goal? If not, then this is irrelevant requirement. Ask the stakeholders if they really want to implement these types of requirements.
In short requirements specification (SRS) doc should address following:
Project functionality (What should be done and what should not)
Software, Hardware interfaces and user interface
System Correctness, Security and performance criteria
Implementation issues (risks) if any
Conclusion:
I have covered all aspects of requirement measurement. To be specific about requirements, I will summarize requirement testing in one sentence:
“Requirements should be clear and specific with no uncertainty, requirements should be measurable in terms of specific values, requirements should be testable having some evaluation criteria for each requirement, and requirements should be complete, without any contradictions”
Test Case Template
| test case id | test case name | test case desc |
test steps |
test case status | test status (P/F) | test priority | defect severity | ||
|
step |
expected |
actual |
|||||||
JMeter-First Load Test
JMeter proxy step-by-step
This short tutorial attempts to explain the exact steps for JMeter's proxy. For those new to Jmeter, one easy way to create a test plan is to use the Proxy. What the proxy does is it
records the requests sent to the server. Jmeter's proxy currently does not support recording HTTPS. Since HTTPS is secure, the proxy is unable to decrypt the traffic and record the request parameters or cookies. There are several ways to get around this limitation.
Basic Proxy Instructions
1. start jmeter with jmeter on windows and jmeter.sh on unix
2. select “test plan” on the tree
3. right click on the “test plan” and add a new thread group: add -> thread group
4. select the thread group
5. right click “add -> config element -> Http Request Defaults”
6. Protocol – enter “HTTP”
7. Server name – enter “jakarta.apache.org”
8. Path – leave blank
9. Port number – enter “80”
10.select workbench
11.right click on workbench and add the Http proxy: add -> non-test elements -> Http Proxy Server
12.Port field - Enter “9090”
13.Target Controller – click on the drop down and select “test plan > thread group”
14.Click the “add” button in “Patterns to include”. This will create a blank entry.
15.Enter “.*\.html”
16.Click the “add” button in “Patterns to exclude”. This will create a blank entry.
17.Enter “.*\.gif”
18.Click the “start” button at the bottom
19.Start Internet Explorer, but do not close Jmeter.
Note: make sure the include and exclude patterns are correct. Below are some common patterns for images and types of pages.
.* - all
.*\.png – png images
.*\.gif – gif images
.*\.jpg – jpeg images
.*\.php
.*\.jsp
.*\.html
.*\.htm
.*\.js
As a general tip, it is a good idea to set the homepage for your browser to a blank page. This way, it reduces the number of unwanted pages Jmeter records during the session. It is a good idea to try the proxy with different sites and get comfortable with different filtering patterns.
20.From the tool bar, click “tools -> internet options”. This should bring up the options.
21.Select the “connection” tab
22.Click “lan settings” button near the bottom.
23.On the connections tab, check “Use a proxy server for your LAN”. The address and port fields should be enabled now.
24.Address – enter “Localhost” or the IP address of your system
25.Port – enter “9090”.
26.Click “ok” button
27.Click “ok” button again. This should return you to the browser
28.In the “Address” bar at the top, enter “http://jakarta.apache.org/jmeter/index.html” and hit the “enter” key.
29.Click on a few links on Jmeter's page.
30.Close internet explorer and bring up the Jmeter window.
Reviewing the Test Plan
Expand the thread group and there should be several samplers. At this point, the test plan
can be saved as is. If you forget to add default http request settings, you will have to
manually delete the servername, and port. In this sample, there aren't any default request
parameters. If a particular request parameter is required by all pages, the request defaults is where one would add the entries.
1. Select “thread group”
2. Right click “add -> listener -> aggregate report” to add an aggregate listener. The aggregate listener will show some basic statistics.
3. Select “thread group”
4. Number of threads – enter “2” OR any digit that you like.
5. Ramp up Period – upon your choice
6. Loop count – enter 10
Running the Test
At this point, we are ready to run our test plan and see what happens. Save the test plan.
When you're ready to run the test, there's two ways:
1. Run -> Start
2. CRTL – R
Before you start the test, select “aggregate report”. As the test runs, the statistics will change until the test is done. At the end of the test, the aggregate report should look like this. While the test is running, in the upper right-hand corner, there should be a green square. When the test is done, the box should be grey.
2. select “test plan” on the tree
3. right click on the “test plan” and add a new thread group: add -> thread group
4. select the thread group
5. right click “add -> config element -> Http Request Defaults”
6. Protocol – enter “HTTP”
7. Server name – enter “jakarta.apache.org”
8. Path – leave blank
9. Port number – enter “80”
10.select workbench
11.right click on workbench and add the Http proxy: add -> non-test elements -> Http Proxy Server
12.Port field - Enter “9090”
13.Target Controller – click on the drop down and select “test plan > thread group”
14.Click the “add” button in “Patterns to include”. This will create a blank entry.
15.Enter “.*\.html”
16.Click the “add” button in “Patterns to exclude”. This will create a blank entry.
17.Enter “.*\.gif”
18.Click the “start” button at the bottom
19.Start Internet Explorer, but do not close Jmeter.
Note: make sure the include and exclude patterns are correct. Below are some common patterns for images and types of pages.
.* - all
.*\.png – png images
.*\.gif – gif images
.*\.jpg – jpeg images
.*\.php
.*\.jsp
.*\.html
.*\.htm
.*\.js
As a general tip, it is a good idea to set the homepage for your browser to a blank page. This way, it reduces the number of unwanted pages Jmeter records during the session. It is a good idea to try the proxy with different sites and get comfortable with different filtering patterns.
20.From the tool bar, click “tools -> internet options”. This should bring up the options.
21.Select the “connection” tab
22.Click “lan settings” button near the bottom.
23.On the connections tab, check “Use a proxy server for your LAN”. The address and port fields should be enabled now.
24.Address – enter “Localhost” or the IP address of your system
25.Port – enter “9090”.
26.Click “ok” button
27.Click “ok” button again. This should return you to the browser
28.In the “Address” bar at the top, enter “http://jakarta.apache.org/jmeter/index.html” and hit the “enter” key.
29.Click on a few links on Jmeter's page.
30.Close internet explorer and bring up the Jmeter window.
Reviewing the Test Plan
Expand the thread group and there should be several samplers. At this point, the test plan
can be saved as is. If you forget to add default http request settings, you will have to
manually delete the servername, and port. In this sample, there aren't any default request
parameters. If a particular request parameter is required by all pages, the request defaults is where one would add the entries.
1. Select “thread group”
2. Right click “add -> listener -> aggregate report” to add an aggregate listener. The aggregate listener will show some basic statistics.
3. Select “thread group”
4. Number of threads – enter “2” OR any digit that you like.
5. Ramp up Period – upon your choice
6. Loop count – enter 10
Running the Test
At this point, we are ready to run our test plan and see what happens. Save the test plan.
When you're ready to run the test, there's two ways:
1. Run -> Start
2. CRTL – R
Before you start the test, select “aggregate report”. As the test runs, the statistics will change until the test is done. At the end of the test, the aggregate report should look like this. While the test is running, in the upper right-hand corner, there should be a green square. When the test is done, the box should be grey.
Monday, 28 April 2014
Establish QA Process
1. CURRENT SITUATION
The first thing you should do is to put what you currently do in a piece of paper in some sort of a flowchart diagram. This will allow you to analyze what is being currently done.
2. DEVELOPMENT PROCESS STAGE
Once you have the "big picture", you have to be aware of the current status of your development project or projects. The processes you select will vary depending if you are in early stages of developing a new application (i.e.: developing a version 1.0), or maintaining an existing application (i.e.: working on release 6.7.1).
3. PRIORITIES
The next thing you need to do is identify the priorities of your project, for example: - Compliance with industry standards - Validation of new functionality (new GUIs, etc) - Security - Capacity Planning (You should see "Effective Methods for Software Testing" for more info). Make a list of the priorities, and then assign them values of (H)igh, (M)edium and (L)ow.
4. TESTING TYPES
Once you are aware of the priorities, focus on the High first, then Medium, and finally evaluate whether the Low ones need immediate attention.
Based on this, you need to select those Testing Types that will provide coverage for your priorities.
5. WRITE A TEST PLAN
Once you have determined your needs, the simplest way to document and implement your process is to elaborate a "Test Plan" for every effort that you are engaged into (i.e.: for every release).
For this you can use generic Test Plan templates available in the web that will help you brainstorm and define the scope of your testing:
- Scope of Testing (defects, functionality, and what will be and will not be tested).
- Testing Types (Functional, Regression, etc).
- Responsible people
- Requirements traceability matrix (match test cases with requirements to ensure coverage)
- Defect tracking
- Test Cases
6. DURING AND POST-TESTING ACTIVITIES
Make sure you keep track of the completion of your testing activities, the defects found, and that you comply with exit criteria prior to moving to the next stage in testing (i.e. User Acceptance Testing, then Production Release)
Make sure you have a mechanism for:
- Test tracking
- Reporting
Subscribe to:
Posts (Atom)