Testing Concepts

Software Testing

"Primary role of testing is not demonstration of correct performance, but the exposure of hidden defects."

- G. J. Myers

Software Testing

n It is not sufficient to demonstrate that the software is doing what it is supposed to do.

n It is more important to demonstrate that the software is not doing what it is not supposed to do.

Traditional Definitions

n Establishing confidence that a program does what it is supposed to do

n Confirming that a program performs its intended functions correctly

n THESE DEFINITIONS ARE INCOMPLETE AND DESCRIBE ALMOST THE OPPOSITE OF WHAT TESTING SHOULD BE!!

What Testing Actually Is

n Identifying the differences between expected and actual results

n Process of executing a program (or part of it) with the intention or goal of finding defects

Principles Of Software Testing

n Testing is not something done as a formality at the end of development; it is done throughout the SDLC to detect defects as early as possible and prevent defect migration to subsequent phases

n SOFTWARE TESTING =

n Verification (Static Testing)

n +

n Validation (Dynamic Testing)

Verification And Validation

n Software V & V is a disciplined approach for evaluation and assessment of each software product during each phase of the life cycle

n Methods: Reviews, walkthroughs, inspections and testing

n Objective: to detect and remove defects as early as possible during the life cycle

Verification

n The process of evaluating a system or component to determine whether the product of a given development phase satisfy the conditions imposed at the start of that phase.

n Focus is on Interim deliverables eg. Requirement doc, Design doc etc.

n Method: Reviews, walkthrough & inspection

Verification

n A ‘filter’ applied at various points during the SDLC to ‘purify’ the product as it progresses through various phases

Validation

n The process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements

n Focus is on the executable

n Method: Testing

Test Methods

Black box testing (functional testing):

n Test data derived from the specifications

White box testing (glass box testing):

n Test data derived from the program code, logic

Black Box Testing

n Ignores the internal mechanism of a system or component

n Find out circumstances in which program does not behave according to its specifications

Black Box Testing

n Test Case Design Methods:

n Equivalence Partitioning Method

n Boundary value Analysis

n Error guessing

White Box Testing

n Testing that takes into account internal mechanism of a system or component

White Box Testing

n Statement coverage

n Decision coverage

n Condition coverage

n Multiple condition coverage

n Path coverage

Levels Of Testing

n Unit/Component Testing - at individual component/module level.

n Integration Testing - conducted by integrating the components / modules

n System Testing – Conducted on complete system

n Acceptance Testing - conducted by customer

Unit Testing

n Testing of a individual component or module

n Validates that the individual component performs as designed

n Deliverable: Software units ready for testing with other components

Integration Testing

n Testing of a related components or modules to evaluate interaction between them

n Validates that multiple parts of the system perform according to the system design

n Test interfaces between the components / modules i.e. data is passed correctly to one another

System Testing

n Testing conducted on a complete, integrated system to evaluate the system’s compliance with its specified requirements

n Includes load test, volume test, performance test, security test, usability test etc.

Acceptance Testing

n Formal testing conducted to enable a user, customer, or other authorized entity to determine whether or not to accept the system

n Can be Alpha testing (at developer’s site) and/or Beta testing (at customer’s site)

“V” Model

n Life Cycle testing: CONTINUOUS testing throughout the SDLC

n Goes hand-in-hand with formalized system development process

n Need to plan the testing activities parallel with the SDLC phases

The V Model

Testing During Requirements

n Verification of requirements definition is extremely significant.

n Requirements must be:

n Adequate

n Clear, complete and consistent

n Unambiguous

n Feasible

n Testable

Testing During Requirements

n Requirements must include constraints, performance criteria, quality attributes, etc.

Testing During Design

n Develop test plan, test schedules, testing strategies, test methods, test teams

n Analyze the design for errors

Testing During Design

n Aspects to be checked:

n Missing test cases

n Faulty logic

n Module interface mismatch

n Data structure inconsistencies

n Erroneous input / output

n User interface inadequacies

n Correctness of decisions, conditions along all paths

n Inconsistencies with requirements and high level design

Testing During Coding

n Static analysis: code walkthroughs / inspections execute the code

n Conducted by peer or experienced developer

Examples Of Defects

n User gives wrong / incomplete requirements

n Analyst interprets requirement incorrectly

n Requirements are not recorded correctly

n Incorrect design specs

n Incorrect program specs

n Errors in coding

n Errors in testing: falsely detect an error / fail to detect existing errors

n Mistakes in error correction

What Is a Test Case

n Documentation specifying inputs, predicted results, and a set of execution conditions for a test item

What Is a Good Test Case

n High probability of finding a defect

n Not redundant

n Cover specifications

What Is a Test Plan

n A Document describing the scope, approach, resources, and schedule of intended test activities. It identifies test items, the features to be tested, the testing tasks, who will do each test, and any risks requiring contingency planning

Contents of a Test Plan

n Scope

n Objectives

n Assumptions

n Resources

n Schedules

n Environments

n Risk plan

n Tools

n Training plan

What Is a Defect

n Producer’s view: Deviation from the specifications

n Customer’s view: Anything that causes dissatisfaction, whether in requirements or not

Defect Reporting

n RECORDED FOR 4 MAJOR PURPOSES

n To Correct

n To Report Status Of The Application

n To Gather Statistics

n To Improve Software Development Process

Defect Attributes

n Defect ID

n Short description

n Status

n Severity

n Priority

n Program/Module name

n OS/Browser

n Detail description

n Steps to reproduce

Defect Attributes

n Tested by

n Assigned to

n Snap shots, links, logs

n Comments

Severity And Priority

n Severity: The degree of impact that a defect has on the operation of a system

n Severity is decided by Tester

n Priority: The level of importance assigned to a defect

n Priority is decided by Customer

Defect Resolution Process/Defect Status

• New
• Open
• Fix
• Retest
• Reopen or Closed

Types of Testing

n Load testing: Stimulate virtual users

n Volume testing: To determine whether the program can handle the required volume of data, requests, etc.

n Stress testing: To identify peak load conditions at which the program will fail to handle required processing loads with in required time spans

Types of Testing

n Performance testing: To determine whether the program meets its performance requirements.

(Performance means speed, accuracy, or memory usage with which a given function must be performed)

Types of Testing

n Usability testing (Human factor testing): To identify those operations that will be difficult or inconvenient for users

n Compatibility testing: Testing whether the software is compatible with other elements of systems with which it should operate eg. Browsers, O.S.

Types of Testing

n Regression testing: Retesting a previously tested program after modifications to ensure defects haven’t been introduced as a result of changes made

n Unit regression testing

n Regional regression testing

n Full regression testing

Automated Test Tools

n Functionality testing: Win runner, Rational Robot, Silk test

n Load testing: Load runner

n Test management: Test director, Rational test manager

n Defect management: Test director, Rational clear quest

Testing Never Ends…