generic a pi test tool
DESCRIPTION
Generic a Pi Test ToolTRANSCRIPT
-
Generic API Test toolBy Moshe SapirAlmog Masika
-
AgendaTesting practice in the industryThe Automatic API test tool conceptAttributes supplied to the developerTest coverageDesign and implementationTool demonstration
-
Testing practice in the industryTesting software module requires deep understanding of the modules inner logic and its interfaces semanticsCurrently, dedicated tests are written to each software componentUsually the tests do not try to retrieve functional information from the modules codeChanges in the software logic are resulted in changes in the test automation logic
-
Automatic API Test Tool Concept Enable the developer to add metadata to the modules codeUse .NET attributes to represent the metadata The metadata should contain information which will enable automatic testing of the moduleCreate automatic test tool that given the module and attributes can generate tests
-
Attributes supplied to the developer Function level attributesInput-Output Assertion:Assertion is defined by three expressions
If P is true before the function is run, then Q is true after the function is run, or one of the exceptions in L is thrown during execution For double Sqrt(double x) the developer will define:P: x > 0Q: ret ^ 2 = xL = {DivisionByZero }
-
Attributes supplied to the developer Function level attributes (cont)Input Parameters attributeProvide set of interesting assignments to the methods parametersFor double tan(double x) the developer will define the following assignments:X = piX = 0X = pi/2X = -1234.45*pi
-
Attributes supplied to the developer Class level attributesClass InstancesProvide set of interesting instances of the classFor class Int32 the developer will define the following instances:01-12^31(-2^31) + 112345-780
-
Attributes supplied to the developer Class level attributes (cont)Class InvariantsDefine set of Boolean expressions that should be true after and before executing any member function on an instance of the classFor Account class:class Account {string Owner;int Balance; }
The developer will define:Owner != string.EmptyBalance >= 0
-
Test Coverage Method level coverageGeneral testRun the method as many times as possible, using parameters supplied by the developer. If no parameters were supplied, try to generate the them
Input Output Assertion testFor each defined on the method, try to find an assignment that satisfies P, run the method and validate Q and L.
-
Test Coverage Assembly level coverageFunction call compositionTry to build a call graph of the functions in the assembly and generate the call paths. Validate input-output assertions and class invariants during the calls:For the classes:A, B, C, D And the functions:M1: A x B -> A M2: B x C -> D M3: A x D -> A Build the following graph:
-
Design and implementation Functional decompositionTest Attributes- Attributes that are available to the assembly developer.User Interface- Retrieve from the user the input required to run the test and to present him the test results. Test Engine- Responsible to run the tests and produce the test results.Results Reporting: Responsible to save the results in persistent file and redirect them to the graphic user interface.
Tester User Interface
Test Engine
Result Reporting
Test Attributes
-
Design and implementation Functional decompositionData flow between the different components:
User Interface
Test Parmeters
Result Reporting
Test Engine
Test Results
Processed Results
-
Test Engine Design: The Algorithm design pattern
+Run()
-TestedMethod
FunctionRunner
TestMatrix
+Generate() : TestMatrix
MatrixGenerator
+ExecuteTest()
-TestParameters-MatGenerator : MatrixGenerator-FuncRunner : FunctionRunner
TestExecuter
-TestedMethod
MethodTester
-TestedClass
ClassTester
-TestedAssembly
AssemblyTester
GeneralMethodTester
InputOutputAssertionsTester
ConstCorrectnessTester
1
1
1
1
-Generates
1
*
-Consume
1
*
ObjectBehaviorTester
FunctionCompositionTester
+Execute(in TestParameter)
-ExecutersList-TestFinishedEvent
TestEngine
1
*
-
Test Attributes Hierarchy
System.Attribute
MethodAttribute
-InterestingInstances
Instances
ClassAttribute
-Q-P-L
InputOutputAssertion
-AssingmenstList
InputParameters
-IsConst : bool
Const
-InvariantList
Invariants
-MethodCallSequence
ObjectBehavioure
-
Design and implementation User Interface Structure
-TestEngine
CmdLineParser
-TestEngine-TestResultsBox-AssemblyTreeView
MainWindow
AssmeblyTreeView
TestResultBox
TestParametersDialog
1
1
1
1
1
*
TestEngine
1
1
1
1
User Interface Component
Test Engine Component
-
Test execution sequence diagram
MainWindow
InputOutputAssertionTester
Execute(TespParams)
TestEngine
ExecuteTest
ExcelReporter
UIReporter
HandleTestResults(TestResult)
HandleTestResults(TestResult)
UpdateFile
UpdateUI
MatrixGenerator
Generate
FunctionRunner
Run
SetTestParams(TesParams)
-
Demonstration . . .
-
Conclusions.NET attributes can be used to keep information that will enable automatic testing of softwareThe technique we have shown is limited to platforms that allow introduction of Meta data in the binary code (.NET, Java) The Syntax of defining expressive test attributes can be error prone (mainly due to the use of strings as function identifiers)The ability of such tools to verify large scale software is still to be investigated. Currently is seems can be used as first tire testing (BVT).