Popular in Course
Popular in ComputerScienence
This 37 page Class Notes was uploaded by Jaden Jakubowski on Thursday October 15, 2015. The Class Notes belongs to CSC 591A at North Carolina State University taught by Staff in Fall. Since its upload, it has received 16 views. For similar materials see /class/223824/csc-591a-north-carolina-state-university in ComputerScienence at North Carolina State University.
Reviews for SPTP
Report this Material
What is Karma?
Karma is the currency of StudySoup.
You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!
Date Created: 10/15/15
An Introduction To Software Agitation Alberto Savoia Chief Technology Officer Agitar Software Inc wwwagitarcom Agitar SOFTWARE Agenda The Developer Testing Revolution 0 Four Types of Tests Software Agitation The Developer Testing Revolution 0 AgileXP methodology are cool Developer testing is part of AgileXP 9 Developer testing is cool Developer Testing Trends 5 years ago Ignorance and resistance Today The Developer Testing Paradox 5 Years from now Common practice Test It s your code and your responsibility Do it for your current colleagues Do it for future generations of colleagues Do it for yourself My View of Unit Tes ng What s a Unit in Unit Testing What s a Unit A single methodfunotionprocedure A collection or related methodsfunotionsprocedures eg a Java class In an ideal world A unit is independent selfsufficient standalone No need to deal with other units fortesting purposes In the real world Most code has lots of dependencies Testing a unit typically involves other units Basic Structure of Unit Tests 1 Setup Create initial state Initialize method parameters Store preexecution values 2 Execute code under test 3 Compare actual results against expected results Partial Correctness Assertions Notation introduced by CAR Hoare in the context of formal verification P S Q If P is true at the time S executes then Q must be true after S completes IntStack s ssize 0 spush42 int val stop val 42 ssize 1 Think of tests as executable PCAs Make P true Execute S Check if Q is true Example public void testlntStackPushTop setup Hake P true IntStack s new IntStaCk execute code under test S spush42 int val stop compare actual vs expected results check Q assertEquals42 val assertTruessize Basic Components of Unit Testing Code under test Test data Several interestin instances of each of the types and objects needed to execute the co e under test Test assertions Booleanvalued expressions built from a dictionary of predicates and various logicalmat ematical operators Four Test Modes Test Data can be specific or general Specific int acctNum 1234 General forall int acctNum acctNum gt 0 Test Assertions can be weak or strong Weak getBalanceacctNum gt MINBALANCE Strong getBalanceacctNum 34432 Weak Assertions Weak Assertions An assertion is considerecl weak if it can evaluate to true even if the aspect of the Implementation that it s testing is incorrect Example of weak assertion after the withdraw operation completes getBalanceacctNum gt MINBALANCE Weak assertions are still useful because they can detect problems when they evaluate to false WAfalse 9 bug Bug 9 WA false Strong Assertions Strong Assertions An assertion is considered strong if it will evaluate to true iff the aspect of implementation that it s testing is correct for the test data used in the test case Example after the withdraw operation completes getBalanceacctNum 34432 SAfalse 9 bug Bug 9 SAfalse Strong Infallible g assgossible to have a faulty implementation even if all strong assertions Consider the following test Bank bank banktotalDeposits 100000000 bankgetBalance1234 10000 bankdeposit1234 5000 banktotalDeposits 10005000 bankgetBalance1234 15000 The 2 strongdass rtiglps can evaluate to true even though deposit has unwante SI e e ects Four Test Modes General Test Data Specific GW GS Test by Contract SW Test by Example 88 Weak Strong Test Assertions Two Basic Types Of Tests Test by Example IntStack s ssize 0 spush42 int val stop val 42 ssize 1 Characterized by Single initial state Singlespecific set of test data Strong assertions Test by Contract IntStack 5 int n ssize lt IntStackMAXSIZE spushn int val stop val n ssize PREssize Characterized by Multiple initial states Multiple sets of test data Strong and weak assertions l Four Modes Exercise Let s create tests for the method int nextPrimeint n Test by Example Specific data Test by Contract General data Test by Example for nextPrimeint n n 0 n 2 n 31 np nextPrimen np nextPrimen np nextPrimen np 2 hp 3 mp 37 n 0 n 2 n 3 np nextPrime 17 mp nextPrime 17 mp nextPrime n np gt 0 isPrime np np 2 1 Test by Contract for nextPrimeint n n gt 0 mp nextPrimen np 2 1 n gt 0 hp nextPrimen isPrimenp np gt n isPrimen np nextPrimen prevPrimenp n Can you spot the bug in this example Test by Contract By combining multiple weak assertions you can create a strong test n gt 0 mp nextPrimen isPrimenp np gt n numOfPrimesBetweenn np 0 4 Modes Summary General Test Data Specific GW GS nextPrimen gt n isPrimemH prevPrimenextPrimen n IsPrImenextPrImen SW SS nextPrime7 gt 7 isPrimenextPrime7 nextPrime7 11 Weak Strong Test Assertions Class Invariants A class invariant is a property that is true of all objects of a given class before and after each public method call Examples for lntStack class size gt 0 size lt MAXSIZE Examples for Employee class hourlySalary gt HRSystemMINIMUMWAGE getManager null getSSNmatches O 93 O 92 O 94 Class invariants are a cheap and powerful testing tool but unfortunately rarely used in manual unit testing The Bottom Line Unit testing is not easy Solid testing effort gt implementation effort 3 4 lines of JUnit for every 1 lines of Java for 90100 code coverage Good set of tests include Specific and general test data combined with Strong and weak assertions general test data strong assertions is best but can be hard to achieve Additional challenges ggdgifficult to create test datatests for situations you did not consider when writing the It s difficult to be aware of all the properties and behaviors of code that you depend on As a result Currently gt90 of unit tests are ofthe testbyexample variety Most unit tests focus on normal conditions and most common paths The Bottom Line The nature and challenges of unit testing make automation and computer assistance a necessity Unit testing tools can help Simplify test creation Test data generation Assertion generation Test execution Test analysis Code coverage Results Unit Testing Tools Testing Frameworks eg JUnit Simplify test creation Automate test execution and reporting Automated Test Generators Automate and accelerate test creation Enable exploratory testing Use unexpected test data Discover method and class invariants Automate test execution and reporting Modern Automated Test Generation Historically ATG 9 Automated Test Data Generation 0 More recently ATG 9 Automated Test Data Generation amp Automated Assertion Generation Automated Assertion Generation Execute Several Times amp Track State Changes Code Tests or Code Discovered Exerciser Invariants Test Generator Regression Help w Bug Test for Likely Detection in Likely Specification Current Version Specification Candidate Assertions Execution 9 Spec Tests via invariants discovery Michael Ernst MIT David Notkin Tao Xie University of Washington North Carolina State University Johannes Henkel Amer Diwan University of Colorado at Boulder TYPE IntStack AXIOMS forall s IntStack i int poppushsistateretval i poppushsistatestate s popIntStackstateretval gt EmptyStackException SAMPLE TEST Stack s spush7 spush9 Sp0p assertTruespOp 7 new Stack Too Much Automation 0 There is such a thing as too much test automation Developer out of the loop 9 tests that verify that the code does what the code does The developer should be involved in reviewing approving and if necessary modify and augment the tests My Automated Test Generation PhHosophy I Automate everything that Tesmummat39on can and should be automated Test Amplification effICIent as pOSSIbIe Software Agitation Making Invariants Detection Idea Practical Automated invariants detection is a brilliant idea but Need an automated way to executedrive the code Must use developerfriendly languageexpressions Must be integrated with IDE to maximize adoption The developer needs to be involved somewhere in the cycle Software Agitation combines invariants detection with Test data generation Automated execution Invariants in programmerfriendly syntax IDE integration Execution 9 Observations 9 Asserts Code Observations on code behavror If an observation reveals a bug fix it T If it describes desired behavior click to promote it to an assertion Agitator Demo Conclusion Early stage deveoperunit testing is critical to software quality Trend OldCurrent pass the buck to QA Future pass the buck to QA after unit testing Unit testing can be as challenging and often more challenging that development lots of Interesting theory and problems New test automation technoo y can developerunit testing eaSIer aster and more thorough Final Word Good test automation should not replace human intelligence creativity and developer participation in the testing process Instead it should help developers focus on the activities that require human intelligence creativity and insig t QampA Contact albertoagitarcom
Are you sure you want to buy this material for
You're already Subscribed!
Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'