New User Special Price Expires in

Let's log you in.

Sign in with Facebook


Don't have a StudySoup account? Create one here!


Create a StudySoup account

Be part of our community, it's free to join!

Sign up with Facebook


Create your account
By creating an account you agree to StudySoup's terms and conditions and privacy policy

Already have a StudySoup account? Login here

Advanced Computer Graphics

by: Aliza Ruecker

Advanced Computer Graphics CSCI 780

Aliza Ruecker

GPA 3.56


Almost Ready


These notes were just uploaded, and will be ready to view shortly.

Purchase these notes here, or revisit this page.

Either way, we'll remind you when they're ready :)

Preview These Notes for FREE

Get a free preview of these Notes, just enter your email below.

Unlock Preview
Unlock Preview

Preview these materials now for free

Why put in your email? Get access to more of this material and other relevant free materials for your school

View Preview

About this Document

Class Notes
25 ?




Popular in Course

Popular in ComputerScienence

This 56 page Class Notes was uploaded by Aliza Ruecker on Thursday October 29, 2015. The Class Notes belongs to CSCI 780 at College of William and Mary taught by Staff in Fall. Since its upload, it has received 25 views. For similar materials see /class/231167/csci-780-college-of-william-and-mary in ComputerScienence at College of William and Mary.


Reviews for Advanced Computer Graphics


Report this Material


What is Karma?


Karma is the currency of StudySoup.

You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!

Date Created: 10/29/15
The Spiral Model of Software Development and Enhancement Barry W Boehm TRW Defense Systems Group 1988 Outline Introduction Previous Models The Spiral Model TRWSP8 Application Advantages and Difficulties Risk Management Conclusions Future of the Spiral Model Discussion A RiskDriven Approach Different idea of software development How does this project affect the developers and the clients How does each step in the project affect its overall development Not used in previous development models Usually codedriven or documentdriven Software Process Model Used to determine the order ofthe stages and to establish the transition criteria What do we do next How long shall we continue doing it Provides guidance between the different phases of a project As opposed to a software methodology Previous Software Process Models 0 An evolution of models Code amp Fix Stagewlse amp Waterfall Evolutlonary Development Transform Model Code amp Fix First elementary model Write code now fix it later No planning involved Problems Code is poorly structured The software developed was usually a poor match for the users needs Stagewise amp Waterfall Born out ofthe shortsightedness ofthe Code amp Fix model need for a design phase requirements phase and a testing phase First used to develop SAGE Semi Automated Ground Environment an early warning system forthe Cold War era Stagewise A development process ofsuccessive hases Phases included operational plan operational specs coding specs coding parame er testing assembly testing shakedown system evaluation Undenlvent two re nements in 1970 Now referred to as the Waterhll Model Waterfall Model Introduced 7 Feedback lumps aeruss multlple stages Valldatlun and verlneatlun steps 7 Prututyplng Vla a pullu ltthee step alungslde pt requlrements and deslgn Dif culties exposed even as revisions were made 0 e model 7 Requlreu elapurateu upeuments Documentrdrlven 7 Led tn pursuan stages ufdevelupment ln the wrung uruer Waterfall Model 62 we 4 Evolutionary Development Evolution ofthe system in directions based on experience Provides rapid initial operational capability I can t tell you what I want but I ll know it when see It Flexible yet uncertain approach Evolutionary Development Problems No formal design phase same problem as Code amp Fix One bad assumption the unplanned paths willquot be compatible Hardtochange code resulted Many problems when new software was incrementally replacing old software The solution to the Evolution The Transform Model contains A formal speci c Transform Model ation Automatic transformation ofthe speci cations 39 0 code An iterative loop for improved performance An outer iterative loop for adjustments Modi cations are made to the speci cations ary Model Transform Model Dif culties arose as in the other models The automatic transformation isn t so easy Unplanned paths still can cause a problem ie the Evolutionary Model s bad assumption A knowledgebasemaintenance problem would result Problem ofchoosing the best option at each transformation point Spiral Model The RiskDriven Approach Spiral Model A different approach born out of the evolution ofthe Waterfall Model Encompasses the previous models as special cases and can make use ofa combination of models Risk analysis asks What are the areas of uncertainty and what is the probability that they will slow the progress of developmentquot RiskAnaysis Prototype DesignNalidatit lanning Alternatives And repeat Meas re of A Typical Cycle u Cumulative CO t and Progress InitiatingTerminating the Process lnitiating the process esize that a particular operational missions can be improved by software effort Test this hypothesis throughout Terminat39ng If a spiral violates its hypothesis then the spiral is termin ted Otherwise it ends with the installation of a new or modi ed software product Cycle Requirements If alternatives or uncertainties are found they must be resolved Riskdriven subsetting allows a mixture of other so ware rocess mo els as necessary until a highrisk situation is resolved 7 Speeifieatieirreirierited Trarisfeirrn DrStagEWlSE 7 Rretetypeerierited Waterfall 7 Auternatietrarisfeirrnatieiri eirierited Trarisfeirrn 7 Simulatieirierierited Evelutieiriaiy Cycle Requirements Each cycle is completed by a review by the people concerned with the project Plans forthe next cycle should be introduced With each succeeding level in the spiral the level ofdetail increases Overlapping Spirals Necessary for Applied to TRWSPS An example of how the Spiral model works in a large system Software Productivity System SPS a group of integrated software development tools for use i h39n TRW as well as for other clients Spiral Model rounds 7 Reurids EDrrESpDnd tei a level iri the splral 7 lri this ease a Reurid El Was eeded tei determlne iriitial feasibility ef the TRWSRS preiieet but is net necessaryfur all preiieets TRWSPS Round 0 7 censiiaims 7 Alternatlves 7 Risks 7 cemmnmem Round 1 7 Alternatlves 7 Risks Risk YESDlutlun cemmnmem Round 2 Round 2 Toplevel requirements spec More detail yet 7 Manrrnonths l ear l5 eo le 7 Objectlves Userrfrlendlysystern lntegratedtools project and personnel support 7 Constralrlts Customerrdellverable SDE rellaolllty e Alternatlves Change ll l os host target workstatlons e Rlsks l rna ch nee s and prlorl leS userrurlfrlerldly system Ul llX performance workstatlon oompatlolllty Mll llrspll al spawned e Rlskresolutlon Survey ofUl39leruSll39lg orgarllzatlorls workstatlm study 7 Rlsk resolutlon results Use Ul llX oased lrlterfaces use tools to su ort earlyphases e Planfor next phase Tools SREM RTTPDL front end support tool N equlprnerlt raollltles e Cornrnltrnerlt Proceed wlth plans gtgtFeatures More Rounds Succeeding rounds may be necessary Depends on the amount of risk Ex The risk alleviation of the R l39l39 preliminary design specification More attractive alternatives may be found Ex The change in UDF tool requirements Spiral Model Features Balances all of the risk elements ie the highrisk elements must be lowered first Offers prototyping as a riskreduction option at any stage of development It allows reworks of earlier stages as more attractive alternatives are identified Detail isn t necessary until detail is needed Round 0 Results The Software Productivity System SPS has grown to over 300 tools and 13 million instructions Over 25 projects have used SPS with overall productivity up 50 most projects have doubled productivity One underestimation of Unix compatibility led to some TRW projects not using SPS Other Models as a Subset ofthe Spiral Model Once certain risk areas are removed other models can replace the Spiral If project is lowrisk in userinter ce and performance requirements but highrisk in budget and schedule Waterfall Model If requirements are stable ie a lowrisk of design and errors produce highrisk Tworleg rnod If project is lowrisk in budget and schedule and high risk in userinterface Evolutlorlary Model If automated so ware generation is available Transform Model 9 Advantages It promotes reuse of existing software in early stages of development Allows quality objectives to be formulated during development Provides preparation for eventual evolution of the software product Eliminates errors and unattractive alternatives early Advantages lt balances resource expenditure Doesn t involve separate approaches for software development and software maintenance Provides a viable framework for integrated hardwaresoftware system development Disadvantages ozoSpiral model not yet complete in 1988 oz Matching to contract software Internal projects have more 39eedom Contract so ware demands total control and a full picture ofthe deliverables in advance oz Relying on riskassessment expertise oz Need for further elaboration of spiral model Milestones and speci cations Guidelines and checklists Risk Management The Spiral model relies heavily on the assessment of risks It provides early identification of the top risk 39tems Improper evaluation of risks may lead inexperienced developers down the wrong path May give an illusion of progress How can a group enhance their risk management skillslevel Software Risk Management Plan 1 Identify the project s top 10 risk items 2 Present a plan for resolving each risk item 3 Update list of top risk items plan and results monthly 4 Highlight riskitem status in monthly project reviews 5 Initiate appropriate corrective actions obvious Conclusions The paper draws four conclusions 1 The riskdriven nature provides adaptability for a full range of software projects The model has been successful in a large application the TRW SPS The model is not yet fully elaborated Even partial implementations of the model such as the risk management plan are compatible with the other process models N at The Spiral Model Today Still a riskdriven model Two definitions One is a c clic approach for incrementally growing a system 5 egree of de nition and implementation while decreasing its degree of risk The other is a set of anchor point milestones for ensuring stakeholder commitment to feasible and mutually satisfactory system solutions Used by the Department of Defense 50002 project The Six Spiral Essential introduced by Barry Boehm in 2000 TestEra A Novel Fra ework for Testing Java Pr ams Sarfraz Khurshid and Darko Mari nov MlT Lab fur Cumputer Sciences invited submissiun tn the Autumated Sumvars Engineering Juurrral Pruceedings Elf the 1B lEEE Cunference urr Autumated Sumvane Engineering ASE 2mm runm Outline Introduction Alloy TestEra Analysis Case Studies Implementation amp Performance Conclusions runm Introduction Problem Manual software testing and test data generation are laborintensive processes Solution Automated testing that will significantly reduce the cost of software development and maintenance TestEra automatically tests Java programs using speci cationbased testing mm m TestEra Uses Alloy to express invariants and prepost conditions Problem manually modeling using Alloy is complicated Solution use Alloy Analyzer to automatically generate test cases runm TestEra Can skip test cases that lead to bugs Generates data structures from structural invariants 7 Test code at concrete data type level 7 Do not need to construct using me had calls mm m Alloy Firstorder declarative language 7 Atoms gt types 7 Relations gt variables 7 Signatures gt userde ned type 7 Functions gt methods 7 Speci cations gt classes Alloy Analyzer fully automatic tool that nds instances of Alloy speci cations runm Linked List Test Cas Alloy Specification uute he open javaprmltlvelnteger header optan uueie fun metnitepoki i acyc11c all n thlsheader next i n 1n nmnext Linked List Test Case Merge Sort sorted all n headernext i some nnext gt ne exte1eni output le permutation of the input 11 i Integer i n headernext i nelem i in header39next39 i nelem39 ii 1 rm 8 TestEra Analysis Two Phases of TestEra o At command line TestEra is given o Phase I generate input using Alloy Analyzer 7 Method deciaratiori e Concretizatiori transiatiori a2 39 Memumem 0 Phase II Java test driver tests each instance 39 MW name ofthe input and check for correctness Parametertypes Ab t t t 2 7 Name of Java Source fiie 7 5 rec 0 rang a 0 j a Memmbuw e it checkfaiis report a counterexampie Ciass iWariaritS e it check succeeds test next inpu PrEPust ecenditiems e Bound on input SiZe 9 two 10 one Case Study Linked List Case Study RedBlack Trees o Ciass invariant amp precondition acvciic o Postcondition acvciic Sortediri ascending order perrnutation or originai iist o No counterexampies initiaiivror mergesort method 0 introduce arr error e Reverse ernpansenrrern lt to gt o Errordetecte outputraiied i or a pustrcuriditiuris Counterexarnpiesreund 11 12 itvvm iunoa Case Study RedBlack Trees 0 Class invariants parent eld must be contained in either a left or right subtree tree must be a BST and must satisfy the four RBT properties 0 TE does not produce any counterexamples on remove method 0 Explicitly create an error by swapping RED and BLACK in restructure command 0 TE detects error and generates counterexamples 10104 Case Study Intentional Naming System INS o Naming system intended for naming and discovering resources in networks of devices and computers 0 Design flaws found in Lookup Name function Wildcards Adding services to database 10104 Case Study Alloyalpha Analyzer 0 Tested two conjectures Instances have distinct basic signatures Atoms in signature are unique 0 Generate simpli ed speci cations then rerun them through Analyzerto see if same specifications generated 0 Two counterexamples found that violated uniqueness properties 10104 Implementation amp Performance mommmucy of addltlan Laokupr ame longmal Table 1 Summary ofTestEra s analyses 10104 Test Cases 0 When TestEra detects a violation ofthe property being tested it generates concrete counterexamples o lfno violation detected can increase confidence in implementation by generating test inputs in a larger space Rule out inputs generated by smaller bounds 0 Can perform test generation on an inputby input basis 10104 Conclusions Several programs were quickly and effectively analyzed for the given small input bounds Violations of correctness property generated concrete counterexamples 0 Approach promises scalability and wide application Computation is not modeled can be arbitrarily complex Plan to extend analysis to report on structural code coverage Help users decide when program has been suf ciently tested 10104 On the Criteria To Be Used in Decomposing Systems into Modules By DL Parnas Outline Introduction Perspective Approaches Example System Modularizations Comparison Conclusion Introduction What is modular programming A wellde ned segmentation ofthe project effort ensures s stem me u arity Eac task forms a separate distinct program module At im Iementation time each module and its inputs and outputs are well delined there is no confusion in the intended interface with other system mo ules tcheckouttimet e integrity ofthe module is tested independently here are few scheduling problems in synchronizing the Finally the system is maintained In mo ular s Ion sy em errors and e ciencies can be traced to speci c modules thus limiting the scope ofdetailed error searchin quot Designing Systems Programs 1970 Introduction Important Points Each task is created as a separate module Each module includes welldefined input and output Each module is tested individually The system is maintained in a modular fashion Introduction Important Points Each task is created as a separate module Each module includes welldefined input and output Each module is tested individually The system is maintained in a modular fashion Introduction Important Points Each task is created as a separate module This was the conventional approach during the early 19705 What was the state of the art at the time Pers pectlve Important Events 1971 UNIX operating system is released 1972 Dennis Ritchie at Bell Labs creates the C programming language Atari introduces Pong video game system Decomposing Systems into Modules published 19 Bob Metcalfe at Xerox PARC invents Ethernet Approaches The functional approach Referred to as conventional The system design can be generated from a flowchart Each step in the flowchart becomes a separate module Approaches The information hiding approach Referred to as unconventional Each module has certain knowledge about the system The interface of each module is designed to hide the inner workings Example System KWIC Key Words In Context The KWIC system accepts an ordered set of lines Each line is an ordered set of words Each word is an ordered set of characters Example System KWIC Key Words In Context Any line may be circularly shifted by repeatedly removing the first word and appending it atthe end of the line The KWIC system outputs a listing of all circular shifts ofall lines in alphabetical order Modularizations How is this broken down Functional Modularization Information Hiding Modularization Functional Modularization Modules Input Circular Shift Alphabetizing Output MasterControI Functional Modularization A MASTER CONTROL p H r CORE MASTER CONTROL A A A lt INPUT 23 ALPHAEEI IZER lt OUTPUT V v V CORE A INPUT Module Reads the data lines from the input medium and stores them in core Packs characters four to a wor Creates an index for the starting address of each line a 53923 gt ltALPHABETIZER 6 V CORE CIRCULAR SHIFT Module Creates an index for the address of the first character of each circular shift Outputs words in pairs to core MASTER CONTROL q MASTER lt CONTROL V SHIFT V CORE A ALPHABETIZING Module Uses data created by Input and Circular Shift modules Produces an alphabetized array in the same format as Circular Shift b MASTER CONTROL 3 lt 038 gt ALPHABETIZER OUTPUT V CORE yr OUTPUT Module Uses arrays created by Input and Alphabetizing modules Outputs list of all circular shifts quot MASTER K CONTROL V INPUT C39RCULAR gt ltALPHAEEHZERgt lt OUTPUT SHlFT v L CORE MASTER CONTROL Module Controls the sequence of the other modules Stores all data shared between modules Information Hiding Modularization Modules Line Storage Input Circular Shifter Alphabetizer Output MasterControl Information Hiding Modularization A CONTROL A A 1 QC39Z igAR ALPHABETIZER OUTPUT 6E STORAGEX V MASTER CONTROL a 5392 ltALPHABETIZER 6 V 39 LINE STORAGE L NE STORAGE Module CHARrwc returns the value for the cth character of the Ith line and wth wor SETCHARrwcd sets the value for the cth character of the Ith line and wth word to the character value for MASTER CONTROL V V yr K LINE STORAGE V LINE STORAGE Module WORDSr returns the number of words in the Ith line DELINE and DELWRD deletes portions of lines or words that are already stored b CONTROL X 1A INPUT lt 038 ltALPHABETIZER OUTPUT V V Vh INPUT Module Reads the data lines from the input medium and calls the Line Storage module to store them quot MASTER K CONTROL V A T lt INPUT 6 ltALPHABUIZERgt lt OUTPUT V v gt LINESTORAGE v39 CIRCULAR SHIFTER Module CSCHARwc Returns the value for the cth character in the wth word and Ith circular shift K CONTROL 4 lt INPUT lt 023 ALPHABETlZER OUTPUT v LINE STORAGE ALPHABETIZER Module ALPH Defines the value for ITH function TH Returns the index of the alphabetically ith circular shift MASTER CONTROL 4 V lt INPUT ll 23 ALPHAEEFIZER Q OUTPUT v V e V A lt LINE STORAGE gt OUTPUT Module Outputs list of all circular shifts MASTER CONTROL Module Controls the sequence of the other modules Comparison Similarities The two modularizations both work and both reduce the problem into manageable pans The algorithms for each may be the same The resulting programs may function the same Comparison Differences Changeability Comprehensibility Changeability What if one of the design choices changed Input format Storing all lines in core Packing characters four to a word Using an index for circular shift versus changing stored data for shifts Alphabetizing the list once Changeability 1 Inputformat Functional Input Information Hiding Input 2 Storing all lines in core Functional All Modules Information Hiding Line Storage Changeability 3 Packing characters fourto a word Functional All Modules Information Hiding Line Storage 4 Changing stored data for shifts Functional Alphabetizer Circular Shift and Output Information Hiding Circular Shift Changeability 5 Alphabetizing the list once Functional Alphabetizer Output Information Hiding Alphabetizer Changeability Functional most changes affect all modules Information Hiding each change affects only one module Comprehensibility Functional Modularization The formats are complicated This makes design decisions more difficult Interactions between module developers is important Comprehensibility Information Hiding Modularization The interfaces are primarily abstract Interaction between module developers is minimized Independent development can begin earlier Decomposing Systems into Modules 39Paper One On the Cnterra To Be Used m Decomposlng systems rnto Modules eebyD L Pamas 39Pap er Two The Modular structure of Complex systems 77 by D L Pamas P c clements and D M Welss Overview Purpose A system design problem 7 KWIC index Conventional approach Unconventional approach Comparison Criteria used in discomposing the system Another system design problem 7 OFF Conclusion Purpose 0 Modularization I a mechanism for enhancing exibility and comprehensibility I Allowing shortening of development time 0 Effectiveness of modularization I Depend on the criteria used in decomposing the system Philosophy of Modular Pro gramming 0 A 1970 textbook by Goutbier and Pont Awelledefmed segmentation othe ro ect effort ensures syst m modulanty Each task forms a separate dlstlnctprogam module At rmplementatron tlme each module and lts lnputs and outputs are welle ned ere ls no confulen in the rntended rnterface wlth other system mpletlon ofseveml tasksbefore checkout canbegln Frnally the system ls malntalned ln modular fashron system errors and de crencres can be trace to speclflc system modules thus llmltlng the scope othe detalled error searchlng the area of Modular Programming 0 The development of coding techniques and assemblers allows one module to be rewritten With little knowledge ofthe code in another mo Modules to be reassembled and replaced Without reassembly of the entire system Primary Goal of Decomposing the System into Modules anagerial Lrttle need fur enmmunreatrnn O Product exibility Changeto one module wlll not cause change m another Comprehensibili w u e system can hehetter unda39stuud Better designed All in all lower cost Example Concepts 0 A KWIC index system is studied 0 Module I Responsibility assignment 0 Modularization I System description I Includes design decisions Some basic concepts used in the analysis I System level decisions Now the KWIC index system How does it work Input 0 Input Accepts an ordered sets oflines I Each line an ordered set ofwords I Each word an ordered set of characters Any line can be circularly shi ed Output I a listing ofall circular shifts ofall lines In alphabetical order How does it work H Cool and Useful Output work All play And no no play And Makes Jack a dullbuy Hn39 wwwntdzllz eanam m quot 39 39 dull buy Makes Jack a akes Jack a dull a dull buy Makes Jack Modularization One Mod I 0 Input 0 Circular shift 0 Alphabetizing 0 Output 0 Master control Mod I Circular Shift 0 Prepares an index giving the starting address of each circular shift 0 Original index of the line in Module I 0 Output of this module saved in core Mod I Output Module 0 Using arrays in Module III andI 0 Generates an output listing all the circular shifm Mod I Input Module 0 Reads the data from the input medium 0 Stores them in core 0 The characters are packed four to a word 0 End of word indicator 0 Index shows the starting address of each line Mod I Alphabetizing 0 Takes as input the arrays produced by previous two modules 0 Outputs an index giving the starting address of circular shifts in alphabetical order 0 Same format as of Module II Mod I Master Control 0 Control the sequencing among the four modules 0 May handle error messages 0 Space allocation 0 And things like that Diagram One Modularization Two Mod 11 0 Line Storage 0 Input 0 Circular Shifter 0 Alphabetizer 0 Output 0 Master Control Looks familiar Mod 11 Line Storage 0 Consists offunctions or subroutines Provides means by which user ofthe module may call on it o Examples CHARrWc saranmpv c a wowso Restrictions in calling the routines o 0 But is it the same as Mod 1 d5 yln N beruflinescunentlystured N h f actersmanywud DEL DELWRD Mod H Input Module Mod 11 Circular Shifter 0 Reads the original lines 0 Functions 0 Calls line storage module analogs of mctions in module 1 0 Have them store internally CS CHAROWC CS S ETU P 0 create a line holder containing all of the circular shifts of lines Mod H Alphabetizer Mod H Output Module 0 Consists of two functions 0 ALPH I Called rst I Alphabetizing TH selves as an index Give the desired printing of circular shifts I ITHi gives the index of the circular shi which comes ith in the alphabetical ordering Mod 11 Master Control Diagram Two aster C ontro Similar to that of modularization one Diagram One Comparison 0 Both will work 0 Algorithms used might be same 0 Identical runnable representation 0 Yet other representation different I understandng Things that may change 0 Input format 0 Decision to have all lines stored in core 0 The decision to pack 4 characters to a word 0 index circular shifts rather than store them 0 Alphabetize the list once Differences Changeability I Input Format change both affect inputmodule If not store all lines in core Not pack four characters a word Some system uses noise function to get rid ofnoise words May use symbol table within the storage module Mod 1 changes needed in every module Mod 11 only affects module one DifferencesChangeability II 0 Index the circular shifts vs store them as such I Mod 11 change only affects circular shi er I Mod 1 alphabetizer output also need to know 0 Alphabetize once I Mod 1 output module can begin a er alphabetizing is done I Mod 11 no change is need DifferencesComprehensibility 0 ModI I need to understand other modules I When understanding one 0 Mod II I Each module has the knowledge of their own I Hiding from other module Difference Independent Development Mod 1 I Interfaces between the modules are complex formats joint effort among several development groups Mod 11 Interface more abstth I Consist in the function names Independent development ofmodules Criteria 0 In Mod I I Each major step is a module I Common approach I Flow chart I Good for system with 500010000 instructions I Insuf cient for larger projects I Big Projecw start with a module guide Criteria Conclusion 0 Mod H Information Hiding 1H 0 Advise against I No longer correspond to steps in the processing I decomposing a system based on owchart I Every module knows its design decision 0 Propose I Other modules don t have this knowledge I Taking the 1H approach I Interface reveals as little as possible I Begin with a list of dif th design decisions I Or design decisions which are likely to change I Each module hides such decision from others Complex System OFP 0 More about information hiding Stands for O boafd Flight Program A project for A7E aircra SW by the Naval Research Lab A realtime program processes ight data Controls displays for the pilot Computes the aircra position Using internal navigation system Must be accumteH 0 An example OFP for A7E aircraft Module Guide Second Level So are Design Mo 1e SM Behavior Module BM 0 Top Level ardware Module Design Principle Criterion We mentioned before 7 1H Likelihood to change System details likely to change independently shouldbe secrets to itself I Interfaces between modules include information less likely to change Data structure I Only accessible to modules inside itself I Others obtain information by calling its access methods Three Ways to Describe a Modular Structure 0 Roles played by individual modules 0 Secrem associated with each module 0 Facilities provided by each module Also called module speci cations HoW to use the module What does the module do Two Kinds of Secrets 0 Primary secrets I Hidden information I Speci ed to the software designer 0 Secondary secrem I Implementation decision I When implementing the module Summary 0 Two system examples 0 Two modularizations 0 TH criterion is advisable 0 Flowchart approach is less desirable 0 Likelihood to change is an measurement of effectiveness of modularization Resources The Modular Structure of Complex Systems7 byD L Pamas P c Clements D M Weiss Enhancing Reusability With Information Hiding 7 by D L PamasPc ClementsD M Weiss httpfxvcl sourceforge netkwicIndex html A http diagram htm mm in iui 39 39 39 html tt quot Palm hnme h DWwwcac 39 39 assignmentZkwicl 9algha doc Special Thanks Help from Dr Coppit Korat Automated Testing Based on ava Predicates Chandrasekhar Boyapati Sarfraz Khurshid Darko Marinov CSci 780 Advanced Software Engineering 11142m3 Outline MulSaw Project Problem Example Algorithm Testing Methods Results TestEra Alloy Analyzer and others Conclusions Discussion 11142m3 2 MulSaw Project MIT Students Goal design and implement tools for checking code conformance Korat TestEra Alloy Annotation LanguageAAL MulSaw Web page Junmm Problem Traditional Testing Manual software testing Test data generation Labor Intensive Software Engineering Utopia cheaper and better testing methods 11142m3 4 Proposed Solution Automated Testing Reduce the cost of development and maintenance Junmm General Idea behind Korat Specificationbased testing Current prototype uses Java Modeling LanguageJML Method Precondition Generate all nonisomorphic test cases Execution Method Postcondition 11142m3 5 Introductory Illustration Method for removing the minimum element from a balanced binary tree Precondition input is a binary tree and it is balanced Generate a balanced binary lies Execute method on all cass Postcondition lt1 Examples nxmnnz 7 uwznnz H x Binary Tree example Examples Binaw Tree l Linked data mums WW We mg WW Number of W m the Heap Arra mm class u e lt mm m 1m m D Arraybased data structure man 9 mm Prlvace Nude rlghc lghc chlld gt pub haulean Epokll lt Unless Inverlanr gt nmmnz m Binary Tree example2 Generate test cass Fin ItIZatIon Public 51am Flnmzallun nBlnaWTveeUm NuMiNudeM FlnlllEllnn l neWFlmllzallunGmaleee class Ob Sel nudes lcvealeobleclsC N de NuMiNude u and NUM7Nude nudes addnulll lselC vuul nudes lvuul m null o Nude lselC slm NuMiNude ll sze NuMiNude lselC Nude lell udes lNude lell in null o Nude lselC Nude vlgm nudes lNude right in nullo Nude velum a l Mostly generated automatically man n Binary Tree example3 Finitization function generated a nonisomorphic trees 5 nBinanTree3 5 lies nBinanTree7 429 trees in lt 1 sec nmmnz x2 Binary Tree example4 Checking correctness of a method removeNode n u Public invariani venom Hciass invanani linvEmavyTvee t Public nuimaijehavinv llspecmcalmn in remove requires ham Hmemndmnn ensues mam ipnammmnn I Publicvmdvemnv Wnde mi method may i Korat translates JML into assertions mm B Binary Tree exampleS Invoke the method with each test case enerated Report concrete counterexample if 39 n ru ntime assertlo ils 1 o m i 11L x o x m o 3 9m 1 5 m m a m n m m n m correctness criteria before and after the method nmmnz m s Algo thm umznm 5 Algorithm Key to test case generation From a Java predicaterep0k or CheckRep enerate a nonisornorphic inpuis Finitization State Space Smrch Nonisomorphism Instrumentation nmmnz ls Finitization There must be a set of bounds for limiting size of inputs Inputs can be from several classes Must know number ofobjecis from Field domain mm n Finitization Java familia ty Korat automatically generates skeleton function from Java code for your class Korat allows nitization to be written in ri nmmnz xx he quotthan u but we new but m xmnmugue mm but M W me new we r Programmer carr speoahze the nnzanm to meet certarrr arterracrr Java State Space ct state space for BmarvTreeB D e Specmed obJecE from thrtrzatrm me BmaryTree three Nodes Creates candda vermr cardme vedu for thrs examp e has 8 he d root srze 3 eft rrchtj 1hspe exam State Space2 Smte space o npuB rs aH possrpxe assrghmerrs to the e ds 4 44 4 4 4144J2v2 12m Possrpxe vames of dei may State Space3 Candwdates can be vahd State Space4 or not vahd hm S earth the State dass and new domarhs are ordered h the c vermr such that each v s an ndex to the e d domah Ihtrah aH va ues are set to 0 Korat seeks to svsternatrcaHv exp ore space SearchZ Korat sets the objects according to the vector values Egtltecutes repOk method on candidate Records the fields accessed during invocation of repOk Generates negtltt vector th rou h backtracking increments field that was accessed last Search3 Change only fields that were accessed Prunes the search space dramatically Does NOT rule out any valid data structure Quality of repOk method determines extent of pruning lllHZEm 25 HimZEN 26 Nonisomorphism Nonisomorphism2 Further optimization of generation I What this means isomorphism is process defined by the root object Definition of isomorphism 39 Candidates Partltl ned by the Tm mnauum a mi on are mummyM in them i i permutation 1r on 0 mt 39 objcru ham 0 m nlycctx mm 0 m iii 1 lt i 5 1 nitii mm Vom e 06 Vf E flk hL ZCD Vp E P o 5 llCllkl 1F0f 0 m 039 m ofmeii39f7rof me nHzma 27 gomorphism criteria nly the smallest from a partition is generate finBinaryTree3 example 1 tree generated per every 3 distinct trees min2cm 25 Instrumentation Korat adds a special constructor identi er eld and get and set methods or any classes in the finitization and repOk nHzma 29 Testing Methods Generation of nitization for methods Small amount of work left for user When Korat cannot determine a eld automatically inserts U er must replace this with the appropriate eld min2cm 3n I Testing Methods Dependent and Independent Parameters Koratindependent generate separately and take cross product Other methodsdependent and independent use the cross product by manually constructing all possibilities 11142nu3 31 Checking Correctness Generate all valid inputs Invoke the method with each input Check the output with the test oracle postcondition or class invariant Results of false also provide concrete counterexample in the form of the input 11142nu3 32 I Checking Correctness Korat combines and adds to the ideas of testing from JUnit and JMLJUnit Adds automation of test cases 11142nu3 33 Results Benchmarks BinaryTree HeapArray LinkedList TreeMap HashSet AVTree 11142nu3 34 Results Due to backtracking Korat can cover the whole state BlnaryTree m HeapArray LmkedLls Results The specifications of correctness for methods were simple containment properties plus the class invariants l cnclumlk llk lliml BinaryTree remove esp ras extract ax munm 1 I lat reverse s n w 11142nu3 35 1 Introduction One of the goals of research in instructional technology is to develop and study new ways of utilizing technology to support effective instruction Recently using the Internet to give learners efficient access to anywhere anytime learning has come to the forefront of this research agenda Although there are several positions taken by instructional technology researchers most share a common focus small digital reusable educational resources This approach is known as Educational software components Roschelle 2001 Sharable content objects ADL 2001 Knowledge objects Merrill 2001 Educational objects Sphorer 2001 Learning objects IEEELTSC 2001 In my work Ihave called these resources learning objects following the IEEE s Learning Technology Standards Committee s terminology defined them as any digital resource which can be reused to facilitate learning Wiley 2000 and have argued that this definition is a significant improvement over the IEEELTSC s definition which fails to exclude any person place thing or idea that has existed at anytime in the history of the universe Wiley in pressa These nonrival educational resources show great promise for helping instructional technologists reach their goal of efficient effective appealing instruction Reigeluth 1999 anywhere anytime However the field is in desperate need of new theoretical frameworks for using these resources In this proposal Iwill argue that current approaches to using learning objects are decontexutalized isolationist and dehumanizing and have failed to yield the educational fruit learning objects are capable of bearing namely enabling high quality anywhere anytime learning environments Therefore I propose to conduct a basic research program directed at devising theoretical frameworks and methodological strategiesfor designing and employing learning objects in online learning environments To develop these new frameworks and strategies I will undertake an empirical program based on ethnographic analyses of existing online learning communities employing the quotpersonactingwithmediationalmeans as the unit of analysis Wertsch 1991 Rather than viewing learning objects as decontextualized artifacts existing independent of learners interactions with them Wertsch s analytic method will allow the research to focus on the ways in which the resources mediate the achievement of online learners goals within their broader historical cultural and institutional context Instead of engaging in new development or community building efforts as part of the research I will carry out the research in the context of existing online learning and work communities including those forming around our Instructional Architect system Recker amp Wiley in press funded under NSF grant DUE0085855 Specifically there will be three broad types of research outputs from the proposed work 1 Development oflearning objects design and utilization frameworks and techniques The first product of this research will be a group of theoretical frameworks and strategies for designing and planning for the use of learning objects To a large extent the exact nature of the new frameworks and techniques must develop as the project progresses However I will employ Hine s 2000 virtual ethnography method in conjunction with Wertsch s 1991 1998 mediated action analytic framework This research will build on my previous work in building taxonomies of learning objects types Wiley in pressa and describing problemcentered possibilities for their use Wiley in pressb 2 Application of the frameworks Second in parallel to their development Iwill apply the frameworks to separate online learning and work communities in order to formatively evaluate their explanatory and predictive power in answering questions including in what ways do learning objects facilitate the accomplishment of goals in online learning and work communities In what ways do they interfere with the accomplishment of these goals What roles does the broader community context play in m leamlng objects seleeted foruse more frequendythan others7 3 othe ernstrng domarns l u learmng obreetswrll d sen NSF39 lerary T T successful deslgn and deployment oflearmng objects In past work Ihave focused on descnblng the types of 00C erey m pressra Th lr 1 2 Why do we need new frameworks varrety rquot w Tum seeuonlwrll argue Tle m w wmlm mm 2 1 Current approaehes uslng leamlng ob eets r rl leamlng object uulrzatron and promdes a summary of other approaehes Due m partto rts own rntemal trammg demands Crseo Systems Inc has beeome amajor player m the onlrne leamlng spaee Bulldlng on ernsung lnstxucnonal deslgn researeh ChuekBanrtt andDeborah Lewrs developed 39 earnt l r 2000 Therr frameworle outllned as follows R L S l l quot c runs 7 3 712 7 I E The RLO Strategy ts hutlt upun the Reusable lnturmattun Obje me An Rlo ts granular reusable LHuHK m n Rlo and cuntent KEer Individual RIOs are then combined to form a larger structure called a Reusable Learning Object RLO A Reusable Learning Object is created by combining an Overview Summary Assessment and five to nine 72 RIOs An RLO is based on a single objective derived from a specific job task Each RIO is built upon an objective that supports the RLOs objective p 24 The RIORLO design process begins by analyzing a task down into its constituent parts and learning objectives are designed from these constituent tasks RlOs are then designed in order to teach the specific objectives and practice and assessments are designed in order to measure student mastery of the objective As evidenced in the graphic from the white paper the R10 is presented in sequence with 4 8 others in order to comprise an RLO or complete learning experience This methodology could provide significant benefits to Cisco s instructional developers by allowing them to reuse existing educational resources much they way software engineers reuse objects in the object oriented programming paradigm The Cisco paper acknowledges work by Mager 1975 Merrill 1983 and Clark 1989 as its theoretical foundations Barritt amp Lewis 2000 212 Summary of learning objects approaches The methods explicated by Cisco and other employers of learning objects are surprisingly similar drawing largely on the same theoretical work from the 1980s or earlier including work done by Mager 1975 Bloom 1956 Merrill 1983 Clark 1989 and others working in behaviorist or cognitivist instructional paradigms The United States Department of Defense s Advanced Distributed Leaming Network Initiative is similarly positioned The following quote from its Sharable Courseware Object Reference Model SCORM39 ADL 2001 specification summarizes the assumptions behind current learning objects approaches Empirical studies have raised national interest in employing education and training technologies that are based on the increasing power accessibility and affordability of computer and networking technologies These studies suggest that realizing the promise of improved learning efficiency through the use of instructional technologies such as computerbased instruction interactive multimedia instruction and intelligent tutoring systems depends on the ability ofthose technologies to tailor instruction to the needs of individuals In contrast to classroom learning these approaches enable the pace sequence content and method of instruction to better fit each students learning style objectives and goals The dilemma presented by individually tailored instruction is that it combines an instructional imperative with an economic impossibility With few exceptions one instructor for every student despite its advantages is not affordable Instructional technology promises to provide most of the advantages of individualized instruction at affordable cost while maintaining consistent measurable highquality content p 1718 There are three significant implicit assumptions in this statement 1 A oneonone instructional model is preferable above others 2 human interaction in large scale learning environments is economically impossible and 3 automation via intelligent instructional systems is the only viable solution to providing anywhere anytime learning The importance of these assumptions cannot be overstated as they color all the design decisions made by participants in the SCORM and other leaming objects efforts eg Cisco NETg Click2Learn sar Dlsappomungly 19801 rum r rh 2 2 1 Decontextuallzedleamlng The mstrucuonal Thls ls true reuse As Ihave demonstratedm prevlous research wth Drs Mtrnr Recker and Andrew Glbbons wuey Recker a Glbbons 2000 leamlng object use ls better descnbed as contextuallzatzon quot Thatrs when an mstrucuonal deslgner or automated system uses alearmng object they are actually placlng the object mto an context mto whlch 1t 15 belng placed determmes whether or not the object flts mto that context The less specl c the mtemal t quot u r l W111 flt quot Flgure l 1 A Less speci c B More specific internal context internal context Figure 1 T ml 1 m contexts vt wt n W H contextrnatchmg rts own m terms of shape and pattern contexts than an entrre lesson on molecular bondlng Instrucuonal soclal contextquot Vygotsky 1981 cultura1hlstonca1 andmstrtuuonal setungquot e g Wertsch 1991 and sltuatedness e g Laye a Wenger 1990 Jonassen 1991 Whlle far transfer lmplylng atype of context Fm Mu tru n w m l l M mm l ulner The slmple contextfor leamlng from Lhe standpamt afsmdent zeammg 2 2 2 Me aphone not medator Leamlng objects are generally deployed as content chunksquot or lnformatzon contamers quot Thatrs they are A or quot distributed learning required were a larger megaphone for the instructor As learning theorists push for more contextualized realworld authentic instruction instructional strategies such as casebased scenarios Schank Berman amp Macpherson 1999 or problembased learning Albanese and Mitchell 199339 Vernon and Blake 1993 have emerged in response When leaming is understood in the context of problem solving learning objects and other resources change from infocapsules that transfer inert knowledge from expert to novice into semiotic tools that mediate and shape the learners actions W ertsch 1985 like the cards in Vygotsky s 1978 interpretation of Leont ev s 1932 forbidden colors task In the forbidden colors experiment subjects were asked to describe a number of items without using the name of any color more than once Subjects were provided with cards corresponding to colors to use during the experiment Many younger subjects were unable to use the cards successfully but older subjects used the cards as tools to mediate their performance of the task39 for example tuming a card face down once its color had been used This tool aspect of learning objects in other words the manner in which learning objects mediate problems solving activities remains almost completely unexplored Wertsch s 1991 call for social science research to focus on mediated action would suggest that neither learners working in online environments or the resources they use in those online environments can be studied fruitfully in isolation Rather than studying learning objects out of context the research unit of analysis must focus on learners actual uses of the objects within a learning context Wertsch 1991 reminds us that Only by being part ofaction do mediational means come into being andplay their role They have no magical power in and ofthemselves p 119 223 Scaling through automation Many individuals and institutions pursue learning objects research with the goal of reaching anywhere anytime learning through computerautomated assembly of leaming objects personalized for individual learners eg Martinez in press Hodgins 200039 IEEELTSC 200139 ADL 2001 And the cost savings of automating instructional design are obvious But while the model of one learner interacting with one computer matches very well with the 1970s view of computerbased instruction refs an isolationist approach is at odds with what modern learning theorists are increasingly emphasizing 7 the importance of collaboration eg Nelson 1999 cooperative learning Johnson amp Johnson 199739 Slavin 1990 communities ofleamers Brown 1994 social negotiation Driscoll 1994 and apprenticeship Rogoff 1990 in learning Even with signi cant pedagogical considerations set aside it seems paradoxical that we would we put hun re thousands or millions of learners in front of advanced communications technology so that they can retrieve data from a supposedly intelligent machine instead of interacting with other people 23 Other practical problems with current leaming objects In addition to the disconnect between current learning objects approaches and current research in teaching and learning there are several practical obstacles to successfully implementing current learning objects models This section describes two such problems 231 Specially designed for reusability While the primary design criterion of learning objectsbased approaches is generally reusability considerations of granularity ie how big the learning object should be and architecture ie the structure according to which the objects should be assembled frequently require designers to reformat all existing content before it can be reused in a given learning objects system For example an existing PDF user manual for a piece of software or hardware may be broken up into several smaller chunks converted into XML and stored in a database Wiley 2000 criticized Merrill s 1999 Instructional Transaction Theory of being particularly guilty of this problem requiring literally every object to be specially prepared and formatted In other words forty some odd terabytes of existing media on the publicly accessible Internet would be unusable without extensive retooling and this is true of other leaming objects approaches as well Cisco NETg Click2Learn SCORM The vast majority of existing digital educational resources could not be reused in current learning objects systems designed to support reusability such as those based on SCORM 232 The reusability paradox Because the primary design goal of learning objects is reusability in a variety of diverse learning contexts learning objects are generally designed in a highly decontextualized manner eg South amp Monson in press Reigeluth and Nelson 1997 have argued that when working with instructional media of any kind educators first deconstruct the materials into component parts in order to reassemble the media according to their individual needs By designing predeconstructed instructional media it is believed greater development efficiency can be achieved as educators bypass the step of personally deconstructing media However Wiley Recker and Gibbons 2001 have argued that extremely decontextualized media are actually more costly and difficult to utilize in instructional development because of a difficulties in indexing extremely decontextualized media for human discovery and use and b computers inability to make meaning and therefore combine primitive media into instructionally meaningful units In the semiotic sense learning objects and other educational resources are signs whether they be text graphics audio animation or otherwise The learning objects user s task of combining individual resources into instructionally meaningful lessons is similar to the speaker s task of combining individual words and utterances into meaningful communication Inasmuch as this is true Vygotsky s 1962 notion of the in ux of sense applies to learning object assembly In language the meanings of words and sentences that proceed and follow an individual word such as the word sense in the proceeding sentence color the meaning of that word In other words proceeding and following utterances significantly alter the meaning of a word or other utterance Vygotsky 1962 wrote The senses of different words flow into one another literally influence one another so that the earlier ones are contained in and modify the later ones Thus a word that keeps recurring in a book or a poem sometimes absorbs all the variety of sense contained in it and becomes in a way equivalent to the work itself Creating a meaningful utterance becomes an act in which words and other utterances with overlapping and contextabsorbing meanings are intermingled to create meaning Returning to learning objects the combination or sequencing of educational resources creates a context in which the resources color and absorb each other s meanings In all but the most basic instructional applications such as drillandkill memorization training computers have no hope of engaging in the type of complicated meaning making required to create meaning full instruction from leaming objects This implies that humans will have to assemble learning objects by hand for all but the most rudimentary instructional content Surprisingly while the most decontextualized learning objects are reusable in the greatest number of learning contexts they are also the most expensive and di icult for instructional designers to reuse 24 Toward a new theoretical framework In summary though there is much to build on there is still a great deal of work to be done in designing theoretical frameworks for the design and utilization of leaming objects For the present project existing approaches must be reexamined in light of current research in teaching and leaming take into account the difficulties with current approaches described above and grounded in accounts of actual use The unit of analysis can be neither the online leamer nor the leaming object itself but must be the personactingwith mediationalmeans Wertsch l99l or more specifically the onlinelearnerproblemsolvingusinglearning objectsastools To build the new framework Iwill begin from my previous research In prior work on my own and with both the Digital Learning Environments Research Group at Brigham Young University and the Reusability Collaboration and Learning Troupe at Utah State University Ihave attempted to describe both ontological and functional aspects of learning objects With partial funding from the Edumetrics Institute Ihave created a taxonomy of learning object types available to instructional designers along with strategies for instructional designers using learning objects in complex domains Wiley 2000 I have critically examined the prevailing metaphor of learning objects the LEGO block suggesting a more robust metaphor built around molecular bonding which captures the notions of fit in learning object contextualization Wiley in pressa With funding from Utah State University Ihave demonstrated a system for the peertopeer sharing of learning objects and built democratic models of object use directly by leamers Wiley in pressb With funding from the Cisco Learning Institute I am analyzing individuals understandings of learning objects and designing a beginning course for persons unfamiliar with the learning object concept Finally in my edited volume The Instructional Use of Learning Objects and in later work I have pioneered the application of constructivist learning theory to learning objectsbased approaches to instructional design Wiley in pressa39 Wiley in pressb This broad base of learning objects research positions me well to carry out the proposed activities I believe that a more rigorous understanding of the ways in which online leamers actually use learning objects that is a more rigorous understanding of the ways that learning objects mediate online learners pro em solving activities will provide significant value to science mathematics and technology education and serve to launch a very productive course of personal research In the next section an example of online leamers using learning objects to solve problems is presente 3 An example of online learners problem solving using learning objects as tools To this point I have asserted that communities of learners who use learning objects to problem solve exist online In this section Ipresent a brief example of an interaction from such a community comment on the example in terms of the obstacles described above and compare the case to current learning objects approaches 31 Example case The following excerpt is taken from the PerlMonks website section titled Seekers of Perl Wisdom PerlMonks is an online community where a variety of people leaming the PERL programming language and working through the problems common to those learning the obtuse language interact and collaborate to solve problems httpwww nerlmnnk nru inde n17node id98134amplastnode id479 Portions of the interaction have been removed in order to preserve space In this example we see a condensed form of the collaborative problem solving process described by Nelson 1999 Problem solving group membership is implied by membership in the community learners negotiate a common understanding of the problem through a series of questions and restatements 0 learners roles in the problem solving are implied as one learner poses the problem and responds with further clarifications thoughts or ideas 0 learners gather information from a variety of sources including PERL modules code samples Postgres output and Postgres documentation a solution is agreed upon and implemented and further questions are raised beginning the problemsolving cycle anew Unique ID by nlafferty on Jul 19 2001 at 1639 nlafferty has asked for the wisdom of the Perl Monks conceming the following question I want to know how to generate a unique id for every row i enter into my postgreSQL DB So that a unique id is specified to a variable which i can INSERT into the row along with my other information Re Unique 1D by lhoward on Jul 19 2001 at 1646 The two easy ways to do this with Postgres One is to specify the 1D col as serial type which will cause it to be autoincrementing ie when you insert a row it will automagically get a new unique val in the serial col Additionally you can use a postgres sequence to keep track of the 1D and query it with postgres39s nextval function whenever you need a new value my sthdbhgtpreparequotselect nextval 39 sthgtexecutequotseqiname my idsthgtfetchrow39 sthgtfinish39 Of coruse you can also do this the oldschool way too but its less efficient to do it this way Have a table to store the last assigned ID When you need an ID lock that table read the value out of it write the value1 back to the table and then unlock it Re Re Unique ID by nlafferty on Jul 19 2001 at 1705 So ifi use a serial type then i do not have to use the sequence as well Or do ihave to use a combination of both Re Re Re Unique 1D by lhoward on Jul 19 2001 at 1728 The serial type creates and uses a sequence behing the scenes but its operation is transparent to you You do not need to create an additional sequence to use it See using psql Postgres39s command line tool test create table fooID serial NOT NULLbar text constraint foo 13k primary ltprogram output removed to preserve spacegt I find postgres sequences to be most useful when you want to share one unique ID across tables and when you don39t have a master table to store it in or don39t have a table that will always be inserted first Or when you want to assign something outside of the DB at least at the time of assignment a unique ID Re Unique 1D by lachoy on Jul 19 2001 at 1700 ObPerl You can also use a wrapper like plug plug SPOPSlink module to do this for you there is example code at this nodelink Chris Mx autobsmode Re Unique 1D by lestrrat on Jul 19 2001 at 1707 If this unique 1D does not have to be any thing in particular you might as well use the oid column oid is unique for every single row in the database and is created when you insert a row It39s sort ofa quothiddenquot field so when you query you have to do SELECT oid FROM table39 ifyou already know your oid SELECT FROM table WHERE oid x39 This is so much easier than maintaining a sequence and is universal for Postgres Re Re Unique ID by nlafferty on Jul 19 2001 at 1714 This is originally how i thought would be a good way to handle this I39ll give it a shotthank you 39 Re Re Unique ID by nlafferty on Jul 19 2001 at 1936 So how would i do a delete statement WHERE oid quotoid 7 32 Discussion of the example case In this section the example case is discussed in terms of the research context established above 321 Learning objects Thinking about the case in terms of learning objects it immediately becomes apparent that educational resources are generally defined in an unnecessarily narrow manner Some digital content such as Internet accessible PERL modules sample code and the Postrges database s user manual are referenced and discussed Also the discussion itself as captured on the web board may be seen as a learning object in the traditional sense This utilization agrees well with the notion of leaming objects as content nuggets However the Postgres database program itself is also used as an educational resource in the dialog above39 specifically relevant output from the software is recorded and shared for use in the problem solving process Additionally when stepping back the researcher is prone to notice that the website software itself is mediating the problem solving process by taking questions and responses displaying these in an easy to navigate threaded manner etc These environmental affordances are important to consider in defining the mediational boundaries The first takeaway from this examination is that our conception of learning object or mediator must be broadened from the traditional concept of small information chunk 322 Problems relating to current research in teaching and learning 3221 Decontextualized learning The various resources employed in the problem solving process in the example are not highly decontextualized chunks of content presented in a simple sequence The resources are highly contextualized in their own right e g an entire database user guide but more importantly they are held together by the context provided by the problem solving activity For example links to PERL modules appear with a contextdependent explanation of why the resource is relevant to the current problem As opposed to a LEGO metaphor in which the resources connect directly into a meaningful structure these objects are used like bricks held together and made meaningful by a contextual mortar39 specifically the prefacing and proceeding material around each resource provides scaffolding context that supports the leamers meaning making 3222 Megaphone not mediator While some of the resources in the case are simply information dumps many of them are functional directives that nlafferty can employ in his own problem solving activity Additionally the resources are not just presented as static information They serve as foci of discussion stimulating new ideas and mediating the thinking of the group 3223 Scaling through automation The most significant departure of the example above from conventional learning objects approaches is that it relies on human beings to locate assemble and contextualize the resources Althoug tragedy of the commons Hardin 1968 would suggest that such voluntary collaborations are not sustainable over time the emergence of the Internet and speci cally the Free or Open Source Software movements have shown peertopeer communications technology s ability to put people in symbiotic you answer my question I ll answer yours relationships The gift culture described by ethnographers of the FreeOpen Source movements such as Raymond 1999 and Himanen 2001 is one explanation of this phenomenon lhave argued elsewhere that a distributed expertise model obtains in sufficiently large distributed learning communities meaning that because expertise exists across the community no individual community member is burdened with primary responsibility for answering questions or providing feedback As problems arise related to the expertise of an individual that individual may or may not choose to provide help lfthe community is of sufficient size as PerlMonks and many other online communities are the distribution of expertise and effort provides timely problem solving support without unduly burdening any individual Wiley in pressb 323 Problems relating to the practical use of learning objects 323l Specially designed for reusability The learning objects employed by learners in this example have not been taken apart and made to conform to a standard grain size Neither have they been forced to conform to an architectural standard which requires special JavaScript hooks to allow the objects to communicate with a learning management system Existing resources have been reused just as they existed previously without any special preparation 3232 The reusability paradox Finally while there is no direct evidence that the learners who volunteered leaming objects and explanations of their relevance to the given problem had an easy time locating and using the learning objects given the voluntary nature of the community and lacking any indication of extrinsic motivation we may assume they did so 33 omnari on of the sample case and current learning objects l have argued above that current learning objects approaches suffer from a number of practical and pedagogical diffrculties However when reviewing the example case in light of these various criticisms none seemed to apply It would appear that learning object use in the wild that is educational resource use unmarred by instructional design and development methodologies exhibits none of the weaknesses of contrived approaches The sample case includes a large number of learners yet scalability in terms of teacherbandwidth is not an issue Leamers are provided with meaningful learning support anytime anywhere yet the case is rich with humantohuman interaction Learning objects are successfully embedded in a meaningful leaming context but the discovery and use of the objects is done by humans without over burdening any individual ie wit out exhibiting scalability problems It is because these naturally occurring methods seem superior to existing approaches that I believe the research plan outlined in this proposal to be so significant both to the field at large and to myself as a launching point for my career 4 The plan of work The goal of the proposed work is to devise new theoretical frameworks and slralegiesfor designing and using learning objects As lhope to have suggested in previous sections the work has the potential to contribute significantly to basic research in instructional technology and specifically science mathematics and information technology teaching and learning If we can improve our understanding of the ways in which learning objects can be effectively designed and used significant improvements in the quality and availability of online education may be effected In line with the goals of the CAREER program Ihave in mind a longterm effort Therefore to a certain extent the plan of work must develop over the period of the grant However the broad shape of the research is clear I know the context for the research and I can build off of promising prior work In this section of the proposal I will lay out the proposed research activities 41 Research Context As stated above Iwill carry out the work in the context of existing online leaming communities rather than first working to create new communities This is advantageous both because the research will hit the ground running and because any learning community I designed would be colored in a way making it inappropriate for grounded research The following online leaming communities have been identified as candidates for the pr0ject 0 The Linux Kernel project an international project in which programmers are collaborating to research and implement the core kemel of a completely free Unixlike operating system httpwwwtuxorglkml and httpwwwkernelorg 0 The Gnome Project an intemational project in which programmers are collaborating to research and implement a completely free windowed desktop environment for Unixlike operating systems httpwwwgnomeorg 0 Slashdot a large webbased community 30000 registered in which people with interests in information technology developments can track recent events ask questions on a variety of technologyrelated topics and receive answers httpslashdotorg 0 Perl Monks a webbased community in which programmers collaborate to help each other leam to use and solve problems involving the PE 1 39 anguage httn39 nerlmnnk my 0 Microsoftnewsgroups in which users of Microsoft software collaborate to help each other learn to use and troubleshoot various Microsoft products including operating systems productivity applications and integrated development environments for a number of programming languages These communities may not seem like traditional online learning communities because they are not They are communities of problem solvers learning in real world contexts mediated by a variety of online resources and communications tools As such they are appropriate targets for the current research Additionally they are all communities of information technology leaming While the main goal of the research is developing new theoretical frameworks for leaming object design and use the selection of information technology communities will likely yield important insights into information technology learning The Department of Instructional Technology at Utah State University one of the top five instructional technology programs in the country provides an ideal setting for the proposed project In addition to myself three of the most respected and active academics in the nation in the area of leaming objects are on faculty Drs David Merrill Andrew Gibbons and Mimi Recker We four and a number of graduate students comprise the Reusability Collaboration and Learning Troupe a research group focusing on learning objects research with existing partnerships with the University of Califomia under NSF DUE0085855 the Cisco Learning Institute under contract Microsoft Research the Utah Educators Network a statewide K12 organization and others The department and research group provide access to top quality graduate students research partners and public schools 42 The core empirical work The core empirical efforts will be based on techniques similar to those used in the examination of the sample case above though much more thorough Initial data gathering will be accomplished through an ethnographic approach modified for use in online settings Hine 2000 While acknowledging the host of methodological problems involved with online ethnographic research Hine describes her approach as adequate for the practical purpose of exploring the relations of mediated action p 65 The five years will be broken into three phases During Phase I which will last for the first two years of the project the empirical and analytical techniques will be employed and refined appropriately In Phase II which will run for the third and fourth years the revised empirical and analytical techniques will employed again during a second iteration of study and analysis Finally during Phase III year 5 analyses will be finalized and published See Table l for the project timeline Research Activities qucation Activities Phase I 0 Selection of two online learning 0 Integrate data gathering analysis and Years 12 communities framework building into courses as 0 Ethnographic study of selected appropriate communities 0 Implement new framework within 0 Analysis of data following Wertsch courses 1991 0 Integrate evaluation of new 0 First draft of new framework framework into courses as appropriate 0 Article writing 0 Develop digital library of exemplary resource use cases Phase II 0 Selection of two online learning 0 Iterate Phase I activities Years 34 communities 0 Ethnographic study of selected communities 0 Analysis of additional data following Wertsch 1991 0 Second draft of new framework 0 Article writing Phase III 0 Finalize new framework 0 Iterate Phase I activities Year 5 0 Article writing Table I Timelinefor the proposedprojecl 421 Phase I Years 1amp2 First iteration and improvement of techniques During Phase I I will select two online learning communities for ethnographic study Because the communities to be studied are at least two years old with public full archives Iwill begin by analyzing the existing record of interactions analysis techniques are explained below I will then join the community as both researcher and participant fully disclosing my intentions Herring 1996 This realtime engagement will allow targeted questions regarding others participation broader interviews and a general responsiveness not available through document review Boundaries of the communities will not be assumed a priori but examined over the course of the study Again following Hine 2000 I will also draw on my own online mediated problem solving experience during the research as a data point I will re ect on the appropriateness and utility of the techniques employed during Phase I and plan Phase II research activities accordingly 422 Phase II Years 3amp4 Second iteration Phase II will involve the same empirical work as Phase I using two additional online leaming communities and an improved data gathering methodology 423 Phase III Year 5 Final analysis and publication In Phase III I will draw on data from the first two cycles to finalize the framework Significant time will be devoted to writing papers and planning future research 43 Analytic techniques As mentioned above Wertsch s 1991 1998 mediated action framework will be employed in the analysis of the ethnographic data This involves focusing on the personactingwithmediationalmeans as opposed to either the learner or the learning objects The lenses through which the analysis proceeds are those of voice dialogicality social languages speech genres and privileging Voice includes considerations of the speaking personality and the addressee of the speaker s utterance Dialogicality involves the manner in which one speaker s concrete utterances come into contact with or interanimate the utterances of another Wertsch 1991 p54 Social languages deal with styles of discourse particular to a specific stratum of society professional age group etc within a given social system at a given time Holquist amp Emerson 1981 p 430 for example thejargon of physicists In contrast to social languages speech genres are characterized not by social strata of those who employ them but the typical social situations in which they are employed for example the language of the formal science classroom Finally privileging concerns choices persons make among mediational means when alternatives are available for example why communicate via a web board when one learner could email another directly The results of these analyses will be used as inputs into a method for creating coherent instructional design models Wiley 2000 1 have previously extended Nelson s 1998 instructional design theory development method and used the resulting methodology fruitfully with approaches to using learning objects in complex domains Wiley 2000 The analysis will generally proceed as follows mediated interactions within the qualitative data are examined for patterns or categories through the lenses described above Categories are constructed linked and densified Corbin amp Strauss 1998 much as factor analytic data are rotated Finally the resulting categorical data are used as inputs into the instructional design model creation methodology Through the instructional design model building method the categories of learning object design and use strategies identified in the analysis are brought into a coherent whole as a group of methods Along with these methods statements of the model s overall goals preconditions for use and values are articulated Finally the goals preconditions values and methods of the model are brought into a coherent form ready for peer review and publication 44 Educational activities 1 see research as completely integral to my teaching While 1 am just beginning my position as Assistant Professor this fall 1 have spent the year since graduation as a Postdoctoral Fellow under NSF DUE0085855 my prior teaching in Education and Computer Science and learning experience has shown that integrating research activities into learning provides additional context and motivation that greatly enhance the effectiveness efficiency and appeal of education Within the context of the project I feel that 1 can positively in uence a number of learners in the following ways 0 Student apprenticeships Wherever possible lwill involve students as research apprentices in the proposed project This includes PhD and Masters students on paid assistantships funded by the grant as well as undergraduates potentially serving in research capacities 0 Project as class research context The proposed project will also provide a research context for projects in various classes lwill teach For example in fall 2001 lwill teach a course on designing instruction using learning objects and in spring 2002 1 will teach a course on qualitative research methods The learning objects course will be centered around reviewing and critiquing existing approaches to learning object use as well as creating exemplary models of learning objectsbased instruction The qualitative methods course will focus heavily on Wertsch s among other interpretivist analytical methodologies Both courses provide natural spaces in which students can engage in research in the course of their class experience 0 Sample framework implementations As the new framework takes shape 1 will design portions of courses andor entire courses around the framework as appropriate These implementations will allow students to engage re exively in their research as they learn within the environment they are researching Additionally these implementations will provide important formative feedback to the framework development process Wiley 200039 Nelson 1998 0 Digital library ofexemplary resource use cases As exemplary situations in which leaming objects are used to mediate problem solving and other learning are discovered within the online community records these will be stored in a digital library and annotated in light of the emerging framework This collection will provide teachers and leamers interested in various aspects of the framework to see exemplary cases of various portions of the framework in action 45 Role of the advisog board For this project I have assembled an advisory board of five members Dr Laurie Nelson Indiana University Dr David Merrill Utah State University Dr Mimi Recker Utah State University Dr Brent Wilson University of Colorado Denver and Dr Marcy Driscoll Florida State University The advisory board will meet annually during the summers between each academic year of the grant four times This will provide me an opportunity to present current work gather feedback and suggestions and generally improve the quality of project work 46 Project organization and management I will be responsible for overseeing all aspects of the research The staff will include three graduate students during the five years of the project These graduate students will help with all aspects of the research including data gathering analysis framework building digital library building publishing and presenting research at conferences 5 Summary At the beginning of this proposal I asserted that the main interest in learning objects was around their ability to facilitate anywhere anytime learning I argued that several organizations have attempted to meet this goal through a process of decontextualization isolation and automation I then presented an example of leaming objects used to facilitate learning in a realworld problemsolving environment rich with context and human interaction I suggested that this environment is capable of scaling to extremely large numbers of learners and has in fact already done so the community Slashdot has over 30000 registered users Ibelieve that a more rigorous understanding of the ways in which learning objects function in these existing interaction rich communities could provide the basis for instructional strategies with the potential to revolutionize online learning Specifically the research will shed significant light on the role of learning objects in mediating problem solving within the information technology leaming domain Last but not least the project will provide solid grounding to a lifelong research agenda dedicated to understanding the effects and potential benefits of technology on learning 6 Summary of research accomplishments My prior research has included most aspects of learning objects research including work on metadata concepts and standards W ason amp Wiley 2000 nonauthoritative metadata as a method of capturing learning objects use contexts Recker amp Wiley in press metadata search interfaces and algorithms Wiley 1999 fundamental learning objects concepts Wiley South Bassett Nelson Seawright Peterson amp Monson 1999 as well as a taxonomy of learning object types and detailed directions for their design and implementation Wiley 2000 My research projects have been favorably mentioned by The New York Times MIT Technology Review The Economist and other news media My work with learning objects is widely recognized in the field of instructional design and technology I have written a number of invited articles and chapters on the topic Wiley in pressb in pressc39 in pressd39 Recker amp Wiley in press Wason amp Wiley 2000 in addition to writing one of the first dissertations Wiley 2000 on the topic and editing the first book Wiley in press about leaming objects In addition to publishing several noninvited pieces e g Recker Walker amp Wiley in press I am active in several professional organizations and have presented leaming objectsrelated research at several conferences including AERA Internet2 AECT Cleanroom Process Model pmde Chaman Singh Verma 24w October 2cm Outline gt Introduction gt Philosophy gt Hocess y Success Stories Discussions Traditional Software Development At Work Cleanroom Cleanroom gt Manufacturing areas in microelectronics pharmaceutical and biotechnology allow only zero tolerant contamination in the room gt Everything is suspected even your own body air condi ns fans your shoes even your formal L clo gt Everyone has to pass through strictly controlled cl ambers to make sure that everything is de contaminate BugCentered Computing gt Defects are inevitable gt Defect removal are part of so tware development l39uiiumiml rm Are bugs in software unavoidable gt Bugs are symptoms of mismanagement and unpredictable environment gt It is difficult to eliminate them but good software engineering can control their insatiable desire for reproduction Let us understand how Cleanroom Process accomplishes that Cleanroom Philosophy Avoid dependence on costly defectremoval process bby writin code increments RIGHT F0 FIRS TIME and verify their correctness before testing ormal specifications and design Corr tness verification Statistical testing The theoretical foundations of cleanroom are F Notice bug removal is absent because they are not supposed to appear in Cleanroom Cleanroom gt Team correctness verif cation takes the place of unit testing and debu in Software enters m testing directly with no execution by development team All errors are accounted for from first execution on with no private debugging permitted Cleanroom process chart Quality Comparison gt Comparison is meaningful only at the time of rst execution gt Traditional software exhibit 2535 or more errors per KLOC at the time of unit testing gt Cleanroom process demonstrated 23 errors per KLOC in million lines of code This num er represent all the errors measured from first ever execution through test completion Quality Comparison gt Errors left behind by Cleanroom correctness verification are not because of complex design or interface errors They are simple errors which are easy to find and fix by statistical testing Cleanroom An incremental approach gt The cleanroom process enforces developing and certifying a pipeline of software increments that accumulates into the final system gt The increments are developed and certified by small independent teams with teams of teams for large projects Incremental approach gt System integration is continual and functions grows with the addition of successive increments gt Any future increment are predefined by the existing system It minimizes the interface and design errors gt Develop high quality software with Quick and Cleanf approa h and new version is released which incorporates new requirements from users experience Cleanroom Requirements are paramount gt A cleanroom teams analyzes and clarifies customer requirements to the minute details after lots of interactions and feedback gtIn case ofdoubt a team can develop a rapid prototype to get more feedback from the users Cleanroom process activities Cleanroom development involves two cooperating teams and five major activities gtSpecification yIn cremental planning gt Design and verification gtQuality certification gtFeedback Specification Functional Specification gt Developed by development team gt De nes the required external system behavior in all circumstances of use gt It is the basis for incremental software development Usage Specification gt Developed by certi cation tam h De nes all possible usage scenarios and their probabilities both correct and incorrect gt It is the basis for generating test cases for statistical tjsting and quality certi cation Incremental Planning gt On the basis of speci cations both development and certification teams define initial plans for developing increments y Break the project into small pieces according to their sizes and complexities gt Paralleldevelopments are allowed Design and verification gt The development team carries out design and correctness veri cation cycle for each increment gt The certi cation team work in parallel using the usage speci cation to generate test cases that re ect on the expected use of accumulating increments Quality Certification gt Periodically the development team integrates a ompleted increment with prior increments and hand over to the certi cation team gt Certi cation team executes the entire software using statistical test cases The results are checked against the functional speci cation gt Inlerfail time ie the elapsed time between failures are pass to a quality certi cation model that computes objective statistical measures ofquali gt The quality certi cation model employs a reliability grthh estimator to derive statistical measures ofquality Quality Certification gt Contrary to other models cleanroom certifications are done continuously over the life of the project gt Higher level increments enters the certi cation first and therefore major architectural and design decisions are validated in execution f re development teams work on t em gtSince certi cation is done for all increments as they accumulate higher level are subjected to more testing than the lower level increments Feed back gt Certification team passes errors to the development team If the quality is low then the managers and team initiate process improvement gtAt every level of increment we have assured quality and intellectual control Functional Specification gtThe cleanroom process is built upon function theory where each program is treated as rules for mathematical functions subject to stepwise refinement and verification gtCleanroom uses Box Structure specification and design model 1 Black Box 2State Box 3Cear Box 2 Box Structure Three principles that govern use of box structure 1 All data defined in a design is encapsulated in boxes 2 All processing is de ned by using boxes sequentially or concurrently 3 Each box occupies a distinct place in a system s usage hierarchy Black Box m gt A black box is a precise speci cation of extern uservisible behavior in all possible circumstances of its use gt The black box transition function H 399 R WhereS is set of all possible stimulus SH A set of stimulus history ilk A set of response which depends on S 39 5H Black Box gtThe objective of blackbox speci cation is to the responses produce for even possible stimulus and stimulus histo including erroneous and unexpected stimuli gtBy de ning behavior in terms ofstimulus histories blackbox speci cations neither epen on nor prematurely de ne design internals Black Box To record large specifications classes of behavior are grouped into neste tables State Box gt Transition function sos9 RNS Where OS Old State NS New State gt This state is same as Black Box but all the histories are referenced by old state gt From the 00 point of view this state encapsulate s te data and services on that data Clear Box gt The transition function i l s SOS9RNS by procedJre gt A clear box is simply a program that implements the corresponding state box gt A clear box may invoke black boxes gt A clear box allows defining new objects or extensions to existing ones Box Verifications gt Each state box is verified with respect to the39 black boxes and clear boxes with respect to their state boxes Bo cture bring correctness verification to object architecture Correctness Verification gt Basic control structures in programming are sequence alternation if thenelse and iterations whiledo gt Ever control structure is veri ed in turn by all the tams The requirement for unanimous agreement results in software at has fewer or no de ects y Functional veri cation is superior than unit testing because unit testing checks only a few test paths out of many possible paths but a function checking requires only one check gt Most functional veri cations conditions can be checked in a few minutes but uni 53 take substantial time to prepare execute and chec Quality Certification Statistical approach is adopted as there are too many tests to perform How long on an average will a software product execute before it fails Sampling All possible executions correct and gt incorrect 2 Measurement Measure the quality by determining if the executions are correct Extrapolation Extrapolate the quality of sample to the population of possible execution Correction Identify and correct aws in the development process V tquot E 5 Quaity Certification gt User Centric Statistical usage testing focuses n the way users intend to use the software gt It is external behavior testing not the internals of design and implementation gt Cleanroom testers should have good knowledge about the usage of the software and not how it was developed Test case generation gt Test cases are generated from UPD which simulates users experience with the product Since it requires only UPD producing them is mechanical and automation is possible gt For large systems usage grammar or Markov chains can be used to generate test cases Usage Probability Distribution Focus on usage of the system It defines gt All usage patterns and conditions correct and incorrect with the probabilities of occurrence gt There sources are specifications interviews with prospective users and the pattern of uses in the previous version gt For largesystems UPD are often recorded in formal grammars or Markov chains for analysis and automatic processing Certain lowprobability but highrisk failure can be given higher Importance Statistical testing gt Highrate errors are responsible for nearly 23 of software failures even though they comprise less than 3 of total errors Because statistical testing amount to testing softWare the way users will use it highrate errors tend to be found first gtTraditional coverage testing find errors in random order it is less effective than statistical testing Embracing Changewllh Extreme Programming By Kent Beck Outline mom none m m oMMWmmm onne ommmm onnonn wanna oanm The Need Waterfall Model anemonnnnwn cmwmstnm ommmmwwmwm wnnmnNMwmmm Waterfall Model oThe steps are not followed lolng Plase lnleglallon leoeolanoe Deslgn Coolng Tesllng Tesllng lnleglallon Tesng ll lll ll 25 Codng 89 703 no Deslgn 493 34st lUll Incremental Development oWwWa mnmmwn mntm MmmmWWWMMs onenmoonnnonoo mommnmnnny Incremental Development oStncet proceeds in small steps it is easier 0 incorporate change Change in requirements Changeinlunctionality Changein deadline oil is ready for surprises t t Next Step1XP oExtreme Programming takes the idea otincremental development to the next level oPlan analyze and design a little at a time oXP has 12 or 13 simple rules The Rules The Planning Game oSchedule sma tasks to he completed during the current iteration oProgrammers will locus their attention on the tasks at hand oList of tasks is updated regularly Small Releases ell functional system is produced alter a ten months oSystem is released before the whole problem is solved oNew releases regularly daily to monthly lletaphor oUse metaphors to describe how the system should work oThese analogies express the functionality of the system Simple Design vThe code should pass all tests and fulfill certain functionality while maintaining No duplicate code Fewest possible classes and methods Say everything once and only oncequot Tests viests are continuously written with the system with tests are run together at every step vCustomers write tests that will convince them the system works vDon t proceed unti current system passes ALL tests Relaotoring vThe code may be changed atany time to provide Pair Programming with programming is done with two coders at the same machine Simplilication thrillll vThe programmersmust share one deduced redundalcl mouse keyboard screen etc Continuous integration Collective Ownership vNewly nished code is integrated immedia ely vSystem is rebuilt from scratch for every addition vNew system must pass all tests or new code is discarded with workers can access any of the code vllny programmer can change any part of the system it an opportunity for Improvement exists OnSite Customer oAt least one customer is always present ofhis customer is avaiiahie futime to answer questions about the system 40 HourWeehs oConsecutive weeks of overtime is not allowed ofhe need for overtime is a symptom ofadeeperprohlem Open Workspace Work on computers set up in the midde of a large room with cubicles around the edges Coding Standards oAgree upon standards forcoding stpies oPromotes ease of understanding and uniformity Just Rules ofheserulesarejustrules oXP teammates agree to him add the rules oAn agreement can he made to change the rules Must address side effects of rule change The Game XP How it Works Development Cycle ommmmmmmmmm mWwWWe oMmmmmmmmmm MWWWW Development Cycle MMWWMWWWW mememmemm memmmmmwmr WWW 9MMWMWWMm memenemmew Stories Stories wmmwm OWWWMWWMW MWWW em WWWWMWW ewe mmm emmememx m wmmmmwmmmw We Release eWMmmwmmwm mmmmmmmm WmMmmMmMMWWm mMMWWWMe WWW Mmmmmmmmm Release mmemmwwmmem wmmmmmmw wmeMM emmmmmmm mmmm Iteration ammonWanmom EMmmmnn oWMmmwmmwm WWMWWMM oamaanamooonoo one Iteration wmmmmmmmamn emmwmnm It one coder has a smaller time frame he chooses more tasks nmmmmmmmwm onnmommmmnnm mmwwm Tasks MMMMWWMWm WMmr nmmmmmnmmnr mews oMWWWWMmW rm nmmnmnmmomMm nmm Test HM WmMMT ommmwmmmmmn mmMmmmnmo Hononwmnommnn Test manMmemm mewmn omMMWMWM ommnnnmnnnn WWWWWMWM oWWmMmmmm WWMMWMMWW HandlingProblems Underesiimaiion wwwm mmnummmm Wan WWMMemmwan MW mnemnnmwnn nmmmmwne 0ther stories will be finished later Uncooperaiire Customers 6mmwmmm mmwme oneenmn emanmmwmnw dmmmmwmmnmn nmnnnnmnmnmng Mi Turnover HmnnmneMmewmi nwmmnmmmnnm Mm HmMMmemn inmmnmmnnmm nnmnmnwmnnm mmemwmmdnwmmmee Changing Requirements enmnmmnnnnmn newnmnmnn nmnwmmnmnnn Mnmmmwnmm eMM mememn name SueeessSiories Chrysleis Payroll System ewnwmmnwwn anMR 6mnmmmmnmmeR nmmnmwnmmn mmmmmn


Buy Material

Are you sure you want to buy this material for

25 Karma

Buy Material

BOOM! Enjoy Your Free Notes!

We've added these Notes to your profile, click here to view them now.


You're already Subscribed!

Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'

Why people love StudySoup

Jim McGreen Ohio University

"Knowing I can count on the Elite Notetaker in my class allows me to focus on what the professor is saying instead of just scribbling notes the whole time and falling behind."

Kyle Maynard Purdue

"When you're taking detailed notes and trying to help everyone else out in the class, it really helps you learn and understand the I made $280 on my first study guide!"

Bentley McCaw University of Florida

"I was shooting for a perfect 4.0 GPA this semester. Having StudySoup as a study aid was critical to helping me achieve my goal...and I nailed it!"


"Their 'Elite Notetakers' are making over $1,200/month in sales by creating high quality content that helps their classmates in a time of need."

Become an Elite Notetaker and start selling your notes online!

Refund Policy


All subscriptions to StudySoup are paid in full at the time of subscribing. To change your credit card information or to cancel your subscription, go to "Edit Settings". All credit card information will be available there. If you should decide to cancel your subscription, it will continue to be valid until the next payment period, as all payments for the current period were made in advance. For special circumstances, please email


StudySoup has more than 1 million course-specific study resources to help students study smarter. If you’re having trouble finding what you’re looking for, our customer support team can help you find what you need! Feel free to contact them here:

Recurring Subscriptions: If you have canceled your recurring subscription on the day of renewal and have not downloaded any documents, you may request a refund by submitting an email to

Satisfaction Guarantee: If you’re not satisfied with your subscription, you can contact us for further help. Contact must be made within 3 business days of your subscription purchase and your refund request will be subject for review.

Please Note: Refunds can never be provided more than 30 days after the initial purchase date regardless of your activity on the site.