1、Techniques for Validating the Security Quality of Infrastructure Software,John D. McGregor johnmccs.clemson.edu,Outline,Motivation Proposed strategy Detailed actions Conclusion,Theme Life Cycle Threats,There is not a means for automated testing of large software, both static and mobile code, to dete
2、ct, identify malicious code, sleeper codes, and exploitable vulnerabilities and to determine and understand the potential impact on the life-cycle of the codes. Current testing approaches are largely manual rather than automated. CSIIR Workshop Themes document,A Recommended Strategy,Software securit
3、y vulnerabilities are often caused by defective specification, design, and implementation. require that software be designed with security at the very heart of the design process Establish a security verification and validation program to evaluate different software development processes and practic
4、es for effectiveness in producing secure software. Certify those processes demonstrated to be effective for producing secure software.Security Across the Software Development Lifecycle Task Force,Recommended Practices,Statistical testing - Usage based testing permits valid statistical estimation of
5、quality with respect to all the executions not tested and tends to find any remaining high-failure-rate defects early. Production testing - Two strategies are testing security functionality with standard functional testing techniques, and risk-based security testing based on attack patterns and thre
6、at models. A good security test plan (with traceability back to requirements) uses both strategies. Process models - organizations can use the goals and attributes defined in process models as high-level guides for defining and improving their management and engineering processes in the ways they fe
7、el are most appropriate for them.Security Across the Software Development Lifecycle Task Force,SEI,The security of a software-intensive system is directly related to the quality of its software1. Over 90% of software security incidents are caused by attackers exploiting known software defects. Analy
8、sis of 45 e-business applications showed that 70% of security defects were design defects. Experienced and capable software engineers inject, on average, one defect every nine lines of code. A one million line of code systems typically contains 1,000-5,000 defects when shipped.1 http:/www.sei.cmu.ed
9、u/tsp/tsp-security.html,A Final Source,“One of the key things that developers can do to help secure their systems is to write code that can withstand attack and use security features properly.” Defend Your Code with Top Ten Security Tips Every Developer Must Know 8 out of 10 tips are directly progra
10、mming issueshttp:/ Assumptions,Testing early is cheaper than testing later. System test of a system with j modules and k experiments (iterations or different implementations for a given interface) results in the following cost of testing,Where ct is the cost per test. Module-level tests with j modul
11、es and k experiments results in,Baldwin and Clark. Design Rules The Power of Modularity, volume 1, MIT Press, 2000.,Basic assumptions - 2,Though estimates vary, the cost of removing defect increases dramatically later in the life cycle.,Premise,Our premise is that poorly written software will contai
12、n more vulnerabilities than well written software where the security quality attribute is a design driver. Current views of security often take a defensive approach. Some of the security infrastructure even adds to the security risk due to the complexity it adds to the product. We propose an offensi
13、ve approach in which security is a key design driver and a priority throughout the development process.,Context,A chain of quality should be threaded through the entire process so that validation is most effective and efficient.,A Proposed Strategy,Develop method engineering tactics and guidelines t
14、hat enhance the security quality of the software through improved processes. Structure architecture evaluation techniques to focus on security by searching for static security patterns. Discover and capture test patterns that correspond to dynamic security patterns. Develop focused test techniques t
15、o effectively explore security test patterns while reducing the test suite size. Create a defect model for security that can be used to predict types and number of security vulnerabilities in scientific codes.,Action Develop method engineering techniques,Method engineers create custom-made processes
16、 to help a project achieve specific goals. The goal of being “secure” needs to be operationalized so that these engineers can assemble methods in ways that enhance the security of the software product built using their processes. This task would involve extending the Software Process Engineering Met
17、aModel1 (SPEM) standard to define security-specific constructs. The model would be automated using existing tools such as MetaEdit+.1 Software Process Engineering Metamodel, version 1.1, Object Management Group (OMG), 2005.,Action Develop method engineering techniques - 2,The security-oriented metho
18、d fragments would be suitable to integrate into processes defined in the context of the SPEM. Process audits could evaluate the strength of the security aspect of the process, once it is explicitly embedded in the process, just as other qualities are validated. Deliverables: A process definition gui
19、de that would show how to design security-centric development process fragments, an assembly guide, and an example process.,Action Create architecture security analysis,Architects have techniques that they apply to an architecture in order to improve the behavior of the architecture with respect to
20、classes of security threats1. Assuring system survivability requires showing that the system architecture is adequately resilient to likely patterns of attack2. One approach to architecture design is to identify quality attributes for the architecture and make design decisions that enhance the desir
21、able qualities and degrade the less desirable ones. Architecture evaluation techniques such as the architecture trade-off analysis method (ATAM) can be used to focus architecture evaluations on the security quality through security-specific scenarios. One aspect of this task would be to develop a se
22、t of architecture-level security scenarios that would guide the evaluation of an architecture for the security quality.1 Security and Survivability Reasoning Frameworks and Architectural Design Tactics. CMU/SEI-2004-TN-022 2 Architectural Refinement for the Design of Survivable Systems CMU/SEI-2001-
23、TN-008,Action Create architecture security analysis - 2,This evaluation can be automated if the architects use a formal architecture description language (ADL). Our current implementation uses the Architectural Analysis and Design Language (AADL) from the Society of Automotive Engineers (SAE) and Ec
24、lipse plug-ins. A second part of the task could be to identify standard mechanisms for representing security in an architecture description so that security patterns could be automatically recognized. Deliverables: A set of validation scenarios and specific architecture tactics for recognizing secur
25、ity vulnerabilities at the architecture level,Current implementation as Eclipse plug-in for AADL,Action Create architecture security analysis - 3,This analysis could also be conducted on pre/post-test basis Use an architecture re-construction technique such as OAR1 to extract the “as built” architec
26、ture from the implementation. Compare to the “as designed” architecture to determine if changes have occurred1 Options Analysis for Reengineering (OAR): A Method for Mining Legacy Assets CMU/SEI-2001-TN-013,Action Discover test patterns for security,Test patterns are solutions to testing problems in
27、 context. They often correspond to design patterns. Test patterns are language independent Content of a test pattern1 Problem - Description of pattern to be tested Context - Special testing conditions Forces - What types of faults are we looking for? Solution - Test case selection strategy that test
28、s the interactions among the components that implement the pattern Example A sample implementation1 John McGregor and David Sykes Practical Guide to Testing Object-Oriented Software, Addison-Wesley, 2001.,Action Discover test patterns for security - 2,Problem: The synchronous communication between t
29、wo objects is modified to be asynchronous by adding a callback object. How should this be tested?,Client,Server,Client,Server,Callback,Action Discover test patterns for security - 3,Context An intermediate object forwards messages Forces Possible to intermingle successive messages/responses Temporal
30、 considerations must be added to the test case Solution Construct tests that exercise the callback in a variety of states Construct multiple clients, submit multiple requests through multiple callback objects Build test cases that submit a second message prior to receiving the response from the firs
31、t message,Action Discover test patterns for security - 4,In this task, security vulnerabilities and standard designs to correct them would be studied. The standard designs would lead to a catalog of specific test patterns. Users of the test patterns could implement them in whatever language was in u
32、se. Deliverables: test pattern catalog,Task force identified security patterns,1. Make the Client Invisible 2. Target Programs That Write to Privileged OS Resources 3. Use a User-Supplied Configuration File to Run Commands That Elevate Privilege 4. Make Use of Configuration File Search Paths 5. Dire
33、ct Access to Executable Files 6. Embedding Scripts within Scripts 7. Leverage Executable Code in Non-executable Files 8. Argument Injection 9. Command Delimiters 10. Multiple Parsers and Double Escapes,11. User-Supplied Variable Passed to File System Calls 12. Postfix NULL Terminator 13. Postfix, Nu
34、ll Terminate, and Backslash 14. Relative Path Traversal 15. Client-Controlled Environment Variables 16. User-Supplied Global Variables (DEBUG=1, PHP Globals, and So Forth) 17. Session ID, Resource ID, and Blind Trust 18. Analog In-Band Switching Signals (aka “Blue Boxing”) 19. Attack Pattern Fragmen
35、t: Manipulating Terminal Devices 20. Simple Script Injection,49 security patterns have been identified. Here are a few.,Security Across the Software Development Lifecycle Task Force,Action Develop focused test techniques - 1,Operational profiles establish a relative frequency of operations for a par
36、ticular type of user or application. Test cases are chosen with the same relative frequency to mimic actual use or to mimic attack scenarios. This narrows the range of values to cover in test.,Action Develop focused test techniques - 2,Testing all possible values even once, much less all possible co
37、mbinations of values, is usually impossible. Combinatorial test designs can systematically sample test values to ensure the maximum coverage with minimum test cases. Orthogonal array testing is one combinatorial approach.,OATS tool - 1,Three variables, each with three possible values, would require
38、27 test cases for all combinations coverage.,OATS tool - 2,OATS reduces this to 9 test cases, testing pair-wise but remains approximately 96% as effective as “all combinations” coverage at finding defects.,Action Develop focused test techniques - 3,Pair-wise value combinations can reduce the size of
39、 the test suite dramatically These techniques have been applied in domains such as telecommunications. In this task we would develop an industrial strength tool that allows the design of combinatorial test suites.,Action Develop focused test techniques - 4,This technique is used in conjunction with
40、traditional functional and structural strategies for test case selection. Functional tests ensure that all specified functions are correct and structural tests ensure that all feasible paths are valid. The result is confidence that the software does everything it is supposed to and nothing that it I
41、s not supposed to. Combinatorial techniques increase the coverage of large software products without significant increases in test effort. This approach allows for risk-based test selections. Deliverables: software tool that uses combinatorial techniques and risk-based techniques to define effective
42、 test cases.,Action Develop a security defect model for scientific codes,If you just found the 73rd defect in your 50,000 LOC program, do you feel good about it? With a validated defect model you would at least know how to feel. You would know approximately how many defects to expect and what types
43、of defects to look for.,Action Develop a security defect model for scientific codes - 2,Using the relative frequency and estimated size, estimates can be made of the number of defects and the test case selection process be more precisely directed.,Action Develop a security defect model for scientifi
44、c codes - 3,In this task we would develop a defect model framework for scientific codes using historic data. The framework would be packaged with a specialization method for tailoring the framework to a specific development context. This would allow us to provide statistical estimates of the number
45、of defects remaining in a software product and to provide a confidence interval on statistics such as reliability. Deliverables: defect model,Conclusion,No single action or algorithm will produce secure software. There must be a chain of quality activities. The outlined strategy places security qual
46、ity gates at several places in the software product development life cycle.,Conclusion - 2,Implementing any part of this strategy will improve the security of the software products produced.Develop method engineering tactics and guidelines that enhance the security quality of the software through im
47、proved processes. Structure architecture evaluation techniques to focus on security by searching for static security patterns. Discover and capture test patterns that correspond to dynamic security patterns. Instrument combinatorial test techniques to effectively explore security test patterns while reducing the test suite size. Create a defect model for security that can be used to predict types and number of security vulnerabilities in a given product.,
