Flawless prevention of security issues at some point is truly impossible, but controlling how quickly we react is very possible. Software development and delivery processes should be long past the days of assigning low priority to security design and implementation. We have all heard horror stories about what happens when development projects fail to include security planning at the earliest stages. Still, zero-day and other security disasters occur at an alarming rate, and post-hoc investigations of such incidents clearly shows when Engineering teams ignored or paid only minimal attention to security considerations during development and delivery processes. This will be a multi-part discussion on High Security DevOps (HSDO) and how it can be achieved.
Your organization must consider security in the early stages of design, planning, and development for both new products and existing ones. This can be challenging, because there are many reasons why organizations fail to give security the necessary priority. This blog post identifies some of these considerations, along with some opportunities and goals for HSDO in improving the security of software development and delivery processes because HSDO can play a key role in facilitating, collaborating, and integrating security considerations in software development.
Considerations for High Security DevOps
- Security requirements can vary significantly, depending on the technology involved.
- An organization’s developers can sometimes lack security expertise or access to a Security Team.
- Development, Dev Ops, QA, and Security teams often have conflicting priorities. For example, even when an organization has a Security team, that team’s expertise may be dedicated to incident management rather than to planning and development.
- Both manual and automated testing mainly focus on the functionality of features and can ignore security testing or fail to include tests needed to identify specific security risks related to the product and its design.
- Audit requirements may conceal deficiencies in security planning: although it is important for project teams to understand what auditors want, security should never become only a matter of checking items on a list.
Development, Security and Operations (Dev, Ops, and Sec) each has their own priorities that can conflict with one another. Understanding how to apply their tools and processes to development workflows will improve collaboration and improve change as your product and organization grow. This will improve overall security posture, quality, and trust.
Opportunities for High Security DevOps
- DevOps is ideally positioned to minimize the risk of mismatches between authentication systems in different environments. Such mismatches can result in missed deadlines (extending time to market) and in the need for quick fixes that can introduce additional security risks.
- DevOps can minimize the risk of artifact poisoning or other problems that can result from insecure or improper artifact management.
- DevOps can include testing for “Evil Cases” in all stages where testing occurs while accurately tracking and reporting that data. Rerunning some security tests should also occur prior to deployment or publishing.
- DevOps can make sure that tools and automation used in development align with operational and security processes.
- DevOps can help minimize some risks by assuring that encryption requirements such as SSL and non-SSL coding standards are addressed early in the development process
- DevOps can provide or integrate secure storage of passwords and keys, e.g., “secrets,” that can be managed, monitored and trusted.
- DevOps can introduce thorough failure injection testing into the process.
- DevOps can introduce security and application-centric desired state configuration management.
Goals for High Security DevOps
- Provide a single source of truth for security and security testing in development and delivery processes.
- Introduce security at the design stage by conducting project security briefings for Developers, Project Managers, QA teams, and DevOps teams – educating teams with examples of potential disasters (if necessary).
- Facilitate collaborative cross-team processes.
- Isolate secure development environments and require secure communications between development and master resources.
- Tight integration of tools and services used between Dev, Ops, and Sec.
- Implement immutable infrastructure for faster patching and closure of any security issues.
- Fast, transparent feedback and intake process that include latest threats and discovery of issues.
- Collaborative process that enables innovation while fostering risk management and operational needs.
The Results
- Optimized, data-driven decision gates for code promotion resulting in faster release and rollback decisions.
- Secure and fast continuous delivery.
- Faster patch, and closure of security issues at any point.
- High standards for secure automated processes.
- Improved product quality, trust, and responsiveness.
- Effective collaboration of Developer, DevOps, QA, Operations and Security teams, with minimal misallocation and duplication of time and effort.
Security should be a habit that is naturally a part of how we do things. Perhaps in how we habitually look both ways when crossing an intersection. We should and can proactively work to prevent incidents, but how quickly we can react will also be a measure of success in preventing unforeseen challenges. In thinking about this, ask this question: “How fast can your teams respond to the next “Heartbleed” like zero-day vulnerability?” Remember, when you consider this, you must consider both public facing services and internal services. This is what secure DevOps is all about. The ability address such a zero day in hours to days, rather than weeks. At Excella we are very much about making an impact, but the impact of this scenario is one we want to be proactive in preventing.
Stay tuned for my next blog post, “No Budget Security for DevOps and Developers” that will explore how software developers and DevOps teams can best work together to enhance security, with minimal cost and disruption.
Personal interviews
While writing this article, I conducted several personal interviews with the following experts:
- Lane, D. – Sr. Consultant, Excella (2016, March 12).
- Althouse, J. – Sr. Security Scientist, Salesforce Incident Response (2016, April 1).
- Kleiner, D. – Forensics Lab Manager, Fireeye (2016, March 15).