ABC/ORAs Update
An Update on the Acceptance Baseline Criteria (ABC) and Overall Risk Assessment (ORA)
In May 2022, we released the Acceptance Baseline Criteria (ABC) and Overall Risk Assessment (ORA) as a Minimally Viable Product (MVP). You can find the original blog post here (take time to look through the rest of our site…we are working hard to keep our documentation and guides up to date!). Since then we have gotten good feedback from our vendors and contributors, but now we need feedback from our users - engineers, programs, and authorizing officials in and supporting the U.S. Department of Defense. Here are a few things you should know…
A quick review of the ABC/ORA
The ABC formally document the hardening and contribution requirements for container images in Iron Bank. The ORA is adapted from the Open Source Security Foundation Scorecard. The ABC is our criteria while the ORA helps inform the risk a program assumes by using the container image. Why does this matter? At Iron Bank we want to establish standards for container security and provide transparency into the security of the Iron Bank images you use. You can find the docs here:
Am I getting an Approval or Verification?
We have been getting a lot of feedback and requests for approvals and verifications. Since these words are sometimes used interchangeably, we thought it would be helpful to clarify our definitions:
An approval is for an image that is being used inside of Platform One - usually by Party Bus or Big Bang. The approval is for our local authorization so that images can be used within our cyber boundary.
A verification is an independent review of the patches and justifications for vulnerabilities in an Iron Bank image. Think of this as a way to show a potential user or customer that someone outside of your organization has independently reviewed and verifies your hard work hardening an image.
Note
An unverified image is just an image we haven't gotten to yet. Our container security team works hard to verify fixes and justifications.
Impacts
- All containers supported by Big Bang (and Platform 1 internally) will continue to receive an Approval or Conditional Approval reflecting detailed government review of the container.
- Other containers will vary in the level of verification, see above section for definitions of verification and unverified. These containers may be marked as Verified or Unverified reflecting the level of review of the container and findings justifications.
- The Iron Bank ABC policy provides formal documentation of the Iron Bank container hardening requirements. We expect this clarified criteria will enable better reciprocity across the DoD.
- Container updates will be immediately available in Iron Bank, before all findings are justified. The ABC process provides timelines to justify, mitigate and remediate findings after submission. Containers that fail these criteria will be marked Non-Compliant, but will not be deleted from Registry1.
- The new ORA score can be used to perform a risk analysis of containers. Data about individual CVE findings and their vendor justifications are also available.
We have changed a few things since the Beta release
We removed two metrics:
- 1a - History of Resolving Vulnerabilities
- 2b - Number of Closed Findings Discovered in the last 90 Days
The calculations for these metrics had an undesired affect of penalizing images that had no vulnerabilities to resolve in the 90 day window and were confusing to end users.
As an alternative, we propose to score the ages of open vulnerabilities as a benchmark for how well an image is remediating findings. The assumption is that images with older open findings have a poorer history of remediating vulnerabilities compared to an image with only newer findings. See Proposal 3 below for more details.
We want your thoughts on a few ideas
See the bottom of the blog for contact information to share your feedback.
Proposal 1 - ABC Notices - Adjustments to Failures and Notice for Vulnerabilities
The guidelines defined by ABC are intentionally strict since security is the most important aspect of Iron Bank. However, in certain cases the guidelines are arguably too strict. This decreases the overall usefulness of a boolean compliant vs. non-compliant rating since it does not always align with what is accepted in practice. Instead of obfuscating lower severity items as "non-issues", a "notice" type is added to provide more granularity/awareness while keeping the boolean score in place.
We propose using the following schema for failing or flagging with a warning for vulnerabilities found in images.
| Critical | High | Medium | Low | |
|---|---|---|---|---|
| True Positive | F | F | F | N |
| No Fix Available | F | F | F | N |
| Won't Fix | F | F | F | |
| Distro Package | F | F | F | |
| Distro Package - Won't Fix | F | F | F | |
| Pending Resolution | F | F | F | N |
| Unreleased | F | F | F | N |
| Mitigated | N | N | N | |
| Not Vulnerable | N | N | N | |
| Not Applicable | ||||
| Policy Exception | ||||
| False Positive | ||||
| NVD Disputed |
F - Failure
N - Notice
Proposal 2 - Modifications to Standards
- ABC Score - Remove failure due to low findings
- Lows should not fail a container for compliance because they are lower risk
Proposal 3 - Modifications to ORA Score
-
ORA Score - Vulnerability Age/Expiration
- Vulnerabilities that are left unpatched for long periods of time indicates poor image hygiene. The existing vulnerability metric does not distinguish between a new finding or an old finding. This metric specifically considers the age of a finding.
- Use an s-curve for scoring and weighting each finding based on age:
100 - (sum (alpha / (1 + e**-k*(x - x0)))) where x = age/D * 3.0 (normalize x such that midpoint is of curve is 2*D) where D = remediation guidelines in days (critical=45, high=90, medium=180, low=360) where k = 1.0, x0 = 6.0 (k is how steep the s-curve is, x0 shift s-curve since age is not negative)
-
ORA Score - Commit Age
- One of the factors for determining how well maintained an image is to look at how often the software is released. This metric uses the date of the last commit to measure how old the most recent change to an image is. This includes version updates and/or bug fixes.
- Commits less than 3 months old are given a maximum score.
- Commits 12 months and older are given a minimum score.
- One of the factors for determining how well maintained an image is to look at how often the software is released. This metric uses the date of the last commit to measure how old the most recent change to an image is. This includes version updates and/or bug fixes.
Proposal 4 - Data on the below items should be collected and displayed but not contribute to the ORA score
- Scan Age: Scores an image based on when the last time it was scanned
- Justification Submission: Scores whether or not justifications are submitted
- Justification Responsiveness: Scores whether or not justifications were provided on time
- Justification Acceptance: Scores how many justifications have been reviewed or approved
More Information
Have feedback or questions about Iron Bank's ABC and ORA? Please fill out our simple form: ABC/ORA Survey
The Iron Bank ABC ORA FAQ is available here. It will be updated whenever the team receives questions or feedback about the policies and implementation.
The Iron Bank AMA is held every Wednesday at 2:30pm MT. Click here to register (opens external link to Zoomgov Meeting Registration page for the AMA).
The Iron Bank Container Hardening Team Public IL2 MatterMost Channel can be used to ask direct questions to the team.