Security Agents for Open Source Projects

Security has become a forefront as an important and essential element of the software development lifecycle. Traditionally, organizations have been focusing on security of their applications only at the time of performance testing phase — when the development phase is already over and the application is ready for use. Due to security being brought late in the game, it is either difficult to find vulnerabilities or even if found it becomes more expensive and difficult to fix. The earlier bugs and vulnerabilities are introduced, the more impact they create and the more complex they can be to resolve. This is why we need to address security from the early development stages till the end when the application is already running.

The two approaches we will be taking to tackle security are static application security testing (SAST) and dynamic application security testing (DAST). SAST is open box testing in which we analyze the program source code to identify security vulnerabilities before compilation or execution. This approach discovers source code security flaws early and avoids leaving security issues to later development phases. DAST scans software applications in real-time against leading vulnerability sources. DAST is a form of closed box testing in which we test the application from an outside attacker’s perspective. The application’s inner functions are not known and thus, detects security vulnerabilities such as those that appear only during the program runtime.

Bahmni

Bahmni is an open source EMR & hospital system. It combines and enhances existing open source products into a single solution. Bahmni is made up of four services which are integrated with each other. These four services are OpenMRS for electronic medical records and patient management, OpenERP for inventory, billing and accounting, dcm4Chee for DICOM & PACS, OpenELIS for laboratory management.

Bahmni Architecture

Our Use Case:

While we have guidance documented for right security practices while installing Bahmni e.g. setting up right permissions, root access etc. but there was no way to keep the respective implementation support / maintenance team informed about possible vulnerabilities. That is when we decided to look for tools that would act as Bahmni Security Agents and would run on Linux machines (on-prem or cloud) where bahmni is installed and would scan for possible vulnerabilities with respect to

• Policies (inappropriate permissions, root access etc)
• Static Code Analysis
• Open ports
• Libraries / Dependencies used by the application that have known vulnerabilities
• OS Security updates missing / misconfigurations
• Container image scanning
• Secrets scanning
• Kubernetes Cluster scanning
• Open Source/Free to use

Suitable Tools:

Instead of going with one tool that meets all the requirements, we decided to go with several lightweight agents. The agents specialize in one or more requirements. Given below is the tools that are well suited for Bahmni and are categorized as per the requirement.

Vulnerability Checks

Dependency Check is Software Composition Analysis that attempts to detect publicly disclosed vulnerabilities contained within a project’s dependencies. It does this by determining if there is a Common Platform Enumeration (CPE) identifier for a given dependency. If found, it will generate a report linking to the associated CVE entries.

Trivy

Trivy is a simple and comprehensive vulnerability/ misconfiguration scanner. It is a product of Aquasec. Bahmni already uses tfsec, another tool from Aquasec which is a static analysis security scanner for Terraform code (Infrastructure as code). Trivy detects vulnerabilities of type:

• OS packages (Alpine, Red Hat Universal Base Image, Red Hat Enterprise Linux, CentOS, AlmaLinux, Rocky Linux, CBL-Mariner, Oracle Linux, Debian, Ubuntu, Amazon Linux, openSUSE Leap, SUSE Enterprise Linux, Photon OS and Distroless) installed by package managers like apk, yum/rpm, zypper, tdnf, apt/apt-get/dpkg
• Language-specific packages (Bundler, Composer, Pipenv, Poetry, npm, yarn, pnpm, Cargo, NuGet, Maven, and Go)

Installation of Trivy is very straightforward. Follow the instruction based on your distribution.

sudo apt-get install wget apt-transport-https gnupg lsb-release
wget -qO — https://aquasecurity.github.io/trivy-repo/deb/public.key | sudo apt-key add -
echo deb https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main | sudo tee -a /etc/apt/sources.list.d/trivy.list
sudo apt-get update
sudo apt-get install trivy

Scan a local project including language-specific files

$ trivy fs /path/to/project

Click here to know more about file system scanning.

Trivy is considered to be used in CI. It will scan the code on every Push/ merge of a pull request. That way, we can make sure that the source code has been analyzed. Bahmni uses Github Actions for CI/CD. Here is the link to Trivy Action.

Scanning OS

Operating Systems should be audited for compliance. A compliance audit is a process of determining whether a given object follows all the rules specified in a compliance policy. The compliance policy is defined by security professionals who specify the required settings, often in the form of a checklist, that a computing environment should use.

OpenSCAP

OpenSCAP is a Configuration Compliance tool by Red Hat Enterprise Linux and is certified by NIST. It is designed to perform configuration and vulnerability scans on a local system, to validate configuration compliance content, and to generate reports and guides based on these scans and evaluations. It utilizes the Extensible Configuration Checklist Description Format (XCCDF). XCCDF is a standard way of expressing checklist content and defines security checklists. It also combines with other specifications such as CPE, CCE, and OVAL, to create a SCAP-expressed checklist that can be processed by SCAP-validated products.

• OpenSCAP can remediate the infrastructure with security compliance such as CIS benchmark guidelines.

To install OpenSCAP, follow their detailed installation guide here.

SCAP Content is basically the security policies in SCAP format. There are many providers of SCAP content. We will be using SSG (SCAP Security Guide) SCAP content. Many Linux distributions have scap-security-guide package that needs to be installed and the SCAP content for that particular distribution is shipped in the scap-security-guide. For other platforms, security policy content can be found here.

After getting the SCAP content for your Operating System, the file content can be displayed by:

oscap info /usr/share/xml/scap/ssg/content/ssg-rhel8-ds.xml

Where /usr/share/xml/scap/ssg/content is the path where SCAP content is present and ssg-rhel8-ds.xml is SCAP Security Guide file for Red Hat Enterprise Linux 8

We need to get the profile id for the profile we will be using for scanning. Follow a detailed guide for performing a scan here.

oscap xccdf eval command is used to perform a scan. The scan generates html formal results. The results can be viewed in a browser.

Scanning Kubernetes Cluster

Trivy

Trivy has a native Kubernetes Operator which continuously scans your Kubernetes cluster for security issues, and generates security reports as Kubernetes Custom Resources. It does it by watching Kubernetes for state changes and automatically triggering scans in response to changes, for example initiating a vulnerability scan when a new Pod is created.

Trivy Operator can be installed using Helm, kubectl. To install the Operator, follow their detailed guide here. After the Operator is installed, it will start scanning all your namespaces except kube-system and trivy-system. It scans all the existing container images, in addition to new images that will be added in future. The scan results can be viewed by:

kubectl get vulnerabilityreports -n <namespace> -o wide

It also exposes metrics which are picked up by Prometheus. These metrics can then be visualized by using Grafana. Just make sure that the service monitor is enabled while installing Trivy Operator. For Helm, –set=”serviceMonitor.enabled=true”

Scanning Container Images

Trivy

Trivy identifies the package and version in the image. It then cross-references with the vulnerability database and downloads it.

Get Umair Fayaz’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Container Images can be scanned in two ways

• By simply specifying an image name (and a tag).

$ trivy image python:3.4-alpine

• (Preferred Way) By setting up Trivy Scanner in your CI/CD Pipelines

Following steps are to be followed in the workflow:

- Build Image

- Check and scan security vulnerabilities using Trivy

- Push to Dockerhub/ other registries.

Example YAML file

name: Scan Container Image
on:
 push:
 branches: [master]
 pull_request:
 branches: [master]
jobs:
 build:
 name: Build
 runs-on: ubuntu-latest
 steps:
 — name: Checkout code
 uses: actions/checkout@v2
— name: Build the Docker image
 run: docker build -t <image-tag> .
— name: Run Trivy vulnerability scanner
 uses: aquasecurity/trivy-action@master
 with:
 image-ref: “memcached-admin:${{ github.sha }}”
 exit-code: “1”
 ignore-unfixed: true
 vuln-type: “os,library”
 severity: “CRITICAL,HIGH”
 format: “template”
 template: “@/contrib/sarif.tpl”
 output: “trivy-results.sarif”
— name: Upload Trivy scan results to GitHub Security tab
 uses: github/codeql-action/upload-sarif@v1
 with:
 sarif_file: “trivy-results.sarif”

The output of the scan results is uploaded as SARIF format and can be seen in the security tab of the project. After this the images can be published in another workflow, which can either be triggered after successful completion of the previous one or triggered manually.

Scanning Secrets

Trivy

Trivy scans any container image, filesystem and git repository to detect exposed secrets like passwords, api keys, and tokens. Secret scanning is enabled by default.

Trivy will scan every plaintext file, according to builtin rules or configuration. There are plenty of builtin rules:

• AWS access key
• GCP service account
• GitHub personal access token
• GitLab personal access token
• Slack access token etc.

When we perform a scan, like we did for file system, Kubernetes cluster, and image scanning, Truvy’s secret scanner works by default. If it finds any secret exposed in the code, it will report that.

Talisman

Talisman is a tool that installs a hook to your repository to ensure that potential secrets or sensitive information do not leave the developer’s workstation.

It validates the outgoing changeset for things that look suspicious — such as potential SSH keys, authorization tokens, private keys etc.

Talisman can be configured as a pre-commit hook in our machines as well as in Ci/CD pipelines that will scan the code for sensitive secrets whenever developers push their code.

Other tools explored

We explored some other tools during the spike but they didn’t fit our use case. Given below is a list of the best ones of those tools with a brief explanation.

OpenVAS

OpenVAS is a tool by Greenbone. It was previously known as NESSUS but after 2008, NESSUS became proprietary and Greenbone has been maintaining OpenVAS as an open source project. It does many sorts of vulnerability assessments especially those inclined towards penetration testing. It didn’t meet our use case because of the following reasons:

• Resource hungry

OpenVAS uses about 5 individual services to run i.e nmap, postgresql db, redis etc which makes it slower if significant memory and processing power is not available.

• It behaves more like a penetration testing tool and assumes the provided host as a black box and thus provides a gamut of false-positives and false-negatives which might increase our efforts further.
• We had better options for performing dependency checks, container image scanning which OpenVAS was not doing or was not at par with. We were more interesting in a light-weight agent and OpenVAS behaves as a one-stop-solution.

Vulners plugin for Nmap

Network mapper or Nmap or Zenmap (GUI of Nmap) is an open source network scanning tool. It can scan an entire network for hosts and can run available or custom scripts on them to check for vulnerabilities. It has various capabilities that include bypassing firewalls with misconfigurations, providing metasploit exploits. It is infamous for its number of false-positives and false-negatives. Also, the user must have a decent knowledge of all the variety of scans that it offers which might be overwhelming for the end users of Bahmni.

Vuls.io

Vuls is an open-source, agentless vulnerability scanner based on information from NVD, OVAL, etc. and it is designed in the Go language. It checks for outdated dependencies. Its remote scan mode requires us to install Vuls on one machine and then a connection needs to be made with the remote target using SSH. For our use case we wouldn’t want to ssh into targets every time we need to make a scan.

Summary:

For securing a project a lot of aspects need to be kept in mind. People might choose to use an enterprise edition of a tool or a combination of such tools to secure their entire on-premise or cloud infrastructure and applications using tools like UTFs. For open source projects a lot of developers are reluctant to invest in the security aspect which makes the applications vulnerable to attacks; which is a major reason why most companies don’t use open source applications directly. We have presented a way in which an open-source project can be secured using open source tools throughout multiple phases of the software development lifecycle.

• Purpose: Want to identify the right tool (agent) to identify various security issues

Types of security issues:

• Policies
• OS vulnerabilities
• Secrets scanning
• Static code analysis
• Vulnerable libraries/dependencies
• Vulnerable Container image

Tools Explored:

• OpenVAS:

1. Great tool but resource hungry
2. Potential false negatives/positives due to underneath Nmap — results require subsequent investigation
3. More complex and steeper learning curve
4. May not suit our target users (Bahmni’s target users are doctors, receptionists working in hospitals and clinics)

• Trivy:

1. Lightweight — comprehensive vulnerability assessment which supports container, filesystem and remote git-repo scanning,
2. Recommended in Thoughtworks Tech Radar
3. Also supports K8s scanning (Helm charts available) and can be integrated with Prometheus to visualize in Grafana

• OpenSCAP: OS scanning

1. Policies maintained by NIST
2. Compliance as Code approach
3. Lightweight and fast

• Talisman: Secrets scanning integrated as local hooks and CI