add container signing POC

Proof-of-concept focusing on enabling container signing with any
signing backend with external signer plugin.

1. Add Notation External Signer plugin and sign images with that
2. Validate the Signature runtime with Kyverno admission control

Signed-off-by: Tuomo Tanskanen <tuomo.tanskanen@est.tech>

security/container-signing-poc/Makefile

@@ -0,0 +1,24 @@
+# top level makefile to run e2e test
+# notation -> oras copy -> kyverno
+
+NOTATION_DIR := notation
+ORAS_DIR := oras
+KYVERNO_DIR := kyverno
+
+SHELL := /bin/bash
+
+.PHONY: all e2e clean
+
+all:
+	@echo "targets: e2e clean"
+
+e2e:
+	make -C $(NOTATION_DIR) e2e
+	make -C $(ORAS_DIR) e2e
+	make -C $(KYVERNO_DIR) e2e
+	@echo "e2e test done!"
+
+clean:
+	make -C $(NOTATION_DIR) clean-e2e
+	make -C $(ORAS_DIR) clean-e2e
+	make -C $(KYVERNO_DIR) clean-e2e

security/container-signing-poc/README.md

@@ -0,0 +1,64 @@
+# Container Signing POC
+
+This experiment is two-fold:
+
+1. [Signing images with Notation, especially with custom signers](notation/README.md)
+1. [Moving signatures from one registry to another](oras/README.md)
+1. [Validating the signing with Kyverno, in a Kind cluster](kyverno/README.md)
+
+## TL;DR of the POC
+
+Basically, the POC is finding the following:
+
+1. Notation is extendable with their plugin system
+1. Plugin that can call any external script or binary to produce a signature has
+   been implemented.
+1. This allows any in-house, custom integration to private signer, regardless
+   of the interface, even manual/email works (despite being brittle), without
+   writing a full-fledged plugin with Go.
+1. Kyverno can easily be configured to verify Notation signatures runtime, via
+   their admission controller and pluggable policies.
+1. Oras can be used to move containers and signatures from CI to production
+
+## e2e test
+
+End-to-end test for this POC can be run with `make e2e` from this directory.
+This does the following:
+
+1. Build and install Notation plugin
+1. Sign busybox image on local registry at port `5002`
+1. Copy container and signature to another registry at port `5001`
+1. Launch Kind cluster, install Kyverno and add ClusterPolicy for signature
+   verification, and then run workloads that pass the verification and workloads
+   that fail the verification to check, if the signing works e2e
+
+End-to-end test will use the same certificates and same signature through the
+whole chain for verify end-to-end functionality.
+
+## Notes
+
+### Notes on Notation
+
+Notary V1 was part of TUF (The Update Framework) organization, yet separate
+project in CNCF. Notary V2 has been rebranded as Notation to make the branding
+clearer that it is based on different principles than Notary V1. Notary V1/TUF
+has more strict "security principles", which also made it hard to use, and it
+did not gain traction.
+
+It should be noted that Notary V1 and Notary V2/Notation are completely different
+projects, with claims of [hostile takeover](https://github.com/cncf/toc/issues/981)
+by the new maintainers (Microsoft/Docker). CNCF TOC did not see it that way,
+but funded a
+[security audit](https://www.cncf.io/blog/2023/07/11/announcing-results-of-notation-security-audit-2023/)
+for Notation, with no critical findings, but the argument from the Notary V1
+folks is that the foundation of Notation security model is not good enough. This
+design aspect was not part of the audit based on the findings.
+
+### Notes on Kyverno
+
+[Kyverno Image Signature verification](https://kyverno.io/docs/writing-policies/verify-images/)
+ is in beta.
+
+Kyverno can also verify Sigstore Cosign signatures.
+
+Kyverno is generic policy engine, capable of replacing OPA Gatekeeper etc.

security/container-signing-poc/kyverno/Makefile

@@ -0,0 +1,105 @@
+# build and install notation-external-signer
+
+SCRIPTS_DIR := scripts
+POLICY_DIR := policy
+
+NOTATION_DIR := ../notation
+EXAMPLES_DIR := $(NOTATION_DIR)/examples
+NOTATION_SIGNER := $(EXAMPLES_DIR)/rsassa-pss-sha512.sh
+
+TEST_REGISTRY := 127.0.0.1:5001
+TEST_REGISTRY_NAME := kind-registry
+TEST_REGISTRY_IMAGE := registry:2
+TEST_IMAGE_UNSIGNED := busybox:1.36.0-glibc
+TEST_IMAGE_SIGNED := busybox:1.36.1-glibc
+TEST_DIGEST := sha256:db16cd196b8a37ba5f08414e6f6e71003d76665a5eac160cb75ad3759d8b3e29
+
+CLUSTER_REGISTRY := 172.19.0.3:5000
+
+SHELL := /bin/bash
+
+.PHONY: all test setup check-tools sign clean clean-tests
+
+all: check-tools
+	@echo "targets: test clean"
+
+test: check-tools setup sign tests
+e2e: check-tools setup-e2e tests
+
+.PHONY: tests test-pod test-deployment
+
+setup: setup-registry create-cluster install-kyverno install-notation-plugin certificates
+setup-e2e: create-cluster install-kyverno
+
+.PHONY: setup-registry create-cluster install-kyverno install-notation-plugin install-policy certificates
+
+check-tools:
+	@type -a helm &>/dev/null || echo "error: Install helm: https://helm.sh/docs/intro/install/"
+	@type -a docker &>/dev/null || echo "error: Install docker: https://docs.docker.com/engine/install/"
+	@type -a kind &>/dev/null || echo "error: Install kind: https://kind.sigs.k8s.io/docs/user/quick-start/"
+	@type -a notation &>/dev/null || echo "error: Install notation: https://notaryproject.dev/docs/user-guides/installation/cli/"
+
+setup-registry:
+	docker run -d -p $(TEST_REGISTRY):5000 --name $(TEST_REGISTRY_NAME) $(TEST_REGISTRY_IMAGE)
+	for image in $(TEST_IMAGE_UNSIGNED) $(TEST_IMAGE_SIGNED); do \
+		docker pull $${image}; \
+		docker tag $${image} $(TEST_REGISTRY)/$${image}; \
+		docker push $(TEST_REGISTRY)/$${image}; \
+	done
+
+create-cluster:
+	./scripts/kind-cluster.sh
+
+install-kyverno:
+	helm repo add kyverno https://kyverno.github.io/kyverno/
+	helm repo update
+	helm install kyverno kyverno/kyverno -n kyverno --create-namespace
+	sleep 60
+	# NOTE: we need to edit Kyverno config to allow insecure registries
+	kubectl -n kyverno get deployment kyverno-admission-controller -o yaml | \
+		sed -e 's/allowInsecureRegistry=false/allowInsecureRegistry=true/' | \
+		kubectl apply -f -
+	sleep 30
+
+install-notation-plugin:
+	make -C $(NOTATION_DIR) install
+
+certificates:
+	make -C $(NOTATION_DIR) certificates
+
+install-policy:
+	# replace example cert with the generated certs
+	cat $(POLICY_DIR)/kyverno-policy.yaml | \
+		sed -re '/-----BEGIN/,/END CERTIFICATE-----/d' | \
+		{ cat -; cat $(EXAMPLES_DIR)/ca.crt | sed -e 's/^/              /g'; } | \
+	kubectl apply -f -
+	sleep 30
+
+sign:
+	make -C $(NOTATION_DIR) sign TEST_IMAGE_SIGN=$(TEST_REGISTRY)/$(TEST_IMAGE_SIGNED)
+
+tests: install-policy test-pod test-deployment
+	sleep 5
+	kubectl get pods -A
+	@echo "Success (if only pods with success are visible - ignore the ImagePull issues)"
+
+test-pod:
+	kubectl run --image $(CLUSTER_REGISTRY)/$(TEST_IMAGE_UNSIGNED) pod-fail || true
+	kubectl run --image $(CLUSTER_REGISTRY)/$(TEST_IMAGE_SIGNED) pod-success
+
+test-deployment:
+	kubectl create deployment --image $(CLUSTER_REGISTRY)/$(TEST_IMAGE_UNSIGNED) deployment-fail || true
+	kubectl create deployment --image $(CLUSTER_REGISTRY)/$(TEST_IMAGE_SIGNED) deployment-success
+
+clean-tests:
+	-kubectl delete pod pod-fail
+	-kubectl delete deployment deployment-fail
+	-kubectl delete pod pod-success
+	-kubectl delete deployment deployment-success
+
+clean: clean-e2e
+	make -C $(NOTATION_DIR) clean
+
+clean-e2e:
+	-docker rm -f $(TEST_REGISTRY_NAME)
+	-kind delete cluster