A few weeks ago, I wrote a post on using GitLab's built in identity to with Kubernetes via OpenUnison. In that post, we focussed on using the identity provided to each GitLab workflow to authenticate to Kubernetes via the OpenUnison running on cluster that's designed for use by developers and administrators. I encourage you to go through that post, but here are the highlights:
- Kubernetes has no built in way to securely authenticate remote workflows - Certificates are insecure because Kubernetes can't check for revocation. ServiceAccount tokens are not designed to be used from outside of the cluster and are too often long lived.
- User authentication is clunky - Authenticating with Okta or EntraID is fine, but those aren't really meant for machine identity.
- OpenUnison can be customized to provide a Security Token Service (STS) - OpenUnison is a great platform for building all kinds of identity integration services. You can use it as an STS to exchange your workflow's identity for a kubectl configuration usable directly by pretty much any Kubernetes client SDK.
To create our STS, we:
- Copied the token Application - We started by duplicating the token Application, which provides a RESTful API for retrieving a kubectl configuration. We tweaked the configuration a bit for use by an API instead of from the browser.
- Created a JWT AuthenticationChain - Creating a JWT AuthenticationChain allows OpenUnison to validate tokens from GitLab.
- Authorized access based on claims in the token - Based on the claims in the token, we limited access to our cluster from a specific project in GitLab.
With all that done, we were able to provide secure access to our cluster from GitLab. So now, how do we do this with GitHub? First, let's look how GitHub provides an identity to it's workflows.
GitHub Action Workflow Identity
Similar to how each Kubernetes Pod has its own unique identity, each GitHub action workflow has a unique identity. Just as with Kubernetes, this identity is meant to be used to interact with the GitHub API, but you can also get JWTs for other services. This has become very popular especially when working with remote cloud services like AWS. Generating an identity is pretty easy. First in your workflow you need to declare that you need an identity:
permissions:
id-token: write
This tells GitHub you want to make your action's identity available in the ACTIONS_ID_TOKEN_REQUEST_TOKEN environment variable. While you technically could use this token directly, it's not scoped for our cluster. Since every token is a potential attack vector, we want to limit how useful any given token is. GitHub makes it pretty straight forward to get a token:
- name: get oidc token
run: |
OIDC_TOKEN=$(curl -sLS "${ACTIONS_ID_TOKEN_REQUEST_URL}&audience=cicd.tremolo.dev" -H "User-Agent: actions/oidc-client" -H "Authorization: Bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN")
JWT=$(echo $OIDC_TOKEN | jq -j '.value')
echo "JWT=$JWT" >> $GITHUB_ENVThis code is pretty straightforward. Use the curl command to call the GitHub action's own STS to generate a token for the audience cicd.tremolo.dev. This is pretty repetitive, and I don't like coping and pasting repetitive code. Tremolo Security published an action to automate this for us! Now, our step to generate a JWT is much cleaner:
- name: get identity
uses: tremolosecurity/action-generate-oidc-jwt@v1.1
with:
audience: https://k8sou.domain.com/
environmentVariableName: "JWT"Before we complete our GitHub action workflow, let's take a look at the claims available in the token we get:
{
"jti": "89e61a91-ddf7-4205-9b1a-bf180e0ce91d",
"sub": "repo:mlbiam/kube-wf-identity:ref:refs/heads/main",
"aud": "https://k8sou.domain.com/",
"ref": "refs/heads/main",
"sha": "1b3e0ba075532afeed2a3dbeb83831304b819641",
"repository": "mlbiam/kube-wf-identity",
"repository_owner": "mlbiam",
"repository_owner_id": "8249283",
"run_id": "12150893094",
"run_number": "1",
"run_attempt": "3",
"repository_visibility": "public",
"repository_id": "898196205",
"actor_id": "8249283",
"actor": "mlbiam",
"workflow": "kube-workflow",
"head_ref": "",
"base_ref": "",
"event_name": "push",
"ref_protected": "false",
"ref_type": "branch",
"workflow_ref": "mlbiam/kube-wf-identity/.github/workflows/kube-wf.yml@refs/heads/main",
"workflow_sha": "1b3e0ba075532afeed2a3dbeb83831304b819641",
"job_workflow_ref": "mlbiam/kube-wf-identity/.github/workflows/kube-wf.yml@refs/heads/main",
"job_workflow_sha": "1b3e0ba075532afeed2a3dbeb83831304b819641",
"runner_environment": "github-hosted",
"iss": "https://token.actions.githubusercontent.com",
"nbf": 1733273599,
"exp": 1733274499,
"iat": 1733274199
}There's quite a bit to work with here. One thing that you'll notice is that there are no groups of any kind. While we can authorize access at the OpenUnison layer based these attributes, we can't write RBAC rules based on them. We'll come back to that later. If you want to know what each of these claims provides, GitHub has a great guide in its documentation.
Now that we have our token generated, the next step is to create an AuthenticationChain that can validate a token from our GitHub action.
Validating a GitHub Action's Token
We've got a token from Github, now we need to validate it in OpenUnison. There's very little change from the GitLab AuthenticationChain we created in the last post:
apiVersion: openunison.tremolo.io/v1
kind: AuthenticationChain
metadata:
name: github-token
namespace: openunison
spec:
authMechs:
- name: oauth2jwt
params:
audience: https://k8sou.domain.com/
defaultObjectClass: inetOrgPerson
fromWellKnown: "true"
issuer: https://token.actions.githubusercontent.com
linkToDirectory: "false"
lookupFilter: (sub=${sub})
noMatchOU: oauth2
realm: kubernetes
scope: auth
uidAttr: sub
userLookupClassName: inetOrgPerson
required: required
secretParams: []
- name: map
params:
map:
- uid|composite|${sub}
- mail|composite|${actor}
- givenName|composite|${repository}
- sn|composite|${workflow_ref}
- displayName|composite|${sub}
required: required
- name: az
params:
rules:
- filter;(&(ref=refs/heads/main)(repository=mlbiam/kube-wf-identity))
required: required
- name: jit
params:
nameAttr: uid
workflowName: jitdb
required: required
- name: genoidctoken
params:
idpName: k8sidp
trustName: kubernetes
required: required
level: 1
There's only a few changes:
- Name - We changed the name from gitlab-token to github-token.
- Issuer - Instead of looking up GitLab's issuer, we are looking up GitHub's issuer.
- Mapping - The claims we're mapping into the "user" object changed
- Authorization - We're still authorizing based on the branch being main and the workflow running in a specific project
We mentioned earlier that there are no groups. this makes it difficult to authorize access to resources in Kubernetes with anything other then a workflow's sub claim, which is generally an anti-pattern. The good news is we can adapt our AuthenticationChain to include some javascript that will let us generate groups from the claims:
apiVersion: openunison.tremolo.io/v1
kind: AuthenticationChain
metadata:
name: github-token
namespace: openunison
spec:
authMechs:
- name: oauth2jwt
params:
audience: https://k8sou.domain.com/
defaultObjectClass: inetOrgPerson
fromWellKnown: "true"
issuer: https://token.actions.githubusercontent.com
linkToDirectory: "false"
lookupFilter: (sub=${sub})
noMatchOU: oauth2
realm: kubernetes
scope: auth
uidAttr: sub
userLookupClassName: inetOrgPerson
required: required
secretParams: []
- name: js
params:
js: |-
function doAuth(request,response,as) {
// setup classes that we can use from Java
Attribute = Java.type("com.tremolosecurity.saml.Attribute");
ProxyConstants = Java.type("com.tremolosecurity.proxy.util.ProxyConstants");
GlobalEntries = Java.type("com.tremolosecurity.server.GlobalEntries");
HashMap = Java.type("java.util.HashMap");
System = Java.type("java.lang.System");
// get the session data needed
var session = request.getSession();
var holder = request.getAttribute(ProxyConstants.AUTOIDM_CFG);
var ac = request.getSession().getAttribute(ProxyConstants.AUTH_CTL);
memberOf = new Attribute("memberOf");
memberOf.getValues().add(ac.getAuthInfo().getAttribs().get("repository_owner").getValues().get(0));
memberOf.getValues().add(ac.getAuthInfo().getAttribs().get("repository").getValues().get(0));
ac.getAuthInfo().getAttribs().put("memberOf",memberOf);
as.setExecuted(true);
as.setSuccess(true);
holder.getConfig().getAuthManager().nextAuth(request, response,session,false);
}
required: required
- name: map
params:
map:
- uid|composite|${sub}
- mail|composite|${actor}
- givenName|composite|${repository}
- sn|composite|${workflow_ref}
- displayName|composite|${sub}
required: required
- name: az
params:
rules:
- filter;(&(ref=refs/heads/main)(repository=mlbiam/kube-wf-identity))
required: required
- name: jit
params:
nameAttr: uid
workflowName: jitdb
required: required
- name: genoidctoken
params:
idpName: k8sidp
trustName: kubernetes
required: required
level: 1Our javascript creates a memberOf attribute for our user that includes both the repository owner and the repository its self. This way we can write an RBAC rule against the repository. As an example, if we want every workflow in our repository to be able to act as an admin in our Namespace:
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: admin-binding
namespace: github-wf
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: admin
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: mlbiam/kube-wf-identity
With OpenUnison now setup to exchange our token for a kubectl configuration, and our cluster configured to authorize access, let's do something useful in our action!
Working With Kubernetes
So far we've configured our GitHub action workflow to get a token scoped for OpenUnison and created an AuthenticationChain to validate our token. The last steps are to update our STS Application to authenticate via GitHub tokens and to use our new short lived configuration from inside of a workflow:
apiVersion: openunison.tremolo.io/v2
kind: Application
metadata:
name: wftoken
namespace: openunison
spec:
azTimeoutMillis: 3000
cookieConfig:
cookiesEnabled: false
domain: '#[OU_HOST]'
httpOnly: true
keyAlias: session-unison
logoutURI: /logout
scope: -1
secure: true
sessionCookieName: tremolosession
timeout: 900
isApp: true
urls:
- authChain: github-token
azRules:
- constraint: o=Tremolo
scope: dn
filterChain:
- className: com.tremolosecurity.scalejs.token.ws.ScaleToken
params:
displayNameAttribute: sub
frontPage.text: Use this kubectl command to set your user in .kubectl/config. Refresh
this screen to generate a new set of tokens. Logging out will clear all
of your sessions.
frontPage.title: Kubernetes kubectl command
homeURL: /scale/
k8sCaCertName: '#[K8S_API_SERVER_CERT:unison-ca]'
kubectlTemplate: ' export TMP_CERT=\$(mktemp) && echo -e "$k8s_newline_cert$"
> \$TMP_CERT && kubectl config set-cluster #[K8S_CLUSTER_NAME:kubernetes]
--server=#[K8S_URL] --certificate-authority=\$TMP_CERT --embed-certs=true
&& kubectl config set-context #[K8S_CLUSTER_NAME:kubernetes] --cluster=#[K8S_CLUSTER_NAME:kubernetes]
--user=$user_id$@#[K8S_CLUSTER_NAME:kubernetes] && kubectl config set-credentials
$user_id$@#[K8S_CLUSTER_NAME:kubernetes] --auth-provider=oidc --auth-provider-arg=client-secret=
--auth-provider-arg=idp-issuer-url=$token.claims.issuer$ --auth-provider-arg=client-id=$token.trustName$
--auth-provider-arg=refresh-token=$token.refreshToken$ --auth-provider-arg=id-token=$token.encodedIdJSON$ --auth-provider-arg=idp-certificate-authority-data=#[IDP_CERT_DATA:$ou_b64_cert$] &&
kubectl config use-context #[K8S_CLUSTER_NAME:kubernetes] && rm \$TMP_CERT'
kubectlUsage: Run the kubectl command to set your user-context and server
connection
kubectlWinUsage: |
\$TMP_CERT=New-TemporaryFile ; "$k8s_newline_cert_win$" | out-file \$TMP_CERT -encoding oem ; kubectl config set-cluster #[K8S_CLUSTER_NAME:kubernetes] --server=#[K8S_URL] --certificate-authority=\$TMP_CERT --embed-certs=true ; kubectl config set-context #[K8S_CLUSTER_NAME:kubernetes] --cluster=#[K8S_CLUSTER_NAME:kubernetes] --user=$user_id$@#[K8S_CLUSTER_NAME:kubernetes] ; kubectl config set-credentials $user_id$@#[K8S_CLUSTER_NAME:kubernetes] --auth-provider=oidc --auth-provider-arg=client-secret= --auth-provider-arg=idp-issuer-url=$token.claims.issuer$ --auth-provider-arg=client-id=$token.trustName$ --auth-provider-arg=refresh-token=$token.refreshToken$ --auth-provider-arg=id-token=$token.encodedIdJSON$ --auth-provider-arg=idp-certificate-authority-data=$ou_b64_cert$ ; kubectl config use-context #[K8S_CLUSTER_NAME:kubernetes] ; Remove-Item -recurse -force \$TMP_CERT
logoutURL: /logout
oulogin: kubectl oulogin --host=#[OU_HOST]
tokenClassName: com.tremolosecurity.scalejs.KubectlTokenLoader
uidAttributeName: uid
unisonCaCertName: unison-ca
warnMinutesLeft: "5"
hosts:
- '#[OU_HOST]'
results:
auFail: default-login-failure
azFail: default-login-failure
uri: /wftoken/tokenThe only change we made from GitLab is that spec.urls[0].authChain is now github-token. Everything else is the exact same. Finally, let's add a step to our action to generate a kubectl configuration and get a ConfigMap from our Namespace:
- name: call kubernetes
run: |
export KUBECONFIG=$(mktemp)
curl -H "Authorization: Bearer $JWT" https://k8sou.domain.com/wftoken/token/user 2>/dev/null | jq -r '.token["kubectl Command"]' | sh
kubectl get cm -n github-wfAfter committing the updated action and pushing into our repository, let's see the output:
Run export KUBECONFIG=$(mktemp)
export KUBECONFIG=$(mktemp)
curl -H "Authorization: ***" https://k8sou.domain.com/wftoken/token/user 2>/dev/null | jq -r '.token["kubectl Command"]' | sh
kubectl get cm -n github-wf
shell: /usr/bin/bash -e {0}
env:
JWT: ***
Cluster "openunison-cp" set.
Context "openunison-cp" created.
User "repox-58-xmlbiamx-47-xkube-wf-identityx-58-xrefx-58-xrefsx-47-xheadsx-47-xmain@openunison-cp" set.
Switched to context "openunison-cp".
NAME DATA AGE
kube-root-ca.crt 1 2m47sThere you have it! Our GitHub action can now securely interact with our cluster! What's next? You probably have more then one CI/CD took I bet!
After GitHub
There are more CI/CD tools then just GitLab and GitHub! Next time, we'll cover how to validate access from a workflow running on another Kubernetes cluster!
If you want to learn more about deploying OpenUnison, check out our documentation site. If you're interested in a commercial support contract for your deployment, we'd love to hear from you!
