Charmed Kubernetes on AWS
Charmed Kubernetes will run seamlessly on AWS. With the addition of the
aws-integrator
and its companion charms, your cluster will also be able
to directly use AWS native features.
AWS integrator
The aws-integrator
charm simplifies working with Charmed Kubernetes on
AWS. Using the credentials provided to Juju, it acts as a proxy between
Charmed Kubernetes and the underlying cloud, granting permissions to
dynamically create, for example, EBS volumes.
AWS K8S Storage
The aws-k8s-storage
charm moves the AWS specific functions of the EBS csi-driver
out-of-tree. Using this charm, the drivers are installed as workloads in the Kubernetes
cluster instead of as natural code paths of the Kubernetes binaries.
AWS Cloud Provider
The aws-cloud-provider
moves the AWS specific functions of the cloud-provider
out-of-tree. The AWS cloud provider provides the interface between a Kubernetes cluster
and AWS service APIs. This project allows a Kubernetes cluster to provision,
monitor and remove AWS resources necessary for operation of the cluster.
Version support
From Kubernetes 1.27
The in-tree cloud-provider is no longer available, and must be deployed
as container workloads in the cluster. Charmed Kubernetes recommends
using the aws-cloud-provider
charm to access AWS Service APIs.
Prior to Kubernetes 1.27
The in-tree cloud-provider is natively available in Kubernetes until the 1.27
release, and it is not necessary to deploy the aws-cloud-provider
charm as in the
above overlay.
Installing
If you install Charmed Kubernetes [using the Juju bundle][install], you can add the aws-integrator at the same time by using the following cloud-provider overlay file ([download it here][asset-aws-overlay]):
description: Charmed Kubernetes overlay to add native AWS support.
applications:
aws-integrator:
charm: aws-integrator
num_units: 1
trust: true
aws-cloud-provider:
charm: aws-cloud-provider
relations:
- ['aws-integrator', 'kubernetes-control-plane']
- ['aws-integrator', 'kubernetes-worker']
- ["aws-cloud-provider:certificates", "easyrsa"]
- ["aws-cloud-provider:kube-control", "kubernetes-control-plane"]
- ["aws-cloud-provider:external-cloud-provider", "kubernetes-control-plane"]
- ["aws-cloud-provider:aws-integration", "aws-integrator"]
As well as the storage overlay file ([download it here][asset-aws-storage-overlay]):
description: Charmed Kubernetes overlay to add native AWS support.
applications:
kubernetes-control-plane:
options:
allow-privileged: "true"
aws-integrator:
charm: aws-integrator
num_units: 1
trust: true
aws-k8s-storage:
charm: aws-k8s-storage
trust: true
options:
image-registry: public.ecr.aws
relations:
- ['aws-k8s-storage:certificates', 'easyrsa:client']
- ['aws-k8s-storage:kube-control', 'kubernetes-control-plane:kube-control']
- ['aws-k8s-storage:aws-integration', 'aws-integrator:aws']
# Include the following relations if not using the aws-cloud-provider charm
# - ['aws-integrator', 'kubernetes-control-plane']
# - ['aws-integrator', 'kubernetes-worker']
To use these overlays with the Charmed Kubernetes bundle, it is specified during deploy like this:
juju deploy charmed-kubernetes --overlay ~/path/aws-overlay.yaml --overlay ~/path/aws-storage-overlay.yaml --trust
… and remember to fetch the configuration file!
juju ssh kubernetes-control-plane/leader -- cat config > ~/.kube/config
For more configuration options and details of the permissions which the integrator uses, please see the [charm readme][aws-integrator-readme].
Using EBS volumes
Many pods you may wish to deploy will require storage. Although you can use any type of storage supported by Kubernetes (see the [storage documentation][storage]), you also have the option to use the native AWS storage, Elastic Block Store (EBS).
Beginning in Kubernetes 1.25
The aws-k8s-storage
charm will need to be installed to make use of EBS Volumes.
Amazon removed CSIMigration away from the in-tree binaries but made them available
as container workload in the cluster. This charm installs and relates to the
existing integrator charm.
A StorageClass will be created by this charm named csi-aws-ebs-default
You can confirm this has been added by running:
kubectl get sc
which should return:
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
csi-aws-ebs-default ebs.csi.aws.com Delete WaitForFirstConsumer false 9s
Prior to Kubernetes 1.25
First we need to create a storage class which can be used by Kubernetes. To start with, we will create one for the ‘General Purpose SSD’ type of EBS storage:
kubectl create -f - <<EOY
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: ebs-gp2
provisioner: kubernetes.io/aws-ebs
parameters:
type: gp2
EOY
You can confirm this has been added by running:
kubectl get sc
which should return:
NAME PROVISIONER AGE
ebs-gp2 kubernetes.io/aws-ebs 39s
You can create additional storage classes for the other types of EBS storage if needed, simply give them a different name and replace the ‘type: gp2’ with a different type (See the [AWS website][ebs-info] for more information on the available types).
Creating a PVC
To actually create storage using this new class, you can make a Persistent Volume Claim:
kubectl create -f - <<EOY
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: testclaim
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 100Mi
storageClassName: ebs-gp2
EOY
This should finish with a confirmation. You can check the current PVCs with:
kubectl get pvc
…which should return something similar to:
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
testclaim Bound pvc-54a94dfa-3128-11e9-9c54-028fdae42a8c 1Gi RWO ebs-gp2 9s
This PVC can then be used by pods operating in the cluster. As an example, the following
deploys a busybox
pod:
kubectl create -f - <<EOY
apiVersion: v1
kind: Pod
metadata:
name: busybox
namespace: default
spec:
containers:
- image: busybox
command:
- sleep
- "3600"
imagePullPolicy: IfNotPresent
name: busybox
volumeMounts:
- mountPath: "/pv"
name: testvolume
restartPolicy: Always
volumes:
- name: testvolume
persistentVolumeClaim:
claimName: testclaim
EOY
<span class="p-notification__title">Note:</span>
<p class="p-notification__message">If you create EBS volumes and subsequently tear down the cluster, check with the AWS console to make sure all the associated resources have also been released.</p>
Using ELB Loadbalancers
With the aws-integrator charm in place, actions which invoke a loadbalancer in Kubernetes will automatically generate an AWS Elastic Load Balancer. This can be demonstrated with a simple application. Here we will create a simple application and scale it to five pods:
kubectl create deployment hello-world --image=gcr.io/google-samples/node-hello:1.0
kubectl scale deployment hello-world --replicas=5
You can verify that the application and replicas have been created with:
kubectl get deployments hello-world
Which should return output similar to:
NAME READY UP-TO-DATE AVAILABLE AGE
hello-world 5/5 5 5 2m38s
To create a LoadBalancer, the application should now be exposed as a service:
kubectl expose deployment hello-world --type=LoadBalancer --name=hello --port 8080
To check that the service is running correctly:
kubectl describe service hello
…which should return output similar to:
Name: hello
Namespace: default
Labels: run=load-balancer-example
Annotations: <none>
Selector: run=load-balancer-example
Type: LoadBalancer
IP: 10.152.183.134
LoadBalancer Ingress: ad5fc7750350611e99768068a686bb67-239702253.eu-west-1.elb.amazonaws.com
Port: <unset> 8080/TCP
TargetPort: 8080/TCP
NodePort: <unset> 31203/TCP
Endpoints: 10.1.13.4:8080,10.1.13.5:8080,10.1.35.8:8080 + 2 more...
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
You can see that the LoadBalancer Ingress is now associated with an ELB address in front of the five endpoints of the example deployment. Leaving a while for DNS propagation, you can test the ingress address:
curl http://ad5fc7750350611e99768068a686bb67-239702253.eu-west-1.elb.amazonaws.com:8080
Hello Kubernetes!
See the guide to contributing or discuss these docs in our public Mattermost channel.