Terminology

We have learned that Kubernetes is an orchestration system to deploy and manage containers. Containers are not managed individually; instead, they are part of a larger object called a Pod. A Pod consists of one or more containers which share an IP address, access to storage and namespace. Typically, one container in a Pod runs an application, while other containers support the primary application.

Orchestration is managed through a series of watch-loops, also known as operators or controllers. Each operator interrogates the kube-apiserver for a particular object state, modifying the object until the declared state matches the current state. The default, newest, and feature-filled operator for containers is a Deployment. A Deployment deploys and manages a different operator called a ReplicaSet. A replicaSet is an operator which deploys multiple pods, each with the same spec information. These are called replicas. The Kubernetes architecture is made up of many operators such as Jobs and CronJobs to handle single or recurring tasks, or custom resource definitions and purpose-built operators.

There are several other API objects which can be used to deploy pods, other than ensuring a certain number of replicas is running somewhere. A DaemonSet will ensure that a single pod is deployed on every node. These are often used for logging, metrics, and security pods. A StatefulSet can be used to deploy pods in a particular order, such that following pods are only deployed if previous pods report a ready status. This is useful for legacy applications which are not cloud-friendly.

To easily manage thousands of Pods across hundreds of nodes can be difficult. To make management easier, we can use labels, arbitrary strings which become part of the object metadata. These are selectors which can then be used when checking or changing the state of objects without having to know individual names or UIDs. Nodes can have taints, an arbitrary string in the node metadata, to inform the scheduler on Pod assignments used along with toleration in Pod metadata, indicating it should be scheduled on a node with the particular taint.

There is also space in metadata for annotations, which remain with the object, but cannot be used as a selector; however, they could be leveraged by other objects or Pods.

While using lots of smaller, commodity hardware could allow every user to have their very own cluster, often multiple users and teams share access to one or more clusters. This is referred to as multi-tenancy. Some form of isolation is necessary in a multi-tenant cluster, using a combination of the following, which we introduce here but will learn more about in the future:

Namespace

A segregation of resources, upon which resource quotas and permissions can be applied. Kubernetes objects may be created in a namespace or cluster-scoped. Users can be limited by the object verbs allowed per namespace. Also the LimitRange admission controller constrains resource usage in that namespace. Two objects cannot have the same Name: value in the same namespace.

Context

A combination of user, cluster name and namespace. A convenient way to switch between combinations of permissions and restrictions. For example you may have a development cluster and a production cluster, or may be part of both the operations and architecture namespaces. This information is referenced from ~/.kube/config.

Resource Limits

A way to limit the amount of resources consumed by a pod, or to request a minimum amount of resources reserved, but not necessarily consumed, by a pod. Limits can also be set per-namespaces, which have priority over those in the PodSpec.

Pod Security Admission

A beta feature to restrict pod behavior in an easy-to-implement and easy-to-understand manner, applied at the namespace level when a pod is created. These will leverage three profiles: Privileged, Baseline, and Restricted policies.

Network Policies

The ability to have an inside-the-cluster firewall. Ingress and Egress traffic can be limited according to namespaces and labels as well as typical network traffic characteristics.