Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions dev-docs/k8s/pod-resources.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Pod resources

## Background

There has been some confusion around the use of memory limits in Kata.
In this doc, you'll find some pointers to how pod resource limits are determined by Kubernetes and how they're implemented in Kata.

## How memory limits are calculated

* Entrypoint: `ResourceConfigForPod` in <https://github.com/kubernetes/kubernetes/blob/v1.31.5/pkg/kubelet/cm/helpers_linux.go#L121>.
* Formula: `PodLimits` in <https://github.com/kubernetes/kubernetes/blob/v1.31.5/pkg/api/v1/resource/helpers.go#L152>.

## How memory limits are propagated

From Kubelet to containerd via CRI method `RunPodSandbox`:

```go
cri.RunPodSandboxRequest{
Config: &cri.PodSandboxConfig{
Linux: &cri.LinuxPodSandboxConfig{
Resources: &cri.LinuxContainerResources{
MemoryLimitInBytes: 1234,
},
},
},
}
```

From containerd to Kata via sandbox container (that is, pause container) annotation: <https://github.com/containerd/containerd/blob/v2.0.0/internal/cri/server/podsandbox/sandbox_run_linux.go#L186>.

Kata converts the annotation to MiB and uses it to calculate the VM size: <https://github.com/kata-containers/kata-containers/blob/3.18.0/src/runtime/pkg/oci/utils.go#L1391>.

## Examples

`default_memory` in our config is 512, the experiments were conducted from <https://github.com/edgelesssys/contrast/commits/a0021ad6b32afaa45b031ba015087de8d7502c8e>.

| Main Container Limit | Init Container Limit | Sidecar Container Limit | Annotation Value | QEMU Parameter |
| -------------------- | -------------------- | ----------------------- | ---------------- | -------------- |
| 100 MB | 200 MB | — | 209715200 | `-m 712M` |
| 100 MB | — | 200 MB | 314572800 | `-m 812M` |
| 100 MB | — | No limit | 104857600 | `-m 612M` |
2 changes: 1 addition & 1 deletion dev-docs/nvidia/cdi.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# CDI

This page explains how devices end up in containers when using CDI annotations.
Reading [life of a confidential container](../aks/life-of-a-confidential-container.md) first is recommended to understand the flow presented here.
Reading [life of a confidential container](../k8s/life-of-a-confidential-container.md) first is recommended to understand the flow presented here.

## The journey begins

Expand Down
Loading