Kubernetes

The OpenEng Kubernetes app is a full cluster console — dashboard, resource explorer, topology, nodes, an interactive pod terminal, and live events — split across two halves that trust each other and almost nothing else: a local sealed engine bound to 127.0.0.1:50054 that does all cluster access, and a thin web UI at k8s.openeng.app that only renders panels and issues operations. Your kube credentials — tokens, client certs, exec-credential plugins — are resolved entirely inside the engine, on your machine, from your own kubeconfig. They never cross the wire to the browser, and they never egress to OpenEng.

Overview

The browser cannot reach a Kubernetes API server directly, and you would not want it to hold your cluster credentials if it could. The app resolves that by keeping the two concerns physically apart:

Every call between them travels the same single encrypted boundary. The transport is gRPC + gRPC-Web over loopback HTTPS, with each frame sealed under a per-session key exchanged in CreateSession (the client sends client_pub, the engine returns engine_pub; every subsequent RPC is sealed per-frame). A default-deny auth interceptor guards the port, and the browser pairs by fetching a 0600 handshake token the engine mints at boot. The security properties are covered under Security model and engine updates.

Connections and clusters

The engine presents a single flat list of clusters via ListClusters, blending two sources into one uniform Cluster row. The active one is the current context; you switch with SetContext and the header cluster picker.

Switching and connecting

The active context is URL-addressable: ?ctx= in the UI drives it, and the header switcher writes it. Switching to a context runs a small pipeline — SetContext, then ConnectCluster (which builds/validates the client, or for a connector calls cluster_info), then discovery (ListNamespaces + ListResourceKinds). The relevant RPCs:

ListClusters      list every kubeconfig context + connector cluster
GetContext        the engine's current context
SetContext        switch the active context (validated against known contexts)
ConnectCluster    build + verify the client; returns the API server version
ClusterHealth     api reachable? server version, nodes ready / total

Both kinds switch identically from the UI's point of view — the header picker, the Settings clusters list, and a deep-linked ?ctx all funnel through the same switch.

Importing a kubeconfig

You do not have to depend on whatever is already in ~/.kube/config. In Settings → Clusters, paste a kubeconfig YAML with a name and click Add kubeconfig; the UI calls AddKubeconfig. The engine:

1. Parses the YAML and rejects anything with no contexts (or invalid syntax).
2. Writes the fragment 0600 under ~/.openeng/kubernetes/kubeconfigs/ so it survives restarts — the file may carry credentials, so it is never logged.
3. Merges it into the live registry and returns the newly available context names, which appear immediately in ListClusters.

Cloud clusters via connector

A connector cluster is a cluster reached through an OpenEng cloud-Kubernetes connector — a deployed service (e.g. a Lambda) that runs inside your own cloud/VPC, speaks the OpenEng tools protocol over HTTPS, and reaches the API server from there. The engine never holds the cluster's credentials; it only holds your key to your connector, and it sees only the connector's trimmed JSON responses.

Adding one

Settings → Clusters → Add a cloud cluster calls AddConnectorCluster. The form fields map directly onto the request:

On add, the engine probes the connector's GET /manifest to (a) validate reachability and the credentials and (b) detect whether the connector is serving read-only. It then persists the connector 0600 under ~/.openeng/kubernetes/connectors/ and returns the assigned context plus the detected read_only flag. A 401/403 comes back stamped with the CLUSTER_UNAUTHORIZED sentinel so the UI can say "check the API key" rather than "unreachable".

How reads are served

For a connector context the engine routes each operation to a connector tool over POST /call-stream (NDJSON), reducing the streamed frames back into one result for unary RPCs. An older connector that predates streaming (which 404s /call-stream) is transparently retried over the buffered POST /call, so a not-yet-redeployed connector is never broken. A connector exposes no discovery API, so the engine advertises a static set of common kinds for the Explorer.

ListNamespaces  -> list_namespaces        ConnectCluster/Health -> cluster_info
ListResources   -> list_<plural>          GetResource/Describe  -> get_<kind>
ListPods        -> list_pods              GetPodLogs (stream)   -> pod_logs
ListNodes       -> list_nodes             GetMetrics            -> node_metrics | pod_metrics
ExecPod         -> pod_exec (per-command) ApplyResource         -> apply_manifest
DeleteResource  -> delete_resource        Cordon/Uncordon       -> node_cordon | node_uncordon

Re-authentication

When a connector later rejects the stored credentials mid-session (any RPC returning CLUSTER_UNAUTHORIZED), the UI raises a re-auth dialog — but only if that connector is still the active cluster, so a stale late 401 from a cluster you just left cannot pop a dialog. GetConnector reads the stored api_key, auth_token, and write_key back to the local UI to pre-fill the dialog; you edit, save, and retry. That echo stays strictly on the loopback seal boundary — the same trust domain that entered the values — and echoing the write key back means a re-auth save never silently drops your write access. Remove a connector with RemoveConnectorCluster (it deletes the persisted file).

Scopes: read-only, write, and gating

Local kubeconfig clusters carry no OpenEng-level scope — whatever your kube RBAC allows, the engine allows; RBAC is your authority. Connector clusters, by contrast, have an explicit read-only / write scope enforced at two layers.

Live read-only re-probe

Read-only status is not frozen at add time. Every ListClusters re-probes each connector's /manifest, so a connector that flips read-only ⇄ write (e.g. its env changed) takes effect on the next refresh — the UI enables or disables mutating actions immediately, with no re-add.

Force-exec prefixes

A read-only connector can still be allowed to exec into specific pods via its K8S_FORCE_EXEC allowlist — a set of pod-name prefixes surfaced on each cluster row as force_exec (refreshed on every ListClusters manifest re-probe). The UI uses these to decide, per pod, whether to offer an exec console even on a read-only cluster. Because the connector enforces the allowlist itself and refuses non-matching pods, the engine deliberately does not apply guard_read_only to connector exec — a refused pod comes back as an error in PtyDone.status.

The workspace

Once a cluster is active, the console is organised into sections; each (except Settings) requires a connected context and otherwise prompts you to pick one.

Settings is reachable without a connected cluster — it hosts the local engine status and the clusters management (kubeconfig import + connector add / switch / remove). The current context (?ctx) and namespace (?ns) are carried across sections in the URL; per-view state like the open drawer is dropped when you navigate.

Namespaces and resources

Discovery is two RPCs: ListNamespaces for the namespace switcher and ListResourceKinds for the kind picker (group / version / kind / plural / namespaced). A local cluster enumerates kinds from the API server's discovery; a connector advertises a static common set.

Generic resource operations

Everything in the Explorer runs through a small, kind-agnostic set of RPCs that work for any kind via the dynamic API:

Pods, logs, and exec

ListPods returns the compact pod shape — phase, node, IP, restarts, ready/total, age, containers — for the pods pane. Two richer surfaces sit on top.

Streaming logs

GetPodLogs streams LogEvents. It takes container, follow (tail), previous (the crashed container's prior instance), tail_lines, and since (e.g. 1h, 10m). On a connector the logs stream over /call-stream, forwarding each NDJSON text chunk into the same LogEvent stream as it arrives, with the same tail semantics.

Interactive exec

The exec surface mirrors the Terminal engine's PTY shape: a server stream of PtyEvent (started / output / done) for output, and unary RPCs for input — WriteExec, ResizeExec, CloseExec. There are two execution modes depending on the cluster kind:

Tab-completion for the connector console

Because a connector console has no live shell to complete paths, the engine serves completion explicitly via CompleteExec. Given the input line up to the cursor it resolves the trailing path token against the pod's filesystem with a single ls run under the session's stored cwd/env — deliberately without advancing that state (completion must never mutate the session it inspects). It returns the directory's full listing (dirs end in /, which the UI caches and filters by prefix), the fragment being completed, and the current cwd. A first bare word is treated as a command name and completed by the UI from its own catalog + history. Failures come back as a message, never a hard error — a completion that can't answer is a no-op at the prompt, not a broken console.

Port-forwarding and events

Port-forward

StartPortForward opens a local listener that tunnels local_port → pod_port through the engine; ListPortForwards enumerates active tunnels and StopPortForward tears one down. Port-forward is served through a local kube::Client, so it is available on kubeconfig clusters only.

Events

WatchEvents streams cluster Events for a namespace — type (Normal / Warning), reason, message, involved object, last-seen, and count — feeding the Events pane and the Overview's recent-activity views. Like watch and port-forward, the events stream runs through the local client and is not routed to connectors.

Helm releases

The Helm pane manages releases through four RPCs: HelmList, HelmInstall and HelmUpgrade (each a ProgressEvent stream of live output lines), and HelmUninstall. Install/upgrade take release, chart, version, and values; uninstall takes an optional keep_history.

Nodes and metrics

Nodes

ListNodes returns each node's readiness, schedulability, roles, kubelet version, age, CPU/memory capacity, taints, and conditions. Node operations:

• CordonNode / UncordonNode — mark a node un/schedulable. Routed to node_cordon / node_uncordon on a connector, and write-gated by guard_read_only there.
• DrainNode — evict pods, streaming progress. Served through the local client only (kubeconfig clusters).

Metrics

GetMetrics takes a scope of node or pod and returns CPU cores, memory bytes, and their percentages per entity — powering the Overview sparklines, top-consumers, and node-capacity views. On a local cluster it reads the metrics API (a metrics-server must be present); on a connector it routes to node_metrics / pod_metrics. Metrics is a read and is never read-only-gated.

Local vs connector capability matrix

Because a connector reaches your cluster through a tool API rather than a live client, a handful of streaming/tunnelling operations are available only on kubeconfig clusters. The full picture:

Reusing a cloud cluster in Chat

A connector cluster you add here is not private to this app. Because connectors are persisted by the engine under ~/.openeng/kubernetes/connectors/ and exposed through the standard ListClusters RPC, the OpenEng Chat app's coding agent can drive the very same cluster — without ever re-entering the credentials.

Chat's engine opens its own sealed loopback session to the Kubernetes engine (the same 127.0.0.1:50054, addressed as engine.openeng.app:50054 pinned to loopback, SNI + Host engine.openeng.app) and calls the typed RPCs on your behalf: ListClusters, ListNamespaces, ListPods, DescribeResource, GetPodLogs, ListResources, and so on. A Chat group's config carries a connection_id that is exactly the Kubernetes context id — a plain context like prod, or a connector like connector:my-cloud — and an empty id means the K8s engine's current context.

The key property: the connector's API key, bearer, and write key never enter Chat. Chat asks the Kubernetes engine by context id; the Kubernetes engine is the sole holder of those credentials and performs the connector call itself. The read-only / write scope you configured here is enforced there too — if the connector is read-only, the agent's mutations are refused the same way the UI's are.

Security model and engine updates

The no-egress invariant

The proto that defines this boundary has a hard rule, enforced by a schema test: no field may carry a secret, credential, API key, password, private key, or token. Cluster auth is resolved locally by kube from your kubeconfig and never crosses the wire. The legitimate payload is only your own cluster data — resource YAML, logs, exec bytes.

There is exactly one deliberate exception: a connector cluster's api_key / auth_token / write_key, explicitly marked connector-credential and exempted from the egress test. That is your key for your connector — handed to the engine once on AddConnectorCluster, stored locally 0600, and only ever echoed back to the local UI (over the loopback seal) to pre-fill the re-auth dialog. It is not a product secret leaking out.

Boundary hardening

Mandatory engine auto-update

The engine self-updates at boot and cannot be skipped. While it is serving, a newer stable build is downloaded, verified, and staged in the background; CreateSession and GetUpdateStatus report the running version plus any staged_version awaiting a restart. The UI surfaces a "restart to update" banner that calls RestartEngine — the engine re-execs in place into the staged binary, carrying the live session internally so you never re-pair, and the UI re-probes and reconnects to the new version. No credential rides either update RPC; they carry only version strings.