Introducing Gateway API Inference Extension

https://kubernetes.io/blog/2025/06/05/introducing-gateway-api-inference-extension/

Modern generative AI and large language model (LLM) services create unique traffic-routing challenges on Kubernetes. Unlike typical short-lived, stateless web requests, LLM inference sessions are often long-running, resource-intensive, and partially stateful. For example, a single GPU-backed model server may keep multiple inference sessions active and maintain in-memory token caches.

Traditional load balancers focused on HTTP path or round-robin lack the specialized capabilities needed for these workloads. They also don’t account for model identity or request criticality (e.g., interactive chat vs. batch jobs). Organizations often patch together ad-hoc solutions, but a standardized approach is missing.

Gateway API Inference Extension

Gateway API Inference Extension was created to address this gap by building on the existing Gateway API, adding inference-specific routing capabilities while retaining the familiar model of Gateways and HTTPRoutes. By adding an inference extension to your existing gateway, you effectively transform it into an Inference Gateway, enabling you to self-host GenAI/LLMs with a “model-as-a-service” mindset.

The project’s goal is to improve and standardize routing to inference workloads across the ecosystem. Key objectives include enabling model-aware routing, supporting per-request criticalities, facilitating safe model roll-outs, and optimizing load balancing based on real-time model metrics. By achieving these, the project aims to reduce latency and improve accelerator (GPU) utilization for AI workloads.

How it works

The design introduces two new Custom Resources (CRDs) with distinct responsibilities, each aligning with a specific user persona in the AI/ML serving workflow​:

InferencePool Defines a pool of pods (model servers) running on shared compute (e.g., GPU nodes). The platform admin can configure how these pods are deployed, scaled, and balanced. An InferencePool ensures consistent resource usage and enforces platform-wide policies. An InferencePool is similar to a Service but specialized for AI/ML serving needs and aware of the model-serving protocol.

InferenceModel A user-facing model endpoint managed by AI/ML owners. It maps a public name (e.g., "gpt-4-chat") to the actual model within an InferencePool. This lets workload owners specify which models (and optional fine-tuning) they want served, plus a traffic-splitting or prioritization policy.

In summary, the InferenceModel API lets AI/ML owners manage what is served, while the InferencePool lets platform operators manage where and how it’s served.

Request flow

The flow of a request builds on the Gateway API model (Gateways and HTTPRoutes) with one or more extra inference-aware steps (extensions) in the middle. Here’s a high-level example of the request flow with the Endpoint Selection Extension (ESE):

Gateway Routing

A client sends a request (e.g., an HTTP POST to /completions). The Gateway (like Envoy) examines the HTTPRoute and identifies the matching InferencePool backend.

Endpoint Selection

Instead of simply forwarding to any available pod, the Gateway consults an inference-specific routing extension— the Endpoint Selection Extension—to pick the best of the available pods. This extension examines live pod metrics (queue lengths, memory usage, loaded adapters) to choose the ideal pod for the request.

Inference-Aware Scheduling

The chosen pod is the one that can handle the request with the lowest latency or highest efficiency, given the user’s criticality or resource needs. The Gateway then forwards traffic to that specific pod.

This extra step provides a smarter, model-aware routing mechanism that still feels like a normal single request to the client. Additionally, the design is extensible—any Inference Gateway can be enhanced with additional inference-specific extensions to handle new routing strategies, advanced scheduling logic, or specialized hardware needs. As the project continues to grow, contributors are encouraged to develop new extensions that are fully compatible with the same underlying Gateway API model, further expanding the possibilities for efficient and intelligent GenAI/LLM routing.

Benchmarks

We evaluated ​this extension against a standard Kubernetes Service for a vLLM‐based model serving deployment. The test environment consisted of multiple H100 (80 GB) GPU pods running vLLM (version 1) on a Kubernetes cluster, with 10 Llama2 model replicas. The Latency Profile Generator (LPG) tool was used to generate traffic and measure throughput, latency, and other metrics. The ShareGPT dataset served as the workload, and traffic was ramped from 100 Queries per Second (QPS) up to 1000 QPS.

Key results

Comparable Throughput: Throughout the tested QPS range, the ESE delivered throughput roughly on par with a standard Kubernetes Service.

Lower Latency:

Per‐Output‐Token Latency: The ​ESE showed significantly lower p90 latency at higher QPS (500+), indicating that its model-aware routing decisions reduce queueing and resource contention as GPU memory approaches saturation.

Overall p90 Latency: Similar trends emerged, with the ​ESE reducing end‐to‐end tail latencies compared to the baseline, particularly as traffic increased beyond 400–500 QPS.

These results suggest that this extension's model‐aware routing significantly reduced latency for GPU‐backed LLM workloads. By dynamically selecting the least‐loaded or best‐performing model server, it avoids hotspots that can appear when using traditional load balancing methods for large, long‐running inference requests.

Roadmap

As the Gateway API Inference Extension heads toward GA, planned features include:

Prefix-cache aware load balancing for remote caches

LoRA adapter pipelines for automated rollout

Fairness and priority between workloads in the same criticality band

HPA support for scaling based on aggregate, per-model metrics

Support for large multi-modal inputs/outputs

Additional model types (e.g., diffusion models)

Heterogeneous accelerators (serving on multiple accelerator types with latency- and cost-aware load balancing)

Disaggregated serving for independently scaling pools

Summary

By aligning model serving with Kubernetes-native tooling, Gateway API Inference Extension aims to simplify and standardize how AI/ML traffic is routed. With model-aware routing, criticality-based prioritization, and more, it helps ops teams deliver the right LLM services to the right users—smoothly and efficiently.

Ready to learn more? Visit the project docs to dive deeper, give an Inference Gateway extension a try with a few simple steps, and get involved if you’re interested in contributing to the project!

via Kubernetes Blog https://kubernetes.io/

June 04, 2025 at 08:00PM

Week Ending June 1, 2025

https://lwkd.info/2025/20250604

Developer News

The Enhancements subteam issued a reminder about tracking changes. Contributors must opt in for tracking if their KEP includes user-facing changes or behavior affecting improvements, even if the KEP doesn’t graduate to a new stage. Pure refactors, non-behavioural improvements, and bug fixes do not require tracking by the Release Team.

Release Schedule

Next Deadline: PRR Freeze, June 12th

This is the last week-and-a-half before those KEPs need to be ready for production readiness review with all the PRR questions answered.

Cherry-picks for the June Patch Releases are due on June 6.

KEP of the Week

KEP-3331: Structured Authentication Config

This KEP introduces the capability to add structured authentication configuration to the Kubernetes API server, using a new API Object called AuthenticationConfiguration. It supports a JWT token, which serves as the next stage for the OIDC authenticator. Previously, authentication for the API server was performed using command-line flags, which were difficult to manage, validate, and lacked consistency. The KEP implements the Kubernetes API schema for validation and provides a standardized, extensible format, improving configuration clarity.

This KEP is tracked as stable in v1.34.

Other Merges

kuberc adds tests for DefaultGetPreferences

PVCs annotated with node-expand-not-required to not be expanded

Pod admission and resize logic moved into the allocation manager

Stress tests added for VolumeAttributesClass

New –show-swap option for kubectl top subcommands

5s delay of tainting node.kubernetes.io/unreachable:NoExecute reduced when a Node becomes unreachable

kubelet: the –image-credential-provider-config flag can now specify a directory path as well

Moved Scheduler interfaces to staging

agnhost: added server address for conntrack cleanup entries

kube-proxy: Remove iptables cli wait interval flag

DRA: kubelet added validation before changing claim info cache

DRA: Improvements to the implementation of counter management in allocator

Promotions

RelaxedDNSSearchValidation to GA

Version Updates

kube-dns to v1.26.4

Subprojects and Dependency Updates

Python client 32.0.0 Alpha is released, as well as version 33.1.0 Beta.

via Last Week in Kubernetes Development https://lwkd.info/

June 04, 2025 at 09:04AM

My pipelines from GitLab Commit to ArgoCD got beaten by FTP, with David Pech

https://ku.bz/_MWX5m6G_

A sophisticated GitLab CI/CD pipeline integrated with Argo CD was ultimately rejected in favour of simple FTP deployment, offering crucial insights into the real barriers facing cloud-native adoption in traditional organisations.

David Pech, Staff Cloud Ops Engineer at Wrike and holder of all CNCF certifications, shares his experience supporting a PHP team after a company merger. He details how he built a complete cloud-native platform with Kubernetes, Helm charts, and GitOps workflows, only to see it fail against cultural and organizational resistance despite its technical superiority.

You will learn:

The hidden costs of sophisticated tooling - How GitOps pipelines with multiple moving parts can create trust issues when developers lose local control and must rely on remote processes they don't understand

Cultural factors that trump technical benefits - Why customer expectations, existing Windows-based infrastructure, and team readiness matter more than the elegance of your Kubernetes solution

Practical strategies for incremental adoption - The importance of starting small, building in-house operational expertise, and ensuring management advocacy at all levels before attempting cloud-native transformations

Sponsor

This episode is sponsored by Learnk8s — get started on your Kubernetes journey through comprehensive online, in-person or remote training.

More info

Find all the links and info for this episode here: https://ku.bz/_MWX5m6G_

Interested in sponsoring an episode? Learn more.

via KubeFM https://kube.fm

June 03, 2025 at 06:00AM

Start Sidecar First: How To Avoid Snags

https://kubernetes.io/blog/2025/06/03/start-sidecar-first/

From the Kubernetes Multicontainer Pods: An Overview blog post you know what their job is, what are the main architectural patterns, and how they are implemented in Kubernetes. The main thing I’ll cover in this article is how to ensure that your sidecar containers start before the main app. It’s more complicated than you might think!

A gentle refresher

I'd just like to remind readers that the v1.29.0 release of Kubernetes added native support for sidecar containers, which can now be defined within the .spec.initContainers field, but with restartPolicy: Always. You can see that illustrated in the following example Pod manifest snippet:

initContainers:

• name: logshipper image: alpine:latest restartPolicy: Always # this is what makes it a sidecar container command: ['sh', '-c', 'tail -F /opt/logs.txt'] volumeMounts:
• name: data mountPath: /opt

What are the specifics of defining sidecars with a .spec.initContainers block, rather than as a legacy multi-container pod with multiple .spec.containers? Well, all .spec.initContainers are always launched before the main application. If you define Kubernetes-native sidecars, those are terminated after the main application. Furthermore, when used with Jobs, a sidecar container should still be alive and could potentially even restart after the owning Job is complete; Kubernetes-native sidecar containers do not block pod completion.

To learn more, you can also read the official Pod sidecar containers tutorial.

The problem

Now you know that defining a sidecar with this native approach will always start it before the main application. From the kubelet source code, it's visible that this often means being started almost in parallel, and this is not always what an engineer wants to achieve. What I'm really interested in is whether I can delay the start of the main application until the sidecar is not just started, but fully running and ready to serve. It might be a bit tricky because the problem with sidecars is there’s no obvious success signal, contrary to init containers - designed to run only for a specified period of time. With an init container, exit status 0 is unambiguously "I succeeded". With a sidecar, there are lots of points at which you can say "a thing is running". Starting one container only after the previous one is ready is part of a graceful deployment strategy, ensuring proper sequencing and stability during startup. It’s also actually how I’d expect sidecar containers to work as well, to cover the scenario where the main application is dependent on the sidecar. For example, it may happen that an app errors out if the sidecar isn’t available to serve requests (e.g., logging with DataDog). Sure, one could change the application code (and it would actually be the “best practice” solution), but sometimes they can’t - and this post focuses on this use case.

I'll explain some ways that you might try, and show you what approaches will really work.

Readiness probe

To check whether Kubernetes native sidecar delays the start of the main application until the sidecar is ready, let’s simulate a short investigation. Firstly, I’ll simulate a sidecar container which will never be ready by implementing a readiness probe which will never succeed. As a reminder, a readiness probe checks if the container is ready to start accepting traffic and therefore, if the pod can be used as a backend for services.

(Unlike standard init containers, sidecar containers can have probes so that the kubelet can supervise the sidecar and intervene if there are problems. For example, restarting a sidecar container if it fails a health check.)

apiVersion: apps/v1 kind: Deployment metadata: name: myapp labels: app: myapp spec: replicas: 1 selector: matchLabels: app: myapp template: metadata: labels: app: myapp spec: containers:

• name: myapp image: alpine:latest command: ["sh", "-c", "sleep 3600"] initContainers:
• name: nginx image: nginx:latest restartPolicy: Always ports:
• containerPort: 80 protocol: TCP readinessProbe: exec: command:
• /bin/sh
• -c
• exit 1 # this command always fails, keeping the container "Not Ready" periodSeconds: 5 volumes:
• name: data emptyDir: {}

The result is:

controlplane $ kubectl get pods -w NAME READY STATUS RESTARTS AGE myapp-db5474f45-htgw5 1/2 Running 0 9m28s

controlplane $ kubectl describe pod myapp-db5474f45-htgw5 Name: myapp-db5474f45-htgw5 Namespace: default (...) Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 17s default-scheduler Successfully assigned default/myapp-db5474f45-htgw5 to node01 Normal Pulling 16s kubelet Pulling image "nginx:latest" Normal Pulled 16s kubelet Successfully pulled image "nginx:latest" in 163ms (163ms including waiting). Image size: 72080558 bytes. Normal Created 16s kubelet Created container nginx Normal Started 16s kubelet Started container nginx Normal Pulling 15s kubelet Pulling image "alpine:latest" Normal Pulled 15s kubelet Successfully pulled image "alpine:latest" in 159ms (160ms including waiting). Image size: 3652536 bytes. Normal Created 15s kubelet Created container myapp Normal Started 15s kubelet Started container myapp Warning Unhealthy 1s (x6 over 15s) kubelet Readiness probe failed:

From these logs it’s evident that only one container is ready - and I know it can’t be the sidecar, because I’ve defined it so it’ll never be ready (you can also check container statuses in kubectl get pod -o json). I also saw that myapp has been started before the sidecar is ready. That was not the result I wanted to achieve; in this case, the main app container has a hard dependency on its sidecar.

Maybe a startup probe?

To ensure that the sidecar is ready before the main app container starts, I can define a startupProbe. It will delay the start of the main container until the command is successfully executed (returns 0 exit status). If you’re wondering why I’ve added it to my initContainer, let’s analyse what happens If I’d added it to myapp container. I wouldn’t have guaranteed the probe would run before the main application code - and this one, can potentially error out without the sidecar being up and running.

apiVersion: apps/v1 kind: Deployment metadata: name: myapp labels: app: myapp spec: replicas: 1 selector: matchLabels: app: myapp template: metadata: labels: app: myapp spec: containers:

• name: myapp image: alpine:latest command: ["sh", "-c", "sleep 3600"] initContainers:
• name: nginx image: nginx:latest ports:
• containerPort: 80 protocol: TCP restartPolicy: Always startupProbe: httpGet: path: / port: 80 initialDelaySeconds: 5 periodSeconds: 30 failureThreshold: 10 timeoutSeconds: 20 volumes:
• name: data emptyDir: {}

This results in 2/2 containers being ready and running, and from events, it can be inferred that the main application started only after nginx had already been started. But to confirm whether it waited for the sidecar readiness, let’s change the startupProbe to the exec type of command:

startupProbe: exec: command:

• /bin/sh
• -c
• sleep 15

and run kubectl get pods -w to watch in real time whether the readiness of both containers only changes after a 15 second delay. Again, events confirm the main application starts after the sidecar. That means that using the startupProbe with a correct startupProbe.httpGet request helps to delay the main application start until the sidecar is ready. It’s not optimal, but it works.

What about the postStart lifecycle hook?

Fun fact: using the postStart lifecycle hook block will also do the job, but I’d have to write my own mini-shell script, which is even less efficient.

initContainers:

• name: nginx image: nginx:latest restartPolicy: Always ports:
• containerPort: 80 protocol: TCP lifecycle: postStart: exec: command:
• /bin/sh
• -c
• | echo "Waiting for readiness at http://localhost:80" until curl -sf http://localhost:80; do echo "Still waiting for http://localhost:80..." sleep 5 done echo "Service is ready at http://localhost:80"

Liveness probe

An interesting exercise would be to check the sidecar container behavior with a liveness probe. A liveness probe behaves and is configured similarly to a readiness probe - only with the difference that it doesn’t affect the readiness of the container but restarts it in case the probe fails.

livenessProbe: exec: command:

• /bin/sh
• -c
• exit 1 # this command always fails, keeping the container "Not Ready" periodSeconds: 5

After adding the liveness probe configured just as the previous readiness probe and checking events of the pod by kubectl describe pod it’s visible that the sidecar has a restart count above 0. Nevertheless, the main application is not restarted nor influenced at all, even though I'm aware that (in our imaginary worst-case scenario) it can error out when the sidecar is not there serving requests. What if I’d used a livenessProbe without lifecycle postStart? Both containers will be immediately ready: at the beginning, this behavior will not be different from the one without any additional probes since the liveness probe doesn’t affect readiness at all. After a while, the sidecar will begin to restart itself, but it won’t influence the main container.

Findings summary

I’ll summarize the startup behavior in the table below:

Probe/Hook

Sidecar starts before the main app?

Main app waits for the sidecar to be ready?

What if the check doesn’t pass?

readinessProbe

Yes, but it’s almost in parallel (effectively no)

No

Sidecar is not ready; main app continues running

livenessProbe

Yes, but it’s almost in parallel (effectively no)

No

Sidecar is restarted, main app continues running

startupProbe

Yes

Yes

Main app is not started

postStart

Yes, main app container starts after postStart completes

Yes, but you have to provide custom

Gateway API v1.3.0: Advancements in Request Mirroring, CORS, Gateway Merging, and Retry Budgets

https://kubernetes.io/blog/2025/06/02/gateway-api-v1-3/

Join us in the Kubernetes SIG Network community in celebrating the general availability of Gateway API v1.3.0! We are also pleased to announce that there are already a number of conformant implementations to try, made possible by postponing this blog announcement. Version 1.3.0 of the API was released about a month ago on April 24, 2025.

Gateway API v1.3.0 brings a new feature to the Standard channel (Gateway API's GA release channel): percentage-based request mirroring, and introduces three new experimental features: cross-origin resource sharing (CORS) filters, a standardized mechanism for listener and gateway merging, and retry budgets.

Also see the full release notes and applaud the v1.3.0 release team next time you see them.

Graduation to Standard channel

Graduation to the Standard channel is a notable achievement for Gateway API features, as inclusion in the Standard release channel denotes a high level of confidence in the API surface and provides guarantees of backward compatibility. Of course, as with any other Kubernetes API, Standard channel features can continue to evolve with backward-compatible additions over time, and we (SIG Network) certainly expect further refinements and improvements in the future. For more information on how all of this works, refer to the Gateway API Versioning Policy.

Percentage-based request mirroring

Leads: Lior Lieberman,Jake Bennert

GEP-3171: Percentage-Based Request Mirroring

Percentage-based request mirroring is an enhancement to the existing support for HTTP request mirroring, which allows HTTP requests to be duplicated to another backend using the RequestMirror filter type. Request mirroring is particularly useful in blue-green deployment. It can be used to assess the impact of request scaling on application performance without impacting responses to clients.

The previous mirroring capability worked on all the requests to a backendRef.

Percentage-based request mirroring allows users to specify a subset of requests they want to be mirrored, either by percentage or fraction. This can be particularly useful when services are receiving a large volume of requests. Instead of mirroring all of those requests, this new feature can be used to mirror a smaller subset of them.

Here's an example with 42% of the requests to "foo-v1" being mirrored to "foo-v2":

apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: http-filter-mirror labels: gateway: mirror-gateway spec: parentRefs:

• name: mirror-gateway hostnames:
• mirror.example rules:
• backendRefs:
• name: foo-v1 port: 8080 filters:
• type: RequestMirror requestMirror: backendRef: name: foo-v2 port: 8080 percent: 42 # This value must be an integer.

You can also configure the partial mirroring using a fraction. Here is an example with 5 out of every 1000 requests to "foo-v1" being mirrored to "foo-v2".

rules:

• backendRefs:
• name: foo-v1 port: 8080 filters:
• type: RequestMirror requestMirror: backendRef: name: foo-v2 port: 8080 fraction: numerator: 5 denominator: 1000

Additions to Experimental channel

The Experimental channel is Gateway API's channel for experimenting with new features and gaining confidence with them before allowing them to graduate to standard. Please note: the experimental channel may include features that are changed or removed later.

Starting in release v1.3.0, in an effort to distinguish Experimental channel resources from Standard channel resources, any new experimental API kinds have the prefix "X". For the same reason, experimental resources are now added to the API group gateway.networking.x-k8s.io instead of gateway.networking.k8s.io. Bear in mind that using new experimental channel resources means they can coexist with standard channel resources, but migrating these resources to the standard channel will require recreating them with the standard channel names and API group (both of which lack the "x-k8s" designator or "X" prefix).

The v1.3 release introduces two new experimental API kinds: XBackendTrafficPolicy and XListenerSet. To be able to use experimental API kinds, you need to install the Experimental channel Gateway API YAMLs from the locations listed below.

CORS filtering

Leads: Liang Li, Eyal Pazz, Rob Scott

GEP-1767: CORS Filter

Cross-origin resource sharing (CORS) is an HTTP-header based mechanism that allows a web page to access restricted resources from a server on an origin (domain, scheme, or port) different from the domain that served the web page. This feature adds a new HTTPRoute filter type, called "CORS", to configure the handling of cross-origin requests before the response is sent back to the client.

To be able to use experimental CORS filtering, you need to install the Experimental channel Gateway API HTTPRoute yaml.

Here's an example of a simple cross-origin configuration:

apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: http-route-cors spec: parentRefs:

• name: http-gateway rules:
• matches:
• path: type: PathPrefix value: /resource/foo filters:
• cors:
• type: CORS allowOrigins:
• * allowMethods:
• GET
• HEAD
• POST allowHeaders:
• Accept
• Accept-Language
• Content-Language
• Content-Type
• Range backendRefs:
• kind: Service name: http-route-cors port: 80

In this case, the Gateway returns an origin header of "*", which means that the requested resource can be referenced from any origin, a methods header (Access-Control-Allow-Methods) that permits the GET, HEAD, and POST verbs, and a headers header allowing Accept, Accept-Language, Content-Language, Content-Type, and Range.

HTTP/1.1 200 OK Access-Control-Allow-Origin: * Access-Control-Allow-Methods: GET, HEAD, POST Access-Control-Allow-Headers: Accept,Accept-Language,Content-Language,Content-Type,Range

The complete list of fields in the new CORS filter:

allowOrigins

allowMethods

allowHeaders

allowCredentials

exposeHeaders

maxAge

See CORS protocol for details.

XListenerSets (standardized mechanism for Listener and Gateway merging)

Lead: Dave Protasowski

GEP-1713: ListenerSets - Standard Mechanism to Merge Multiple Gateways

This release adds a new experimental API kind, XListenerSet, that allows a shared list of listeners to be attached to one or more parent Gateway(s). In addition, it expands upon the existing suggestion that Gateway API implementations may merge configuration from multiple Gateway objects. It also:

adds a new field allowedListeners to the .spec of a Gateway. The allowedListeners field defines from which Namespaces to select XListenerSets that are allowed to attach to that Gateway: Same, All, None, or Selector based.

increases the previous maximum number (64) of listeners with the addition of XListenerSets.

allows the delegation of listener configuration, such as TLS, to applications in other namespaces.

To be able to use experimental XListenerSet, you need to install the Experimental channel Gateway API XListenerSet yaml.

The following example shows a Gateway with an HTTP listener and two child HTTPS XListenerSets with unique hostnames and certificates. The combined set of listeners attached to the Gateway includes the two additional HTTPS listeners in the XListenerSets that attach to the Gateway. This example illustrates the delegation of listener TLS config to application owners in different namespaces ("store" and "app"). The HTTPRoute has both the Gateway listener named "foo" and one XListenerSet listener named "second" as parentRefs.

apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: prod-external namespace: infra spec: gatewayClassName: example allowedListeners:

• from: All listeners:
• name: foo hostname: foo.com protocol: HTTP port: 80 --- apiVersion: gateway.networking.x-k8s.io/v1alpha1 kind: XListenerSet metadata: name: store namespace: store spec: parentRef: name: prod-external listeners:
• name: first hostname: first.foo.com protocol: HTTPS port: 443 tls: mode: Terminate certificateRefs:
• kind: Secret group: "" name: first-workload-cert --- apiVersion: gateway.networking.x-k8s.io/v1alpha1 kind: XListenerSet metadata: name: app namespace: app spec: parentRef: name: prod-external listeners:
• name: second hostname: second.foo.com protocol: HTTPS port: 443 tls: mode: Terminate certificateRefs:
• kind: Secret group: "" name: second-workload-cert --- apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: httproute-example spec: parentRefs:
• name: app kind: XListenerSet sectionName: second
• name: parent-gateway kind: Gateway sectionName: foo ...

Each listener in a Gateway must have a unique combination of port, protocol, (and hostname if supported by the protocol) in order for all listeners to be compatible and not conflicted over which traffic they should receive.

Furthermore, implementations can merge separate Gateways into a single set of listener addresses if all listeners across those Gateways are compatible. The management of merged listeners was under-specified in releases prior to v1.3.0.

With the new feature, the specification on merging is expanded. Implementations must treat the parent Gateways as having the merged list of all listeners from itself and from attached XListenerSets, and validation of this list of listeners must behave the same as if the list were part of a single Gateway. Within a single Gateway, listeners are ordered using the following precedence:

Single Listeners (not a part of an XListenerSet) first,

Remaining listeners ordered by:

object creation time (oldest first), and if two listeners are defined in objects that have the same timestamp, then

alphabetically based on "{namespace}/{name of listener}"

Retry budgets (XBackendTrafficPolicy)

Leads: Eric Bishop, Mike Morris

GEP

Forget CLIs and GUIs: AI is the New Interface for Developer Platforms

Discover how AI can revolutionize user interactions with Internal Developer Platforms. This video demonstrates using AI to create, observe, and delete services through natural language conversations. Watch as we leverage custom AI commands to dynamically generate Kubernetes manifests, monitor service status, and manage resources, all without users needing deep platform knowledge. Learn about the pros and cons of this approach, including flexibility vs. determinism. See how AI can simplify complex operations, making platform interactions more intuitive for developers. This approach showcases the potential of AI in streamlining DevOps processes and enhancing developer experiences. We'll use Claude Code custom commands today, but the same principles could be applied to other agents as well.

▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ Sponsor: TestSprite 🔗 https://testsprite.com ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

InternalDeveloperPlatform, #AI, #PlatformEngineering

Consider joining the channel: https://www.youtube.com/c/devopstoolkit/join

▬▬▬▬▬▬ 🔗 Additional Info 🔗 ▬▬▬▬▬▬ ➡ Transcript and commands: https://devopstoolkit.live/internal-developer-platforms/forget-clis-and-guis-ai-is-the-new-interface-for-developer-platforms 🔗 Claude Code: https://claude.ai 🎬 Claude Code: AI Agent for DevOps, SRE, and Platform Engineering: https://youtu.be/h-6LP133o6w

▬▬▬▬▬▬ 💰 Sponsorships 💰 ▬▬▬▬▬▬ If you are interested in sponsoring this channel, please visit https://devopstoolkit.live/sponsor for more information. Alternatively, feel free to contact me over Twitter or LinkedIn (see below).

▬▬▬▬▬▬ 👋 Contact me 👋 ▬▬▬▬▬▬ ➡ BlueSky: https://vfarcic.bsky.social ➡ LinkedIn: https://www.linkedin.com/in/viktorfarcic/

▬▬▬▬▬▬ 🚀 Other Channels 🚀 ▬▬▬▬▬▬ 🎤 Podcast: https://www.devopsparadox.com/ 💬 Live streams: https://www.youtube.com/c/DevOpsParadox

▬▬▬▬▬▬ ⏱ Timecodes ⏱ ▬▬▬▬▬▬ 00:00 AI for Internal Developer Platforms (IDPs) 01:40 TestSprite (sponsor) 02:41 AI for Internal Developer Platforms (IDPs) (cont.) 03:25 AI to Create Platform Services 08:21 AI to Observe Platform Services 10:32 AI to Fix Issues with Platform Services 11:31 AI to Fix Delete Platform Services 12:52 How Did It All Happen? 14:43 Pros and Cons of AI as Platform Interface

via YouTube https://www.youtube.com/watch?v=ApjnCa-a2xI