Mastering Developer Portals: Discover & Integrate API Schemas with Port

The ability to discover resource definitions makes a huge difference. It allows me to stop wasting my time rewriting "stuff" from one format to another but focus on creating APIs and let others discover what's behind them.

Port just released a feature that enables just that. It allows us to discover schemas from APIs and convert them into data-models and actions.

In this video use the new "discovery" feature in Port to combine it with the whole life-cycle of resources. We'll see how to combine it with Crossplane, Argo CD, GitHub Actions, and, probably, a few other tools. We'll see how we can enable developers to manage anything, be it applications, databases, clusters, or anything else. We are about to build a whole platform, not only the console.

DeveloperPortal #InfrastructureAsCode #DevOpsAutomation #Crossplane

Consider joining the channel: https://www.youtube.com/c/devopstoolkit/join

▬▬▬▬▬▬ 🔗 Additional Info 🔗 ▬▬▬▬▬▬ 🔗 Port: https://getport.io ➡ Transcript and commands: https://devopstoolkit.live/internal-developer-platforms/mastering-developer-portals-discover--integrate-api-schemas-with-port 🎬 Getting Started with Crossplane: A Glimpse Into the Future: https://youtu.be/bBpE0rfE-JM 🎬 Crossplane Providers and Managed Resources: https://youtu.be/o53_7vuWjw4 🎬 Crossplane Compositions: https://youtu.be/X7E6YfXWgvE 🎬 Github Actions Review And Tutorial: https://youtu.be/eZcAvTb0rbA 🎬 Argo CD playlist: https://youtube.com/playlist?list=PLyicRj904Z9_dGuNs6AN5Khljjn9ssbQ6

▬▬▬▬▬▬ 💰 Sponsorships 💰 ▬▬▬▬▬▬ If you are interested in sponsoring this channel, please use https://calendar.app.google/Q9eaDUHN8ibWBaA7A to book a timeslot that suits you, and we'll go over the details. Or feel free to contact me over Twitter or LinkedIn (see below).

▬▬▬▬▬▬ 👋 Contact me 👋 ▬▬▬▬▬▬ ➡ Twitter: https://twitter.com/vfarcic ➡ LinkedIn: https://www.linkedin.com/in/viktorfarcic/

▬▬▬▬▬▬ 🚀 Other Channels 🚀 ▬▬▬▬▬▬ 🎤 Podcast: https://www.devopsparadox.com/ 💬 Live streams: https://www.youtube.com/c/DevOpsParadox

▬▬▬▬▬▬ ⏱ Timecodes ⏱ ▬▬▬▬▬▬ 00:00 Issues With Developer Portal Consoles 03:04 Discover Data Models From API Schemas 10:07 Create Service Instances 12:46 What Happened? 14:19 How Did It Happen? 19:00 Update and Delete Service Instances

via YouTube https://www.youtube.com/watch?v=PV1sBiC85Yc

Go to big city then build the #lego version is something I can get behind.

I gotta figure out a new place for the football.

June 21, 2024 at 05:57PM

via Instagram https://instagr.am/p/C8fmADYv9--/

Valkey gains momentum with broadening band of backers

Momentum is building behind open source Redis alternative Valkey with confirmation of new backers including Broadcom, the AlmaLinux OS Foundation, and…

June 21, 2024 at 02:29PM

via Instapaper

CDK Global hacked again while recovering from first cyberattack

June 21, 2024 at 09:12AM

via Instapaper

Cyberattack led to harrowing lapses at Ascension hospitals, clinicians say

Marvin Ruckle, who has worked as a NICU nurse at an Ascension hospital in Kansas, said problems caused by the cyberattack nearly led him to administer the wrong…

June 20, 2024 at 01:34PM

via Instapaper

Dozens of metro Detroit car dealers impacted by cyberattack on software company

DEARBORN, Mich. (WXYZ) — A cyberattack on a little-known company to most Americans is having a huge impact on car dealerships — and potentially their customers…

June 20, 2024 at 01:34PM

via Instapaper

Hundreds of PC, Server Models Possibly Affected by Serious Phoenix UEFI Vulnerability

Hundreds of PC and server models that use Intel processors could be affected by a high-severity vulnerability found recently in Phoenix Technologies’ SecureCore…

June 20, 2024 at 10:13AM

via Instapaper

Spotlight on SIG Node

https://kubernetes.io/blog/2024/06/20/sig-node-spotlight-2024/

In the world of container orchestration, Kubernetes reigns supreme, powering some of the most complex and dynamic applications across the globe. Behind the scenes, a network of Special Interest Groups (SIGs) drives Kubernetes' innovation and stability.

Today, I have the privilege of speaking with Matthias Bertschy, Gunju Kim, and Sergey Kanzhelev, members of SIG Node, who will shed some light on their roles, challenges, and the exciting developments within SIG Node.

Answers given collectively by all interviewees will be marked by their initials.

Introductions

Arpit: Thank you for joining us today. Could you please introduce yourselves and provide a brief overview of your roles within SIG Node?

Matthias: My name is Matthias Bertschy, I am French and live next to Lake Geneva, near the French Alps. I have been a Kubernetes contributor since 2017, a reviewer for SIG Node and a maintainer of Prow. I work as a Senior Kubernetes Developer for a security startup named ARMO, which donated Kubescape to the CNCF.

Gunju: My name is Gunju Kim. I am a software engineer at NAVER, where I focus on developing a cloud platform for search services. I have been contributing to the Kubernetes project in my free time since 2021.

Sergey: My name is Sergey Kanzhelev. I have worked on Kubernetes and Google Kubernetes Engine for 3 years and have worked on open-source projects for many years now. I am a chair of SIG Node.

Understanding SIG Node

Arpit: Thank you! Could you provide our readers with an overview of SIG Node's responsibilities within the Kubernetes ecosystem?

M/G/S: SIG Node is one of the first if not the very first SIG in Kubernetes. The SIG is responsible for all iterations between Kubernetes and node resources, as well as node maintenance itself. This is quite a large scope, and the SIG owns a large part of the Kubernetes codebase. Because of this wide ownership, SIG Node is always in contact with other SIGs such as SIG Network, SIG Storage, and SIG Security and almost any new features and developments in Kubernetes involves SIG Node in some way.

Arpit: How does SIG Node contribute to Kubernetes' performance and stability?

M/G/S: Kubernetes works on nodes of many different sizes and shapes, from small physical VMs with cheap hardware to large AI/ML-optimized GPU-enabled nodes. Nodes may stay online for months or maybe be short-lived and be preempted at any moment as they are running on excess compute of a cloud provider.

kubelet — the Kubernetes agent on a node — must work in all these environments reliably. As for the performance of kubelet operations, this is becoming increasingly important today. On one hand, as Kubernetes is being used on extra small nodes more and more often in telecom and retail environments, it needs to scale into the smallest footprint possible. On the other hand, with AI/ML workloads where every node is extremely expensive, every second of delayed operations can visibly change the price of computation.

Challenges and Opportunities

Arpit: What upcoming challenges and opportunities is SIG Node keeping an eye on?

M/G/S: As Kubernetes enters the second decade of its life, we see a huge demand to support new workload types. And SIG Node will play a big role in this. The Sidecar KEP, which we will be talking about later, is one of the examples of increased emphasis on supporting new workload types.

The key challenge we will have in the next few years is how to keep innovations while maintaining high quality and backward compatibility of existing scenarios. SIG Node will continue to play a central role in Kubernetes.

Arpit: And are there any ongoing research or development areas within SIG Node that excite you?

M/G/S: Supporting new workload types is a fascinating area for us. Our recent exploration of sidecar containers is a testament to this. Sidecars offer a versatile solution for enhancing application functionality without altering the core codebase.

Arpit: What are some of the challenges you've faced while maintaining SIG Node, and how have you overcome them?

M/G/S: The biggest challenge of SIG Node is its size and the many feature requests it receives. We are encouraging more people to join as reviewers and are always open to improving processes and addressing feedback. For every release, we run the feedback session at the SIG Node meeting and identify problematic areas and action items.

Arpit: Are there specific technologies or advancements that SIG Node is closely monitoring or integrating into Kubernetes?

M/G/S: Developments in components that the SIG depends on, like container runtimes (e.g. containerd and CRI-O, and OS features are something we contribute to and monitor closely. For example, there is an upcoming cgroup v1 deprecation and removal that Kubernetes and SIG Node will need to guide Kubernetes users through. Containerd is also releasing version 2.0, which removes deprecated features, which will affect Kubernetes users.

Arpit: Could you share a memorable experience or achievement from your time as a SIG Node maintainer that you're particularly proud of?

Mathias: I think the best moment was when my first KEP (introducing the startupProbe) finally graduated to GA (General Availability). I also enjoy seeing my contributions being used daily by contributors, such as the comment containing the GitHub tree hash used to retain LGTM despite squash commits.

Sidecar containers

Arpit: Can you provide more context on the concept of sidecar containers and their evolution in the context of Kubernetes?

M/G/S: The concept of sidecar containers dates back to 2015 when Kubernetes introduced the idea of composite containers. These additional containers, running alongside the main application container within the same pod, were seen as a way to extend and enhance application functionality without modifying the core codebase. Early adopters of sidecars employed custom scripts and configurations to manage them, but this approach presented challenges in terms of consistency and scalability.

Arpit: Can you share specific use cases or examples where sidecar containers are particularly beneficial?

M/G/S: Sidecar containers are a versatile tool that can be used to enhance the functionality of applications in a variety of ways:

Logging and monitoring: Sidecar containers can be used to collect logs and metrics from the primary application container and send them to a centralized logging and monitoring system.

Traffic filtering and routing: Sidecar containers can be used to filter and route traffic to and from the primary application container.

Encryption and decryption: Sidecar containers can be used to encrypt and decrypt data as it flows between the primary application container and external services.

Data synchronization: Sidecar containers can be used to synchronize data between the primary application container and external databases or services.

Fault injection: Sidecar containers can be used to inject faults into the primary application container in order to test its resilience to failures.

Arpit: The proposal mentions that some companies are using a fork of Kubernetes with sidecar functionality added. Can you provide insights into the level of adoption and community interest in this feature?

M/G/S: While we lack concrete metrics to measure adoption rates, the KEP has garnered significant interest from the community, particularly among service mesh vendors like Istio, who actively participated in its alpha testing phase. The KEP's visibility through numerous blog posts, interviews, talks, and workshops further demonstrates its widespread appeal. The KEP addresses the growing demand for additional capabilities alongside main containers in Kubernetes pods, such as network proxies, logging systems, and security measures. The community acknowledges the importance of providing easy migration paths for existing workloads to facilitate widespread adoption of the feature.

Arpit: Are there any notable examples or success stories from companies using sidecar containers in production?

M/G/S: It is still too early to expect widespread adoption in production environments. The 1.29 release has only been available in Google Kubernetes Engine (GKE) since January 11, 2024, and there still needs to be comprehensive documentation on how to enable and use them effectively via universal injector. Istio, a popular service mesh platform, also lacks proper documentation for enabling native sidecars, making it difficult for developers to get started with this new feature. However, as native sidecar support matures and documentation improves, we can expect to see wider adoption of this technology in production environments.

Arpit: The proposal suggests introducing a restartPolicy field for init containers to indicate sidecar functionality. Can you explain how this solution addresses the outlined challenges?

M/G/S: The proposal to introduce a restartPolicy field for init containers addresses the outlined challenges by utilizing existing infrastructure and simplifying sidecar management. This approach avoids adding new fields to the pod specification, keeping it manageable and avoiding more clutter. By leveraging the existing init container mechanism, sidecars can be run alongside regular init containers during pod startup, ensuring a consistent ordering of initialization. Additionally, setting the restart policy of sidecar init containers to Always explicitly states that they continue running even after the main application container terminates, enabling persistent services like logging and monitoring until the end of the workload.

Arpit: How will the introduction of the restartPolicy field for init containers affect backward compatibility with existing Kubernetes configurations?

M/G/S: The introduction of the restartPolicy field for init containers will maintain backward compatibility with existing Kub

Blog: Spotlight on SIG Node

https://www.kubernetes.dev/blog/2024/06/20/sig-node-spotlight-2024/

In the world of container orchestration, Kubernetes reigns supreme, powering some of the most complex and dynamic applications across the globe. Behind the scenes, a network of Special Interest Groups (SIGs) drives Kubernetes’ innovation and stability.

Today, I have the privilege of speaking with Matthias Bertschy, Gunju Kim, and Sergey Kanzhelev, members of SIG Node, who will shed some light on their roles, challenges, and the exciting developments within SIG Node.

Answers given collectively by all interviewees will be marked by their initials.

Introductions

Arpit: Thank you for joining us today. Could you please introduce yourselves and provide a brief overview of your roles within SIG Node?

Matthias: My name is Matthias Bertschy, I am French and live next to Lake Geneva, near the French Alps. I have been a Kubernetes contributor since 2017, a reviewer for SIG Node and a maintainer of Prow. I work as a Senior Kubernetes Developer for a security startup named ARMO, which donated Kubescape to the CNCF.

Gunju: My name is Gunju Kim. I am a software engineer at NAVER, where I focus on developing a cloud platform for search services. I have been contributing to the Kubernetes project in my free time since 2021.

Sergey: My name is Sergey Kanzhelev. I have worked on Kubernetes and Google Kubernetes Engine for 3 years and have worked on open-source projects for many years now. I am a chair of SIG Node.

Understanding SIG Node

Arpit: Thank you! Could you provide our readers with an overview of SIG Node’s responsibilities within the Kubernetes ecosystem?

M/G/S: SIG Node is one of the first if not the very first SIG in Kubernetes. The SIG is responsible for all iterations between Kubernetes and node resources, as well as node maintenance itself. This is quite a large scope, and the SIG owns a large part of the Kubernetes codebase. Because of this wide ownership, SIG Node is always in contact with other SIGs such as SIG Network, SIG Storage, and SIG Security and almost any new features and developments in Kubernetes involves SIG Node in some way.

Arpit: How does SIG Node contribute to Kubernetes’ performance and stability?

M/G/S: Kubernetes works on nodes of many different sizes and shapes, from small physical VMs with cheap hardware to large AI/ML-optimized GPU-enabled nodes. Nodes may stay online for months or maybe be short-lived and be preempted at any moment as they are running on excess compute of a cloud provider.

kubelet — the Kubernetes agent on a node — must work in all these environments reliably. As for the performance of kubelet operations, this is becoming increasingly important today. On one hand, as Kubernetes is being used on extra small nodes more and more often in telecom and retail environments, it needs to scale into the smallest footprint possible. On the other hand, with AI/ML workloads where every node is extremely expensive, every second of delayed operations can visibly change the price of computation.

Challenges and Opportunities

Arpit: What upcoming challenges and opportunities is SIG Node keeping an eye on?

M/G/S: As Kubernetes enters the second decade of its life, we see a huge demand to support new workload types. And SIG Node will play a big role in this. The Sidecar KEP, which we will be talking about later, is one of the examples of increased emphasis on supporting new workload types.

The key challenge we will have in the next few years is how to keep innovations while maintaining high quality and backward compatibility of existing scenarios. SIG Node will continue to play a central role in Kubernetes.

Arpit: And are there any ongoing research or development areas within SIG Node that excite you?

M/G/S: Supporting new workload types is a fascinating area for us. Our recent exploration of sidecar containers is a testament to this. Sidecars offer a versatile solution for enhancing application functionality without altering the core codebase.

Arpit: What are some of the challenges you’ve faced while maintaining SIG Node, and how have you overcome them?

M/G/S: The biggest challenge of SIG Node is its size and the many feature requests it receives. We are encouraging more people to join as reviewers and are always open to improving processes and addressing feedback. For every release, we run the feedback session at the SIG Node meeting and identify problematic areas and action items.

Arpit: Are there specific technologies or advancements that SIG Node is closely monitoring or integrating into Kubernetes?

M/G/S: Developments in components that the SIG depends on, like container runtimes (e.g. containerd and CRI-O, and OS features are something we contribute to and monitor closely. For example, there is an upcoming cgroup v1 deprecation and removal that Kubernetes and SIG Node will need to guide Kubernetes users through. Containerd is also releasing version 2.0, which removes deprecated features, which will affect Kubernetes users.

Arpit: Could you share a memorable experience or achievement from your time as a SIG Node maintainer that you’re particularly proud of?

Matthias: I think the best moment was when my first KEP (introducing the startupProbe) finally graduated to GA (General Availability). I also enjoy seeing my contributions being used daily by contributors, such as the comment containing the GitHub tree hash used to retain LGTM despite squash commits.

Sidecar containers

Arpit: Can you provide more context on the concept of sidecar containers and their evolution in the context of Kubernetes?

M/G/S: The concept of sidecar containers dates back to 2015 when Kubernetes introduced the idea of composite containers. These additional containers, running alongside the main application container within the same pod, were seen as a way to extend and enhance application functionality without modifying the core codebase. Early adopters of sidecars employed custom scripts and configurations to manage them, but this approach presented challenges in terms of consistency and scalability.

Arpit: Can you share specific use cases or examples where sidecar containers are particularly beneficial?

M/G/S: Sidecar containers are a versatile tool that can be used to enhance the functionality of applications in a variety of ways:

Logging and monitoring: Sidecar containers can be used to collect logs and metrics from the primary application container and send them to a centralized logging and monitoring system.

Traffic filtering and routing: Sidecar containers can be used to filter and route traffic to and from the primary application container.

Encryption and decryption: Sidecar containers can be used to encrypt and decrypt data as it flows between the primary application container and external services.

Data synchronization: Sidecar containers can be used to synchronize data between the primary application container and external databases or services.

Fault injection: Sidecar containers can be used to inject faults into the primary application container in order to test its resilience to failures.

Arpit: The proposal mentions that some companies are using a fork of Kubernetes with sidecar functionality added. Can you provide insights into the level of adoption and community interest in this feature?

M/G/S: While we lack concrete metrics to measure adoption rates, the KEP has garnered significant interest from the community, particularly among service mesh vendors like Istio, who actively participated in its alpha testing phase. The KEP’s visibility through numerous blog posts, interviews, talks, and workshops further demonstrates its widespread appeal. The KEP addresses the growing demand for additional capabilities alongside main containers in Kubernetes pods, such as network proxies, logging systems, and security measures. The community acknowledges the importance of providing easy migration paths for existing workloads to facilitate widespread adoption of the feature.

Arpit: Are there any notable examples or success stories from companies using sidecar containers in production?

M/G/S: It is still too early to expect widespread adoption in production environments. The 1.29 release has only been available in Google Kubernetes Engine (GKE) since January 11, 2024, and there still needs to be comprehensive documentation on how to enable and use them effectively via universal injector. Istio, a popular service mesh platform, also lacks proper documentation for enabling native sidecars, making it difficult for developers to get started with this new feature. However, as native sidecar support matures and documentation improves, we can expect to see wider adoption of this technology in production environments.

Arpit: The proposal suggests introducing a restartPolicy field for init containers to indicate sidecar functionality. Can you explain how this solution addresses the outlined challenges?

M/G/S: The proposal to introduce a restartPolicy field for init containers addresses the outlined challenges by utilizing existing infrastructure and simplifying sidecar management. This approach avoids adding new fields to the pod specification, keeping it manageable and avoiding more clutter. By leveraging the existing init container mechanism, sidecars can be run alongside regular init containers during pod startup, ensuring a consistent ordering of initialization. Additionally, setting the restart policy of sidecar init containers to Always explicitly states that they continue running even after the main application container terminates, enabling persistent services like logging and monitoring until the end of the workload.

Arpit: How will the introduction of the restartPolicy field for init containers affect backward compatibility with existing Kubernetes configurations?

M/G/S: The introduction of the restartPolicy field for init containers will maintain backward compatibility with

Week Ending June 16, 2024

https://lwkd.info/2024/20240618

Developer News

Kubelet node IP address assignment is still “fragile and broken,” with yet another PR this week. As such, Antonio Ojea would like people who work on external cloud providers to weigh in on a long-term solution.

SIG-Etcd and SIG-Cluster Lifecycle have proposed a new Etcd Operator WG in order to create one.

The call for session proposals for the Contributor Summit in Salt Lake City is open. This includes proposals for presentations, demos, discussions, workshops, and SIG/team meetings; really, anything that should go on the KCS schedule except for the Doc Sprints (which are already there). Submit before September 15.

Release Schedule

Next Deadline: Docs Deadline for placeholder PRs, June 27th

Kubernetes v1.31.0-alpha.2 is live!

We are in Enhancements Freeze now, and currently have 71 opted-in, 54 tracked, and 7 removed features. If your feature missed the deadline, you need to file an Exception.

Patch releases for 1.27.15, 1.28.11, 1.29.6 and 1.30.2 came out last week, with an update to Go versions 1.21.11 and 1.22.4.

Featured PRs

125029 kubeadm: enable the v1beta4 API

This PR implements the necessary changes to support the new v1beta4 kubeadm API in Kubernetes. Key updates include the conversion of extraArgs to a list of name/value pairs, the replacement of several configuration fields for improved clarity, and the introduction of the kubeadm config migrate command for seamless migration from v1beta3 to v1beta4. While v1beta3 is deprecated, it remains supported until version 1.34 or later.

KEP of the Week

KEP 4444: Traffic Distribution for Services

This KEP aims to introduce a new field, trafficDistribution, to the Kubernetes Service specification. This enhancement is designed to supersede the existing service.kubernetes.io/topology-mode annotation and the deprecated topologyKeys field to provide a more flexible, user-friendly approach to routing traffic within Kubernetes clusters.

This KEP is tracked for beta release in the upcoming v1.31.

Other Merges

client-go can use API Streaming for lists

ControllerRevision objects must be valid JSON

Measure scheduler performance with many gated pods

DRA apiGroup validates group name

Fix namespace watch regression

kubeadm: allow --yes --config

OTel can trace syncPod errors

Fix container log rotation on Windows

Promotions

Portforward over Websockets to beta

LogarithmicScaleDown to GA

KubeProxyDrainingTerminatingNodes to GA

Deprecated

Remove incorrect deprecation warning for v1 pod bindings

Subprojects and Dependency Updates

cloud-provider-vsphere v1.28.1: move to new resource pool and folder in VMC

etcd v3.4.33: memberlist results not updated when proxy node down

via Last Week in Kubernetes Development https://lwkd.info/

June 18, 2024 at 06:00PM

Introducing Distill CLI: An efficient, Rust-powered tool for media summarization

A few weeks ago, I wrote about a project our team has been working on called Distill. A simple application that summarizes and extracts important details from…

June 18, 2024 at 03:06PM

via Instapaper