1_r/devopsish

Developer News

Jim Farley spoke at The Wall Street Journal's Future of Everything Festival about competition and technology within the EV market.

Authors: Katrina Verey (independent) and Justin Santa Barbara (Google) Declarative configuration management with the kubectl apply command is the gold standard approach to creating or modifying Kubernetes resources. However, one challenge it presents is the deletion of resources that are no longer needed. In Kubernetes version 1.5, the --prune flag was introduced to address this issue, allowing kubectl apply to automatically clean up previously applied resources removed from the current configuration. Unfortunately, that existing implementation of --prune has design flaws that diminish its performance and can result in unexpected behaviors. The main issue stems from the lack of explicit encoding of the previously applied set by the preceding apply operation, necessitating error-prone dynamic discovery. Object leakage, inadvertent over-selection of resources, and limited compatibility with custom resources are a few notable drawbacks of this implementation. Moreover, its coupling to client-side apply hinders user upgrades to the superior server-side apply mechanism. Version 1.27 of kubectl introduces an alpha version of a revamped pruning implementation that addresses these issues. This new implementation, based on a concept called ApplySet , promises better performance and safety. An ApplySet is a group of resources associated with a parent object on the cluster, as identified and configured through standardized labels and annotations. Additional standardized metadata allows for accurate identification of ApplySet member objects within the cluster, simplifying operations like pruning. To leverage ApplySet-based pruning, set the KUBECTL_APPLYSET=true environment variable and include the flags --prune and --applyset in your kubectl apply invocation: KUBECTL_APPLYSET = true kubectl apply -f directory/ --prune --applyset= name By default, ApplySet uses a Secret as the parent object. However, you can also use a ConfigMap with the format --applyset=configmaps/name . If your desired Secret or ConfigMap object does not yet exist, kubectl will create it for you. Furthermore, custom resources can be enabled for use as ApplySet parent objects. The ApplySet implementation is based on a new low-level specification that can support higher-level ecosystem tools by improving their interoperability. The lightweight nature of this specification enables these tools to continue to use existing object grouping systems while opting in to ApplySet's metadata conventions to prevent inadvertent changes by other tools (such as kubectl ). ApplySet-based pruning offers a promising solution to the shortcomings of the previous --prune implementation in kubectl and can help streamline your Kubernetes resource management. Please give this new feature a try and share your experiences with the community—ApplySet is under active development, and your feedback is invaluable! Additional resources For more information how to use ApplySet-based pruning, read Declarative Management of Kubernetes Objects Using Configuration Files in the Kubernetes documentation. For a deeper dive into the technical design of this feature or to learn how to implement the ApplySet specification in your own tools, refer to KEP 3659 : ApplySet: kubectl apply --prune redesign and graduation strategy . How do I get involved? If you want to get involved in ApplySet development, you can get in touch with the developers at SIG CLI . To provide feedback on the feature, please file a bug or request an enhancement on the kubernetes/kubectl repository.

At Siemens, we strongly believe that Open Source software is key to our success. That's why we are thrilled to announce the launch of the Siemens Open…

Networking is one of the core pillars of Kubernetes, and the Special Interest Group for Networking (SIG Network) is responsible for developing and maintaining the networking features of Kubernetes. It covers all aspects to ensure Kubernetes provides a reliable and scalable network infrastructure for containerized applications. In this SIG Network spotlight, Sujay Dey talked with Shane Utt , Software Engineer at Kong, chair of SIG Network and maintainer of Gateway API, on different aspects of the SIG, what are the exciting things going on and how anyone can get involved and contribute here. Sujay : Hello, and first of all, thanks for the opportunity of learning more about SIG Network. I would love to hear your story, so could you please tell us a bit about yourself, your role, and how you got involved in Kubernetes, especially in SIG Network? Shane : Hello! Thank you for reaching out. My Kubernetes journey started while I was working for a small data centre: we were early adopters of Kubernetes and focused on using Kubernetes to provide SaaS products. That experience led to my next position developing a distribution of Kubernetes with a focus on networking. During this period in my career, I was active in SIG Network (predominantly as a consumer). When I joined Kong my role in the community changed significantly, as Kong actively encourages upstream participation. I greatly increased my engagement and contributions to the Gateway API project during those years, and eventually became a maintainer. I care deeply about this community and the future of our technology, so when a chair position for the SIG became available, I volunteered my time immediately. I’ve enjoyed working on Kubernetes over the better part of a decade and I want to continue to do my part to ensure our community and technology continues to flourish. Sujay : I have to say, that was a truely inspiring journey! Now, let us talk a bit more about SIG Network. Since we know it covers a lot of ground, could you please highlight its scope and current focus areas? Shane : For those who may be uninitiated: SIG Network is responsible for the components, interfaces, and APIs which expose networking capabilities to Kubernetes users and workloads. The charter is a pretty good indication of our scope, but I can add some additional highlights on some of our current areas of focus (this is a non-exhaustive list of sub-projects): kube-proxy & KPNG Those familiar with Kubernetes will know the Service API, which enables exposing a group of pods over a network. The current standard implementation of Service is known as kube-proxy , but what may be unfamiliar to people is that there are a growing number of disparate alternative implementations on the rise in recent years. To try and give provisions to these implementations (and also provide some areas of alignment so that implementations do not become too disparate from each other) upstream Kubernetes efforts are underway to create a more modular public interface for kube-proxy . The intention is for implementations to join in around a common set of libraries and speak a common language. This area of focus is known as the KPNG project, and if this sounds interesting to you, please join us in the KPNG community meetings and #sig-network-kpng on Kubernetes Slack . Multi-Network Today one of the primary requirements for Kubernetes networking is to achieve connectivity between pods in a cluster, satisfying a large number of Kubernetes end-users. However, some use cases require isolated networks and special interfaces for performance-oriented needs (e.g. AF_XDP , memif , SR-IOV ). There’s a growing need for special networking configurations in Kubernetes in general. The Multi-Network project exists to improve the management of multiple different networks for pods: anyone interested in some of the lower-level details of pod networking (or anyone having relevant use cases) can join us in the Multi-Network community meetings and #sig-network-multi-network on Kubernetes Slack. Network Policy The NetworkPolicy API sub-group was formed to address network security beyond the well-known version 1 of the NetworkPolicy resource. We’ve also been working on the AdminNetworkPolicy resource (previously known as ClusterNetworkPolicy ) to provide cluster administrator-focused functionality. The network policy sub-project is a great place to join in if you’re particularly interested in security and CNI, please feel free to join our community meetings and the #sig-network-policy-api channel on Kubernetes Slack. Gateway API If you’re specially interested in ingress or mesh networking the Gateway API may be a sub-project you would enjoy. In Gateway API , we’re actively developing the successor to the illustrious Ingress API, which includes a Gateway resource which defines the addresses and listeners of the gateway and various routing types (e.g. HTTPRoute , GRPCRoute , TLSRoute , TCPRoute , UDPRoute , etc.) that attach to Gateways. We also have an initiative within this project called GAMMA, geared towards using Gateway API resources in a mesh network context. There are some up-and-coming side projects within Gateway API as well, including ingress2gateway which is a tool for compiling existing Ingress objects to equivalent Gateway API resources, and Blixt, a Layer4 implementation of Gateway API using Rust/eBPF for the data plane, intended as a testing and reference implementation. If this sounds interesting, we would love to have readers join us in our Gateway API community meetings and #sig-network-gateway-api on Kubernetes Slack. Sujay : Couldn’t agree more! That was a very informative description, thanks for highlighting them so nicely. As you have already mentioned about the SIG channels to get involved, would you like to add anything about where people like beginners can jump in and contribute? Shane : For help getting started Kubernetes Slack is a great place to talk to community members and includes several #sig-network-project channels as well as our main #sig-network channel. Also, check for issues labelled good-first-issue if you prefer to just dive right into the repositories. Let us know how we can help you! Sujay : What skills are contributors to SIG Network likely to learn? Shane : To me, it feels limitless. Practically speaking, it’s very much up to the individual what they want to learn. However, if you just intend to learn as much as you possibly can about networking, SIG Network is a great place to join in and grow your knowledge. If you’ve ever wondered how Kubernetes Service API works or wanted to implement an ingress controller, this is a great place to join in. If you wanted to dig down deep into the inner workings of CNI, or how the network interfaces at the pod level are configured, you can do that here as well. We have an awesome and diverse community of people from just about every kind of background you can imagine. This is a great place to share ideas and raise proposals, improving your skills in design, as well as alignment and consensus building. There’s a wealth of opportunities here in SIG Network. There are lots of places to jump in, and the learning opportunities are boundless. Sujay : Thanks a lot! It was a really great discussion, we got to know so many great things about SIG Network. I’m sure that many others will find this just as useful as I did.

WebAuthn Explained. Everything a developer needs to know about the Web Authentication API.

Author: Xing Yang (VMware) Volume group snapshot is introduced as an Alpha feature in Kubernetes v1.27. This feature introduces a Kubernetes API that allows users to take crash consistent snapshots for multiple volumes together. It uses a label selector to group multiple PersistentVolumeClaims for snapshotting. This new feature is only supported for CSI volume drivers. An overview of volume group snapshots Some storage systems provide the ability to create a crash consistent snapshot of multiple volumes. A group snapshot represents “copies” from multiple volumes that are taken at the same point-in-time. A group snapshot can be used either to rehydrate new volumes (pre-populated with the snapshot data) or to restore existing volumes to a previous state (represented by the snapshots). Why add volume group snapshots to Kubernetes? The Kubernetes volume plugin system already provides a powerful abstraction that automates the provisioning, attaching, mounting, resizing, and snapshotting of block and file storage. Underpinning all these features is the Kubernetes goal of workload portability: Kubernetes aims to create an abstraction layer between distributed applications and underlying clusters so that applications can be agnostic to the specifics of the cluster they run on and application deployment requires no cluster specific knowledge. There is already a VolumeSnapshot API that provides the ability to take a snapshot of a persistent volume to protect against data loss or data corruption. However, there are other snapshotting functionalities not covered by the VolumeSnapshot API. Some storage systems support consistent group snapshots that allow a snapshot to be taken from multiple volumes at the same point-in-time to achieve write order consistency. This can be useful for applications that contain multiple volumes. For example, an application may have data stored in one volume and logs stored in another volume. If snapshots for the data volume and the logs volume are taken at different times, the application will not be consistent and will not function properly if it is restored from those snapshots when a disaster strikes. It is true that you can quiesce the application first, take an individual snapshot from each volume that is part of the application one after the other, and then unquiesce the application after all the individual snapshots are taken. This way, you would get application consistent snapshots. However, sometimes it may not be possible to quiesce an application or the application quiesce can be too expensive so you want to do it less frequently. Taking individual snapshots one after another may also take longer time compared to taking a consistent group snapshot. Some users may not want to do application quiesce very often for these reasons. For example, a user may want to run weekly backups with application quiesce and nightly backups without application quiesce but with consistent group support which provides crash consistency across all volumes in the group. Kubernetes Volume Group Snapshots API Kubernetes Volume Group Snapshots introduce three new API objects for managing snapshots: VolumeGroupSnapshot Created by a Kubernetes user (or perhaps by your own automation) to request creation of a volume group snapshot for multiple persistent volume claims. It contains information about the volume group snapshot operation such as the timestamp when the volume group snapshot was taken and whether it is ready to use. The creation and deletion of this object represents a desire to create or delete a cluster resource (a group snapshot). VolumeGroupSnapshotContent Created by the snapshot controller for a dynamically created VolumeGroupSnapshot. It contains information about the volume group snapshot including the volume group snapshot ID. This object represents a provisioned resource on the cluster (a group snapshot). The VolumeGroupSnapshotContent object binds to the VolumeGroupSnapshot for which it was created with a one-to-one mapping. VolumeGroupSnapshotClass Created by cluster administrators to describe how volume group snapshots should be created. including the driver information, the deletion policy, etc. These three API kinds are defined as CustomResourceDefinitions (CRDs). These CRDs must be installed in a Kubernetes cluster for a CSI Driver to support volume group snapshots. How do I use Kubernetes Volume Group Snapshots Volume group snapshots are implemented in the external-snapshotter repository. Implementing volume group snapshots meant adding or changing several components: Added new CustomResourceDefinitions for VolumeGroupSnapshot and two supporting APIs. Volume group snapshot controller logic is added to the common snapshot controller. Volume group snapshot validation webhook logic is added to the common snapshot validation webhook. Adding logic to make CSI calls into the snapshotter sidecar controller. The volume snapshot controller, CRDs, and validation webhook are deployed once per cluster, while the sidecar is bundled with each CSI driver. Therefore, it makes sense to deploy the volume snapshot controller, CRDs, and validation webhook as a cluster addon. I strongly recommend that Kubernetes distributors bundle and deploy the volume snapshot controller, CRDs, and validation webhook as part of their Kubernetes cluster management process (independent of any CSI Driver). Creating a new group snapshot with Kubernetes Once a VolumeGroupSnapshotClass object is defined and you have volumes you want to snapshot together, you may request a new group snapshot by creating a VolumeGroupSnapshot object. The source of the group snapshot specifies whether the underlying group snapshot should be dynamically created or if a pre-existing VolumeGroupSnapshotContent should be used. A pre-existing VolumeGroupSnapshotContent is created by a cluster administrator. It contains the details of the real volume group snapshot on the storage system which is available for use by cluster users. One of the following members in the source of the group snapshot must be set. selector - a label query over PersistentVolumeClaims that are to be grouped together for snapshotting. This labelSelector will be used to match the label added to a PVC. volumeGroupSnapshotContentName - specifies the name of a pre-existing VolumeGroupSnapshotContent object representing an existing volume group snapshot. In the following example, there are two PVCs. NAME STATUS VOLUME CAPACITY ACCESSMODES AGE pvc-0 Bound pvc-a42d7ea2-e3df-11ed-b5ea-0242ac120002 1Gi RWO 48s pvc-1 Bound pvc-a42d81b8-e3df-11ed-b5ea-0242ac120002 1Gi RWO 48s Label the PVCs. % kubectl label pvc pvc-0 group=myGroup persistentvolumeclaim/pvc-0 labeled % kubectl label pvc pvc-1 group=myGroup persistentvolumeclaim/pvc-1 labeled For dynamic provisioning, a selector must be set so that the snapshot controller can find PVCs with the matching labels to be snapshotted together. apiVersion : groupsnapshot.storage.k8s.io/v1alpha1 kind : VolumeGroupSnapshot metadata : name : new-group-snapshot-demo namespace : demo-namespace spec : volumeGroupSnapshotClassName : csi-groupSnapclass source : selector : matchLabels : group : myGroup In the VolumeGroupSnapshot spec, a user can specify the VolumeGroupSnapshotClass which has the information about which CSI driver should be used for creating the group snapshot. Two individual volume snapshots will be created as part of the volume group snapshot creation. snapshot-62abb5db7204ac6e4c1198629fec533f2a5d9d60ea1a25f594de0bf8866c7947-2023-04-26-2.20.4 snapshot-2026811eb9f0787466171fe189c805a22cdb61a326235cd067dc3a1ac0104900-2023-04-26-2.20.4 How to use group snapshot for restore in Kubernetes At restore time, the user can request a new PersistentVolumeClaim to be created from a VolumeSnapshot object that is part of a VolumeGroupSnapshot. This will trigger provisioning of a new volume that is pre-populated with data from the specified snapshot. The user should repeat this until all volumes are created from all the snapshots that are part of a group snapshot. apiVersion : v1 kind : PersistentVolumeClaim metadata : name : pvc0-restore namespace : demo-namespace spec : storageClassName : csi-hostpath-sc dataSource : name : snapshot-62abb5db7204ac6e4c1198629fec533f2a5d9d60ea1a25f594de0bf8866c7947-2023-04-26-2.20.4 kind : VolumeSnapshot apiGroup : snapshot.storage.k8s.io accessModes : - ReadWriteOnce resources : requests : storage : 1Gi As a storage vendor, how do I add support for group snapshots to my CSI driver? To implement the volume group snapshot feature, a CSI driver must : Implement a new group controller service. Implement group controller RPCs: CreateVolumeGroupSnapshot , DeleteVolumeGroupSnapshot , and GetVolumeGroupSnapshot . Add group controller capability CREATE_DELETE_GET_VOLUME_GROUP_SNAPSHOT . See the CSI spec and the Kubernetes-CSI Driver Developer Guide for more details. a CSI Volume Driver as possible, it provides a suggested mechanism to deploy a containerized CSI driver to simplify the process. As part of this recommended deployment process, the Kubernetes team provides a number of sidecar (helper) containers, including the external-snapshotter sidecar container which has been updated to support volume group snapshot. The external-snapshotter watches the Kubernetes API server for the VolumeGroupSnapshotContent object and triggers CreateVolumeGroupSnapshot and DeleteVolumeGroupSnapshot operations against a CSI endpoint. What are the limitations? The alpha implementation of volume group snapshots for Kubernetes has the following limitations: Does not support reverting an existing PVC to an earlier state represented by a snapshot (only supports provisioning a new volume from a snapshot). No application consistency guarantees beyond any guarantees provided by the sto...

Presidents, prime ministers, and dictators will all have to wait to join Bluesky, the buzzy Twitter competitor.

I recently implemented a Brainfuck interpreter in the Crystal programming language and I’d like to share my honest opinion about working in the language. When looking at a new programming language I am interested in these things in no particular order

strengthening digital self-defense | research and development | providing privacy-focused goods and services

Building evolvable software systems is a strategy, not a religion. And revisiting your architectures with an open mind is a must.

The bid includes large players such as Airbus Defence and Space, Eutelsat, and SES.

Westinghouse announced on Thursday it is launching a small nuclear reactor, a miniature version of its flagship AP1000.

Let’s get it out of the way early: it’s not always clear how you can best plug into organizations like OpenSSF. That’s why I’m writing this guest blog post as an “outsider.” I’m just your average tech employee who has become progressively more involved since my company, Sonatype, became members of OpenSSF. If you’re connecting for your first time, the recommended engagement path is effectively “choose your adventure!”

Taiwan Semiconductor Manufacturing Co is in talks with partners to invest as much as 10 billion euros ($11.04 billion) to build a chip fabrication plant in Germany, Bloomberg News reported on Wednesday, citing people familiar with the matter.

Call any LLM with a single API. Zero dependencies.

Discord is dropping the four-digit suffix that followed each username.

macOS Internals. GitHub Gist: instantly share code, notes, and snippets.

The leaks of classified documents online by a Massachusetts Air National Guard member have had an emotional impact on the government agencies that produce those products, the director of national intelligence told Congress on Thursday.

Taiwan-based personal computer vendor Asus has unveiled its latest Zenbook S 13 OLED globally, touting it as the world’s thinnest laptop.

The company cut ties with the rapper last year over his antisemitic and offensive comments. Its “options are narrowing" on what to do with the unsold sneakers.

Element is one of 14 messaging apps blocked by the Central Indian Government which - we believe from media reports - relates to Section 69A of the Information Technology Act, 2000.

Cinema chain Odeon may have shared more information than it intended in the release notes accompanying its latest iOS app update.