
1_r/devopsish
Updating an old Ubuntu to a supported version
https://anonymoushash.vmbrasseur.com/2025/07/old-ubuntu-upgrade.html
I host my own Mastodon instance, which generally is pretty easy to maintain. The great team in the Mastodon community does a super job in making it easy to upgrade as they release new versions. I’ve therefore been keeping my Mastodon installation up to date. Go me!
Unfortunately, I haven’t been keeping up on my operating system updates, so my Digital Ocean droplet was still way back on Ubuntu-22.10. The latest LTS release is 24.04. Ooooops.
More unfortunate, the standard do-release-upgrade won’t work between releases that are as far apart as mine is from the latest release. What to do?
The answer is to work my way through the version upgrades manually. This answer on Ask Ubuntu was especially helpful for figuring out how to do this.
Is this a tedious pain in the ass? Yes, yes it is.
Is it entirely my fault for not keeping my OS up-to-date? Also very much yes.
How I did it
Checking the meta-release file for Ubuntu, I see that I need to do two manual upgrades (from kinetic to lunar, then lunar to mantic), then I should be able to use do-release-upgrade from mantic to noble (aka Noble Numbat, aka the current LTS release).
So for each of lunar and mantic, I did these things…
Downloaded the appropriate UpgradeTool from the meta-release file for Ubuntu
Created a directory then unpacked the upgrade tool tar.gz file into it
Solved problems along the way (see below)
Ran the upgrade tool
Then I was able to run do-release-upgrade and, finally after hours of putzing about trying to get the Ubuntu upgrade going, then update Mastodon. Success!
Problems I solved along the way
Irritatingly, the yarn and postgres errors below needed to be fixed before the kinetic to lunar upgrade tool would run successfully.
The yarn gpg key was expired, causing an error during upgrade
The error in question included this line:
The following signatures were invalid: EXPKEYSIG 23E7166788B63E1E Yarn Packaging yarn@dan.cx
According to this issue in the yarn repo, I should’ve just been able to curl the latest GPG key, run apt-key, and all would be well with the world. Except that didn’t work. No, I don’t know why and I don’t much care. I just wanted to get this thing done.
I found an It’s FOSS article about dealing with GPG keys. It’s not the error I was working on, but it was the information I needed to put the correct key in place:
First I backed up the existing key, which was /usr/share/keyrings/yarnkey.gpg.
Then I downloaded and added the latest GPG key:
curl -sS curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | gpg --dearmor | sudo tee /usr/share/keyrings/yarnkey.gpg
And finally, I updated /et/apt/sources.list.d/yarn.list to use that key for decryption:
deb [signed-by=/usr/share/keyrings/yarnkey.gpg] https://dl.yarnpkg.com/debian stable main
That solved the expired key problem. At some point I’ll need to change that back to remove the signed-by bit since I doubt that manually updated key will get any automatic updates.
The postgresql source for apt no longer had a release file for kinetic
The sources for a Postgres update were just as out of date as the ones for Ubuntu. This resulted in this error:
Ign http://apt.postgresql.org/pub/repos/apt kinetic-pgdg InRelease Err http://apt.postgresql.org/pub/repos/apt kinetic-pgdg Release 404 Not Found [IP: 2a04:4e42:2f::820 80]
The answer for this one was pretty easy, once I finally bothered to read the Postgres wiki page for apt. I needed to change /etc/apt/sources.list.d/postgresql.list to point to apt-archive.postgresql.org instead of apt.postgresql.org. The final file contents look like this:
deb [signed-by=/usr/share/postgresql-common/pgdg/apt.postgresql.org.asc] https://apt-archive.postgresql.org/pub/repos/apt kinetic-pgdg main
Fixing that resolved all of the errors and the kinetic to lunar upgrade tool worked without any problems at all.
via {anonymous => 'hash'}; https://anonymoushash.vmbrasseur.com/
July 10, 2025 at 03:00AM
Week Ending July 6, 2025
https://lwkd.info/2025/20250709
Developer News
SIG-Architecture group proposes to form a new Working Group focused on AI Conformance Certification. The WG would define a standardized set of capabilities, APIs, and configurations that Kubernetes clusters must support to reliably and efficiently run AI/ML workloads.
Kubernetes has formed a dedicated Checkpoint/Restore Working Group to integrate native Checkpoint/Restore functionality, enabling container migration and workload pre-emption to improve resource efficiency and support advanced use cases like AI/ML.
Release Schedule
Next Deadline: Code and Test Freeze, July 24/25
Code and Test Freeze starts at 0200 UTC on Friday, July 25. Your PRs should all be merged by then. Vyom Yadav has shared mid-cycle status, including 72 tracked changes. Because this means an extra-long Release Blog, the Comms Team requests that leads submit their release highlights early, if you can.
Cherry-picks for the July Patch Releases are due on July 11.
Featured PRs
131641: DRA kubelet: add dra_resource_claims_in_use gauge vector
This PR introduces a new gauge vector metric, dra_resource_claims_in_use, to the Kubelet; This metric tracks active DRA drivers and informs administrators when a driver is in use, ensuring safe removal of drivers without impacting pod operations; This metric is useful to determine if drivers have active ResourceClaims, preventing issues during the driver removal process.
KEP of the Week
KEP-2831: Kubelet Tracing
This KEP adds support for distributed tracing in the kubelet to help diagnose node-level issues like pod creation latency or container startup delays. It solves the problem of limited visibility into how the kubelet talks to the API server and container runtime by exporting trace data. The implementation uses OpenTelemetry to generate and export spans in the OTLP format. An OpenTelemetry Collector, typically deployed as a DaemonSet, receives and forwards this data to a tracing backend.The feature is enabled through the KubeletTracing feature gate and configured using the TracingConfiguration in the kubelet configuration file.
This KEP is tracked as stable in v1.34.
Other Merges
logger.Error replaced with utilruntime.HandleErrorWithXXX where errors cannot be returned
Fix for validation error when specifying resource requirements at the container level for a resource not supported at the pod level
Declarative Validation enabled for CertificateSigningRequest
Names of new Services are validated with NameIsDNSLabel() relaxing pre-existing validation when RelaxedServiceNameValidation feature gate is enabled
allocationManager’s IsPodResizeInProgress method unexported
New dra_resource_claims_in_use kubelet metrics to inform about active ResourceClaims
Statefulset now respects minReadySeconds
CSIDriverRegistry cleaned up
Function to translate named port to port number cleaned up to avoid duplication
Unit tests for VolumePathHandler
Deprecated
In a major refactoring effort, replaced the deprecated package ‘k8s.io/utils/pointer’ with ‘k8s.io/utils/ptr’ across multiple components
Deprecated gogo protocol definitions removed from k8s.io/externaljwt and k8s.io/cri-api
Subprojects and Dependency Updates
cluster-api v1.11.0-alpha.2: releases alpha version for testing
cluster-api-provider-vspherev1.14.0-alpha.2 : releases alpha version for testing
kustomize [shlex] (https//github.com/google/shlex) has been replaced with carapace-shlex, Bump to viper v1.20.0 and Drop usage of forked copies of goyaml.v2 and goyaml.v3
Shoutouts
No shoutouts this week. Want to thank someone awesome in the community? Tag them in the #shoutouts channel.
via Last Week in Kubernetes Development https://lwkd.info/
July 08, 2025 at 10:56PM
Ep28 - Ask Me Anything About Anything with Scott Rosenberg
There are no restrictions in this AMA session. You can ask anything about DevOps, AI, Cloud, Kubernetes, Platform Engineering, containers, or anything else. Scott Rosenberg, regular guest, will be here to help us out.
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ Sponsor: Codefresh 🔗 GitOps Argo CD Certifications: https://learning.codefresh.io (use "viktor" for a 50% discount) ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
▬▬▬▬▬▬ 👋 Contact me 👋 ▬▬▬▬▬▬ ➡ BlueSky: https://vfarcic.bsky.social ➡ LinkedIn: https://www.linkedin.com/in/viktorfarcic/
▬▬▬▬▬▬ 🚀 Other Channels 🚀 ▬▬▬▬▬▬ 🎤 Podcast: https://www.devopsparadox.com/ 💬 Live streams: https://www.youtube.com/c/DevOpsParadox
via YouTube https://www.youtube.com/watch?v=-u7zcjeAEh8
Vibe Coding Explained: AI Coding Best Practices
Vibe coding is transforming software development by enabling us to interact with AI through simple, natural language instructions. Instead of manually writing code line by line, we can now direct AI agents to generate code, conduct tests, and manage various software development lifecycle operations. In this video, we'll explore essential best practices for vibe coding, including effective session management, the importance of detailed product requirements, memory and context management strategies, testing guidelines, and tips for making the most of "thinking" AI models.
Discover how to leverage vibe coding efficiently and avoid common pitfalls, such as overtrusting AI or neglecting proper code review. Learn why maintaining a small, manageable codebase, regularly updating AI memory, and consistently seeking second opinions are crucial for success. Whether you're new to vibe coding or already integrating it into your workflow, these practical guidelines will help you collaborate more effectively with AI and elevate your development practices.
VibeCoding #AIProgramming #BestPractices
Consider joining the channel: https://www.youtube.com/c/devopstoolkit/join
▬▬▬▬▬▬ 🔗 Additional Info 🔗 ▬▬▬▬▬▬ ➡ Transcript and commands: https://devopstoolkit.live/ai/vibe-coding-explained-ai-coding-best-practices 🎬 The Missing Link: How MCP Servers Supercharge Your AI Coding Assistant: https://youtu.be/n0dCFY6wMeI 🎬 From Shame to Fame: How I Fixed My Lazy Vibe Coding Habits with Taskmaster: https://youtu.be/0WtCBbIHoKE 🎬 Outdated AI Responses? Context7 Solves LLMs' Biggest Flaw: https://youtu.be/DeZ-gw_aop0
▬▬▬▬▬▬ 💰 Sponsorships 💰 ▬▬▬▬▬▬ If you are interested in sponsoring this channel, please visit https://devopstoolkit.live/sponsor for more information. Alternatively, feel free to contact me over Twitter or LinkedIn (see below).
▬▬▬▬▬▬ 👋 Contact me 👋 ▬▬▬▬▬▬ ➡ BlueSky: https://vfarcic.bsky.social ➡ LinkedIn: https://www.linkedin.com/in/viktorfarcic/
▬▬▬▬▬▬ 🚀 Other Channels 🚀 ▬▬▬▬▬▬ 🎤 Podcast: https://www.devopsparadox.com/ 💬 Live streams: https://www.youtube.com/c/DevOpsParadox
▬▬▬▬▬▬ ⏱ Timecodes ⏱ ▬▬▬▬▬▬ 00:00 Intro to Vibe Coding 04:10 UpCloud (sponsor) 05:09 New Session 08:23 Development 11:28 Memory and Context Management 14:38 Miscelaneous 19:27 Vibe Coding Best Practices and Rules
via YouTube https://www.youtube.com/watch?v=W1105cy1D84
Week Ending June 29, 2025
https://lwkd.info/2025/20250704
Developer News
Kubernetes is auditing and cleaning up inactive GitHub organization members in the first week of July 2025 to ensure active and accurate community representation. Contributors who are still active but not tracked by Dev-Stats can request an exception by commenting on the cleanup issue before the deadline on July 18, 2025.
The KubeCon North America 2025 Project Lightning Talk and Maintainer Track CFP is now open and closes soon on July 7th. Make sure to submit your talks before the deadline!
Ongoing discussion in the Kubernetes community regarding Slack migration is closed now. Since Salesforce has postponed the downgrade. Any future conversations about potential migration will take place later on a more relaxed timeline.
Release Schedule
Next Deadline: Feature Blog Placeholders, July 11th
1.34-alpha.2 was released this week, in case you want to play around with the new version.
Featured PRs
12937: feature(kubectl): support –cpu, –memory flag to kubectl autoscale
This PR introduces support for the --cpu and --memory flags in the kubectl autoscale command; The new flags allow users to specify CPU and Memory metrics for horizontal pod autoscaling; The update supports both percentage-based utilization and fixed resource values, thus streamlining resource management; This PR also deprecates the --cpu-percent flag, following the new approach for defining resource targets.
132351: bugfix(hpa): introduce buildQuantity helper for consistent resource quantity
This PR introduces the buildQuantity helper function in the Horizontal Pod Autoscaler (HPA) controller to ensure consistent handling of resource quantities; Before this change, resource quantities were created inline, which causes inconsistencies in handling CPU and memory metrics; With this update, the buildQuantity function standardizes the process by converting raw memory values to KiB and use BinarySI and handling CPU and other resources in milli-units with DecimalSI; Memory metrics are now displayed correctly in Ki, instead of incorrectly appending the “m” suffix thus improving consistency in metric calculations and display.
131837: Deny pod admission for static pods referencing API objects
Static pods that reference API objects are now denied admission by the kubelet. This is to prevent static pods silently running even after the mirror pod creation fails. Currently, mirror pod reconciliation for static pods which reference API objects will fail. However the pod itself is not denied admission and the node would be silently running the static pod’s container. A new feature gate PreventStaticPodAPIReferences is introduced to enable stricter validation for static pods. Enabling this feature gate ensures that the static pod container is not created when the mirror pod creation fails.
KEP of the Week
KEP-3902: Decouple Taint-based Pod Eviction from Node Lifecycle Controller
This KEP splits the existing NodeLifecycleController into two controllers: NodeLifecycleController (to add taints to unhealthy nodes) and TaintEvictionController (to evict pods from tainted nodes). Previously, both tainting and eviction were handled by a single controller, but the main goal of this change is to separate responsibilities for better clarity, organization, and maintainability. The new TaintEvictionController is created from existing taint-manager code and now runs separately. A feature gate called SeparateTaintEvictionController lets you enable or disable the new setup. From Kubernetes v1.29, the taint-based eviction is still enabled by default, but cluster admins can disable the default TaintEvictionController using the --controllers=-taint-eviction-controller flag in kube-controller-manager if needed.
This KEP is tracked as stable in v1.34.
Other Merges
Commonize filtering of Pods by Owner with all orphans in namespace
Fix validation for Job with suspend=true, and completions=0 to set the Complete condition
DRA: the v1alpha4 kubelet gRPC API is no longer supported
Bug fix for replica set failing to be created when a deployment name is too long
Deprecated package ‘k8s.io/utils/pointer’ replaced with ‘k8s.io/utils/ptr’ for the kube-apiserver
More usages of deprecated function ExtractCommentTags migrated to ExtractFunctionStyleCommentTags
Defunct make vet target removed
New SchedulerAsyncAPICalls feature gate added
Code coverage increased for kubelet_client
Validation error message for required fields simplified by removing redundant messages
Flags added to kube-apiserver to make coordinated leader election timers configurable
SizeBasedListCostEstimate feature gate to allow apiserver to estimate sizes of objects to calculate cost of LIST requests
HPA status now displays memory metrics with proper units
ClusterEvent type moved to staging repo
Code and status moved from pkg/scheduler/framework to staging repo
DRA: the kubelet now also cleans up ResourceSlices in some additional failure scenarios
Objects are transformed prior to storage in SharedInformers if a transformer is provided and WatchList is activated
kubectl debug: label added for debugger pod for making cleanup easier
podSpec validation added during StatefulSet creation
Promotions
StreamingCollectionEncodingToJSON and StreamingCollectionEncodingToProtobuf to GA
WaitForAllControlPlaneComponents to GA
Deprecated
StreamingConnectionIdleTimeout field of the kubelet config deprecated
Version Updates
etcd to v3.6.1
kube-openapi bumped
Shoutouts
Jenny Shu (@Jenny Shu) : A little belated, but I want to give a big shout-out to the 1.34 Enhancements Shadows: Drew Hagen(@Drew Hagen), Faeka Ansari (@Faeka Ansari), Josh Michielsen (@jmickey), Rayan Das (@rayandas), Sean McGinnis (@Sean McGinnis), for all their hard work leading up to Enhancements Freeze last week! Keep up the great work!
via Last Week in Kubernetes Development https://lwkd.info/
July 04, 2025 at 02:49AM