GPUs in Kubernetes for AI Workloads

Today we dive into running AI models on Kubernetes with GPU support. Learn how to manage GPUs in Kubernetes clusters, create GPU nodes, and optimize resource usage without breaking the bank. We'll walk you through setting up a Google Cloud Kubernetes cluster (the same logic should apply to other Cloud providers), deploying AI models like Ollama's Llama2, and handling GPU partitioning. Watch now to master GPU-based AI workloads in Kubernetes!

Kubernetes #GPU #AI

▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ Sponsor: CAST AI 🔗 https://cast.ai ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

Consider joining the channel: https://www.youtube.com/c/devopstoolkit/join

▬▬▬▬▬▬ 🔗 Additional Info 🔗 ▬▬▬▬▬▬ ➡ Transcript and commands: https://devopstoolkit.live/ai/unlock-the-power-of-gpus-in-kubernetes-for-ai-workloads

▬▬▬▬▬▬ 💰 Sponsorships 💰 ▬▬▬▬▬▬ If you are interested in sponsoring this channel, please visit https://devopstoolkit.live/sponsor for more information. Alternatively, feel free to contact me over Twitter or LinkedIn (see below).

▬▬▬▬▬▬ 👋 Contact me 👋 ▬▬▬▬▬▬ ➡ Twitter: https://twitter.com/vfarcic ➡ LinkedIn: https://www.linkedin.com/in/viktorfarcic/

▬▬▬▬▬▬ 🚀 Other Channels 🚀 ▬▬▬▬▬▬ 🎤 Podcast: https://www.devopsparadox.com/ 💬 Live streams: https://www.youtube.com/c/DevOpsParadox

▬▬▬▬▬▬ ⏱ Timecodes ⏱ ▬▬▬▬▬▬ 00:00 AI Inference with GPUs 01:30 CAST AI (sponsor) 02:29 Using GPUs for AI Inference in Kubernetes

via YouTube https://www.youtube.com/watch?v=zuRKdveFuZ4

Data Pipelines - Feat. Fluent Bit, Otel Collector, Logging Operator (You Choose!, Ch. 04, Ep. 05)

Data Pipelines - Choose Your Own Adventure: The Observability Odyssey

In this episode, we'll go through data pipelines. The contestants are Fluent Bit, Otel Collector, and Logging Operator.

Vote for your choice of a tool for signing artifacts at https://cloud-native.slack.com/archives/C05M2NFNVRN. If you have not already joined CNCF Slack, you can do so from https://slack.cncf.io.

This and all other episodes are available at https://www.youtube.com/playlist?list=PLyicRj904Z9-FzCPvGpVHgRQVYJpVmx3Z.

More information about the "Choose Your Own Adventure" project including the source code and links to all the videos can be found at https://github.com/vfarcic/cncf-demo.

٩( ᐛ )و Whitney's YouTube Channel → https://www.youtube.com/@wiggitywhitney

fluentbit #otel #loggingoperator

▬▬▬▬▬▬ 🔗 Additional Info 🔗 ▬▬▬▬▬▬ 🔗 Tracin: https://github.com/vfarcic/cncf-demo/tree/main/manuscript/logging/README.md

via YouTube https://www.youtube.com/watch?v=fhIuylaJ1xA

Resources - Linux Foundation - Education

October 4, 2024 at 09:22AM

via Instapaper

Comparing GitOps: Argo CD vs Flux CD, with Andrei Kvapil

https://kube.fm/flux-vs-argo-andrei

Dive into the world of GitOps and compare two of the most popular tools in the CNCF landscape: Argo CD and Flux CD.

Andrei Kvapil, CEO and Founder of Aenix, breaks down the strengths and weaknesses of Argo CD and Flux CD, helping you understand which tool might best fit your team's needs.

You will learn:

The different philosophies behind the tools.

How they handle access control and deployment restrictions.

Their trade-offs in usability and conformance to infrastructure as code.

Why there is no one-size-fits-all in the GitOps world.

Sponsor

This episode is sponsored by DigitalOcean — learn how GPUs for DigitalOcean Kubernetes can enable your AI/ML workloads.

More info

Find all the links and info for this episode here: https://kube.fm/flux-vs-argo-andrei

Interested in sponsoring an episode? Learn more.

via KubeFM https://kube.fm

October 01, 2024 at 06:00AM

Geomagnetic Storm Watch issued; Prepare for northern lights over Michigan, southern Midwest

A strong solar flare is likely going to blow enough solar energy into our atmosphere to produce widespread northern lights in the next few days. The NOAA’s…

October 3, 2024 at 12:31PM

via Instapaper

GitOps: Branches, directories, or different repositories for the desired state of environments?

Today, we tackle a common GitOps question: "Should we use branches, directories, or different repositories to store the desired state of different environments?" We'll explore the pros and cons of each approach, focusing on the best practices for maintaining application and environment-specific manifests.

GitOps #DevOpsPractices #ArgoCD #BranchingStrategies

Consider joining the channel: https://www.youtube.com/c/devopstoolkit/join

▬▬▬▬▬▬ 💰 Sponsorships 💰 ▬▬▬▬▬▬ If you are interested in sponsoring this channel, please visit https://devopstoolkit.live/sponsor for more information. Alternatively, feel free to contact me over Twitter or LinkedIn (see below).

▬▬▬▬▬▬ 👋 Contact me 👋 ▬▬▬▬▬▬ ➡ Twitter: https://twitter.com/vfarcic ➡ LinkedIn: https://www.linkedin.com/in/viktorfarcic/

▬▬▬▬▬▬ 🚀 Other Channels 🚀 ▬▬▬▬▬▬ 🎤 Podcast: https://www.devopsparadox.com/ 💬 Live streams: https://www.youtube.com/c/DevOpsParadox

via YouTube https://www.youtube.com/watch?v=c6tnnB-t470

Atlanta likely to be covered by haze and chlorine odor from chemical lab fire, officials warn

Haze and a chlorine odor are likely to descend upon Atlanta on Thursday as officials struggle to contain the fallout from a chemical lab fire that ignited Sunday.

Tags:

via Pocket https://www.nbcnews.com/news/us-news/atlanta-likely-covered-haze-chlorine-odor-chemical-lab-fire-officials-rcna173672

October 03, 2024 at 09:34AM

Valkey 8.0 Now GA with Improved Memory Efficiency

Bookmarks Oct 05, 2024 2 min read by Renato Losio InfoQ Staff Editor | Cloud Expert | AWS Data Hero Write for InfoQ Feed your curiosity. Help 550k+ global…

October 3, 2024 at 08:52AM

via Instapaper

Announcing the 2024 Steering Committee Election Results

https://kubernetes.io/blog/2024/10/02/steering-committee-results-2024/

The 2024 Steering Committee Election is now complete. The Kubernetes Steering Committee consists of 7 seats, 3 of which were up for election in 2024. Incoming committee members serve a term of 2 years, and all members are elected by the Kubernetes Community.

This community body is significant since it oversees the governance of the entire Kubernetes project. With that great power comes great responsibility. You can learn more about the steering committee’s role in their charter.

Thank you to everyone who voted in the election; your participation helps support the community’s continued health and success.

Results

Congratulations to the elected committee members whose two year terms begin immediately (listed in alphabetical order by GitHub handle):

Antonio Ojea (@aojea), Google

Benjamin Elder (@BenTheElder), Google

Sascha Grunert (@saschagrunert), Red Hat

They join continuing members:

Stephen Augustus (@justaugustus), Cisco

Paco Xu 徐俊杰 (@pacoxu), DaoCloud

Patrick Ohly (@pohly), Intel

Maciej Szulik (@soltysh), Defense Unicorns

Benjamin Elder is a returning Steering Committee Member.

Big thanks!

Thank you and congratulations on a successful election to this round’s election officers:

Bridget Kromhout (@bridgetkromhout)

Christoph Blecker (@cblecker)

Priyanka Saggu (@Priyankasaggu11929)

Thanks to the Emeritus Steering Committee Members. Your service is appreciated by the community:

Bob Killen (@mrbobbytables)

Nabarun Pal (@palnabarun)

And thank you to all the candidates who came forward to run for election.

Get involved with the Steering Committee

This governing body, like all of Kubernetes, is open to all. You can follow along with Steering Committee meeting notes and weigh in by filing an issue or creating a PR against their repo. They have an open meeting on the first Monday at 8am PT of every month. They can also be contacted at their public mailing list steering@kubernetes.io.

You can see what the Steering Committee meetings are all about by watching past meetings on the YouTube Playlist.

If you want to meet some of the newly elected Steering Committee members, join us for the Steering AMA at the Kubernetes Contributor Summit North America 2024 in Salt Lake City.

This post was adapted from one written by the Contributor Comms Subproject. If you want to write stories about the Kubernetes community, learn more about us.

via Kubernetes Blog https://kubernetes.io/

October 02, 2024 at 04:10PM

Blog: Announcing the 2024 Steering Committee Election Results

https://www.kubernetes.dev/blog/2024/10/02/steering-committee-results-2024/

The 2024 Steering Committee Election is now complete. The Kubernetes Steering Committee consists of 7 seats, 3 of which were up for election in 2024. Incoming committee members serve a term of 2 years, and all members are elected by the Kubernetes Community.

This community body is significant since it oversees the governance of the entire Kubernetes project. With that great power comes great responsibility. You can learn more about the steering committee’s role in their charter.

Thank you to everyone who voted in the election; your participation helps support the community’s continued health and success.

Results

Congratulations to the elected committee members whose two year terms begin immediately (listed in alphabetical order by GitHub handle):

Antonio Ojea (@aojea), Google

Benjamin Elder (@BenTheElder), Google

Sascha Grunert (@saschagrunert), Red Hat

They join continuing members:

Stephen Augustus (@justaugustus), Cisco

Paco Xu 徐俊杰 (@pacoxu), DaoCloud

Patrick Ohly (@pohly), Intel

Maciej Szulik (@soltysh), Defense Unicorns

Benjamin Elder is a returning Steering Committee Member.

Big thanks!

Thank you and congratulations on a successful election to this round’s election officers:

Bridget Kromhout (@bridgetkromhout)

Christoph Blecker (@cblecker)

Priyanka Saggu (@Priyankasaggu11929)

Thanks to the Emeritus Steering Committee Members. Your service is appreciated by the community:

Bob Killen (@mrbobbytables)

Nabarun Pal (@palnabarun)

And thank you to all the candidates who came forward to run for election.

Get involved with the Steering Committee

This governing body, like all of Kubernetes, is open to all. You can follow along with Steering Committee meeting notes and weigh in by filing an issue or creating a PR against their repo. They have an open meeting on the first Monday at 8am PT of every month. They can also be contacted at their public mailing list steering@kubernetes.io.

You can see what the Steering Committee meetings are all about by watching past meetings on the YouTube Playlist.

If you want to meet some of the newly elected Steering Committee members, join us for the Steering AMA at the Kubernetes Contributor Summit North America 2024 in Salt Lake City.

This post was adapted from one written by the Contributor Comms Subproject. If you want to write stories about the Kubernetes community, learn more about us.

via Kubernetes Contributors – Contributor Blog https://www.kubernetes.dev/blog/

October 02, 2024 at 04:10PM

Week Ending September 29, 2024

https://lwkd.info/2024/20240930

Developer News

SIG-Storage plans to move VolumeGroupSnapshot to Beta in 1.32. If you are a CSI driver author who has implemented VGS, please reply to Xing Yang.

For US Deaf Awareness Month, read this interview with some Deaf/Hard of Hearing contributors

Release Schedule

Next Deadline: Production Readiness, October 3

PRR questionnaires are due this week. Make sure you have one for your enhancement.

Featured PRs

KEP of the Week

KEP-4815: DRA: Add support for partitionable devices

One of the original use cases of DRA was to dynamically create and allocate partitions of a full GPU device on demand. As DRA evolved into “structured parameters” this ability to dynamically partition devices has been lost. This KEP proposes introducing the ability for “structured parameters” DRA to allocate both full devices and fixed-size partitions of full devices (across multiple dimensions). This is planned to be done by introducing a new type called PartitionableDevice which would have all the same fields as BasicDevice plus two new fields Includes and ConsumesCapacityFrom. With these new fields, the scheduler has everything it needs to support the dynamic allocation of both full devices and their fixed-size partitions.

This KEP is aiming for alpha in the ongoing v1.32 release cycle.

Other Merges

Deleted UDP endpoint detection is fixed

Allow to filter pods using labels on barrierOp in scheduler_perf and while collecting metrics

Moved InMemoryResponseWriter into own package

Warning about the disablement of internal cloud providers, when setting a value other than external for the --cloud-provider flag

Fixed: expected-actual rule from the testifylint in linter in k8s.io/apimachinery & k8s.io/kubernetes

Allow to set sleepOp duration using parameter

Local reference to hcs structs in windows cri stats test

Dual-stack Services or Services with IPs can be created using the beta MultiCIDRServiceAllocator feature

Test added: covers cronjob-scheduled-timestamp

Bookmark event: new annotation kubernetes.io/initial-events-list-blueprint added and send all items in watchCache store

Test added: unintended changes to dynamic client response handling

Fixed: nil-compare rule and error-nil rules from testifylint in k8s.io/kubernetes

DRA Scheduler: improved queue hint unit test & fixed queuing hint support

Checks added: whether the peer URL for the added etcd member already exists

Improvement: Node QueueHint in the NodeResource Fit plugin

Filtered informers tests added for CRD field selectors

Command line flags --healthz-port and --metrics-port removed

Reimplementation of merge functionality from mergo

Support added: nondeterministic encode for the CBOR serializer

Node Shutdown Controller: attempt to wait for CSI drivers to complete volume teardown

Promotions

Custom Resource Field Selectors to GA

RetryGenerateName to GA

Deprecated

Remove GA feature gate ComponentSLIs

Removed unneeded permissions for volume controllers

Version Updates

cri-tools to v1.31.1

Subprojects and Dependency Updates

grpc v1.66.2 added support for Python 3.13

cloud-provider-openstack v1.31.1 allow Cinder CSI node service to run without OpenStack credentials

kubespray v2.24.3 possibility to fix nodePort of ingress-nginx service with property

cri-o v1.31.0 removed crio config --migrate-defaults command

containerd v1.7.22 cumulative stats can’t decrease

kops v1.30.1 fix conversion for instance-selector flags

via Last Week in Kubernetes Development https://lwkd.info/

September 30, 2024 at 06:00PM

October 2, 2024 at 04:24PM

via Instapaper

Redis Users Want a Change

October 2, 2024 at 04:23PM

via Instapaper

saoudrizwan/claude-dev: Autonomous coding agent right in your IDE, capable of creating/editing files, executing commands, and more with your permission every step of the way.

Claude Dev Thanks to Claude 3.5 Sonnet's agentic coding capabilities Claude Dev can handle complex software development tasks step-by-step. With tools that let…

October 2, 2024 at 09:49AM

via Instapaper

AI coding assistants do not boost productivity or prevent burnout, study finds

Serving tech enthusiasts for over 25 years. TechSpot means tech analysis and advice you can trust. In a nutshell: Developers were supposed to be among the…

October 2, 2024 at 09:28AM

via Instapaper

AWS at KubeCon SLC 2024

AWS at KubeCon + CloudNativeCon North America 2024 Nov 12-15 | Salt Lake City, Utah Meet us at Booth F1 Join us at KubeCon Salt Lake City at Booth F1, where you…

October 1, 2024 at 04:23PM

via Instapaper

(54) Valkey: Posts | LinkedIn

0 notifications total…

October 1, 2024 at 04:13PM

via Instapaper

SSH

SSH Table of contents We’ve integrated liboqs into forks of OpenSSH and libssh to provide prototype post-quantum and hybrid key exchange in the SSH protocol.…

October 1, 2024 at 03:52PM

via Instapaper

Photos of devastation caused by Helene’s floodwaters in North Carolina

After battering Florida as a Category 4 hurricane Thursday, Helene dumped historic rainfall on Georgia, the Carolinas and other states, spurring flash flooding and leaving scores of people confirmed dead, with others still missing.

Tags:

via Pocket https://www.washingtonpost.com/photography/2024/09/30/helene-photos-nc-flood/

October 01, 2024 at 10:45AM

Y Combinator is being criticized after it backed an AI startup that admits it basically cloned another AI startup | TechCrunch

Startups 5:13 PM PDT • September 30, 2024 Image Credits: Pedro Fiúza/NurPhoto / Getty Images A Y Combinator startup named PearAI launched with an X post thread…

October 1, 2024 at 10:37AM

via Instapaper

Kubernetes is simple: it's just Linux, with Eric Jalal

https://kube.fm/kubernetes-just-linux-eric

Eric Jalal, an independent consultant and Kubernetes developer, explains how Kubernetes is fundamentally built on familiar Linux features. He discusses why understanding Linux is crucial for working with Kubernetes and how this knowledge can simplify your approach to cloud-native technologies.

You will learn:

Why Eric considers Kubernetes to be "just Linux" and how it wraps existing Linux technologies.

The importance of understanding Linux fundamentals (file systems, networking, storage).

How Kubernetes provides a standard and consistent interface for managing Linux-based infrastructure.

Why learning Linux deeply can make Kubernetes adoption an incremental step rather than a giant leap

Sponsor

This episode is sponsored by Learnk8s — get started on your Kubernetes journey through comprehensive online, in-person or remote training.

More info

Find all the links and info for this episode here: https://kube.fm/kubernetes-just-linux-eric

Interested in sponsoring an episode? Learn more.

via KubeFM https://kube.fm

October 01, 2024 at 06:00AM

Massive Verizon outage — live updates and what to do with iPhone in SOS

Some Verizon customers are waking up this morning to their phones in SOS mode due to an outage. Service tracker Downdetector is showing a huge spike starting at 10:00 a.m. ET, with many customers saying on X and Threads that they’re having issues placing phone calls.

Tags:

via Pocket https://www.tomsguide.com/phones/live/massive-verizon-outage-live-updates-and-what-to-do-with-iphone-in-sos

September 30, 2024 at 03:29PM

The Future of Shells with Nushell! Shell + Data + Programming Language

Discover the power of Nushell! 🌟 Learn how to set up and use this innovative shell that combines the best of traditional shells with a fully-typed scripting language. We'll explore:

• Setting up Nushell and Docker
• Running commands in Zshell and Nushell
• Filtering and sorting data with ease
• Using structured data for efficient data manipulation
• Auto-complete and error detection features
• Running external commands like kubectl
• Writing and executing Nushell scripts

Nushell #CommandLine #DataScripting #TechGuide

Consider joining the channel: https://www.youtube.com/c/devopstoolkit/join

▬▬▬▬▬▬ 🔗 Additional Info 🔗 ▬▬▬▬▬▬ ➡ Transcript and commands: https://devopstoolkit.live/terminal/discover-the-future-of-shells-with-nushell 🔗 Nushell: https://www.nushell.sh

▬▬▬▬▬▬ 💰 Sponsorships 💰 ▬▬▬▬▬▬ If you are interested in sponsoring this channel, please visit https://devopstoolkit.live/sponsor for more information. Alternatively, feel free to contact me over Twitter or LinkedIn (see below).

▬▬▬▬▬▬ 👋 Contact me 👋 ▬▬▬▬▬▬ ➡ Twitter: https://twitter.com/vfarcic ➡ LinkedIn: https://www.linkedin.com/in/viktorfarcic/

▬▬▬▬▬▬ 🚀 Other Channels 🚀 ▬▬▬▬▬▬ 🎤 Podcast: https://www.devopsparadox.com/ 💬 Live streams: https://www.youtube.com/c/DevOpsParadox

▬▬▬▬▬▬ ⏱ Timecodes ⏱ ▬▬▬▬▬▬ 00:00 Introduction to Nushell 00:59 Pretty Outputs with Nushell 02:22 Mend Renovate (sponsor) 03:22 Sructured Data In Nushell 11:43 Nushell Auto-Complete, Error Detection 14:00 Nushell Language 16:44 Nushell External Commands 18:27 Nushell Works Everywhere 20:10 Nushell Pros and Cons

via YouTube https://www.youtube.com/watch?v=zoX_S6d-XU4