OpenTelemetry

Learn how to export and rehydrate telemetry data to and from AWS S3 using the OpenTelemetry (OTel) Collector.

With this update to Honeycomb Telemetry Pipeline, easily build & reconfigure telemetry pipelines & sample with the ability to pull data when you need it.

Continuing to answer questions while I'm running our social media this week, here's one from Jeremy Morrell at Cloudflare, formerly at Heroku. "I'd be curious about writing the full unsampled firehose to S3. That can be a lot of data! Were there any hurdles there?" And yes, it's such an interesting and deep one once you get to a sufficient scale of data. The first knob to tweak is using the most efficient encoding -- this means using OTLP/proto instead of OTLP/json. But there's a lot more tweaking to do beyond that. Beyond using OTLP/proto there's some other things you should do with large data volume. For instance, use AWS PrivateLink or gateway endpoints both to talk to Honeycomb and to S3. You definitely don't want that volume of data passing to the public internet (even TLS encrypted, this isn't about security), because as Corey Quinn says, NAT gateways cost your firstborn child. So use PrivateLink to make sure that the data takes the most cost-effective path. You also will need to, at sufficiently large scales, use a prefix in your S3 buckets for the collector ID, so that you are not "laser pointer of death" blasting the S3 infrastructure by writing to /year=2025/month=06/day=27/hour=18/minute=01 etc. from all collectors at once; prefix your writes with id=abc012 where abc012 is a stand-in for the collector's pod name, so each parallel writer uses a different key. and you'll probably want to (again, at very large scales) talk to your AWS account rep about making sure that your S3 buckets are manually partitioned so that they will not hotspot on individual S3 metadata servers. But at smaller scales, writing raw telemetry to S3 just works. Happy observing, -- Liz Fong-Jones

Contrib repository for the OpenTelemetry Collector - open-telemetry/opentelemetry-collector-contrib

Observability isn’t just about collecting data. It’s about making that data useful. In a world full of endless system events, how can we actually understand ...

An example of how to run a Ruby project in Docker in a Buildkite pipeline - buildkite/ruby-docker-example

Due to the significant number of modifications and the extensive user base affected by them, existing HTTP instrumentations published by OpenTelemetry are required to implement a migration plan that will assist users in transitioning to the stable HTTP semantic conventions. Specifically, when existing HTTP instrumentations published by OpenTelemetry are updated to the stable HTTP semantic conventions, they: SHOULD introduce an environment variable OTEL_SEMCONV_STABILITY_OPT_IN in their existing major version, which accepts: http - emit the stable HTTP and networking conventions, and stop emitting the old HTTP and networking conventions that the instrumentation emitted previously. http/dup - emit both the old and the stable HTTP and networking conventions, allowing for a phased rollout of the stable semantic conventions. The default behavior (in the absence of one of these values) is to continue emitting whatever version of the old HTTP and networking conventions the instrumentation was emitting previously. Need to maintain (security patching at a minimum) their existing major version for at least six months after it starts emitting both sets of conventions. May drop the environment variable in their next major version and emit only the stable HTTP and networking conventions. [!NOTE] OTEL_SEMCONV_STABILITY_OPT_IN is only intended to be used when migrating from an experimental semantic convention to its initial stable version.

Contrib Packages for the OpenTelemetry Ruby API and SDK implementation. - open-telemetry/opentelemetry-ruby-contrib

Join Johannes and Raphael in this episode as they dive into the world of CI/CD observability using GitHub Actions and OpenTelemetry. They discuss the importa...

We benchmarked the overhead of OpenTelemetry in a high-load Go application and compared it to eBPF-based instrumentation. The results may surprise you - tracing adds cost, but it’s not always a dealbreaker

The goal of this blog post is to demonstrate how you can expose an OpenTelemetry (OTel) Collector running inside Kubernetes to the outside world securely, using the Kubernetes Gateway API and mutual TLS (mTLS) for authentication and encryption. As observability becomes increasingly critical in modern distributed systems, centralizing telemetry data via OTel Collectors deployed in one or many Kubernetes clusters is common practice. Often, services or agents running outside your Kubernetes cluster need to send data to these Collectors. Exposing internal services requires careful consideration of security and standardization. This is where the Kubernetes Gateway API and mTLS shine.

Contextual information that is passed between signals.

Learn about sampling and the different sampling options available in OpenTelemetry.

Learn how semantic conventions improve database observability, then follow a demo on how to instrument a simple app with OpenTelemetry and visualize it in Grafana.

Learn how to monitor SNMP easily with open source tools by using the observIQ distribution of OpenTelemetry.

Configuring Your OTel Collector’s Internal Telemetry Endpoint

Component(s) auto-instrumentation Is your feature request related to a problem? Please describe. I'd like to open a feature request to include Ruby autoinstrumentation in the operator's autoinstrum...

Contrib Packages for the OpenTelemetry Ruby API and SDK implementation. - open-telemetry/opentelemetry-ruby-contrib

Find libraries, plugins, integrations, and other useful tools for using and extending OpenTelemetry.

Learn how to collect telemetry data from applications written in Ruby.

Tutorial for using OpenTelemetry auto-instrumentation for Ruby

An implementation of auto-instrumentation using the OpenTelemetry Operator.

A language-specific implementation of OpenTelemetry in Ruby.

OpenTelemetry tools & library for Nix. Contribute to FriendsOfOpenTelemetry/opentelemetry-nix development by creating an account on GitHub.

One of the secret pleasures of life is to be paid for things you would do for free. On a completely unrelated note, this blog post documents my time figuring out how to monitor a Minecraft server with OpenTelemetry, Prometheus and Dash0.

HTTP/GRPC/UDP

Good example of creating an otel component that is testable

Build custom OpenTelemetry Collector Distributions from manifest files with a local build utility, Docker, Google Cloud Build, or a GitHub Action. - observIQ/otel-distro-builder