DevOps & SRE Weekly Digest — 2026-09-23

Table of Contents

The Week in One Paragraph

Pre-release and hardening week. OpenTofu v1.13.0-rc1 leads with real upgrade notes: WinRM provisioner support removed, base64gzip output change that can trigger resource diffs, and the start of 32-bit deprecation. Backstage v1.55.0 ships breaking catalog API removals and an Elasticsearch 8 requirement — platform teams need a migration pass before upgrading. On the security side, the gRPC v1.83.2 advisory (GHSA-2v4p-qf9q-27wj) propagated through the ecosystem this week: Vault v2.1.1, Packer v1.16.1, Loki v3.7.8/v3.6.17, and cert-manager v1.20.4 all picked it up along with x/crypto fixes. Prometheus 3.15.0-rc.1 fixes Agent-mode WAL corruption, Alertmanager v0.34.1 repairs inhibition bugs that were un-muting alerts, and Istio 1.31.1/1.30.5/1.29.8 dropped a coordinated patch day across three lines.

All items below were published between September 16 and September 23, 2026, and every claim links to a primary source.

🚨 Top Stories

1. OpenTofu v1.13.0-rc1 — WinRM removed, base64gzip diff churn, 32-bit builds deprecated

What happened: OpenTofu v1.13.0-rc1 shipped September 17, 2026. Upgrade notes:

• The winrm connection type for provisioners is removed. Deprecated in v1.12; upstream libraries are no longer maintained. OpenTofu suggests migrating Windows targets to OpenSSH.
• base64gzip now emits a different (but decompresses-to-identical) compressed form due to a new optimized DEFLATE implementation — if you feed it into a managed resource argument, expect update/replace diffs.
• macOS now requires 13 Ventura or later.
• v1.13 is the final series with official 32-bit builds (*_386, *_arm). The v1.13 series is supported until August 1, 2027.
• Stricter SSH file/wire-format parsing for remote provisioners; symbol libraries arrive as an experiment.

Why it matters: The base64gzip change is a classic “equivalent but not equal” trap — plans may propose noisy replacements after upgrade, and you should know that before rolling the release into CI. WinRM removal is a hard breaking change for Windows provisioner workflows.

Who is affected: Anyone provisioning Windows over WinRM, anyone using base64gzip in resource arguments, and anyone pinned to 32-bit CI runners.

Recommended action: Test v1.13.0-rc1 in CI now. Inventory base64gzip usage and pre-communicate expected one-time diffs. Plan WinRM → SSH migration before GA. If you run 32-bit, plan migration to amd64/arm64 before the v1.13 series EOL.

2. Terraform v1.16.4 — stacks deferred-error fix, TFE policy-render compatibility

What happened: Terraform v1.16.4 shipped September 23, 2026 — the fifth weekly release on the 1.16 line. Fixes:

• Stacks: invalid deferred error when a provider returns a deferral and a resource has unknown count/for_each (#39237).
• Failure rendering policy evaluation outcomes against older Terraform Enterprise versions (#39095).

Why it matters: The stacks deferral bug can block plans/applies outright in stacks adoption; the TFE fix matters for organizations mixing current CLI with older Enterprise installs running policy checks.

Recommended action: Stay on the 1.16 fast-follow train — pin 1.16.4 if you run stacks or TFE policy evaluation. The weekly cadence confirms HashiCorp is still ironing out the 1.16 line; don’t treat it as “stable and done.”

3. Backstage v1.55.0 — breaking catalog API removals, Elasticsearch 8 required

What happened: Backstage v1.55.0 shipped September 15 with a v1.55.1 patch on September 21. Breaking changes:

• Deprecated catalog APIs removed: catalog.enableRelationsCompatibility compatibility layer gone — entity relations now always return the standard targetRef format. CodeOwnersProcessor and AnnotateScmSlugEntityProcessor removed (use the community modules instead). catalog.stitchingStrategy.mode config key removed.
• Search backend Elasticsearch client upgraded to v8, requiring at least Elasticsearch 8.19 — the only 8.x line still under support after 7.x and earlier 8.x EOL.
• v1.55.1 fixes the Yarn patch verifier requiring a separate Yarn CLI install and TechDocs addons not rendering on the new frontend system.

Why it matters: Internal developer portals are production infrastructure now. If your catalog config predates the deprecations or your search backend sits on Elasticsearch 7.x/early-8.x, this upgrade breaks the portal until you migrate.

Recommended action: Before upgrading: grep your app-config for enableRelationsCompatibility and stitchingStrategy.mode, swap removed processors for the community modules, and verify your Elasticsearch cluster is ≥ 8.19. Apply v1.55.1 rather than v1.55.0.

4. Prometheus 3.15.0-rc.1 — Agent-mode WAL corruption fix; Alertmanager v0.34.1 fixes inhibition bugs

What happened: Prometheus 3.15.0-rc.1 shipped September 22 fixing WAL corruption after a failed WAL write in Agent mode (#19700). Separately, Alertmanager v0.34.1 (September 17) fixes several inhibition bugs that caused alerts to be improperly un-muted in some cases (#5542, #5449, #5559).

Why it matters: Two failure modes SREs hate: corrupt WALs (data loss / crash-loop risk for agent fleets pushing remote-write) and silenced alerts spontaneously paging again (alert trust erosion). The Alertmanager fix is a straight upgrade for anyone on 0.34.0.

Recommended action: Upgrade Alertmanager to 0.34.1 — it’s a one-bugfix patch, low risk. Test 3.15.0-rc.1 on a canary Prometheus Agent before GA if you run agents at scale; WAL corruption fixes are exactly what rc cycles are for.

5. HashiCorp Vault v2.1.1 — gRPC and x/crypto security patches, raft retry-join concurrency limit

What happened: Vault v2.1.1 shipped September 16, 2026. Security fixes:

• github.com/apache/thrift → v0.24.0 for GHSA-8wv5-x4w7-5gww.
• golang.org/x/crypto → v0.56.0 for GO-2026-6354 and GO-2026-6355.
• google.golang.org/grpc → v1.83.2 for GHSA-2v4p-qf9q-27wj; UI dompurify bump for SECVULN advisories.

Reliability changes: raft retry-join workers capped at 20 concurrent attempts (further attempts error out), and secrets import gains allowed_ipv4_cidrs for SSRF-safe private connectivity.

Why it matters: Vault is your secrets root of trust — security patches there are never optional. The raft retry-join cap also changes behavior during large unseal/rejoin storms: instead of unbounded retries, you get explicit errors, which is better for alerting but worth knowing.

Recommended action: Schedule the v2.1.1 upgrade in your next maintenance window. Verify raft retry-join alerting after upgrade; if you run automated recovery loops, make sure they handle the new “too many concurrent raft retry joins” error.

☸️ Kubernetes & Cloud Native

• Istio 1.31.1 / 1.30.5 / 1.29.8 — coordinated patch releases September 21 across all three supported lines. Patch-day cadence across every line usually signals fixes the project wants applied promptly; check the release notes for your line and roll sidecars/gateways in the next window.
• cert-manager v1.20.4 — September 16. Dependency security bumps (x/crypto, x/net, grpc v1.83.2, cel-go, go-pkcs12) plus a fix where ingress-shim removed the applyset label from cached Ingress/Gateway objects. Notably, the release documents three golang.org/x/crypto scanner findings it will not fix on 1.20 (deadlocks in the ssh multiplexer — cert-manager never opens SSH connections; govulncheck confirms the vulnerable functions aren’t called). 1.21 already uses x/crypto v0.56.0 — upgrade to the 1.21 line if your compliance gates require a clean scan.
• Crossplane v2.4.2 — September 22. Fixes ownership of ServiceAccounts not transferring to newly active package revisions, makes claims fail closed when an XRD cannot be listed (#7862), and backports otlptrace security bumps. Fail-closed beats fail-open for platform RBAC — take the patch.
• External Secrets Operator v2.11.0 — September 18. Adds OpenStack Barbican authentication via Application Credentials, an Infisical option to keep duplicate secret keys, webhook GetAllSecrets now returns “not supported” instead of misbehaving, and gRPC bumps.
• Argo CD v3.5.3 / v3.4.9 — September 14 (just before this window but worth noting): health-check corrections (stopped KubeVirt VMs and suspended FlinkDeployments now Healthy), repository cleanup on revision change, and Crossplane managed resources now correctly report Progressing while provisioning.
• Argo Workflows v4.1.4 / v4.0.12 — September 18. Patch releases on both supported lines; consult the changelog for your line.
• Rook v1.21.0-beta.0 — September 16. First beta of the next Ceph operator line; lab only.

☁️ Cloud Platforms

AWS, Azure, Google Cloud, and Cloudflare official update channels were not reachable from this run’s research environment, so this section is intentionally short rather than padded with unverified items. Two adjacent signals worth tracking:

• Kubernetes patch cadence: no new Kubernetes minor/patch releases this window (v1.37.0 shipped August 26); the next patch wave will be worth watching given managed-provider lag. If you run EKS/GKE/AKS, map the upcoming v1.36.x/v1.35.x/v1.34.x patches against your control-plane auto-upgrade settings.
• Karpenter: no new release since v1.14.1 (August 21) — stable signal after the v1.14 line’s rapid early patches.

🏗️ Infrastructure as Code & Platform Engineering

• Terraform v1.16.4 — see Top Story #2. The 1.16 line has now shipped five weekly patches; keep absorbing them and re-evaluate the v1.17 beta (-minimal-refresh, variables in provider requirements) as it matures toward GA.
• OpenTofu v1.13.0-rc1 — see Top Story #1. This is the pre-release to test this week if you’re on OpenTofu.
• Backstage v1.55.0 / v1.55.1 — see Top Story #3.
• Packer v1.16.1 — September 18. Plugin getter now supports installing from non-GitHub HTTPS sources (mirrors and artifact repositories exposing the HashiCorp release directory format) — useful for air-gapped builds. Security: gRPC → v1.83.2 to reject requests missing both :authority and Host headers; x/crypto and x/net bumps. Also fixes segfaults on Apple M5 hardware and restores legacy {{ user "name" }} interpolation in plugin-managed templates.
• Ansible v2.22.0b1 — September 18. First beta of the next ansible-core line; check the changelog against your collections before the GA window.

📊 Observability & SRE

• OpenTelemetry Collector v0.161.0 / v1.67.0 — September 16. Breaking changes confined to Go module APIs (deprecated pkg/service ZapOptions, config-watcher interfaces, mdatagen builder config removed) — most operators consume the distro, but if you build custom collectors or use mdatagen, sweep before upgrading. Bug fix worth knowing: shutdown no longer gets randomly delayed by an extra scrape when a scrape outruns the collection interval.
• Grafana Tempo v3.1.0-rc.0/rc.1 — September 17. Big feature release candidate: query-based trace redaction (redact via TraceQL selector instead of enumerating trace IDs — think incident response for accidentally logged secrets in spans), TraceQL metrics arithmetic with sampling-aware extrapolation, Kafka SASL/TLS improvements, and vParquet5 as the default block format for new writes (reads of vParquet3/4 continue, no migration needed).
• Grafana Alloy v1.20.0-rc.0 — September 22. Pre-release of the next collector line; test-only.
• Loki v3.7.8 / v3.6.17 — September 17. Security-focused patch releases: HIGH-severity gRPC bumps to v1.83.2, containerd bumps, x/net and x/crypto updates. Straight upgrade if you run either line.

🔐 DevSecOps & Infrastructure Security

The week’s pattern is one gRPC advisory rippling through the Go ecosystem. Track it once, patch everywhere:

• GHSA-2v4p-qf9q-27wj (google.golang.org/grpc) — fixed in v1.83.2; Packer’s notes describe the fix as rejecting requests missing both :authority and Host headers. This week alone it landed in Vault v2.1.1, Packer v1.16.1, Loki v3.7.8/v3.6.17, and cert-manager v1.20.4. If you run any of these, their patch releases are your remediation path.
• Vault v2.1.1 additionally fixes GHSA-8wv5-x4w7-5gww (apache/thrift → v0.24.0) and GO-2026-6354 / GO-2026-6355 (golang.org/x/crypto → v0.56.0).
• cert-manager v1.20.4 bumps grpc to v1.83.2 for CVE-2026-84304, CVE-2026-84445, CVE-2026-84303, and x/crypto to v0.55.0 for CVE-2026-46600, CVE-2026-56852, CVE-2026-56854. The release also transparently documents three x/crypto scanner findings that do not affect cert-manager (ssh multiplexer deadlocks — no SSH connections are ever opened) — a good template for how to communicate scanner noise to auditors.

Operational guidance: if your dependency-scanning gates flag gRPC < 1.83.2 as HIGH this week, that finding is legitimate — this is a real coordinated fix, not scanner noise. Prioritize Vault (root of trust), then anything exposing gRPC endpoints beyond your cluster boundary.

🛠️ Tools & Releases Worth Watching

• Grafana Tempo v3.1.0-rc.1 — query-based trace redaction is the standout: for teams with compliance requirements, being able to redact spans by TraceQL selector instead of hunting trace IDs is a genuine operational capability. RC for testing. grafana/tempo
• External Secrets Operator v2.11.0 — Barbican Application Credentials support matters if you run OpenStack; the webhook GetAllSecrets contract fix matters if you integrate custom providers. external-secrets/external-secrets
• Packer v1.16.1 — non-GitHub plugin sources unblock air-gapped and mirror-based image pipelines. hashicorp/packer
• OpenTelemetry Collector v0.161.0 — allocation-avoidance in pdata WriteInt64/WriteUint64 is small but compounds on high-throughput pipelines; also fixes the long-scrape shutdown delay. open-telemetry/opentelemetry-collector
• Alertmanager v0.34.1 — single-purpose inhibition fix; if you’re on 0.34.0, take it. prometheus/alertmanager

💡 SRE Takeaways of the Week

1. Patch gRPC everywhere. GHSA-2v4p-qf9q-27wj landed in Vault, Packer, Loki, and cert-manager this week — if your scanners flag gRPC < 1.83.2, treat it as real and use the vendor patch releases as your path.
2. Test OpenTofu v1.13.0-rc1 in CI now. Inventory base64gzip usage (expect one-time replace diffs), plan WinRM → SSH migration, and start any 32-bit → 64-bit move before the v1.13 series EOL (August 1, 2027).
3. Upgrade Alertmanager to v0.34.1. Inhibitions improperly un-muting alerts is a trust-erosion bug; the patch is small and safe.
4. Gate the Backstage v1.55 upgrade on a deprecation sweep — removed catalog processors, targetRef-only relations, and Elasticsearch ≥ 8.19 are all breaking if you haven’t migrated.
5. Watch Prometheus 3.15 GA for Agent fleets. The rc.1 WAL-corruption fix is exactly the kind of change to canary on one agent before wide rollout.

Sources

• OpenTofu v1.13.0-rc1 — opentofu/opentofu
• Terraform v1.16.4 — hashicorp/terraform
• Backstage v1.55.0 — backstage/backstage
• Backstage v1.55.1 — backstage/backstage
• Prometheus 3.15.0-rc.1 — prometheus/prometheus
• Alertmanager v0.34.1 — prometheus/alertmanager
• Vault v2.1.1 — hashicorp/vault
• Packer v1.16.1 — hashicorp/packer
• Istio 1.31.1 — istio/istio
• cert-manager v1.20.4 — cert-manager/cert-manager
• Crossplane v2.4.2 — crossplane/crossplane
• External Secrets Operator v2.11.0 — external-secrets/external-secrets
• Argo CD v3.5.3 — argoproj/argo-cd
• Argo Workflows v4.1.4 — argoproj/argo-workflows
• Rook v1.21.0-beta.0 — rook/rook
• Ansible v2.22.0b1 — ansible/ansible
• OpenTelemetry Collector v0.161.0 — open-telemetry/opentelemetry-collector
• Grafana Tempo v3.1.0-rc.1 — grafana/tempo
• Grafana Alloy v1.20.0-rc.0 — grafana/alloy
• Loki v3.7.8 — grafana/loki
• Kubernetes releases — kubernetes/kubernetes
• GitHub Security Advisories — github.com/advisories