Article: Two Misconfigurations That Caused Spark OOM Failures on Kubernetes

Pranav Bhasker — Wed, 03 Jun 2026 09:00:00 GMT

After migrating Spark pipelines to Azure Kubernetes Service, two infrastructure settings interacted destructively: spark.kubernetes.local.dirs.tmpfs=true backed shuffle spill with RAM instead of disk, and a hard podAffinity rule forced all executors onto one node. Together, they caused repeated OOM kills invisible to standard diagnostics.

By Pranav Bhasker

Article: Stragglers, Not Failures: How Adaptive Hedged Requests Reduce p99 Latency by 74 Percent

Prathamesh Bhope — Thu, 28 May 2026 09:00:00 GMT

In fan-out microservice architectures, slow-but-completing requests accumulate across services and drive p99 latency far higher than per-service metrics suggest. This article presents an adaptive hedging mechanism that uses DDSketch for real-time quantile estimation, windowed rotation to handle distribution drift, and a token-bucket budget to prevent load amplification.

By Prathamesh Bhope

InfoQ - Cloud - Articles

Article: Two Misconfigurations That Caused Spark OOM Failures on Kubernetes

Article: Stragglers, Not Failures: How Adaptive Hedged Requests Reduce p99 Latency by 74 Percent