Senior Splunk Infrastructure Engineer

Required Qualifications

• Splunk Expertise: 5+ years of experience administering large-scale Splunk Enterprise environments. Proven experience with Indexer Clustering and Search Head Clustering.
• Advanced SPL (Splunk Processing Language): Proficiency in writing, debugging, and optimizing complex SPL queries to support dashboarding, alerting, and data analysis.
• Splunk Enterprise Security (ES): Hands-on experience administering and maintaining Splunk ES environments.
• Configuration Management: Strong proficiency with SaltStack (Salt). You must be comfortable writing custom states and managing configurations for thousands of nodes.
• Unix/Linux Internals: Deep understanding of Linux administration (RHEL/CentOS/Ubuntu). Ability to debug resource contention, file system issues, and network bottlenecks.
• Cloud Infrastructure: Hands-on experience with Google Cloud Platform (GCP), specifically GCE and networking.
• Scripting: Proficiency in Python or Bash for automation tasks.

Preferred Qualifications

• Terraform & Infrastructure Workflow: Experience using Terraform for infrastructure provisioning, combined with SaltStack for configuration management. Ideal candidates understand the distinction and interaction between provisioning (Terraform) and configuration (Salt).
• Kubernetes & GitOps: Experience with Helm and Flux for managing Kubernetes resources, utilizing GitOps methodologies to drive consistent and automated deployments.
• "Done For You" / Platform Engineering: Experience building "Done for you" solutions or paved paths that abstract infrastructure complexity for internal customers, enabling self-service and standardization.
• Multi-Cloud Architecture: Experience managing infrastructure across multiple cloud providers (e.g., GCP, AWS, Azure), understanding the nuances of hybrid networking and data locality.
• OpenTelemetry (OTEL): Knowledge of migration strategies from Splunk Universal Forwarders to OTEL Collectors to standardize data formats.
• Experience working in a highly regulated fintech environment.