Sr. Data Platform Engineer

About the Company

A fast-growing AI technology company building advanced analytics and optimization platforms for complex, large-scale systems. The team operates globally and focuses on delivering enterprise-grade, data-driven solutions.

As a Senior Data Platform Engineer, you will design and scale the core data infrastructure powering the company's AI and analytics platform. Your work will focus on taking an existing system to an enterprise-grade, highly reliable data platform supporting large-scale workloads across cloud and on-prem environments.

You'll build high-performance data connectors, governance frameworks, object storage and lakehouse components, and backend services. This includes developing S3-compatible object storage, Delta Lake-compatible services, and integrations with modern data ecosystems such as Snowflake, Databricks, and BigQuery.

*Unfortunately, this role does not offer sponsorships. U.S. Citizens or Greencard Holders Only!*

Details:

• Full-Time, Permanent Position
• $180,000 - $200,000 + Equity + Benefits
• San Francisco, CA | 5 Days On-Site

Key Responsibilities

• Build secure, scalable data connectors and ingestion frameworks across cloud and on-prem systems.
• Develop core data platform components, including S3-compatible object storage and table/metadata services.
• Implement Delta Lake-compatible services (ACID transactions, schema evolution, time travel).
• Create platform APIs supporting Spark, Pandas, and distributed compute frameworks.
• Optimize performance, reliability, and scalability in multi-cloud and hybrid environments.
• Improve observability and high-availability for mission-critical data services.

Qualifications

• 10+ years of experience in data platform engineering, data infrastructure, or distributed systems.
• Strong backend development skills (Python or similar).
• Deep understanding of lakehouse architectures (Delta Lake, Iceberg) and object storage.
• Experience building large-scale data platforms or pipelines.
• Strong cloud experience (AWS, Azure, or GCP).

Preferred

• Experience with data governance or metadata platforms.
• Familiarity with streaming, CDC, and modern ingestion patterns.
• Experience with Spark, Kubernetes, or performance tuning.

Oscar Associates Limited (US) is acting as an Employment Agency in relation to this vacancy.