Staff Software Engineer, Data Infrastructure

Peregrine Technologies
New York, New York, United States
Full Time
On Site
Senior
USD 200,000 – 275,000
Data & Analytics Information Technology Telecommunications

Posted 3 days ago

Apply to this job and hundreds more automatically

Get Started
Excellent 4.7

Job summary

The role is for a Staff Data Infrastructure Engineer responsible for architecting and building scalable data systems for real-time operational data ingestion, storage, and processing. The candidate will work on complex challenges including high-throughput data integration, scalable storage solutions, and data pipeline optimization, primarily using technologies like Apache Spark, Kafka, Iceberg, and AWS infrastructure.

Responsibilities

The Staff Data Infrastructure Engineer will architect and build systems to ingest, store, and serve massive volumes of real-time operational data, taking deep ownership over the entire data layer that underpins the company's platform. This involves designing high-throughput integration platforms, architecting scalable open table format layers, and optimizing distributed data processing pipelines using technologies like Spark and Kafka.

Qualifications

Apache Iceberg Apache Kafka Apache Spark AWS Data Infrastructure Data Pipeline Orchestration Kubernetes Python Scala Streaming Technologies

Candidates must have 8+ years of experience architecting and operating large-scale production data infrastructure systems, with deep expertise in open table formats like Apache Iceberg and extensive hands-on experience with Apache Spark for batch and streaming workloads. Strong software engineering fundamentals in Python and/or Scala, coupled with solid experience in cloud platforms like AWS, are required.

English

Benefits

  • Benefits
  • Equity
  • Bonus

Company

Peregrine Technologies

Peregrine Technologies

linkedin
Software Development
Location
New York, New York, United States

Apply to this job and hundreds more automatically

Get Started
Excellent 4.7
Pete DeQuarto avatar
Pete DeQuarto

AIApply is a total game-changer. Thanks to Auto-Apply, I have been applying to jobs I would never even have come across otherwise!

Akarshak Tanwar avatar
Akarshak Tanwar

In one week I applied to over 150 jobs and locked in 2 confirmed interviews, with a support team that was quick to respond. The auto-apply feature is fascinating and a real pleasure to use!

JC
Julian Carlin

It really worked. In under 48 hours it had applied to 100 jobs on my behalf without any time or effort from me. I am looking forward to seeing how it keeps improving.