Senior Engineer II, Inference Engine - Serving Engine

DigitalOcean
San Francisco, California, United States
Full Time
Remote
Senior
USD 167,200 – 209,000
Engineering Information Technology Telecommunications

Posted 3 days ago

Apply to this job and hundreds more automatically

Get Started
Excellent 4.7

Job summary

The role involves leading the design, development, and deployment of large-scale data plane services for AI inference in a cloud environment, focusing on performance, resiliency, and system architecture, using distributed systems expertise and AI hardware knowledge.

Responsibilities

The engineer will act as a technical leader, driving the design, development, and delivery of critical data plane components hosting large generative AI models. Responsibilities also include architecting high-scale, multi-tenant inference cloud systems and optimizing distributed inference hosting.

Qualifications

Databases Distributed inference serving frameworks such as llm-d, NVIDIA Dynamo, Ray Serve Distributed Systems GPU-level optimization Hosting large language or multimodal models Inference engines like vLLM, SGLang, Modular Infrastructure as code Interconnect technologies like NVlink, XGMI, RoCE Messaging Systems Microservices

Candidates must possess strong expertise in distributed systems, including microservices and messaging, along with hands-on experience hosting large models using inference engines like vLLM or SGLang. Expert proficiency in GoLang or Python and familiarity with GPU-level optimization are also required.

English

Benefits

  • Employee Assistance Program
  • Local Employee Meetups
  • Flexible Time Off Policy
  • Bonus
  • Equity Compensation
  • Employee Stock Purchase Program

Company

DigitalOcean

DigitalOcean

linkedin
Software Development
Location
San Francisco, California, United States

Apply to this job and hundreds more automatically

Get Started
Excellent 4.7
Pete DeQuarto avatar
Pete DeQuarto

AIApply is a total game-changer. Thanks to Auto-Apply, I have been applying to jobs I would never even have come across otherwise!

Akarshak Tanwar avatar
Akarshak Tanwar

In one week I applied to over 150 jobs and locked in 2 confirmed interviews, with a support team that was quick to respond. The auto-apply feature is fascinating and a real pleasure to use!

JC
Julian Carlin

It really worked. In under 48 hours it had applied to 100 jobs on my behalf without any time or effort from me. I am looking forward to seeing how it keeps improving.