This role is for a Machine Learning Engineer focused on AWS inference optimization. You will work on improving inference performance for large language OSS models, with ownership of price-performance for Amazon LLM inference across SageMaker, Hyperpod, and Bedrock.
What You’ll Do
- Develop model servers.
- Build CUDA kernels in vLLM and SGLang.
- Create services for GenAI optimization and benchmarking.
- Create or modify a mid-size application or service.
- Optimize large language OSS models such as Qwen, Gemma, gptoss, and others by improving inference stacks including vLLM and SGLang.
- Perform performance tuning for GenAI inference on Amazon SageMaker, Amazon Bedrock, and Hyperpod.
- Implement inference optimization techniques including Speculative decoding, Disaggregated Inference, and CUDA kernel tuning.
- Own price-performance for Amazon LLM inference deployed on SageMaker, Hyperpod, and Bedrock.
- Deliver production solutions that leverage existing academic and industrial research or apply practical, original thinking.
- Deliver solutions into customer-facing products.
Key Technologies
- vLLM, SGLang
- Amazon SageMaker, Amazon Bedrock
- Hyperpod
- CUDA
- C#, C++, Java, Perl
- Transformer architecture
Requirements
- 3+ years of non-internship professional software development experience.
- 2+ years of non-internship design or architecture experience, including design patterns and reliability and scaling of new and existing systems.
- 1+ year of software development engineer or related occupational experience.
- 1+ year designing and developing large-scale, multi-tiered, multi-threaded, embedded or distributed software applications, tools, systems, and services using C#, C++, Java, or Perl.
- Bachelor’s degree (or foreign equivalent) in Computer Science, Engineering, Mathematics, or a related field.
- Knowledge of machine learning and LLM fundamentals, including transformer architecture, training/inference lifecycles, and optimization techniques.
Preferred Qualifications
- 3+ years of full software development life cycle experience, including coding standards, code reviews, source control management, build processes, testing, and operations.
- Bachelor’s degree in computer science or equivalent.
Education
Bachelor’s degree (Computer Science, Engineering, Mathematics, or related field).
Location and Work Mode
- Santa Clara, CA (onsite)
Compensation
USD 165,200 - 223,600 per year.
Benefits
- Sign-on payments
- Restricted stock units (RSUs)
- Health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance, option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage)
- 401(k) matching
- Paid time off
- Parental leave
- Workplace accommodation or adjustment during the application and hiring process (including support for the interview or onboarding process)
Los Angeles County Applicants
Job duties for this position include:
- Work safely and cooperatively with other employees, supervisors, and staff.
- Adhere to standards of excellence despite stressful conditions.
- Communicate effectively and respectfully with employees, supervisors, and staff to ensure exceptional customer service.
- Follow all federal, state, and local laws and Company policies.
- Criminal history may have a direct, adverse, and negative relationship with some of the material job duties of this position, including the abilities and responsibilities listed above, as well as the abilities to adhere to Company policies, exercise sound judgment, effectively manage stress and work safely and respectfully with others, exhibit trustworthiness and professionalism, and safeguard business operations and the Company’s reputation.
- Pursuant to the Los Angeles County Fair Chance Ordinance, qualified applicants with arrest and conviction records will be considered for employment.
#J-18808-Ljbffr