Logo
job logo

Director, AI Product Development

NYC Health + Hospitals, New York, NY, United States


The Director of Product Development, Artificial Intelligence leads the full lifecycle of Artificial Intelligence (AI) prototypes and post‑deployment evaluation. This role directs team activities from problem intake and definition, AI solution design, prototype development and evaluation. The Director oversees post‑deployment monitoring, dashboard development, and the creation of plans for system‑wide scaling. The Director ensures that all prototypes, pilot evaluations, and monitoring frameworks meet scientific, clinical, and patient‑safety standards.

Essential Duties and Responsibilities:

  1. Lead the AI Product Development lifecycle from intake through post‑deployment monitoring.
  2. Serves as strategic lead, presenting AI product proposals, evaluations, and roadmaps to executive leadership, funding bodies, and clinical steering committees to secure resource alignment and strategic approval.
  3. Translates clinical and operational issues into AI solution designs.
  4. Oversees the development of prototypes using Machine Learning (ML), Large Language Model (LLM), and Agentic AI techniques through iterative model development and refinement.
  5. Oversees development of applications written in Spark‑Scala, PySpark, Java, and Python.
  6. Guides the creation of Agentic AI prototypes, including Retrieval‑Augmented Generation (RAG), vector database integrations, embedding pipelines, and orchestration logic.
  7. Leads retrospective evaluation including cohort creation, metric design, workflow simulation, and performance assessment.
  8. Prepares prototypes for deployment by Machine Learning Operations (MLOps) Engineering, ensuring clear packaging, documentation, and acceptance criteria.
  9. Owns the documentation for model cards, data lineage, and risk assessment for all prototypes prior to handover to MLOps for production deployment.
  10. Designs and executes silent and active pilots to validate application behavior, workflow fit, suppression logic, thresholds, and performance.
  11. Designs and develops continuous monitoring systems to track performance, patient safety, workflow fit, utilization, Key Performance Indicators (KPIs), clinical impact, and Return on Investment (ROI).
  12. Defines strategies for healthcare setting deployment of validated AI products.
  13. Collaborates with Product Management, Platform Engineering, Interoperability, and MLOps Engineering teams to ensure implementation.
  14. Manages a team of ML Scientists, LLM Scientists, and MLOps Scientists.
  15. Performs other related duties as assigned.

Minimum Qualifications:

  1. Master’s degree from an accredited college or university in Computer Science, Biomedical Engineering, Applied Mathematics, Statistics, or a related technical discipline; and
  2. Five (5) years of experience developing Machine Learning (ML)/Artificial Intelligence (AI) applications with hands‑on prototyping and experimentation or ten (10) years of experience in Machine Learning and Technology development.

Certifications Preferred:

  1. Professional certifications in cloud architecture, ML/AI engineering, or DevOps from leading cloud platforms.

Preferred Knowledge Areas, Skills, Abilities, and other Qualifications:

  1. Expert programming skills in Python, Java, and Spark‑Scala/PySpark with experience building distributed systems and AI pipelines.
  2. Experience with iterative model development, experimental design, evaluation frameworks, and retrospective validation.
  3. Ability to design Retrieval‑Augmented Generation (RAG) workflows, vector database integrations, embedding pipelines, and orchestration layers.
  4. Experience building dashboards and analytics tools using modern Python‑based visualization and User Interface (UI) frameworks.
  5. Experience working with development infrastructure, including containerization (e.g., Docker), dependency and package management, build tooling, and adherence to industry‑standard software engineering best practices.
  6. Experience leading workflow mapping, cross‑functional working groups, stakeholder engagement, and change‑management processes to support safe and effective AI adoption in clinical settings.
  7. Demonstrated leadership in managing multidisciplinary engineering and research teams.
  8. Strong communication and stakeholder engagement skills, especially with clinical teams.
  9. Experience deploying AI systems in healthcare, public‑sector environments, or other highly regulated systems.
  10. Experience building or operating large‑scale RAG or agentic AI systems in production.

Preferred Knowledge Areas, Skills, Abilities, and other Qualifications (continued):

  1. Familiarity with Plotly visualization, clinical note processing, or multimodal clinical models.
  2. Experience Using the Following Software and/or Platforms:

Experience Using the Following Software and/or Platforms:

  • Python, Java, Scala, PySpark, Structured Query Language (SQL).
  • RAG frameworks; vector databases (Pinecone, FAISS, mongoDB); embedding workflows; Python‑based visualization/UI frameworks (Plotly, Dash); Docker; dependency & package management tools; build tools (Maven, Sbt); version control (Git); orchestration layers for AI agents.

#J-18808-Ljbffr