Job Description:
Responsibilities:
• Evaluate and test Generative AI POC models built using Open AI and Vertex LLM Models.
• Design, develop, and execute detailed test plans to validate the model performance with real customer data.
• Perform data validation, ensuring accuracy and completeness of the input/output data flow through AI models.
• Collaborate with data scientists and engineers to ensure the model's output is aligned with the expected results.
• Create test cases to assess model accuracy, bias, performance, and edge cases.
• Identify model weaknesses, inaccuracies, and areas for optimization.
• Report bugs, issues, and improvement areas, providing detailed feedback to development teams.
• Use automated testing tools and frameworks for model testing.
• Maintain comprehensive documentation of the QA process and results.
Requirements:
• Proven experience in testing Machine Learning/AI models.
• Familiarity with Generative AI models like Open AI GPT, Vertex AI models.
• Strong knowledge of data validation, model performance, and quality assurance practices.
• Experience with model evaluation metrics such as accuracy, precision, recall, F1 score, and bias analysis.
• Proficiency in Python or other relevant programming languages.
• Familiarity with testing frameworks and automated testing tools for AI models.
• Strong analytical and problem-solving skills.
Preferred:
• Experience in testing AI models with real-world datasets.
• Knowledge of model versioning, deployment, and monitoring.
Location: Dallas, TX
Posted: Nov. 14, 2024, 10:38 p.m.
Apply Now Company Website