Chatbot Evaluation through Q&A Test Sets
Main contact

Project scope
Categories
Data analysis Machine learning Artificial intelligence Data scienceSkills
data science trend analysis large language modeling machine learning chatbot generative artificial intelligence data visualization chatgptJoin our team to help advance the evaluation of a cutting-edge data analysis chatbot. This is a hands-on opportunity to work in a fast-paced environment, contributing to a project that spans multiple industries and enhances our chatbot's ability to provide accurate, relevant insights across diverse data domains.
In this role, you’ll develop a collection of Question & Answer (Q&A) sets designed to test and benchmark the performance of our AI-driven chatbot. You’ll work directly with structured datasets across specific industries to create targeted Q&A sets, helping us improve and refine our chatbot’s data analysis capabilities.
Who Should Apply:
- Experience using ChatGPT, Llama, and/or other LLMs.
- Students in fields such as Data Science, Computer Science, Business Analytics, or similar.
- Strong analytical and problem-solving skills, with an interest in data science and machine learning.
- Experience with large datasets and familiarity with data visualization and trend analysis.
- Working experience in industries relating to healthcare, environment, marketing, and others.
- Craft Comprehensive Q&A Test Sets: Create detailed question sets and reference answers, working with various Large Language Models (LLMs) and verifying responses to ensure accuracy.
- Benchmark and Validate: Ensure consistency in answers through manual validation, helping us establish a ground truth for effective evaluation.
Providing specialized, in-depth knowledge and general industry insights for a comprehensive understanding.
Sharing knowledge in specific technical skills, techniques, methodologies required for the project.
Direct involvement in project tasks, offering guidance, and demonstrating techniques.
Providing access to necessary tools, software, and resources required for project completion.
Scheduled check-ins to discuss progress, address challenges, and provide feedback.
About the company
Ample Insight is a leading Canadian data consulting firm helping organizations across diverse industries build and scale AI and data solutions. We have a proven track record of solving some of the most complex technical and business problems for leading startups and Fortune 500 companies.
Main contact
