QA Analyst jobs in Ottawa, ON

Location: Remote in Ontario
Hours: Monday - Friday, 37.5 hrs per week
Salary: CAD $105,000 to $125,000 per annum
Reporting To: QA Manager

About Kortext

Founded in 2013, Kortext is a global leader in digital learning and student engagement solutions. We support institutions worldwide with our AI-powered, cutting-edge content discovery and study products, market-leading learner analytics, and streamlined workflows.

Our mission is simple: to help every student fulfil their potential. We’re growing quickly and looking for passionate individuals who share our vision. You’ll join a team that collaborates with our education partners across the globe, working together to make a difference.

The Role
We are seeking an experienced and forward-thinking Senior Automation Specialist to join our QA team. The ideal candidate combines deep hands-on expertise in test automation, particularly Playwright with TypeScript or JavaScript, and modern CI/CD workflows using Git and Azure DevOps, with practical experience applying AI and Large Language Models (LLMs) to quality engineering. You will design and evolve our automation platform, use agentic AI and AI-assisted testing capabilities, and mentor the team in adopting them safely and effectively as part of our Continuous Deployment and Synthetic Monitoring initiatives.

Key Responsibilities

Test automation & quality engineering

• Design, develop, and maintain scalable automated test suites using Playwright with TypeScript/JavaScript for functional, regression, API, security, and performance testing.
• Own and evolve the test automation framework, standards, and reusable libraries; lead code reviews for automation code.
• Integrate automated tests into CI/CD pipelines in Azure DevOps as quality gates for Continuous Deployment, and extend synthetic monitoring with functional scenarios in production.
• Execute automated tests, analyze results, triage failures, perform root cause analysis, and drive timely resolution.
• Collaborate with QA Analysts, developers, and product owners to translate requirements and risk into effective automated coverage.

AI-driven testing & agentic automation

• Design, build, and operate AI testing agents (agentic AI) that can autonomously explore the application, generate and self-heal test scenarios, triage failures, and propose defect reports for human review.
• Apply LLMs to quality engineering: generating test cases and test data from user stories and acceptance criteria, summarizing test results, classifying and de-duplicating defects, and assisting with root cause analysis.
• Integrate AI capabilities into the automation platform and pipelines using tools such as Playwright MCP, agent frameworks, and LLM APIs (e.g., OpenAI, Azure OpenAI, Anthropic), with appropriate guardrails, cost controls, and observability.
• Design and execute test strategies for AI-powered product features: evaluating LLM outputs for correctness, relevance, consistency, bias, and safety; building evaluation datasets and automated evals; and testing prompt/RAG-based workflows for regressions.
• Establish responsible-AI practices for QA: prompt versioning, human-in-the-loop review, data privacy and security of test data used with AI services, and clear documentation of AI-assisted results.

Leadership

• Act as technical lead for automation initiatives; coach and mentor automation specialists and QA analysts in automation and AI-assisted testing.
• Contribute to the QA strategy and roadmap, evaluate emerging tools and AI capabilities, and advocate for best testing practices across workstreams.

Skills & Requirements

Essential

• 5+ years of experience in test automation, including 2+ years in a senior or lead capacity.
• 3+ years of hands-on experience with Playwright (or comparable framework with a strong Playwright track record).
• 3+ years of experience with TypeScript or JavaScript; comfortable with Python for AI tooling and scripting.
• Proven experience using LLMs and generative AI in a testing or engineering context (e.g., GitHub Copilot, ChatGPT/Claude, Azure OpenAI), including prompt engineering and evaluation of AI outputs.
• Hands-on experience building or integrating AI agents or agentic workflows (e.g., Playwright MCP, Model Context Protocol tools).
• Experience testing AI/LLM-based features: evaluation frameworks and metrics, golden datasets, non-deterministic output validation, hallucination and bias detection.
• Strong understanding of CI/CD; experience with Azure DevOps pipeline configuration, test integration, and reporting.
• Proficient in Git and version control workflows.
• Proficient in API testing (REST/GraphQL) and writing SQL queries against relational databases.
• Strong knowledge of QA methodologies, risk-based testing, and shift-left/shift-right practices.
• Excellent communication, problem-solving, and analytical skills; able to explain AI-assisted approaches and their limitations to technical and non-technical stakeholders.

Desirable

• Experience with AI-native or self-healing test tools (e.g., Playwright agents, Testim, mabl, Functionize, Applitools Visual AI).
• Experience with vector databases, RAG pipelines, or fine-tuning/evaluating models (e.g., OpenAI Evals, DeepEval, Ragas, promptfoo).
• Experience with performance or load testing and observability tools (Artillery, Grafana).
• Knowledge of cloud platforms (Azure preferred) and containerization (Docker).
• Experience working in Agile or Scrum development environments.
• ISTQB (including AI Testing certification) or similar QA certification.

Kortext Benefits

21 days’ vacation (rising to 24 days after 5 years)

Extended healthcare benefits, healthcare spending account and Company sick pay

Flexible, hybrid & remote working options to support productivity and wellbeing

Service awards to recognize and celebrate long-term commitment

Referral bonus for introducing great people to our team

Learning and development opportunities

Equal Opportunities Employer: We are committed to building an inclusive workplace where people feel valued, respected, and empowered to thrive. We celebrate diversity and believe that different perspectives and experiences drive innovation.

Accommodation: We strive to ensure our hiring process and workplace are accessible to all and encourage you to speak with us should you require any accommodation during the hiring process or during your employment.

See more about Kortext via our Website and LinkedIn

*Disclaimer: AI may be used in the recruitment process

Pay: $105,000.00-$125,000.00 per year

Application question(s):

• Are you legally entitled to work in Canada?
• Do you have hands on experience with using agentic AI in testing?

Experience:

• Senior or lead: 2 years (required)
• Test automation: 5 years (required)

Location:

• Ontario (required)

Work Location: Remote