Retrying Intermittently Failing Tests with VSTest's retryFailedTests
Your Azure DevOps pipeline uses the 'DotNetCoreCLI@2' task to run unit tests. Some tests are failing intermittently. You suspect test flakiness due to race conditions. What should you do to automatically retry failed tests without rewriting the tests?
Quick Answer
Switching to the VSTest@2 task and enabling its built-in retryFailedTests option automatically re-runs failed tests a set number of times, directly addressing race-condition flakiness without touching the test code itself — DotNetCoreCLI@2 has no equivalent retry mechanism, which is why the switch matters here.
⚠ Common exam trap
Test-takers frequently assume the 'DotNetCoreCLI@2' task has a retry option or that a generic pipeline setting exists for test retries, when in fact only the 'VSTest@2' task provides this specific capability, and the other options are either non-existent or overly complex workarounds.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Use the 'VSTest@2' task with 'retryFailedTests: true' option.
The 'VSTest@2' task in Azure DevOps includes a built-in 'retryFailedTests' property that automatically re-runs failed tests a specified number of times. This directly addresses intermittent test flakiness caused by race conditions without requiring any code changes to the tests themselves. The 'DotNetCoreCLI@2' task does not support this retry mechanism, so switching to 'VSTest@2' is the appropriate solution.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Use the 'VSTest@2' task with 'retryFailedTests: true' option.
Why this is correct
The VSTest@2 task supports automatic retry of failed tests.
- ✗
Create a custom script that runs each test individually and retries on failure.
Why it's wrong here
This is complex and not recommended; built-in support exists.
- ✗
Enable 'test retry' in the pipeline settings under 'General'.
Why it's wrong here
There is no such setting in pipeline general settings.
- ✗
Wrap the test task in a 'Retry' loop using a 'each' expression.
Why it's wrong here
This would retry the entire task, not individual tests.
Go deeper
Related to this question
Learn chapter
Introduction to DevOps and Azure DevOps
Key term
Task
A Task in Azure DevOps is a predefined, reusable step that performs a specific action during a build or release pipeline, like compiling code or running tests.
Key term
DevOps
DevOps is a set of practices that combines software development (Dev) and IT operations (Ops) to shorten the development lifecycle and deliver high-quality software continuously.
About these practice questions
Courseiva writes every AZ-400 question from scratch — 823 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on AZ-400
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. Your team uses Azure Pipelines with a YAML-based build pipeline. The pipeline builds a .NET application and runs unit tests. Recently, the unit tests are failing intermittently due to flaky tests. You need to ensure that the pipeline fails only if the same test fails in two consecutive runs. Which feature should you configure?
hard- A.Implement a GitHub Actions workflow with 're-run' trigger.
- B.Use the 'Re-run failed stages' option in the pipeline run.
- C.Enable 'Automatically rerun failed jobs' in the pipeline settings.
- ✓ D.Configure the 'retry failed tests' setting in the pipeline's test tab.
Why D: The correct feature is the 'Retry failed tests' setting in the pipeline's test tab. This allows you to configure the number of times a failed test is automatically retried. If the test passes on retry, the pipeline is marked as succeeded with warnings, effectively requiring two consecutive failures for the pipeline to fail. Option A is incorrect because GitHub Actions workflows are not relevant to Azure Pipelines. Option B is for rerunning failed stages, not individual tests. Option C reruns failed jobs, not tests.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AZ-400 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AZ-400 exam.