This chapter covers re-testing and validation testing, a critical phase in the penetration testing lifecycle that verifies the effectiveness of remediation efforts. For the PT0-002 exam, understanding the process of re-testing, including timing, scope, and reporting, is essential because it directly impacts the validity and completeness of a penetration test. Approximately 5-10% of exam questions touch on re-testing and validation, often focusing on when re-testing should occur, what constitutes a valid re-test, and how to document results.
Jump to a section
Re-testing and validation testing in penetration testing is like a surgeon performing a follow-up operation after a tumor removal. Initially, the surgeon (penetration tester) identifies and removes a tumor (vulnerability). However, the surgeon must later re-enter the site to verify that no cancerous cells remain and that the surrounding tissue has healed properly. In the first operation, the surgeon may have removed the visible tumor, but microscopic remnants could be left behind. The follow-up procedure involves taking biopsies (re-scanning) from the exact same area and adjacent regions to ensure complete eradication. The surgeon also checks that the sutures (patches) are holding and that no new growth (regression) has occurred. If any residual cells are found, the surgeon must decide whether to remove them immediately (remediate) or schedule another procedure (escalate). The patient (organization) is monitored post-surgery for recurrence, and the surgeon documents every finding for the medical record (report). This mirrors how a penetration tester must re-execute exploits, re-scan ports, and re-verify configurations to confirm that vulnerabilities are fully remediated and no new issues were introduced during the fix.
What is Re-Testing and Validation Testing?
Re-testing, also known as validation testing, is the process of re-evaluating a target system after vulnerabilities have been reported and remediated by the client. Its purpose is to confirm that the previously identified vulnerabilities have been properly fixed and that no new vulnerabilities were introduced during the remediation process. This phase is not always included in a penetration test engagement; it is typically an optional add-on service or part of a multi-phase assessment.
Why Re-Testing Exists
Penetration tests are not just about finding vulnerabilities; they are about enabling the client to improve their security posture. Without re-testing, the client cannot be certain that their remediation efforts were successful. Re-testing provides: - Verification of Fixes: Confirms that the specific vulnerabilities reported have been eliminated. - Regression Testing: Ensures that the fixes did not introduce new vulnerabilities or break existing functionality. - Compliance: Many regulatory frameworks (e.g., PCI DSS) require validation that vulnerabilities have been remediated. - Closure: Provides a formal end to the engagement with a validated outcome.
How Re-Testing Works Internally
Re-testing follows a structured process:
Remediation Period: After the initial penetration test report is delivered, the client is given a window (typically 30-90 days) to implement fixes. The tester may provide guidance but does not perform the fixes.
Re-Test Planning: The tester reviews the original report and creates a re-test plan. This plan specifies which vulnerabilities to re-test, the methods to use (e.g., same exploit, different exploit, or just verification scanning), and the scope (only affected systems or full re-test).
3. Reconnaissance and Scanning: The tester repeats the initial reconnaissance and scanning phases, but focused on the previously vulnerable systems. This includes: - Port Scanning: Re-scan the same ports to see if services are still running or have changed. - Vulnerability Scanning: Run the same vulnerability scanners to check if the specific vulnerability signatures are still present. - Manual Verification: For critical vulnerabilities, the tester may attempt to exploit the same vulnerability using the same or a different payload.
Exploitation Attempts: The tester attempts to exploit the same vulnerabilities using the same techniques as before. If the exploit succeeds, the vulnerability is not remediated. If it fails, the tester must determine why:
The vulnerability was patched (correct remediation).
The vulnerability was mitigated by a workaround (e.g., WAF rule blocking the exploit).
The vulnerability was removed by disabling the service entirely (acceptable but may impact functionality).
Validation of New Issues: During re-testing, the tester must also look for new vulnerabilities that may have been introduced by the remediation. For example, a patch might open a new port or a configuration change might weaken authentication.
Reporting: The tester produces a re-test report that documents:
Each vulnerability from the original report and its current status (Remediated, Partially Remediated, Not Remediated, or New Finding).
Evidence of the re-test (e.g., screenshots, logs, exploit output).
Any new vulnerabilities discovered.
Recommendations for unresolved issues.
Key Components, Values, and Timers
- Remediation Window: Typically 30, 60, or 90 days. The PT0-002 exam may ask about appropriate timelines. Standard best practice is 30 days for high-risk vulnerabilities, 60 days for medium, and 90 days for low. - Re-Test Validity Window: A re-test is only valid if performed within a certain period after remediation. If too much time passes, the environment may have changed, invalidating the re-test. Common window: 30 days after the client reports remediation complete. - Scope of Re-Test: Two common approaches: - Focused Re-Test: Only the specific systems and vulnerabilities from the original report. - Full Re-Test: The entire original scope is re-assessed. This is more thorough but more expensive. - Evidence Requirements: The PT0-002 exam emphasizes that re-testing must produce evidence. This includes:
Screenshots showing the vulnerability is no longer present.
Logs of failed exploit attempts.
Output from vulnerability scanners showing a clean scan.
Regression Testing: The tester should verify that the fix did not break any business functionality. For example, if a patch was applied to a web server, the tester should ensure the web application still works as expected.
Configuration and Verification Commands
While re-testing is not about configuration, the tester uses tools to verify fixes. Common commands and tools include:
- Nmap: To verify that a port that was previously open is now filtered or closed.
nmap -p 445 --reason 192.168.1.10- Metasploit: To re-attempt an exploit that previously worked.
msfconsole -q
use exploit/windows/smb/ms17_010_eternalblue
set RHOSTS 192.168.1.10
check
exploitBurp Suite: To re-test a web application vulnerability such as SQL injection by re-sending the same payload and observing the response.
Nessus / OpenVAS: To run a scan with the same policy used in the initial test and compare results.
Interaction with Related Technologies
Re-testing interacts with: - Patch Management: The client's patch management process determines how quickly vulnerabilities are fixed. The tester must verify that patches were applied correctly. - Configuration Management: Changes to configuration files (e.g., firewall rules, web server configs) must be validated. - Change Management: The client should have a change management process that documents all remediation actions. The tester can review these records to understand what was changed. - Vulnerability Management: The client's vulnerability management platform may track the status of vulnerabilities. The tester's re-test results should feed back into this system.
Special Considerations
Partial Remediation: Sometimes a vulnerability is only partially fixed. For example, a patch may be applied but the underlying misconfiguration remains. The tester must document the residual risk.
False Positives: During re-testing, a vulnerability that was previously confirmed may appear to be fixed, but only because the tester's exploit method changed. The tester must ensure they are using the same or equivalent technique.
New Vulnerabilities: The remediation process may introduce new vulnerabilities. For example, applying a security patch might disable a required feature, leading the client to implement a risky workaround. The tester must identify these.
Scope Creep: The client may request additional testing during the re-test phase. The tester should adhere to the original scope unless a change order is approved.
PT0-02 Exam Focus
The PT0-02 exam tests re-testing in the context of the reporting and communication domain (Objective 4.2). Key topics include:
When to perform re-testing (after remediation window).
What constitutes a valid re-test (same scope, same methods, evidence).
How to document re-test results (status categories: Remediated, Partially Remediated, Not Remediated, New Finding).
The importance of regression testing.
The difference between a focused re-test and a full re-test.
Common exam traps:
Thinking that a re-test is always required (it is not; it is optional).
Believing that a re-test involves only automated scanning (manual verification is often required for critical vulnerabilities).
Assuming that if an exploit fails, the vulnerability is fixed (it may be mitigated by a WAF or other control, not patched).
Plan Re-Testing Scope
Review the original penetration test report and identify all vulnerabilities that were reported. Determine which vulnerabilities will be re-tested. Typically, all vulnerabilities with a risk rating of High or Critical are re-tested, and a sample of Medium and Low may be re-tested. The scope may also include systems that were not originally vulnerable but are related to the fix. The tester must also decide whether the re-test will be focused (only affected systems) or full (entire original scope). This decision is documented in a re-test plan.
Verify Remediation Actions
Obtain from the client a list of remediation actions taken for each vulnerability. This may include patch IDs, configuration changes, or workarounds. The tester reviews this list to understand what was done and to plan the verification approach. For example, if a patch was applied, the tester may check the patch level via a registry key or file version. If a configuration change was made, the tester may manually inspect the configuration file or use a tool to verify the setting.
Execute Re-Scanning
Run automated vulnerability scanners against the target systems using the same scanning policies as the initial test. Compare the new scan results with the original findings. Any vulnerability that no longer appears in the scan results is a candidate for being remediated. However, the tester must be cautious: a scanner may miss a vulnerability if the scanner's signature was updated or if the vulnerability was only partially fixed. Therefore, automated scanning is only a first pass; manual verification is required for critical findings.
Manual Verification of Exploits
For each critical or high-risk vulnerability, the tester attempts to manually exploit the vulnerability using the same or similar technique as in the initial test. For example, if SQL injection was found, the tester re-sends the same payload and observes the response. If the exploit succeeds, the vulnerability is not remediated. If it fails, the tester must determine the reason: the vulnerability was patched, a WAF blocked the exploit, or the service was removed. The tester documents the outcome with evidence such as screenshots or logs.
Regression Testing
Verify that the remediation did not break any business functionality or introduce new vulnerabilities. For example, if a patch was applied to a web server, the tester checks that the web application still functions correctly (e.g., login works, pages load). Additionally, the tester performs a quick scan for new vulnerabilities that may have been introduced. This step is crucial because a fix that breaks functionality may be rolled back, or a new vulnerability may be more severe than the original.
Document and Report Results
Create a re-test report that lists each vulnerability from the original report along with its current status. Use standard categories: Remediated, Partially Remediated, Not Remediated, or New Finding. For each, provide evidence (screenshots, logs, scanner output). Include a summary of regression testing results. If any vulnerabilities remain un-remediated, provide updated risk ratings and recommendations. The report should be delivered to the client along with the original report for reference.
In a large enterprise, re-testing often occurs after a major patch cycle, such as Microsoft's Patch Tuesday. For example, a penetration test of a financial institution's internal network found multiple critical vulnerabilities, including an unpatched SMB vulnerability (EternalBlue) and a weak SNMP community string. The client applied the SMB patch and changed the SNMP string. The re-test involved scanning the same IP range with Nessus and manually attempting to exploit the SMB vulnerability using Metasploit. The exploit failed, and the scanner showed the SMB service was patched. For SNMP, the tester used snmpwalk with the old community string and received an authentication error, confirming the change. However, during regression testing, the tester discovered that the SMB patch had disabled file sharing on a critical server, causing business disruption. The client had to re-enable sharing with a more restrictive configuration, which introduced a new risk that required further testing.
Another scenario involves web application testing. A retail company's e-commerce site had a SQL injection vulnerability in the product search function. The developer implemented parameterized queries to fix it. During re-test, the tester sent the same SQL injection payload and received a 200 OK with no error, but the payload was reflected in the response as a literal string, indicating that input was properly escaped. However, the tester also tested for second-order SQL injection by storing malicious input in the database and later retrieving it, but no injection occurred. The re-test report marked the vulnerability as Remediated. However, the tester also noticed that the fix introduced a new XSS vulnerability because the developer didn't sanitize output in the search results. This new finding was documented.
A common misconfiguration in re-testing is when the client implements a workaround instead of a proper fix. For example, instead of patching a vulnerable Apache version, the client placed a WAF in front of the server to block the exploit. During re-test, the tester's exploit attempt failed because the WAF blocked the malicious payload. However, the underlying vulnerability still exists, and the WAF could be bypassed or misconfigured. The tester should note this as Partially Remediated and recommend an actual patch.
Performance considerations: Re-testing can be time-consuming if the original scope was large. A full re-test may take as long as the initial test. Therefore, many clients opt for a focused re-test. However, focused re-tests risk missing new vulnerabilities introduced elsewhere. The tester should advise the client on the trade-offs.
What goes wrong: The most common issue is miscommunication about the scope of the re-test. The client may expect a full re-test but only paid for a focused one. Another issue is timing: if the re-test is delayed, the environment may have changed, making the results invalid. Also, if the client does not provide clear documentation of remediation actions, the tester may waste time trying to understand what was changed.
The PT0-002 exam tests re-testing and validation under Objective 4.2: 'Explain the importance of communication during the penetration testing process.' Re-testing is a key communication milestone. The exam expects you to know:
The purpose of re-testing: to validate remediation.
When re-testing should occur: after the client has had time to remediate (typically 30-90 days).
The difference between a focused re-test and a full re-test.
The four status categories for vulnerabilities in a re-test report: Remediated, Partially Remediated, Not Remediated, and New Finding.
The importance of evidence (screenshots, logs) in the re-test report.
The concept of regression testing: ensuring the fix didn't break anything.
Common wrong answers on exam questions: 1. Assuming re-testing is always required: Many candidates think that every penetration test includes re-testing. In reality, re-testing is an optional add-on service. The exam may present a scenario where the client does not pay for re-testing, and the correct answer is that the tester should not perform re-testing without a new contract. 2. Believing that a failed exploit means the vulnerability is fixed: A WAF or IPS could block the exploit while the underlying vulnerability remains. The correct answer is that the tester should attempt to bypass the control or verify the patch level directly. 3. Thinking that automated scanning alone is sufficient for re-testing: The exam emphasizes that manual verification is required for critical vulnerabilities. Automated scans can produce false negatives. 4. Confusing re-testing with continuous monitoring: Re-testing is a point-in-time activity, not ongoing.
Specific numbers and terms that appear on the exam:
Remediation window: 30 days for high-risk, 60 for medium, 90 for low.
The term 'regression testing' is used explicitly.
The four status categories: Remediated, Partially Remediated, Not Remediated, New Finding.
The phrase 'focused re-test' vs 'full re-test'.
Edge cases:
What if the client remediates a vulnerability by removing the service entirely? That is considered remediation, but the tester should verify that the removal did not impact other systems.
What if a vulnerability is no longer exploitable due to a network change (e.g., firewall rule)? The vulnerability still exists but is mitigated. This is Partially Remediated.
What if the tester discovers a new vulnerability during re-testing? It must be documented as a New Finding.
How to eliminate wrong answers: Focus on the mechanism. If an answer says 're-testing is always part of the engagement,' eliminate it because it's optional. If an answer says 'automated scan confirms fix,' look for a more precise answer involving manual verification. If an answer says 'vulnerability is fixed because exploit failed,' consider whether a WAF could be the reason.
Re-testing validates that reported vulnerabilities have been remediated and that no new vulnerabilities were introduced.
The four status categories for vulnerabilities in a re-test report are: Remediated, Partially Remediated, Not Remediated, and New Finding.
A focused re-test only checks the original vulnerable systems; a full re-test reassesses the entire scope.
Automated scanning alone is insufficient; manual verification is required for critical vulnerabilities.
Regression testing ensures that remediation did not break business functionality or introduce new risks.
Re-testing is typically performed 30-90 days after the initial report, depending on risk level.
If an exploit fails, it may be due to a security control, not a patch; verify the actual fix.
These come up on the exam all the time. Here's how to tell them apart.
Focused Re-Test
Only tests systems and vulnerabilities identified in the original report.
Less time-consuming and less expensive.
May miss new vulnerabilities introduced elsewhere in the environment.
Suitable for clients with limited budget or time.
Provides a quick confirmation that specific fixes were applied.
Full Re-Test
Re-assesses the entire original scope, including all systems and all vulnerability types.
More thorough and time-consuming, thus more expensive.
Can identify new vulnerabilities that were not present during the initial test.
Suitable for clients who want a comprehensive validation of their security posture.
Provides a higher level of assurance that the environment is secure.
Mistake
Re-testing is always included in a penetration test.
Correct
Re-testing is typically an optional add-on service. The initial penetration test engagement often does not include re-testing; it is a separate phase that may require a new contract or change order.
Mistake
If an exploit fails during re-test, the vulnerability is definitely fixed.
Correct
The exploit may fail due to a security control like a WAF or IPS blocking the payload, while the underlying vulnerability remains unpatched. The tester should verify the patch level or attempt to bypass the control.
Mistake
Automated vulnerability scanning is sufficient for re-testing all vulnerabilities.
Correct
Automated scanning can miss vulnerabilities due to signature changes or false negatives. Manual verification, especially for critical vulnerabilities, is required to confirm remediation.
Mistake
Re-testing should be performed immediately after the client reports remediation.
Correct
The tester should allow a reasonable remediation window (e.g., 30 days) and then schedule the re-test. Immediate re-testing may not give the client enough time to implement proper fixes.
Mistake
A focused re-test only checks the specific vulnerabilities reported.
Correct
A focused re-test checks the specific systems and vulnerabilities from the original report, but it may also include a quick check for new vulnerabilities on those systems. However, it does not re-assess the entire scope.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
A focused re-test only examines the specific systems and vulnerabilities that were identified in the original penetration test report. It is quicker and less expensive but may miss new vulnerabilities introduced elsewhere. A full re-test reassesses the entire original scope, including all systems and vulnerability types, providing a more comprehensive validation but at a higher cost and time commitment.
Re-testing should be performed after the client has had a reasonable remediation window, typically 30 days for high-risk vulnerabilities, 60 days for medium-risk, and 90 days for low-risk. The re-test should occur soon after the client reports that remediation is complete, ideally within 30 days, to ensure the environment hasn't changed significantly.
A re-test report should include evidence for each vulnerability's current status. This may include screenshots of scanner output showing the vulnerability is no longer present, logs of failed exploit attempts, output from manual verification steps (e.g., command output), and any new findings discovered during re-testing. The evidence must be clear and verifiable.
Partially Remediated means that some action was taken to reduce the risk of the vulnerability, but the vulnerability still exists or is not fully resolved. For example, a WAF was added to block exploits, but the underlying vulnerable software was not patched. The vulnerability is still present but harder to exploit.
Yes, many compliance standards require validation that vulnerabilities have been remediated. For example, PCI DSS requires that a follow-up scan be performed after remediation to confirm that vulnerabilities are resolved. However, this is often a separate scan, not necessarily a full penetration test re-test.
Yes, it is common for the same tester to perform the re-test because they are already familiar with the environment and the vulnerabilities. However, some organizations prefer a different tester to provide an independent perspective. The PT0-002 exam does not mandate either approach.
The tester should document the new vulnerabilities as 'New Finding' in the re-test report and include them in the results. The client should be informed, and these new findings may require a separate remediation and re-test cycle.
You've just covered Re-Testing and Validation Testing — now see how well it sticks with free PT0-002 practice questions. Full explanations included, no account needed.
Done with this chapter?