Security Assessment and Testing practice questions

Penetration test

Why wrong: A penetration test is a proactive security assessment designed to identify exploitable vulnerabilities in systems, applications, or networks by simulating real-world attacks. While crucial for technical risk identification and validating defensive measures, it focuses on specific technical weaknesses rather than a holistic evaluation of an organization's entire control framework or its Information Security Management System (ISMS). Therefore, it is not considered an audit and cannot fulfill the requirements for a comprehensive control assessment or certification against standards like ISO 27001.

SOC 2 Type II audit

Why wrong: A SOC 2 Type II audit specifically assesses a service organization's controls relevant to security, availability, processing integrity, confidentiality, or privacy, based on the AICPA's Trust Services Criteria. This report is primarily intended for user entities and their auditors to understand the controls at the service organization. While valuable for demonstrating control effectiveness, it operates under a different framework and set of criteria than ISO 27001, making it unsuitable for directly assessing compliance with or achieving certification for the ISO 27001 standard.

Internal audit

Why wrong: An internal audit is a systematic, independent evaluation performed by an organization's own personnel or designated internal resources to assess the effectiveness of its governance, risk management, and control processes. While vital for continuous improvement, identifying non-conformities, and preparing for external assessments, it lacks the requisite independence and accreditation required for formal external certification against standards such as ISO 27001. Its primary role is to provide assurance to management and the board, not external stakeholders for certification purposes.

Vulnerability assessment

Why wrong: A vulnerability assessment systematically identifies, quantifies, and prioritizes security weaknesses or flaws within systems, applications, or networks. Its primary focus is on discovering potential technical vulnerabilities that could be exploited, rather than evaluating the adherence to security policies or the effectiveness of existing controls against a predefined standard. This process typically involves scanning tools and manual checks to catalog weaknesses without attempting to exploit them.

Penetration test

Why wrong: A penetration test, or pen test, is an authorized simulated cyberattack on a computer system, network, or web application to evaluate its security. Unlike an audit, its objective is to actively exploit identified vulnerabilities to determine if unauthorized access can be gained and what data could be compromised, thereby demonstrating real-world attack vectors. This method focuses on the practical exploitability of flaws, not merely their existence or control compliance.

Security review

Why wrong: A security review involves a less formal examination of an organization's security posture, often conducted internally to assess documentation, configurations, or processes against best practices or internal guidelines. While it aims to identify areas for improvement, it typically lacks the independence, rigorous methodology, and objective evidence collection required for a formal audit or the active exploitation of a penetration test. It serves as a preliminary check rather than a definitive compliance evaluation.

Physical security assessment

Why wrong: A physical security assessment focuses on evaluating the tangible controls protecting an organization's assets, such as data centers, servers, and network infrastructure. This includes examining access controls, surveillance systems, environmental safeguards, and personnel security. While crucial for overall organizational security, it does not directly analyze the source code or runtime behavior of a custom web application for software vulnerabilities.

Social engineering

Why wrong: Social engineering involves manipulating individuals into performing actions or divulging confidential information, often through psychological manipulation. Techniques like phishing, pretexting, or baiting exploit human trust and susceptibility rather than technical vulnerabilities within software. Therefore, it is not a method for directly assessing the security of a custom web application's code or its underlying architecture.

Network penetration testing

Why wrong: Network penetration testing involves simulating real-world attacks against an organization's network infrastructure to identify vulnerabilities in network devices, operating systems, and configurations. This type of assessment evaluates perimeter defenses, internal network segmentation, and system hardening. While critical for overall security, it primarily focuses on the underlying network and host layers, not the specific application logic or code flaws within a custom web application.

NVD

Why wrong: NVD is a database that uses CVSS.

CVE

Why wrong: CVE is an identifier, not a scoring system.

OWASP

Why wrong: OWASP is a community, not a scoring system.

Scan once per year to minimize operational impact

Why wrong: Scanning only once per year leaves an organization exposed to newly discovered vulnerabilities and configuration drift for an unacceptably long period. In dynamic IT environments, new assets are deployed, configurations change, and zero-day exploits emerge frequently, rendering annual scans largely ineffective and reactive. This infrequent approach fails to provide timely visibility into the evolving threat landscape, significantly increasing the risk of compromise.

Scan only when vulnerabilities are publicly disclosed

Why wrong: Waiting for public disclosure of vulnerabilities before initiating scans is a fundamentally reactive strategy that leaves systems exposed during the critical period between vulnerability existence and public awareness. This approach fails to identify internally introduced misconfigurations or unknown vulnerabilities, relying solely on external intelligence. Such a delay provides attackers with a significant window of opportunity to exploit unpatched systems before an organization even begins to assess its exposure.

Use the same interval for all systems regardless of criticality

Why wrong: Applying a uniform scanning interval across all systems, irrespective of their criticality, data sensitivity, or exposure, is an inefficient and potentially risky strategy. Highly critical systems, such as those processing sensitive data or directly exposed to the internet, demand more frequent and thorough scrutiny due to their higher impact if compromised. A blanket approach wastes resources on less critical assets while neglecting the necessary attention for those posing the greatest risk.

Reviewing logs only after a security incident

Why wrong: Reviewing logs only after a security incident is a reactive approach that severely limits an organization's ability to detect and prevent threats proactively. This practice means potential breaches or system anomalies could go unnoticed for extended periods, increasing the impact and cost of an eventual incident response. Effective security requires continuous, periodic log analysis to identify suspicious patterns and indicators of compromise before they escalate.

Storing logs in plaintext without access controls

Why wrong: Storing logs in plaintext without robust access controls creates a critical security vulnerability, compromising the confidentiality and integrity of vital security information. Unauthorized individuals could easily read sensitive event data, modify logs to cover their tracks, or delete them entirely, hindering incident detection and forensic analysis. Essential log data must be protected with encryption at rest and in transit, alongside strict role-based access controls.

Security review

Why wrong: A security review involves a high-level, often proactive, examination of security documentation, system designs, configurations, or policies to identify potential weaknesses or non-compliance. This process typically does not involve active scanning or exploitation of live systems but rather a critical analysis of existing security artifacts. It aims to uncover architectural flaws or policy gaps that could lead to vulnerabilities, often performed early in the development lifecycle or as part of a governance process.

Security audit

Why wrong: A security audit is a formal, systematic evaluation comparing an organization's security controls and practices against established policies, industry standards, or regulatory requirements. Its objective is to determine compliance, identify control deficiencies, and assess the effectiveness of implemented security measures. This process often involves reviewing documentation, interviewing personnel, and examining evidence of control operation, rather than actively scanning for technical vulnerabilities.

Penetration test

Why wrong: A penetration test is an authorized, simulated cyberattack designed to actively exploit identified vulnerabilities in a system or network to determine the extent to which an attacker could gain unauthorized access or cause damage. Unlike a vulnerability assessment, it goes beyond identification by attempting to compromise systems, escalate privileges, and exfiltrate data. This process provides a realistic view of an organization's security posture and the potential business impact of a successful breach.

Reconnaissance

Why wrong: Reconnaissance focuses on passive or active information gathering before any exploitation occurs, such as scanning for open ports or enumerating services. The scenario describes a tester who has already obtained initial access and is now moving laterally, which falls under the post-exploitation or lateral movement phase. This option is tempting because reconnaissance is often confused with the entire testing process, and in a scenario where no access has been gained, it would be the correct initial phase.

Reporting

Why wrong: Reporting is the formal documentation phase of a penetration test, occurring after all technical activities, including post-exploitation and cleanup, have concluded. It involves compiling findings, identified vulnerabilities, associated risks, and actionable recommendations into a comprehensive report for the client. This phase does not involve active system interaction or movement within the target network, as the technical assessment is already complete.

Exploitation

Why wrong: Exploitation is the process of actively leveraging identified vulnerabilities to gain unauthorized access or control over a target system. This phase involves executing specific exploits against discovered weaknesses to establish an initial foothold, often resulting in remote code execution or shell access. Since the question states the tester "has obtained initial access," the exploitation phase has already been successfully completed, and the tester is now operating beyond that initial breach point.

IAST

Why wrong: IAST (Interactive Application Security Testing) is a hybrid analysis method that requires the application to be running and actively exercised, instrumenting it to observe internal behavior and data flow. While it offers deep insights by combining static and dynamic analysis techniques, it is not suitable for identifying vulnerabilities purely from static code analysis before any execution or comprehensive testing.

RASP

Why wrong: RASP (Runtime Application Self-Protection) functions as a security control embedded within the application itself, actively monitoring and blocking attacks in real-time during production. It is a defensive mechanism designed to protect against exploitation of vulnerabilities, rather than a testing tool designed to proactively discover vulnerabilities in the source code before deployment.

DAST

Why wrong: DAST (Dynamic Application Security Testing) scans a running application from the outside, simulating attacks to identify vulnerabilities visible at the application's external interfaces. While effective for finding runtime issues and configuration flaws, it does not analyze the underlying source code directly and requires a fully deployed, executable application to perform its tests.

Exploitation techniques to use

Why wrong: While specific exploitation techniques might be outlined within a detailed Rules of Engagement (RoE) document to clarify permissible actions, they are not considered a key component in the same vein as scope, authorization, or emergency procedures. The RoE primarily establishes the legal and ethical boundaries, objectives, and communication protocols for an engagement, rather than serving as a tactical playbook for specific attack methods. Defining the types of techniques allowed (e.g., social engineering, network scanning) is more aligned with scope than the granular details of how to execute them.

CVSS score of vulnerabilities

Why wrong: The Common Vulnerability Scoring System (CVSS) score of vulnerabilities is a metric used to quantify the severity of discovered weaknesses, typically generated after a vulnerability assessment or penetration test is conducted. It is a crucial element of the final reporting phase, informing remediation priorities, but it is not a component of the Rules of Engagement. RoE documents are established before testing begins to define the scope, authorization, and operational parameters, not to report on findings.

Number of vulnerabilities found

Why wrong: The number of vulnerabilities found is an outcome or metric derived from the execution of a security assessment, serving as a quantitative measure of the findings. This figure is a result of the testing process and is typically included in the final report to stakeholders. It cannot be a component of the Rules of Engagement, which are pre-defined agreements and guidelines established before the assessment commences to govern its conduct, scope, and authorized activities.

NVD

Why wrong: The National Vulnerability Database (NVD) is a U.S. government repository of standards-based vulnerability management data, including CVEs and their associated CVSS scores. While NVD aggregates and presents these scores, it is not the system that defines or calculates the standardized severity score itself. Instead, it serves as a comprehensive database for public consumption, integrating various vulnerability-related information from multiple sources.

CVE

Why wrong: Common Vulnerabilities and Exposures (CVE) provides a unique, standardized identifier for publicly known cybersecurity vulnerabilities. Its primary function is to enable consistent referencing and data sharing across different security tools and databases, facilitating communication about specific vulnerabilities. However, CVE IDs themselves do not contain any information regarding the severity, impact, or exploitability of a vulnerability; they are purely for identification.

CWE

Why wrong: Common Weakness Enumeration (CWE) is a community-developed list of common software weaknesses and vulnerability types, serving as a dictionary of potential flaws in software design, architecture, or implementation. Its purpose is to help developers and security professionals identify, mitigate, and prevent these weaknesses in code. Unlike CVSS, CWE focuses on categorizing the types of weaknesses rather than assigning a severity score to specific instances of discovered vulnerabilities.

Patch compliance percentage

Why wrong: Patch compliance percentage indicates the proportion of systems that have successfully applied all required security patches within a specified timeframe. While a critical indicator of general system hygiene and a component of vulnerability management, this metric primarily measures the breadth of patch application, not the speed of response specifically to newly discovered or critical vulnerabilities. It doesn't inherently reflect the time taken to identify, prioritize, and fully remediate a critical vulnerability beyond just applying a patch.

ROI of security controls

Why wrong: ROI of security controls quantifies the financial return on investment for security expenditures, comparing the cost of controls against the avoided losses or increased business value. While essential for budget justification and demonstrating the financial prudence of security initiatives, it does not directly measure the operational speed or efficiency with which an organization identifies and resolves vulnerabilities. This metric focuses on economic impact rather than the agility of the vulnerability response process.

Number of open vulnerabilities by severity

Why wrong: The number of open vulnerabilities by severity provides a snapshot of the current risk backlog and the overall volume of unaddressed security issues within an environment. While this metric is crucial for understanding the overall risk posture and resource allocation needs, it does not inherently measure the speed or effectiveness of the remediation process itself. It indicates the quantity of problems, not the velocity at which those problems are being resolved once identified.

SOC 2 Type II

Why wrong: SOC 2 Type II reports provide a detailed assessment of a service organization's controls over a specified period, typically 6-12 months, including the suitability of design and operating effectiveness. These comprehensive reports are intended for restricted use by user entities and their auditors to evaluate the impact of the service organization's controls on their own operations, making them unsuitable for public disclosure.

SOC 1

Why wrong: SOC 1 reports, governed by SSAE 18 (Statement on Standards for Attestation Engagements No. 18), focus exclusively on controls relevant to a service organization's financial reporting. These reports are primarily for user entities' financial statement auditors to assess the impact of the service organization's controls on the user entity's financial statements and are not designed for public consumption or a broader summary of security posture.

SOC 2 Type I

Why wrong: A SOC 2 Type I report evaluates the suitability of the design of a service organization's controls at a specific point in time, addressing one or more of the Trust Services Criteria (TSC). While it provides assurance regarding control design, it does not assess operating effectiveness and, like its Type II counterpart, is intended for restricted distribution to user entities and their auditors, not for public release.

Log review frequency

Why wrong: Log review frequency dictates how often security personnel or automated systems analyze collected logs for anomalies, security incidents, or policy violations. While crucial for proactive threat detection and operational security, it does not directly specify the mandated duration for which these log records must be stored. This metric focuses on the *act* of examination, not the *length* of storage, making it distinct from retention periods.

Log format standardization

Why wrong: Log format standardization ensures that security logs from various sources adhere to a consistent structure, schema, or syntax (e.g., CEF, LEEF, Syslog RFCs). This consistency significantly improves the efficiency of parsing, correlation, and automated analysis across different systems and tools. However, standardizing the format addresses the *interoperability and readability* of logs, not the regulatory or policy-driven duration for which they must be archived.

Centralized log management

Why wrong: Centralized log management involves consolidating security logs from multiple disparate sources into a single, secure repository or Security Information and Event Management (SIEM) system. This approach enhances efficiency in collection, storage, analysis, and incident response by providing a unified view of security events. While it facilitates the *management* of logs, including their storage, it does not inherently define or dictate the specific duration for which those logs must be retained according to organizational policy or external regulations.

External scan

Why wrong: An external scan is inherently limited as it simulates an attack from outside the network perimeter, typically without any credentials. While crucial for identifying internet-facing vulnerabilities and misconfigurations, it cannot access internal system configurations, patch levels, or vulnerabilities that are only exploitable from within the network or with authenticated access. Therefore, it provides only a perimeter-level view, not a comprehensive internal assessment.

Passive scan

Why wrong: Passive scanning involves monitoring network traffic and system behavior without actively sending probes or interacting with the target system. This method is excellent for detecting anomalies, exposed services, or misconfigurations that manifest in network communications, but it cannot actively discover hidden vulnerabilities, unpatched software, or internal configuration weaknesses that do not generate observable network traffic. Its non-intrusive nature means it lacks the depth of active, direct system inspection.

Unauthenticated scan

Why wrong: An unauthenticated scan simulates an attacker with no prior knowledge or access credentials, probing network services and applications from an external perspective. While effective for identifying publicly exposed vulnerabilities and misconfigurations visible from the network perimeter, it cannot delve into the internal workings of a system. This limitation means it will miss critical vulnerabilities residing within the operating system, applications, or databases that require authenticated access to detect, thus providing an incomplete security picture.

White box

Why wrong: White box testing provides the assessor with complete and comprehensive knowledge of the target environment, including full access to system architecture diagrams, network configurations, and often source code. This deep insight allows for thorough analysis of internal logic, security controls, and potential vulnerabilities that might be missed by external testing. While highly effective for detailed code review and internal security audits, it does not simulate an external attacker's perspective, which is often the primary objective for a general third-party assessment.

Grey box

Why wrong: Grey box testing involves the assessor having partial or limited knowledge of the target system, such as user-level credentials, network diagrams for specific segments, or access to internal documentation. This approach aims to simulate an attacker who has gained some initial foothold or an insider threat with restricted access. While it offers a balance between efficiency and realism by focusing on specific areas, it is not the most appropriate choice when the goal is to assess an organization's defenses from a completely unknown, external perspective, as implied by a general third-party assessment without specified knowledge.

Internal test

Why wrong: An internal test refers to an assessment conducted from within the organization's network perimeter, simulating an insider threat or an attacker who has successfully breached the external defenses. While the knowledge level for an internal test can vary (black, grey, or white box), the term primarily defines the *location* of the assessment, not the *information* provided to the tester. The question describes a third party performing an assessment without specifying location or initial knowledge, making 'black box' a more direct and common fit for evaluating external defenses from an unknown perspective.

Vulnerability scoring methodology

Why wrong: A vulnerability scoring methodology, such as CVSS, is primarily applied during the analysis and reporting phases of a penetration test, not during the initial planning or establishment of rules of engagement. While crucial for prioritizing findings and communicating risk post-test, the specific method for assigning severity scores does not dictate the scope, authorization, or boundaries of the testing activities themselves. The planning phase focuses on *what* will be tested and *how* it will be authorized, not *how* findings will be rated.

Previous test results

Why wrong: While previous penetration test results can offer valuable context for understanding an organization's historical security posture and identifying recurring issues, they are not a fundamental component of establishing the rules of engagement for a *new* test. The rules of engagement define the parameters, scope, and authorization for the *current* specific assessment. Previous results might inform the *objectives* or *scope* of a new test, but they do not constitute the rules themselves.

List of tools to be used

Why wrong: A detailed list of specific tools to be used, while important for the technical execution plan and potentially for client awareness, is generally not a core element of the high-level rules of engagement. The rules of engagement focus on the overarching agreement, scope, authorization, and communication protocols, rather than the granular technical implementation details. Specifying tools is typically part of the more detailed test plan or methodology document, which is developed *after* the rules of engagement are established.

Number of employees in IT security

Why wrong: Headcount describes program capacity or investment, not program effectiveness — a team of ten people running a mature, well-tuned vulnerability management process can outperform a team of thirty using ad hoc, unmeasured processes. The board is asking whether the program is working, and staffing levels say nothing about outcomes like remediation speed or residual risk.

Budget for security tools

Why wrong: Dollar spend on tooling reflects investment, not results — an organization can license expensive scanning and patching platforms and still leave critical vulnerabilities unremediated for months if the underlying process is broken. Effectiveness metrics need to measure what the program actually achieves, such as detection coverage, remediation speed, and residual exposure, not how much was spent trying to achieve it.

Monthly internal vulnerability scans

Why wrong: While internal vulnerability scans are a crucial PCI DSS requirement (Requirement 11.2.1), the standard mandates them at least quarterly, not monthly. Performing them monthly exceeds the minimum requirement but is not the specific frequency mandated for a PCI DSS assessment, making this option incorrect as a required preparation step if interpreted as the minimum frequency. The key is the quarterly cadence.

SOC 2 Type II report

Why wrong: A SOC 2 Type II report, while demonstrating a service organization's controls over security, availability, processing integrity, confidentiality, or privacy, is not a direct requirement for a company preparing for a PCI DSS assessment. PCI DSS has its own specific set of compliance requirements and validation procedures, which are distinct from the AICPA's Trust Services Criteria addressed by SOC 2 reports. One does not substitute for or directly fulfill the other's assessment criteria.

Annual penetration test of all systems

Why wrong: PCI DSS Requirement 11.3 mandates annual penetration testing, but specifically within the scope of the Cardholder Data Environment (CDE) and any systems that could impact its security. The phrase "all systems" is overly broad and inaccurate for PCI DSS compliance, as it would include systems entirely outside the CDE and its connected components, leading to unnecessary effort and misinterpreting the standard's scope. The focus must remain on the CDE and critical systems connected to it.

External scan

Why wrong: An external scan is conducted from outside the organization's network perimeter, simulating an attacker's perspective on internet-facing assets. While effective for identifying network-level vulnerabilities like open ports, misconfigured firewalls, or exposed services, it typically lacks the internal access or credentials required to thoroughly assess vulnerabilities within web applications or internal systems, which often require authenticated access to reveal deeper flaws.

Passive scan

Why wrong: A passive scan monitors network traffic and application responses without actively sending malicious payloads or probing for weaknesses. This method can identify certain vulnerabilities, such as insecure HTTP headers, exposed sensitive information in responses, or outdated server banners. However, it cannot uncover vulnerabilities that require active interaction, like SQL injection, cross-site scripting (XSS), or authentication bypasses, as it doesn't manipulate inputs or states.

Unauthenticated scan

Why wrong: An unauthenticated scan attempts to find vulnerabilities without providing any login credentials, simulating an attacker who has not yet gained access to the system. While effective for discovering publicly exposed weaknesses like default credentials, misconfigured public services, or vulnerabilities on login pages, it cannot access or test the vast majority of an application's functionality or data that resides behind an authentication barrier, leading to a significant blind spot for internal application flaws.