Information Systems Acquisition, Development, and Implementation practice questions

Impact analysis

Why it fails: Impact analysis is important but not always required for emergency; often done post-change.

Pre-approval from CAB

Why it fails: Emergency changes may be approved post-implementation or by a subset.

The report includes a description of the vendor's business continuity plan

Why it fails: Not required for reliability.

The report includes a list of the vendor's customers

Why it fails: Not relevant to reliability.

The report contains the vendor's internal control objectives

Why it fails: SOC 2 reports always include control objectives; this is not a reliability check.

Greater control over customization

Why it fails: COTS packages often limit customization to maintain vendor support.

Lower total cost of ownership

Why it fails: TCO can be higher due to licensing, maintenance, and integration costs.

Complete alignment with business processes

Why it fails: COTS may require process changes to fit the software.

Risk mitigation plans are not documented

Why it fails: While important, the identification of risks is foundational.

The project uses a combination of waterfall and prototyping

Why it fails: This is typical of the spiral model.

Risk analysis is performed by the project manager alone

Why it fails: While not best practice, it is less critical than failing to reassess risks.

Availability of skilled developers to maintain the custom solution

Why it fails: This is relevant for custom development but not the most important when evaluating COTS.

User training requirements for the new system

Why it fails: Training is manageable and not a primary decision factor compared to long-term vendor dependence.

Total cost of ownership including maintenance and licensing

Why it fails: While important, cost is not the most critical risk compared to vendor dependency, which can have broader strategic implications.

Perform a penetration test after go-live

Why it fails: Penetration testing is a point-in-time assessment and cannot fix requirements gaps.

Conduct a vulnerability scan after deployment to identify security gaps

Why it fails: Vulnerability scanning is reactive and may not address design flaws that stem from missing requirements.

Implement a firewall and intrusion detection system

Why it fails: Network controls are important but do not compensate for lack of security requirements in the system design.

The change request included a rollback plan

Why it fails: A rollback plan is important but not the most critical post-implementation control for governance.

The change was tested in a production-like environment before deployment

Why it fails: Testing is important, but due to the emergency nature, some testing may be limited; retrospective review is key.

The change was documented and approved by the change manager after implementation

Why it fails: Documentation is necessary but CAB review provides broader oversight.

Time-and-materials contract

Why it fails: This type allows flexibility as the buyer pays for actual effort, reducing risk of scope disputes.

Cost-plus contract

Why it fails: Similar to time-and-materials, cost-plus reimburses costs plus fee, which is less risky for undefined requirements.

Outcome-based contract

Why it fails: Outcome-based contracts tie payment to results, which can be risky if metrics are not clear, but generally align incentives.

Require reconfiguration of SoD controls before the next audit

Why it fails: While reconfiguration is ideal, immediate compensating controls are needed to mitigate current risk.

Document the issue and accept the risk

Why it fails: Accepting risk may be an option but the auditor should first recommend controls to reduce risk to an acceptable level.

Advise management to terminate the project manager

Why it fails: Personnel actions are not within the auditor's scope; the focus should be on risk mitigation.

The vendor's marketing plan for the solution

Why it fails: Marketing is irrelevant to service delivery.

The vendor's employee training program details

Why it fails: Training details are not typically in an SLA; they may be in a separate security appendix.

The vendor's disaster recovery testing schedule

Why it fails: DR testing schedule may be in a separate document, not usually in SLA.

Daily standup meetings

Why it fails: Standups are for coordination, not control.

Retrospective

Why it fails: Retrospectives focus on process improvement, not acceptance.

Product backlog

Why it fails: The backlog is a prioritization tool, not a control.

Security awareness training

Why it fails: Training is preventive.

Threat modeling

Why it fails: Threat modeling is preventive.

Code review

Why it fails: Code review is a preventive control.

Burndown charts

Why it fails: Burndown charts track progress but are not controls.

Daily standup

Why it fails: Standups are coordination meetings, not controls.

A post-implementation review scheduled six months later

Why it fails: Post-implementation review is important but not specifically required for emergency change documentation.

A formal change request approved by the Change Advisory Board (CAB) before implementation

Why it fails: Emergency changes are fast-tracked; CAB approval often occurs after.

A complete test plan executed before deployment

Why it fails: Emergency changes may not allow full testing.

The IT manager may not have sufficient technical expertise to approve emergency changes.

Why it fails: The process already requires CAB ratification, so expertise gaps are mitigated; the greater risk is that the same person authorising emergency changes may also implement them, removing segregation of duties. It is tempting because technical competence is a genuine approval consideration in change management.

Emergency changes may be delayed while waiting for CAB ratification.

Why it fails: Emergency changes are already implemented before ratification, so delay in CAB ratification does not prevent or mitigate the change itself; the exposure arises from unauthorised or unreviewed implementation. The option is tempting because CAB ratification backlogs genuinely slow routine changes, making this the correct concern where emergency changes await prior approval rather than post-hoc review.

The CAB may not have enough time to review emergency changes properly.

Why it fails: CAB ratification occurs after emergency changes are already deployed, so insufficient review time does not expose the application to unauthorised or untested changes during the emergency window. It is tempting because review quality is a recognised change management control concern.

Formal phase-gate approvals between design and development

Why it fails: Phase-gate approvals are typical of waterfall, not agile.

Detailed requirements documentation approved before coding

Why it fails: Agile favors working software over comprehensive documentation; requirements evolve.

Pair programming for all critical code

Why it fails: Pair programming is a technique but not a mandatory control in all agile projects.

The data migration plan includes a rollback strategy in case of failure.

Why it fails: A data migration plan with rollback is essential, but it is developed during later project phases. At the business case and feasibility stage, the auditor's primary concern is whether the project is justified and feasible, not the detailed execution plans, which come after approval.

The SaaS contract includes a service level agreement (SLA) with uptime guarantees.

Why it fails: While SLAs are important, at this early stage the primary concern is whether the business case accurately reflects the total cost of ownership and alignment with strategic objectives. An SLA is a later contractual detail and does not address the fundamental justification and feasibility of the acquisition.

The project team has selected a vendor based on a demo that met all functional requirements.

Why it fails: Vendor selection should follow a structured evaluation, not just a demo. However, at this stage, the primary concern is the business case and feasibility, not the vendor selection process, which typically occurs later. The auditor would focus on whether the justification is sound before proceeding to procurement.

The parallel run may cause confusion among employees regarding which system is authoritative.

Why it fails: While confusion can occur, it is typically mitigated through clear communication and procedures. The more significant risk is the potential for data discrepancies between the two systems, which could lead to incorrect payments and compliance issues if not reconciled properly.

The legacy system may not be able to handle the additional processing load during the parallel run.

Why it fails: Legacy systems are often robust, and if they can handle normal operations, they can usually handle a parallel run. The critical risk is not processing capacity but the accuracy of data comparison and reconciliation between the two systems.

The cost of running two systems simultaneously may exceed the project budget.

Why it fails: Cost overrun is a concern, but it is a project management issue rather than a control risk. The primary risk in a parallel conversion is the integrity of the data and the reconciliation process, which directly affects payroll accuracy and employee trust.

Code changes are peer-reviewed before being merged into the main branch.

Why it fails: Peer review is a valuable control, but in a CI/CD environment, it can be bypassed or rushed. It is not sufficient on its own to guarantee integrity; additional controls like segregation of duties and automated gates are needed.

Automated unit tests are run as part of the build pipeline.

Why it fails: Automated unit tests are important for quality but do not ensure the integrity of code changes from a security and authorization perspective. They help catch bugs but do not prevent unauthorized or malicious code from being deployed if other controls are missing.

The deployment process includes a rollback mechanism in case of failure.

Why it fails: A rollback mechanism is important for availability and recovery, but it does not ensure the integrity of code changes. It addresses the aftermath of a failed deployment, not the prevention of unauthorized or flawed changes from being deployed in the first place.

Regression testing

Why it fails: Regression testing is important when changes are made to ensure existing functionality still works. However, in this scenario, the project is in the final testing phase before production, and regression testing may have been performed after system testing. While it can be beneficial, it is not as critical as UAT and performance testing for initial go-live readiness.

Disaster recovery testing

Why it fails: Disaster recovery testing validates the ability to recover from a major disruption and is typically conducted as part of business continuity management, not system development testing. While important, it is not a standard pre-production test for a new system. The focus should be on UAT and performance testing to ensure the system meets business and operational requirements.

Code review

Why it fails: Code review is typically conducted during development, before unit testing, to identify defects early. While valuable, it is not an additional test after unit and system testing. The focus at this stage should be on validating the integrated system against business and performance requirements, not on reviewing individual code modules.