Storage protocols and virtualisation solve the problem of making data move quickly and flexibly between servers and storage devices over a network. For the 350-601 exam, you must understand how different protocols like iSCSI and NVMe over Fabrics achieve this, and how virtualisation abstracts physical storage so that it can be managed like a single pool of resources. These concepts are the backbone of modern data centre storage architecture.
Jump to a section
A simple way to picture Storage Protocols and Virtualization
A busy city's central mail sorting facility. Workers sort envelopes and parcels from countless senders, then load them onto trucks, planes, and trains for delivery.
In the old days of computer storage, data was treated like a single, massive block of stone that you had to carve messages into and carry to another location. This was similar to how early networks handled storage using a protocol called Fibre Channel, which required dedicated roads (special cables and switches) and spoke its own private language that only special storage devices understood. It worked, but was expensive and inflexible.
Now consider a modern postal system. You write a message, put it in an envelope, and drop it in any mailbox. The postal service doesn't care if your letter goes by truck, plane, or train – it just uses whatever is available. This is exactly how iSCSI works. It takes your storage commands (read this file, write that data) and wraps them up like letters in an envelope using the standard network protocol (TCP/IP – the same language the internet speaks). The iSCSI storage data then travels over the exact same cables and switches that your email and web browsing use. No special roads required.
NVMe over Fabrics is like upgrading from regular mail to express courier. It uses a much more efficient addressing system that puts multiple commands on the same trip, rather than making one trip per task. The fabric part is the map of all possible routes across the city (the network), allowing the courier to take the fastest path not just a single predetermined road.
Storage virtualisation is like having a central package receiving depot for an entire apartment block. Instead of each apartment having its own separate mail slot that could fill up or get jammed, all packages go to the depot. The depot manager can then reorganise the incoming parcels, split large packages into smaller deliveries, and store extra packages in a communal holding area. To the outside world, the depot appears as one gigantic address that never fills up, even though behind the scenes it might be using ten different lockers in the basement.
Storage protocols are the rules and languages that computers use to talk to storage devices across a network. Imagine you have a hard drive inside your laptop. Your laptop's operating system uses a protocol called SATA (Serial ATA) to send commands directly to that drive over a short cable. Now imagine that hard drive is in another building, or another city. You cannot use a SATA cable that long – you need a protocol designed for sending storage commands over a network. This is where iSCSI and NVMe over Fabrics come in.
iSCSI stands for Internet Small Computer System Interface. The SCSI part is a very old set of commands for talking to storage devices – think of it as a universal language for saying 'write this data to sector 5' or 'read 64 kilobytes starting at block 1024'. The 'Internet' part means these commands are wrapped up inside standard TCP/IP packets – the same kind of packets used for loading a webpage or sending an email. A server that wants to use storage located elsewhere runs an iSCSI initiator (the client software). The storage device runs an iSCSI target (the server software). The initiator connects to the target over a standard Ethernet network, and from the server's perspective, the remote storage looks and behaves exactly like a local hard drive attached to its own motherboard.
The key characteristics of iSCSI are:
It uses standard Ethernet cables and switches – no special storage-only hardware required.
It typically runs over TCP/IP, which provides reliable, ordered delivery of data – but this adds some overhead (extra processing) compared to faster protocols.
It can be slow and inefficient if the network is congested, because every storage command requires a separate network transaction.
NVMe over Fabrics (NVMe-oF) solves the performance problem. NVMe stands for Non-Volatile Memory Express, a protocol originally designed for super-fast solid-state drives (SSDs) that are directly attached to a server's motherboard. NVMe is much more efficient than SCSI because it uses multiple queues to handle thousands of commands at the same time, rather than one queue that processes commands one after another.
NVMe over Fabrics extends the same NVMe commands over a network fabric. A fabric is just a fancy word for a network that connects many devices together. Instead of wrapping commands in TCP/IP (like iSCSI does), NVMe-oF can use a more efficient transport such as RDMA (Remote Direct Memory Access) – a technology that lets one computer read data directly from another computer's memory without involving the remote computer's CPU. This drastically reduces latency (delay) and CPU overhead.
The two most common transports for NVMe-oF are:
Fibre Channel (FC-NVMe) – using dedicated high-speed fibre optic cables and switches designed specifically for storage traffic.
RDMA over Converged Ethernet (RoCE) – using standard Ethernet cables but with special network cards that support RDMA.
Storage virtualisation decouples the logical view of storage from the physical hardware. Instead of a server directly accessing a specific hard drive, it talks to a virtual storage pool. A storage virtualisation appliance or software sits between the servers and the physical drives. This layer presents a single, large, virtual hard drive to all servers, while behind the scenes it can distribute the data across many different physical drives (some fast SSDs, some slower hard disks), from different vendors, in different locations.
The benefits of storage virtualisation include:
Simplified management – administrators can create, resize, or move storage volumes without touching physical hardware.
Better utilisation – physical drives that were previously half-empty (because one server could not use another server's unused space) can now be pooled together and shared.
Easier migration – data can be moved from old drives to new ones while servers keep working, without any downtime.
On the 350-601 exam, you need to know the differences between block-level storage (where raw storage volumes are presented to servers, as in iSCSI and NVMe-oF) and file-level storage (where the storage device manages a file system, as in NAS). You also need to understand that storage virtualisation is often delivered via a storage controller or a virtual SAN (Storage Area Network) appliance.
Server Initiates Storage Request
A server's operating system requests a data read or write from a storage volume. This request can either be a SCSI command (for iSCSI) or an NVMe command (for NVMe over Fabrics). The operating system does not know whether the storage is local or remote – it just issues the command.
Protocol Encapsulation
For iSCSI, the SCSI command is wrapped in an iSCSI Protocol Data Unit (PDU), which is then encapsulated inside a TCP segment, then an IP packet, and finally an Ethernet frame. For NVMe over Fabrics, the NVMe command is encapsulated in a Fabric frame (either Fibre Channel frame or RDMA packet), which is then placed on the physical network.
Network Transport
The encapsulated command travels across the network. For iSCSI, it passes through standard Ethernet switches where each switch reads the IP destination address and forwards the packet. For NVMe-oF over RoCE, the switch must support RDMA features like priority flow control (PFC) to avoid packet loss that would destroy performance.
Storage Target Reception and Processing
The storage target (a storage array or a virtualisation appliance) receives the network packet, strips off the network headers to expose the original SCSI or NVMe command, and then processes the command against its physical storage drives. If the request is a read, it retrieves the data from the drive; if it is a write, it stores the data and may copy it to multiple drives for redundancy.
Response and Data Return
The storage target sends a response back to the initiating server. For a read request, the response includes the requested data. This response follows the same path in reverse: encapsulation, network transport, and decapsulation at the server. The server's operating system receives the data as if it came from a local drive.
Virtualisation Layer Handling (if applicable)
When storage virtualisation is in place, the target is not a physical drive but a virtualisation appliance. The appliance receives the command, then performs its own mapping to determine which physical drive or drives hold the data. It may also apply compression, deduplication, or snapshots at this point before reading or writing to the physical media.
Consider a medium-sized company called DataMovers Inc. that runs an online retail platform. They have 100 virtual servers running in their data centre, all needing fast access to a central database that holds customer orders, inventory, and payment records.
The old setup was a nightmare. Each virtual server was configured to use a specific hard drive in a specific storage array. When a drive failed, that server's database would crash, and the IT team had to manually restore from backup. When they bought new storage, they had to reconfigure every server to see the new drives. The storage administrator, Priya, spent most of her time physically cabling and zoning (configuring access permissions) rather than solving interesting problems.
Priya decides to implement storage virtualisation using a software-defined storage (SDS) platform. Here is what happens step by step:
She installs the SDS software on three standard x86 servers, each with a mix of SSDs and traditional hard drives. These three servers together form a cluster.
The SDS software aggregates all the physical drives from all three servers into a single virtual storage pool. The pool appears to the network as one massive repository with 50 terabytes of capacity.
Priya creates virtual storage volumes from this pool. Each volume is a logical chunk of storage that looks like a local hard drive to any server that connects to it. She creates one 500 GB volume for the database and several smaller volumes for application logs and backups.
She configures iSCSI targets on the SDS cluster to expose each volume across the network. The virtual servers that host the database are configured with iSCSI initiators. They connect to the SDS cluster over the company's existing 10 Gigabit Ethernet network.
When a virtual server wants to write a customer order, it sends an iSCSI command over the network. The SDS cluster receives the command and immediately writes the data to two different physical drives on two different servers – this mirroring ensures that if one drive fails, no data is lost.
Six months later, the database has grown. Priya logs into the SDS management dashboard, selects the database volume, and expands it from 500 GB to 750 GB with a slider. The change takes effect instantly. The virtual server sees the extra space without needing to reboot or be reconfigured.
When a physical SSD in one of the SDS servers starts showing early signs of failure, Priya triggers a data migration. The SDS software automatically moves the data from the failing drive to a spare drive in another server, while all virtual servers continue serving customers without any interruption.
The key tools and actions Priya uses:
Software-defined storage platform (e.g., VMware vSAN, NetApp ONTAP, or open-source Ceph).
iSCSI initiator configuration on the virtual servers (usually in the operating system or hypervisor).
Zoning or network segmentation to ensure only authorised servers can access the iSCSI targets.
Monitoring dashboards to track capacity, performance latency, and drive health.
For a more performance-critical workload like a financial trading platform, Priya would replace iSCSI with NVMe over Fabrics using RDMA over Converged Ethernet (RoCE). The setup is similar, but the storage devices would be all-flash NVMe SSDs, and the network cards would support RDMA. The servers would use NVMe-oF initiators instead of iSCSI ones, and the latency would drop from several milliseconds to under 100 microseconds.
The 350-601 exam tests storage protocols and virtualisation in the context of Cisco Data Center technologies. You must understand not just the theory, but how these concepts are implemented in Cisco's products and architectures, particularly the Cisco MDS switches and the Nexus series of switches.
The exam loves to test the following topics with specific, pick-the-best-answer questions:
Difference between iSCSI and NVMe over Fabrics in terms of performance overhead and protocol stack.
Benefits of storage virtualisation, especially relating to storage utilisation and ease of management.
Understanding the role of a storage controller in virtualisation.
The concept of a virtual SAN (VSAN) in Fibre Channel environments – not the same as storage virtualisation.
The two main transport options for NVMe-oF: Fibre Channel and RoCE.
Common exam traps:
The exam often presents a scenario where storage performance is critical and asks which protocol to use. Many candidates incorrectly choose iSCSI because it is simpler. The correct answer for high performance is always NVMe over Fabrics (often FC-NVMe or RoCE).
Another trap is confusing storage virtualisation (pooling physical storage) with a virtual SAN (VSAN, which is about logically separating traffic on a Fibre Channel network). They are different concepts.
The exam may list several benefits and ask which one is NOT a benefit of storage virtualisation. Watch out for answers like 'removes the need for backup' – virtualisation does not eliminate the need for backups, it just makes them easier.
Key definitions to memorise for the exam:
iSCSI: SCSI commands over TCP/IP networks.
NVMe-oF: NVMe commands over a fabric network (FC or Ethernet with RDMA).
RDMA: Remote Direct Memory Access – allows direct data transfer between computers without involving the remote CPU.
Storage Virtualisation: Logical abstraction of physical storage resources into a single pool.
Block-level storage: Raw storage volumes presented to servers, no file system managed by the storage device.
The question types you will see include:
Multiple choice (single best answer) – e.g., 'Which storage protocol uses TCP/IP to transport SCSI commands?'
Multiple choice (select two) – e.g., 'Which two are valid transport options for NVMe over Fabrics?'
Drag and drop – match protocols to their characteristics.
Simulation – given a network diagram and a high-level goal, you choose the correct protocol and configuration.
The exam focus is on choosing the right protocol for a given workload. If you see 'low latency', 'high IOPS', 'flash storage', pick NVMe-oF. If you see 'existing Ethernet infrastructure', 'cost-sensitive', 'compatibility with legacy systems', pick iSCSI. Always justify your answer based on the scenario description.
iSCSI encapsulates SCSI storage commands inside standard TCP/IP packets, allowing storage traffic to run over existing Ethernet networks without special hardware.
NVMe over Fabrics uses the NVMe command set designed for flash storage and can utilise RDMA to bypass the remote server's CPU, dramatically reducing latency.
Storage virtualisation abstracts physical storage resources into a single logical pool, enabling features like thin provisioning, snapshots, and non-disruptive data migration.
The two primary transport options for NVMe over Fabrics are Fibre Channel (FC-NVMe) and RDMA over Converged Ethernet (RoCE).
Block-level storage (used by iSCSI and NVMe-oF) presents raw devices to servers, giving the server full control over the file system.
Storage virtualisation does not eliminate the need for backups or failure management – it provides tools to make those tasks easier and less disruptive.
On the 350-601 exam, choose iSCSI for low-cost compatibility with existing Ethernet, and NVMe over Fabrics for high-performance flash storage workloads.
A vSAN (virtual SAN) is a specific implementation of storage virtualisation that pools local drives from multiple servers into a shared datastore, often used in hyper-converged infrastructure.
RDMA allows a computer to read data directly from another computer's memory without involving the remote CPU, reducing latency and CPU overhead.
Fibre Channel is a dedicated storage network technology that can carry both SCSI (FCP) and NVMe (FC-NVMe) commands.
These come up on the exam all the time. Here's how to tell them apart.
iSCSI
Uses SCSI command set originally designed for hard disk drives (HDDs).
Encapsulates commands in TCP/IP, adding CPU overhead and higher latency.
Supports standard Ethernet hardware without special RDMA capabilities.
NVMe over Fabrics
Uses NVMe command set optimised for solid-state drives (SSDs) with multiple queues.
Can use RDMA to bypass remote CPU, reducing latency to under 100 microseconds.
Requires RDMA-capable network adapters (RoCE, iWARP, or Fibre Channel).
Storage Virtualisation
A logical layer that pools physical storage resources from multiple sources.
Provides features like snapshots, thin provisioning, and data migration.
Can work with any storage backend: local drives, SAN, NAS, or cloud.
Storage Area Network (SAN)
A dedicated network that connects servers to storage devices (typically Fibre Channel or iSCSI).
Provides block-level access to storage volumes without abstraction.
Requires specialised switches and configuration (e.g., zoning, VSANs).
Fibre Channel Protocol (FCP)
Carries SCSI commands over Fibre Channel networks using FCP.
Typically used with spinning disk drives or mixed workloads.
Older standard, well-established with many legacy devices.
Fibre Channel NVMe (FC-NVMe)
Carries NVMe commands over Fibre Channel networks.
Optimised for all-flash arrays and low-latency workloads.
Newer standard, requires compatible FC switches and HBAs (Host Bus Adapters).
Mistake
iSCSI and NVMe over Fabrics are basically the same thing, just different speed versions.
Correct
iSCSI sends SCSI commands over TCP/IP, which has significant overhead due to protocol processing at both ends. NVMe over Fabrics sends NVMe commands over a fabric and can use RDMA to bypass the remote CPU, resulting in much lower latency and higher throughput.
Both protocols allow storage access over a network, so beginners assume they are just different speeds of the same thing. The underlying command sets and transport mechanisms are fundamentally different.
Mistake
ESXi (vSphere) does not support NVMe over Fabrics because it is too new.
Correct
VMware vSphere 6.5 and later fully support NVMe over Fabrics, both over Fibre Channel (FC-NVMe) and RDMA over Converged Ethernet (RoCE).
Many people think VMware only supports iSCSI and Fibre Channel because those are older and more documented. NVMe-oF support has been standard for years now.
Mistake
Storage virtualisation is the same thing as having a storage area network (SAN).
Correct
A SAN is a dedicated network that connects servers to storage devices. Storage virtualisation is a technology that can run on top of a SAN (or any storage) that pools physical storage and presents it logically. You can have a SAN without virtualisation, and you can have virtualisation without a SAN (e.g., using local drives in a cluster).
Both terms involve storage and networking, so beginners mentally group them together. They solve different problems.
Mistake
RDMA only works with Fibre Channel networks.
Correct
RDMA is a technology that can run over multiple network types, including Fibre Channel (as FC-NVMe), InfiniBand, and Ethernet (as RoCE or iWARP). RoCE is very common in data centres using standard Ethernet.
People hear 'RDMA' and associate it with 'special fibre channel hardware' because of the historical dominance of Fibre Channel in storage. Modern Ethernet can also support RDMA.
Mistake
When you use storage virtualisation, you no longer need to worry about physical drive failures.
Correct
Storage virtualisation provides features like replication, snapshots, and migration that make it easier to handle drive failures, but the physical drives still fail and can cause data loss if not managed correctly. You still need RAID or erasure coding and backups.
The abstraction layer gives a false sense of invincibility. Beginners think the virtual pool automatically fixes everything.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
iSCSI stands for Internet Small Computer System Interface. The main difference is that iSCSI runs over standard Ethernet (TCP/IP), while Fibre Channel requires dedicated fibre optic cables and switches. iSCSI is cheaper and easier to manage; Fibre Channel offers higher reliability and performance for critical storage.
It depends on the transport. For Fibre Channel NVMe, you need Fibre Channel cables and switches. For NVMe over Fabrics using RoCE (RDMA over Converged Ethernet), you need standard Ethernet cables and switches that support RDMA features (like priority flow control). For iWARP, standard Ethernet is used as well.
Yes, you can run both protocols on the same physical network. However, you need to be careful about network congestion. iSCSI can cause TCP packet loss, which may disrupt RDMA traffic. Many administrators use separate VLANs or dedicated ports for each protocol to ensure quality of service.
RAID (Redundant Array of Independent Disks) is a technology that combines multiple physical drives into one logical unit for redundancy or performance. Storage virtualisation is a broader concept that pools storage from many different sources (local drives, SAN, NAS) and presents it as a single resource. RAID can be a part of a virtualised storage system, but they are different layers.
Yes, vSphere 6.5 and later support NVMe over Fabrics. You can configure NVMe over Fibre Channel (FC-NVMe) or NVMe over RDMA (RoCE v2). Check VMware's Hardware Compatibility List for supported adapters.
You would choose iSCSI if you have a limited budget, existing Ethernet infrastructure, and your workload does not require extremely low latency (e.g., backups, file shares). iSCSI is also easier to set up and troubleshoot. NVMe over Fabrics is only beneficial when you have flash storage and need high IOPS and low latency.
You've finished Storage Protocols and Virtualization. Continue through the 350-601 study guide to build a complete picture of the exam.
Done with this chapter?