AWS Nitro System: Secure And Efficient Virtualization Technology
Learn more about virtualization technology that supports high-level security, performance and efficiency of EC2 instances.
The AWS Nitro System transforms EC2 virtualization with dedicated and optimized hardware and software. This setup is offloading many tasks to specialized components. Let’s explore the key parts and check how they work.
What is it?
AWS Nitro System is a collection of hardware and software components from AWS that are used for the virtualization of the EC2 servers. The Nitro System enables higher performance, availability and security than traditional virtualization.
What is virtualization?
Virtualization is a technology that you can use to create virtual representations of servers, storage, networks, and other physical machines. Virtual software mimics the functions of physical hardware to run multiple virtual machines simultaneously on a single physical machine - link.
![Traditional virtualization Traditional virtualization](https://substackcdn.com/image/fetch/w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe5459a49-ce8f-4e7a-948b-a3e72f8bc94f_631x367.png)
The Nitro System is the underlying platform for a vast majority of EC2 instances. If you’d like to check which instances support the Nitro system - check this link.
![Nitro System virtualization Nitro System virtualization](https://substackcdn.com/image/fetch/w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7004ef25-98b8-4263-b45a-3a8a15c06505_666x634.png)
It ensures the confidentiality, security and performance of the EC2 instances by moving most of the traditional virtualization functions to the dedicated hardware and firmware. This allows the core system to have more power for the actual workloads.
Three main parts of the Nitro are:
Nitro Cards
Nitro Security Chip
Nitro Hypervisor
There are 2 additional parts of the system:
Nitro TPM
Nitro Enclaves
Let’s check how they are designed and what their responsibilities are within the Nitro System.
Nitro cards
EC2 server consists of 2 parts:
System board (bare-metal)
Nitro Cards
System boards have the CPU and memory.
Nitro Cards are a crucial component in the Nitro System. They function as separate components on the system boards. Some of the key characteristics and functions are:
1. System control and I/O Virtualization
Nitro cards provide overall system control and input/output virtualization independent of the system board. They have their CPU and memory. This separation enhances the performance of the core system.
2. Dedicated hardware and firmware
Nitro cards have their own SoC - System on Chip package which is developed in-house by the dedicated team within AWS - Annapurna Labs. Those SoCs need firmware to run - that firmware is created by the AWS dedicated teams too, meaning that the whole system is performance and security-optimized.
3. Specialized Nitro Cards for different I/O functions
There are specialized Nitro Cards too (with the same SoC and base firmware designs), specifically optimized for different workloads:
Nitro Cards for VPC - networking
Nitro Cards for EBS - storage
Nitro Cards for Local NVMe Storage
4. Hardware-Accelerated Encryption
Nitro cards use hardware offload engines for data encryption in networking and storage. This means that data and the traffic are encrypted without system performance impact. Secure key storage is integrated into the SoC.
Nitro Controller
Nitro Controller is the primary Nitro Card in the AWS Nitro System and with that - the hardware root of trust for the entire system. It manages various Nitro components and their interactions, including the loaded firmware.
What are the responsibilities of a Nitro Controller?
![AWS Nitro System control architecture AWS Nitro System control architecture](https://substackcdn.com/image/fetch/w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F84612cdf-134a-495d-8266-47de10d20089_666x281.png)
The Nitro Controller presents a set of strongly authenticated and encrypted APIs for system management.
On the left is an EC2 Control Plane - a set of services responsible for managing customer configuration (e.g. creating, modifying, deleting resources).
On the right is a Nitro Hypervisor. We’ll talk about it a bit later.
Nitro Security Chip
Please note that:
The nitro Controller and other Nitro Cards together operate as one domain in a Nitro System and the system main board with processors, where customer workloads are run.
The Nitro Security Chip is a critical component of the AWS Nitro System. It serves as a bridge between the Nitro Controller and the system board by extending the root hardware of trust (Nitro Controller chain of trust) and control over to the system main board.
The NSC (Nitro Security Chip) sits between the BMC (Baseboard Management Controller) and the main system CPU, on the high-speed PCI connection.
You probably guessed why - by being there, Nitro Security Chip is effectively acting as a firewall on production systems. Because it is controlled by the Nitro Controller, NSC can validate updates to the firmware or other programming of the non-volatile devices (ones that retain information even when the power is off), either on the system main board or the other Nitro Cards.
Secure Boot and Reset control
One of the most important things when it comes to the security of the system is BIOS integrity. It is the heart of the system in a way.
So, how does the NSC help here?
It controls the physical reset pins of the system main board, including the CPU and BMC.
This allows the Nitro Controller to complete its secure boot and verify the integrity of the BIOS, BMC and every other system firmware. Only after all checks are complete, NSC is instructed to release CPUs and BMC from being held in reset.
Note: Nitro Security Chip is playing a crucial role in the EC2 bare-metal mode too. There is no hypervisor as the EC2 instance system board is not shared, so NSC is making sure that the main system CPUs cannot update the firmware in bare-metal mode. But even when Nitro Hypervisor is present (shared system board), NSC has the same role. Just a different approach.
Nitro Hypervisor
Who is running and managing EC2 instances on shared hardware? Nitro Hypervisor - another part of the Nitro System that is purpose in-house built.
The Nitro Hypervisor is a limited and carefully designed component that has been intentionally minimized and purpose built with the capabilities needed to perform its assigned functions, and no more.
The Security Design of the AWS Nitro System - page 16 - link
This system minimizes the possibility of vulnerabilities that could be exploited by attackers. It manages the allocation of resources (CPU, memory) and is controlled by the Nitro Controller.
Think of the AWS Nitro System as a busy office with specialized roles to ensure efficiency:
• Nitro Hypervisor: Like a specialized and efficient receptionist, its primary task is to allocate resources (meeting rooms and equipment) to different teams (virtual machines). It ensures security by maintaining strict boundaries between teams but does not handle tasks such as taking phone calls or managing schedules.
• Nitro Controller: Think of it as the office manager setting the policies and ensuring that the environment is efficiently operating. It sets the ground rules and policies for resource usage but also controls the coordination between the receptionist (Nitro Hypervisor) and other office functions, ensuring everything runs smoothly.
Let’s break down Nitro Hypervisor functions and characteristics:
Resource Management: Manages CPU and memory allocation as instructed by the Nitro Controller.
Hardware Device Assignment: Assigns GPUs to the VMs and manages error recovery.
Simple design, optimized security and performance: There is no networking stack, file system of any sort or any drivers. Meaning? It’s simple, secure and performant.
Storage: Firmware is stored on an encrypted storage device connected to the Nitro Controller. It is linked to the NC’s root of trust too, which means its integrity is validated.
Firmware updates: Live updates are possible. That means that Nitro Hypervisor firmware can be updated or even replaced without disruption to running instances!
Bare Metal instances: In this case, system management, I/O virtualization and monitoring are handled by the Nitro Cards directly. Nitro Hypervisor is an optional discrete component in this case.
Passive communication: Passive communication design principle means that during production operation, components of the system (any system) do not initiate outbound communication. Nitro Hypervisor follows this principle. Remember - there is no networking stack. It waits for commands from the Nitro Controller via a private channel over the PCIe interface.
Nitro TPM
This is another important part of the Nitro System and is utilized during the Secure Boot process.
Let’s check Nitro TPMs function:
Measurement and Verification: During system initialization, the TPM records measurements of the initial boot code. These measurements are then used to verify the integrity of the next stages of system firmware as they are loaded.
System Integrity: The TPM utilizes its embedded tamper-resistant cryptographic keys to generate digital signatures for the set of system measurements taken during the boot process. This digitally signed file serves as a reference point for future boots. Think of it as a boot history/archive.
Boot-Time Integrity Checks: Each time the system reboots, the TPM compares the newly generated system measurements against the signed file created during the initial secure boot. This comparison allows the system to detect any unauthorized changes or modifications to the system firmware.
Controlled Decryption: If the TPM verifies that no unauthorized changes have occurred, it enables the decryption of additional system data, allowing the boot process to proceed. But, if any mismatch is detected, this decryption process (including the Secure Boot) is halted and the system is taken offline, preventing it from hosting customer workloads.
Note: In the case of AWS Outposts deployments, a Nitro Security Key is also used along with a TPM and the secure boot features of the SoC to protect the encryption key for the SSD, which is connected directly to the Nitro Controller.
The Security Design of the AWS Nitro System - page 12 link
It checks the integrity of the firmware. If something has been changed since the initial boot, without authorization, it is clear that something is wrong and the machine becomes unavailable.
Nitro Enclaves
Nitro Enclaves is a capability of the Nitro System that enables users to separate workloads into distinct components. They also allow the processing of data and highly trusted code running to which even the EC2 instance administrators do not have access to.
Nitro Enclaves characteristics:
Enhanced Isolation: Nitro Enclaves inherit the robust isolation specific to EC2 instances operating on the Nitro System. This means that there is a high level of security and protection for sensitive workloads.
Resource Allocation: Parent EC2 instance allocates a dedicated set of virtual CPUs (minimum one full core) and memory pages. These resources are isolated from the parent instance and dedicated to the enclave's operation.
Secure Communication: There is no IP networking capability, intentionally. This limits the attack surface and provides a secure communication model with the parent instance or other authorized systems/devices.
Persistence: Nitro Enclaves don’t have persistent storage, by design. This guarantees that sensitive data is not stored within the enclave's environment when it is deleted.
So, the primary objective of Nitro Enclaves is to provide an additional layer of isolation (and therefore security) to the Nitro System for (very) sensitive tasks. One of the key features is to specifically safeguard the Enclave against unauthorized access from users and software running on the ‘main’ EC2 instance.
Usage
You might ask - where is this useful?
Healthcare: Processing this highly-sensitive data securely - patient records, medical history, test results…
Financial services: Processing sensitive financial data - transactions, account information, financial analytics…
Machine learning: Running machine learning models on sensitive data without exposing the data to the host system or users.
Confidential collaboration: Enclaves enable multiple parties to collaborate on sensitive data without exposing it to each other or the underlying infrastructure. For example, imagine two companies, one has the data, other one has the ML model. They do not want to share their IP (intellectual property)/data. In that case, Nitro enclaves provide a way for them to collaborate while ensuring confidentiality for both parties.
So, there you have it – the AWS Nitro System in a nutshell. It’s a complex virtualization system that provides a high level of security, performance and efficiency.
Whether you’re handling (or planning to) sensitive data in healthcare, finance, or any other industry, Nitro System ensures your workloads are secure and running smoothly.