SSM Inventory and Managed Instance Management

AWS Systems Manager Inventory is a feature that automates the collection of metadata from your managed instances. It provides a centralized view of software installed, OS configuration, file details, and other system-level information across your entire fleet. Inventory is not a configuration management tool; it is purely a data collection service. It integrates with AWS Config, Amazon Athena, and Amazon S3 for querying and analysis.

Before Inventory, administrators had to manually connect to each instance or use third-party tools to gather inventory data. At scale, this becomes impractical. Inventory solves this by:

Inventory relies on the SSM Agent running on each instance. The agent collects metadata based on inventory types you specify and sends it to the Systems Manager service. The service then stores the data in an S3 bucket (if configured) and/or in AWS Config as configuration items.

Step-by-step mechanism: 1. You create an Inventory association (via State Manager or explicitly as an Inventory configuration). 2. The association defines which inventory types to collect (e.g., AWS:Application, AWS:NetworkConfig), the schedule (rate or cron), and the target instances (by tags, resource groups, or instance IDs). 3. The SSM Agent on each target instance receives the association and runs the inventory collection on the defined schedule. 4. The agent gathers data (e.g., list of installed applications) and sends it to the Systems Manager service as a JSON payload. 5. The service processes the payload and stores it in the S3 bucket you specified (optional but recommended) and/or in AWS Config. 6. You can then query the data using the console, CLI, or AWS Config advanced queries.

Creating an Inventory association using the AWS CLI:

aws ssm create-association \
    --name AWS-GatherSoftwareInventory \
    --targets Key=tag:Environment,Values=Production \
    --schedule-expression "rate(1 hour)" \
    --parameters '{"applications": ["Enabled"], "awsComponents": ["Enabled"], "networkConfig": ["Enabled"], "files": ["Enabled"], "windowsRegistry": ["Enabled"], "instanceDetailedInformation": ["Enabled"], "customInventory": ["Enabled"], "services": ["Enabled"], "windowsRoles": ["Enabled"], "tags": ["Enabled"], "update": ["Enabled"], "compliance": ["Enabled"]}'

Verifying inventory data collection:

aws ssm get-inventory \
    --instance-id i-1234567890abcdef0 \
    --type-name AWS:Application

Listing all inventory instances:

aws ssm list-inventory-entries \
    --instance-id i-1234567890abcdef0 \
    --type-name AWS:InstanceInformation

Querying with AWS Config advanced query:

SELECT
  resourceId,
  resourceType,
  configuration.application
WHERE
  resourceType = 'AWS::SSM::ManagedInstanceInventory'

Managed instances are EC2 instances or on-premises machines that have the SSM Agent installed and are registered with Systems Manager. To manage an instance, you need:

Key concepts: - Hybrid Activations: For on-premises servers, you create a hybrid activation to register them as managed instances. - Managed Instance ID: Each managed instance gets a unique ID (e.g., mi-1234567890abcdef0). - Ping Status: You can check connectivity with aws ssm describe-instance-information. - Association Status: Use aws ssm describe-association-execution-targets to verify association compliance.

Common issues: - Inventory not collected: Check SSM Agent logs (/var/log/amazon/ssm/ on Linux, %PROGRAMDATA%\Amazon\SSM\Logs\ on Windows). Ensure the instance has proper IAM permissions (AmazonSSMManagedInstanceCore). Verify network connectivity (use aws ssm describe-instance-information to see ping status). - Data not appearing in S3: Confirm S3 bucket policy allows Systems Manager to write. Check that you specified a bucket when creating the association. - Missing inventory types: Ensure the SSM Agent version supports the type (e.g., AWS:File requires agent version 2.2.x or later). - Custom inventory not working: The custom inventory script must output JSON to stdout in the correct format.

Scenario 1: Enterprise Software License Compliance A large enterprise with thousands of EC2 instances needs to track which instances have licensed software (e.g., Oracle Database, Microsoft SQL Server). They configure Inventory to collect AWS:Application data every 6 hours and store it in an S3 bucket. They use Amazon Athena to run weekly queries that list all instances with Oracle installed, cross-referencing with an external license database. The challenge is that some instances are in private subnets without internet access, so they must use VPC endpoints for Systems Manager and S3. They also set up a Config rule to flag instances that have unlicensed software versions. The main issue they face is that Inventory only reports what is installed, not whether the license is valid; they need to combine with other data sources.

Scenario 2: OS and Patch Compliance A company wants to ensure all Windows servers have a specific set of security updates. They enable Inventory with AWS:Update type to collect installed updates. They also use Patch Manager to apply missing patches. Inventory shows which patches are installed, while Compliance shows whether the instance is compliant with a patch baseline. They set a schedule to collect inventory daily and patch weekly. A common misconfiguration is not enabling the AWS:Update type, so the compliance dashboard shows no data. They also learn that Inventory data for updates is only collected after a patch operation; it does not automatically reflect real-time changes. They must ensure the association runs after patching.

Scenario 3: Custom Software Inventory A DevOps team uses a custom script to collect deployment metadata (e.g., version of a custom application, environment variables). They implement a custom inventory type using AWS:CustomInventory. The script runs on each instance and outputs JSON to stdout. They configure Inventory to execute this script every hour. The data is stored in S3 and queried via Athena. The challenge is that the custom inventory script must be idempotent and handle errors gracefully. If the script fails, Inventory does not collect data for that cycle. They also need to ensure the script is present on all instances; they use a State Manager association to distribute the script. A common mistake is not setting the correct working directory or environment variables for the script.