AWS Transfer Family (SFTP/FTP/FTPS)

AWS Transfer Family is a fully managed service that provides support for transferring files over SFTP (SSH File Transfer Protocol), FTP (File Transfer Protocol), and FTPS (FTP over SSL/TLS). Its primary purpose is to allow external partners, legacy systems, or on-premises applications to transfer files to and from AWS storage (S3 or EFS) using standard file transfer protocols they already use, without requiring them to adopt AWS-specific APIs or SDKs.

Many enterprises have existing business processes that rely on SFTP/FTP for data exchange with trading partners, supply chain systems, or internal tools. Rewriting these to use S3 APIs is costly and disruptive. Transfer Family bridges the gap by exposing an S3 bucket (or EFS file system) as a traditional file server endpoint. The service handles the protocol translation, authentication, and scaling, while the actual data is stored natively in S3 or EFS.

Transfer Family operates by deploying one or more endpoints in your AWS account. Each endpoint is a network interface in a VPC (or internet-facing) that listens for incoming connections on the chosen protocol(s). When a client connects:

1. Authentication: The endpoint authenticates the user. Transfer Family supports three identity providers: - Service-managed: You create users directly within Transfer Family, storing SSH public keys or passwords. This is simple but limited to 500 users per server. - AWS Directory Service: You integrate with Microsoft Active Directory for existing corporate credentials. - Custom identity provider: You provide an HTTP-based API that Transfer Family calls to authenticate users and return their home directory (S3 bucket prefix or EFS path). This allows integration with any identity store (e.g., Okta, LDAP).

Server: A Transfer Family server is the logical entity that accepts connections. Each server has an endpoint (one or more) and a set of users. You can create up to 10 servers per account (soft limit).

Endpoint Types: - Public: Accessible from the internet. The endpoint gets a public DNS name and optional Elastic IP addresses. Use for external partners. - VPC: Accessible only within your VPC via a private IP. Use for internal transfers (e.g., from on-premises via Direct Connect or VPN). - VPC with internal DNS: Same as VPC but uses a custom DNS name resolved via Route 53 Resolver.

Protocols: You can enable one or more of SFTP, FTP, FTPS on a single server. Each protocol has its own port:

Security Policies: Transfer Family allows you to specify a security policy that defines the allowed ciphers, key exchange algorithms, and MACs for SFTP/FTPS. You can choose from predefined policies (e.g., TransferSecurityPolicy-2020-06) or create a custom one.

Idle Timeout: The default idle timeout for a connection is 120 seconds. If no data is transferred within that time, the connection is closed.

File Size Limits: You can set a MaxFileSize for uploads (in bytes). The default is 0 (no limit).

VPC Endpoint Configuration: For VPC endpoints, you must specify the VPC, subnets (at least two for high availability), and security groups. The endpoint gets an Elastic Network Interface (ENI) in each subnet.

CloudWatch Logging: You can enable logging to a CloudWatch log group. By default, logging is disabled.

To create a server using AWS CLI:

aws transfer create-server \
    --endpoint-type VPC \
    --identity-provider-type SERVICE_MANAGED \
    --logging-role arn:aws:iam::123456789012:role/TransferLoggingRole \
    --protocols SFTP

To add a user (service-managed):

aws transfer create-user \
    --server-id s-01234567890abcdef \
    --user-name alice \
    --role arn:aws:iam::123456789012:role/TransferUserRole \
    --home-directory /my-bucket/alice-home \
    --ssh-public-key-body "ssh-rsa AAAAB3..."

To test connectivity (from a client):

sftp -i private-key.pem alice@s-01234567890abcdef.server.transfer.us-east-1.amazonaws.com

Transfer Family charges based on: - Provisioned endpoints: Per endpoint per hour (e.g., ~$0.30/hour for a public SFTP endpoint; varies by region). - Data transferred: Per GB uploaded/downloaded through the endpoint (e.g., ~$0.04/GB). - Additional features: Elastic IP addresses (standard EC2 charges), CloudWatch logs (standard charges).

There is no charge for the number of users or servers (only endpoints).

Transfer Family endpoints are highly available within a single region. For VPC endpoints, you must deploy at least two subnets in different Availability Zones. The service automatically scales to handle the number of concurrent connections; you do not need to provision capacity. However, there is a soft limit of 1000 concurrent connections per server (can be increased by request).

A large retailer exchanges purchase orders and invoices with thousands of suppliers using SFTP. Their on-premises SFTP server is reaching capacity and requires frequent patching. They migrate to AWS Transfer Family with a public endpoint. Each supplier gets a service-managed user with an SSH key. The home directory is a prefix in an S3 bucket (e.g., s3://retail-edi/supplier123/). An AWS Lambda function is triggered on S3 PUT events to process incoming EDI files and send acknowledgments. The retailer saves on hardware maintenance and gains scalability—the endpoint automatically handles peak loads during month-end. They enable CloudWatch logging for audit compliance. A common pitfall: forgetting to set a bucket policy that denies public access; they use S3 Block Public Access and rely on the IAM role for access.

A financial services firm needs to transfer sensitive reports between departments. They use FTPS (FTP over TLS) for compliance, but their internal apps cannot use S3 APIs. They deploy a VPC endpoint for Transfer Family, accessible only within their corporate network via Direct Connect. They integrate with their existing Microsoft Active Directory for authentication (using AWS Directory Service). Each department's files land in a separate S3 bucket prefix, with IAM roles restricting cross-department access. They set a maximum file size of 100 MB to prevent abuse. The security group allows inbound FTPS on port 990 only from internal IP ranges. They monitor CloudWatch logs for failed login attempts. A common mistake: using FTP instead of FTPS over the internet—data is sent in clear text; they ensure FTPS is enforced.

A media company uses Transfer Family with Amazon EFS as the storage backend to support file locking and POSIX permissions required by their legacy rendering applications. They have an on-premises NFS server that they want to retire. They create a VPC endpoint and mount the same EFS file system on both Transfer Family and their EC2 rendering farm. Artists upload raw footage via SFTP from remote locations, and rendering jobs access the same files via EFS. They use a custom identity provider to authenticate users against their existing LDAP. The challenge is EFS performance—they need to provision sufficient throughput (e.g., 100 MB/s per TB) to avoid bottlenecks. They also set up lifecycle management to move older files to EFS Infrequent Access.