Study guide
11.4 Private Network Connectivity & Isolation with AWS PrivateLink & VPC Endpoints
AWS Certified Data Engineer - Associate DEA-C01
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Streaming Data Ingestion
1.1 Kinesis Data Streams vs. Firehose Architecture & Partitioning 1.2 Amazon MSK (Managed Streaming for Apache Kafka) & Serverless Patterns 1.3 Event-Driven Streaming Ingestion with DynamoDB Streams & Lambda 1.4 Streaming Distribution, Replay & Processing State
2 Chapter 2: Batch Data Ingestion & Migration
2.1 AWS DMS (Database Migration Service) & Change Data Capture (CDC) 2.2 S3 Batch Operations & High-Throughput Bulk Data Ingestion 2.3 Hybrid Data Transfer with Snow Family & AWS DataSync 2.4 Application, API & Managed File Ingestion
3 Chapter 3: ETL Transformation with AWS Glue
3.1 AWS Glue Jobs, DynamicFrames & PySpark Transformations 3.2 Glue Crawlers, Data Catalog & Schema Evolution 3.3 Data Preparation & Quality Rules with AWS Glue DataBrew & Data Quality 3.4 Incremental Processing with AWS Glue Job Bookmarks & Partition Processing 3.5 Technical and Business Catalogs, Connections & Lineage
4 Chapter 4: Big Data Processing & Workflow Orchestration
4.1 Amazon EMR Cluster Architecture, Spark & Hadoop Ecosystem 4.2 EMR Serverless & Instance Selection (On-Demand vs. Spot Optimization) 4.3 Serverless Orchestration with AWS Step Functions 4.4 Airflow Pipeline Orchestration with Amazon MWAA 4.5 Containers, Lambda & Distributed Processing Performance 4.6 Data Engineering Code, IaC & CI/CD
5 Chapter 5: Object Storage & Data Lake Architecture
5.1 Amazon S3 Storage Classes, Lifecycle Policies & Cost Optimization 5.2 S3 Data Lake Partitioning, Bucketing & Layout Strategies 5.3 Efficient Columnar Formats: Parquet, ORC, and Avro Schema Design 5.4 S3 Performance Optimization, Multipart Uploads & Transfer Acceleration 5.5 Apache Iceberg, S3 Tables & Open Table Operations
6 Chapter 6: Data Warehousing with Amazon Redshift
6.1 Amazon Redshift Architecture, RA3 Nodes & Managed Storage 6.2 Redshift Data Modeling: Distribution Styles (KEY, EVEN, ALL) & Sort Keys 6.3 Loading & Unloading Data with COPY/UNLOAD & Redshift Spectrum
7 Chapter 7: NoSQL, Relational & Specialty Data Stores
7.1 Amazon DynamoDB Partition Keys, Global Secondary Indexes & Data Modeling 7.2 Amazon RDS & Aurora Operations, Read Replicas & Multi-AZ 7.3 Graph & Analytical Querying with Amazon Neptune, OpenSearch & Athena 7.4 Vector Stores, Indexes & LLM Data Processing
8 Chapter 8: Pipeline Operations, Monitoring & Observability
8.1 Monitoring Pipeline Health with Amazon CloudWatch Metrics & Logs 8.2 Event-Driven Alerting & Automation with EventBridge & CloudWatch Alarms 8.3 Centralized Audit Logging with AWS CloudTrail
9 Chapter 9: Troubleshooting, Data Quality & Performance Tuning
9.1 Diagnosing & Resolving AWS Glue, EMR & Lambda Execution Failures 9.2 Redshift & Athena Query Performance Tuning & Memory Management 9.3 Implementing Data Quality Frameworks & Automated Validation Checks 9.4 Visualization, Notebooks, SQL Analytics, Sampling & Skew
10 Chapter 10: High Availability, Backup & Cost Optimization
10.1 Disaster Recovery, Cross-Region Replication & Backup Strategies 10.2 Designing High Availability Architectures Across AWS Data Services 10.3 Cost Optimization & Resource Utilization across Analytics Workloads
11 Chapter 11: Data Security, Encryption & Identity Management
11.1 Fine-Grained Access Control with AWS IAM Policies, Roles & Conditions 11.2 Data Encryption at Rest & in Transit with AWS KMS & S3 Bucket Policies 11.3 Secure Credentials & Secrets Management with AWS Secrets Manager & Parameter Store 11.4 Private Network Connectivity & Isolation with AWS PrivateLink & VPC Endpoints
12 Chapter 12: Data Governance, Privacy & Compliance
12.1 Centralized Data Lake Governance & Access Control with AWS Lake Formation 12.2 Automated Sensitive Data Discovery & PII Protection with Amazon Macie 12.3 Configuration Auditing & Compliance Enforcement with AWS Config 12.4 Data Privacy, Sovereignty, Sharing & Retention Guardrails 12.5 SageMaker Unified Studio Domains, Projects & Catalog Governance
11.4 Private Network Connectivity & Isolation with AWS PrivateLink & VPC Endpoints
Key Takeaways Explain This Topic AI
Gateway VPC Endpoints route traffic to Amazon S3 and DynamoDB via VPC route table entries at no extra cost, requiring no Public IPs, NAT Gateways, or Internet Gateways.
Interface VPC Endpoints (powered by AWS PrivateLink) deploy Elastic Network Interfaces (ENIs) with private IP addresses in your VPC subnets to connect privately to AWS services like KMS, Glue, Kinesis, and Redshift.
Gateway VPC Endpoint Policies provide network-level access control, allowing organizations to restrict data transfers exclusively to corporate-owned S3 buckets and block external data exfiltration.
Running AWS Glue ETL jobs or Amazon EMR clusters inside private subnets without Internet Gateways requires VPC endpoints for S3, KMS, Glue Data Catalog, and CloudWatch logs to function properly.
AWS PrivateLink enables cross-VPC and cross-account data sharing by exposing internal microservices and data pipelines behind Network Load Balancers (NLBs) without VPC Peering or public routing.
Last updated: August 2026
Private Network Connectivity & Isolation with AWS PrivateLink & VPC Endpoints
Security best practices require that enterprise data lakes, data warehouses, and ETL processing clusters operate within completely isolated network environments. Traffic between compute nodes (EMR, EC2, Glue) and storage engines (S3, Redshift, DynamoDB) must never traverse the public internet. AWS Virtual Private Cloud (VPC) Endpoints and AWS PrivateLink provide private, secure connectivity to AWS services and external SaaS applications entirely within the AWS global network backbone.
Gateway VPC Endpoints vs Interface VPC Endpoints
Understanding the fundamental architectural and pricing differences between Gateway Endpoints and Interface Endpoints is mandatory for the DEA-C01 exam:
Architectural Feature Gateway VPC Endpoints Interface VPC Endpoints (AWS PrivateLink)
Supported AWS Services Amazon S3 and Amazon DynamoDB ONLY. KMS, Glue, Kinesis, Redshift, Athena, Secrets Manager, 100+ services.
Underlying Mechanism Route Table target entry ( vpce-xxx ) in VPC. Elastic Network Interface (ENI) with private IP in subnet.
Security Groups No (Uses Endpoint Policies & Route Tables). Yes (Attaches standard VPC Security Groups).
Pricing / Hourly Cost 100% FREE (No hourly or data processing fee). Hourly fee per endpoint + per-GB data processing fee.
On-Premises Access Cannot be accessed directly via Direct Connect / VPN. Accessible from on-premises via Direct Connect / VPN (Private IP).
Cross-Region Access Regionally bound to the VPC. Supports VPC Peering / Transit Gateway cross-region routing.
Gateway VPC Endpoints for S3 & Data Exfiltration Prevention
To configure a Gateway VPC Endpoint for S3:
Create the endpoint specifying the target VPC and S3 service identifier ( com.amazonaws.us-east-1.s3 ).
Attach the endpoint to the private subnet Route Tables .
The AWS network automatically injects a prefix list route directing all S3 traffic directly to the private AWS network backbone. [ Private Subnet EC2 / EMR ] --> [ Route Table: Pl-63a5400a -> vpce-12345 ] --> [ AWS Backbone ] --> [ Amazon S3 ]
Data Exfiltration Prevention via Gateway Endpoint Policies
Without an Endpoint Policy, a compromised EC2 instance in a private subnet could upload sensitive corporate data to a malicious actor's personal S3 bucket. A VPC Endpoint Policy enforces resource restrictions at the network layer: {
- "Version": "2012-10-17",
- "Statement": [
- {
- "Sid": "RestrictAccessToCorporateBucketsOnly",
- "Effect": "Allow",
- "Principal": "*",
- "Action": ["s3:GetObject", "s3:PutObject", "s3:ListBucket"],
- "Resource": [
- "arn:aws:s3:::corporate-data-lake-prod",
- "arn:aws:s3:::corporate-data-lake-prod/*"
- ]
- }
- ]
- }
A request routed through this endpoint to an unlisted bucket is blocked even if the application has broad IAM permissions. Pair the endpoint policy with route, egress, bucket-policy, and Organizations controls because it does not govern traffic that bypasses the endpoint.
Interface VPC Endpoints (AWS PrivateLink)
Interface VPC Endpoints deploy private Elastic Network Interfaces (ENIs) into chosen availability zones inside your private subnets. Private DNS names map standard AWS service endpoints (e.g., kms.us-east-1.amazonaws.com ) directly to the ENI's private IP addresses ( 10.0.1.50 ).
Essential Interface Endpoints for Private Data Pipelines
When deploying AWS Glue ETL jobs or EMR clusters inside a completely private subnet (without NAT Gateway or Internet Gateway), you MUST create Interface VPC Endpoints for all dependent services:
com.amazonaws.region.kms : Required for KMS key decryption during data processing.
com.amazonaws.region.glue : Required for AWS Glue Data Catalog metadata queries.
com.amazonaws.region.logs : Required for pushing CloudWatch logs.
com.amazonaws.region.kinesis-streams : Required for private real-time stream ingestion.
AWS PrivateLink for Custom Cross-VPC Data Services
AWS PrivateLink also allows enterprise architectures to share data microservices or data ingestion APIs across different AWS accounts and VPCs without establishing complex VPC Peering or Transit Gateway routing tables. [ Consumer VPC (Account A) ] [ Service Provider VPC (Account B) ]
- Private Subnet Private Subnet
- [ Interface Endpoint ] <=== PrivateLink AWS Backbone ===> [ Network Load Balancer (NLB) ]
- |
- v
- [ Data Ingestion API / EKS ]
How PrivateLink Provider Services Work:
The service provider exposes their data ingestion microservice behind a Network Load Balancer (NLB) .
The provider creates an Endpoint Service configuration referencing the NLB.
The consumer account creates an Interface VPC Endpoint pointing to the Endpoint Service.
Traffic flows unidirectionally from consumer to provider over the private AWS backbone, completely isolated from IP overlapping conflicts.
- Network Isolation Design Patterns for AWS Data Services
- 1. Amazon EMR in Private Subnets
EC2 instances run in private subnets with no public IPs.
Security groups block all inbound traffic except internal cluster communication.
Gateway Endpoint for S3 handles data storage access.
Interface Endpoints handle KMS, CloudWatch, and EMR service communication.
2. AWS Glue Connection in VPC Mode
Glue ETL jobs attach to a specified Subnet and Security Group via Elastic Network Interfaces.
Requires a Gateway Endpoint for S3 in the subnet route table to prevent Glue jobs from timing out when fetching dependencies or reading S3 Parquet files.
Summary Checklist for DEA-C01 Exam
Gateway Endpoints : S3 and DynamoDB ONLY. Free. Uses VPC Route Tables.
Interface Endpoints (PrivateLink) : Uses ENIs + Private IPs + Security Groups. Hourly fee.
Data Exfiltration Prevention : Attach an Endpoint Policy to Gateway Endpoints restricting S3 access strictly to allowed corporate bucket ARNs.
On-Premises Access : Requires Interface Endpoints (PrivateLink) because Gateway Endpoints cannot be targeted from Direct Connect.