How to Invoke AWS Glue Job with SNI Filtering Configuration

Select Language:

When you’re running an AWS Glue job from an EC2 instance using the AWS CLI and filtering with Server Name Indication (SNI) through an Envoy proxy, there’s a key detail to keep in mind. You’ll want to allow access to all subdomains under glue.ca-central-1.amazonaws.com by permitting *.glue.ca-central-1.amazonaws.com instead of just glue.ca-central-1.amazonaws.com.

This is important because AWS Glue uses multiple service endpoints with different subdomains. These serve various purposes and help ensure smooth communication between your EC2 instance and the Glue service.

Here’s why you need this wildcard pattern:

Multiple API endpoints: AWS Glue may direct requests to different endpoints with specific subdomains. These can vary based on the region or the particular service operation you’re performing.
Job execution infrastructure: When you start a job with commands like “aws glue start-job-run,” the service interacts with different backend systems via various subdomains.
Service architecture: AWS often uses multiple subdomains to load balance, provide redundancy, and separate different parts of its services. Without allowing all subdomains, your proxy might block some important connections, leading to job failures.

Allowing the wildcard *.glue.ca-central-1.amazonaws.com ensures that your Envoy proxy can reach all necessary Glue service endpoints. This setup will help your EC2 instance run jobs smoothly and manage them effectively without connection issues.

For more details, you can refer to the official AWS documentation on generating test data using AWS Glue and Python.