Skip to main content

How to use a Private S3 Bucket with RESTful Engine in AWS?

Overview​

There may be security, performance and/or maintenance reasons for not passing in the actual Document Template File or Data File to the Windward RESTful engine. For these reasons, the Windward RESTful engine allows the Document Template File and/or Document Data File to be provided by URI. If you decide to use AWS and S3 to hold your files needed for the RESTful engine you should be using private S3 buckets. The following explains how to use the Windward RESTful engine with private S3 buckets.

Requirements​

  • AWS Account
  • Private S3 Bucket
  • EC2 instance with RESTful engine

1. Create a private S3 bucket​

A private S3 bucket can be created by following the documentation on creating a private S3 bucket. The bucket will default to as not allowing public access. It can be verified by going to the bucket and then found under the “Permissions” tab.

2. Create an AWS VPC Endpoint​

Every AWS account has one default VPC for each AWS Region. Here’s an article on how to create the VPC endpoint for S3. The VPC Endpoint is a gateway endpoint. It adds an entry to the route table of a subnet and forwards S3 traffic to the S3 VPC endpoint. So that the communication between the RESTful engine and S3 stays on the VPC and will not communicate over the internet.

3. Verify Change to Bucket Policy​

As shown in the article above on how to create the VPC endpoint for S3 it contains the modifications to the bucket policy. Below is an example bucket policy that allows access from resources on the Virtual Private Cloud.

{
    "Version": "2012-10-17",
    "Id": "Policy1565791234567",
    "Statement": [
        {
            "Sid": "Stmt1565791234567",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::windward-test/*",
            "Condition": {
                "StringEquals": {
                    "aws:sourceVpce": "vpce-1234567890abcdef0"
                }
            }
        }
    ]
}

The DOC-EXAMPLE-BUCKET under "Resource" will need to be changed to the name of the private bucket. The vpce1a2b3c4d under aws:sourceVpce will need to be changed to your VPCID which can be found. It will be under “VPC” → “Endpoints” → “Endpoint ID” as shown in the image below

4. Try out the RESTful Engine​

The RESTful engine can then use the S3 URI for the private bucket the same way as a public bucket. The URL below is an example of what can be passed to the RESTful engine to reach a private S3 bucket.

https://wnd-unittest-resources.s3.amazonaws.com/windward-private-bucket/Southwind_JSON.docx

Further information on the RESTful engine can be found in our RESTful Engine documentation. Or try out a working RESTful engine using a private S3 bucket on our swagger documentation for the RESTful engine.