Using S3 as a source action for CodePipeline

 

Rather new feature is to set S3 bucket to trigger CodePipeline (AWS blog).  There have been around scheduled pulls option for CodePipeline and S3 connection but since March 2018 also ‘push’ option is possible. There are good references available to set CodePipeline and that is therefore skipped in this blogpost. Lets run through a scenario where a new version arrives by Lambda. Let say the s3 bucket to be used is s3sourcebucket and the code is in the file.zip.

At first, versioning has to be turned on for S3 bucket. CodePipeline job can be identified by the version identifier. So after versioning turned on the S3 bucket is all set.

Then the focus is on CodePipeline. Lets use the CodePipeline console to configure the s3://s3sourcebucket/file.zip as the code source as (according to the documentation) CloudTrail and CloudWatch Events are automatically created and linked to the CodePipeline. Whenever new version is uploaded to the bucket, CloudTrail logs the API call, CloudWatch Event rule catch the CloudTrail logged trail and invokes the correspondent CodePipeline.

Below is an example of the automatically generated CloudWatch event rule. As default,  event pattern is built to match the entire event.

{
  "source": [ "aws.s3" ],
  "detail-type": [ "AWS API Call via CloudTrail" ],
  "detail": {
    "eventSource": [ "s3.amazonaws.com" ],
    "eventName": [ "PutObject" ],
    "resources": {
       "ARN": [ "arn:aws:s3:::s3sourcebucket/file.zip" ]
       }
    }
}

As the snippet points out, CloudWatch event rule is looking for “PutObject” as an EventName. That is the correct event name for ‘put’, as if IAM user is uploading a fresh file to the bucket. However, if the file transfers to the bucket by a copy operation, as if Lambda copies the file from somewhere to the bucket, then the EventName is not match. And the CodePipeline won-t be invoked. The correct event name is “CopyObject” for that occasion.  As a general hint for debugging, double-check the CloudTrail log for the correct event name and confirm that the same phrase is used in the CloudWatch Event rule.

-Tero

Leave a Reply

Your email address will not be published. Required fields are marked *