Automated Video Transcoding using Amazon Elastic Transcoder & AWS Lambda

Many AWS customers use Amazon Cloud Front to stream videos to various devices. As you may know, video streaming can be implemented using a wide variety of protocols that are layered on top of HTTP.

In this post, I would like to show you how multiple AWS services can be used to implement video streaming. First, we will take a look at the on-demand streaming of content stored in Amazon S3.

Recently I got a chance to work with the transcoding process and make it automated using Elastic Transcoder, S3, and Lambda. The process not only transcodes videos to 720p, WebM and HLSv3 but also generates thumbnails, watermarks our videos and nicely organizes everything in S3.

Amazon Elastic Transcoder:

Elastic Transcoder serves as the backbone of our implementation. To get it up and running you need to do two things: define a pipeline and create a job. A pipeline essentially defines a queue for future jobs. To create a pipeline you need to specify the input bucket (where the source videos will be), the output bucket and a bucket for thumbnails.

Having created a pipeline you can immediately create a job and kick off a transcoding task. The job involves specifying the name of the file, selecting one or more of the transcoding presets (e.g. Generic 720p, WebM, HLS, etc..), setting up a playlist, metadata or overriding input parameters such as the frame rate or aspect ratio.

The above process is a manual process where for each video you need to manually create a job.

Elastic Transcoder Dashboard

To create a job automatically for newly added objects in the S3 bucket, we have a great serverless approach called “Lambda” to run custom code.

At the moment, there is no way to automatically run an Elastic Transcoder job by uploading a file to a bucket. However, it is easy to invoke Lambda from an S3 event and from a Lambda function invoke Elastic Transcoder.

Here we have created an architecture of the video transcoding process. Do have a look at the process below.

AWS Architecture

Invoking a Lambda function from S3 Bucket:

Invoking a Lambda function from an S3 bucket is straightforward — open the required bucket in S3 and click on Properties. From there click on Events, Lambda and from the drop-down select the relevant Lambda function you want to execute. Finally, select the event (ObjectCreated (All)) that will trigger the process.

S3 Bucket Event Trigger

Lambda:

Previously I mentioned that you can create and execute jobs using the Elastic Transcoder console. But to automate the process of transcoding using Lambda, Thankfully it’s pretty easy to do using the AWS JavaScript SDK. Here are the main steps you would go through:

1. Go to Lambda console, create a new function from the predefined blueprint and add the AWS SDK to it.
2. Here we will select a blank function, Click on “Author From Scratch”.

Lambda Blank Function

An example Lambda function that meets our requirements is given below. Note that it creates a job with outputs— generic 720p.

var AWS = require(‘aws-sdk’);
var s3 = new AWS.S3({
apiVersion: ‘2012–09–25’
});
var eltr = new AWS.ElasticTranscoder({
apiVersion: ‘2012–09–25’,
region: ‘ap-south-1’
});
exports.handler = function(event, context) {
console.log(‘Executing Elastic Transcoder Orchestrator’);
var bucket = event.Records[0].s3.bucket.name;
var key = event.Records[0].s3.object.key;
var pipelineId = <Your Pipeline ID>;
if (bucket !== <Your Bucket Name>) {
context.fail(‘Incorrect Video Input Bucket’);
return;
}
var srcKey = decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, “ “)); //the object may have spaces
var newKey = key.split(‘.’)[0];
var todayTime = Math.round((new Date()).getTime() / 1000);
var params = {
PipelineId: pipelineId,
//OutputKeyPrefix: todayTime + ‘/’,
Input: {
Key: srcKey,
FrameRate: ‘auto’,
Resolution: ‘auto’,
AspectRatio: ‘auto’,
Interlaced: ‘auto’,
Container: ‘auto’
},
Outputs: [{
Key: ‘videos/’ + newKey + ‘.mp4’,
ThumbnailPattern: ‘thumbs/’ + newKey + ‘-{count}’,
PresetId: ‘<your Preset ID>’, //Generic 720p
}]
};
console.log(‘Starting Job’);
eltr.createJob(params, function(err, data){
if (err){
console.log(err);
} else {
console.log(data);
}
context.succeed(‘Job well done’);
});
};

ETS Presets:

One detail you might notice in the Lambda function above is the use of presets. Presets describes how to encode the given file. The full list of available presets can be found in AWS documentation. You will also get that list on elastic transcoder Page under “Presets” tab.

It’s also possible to create your own preset too. You can do so by clicking on the Presets link in the Elastic Transcoder console. You’ll see the full list of system presets there and you’ll be able to create your own by hitting the Create New Preset button.

you can also make a copy of existing presets to modify as per your requirement.

Test:

Now you can add the object to the S3 buckets and check if the lambda function is getting triggered further triggering to the amazon elastic transcoder.

You can check the job status of the object under the “Jobs” section on the ETS console.

ETS Jobs Status

Have you used the Elastic Transcoder for your project? I am keen to hear about your experience.