A New Zealand insurance company has released a web service to allow partners, brokers and agents to submit applications online from their portal. When submitting an application, advisers are sometimes required to upload supporting documents needed for assessing the application before a policy is issued. Because they can contain malicious content, all files must be scanned for viruses before they are stored.
The customer was looking for a virus scanning solution that was cost effective and had a quick turn-around time. To find this solution, they turned to OSS Group, the team behind their upgrade to an enterprise data centre built on AWS.
“We were happy to use OSS Group as they had a good understanding of requirements and had successfully delivered the original AWS environment.”
The easiest solution the team at OSS Group could have implemented was running the antivirus software in an Elastic Compute Cloud (EC2). However, this would have required paying for the continual running of the EC2.
“This was an unnecessary expense,” says Filipe Silva, a technical consultant with OSS Group. “We knew we could do better.”
Instead, the team recommended the use of AWS Lambda, a serverless computing platform. While an EC2 is constantly on and running, Lambda is triggered by an event. When it is triggered, it responds to the event by executing predetermined code.
The first design was this: clients would upload their files, which would be encoded. The arrival of a file in an S3 bucket would trigger the Lambda function. Lambda would copy the file, decode it, and download the latest virus definition database. The virus scan would run inside Lambda on the copy of the document. If the file was found to be dirty, the copy would be deleted and the original in the untrusted bucket would also be deleted. If clean, Lambda would delete its copy and the original file would be moved from the untrusted bucket to a trusted bucket.
“The first one million requests are free, so until the customer exceeds that amount, it does not have to pay for Lambda.”
Even when it does reach that number of requests, the costs of running Lambda will remain minimal. They only need to pay for the time Lambda is running, from when the code is triggered to the completion of the task.
Although using Lambda on its own was the most cost-effective option, there was a business limitation that prevented this solution from being deployed.
“Lambda returned a response in 30 to 45 seconds,” says Silva. “This was too slow for the customer.”
Since Lambda terminates after responding to an event, the virus definition database had to be downloaded again whenever a new event was triggered. This task was taking the most time.
To speed things up, the team introduced Docker containers with AWS Fargate as a container management platform. For this solution, Lambda copies the file, decodes it and sends it via a TCP socket stream to a pair of Docker containers that are running the antivirus. One container scans the file and the other container keeps the database up to date by downloading and updating the virus definitions.
This solution resulted in a tremendous leap in speed. By using containers, the OSS Group team dropped the response time from 30 to 45 second down to an average of 2 to 3 seconds.
The cost for this increase in speed was minimal – the only added cost was the container manager, which came with additional benefits.
“By using Fargate, the self-managed container solution for AWS, I do not need to manage an EC2 instance, and I don’t need to manage the underlying infrastructure of the containers."
While one of the pair of Docker containers is responsible for updating the virus definition database, sometimes the antivirus software itself needs to be updated to a new version. A safe way of doing this is to use blue/green deployment. In blue/green deployment, the “blue” version runs a copy of the application in production while the “green” version runs the updated version in a non-production environment. The green version can be tested without impacting the production environment. When it is ready, the blue version is switched with the green version, creating a seamless transition to the updated version of the software.
“I can implement blue/green deployment using AWS Code Deploy to roll out a new version without effecting the process,” says Silva.
Since blue/green deployment requires that a copy be made of the production environment, using containers is a cost-effective solution.
“If the antivirus solution were being run in a full stack, I would need to pay to run a copy of the entire stack,” says Silva.
By using containers, running a copy of the environment is significantly cheaper.
Although containers allowed for the antivirus check to be done at speed, Silva insists that Lambda is the powerhouse behind this specific solution.
“Lambda is the key differentiator here,” says Silva.
Instead of continually running an EC2, Lambda’s serverless, event-triggered properties allowed the OSS Group team to consider options that optimised cost and turn-around time. And by providing a solution that fulfilled the client’s requirements, the team at OSS Group has satisfied yet another customer.
“End to end performance was a key success criterion for our project. Without the solution provided by OSS, the original solution would have been untenable.”
These Stories on AWS
+64 9 307 3500
Ground Floor, 126 Vincent Street, Auckland 1010
P.O. Box 8833, Auckland 1149
+64 4 260 5300
Level 5, Featherston House, 119 Featherston Street, Wellington 6011
Comments (1)