_______ ___ _______ __ __ _______ _______ _______ __ _
| | | | | _ || |_| | | || || _ || | | |
| _____| | | | |_| || | | _____|| || |_| || |_| |
| |_____ | | | || | | |_____ | || || |
|_____ | | |___ | || | |_____ || _|| || _ |
_____| | | || _ || ||_|| | _____| || |_ | _ || | | |
|_______|3 |_______||__| |__||_| |_|bda Clam|_______||_______||__| |__||_| |__|
The goal of this project is to efficiently virus scan files that are uploaded to a S3 bucket and notify the results of the scan.
This can be achieved in a reasonably cost effictive manner using Lambda, node and clamscan.
S3 is configured to call a node handler when a S3 PUT event is received. The node handler calls out to clamscan and then publishes to SNS with the results. SNS can be configured to POST to a webhook or PUT in a SQS queue for later processing.
Unfortunately due to size limitations, its not possible to keep the virus definitions in the package, but rather they need to be uploaded to S3 where the Lambda process can then download. If you're processing files quite closely together, the Lambda container may still be around and so the virus definitions won't need to be re-downloaded.
clamscan is currently the only supported virus scanning engine and need to be configured to
run under Lambda. There are a couple of ways to do this:
docker pull amazonlinuxdocker run -it amazonlinux- Follow the build/update instructions below.
- Spin up an EC2 instance and assume the
ec2-user. sudo yum groupinstall "Development Tools"sudo yum install openssl openssl-devel wgetwget https://www.clamav.net/downloads/production/clamav-0.99.2.tar.gztar -xvf clamav-0.99.2.tar.gzcd clamav-0.99.2./configure --enable-static=yes --enable-shared=no --disable-unrar --prefix=/var/taskmakesudo make install
sudo chown -R ec2-user /var/tasktouch /var/task/etc/freshclam.conf- or
cp /var/task/etc/freshclam.conf.sample /var/task/etc/freshclam.confand follow the instruction to#Comment or remove the line below
- or
mkdir /var/task/share/clamav/var/task/bin/freshclam/var/task/bin/clamscan /var/task/test/resources/EICAR-AV-Test- Should return
/var/task/test/resources/EICAR-AV-Test: Eicar-Test-Signature FOUND
- Should return
- You'll need to upload the virus definition files from
/var/task/share/clamavto a location in S3, or some HTTP/HTTPS accessible location.- Make sure that the
lambda_execIAM role hasreadpermissions on these files. - Add the URIs to the files to your configuration under
db-files
- Make sure that the
- You'll need to copy the
clamscanbinary in/var/task/bin/clamscanto thebindirectory in this project - You'll need to copy the
lib64libraries in/var/task/lib64to thelib64directory in this project
- S3 Bucket with files to scan
- SNS Topic to notify on infected file discovery
- IAM Resources
- An IAM user with an access key/secret so you can
node-lambda runandnode-lambda deploy- Quick Start:
AWSLambdaFullAccess
- Quick Start:
- A
lambda_execrole (Lambda Service type role)- Quick Start:
AmazonS3ReadOnlyAccessAmazonSNSFullAccess
- Better Security:
- S3 Bucket:
ListBuckets&GetObject - SNS Topic: Publish access
- S3 Bucket:
- Quick Start:
- An IAM user with an access key/secret so you can
- CloudWatch is very useful for debugging - you will need to add permissions for that as well if desired.
# Install dependencies
brew install node
npm install -g node-lambda
# Setup the package
npm install # or `yarn install`
# Provide your `slamscan` configuration by changing the `DEFINE-ME` values in default.yaml to the relevant ones for you
cp config/test.yaml config/local.yaml
emacs config/local.yaml # or `vim` or `nano` or whatever.
# Initialize and provide some `node-lambda` configuration
node-lambda setup
emacs .env # or `vim` or `nano` or whatever.
# You'll want to set the following, but experimentation is encouraged
# AWS_MEMORY_SIZE=1024 # `clamscan` is pretty RAM hungry these days, per https://github.com/widdix/aws-s3-virusscan/issues/12
# AWS_TIMEOUT=120 # `clamscan` takes a bit of time to spin up, plus downloading your virus definitions & files to scan might take a while
# AWS_RUN_TIMEOUT=120
# AWS_PROFILE=<your local `aws-cli` configured credentials in a profile>
cp test/resources/event.json ./event.json
# Might as well use the skeleton that's already there and just change the `DEFINE-ME`s
emacs event.json # or `vim` or `nano` or whatever.
# Run your lambda locally
node-lambda run
# Deploy your lambda
node-lambda deploy
Login to the AWS Console and add the appropriate (S3 PUT) triggers here.
Pull requests are welcome. Please ensure existing standards and tests pass by running npm test.