How To Upload An S3 Using Node
Uploading files to AWS S3 using Nodejs
AWS S3. A place where you tin store files. That's what most of you already know about it. S3 is one of the older service provided past Amazon, before the days of revolutionary Lambda functions and game changing Alexa Skills.You lot can store most whatever blazon of files from dr. to pdf, and of size ranging from 0B to 5TB.
According to their official docs, AWS S3 is,
"S3 provides comprehensive security and compliance capabilities that run across even the almost stringent regulatory requirements. It gives customers flexibility in the mode they manage data for cost optimization, access control, and compliance. " - AWS Docs
If I try to put it in simple terms, AWS S3, is an object based storage system where every file your store is saved as object not file. There are many big tech-wigs, which uses S3, and Dropbox is one of them. Recently, Dropbox starts saving metadata of their file in their own service but they are still saving chief data in S3. Why? Well, S3 is not that expensive and it's 99.9% available. Plus, yous get the change to use services similar Glacier which tin save data and charge almost $0.01 per GB.
So far, if I have gotten your attention, and you're thinking how to use S3 in my nodejs application. Well, you don't have to expect for long.
AWS has official packet which exposes S3 apis for node js apps and makes it too easy for developers to access S3 from their apps.
Source: https://youtu.be/FLolHgKRTKg
In adjacent few steps, I volition guide you to build a nodejs based app, which can write any file to AWS S3.
1. Set up node app
A basic node app usually accept ii file, bundle.json (for dependencies) and a starter file (like app.js, index.js, server.js).
You can use your OS'due south file managing director or your favourite IDE to create project only I unremarkably prefer CLI. And so, let's go into our crush and follow these commands:
mkdir s3-contacts-upload-demo
cd s3-contacts-upload-demo
touch index.js .gitignore
npm init
If you didn't get any fault in in a higher place commands, you volition have a folder by the name of s3-contacts-upload-demo, iii files in it, package.json, .gitignore andindex.json. You lot can utilise this file to add file in your .gitignore file, so that these won't go committed to github or some other version control.
npm init creates a bundle.json, which have projection'south details, you lot tin can merely hit enter in your shell for default values if y'all wish.
2. Install dependencies
Let's start past installing the NPM package:
npm install --relieve aws-sdk
Subsequently successful installation of this package, you can as well cheque your package.json file, information technology will have aws-sdk listed in "dependencies" field.
This npm bundle can exist used to access any AWS service from your nodejs app, and hither we volition use it for S3.
3. Import packages
Once installed, import the package in your code:
const fs = require('fs');
const AWS = require('aws-sdk');
const s3 = new AWS.S3({
accessKeyId: process.env.AWS_ACCESS_KEY,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
});
As you can notice, nosotros accept also imported fs parcel which volition exist used to write file information in this app. We are also using environment variables, to set up up AWS access and underground access primal, as it is a bad practice to put them on version control similar Github or SVN.
Now, you lot take your S3 instance, which tin can access all the buckets in your AWS account.
iv. Pass bucket data and write business concern logic
Below is a unproblematic prototype of how to upload file to S3. Here, Bucket is proper noun of your bucket and primal is name of subfolder. So, if your bucket name is "exam-saucepan" and you want to save file in "examination-bucket/folder/subfolder/file.csv", so value of Cardinal should be "older/subfolder/file.csv".
Annotation: S3 is object based storage and not file based. Then, even in AWS console you can see nested folders, behind the scene they never get saved like that. Every object has two fields: Primal and Value. Key here is simply name of file and Value is information which is getting stored.
And then, if a saucepan "bucket1" has cardinal "key1/key2/file.mp3", y'all tin can visualize it like this:
{
"bucket1": {
"key1/key2/file.mp3": "<mp3-data>"
}
}
Below is unproblematic snippe to upload a file,using Key and BucketName.
const params = {
Bucket: 'bucket',
Key: 'key',
Body: stream
};
s3.upload(params, function(err, data) {
console.log(err, data);
});
5. File to upload to S3
Commencement, create a file, let's say contacts.csv and write some data in it.
Higher up is some dummy data for you to get started. At present you have a contacts.csv file, allow'due south read information technology using fs module and salve it to S3.
S3 upload method returns error and data in callback, where data field contains location, bucket and key of uploaded file. For consummate API reference, refer their official docs.
Now run this app by post-obit control:
AWS_ACCESS_KEY=<your_access_key>
AWS_SECRET_ACCESS_KEY=<your_secret_key> node index.js
Nosotros have passed our AWS keys as surround variables.
If you lot don't become any error in above snippet, your saucepan should have the file.
And that'southward it, under thirty lines of code y'all uploaded a file to AWS S3.
Yous can find the full project here.
There are lot many things you tin do with this package similar
-
List out buckets and objects
-
Set permissions on bucket
-
Create, go or delete bucket and much more
I would highly recommend you to through this doc for APIs, they are very well explained with parameters you can pass to each API and response format they render.
I promise you plant this postal service useful and please do let united states of america know in instance of any question or query.
- Node JS
Useful Links
Source: https://www.zeolearn.com/magazine/uploading-files-to-aws-s3-using-nodejs
Posted by: bouckwaidelve.blogspot.com
0 Response to "How To Upload An S3 Using Node"
Post a Comment