How to successfully upload an image to AWS S3 using NestJS?

Question

How to successfully upload an image to AWS S3 using NestJS?

I'm currently working on implementing an image upload feature to AWS S3 using multer-s3 within my NestJS API. I have experimented with aws-sdk as well. I am utilizing FileInterceptor and UploadedFile decorator to handle the file request. This is what I have achieved so far:

// Controller
 @Post()
 @UseInterceptors(FileInterceptor('file', multerOptions))
    uploadImage(@UploadedFile() file) {
        console.log(file);
    }

// multerOptions in multer.ts file
const configService = new ConfigService();

export const accessParams = {
    accessKeyId: configService.get('AWS_ACCESS_KEY_ID'),
    secretAccessKey: configService.get('AWS_SECRET_ACCESS_KEY'),
    region: configService.get('AWS_REGION'),
};

const imageMimeTypes = [
    'image/jpg',
    'image/jpeg',
    'image/png',
    'image/bmp',
];

AWS.config.update(accessParams);
export const s3 = new AWS.S3();

export const multerOptions = {
    fileFilter: (req: any, file: any, cb: any) => {
        const mimeType = imageMimeTypes.find(im => im === file.mimetype);

        if (mimeType) {
            cb(null, true);
        } else {
            cb(new HttpException(`Unsupported file type ${extname(file.originalname)}`, HttpStatus.BAD_REQUEST), false);
        }
    },
    storage: multerS3({
        s3: s3,
        bucket: configService.get('S3_BUCKET_NAME'),
        acl: 'read-public',
        metadata: function (req, file, cb) {
            cb(null, { fieldName: file.fieldname })
        },
        key: (req: any, file: any, cb: any) => {
            cb(null, `${Date.now().toString()}/${file.originalname}`);
        },
        contentType: multerS3.AUTO_CONTENT_TYPE
    }),
};

However, this implementation results in the following error being triggered:

{
  "message": null,
  "code": "InvalidArgument",
  "region": null,
  "time": "2020-04-24T05:34:19.009Z",
  "requestId": "DH224C558HTDF8E3",
  "extendedRequestId": "JKHKJH6877-LKJALDNC765llLKAL=",
  "statusCode": 400,
  "retryable": false,
  "retryDelay": 6.790294010827713,
  "storageErrors": []
}

Would appreciate any insights or suggestions to resolve this issue. Thank you.

typescript amazon-s3 nestjs multer multer-s3

Answer 1

Answer №1

To implement file uploading functionality in NestJS, you can start by creating a controller:

import { Post, UseInterceptors, UploadedFile } from '@nestjs/common';

@Post('upload')
@UseInterceptors(FileInterceptor('file'))
async upload(@UploadedFile() file) {
  return await this.fileUploadService.upload(file);
}

Your FileUploadService should be structured as follows:

import { S3 } from 'aws-sdk';
import { Logger, Injectable } from '@nestjs/common';

@Injectable()
export class FileUploadService {
    async upload(file) {
        const { originalname } = file;
        const bucketS3 = 'my-aws-bucket';
        await this.uploadToS3(file.buffer, bucketS3, originalname);
    }

    async uploadToS3(file, bucket, name) {
        const s3 = this.getS3();
        const params = {
            Bucket: bucket,
            Key: String(name),
            Body: file,
        };
        return new Promise((resolve, reject) => {
            s3.upload(params, (err, data) => {
            if (err) {
                Logger.error(err);
                reject(err.message);
            }
            resolve(data);
            });
        });
    }

    getS3() {
        return new S3({
            accessKeyId: process.env.AWS_ACCESS_KEY_ID,
            secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
        });
    }
}

Answer 2

To implement file uploading functionality in NestJS, you can start by creating a controller:

import { Post, UseInterceptors, UploadedFile } from '@nestjs/common';

@Post('upload')
@UseInterceptors(FileInterceptor('file'))
async upload(@UploadedFile() file) {
  return await this.fileUploadService.upload(file);
}

Your FileUploadService should be structured as follows:

import { S3 } from 'aws-sdk';
import { Logger, Injectable } from '@nestjs/common';

@Injectable()
export class FileUploadService {
    async upload(file) {
        const { originalname } = file;
        const bucketS3 = 'my-aws-bucket';
        await this.uploadToS3(file.buffer, bucketS3, originalname);
    }

    async uploadToS3(file, bucket, name) {
        const s3 = this.getS3();
        const params = {
            Bucket: bucket,
            Key: String(name),
            Body: file,
        };
        return new Promise((resolve, reject) => {
            s3.upload(params, (err, data) => {
            if (err) {
                Logger.error(err);
                reject(err.message);
            }
            resolve(data);
            });
        });
    }

    getS3() {
        return new S3({
            accessKeyId: process.env.AWS_ACCESS_KEY_ID,
            secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
        });
    }
}

Answer 3

Answer №2

Utilizing typings and stream functionality:

import { ReadStream } from 'fs';
import * as AWS from 'aws-sdk';
import { PromiseResult } from 'aws-sdk/lib/request';
import { Injectable, InternalServerErrorException } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';

import { TConfig, TStorageConfig } from '../../config';

@Injectable()
export class CloudStorageService {
    private S3: AWS.S3;
    private BUCKET: string;

    constructor(private configService: ConfigService<TConfig>) {
        this.S3 = new AWS.S3({
            // Customize your configuration options here
            accessKeyId: this.configService.get<TStorageConfig>('storage').accessKeyId,
            secretAccessKey: this.configService.get<TStorageConfig>('storage').secretAccessKey,
            endpoint: this.configService.get<TStorageConfig>('storage').endpoint,
            s3ForcePathStyle: true,
            signatureVersion: 'v4',
        });
        this.BUCKET = this.configService.get<TStorageConfig>('storage').bucket;
    }

    async getBlob(key: string): Promise<PromiseResult<AWS.S3.GetObjectOutput, AWS.AWSError>> {
        const params = { Bucket: this.BUCKET, Key: key };
        const blob = await this.S3.getObject(params).promise();
        
        return blob;
    }

    async putBlob(blobName: string, blob: Buffer): Promise<PromiseResult<AWS.S3.PutObjectOutput, AWS.AWSError>> {
        const params = { Bucket: this.BUCKET, Key: blobName, Body: blob };
        const uploadedBlob = await this.S3.putObject(params).promise();

        return uploadedBlob;
    }

    // To upload a stream, you can utilize file.createReadStream()
    async putStream(key: string, stream: ReadStream): Promise<AWS.S3.PutObjectOutput> {
        const file = await new Promise<AWS.S3.PutObjectOutput>((resolve, reject) => {
            const handleError = (error) => {
                reject(error);
            };
            const chunks: Buffer[] = [];

            stream.on('data', (chunk: Buffer) => {
                chunks.push(chunk);
            });

            stream.once('end', async () => {
                const fileBuffer = Buffer.concat(chunks);

                try {
                    const uploaded = await this.putBlob(key, fileBuffer);

                    resolve(uploaded);
                } catch (error) {
                    handleError(new InternalServerErrorException(error));
                }
            });

            stream.on('error', (error) => handleError(new InternalServerErrorException(error)));
        });

        return file;
    }
}

Answer 4

Utilizing typings and stream functionality:

import { ReadStream } from 'fs';
import * as AWS from 'aws-sdk';
import { PromiseResult } from 'aws-sdk/lib/request';
import { Injectable, InternalServerErrorException } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';

import { TConfig, TStorageConfig } from '../../config';

@Injectable()
export class CloudStorageService {
    private S3: AWS.S3;
    private BUCKET: string;

    constructor(private configService: ConfigService<TConfig>) {
        this.S3 = new AWS.S3({
            // Customize your configuration options here
            accessKeyId: this.configService.get<TStorageConfig>('storage').accessKeyId,
            secretAccessKey: this.configService.get<TStorageConfig>('storage').secretAccessKey,
            endpoint: this.configService.get<TStorageConfig>('storage').endpoint,
            s3ForcePathStyle: true,
            signatureVersion: 'v4',
        });
        this.BUCKET = this.configService.get<TStorageConfig>('storage').bucket;
    }

    async getBlob(key: string): Promise<PromiseResult<AWS.S3.GetObjectOutput, AWS.AWSError>> {
        const params = { Bucket: this.BUCKET, Key: key };
        const blob = await this.S3.getObject(params).promise();
        
        return blob;
    }

    async putBlob(blobName: string, blob: Buffer): Promise<PromiseResult<AWS.S3.PutObjectOutput, AWS.AWSError>> {
        const params = { Bucket: this.BUCKET, Key: blobName, Body: blob };
        const uploadedBlob = await this.S3.putObject(params).promise();

        return uploadedBlob;
    }

    // To upload a stream, you can utilize file.createReadStream()
    async putStream(key: string, stream: ReadStream): Promise<AWS.S3.PutObjectOutput> {
        const file = await new Promise<AWS.S3.PutObjectOutput>((resolve, reject) => {
            const handleError = (error) => {
                reject(error);
            };
            const chunks: Buffer[] = [];

            stream.on('data', (chunk: Buffer) => {
                chunks.push(chunk);
            });

            stream.once('end', async () => {
                const fileBuffer = Buffer.concat(chunks);

                try {
                    const uploaded = await this.putBlob(key, fileBuffer);

                    resolve(uploaded);
                } catch (error) {
                    handleError(new InternalServerErrorException(error));
                }
            });

            stream.on('error', (error) => handleError(new InternalServerErrorException(error)));
        });

        return file;
    }
}

Answer 5

Answer №3

To start, develop a class named AWSS3Utils with the following structure:

import * as multerS3 from 'multer-s3-transform'
import * as sharp from 'sharp'
import * as AWS from 'aws-sdk' 
export default class AWSS3Utils {
  
 static uploadFile(bucket:string,transform:boolean,acl:string)
  {
    return multerS3({
      s3: new AWS.S3({
        accessKeyId: 'accessKeyId'  ,
        secretAccessKey: 'secretAccessKey',
      }),
      bucket: bucket,
      shouldTransform: true,
      acl: acl,
      transforms: [
        {
          id: 'original',
          key: function (req, file, cb) {
            cb(null, `${file.originalname}` )
          },
          transform: function (req, file, cb) {
            cb(null, sharp().png())
          }
        },
        {
          id: 'large',
          key: (req, file, cb) => cb(null, new Date().getTime() + `_large_${file.originalname}`),
          transform: (req, file, cb) => cb(null, sharp().resize(1200, 900).png())
        },
        {
          id: 'small',
          key: (req, file, cb) => cb(null, new Date().getTime() + `_small_${file.originalname}`),
          transform: (req, file, cb) => cb(null, sharp().resize(400, 300).png())
        }
      ]
    })
  }
}

Now you can seamlessly integrate it with Nest FileInterceptor

 @Post('upload')
  @UseInterceptors(FileInterceptor('file', { storage: AWSS3Utils.uploadFile('nest-upload-tutorial',true,'public-read') }))
  uploadFile(
    @UploadedFile(
      new ParseFilePipe({
        validators: [new FileTypeValidator({ fileType: 'png' })],
      }),
    )
    file: Express.Multer.File,
  ) {
    return file
  }

Ensure to create a bucket with proper ACL permissions or omit the acl flag from the uploadSingle function.

Answer 6

To start, develop a class named AWSS3Utils with the following structure:

import * as multerS3 from 'multer-s3-transform'
import * as sharp from 'sharp'
import * as AWS from 'aws-sdk' 
export default class AWSS3Utils {
  
 static uploadFile(bucket:string,transform:boolean,acl:string)
  {
    return multerS3({
      s3: new AWS.S3({
        accessKeyId: 'accessKeyId'  ,
        secretAccessKey: 'secretAccessKey',
      }),
      bucket: bucket,
      shouldTransform: true,
      acl: acl,
      transforms: [
        {
          id: 'original',
          key: function (req, file, cb) {
            cb(null, `${file.originalname}` )
          },
          transform: function (req, file, cb) {
            cb(null, sharp().png())
          }
        },
        {
          id: 'large',
          key: (req, file, cb) => cb(null, new Date().getTime() + `_large_${file.originalname}`),
          transform: (req, file, cb) => cb(null, sharp().resize(1200, 900).png())
        },
        {
          id: 'small',
          key: (req, file, cb) => cb(null, new Date().getTime() + `_small_${file.originalname}`),
          transform: (req, file, cb) => cb(null, sharp().resize(400, 300).png())
        }
      ]
    })
  }
}

Now you can seamlessly integrate it with Nest FileInterceptor

 @Post('upload')
  @UseInterceptors(FileInterceptor('file', { storage: AWSS3Utils.uploadFile('nest-upload-tutorial',true,'public-read') }))
  uploadFile(
    @UploadedFile(
      new ParseFilePipe({
        validators: [new FileTypeValidator({ fileType: 'png' })],
      }),
    )
    file: Express.Multer.File,
  ) {
    return file
  }

Ensure to create a bucket with proper ACL permissions or omit the acl flag from the uploadSingle function.

How to successfully upload an image to AWS S3 using NestJS?

Answer №1

Answer №2

Answer №3

Similar questions

Utilizing Dependency Injection with TypeScript and Angular 5 in an Abstract Class

Can you explain the significance of declaring messages as a string array in an Angular class?

Retrieving the chosen option using a change event listener

Tips for successfully passing a closure as a parameter in a constructor

Using JSON data in an ArrayBuffer with TypeScript

Inversify: class-based contextual dependency injection

Angular 5's data display glitch

Utilizing AngularJS and TypeScript for seamless two-way data binding: a guide for synchronizing controller and directive interaction

The behavior of the Ionic checkbox in version 5 seems to be quite delayed

Creating a dynamic link for a button based on the selected value from a dropdown menu

Routing with nested modules in Angular 2 can be achieved by using the same

Uploading multiple strings to an Amazon S3 bucket using Node.js by piping a string

Is there a specific method for conducting a production build using AngularCLI rc.1?

Getting the local path of a file from an input file in Angular 7

Can you use setValidators() in Angular to validate two patterns simultaneously?

Error: The property 'combine' of 'winston_1.default.format' cannot be destructured since it is not defined

Angular - Is there a specific type for the @HostListener event that listens for scrolling on the window?

Limiting the combinations of types in TypeScript

Updating the main window in Angular after the closure of a popup window

I'm facing an issue with Angular2 where I am unable to include a component from