vi
new
+7
Avatar
NGUYỄN ANH TUẤN @Clarence161095
21.9K 517 370
Đã đăng vào thg 1 17, 2023 3:06 SA 5 phút đọc
203

AWS SQS: The Ultimate Guide for Junior Developers

AWS Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications. SQS eliminates the complexity and overhead associated with managing and operating message oriented middleware, and empowers developers to focus on differentiating work.

Node.js is a JavaScript runtime built on Chrome's V8 JavaScript engine. It allows developers to run JavaScript on the server-side to create fast and scalable network applications. Node.js is widely used for creating back-end services, building real-time apps, and running automation tasks.

By using SQS and Node.js together, you can create powerful and flexible applications that can handle and process large amounts of data and perform multiple tasks simultaneously.

Setting up an SQS Queue in AWS

Before you can use SQS with Node.js, you need to create a new queue in AWS. You can do this by following these steps:

  1. Go to the SQS homepage in the AWS Management Console.
  2. Click the "Create Queue" button.
  3. Give your queue a name, and select the "Standard Queue" option.
  4. Click the "Create Queue" button to create your new queue.

Once your queue is set up, you can start sending and receiving messages to and from it.

Sending Messages to an SQS Queue in Node.js

To send a message to an SQS queue from a Node.js application, you need to use the AWS SDK for JavaScript in Node.js (aws-sdk). The SDK provides a client for SQS that you can use to interact with SQS using Node.js.

Here's an example of how to send a message to an SQS queue using the SDK:

const AWS = require('aws-sdk');
const sqs = new AWS.SQS();

// Create the message to send to the SQS queue
const message = {
  message: 'Hello, SQS!'
};

// Set the parameters for sending the message to the SQS queue
const params = {
  MessageBody: JSON.stringify(message),
  QueueUrl: 'https://sqs.us-east-1.amazonaws.com/1234567890/MyQueue'
};

// Send the message to the SQS queue
sqs.sendMessage(params, (err, data) => {
  if (err) {
    console.log(`Error sending message to SQS: ${err}`);
  } else {
    console.log(`Successfully sent message to SQS: ${data}`);
  }
});

Receiving and Processing Messages from an SQS Queue in Node.js

Once you've sent messages to your SQS queue, you can receive and process them in a Node.js application using the AWS SDK for JavaScript in Node.js (aws-sdk).

Here's an example of how to poll an SQS queue for new messages and process each message as it's received:

const AWS = require('aws-sdk');
const sqs = new AWS.SQS();

// Set the parameters for receiving messages from the SQS queue
const params = {
  QueueUrl: 'https://sqs.us-east-1.amazonaws.com/1234567890/MyQueue',
  MaxNumberOfMessages: 10
};

// Poll the SQS queue for new messages
setInterval(() => {
  sqs.receiveMessage(params, (err, data) => {
    if (err) {
      console.log(`Error receiving message from SQS: ${err}`);
    } else {
      console.log(`Successfully received message from SQS: ${data}`);
      // Process the message
      // ...
    }
  });
}, 5000);

In this example, the setInterval function is used to poll the SQS queue for new messages every 5 seconds. When a message is received, it is logged to the console and then processed by the application.

Use Cases

1. Background Job Processing

SQS can be used to process background jobs asynchronously in a Node.js application. For example, sending an email, generating reports, and cleaning up old data.

In this use case, you can use SQS to process background jobs asynchronously in a Node.js application. For example, sending an email:

const AWS = require('aws-sdk');
const sqs = new AWS.SQS();
const ses = new AWS.SES();

// Create the message to send to the SQS queue
const message = {
  email: 'example@example.com',
  subject: 'Hello, SQS!',
  body: 'This is a background job processed by SQS'
};

// Set the parameters for sending the message to the SQS queue
const params = {
  MessageBody: JSON.stringify(message),
  QueueUrl: 'https://sqs.us-east-1.amazonaws.com/1234567890/emailQueue'
};

// Send the message to the SQS queue
sqs.sendMessage(params, (err, data) => {
  if (err) {
    console.log(`Error sending message to SQS: ${err}`);
  } else {
    console.log(`Successfully sent message to SQS: ${data}`);
  }
});

setInterval(() => {
  sqs.receiveMessage({
    QueueUrl: 'https://sqs.us-east-1.amazonaws.com/1234567890/emailQueue',
    MaxNumberOfMessages: 10
  }, (err, data) => {
    if (err) {
      console.log(`Error receiving message from SQS: ${err}`);
    } else {
      const emailParams = {
        Destination: {
          ToAddresses: [data.Messages[0].email]
        },
        Message: {
          Body: {
            Text: {
              Charset: "UTF-8",
              Data: data.Messages[0].body
            }
          },
          Subject: {
            Charset: 'UTF-8',
            Data: data.Messages[0].subject
          }
        },
        Source: 'example@example.com',
      };
      ses.sendEmail(emailParams, (err, data) => {
        if (err) {
          console.log(`Error sending email: ${err}`);
        } else {
          console.log(`Successfully sent email: ${data}`);
        }
      });
      sqs.deleteMessage({
        QueueUrl: 'https://sqs.us-east-1.amazonaws.com/1234567890/emailQueue',
        ReceiptHandle: data.Messages[0].ReceiptHandle
      }, (err, data) => {
        if (err) {
          console.log(`Error deleting email from SQS: ${err}`);
        } else {
          console.log(`Successfully deleted email from SQS: ${data}`);
        }
      });
    }
  });
}, 5000);

2. Decoupling Microservices

SQS can be used to decouple microservices in a Node.js application by using SQS as a messaging bus between services. This allows services to operate independently and improves the scalability and availability of the overall application.

In this use case, you can use SQS as a messaging bus between different microservices in a Node.js application. This allows the microservices to operate independently and improves the scalability and availability of the overall application. Here's an example of how one microservice could send a message to another using SQS:

const AWS = require('aws-sdk');
const sqs = new AWS.SQS();

// Create the message to send to the SQS queue
const message = {
  message: 'This is a message from Microservice 1 to Microservice 2'
};

// Set the parameters for sending the message to the SQS queue
const params = {
  MessageBody: JSON.stringify(message),
  QueueUrl: 'https://sqs.us-east-1.amazonaws.com/1234567890/microserviceQueue'
};

// Send the message to the SQS queue
sqs.sendMessage(params, (err, data) => {
  if (err) {
    console.log(`Error sending message to SQS: ${err}`);
  } else {
    console.log(`Successfully sent message to SQS: ${data}`);
  }
});

3. Real-time Notification

SQS can be used to send real-time notifications to a Node.js application when certain events occur. For example, sending a notification when a new user signs up, or when a payment is received.

In this use case, you can use SQS to send real-time notifications to a Node.js application when certain events occur. For example, sending a notification when a new user signs up:

const AWS = require('aws-sdk');
const sqs = new AWS.SQS();

// Create the message to send to the SQS queue
const message = {
  message: 'A new user has signed up: John Doe'
};

// Set the parameters for sending the message to the SQS queue
const params = {
  MessageBody: JSON.stringify(message),
  QueueUrl: 'https://sqs.us-east-1.amazonaws.com/1234567890/notificationQueue'
};

// Send the message to the SQS queue
sqs.sendMessage(params, (err, data) => {
  if (err) {
    console.log(`Error sending message to SQS: ${err}`);
  } else {
    console.log(`Successfully sent message to SQS: ${data}`);
  }
});

// Poll the SQS queue for new messages
setInterval(() => {
  sqs.receiveMessage({
    QueueUrl: 'https://sqs.us-east-1.amazonaws.com/1234567890/notificationQueue',
    MaxNumberOfMessages: 10
  }, (err, data) => {
    if (err) {
      console.log(`Error receiving message from SQS: ${err}`);
    } else {
      console.log(`Received notification: ${data.Messages[0].message}`);
      // send this notification  to client
      //...
      sqs.deleteMessage({
        QueueUrl: 'https://sqs.us-east-1.amazonaws.com/1234567890/notificationQueue',
        ReceiptHandle: data.Messages[0].ReceiptHandle
      }, (err, data) => {
        if (err) {
          console.log(`Error deleting notification from SQS: ${err}`);
        } else {
          console.log(`Successfully deleted notification from SQS: ${data}`);
        }
      });
    }
  });
}, 5000);

4. Automated Workflows

SQS can be used to create automated workflows in a Node.js application. For example, processing a user's image and sending it to another service for analysis.

In this use case, you can use SQS to create automated workflows in a Node.js application. For example, processing a user's image and sending it to another service for analysis:

const AWS = require('aws-sdk');
const sqs = new AWS.SQS();
const rekognition = new AWS.Rekognition();

// Create the message to send to the SQS queue
const message = {
  imageUrl: 'https://example.com/image.jpg'
};

// Set the parameters for sending the message to the SQS queue
const params = {
  MessageBody: JSON.stringify(message),
  QueueUrl: 'https://sqs.us-east-1.amazonaws.com/1234567890/imageQueue'
};

// Send the message to the SQS queue
sqs.sendMessage(params, (err, data) => {
  if (err) {
    console.log(`Error sending message to SQS: ${err}`);
  } else {
    console.log(`Successfully sent message to SQS: ${data}`);
  }
});

setInterval(() => {
  sqs.receiveMessage({
    QueueUrl: 'https://sqs.us-east-1.amazonaws.com/1234567890/imageQueue',
    MaxNumberOfMessages: 10
  }, (err, data) => {
    if (err) {
      console.log(`Error receiving message from SQS: ${err}`);
    } else {
      const imageUrl = data.Messages[0].imageUrl;
      rekognition.detectLabels({
        Image: {
          S3Object: {
            Bucket: 'imageBucket',
            Name: imageUrl
          }
        },
        MinConfidence: 90
      }, (err, data) => {
        if (err) {
          console.log(`Error processing image: ${err}`);
        } else {
          console.log(`Image labels: ${data.Labels}`);
        }
      });
      sqs.deleteMessage({
        QueueUrl: 'https://sqs.us-east-1.amazonaws.com/1234567890/imageQueue',
        ReceiptHandle: data.Messages[0].ReceiptHandle
      }, (err, data) => {
        if (err) {
          console.log(`Error deleting image from SQS: ${err}`);
        } else {
          console.log(`Successfully deleted image from SQS: ${data}`);
        }
      });
    }
  });
}, 5000);

5. Load Balancing

SQS can be used as a load balancer in a Node.js application by routing incoming requests to multiple instances of the application. This allows the application to scale automatically to handle large amounts of traffic.

In this use case, you can use SQS as a load balancer in a Node.js application by routing incoming requests to multiple instances of the application. This allows the application to scale automatically to handle large amounts of traffic. Here's an example of how to route incoming requests to different instances of a Node.js application:

const AWS = require("aws-sdk");
const sqs = new AWS.SQS();

// Create the SQS queue
sqs.createQueue(
  {
    QueueName: "requestQueue",
  },
  (err, data) => {
    if (err) {
      console.log(`Error creating SQS queue: ${err}`);
    } else {
      console.log(`Successfully created SQS queue: ${data}`);
    }
  }
);

// Send a request to the SQS queue
const request = {
  requestUrl: "/example",
  requestMethod: "GET",
};

const params = {
  MessageBody: JSON.stringify(request),
  QueueUrl: "https://sqs.us-east-1.amazonaws.com/1234567890/requestQueue",
};

sqs.sendMessage(params, (err, data) => {
  if (err) {
    console.log(`Error sending request to SQS: ${err}`);
  } else {
    console.log(`Successfully sent request to SQS: ${data}`);
  }
});

// Poll the SQS queue for new requests
setInterval(() => {
  sqs.receiveMessage(
    {
      QueueUrl: "https://sqs.us-east-1.amazonaws.com/1234567890/requestQueue",
      MaxNumberOfMessages: 10,
    },
    (err, data) => {
      if (err) {
        console.log(`Error receiving request from SQS: ${err}`);
      } else {
        // Route request to a free instance of the application
        //...
        sqs.deleteMessage(
          {
            QueueUrl: "https://sqs.us-east-1.amazonaws.com/1234567890/requestQueue",
            ReceiptHandle: data.Messages[0].ReceiptHandle,
          },
          (err, data) => {
            if (err) {
              console.log(`Error deleting request from SQS: ${err}`);
            } else {
              console.log(`Successfully deleted request from SQS: ${data}`);
            }
          }
        );
      }
    }
  );
}, 5000);

As you can see in the code samples, AWS SQS can be used for various use cases like background job processing, decoupling microservices, real-time notifications, automated workflows, and load balancing. The code samples I've provided are just a starting point and can be customized to meet the specific requirements of your application.

Conclusion

In this article, we've discussed how AWS Simple Queue Service (SQS) and Node.js can be used together to create powerful and flexible applications. We've shown how to create an SQS queue in AWS, how to send and receive messages to and from the queue using Node.js, and we've provided real-world use cases where SQS can be used to solve common problems. Remember that SQS is a fully managed service and it can help you to build microservices and scalable applications.

I hope this information helps you to understand the basics of using SQS and Node.js together and give you a foundation for creating your own applications. If you want to learn more about SQS and Node.js, I recommend you check out the official AWS SQS and AWS SDK for Node.js documentation.

Mình hy vọng bạn thích bài viết này và học thêm được điều gì đó mới.

Donate mình một ly cafe hoặc 1 cây bút bi để mình có thêm động lực cho ra nhiều bài viết hay và chất lượng hơn trong tương lai nhé. À mà nếu bạn có bất kỳ câu hỏi nào thì đừng ngại comment hoặc liên hệ mình qua: Zalo - 0374226770 hoặc Facebook. Mình xin cảm ơn.

Momo: NGUYỄN ANH TUẤN - 0374226770

TPBank: NGUYỄN ANH TUẤN - 0374226770 (hoặc 01681423001)

image.png

Amazon Web Services (AWS) Node.js Simple English LEARN ENGLISH

All rights reserved

Mục lục


Viblo
Hãy đăng ký một tài khoản Viblo để nhận được nhiều bài viết thú vị hơn.
Đăng kí