SQS FIFO vs Standard Queue: A Complete Guide

We will continue our exploration of AWS services by diving into a specific type of SQS queue called First In, First Out (FIFO). This queue type helps us decouple services and applications with guaranteed ordering. While our previous article covered the standard SQS queue, we’ll now focus on message ordering and duplication handling in FIFO queues.

Key Differences: SQS FIFO vs Standard

Let’s consider a practical example to understand the differences. In a standard SQS queue, if we send five messages in sequence (1, 2, 3, 4, 5):

When polling from a standard queue, messages might come out in any order (e.g., 3, 5, 1, 2, 4) and could potentially be duplicated. Here’s a quick comparison:

When to Choose Each Queue Type?

Use Standard Queue When:

• Order doesn’t matter (e.g., processing social media hashtags)
• You need high throughput
• Cost is a primary concern

Use FIFO Queue When:

• Order is critical (e.g., financial transactions)
• You need exactly-once processing
• Your throughput is under 300 messages/sec

First In, First Out.

If we sent five messages on this order 1,2,3,4,5 using FIFO queue.

When we poll the messages, they come out 1, 2, 3, 4.

So, FIFO ensures that ordering and FIFO ensure no duplicate messages.

Right now, you may be thinking, ok… So why would we use the standard queue if the SQS FIFO looks pretty better?

Well, it depends, let’s analyze the main differences between them.

So, using the standard SQS queue, we might get duplicated messages and best-effort ordering. There is no guarantee.

However, the SQS Standard offers practically unlimited transactions per second, implying that we can have several instances of reading and writing, and we will not face any issues.

On another side, with FIFO, the order and no duplication is guaranteed.

However, it restricts you to approximately 300 messages per second. Also, it’s more expensive than the standard because AWS performs all the hard work to keep the message in the same order and avoid duplication.

So, FIFO is easier because we don’t need to manage it, but it may increase the cost and can reach the throughput.

Understanding FIFO Queue Features

1. Message Group ID

Message Group ID identifies that all messages belong to a specific set. So when we pull that Set, none of the messages are duplicated. All of the messages are in their original sequence. Now, there are some extra fields that we have to specify its value. The final configuration is the message deduplication ID.

Important: You cannot request messages with a specific group ID directly.

2. Message Deduplication ID

This specific value guarantees that all of your messages aren’t duplicated.

The deduplication gap is a five-minute window, where if SQS sees the same ID numerous times, the SQS will receive the messages successfully. Still, SQS will not send these messages to the consumer.

For example, for one message with a group ID of 1 and a deduplication ID of 1, if we send this message multiple times, the consumer will only receive the first message.

Amazing!

So, we have learned the main difference and when to employ standard queues versus when to employ FIFO.

Conclusion and Exam Tips

It’s worth highlighting that FIFO is the only option that comes to message order. So if you see a question about that message ordering or replication of the messages being an issue, you should be peeking for FIFO in the solutions.

The FIFO does not have the highest performance. If you see some questions about a scenario where we require thousands of transactions per second, the FIFO queue is not going to be the correct answer for you because the limit is about 300 messages per second.

One rule that must never forget is that the name of a FIFO queue must use the .fifo suffix. The suffix counts towards the 80-character queue name quota. So, if we need to know if one queue is FIFO or not, we can look if the name ends with .fifo.

Remember that message group ID guarantees that all messages with that same ID are bundled into one batch when we poll and process in the proper order.

However, there is no promise the SQS will process those batches of messages in the order. So, the sets might arrive out of order, but the messages inside of those batches will consistently be in order.

Always think about the cost. Just recall: SQS FIFO does cost us more because AWS is spending that computation power to guarantee the non-duplication of those messages and the order of those messages.

Next Topics About SQS

To take advantage of AWS SQS, understand what is the standard SQS and when we need to use the SQS Dead-Letter Queue.

Become AWS Expert

What is High Availability on AWS?