We will continue today to talk about one specific service from AWS, that helps us to decouple services and applications. This article will learn about a particular type of SQS queue called First In, First Out (FIFO). In our last article, we learned about the standard SQS queue. However, we didn’t cover the message ordering and message duplication because it makes more sense to discuss them when we are analyzing the FIFO.
SQS FIFO vs Standard
So, there are two topics that we have not entirely covered with SQS in our previous article. Let’s go ahead and learn that right now. First, let’s consider this situation:
We have a standard SQS queue, and we sent five messages. Each message contains a sequence number, like the order in the draw below:
So, when we polled SQS, they went in 1, 2, 3, 4, 5, but they might come out 3, 5, 1, 2, 4. Also, we might get a duplicated message. But it doesn’t happen very often. However, we don’t have the guarantee that the order will be kept when we poll the queue.
| Standard | FIFO |
| Not Ordering | Ordering |
| Duplication | No duplication |
| Unlimited Messages per Seconds | 300 messages/s |
So, what is the impact if we don’t receive that message in the order that we sent? And, if we got duplicated messages? So, you should analyze the kind of question in your scenario and your requirements.
For example, more than once could be a problem for us depending on our application.
Let’s imagine we are building one website that will display statistics from a specific hashtag on Twitter. And we would like to show it in real-time or close to it every time we receive one message with the hashtag. Here, it doesn’t matter the order to use the standard SQS queue.
But let’s analyze another scenario.
Suppose that we are working on the backend service for a big bank. That receives online payments. What happens if one customer pays for your order and this transaction is processed twice on our backend, or if this transaction is not in the proper order in a report, for sure, that customer wouldn’t be happy.
First In, First Out.
If we sent five messages on this order 1,2,3,4,5 using FIFO queue.
When we poll the messages, they come out 1, 2, 3, 4.
So, FIFO ensures that ordering and FIFO ensure no duplicate messages.
Right now, you may be thinking, ok… So why would we use the standard queue if the SQS FIFO looks pretty better?
Well, it depends, let’s analyze the main differences between them.
So, using the standard SQS queue, we might get duplicated messages and best-effort ordering. There is no guarantee.
However, the SQS Standard offers practically unlimited transactions per second, implying that we can have several instances of reading and writing, and we will not face any issues.
On another side, with FIFO, the order and no duplication is guaranteed.
However, it restricts you to approximately 300 messages per second. Also, it’s more expensive than the standard because AWS performs all the hard work to keep the message in the same order and avoid duplication.
So, FIFO is easier because we don’t need to manage it, but it may increase the cost and can reach the throughput.
Let’s go through some configurations we should know how to use: the throughput limit, duplication, and message group ID.
Message group ID
Message Group ID identifies that all messages belong to a specific set. So when we pull that Set, none of the messages are duplicated. All of the messages are in their original sequence. Now, there are some extra fields that we have to specify its value. The final configuration is the message deduplication ID.
How can we request messages with Group ID?
We can’t request to obtain messages with a specific message group ID.
When receiving messages from a FIFO queue with numerous message group IDs, AWS SQS first tries to retrieve multiple messages with the identical message group ID as feasible. This permits different consumers to process messages with a distinct message group ID. So, we receive a message with a message group ID, and no additional messages for the exact message group ID are retrieved unless we remove the message or it becomes visible.
Message Deduplication ID
This specific value guarantees that all of your messages aren’t duplicated.
The deduplication gap is a five-minute window, where if SQS sees the same ID numerous times, the SQS will receive the messages successfully. Still, SQS will not send these messages to the consumer.
For example, for one message with a group ID of 1 and a deduplication ID of 1, if we send this message multiple times, the consumer will only receive the first message.
Amazing!
So, we have learned the main difference and when to employ standard queues versus when to employ FIFO.
Conclusion and Exam Tips
It’s worth highlighting that FIFO is the only option that comes to message order. So if you see a question about that message ordering or replication of the messages being an issue, you should be peeking for FIFO in the solutions.
The FIFO does not have the highest performance. If you see some questions about a scenario where we require thousands of transactions per second, the FIFO queue is not going to be the correct answer for you because the limit is about 300 messages per second.
One rule that must never forget is that the name of a FIFO queue must use the .fifo suffix. The suffix counts towards the 80-character queue name quota. So, if we need to know if one queue is FIFO or not, we can look if the name ends with .fifo.
Remember that message group ID guarantees that all messages with that same ID are bundled into one batch when we poll and process in the proper order.
However, there is no promise the SQS will process those batches of messages in the order. So, the sets might arrive out of order, but the messages inside of those batches will consistently be in order.
Always think about the cost. Just recall: SQS FIFO does cost us more because AWS is spending that computation power to guarantee the non-duplication of those messages and the order of those messages.
Next Topics About SQS
To take advantage of AWS SQS, understand what is the standard SQS and when we need to use the SQS Dead-Letter Queue.
