|
| 1 | +# Topics and Queues Cross-Service Scenario |
| 2 | + |
| 3 | +## Overview |
| 4 | + |
| 5 | +Publish and subscribe is a mechanism for passing information. It’s used in social media, and it’s also used internally in software applications. A producer publishes a message, and the subscribers receive the message. In software, publish and subscribe notifications make message passing flexible and robust. The producers of messages are decoupled from the consumers of messages. |
| 6 | + |
| 7 | +Use the sample code in this folder to explore publishing and subscribing to a topic by using filters and queues. This tutorial does not create a complete end-to-end application. Instead, you can use it to play around with a publish and subscribe architecture. |
| 8 | + |
| 9 | +You can create an Amazon Simple Notification Service (Amazon SNS) topic and subscribe two Amazon Simple Queue Service (Amazon SQS) queues to the topic. You can enable FIFO (First-In-First-Out) queueing, and you can add filtered subscriptions. Then, you can publish messages to the topic and see the results in the queues. |
| 10 | + |
| 11 | +You can publish and subscribe using Amazon SNS alone. But combining Amazon SNS with Amazon SQS gives you more flexibility in how the messages are consumed. |
| 12 | + |
| 13 | +Amazon SNS is a push service. It pushes to endpoints such as email addresses, mobile application endpoints, or SQS queues. (For a full list of endpoints, see [SNS event destinations](https://docs.aws.amazon.com/sns/latest/dg/sns-event-destinations.html)). |
| 14 | + |
| 15 | +With Amazon SQS, messages are received from a queue by polling. With polling, the subscriber receives messages by calling a receive message API. Any code can poll the queue. Also, the messages stay in the queue until you delete them. This gives you more flexibility in how the messages are processed. |
| 16 | + |
| 17 | +The sample code builds a command line application that asks you for input. This is implemented in multiple programming languages, and the interface can vary slightly between languages. The following shows the interface for the Python implementation. |
| 18 | + |
| 19 | +### Create an SNS topic |
| 20 | + |
| 21 | +``` |
| 22 | +Would you like to work with FIFO topics? (y/n): |
| 23 | +``` |
| 24 | + |
| 25 | +You configure FIFO (First-In-First-Out) topics when you create them. Choosing a FIFO topic enables other options, too. To learn more, see [FIFO topics example use case](https://docs.aws.amazon.com/sns/latest/dg/fifo-example-use-case.html). |
| 26 | + |
| 27 | + |
| 28 | +``` |
| 29 | +Use content-based deduplication instead of entering a deduplication ID? (y/n): |
| 30 | +``` |
| 31 | + |
| 32 | +Deduplication is only available for FIFO topics. Deduplication prevents the subscriber from responding more than once to events that are determined to be duplicates. If a message gets published to an SNS FIFO topic and it’s found to be a duplicate within the five-minute deduplication interval, the message is accepted but not delivered. For more information, see [Message deduplication for FIFO topics](https://docs.aws.amazon.com/sns/latest/dg/fifo-message-dedup.html). |
| 33 | + |
| 34 | +Content-based deduplication uses a hash of the content as a deduplication ID. If content-based deduplication is not enabled, you must include a deduplication ID with each message. |
| 35 | + |
| 36 | +``` |
| 37 | +Enter a name for your SNS topic: |
| 38 | +``` |
| 39 | + |
| 40 | +Topic names can have 1-256 characters. They can contain uppercase and lowercase ASCII letters, numbers, underscores, and hyphens. If you chose a FIFO topic, the application automatically adds a “.fifo” suffix, which is required for FIFO topics. |
| 41 | + |
| 42 | +### Create two SQS queues |
| 43 | + |
| 44 | +Now, configure two SQS queues to subscribe to your topic. Separate queues for each subscriber can be helpful. For |
| 45 | +instance, you can customize how messages are consumed and how messages are filtered. |
| 46 | + |
| 47 | +``` |
| 48 | +Enter a name for SQS queue #1: |
| 49 | +``` |
| 50 | + |
| 51 | +Queue names can have 1-80 characters. They can contain uppercase and lowercase ASCII letters, numbers, underscores, and hyphens. If you chose a FIFO topic, the application automatically adds a “.fifo” suffix, which is required for FIFO queues. |
| 52 | + |
| 53 | + |
| 54 | +``` |
| 55 | +Filter messages for <queue name>'s subscription to the topic? (y/n): |
| 56 | +``` |
| 57 | + |
| 58 | +If you chose FIFO topics, you can add a filter to the queue’s topic subscription. There are many ways to filter a topic. In this example code, you have the option to filter by a predetermined selection of attributes. For more information about filters, see [Message filtering for FIFO topics](https://docs.aws.amazon.com/sns/latest/dg/fifo-message-filtering.html). |
| 59 | + |
| 60 | + |
| 61 | +``` |
| 62 | +You can filter messages by one or more of the following TONE attributes. |
| 63 | +Enter a number to add a TONE filter, or enter 0 to stop adding filters. |
| 64 | + 1. cheerful |
| 65 | + 2. funny |
| 66 | + 3. serious |
| 67 | + 4. sincere |
| 68 | +Your choice: |
| 69 | +``` |
| 70 | + |
| 71 | +If you add a filter, you can select one or more “tone” attributes to filter by. When you’re done, enter “0’” to continue. |
| 72 | + |
| 73 | +The application now prompts you to add the second queue. Repeat the previous steps for the second queue. |
| 74 | + |
| 75 | +The following diagram shows the topic and queue options. |
| 76 | + |
| 77 | + |
| 78 | +After you create the topic and subscribe both queues, the application lets you publish messages to the topic. |
| 79 | + |
| 80 | + |
| 81 | +``` |
| 82 | +Enter a message to publish: |
| 83 | +``` |
| 84 | + |
| 85 | +All configurations include a message text. |
| 86 | + |
| 87 | + |
| 88 | +``` |
| 89 | +Enter a message group ID for this message: |
| 90 | +``` |
| 91 | + |
| 92 | +If this is a FIFO topic, then you must include a group ID. The group ID can contain up to 128 alphanumeric characters `(a-z, A-Z, 0-9)` and punctuation `(!"#$%&'()*+,-./:;<=>?@[\]^_``{|}~)`. |
| 93 | +For more information about group IDs, see [Message grouping for FIFO topics](https://docs.aws.amazon.com/sns/latest/dg/fifo-message-grouping.html). |
| 94 | + |
| 95 | + |
| 96 | +``` |
| 97 | +Enter a deduplication ID for this message: |
| 98 | +``` |
| 99 | + |
| 100 | +If this is a FIFO topic and content-based deduplication is not enabled, then you must enter a deduplication ID. The message deduplication ID can contain up to 128 alphanumeric characters `(a-z, A-Z, 0-9)` and punctuation `(!"#$%&'()*+,-./:;<=>?@[\]^_``{|}~)`. |
| 101 | + |
| 102 | + |
| 103 | +``` |
| 104 | +Add an attribute to this message? (y/n): |
| 105 | +``` |
| 106 | + |
| 107 | +If you added a filter to one of the subscriptions, you can choose to add a filtering attribute to the message. |
| 108 | + |
| 109 | + |
| 110 | +``` |
| 111 | +Enter a number for an attribute: |
| 112 | + 1. cheerful |
| 113 | + 2. funny |
| 114 | + 3. serious |
| 115 | + 4. sincere |
| 116 | +Your choice: |
| 117 | +``` |
| 118 | + |
| 119 | +Select a number for an attribute. |
| 120 | + |
| 121 | + |
| 122 | +``` |
| 123 | +Send another message? (y/n): |
| 124 | +``` |
| 125 | + |
| 126 | +You can post as many messages as you want. |
| 127 | + |
| 128 | +When you are done posting messages, the application polls the queues and displays their messages. |
| 129 | + |
| 130 | + |
| 131 | +## ⚠️ Important |
| 132 | + |
| 133 | +* Running this code might result in charges to your AWS account. |
| 134 | +* Running the tests might result in charges to your AWS account. |
| 135 | +* We recommend that you grant your code least privilege. At most, grant only the minimum permissions required to perform the task. For more information, see [Grant least privilege](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#grant-least-privilege). |
| 136 | +* This code is not tested in every AWS Region. For more information, see [AWS Regional Services](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services). |
| 137 | + |
| 138 | +## Prerequisites |
| 139 | + |
| 140 | +For general prerequisites, see the [README](../../README.md#prerequisites) in the `python` folder. |
| 141 | + |
| 142 | +### Install Python dependencies |
| 143 | + |
| 144 | +The requirements for this example are listed in the accompanying `requirements.txt` file. |
| 145 | + |
| 146 | +Install the requirements by running the following in a virtual environment: |
| 147 | + |
| 148 | +``` |
| 149 | +python -m pip install -r requirements.txt |
| 150 | +``` |
| 151 | + |
| 152 | +This installs the AWS SDK for Python (Boto3) and other required packages. |
| 153 | + |
| 154 | +## Instructions |
| 155 | + |
| 156 | +For general instructions to run the examples, see the |
| 157 | +[README](../../README.md#run-the-examples) in the `python` folder. |
| 158 | + |
| 159 | +Run this example by running the following command in the folder that contains this README: |
| 160 | + |
| 161 | +``` |
| 162 | +python topics_and_queues_scenario.py |
| 163 | +``` |
| 164 | + |
| 165 | +This starts an interactive scenario that walks you through creating topics and queues, |
| 166 | +publishing messages with filters, and viewing the results. |
| 167 | + |
| 168 | +Running this example requires AWS Identity and Access Management (IAM) permissions for both SNS and SQS. |
| 169 | + |
| 170 | +## Additional resources |
| 171 | + |
| 172 | +* [Amazon SNS Developer Guide](https://docs.aws.amazon.com/sns/latest/dg/welcome.html) |
| 173 | +* [Amazon SNS API Reference](https://docs.aws.amazon.com/sns/latest/api/welcome.html) |
| 174 | +* [Amazon SQS Developer Guide](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/welcome.html) |
| 175 | +* [Amazon SQS API Reference](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/Welcome.html) |
0 commit comments