AWS IoT Core – Using Basic Ingest to reduce costs

Introduction

If you pay attention to the AWS IoT Core pricing, you will notice the following (up to a billion messages, for us-east-2):

  1. Messaging cost: $1/million messages
  2. Rules triggered: $0.15/million rules triggered/million actions executed
  3. Actions executed: $0.15/million rules triggered/million actions executed

Therefore, if you have a certain group of messages which always trigger one rule and execute one action, you will pay ($1+$0.15+$0.15 = $1.30) for a million messages sent to AWS IoT Core. As you can see, the messaging charges are nearly 77% of the total charges you bear. This percentage would be even higher if not all rule triggers lead to action. Wouldn’t it be nice if you could somehow bypass the messaging charges? Basic Ingest lets you do exactly that.

If you are sure that a certain message will definitely trigger a rule, then instead of sending the message to the topic, you can directly send the message to the corresponding rule. This way, you will avoid the messaging cost, and only pay for the rule triggering and action execution.

How do you implement basic ingest? You simply post the message to the following topic:

$aws/rules/rule-name/topic

where rule-name is the name of the rule you defined for that message, and topic is the name of the topic to which the message was originally going to be published

Example

In the Triggering a Lambda Using Rules in AWS IoT Core article, we defined a rule lambda_trigger on the esp32/pub topic. The SQL for the rule was:

SELECT * FROM ‘esp32/pub’ where value>800

Now, with basic ingest, all I need to do is post to $aws/rules/lambda_trigger/esp32/pub instead of posting to esp32/pub. Note that the thing should have the publish permissions on this new topic. You can edit the policy document of the thing you will use for testing, and add the following (replace your-region and your-account with the correct values):

{
      "Effect": "Allow",
      "Action": "iot:Publish",
      "Resource": "arn:aws:iot:your-region:your-account:topic/$aws/rules/*"
}

Once the publish permission is provided, and the topic is changed in the thing code, you will see that there are no messages sent to the original topic (esp32/pub). You can check on MQTT Test Client.

Original Topic

However, if you check the lambda invocations, you will find that it got invoked whenever the value crossed 800.

Lambda Invocation

Thus, we are bypassing the messaging and directly dealing with the rules and actions. Thus, basic ingest works!

Refer to this AWS reference document for more information on using basic ingest.


I hope you liked this article. For more tutorials on AWS, check out https://iotespresso.com/category/aws/. Also, if you are planning to become a certified AWS Solutions Architect, I’d recommend that you check out this course on Udemy. I took this course and found the lectures to be lucid, to-the-point, and fun. I hope they will help you as well.

Leave a comment

Your email address will not be published. Required fields are marked *