If you have an app on Elastic Beanstalk, and click on the ‘Logs’ option from the menu, you get an option to see either the last 100 lines of the log, or the Full logs. With either of these, you can only get logs till the point of download. In other words, these logs are not real time, only historical.
Now, there may be scenarios where you may want to see in real time what’s really happening. This is especially true if you are facing a bug that needs to be addressed quickly. Therefore, real time streaming of logs becomes a necessity. One way to do that is through SSH. You just SSH into the EC2 instance, locate the relevant log file and stream it. A tutorial for the same is provided here.
However, there may be cases where you may not want to use SSH. Sometimes you may not have enough privileges. Sometimes, your organization would like to avoid SSH into a production instance. How can logs still be streamed? Let’s find out.
Step 1: Enable log streaming to CloudWatch logs
Go to your Elastic Beanstalk environment in the AWS Console. From the left menu, click on ‘Configuration’ and against the ‘Software’ category, click on ‘Edit’.
Now, click on the checkbox to enable the streaming of logs to CloudWatch.
You can specify the number of days for which you wish to retain the logs, and also what happens to the logs if the environment itself is terminated. The longer you retain the logs, the higher will be the storage charges incurred.
This will start streaming the logs to AWS CloudWatch. If you then go to CloudWatch on the AWS Console, under ‘Log groups’, you should be able to see the new groups created for your Elastic Beanstalk environment.
Step 2: Stream the logs on AWS CLI
Now that your logs are getting stored in CloudWatch, the next step is pretty simple. All you need to do is use the AWS CLI to stream the log group of your interest. Now, the applications logs will typically be found in the /web.stdout.log group, for languages like Node.js and Python. This may however change depending on the language used for your app. If you are unsure, you can download the last 100 lines of the logs from the Elastic Beanstalk console, and see which group contains your application logs.
Once you have identified the correct group, you can stream all the logs posted to that group by running the following CLI command:
aws logs tail $group_name --followReplace the group name with the fill name you see in the CloudWatch console. With reference to the image above, the command will be:
aws logs tail /aws/elasticbeanstalk/<app_name>-env/var/log/web.stdout.log --followThat’s it. Logs will now start streaming in your terminal.
I hope you liked this article. For more tutorials on AWS, check out https://iotespresso.com/category/aws/.