In this article, I will explain how you can Notify a Pagerduty oncall by mentioning @oncall in Slack channel with help of Hubot. If you are a DevOps, Developer or any engineer who works on infrastructure which demands high availability and stability, you must be familiar with Pagerduty. Pagerduty takes care of oncall management and notify the Oncall person via Phone call, SMS, Mail …etc. And an Oncall is someone who acts on alerts which is received via Pagerduty notification methods. I think I don’t have to explain much more about Oncall person because nowadays every company has oncall culture.
Assume you are a developer, and you are seeing app owned by you is not behaving as usual. Now you want to talk to the DevOps Oncall to check for any system level issues. You know the Slack channel for DevOps, and you messaged there about this abnormality. If we don’t know the DevOps Oncall name, generally people mention @here or @channel to notify everybody in the slack channel. Why are we disturbing others, if someone can help you notify the DevOps Oncall alone when you mention @oncall in your message. Yes, we can setup Hubot to do this.
This funny Pic clearly explains what we are going to do here 😛
Note: Here Jarvis is name of the Hubot chatbot.
Step 1: Integrate Hubot with Slack
Before we start, you need to have Hubot setup and integrated with Slack. If you don’t, go through this easy tutorial from easyaslinux >>.
Step 2: Write a JavaScript to do the work
Once you have integrated Hubot with Slack, go to the scripts directory, create a JavaScript file at_oncall.js and add below content.
var request = require('request'); /* Your inputs here */ var pagerduty_api_key = "xxxxxxxxxxxxxxxxx" var escalation_policy_name = "xxxxxxxxxxxxxx" /* =================== */ function default_headers(url) { var headers = { url: url, method: 'GET', headers: { 'Authorization': 'Token token=' + pagerduty_api_key, 'Accept': 'application/vnd.pagerduty+json;version=2' } } return headers } function oncall(Id, msg) { url = 'https://api.pagerduty.com/oncalls' headers = default_headers(url) headers['qs'] = {'escalation_policy_ids[]': Id, "include[]": "users"} request(headers, function (error, response, body) { if (error) {} else { json_data = JSON.parse(body) for (i in json_data['oncalls']) { if (json_data['oncalls'][i]['escalation_level'] == '1') { var slack_username = (json_data['oncalls'][i]['user']['email']).split("@") msg.send('@' + slack_username[0]) } } } }) } function escalation_id(team, msg) { url = 'https://api.pagerduty.com/escalation_policies' headers = default_headers(url) headers['qs'] = {'query': team} request(headers, function (error, response, body) { if (error) { } else { json_data = JSON.parse(body) if (json_data['escalation_policies'].length > 0) { Id = json_data['escalation_policies'][0]['id'] oncall(Id,msg) } } }) } module.exports = function(robot) { robot.hear(/@oncall/i, function(msg) { escalation_id(escalation_policy_name,msg) } ); }
Note: I developed this script assuming that same username is used to authenticate both Pagerduty and Slack.
You need to give values for two variables pagerduty_api_key and escalation_policy_name to get the script working.
var pagerduty_api_key = "xxxxxxxxxxxxxxxxx" #Generate Pagerduty API key from here >> var escalation_policy_name = "xxxxxxxxxxxxxx" #input pagerduty escalation policy name here
What the JavaScript does:
- Hubot hear for @oncall mention in the slack.
- Once someone mention @oncall , script finds escalation policy ID by making an API call to Pagerduty RestAPI with Escalation policy name as input.
- Then gets Oncall person’s login Email ID from Pagerduty RestAPI by making another API call.
- Then split the email ID with ”@” and first part is sent back to slack(After prefix ‘@’ added).
- And Jarvis put the message in the channel.
Step 3: Install required NPM packages
Fortunately, we need to install only one node.js package for the script which is request package. Go to the directory where bot installed and run below command.
npm install request --save
Now we have everything ready. Let’s restart Hubot service to detect the newly added script.
sudo systemctl restart hubot
Step 4: Let’s test it
Now go to slack channel where Hubot chatbot is present, ask Oncall to check something and see what happens.
See Hubot has tagged the Oncall person for you. You don’t have to go around and ask “who is oncall?”. This will be useful for managers also. You can customise the JavaScript to tag Dev/DevOps oncall, tag oncall based on the channel …etc. Do explore!
Thanks for the time taken to read my blog. Subscribe to this blog so that you don’t miss out anything useful (Checkout Right Sidebar for the Subscription Form) . Please also put your thoughts on this article as comments .