Published: Mar 7, 2023 by Isaac Johnson

We have handled Slack and Github Issues thus far and we’ve leveraged Azure DevOps in the past as well. Today, let’s focus on JIRA and Discord. We’ll set up notifications on Slack, Discord, Keybase and Teams. Then we’ll tackle Datadog Metrics and wrap up with better error handling and alerting.

JIRA

There are several ways to update JIRA. We can go to start.atlassian.com. If you haven’t created an account, you can do so there. You’ll be given a start page with all the products available

There is a subtle distinction between JIRA Service Management and just JIRA. I’ll be using JIRA (with the Diamond Icon).

Here I created a basic project (MyJIRAProject)

We can use the Create button to make a new Issue or Task

Rest API

To use the REST API, we’ll need an API token. We can get that by going to user settings

We then click the link to create and manage API tokens

I’ll click “Create API Token”

We can give it a lable then click create

I’ll copy to the clipboard

We can refer to the REST Documentation

I tried using ‘user:pass’ as a base64 Bearer token, using the API token and base64’ed API token - all my tests trying to use --header "Authorization: Bearer (thing to try) failed. I have a fealing only a valid OAuth2 flow will work now.

However, using -u user:(API Token) worked.

$ curl --request GET --url "https://freshbrewed.atlassian.net/rest/agile/1.0/board/" -u 'isaac@freshbrewed.science':"sdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdf" --header 'Accept: application/json'
{"maxResults":50,"startAt":0,"total":1,"isLast":true,"values":[{"id":1,"self":"https://freshbrewed.atlassian.net/rest/agile/1.0/board/1","name":"TPK board","type":"kanban","location":{"projectId":10000,"displayName":"MyJIRAProject (TPK)","projectName":"MyJIRAProject","projectKey":"TPK","projectTypeKey":"software","avatarURI":"https://freshbrewed.atlassian.net/rest/api/2/universal_avatar/view/type/project/avatar/10425?size=small","name":"MyJIRAProject (TPK)"}}]}

Running through jq

{
  "maxResults": 50,
  "startAt": 0,
  "total": 1,
  "isLast": true,
  "values": [
    {
      "id": 1,
      "self": "https://freshbrewed.atlassian.net/rest/agile/1.0/board/1",
      "name": "TPK board",
      "type": "kanban",
      "location": {
        "projectId": 10000,
        "displayName": "MyJIRAProject (TPK)",
        "projectName": "MyJIRAProject",
        "projectKey": "TPK",
        "projectTypeKey": "software",
        "avatarURI": "https://freshbrewed.atlassian.net/rest/api/2/universal_avatar/view/type/project/avatar/10425?size=small",
        "name": "MyJIRAProject (TPK)"
      }
    }
  ]
}

It took me some testing, but I teased out the minimum set of fields for a new Issue

{
  "fields": {
    "description": {
      "content": [
        {
          "content": [
            {
              "text": "This is a test",
              "type": "text"
            }
          ],
          "type": "paragraph"
        }
      ],
      "type": "doc",
      "version": 1
    },
    "issuetype": {
      "id": "10001"
    },
    "labels": [
      "UserAsk"
    ],
    "project": {
      "id": "10000"
    },
    "summary": "This is a summary"
  },
  "update": {}
}

which when testing, created issue “TPK-3”

$ curl --request POST --url "https://freshbrewed.atlassian.net/rest/api/3/issue" -u 'isaac@freshbrewed.science':"aasdfsadfasdfsadfasdf" --header 'Accept: application/json' --header 'Content-Type: application/json' -d @jiraIssueBasic.json 
{"id":"10002","key":"TPK-3","self":"https://freshbrewed.atlassian.net/rest/api/3/issue/10002"}

The nice thing about doing it this way with the label is that we can organize all the external customer asks by searching on lable

So that I can now add that to flows, I’ll put the API key in the Azure Key Vault

$ az keyvault secret set --vault-name wldemokv --name jiraapitoken --description "GH Token. fb.atlassian.net" --value asdfasdfasdfasdfasdfasdfasdfasdfasdfasdf

I can create a basic step in the Github Workflow for processing form data. I likely need to optimize and sanitize.

      - name: 'JIRA Create'
        run: |
          az keyvault secret show --vault-name wldemokv --name jiraapitoken -o json | jq -r .value > jiraapitoken

          # Create in Hosted Atlassian
          cat >jiraJSON <<EOT
          {
            "fields": {
              "description": {
                "content": [
                  {
                    "content": [
                      {
                        "text": "$ :: Requested by $",
                        "type": "text"
                      }
                    ],
                    "type": "paragraph"
                  }
                ],
                "type": "doc",
                "version": 1
              },
              "issuetype": {
                "id": "10001"
              },
              "labels": [
                "UserAsk"
              ],
              "project": {
                "id": "10000"
              },
              "summary": "$"
            },
            "update": {}
          }
          EOT

          curl --request POST --url "https://freshbrewed.atlassian.net/rest/api/3/issue" -u 'isaac@freshbrewed.science':"`cat jiraapitoken | tr -d '\n'`" --header 'Accept: application/json' --header 'Content-Type: application/json' -d @jiraJSON 

Then I’ll push to main and give it a test from the Feedback form

This is equivalent to just using a curl call:

$ curl -X POST -H "Accept: application/vnd.github+json" -H "Authorization: Bearer `az keyvault secret show --vault-name idjakv --name GithubToken-MyFull90d -o json | jq -r .value | tr -d '\n'`" -H "X-GitHub-Api-Version: 2022-11-28" https://api.github.com/repos/idjohnson/workflowTriggerTest/dispatches -d '{"event_type":"on-demand-payload","client_payload":{"summary":"this is the summary","description":"this is the descriptionn","userid":"isaac@freshbrewed.science"}}'

We can then see the result

Discord

Making a webhook notification is quite easy. We just need to click the Gear icon next to a channel.

From there we can click on the Integrations settings and go to webhooks

Click on new webhook to create a new one

I’ll give it an icon and name

then copy the Webhook URL

I’ll test

curl -X POST -H "Content-Type:application/json" --data '{"content": "Hello World"}' https://discord.com/api/webhooks/1081944728741888090/asdfsadfasdfasdfasdfasdfsadfsadfasdfsadfasdfsadf

And we can see that after I save the icon change, we get a nice Hello world message

As before, I’ll set an AKV secret to the Discord URL

$ az keyvault secret set --vault-name wldemokv --name discordwhu
rlgeneral --description "PrincessKing - General channel - GH notifications" --value 'https://discord.com/api/webhooks/10
81944728741888090/asdfsadfasdfasdfasdfasdfasdfasdfasdfasdfp'

Then I can add it to the Feedback form pipeline

I’ll add some blocks for positive

          # Discord
          discorddata="$( jq -nc --arg content "Feature Request $GHURL: $ Requested by $" '$ARGS.named')"
          cat >discorddata.json <<EOTT
          $discorddata
          EOTT

          curl -X POST -H "Content-Type:application/json" -d @discorddata.json `az keyvault secret show --vault-name wldemokv --name discordwhurlgeneral -o json | jq -r .value | tr -d '\n'`

and in the failure block as well

          # Discord Notification
          discorddata="$( jq -nc --arg content "Feature Request Issue Create Failure: $ Requested by $" '$ARGS.named')"
          cat >discorddata.json <<EOTT
          $discorddata
          EOTT

          curl -X POST -H "Content-Type:application/json" -d @discorddata.json `az keyvault secret show --vault-name wldemokv --name discordwhurlgeneral -o json | jq -r .value | tr -d '\n'`

I’ll give a test

curl -X POST -H "Accept: application/vnd.github+json" -H "Authorization: Bearer `az keyvault secret show --vault-name idjakv --name GithubToken-MyFull90d -o json | jq -r .value | tr -d '\n'`" -H "X-GitHub-Api-Version: 2022-11-28" https://api.github.com/repos/idjohnson/workflowTriggerTest/dispatches -d '{"event_type":"on-demand-payload","client_payload":{"summary":"this is the summary","description":"this is the descriptionn","userid":"isaac@freshbrewed.science"}}'

Which properly sent a notificaton

Teams

Teams webhook are quite similar.

We’ll click the 3 dots and choose connectors

We’ll configure an incoming webhook

However, I just get errors…

It crashed even on a reinstall

And on the web browser…

I went back and tried every Teams URL I still had in my passwords and old builds.

In all cases they vomited as well

$ curl -X POST -H "Content-Type: application/json" -d @post.json https://outlook.office.com/webhook/ad403e91-2f21-4c3c-92e0-02d7dce003ff@ad403e91-2f21-4c3c-92e0-02d7dce003ff/IncomingWebhook/ad403e91-2f21-4c3c-92e0-02d7dce003ff/ad403e91-2f21-4c3c-92e0-02d7dce003ff
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"/>
<title>500 - Internal server error.</title>
<style type="text/css">
<!--
body{margin:0;font-size:.7em;font-family:Verdana, Arial, Helvetica, sans-serif;background:#EEEEEE;}
fieldset{padding:0 15px 10px 15px;} 
h1{font-size:2.4em;margin:0;color:#FFF;}
h2{font-size:1.7em;margin:0;color:#CC0000;} 
h3{font-size:1.2em;margin:10px 0 0 0;color:#000000;} 
#header{width:96%;margin:0 0 0 0;padding:6px 2% 6px 2%;font-family:"trebuchet MS", Verdana, sans-serif;color:#FFF;
background-color:#555555;}
#content{margin:0 0 0 2%;position:relative;}
.content-container{background:#FFF;width:96%;margin-top:8px;padding:10px;position:relative;}
-->
</style>
</head>
<body>
<div id="header"><h1>Server Error</h1></div>
<div id="content">
 <div class="content-container"><fieldset>
  <h2>500 - Internal server error.</h2>
  <h3>There is a problem with the resource you are looking for, and it cannot be displayed.</h3>
 </fieldset></div>
</div>
</body>
</html>

Getting past the errors…

However, I went to an infrequently accessed channel. One I rarely use for notifications (General).

I chose Connectors

Then configure … here I usually got a crash

I gave it a name and Icon, then clicked Create

The URL looked different than my prior ones, so perhaps something happened in a bigger way behind the scenes when I was forced to migrate Teams accounts.

https://princessking.webhook.office.com/webhookb2/asdfasdfasdfasdfasdf@asdfasdfasdfasdasdfasdfasdfasdf/IncomingWebhook/asdfasdfasdfasdfasdfasdfasdfasdfasdasdasdasdf

I’ll fire a test post to it

$ cat post.json 
{
        "@type": "MessageCard",
        "@context": "https://schema.org/extensions",
        "summary": "1 new build message",
        "themeColor": "0078D7",
        "sections": [
                {
                        "activityImage": "https://ca.slack-edge.com/T4AQPQN8M-U4AL2JFC3-g45c8854734a-48",
                        "activityTitle": "My Title",
                        "activitySubtitle": "My Subtitle",
                        "facts": [
                                {
                                        "name": "Keywords:",
                                        "value": "Some Keywords"
                                },
                                {
                                        "name": "Group:",
                                        "value": "Some Group"
                                }
                        ],
                        "text": "Build 1234 has completed. Pipeline started at . The world now has just that much more joy in it.\n<br/>\n  Peace and love.",
                        "potentialAction": [
                                {
                                        "@type": "OpenUri",
                                        "name": "View conversation"
                                }
                        ]
                }
        ]
}
$ curl -X POST -H "Content-Type: application/json" -d @post.json https://princessking.webhook.office.com/webhookb2/asdfasdfasdfasdfasdf@asdfasdfasdfasdasdfasdfasdfasdf/IncomingWebhook/asdfasdfasdfasdfasdfasdfasdfasdfasdasdasdasdf

And see that it works

Since I’m already in the AKV UI, I’ll just create a manual secret there

I created a block for passing Forms:

          # Teams
          export SUMMARY="`jq -nc --arg text "$" '$ARGS.named' | jq -r .text | sed 's/[^[:alnum:]_-]/ /g'`"
          export DESCRIPTION="`jq -nc --arg text "$" '$ARGS.named' | jq -r .text | sed 's/[^[:alnum:]_-]/ /g'`"

          cat > ./post.json <<endmsg
          {
                  "@type": "MessageCard",
                  "@context": "https://schema.org/extensions",
                  "summary": "New Feedback Form",
                  "themeColor": "0078D7",
                  "sections": [
                          {
                                  "activityImage": "https://ca.slack-edge.com/T4AQPQN8M-U4AL2JFC3-g45c8854734a-48",
                                  "activityTitle": "Feedback Form Submission",
                                  "activitySubtitle": "New Issue",
                                  "facts": [
                                          {
                                                  "name": "Requestor:",
                                                  "value": "$"
                                          },
                                          {
                                                  "name": "Summary:",
                                                  "value": "$SUMMARY"
                                          }
                                  ],
                                  "text": "$GHURL - $DESCRIPTION",
                                  "potentialAction": [
                                          {
                                                  "@type": "OpenUri",
                                                  "name": "View conversation"
                                          }
                                  ]
                          }
                  ]
          }
          endmsg

          curl -X POST -H "Content-Type: application/json" -d @post.json `az keyvault secret show --vault-name wldemokv --name msteamsgeneralfeedbackform -o json | jq -r .value | tr -d '\n'`

and ones that error

          curl -X POST -H "Content-Type:application/json" -d @keybasedata.json `az keyvault secret show --vault-name wldemokv --name keybasebottpk -o json | jq -r .value | tr -d '\n'`
          # Teams
          export SUMMARY="`jq -nc --arg text "$" '$ARGS.named' | jq -r .text | sed 's/[^[:alnum:]_-]/ /g'`"
          export DESCRIPTION="`jq -nc --arg text "$" '$ARGS.named' | jq -r .text | sed 's/[^[:alnum:]_-]/ /g'`"

          cat > ./post.json <<endmsg
          {
                  "@type": "MessageCard",
                  "@context": "https://schema.org/extensions",
                  "summary": "New Feedback Form (error)",
                  "themeColor": "0078D7",
                  "sections": [
                          {
                                  "activityImage": "https://ca.slack-edge.com/T4AQPQN8M-U4AL2JFC3-g45c8854734a-48",
                                  "activityTitle": "Feedback Form Submission (error)",
                                  "activitySubtitle": "New Issue",
                                  "facts": [
                                          {
                                                  "name": "Requestor:",
                                                  "value": "$"
                                          },
                                          {
                                                  "name": "Summary:",
                                                  "value": "$SUMMARY"
                                          }
                                  ],
                                  "text": "$DESCRIPTION",
                                  "potentialAction": [
                                          {
                                                  "@type": "OpenUri",
                                                  "name": "View conversation"
                                          }
                                  ]
                          }
                  ]
          }
          endmsg

          curl -X POST -H "Content-Type: application/json" -d @post.json `az keyvault secret show --vault-name wldemokv --name msteamsgeneralfeedbackform -o json | jq -r .value | tr -d '\n'`

Keybase.io

Let’s go to Keybase.io and add a team

I debated between the first and last option. I decided to make it the first (A Squad)

I’ll give it a name

Then I’ll add some team members

I can add from a variety of sources. For now, I’ll just add myself

We can now see the channel listed among my other private chats

I can now click the gear icon on the right hand side of the Team name and choose “Team Info”

From there, I go to the Bots tab and Install Bot

Then search for and add the Webhook bot

I’ll review and Install

We can now use the Bot to create a Webhook for us with the !webhook create command

The bot, when successful, will then send the link to the URL directly to me

I can then test with curl

$ curl -X POST  --header 'Accept: application/json' --data '{"msg":"This is a test"}' https://bots.keybase.io/webhookbot/asdfasdfasdfasdfasdf

And set the alert show up

I’ll now set in AKV

$  az keyvault secret set --vault-name wldemokv --name keybasebottpk --value 'https://bots.keybase.io/webhookbot/asdfasdfsadfasdfasdfasdfasdf   '

I’m sure you see the pattern.

          # Keybase
          keybasedata="$( jq -nc --arg msg "Feature Request $GHURL: $ Requested by $" '$ARGS.named')"
          cat >keybasedata.json <<EOTT
          $keybasedata
          EOTT

          curl -X POST -H "Content-Type:application/json" -d @keybasedata.json `az keyvault secret show --vault-name wldemokv --name keybasebottpk -o json | jq -r .value | tr -d '\n'`

and a similar failure message

          # Keybase
          keybasedata="$( jq -nc --arg msg "Feature Request Issue Create Failure: $ Requested by $" '$ARGS.named')"
          cat >keybasedata.json <<EOTT
          $keybasedata
          EOTT

          curl -X POST -H "Content-Type:application/json" -d @keybasedata.json `az keyvault secret show --vault-name wldemokv --name keybasebottpk -o json | jq -r .value | tr -d '\n'`

Bringing it all-together

Let’s now test the flow of submitting a new feedback form that should:

trigger a new Github Issue
and a JIRA issue
Notify us by
1. email
2. slack
3. discord
4. keybase

And while I forgot to show it above, this email was sent to the requestor

and myself

Datadog

I decided I wasn’t quite done yet. I want to add some Datadog metrics.

I added the key for my Kasa endpoint as well as my Datadog API key to the GH Env Secrets

Then I added a block of Passing Metrics

      - name: Build count
        uses: masci/datadog@v1
        with:
          api-key: $
          metrics: |
            - type: "count"
              name: "prfinal.runs.count"
              value: 1.0
              host: $
              tags:
                - "project:$"
                - "branch:$"
      - name: Datadog-Pass
        uses: masci/datadog@v1
        with:
          api-key: $
          events: |
            - title: "Passed building Feedback"
              text: "Branch $ passed build"
              alert_type: "info"
              host: $
              tags:
                - "project:$"

And for failing

      - name: Datadog-Fail
        uses: masci/datadog@v1
        with:
          api-key: $
          events: |
            - title: "Failed building Feedback"
              text: "Branch $ failed to build"
              alert_type: "error"
              host: $
              tags:
                - "project:$"
      - name: Fail count
        uses: masci/datadog@v1
        with:
          api-key: $
          metrics: |
            - type: "count"
              name: "prfinal.fails.count"
              value: 1.0
              host: $
              tags:
                - "project:$"
                - "branch:$"

As well as turning off and on the build light

      - name: Turn on build light
        run: |
          curl -s -o /dev/null "https://kasarest.freshbrewed.science/on?devip=192.168.1.3&apikey=$"
      - name: Turn off build light
        run: |
          curl -s -o /dev/null "https://kasarest.freshbrewed.science/off?devip=192.168.1.3&apikey=$"

Now when run, the build light went on/off, I received all the notifications we’ve demoed thus far, but also dropped a metric into DD which I can see in my dashboards

But also in Workflow Explorer

If I send a giant mangled payload which I fully expect to fail

It did manage to fail both stages

But not before posting to Slack

And updating Datadog Events

I can see a time and the Project in Datadog

And use the Build ID to see what was the cause of the failure in Github Actions

Which, in this case, wouldn’t solve it as the content was garbage

But if a user requested a feature and it just wasn’t parsable, I could do something about it.

Also, my Datadog is setup to Alert on subsequent failures. Had it happened multiple times it could escelate

In some select cases, I also have issue trigger pagerduty, but I’m cautious there as I will get woken up at night so this isn’t a case where I’ll use the PagerDuty integration.

Summary

We covered a lot today. We rolled out notifications to Slack, Discord, Keybase and Teams. We Setup JIRA for issues and Datadog as an event collector. I even setup better error handling and build light (as in a bulb tied to a Kasa plug).

Here is a final run of all the parts together:

While my corporate O365 flagged it as Junk, I did get an email to my work address:

You can see we used the live static form to send feedback which triggered an ephemeral Github Action. This:

Created a JIRA Task
Created a Github Issue
Notified me by:
1. Discord
2. Slack
3. Keybase
4. MS Teams
5. Email
Updated Datadog Metrics
(and while you couldn’t see it) turned on and off a physical ‘build’ lamp at my desk.

I think this might be enough for a basic feedback form, perhaps a bit too much. That said, as always, the feedback form is alive and well. If you use it, clearly I’ll hear you.