HCL Compass Webhooks in Action

Introduction

This article talks about Webhooks in HCL Compass. Webhooks are nothing but automated data sent from an app when something is triggered. In our case, HCL Compass is the app which sends the Webhook data, which is also called as Payload. A Webhook subscribes to Compass Application and waits until a specific condition is met to send the payload. The payload usually is a JSON with record data and sent as HTTP POST request, which can be caught by any third party application or your custom script and process it to do useful things. This is different from RESTAPI, where the requests are sent from an Application to Compass for specific information or to do some specific actions. Here, you just wait for the data from Compass at regular intervals based on the condition defined in  Compass Webhook records.

To demonstrate this, we are going to first enable Webhooks in an HCL Compass instance. We will go through the steps required to do this. Once we have everything set up, we will use a simple Python Flask Application to listen to the POST request from Compass Webhooks and display it, just to prove that it works.

Enabling Webhooks in Compass

Let’s go through how to enable Webhooks in  HCL compass step by step. Most of these steps are specific to Windows, but translate the same way to Unix environments as well.

1. In order to use Webhooks you need to have the HCL Compass RESTAPI server component installed.

You can go to Start -> Programs -> IBM Installation Manager -> View Installed Packages to verify this,

Once the page opens, see the following to confirm RESTAPI server is installed,

If this is not installed, you would need to install it via the Installation Manager, if not continue to the next step.

2. The next step is to install the package, Webhooks 1.0 into your Schema Repository. To do this, Open your Compass Designer, click on start->programs->HCL Compass-> Compass Designer,

3. Connect to the Schema Repository using your credentials and click on Schema -> Version -> Click on Packages – > Apply Package,

4. Select Webhooks package 1.0 and apply it,

5. Select the record type to enable for the package,

6. Once Package is applied, checkin the schema and upgrade the database,

Creating Webhook configuration

The Webhooks package allows other systems to receive JSON payloads about Compass records when certain conditions are met, such as a specific action being executed on a record, or a record changing state.

Once the package is successfully applied, It’s time to Create the records required to get Webhooks up and running.

With an Administrator account, login to cqweb and add a record of type, WebhookMaster. Add the Groups Allowed to create WebhookConfig records (This is not mandatory). The users in this group can create Webhook configurations. You also need to provide a Callback URL, which is nothing but your Compass RestAPI url. We have it locally on the machine, hence the name localhost. Basically, Compass is sending data to a third party application and waiting to see if the message is being delivered or not. So that response is expected in the Callback URL.

Next step is to create a WebhookConfig record,

This is where you define your Webhook parameters, such as , conditions for the enabled record to trigger a Webhook, a url to your third party application etc. So as per the below screenshots, we entered a name, compassdefect as Defect is our record type which needs to be enabled for Webhooks. You can have any name here. The next field is URL, which is nothing but the URL at which your third party application would be listening to the POST requests from Compass Webhooks.  In this case, it is a simple Python Flask application which listens to the compassdefect route. The 127.0.0.1 is the loopback address since the application is running on the same system as Compass. In your case, this might be a different IP or hostname. There is also an optional SECRET field, this is used for securing the Webhook data. This is strongly recommended to be enabled in a Production environment. The secret you provide is used to hash the payload by using the HMAC-SHA1 hash function and sent via signature header along with the payload. In your application, you would need to pass the secret to a hash function and verify if it is matching with the one provided by Compass in the header.

You can also provide a Description to your Webhook in the Description field. Provide the record type which needs to be tracked. In this case, this is the Defect. Choose the Webhook type as Commit or Notification. The difference is that Commit will only send the payload if the Action mentioned in the Controls is successful while Notification is supposed to sent the payload irrespective of the status. In this case, Commit is selected.

In the Controls tab, in this case, Action selected is Submit and state is Submitted. So, if you submit a Defect record , it will trigger a Webhook payload to the Flask server. There is also a defect associated with the controls tab and the steps to tackle it, is mentioned in the documentation url below,

https://help.hcltechsw.com/compass/2.0.3/com.hcl.compass.doc/webhelp/oxy_ex-1/com.ibm.rational.clearquest.oslc_cmrest_api.doc/topics/t_create_webhook.html

Using any API client, such as Postman, create a HTTP POST request to the REST API server’s hooksetup end point. For this you need to start the RESTAPI server first.  Go to the Compass Installation directory and navigate to the following path in the command line, and run start.bat,

When you start the REST server, you would see a line like, Server is NOT configured for Webhooks.

Once the server is up, from an API client run an HTTP POST Request with the following endpoint,

Method : POST

URL : https://localhost:8190/ccmweb/rest/hooksetup

Body : {

“username”: “admin”,

“password”: “”,

“repo”: “Gitintegration”,

“db”: “GITIN”

}

In this case GITintegration is the Schema Repository and GITIN is the user database, admin is the user with no password.

In this case, REST server is running on the same machine, hence the name localhost. This enables the server for use with the Webhooks package by creating a system user that can execute the necessary actions in Compass.

Restart the RESTAPI server,

This time you can see that Webhooks  is now Enabled.

Execution of the Webhooks

Everything from this point onwards, should be automatic. Compass should trigger a Webhook payload, the moment, there is a new record submitted and we will check if this payload is sent successfully and is able to be read in our Flask application and remember for all this to work, RESTAPI server should be up and running and so does the Flask application.

1. Create a new defect in Compass

2) Start RESTAPI server if it’s not started yet and you can see that Webhooks is running,

And we can see the same data received in the Flask Application,

How can you troubleshoot Webhooks

You can create a Query in Compass to track the Webhooks. You need to base the query on a record type, WebhookData. When you run this query, you can see if there are any pending Webhooks to be delivered and how many times it has tried to deliver to your Application and if there are any errors.

If there is nothing to deliver, you won’t see any records listed.

Here is an example of a failure. This time the Flask server was not running, hence the response code of 500 and retry of 1.

Here is the sample FLASK Application code,

Conclusion

NB : Please note that this is only a sample code for demonstration purpose and not to be used in production.

As you can see from above, the application does nothing other than just printing the data into the console . It could have done many other things such as storing into a DB or using it for any other purpose to display on a webpage. But in the real world, the possibilities are endless with having to set up integration of HCL Compass with Third party applications. We hope that this will get you started with Webhooks and explore the possibilities of it to work with HCL Compass Data.

References

https://help.hcltechsw.com/compass/2.0.3/com.hcl.compass.doc/webhelp/oxy_ex-1/com.ibm.rational.clearquest.oslc_cmrest_api.doc/topics/c_webhooks.html