Testing your outgoing Webhooks and APIs with Docker, RequestBin, and ngrok

Example of how to use Docker, RequestBin and ngrok to test your outgoing webhooks and APIs locally.

Lately, I’ve been doing lots of work with APIs to connect different systems. Lots of websites have great API documentation with great examples.

http://developer.jamf.com/#/ for an Admin example.
https://developer.okta.com/docs/api/resources/users for a SAML example.

Others, like Okta have great Postman support to test all your GET, POST and PUSH desires. You just add the API reference into Postman and all the formatting is done for you and you are free to build!

The infrastructure to send out requests is great, but how do you securely test incoming requests? If I fire an outgoing webhook, or program a POST request from a Lambda, how can I make I’ve got the formatting right and the data is being received correctly?

To fill that gap, I’ve decided to use RequestBin downloaded and run from Docker, and ngrok. I am using macOS 10.13. Instructions below.

Why go through all this hassle?

If you are testing something and need to use production data rather than test data, there are likely restrictions on where that data can go and where it can exist. For instance, if you wanted to test outgoing webhooks from a production Jamf Pro instance, you could use this method to receive the data locally on your machine rather then having it exist, even in a transitive state, on a third party website.

Additionally, you can use this method to get the explicit output from a third party service to make sure that the formatting is as expected, in case something has changed but hasn’t been reflected in the reference documentation. You can also confirm that your POST data matches up with the reference documents for troubleshooting.

Tools:

• Docker
  • Docker is a container tool, it will allow us to download and run a contained RequestBin that we can spin up when needed, and spin down when we are done with it.
• RequestBin
  • “RequestBin gives you a URL that will collect requests made to it and let you inspect them in a human-friendly way.
    Use RequestBin to see what your HTTP client is sending or to inspect and debug webhook requests.”
• ngrok
  • ngrok allows you to create a tunnel from a public URL directly to the specific port on your local machine.

Building and Running RequestBin:

• Download and Install Docker.
• Use git to clone the RequestBin repo:
  • Instructions here

git clone git://github.com/Runscope/requestbin.git

• Build the Docker container, and then bring up the container

sudo docker-compose build

sudo docker-compose up -d

• You will now have a Docker container running with all the pieces you need. By default, RequestBin will broadcast locally at  http://localhost:8000

Ngrok and tunnel building:

• Download the ngrok binary
  • As per the download site ” a single binary with zero run-time dependencies”, just download and run!
• In Terminal, change to the directory where you’ve placed ngrok and run.

./ngrok http 8000

• ngrok will create a tunnel for 8 hrs on the free tier, with an http and https link that will look like

https://1a2b34cd.ngrok.io

• ngrok make an http endpoint and will take all traffic from the provided address and send it to the local machine and port specified (8000 in the example above). Go to the URL you are provided and you should be looking at your local RequestBin instance!
  • ngrok free will give you different URLs each time you run it. There is a paid version which will let you specify the URLs and make them persistent.

Create a RequestBin and start testing!

• Click “Create a RequestBin”, and you’ll be provided a URL and many sample tests. The URL with be the ngrok URL plus more random characters.
• From here, you can fire an outgoing webhook to this URL or POST to it from Postman or a different service and see exactly what your output looks like when it arrives at the server!

Now you can test locally without having your information ending up on a random endpoint and have better visibility of just exactly what is being sent and how its received!

NOTE: When you are done testing, don’t forget to disable/remove the any webhooks!

Since you are sending your data out.

Example – Jamf Pro Webhook

Once you have a functioning RequestBin instance, you can go to your Jamf Pro server to test the outgoing webhooks.

https://YOURJAMFPRO/webhooks.html

Here you can add your RequestBin ngrok URL to a new webook policy.

If you want quick results, I can suggest using ComputerCheckIn if you have a large fleet, or ComputerInventoryComplete and a sudo jamf recon on an enrolled device. Refresh the RequestBin page and you should see results pouring in.