Writing a new integration

Integrations are one of the most important parts of a group chat tool like Zulip, and we are committed to making integrating with Zulip and getting your integration merged upstream so that everyone else can benefit from it, as easy as possible while maintaining the high quality of the Zulip integrations library.

On this page you'll find:

• An overview of the different types of integrations possible with Zulip.
• General advice for writing integrations.
• Details about writing webhook integrations.
• Details about writing Python script and plugin integrations.
• A guide to documenting your integration is on a separate page.

A detailed walkthrough of a simple "Hello World" integration can be found in the webhook walkthrough.

Contributions to this guide are very welcome, so if you run into any issues following these instructions or come up with any tips or tools that help writing integration, please email zulip-devel@googlegroups.com, open an issue, or submit a pull request to share your ideas!

Types of integrations

We have several different ways that we integrate with 3rd party products, ordered here by which types we prefer to write:

1. Webhook integrations (examples: Freshdesk, GitHub), where the third-party service supports posting content to a particular URI on our site with data about the event. For these, you usually just need to create a new python package in the zerver/webhooks/ directory. You can easily find recent commits adding new integrations to crib from via git log zerver/webhooks/.

2. Python script integrations (examples: SVN, Git), where we can get the service to call our integration (by shelling out or otherwise), passing in the required data. Our preferred model for these is to ship these integrations in the Zulip Python API distribution, within the integrations directory there.

3. Plugin integrations (examples: Jenkins, Hubot, Trac) where the user needs to install a plugin into their existing software. These are often more work, but for some products are the only way to integrate with the product at all.

General advice

• Consider using our Zulip markup to make the output from your integration especially attractive or useful (e.g. emoji, markdown emphasis, @-mentions, or !avatar(email)).

• Use topics effectively to ensure sequential messages about the same thing are threaded together; this makes for much better consumption by users. E.g. for a bug tracker integration, put the bug number in the topic for all messages; for an integration like Nagios, put the service in the topic.

• Integrations that don't match a team's workflow can often be uselessly spammy. Give careful thought to providing options for triggering Zulip messages only for certain message types, certain projects, or sending different messages to different streams/topics, to make it easy for teams to configure the integration to support their workflow.

• Consistently capitalize the name of the integration in the documentation and the Client name the way the vendor does. It's OK to use all-lower-case in the implementation.

• Sometimes it can be helpful to contact the vendor if it appears they don't have an API or webhook we can use -- sometimes the right API is just not properly documented.

• A helpful tool for testing your integration is UltraHook, which allows you to receive webhook calls via your local Zulip development environment. This enables you to do end-to-end testing with live data from the service you're integrating and can help you spot why something isn't working or if the service is using custom HTTP headers.

Webhook integrations

A webhook allows a third-party service to push data to you when something happens. It's different from making a REST API call, where you send a request to the service's API and wait for a response. With a webhook, the third-party service sends you an HTTP POST when it has something for you. Your webhook integration defines the URI the service uses to communicate with Zulip, and handles that incoming data.

New Zulip webhook integrations can take just a few hours to write, including tests and documentation, if you use the right process.

For detailed instructions, check out the "Hello World" webhook walkthrough.

For a quick guide, read on.

• First, use http://requestb.in/ or a similar site to capture an example webhook payload from the service you're integrating. You can use these captured payloads to create a set of test fixtures for your integration under zerver/webhooks/mywebhook/fixtures/.

• Then write a draft webhook handler under zerver/webhooks/; there are a lot of examples in that directory. We recommend templating off a short one (like stash or zendesk), since the longer ones usually just have more complex parsing which can obscure what's common to all webhook integrations. In addition to writing the integration itself, you'll need to create Integration object and add it to WEBHOOK_INTEGRATIONS in zerver/lib/integrations.py; search for webhook in that file to find the existing ones (and please add yours in the alphabetically correct place).

• Then write a test for your fixture in the tests.py file in the zerver/webhooks/mywebhook directory. You can now iterate on debugging the tests and webhooks handler until they work, all without ever needing to post directly from the service you're integrating with to your Zulip development machine. You can run just the tests for one integration like this:

  test-backend zerver/webhooks/pagerduty/

  Hint: See this guide for more details on the Zulip test runner.

• Once you've gotten your webhook working and passing a test, capture payloads for the other common types of posts the service's webhook will make, and add tests for them; usually this part of the process is pretty fast. Webhook integration tests should all use fixtures (as opposed to contacting the service), since otherwise the tests can't run without Internet access and some sort of credentials for the service.

• Finally, write documentation for the integration; there's a detailed guide.

Files that need to be created

Select a name for your webhook and use it consistently. The examples below are for a webhook named 'MyWebHook'.

• static/images/integrations/logos/mywebhook.svg: An image to represent your integration in the user interface. Generally this should be the logo of the platform/server/product you are integrating. See Documenting your integration for details.
• static/images/integrations/mywebbook/001.svg: A screen capture of your integration for use in the user interface. You can add as many images as needed to effectively document your webhook integration. See Documenting your integration for details.
• zerver/webhooks/mywebhook/fixtures/messagetype.json: Sample json payload data used by tests. Add one fixture file per type of message supported by your integration. See Testing and writing tests for details.
• zerver/webhooks/mywebhook/__init__.py: Empty file that is obligatory part of every python package. Remember to git add it.
• zerver/webhooks/mywebhook/view.py: Includes the main webhook integration function including any needed helper functions.
• zerver/webhooks/mywebhook/tests.py: Add tests for your webbook. See Testing and writing tests for details.
• zerver/webhooks/mywebhook/doc.html: Add end-user documentation. See Documenting your integration for details.

Files that need to be updated

• zerver/lib/integrations.py: Add your integration to WEBHOOK_INTEGRATIONS to register it. This will automatically register a url for the webhook of the form api/v1/external/mywebhook and associate with the function called api_mywebhook_webhook in zerver/webhooks/mywebhook/view.py.

Python script and plugin integrations

For plugin integrations, usually you will need to consult the documentation for the third party software in order to learn how to write the integration. But we have a few notes on how to do these:

• You should always send messages by POSTing to URLs of the form https://zulip.example.com/v1/messages/.

• We usually build Python script integration with (at least) 2 files: zulip_foo_config.py containing the configuration for the integration including the bots' API keys, plus a script that reads from this configuration to actually do the work (that way, it's possible to update the script without breaking users' configurations).

• Be sure to test your integration carefully and document how to install it (see notes on documentation below).

• You should specify a clear HTTP User-Agent for your integration. The user agent should at a minimum identify the integration and version number, separated by a slash. If possible, you should collect platform information and include that in ()s after the version number. Some examples of ideal UAs are:

ZulipDesktop/0.7.0 (Ubuntu; 14.04)
ZulipJenkins/0.1.0 (Windows; 7.2)
ZulipMobile/0.5.4 (Android; 4.2; maguro)