What is a webhook? How do webhooks work?
Real-time notifications have changed the way people interact with services. For example, your phone instantly notifies you when an SMS arrives, saving you from constantly checking for new messages. Similarly, you rely on package tracking notifications instead of refreshing the courier website to track a parcel.
A webhook brings this same efficiency to software. With webhooks, your system no longer wastes time and resources checking for updates. Instead, other systems directly notify your system when something important happens—just like those helpful text notifications!
What is a webhook?
A webhook is a mechanism that allows applications to transmit real-time data to other applications when specific events occur. It functions as a digital notification system, enabling one application to inform another that something important has happened. This helps automate tasks and keep systems in sync without needing constant manual checks.
Without a webhook, applications are forced to poll other systems for updates by sending regular requests to those systems’ APIs at fixed intervals. When new data or an event is received, it is processed, and the polling process continues. However, this approach has some drawbacks. Servers potentially waste resources fetching data that might not have changed, introducing a delay between events and their processing. Attempting to mitigate this by requesting updates more frequently provides marginal benefits and wastes computing resources.
Webhooks offer a better solution by letting systems send notifications (webhooks) to other applications only when new data becomes available or a specific event occurs. These notifications eliminate unnecessary requests, conserve server resources, and enable instantaneous real-time communication between systems.
In this guide, you’ll learn all about the benefits of webhooks and explore how and when to use webhooks. You’ll also take a look at a few real-world webhook examples to better understand the advantages that webhooks offer.
Why are webhooks useful?
Why are webhooks useful?To understand the benefits of webhooks, you must first understand how these real-time systems were built using application programming interfaces (APIs).
What is an API? API architecture for communication
An API is a component in your system that lets other systems and software communicate with it using rules and protocols. The API type determines details like authentication requirements for clients, and you can choose from multiple protocols when building an API. The most popular protocols are REST and GraphQL. However, Remote Procedure Call (RPC) and SOAP are also options.
Architecture diagram demonstrating communication between the API client and API server
What are webhooks? Webhook architecture for communication
Like APIs, webhooks enable communication between systems. However, unlike APIs, where the client polls the server for new information, webhooks let the webhook provider (or server) notify the webhook consumer (or client) when new information is available.
Architecture diagram demonstrating communication between the webhook provider and consumer
Webhook vs. API: What’s the difference?
The following table summarizes the difference between webhook vs. API:
Webhook use cases
Financial systems also use webhooks. When making payments using a payment gateway, the gateway sends webhook notifications as the payment status changes. If a payment is successful, a webhook notification can be sent to your system to update the order status to Paid. Your system can also receive a webhook notification if the payment fails so that it can notify the user. These webhook notifications enable a level of instantaneous communication that’s not possible with APIs.
Webhooks can also be used for data processing. A background service that encodes large video files can send webhook notifications as videos are processed, providing updates on the encoding process and letting the user know when the process finishes encoding the video.
Benefits and challenges of webhooks
In a nutshell, webhooks’ main benefit is their efficiency in real-time communication compared to APIs. They use a common web protocol (HTTP), making it simple to implement in different technologies and frameworks.
Webhooks are also a good choice for scalable systems since they allow for asynchronous processing, as a system can trigger a process in another system and forget about it.
Here are some of the benefits of webhooks in more detail:
Real-time data transfer: Webhooks provide immediate notifications when an event occurs, allowing instant data transfer between systems. This is beneficial for applications that require up-to-the-minute information, such as payment processors, business messages, in-app chat, custom AI chatbots, or inventory management systems.
Reduced load on client and server: Instead of continuously polling an API for updates, webhooks push data when events happen, reducing the load on both the client and server. This leads to more efficient use of resources and bandwidth.
Automation: Webhooks enable automated workflows by triggering predefined actions when specific events occur. For instance, they can automatically update a database, send emails, or trigger other processes without manual intervention.
Simplicity: Setting up a webhook is usually straightforward, involving configuring a URL endpoint to receive the data. This simplicity makes webhooks accessible even for those with basic programming knowledge.
Scalability: Webhooks can handle large volumes of events efficiently. They allow systems to scale without significant changes to the underlying architecture since the push mechanism is inherently more scalable than polling.
Customization: Webhooks can be tailored to specific needs, sending only the relevant data for the event in question. This customization reduces unnecessary data transfer and processing.
Integration: Webhooks facilitate integration between different services and applications, making it easier to create interconnected systems. This is especially useful in microservices architectures where different components need to communicate seamlessly.
Cost-effectiveness: By reducing the need for constant polling and minimizing the processing power required to check for updates, webhooks can lead to cost savings in terms of both infrastructure and operational expenses.
Despite their benefits, you may encounter particular challenges when using webhooks. One such challenge is compatibility between systems. While webhooks use the HTTP protocol, how they use it can differ. These differences include payload encoding and authentication. Additionally, you must ensure that the webhook notifications sent to a system never overload it, which could lead to data loss.
Here are some challenges of webhooks in more detail:
Error handling: If the receiving server is down or there are network issues, webhook data might be lost or delayed. Implementing robust retry mechanisms and logging failures is essential to ensure reliable delivery and processing of events.
Scalability: Handling a high volume of webhook events can be challenging. The receiving server must be able to process numerous simultaneous requests without performance issues, requiring efficient resource management and infrastructure planning.
Idempotency: Webhooks may deliver the same event multiple times due to retries, leading to potential data inconsistencies or duplication. Ensuring that processing the same event multiple times has the same effect as processing it once is crucial for data integrity.
Latency: Although webhooks are designed for real-time communication, network latency can sometimes cause delays. Ensuring low latency and high-speed communication requires robust infrastructure and efficient network configurations.
Debugging and monitoring: Diagnosing issues with webhooks can be complex due to the involvement of multiple systems. Effective monitoring, logging, and debugging tools are necessary to track events, identify problems, and ensure smooth operation.
How do webhooks work?
Now that you understand the purpose and benefits of using webhooks over traditional architectures like APIs, the following sections will delve into the practical side of webhooks.
Anatomy of a Webhook
Webhooks have three components: an event, a payload, and an endpoint URL.
Event: An event is any action in a system that sends a webhook notification. For example, a system can raise an event when a user places a new order on an e-commerce site, a payment is processed successfully, or a customer updates their payment details.
Payload: Information about the event is sent in the webhook payload.
Endpoint URL: The endpoint URL is an HTTP endpoint on the consumer that receives the webhook notification and its payload.
Diagram showing the different webhook components
How webhooks work
The following diagram illustrates how one might use webhooks when integrating a support system, chatbot, and CRM:
Diagram illustrating webhooks being used in an AI chatbot
In the first step, the customer starts talking to the AI chatbot. The chatbot tries to answer using its knowledge base. When it can’t answer, it sends a webhook request to the organization’s support system to create a new ticket. The webhook request is a GET or POST HTTP request, with the payload either in the body or query parameters of the request. For example, a webhook using a POST request might look like this:
The first line includes the endpoint URL on the consumer. The remaining lines are the JSON payload sent in the request body.
The support system creates a ticket in the support system using the information provided in the webhook. The system raises an IssueCreated event once the ticket is created. This event triggers another webhook notification to the organization’s CRM to make a note of the issue in the CRM system. If the webhook notification uses a GET request, it would look like this:
This webhook notification encodes the payload and adds it to the endpoint URL as query parameters instead of the request body. This technique is necessary because GET methods aren’t typically allowed to have a request body. The CRM system receives this webhook notification and notes the issue on the client’s file.
A support agent is notified of the new support ticket (possibly with a webhook) and investigates the issue. They solve the problem and change the ticket’s status to resolved, triggering an IssueResolved event in the support system. This event sends another webhook notification to the CRM to update the note on the client.
This scenario demonstrates how webhooks share new information with other systems instantaneously. Without webhooks, the CRM system would need to constantly poll the support system for the ticket’s latest status. If the support system didn’t use webhooks, it would also have to poll the chatbot for the latest unresolved queries.
Setting up and configuring webhooks
Before your systems can communicate using webhooks, you must create an HTTP endpoint in your consumer system to receive webhook notifications. Once you’ve set up the endpoint, you have to share the URL with the provider so it can send messages there. If you’re using Sendbird’s AI chatbot, for example, you must configure your system’s endpoint URL in Sendbird so it knows where to send function calls. You do this in the Sendbird dashboard by navigating to your project’s Function calls page and clicking the Create function button:
Creating a function in the Sendbird Dashboard
After you click Create function, you’ll be prompted to configure your webhook. Sendbird provides some AI chatbot-specific fields—such as Function name, Function key, and Trigger prompt—that let you specify when and how the custom AI chatbot should trigger your webhook. You configure the webhook endpoint by specifying the HTTP method and URL to call when sending the webhook. You can also define custom headers to send with the webhook payload, which are helpful for authentication.
Once the webhook is configured, test it using the Send request button:
Testing the webhook
Once you’re happy with the configuration, click Create. Your chatbot can now integrate with other systems in real time:
Request result after clicking the “Send request” button in Sendbird
Securing webhooks
Since webhook endpoints are publicly accessible HTTP endpoints in your system, security is essential to ensure that only authorized systems can send notifications. Always use HTTPS for your endpoints, regardless of your authentication method. HTTPS encrypts the payload while in transit and prevents man-in-the-middle attacks by malicious actors.
A couple of authentication options are available and described below.
Shared secrets
The shared secrets option is the most straightforward authentication method to implement. This method involves sharing a random string with the webhook provider and consumer.
Instead of a random string, you can also use basic authentication. When the provider sends a webhook notification, it includes the shared secret in the header or payload of the request. The consumer receives the request, validates the secret, and only processes it if the string matches its string.
Signature verification
A more secure option is signature verification, also known as hash-based message authentication code (HMAC). The provider hashes the payload and includes the hash in a header on the request. When the consumer receives the request, the payload hash is recalculated to ensure it matches the hash in the request. If they match, the consumer proceeds to process the request.
You can strengthen HMAC security by implementing asymmetric keys for creating the hash and including a timestamp in the hash for replay prevention.
Security in Sendbird
Sendbird lets you specify custom headers when configuring webhooks for an AI chatbot. These custom headers let you implement shared secrets for your webhook calls:
An authorization header configured in Sendbird
When receiving a webhook call, the consumer can verify that the secret in the header is correct before processing the notification.
Handling webhook data
The exact implementation of your webhook endpoint will differ depending on your programming language and framework. Regardless of the technology stack, it’s crucial to check authentication, validate the incoming payload, implement error handling, and add retry logic.
The following snippet shows a possible endpoint implementation in Node.js using Express for a webhook notification from Sendbird:
The endpoint accepts webhooks using POST requests. It ensures that the secret in the Authentication header matches the configured one. Then, the code retrieves the payload from the request body and validates that all the required fields are specified. The payload is processed, and the code returns a 200 HTTP code if the processing is successful. A try block surrounds the implementation to log any errors. It’s recommended that you also implement retry logic here.
Top five webhook best practices
When using webhooks, consider implementing the following best practices to avoid common pitfalls.
1. Build scalable webhook endpoints
Suppose your consumer receives an excessive number of webhook requests. Your system might be unable to handle the volume and crash, potentially resulting in your losing valuable data. Building webhook consumers that dynamically increase their computing capacity based on incoming requests can help mitigate this issue.
2. Implement rate limiting
You can also mitigate the risk of overloading your consumer by implementing rate limiting on the webhook endpoint. Rate limiting prevents the consumer from receiving more requests than it can manage in a certain time period.
3. Plan for errors by implementing logging and monitoring
No system is perfect, and yours is bound to encounter errors at some point. Proper monitoring and logging are essential for detecting and resolving these issues. Observability tools like Grafana, Amazon CloudWatch, or Azure Monitor make it easy to monitor your webhook endpoints. You can also use these tools to identify and send notifications if errors occur.
4. Use retry logic to handle transient errors
5. Secure your endpoints
Webhooks are often public URLs since external applications need to call them. Securing them is essential so malicious actors don’t exploit these open endpoints. Security measures like HTTPS ensure data is encrypted while in transit. You can also secure an endpoint using secret keys or strings that are verified before processing the webhook message.
Tools for working with webhooks
Developing and debugging webhook endpoints can be complicated because they involve multiple systems. Thankfully, several tools on the market can help.
RequestBin lets you inspect webhooks by configuring your provider to send webhooks to RequestBin. The tool visualizes the headers and payload sent in each webhook request. You can use this to see how you should parse and validate the payloads in your system. Webhook.site is a similar tool for receiving webhooks from your provider and displaying headers and payloads.
When sending test webhook requests during development, you can use any HTTP client, like curl or Postman. Just ensure you use the same HTTP method and send the same payload and headers the provider sends.
Debugging an endpoint on your local machine is also helpful. A local development reverse proxy lets you expose a local endpoint on the web so your provider can send messages to that endpoint. These messages get processed in your local environment, where you can debug them. This includes inspecting the headers and data in a webhook message so you can understand how it is encoded and formatted to process it correctly. This is ideal, especially when working with properties or data in a webhook message with little or no documentation. Such reverse proxies include ngrok, localtunnel, and Loophole.
You can also use a comprehensive webhook testing solution like Hookdeck, which includes all the features above in a single service, simplifying your webhook development workflow.
Build real-time communication with webhooks
Webhooks are vital for building modern web applications, and it’s critical to stay ahead of the curve and understand how they work and when to use them. In this article, we answered the question ‘What is a webhook?’ and touched briefly on the webhook vs. API debate. We also talked about how webhooks are more efficient and instant than the traditional method of polling an API. We also learned how to implement webhooks and secure endpoints, and we discussed best practices and tools to use when doing so. To learn more about webhooks, check out this video.
Sendbird uses webhooks in several product offerings, such as AI chatbots and Business Messaging. When your business decides that the time is right to build in-app communication such as chat, calls, omnichannel business messaging capabilities, Sendbird is ready to provide customer communications solutions - including a chat API and fully customizable AI chatbot - that you can build on. You can send your first message today by creating a Sendbird account to get access to valuable (free) resources with the Developer plan. Become a part of the Sendbird developer community to tap into more resources and learn from the expertise of others. You can also browse our demos to see Sendbird Chat in action. If you have any other questions, please contact us. Our experts are always happy to help!