Guides

Webhook E2E Strategy - Technical Overview

An asynchronous approach for at least once delivery designed for high throughput.

🚧

Under construction and subject to changes. Speak to the team for more available ASYNC strategies.

Summary

The Alert to Ticket Module provides Integrators with the capabilities to automatically create PSA tickets from your alerts. The system features multi-layered control mechanisms, intelligent duplicate handling, and self-healing capabilities to streamline Partner incident management workflows. This flow requires a webhook endpoint to be provided by the Integrator such that MSPC can provide information to the Integrator regarding ticket management throughout the lifecycle of an alert/ticket

Core Functionality

Automated Ticket Creation: Integrators can programmatically create PSA tickets through MSPCentric's oem alerting API, with intelligent routing to appropriate PSA queues/boards based on rule creation by the user

Smart Duplicate Handling: The system prevents ticket spam by appending related alerts to existing tickets rather than creating duplicates, configurable per Partner preferences.

Self-Healing Resolution: When underlying issues resolve automatically, vendors can close associated tickets programmatically, reducing manual intervention requirements.

Layered Control System: Multiple configuration layers ensure Partners maintain full control:

Webhook Informational System: Provides detailed information of key events during the lifecycle of an alert/ticket.

  • Integration-level alerting enablement
  • Status-based suppression rules
  • Individual feature toggles (ticket creation, self-healing, de-duplication)
  • Granular account-to-PSA-queue mapping

Technical Architecture

API Structure: RESTful endpoints hosted on MSPCentric platform (/api/oem-alerts-api/*) with secure authentication and rate limiting.

Multi-PSA Support: Compatible with ConnectWise Manage, Datto Autotask(tbd), Halo PSA (tbd), Kaseya BMS, Syncro PSA (tbd), and Pulseway PSA.

Audit & Visibility: Comprehensive logging of all operations with dashboard visibility for Partners, including success/failure metrics and complete event history.

Rules Engine: Configurable suppression logic allowing MSPs to prevent ticket creation based on Product Groups, alert statuses, or custom business rules.

Integration Requirements

Vendors must implement alert detection logic, API integration for ticket operations, robust error handling with retry mechanisms, and correlation ID management for tracking ticket lifecycles. The system supports both real-time alert processing and historical status queries. Vendor must provide a webhook endpoints such that the Alerts system can provide feedback regarding key events during the ticket lifecycle.

What am I implementing?

Scope: Implement API integration to automatically create PSA tickets from alerts.

API Endpoints (Hosted on MSPCentric Platform)

  • Create Alert POST /api/oem-alerts-api/alert Alert endpoint that handles creating tickets, smart appending and auto-healing
  • List Tickets: GET /api/oem-alerts-api/alert - List all tickets for a partner
  • Get Ticket: GET /api/oem-alerts-api/alert/{correlationId}- Get ticket for Partner by correlationId
  • Integration Check: GET /api/oem-api/integration/list - Validate alerting enabled

Webhook endpoint (Provided by Vendor)

  • Provide a single endpoint such that the Alerting system can provide key informational updates during the lifecycle of the Alert/Ticket. This is a key feature that will allow the Integrator to receive critical updates to help manage ticket lifecycle.

During implementation, we will implement your webhook and authentication necessary for the webhook.
Ideally, you will provide the same authentication method as your normal API and the same key that we use for the integration can be used here to identify the user when we send a webhook response.

Vendor Implementation Requirements

1. Alert Detection & Filtering

Key Considerations:

  • Consider alert frequency to avoid ticket spam

2. Determining Account IDs

Account IDs in our public api are managed by the Integrator and will be the Vendor side identifiers used on the Integrators platform. Therefore, it is a requirement that you provide unique identifiers via the Vendor API.

Retrieve Accounts

When calling the alert endpoint, you will provide the internal Vendor identifier that uniquely identifies the company of your Partner.

All account Mappings created are retrievable via the Vendor API using Integrator side IDs.


3. Product Groups and Alert Types

Product Groups and Alert Types enable Partners to create rules that target meaningful groupings of products or services. Any Rule Set evaluated, will be evaluated based on rule precedence, and the incoming Alert's Product Group and Alert Type. When a Rule is matched, the Rule's configuration will determine which Board a ticket will be created in, and what metadata will be set on the newly created ticket.

It is the Integrator's responsibility to provide us a list of Product Groups and a list of corresponding Alert Types that match those Product Groups. This means that each Product Group has child Alert Types which can be associated to it.

We will store this as a list of comma separated values for use during runtime to help the rules engine determine which rule to use when calling the alert endpoint.

🚧

Advanced Feature Available:

Due to some platforms having an exorbitant number of potential Product Groups and Alert Types per Partner, we have an alternative, non-static, retrieval method for Product Groups and Alert Types which takes a Vendor Account ID as an argument for targeted retrieval of Product Groups. This implementation requires Integrator API's.

Dynamic Product Groups and Alert Types

4. API Integration Development

Required API Calls:

Alert Creation:

POST /api/oem-alerts-api/alert HTTP/1.1
Content-Type: application/json

{
  "clientId": "66abbbae5a078102ccf5790c6a84b4de4cac647110f3db597399cde2010d7d17",
  "clientSecret": "bad6425ac1fd5bd2157f839496580ad9371e9b5b0abd65c3b88fe34d471eeef37c3cf06bda3932782b0cbc003c1980be450c1a3214eae3a0c2781a95efc1859347b81bdc4b8914745fe669cfccf933dfaa46ed0795725412cbfdc7663eb13c28",
  "grantType": "client_credentials",
  "scope": "vendor_manage",
  "organizationOemToken": "bad6425ac1fd5bd2157f839496580ad9371e9b5b0abd65c3b88fe34d471eeef37c3cf06bda3932782b0cbc003c1980be450c1a3214easdas34d39347b81bdc4b8914745fe669cfccf933dfaa46ed0795725412cbfdc7663eb13c28",
  "integrationId": "690b98ea1a4906f08f785ad8",
  "data": {
    "title": "Backup Failed: Agent not checking in",
    "summary": "A descriptive summary of the incident.",
    "alertType": "Backup Failed",
    "accountId": "as23cCS13",
    "productGroup": "firewalls", 
    "incidentStatus": "OPEN"
  }
}
HTTP/1.1 201 OK
Content-Type: application/json

'{
  "message": "created",
  "data": {
    "messageId": "string", <-- unique identifier for this single transaction
    "correlationId": "string" <-- unique identifier for entire lifecycle of an alert/ticket
  }
}'
HTTP/1.1 200 OK
Content-Type: application/json

{
  "message": "suppressed"
  "data": {
    "messageId": "string", <-- unique identifier for this single transaction
    "correlationId": "string" <-- unique identifier for entire lifecycle of an alert/ticket
  }
}
HTTP/1.1 200 OK
Content-Type: application/json

{
  "message": "failed"
  "data": {
    "messageId": "string", <-- unique identifier for this single transaction
    "correlationId": "string" <-- unique identifier for entire lifecycle of an alert/ticket
  }
}
📘

Note: Optionally, you can provide your own unique correlationId in the Create Alert endpoint call to manage the lifecycle of a ticket if that works better within your system.

Smart Appending (appending or updating a ticket):

POST /api/oem-alerts-api/alert HTTP/1.1
Content-Type: application-json

{
  "clientId": "66abbbae5a078102ccf5790c6a84b4de4cac647110f3db597399cde2010d7d17",
  "clientSecret": "bad6425ac1fd5bd2157f839496580ad9371e9b5b0abd65c3b88fe34d471eeef37c3cf06bda3932782b0cbc003c1980be450c1a3214eae3a0c2781a95efc1859347b81bdc4b8914745fe669cfccf933dfaa46ed0795725412cbfdc7663eb13c28",
  "grantType": "client_credentials",
  "scope": "vendor_manage",
  "organizationOemToken": "bad6425ac1fd5bd2157f839496580ad9371e9b5b0abd65c3b88fe34d471eeef37c3cf06bda3932782b0cbc003c1980be450c1a3214easdas34d39347b81bdc4b8914745fe669cfccf933dfaa46ed0795725412cbfdc7663eb13c28",
	"correlationId": "AS23-DSA4-ADF3-SASFD1", <--- pass the correlation id when appending
  "integrationId": "690b98ea1a4906f08f785ad8",
  "data": {
    "title": "Backup Failed: Agent not checking in",
    "summary": "A descriptive summary of the incident.",
    "alertType": "Backup Failed",
    "accountId": "as23cCS13",
    "productGroup": "firewalls", 
    "incidentStatus": "UPDATE" <--- pass update to ensure smart append
  }
}
HTTP/1.1 200 Ok
Content-Type: application/json

{
  "message": "updated",
  "statusCode": "200",
  "data": {
    "messageId": "string", <-- unique identifier for this single transaction
    "correlationId": "string" <-- unique identifier for entire lifecycle of an alert/ticket
	}
}

Auto-Healing:

POST /api/oem-alerts-api/alert HTTP/1.1
Content-Type: application-json

{
  "clientId": "66abbbae5a078102ccf5790c6a84b4de4cac647110f3db597399cde2010d7d17",
  "clientSecret": "bad6425ac1fd5bd2157f839496580ad9371e9b5b0abd65c3b88fe34d471eeef37c3cf06bda3932782b0cbc003c1980be450c1a3214eae3a0c2781a95efc1859347b81bdc4b8914745fe669cfccf933dfaa46ed0795725412cbfdc7663eb13c28",
  "grantType": "client_credentials",
  "scope": "vendor_manage",
  "organizationOemToken": "bad6425ac1fd5bd2157f839496580ad9371e9b5b0abd65c3b88fe34d471eeef37c3cf06bda3932782b0cbc003c1980be450c1a3214easdas34d39347b81bdc4b8914745fe669cfccf933dfaa46ed0795725412cbfdc7663eb13c28",
	"correlationId": "AS23-DSA4-ADF3-SASFD1", <--- pass the correlation id when auto-healing
  "integrationId": "690b98ea1a4906f08f785ad8",
  "data": {
    "title": "Backup Failed: Agent not checking in",
    "summary": "A descriptive summary of the incident.",
    "alertType": "Backup Failed",
    "accountId": "as23cCS13",
    "productGroup": "firewalls", 
    "incidentStatus": "RESOLVED" <--- pass update to ensure auto-healing
  }
}
HTTP/1.1 200 Ok
Content-Type: application/json

{
  "message": "closed",
  "statusCode": "200",
  "data": {
    "messageId": "string", <-- unique identifier for this single transaction
    "correlationId": "string" <-- unique identifier for entire lifecycle of an alert/ticket
	}
}

3. Define Webhook endpoint

Required Implementation:

In order to use the webhook flow, you must provide a webhook endpoint such that we can send updates on a regular basis during the lifecycle of the alert/ticket.

Webhooks requests will be sent to you in a consistent format.

interface InformEventDto<AlertResponseDto> {
  module: string;
  eventType: string;
  payload: {
    event: AuditTrailEventType; 
    messageId: string;
  	correlationId: string;
  	vendorOrganizationId: string;
  	vendorUserOrganizationId: string;
    integrationId: string;
    payload: {
      status: number;
			message: string
		}
	}
}
interface InformEventDto<AlertStateSnapshotPayloadDto> {
  module: string;
  eventType: string;
  payload: {
		isIntegrationActive: boolean;
		isAlertsEnabled: boolean;
		productGroups: Map<string, string[]> // a map holding productGroup as key, alert type as string[]
		accounts: string[];
		vendorOrganizationId: string;
		vendorUserId: string;
		integrationId: string;	
	}
}

Examples of possible responses:

Content-Type: application/json
{
	module: "alerts",
  eventType: "status_update",
  payload: {
    event: "Ticket Created",
    messageID: XRAY.TraceId,
    correlationId: "fake-correlation-id",  	
    vendorOrganizationId: "123456789A",
    vendorUserOrganizationId: "U54634646A",
    integrationId: "69fcd2179a3f408700dc0511",
    payload: {
      status: "success",
      message: "Ticket created Successfully",
    }
	}
}
Content-Type: application/json
{
	module: "alerts",
  eventType: "status_update",
  payload: {
    event: "Failed To Create Ticket",
    messageID: XRAY.TraceId,
    correlationId: "fake-correlation-id",  	
    vendorOrganizationId: "123456789A",
    vendorUserOrganizationId: "U54634646A",
    integrationId: "69fcd2179a3f408700dc0511",
    payload: {
      status: "suppressed",
      message: "Ticket suppressed",
    }
	}
}
Content-Type: application/json
{
	module: "alerts",
  eventType: "status_update",
  payload: {
    event: "Ticket Not Found",
    messageID: XRAY.TraceId,
    correlationId: "fake-correlation-id",  	
    vendorOrganizationId: "123456789A",
    vendorUserOrganizationId: "U54634646A",
    integrationId: "69fcd2179a3f408700dc0511",
    payload: {
      status: "failed",
      message: "Ticket not found",
    }
	}
}

4. Customer Configuration Interface

Recommended UI Elements:

  • Test alert button
  • Alert preview/simulation

5. Error Handling & Retry Logic

Required Implementation:

  • Handle HTTP error responses (403 Forbidden, 500 errors, rate limits)
  • Implement retry logic for temporary failures
  • Log failed API calls for customer troubleshooting
  • Graceful degradation when MSPCentric API is unavailable

Error Scenarios to Handle:

  • Integration alerting disabled (409 Forbidden)
  • Alert suppressed by rules (409 OK, but no ticket created)
  • PSA system unavailable (500/502 errors)
  • Network connectivity issues

5. Correlation Id Management

Critical Requirement:

  • Store the correlationId returned from alert creation calls
  • Use correlationId for self-healing and smart-appending
  • Maintain mapping between your internal alerts and MSPCentric correlationIds
  • Implement correlation ID cleanup for resolved/closed alerts

Runtime Alert Processing:

  1. Alert Triggered: Your system detects an issue (e.g., backup failure)
  2. Create Ticket: Call MSPCentric API to create/append ticket
  3. Store Correlation: Save the returned correlationId for future reference
  4. Monitor Resolution: When issue resolves, call self-healing endpoint
  5. Handle Errors: Log any API failures and retry if appropriate

Error Handling

This section contains a list of possible known errors that our system will throw given certain scenarios when working with the Alerts API.

  1. 404 OemNotFoundException - Thrown when:
    1. No PSA found on the organization
    2. Account was not found in our system
    3. Account was not mapped by user and cannot be communicated to the psa
    4. User has no rules defined for a given Product Group
    5. User has turned off all rules for a Product Group
    6. Ticket not found that corresponds to the correlationId provided
    7. Rule was not found during smart append or auto-healing, means rule was likely deleted by the user
    8. Integration not found during smart append or auto-healing, means integration was likely removed by user.
  2. 409 OemConflictException - Thrown when:
    1. Integration is not in an active state to receive Alerts
    2. Alerts is not enabled for a user
  3. 500 OemInternalServerException - Throw when:
    1. Server encounters an unknown error