Thanks for taking the time and for your help improving this project!
If you have a question about rudder or have encountered problems using it, start by asking a question on Slack.
To contribute to this project, we need you to sign to Contributor License Agreement (“CLA”) for the first commit you make. By agreeing to the CLA we can add you to list of approved contributors and review the changes proposed by you.
See the project's README for further information about working in this repository.
- Submit an issue describing your proposed change.
- We will try to respond to your issue promptly.
- Fork this repo, develop and test your code changes. See the project's README for further information about working in this repository.
- Submit a pull request against this repo's
mainbranch.- Include instructions on how to test your changes.
- Your branch may be merged once all configured checks pass, including:
- A review from appropriate maintainers
- Fork this repo, develop and test your code changes. See the project's README for further information about working in this repository.
- Submit a pull request against this repo's
mainbranch.- Include instructions on how to test your changes.
- Your branch may be merged once all configured checks pass, including:
- A review from appropriate maintainers
We prefer squash or rebase commits so that all changes from a branch are committed to main branch as a single commit. All pull requests are squashed when merged, but rebasing prior to merge gives you better control over the commit message.
Now, that you have a basic overview of the contibution guidelines. Let's dive into a detailed guide to contribute by creating a new custom RudderStack integration.
Before starting to work on your first RudderStack integration, it is highly recommended to get a high-level overview of RudderStack Event Specification.
- When developing a source integration, you’ll be transforming your events data received from the source to this specification.
- When developing a destination integration, you’ll be parsing the event data according to this event spec and transforming it to your destination’s data spec.
- Add integration code to rudder-transformer
src/v1/sourcesorsrc/cdk/v2/destinationsfolder
This is the codebase that controls how raw event data (received from the source) is transformed to RudderStack Event Data Specification and then finally to the destination specific data format - Add RudderStack UI configurations in rudder-integrations-config
src/configurations/sourcesorsrc/configurations/destinationsfolder
This enables your integration users to setup/configure the integration via RudderStack Dashboard - Write the documentation for your integration (or share the integration plan document with the RudderStack team)
- RudderStack team will deploy your integration to production and post an announcement in the Release Notes
RudderStack team will be available to help you by giving feedback and answering questions either directly on your GitHub PR or the RudderStack Slack community. Before diving into code, writing an integration plan document helps a lot, here’s an example document.
Setup rudder-transformer on your local machine
- Clone this repository
- Setup the repository with
npm run setup - Build the service with
npm run build:clean - Start the server with
npm start
Understand the code structure
src/v1/sources- Source integrationssrc/cdk/v2/destinations- Destination integrationssrc/v0andsrc/v1have older integrations, will be useful only if you’re fixing a bug or adding feature in older integrationstest/integrations/sources- Integration tests for source integrationstest/integrations/destinations- Integration tests for destination integrations
Here, we will explain the code for a v0 source integration but most of the learning will apply to other integrations as well (destination and v1).
- Create a new directory with the integration name under
src/v0(follow the snake case naming convention) - Add
transform.js, add aprocess(eventData)function and export it. This is the function which will be called to transform the event data received from the source - Add
config.jsto separate the source configuration and mapping
A simple example of process function and transform.js file looks like this
/**
* src/v0/sources/slack/transform.js
* An example transform.js file for Slack integration with v0 integration method
*/
const Message = require('../message'); // Standard RudderStacj event message object
const { TransformationError } = require('@rudderstack/integrations-lib'); // Standard errors
const { generateUUID, removeUndefinedAndNullValues } = require('../../util'); // Common utilities for v0 source integrations
/**
* Transform the input event data to RudderStack Standard Event Specification - https://www.rudderstack.com/docs/event-spec/standard-events/
* @param {Object} eventPayload - The event object received from the source.
* For example, Slack sends this payload which Slack source integration transforms to match the RudderStack Event Specs (e.g. Slack's team_join event to RudderStack's IDENTIFY event spec)
* {
* "event": {
"type": "team_join",
* "user": {
* "id": "W012CDE",
* "real_name": "John Doe"
* }
* },
* "type": "event_callback",
* ...restOfTheEventPayload
* }
* @returns {Object} transformedEvent - Transformed event
*/
function process(eventPayload) {
const transformedEvent = new Message(`SLACK`); // SLACK is the integration name here. It will be different for your integration.
if (!eventPayload?.event) {
throw new TransformationError('Missing the required event data');
}
switch (eventPayload.event.type) {
case 'team_join':
// Transform Slack's team_join event to RudderStack's standard IDENTIFY event schema
transformedEvent.setEventType(EventType.IDENTIFY);
break;
}
// Normalize event names e.g. "team_join" to "Team Join"
transformedEvent.setEventName(normalizeEventName(eventPayload.event.type));
const stringifiedUserId = eventPayload?.event?.user?.id;
// anonymousId is an important property of RudderStack event specification to identify user across different user events
transformedEvent.setProperty(
'anonymousId',
stringifiedUserId ? sha256(stringifiedUserId).toString().substring(0, 36) : generateUUID(),
);
// externalId is another important property of RudderStack event specification that helps connect user identities from different channels
transformedEvent.context.externalId = [
{
type: 'slackUserId',
id: stringifiedUserId,
},
];
// Set the standard common event fields. More info at https://www.rudderstack.com/docs/event-spec/standard-events/common-fields/
// originalTimestamp - The actual time (in UTC) when the event occurred
transformedEvent.setProperty(
'originalTimestamp',
tsToISODate(eventPayload.event.ts)
);
// Map the remaining standard event properties according to mappings for the payload properties
const mapping = [
{
"sourceKeys": "event.type",
"destKeys": "event"
},
{
"sourceKeys": "event.user.profile.real_name",
"destKeys": "context.traits.name"
},
]
// Using a fixed json mapping structure (sourceKeys <> destKeys), it is the quickest way to map source fields with the destination fields
transformedEvent.setPropertiesV2(eventPayload, mapping);
// Copy the complete event payload to transformedEvent.properties
Object.assign(transformedEvent.properties, eventPayload.event);
return removeUndefinedAndNullValues(transformedEvent);
}
// Make sure to export the `process` function. All other details of the integration code can remain hidden.
exports.process = process;You’ll need some API request client (e.g. Postman, Bruno, etc.) to make a POST test request to
/{$integration-version-type e.g. v0 or v1}/{$integration-type e.g. sources or destinations}/{$integration-name e.g. slack}
- Request endpoint example for Slack source developed under v0 folder -
POST /v0/sources/slack - Body - An array of event data object received from the source i.e.
[{ …eventData }] - Headers -
Content-Type: application/json
Depending upon your integration behavior for different types of event, you can get different types of output.
This is what you’d want to do most of the time. The standard case, when you only transform the incoming event and hand it over to RudderStack to deliver to the destination.
For a successful event processing in such case, you should receive HTTP 200 OK response status with data in the following structure
[
{
"output": {
"batch": [
{
...transformedEventData
}
]
}
}
]You can customize the response to the HTTP request from the source beyond the standard event response format sent by RudderStack. You can customize the response body, content type header, status code, etc. Such customization is useful in some cases when you do not want to receive the standard response from RudderStack e.g. url_verification event for Slack webhook url ownership verification needs to respond back with the challenge parameter in the response body and the HTTP response status code to be 200.
In such a case, you need to return the response in a specific format containing outputToSource and statusCode. Your integration code can send the outputToSource.body and outputToSource.contentType as per your requirements.
In this case, a successful event response should match the following structure
[
{
outputToSource: {
body: ...base64EncodedResponseBody, // e.g. eyJjaGFsbG2UiOiIzZVpicncxYUIxMEZFTUFHQVpkNEZ5RlEifQ (base64 encoding of the response body)
contentType: ...contentType // e.g. application/json
},
statusCode: ...httpStatusCode, // e.g. 200
}
]Follow the test structure similar to other integrations in test/integrations/sources or test/integrations/destinations.
You may reuse the same request data from the manual tests i.e. use them in place of replaceThisWithYourEventPayloadProps and replaceThisWithYourTransformedEventOutput (including the enclosing [{ }]). But make sure to redact any personal or secret information.
Here’s an example
/** test/sources/slack/data.ts **/
export const data = [
/** Test 1 - Testing standard event response **/
{
name: 'slack', // Replace with your integration name
description: 'Team joined event', // Replace with your event description
module: 'source', // Replace with your integration type - destination or source
version: 'v0', // Replace with your integration approach - v0 or v1
input: {
request: {
body: ...replaceThisWithYourEventPayload
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
},
pathSuffix: '',
},
output: {
response: {
status: 200,
body: [
{
output: {
batch: ...replaceThisWithYourTransformedEventOutput
},
},
],
},
},
},
/** Test 2 - Testing custom event response **/
{
name: 'slack', // Replace with your integration name
description: 'Webhook url verificatin event (not a standard RudderStack event, returns a custom response)', // Replace with your event description
module: 'source', // Replace with your integration type - destination or source
version: 'v0', // Replace with your integration approach - v0 or v1
input: {
request: {
body: ...replaceThisWithYourEventPayload
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
},
pathSuffix: '',
},
output: {
response: {
status: 200,
body: [
{
outputToSource: {
body: 'eyJjaGFsb2UiOiIzZVpicncxYUIxMEZFTUFHQVpkNEZ5RlEifQ==', // Replace this with the Base64 encoding of the response body your integrations sends
contentType: 'application/json', // Replace this with the content type your integration sends
},
statusCode: 200, // Replace this with the custom response status your integration sends for this event
},
],
},
},
},
];You can run tests only for the specific integration, for example
- To test Slack destination -
npm run test:ts -- component --destination=slack - To test Slack source -
npm run test:ts -- component --source=slack
These tests will automatically be run on each commit, making sure that any new development does not break the existing integrations.
Add configuration for your integration in rudder-integrations-config repo under src/configurations/sources/ or src.configurations/destinations. At bare minimum, a db-config file is needed, here’s an example of the same for Slack source. Duplicate it in the directory for your integration config folder and change the relevant values for your integration.
Alternatively, the easier path to do this is by running a script which will generate these files for you. For this, first create a copy of this [test/configData/inputData.json](https://github.com/rudderlabs/rudder-integrations-config/blob/develop/test/configData/inputData.json) and adjust it to what you want to see in the RudderStack dashboard ui. And use that placeholder file in the following command
python3 scripts/configGenerator.py <path of the placeholder file>
Run this command from the root directory of the rudder-integrations-config repo. The PR can then be raised after checking if everything looks good.
nvm use v20
npm ci
npm run build:start
- [Contributing.md](https://github.com/rudderlabs/rudder-server/blob/master/CONTRIBUTING.md\) in all github repositories
- https://www.rudderstack.com/docs/resources/community/
- https://www.rudderstack.com/docs/event-spec/standard-events/
- Recording of the community event for new contributors
Before diving into this tutorial, it is highly recommended to get a high-level overview of RudderStack Event Specification.
- When developing a destination integration, you’ll be transforming the data from RudderStack Event Specification to your target destination’s data specification.
- When developing a source integration, you’ll be transforming your events data received from the source to RudderStack Event Specification.
In this tutorial, we will specifically focus on developing a destination integration.
- Add integration code to rudder-transformer
src/cdk/v2/destinationsfolder. This is the codebase that controls how raw event data from RudderStack Event Data Specification to destination specific data format - Add RudderStack UI configurations in rudder-integrations-config
src/configurations/destinationsfolder
This enables your integration users to setup/configure the integration via RudderStack Dashboard - Write the documentation for your integration (or share the integration plan document with the RudderStack team)
- RudderStack team will deploy your integration to production and post an announcement in the Release Notes
RudderStack team will be available to help you by giving feedback and answering questions either directly on your GitHub PR or the RudderStack Slack community. Before diving into code, writing an integration plan document helps a lot, here’s an example document.
Setup rudder-transformer on your local machine
- Clone this repository
- Setup the repository with
npm run setup - Build the service with
npm run build:clean - Start the server with
npm start
Understand the code structure
src/cdk/v2/destinations- Destination integrationssrc/v1/sources- Source integrationssrc/v0andsrc/v1- Older integrations. If you’re fixing bugs in one of the older integrations, you might find your integration in these folders if it is an older integration.test/integrations/sources- Integration tests for source integrationstest/integrations/destinations- Integration tests for destination integrations
Here, we will explain the code for a destination integration for single event processing as well as batch events processing.
Before diving into the code, we recommend exploring the code for different destinations under the src/cdk/v2/destinations folder. This is the same structure you'll follow for your integration. This is a high-level overview of the code structure which you’ll be creating
src/cdk/v2/destinations/**my_destination**(follow the snake_case naming convention) - Your integration code formy_destinationsrc/cdk/v2/destinations/**my_destination/procWorkflow.yaml**(processor workflow) - This is where you write theyamlconfiguration specifying how to transform an event from standard RudderStack format to destination format. Thisyamlcode will later automatically get converted to javascript code using our RudderStack Workflow Engine. You will need to familiarize yourself with RudderStack Workflow Engine to understand what key/value you should be using. We will cover a quickstart for the same in this tutorial but whenever in doubt, refer to the RudderStack Workflow Engine docs/readme.src/cdk/v2/destinations/**my_destination/rtWorflow.yaml**(router workflow) - For batch events integration code. This will call the processor workflow to process individual events.src/cdk/v2/destinations/**my_destination/config.js**- Configurations for the integration
Before we dive into the code for a simple transformation for a destination, let’s address the potential questions you might have at this point
Why is the integration code for destination transformation written in yaml instead of javascript?
yaml is a great language for data serialization. It also makes the code short and consistent to read. This is why we moved from javascript to yaml for the data transformation code. It helped us move to a config-driven approach.
Does using yaml limit our flexibility to write complex transformation logic?
No. We use a templating language to support any kind of complex transformation logic in yaml, so we have the extensibility as well as the key benefit - concise and simple yaml for common transformation requirements. This was made possible by the RudderStack Workflow Engine.
What exactly is RudderStack Workflow Engine?
In the rudder-transformer service, we handle data transformation from customer events to destination events. The rudder-transformer service uses the RudderStack Workflow Engine to organize this transformation process into clear, manageable steps. This includes validation, data mapping, enriching with API calls, and handling various event types. These steps were earlier implemented in JavaScript, which, while flexible, was challenging to maintain and standardize. To improve readability, testability, and development speed, we transitioned to a config-driven workflow engine using template languages like JSONata and JsonTemplate, with the capability to extend to additional languages. In short, this is what helped us move transformation code from javascript to yaml.
To develop an integration, how much do I need to understand about RudderStack Workflow Engine?
Not more than what can not be covered in 10 mins. The RudderStack Workflow Engine will convert the yaml file to javascript for transformation. The key question is - how to write this yaml file? And the answer lies in the keys you can use in the yaml. Get familiar following keys, and you should be able to write your first integration
-
bindings- To import external javascript functions and datapath- The file path where we want to import from e.g../config(can omit the extension.js)name- The name by which we want import e.g.Config(can be accessed as$Configor$Config.something)exportAll- Iftrue, imports everything from the file. Default:falsei.e. imports only the object matching thename
-
steps- To express the destination transformation logic as a series of steps. If you have used GitHub action, this will sound familiar. 4.name(mandatory) - A name to track outputs 5.description- Mention what does this step do 6.functionName- The function we want to execute in this step. This function must be defined in the bindings and must have following definition
javascript (input: any, bindings: Record<string, any>) => { error?: any, output?: any }condition- The step will get executed only when this condition is satisfiedinputTemplate- To customize the input. Passed while executing the step. By default, all steps receive the same input as the workflow input, but when we want to modify the input before executing the step, we can use this feature.contextTemplate- By default, all steps receive the current context, but we can use this feature when we want to modify the context before executing the step. This is useful when using external workflows, workflow steps, or template paths.loopOverInput- We can use this feature when the input is an array, and we want to execute the step logic for each element independently. This is mainly used for batch processing.steps- You may define a series of steps inside a step. We will call such a step aWorkflowStepas opposed toSimpleStep.workflowStepPath- We can import the steps from anotheryamlfilebatches- To batch the inputs using filter and by lengthkey- The input name e.g. “heroes”filter- The filter logic e.g..type === “hero”size- The size of the batch
A simple example of procWorkflow.yaml looks like this:
bindings:
- name: EventType
path: ../../../../constants
- path: ../../bindings/jsontemplate
exportAll: true
- name: removeUndefinedAndNullValues
path: ../../../../v0/util
steps:
- name: validateInput
template: |
$.assert(.message.type, "message Type is not present. Aborting message.");
$.assert(.message.type in {{$.EventType.([.TRACK, .IDENTIFY])}},
"message type " + .message.type + " is not supported");
- name: prepareContext
template: |
$.context.messageType = .message.type.toLowerCase();
$.context.payload = {};
$.context.finalHeaders = {
"authorization": "Basic " + .destination.Config.apiKey,
"content-type": "application/json"
};
- name: identifyPayload
condition: $.context.messageType == "identify"
template: |
$.context.endpoint = "https://api.fullstory.com/v2/users";
$.context.payload.properties = .message.traits ?? .message.context.traits;
$.context.payload.uid = .message.userId;
$.context.payload.email = .message.context.traits.email;
$.context.payload.display_name = .message.context.traits.name;
- name: trackPayload
condition: $.context.messageType == "track"
template: |
$.context.endpoint = "https://api.fullstory.com/v2/events";
$.context.payload.name = .message.event;
$.context.payload.properties = .message.properties;
$.context.payload.timestamp = .message.originalTimestamp;
$.context.payload.context = {};
- name: validateEventName
condition: $.context.messageType == "track"
template: |
$.assert(.message.event, "event is required for track call")
- name: mapContextFieldsForTrack
condition: $.context.messageType == "track"
template: |
$.context.payload.context.browser = {
"url": .message.context.page.url,
"user_agent": .message.context.userAgent,
"initial_referrer": .message.context.page.initial_referrer,
};
$.context.payload.context.mobile = {
"app_name": .message.context.app.name,
"app_version": .message.context.app.version,
};
$.context.payload.context.device = {
"manufacturer": .message.context.device.manufacturer,
"model": .message.context.device.model,
};
$.context.payload.context.location = {
"ip_address": .message.context.ip,
"latitude": .message.properties.latitude,
"longitude": .message.properties.longitude,
"city": .message.properties.city,
"region": .message.properties.region,
"country": .message.properties.country,
};
- name: mapIdsForTrack
condition: $.context.messageType == "track"
template: |
$.context.payload.session = {
"id": .message.properties.sessionId,
"use_most_recent": .message.properties.useMostRecent,
};
$.context.payload.user = {
"uid": .message.properties.userId ?? .message.userId,
}
- name: cleanPayload
template: |
$.context.payload = $.removeUndefinedAndNullValues($.context.payload);
- name: buildResponseForProcessTransformation
template: |
$.context.payload.({
"body": {
"JSON": .,
"JSON_ARRAY": {},
"XML": {},
"FORM": {}
},
"version": "1",
"type": "REST",
"method": "POST",
"endpoint": $.context.endpoint,
"headers": $.context.finalHeaders,
"params": {},
"files": {}
})You’ll need some API request client (e.g. Postman, Bruno, curl, etc.) for manual testing.
- Make a
POSTrequest to this endpoint -POST /v1/destinations/my_integration - Body - An array of event data object received from the source i.e.
[{ …eventData }] - Headers -
Content-Type: application/json
Depending upon your integration behavior for different types of event, you can get different types of output.
This is what you’d want to do most of the time. The standard case, when you only transform the incoming event and hand it over to RudderStack to deliver to the destination.
For a successful event processing in such case, you should receive HTTP 200 OK response status with data in the following structure
[
{
"output": {
"batch": [
{
...transformedEventData
}
]
}
}
]You can customize the response to the HTTP request from the source beyond the standard event response format sent by RudderStack. You can customize the response body, content type header, status code, etc. Such customization is useful in some cases when you do not want to receive the standard response from RudderStack e.g. you want a custom response body and the HTTP response status code.
4. Write automated tests for your destination integration {#4-write-automated-tests-for-your-destination-integration}
Follow the test structure similar to other integrations in test/integrations/destinations. Here’s an overview of the test code structure which you will be creating
test/integrations/destinations/my_destination- To contain all your test code formy_destinationtest/integrations/destinations/my_destination/processor- Test code for single event processing logictest/integrations/destinations/my_destination/router- Test code for batch events processing logic
You may reuse the same request data from the manual tests i.e. use them in place of replaceThisWithYourEventPayloadProps and replaceThisWithYourTransformedEventOutput (including the enclosing [{ }]). But make sure to redact any personal or secret information.
Here’s an example
export const data = [
{
name: 'fullstory',
description: 'Complete track event',
feature: 'processor',
module: 'destination',
version: 'v0',
input: {
request: {
body: [
{
message: {
anonymousId: '78c53c15-32a1-4b65-adac-bec2d7bb8fab',
channel: 'web',
context: {
app: {
name: 'RSPM',
version: '1.9.0',
},
campaign: {
name: 'sales campaign',
source: 'google',
medium: 'medium',
term: 'event data',
content: 'Make sense of the modern data stack',
},
ip: '192.0.2.0',
library: {
name: 'RudderLabs JavaScript SDK',
version: '2.9.1',
},
locale: 'en-US',
device: {
manufacturer: 'Nokia',
model: 'N2023',
},
page: {
path: '/best-seller/1',
initial_referrer: 'https://www.google.com/search',
initial_referring_domain: 'google.com',
referrer: 'https://www.google.com/search?q=estore+bestseller',
referring_domain: 'google.com',
search: 'estore bestseller',
title: 'The best sellers offered by EStore',
url: 'https://www.estore.com/best-seller/1',
},
screen: {
density: 420,
height: 1794,
width: 1080,
innerHeight: 200,
innerWidth: 100,
},
userAgent:
'Dalvik/2.1.0 (Linux; U; Android 9; Android SDK built for x86 Build/PSR1.180720.075)',
},
event: 'Product Reviewed',
integrations: {
All: true,
},
messageId: '1578564113557-af022c68-429e-4af4-b99b-2b9174056383',
properties: {
userId: 'u001',
sessionId: 's001',
review_id: 'review_id_1',
product_id: 'product_id_1',
rating: 5,
review_body: 'Sample Review Body',
latitude: 44.56,
longitude: 54.46,
region: 'Atlas',
city: 'NY',
country: 'USA',
},
originalTimestamp: '2020-01-09T10:01:53.558Z',
type: 'track',
sentAt: '2020-01-09T10:02:03.257Z',
},
destination: {
ID: '1pYpzzvcn7AQ2W9GGIAZSsN6Mfq',
Name: 'Fullstory',
DestinationDefinition: {
Config: {
cdkV2Enabled: true,
},
},
Config: {
apiKey: 'dummyfullstoryAPIKey',
},
Enabled: true,
Transformations: [],
},
metadata: {
destinationId: 'destId',
workspaceId: 'wspId',
},
},
],
},
},
output: {
response: {
status: 200,
body: [
{
output: {
body: {
JSON: {
name: 'Product Reviewed',
properties: {
userId: 'u001',
sessionId: 's001',
review_id: 'review_id_1',
product_id: 'product_id_1',
rating: 5,
review_body: 'Sample Review Body',
latitude: 44.56,
longitude: 54.46,
region: 'Atlas',
city: 'NY',
country: 'USA',
},
timestamp: '2020-01-09T10:01:53.558Z',
context: {
browser: {
url: 'https://www.estore.com/best-seller/1',
user_agent:
'Dalvik/2.1.0 (Linux; U; Android 9; Android SDK built for x86 Build/PSR1.180720.075)',
initial_referrer: 'https://www.google.com/search',
},
mobile: {
app_name: 'RSPM',
app_version: '1.9.0',
},
device: {
manufacturer: 'Nokia',
model: 'N2023',
},
location: {
ip_address: '192.0.2.0',
latitude: 44.56,
longitude: 54.46,
city: 'NY',
region: 'Atlas',
country: 'USA',
},
},
session: {
id: 's001',
},
user: {
uid: 'u001',
},
},
JSON_ARRAY: {},
XML: {},
FORM: {},
},
version: '1',
type: 'REST',
method: 'POST',
endpoint: 'https://api.fullstory.com/v2/events',
headers: {
authorization: 'Basic dummyfullstoryAPIKey',
'content-type': 'application/json',
},
params: {},
files: {},
userId: '',
},
metadata: {
destinationId: 'destId',
workspaceId: 'wspId',
},
statusCode: 200,
},
],
},
},
},
{
name: 'fullstory',
description: 'Missing event name',
feature: 'processor',
module: 'destination',
version: 'v0',
input: {
request: {
body: [
{
message: {
channel: 'web',
context: {
device: {
manufacturer: 'Nokia',
model: 'N2023',
},
userAgent:
'Dalvik/2.1.0 (Linux; U; Android 9; Android SDK built for x86 Build/PSR1.180720.075)',
},
integrations: {
All: true,
},
properties: {
userId: 'u001',
latitude: 44.56,
longitude: 54.46,
region: 'Atlas',
city: 'NY',
country: 'USA',
},
originalTimestamp: '2020-01-09T10:01:53.558Z',
type: 'track',
sentAt: '2020-01-09T10:02:03.257Z',
},
destination: {
ID: '1pYpzzvcn7AQ2W9GGIAZSsN6Mfq',
Name: 'Fullstory',
DestinationDefinition: {
Config: {
cdkV2Enabled: true,
},
},
Config: {
apiKey: 'dummyfullstoryAPIKey',
},
Enabled: true,
Transformations: [],
},
metadata: {
destinationId: 'destId',
workspaceId: 'wspId',
},
},
],
},
},
output: {
response: {
status: 200,
body: [
{
metadata: {
destinationId: 'destId',
workspaceId: 'wspId',
},
statusCode: 400,
error:
'event is required for track call: Workflow: procWorkflow, Step: validateEventName, ChildStep: undefined, OriginalError: event is required for track call',
statTags: {
errorCategory: 'dataValidation',
errorType: 'instrumentation',
implementation: 'cdkV2',
destType: 'FULLSTORY',
module: 'destination',
feature: 'processor',
destinationId: 'destId',
workspaceId: 'wspId',
},
},
],
},
},
},
{
name: 'fullstory',
description: 'Complete identify event',
feature: 'processor',
module: 'destination',
version: 'v0',
input: {
request: {
body: [
{
message: {
userId: 'dummy-user001',
channel: 'web',
context: {
traits: {
company: 'Initech',
address: {
country: 'USA',
state: 'CA',
street: '101 dummy street',
},
email: '[email protected]',
name: 'dummy user',
phone: '099-999-9999',
},
userAgent:
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36',
},
integrations: {
All: true,
},
originalTimestamp: '2020-01-27T12:20:55.301Z',
receivedAt: '2020-01-27T17:50:58.657+05:30',
request_ip: '14.98.244.60',
sentAt: '2020-01-27T12:20:56.849Z',
timestamp: '2020-01-27T17:50:57.109+05:30',
type: 'identify',
},
destination: {
ID: '1pYpzzvcn7AQ2W9GGIAZSsN6Mfq',
Name: 'Fullstory',
DestinationDefinition: {
Config: {
cdkV2Enabled: true,
},
},
Config: {
apiKey: 'fullstoryAPIKey',
},
Enabled: true,
Transformations: [],
},
metadata: {
destinationId: 'destId',
workspaceId: 'wspId',
},
},
],
},
},
output: {
response: {
status: 200,
body: [
{
output: {
body: {
JSON: {
properties: {
company: 'Initech',
address: {
country: 'USA',
state: 'CA',
street: '101 dummy street',
},
email: '[email protected]',
name: 'dummy user',
phone: '099-999-9999',
},
uid: 'dummy-user001',
email: '[email protected]',
display_name: 'dummy user',
},
JSON_ARRAY: {},
XML: {},
FORM: {},
},
version: '1',
type: 'REST',
method: 'POST',
endpoint: 'https://api.fullstory.com/v2/users',
headers: {
authorization: 'Basic fullstoryAPIKey',
'content-type': 'application/json',
},
params: {},
files: {},
userId: '',
},
metadata: {
destinationId: 'destId',
workspaceId: 'wspId',
},
statusCode: 200,
},
],
},
},
},
{
name: 'fullstory',
description: 'Identify event with needed traits',
feature: 'processor',
module: 'destination',
version: 'v0',
input: {
request: {
body: [
{
message: {
userId: 'dummy-user001',
channel: 'web',
context: {
traits: {
email: '[email protected]',
name: 'dummy user',
phone: '099-999-9999',
},
},
timestamp: '2020-01-27T17:50:57.109+05:30',
type: 'identify',
},
destination: {
ID: '1pYpzzvcn7AQ2W9GGIAZSsN6Mfq',
Name: 'Fullstory',
DestinationDefinition: {
Config: {
cdkV2Enabled: true,
},
},
Config: {
apiKey: 'fullstoryAPIKey',
},
Enabled: true,
Transformations: [],
},
metadata: {
destinationId: 'destId',
workspaceId: 'wspId',
},
},
],
},
},
output: {
response: {
status: 200,
body: [
{
output: {
body: {
JSON: {
properties: {
email: '[email protected]',
name: 'dummy user',
phone: '099-999-9999',
},
uid: 'dummy-user001',
email: '[email protected]',
display_name: 'dummy user',
},
JSON_ARRAY: {},
XML: {},
FORM: {},
},
version: '1',
type: 'REST',
method: 'POST',
endpoint: 'https://api.fullstory.com/v2/users',
headers: {
authorization: 'Basic fullstoryAPIKey',
'content-type': 'application/json',
},
params: {},
files: {},
userId: '',
},
metadata: {
destinationId: 'destId',
workspaceId: 'wspId',
},
statusCode: 200,
},
],
},
},
},
];You can run tests only for the specific integration, for example
- To test Slack destination -
npm run test:ts -- component --destination=my_destination
These tests will automatically be run on each commit, making sure that any new development does not break the existing integrations.
Add configuration for your destination integration in rudder-integrations-config repo under src.configurations/destinations. At bare minimum, a db-config file is needed, here’s an example of the same for FullStory destination. Duplicate it in the directory for your integration config folder and change the relevant values for your integration.
Alternatively, the easier path to do this is by running a script which will generate these files for you. For this, first create a copy of this [test/configData/inputData.json](https://github.com/rudderlabs/rudder-integrations-config/blob/develop/test/configData/inputData.json) and adjust it to what you want to see in the RudderStack dashboard ui. And use that placeholder file in the following command
python3 scripts/configGenerator.py <path of the placeholder file>
Run this command from the root directory of the rudder-integrations-config repo. The PR can then be raised after checking if everything looks good.
nvm use v20
npm ci
npm run build:start
- [Contributing.md](https://github.com/rudderlabs/rudder-server/blob/master/CONTRIBUTING.md\) in all github repositories
- https://www.rudderstack.com/docs/resources/community/
- https://www.rudderstack.com/docs/event-spec/standard-events/
- RudderStack Workflow Engine
- RudderStack JSON Templating Engine
- Recording of the community event for new contributors
We look forward to your feedback on improving this project.