Data Mapping Engine (For DSYNC API and Developer Portal click here)
DSYNC is equipped with fully configurable mapping engine where you can setup different transformation and validation rules to modify your data between a source and target endpoints.
Helpful links for mapping APIs using DSYNC:
Mapping engine is part of the connection/link and as such is accessible via the connecting line between endpoints. More precisely, it’s the part of the link which runs from the middle check icon to the destination. Click on the part of the link which runs from the middle of the connection to your destination (or Slave) endpoint and choose the action you want to perform on the right sidebar – create a new mapping template, select from existing, or edit existing mapping template.
Tip: double-click an element on the canvas to quickly access its data layout.
Main Building Blocks
|Data Layout||Schema||Defines all available attributes for an endpoint|
|Mapping Template||Link||Defines the mapping of data layout fields across two or more systems|
The DSYNC platform has been developed to rapidly integrate systems together. Different systems have different data elements. For example, in one system, the data element may be called a “Contact” in another system it may be referred to as a “Customer”. DSYNC has a mapping engine where you can dictate to our platform how and where you would like the data to flow. As part of our mapping engine we also have the ability to set rules and functions to the data you are joining.
As a high level, there are a few key steps to our mapping engine.
Step 1 Ensure there are two systems to link: Before we can begin mapping we must have loaded two different systems onto the DSYNC dashboard. The systems must be authorized to communicate to DSYNC via our API keys and the systems own authorization.
Step 2 Have you authorized the two systems? For security, we have a two-way authorization process meaning we need the system to approve DSYNC accessing the data & secondly, we generate our own secret API keys to ensure the security of your data. The DSYNC API keys are generated through the “My Account Section”. Once you have generated a DSYNC API key for each system please copy and paste the key into the system you are trying to link. Different systems require different authorisations. We commonly work with OAUTH.
Here are some of the common questions about API keys:
- Where do I generate an API key? The DSYNC API key is generated from my account section
- Do I need an API key for each system I am looking to sync data with? Yes, you will need to create and name an individual key for each system you would like to link.
- What should I name my DSYNC API key? You can name the key anything you would like however it is strongly advised to pick a standard naming convention so you don’t get confused later.
Step 3 Let’s begin the mapping:
Mapping is where you can control the data flows between one system and another. To start mapping one data element to another the first step is to click on the data entity you would like to synchronize. One of the features of DSYNC is that you can transform the names of the data between the endpoints. For Example, if one system is called “Customer” and in another system the data point is called “Contact” or is in capitals, or needs additional spaces or fields DSYNC can transform the data to multiple endpoints almost instantly.
- Click on the FIRST data element you would like to connect example “Customer”
- Once you have clicked on the first element then select “Create Relation” in the right toolbar
- Then click on the SECOND data element you would like to connect example “Contact”
This creates a bridge between the two systems and creates the first link to generating the data flow.
Step 4: Name your link: As soon as you have created your first link between the two systems a pop up box will automatically appear and it will ask you to name your system.
Step 5 Create a map between the link (relation): Now that you have created a link between the data you would like to link and named the link. It is time to create a “Map” to pair the attributes together.
- Click on the lines between the two systems
- Then select “New”
- You will then need to name your map
- Maps can be named anything – just make sure it is something standardized so you don’t get lost later.
Once you have named your map you can also set the direction of the data flow. We have set up the system to accommodate Master, Slave and Both. This simply gives you the control of the data flow. The purpose of the mapping engine is to pair one data entity to another.
Step 6: Now we need to “Edit the Map”
- Click on the lines between the system
- On the right-hand side column, there is a button called “Edit” by selecting this field you will now be in the Edit Map section
- The purpose of this section is to overlay the two systems elements you are trying to connect. We would like to pair different data that you would like to flow from one system to another.
- For systems to talk there is always required fields. We indicate the required fields by an asterix next to each of the data elements.
- The best way to test your data mapping skills is to start with the required fields.
- Different systems have different required fields so be sure to check the red asterix when pairing.
- The DSYNC system is drag and drop so you can easily click on multiple elements and then place them next to corresponding field. The intention is to map corresponding fields so the data can be synchronized
- When you click on the data elements the elements are placed for you to match on the behind layout. Then simply drag and drop the data elements to the right positioning.
- Don’t forget to select save
Step 7 Map the opposite direction
In step 6 we mapped one direction of the data. You can always check the direction of the data by the header columns that are called Source and Destination. DSYNC is a bidirectional system meaning that we push data in both directions if the system will let us! In this next section, we will show you how to pair the data in the reverse direction.
- To switch to the opposite direction – Please click on the two directional arrows between the source and destination. This function will show you the opposite direction of the data flows and allow you to perform the mapping on the reverse angle.
- In some cases, there are systems that need to be identically mapped from a data perspective. We have a handy trick where we can copy the original mapping over to the reverse angle called “Pairing Map”
- To save time DSYNC have implemented a copy button – When selecting “Pairing Map” you can copy the existing mapping from one side to another.
- Some of our customers prefer one way mapping or even customizing the direction of the data flow.
Step 8 Turn the Job/map on and test
After you have created the mapping we need to initialize and turn on the new mapping.
- To turn on the data sync please select the circle in the right corner to turn on the map.
Welcome to another DSYNC demonstration where we will be showcasing how our transformation engine works. Transformation is when you would like to set business formatting rules for data you are sending between the systems. With DSYNC you can set multiple transformation rules to different systems this enables customers to use one set of data to be transformed to meed multiple applications required endpoints.
As a high level our transformation engine allows certain rules to be applied in the mapping section that transforms the data to a required format.
There are many use cases for transformation from changing a birthdate format from “American format” to the “English format” or simply adding additional fields for apartment, suite or building number fields. The DSYNC transformation section enables you to alter the data to suite multiple end points. For example, if a customer changes an American birthdate in a CRM then the POS, Website and shipping application can be updated at the same time with the right formatted data.
In this video demonstration, we will be showcasing a simple rule of changing one filed from lower case lettering to UPPER case lettering. This may be to suite a condition required in one system.
- Step one is to visit the DSYNC dashboard and enter the mapping section
- Step two is to click on the line between the two systems you would like to transform the data.
- Step three is to find the specific data element you would like to transform. In this tutorial example, we will be making a rule to change first name of the “customer” to be transformed to show the data in capitals.
- Step four click on the “FX” icon next to the data you would like to set the rule for.
- In this example, we will be setting the rule to make all first names sent to another system in all upper-case letters.
- As you select the FX icon an “Edit Functions” pop up box will appear. Within the edit functions box there is a list of different rules you can set rules for ranging from Capitalize, Lower Case, Lower Case Position, Lowercase sub-string, Upper case, Upper case position, Uppercase Substring and more.
- For this example, we will be utilizing the Upper-Case function.
- Select Add Function – Then select OK
- In this example, we will also be setting the same transformation rule to the reverse direction. After saving the changes to the new map simply change the direction of the Map by utilizing the two arrows to switch mapping direction.
- It is easy to apply the same rules by selecting the FX icon, then in the “Edit Functions” box apply the Upper-case rule as applied before.
- Don’t forget to save the new edits to the map.
- The transformation rules should now be set for testing. For testing purposes let’s visit the first system and change Roberts last name to Pineapple. As we save and refresh the second system you can see that Roberts fist name is all in upper case or capitals.
- There are several use cases for this type of feature ranging from ensuring your SKUs are in the right format, your customers data details are entered in the right format. Or your $ value for your sales orders are in the right format to be moved across systems.
Now it is time to test the integration