Full DSYNC tutorial for developers to understand how to operate the system:
What is the mapping section? The DSYNC data mapping section is where you can map, pair, and transform data between your two systems data layouts. Maps enable the user extra flexibility for integrating systems and allow customized options to be entered.
Maps are created between two systems to identify what data should move and how it be presented to the destination endpoint.
Do I have to map all o the data? No only sync and map the data that you need to be synchronized across. Within the mapping section (data layout) you can disable data that is unnecessary to prevent an error being sent back.
Do I have to SYNC every element / field? NO in-fact we recommend you do DISABLE data fields you do not want to sync across to another system this will improve any errors you will see headed back. Here is a video showing how to disable data you do not want to sync
The mapping engine allows you to map one or more source fields onto each destination field. Each destination field has its’ name and data type visible. Data types can be set on appropriate data layout when adding/editing a field. Mapping engine does not allow you to modify the data layout in any way.
Data types play an important role when mapping source field values onto destination. DSYNC allows you to map different data types but what goes out to a destination system is ultimately formatted based on the destination data layout and its’ fields. This means that you can map a number to a text field however a textual value (string) will be outputted and sent to the destination system.
Text: any textual value (string)
Number: any numeric value, allows user to perform calculations (int, double, float, ...)
URL: any URL (must have correct format; ie. http://mydomain.com)
Email: an email address (must have correct format; ie. firstname.lastname@example.org)
Date: date value in specific format (user specified format)
Time: time value in specific format (user specified format)
Datetime: date-time value in specific format (user specified format)
Boolean: logical value - true or false (user can specify mode: standard true/false, numeric 1/0, custom)
Object: special data type - container for data fields, used to model parent-child relationship
When you first open the mapping page you are presented with all the destination fields as per your destination data layout. All fields are unmapped by default and show ‘Click to add field’ link.
To map a field, click the ‘Click to add field’ link. A modal window opens where you can see your entire source data layout and all your source fields.
By clicking on source fields, you start to add/map fields onto the destination field you have selected. From the example above, clicking on ‘attribute_1’ field maps that field to ‘TrackQtyOnHand’ destination field.
What this means is that when the job runs, the value from ‘attribute_1’ is mapped to the ‘TrackQtyOnHand’ field. When DSYNC sends data to the destination system, there will be a ‘TrackQtyOnHand’ field and the value of this field will be the same as the value of ‘attribute_1’ field which came into DSYNC from the source system.
As mentioned previously, you can map more source fields onto one destination field. If for example your destination system requires Full Name and all you get from source is First and Last names, you can add both source fields and concatenate these by space like so…
The space between the first name and last name is a static value which you can add using the ‘Add custom field’ option. Just select the ‘Add custom field’, type space in the ‘String constant’ field, and click the ‘ADD’ button.
Using the ‘Add custom field’, not only you can add static values but also perform numeric calculations. Imagine for example that you run an online shop and your supplier is sending you a daily products feed with their own prices. Your product mark-up is 45% and you want to import that feed straight into your web shop. You can take the field which holds the supplier’s product price and add 45% on top of it easily by performing a numeric calculation on the field like so…
The value of price is multiplied by 1.45 and the result gets put into the ‘UnitPrice’ destination field (sent to your online store).
Setting Default Values
Sometimes you will not get data from your system which can be mapped to a destination field however some systems may require you to send a value anyways. You can either use the ‘Add custom field’ however that will prevent you from mapping any dynamic source field onto your destination.
Each destination field can be set a default value which is put into the output data when there is no source value present. To access the Default value settings, click on the fx icon next to the destination field name. Alternatively, you can also select the pencil icon at the end of destination field.
A modal window opens where you can set a Default value.
Not all fields from the destination data layout must be mapped. Required fields are easily identifiable by red asterisk *next to the destination field name. These should be always mapped. Other fields are optional which means the destination system does not expect a value to be sent in that field or does not need the field to be sent at all.
You can easily disable/enable a field by clicking the ‘disable’ icon at the end of the destination field.
Disabled fields are ignored by DSYNC and will not be sent to the target system at all. This is especially useful when dealing with strict systems. Some systems may not require a field which holds a date but sending that field with no value causes the system to return ‘Invalid Date’ error. Disabling the field all together fixes this issue.
Core steps when mapping data between the two systems
- Create Relation/ Link
- Create a map (template link below)
- Edit the map
- Match the required fields
- Set the reverse direction
- Set the schedule
- Enable the sync
Data Mapping FAQs
Where do I find ready made mapping templates?
The DSYNC team have prepared commonly used mapping templates that can be imported directly into your mapping section. If you are struggling with mapping one system to another please reach out to our team.
How do I identify the required fields? The required fields have a small red asterisk to the right of the data entity. Required fields are labelled with a small red asterisks *
Will my synchronization work if I have not matched all the required fields? It will probably fail, required fields are usually indicated as a required field because the system is requesting information to be passed through.
I cannot find a matching data point for a required field, what should I do? Data fields between systems do not have to match. Some systems will require the endpoint to be displayed in a particular format. For example the data element you are trying to match may be required in a number format. Within the mapping section you will be able to see that these fields are labelled with "123" on the left side of the element.
How do I check the direction of the data flow in the mapping screen? The direction is indicated in the top left of the mapping engine. The “source” is where the data is coming from and the “destination” is where the data is going to.
Will the synchronization run automatically? No, you must enable and schedule the job to run at a specific time. To activate the specific job please select the “enable” circle after highlighting your job.
When would I use a Custom field? Custom fields can be used when there is not a matching data element. You can also use the custom field function to enter a formula to change your data during your data sync. For more information on how to set formulas with your custom fields here.
What is a String constant? A String constant is the type of data you can enter into the custom field section.
What are some examples of different fields I can map? Please see example grid below for example a ZIPCODE can be mapped to POSTCODE, PostCode or an English date can be mapped to an American data or you can combine data fields a common example is Full name can be mapped to "FirstName" + "LastName" or you can transform lowercase data to all be capitalized. There are many different variations where DSYNC can be used.
What is the (fx) functions button? Within this section you can set rules and calculations to change your data. The FX function is used to set rules and transform your data.
I would like to format my data so there is a space between my two data layouts, how do I do this? A common scenario is when formatting data to represent a “Full name”. A full name is usually the first name and the last name joined together. Rather than having the output as firstnamelastname you can format the data with a “Space” so the result would be firstname last name. To enter a space between two data points please use the “Add custom field” button. By selecting “Add custom field” then leave the string constant section blank and select “Add Field”. This will activate a space that can be positioned in the top toolbar to help you format the data.
If I don’t have a second email address and there is nothing to pair the element with what do I do? For testing purposes, you can simply add a custom field and enter a bypass email address.
What is the Pairing Map? The pairing map is a copy function that can be used to save time when mapping data in the opposite direction. The pairing map will reflect the data you have paired within the original mapping template.
Will there be required fields on one system but not another? Yes, each system will have its own dependencies. For this reason, it is important to check the required fields for both directions of the data.
I would only like my data pushed in one direction, is this possible? Yes, within the DSYNC dashboard you can select To Master, To Slave and both to indicate the direction of the data flow.
Can I name my map anything? Yes, both the name and the description is entirely up to you. We would encourage some naming conventions to help you identify the templates in the future.
What is a Relation? A Relation is the high-level link between the two systems. Creating a relation is the first step in the mapping process. Within this step, we indicate at a high level what is going to be synchronized. For example, Customer in one system to Contacts in another. Or Inventory in one system to Products in another.
Do I have to match every data point with another data point? No, only the required fields need to be mapped to another data point. Required fields are indicated by a small red asterisk.
One of my systems is asking for a middle name, how can I insert into the data mapping properly? First start by selecting all the parts of the name example First, Middle and Last name. Then select “Add custom field” by leaving the string constant blank and selecting “Add” you will create spaces to be added to the top toolbar. Organise the top toolbar with the correct layout. First “Space” middle ”space” then last name.
To access element’s mapping template please refer to ‘Working with DSYNC’ section.
Once you have created a link between two endpoints the last configuration you need to do is map individual fields from one data layout to another. There are two ways of defining a mapping for a connection:
- Use Existing Map Template
- Create Map Template Manually
Using Existing Map Template
DSYNC system comes with predefined mapping templates which can be easily used on any of your connections. To apply an existing map template to a connection,
- Select the link between the endpoints you want to apply the mapping template to.
- In the right sidebar under the ‘MAP’ section, click the ‘SELECT’ button.
- A modal window appears with available mapping templates. Select the template you want to use for your connection by clicking on its’ name.
- Mapping template is applied to your connection. You can review your mapping template by clicking on the ‘EDIT’ button under the ‘MAP’ section in the right sidebar.
Creating Map Template Manually
For all your custom connections, you will be required to define your map template manually. To create a new mapping template,
- Select the link between the endpoints for which you want to create a mapping template.
- In the right sidebar under the ‘MAP’ section, click the ‘NEW’ button.
- Enter the name and optional description for the mapping template and click ‘OK’.
- A new mapping template is created and the mapping page opens.
- Map your fields as per your requirements and save the template. To learn more about mapping please refer to the ‘Mapping Engine’ section.
Note: It is recommended to use descriptive names for your mapping templates as you may want to reuse map templates in other connections. The best practice is to list the systems you are connecting and the resource/object being synchronized. Example: Magento > VendHQ Sales Orders.
Reusing Mapping Templates
Every single mapping template you create in your account is added to your mapping template library.
The library is accessible via the ‘SELECT’ button under the ‘MAP’ section. The library also opens every time you create a new link between two endpoints.
You can import new mapping templates into the library using the ‘IMPORT’ button, export mapping template in a JSON file using the export icon, or delete a mapping template using the delete icon next to the template’s name. Some map templates cannot be deleted – these are the DSYNC prepared predefined mapping templates available in your account from the start.