Data Transformation

Upper Case

Description

This operations helps to convert a given key value into Upper Case from any JSON Dataset. This operations take place while data is in-flight

Number of Parameters : 1

Parameter : Uppercase

Provide comma separated list of keys in double quotes to convert the defined key values into Upper Case

Below is a example where we are converting the value of first_name and last-name into Upper Case

"first_name", "last_name"

Lower Case

Description

This operations helps to convert a given key value into Lower Case from any JSON Dataset. This operations take place while data is in-flight

Number of Parameters : 1

Parameter : Lowercase

Provide comma separated list of keys in double quotes to convert the defined key values into Lower Case

Below is a example where we are converting the value of first_name and last_name into Lower Case

 "first_name", "last_name"

Data Type

Description

Data type operation can convert any key's value into it's data type like 

1. String type Boolean into Boolean data type 
2. String type Float into Float  data type
3. String type Integer into Integer data type 
4. String type Datetime into Datetime data type

Number of Parameters : 4

Parameter : Boolean

A string type Boolean can be converted into Boolean data type .

String type Boolean examples: "True","False","0","1" etc.

Booleans datatype do not contain double quotes.

Below is a example where we are converting the values of test_passed key into Boolean data type .

"test_passed"

Parameter : Float

A string type Float can be converted into Float data type 

Below is a example where we are converting the value of Amount into JSON data type float.

"Amount"

Parameter : Integer

A string type Integer can be converted into Integer data type. 

Below is a example where we are converting the value of Quantity into JSON data type Integer.

"Quantity"

Parameter : Date Time

A string type Date Time can be converted into Date Time data type . 

From date format : The date/datetime format in which the user is defining their date/datetime.

To date format : The date/datetime format in which the user wants to convert their date/datetime.

Example: ["key_name","From date format","To date format"]

Below is a example where we are converting From date format to  To date format inside JSON data type.

"startweekdate1","%Y-%m-%d %H:%M:%S.%f%z","%Y-%m-%d %H:%M:%S","startweekdate2","%Y-%m-%d %H:%M:%S","%Y-%m-%d"

Append

Description: 

Append operation adds a new key on the fly with it's value as dynamic value or static value. This will add key and it's value for each record.

Number of Parameters : 1

Parameter : Append

By adding new elements to the end of an existing data structure, the append operation can help to extend or modify the data structure in a flexible and efficient way.

For dynamic value the user can use {%column_name%} where column_name is the incoming column in the data pipeline

Below is an example where we are adding a new key in-flight with its fixed/static value.

"export_flag_y":"Y","export_flag_p":"P"

Below is an example where we are adding an array with key's value with dynamic values to the keys. We can append multiple keys at a time.

"array_key": [{"transportmode":"{%MODE_OF_TRANSPORT%}","orderType": "{%ORDER_TYPE%}"}]

Below is an example where we are adding an object with key's value with dynamic values to the keys. We can append multiple keys at a time.

"keyname": {
                "id": "{%id%}",
                "email": "{%email%}",
                "first_name": "{%first_name%}"
            }

Title Case

Description:

Title Case operation helps in the converting a given key's value into title case. 

Number of Parameters : 1

Parameter : Title Case

Provide comma separated list of keys in double quotes to convert the defined key values into Title Case

Below is an example where we are converting the values of amount and first_name into Title Case.
 

"amount","first_name"

Data Extractor

Description: 

Data Extractor operation is designed to extract specific data from JSON response. 

Number of Parameters : 2

Parameter : Data_Extractor

Data_Extractor is used to extract keys and its value from a JSON response.

Parameter :Data_Extractor_Keys

This helps to provide user defined keys. If left blank it will auto generate keys.

Below is an example where we are extracting the values of access_token and feedDocumentId.

"['access_token']","['feedDocumentId']"

Trim 

Description: 

Trim operation helps in removing unnecessary parts from the given key's value as defined by the user.

Number of Parameters : 1

Parameter : Trim

Provide comma separated list of keys in double quotes to trim the defined key values.

Below is an example where we will trim the defined key value first_name

"first_name"

JSON to String

Description: 

JSON to String operation is used to convert JSON into a String.

Number of Parameters : 1

Parameter : JSON to String

Provide comma separated list of keys in double quotes to convert the defined key's value from JSON to String.

Below is an example where we are converting the values of key1 and key2 from JSON to String

"key1","key2"

String to JSON

Description: 

String to JSON operation is used to convert a String into a JSON . 

Number of Parameters : 1

Parameter : String to JSON

Provide comma separated list of keys in double quotes to convert the defined key's value from String to JSON

Below is an example where we are converting the value of key1 and key2 from JSON to String

"key1","key2"

JSON to XML

Description: 

JSON to XML operation helps to convert JSON object or value into XML.

Number of Parameters : 2

Parameter : Key_data

Key_data converts the provided key's value from JSON to XML.

Below is an example where we are converting the value of product_data_response from JSON to XML.

product_data_response

 

Parameter : Response_key

Response_key holds the converted XML value under the specified key name which is easy to access.

Below is an example of the key name data_response which will hold the converted XML value, 

data_response

XML to JSON

Description: 

XML to JSON is used to convert XML into JSON. 

Number of Parameters : 2

Parameter : Get_key

Get_key converts the provided Key's value from XML to JSON
Below is an example where we are converting the value of product_data_response from XML to JSON.

product_data_response

Parameter : Response_key

Response_key take note of the converted value under the specified key name which is easy to access.
Below is an example of the key name data_response which will hold the converted JSON value, 

data_response

Base64 Encoding

Description: 

Base64 operation is used to convert some specific key to base64 encoded, the user can give multiple 
keys if they require.

Number of Parameters : 1

Parameter : Base64_encode

Used to encode the values of given key into base64.

Below is an example where we are encoding the value of email into Base64.

"email"

Base64 Decoding

Description: 

Base64 decoding operation is used to convert a Base64-encoded string back to its original data format, the user can give multiple 
keys if they require.

Number of Parameters : 1

Parameter : Base64_decode

Below is an example where we are decoding the Base64 encoded value of email back to original data.

"email"

Generate Array Sequence Number

Description:

Generate Array Sequence Number operation helps in generating sequence number for each row.

Number of Parameters : 2

Parameter : Sequence_key

It is the key name in which the single line data is present whose sequence the user needs to give.
Below is an example of the key_name which holds the single line data.

key_name

Parameter : Sequence_number_key

It is the name of the new key for sequence
Below is an example of the key name DATA which will store the sequence number.

DATA

Send Keys top of Root

Description:

Column_to_root operation helps in bringing the given nested key's value to the top of root.

Number of Parameters : 1

Parameter : Column_to_root

Provide comma separated key name in double quotes to specify the key's value.

Below is an example where we are giving the nested key as key_name whose value we want to bring at the top of the root.

"key_name"

Today Timestamp

Description:

Today Timestamp operation adds a new on the fly with the value of today's date/ date time as specified by the user.

Number of Parameters : 2

Parameter : Date_format

Date_format is used to pass the required format.
Below is an example of a datetime format which can be modified according to user's need.

%Y-%m-%dT%H:%M:%S.%f%z

Parameter : Datetime_key

Datetime_key is the name of key in which the user wants to save their date/ datetime

Below is an example of key name dl_insert_date which will add a new key on the fly with date format value.

dl_insert_date

Round

Description:

Round operation is used to reduce a decimal number to a specific number of decimal places, where the numbers need to be rounded off.

Number of Parameters : 2

Parameter : Round_keys

Round_keys is used to access the specific key that needs to be rounded off, define by the user.
Below is an example of the key name's float_value and int_value whose value we want to round off. 

"float_value","int_value"

Parameter : Decimal_key_number

Decimal_key_number specifies how much decimal places the user needs.
Below is a example of the decimal key number till where we are  rounding off the value. 

2

Calculator

Description: 

Calculator operation is used when we want to calculate any particular process provided by the user depending upon the values which are in the columns.

Number of Parameters : 2

Parameter : Calculation_keys

Calculation_keys holds the calculated formula based on the column names of the provided data.
Below is a example where we are providing calculated formula's Amount1 - Amount2 and Amount1 + Amount2 based on column names of data.

"Amount1-Amount2","Amount1 + Amount2"

Parameter : New_calculation_keys

New_calculation_keys used to  store the calculated values.

Provide comma separated list of keys in double quotes to specify the value.

Below is an example of new keys name key1 and key2 which will store the calculated values.

"key1","key2"

Date Analytics

Description: 

Date analytics helps us to extract information about the date like the financial year, financial month, quarterly information of year as well as financial year and many other relatable information about the date. .

Number of Parameters : 4

Parameter : Data_field_key

Data_field_key

In Data_field_key we provide the key name which holds the date.
Below is the example of the key name Created_datetime which holds the required value.

Created_datetime

Parameter : Fiscal_month_start 

Fiscal_month_start is used to specify the fiscal start month of an organization.
Below is an example of the organization's fiscal start month as April so we are taking the month number of the year i.e. 4.
 

4

Parameter : Date_column

Date_column is used for giving user defined 11 fields for savings data. If left blank by user it will generate 11 fields dynamically.
Below is an example of user defined 11 fields. 

"Calender Month","Calender Month Num","Calender Year","Calender FY Month Num","Calender FY","Calender Week Num","Calender Month Week","Calender Qr Num","Calender FY Qr Num","Calender Qr","Calender FY Qr"

Below is an example when user leaves date_column blank

Parameter : Weekday

Weekday is used to specify the starting of the week, %W (starting of week from Monday) and %U (starting of week from Sunday) .
Below is an example of weekday starting with Monday, 

%W

Repeat First Row Values

Description:

Repeat First Row Values operation is used to repeat the 1st row's value of the specified column's.

Number of Parameters : 2

Parameter : Source_key

Source_key is the key which holds the user's single line data that we need to pass.
Below is the example of  key name that holds the data

['product_data_response']['data']

Parameter : Fields_to_repeat_value

Fields_to_repeat_value is a list of key names comma separated in double quotes whose first row value we want to repeat.

Below is an example of key names whose 1st row value we want to repeat

"month","customer_site"