Data Pipeline Controls
Sleep Time
Description:
Sleep Time operation helps in giving a pause of some seconds which is defined by the user.
Number of Parameters : 1
Parameter : Sleep Time
It specifies the sleep time duration in seconds.
Below is an example where we are keeping sleep time duration of 60 seconds.
60Python Operation
Description:
Python operation helps in when a user has a particular need or requirement to perform a special operation that is not predefined, a user can develop a new operation that will meet their unique requirements.
Number of Parameters : 1
Parameter : Pycode
In the below example we are writing a new python code.
"\nResponsedata=[]\nsum= data.split('\\n\\n')[0].split(':')[1].split('\\n')[1:]\nfor s in sum:\n\trow = (s.strip().split('\\t\\t'))\n\t\n\td2={}\n\tfor i in range(0, len(row), 2):\n\t\td2[row[i]]=row[i + 1]\n\tResponsedata.append(d2)\nerror_records = data.split('\\n\\n')[1].split('\\n')\nerr_headers=error_records[0].split('\\t')\nerr_rows=error_records[1:]\nfor line in err_rows:\n\terr_row = line.split('\\t')\n\tel = zip(err_headers,err_row)\n\td = {}\n\tfor t, f in el:\n\t\td[t] = f.strip()\n\tResponsedata.append(d)\n",