Data Source - Bizintel360 Data Lake

Bizintel360 Data Lake is a search engine based NO-SQL database owned by Bizdata. Users can ingest PBspetabyte, zettabyte, yottabyte  of records both structuresstructured and unstructured for Analytics, Storage, Machine Learning and deep learning.

Bizintel360 Data Lake Source is a connection pool in eZintegrations platform to retrieve data in JSON format.

Bizintel360 Data Lake Source have following Parameters :

Data Lake Version : Data Lake Version is the Data Lake Name and its version assigned to the organization by Bizdata

Index / Table Name: Index or Table name that you want to retrieve data from Data Lake. For the List of table name or index, check `Datalake` section in Bizintel360 Visualization product.

Pagination Wait Time : By default it is 2m, where m is minute. How long need to wait for next page. If the response of Table/Index is very high ( having 100+ keys in a single record) then try increasing the `Pagination Wait Time'. In general `2m` is sufficient to stream data. Use this when you have high network traffic congestion

Timeout : By default it is 2m, where m is minute. In general `2m` is high enough to get response from Bizintel360 Data Lake. Increase this when response from Bizintel360 Data Lake slow. This can happen when the Data Lake Cluster size is small. Reach Bizdata support team to see increase in cluster size of Bizintel360 Data Lake.

Size : By default it is `1000`. Size is number of streaming record count from Bizintel360 Data Lake source. The source will stream the records inside pipeline in size of 1000 chunks and move to operations and finally to Data Target.

Query