Skip to main content

Data Chopping and Aggregation Strategy in eZintegrations

Enterprise integrations often deal with hierarchical business data containing multiple levels of related information.

Examples include:

  • Orders containing Shipments and Items
  • Invoices containing Invoice Lines
  • Claims containing Claim Entries
  • Payments containing Transaction Records

A common implementation pattern is to start processing from the top-level business object and progressively break the data into smaller parts.

While this works functionally, it creates dependency on the parent hierarchy throughout the workflow.

In eZintegrations, the recommended strategy is to process data from the lowest meaningful transactional level and progressively build higher-level business summaries.

Current Processing Pattern

In many integrations, processing typically follows this pattern:



Parent Business Object
   ↓
Break into Sub-Level Records
   ↓
Break into Detailed Records
   ↓
Prepare Final Business Summary


Although the data is eventually split into granular records, the workflow is still driven by the parent hierarchy.

Example: Order Management

Common Processing Flow



Orders
   ↓
Shipments
   ↓
Items
   ↓
Final Order Summary


In this model:

  • Processing starts from Orders
  • Data is progressively broken downward
  • Final business summaries are prepared only at the end

This creates dependency on the complete Order hierarchy during processing.



Items
   ↓
Item-Level Processing
   ↓
Build Shipment Details
   ↓
Build Final Order Summary


Here:

  • Items become the primary processing unit
  • Business calculations originate from transactional records
  • Higher-level summaries are progressively derived from finalized item data

Functional Perspective

From a business perspective, the recommended approach focuses on processing smaller transactional units independently instead of depending on the complete parent business object throughout the workflow.

This creates a cleaner and more manageable processing model for hierarchical business transactions.

Technical Perspective

From a technical architecture perspective, the strategy follows a lowest-level-first processing model.

Data Aggregation Flow



Lowest-Level Records
   ↓
Independent Processing
   ↓
Data Aggregation
   ↓
Derived Parent Summaries


The parent business object becomes a derived summary instead of the primary processing controller.

Key Principle

Data Chopping Alone Is Not Enough

Breaking hierarchical business data into smaller records is only one part of the processing strategy.

The more important aspect is how the business flow is designed after the data is split.

If processing still depends heavily on the parent business hierarchy, the workflow becomes tightly coupled and difficult to manage as transaction volume increases.

  • Process the lowest meaningful transactional records first
  • Perform business processing at the granular level
  • Progressively derive higher-level business summaries from processed records


Conclusion

In eZintegrations, hierarchical business data should not be processed primarily through parent-driven traversal.

Instead, integrations should:



Process Lowest-Level Records
            ↓
Independent Processing
            ↓
Data Aggregation
            ↓
Derived Business Summaries


This forms the recommended data chopping and aggregation strategy in eZintegrations.

Back to top