The data integration consumption monitor helps you track and understand how much data is being replicated or applied through your data integration projects and pipelines. This feature provides visibility into your data usage across your workspace.
The data integration consumption monitor displays metrics about the volume of data being transferred through your replication projects and pipelines. This information helps you understand your data usage patterns and manage your resources effectively. The consumption monitor measures data in bytes, which represents the total amount of data that has been replicated or applied in your workspace. This metric is tracked across all your continuous and mainframe replication projects.
Key benefits
- Track total data volume replicated across your workspace
- View data consumption broken down by time periods
- Monitor consumption at the project and pipeline level
- Understand which replication operations consume the most data
- Plan resource allocation based on actual usage patterns
Ways to view consumption data
You can view your data integration consumption in three different ways:
- DIS Usage Screen: The main usage screen displays your total data integration volume in a dedicated section. You can click on the details to see how your data consumption is distributed across different time periods, including the last hour, last 24 hours, last 30 days, monthly, and your entire billing period.
- Catalogue Asset Level: For continuous replication projects and pipelines, you can view consumption metrics directly in the catalogue. When you search for a replication project asset, you see two fields: transfer unit (in bytes) and transfer volume (total volume). This view is available only for continuous projects, not for mainframe projects.
- Tableau Reports: Advanced reporting is available through Tableau reports, which provide detailed breakdowns of consumption by user and workspace. These reports offer additional filtering and analysis capabilities for deeper insights into your data usage.
Understanding the metrics
- Transfer Unit: The transfer unit is measured in bytes. This represents the raw data volume transferred during replication operations. Bytes are used as the standard unit to ensure precise measurement of all data transfers, regardless of the data type or format. Transfer
- Volume: The transfer volume is the total cumulative amount of data that has been replicated or applied in your workspace. This is the sum of all bytes transferred across all your replication operations.
- Time Period Breakdown: The consumption monitor breaks down your data
usage into the following time periods:
- Last hour
- Last 24 hours
- Last 30 days
- Monthly
- Entire billing period
Monitor data integration consumption
- To view your total data integration consumption on the DIS usage screen:
- From the main navigation menu, click Account→Usage.
- Locate the Data Integration section on the usage screen.
- View the Consumption Volume metric, which displays the total amount of bytes that have been replicated or applied in your workspace.
- To view consumption data broken down by time period:
- On the Data Integration section, click Details.
- Review the consumption breakdown for the following time periods:
- Last hour
- Last 24 hours
- Last 30 days
- Monthly
- Entire billing period
- Use this information to identify usage trends and peak consumption times.
- To view consumption data at the catalogue asset level (for continuous
replication projects only):
- From the main navigation menu, click Catalog.
- Search for or navigate to a Replication Project asset.
- View the asset details page, which displays two consumption-related
fields:
- Transfer Unit: The unit of measurement (bytes)
- Transfer Volume: The total volume of data replicated for this project
- Use these metrics to understand consumption at the project level.
- To view detailed consumption reports through Tableau:
- Access the Tableau reporting interface from your workspace.
- Navigate to the data integration consumption report.
- Use the available filters to analyze consumption by workspace, project, pipeline, connection, database, schema, table, alias, operation (insertion/update/delete), type (continuous/mainframe) and unit(bytes/records).
- Review the detailed breakdowns to identify consumption patterns and optimize your data integration strategy.
- Understand your current data usage patterns
- Identify which projects and pipelines consume the most data
- Plan resource allocation based on actual consumption
- Monitor trends over time to forecast future needs