Upload or generate sample data - Precisely Data Integrity Suite

Data Integrity Suite
Product
Spatial_Analytics
Data_Integration
Data_Enrichment
Data_Governance
Precisely_Data_Integrity_Suite
geo_addressing_1
Data_Observability
Data_Quality
dis_core_foundation
Services
Spatial Analytics
Data Integration
Data Enrichment
Data Governance
Geo Addressing
Data Observability
Data Quality
Core Foundation
ft:title
Data Integrity Suite
ft:locale
en-US
PublicationType
pt_product_guide
copyrightfirst
2000
copyrightlast
2026

Sample data lets you test and configure pipelines without using production data. Upload sample data from a file to include specific characteristics such as duplicates or address errors, or generate sample data directly from your cataloged dataset. Sample data is encrypted and stored securely in the Precisely Cloud or Amazon S3.

Sample data is essential for testing and configuring Data Quality pipelines without risking production data. You can choose to include only the fields from your cataloged dataset that will be processed as input data by the pipeline. This simplifies assembling a pipeline for large datasets with many different fields.

The procedures to create and run Data Quality pipelines are the same regardless of storage location (Precisely Cloud or Amazon S3). Note that steps in a pipeline may create fields not included in the sample dataset. If a field created by a pipeline has the same name as a field already in the cataloged dataset, data in that field may overwrite data in the existing field.

Sample data storage: Sample datasets are encrypted by default and stored on the Precisely Cloud. Optionally, you can store sample datasets in an Amazon S3 cloud storage location. Sample data stored in an S3 bucket are managed by AWS, and subscribers can configure their own encryption and deletion policies.
Note: Sample datasets are automatically deleted 90 days after they are last used or browsed.

Characteristics of a sample file to upload

When uploading sample data, your file must meet these requirements:

  • Contains representative data in a text (.txt) or comma-delimited (.csv) format with fields that correspond to some or all of the fields in a mapped schema.
  • Can contain up to 10 MB of data.
  • Does not have to contain all of the fields from the cataloged dataset.
  • Sample columns may be in any order as long as the first row contains field names exactly as they appear in the cataloged dataset. Any column name in the file that does not match a column name in the cataloged dataset creates an error that prevents the upload.
  • Uploaded sample data must be in the same format as the data type in the dataset. For example, depending on the datasource, Date-time fields are often in ISO format ('1969-07-16T20:17:00').
  • Fields in a text file may be delimited by the comma (,), period (.), pipe (|), semicolon (;), space ( ), or tab character.
  • Text in a field that contains the delimiter may be qualified by single or double quotation characters.
  • Line breaks may use the Unix (or OS X) LF character, the Windows CR and LF characters, or the Macintosh CR character.
  • The first row must contain delimited field names that match fields specified by the mapped schema.
  • The delimited field names and data can be in any order.
Tip: If you are unsure of the encoding used by a file, you can open the file in Windows or Notepad and view the encoding in the Save as dialog. You can also use a code editor (such as Notepad++) to view or change encoding for a file.

Data format guidelines: These guidelines must be followed for data that you want to upload:

  • Must be in the same format as the data type in the dataset. For example, a date-time field is usually in ISO format, such as DateTime ('2011-12-03T10:15:30').

  • Sample data containing a geospatial column type must include the spatial features in GeoJSON format. In Snowflake, data can be exported to either of the following formats:

    • Setting the session before exporting the data: ALTER SESSION SET GEOGRAPHY_OUTPUT_FORMAT = 'GeoJSON' (for more information see, Snowflake documentation).
    • Using the ST_ASGEOJSON function in the SQL used to export the data: SELECT ID, Description, ST_AsGeoJSON(Geom) FROM Mytable.

  • Both Snowflake GEOMETRY and GEOGRAPHY columns are supported. The geometry in the export must be in the original coordinate system of the table. Although the GeoJSON specification requires the geometry to be in the WGS84 coordinate system (SRID = 4326). The Snowflake ST_ASGEOJSON function does not impose this requirement.

Upload sample data

To upload sample data that matches fields in a dataset from a file:
  1. Go to Catalog > Datasets.
  2. Click the cataloged dataset name for which you want to upload sample data.
  3. Click +Upload Sample.
  4. Choose the file that contains the sample data, then click Open.
The data is uploaded and stored in encrypted format in the Precisely Cloud, and displayed in a table. The table columns appear in the same order as the fields appear in the file. The storage location (such as Stored in the Precisely Cloud) shows in the margin above the table. The number of days until the sample data will be purged if it is left unused also shows above the table.
Note:
  • If previously uploaded sample data appears on the tab, you must first click the Delete button to clear the existing data.
  • If the upload fails, a dialog will describe problems with the upload. Common issues that prevent an upload include the following: sample and dataset fields do not match, file size exceeds the maximum allowed file size, or unsupported file type.
  • Sample datasets are by default stored in encrypted format on the Precisely Cloud. They are deleted 90 days after they are last used or browsed.

Generate sample data

When you generate sample data from a cataloged dataset, you can retrieve from 1 to 2000 records from the cataloged dataset. Currently, records can only be sampled sequentially, starting with the first record in the cataloged dataset. When you generate sample data, you can select the dataset fields that you want to include in the sample data. Names of fields in the generated sample data always match the names of the selected fields. The data is encrypted and stored in the Precisely Cloud.
  1. Go to Catalog > Datasets.
  2. Click the cataloged dataset name for which you want to create a sample dataset from source data.
  3. Click Generate Sample. This expands the Generate Sample panel on which you can see Fields in the dataset.
  4. Enter a value for the Number of rows to include in the sample.
    Note:
    • The sample will sequentially retrieve the number of records specified here, starting with the first record in the dataset. You can specify between 1 and 2000 rows. The default is 100. You may choose to increase this value to capture additional variability from the source data.
    • At the Sample Preview and Pipeline Grid section, you will see 500 records if the sample was generated with 500 or more sample count.
  5. In the Connections box, choose the connection to the host that you want to use. You may choose to use a connection that provides the authentication, cluster, or other settings that you need to connect to the host.
  6. Click Generate Sample.
This procedure starts a job that retrieves the specified number of rows of data from the source dataset. Upon successful completion of the job, the sample data appears in table format on the tab. The storage location (such as Stored in the Precisely Cloud) shows in the margin above the table. The number of days until the sample data will be purged if it is left unused also shows above the table. After you complete this procedure, you can click + Create Pipeline to use the sample data to create a pipeline.

Delete sample data

Remove sample data from a cataloged dataset.

  1. Go to Catalog > Datasets.
  2. Click the cataloged dataset name for which you want to delete sample data.
  3. Click Delete Sample.
  4. When prompted whether to delete the sample, click Delete.