Data Uploader

The Data Uploader application is developed as a KNIME Analytics Platform plug-in comprising of several workflow nodes. Its aim is to allow easy data mapping and import of non-IUCLID data into IUCLID in a controlled way, supporting all IUCLID data sections.

The Data Uploader supports the transformation of data on chemicals to the IUCLID format. It is especially useful for conversion of legacy toxicity data into harmonised templates which can be stored in a IUCLID database. This tool can be used by owners of chemical databases who would like to share their information in an internationally recognised format. The application is developed as a KNIME Analytics Platform plug-in, comprising of several workflow nodes.

The main Data Uploader workflow consists of 6 tailor-made KNIME nodes:

Data Uploader Workflow

 

  • Connection to a IUCLID database
  • Selection of relevant IUCLID documents and fields
  • Mapping with an external data source
  • Validation of the data against the IUCLID format
  • Generation of IUCLID files (.i6z)
  • Upload to IUCLID

 

 

The Data Uploader workflow with the Incremental update node supports a stepwise approach and other incremental update scenarios. With the stepwise approach the user is able to upload a set of simple entities, e.g. Reference Substance, Legal entity, Literature, that can be used later as references in SUBSTANCE or TEMPLATE documents. This reduces the complexity of the KNIME workflows to transform the data and improves the overall user experience.

 

Other incremental update scenarios include creating an entity (SUBSTANCE, TEMPLATE…), creating documents (ENDPOINT_STUDY_RECORD, FLEXIBLE_RECORD…), editing fields within an entity/document and deleting entities/documents.

You can find in the following table the fixes and improvements made in the releases of the Data Uploader:

Version Compatibility Improvements
1.4

KNIME 5.1 & 5.2
IUCLID 6.7

- Upgrade to KNIME version 5
- Enhanced data validation capabilities with IUCLID
- Expanded flexibility by adding new field delimiters, including $, ^, !, +, =
- Enabled automatic creation and inclusion of file directories and paths within nodes
1.2

KNIME 4.6.4 & 4.7
IUCLID 6.6 & 6.7

- Upgrade to KNIME 4.6.4 and 4.7
- Compatible with IUCLID 6.7
- Improved incremental update with possibility to edit and delete datasets/values in IUCLID
- Implemented KNIME node cancelation
- Updated use cases
1.1

KNIME 4.6
IUCLID 6.6

- Upgrade to KNIME 4.6
- Alignment with trusted community nodes
- Simplified installation
- Automatic selection of required fields (DocumentSelection node)
1.0

KNIME 4.2.2
IUCLID 6.6

- First released version

 

If you have questions about the Data Uploader, or suggestions on how to improve it, please fill in the contact form on the ECHA website here. Under Request type, select Data Uploader.

Data Uploader Knime plug-in (7th March 2024 - v1.4)

You can now download v.1.4 of the Data Uploader. The installation package contains the following:

  • The installation instructions
  • The Data Uploader KNIME plug-in
  • An introductory use case for creating reference substance documents in IUCLID, guiding the user through the Data Uploader workflow and data preparation process, and explaining the configuration of the Data Uploader nodes. The provided material includes the starting data set and two KNIME workflows, one for the data preparation and one for the data upload into IUCLID.
  • A more advanced use case for creating substance datasets that contain endpoint study records. The provided material includes the starting data set and two KNIME workflows that are described in the accompanying documentation.
Please sign in to download files