SPARC Standards Process

Background: Now that the SPARC Data and Resource Center (DRC) is up and running, recommending standards and best practices will no longer fall under the original Standards Committee but will be under the remit of the curation team. As standards and best practices drive the type of integration that SPARC is required to achieve, standards are identified based on the data submitted and user requirements. Recommendations for standards will be submitted to the Data Standards Committee for approval and then implemented by the DRC. If there is a disagreement between the curation team and Data Standards Committee, the issues will be brought to NIH and the DRC Steering Committee for resolution.

SPARC Overall Standards Strategy

Standards in SPARC should achieve the following:

• Ensure that SPARC data is FAIR through adherence to the FAIR principles, which include rich metadata, FAIR vocabularies, and adherence to community standards.
  • Community standards in use outside of SPARC are preferred when possible. We are particularly monitoring what is coming out of the US BRAIN Initiative, but we should also be looking at other Common Fund projects given the desire to integrate across Common Fund Projects through the Common Fund Data Ecosystem (CFDE).
  • Standards will be developed as necessary for SPARC when none are available. Depending on the nature of the standard, we may create an ad hoc committee of SPARC investigators to work through a problem.
• Improve the user experience by ensuring that SPARC data is harmonized with respect to formats, metadata, etc., to the degree possible, and allows computational tools to be written against SPARC data by users.

Complying with standards can be hard, particularly without computational support, and frequent updating of standards can place a significant burden on both the data submitter and the infrastructure. Decisions about what standards to use and when will therefore be based on the maturity and support for the standard. The INCF has developed a set of criteria that can help evaluate standards under consideration (Abrams et al. 2019).

Standards Process

Overview: The need for data standards will be identified across the DRC cores. Recommendations for data standards will be prepared by the appropriate core and submitted to the Data Standards Committee. The Data Standards Committee can either endorse or reject the standard.

Preparing the Proposal: The submitting core will prepare a proposal that outlines the following:

• The need for the standard.
• Who is impacted by this standard, and what effort will be required to comply?
• Selection criteria: how and why was this chosen/developed?
• Testing protocol, acceptance criteria, and testing results: Does the standard perform as required?

Testing protocol: For each standard, the submitter should outline an appropriate testing protocol to ensure that the proposed standard performs as required. The testing protocol should include the acceptance criteria that the standard will have to satisfy, including usability. That is, the submitting core is required to perform an appropriate level of testing and provide the results to the Standards Committee. For example, a testing protocol for a recommended file format might be tested by: 1) Showing that the format can be read and written by investigators using popular tools; 2) Conversion into that format does not result in loss or distortion of information; 3) It is sufficiently well documented that investigators can use it within custom pipelines written in MatLab or Python. The acceptance criteria are then tested by asking two SPARC investigators who use this type of data to execute these actions. Results are then provided to the Data Standards Committee.

Submitting the proposal: In order to ensure a timely review, the Data Standards Committee should be notified at least two weeks ahead of the submission to give them time to schedule the review. The Committee prefers that the document be submitted as a final version Word document. If the submitters have questions for the committee, they can request preliminary comments on a draft document ahead of formal submission.

Endorsing a SPARC standard: A review of a standard should be completed within three weeks. If endorsed, the standard recommendation will be implemented by the appropriate core. All standard specifications should be accessible via the SPARC Portal. Each standard will be clearly versioned and given a unique identifier for referencing in SPARC systems. If rejected, the committee will provide feedback as to why the proposed standard was rejected. The submitter can then decide whether to revise and resubmit or withdraw the consideration. If the submitter disagrees with the committee’s decision, the issue can be brought to the NIH and DRC Steering Committee for adjudication.