Questions for TSC Regarding the ONC Conformance Testing Project

1. The general approach that was discussed several months ago with the TSC chair, CTO and staff was that HL7 should not be in the business of creating conformance testing. Rather, we should create conformance statements as part of our IG that are then rolled up and distributed to external toolsmiths capable of creating conformance testing tools (probably at the same time that the standards are created). While my statement of the process is likely over simplified, does the TSC agree that this is the preferred approach?

• F.Hall - yes, and this has worked for the US Realm Lab IGs, conformance statements from IGs used by National Institute of Standards and Technology (NIST) to build validation tools. However, conformance statements must be machine computable.
• J.ROberts - Yes; at our best our IGs contain computable conformance statements. Where that is not true we need to work.

2. Assuming that the answer to question 1 is yes, we would likely need to come forward with a new requirement that all IGs provide these conformance statements. Where would such a statement go? Is there a document that currently defines what is required for IGs?

• F.Hall - Would include in multiple documents since we never know where a person will look and we have no definitive source of truth recognized by all. For example: GOM, Co-chair handbook, BAM (?), V2 Style Guide, V3, FHIR (?) TSC could also monitor (or require?) existing role of "Conformance Facilitator" for IG projecs, thus would prompt Work Group to plan for how to achieve conformance statements, even if they assign the WG to complete

3. If we make conformance statements a requirement, how should this be rolled out? Would the TSC/Board ask each WG to update the current IGs to produce the conformance statements? If so, would this be rolled out over the course of a couple of years? While we could make certain this happens going forward, I believe our plan would need to cover the IGs that have already been published. At least those that are used frequently.

• F.Hall - Suggest follow the "Introducing New Processes to HL7" methodology to roll out. Could create expectation that all IGs need CS created, unless exception granted by TSC. That allows WGs to petition for exception and help identify what is not "used frequently".

4. Would a particular WG or WGs in HL7 be responsible for developing a document that provides guidance and direction on how to create the conformance statements? Are there other considerations? For example, to do this correctly would each committee need a conformance facilitator in the same way that we have vocabulary and modeling facilitators?

• F.Hall - I think Conformance & Guidance for Implementation/Testing Work Group (CGIT) is best candidate to develop, their mission is to "... support all conformance activities of users of the HL7 standards." They would likely need input from other groups, could be done as a peer review or comment only ballot on their proposal. On the project scope statement we already have a slot for "Conformance facilitator (for IG projects)", defined as: The Conformance Facilitator, recommended for Implementation Guide projects, is a member of the integration team with in-depth knowledge of interoperability standards and conformance/constraint rules. This individual advises the project team on the use of HL7 standards and the constraint(s) requested to support the stakeholder requirements.

5. Do we need to develop a program to certify the certifiers? If so, which WG within HL7 do you see being responsible for this task?

• F.Hall - Would promote consistency in approach, seems CGIT should supervise this work.

6. ONC would like us to suggest a way that we can deliver annually a way to report on implementation/use of our IGs. I’ve had a very brief discussion with Pew Charitable Trust on this but would like to get input from the TSC on any suggestions they have for collecting this information, including whether the standards maturity model would provide what is needed.