Versioning and Backwards Compatibility in V3

From HL7 TSC
Revision as of 15:56, 23 October 2012 by WoodyBeeler (talk | contribs)
Jump to navigation Jump to search

Notes for V3 Task Force on Data Types and Backwards Compatibility

Version 3 Principles on Compatibility

Document balloted in HL7 in late 90's to express the intended principles for V3. The principles are published as section 5.2 of the Notes to Readers in each V3 Ballot.

Upward Compatibility Among V3

HL7 will provide the maximum degree possible of interoperability among systems operating on older and newer versions of HL7 protocols in the V3 family through compatible enhancement.

  • A message structure that is modified in a newer version of the protocol must be acceptable to a system designed for the prior V3 release. However, a system built for the prior release will only extract the information that was defined for that prior release.
  • A message structure created in accordance with an older version of the V3 protocol must be acceptable to a system designed for a later V3 release. In some cases, this will mean that the system built for the newer release will not receive certain information fields because they were not a part of the older version of the message structure in use by a specific sender.
  • Where compatible enhancement is not possible, HL7 will require evolution in the protocol to happen gradually, so that users can introduce the change into their networks gradually.
  • The messages associated with all interactions that are newly defined in a version of HL7 SHALL NOT be sent to a receiver that conforms to the older version.
  • A message structure MAY be declared obsolescent in one release, with a stated alternative message structure. Both the obsolescent message structure and its alternative can be used by all systems supporting that release.
    • WoodyBeeler 15:56, 23 October 2012 (UTC) The second sentence above begs a number of questions: (i) Did it really men "Both" can be used or should it be "either" (one or the other)? (ii) If "both" does this mean systems MUST send both or that they MAY send both? (iii) Should there be a requirement to provide a transform (in the case of XML) to convert "obsolescent" to "alternative"?
  • The obsolescent message structure MAY be declared obsolete and dropped when still another HL7 version is issued.
  • An obsolescent message structure SHALL NOT be declared obsolete for at least two years from the date of the version that first declared it obsolescent.

The above notwithstanding, if a new Implementation Technology Specification (ITS) is introduced, HL7 may specify that conformance to the ITS does not require dealing with message structures that are obsolescent when the new ITS is introduced. To the maximum degree possible, these restrictions should not impose limitations on the evolution of the overall reference model. There are no restrictions on making changes to the information model if those changes do not impact the message structures that were described in a prior version of the Standard.

Actual "Versioning" in V3

Subsequent to the V3 Principles approval, HL7 undertook the development of V3 specifications. The strategy that was followed was to seek "parallel" development and balloting of the V3 standards. Under this strategy, each of the involved Work Groups would develop and seek approval for its own "domain" content, and the combined effort to become the "V3 Messaging Specifications". Subsequent to the first approvals of V3 specifications, the Modeling and Methodology Work Group evolved the notion of publication packages known as editions. In essence, each ballot would constitute an "implementation edition", and an annual "Normative Edition" would be produced as the formal specification which implementers were expected to use.

Since 2005, there have been seven "Normative Editions" published by HL7. Each has been self consistent. The expressed intent from Modeling and Methodology has been that implementations should be made from content drawn solely from a single Normative Edition. Thus, implicitly, the transition from one Normative Edition to a subsequent Edition might be "non--compatible". In effect, the earlier Normative Edition contains the "obsolescent" messages, and the later Normative Edition is the "newly defined" set of messages.

To be clear, this was never expressed relative to the obsolescence principles cited above.

Implementation of "Deprecation" in the Reference Information Model

In order to support the concepts embodied in the V3 Principles, including the notion of "obsolescence", the Harmonization process, which manages the HL7 Reference Information Model and its associated vocabulary, adopted the notion of "Deprecation" to support the gradual adoption of "breaking" changes in the RIM.

Whenever a "breaking" change is proposed, such as the removal of an attribute, Harmonization adopted with the following steps and expectations:

  1. the element to be removed would be listed as "Deprecated" as of a particular date and release of the RIM
  2. the deprecation notations would contain a statement as to how to represent the "semantics" of the deprecated element in future models. (This might include identification of a new, replacement element, or of an alternate design structure that would express the same thing.)
  3. Effective with the date of deprecation, the selected element was not to be used in the development of any new models. (To be clear, the meaning of "new models" was never fully explored. For example if a model was developed prior to deprecation, and was subsequently (after deprecation) modified within the context of a second ballot, did this constitute a new model?)
  4. Subsequent to deprecation, the element was to remain part of the RIM for at least two Normative Edition. Subsequent to that time, the deprecated element could be removed (deleted) from the RIM.

Semantic Compatibility in Normative Editions

As each Normative Edition is published, the content within that edition is updated to the "current" RIM and Data Types, so long as the changes are semantically equivalent.

The latter distinction was only important in the conversion from data types release one to data types release two( for Normative Edition 2012). In that circumstance, the intention to make these changes was formally announced to the cochairs meeting had several Working Group Meetings, and formally expressed in published announcements to the cochairs. Indeed the actual conversion occurred 20 months after the first announcement, and almost 4 years after the beginning of the development of Data Types Release 2.

One consequence of the management of Normative Edition content has been that several Normative Editions include "deprecated" elements (both attributes and vocabulary terms) that, in effect, form and "obsolescent" message structure.

Implications of ANSI Version Expectations

Recently, HL7 has begun to stumble over the ANSI expectations for versioning. Specifically, ANSI expects each specification to be reviewed at the end of five years, at which time it should either be reaffirmed for five more years or updated. At the end of 10 years, the specification must be updated.

While several Work Groups have followed this pattern, a number of others have not. As a consequence, most of the "common" message content in V3 (with the exception of CMETs) has passed the time at which could be reaffirmed. This includes: messaging wrappers, query wrappers, registry wrappers, control act wrappers, and common messages.

One solution to this problem would be to re--ballot the original release one specifications, but with subsequently deprecated content replaced by its semantic equivalent. This material could become the "obsolescent" specification, while the Infrastructure and Messaging Work Group undertakes to develop the release two content.

Recasting "Draft motion" in Terms of V3 Compatibility

Draft Motion (Hugh):

  • HL7 has not and will not guarantee wire format backward compatibility of V3 payloads as the standard evolves. However HL7 recognises that in some circumstances breaking wire format backward compatibility will cause significant difficulty to users and that this in turn will reflect badly on HL7 and have a substantial negative impact on the adoption of V3. In such circumstances, and where a resolution can be found, HL7 will publish multiple versions of affected schemas so that both new versions and old wire format backward compatible versions are available to the user community that has been affected.
  • Discussion points:
    • The reason for trying to provide a backward compatible version is HL7 self-interest – we lose credibility if we don’t. We are not committing to satisfy all backward compatibility requirements everywhere.
    • We are only committing to provide a solution where one can be found.
    • We are not committing to providing the solution to everyone – but only to the group that has the issue.

Motion Re-vised

In keeping with the "V3 Principles", HL7 believes that standards must be able to evolve as new requirements and new solutions are presented. Wire-format backwards compatibility has served HL7 extremely well in the Version 2 and CDA environments and should remain a core objective for evolving HL7 standards.

Nevertheless, when "breaking changes" are proposed and approved through HL7's standards-setting processes, HL7 has an obligation to provide a reasonable "migration path" for those who have implemented the prior standards. Going forward, the approach will be based upon the declarations of obsolescent and obsolete standards, as discussed in the V3 principles.

Motion For Dealing with Data Types R2b

In the current circumstance, with Data Types Release 2 established as the Primary specification underlying V3, it is too late to undertake the complete obsolescent/obsolete process. Moreover, there is a significant, albeit circumscribed community. that seeks an implementation technology that conforms to Data Types R2, but that is more backwards-compatible to Data Types R1, than is the Normative ITS. The preferred backwards-compatible specification has the short-hand designation "Data Types R2B".

The Task Force recommends that the Data Types R2B go forward to Normative ballot in January 2013. This specification shall be formally recognized as a "fork" of the Data Types R2 ITS, and will be made available to a "defined community of applications" for the duration of the R2B's existence. The definition of the "defined community of applications" includes:

  • A
  • B
  • C