HL7 Edition/Ballot Quality Review

Release: Edition2012

V3 Publishing Co-Chair George Beeler
Beeler Consulting LLC
V3 Publishing Co-Chair Andy Stechishin
Director of Technical Publications Don Lloyd
Health Level Seven, Inc.

Edited: 2011-05-31T09:53:01

Last Published: 20120831 11:51 AM

HL7® Version 3 Standard, © 2012 Health Level Seven® International.  All Rights Reserved.

HL7 and Health Level Seven are registered trademarks of Health Level Seven International Reg. U.S. Pat & TM Off.

Use of these materials is governed by HL7 International's IP Compliance Policy.

Return to Parent Domain

Table of Contents

1   Introduction
    1.1   Overview
    1.2   Reference Types Being Checked
    1.3   Indication of "Fatal" Errors
2   Vocabulary Reference Failure Tests
    2.1.  Vocabulary Testing Overview
        2.1.1   Reference Testing
        2.1.2   Vocabulary Reference Correction
    2.2   Vocabulary Reference Errors
        2.2.1   Introduction
        2.2.2   Results
    2.3   Static Models with Vocabulary Reference Errors
        2.3.1   Introduction
        2.3.2   Results
3   Dynamic Model Binding Reference Failures
    3.1.  Binding Reference Analysis
    3.2   Interaction Binding Reference Failures
        3.2.1.  Failed Binding of "Complete" Interactions
        3.2.2   Results
    3.3   Interaction Binding Reference Warnings
        3.3.1.  Warning on Interactions Without Payload
        3.3.2   Results
    3.4   Example Interactions with Incomplete Bindings
        3.4.1.  Incomplete Example Interactions
        3.4.2   Results
4   CMET Reference Failures
    4.1.  CMET Reference List
    4.2   Incomplete Specification of Listed CMETs
        4.2.1.  CMET Reference List
        4.2.2   Results
    4.3   Incomplete Listing of Referenced CMETs
        4.3.1.  CMET List Omissions
        4.3.2   Results
    4.4   Mis-matched CMET References from Static Models
        4.4.1.  Mis-matched CMET References
        4.4.2   Results
5   Static Model Design and Definition Failures
    5.1.  Static Model Design and Definition Issues
    5.2   Static Model Designs That Cannot be Serialized
        5.2.1.  Serialization Issues
        5.2.2   Results
    5.3   Static Model Designs With MIF-invalid Names
        5.3.1.  MIF Name Validation
        5.3.2   Results
    5.4   Missing or Mismatched Static Model Designs
        5.4.1.  Missing Static Models
        5.4.2   Results
    5.5   Missing Definition for Static Model
        5.5.1.  Missing Definition
        5.5.2   Results
    5.6   Duplicate Design for Static Model
        5.6.1.  Duplicate Design
        5.6.2   Results
6   Missing Files for Graphics and Other References
    6.1.  Other Reference Issues
    6.2   Missing Files in Content Submission
        6.2.1.  Missing graphic and "other" support files
        6.2.2   Results
    6.3   "Uncertain" Hyperlinks
        6.3.1.  "Uncertain" Hyperlinks In Specifications
        6.3.2   Results
7   Summary of Raw Reference Errors Data
    7.1.  Summary Table
    7.2   Results
Reference1 Introduction
 1.1 Overview

In early 2006, the V3 Generator became a more integral part of preparing each V3 Publication. This vehicle draws upon a comprehensive collection of files containing the domain artifacts being processed. As a consequence the Publishing Committee created an analysis of the cross-references embedded in these files, since the "failure" of any cross-reference represents incomplete linkages in the published content, and, therefore, a publication quality issue.

The resulting quality review files have been used to guide the creation of each ballot publication, identifying potential problems or places where the "wrong" source file has been provided. They have also guided the modification of generator processes to correct systematic errors in the content. These files have been provided to the publishing facilitators in the raw XML form. This document represents this content in a form that can guide implementers of the standards and the individual Work Groups responsible for creating the content in their efforts to adapt and improve the content

 1.2 Reference Types Being Checked

At present, the principle forms of reference that have been tested are:

  • Vocabulary References

    Almost half of the attributes used in HL7 static model (message) designs are coded attributes. Each of these carries a constraint specification that constitutes a reference to an element defined in the HL7 Vocabulary. In the HL7 "parallel development" environment these references are frequently inserted in model designs before their "target" has been added to the Vocabulary Content. This portion of the review looks for places where the necessary target is not in the Vocabulary. These arise for one of two reasons:

    1. The reference may be incorrectly entered.
    2. The necessary target has not been defined and approved (through Harmonization) in the Vocabulary.
  • Dynamic Model Bindings (Wrappers)

    Within the messaging dynamic model, HL7 static models are assembled into complete communicable models by "wrapping" models within models. Thus, a transmission wrapper surrounds a control act wrapper that surrounds a payload. These are bindings from "stubs" to their corresponding payload. In the HL7 "parallel development" environment these references are asserted by one Work Group while another Work Group is creating the target. Thus, it is necessary to check that the referenced target exists and that the "type" of the stub matches the type of the root element being bound to that stub.

  • CMET References

    HL7 static models include a significant amount of "re-use" in the form of Common Model Element Type (CMET) references. In the HL7 "parallel development" environment these references are asserted by one Work Group while another Work Group is creating the target. Thus, it is necessary to check that the referenced target exists.

  • Static Model Design-to-Definition References

    HL7 static models are designed in a graphic-editor environment (currently the RMIM Designer in Visio), but the relations of these static models to the rest of the "dynamic model" is documented separately in a "publication data base". This set of reference checks verifies that each static model design being distributed is "defined" in a dynamic model, and, conversely, that there is a static model design for each model defined in a publication data base.

  • Graphic and Other References to Subsidiary Files

    Each specification may, and usually does, have references to numerous subsidiary files that provide graphics or supplemental information for the specification. This set of reference checks verifies that such files were included in the submission of the specification for publication. Further it provides a set of "URL"s for hyper-links within the documents. These cannot be automatically validated, but should be reviewed for continued for current validity.

 1.3 Indication of "Fatal" Errors

In attempt to indicate to reviewers the severity of individual errors. Some of the results tables below have been set to indicate the presence of a "fatal" error by highlighting a portion of the error row in yellow. Items that are deemed "fatal" are those which fulfill one or more of the following criteria:

  • The error will cause an interaction that includes the involved element to be constructed incorrectly.
  • The error will cause the involved element to be invalid.
  • The error will cause designs that "include" the involved element (such as a CMET) to be invalid.

Note that in this context the term "invalid" means that one or more of the following will be true:

  • The schema generated for the element in question or any higher order elements that include it will fail top validate.
  • The MIF file created to represent the element will fail to validate.
  • The model element in question will be invalid according the the HL7 design methodology.
  • The model is duplicated in the package and may cause fatal tooling errors when it is processed.

As an example, selected rows of the vocabulary errors within static models table are flagged with yellow highlight. In these rows, the data type of the RIM attribute is "CS", and the error detected will cause the schema for the that static model to fail to validate because the vocabulary schema (which enumerates the possible values for attributes of type "CS") does not represent the erroneous value. If this static model is a CMET, the schema for any other static model that includes this CMET will also fail to validate, as will any interactions. However, in many of these cases, the MIF file will validate against the MIF schemas, as these do not mandate checking vocabulary enumerations

Reference2 Vocabulary Reference Failure Tests
 2.1.Vocabulary Testing Overview
 2.1.1 Reference Testing

The following sections list Vocabulary domain specifications for which a reference cannot be found in the current Vocabulary content. Almost half of the attributes used in HL7 static model (message) designs are coded attributes, and each of these requires a constraint specification that constitutes a reference to an element defined in the HL7 Vocabulary. As a consequence, an HL7 NormativeEdition will contain 50,000 to 100,000 (estimate) vocabulary constraint references. Many of these references are the same. Nevertheless, there are almost 10,000 (measured) unique reference expressions in these designs.

Sources of Error

In the HL7 "parallel development" environment these references are frequently inserted in model designs before their "target" has been added to the Vocabulary Content. This portion of the quality review looks for places where the necessary target is not in the Vocabulary. These arise for one of three reasons:

  1. The reference may be incorrectly entered.
  2. The necessary target has not been defined and approved (through Harmonization) in the Vocabulary.
  3. The necessary target has been approved through Harmonization, but has not yet been posted to the Vocabulary content.

Note: For Normative Edition 2009, there are about 100 reference errors (just over one per-cent of the unique instances). Of these, over 1/3 (35) are incorrectly entered, and can be "fixed" by editing the static model design. The remainder have either not been proposed (majority) or have been approved but not posted.

Error Analysis

The current analysis is based on a complete list of all vocabulary constraints cited in static model designs. These are extracted from the "Visio" output (xml) files or the "hmd" files exported from RoseTree. The MIF representation for all static model designs is drawn from one of these two sources. The design process specifies whether the constraint is asserted as:

  • A Code System constraint, usually with a specific code reference as well. (These are correct representations for constraints of structural attributes.)
  • A Value Set constraint.
  • A Concept Domain constraint.

In matching these constraints, the mapping process takes into consideration the fact that early constraints were all expressed as domain constraints, whereas more recent tooling and vocabulary directions have made it easier to constrain directly to value sets or codes. Thus, any reference to a "domain" for which there is currently no Concept Domain is also tested to see if there is a Value Set of the same name. If there is, this is treated as a correct reference.

 2.1.2 Vocabulary Reference Correction

In preparing a Normative Edition. an attempt is made to analyze each unique error and recommend a "resolution" for the error. These resolutions may be specific corrections, such as"Use D:ControlActReason" or a simple "Need Proposal" for cases where no appropriate target can be identified in the current vocabulary content. As noted above, over a third of the errors represented invalid expressions for which a correct target could be identified. In selected cases (a total of about 50 static model designs), these corrections were made as a "technical correction" to the Normative Edition content before publications. The requirements for making such a technical correction were:

  1. All of the vocabulary reference errors for that design must be correctable with simple editing of the reference expression
  2. The static model design must have been presented as "design xml" file from Visio, rather than having been processed through the design repository into an "hmd" file. (In this case, one can be sure that other design elements of the model were not affected by the correction.).
 2.2 Vocabulary Reference Errors

Amazingly, there are vocabulary errors detected for this package.

Return to Table of Contents

 2.3 Static Models with Vocabulary Reference Errors
 2.3.1 Introduction

The previous listing showed the existing unique vocabulary reference errors and, if available, possible solutions. The following table lists all of the defined static models that have erroneous vocabulary references, along with the specific errors and, if available, their resolution. There are no new errors introduced in this table, rather it represents formulation that will facilitate repairing the flawed static models. The columns in the following table are:

  1. The identifier of flawed static model. This entry appears only in the "header" row for the sub-table that holds the flawed references
  2. The Name of the Code System, Concept Domain or Value Set that was referenced. This name is prefaced with "C:", "D:", or "V:" to indicate which kind of reference was being made. If the name listed here is erroneous, it will be displayed in bold face type.
  3. The Code being referenced within a Code System (if any). If the code reference is erroneous, it will be displayed in bold face type. If the error is that the code value is empty, this column will show --blank--.
  4. RIM Class.attribute being constrained
  5. Guidance, if available, on how to resolve the error.
 2.3.2 Results
Return to Table of Contents


Reference3 Dynamic Model Binding Reference Failures
 3.1.Binding Reference Analysis

The following sections list message-to-message binding specifications from the dynamic models (interactions) in this package for which a target cannot be found in the current package content.

As defined in the interaction specifications of the dynamic model, HL7 static models are assembled into complete communicable models by "wrapping" models within models. Thus, a transmission wrapper surrounds a control act wrapper that surrounds a payload. These are bindings from "stubs" to their corresponding payload. In the HL7 "parallel development" environment these references are asserted by one Work Group while another Work Group is creating the target. Thus, it is necessary to check that the referenced target exists.

The incomplete bindings are presented in three categories, in declining order of severity.

  1. Complete interactions from a domain but where the binding fails owing to a mismatch between the "stub" type of the wrapper, and the root element type of the target
  2. Interactions that lack a pay-load, but that are probably "OK". Check documentation of interaction,
  3. Incomplete interactions that lack either or both of a Transmission wrapper or ControlAct wrapper and that were probably created to illustrate a story board
 3.2 Interaction Binding Reference Failures

There are no failed interactions in this package.

Return to Table of Contents

 3.3 Interaction Binding Reference Warnings

There are no interactions with warnings in this package.

Return to Table of Contents

 3.4 Example Interactions with Incomplete Bindings
 3.4.1.Incomplete Example Interactions

This table lists incomplete interactions that lack either or both of a Transmission wrapper or ControlAct wrapper. Almost all of these are from "infrastructure" domains and were created to illustrate a story board.

 3.4.2 Results


Listing of Example Interactions ( 33 rows)
Dm Interaction Transmission ControlAct Payload QueryDefinition Resolution
CI MCCI_IN000000UV01 MCCI_MT000100UV01 Neither ControlAct wrapper nor Payload specified
CI MCCI_IN000001UV01 MCCI_MT000100UV01 Neither ControlAct wrapper nor Payload specified
CI MCCI_IN000002UV01 MCCI_MT000200UV01 Neither ControlAct wrapper nor Payload specified
CI MCCI_IN000003UV01 MCCI_MT000100UV01 Neither ControlAct wrapper nor Payload specified
CI MCCI_IN000004UV01 MCCI_MT000300UV01 Neither ControlAct wrapper nor Payload specified
CI MCCI_IN000005UV01 MCCI_MT000100UV01 Neither ControlAct wrapper nor Payload specified
CI MCCI_IN000006UV01 MCCI_MT000200UV01 Neither ControlAct wrapper nor Payload specified
CI MCCI_IN000007UV01 MCCI_MT000300UV01 Neither ControlAct wrapper nor Payload specified
CI MCCI_IN000008UV01 MCCI_MT000200UV01 Neither ControlAct wrapper nor Payload specified
CI MCCI_IN000009UV01 MCCI_MT000100UV01 Neither ControlAct wrapper nor Payload specified
CI MCCI_IN000010UV01 MCCI_MT000300UV01 Neither ControlAct wrapper nor Payload specified
CI MCCI_IN000011UV01 MCCI_MT000300UV01 Neither ControlAct wrapper nor Payload specified
CI MCCI_IN000012UV01 MCCI_MT000100UV01 Neither ControlAct wrapper nor Payload specified
CI MCCI_IN000013UV01 MCCI_MT000200UV01 Neither ControlAct wrapper nor Payload specified
CI MCCI_IN000014UV01 MCCI_MT000300UV01 Neither ControlAct wrapper nor Payload specified
CI MCCI_IN000015UV01 MCCI_MT000200UV01 Neither ControlAct wrapper nor Payload specified
CI MCCI_IN000016UV01 MCCI_MT000300UV01 Neither ControlAct wrapper nor Payload specified
CI MCCI_IN000017UV01 MCCI_MT000200UV01 Neither ControlAct wrapper nor Payload specified
CI MCCI_IN100001UV01 MCCI_MT100100UV01 Neither ControlAct wrapper nor Payload specified
CI MCCI_IN100002UV01 MCCI_MT100300UV01 Neither ControlAct wrapper nor Payload specified
CI MCCI_IN100003UV01 MCCI_MT000200UV01 Neither ControlAct wrapper nor Payload specified
CI MCCI_IN100004UV01 MCCI_MT100200UV01 Neither ControlAct wrapper nor Payload specified
CI MCCI_IN100005UV01 MCCI_MT000200UV01 Neither ControlAct wrapper nor Payload specified
CI MCCI_IN200100UV01 MCCI_MT200100UV Neither ControlAct wrapper nor Payload specified
CI MCCI_IN200101UV01 MCCI_MT200101UV Neither ControlAct wrapper nor Payload specified
Return to Table of Contents


Reference4 CMET Reference Failures
 4.1.CMET Reference List

The following section lists CMET references for which a target cannot be found in the current ballot content. HL7 static models include a significant amount of "re-use" in the form of Common Model Element Type (CMET) references. In the HL7 "parallel development" environment these references are asserted by one Work Group while another Work Group is creating the target. Thus, it is necessary to check that the referenced target exists.

In the case of CMETs, there are three different sources that should agree. Specifically:

  1. Every CMET should be listed in the "CMETinfo" file that defines the package of reusable artifacts from which designers can draw.
  2. Each CMET must have a valid static model design, in order that its schema may be defined and linked to the schemas that refers to it.
  3. Each CMET must be defined in the context of a publication database.

The CMET reference checking looks at these three sources seeking to determine that each CMET is listed in all three places, and, further, that each design that references a CMET points to one of these documented designs. The listings below show instances where a CMET is missing from one or more of these places. (NOTE: There is no test that confirms whether or not a CMET is actually referenced or used within another static model.)

 4.2 Incomplete Specification of Listed CMETs

There are no specification errors for listed CMETs in this package.

Return to Table of Contents

 4.3 Incomplete Listing of Referenced CMETs

There are no unlisted CMETs in this package.

Return to Table of Contents

 4.4 Mis-matched CMET References from Static Models

There are CMET reference warnings in this package.

Return to Table of Contents


Reference5 Static Model Design and Definition Failures
 5.1.Static Model Design and Definition Issues

The following section lists circumstances in which the package content contains a "non-serializable" static model or shows a mismatch between a static model design and a corresponding reference definition (from a Publication Database). There are variants of these errors include:

  1. A static model design cannot be uniquely serialized because it contains one or more associations that are not blocked in either direction.
  2. A static model design contains names (class, association or attribute) whose pattern or length are not valid and will cause the MIF representation of the model to fail schema validation.
  3. A static model design exists (as part of a "Visio xml" or "hmd" file), but there is no equivalent definition in from a publication data base
  4. A definition is provided in a publication data base for which there is no corresponding static model design as part of a "Visio xml" or "hmd" file
  5. There is both a static model design and a definition whose base identifiers are the same (such as "REAL_MT987123UV") but where the version number (final two digits) do not match.
  6. A static model design is duplicated in the submitted package and thus may cause fatal errors in the tooling when it is processed.
 5.2 Static Model Designs That Cannot be Serialized

There are no non-serializable static models in this package.

Return to Table of Contents

 5.3 Static Model Designs With MIF-invalid Names

There are MIF-name-invalid static models in this package.

Return to Table of Contents

 5.4 Missing or Mismatched Static Model Designs

There are no missing static model designs in this package.

Return to Table of Contents

 5.5 Missing Definition for Static Model

There are no undefined static models in this package.

Return to Table of Contents

 5.6 Duplicate Design for Static Model

There are no duplicated static model designs in this package.

Return to Table of Contents


Reference6 Missing Files for Graphics and Other References
 6.1.Other Reference Issues

The following section lists files that are referenced in the specification, but which cannot be found in the appropriate directories submitted for publication; and references to external "URL"s that must be checked manually for current validity. Specificall::

  1. Missing graphic and "other" support files.
  2. External hyperlinks that must be checked and maintained manually. These are deprecated in the "best practices" recomended by the Publishing Work Group, because the continued validity of such links cannot be assured.
 6.2 Missing Files in Content Submission
 6.2.1.Missing graphic and "other" support files

This report lists those graphic and other files that could not be found to satisfy a reference in the specification. These errors will lead to either an unsatisfied hyper-link, or the insertion of a missing graphic icon in the document.

 6.2.2 Results


Missing Files To Match References ( 6 rows)
SpecificationId Reference Type Unsatisfied Reference Directory Searched
uvci models MCCI_RM000100UV*.hmd sourcegraphics/
'' models MCCI_RM000200UV*.hmd sourcegraphics/
'' models MCCI_RM000300UV*.hmd sourcegraphics/
'' models MCCI_RM100100UV*.hmd sourcegraphics/
'' models MCCI_RM100200UV*.hmd sourcegraphics/
'' models MCCI_RM100300UV*.hmd sourcegraphics/
Return to Table of Contents

 6.3 "Uncertain" Hyperlinks
 6.3.1."Uncertain" Hyperlinks In Specifications

This report lists any hyper-links included in the specification that could not readily be validated as being targeted in the current published ballot.

 6.3.2 Results


Unvalidated URL References ( 3 rows)
SpecificationId Reference Type Unvalidated Reference Element Holding Ref
uvci spec IMMCCI xspecref
'' spec transport-mllp xspecref
'' url http://www.hl7.org/dstucomments/index.cfm loc
Return to Table of Contents


Reference7 Summary of Raw Reference Errors Data
 7.1.Summary Table

The following is a summary report for all domains in this publication as it appeared in the console output stream of the V3 Generator.

 7.2 Results

Error Listing from V3 Generator Console Listing for Edition2012:

------------------------------------------------------------------------
Pre-Generation Report of Reference Errors in Generated Package (Summary)
========================================================================
   14 errors involving references to known CMETs
       0 CMETS in cmetInfo.txt, a PubDb and in an hmd/mif file but where the UVnn codes do not agree
       0 CMETS in cmetInfo.txt but not in either PubDb or an hmd/mif file.
       0 CMETS in cmetInfo.txt and in a PubDb but has no hmd/mif file.
       2 CMETS in cmetInfo.txt and in an hmd/mif file but not defined in a PubDb
      12 CMETs referenced in an hmd/mif file but not listed in cmetInfo.txt
    4 warnings for likely CMET reference errors in static models
    3 errors for messages defined in PubDb but not in hmd/mif files
    3 errors for messages in hmd/mif files that are not defined in a PubDb
    8 errors for messages referenced in a PubDB or hmd/mif file that are not in an hmd/mif file
       0 messages referenced in a PubDB that are not in an hmd/mif file
       8 CMETs referenced in an hmd/mif file that are not in another hmd/mif file.

   4 Static models with name elements that are INVALID in MIF.

   100 errors in vocabulary constraints in a design that are not in vocabulary source (13 fatal)
       1 errors for Code Systems used in a design that are not in vocabulary source (0 fatal)
      12 errors for individual Codes used in a design that are not in vocabulary source (4 fatal)
      86 errors for Concept Domains used in a design that are not in vocabulary source (9 fatal)
       1 errors for Value Sets used in a design that are not in vocabulary source (0 fatal)

   43 interactions fail to bind because of stub-target mis-matches
      39 incomplete interaction specifications that cause binding failures
       2 stub-target binding mis-matches that cause interaction binding failures

(for details see Reports/MessageReferenceTestResults xml)
------------------------------------------------------------------------

Return to Table of Contents