CCHRI Reporting Principles

Approved by the CCHRI Executive Committee 6/22/09

CALIFORNIA COOPERATIVE HEALTHCARE REPORTING INITIATIVE GOALS

The mission of CCHRI is to collect and report comparable, reliable performance data.  Members of the collaborative work together to:

• Collect and report standardized, reliable health plan and provider performance data.
• Promote the use of accurate and comparable performance measures.
• Create efficiency in data collection leading to reduced burden and cost to all participants.
• Provide a source for expert advice to consumer reporting entities.

OVERARCHING REPORTING PRINCIPLES

These principles provide general philosophical and methodological direction regarding CCHRI performance measurement and reporting.  More detailed, specific and quantified principles, guided by the overarching principles, are presented in the sections covering each performance domain.

1. Representatives of the entities whose data were analyzed (plans, medical groups, or individual physicians) and other stakeholders should have opportunity for input into the content of reporting.  Stakeholders should also have sufficient time and convenient access to review and provide comments on or corrections to the data before final reporting.
2. The measures selected should be scientifically valid, statistically reliable, and relevant.
3. All measures and reporting methodologies should be publicly available and transparent.
4. Any calculation of performance scores should incorporate statistical significance testing or language that identifies when a difference in scores may not be statistically significant (i.e., may be due to chance alone).
5. Adequate data quality assurance checks of the performance information being reported should be in place.  As relevant, performance measures should undergo required audits.
6. The reports themselves should adhere to the following requirements:
   1. The purpose of the report should be to provide health care performance information to stakeholders.
   2. The report should be timely (i.e., the interval between data collection and report should be as short as possible without compromising the quality of the report).
   3. The report should list its authors and sponsors.
   4. All measures and reporting methodologies should be stated explicitly, especially if benchmarks or composite scores are used.
   5. Reporting of performance measures should follow nationally accepted guidelines, where available, unless there is a strong reason to deviate from these guidelines.
   6. The report should disclose the limitations of the data and any cautions in interpreting the analyses provided.
   7. Reports should, where possible, contain the results of trending analyses (i.e., assessment of statistically reliable changes over time on a comparable measure).
   8. Text should reflect fair and appropriate treatment of all health plans, physician organizations, and physicians.
   9. Language explaining missing data should fairly represent the reason a plan, physician organization or physician rate is not displayed.
   10. Elements of performance displays should have consistent meaning across all presentations (e.g., interpretation of three stars as "good").
7. Periodically, CCHRI will evaluate the effectiveness of its reports and make appropriate modifications.

DOMAIN-SPECIFIC PRINCIPLES

• Health Plan Healthcare Effectiveness Data and Information Set (HEDIS)
• Health Plan Consumer Assessment of Healthcare Providers and Systems (CAHPS)
• Physician Organization Patient Assessment Survey (PAS)
• Provider After-Hours Survey

Subsequent sections of this document provide specific reporting principles directly related to each of these domains.

HEALTH PLAN HEDIS REPORTING

EXTERNAL VALIDATION OF HEDIS DATA

1. All CCHRI health plans must undergo one of the following types of Compliance Audits:
   • A Full Compliance Audit that encompasses the entire HEDIS Reporting Set; or
   • A Partial Compliance Audit that assigns a measure designation to each of the CCHRI-required HEDIS measures.
2. In order for CCHRI to accept a given health plan's HEDIS rate on a given measure, the plan must meet the following criteria:
   • Obtain from its Compliance Auditor a designation of "Report"; and
   • Submit the results and the measure designation to the CCHRI contractor by the CCHRI-established deadline.

REPORTING OF DOMAINS AND RATES

All Product Types

1. CCHRI includes all non survey-based Effectiveness of Care measures in its reports.
   • CCHRI considers exceptions to this default position on a measure-specific basis.
   • CCHRI follows NCQA HEDIS reporting guidelines unless there is strong reason to deviate from these guidelines. Criteria for excluding measures from reporting includes (but is not limited to) the following:
     • Weakness in the strength of the clinical evidence
     • Measure specifications flaws
     • Data capture and/or coding issues that affect validity of rates
     • Conflict with nationally accepted clinical guidelines
   • All Reporting Committee recommendations must be reviewed and approved by the CCHRI Executive Committee.
   • Any decision not to report a specific measure publicly will be forwarded to NCQA with accompanying rationale for input prior to forwarding reporting recommendations to the Executive Committee.
   • CCHRI adopts NCQA's approach to "first-year" measures and does not publicly report performance on measures collected for the first time.
     • CCHRI reports first-year measures blinded internally.
     • Unblinded trending will not be conducted against first-year rates in either the public or internal reports.
2. In addition to the Effectiveness of Care domain, CCHRI may also report HEDIS measures from other domains or survey-based Effectiveness of Care measures.  CCHRI makes these decisions on a measure-specific basis (e.g., Prenatal and Postpartum Care from the Access/Availability domain).
3. If an auditor designates specific HEDIS measure(s) as "NO REPORT", that designation appears as the plan's performance in CCHRI Reports.
4. CCHRI reports adopt NCQA policies governing minimum volumes in denominators.  Footnotes accompany displays of NA, explaining that such a designation is not pejorative.

CHANGES TO PERFORMANCE RATES AFTER SUBMISSION OF DATA

If a CCHRI health plan produces a revised, auditor attested rate after the CCHRI deadline that CCHRI is able but NCQA is unable to accommodate, the Reporting Committee determines the CCHRI reporting procedure on a case-by-case basis, with final approval required from the Executive Committee.

ROTATION STRATEGY

1. Plans that elect to collect and submit rotation measures will have those results displayed in CCHRI reports.
2. CCHRI reports incorporate the most recent, auditor-attested rates submitted to NCQA.

REFERENCE POINT FOR DISPLAYS OF COMPARATIVE PERFORMANCE

For purposes of displaying comparative performance, CCHRI generally uses the national percentiles from Quality Compass and the CCHRI all-plan average as a reference point for non survey-based effectiveness of care measures.

HEALTH PLAN CAHPS REPORTING

EXTERNAL VALIDATION OF HEDIS DATA

1. All CCHRI health plans must undergo a Certified HEDIS Compliance Audit that covers the reporting of CAHPS data.
2. In order for CCHRI to accept a health plan's CAHPS results, the plan must obtain from its Compliance Auditor a designation of "Report" for all relevant CAHPS based-measures.

REPORTING DOMAINS AND RATES

1. CCHRI includes all composite and ratings measures as well as results from selected measures as agreed upon by the project and reporting committees.
2. CCHRI reports CAHPS Adult Survey results according to NCQA guidelines (e.g., NCQA-defined composites). If agreed upon by the Reporting Committee, CCHRI may also report the results of individual survey questions.
   • CCHRI requires its plans to collect the Commercial CAHPS Adult Survey but does not require its plans to collect the Management of Menopause or CAHPS Child Surveys.
3. As available, reporting formats should follow national standards, including labeling and any cautionary language.
4. If an auditor designates specific HEDIS measure(s) as "NO REPORT", that designation appears as the plan's performance on CCHRI Reports.
5. CCHRI reports adopt NCQA policies governing minimum volumes in denominators. Footnotes accompany displays of NA, explaining that such a designation is not pejorative.

CHANGES TO PERFORMANCE RATES AFTER SUBMISSION OF DATA

If a CCHRI health plan produces a revised, auditor attested rate after the CCHRI deadline that CCHRI is able but NCQA is unable to accommodate, the Reporting Committee determines the CCHRI reporting procedure on a case-by-case basis, with final approval required from the Executive Committee.

REFERENCE POINT FOR DISPLAYS OF COMPARATIVE PERFORMANCE

For purposes of displaying comparative performance, CCHRI generally uses the all-CCHRI Plan mean as a reference point for CAHPS measures until such time as the NCQA Quality Compass percentiles become available.

PHYSICIAN ORGANIZATION PAS REPORTING

The Patient Assessment Survey (PAS) Project Committee will submit PAS Reporting recommendations to the CCHRI Reporting Committee for review and final recommendation to the CCHRI Executive Committee for approval.

The PAS Reporting Principles would apply to medical group-level PAS reporting beginning with the 2010 PAS project (for 2009 measurement year).

PAS Reporting Principles include the following:

• The patient sample selection process and analytic methods reduce the risk of bias in the estimates and account for differences in the patient population across groups through ensuring: (1) adequate sample sizes to produce a stable estimate; and (2) case mix adjustment of results.
• Performance estimates provide a strong signal of true performance, as determined by: (1) achieving a minimum threshold of reliability (i.e., 0.70 or higher).

PROVIDER AFTER-HOURS SURVEY REPORTING

The objective of the Provider After-Hours survey is to measure patient access to appropriate emergency information and availability of healthcare professionals after-hours for urgent care issues.  The Provider After-Hours Survey Project Committee will submit reporting recommendations to the Reporting Committee for review.

• A target of 50 completed primary care physician surveys is attempted per physician organization.  If a phone number appeared more than once in a particular sample set, that phone number is called only once. If the call is answered by a live person, they will verify that response for each physician listed in the sample is the same and record any differences.
• Only completed interviews were used in the analysis of the data. If the response choice "don't know" or "refused" was selected for a particular question, those responses were not included in the analysis of that particular question.
• In the event that a group uses a call center after-hours is staffed by qualified healthcare professions and requires member identification to access a healthcare professional their score will be based on a description of their call protocol.
• Significance Testing at 95% to determine if a group's score is above or below the state average for all CCHRI participating groups.

Guidelines for Use of Data

CCHRI Senior Manager
Cathie Markow, PBGH
cmarkow@pbgh.org