Real World Testing Results 2022
Covered by this topic
Executed Real World Testing Results 2022 PDF Version
General Information
Plan Report ID Number | |
Developer Name | Medical Informatics Engineering |
Product Name(s) | WebChart EHR |
Version Number(s) | 8.4 |
Certified Health IT Product List ID(s) | 15.04.04.1932.WebC.84.01.0.221117 |
Developer Real World Testing Page URL | https://docs.webchartnow.com/resources/system-specifications/ehr-certification/real-world-testing/ |
Results Submission Date | 02/01/2023 |
Changes to the Original Plan
Summary of Change | Reason | Impact |
QRDA SVAP was not completed during 2022 | Business needs prevented development from finalizing code to meet the new standards. | There was no impact on RWT activities as testing measures did not need to be updated to the newer standard. |
Certification of b.10 and additional CQMs was not completed during 2022 | Business needs prevented development of support for the b.10 criteria and additional CQMs. | There was no impact on RWT activities as these criteria were not scheduled for testing until at least 2023. |
Other changes |
Withdrawn Products
Product Names(s) | WebChart EHR |
Version Number(s) | 7.4 |
CHPL Product Number(s) | 15.04.04.1932.WebC.74.00.0.181219 |
Date(s) Withdrawn | 12/31/2022 |
Inclusion of Data in Results Report | WebChart EHR 7.4 was the primary product version available to customers in 2022 and all data collected during the first three quarters was collected exclusively in v7.4. Customers began transitioning to WebChart EHR 8.4 during the fourth quarter. Data collected in Q4 represents a combination of data from v7.4 and v8.4. Customers continue to use the same technology (database, log files, etc) after transition to v8.4, so there is no interruption to RWT data collection activities. |
Certification Criteria Tested
- ‘‘Transitions of care" (§ 170.315(b)(1) Cures Update);
- “Clinical information reconciliation and incorporation” (§ 170.315(b)(2));
- “Electronic prescribing” (§ 170.315(b)(3) Cures Update);
- “Data export” (§ 170.315(b)(6));
- “CQMs – record and export” (§ 170.315(c)(1));
- “CQMs – import and calculate” (§ 170.315(c)(2));
- “CQMs – report” (§ 170.315(c)(3));
- “View, download, and transmit to 3rd party” (§ 170.315(e)(1));
- “Transmission to immunization registries” (§ 170.315(f)(1));
- “Application access – patient selection” (§ 170.315(g)(7));
- “Application access – data category request” (§ 170.315(g)(8));
- “Application access—all data request” (§ 170.315(g)(9) Cures Update);
- “Direct Project” (§ 170.315(h)(1))
Criteria-Measure Matrix
Criteria | Requirement | Measure |
"Transitions of care" (§170.315(b)(1)) | (b)(1)(i)(A)(Alternative) - Send Using Edge Protocol for SMTP/IXE XDR | 17 |
(b)(1)(i)(B)(Alternative) - Receive Using Edge Protocol for SMTP/IXE XDR | 17 | |
(b)(1)(i)(C)(Conditional) - XDM Processing | 17 | |
(b)(1)(ii)(A) - Receive, Parse, and Process | 7, 19 | |
(b)(1)(ii)(B) - View | 7 | |
(b)(1)(ii)(C) - Section Display | 7 | |
(b)(1)(iii) - Create | 7 | |
(b)(1)(iii)(A) - Assessment, Plan, Goals, Health Concerns | 7 | |
(b)(1)(iii)(B) - Diagnoses | 7 | |
(b)(1)(iii)(C) - Cognitive Status | 7 | |
(b)(1)(iii)(D) - Functional Status | 7 | |
(b)(1)(iii)(E) - Ambulatory Referral Summary | 7 | |
(b)(1)(iii)(F) - Inpatient Discharge Instructions | 7 | |
(b)(1)(iii)(G) - Patient Matching | 7 | |
"Clinical information reconciliation and incorporation" (§170.315(b)(2)) | (b)(2)(ii) - Correct Patient | 7 |
(b)(2)(iii)(A) - Simultaneous Display | 9 | |
(b)(2)(iii)(B) - Reconciled List | 9 | |
(b)(2)(iii)(C) - User Review | 9 | |
(b)(2)(iii)(D) - List Acceptance | 9 | |
(b)(2)(iv) - CCD Creation | 9 | |
"Electronic prescribing" (§170.315(b)(3)) | (b)(3)(ii)(A)(1) - NewRx | 3 |
(b)(3)(ii)(A)(2) - RxChangeRequest, RxChangeResponse | 3 | |
(b)(3)(ii)(A)(3) - CancelRx, CancelRxResponse | 3 | |
(b)(3)(ii)(A)(4) - RxRenewalRequest, RxRenewalResponse | 3 | |
(b)(3)(ii)(A)(5) - RxFill | 3 | |
(b)(3)(ii)(A)(6) - RxHistoryRequest, RxHistoryResponse | 3 | |
(b)(3)(ii)(A)(7) - Status | 3 | |
(b)(3)(ii)(A)(8) - Error | 3 | |
(b)(3)(ii)(A)(9) - Verify | 3 | |
(b)(3)(ii)(C)(1) - Primary/Secondary Diagnosis | 4 | |
(b)(3)(ii)(E) - Metric Units | 5 | |
(b)(3)(ii)(F) - Decimal Format | 6 | |
"Data export" (§170.315(b)(6)) | (b)(6)(i) - Configure and export | 18 |
(b)(6)(ii) - Set Export | 18, 19 | |
(b)(6)(ii)(A) - CCDS | 18, 19 | |
(b)(6)(ii)(B) - Diagnoses | 18, 19 | |
(b)(6)(ii)(C) - Cognitive Status | 18, 19 | |
(b)(6)(ii)(D) - Functional Status | 18, 19 | |
(b)(6)(ii)(E) - Ambulatory Reason for Referral | 18, 19 | |
(b)(6)(ii)(F) - Inpatient Discharge Instructions | 18, 19 | |
(b)(6)(iii)(A) - Timeframe configuration | 18 | |
(b)(6)(iii)(B) - Export summary | 18 | |
(b)(6)(iv) - Save location | 18 | |
"CQMs – record and export" (§170.315(c)(1)) | (c )(1)(i) - Report | 1 |
(c )(1)(ii) - Export | 1 | |
"CQMs – import and calculate" (§170.315(c)(2)) | (c )(2)(i) - Import | 2 |
(c )(2)(ii) - Calculate | 1, 2 | |
"CQMs – report" (§170.315(c)(3)) | (c )(3)(i) - Report | 1, 2 |
"View, download, and transmit to 3rd party" (§170.315(e)(1)) | (e)(1)(i) - Web Content Accessibility | 21 |
(e)(1)(i)(A) - View | 14 | |
(e)(1)(i)(A)(1) - CCDS | 14 | |
(e)(1)(i)(A)(2) - Ambulatory Contact Info | 14 | |
(e)(1)(i)(A)(3) - Inpatient Discharge Instructions | 14 | |
(e)(1)(i)(A)(4) - Lab Report | 14 | |
(e)(1)(i)(A)(5) - Diagnostic Imaging Report | 14 | |
(e)(1)(i)(B)(1)(i) - Download Human Readable | 15 | |
(e)(1)(i)(B)(1)(ii) - Download CCD | 15 | |
(e)(1)(i)(B)(2) - CCD Human Readable | 15 | |
(e)(1)(i)(B)(3) - Inpatient Summary | 15 | |
(e)(1)(i)(C)(1)(i) - Email | 16 | |
(e)(1)(i)(C)(1)(ii) - Encrypted Transmission | 16 | |
(e)(1)(i)(C)(2) - Inpatient Email | 16 | |
(e)(1)(i)(D)(1) - Specific Date | 14, 15, 16 | |
(e)(1)(i)(D)(2) - Date Range | 14, 15, 16 | |
(e)(1)(ii)(A) - Activity Log | 14, 15, 16 | |
"Transmission to immunization registries" (§170.315(f)(1)) | (f)(1)(i) - Create Content | 10 |
(f)(1)(ii) - Query Records | 11 | |
"Application access – patient selection" (§170.315(g)(7)) | (g)(7)(i) - Query processing and response | 20 |
(g)(7)(ii)(A)(1) - Functional Documentation | 8 | |
(g)(7)(ii)(A)(2) - Implementation Requirements | 8 | |
(g)(7)(ii)(A)(3) - Terms of Use | 8 | |
(g)(7)(ii)(B) - Public Link | 8 | |
"Application access – data category request" (§170.315(g)(8)) | (g)(8)(i)(A) - Return CCDS data | 20 |
(g)(8)(i)(B) - Request response | 20 | |
(g)(8)(ii)(A)(1) - Documentation | 8 | |
(g)(8)(ii)(A)(2) - Implementation Requirements | 8 | |
(g)(8)(ii)(A)(3) - Terms of Use | 8 | |
(g)(8)(ii)(B) - Public URL | 8 | |
"Application access—all data request" (§170.315(g)(9)) | (g)(9)(i)(A)(1) - Demonstrate API | 20 |
(g)(9)(i)(A)(3) - Data Classes | 20 | |
(g)(9)(i)(B) - Data Return | 20 | |
(g)(9)(ii)(A)(i) - Documentation | 8 | |
(g)(9)(ii)(A)(ii) - Implementation Requirements | 8 | |
(g)(9)(ii)(B) - Public URL | 8 | |
"Direct Project" (§170.315(h)(1)) | (h)(1)(i) - Send | 12 |
(h)(1)(i) - Receive | 13 | |
(h)(1)(ii) - Message Disposition Notification: Processed | 12 | |
(h)(1)(ii) - Message Disposition Notification: Failed | 12 |
Summary of Testing Methods and Key Findings
WebChart EHR is a cloud-based, fully-inclusive EHR solution. All certified functionality is delivered in all instances of the product regardless of the care setting, size of practice, or required use cases for a given practice. Each production client is maintained in a separate database; however, the implementation of the environment is identical with the exception of optional increased security protocols that a client may choose to add for enhanced data protection. Additionally, the only differences between the client-facing portion of each system are a result of configuration settings that can be selected at go-live or updated at any time during a client’s contract. Due to this philosophy of product delivery, all certified capabilities may not be actively used in all marketed care settings or may not be actively used in any current client production system. To address the Real World Testing requirements, MIE will be using a hybrid approach. Testing will primarily be conducted using de-identified real patient data from production systems as recorded in database tables and log files. For those criteria for which this live production recording is not available or minimal due to lack of client usage, client reported issues will be tracked and reported in addition to enacting automated tests of the certified functionality in a test system in a production environment. The automated tests will be run daily or weekly as appropriate in a system that is identical in substance and delivery to a client production system with the only exception being live real patient data. This blended approach will allow MIE to prove ongoing maintenance of WebChart EHR’s certified technology regardless of the level of implementation by current clients.
Measures Used in Overall Approach
The following measures first summarize the testing methods and justification used for testing each criteria requirement during the 2022 testing period. The real world testing results collected throughout 2022 and a discussion of those results then follow. Please review the Criteria-Measure Matrix above to review which measure(s) will cover a specific requirement.
Measure 1: Clinical Quality Measures Outgoing
Description
This measure will review WebChart EHR’s ability to measure clinical quality and export the required information. Compliance will be tested both manually by developers and clients as well as automatically by reporting bodies and the Cypress CUV+ test system.
Associated Certification Criteria
Certification Criteria | Requirement(s) |
"CQMs – record and export" (§170.315(c)(1)) | (c )(1)(i) - Record |
(c )(1)(ii) - Export | |
"CQMs – import and calculate" (§170.315(c)(2)) | (c )(2)(ii) - Calculate |
"CQMs – report" (§170.315(c)(3)) | (c )(3)(i) - Report |
Test Methodology
First, MIE will install an instance of Cypress 6+ on our production servers following all of our protocols for maintaining the security of PHI. Cypress CUV+ supports the validation of QRDA reports containing PHI and will be used monthly to validate a random selection of QRDAs from the care settings identified. Any errors identified by Cypress CUV+ will be tracked, reported, and addressed, then followed with testing of a larger sample of files.
Additionally, WebChart EHR has two customers that participate in quarterly attestations using both QRDA I and QRDA III reports. These customers regularly inspect their CQM compliance numbers and will alert MIE to any perceived errors. MIE will then collect and track the attestation results from the reporting bodies including any errors so as to report a success/failure rate.
Results
Calculations | QRDA I | QRDA III | |
Client Reported Issues | 0 | 0 | 1 warning, 1 error in a non-certified measure |
Submitted Files | N/A | 10567 | 5 |
Submission Errors | N/A | 0 | 0 |
Tested Files | N/A | 2748 | 48 |
Testing Errors | N/A | 0 | 0 |
Discussion
As expected, no errors were found in formatting or coding of the certified measures. One warning was reported by a client where a provider NPI was included in the QRDA III file, but the provider had been removed from the client reported roster. The provider producing the warning departed the practice mid-year causing data to be tracked in WebChart EHR for encounters that the provider performed prior to their departure. This warning was determined to be of no impact to the eCQM certified functionality. Additionally, one QRDA III error was reported by a client where the performance rate was calculated without removing the exclusions from the denominator prior to calculation; however, since the error occurred on a non-certified measure, this error was determined to be of no impact to the certified eCQM functionality.
Measure 2: Clinical Quality Measures Incoming
Description
This measure will review WebChart EHR’s ability to measure clinical quality and export the required information. Compliance will be tested both manually by developers and clients as well as automatically by reporting bodies and the Cypress CUV+ test system.
Associated Certification Criteria
Certification Criteria | Requirement(s) |
"CQMs – import and calculate" (§170.315(c)(2)) | (c )(2)(i) - Import |
(c )(2)(ii) - Calculate | |
"CQMs – report" (§170.315(c)(3)) | (c )(3)(i) - Report |
Test Methodology
MIE will install an instance of Cypress 6+ on our production servers following all of our protocols for maintaining the security of PHI. Automated testing will download QRDA I files from Cypress for each certified CQM, import the files to WebChart EHR, calculate the CQMs, and export the QRDA files for Cypress validation of both the content and calculations to verify that the import was successful. Any errors identified by Cypress will be tracked, reported, and addressed.
Results
QRDA I | QRDA III | |
Tested Files | 2808 | 48 |
Testing Errors | 0 | 0 |
Discussion
As expected, no errors were found in formatting or coding of the certified measures.
Measure 3: E-Prescribing Messages Sent and Received
Description
This measure will verify that all supported e-prescribing message types are in use in WebChart EHR, including inbound and outbound message types.
Associated Certification Criteria
Certification Criteria | Requirement(s) |
"Electronic prescribing" (§170.315(b)(3)) | (b)(3)(ii)(A)(1) - NewRx |
(b)(3)(ii)(A)(2) - RxChangeRequest, RxChangeResponse | |
(b)(3)(ii)(A)(3) - CancelRx, CancelRxResponse | |
(b)(3)(ii)(A)(4) - RxRenewalRequest, RxRenewalResponse | |
(b)(3)(ii)(A)(5) - RxFill | |
(b)(3)(ii)(A)(6) - RxHistoryRequest, RxHistoryResponse | |
(b)(3)(ii)(A)(7) - Status | |
(b)(3)(ii)(A)(8) - Error | |
(b)(3)(ii)(A)(9) - Verify |
Test Methodology
MIE will report a count of messages for each supported message type:
- NewRx
- RxChangeRequest
- RxChangeResponse
- CancelRx
- CancelRxResponse
- RxRenewalRequest
- RxRenewalResponse
- RxFill
- RxHistoryRequest
- RxHistoryResponse
- Status
- Error
- Verify
The report will also include a count of outbound messages unable to be transmitted due to connectivity issues or other errors, for each message type. This report will be based on the contents of each client’s local database table of stored messages. MIE will run the report for each client under consideration and aggregate the results.
Results
Message Type | Client Message Counts | Total Count | ||||
NewRx | 48926 | 223627 | 120724 | 34765 | 9517 | 437559 |
RxChangeRequest | 22 | 407 | 2665 | 880 | 0 | 3974 |
RxChangeResponse | 4 | 412 | 2420 | 882 | 0 | 3718 |
CancelRx | 195 | 372 | 3 | 7 | 6 | 583 |
CancelRxReponse | 164 | 307 | 2 | 6 | 6 | 485 |
RxRenewalRequest | 12260 | 11711 | 58090 | 6261 | 0 | 88322 |
RxRenewalResponse | 11943 | 11756 | 55118 | 6261 | 0 | 85078 |
RxFill | 0 | 0 | 0 | 1304 | 0 | 1304 |
RxHistoryRequest | 11 | 0 | 9 | 2 | 1 | 23 |
RxHistoryResponse | 8 | 0 | 4 | 1 | 0 | 13 |
Status | 123697 | 432586 | 385387 | 84808 | 17723 | 1044201 |
Error | 563 | 1118 | 3206 | 217 | 25 | 5129 |
Verify | 50418 | 184894 | 145576 | 34552 | 8233 | 423673 |
Error Text | Total Count |
(Error Code: 600) | 3 |
(Error Code: 601) (Desc. Code: 008) | 1334 |
(Error Code: 602) | 3 |
(Error Code: 900) | 94 |
1 errors validating against transport.xsd | 362 |
1 errors validating against transport.xsd 'Address' has invalid child element 'City'. List of possible elements expected: 'AddressLine1'. (Error Code: 900) | 9 |
1 errors validating against transport.xsd 'Address' has invalid child element 'PostalCode'. List of possible elements expected: 'StateProvince'. (Error Code: 900) | 2 |
1 errors validating against transport.xsd 'Address' has invalid child element 'StateProvince'. List of possible elements expected: 'AddressLine1'. (Error Code: 900) | 12 |
1 errors validating against transport.xsd 'Address' has invalid child element 'StateProvince'. List of possible elements expected: 'AddressLine2, City'. (Error Code: 900) | 3 |
1 errors validating against transport.xsd 'MedicationPrescribed' has incomplete content. List of possible elements expected: 'DrugUseEvaluation, DrugCoverageStatusCode, PriorAuthorizationStatus, Sig'. (Error Code: 900) | 1 |
1 errors validating against transport.xsd 'MedicationPrescribed' has invalid child element 'DaysSupply'. List of possible elements expected: 'Quantity'. (Error Code: 900) | 5 |
1 errors validating against transport.xsd 'MedicationPrescribed' has invalid child element 'WrittenDate'. List of possible elements expected: 'Quantity'. (Error Code: 900) | 29 |
1 errors validating against transport.xsd 'Sig' has incomplete content. List of possible elements expected: 'SigText'. (Error Code: 900) | 1 |
1 errors validating against transport.xsd The 'AddressLine1' element is invalid - The value '5633 SOUTH WAYNE AVE APT D' is invalid according to its datatype 'an1..40' - The Pattern constraint failed. (Error Code: 900) | 1 |
1 errors validating against transport.xsd The 'AddressLine1' element is invalid - The value '5633 SOUTH WAYNE AVE APT. D' is invalid according to its datatype 'an1..40' - The Pattern constraint failed. (Error Code: 900) | 2 |
1 errors validating against transport.xsd The 'DaysSupply' element is invalid - The value '2970' is invalid according to its datatype 'n1..3' - The actual length is greater than the MaxLength value. (Error Code: 900) | 2 |
1 errors validating against transport.xsd The 'DrugDescription' element is invalid - The value '' is invalid according to its datatype 'an1..105' - The Pattern constraint failed. (Error Code: 900) | 2 |
1 errors validating against transport.xsd The 'Extension' element is invalid - The value '2606735445' is invalid according to its datatype 'n1..8' - The actual length is greater than the MaxLength value. (Error Code: 900) | 1 |
1 errors validating against transport.xsd The 'Note' element is invalid - The value ' 3 3 @' is invalid according to its datatype 'an1..210' - The Pattern constraint failed. (Error Code: 900) | 1 |
1 errors validating against transport.xsd The 'Note' element is invalid - The value ' Information regarding your request; Drug is covered by current benefit plan. No further PA activity needed' is invalid according to its datatype 'an1..210' - The Pattern constraint failed. (Error Code: 900) | 1 |
1 errors validating against transport.xsd The 'Note' element is invalid - The value 'G ; 2 ; W ; ; { ; ; ; ; $ ;' is invalid according to its datatype 'an1..210' - The Pattern constraint failed. (Error Code: 900) | 1 |
1 errors validating against transport.xsd The 'Note' element is invalid - The value 'Per UpToDate Guidance: Infants 9 months: Oral: 3.5 mg/kg/dose twice daily' is invalid according to its datatype 'an1..210' - The Pattern constraint failed. (Error Code: 900) | 1 |
1 errors validating against transport.xsd The 'Note' element is invalid - The value 'Type 2 diabetes mellitus with hyperglycemia, without long-term current use of insulin E11.65' is invalid according to its datatype 'an1..70' - The actual length is greater than the MaxLength value. (Error Code: 900) | 1 |
1 errors validating against transport.xsd The 'Note' element is invalid - The value 'this was sent to Longs 6/7:; ID 619a21f6897247a9-abe8709427acd660; Date 06-07-2021 by Sarah R Kaplan; Prescriber Sarah R Kaplan; To Longs Drug Store #9663 - 70 E Kaahumanu Ave Kahului HI (808) 877-0068 f: (808)' is invalid according to its datatype 'an1..210' - The Pattern constraint failed. (Error Code: 900) | 1 |
1 errors validating against transport.xsd The 'Number' element is invalid - The value '12605873288' is invalid according to its datatype 'n1..10' - The actual length is greater than the MaxLength value. (Error Code: 900) | 2 |
1 errors validating against transport.xsd The 'Number' element is invalid - The value '26022339862' is invalid according to its datatype 'n1..10' - The actual length is greater than the MaxLength value. (Error Code: 900) | 2 |
1 errors validating against transport.xsd The 'Number' element is invalid - The value '26086711166' is invalid according to its datatype 'n1..10' - The actual length is greater than the MaxLength value. (Error Code: 900) | 9 |
1 errors validating against transport.xsd The 'Number' element is invalid - The value '41923800944' is invalid according to its datatype 'n1..10' - The actual length is greater than the MaxLength value. (Error Code: 900) | 7 |
1 errors validating against transport.xsd The 'Number' element is invalid - The value '41963396028' is invalid according to its datatype 'n1..10' - The actual length is greater than the MaxLength value. (Error Code: 900) | 10 |
1 errors validating against transport.xsd The 'NumberOfRefills' element is invalid - The value '180' is invalid according to its datatype 'n1..2' - The actual length is greater than the MaxLength value. (Error Code: 900) | 3 |
1 errors validating against transport.xsd The 'StateProvince' element is invalid - The value '' is invalid according to its datatype 'an' - The Pattern constraint failed. (Error Code: 900) | 7 |
Cannot get endpoint | 5 |
Cannot get endpoint (Error Code: 900) | 2 |
Cannot get endpoint (id = 6505404965001 uid = 1), please contact support (Error Code: 602) | 1 |
Cannot get endpoint (id = 6888514521004 uid = 1), please contact support (Error Code: 602) | 1 |
Cannot get endpoint (id = 6989612455001 uid = 1), please contact support (Error Code: 602) | 1 |
Cannot get portal (id = 422345), please contact support (Error Code: 602) | 1 |
Controlled Substance Cannot Be Faxed (Error Code: 900) | 3 |
Controlled substance must be signed or Digital Signature Indicator must be true. (Error Code: 900) | 6 |
Controlled substance must have a DEASchedule populated in Medication Prescribed (Error Code: 900) | 43 |
DEA or SSN not found in the message (Error Code: 900) | 80 |
Duplicate Prescription, original message has been processed (Error Code: 900) (Desc. Code: 220) | 1 |
ERROR FORWARDING XML MESSAGE (Error Code: 900) | 13 |
ERX611 ERROR PROCESSING MESSAGE TO RECIPIENT (Error Code: 600) (Desc. Code: 4000) | 4 |
Effective November 19, 2021, Walmart and Sams Club no longer stocks this medication. Please contact your pharmacist to discuss alternatives. (Error Code: 900) | 2 |
Fax receipt could not be confirmed (Error Code: 600) | 47 |
Intermediary system error. (Error Code: 900) (Desc. Code: 4020) | 43 |
Internal Processing Error Occurred (Error Code: 602) | 31 |
Internal error.Contact pharmacy by other means. (Error Code: 600) (Desc. Code: 008) | 38 |
Invalid MedicationPrescribed WrittenDate (Error Code: 900) | 1 |
MAIL-PREVIOUS RESPONSE HAS BEEN PROCESSED. THIS WILL NOT BE ACCEPTED (Error Code: 602) | 8 |
Mailboxed message could not be confirmed after three tries (Error Code: 602) | 3 |
Message is a duplicate. | 327 |
Message type not supported by sender (Error Code: 900) | 1 |
NON PARTICIPATING PHARMACY for C19 ANTIVIRAL REFER TO STATE HEALTH DEPT FOR PARTICIPATING PHARMACY (Error Code: 900) | 4 |
NewRx MedicationPrescribed Quantity must be a non-zero value (Error Code: 900) | 91 |
No NDC or compound or supply code in message for MedicationPrescribed (Error Code: 900) | 41 |
No NDC or compound or supply code in message for MedicationResponse (Error Code: 900) | 2 |
Not able to connect to Backend[DMZ Error] (Error Code: 900) | 7 |
Partner rejected transaction with an error (Error Code: 900) (Desc. Code: 220) | 620 |
Patient DOB does not match with the request (Error Code: 900) | 2 |
Patient address required for controlled substance (Error Code: 900) | 1 |
Pharmacy does not dispense class I or class II controlled substances (Error Code: 900) | 3 |
Pharmacy is Temporarily Unavailable (Error Code: 900) | 4 |
Pharmacy is temporarily down. Please retry. (Error Code: 601) | 1 |
Pharmacy is temporarily down. Please retry. (Error Code: 900) (Desc. Code: 2000) | 43 |
Pharmacy record inactive, may have new active location (Error Code: 900) | 77 |
Prescriber address required for controlled substance (Error Code: 900) | 5 |
PrescriberOrderNumber must be present and populated (Error Code: 900) | 19 |
Prescription no longer active (Error Code: 601) | 464 |
QS1: Transaction unable to be processed (Error Code: 900) | 1 |
Receiving partner does not support this type of message (Error Code: 900) | 86 |
Refill request has been canceled (Error Code: 900) (Desc. Code: 4020) | 34 |
Request has been canceled prior to response. Response not processed. (Error Code: 601) (Desc. Code: 008) | 346 |
Rx Outreach pharmacy does not currently offer this product. (Error Code: 900) | 1 |
SCRIPT Validation Error: See free text for description (Error Code: 900) | 3 |
Storage error. (Error Code: 602) | 1 |
The MedicationPrescribed QuantityUnitOfMeasure Code: C16830 is invalid (Error Code: 900) | 3 |
The MedicationPrescribed QuantityUnitOfMeasure Code: C28253 is invalid (Error Code: 900) | 4 |
The MedicationPrescribed QuantityUnitOfMeasure Code: C48477 is invalid (Error Code: 900) | 6 |
The MedicationPrescribed QuantityUnitOfMeasure Code: C48478 is invalid (Error Code: 900) | 8 |
The MedicationPrescribed QuantityUnitOfMeasure Code: C48481 is invalid (Error Code: 900) | 1 |
The MedicationPrescribed QuantityUnitOfMeasure Code: C48519 is invalid (Error Code: 900) | 1 |
The MedicationPrescribed QuantityUnitOfMeasure Code: C48537 is invalid (Error Code: 900) | 1 |
The MedicationPrescribed QuantityUnitOfMeasure Code: C48540 is invalid (Error Code: 900) | 110 |
The MedicationPrescribed QuantityUnitOfMeasure Code: C48549 is invalid (Error Code: 900) | 4 |
The MedicationPrescribed QuantityUnitOfMeasure Code: C48551 is invalid (Error Code: 900) | 403 |
The MedicationPrescribed QuantityUnitOfMeasure Code: C62275 is invalid (Error Code: 900) | 48 |
The MedicationPrescribed QuantityUnitOfMeasure Code: C97717 is invalid (Error Code: 900) | 6 |
This pharmacy dispenses oxybate products only. Please select different pharmacy (Error Code: 900) | 2 |
TrackingID service error (Error Code: 900) | 1 |
Transaction is a duplicate. (Error Code: 601) (Desc. Code: 220) | 2 |
Transaction rejected. Does not follow NCPDP standard or implementation guide rules. Duplicate Message (Error Code: 900) | 1 |
UIB Trace Number is Invalid (Error Code: 900) | 1 |
Unable To Process (Error Code: 601) (Desc. Code: 210) | 14 |
Unable to associate Response to Request (Error Code: 601) | 1 |
Unable to store the incoming request/image. Error:Unable to locate the NCPDPID: 5710098 (Error Code: 601) (Desc. Code: 4020) | 1 |
Unexpected Exception occured while processing the request. (Error Code: 900) | 2 |
Unexpected Exception occurred while processing the request. (Error Code: 900) | 19 |
Total error rate: 5129 / 2094062 * 100 = 0.24%
Discussion
As expected, all supported message types have a greater than zero total message count, and the total number of messages far exceeds the number of errored messages with a total error rate of 0.24%. Over the four quarters the error rate fluctuated from a low of 0.22% in Q2 to a high of 0.26% in Q3. This shows an overall low error rate for the whole year of 2022. Additionally, the number of NewRx messages is significantly greater than the number of RxChangeResponse, CancelRx, and RxRenewalResponse messages.
Measure 4: E-Prescribing Diagnosis Codes
Description
This measure will verify that all diagnosis elements are present in some e-prescribing messages as required by §170.315(b)(3), including inbound and outbound message types.
Associated Certification Criteria
Certification Criteria | Requirement(s) |
"Electronic prescribing" (§170.315(b)(3)) | (b)(3)(ii)(C)(1) - Primary/Secondary Diagnosis |
Test Methodology
MIE will report the contents of each stored message in a client’s local database table of stored messages, and counts the inbound and outbound messages that include Diagnosis elements. MIE will run the report for each client under consideration and aggregate the results.
Results
Total NewRx Messages | NewRx Messages with Diagnosis Included |
437559 | 126844 |
Discussion
Since the Diagnosis elements are not a required component of a NewRx message, as anticipated, only a subset (28.99%) of the NewRx messages included a diagnosis. The rate of NewRx messages with a diagnosis included fluctuated from 26.53% in Q1 to 31.16% in Q4 showing an increasing rate of using the diagnosis element.
Measure 5: E-Prescribing Oral Liquid Units
Description
This measure will verify that prescriptions for medications with an oral liquid form will have a quantity unit of measurement of mL, not cc or English units as outlined in §170.315(b)(3).
Associated Certification Criteria
Certification Criteria | Requirement(s) |
"Electronic prescribing" (§170.315(b)(3)) | (b)(3)(ii)(E) - Metric Units |
Test Methodology
MIE will create a system report that examines the contents of each stored NewRx message in a client’s local database table of stored messages, limiting to oral liquid medications, and provides a count of each distinct quantity unit of measure used. MIE will run the report for each client under consideration and aggregate the results.
Results
Unit Code | Unit Description | Quantity |
C28254 | Milliliter (ml) | 13891 |
C48155 | Gram (g) | 4 |
C48477 | Bottle | 2 |
C48480 | Capsule | 1 |
C48521 | Packet | 4 |
C48542 | Tablet | 3 |
C64933 | Each | 20 |
Discussion
As expected, C28254 (milliliters) is the most commonly sent unit of measure for oral liquid medications. Non-C28254 units were only sent in 0.24% of oral liquid medication messages. This rate fluctuated from 0.03% in Q3 to 0.55% in Q1.
Measure 6: E-Prescribing Decimal Format
Description
This measure will verify that numeric amounts in prescriptions include leading zeros before decimal points and do not allow trailing zeros after a decimal point.
Associated Certification Criteria
Certification Criteria | Requirement(s) |
"Electronic prescribing" (§170.315(b)(3)) | (b)(3)(ii)(F) - Decimal Format |
Test Methodology
MIE will create a system report that examines the contents of each stored NewRx message in a client’s local database table of stored messages, and provides a count of prescription messages that include inappropriate trailing zeros, and a count of those missing leading zeros. MIE will run the report for each client under consideration and aggregate the results.
Results
Total NewRx Messages | NewRx Messages with Improper Decimal Format |
437559 | 1895 |
Discussion
As expected, the number of NewRx messages sent with inappropriate trailing zeros, or missing leading zeros, occurs rarely in only 0.43% of messages. This rate increased over the four quarters from 0.35% to 0.53%.
Measure 7: CDA Download
Description
This measure will verify that the system can accept a CDA document uploaded into the system, assign it to the appropriate chart in the system as appropriate, and display the document with a standard stylesheet with all sections being accepted and visible.
Associated Certification Criteria
Certification Criteria | Requirement(s) |
"Clinical information reconciliation and incorporation" (§ 170.315(b)(2)) | (b)(2)(ii) - Correct patient. |
‘‘Transitions of care" (§ 170.315(b)(1)) | (b)(1)(ii) - All paragraphs |
(b)(1)(iii) - All paragraphs |
Test Methodology
MIE will report on the number of CDA formatted documents uploaded into tracked Webchart systems and the number of upload attempts that failed as stored in client databases and error log files.
MIE will report on the number of requests to view a CDA document within the system, and the number of times it displayed correctly, and when there were errors in display.
Any errors reported by customers or the recipients of their quarterly attestations will be tracked and reported as a baseline. These test assumptions for customer reporting align with the “visual inspection” aspects of the test lab tests.
Results
Total CDA Documents Uploaded | 43,354 |
Total CDA Document Upload Failures | 0 |
Total CDA Document Views | 8,006 |
Total CDA Document View Reported Errors | 0 |
Total Successful CDA Views | 8,006 |
Discussion
As anticipated, there were no errors in being able to upload and view valid CCDA documents.
Usage increased throughout the year, with the final quarter accounting for more than half of the numbers above (23,760 uploaded CDAs and 5,066 document views).
Measure 8: Application Access Documentation
Description
This measure will verify that WebChart EHR’s API documentation is publicly and perpetually available. Compliance will be recorded by an external uptime monitor and reported quarterly. Upon request, or in the event of downtime, data can additionally be reported in daily, weekly, or monthly increments.
Associated Certification Criteria
Certification Criteria | Requirement(s) |
"Application access – patient selection" (§ 170.315(g)(7)) | (g)(7)(ii)(A)(1) - Functional Documentation |
(g)(7)(ii)(A)(2) - Implementation Requirements | |
(g)(7)(ii)(A)(3) - Terms of Use | |
(g)(7)(ii)(B) - Public Link | |
"Application access – data category request" (§ 170.315(g)(8)) | (g)(8)(ii)(A)(1) - Documentation |
(g)(8)(ii)(A)(2) - Implementation Requirements | |
(g)(8)(ii)(A)(3) - Terms of Use | |
(g)(8)(ii)(B) - Public URL | |
"Application access—all data request" (§ 170.315(g)(9)) | (g)(9)(ii)(A)(i) - Documentation |
(g)(9)(ii)(A)(ii) - Implementation Requirements | |
(g)(9)(ii)(B) - Public URL |
Test Methodology
An external uptime monitor will check the availability of all documentation available at https://docs.webchartnow.com/resources/system-specifications/application-programming-interface-api.html . Both up- and downtime will be logged to be reported quarterly. The cause of any downtime and the duration will also be logged In the event of any downtime, the amount of downtime can be reported at daily, weekly, or monthly intervals in addition to the quarterly reports, and the cause of each downtime occurrence will be reported.
Results
The MIE API documentation was available 99.967% of the time.
The MIE API documentation was available 99.997% of Q1. There was a 3 minute down period on March 17, 2022 due to a connection timeout.
The MIE API documentation was available 100% of Q2.
The MIE API documentation was available 99.902% of Q3. There were several short down-time events during July averaging 17.29 minutes per event all due to Cloudflare timeouts.
The MIE API documentation was available 99.967% of Q4. Three instances of prolonged downtime occurred (10/14/2022: 13 minutes -unannounced US Signal downtime issue , 10/27/2022: 5 minutes - unannounced US Signal downtime issue, 12/6/2022: 25 minutes - **** US Signal HRN Maintenance Window).
Discussion
As expected, the documentation maintained an uptime of greater than 99.9% at 99.967% for the year. During any down time the API documentation would not have been available, but, no effect to end user requests for the information was reported.
Measure 9: Clinical Information Reconciliation and Incorporation
Description
This measure will verify that the system can take a CCDA transition of care/referral summary formatted according to the standards adopted §170.205(a)(3) and §170.205(a)(4) and read the data for medications, allergies, and conditions from the document, reconcile those into the chart, and that the data is fully incorporated into the chart.
Associated Certification Criteria
Certification Criteria | Requirement(s) |
"Clinical information reconciliation and incorporation" (§ 170.315(b)(2)) | (b)(2)(iii)(A), (B), (C), (D) |
(b)(2)(iv) - System Verification |
Test Methodology
MIE will report on the number of CDA formatted documents reconciled via the “Froozhie” tool.
Following each reconcile, a temporary CDA for the chart will be created and validated to ensure the reconciled data can be incorporated into a CDA created free of schematic errors (the CDA document will NOT be kept, only the result of the validation). Additionally, any client complaints that data is not being imported correctly from the tool will be tracked, investigated, and reported
Results
CDA Documents Reconciled | 1,238 |
CDA Documents Generated on a Chart after a Reconcile that had an invalid Validation | 87 |
Client reported Issues with CDA reconciliation | 0 |
Discussion
The use of CDA reconciliation in systems was relatively low, and no issues with the reconciliation of the documents were reported. In the 4th quarter there was a steep decline in reconciliations because of issues with workflow from the main client using the feature. This has been addressed and we plan to see higher numbers in the beginning of 2023 again.
The number of invalid validations were higher than anticipated. Work is continuing to be done to rectify the issues with the CDA generation in these systems, and to put in place a more immediate notification process when invalid CDAs are generated. Many of the issues with invalid CCDAs were because of data missing in the CCDA requests that were not handled well (empty quotes, etc) and those issues have been addressed and should not result in failures going forward.
Measure 10: Transmission to Immunization Registry: Create Content
Description
This measure will verify that the system can generate a VXU conforming to the HL7 v2.5.1 standard, CDC guidance for communication to Immunization Registries and state/local guidance. The VXU messages shall contain information related to the demographics and vaccination administration record.
Associated Certification Criteria
Certification Criteria | Requirement(s) |
"Transmission to immunization registries" (§ 170.315(f)(1)) | (f)(1)(i) - Create Content |
Test Methodology
MIE will report from the database the number of successfully sent VXU messages acknowledged as received by the state immunization registry. MIE will also report from the database on the number of records rejected by the state registry due to error, whether the failure was due to registry internal errors, clinical data entry issues or a not well-formed message. Finally, MIE will report from the database the number of messages which declined to be generated due to data entry issues failing message pre-validation.
Results
Successful transmission to registry | 8766 |
Rejected by the registry | 72 |
Declined to generate | 41 |
Discussion
As expected, very few messages were rejected by the state registry. There is a small portion of the entered injections which were not able to be transmitted to the state registry due to user data entry issues. The overall error rate for the year (rejected and declined) was 1.2%. The rate of Vaccination submission tended to be lighter during the first half of the year and ramped up substantially for the second half.
Measure 11: Transmission to Immunization Registries: Query Records
Description
This measure will verify that the system can generate a QBP conforming to the HL7 v2.5.1 standard, CDC guidance for communication to Immunization Registries and state/local guidance. Furthermore, the system shall be able to retrieve, consume and display to the end user the results of any such query.
Associated Certification Criteria
Certification Criteria | Requirement(s) |
"Transmission to immunization registries" (§ 170.315(f)(1)) | f)(1)(ii) - Query Records |
Test Methodology
MIE will report the number of successful retrievals of evaluated history and forecasting operations from the database. MIE will report the number of failed retrievals, including those resulting from an internal error in the registry resulting in an inability to consume a response from the database. MIE will manually track, resolve and report issues resulting from WebChart EHR application errors as reported by end users.
Results
Successful retrievals | 31417 |
Failed retrievals | 597 |
Discussion
The rate of queries remained fairly consistent with a slight trend upwards as the year progressed. The overall error rate for the year was 1.9%.
Measure 12: Direct Project: Send
Description
This measure will verify that the system can transmit a Direct project conforming S/MIME to a HISP. The measure will also verify the receipt of those transmissions by verifying the status of the resultant MDN messages.
Associated Certification Criteria
Certification Criteria | Requirement(s) |
"Direct Project" (§ 170.315(h)(1)) | (h)(1)(i) - Send |
(h)(1)(ii) - Message Disposition Notification: Processed | |
(h)(1)(ii) - Message Disposition Notification: Failed |
Test Methodology
MIE will report from log files the number of messages transmitted. MIE will report from logs the number of messages which failed to be transmitted whether due to internal error, external failures or inability to verify trust of the recipient. MIE will report from logs the number of Processed MDN messages received. MIE will report from logs the number of Failed MDN messages received.
Results
Messages Transmitted | 7 |
Failed to transmit | 0 |
Failed MDNs received | 0 |
Discussion
The rate of transmission was very low for this year. In the coming year, we are looking into ways to increase the usage of Direct Messaging transmissions by the end users.
Measure 13: Direct Project: Receive
Description
This measure will verify that the system conforms to Direct Project message receipt requirements for validation.
Associated Certification Criteria
Certification Criteria | Requirement(s) |
"Direct Project" (§ 170.315(h)(1)) | (h)(1)(i) - Receive |
Test Methodology
MIE will report from logs the number of messages transmitted to the HISP. MIE will report from logs the number of messages failing to conform to Direct Project specifications. MIE will report from logs the number of messages which are successfully delivered to recipients.
Results
Messages received by HISP | 5 |
Messages failing to conform | 0 |
Messages successfully delivered | 5 |
Discussion
Receipt of Direct Messages for this year was very limited. In 2023 we are looking into ways to increase the ability of outside practitioners to utilize our client endpoints.
Measure 14: Patient Portal View
Description
This measure will verify that a patient can view various document types within the patient portal.
Associated Certification Criteria
Certification Criteria | Requirement(s) |
"View, download, and transmit to 3rd party" (§ 170.315(e)(1)) | (e)(1)(i)(A)(1),(2),(3),(4),(5) |
(e)(1)(i)(D)(1), (2) | |
(e)(1)(ii)(A) |
Test Methodology
MIE will report a number of measurements surrounding documents, including:
- Number of documents sent successfully to patients in the patient portal
- Number of documents unsuccessfully sent to the patient portal
- Number of documents read by patients in the patient portal
- Number of failures in the ability to read messages in the patient portal
Results will be retrieved from database tables and aggregated for reporting. Any failures will be reported from the information found in log files as well as any client reported issues tracked during the testing period.
Results
CDA Documents Sent to the patient portal | 21 |
CDA Documents Unsuccessfully Sent to the patient portal | 0 |
CDA Documents Read by Patients in the Patient Portal | 8 |
CDA Documents Failed to be read in the Patient Portal | 0 |
Discussion
The low number of CDA documents in the NMC patient portal was because workflows were still sending other document types instead of CDA to the patient portal. Client workflows in 2023 are anticipated to greatly increase the number of C-CDA documents being consumed in Patient Portals in 2023. We did not report any known failures to read a CCDA document in the NMC portal in 2023.
Measure 15: Patient Portal Download
Description
This measure will verify that a patient can download various document types within the patient portal.
Associated Certification Criteria
Certification Criteria | Requirement(s) |
"View, download, and transmit to 3rd party" (§ 170.315(e)(1)) | (e)(1)(i)(B)(1), (2), (3) |
(e)(1)(i)(D)(1), (2) | |
(e)(1)(ii)(A) |
Test Methodology
MIE will report a number of measurements surrounding documents, including:
- Number of documents sent successfully to patients in the patient portal
- Number of documents unsuccessfully sent to the patient portal
- Number of documents successfully downloaded from the patient portal
- Number of documents unsuccessful in being downloaded from the patient portal.
Results will be retrieved from database tables and aggregated for reporting. Any failures will be reported from the information found in log files and third party reports as well as any client reported issues tracked during the testing period.
Results
CDA Documents Sent to the patient portal | 21 |
CDA Documents Unsuccessfully Sent to the patient portal | 0 |
CDA Documents Downloaded by Patients in the Patient Portal | 4 |
CDA Documents Failed to be downloaded in the Patient Portal | 0 |
Discussion
The low number of CDA documents in the NMC patient portal was because workflows were still sending other document types instead of CDA to the patient portal. Client workflows in 2023 are anticipated to greatly increase the number of C-CDA documents being consumed in Patient Portals in 2023. 4 documents that were in the portal were exported/downloaded.
Measure 16: Patient Portal CCDA Transmit
Description
This measure will verify that a patient can transmit various document types within the patient portal to other entities.
Associated Certification Criteria
Certification Criteria | Requirement(s) |
"View, download, and transmit to 3rd party" (§ 170.315(e)(1)) | (e)(1)(i)(C)(1), (2) |
(e)(1)(i)(D)(1), (2) | |
(e)(1)(ii)(A) |
Test Methodology
MIE will report a number of measurements surrounding documents, including:
- Number of documents sent successfully to patients in the patient portal
- Number of documents unsuccessfully sent to the patient portal
- Number of documents successfully transmitted from the patient portal
- Number of documents unsuccessful in being transmitted from the patient portal.
Results will be retrieved from database tables and aggregated for reporting. Any failures will be reported from the information found in log files and third party reports as well as any client reported issues tracked during the testing period.
Results
CDA Documents Sent to the patient portal | 21 |
CDA Documents Unsuccessfully Sent to the patient portal | 0 |
CDA Documents Transmitted by Patients in the Patient Portal | 0 |
CDA Documents Failed to be transmitted in the Patient Portal | 0 |
Discussion
The low number of CDA documents in the NMC patient portal was because workflows were still sending other document types instead of CDA to the patient portal. Client workflows in 2023 are anticipated to greatly increase the number of C-CDA documents being consumed in Patient Portals in 2023. With the low number of CDA documents in the portal, there were no reported transmission attempts of CDA documents out of the portal. If transmissions are low, even with the increase in CCDA portal documents in 2023, we will have automated tests of transmissions of test chart documents scheduled in 2023.
Measure 17: Send Using Edge Protocol for SMTP / XDM
Description
This measure will verify that the system is able to utilize a SMTP edge protocol for sending and receiving Direct Project messages. As part of receiving messages, XDM shall be handled when applicable.
Associated Certification Criteria
Certification Criteria | Requirement(s) |
‘‘Transitions of care" (§ 170.315(b)(1)) | (b)(1)(i)(A)(Alternative) - Send Using Edge Protocol for SMTP/IXE XDR |
(b)(1)(i)(B)(Alternative) - Receive Using Edge Protocol for SMTP/IXE XDR | |
(b)(1)(i)(C)(Conditional) - XDM Processing |
Test Methodology
MIE will report from logs the number of messages transmitted to the HISP by SMTP. MIE will report from logs the number of messages received from the HISP by SMTP. MIE will report from logs the number of XDM packages processed. In the case where insufficient real-world data is available, data resulting from regular testing with DirectTrust shall be included in the reporting.
Results
Messages submitted to HISP via SMTP | 7 |
Number of XDM packages processed | 346 |
Discussion
As expected, there was very low usage. XDM packages that were processed included packages from standard regular testing.
Measure 18: Data Export
Description
This measure will verify that a user can use WebChart EHR’s Data Export Tool to pull down groups of patient data from a Webchart EHR system.
Associated Certification Criteria
Certification Criteria | Requirement(s) |
"Data export" (§ 170.315(b)(6)) | (b)(6)(i) |
(b)(6)(ii)(A)-(F) | |
(b)(6)(iii)(A)-(B) | |
(b)(6)(iv) |
Test Methodology
MIE will report from the event log database tables a series of occurrences that indicates use of the WebChart EHR Data Export Tool:
- Event logs of the report to find all patients for Document Export being called.
- Event logs of CDA documents being generated within a certain short time period following the report.
MIE will track customer reports of data expected to be in mass data export downloads that did not download as failures.
Results
Number of times EHR Data Export Tool was used: 0
Discussion
We did not have any incidents of clients using the EHR Data Export Tool available at https://github.mieweb.com/webchart/wcexport . We will be adding some automated tests to test the usage of the export tool during the 2023 calendar year.
Measure 19: CDA Validation
Description
This measure will verify that CDAs both created by and received by a Webchart EHR system pass basic CDA validation.
Associated Certification Criteria
Certification Criteria | Requirement(s) |
‘‘Transitions of care" (§ 170.315(b)(1) Cures Update) | (b)(1)(ii)(A) |
"Data export" (§ 170.315(b)(6)) | (b)(6)(ii), (A)-(F) |
Test Methodology
All CDAs stored within a Webchart EHR will be run through schema validation regardless of the document’s origin. Documents may originate within the WebChart EHR system or be imported from a third party application of manual upload. The schema validator will be installed within the MIE production environment to ensure the security of all PHI contained in the documents. Only results of the validation will be made available, document content will not be revealed to developers during testing.
The number of valid vs. invalid CDAs and their sources will be reported.
Results
All CDAs
CDA Documents Marked Valid in Period | 2628 |
CDA Documents Marked Invalid in Period | 1589 |
% Marked Invalid In Period | 38% |
Webchart Generated CDAs Only
CDA Documents Marked Valid in Period | 26 |
CDA Documents Marked Invalid in Period | 39 |
% Marked Invalid In Period | 60% |
Discussion
The high percentage of invalid validation documents were caused by a few factors. From the majority of CDAs that came from outside 3rd party vendors, these were issues with the content that they sent in. We have been working with these vendors to establish getting alerts to them when CDAs are not passing validation, and we expect to see improvements in those numbers in 2023. For the Webchart generated documents, the documents generated were low, and mostly generated manually. The issues seen were from workflow issues within various systems that generated missing data. We’ve improved the CCDA generation throughout the year to display the data properly for validation when missing. We are also anticipating more automated CCDA generation being implemented as part of workflows in 2023.
It should also be noted that none of the invalid documents caused any noted problems for viewing or ingesting documents during the year. The errors were of a nature such as “non-conformant” empty fields in attributes that did not cause any issues with viewing/importing data.
Measure 20: Patient Data requests VIA API
Description
This measure will verify that the API as outlined in WebChart EHR’s documentation is functional. A valid request for patient information must provide that information.
Associated Certification Criteria
Certification Criteria | Requirement(s) |
"Application access – patient selection" (§ 170.315(g)(7)) | (g)(7)(i) - Query processing and response |
"Application access – data category request" (§ 170.315(g)(8)) | (g)(8)(i)(A) - Return CCDS data |
(g)(8)(i)(B) - Request response | |
"Application access—all data request" (§ 170.315(g)(9)) | (g)(9)(i)(A)(1) - Demonstrate API |
(g)(9)(i)(A)(3) - Data Classes | |
(g)(9)(i)(B) - Data Return |
Test Methodology
To address the overall automated testing, the following test requests will be made daily against a test system in a production environment.
- Issue a request in the browser to search for a patient (patient selection)
- Issue a request in the browser to request demographics of a patient (data category request)
- Issue a request using the export tool described in the documentation.
All API requests made in production systems are recorded in log files. The number of requests logged will be reported against the number of issues with API functionality that are reported.
Results
Production Exports | 3 |
Total Charts Exported | 167268 |
Total Export Errors | 26 |
Discussion
As expected, the production use of the API was rare, but the total number of export errors was minimal. Patient chart exports were successful in 99.9% of cases.
Measure 21: Web Content Accessibility
Description
This measure will verify that all certified content in the patient portal will maintain accessibility conformance as outlined in the Web Content Accessibility Guidelines (WCAG) 2.0.
Associated Certification Criteria
Certification Criteria | Requirement(s) |
(e)(1)(i) - Web Content Accessibility |
Test Methodology
MIE will conduct monthly third-party production accessibility scanning as well as automated nightly internal accessibility scanning of a test system in a production environment.
Results
The internal accessibility scanning of a pre-production test system identified 0 urgent and 0 secondary non-conformance issues in 98.79% of nightly and ad-hoc scans throughout the year. In the remaining 1.21% of scans only secondary issues were uncovered, all of which were eliminated prior to code changes reaching production systems. Production accessibility scanning identified 0 urgent and 0 secondary non-conformance issues for the entire quarter.
Discussion
As expected no urgent non-conformance issues were identified in either live production or pre-production test systems. In the rare occurrence that a secondary non-conformance issue was identified in testing, it was addressed and eliminated prior to reaching live production systems.
Schedule of Key Milestones
Key Milestone | Care Setting | Date/Timeframe | Status |
Release of documentation for the Real World Testing to be provided to ACB and providers | All settings | Nov 15, 2021 | Complete |
Begin collection of information as laid out by the plan | All settings | Jan 1, 2022 | Complete |
New certification of b.7, b.8, b.9 | All settings | Q1, 2022 | Complete |
Certification of additional CQMs to c.1, c.2, c.3 and QRDA SVAP | All settings | Q1, 2022 | Delayed to 2023 |
Attest to Cures update version of b.2, e.1 | All settings | Q1, 2022 | Complete |
Follow-up with providers and authorized representatives to understand any issues arising with the data collection. | All settings | Quarterly, 2022 | Complete |
Data collection and review. | All settings | Quarterly, 2022 | Complete |
New certification of b.10, g.10 | All settings | Q3, 2022 | G.10 complete B.10 planned for 2023 |
End of Real World Testing period/final collection of all data for analysis | All settings | Dec 31, 2022 | Complete |
Data analysis and report creation | All settings | January, 2023 | Complete |
Submission of Real World Testing Results to ACB | All settings | Feb 1, 2023 |
Attestation
This Real World Testing plan is complete with all required elements, including measures that address all certification criteria and care settings. All information in this plan is up to date and fully addresses the health IT developer’s Real World Testing requirements.
Authorized Representative Name | Doug Horner |
Authorized Representative Email | horner@mieweb.com |
Authorized Representative Phone | 260-459-6270 |
Authorized Representative Signature | Doug Horner |
Date | 02/01/2023 |
Enterprise Health Documentation
Last Updated:
Last Build:
Fri, 26 Jul 2024 16:24:35 UTC
WikiGDrive Version: dd69069d725fca5f553df7ded62e130a49d49ca6