ULTROMICS EchoGo Pro v1.0.5 Clinician User Manual

ULTROMICS EchoGo Pro v1.0.5 Clinician

Introduction

The Echo Go Pro Clinicians User Manual contains the information required for the clinical use of the Echo Go Pro software application. Echo Go Pro is a stand-alone software application that utilizes artificial intelligence (AI) and geometric features to assess whether a stress echocardiogram is suggestive of the possibility significant coronary artery disease (CAD). Significant CAD is determined by Echo Go Pro was defined as showing evidence of ≥70% stenosis in the proximal to mid LAD, proximal left circumflex or proximal to mid RCA as measured by invasive angiography performed within 6 months of stress echocardiogram.
The Echo Go Pro application will primarily be delivered as a software as a service (SAAS) and the clinician user will not need to install Echo Go Pro. The requirements of integrating with the clinical site are described below (Section 10) and the stress echo exams should be acquired using standard clinical protocols. An image of the report with the calculated result will be appended to the end of the stress echo exam and will be displayed in the image series when accessed in the routine workflow at the clinical site for the clinician to review.
Trained operators will complete the image processing using Echo Go Pro. An overview of the operator training procedure and workflow is described in this manual to provide the clinician user with a detailed understanding of the analysis process.
For use in operating the software, please refer to Echo Go Pro Operators Manual.

Purpose and Scope

This manual has been designed to provide guidance for the clinical assessment of the left ventricular (LV) analysis utilizing the Echo Go Pro application. Echo Go Pro software provides LV analysis of stress echo exams for patients with suspected coronary artery disease during both the rest and the peak exercise phase and provides results to support clinician interpretation of the stress-echo examinations.
Constraints and requirements that need to be met to operate Echo Go Pro in a way that is compliant with its intended use are marked with an attention icon.
The word “shall” is used for those requirements that need to be met to assure the safe and effective operation of Echo Go Pro.

Glossary of Terms

Device Description

Input to the Echo Go Pro v1.0.5 software is DICOM 3.0 compliant stress echocardiogram (SE) datasets on patients undergoing SE investigations for suspected CAD. The SE should be acquired following the standard clinical SE protocol that provides apical 2 chamber (A2C), 4 chamber (A4C) and parasternal short axis (SAX) views. The datasets are transferred to Ultromics using a third-party transfer medical device data system (MDDS) that also anonymizes the data. The MDDS does not control or alter the functions or parameters of Echo Go Pro and is used solely for the electronic transfer of medical device data.
The incoming DICOM study is checked for consistency and completeness, i.e., whether all required views labels are present in the DICOM meta data (Technical Quality Control (QC)). If required view labels are not present, the study will be rejected, and a rejection report will be generated and sent to the referring physician/clinical site.
Once the technical QC has been performed on the DICOM datasets, the algorithm for automated contour detection of the endocardium of the left ventricle (LV) is applied to identify the region of interest (ROI) which is presented for review and approval by trained operators in the Thumbnail and Contour Screen. An auto-contouring algorithm places a trace around the LV that sufficiently captures the LV shape (ROIs). Contours of the LV are traced for all AP2, AP4 and mid-ventricular SAX views at both rest and peak stress, and a single ED to ES cycle per view is automatically identified to be used in the analysis. The approved contour traces are used in for geometric feature calculation which form the input to the trained prediction algorithm. A report is generated from the results which indicates whether or not the SE is suggestive of CAD. The result is intended as an additional support to standard diagnostic pathways and should only be used by a board-certified cardiologist/clinician.

Intended Use/Indications for Use

Intended Use
Based on an echocardiogram, indicate whether there is evidence of disease as an aid to diagnosis.
Indication for Use
Echo Go Pro v1.0.5 is a machine learning-based decision support system, indicated as an adjunct to diagnostic stress echocardiography for patients undergoing assessment for coronary artery disease (CAD). When utilized by an interpreting physician, this device provides information that may be useful in rendering an accurate diagnosis. Patient management decisions should not be made solely on the results of the Echo Go Pro v1.0.5 analysis. Echo Go Pro v1.0.5 is to be used with stress echo exam protocols that contain A2C, A4C and mid-ventricular short-axis views at rest and at peak stress. Echo Go Pro v1.0.5 is not intended for the assessment of mild or moderate myocardial ischemia, or localization of coronary artery disease, or for the assessment of myocardial perfusion, myocardial viability or valve disease.
Limitations:
Echo Go Pro v1.0.5 has not been validated on patients who underwent previous coronary artery bypass graft (CABG) surgery.

Cautions, Precautions & Limitations

Please carefully read the information in this section before using Echo Go Pro 1.0.5, it contains important information on operating safety and use of the product.
CAUTION

• U.S Federal law restricts this product to sale by or on the order of the clinician.
• The services provided by Ultromics are not regulated by the Food and Drug Administration and are under the jurisdiction of the state and local government. Please ensure that your institution is following the appropriate local laws for the use of software products.

PRECAUTIONS

• The product provides results based on images from a stress echo exam acquired by a professionally accredited echocardiographer/sonographer or medically certified cardiologist/clinician.
• The software provides a result based on image selection and automated contouring that shall be approved by a trained operator.
• The analysis of images shall be rejected by the operator according to specific quality control (QC) acceptanace criteria are no met no calculations will be provided by the software.
• Clinicians can reject the resulting report if there is doubt in the accuracy given the known operator quality acceptance criteria.
• The clinician remains responsible for determining if the provided results are acceptable for the corresponding stress echo exam and for their use in supporting diagnostic decisions.
• Only persons in the intended user group shall use the product. Intended user groups include trained operators who have the required credentials appropriate to use the software to adjudicate contours and verify that a numerical result is generated by the software. The results from Echo Go Pro are intended to only be used for supporting a diagnosis by a licensed medical practitioner.

LIMITATIONS

• Not intended for use without clinician interpretation
• Echo Go Pro v1.0.5 has not been validated on patients who underwent previous coronary artery bypass graft (CABG) surgery.
• All views A2C, A4C and parasternal short axis (SAX) views at both rest and peak stress are required for a result to be provided.

Operator Competency

Accreditation Requirements
Operators will include echocardiographers/sonographers and clinicians. All are required to have accreditation from their recognized professional bodies (or equivalent) to ensure the device is used safely and effectively. Examples of accreditation bodies include the British Society of Echocardiography (BSE) or Registered Diagnostic Cardiac Sonographers (RDCS) through the American Registry for Diagnostic Medical Sonographers (ARDMS). Accreditation certification shall be maintained in an active status.

Training Process
All operators shall undergo a product training session. Operators using Echo Go Pro v1.0.5 and those working within Ultromics in the Software as a Service (SaaS) must meet competency requirements. Copies of training records and certificates of completion will be filed within the Ultromics Operations document retention system.

Refresher Training
Operators are required to maintain their competency. Refresher sessions requiring processing a randomized sample of stress echo exams will be completed. Copies of training records and certificates of completion will be filed within Ultromics document retention system. Re-training will be performed quarterly or at the discretion of the Quality Operations Management team/representative.

Summary of the Echo Go Pro Workflow

Echo Go Pro v1.0.5 provides additional information to the clinician to indicate whether or not stress echocardiograms are suggestive of significant CAD by providing a report back to the clinical site. A summary of the workflow is described below:

1. An accredited/board-certified professional performs a stress echo exam following standard clinical protocols.
2. Digital Imaging and Communications in Medicine (DICOM) 3.0 SE datasets are transferred to the application using an approved third-party transfer system which also anonymizes the data.
3. Technical QC is performed on the received anonymised dataset.
4. The left ventricles (ROIs) are automatically detected.
5. Auto-contours are presented on the contour screen for review.
6. Trained operators perform a quality check which includes review of image quality, review of views and stages being labelled correctly and approval of the auto-contours.
7. From the contours, geometric features are calculated as input to the machine learning model. The output of the model is included into a report that gives a categorical assessment as to whether the data are suggestive of significant CAD or not.
8. Trained operators review and confirm that the result has been generated successfully. Trained operators review and confirm that the result has been generated successfully.
9. Upon approval, a report is generated.
10. The report is returned using the same approved third-party transfer system and is used by the clinician as additional clinically relevant information to interpret the stress echo exam.

Operator User Access and Auditing

Each operator shall be provided with their own Windows account and will be required to use this account before launching EchoGo Pro. The Windows accounts shall not be shared between operators and each account shall have a unique password. Access to the application is restricted to authorized user accounts.
Standard measures will be in place for all users of the system to be compliant with general controls, including but not limited to:

• Each operator is required to have a password that matches their local information security policy.
• All user actions will be audited (e.g., log-ons and studies accessed) and these protected files will be stored as per local legal and organizational guidelines.
• If the Export logs feature is enabled audit logs may be extracted directly from the application by the trained operator in a zip file format by pressing the ‘log’ file button. The audit files are password protected and are kept securely with the IT department to allow for improved technical support.

Records need to be kept that relate the Windows username to the name of the operator. Windows login names shall not be reassigned to another operator. New operators will need a new unique Windows login name.

Retention Periods of Audit Log
The retention period of records in the audit log needs to comply with local legal requirements and organizational requirements.

Software Requirements and Integration at the Clinical Site

Integration at the clinical site will involve utilizing existing or third- party routers locally. The Echo Pro application is not installed at the clinical site as part of the SaaS model.
Data Flow and Software Installation
The clinical site’s local IT staff will coordinate the transfer of a copy of the DICOM images to either an existing router or a third-party router which will anonymize the studies and send them to a DICOM gateway. These routers will be locally installed within the clinical sites firewall. The anonymised studies will then be routed to a DICOM cloud suite which will be accessed by Ultromics so studies can be downloaded, and the images processed through the EchoGo Pro application. Reports will then be returned to the clinical site via the same process.
If the Export logs feature is enabled audit logs may be extracted directly from the application by the trained operator in a zip file format by pressing the ‘log’ file button. The audit files are password protected and are kept securely with the IT department to allow for improved technical support.
Requirements on Data-in
Stress Echocardiogram images shall be acquired using ultrasound imaging systems cleared by the Food and Drug Administration (FDA) or appropriate regulatory body for use in echocardiography.
Stress Echocardiogram images shall be acquired using ultrasound imaging systems for use in echocardiography that have been approved for marketing in their region of use.
Stress Echo exams shall be anonymized before they are sent on to EchoGo Pro for processing. Removal of Protected Health Information (PHI) shall follow national legal requirements like Health Insurance Portability and Accountability Act (HIPAA) and General Data Protection Regulation (GDPR). PHI will be reinserted into the DICOM report object before being added to the original study in the clinical site’s PACS system.
The following tags shall be inserted into all DICOM objects of the study upon incorporation of the client exam. They will be used to relate the anonymized study to the original study at the clinical site if required for maintenance or if an error occurred during transmission. The Ultromics UIDs shall not encode or make use of any PHI.

• Private Creator Data Element (1965,0010) LO “ULTROMICS_506063469”
• Private Tag Ultromics Origin ID (1965,1010) LT: UltromicsOriginUID
• Private Tag Ultromics Study ID (1965,1012) LT: UltromicsStudyUID

UltromicsOriginUID: Each clinical site’s location shall be setup in to have unique identifier; for example, Medical Clinic Fort Lauderdale could be MEDCLIN-FL. The UID is displayed at the bottom right of the report.
UltromicsStudyUID: Each study shall be assigned a unique, sequential number.

Requirements on Data-out
The set of private DICOM tags listed above will be included within the DICOM report object by EchoGo Pro and shall be retained in the re-identification process of the study on its way back into the clinical site’s system. The Ultromics UIDs can be used to relate the study to audit logs stored by EchoGo Pro.
The IT department at the clinical site shall perform verification tests to ensure the DICOM report object gets routed back to its originating site and is stored with the correct original study.

EchoGo Pro Workflow

Technical QC
Received files shall be validated automatically by EchoGo Pro using the following procedure and will be skipped if the file is not a DICOM image or has more than one B-Mode image region. The entire batch of files will be rejected if:

• All files do not have the same StudyInstanceUID, StudyDate, StudyTime or PatientSex.
• There is not at least one file in this study for each required chamber view

If the batch of files is rejected, the operator is presented with a description of the errors encountered and must either correctly select from the pull-down list the reason for rejection or manually enter the error. The operator is prevented from moving away the from the study overview screen and must either cancel or reject the study if it fails the technical QC.

Left Ventricular (LV) Auto-Contouring, Clip Selection Screen, Quality Assessment and Contour Approval

Trained operators shall perform QC on all received datasets. This ensures that all information is present, the datasets have been obtained under the correct protocols, and all images capture the LV to a suitable degree of quality for assessment.
Incoming datasets shall only be processed if they are of sufficient quality. The platform requires quality to be assessed by the trained echocardiographer before any calculations are made.
If a batch of received files pass the technical QC, the algorithm for automated contour detection of the endocardium of the LV (ROI’s) is applied. The auto-contouring is detected at end-systole (ES) and end-diastole (ED) for the apical two chamber (A2C), four-chamber (A4C) and mid-ventricular SAX views at both rest and peak stages. Appendix A provides examples of contours that shall be rejected and accepted.
If there is more than one file in the study for any combination of patient views (A2C, A4C and SAX) then the operator will be presented with a Clip Selection Screen to review all contoured views before choosing the one they would like to proceed with in the workflow.
For each of the views, trained operators will perform quality checks on all images to ensure they are of sufficient quality prior to any calculation being performed. These quality checks include confirming:

• Views and stages have been correctly labelled
• The LV is clearly visualised and in centre of image
• Image quality is sufficient for LV tracing to be performed
• Images contain appropriate end-diastolic (ED) and end-systolic (ES) frame
• Auto-contouring provides valid traces of the LV with correct tracking between ED and ES frames
• There must be a minimum of 4 frames between ED and ES

Figure 1: Example of the Thumbnail screen displaying the different stages and views

Figure 2: Example of the contouring screen used to access the quality of the generated auto-contours

Any deviations detected by an operator shall be flagged on the system and reported. If auto-contours are of suboptimal quality for processing, they will be rejected, a rejection report will be generated and sent to the referring physician.

Reported Analysis
The generated reports present the results of the ROIs analysis as a categorical assessment of whether the SE show consistency with significant CAD, which was defined as showing evidence of ≥70% stenosis in the proximal to mid LAD, proximal left circumflex or proximal to mid RCA as measured by invasive angiography performed within 6 months of stress echocardiogram. The report will state one following outcomes:

• Stress echocardiogram ROIs are suggestive of a higher possibility of significant coronary artery disease
• Stress echocardiogram ROIs are suggestive of a lower possibility of significant coronary artery disease
• Report is not possible for the following reason ‘operator specified reason’

Section 11.4 provides further details on prespecified rejection reasons. The report is reviewed by the trained operator analysing the dataset to confirm that the result is displayed. When the report has been reviewed the operator can accept or reject the result.

Report Generation and Review Screen

A result is only generated upon completion of the following workflow:

• Retrieval of study images
• Quality assessment of auto-contoured LV tracing is completed and accepted
• The algorithm is applied and approved
• Results are reviewed and returned

A report preview is returned to the trained operator to confirm that the calculated measurements are displayed. The operator can then finalize the report by selecting “Send” or abort the report by selecting “Reject” or “Cancel”
The Report Screen displays the following buttons:

• “Send”
• “Cancel”
• “Reject”
• “Contour”

If the operator chooses “Cancel”, the result will be sent back as a DICOM image tagged with the Study
UID, and the application returns the operator to the Study Overview screen in the no studies loaded
status.
The operator can reject a study if quality requirements are not met in the received study. When the Operator rejects a study, the application will generate a rejection report which will be sent to the clinical site.

The Operator will select one of the following reasons for rejection:

• Unable to fully assess the LV border delineation
• Incorrect labelling of anatomical view(s)
• Required image(s) missing from dataset
• Corrupt data set
• Other

If the Operator chooses “Other” there will be a text box which the Operator can use to enter any comments – a comment must be entered.
A report (worksheet) is only generated upon completion of the workflow described above.
A report preview is returned to the trained Operator to confirm that the calculated likelihood information is displayed. The operator can then finalize the report by selecting “Send” or abort the report by selecting “Reject” or “Cancel”.
Once the report is finalized it is sent to the clinical site via the same DICOM gateway.

Figure 3: Example of the EchoGo Pro Reports for ROIs suggestive of a lower possibility of significant CAD, ROIs are suggestive of a higher possibility of CAD, a rejection report and the labelling page

Age is used by the operator to access whether the patient has reached at least 80% of the maximum target heart rate during peak stress. If this heart rate is not reached operators will reject the study. Patient age and sex are not used by the algorithms to determine the outcomes by EchoGo Pro v1.0.5.
Any deviations detected by an operator will be flagged on the system and the result. If images are of suboptimal quality for processing, they will be rejected, and the referring clinician notified of the result.

EchoGo Pro Validation and Performance Testing

EchoGo Pro v1.0.5 is an effective, AI driven tool to help physicians detect cardiac ischaemia in cases of suspected coronary artery disease. Performance evaluation of EchoGo Pro v1.0 showed that it is capable of disease stratification as a standalone device, demonstrated by and area under the receiver operating characteristic curve (AUROC) of 0.934 (Figure 4A), diagnostic specificity of 0.857 (95% CI 0.827, 0.889) and a sensitivity of 0.867 (95% CI 0.802, 0.943).
As an assistive device within the clinic, EchoGo Pro has been demonstrated to effectively improve clinician diagnostic performance and reduce inter- clinician variability. A reader study demonstrated that the assistance of EchoGo Pro during clinical reads of stress echocardiograms (SE) resulted in a 0.54-point increase in AUROC, translating to a 10% increase in sensitivity and 1.4% increase in specificity.

Figure 4: Performance of the CAD classifier on an independent US testing dataset and when utilized by four clinician operators in a reader study. A) ROC curves for the CAD classifier model on UK training (solid line) and US validation (dashed line) datasets. B) ROC curves for clinician performance during interpretation of SE studies from the validation with (solid line) and without (dashed line) the assistance of the CAD classifier. A washout period of 4-weeks of one month was implemented between reads.

Appendix A – Example of Contour Images

Contour Images: Accept

Contour Images: Reject

© Copyright Ultromics 2021
All Rights Reserved

Read User Manual Online (PDF format)

Read User Manual Online (PDF format)  >>

Download This Manual (PDF format)

Download this manual  >>