ABSTRACT

Purpose: To compare the quality of retinal images captured with a smartphone-based, handheld fundus camera with that of retinal images captured with a commercial fundus camera and to analyze their agreement in determining the cup-to-disc ratio for a cohort of ophthalmological patients.
Methods: A total of 50 patients from a secondary ophthalmic outpatient service center underwent a bilateral fundus examination under mydriasis with a smartphone-based, handheld fundus camera and with a commercial fundus camera (4 images/patient by each). Two experienced ophthalmologists evaluated all the fundus images and graded them on the Likert 1-5 scale for quality. Multivariate regression analyses was then performed to evaluate the factors associated with the image quality. Two masked ophthalmologists determined the vertical cup-to-disc ratio of each fundus image, and both the intraobserver (between devices) and interobserver agreement between them was calculated.
Results: Ninety-eight images from 49 patients were processed in this study for their quality analysis. Ten images from five patients (four from commercial fundus camera and one from smartphone-based, handheld fundus camera) were not included in the analyses due to their extremely poor quality. The medians [interquartile interval] of the image quality were not significantly different between those from the smartphone-based, handheld fundus camera and from the commercial fundus camera (4 [4-5] versus 4 [3-4] respectively, p=0.06); however, both the images captured with the commercial fundus camera and the presence of media opacity presented a significant negative correlation with the image quality. Both the intraobserver [intraclass correlation coefficient (ICC)=0.82, p<0.001 and 0.83, p<0.001, for examiners 1 and 2, respectively] and interobserver (ICC=0.70, p=0.001 and 0.81; p<0.001, for smartphone-based handheld fundus camera and commercial fundus camera, respectively) agreements were excellent and statistically significant.
Conclusions: Our results thus indicate that the smartphone-based, handheld fundus camera yields an image quality similar to that from a commercial fundus camera, with significant agreement in the cup-to-disc ratios between them. In addition to the good outcomes recorded, the smartphone-based, handheld fundus camera offers the advantages of portability and low-cost to serve as an alternative for fundus documentation for future telemedicine approaches in medical interventions.

Keywords: Photography/instrumentation; Smartphone; Optic nerve; Telemedicine

RESUMO

Objetivo: Comparar a qualidade das imagens da retina capturadas com um retinógrafo portátil acoplado a um smartphone com aquelas adquiridas com um retinógrafo comercial padrão e analisar a concordância na determinação da relação escavação/ cabeça do nervo óptico em um coorte de pacientes de um serviço oftalmológico.
Métodos: Cinquenta pacientes de um serviço oftalmológico secundário foram submetidos a uma avaliação do fundo de olho bilateral, sob midríase, utilizando o retinógrafo portátil acoplado a um smartphone e o retinógrafo comercial padrão (4 imagens por paciente). Dois oftalmologistas experientes avaliaram a qualidade de todas as imagens e atribuíram a elas uma pontuação entre 1 e 5, de acordo com a escala Likert. Os fatores relacionados a qualidade das imagens foram avaliados utilizando uma análise de regressão multivariada. Dois oftalmologistas determinaram de forma mascarada a relação da escavação/ cabeça do nervo óptico de cada imagem e a concordância intra e interobservador foi calculada.
Resultados: Noventa e oito imagens de 49 pacientes foram utilizadas neste estudo para análise de qualidade. Dez imagens de cinco pacientes (quatro do retinógrafo comercial padrão e um do retinógrafo portátil acoplado a um smartphone) foram excluídas das análises de concordância devido à baixa qualidade das mesmas, mas foram considerados nas análises de qualidade. Dos cinco pacientes com imagens excluídas, quatro foram capturadas pelo retinógrafo comercial padrão e uma pelo retinógrafo portátil acoplado a um smartphone. As medianas (intervalo interquartil) da qualidade das imagens não apresentaram diferença estatística entre o retinógrafo portátil acoplado a um smartphone e o retinógrafo comercial padrão (4 [4-5] versus 4 [3-4] respectivamente, p=0.06). As imagens obtidas com o retinógrafo comercial padrão e o diagnóstico de opacidade de meios apresentou uma correlação negativa com a qualidade da imagem. As concordâncias intraobservador (ICC =0,82, p<0,001 e 0,83, p<0,001, para o examinador 1 e 2, respectivamente) e interobservador (ICC = 0,70, p=0,001 e 0,81, p<0.001, para o retinógrafo portátil acoplado a um smartphone e retinógrafo comercial padrão, respectivamente) foram excelentes e estatisticamente significativas.
Conclusões: Nossos resultados sugerem que o retinógrafo portátil acoplado a um smartphone apresenta uma qualidade de imagem semelhante ao retinógrafo comercial padrão, com concordância significativa na análise da relação escavação-cabeça do nervo óptico. Além dos bons resultados apresentados, o retinógrafo portátil acoplado a um smartphone pode ser considerado uma alternativa portátil de baixo custo para documentação de retina em cenários futuros de telemedicina.

Descritores: Fotografia/instrumentação; Smartphone; Nervo óptico; Telemedicina

INTRODUCTION

The impact of senile chronic diseases in Brazil is becoming increasingly important considering the current aging pattern of the Brazilian population⁽¹⁾. Ocular diseases such as glaucoma, macular degeneration, and diabetic retinopathy, besides cataract, are the leading causes of blindness in individuals aged >50 years⁽²⁾. The increasing prevalence of these diseases reinforces the need for a diagnosis based on fundus examination in national health programs.

Unfortunately, eye health programs are often not well integrated with the health system⁽³⁾. Barriers to access healthcare are derived from the limitations associated with technologies, providers, geographical distances, as well as other cultural, cognitive, and behavioral differences among the health service users⁽⁴⁾.

Previous studies in India⁽⁵⁾ and Kenya⁽⁶⁾ demonstrated that handheld devices can be used for fundus documentation by non-ophthalmologists in areas lacking assistance for the detection of retinal diseases. Nevertheless, non-ophthalmologist healthcare workers could capture high-quality images in children screened for retinopathy of prematurity with a portable fundus camera. Their photographs were uploaded and remotely graded for retinopathy of prematurity by a retina specialist with good sensitivity and specificity levels⁽⁷⁾. All recent advances in mobile devices that may facilitate telemedicine strategies are believed to improve the integration of eye healthcare system in countries with low resources. With this background, the purpose of the present study was to validate a new smartphone, handheld fundus camera (SHFC) by evaluating both the image quality and its agreement with those from a commercial fundus camera (CFC) in determining the cup-to-disc ratio in a cohort of patients.

METHODS

Ethics approval

This cross-sectional study was approved by the Research Ethics Committee of the PIO XII Foundation, Barretos, SP and by the Research Ethics Committee of Hospital das Clínicas da Faculdade de Medicina de Ribeirão Preto, SP. The norms of the Declaration of Helsinki and the International Conference on Standardization Note for Guidance on Good Clinical Practice (ICH, Topic E6, 1995) were followed. All subjects received extensive and detailed oral and written explanations of all project-related events, and they provided with their signed informed consent form.

Participants

A total of 50 patients who were previously scheduled for eye examination by independent ophthalmologists from the ophthalmologic service of the AME, Barretos (São Paulo, Brazil) were included in this study.

The following were the inclusion criteria for the study subjects: age ≥18 years, no cognitive disability, the ability to undertake all necessary examinations, and no previous eye surgery performed in the last 2 months.

Device development

The SHFC was developed using an optical system that could generate high-resolution images within the 45 degree of the fundus view. It was attached to a smartphone (Samsung Galaxy S7; 12-Megapixel camera resolution, 2.6-Megapixel image resolution; Samsung Electronics Co., Suwon, Korea) and its processor, display, global positioning system, and internet access for handling patient, examinations, and data reports were used. The safety of light exposure was previously compared with reference to the energy levels necessary for performing colored fundus imaging between SHFC and other CFC (Topcon retinal camera, 16.2-Megapixel camera resolution, 1.45-Megapixel image resolution; Topcon Healthcare Solutions, Oakland, USA). The light measurements were made in two moments: a preview mode, with continuous homogeneous illumination and a capturing mode, with an instantaneous flash power. All measurements were performed using a handheld power/energy meter (Vega, Ophir Photonics, Newport Co., Jerusalem, Israel) coupled with a thermopile-based laser energy sensor (Model 3A Ophir Photonics; Newport Co., Jerusalem, Israel). The distance between the camera optical system and the sensor was determined based on the optical alignment to ensure that all light from the devices would reach the sensor-sensitive area to enable detection of the highest optical power value. Ten measurements each were taken in both the preview and the capturing modes.

Procedures

All patients scheduled for fundus imaging received at least one prior comprehensive ophthalmological evaluation, performed by the attending ophthalmologists from AME-Barretos, São Paulo, Brazil. On the scheduled day, the patients received one drop of 1% tropicamide and 10% phenylephrine for pupil dilation. After 25 min, a series of 3 photographs, followed by a series of 4 photographs of each eye, were captured by 2 trained nurses in a bright room by using the CFC and the SHFC, separately. The examiners had previously undergone three separate 1-h trainings. A masked examiner selected the best fundus picture of each eye from all patients, captured with both the devices.

The photographs of the anterior segment of both the eyes were also captured, which served as the measurement of the horizontal pupil diameter. The summarized demographic data (such as age and gender) and the individual spherical equivalent and diagnosis (such as glaucoma and suspects, retinal diseases, refraction errors, and cataract) were recorded on the same day and then subsequently analyzed. These data were collected from a summary review of the patients’ medical record, as the comprehensive ophthalmological evaluation was not directly performed by the researchers.

Two experienced, masked ophthalmologists evaluated all the fundus images on the same 19-inch LCD computer monitor. Each examiner graded an individual image based on the quality score with reference to the Likert 1-5 scale: 1 = Poor, unsatisfactory, or impossible to capture; 2 = Regular or partially satisfactory; 3 = Good or satisfactory; 4 = Very good or quite satisfactory; and 5 = Excellent or totally satisfactory.

The examiners also evaluated the randomly assorted individual images and attributed the values to the vertical cup/disc ratio by using a double-masked database of images (for both patient identification and the device used).

Statistical analysis

Variables were described using the mean, median, standard error, 95% confidence interval (95% IC), and frequencies, as necessary. Images with insufficient quality (score <2) were excluded from the agreement analysis, but were included for quality comparisons (using the nonparametric Friedman’s 2-way analysis of variance by ranks). A linear mixed-effects multivariate regression was also performed to identify the factors associated with the image quality, as follows: device type, ocular diagnosis, pupil diameter (after mydriasis), spherical equivalent, and age.

In addition, an intraclass correlation coefficient (ICC) was used to assess interobserver agreement (for each device) as well as the agreement between devices (for each observer) in combination with the Bland-Altmann plot analysis, displaying the mean difference ± limits of agreement (±1.96 × standard deviation) of the vertical cup-to-disc ratio between the two devices. All analyses were performed using the Stata software (Stata 14.2; StataCorp LLC, Texas, USA). Statistical significance was set at p<0.05.

RESULTS

The results of ocular safety when exposed to the SFHC are depicted in table 1. The mean optical power (0.14 ± 0.02 mW) and the mean radiant flash energy (0.29 ± 0.02 mJ) of the SHFC were found to be significantly smaller than those of CFC (0.50 ± 0.03 mW and 6.40 ± 0.05 mJ, respectively; p<0.001). Based on its presented lighting levels, the SHFC device was classified in Group 1 (safe) according to both the ISO 10940 and 15004-2 standards. Accordingly, 98 eyes of 49 patients (33 women [67.3%], mean age: 62.1 ± 10.2 years) were included in this study. Table 2 displays the summarized demographic data and diagnoses.

One patient was excluded from all data analyses due to the loss of his images that were captured with the CFC. Another 5 patients were later excluded from the agreement analyses (for the cup-to-disc ratios; n=44 patients, 88 eyes) due to the poor quality of the acquired images (grade 1 or 2), but they were considered in the quality analyses. Of these five unclassifiable patient images, four were acquired with the CFC device and one with the SHFC device.

The main difficulties reported during the SHFC examinations included the following: manual centralization of the preview image, patient collaboration, pupil size, eye alignment, and screen handling. These reports, however, were not objectively analyzed.

Figure 1 illustrates examples of retinal images captured by the same individual using the two different devices. The median [interquartile interval] score for image quality was 4 [4-5] for the SHFC, and it did not differ from the corresponding values for the CFC (4 [3-4]; p=0.06). Thus, we observed that the use of the CFC device and the diagnosis of “cataract” had a significant negative correlation with the image quality (Table 3).

Figure 1. Examples of fundus images captured from the same patient using the two different devices. On top, an image of participant #27 acquired with the CFC. On the bottom, an image from the same participant acquired with the SHFC.

The interobserver agreement for the evaluation of the cup-to-disc ratio was considered “good” with the SHFC images (ICC=0.70; p=0.001) and “very good” with the CFC images (ICC=0.81; p<0.001). The agreement coefficients between the devices were “very good” (examiner 1: ICC=0.82; p<0.001, examiner 2: ICC=0.83; p<0.001) (Table 4). The Bland-Altmann plot analyses displayed a good agreement between both the devices for both the examiners. However, data for examiner 2 showed a higher mean difference in the vertical cup-to-disc ratio (mean difference ± limit of agreement: 0.02 ± 0.29 and 0.07 ± 0.25 for examiners 1 and 2, respectively), and four images had differences of >0.3 (Figure 2).

Figure 2. Bland-Altmann plots depicting the agreement analyses for the assessment of the vertical cup-to-disc ratios between the two devices for examiner 1 (left panel) and examiner 2 (right panel). Dashed lines represent the mean difference in the values between the devices (CFC-SHFC) and the continuous lines represent the limits of agreement calculated with ±1.96 × Standard Deviation (SD).

DISCUSSION

Recent reports estimate that at least 2.2 billion people will present visual impairment, with age-related macular degeneration, glaucoma, and diabetic retinopathy being the most common causes, besides cataract and refractive error^(8-12). Latin America and the Caribbean countries could have saved $ 6,281 million by 2020 if the blindness prevention programs had been implemented in the past 13 years⁽¹³⁾. These data reinforce the need for an integrated health system that is capable of early detection of these conditions and the prevention of blindness at lower costs. Portable equipment compatible with telemedicine can facilitate low-cost integration of the health system against the burden of ocular diseases⁽¹⁴⁾.

Among a few others, the prototype used in the present study may be considered as a safe alternative for developing a telemedicine-based system for the detection of ocular diseases integrated with the health system. This system can be considered safe owing to its at least 3-times lower optical power than that of the CFC in a preview mode and 22-times lower power in a capturing mode, in addition to its affordability (estimated to be US$ 5,000, which is 6-times lower than that for most other available table retinal cameras).

Regarding the quality of images captured, a significant superiority was noticed in the images captured by the SHFC in comparison to those by CFC, since the multivariate analysis indicated a significant correlation with the device type (SHFC), besides the presence of cataract. The other parameters analyzed (including age, pupil diameter, retinal diseases, and refractive errors) presented no significant association with the image quality (Table 3). A significant negative association was noted between cataract and image quality (p=0.04), however, this point needs to be considered along with the fact that only one patient with this diagnosis was included in the study, which may potentially affect further conclusion.

The clinical validation of the SHFC was verified through the determination of the cup-to-disc ratios in the fundus images in comparison to the CFC across two masked examiners. First, our results showed good significant agreement between the observers for both the devices, with the coefficient being higher for the CFC device (ICC=0.81; p<0.001) than for the SHFC device (ICC=0.70; p=0.001). It is believed that the interobserver agreement in the clinical evaluation of the optic nerve varies, but it can be higher if the analysis is based on retinal images of the fundus^(14,15).

Second, the Bland-Altman plots showed good levels of agreement. Moreover, examiner 2 may have performed worse than examiner 1 considering the four images with the vertical cup-to-disc ratio differences between the devices being >0.3 in his evaluation. One of the examiners is a glaucoma specialist, while the other is a general practice ophthalmologist, both with several years of clinical experience. We thus speculated that the difference in the background of the two examiners indirectly accounted for the higher variability presented as well as for the potential differences in the generation of some images by the different devices. Thus, good ICC values recorded herein accounts for the higher credibility of both the examiners, although examiner 2 demonstrated higher variability during his evaluation.

The intraobserver agreement in determining the cup-to-disc ratio was also good and significant for each examiner (ICC = 0.82; p<0.001 versus 0.83; p<0.001). These consistent results of comparable performance between the devices in producing reliable fundus image (at least for the good determination of the optic nerve excavation boundaries) account for the clinical validation of SHFC, despite the technical differences between them.

Previous studies have evaluated both the interobserver and the intraobserver agreements in cup-to-disc ratios measured in the fundus images captured by different devices. The present results are similar to those described for both the interobserver (0.67-0.9) and intraobserver (0.79-0.92) agreement levels in some past studies^(14,16,17). However, only Shuttleworth et al.⁽¹⁷⁾ presented their coefficient results as ICC, with no comparison between two different devices. Waisbourd et al.⁽¹⁸⁾ reported lower ICC levels of agreement (ICC = 0.71 for the intraobserver and ICC = 0.69 for the interobserver agreement) with the use of a handheld fundus camera, but they did not compare the data with those of other routinely used devices. On the other hand, Miller et al.⁽¹⁹⁾ compared the performance of a non-mydriatic handheld fundus camera to that of a conventional tabletop mydriatic camera and observed slightly lower k values for both the intraobserver (0.64) and interobserver (0.54) agreements. Thus, our good results validate and indicate the potential clinical applications of the SHFC.

The present study, however, has some limitations. First, we included no healthy control group. Our study protocol was applied for the evaluation of all cup-to-disc ratios, because it was not designed for glaucoma diagnosis. In addition, both patients with glaucoma and suspects were included in the regression analyses. Second, despite patients and examiners reporting great comfort with the SHFC device at the very first tests, the factors of comfort and ease of handling were not objectively analyzed. Finally, a new non-mydriatic version of the SHFC (Figure 3) is now commercially available, but it was not tested in the present study.

Figure 3. Photographs of the prototype used in the study (left panel: SHFC) and that of its latest commercial version (right panel: Eyer).

Several portable retinal imaging devices have emerged in the past few years as alternatives for better-integrated eye health care. We believe that an ideal portable device for fundus image should be light-weighted, easy to handle, of low-cost, non-mydriatic, and equipped with the facility of data transfer. Several available portable devices meet some of these features, and the new version of the prototype presented in this study represents an option that fulfills all of them. Further research is warranted to determine the proposed device’s capability to generate good-quality images without mydriasis and to validate its sensitivity and specificity levels for the diagnosis of the most prevalent ocular diseases, as a possible new alternative to the telemedicinal approach for the future.

ACKNOWLEDGMENT

We would kindly like to acknowledge Bruna Melchior Silva, for performing part of the statistical analysis, as well as the nurses Marilene Oliveira Simeão and Denisia de Amorim Morais for their technical help. We also acknowledge the CAPES - Brazil (process 1798846) and FAPESP (PIPE 2016/00985-5) for the research grants.

REFERENCES

1. Bourne RR, Flaxman SR, Braithwaite T, Cicinelli MV, Das A, Jonas JB, et al.; Vision Loss Expert Group. Magnitude, temporal trends, and projections of the global prevalence of blindness and distance and near vision impairment: a systematic review and meta-analysis. Lancet Glob Health. 2017;5(9):e888-97.

2. Schellini SA, Meneghim RLFS, Satto L, Cavinatto P, Galindo-Ferreiro A, Khandekar R. Magnitude of visual impairment, blindness and causes in the southwest region of São Paulo state, Brazil. Arq Bras Oftalmol. 2018;81(5):414-20.

3. Gordois A, Pezzulio L, Cutler H. The global economic cost of visual impairment. Access Economics. International Council of Ophthalmology. 2010. [cited 2020 Sep 7]. Available from: http://www.icoph.org/resources/146/The-Global-Economic-Cost-of-Visual-Impairment.html

4. O’Donnell O. Access to health care in developing countries: breaking down demand side barriers. Cad Saude Publica. 2007;23(12):2820-34.

5. Sengupta S, Sindal MD, Besirli CG, Upadhyaya S, Venkatesh R, Niziol LM, et al. Screening for vision-threatening diabetic retinopathy in South India: comparing portable non-mydriatic and standard fundus cameras and clinical exam. Eye (Lond). 2018;32(2):375-83.

6. Bastawrous A, Giardini ME, Bolster NM, Peto T, Shah N, Livingstone IA, et al. Clinical validation of a smartphone-based adapter for optic disc imaging in Kenya. JAMA Ophthalmol. 2016;134(2):151-8.

7. Prakalapakorn SG, Freedman SF, Hutchinson AK, Wallace DK, Stinnett SS, Riggins JW, et al. Evaluating a portable, noncontact fundus camera for retinopathy of prematurity screening by Nonophthalmologist Health Care Workers. Ophthalmol Retina. 2018;2(8):864-71.

8. World Health Organization (WHO). World report on vision. Geneva: WHO; 2019. [cited 2020 Sep 7]. Available from: https://www.who.int/publications/i/item/world-report-on-vision

9. Wong WL, Su X, Li X, Cheung CM, Klein R, Cheng CY, et al. Global prevalence of age-related macular degeneration and disease burden projection for 2020 and 2040: a systematic review and meta-analysis. Lancet Glob Health. 2014;2(2):e106-16.

10. Flaxman SR, Bourne RR, Resnikoff S, Ackland P, Braithwaite T, Cicinelli MV, et al.; Vision Loss Expert Group of the Global Burden of Disease Study. Global causes of blindness and distance vision impairment 1990-2020: a systematic review and meta-analysis. Lancet Glob Health. 2017;5(12):e1221-34.

11. World Health Organization (WHO). Global report on diabetes. Geneva: WHO; 2016. [cited 2020 Sep 7]. Available from: http://apps.who.int/iris/bitstream/handle/10665/204871/9789241565257_eng.pdf

12. Yau JW, Rogers SL, Kawasaki R, Lamoureux EL, Kowalski JW, Bek T, et al.; Meta-Analysis for Eye Disease (META-EYE) Study Group. Global prevalence and major risk factors of diabetic retinopathy. Diabetes Care. 2012;35(3):556-64.

13. Ottaiano JA, Ávila MP, Umbelino CC, Taleb AC. As Condições de Saúde Ocular no Brasil 2019. São Paulo: Conselho Brasileiro de Oftalmologia; 2019.

14. Tielsch JM, Katz J, Quigley HA, Miller NR, Sommer A. Intraobserver and interobserver agreement in measurement of optic disc characteristics. Ophthalmology. 1988;95(3):350-6.

15. Kumar S, Giubilato A, Morgan W, Jitskaia L, Barry C, Bulsara M, et al. Glaucoma screening: analysis of conventional and telemedicine-friendly devices. Clin Exp Ophthalmol. 2007;35(3):237-43.

16. Varma R, Steinmann WC, Scott IU. Expert agreement in evaluating the optic disc for glaucoma. Ophthalmology. 1992;99(2):215-21.

17. Shuttleworth GN, Khong CH, Diamond JP. A new digital optic disc stereo camera: intraobserver and interobserver repeatability of optic disc measurements. Br J Ophthalmol. 2000;84(4):403-7.

18. Waisbourd M, Bond EA, Sullivan T, Hu WD, Shah SB, Molineaux J, et al. Evaluation of nonmydriatic hand-held optic disc photography grading in the Philadelphia Glaucoma Detection and Treatment Project. J Glaucoma. 2016;25(5):e520-5.

19. Miller SE, Thapa S, Robin AL, Niziol LM, Ramulu PY, Woodward MA, et al. Glaucoma screening in Nepal: cup-to-disc estimate with standard mydriatic fundus camera compared to portable nonmydriatic camera. Am J Ophthalmol. 2017;182:99-106.

Submitted for publication: April 29, 2020.
Accepted for publication: August 10, 2020.

Approved by the following research ethics committee: Pio XII Institution, Hospital do Câncer de Barretos, SP (CAAE: 79711617.5.0000.5437).

Disclosure of potential conflicts of interest: JSP was a technical consultant for Phelcom Inc., São Carlos - Brazil in 2017. DL, FPV, and JAS are the owners and founders of the Phelcom Inc., São Carlos - Brazil, and have direct commercial interests in the production of this prototype.