ABSTRACT

PURPOSE: To evaluate and compare the observers' ability to measure simulations of cup/disc ratios (CDR) as concentric and non-concentric circles. METHODS: In a prospective, random, and masked setting, 43 images representing the CDR spectrum from 0.2 to 0.9 for vertical and horizontal CDR measurements were developed and presented on a computer screen to 171 participants. RESULTS: There were satisfactory agreements according to the kappa coefficient (0.755 and 0.730 for horizontal and vertical cup disc ratios, respectively) and Lin's concordance correlation (R=0.88 and R=0.86 for horizontal and vertical measurements, respectively). However, very poor agreement was found for intermediate CDR values. The worst agreement occurred when the CDR was between 0.4 and 0.6 for both the horizontal and vertical values. The kappa coefficient was 0.37 and 0.39 for 0.4 CDR (horizontal and vertical, respectively), 0.39 and 0.38 for 0.5 CDR (horizontal and vertical, respectively) and 0.45 and 0.41 for 0.6 CDR (horizontal and vertical, respectively). CONCLUSION: Despite a good general agreement between the gold standard and the participants' responses, the absolute agreement for intermediate CDR values was very poor for both horizontal and vertical values.

Keywords: Optic disc; Glaucoma; Optic nerve diseases; Diagnostic techniques, ophthalmological; Internet; Measures

RESUMO

OBJETIVO: Avaliar e comparar a habilidade de observadores em medir a razão escavação/disco (CDR) por meio de figuras esquemáticas. MÉTODOS: Em um estudo prospectivo, randomizado e mascarado, 43 imagens representado CDR horizontais e verticais entre 0,2 e 0,9 foram desenvolvidas e apresentadas em uma tela de computador para 171 participantes. RESULTADOS: Para todos os intervalos de CDR a concordância foi satisfatória para análise kappa (0,755 e 0,730 para CDR horizontais e verticais, respectivamente) e para concordância de Lin (R=0,88 e R=0,86 para medidas horizontais e verticais respectivamente). No entanto, a concordância foi fraca para valores intermediários de CDR. A pior concordância ocorreu para CDR horizontais e verticais entre 0,4 e 0,6. CONCLUSÃO: Apesar da boa concordância geral entre as respostas corretas e as respostas dadas pelos participantes, a concordância absoluta para valores intermediários de CDR mostrou-se muito fraca tanto para figuras horizontais como verticais.

Descritores: Disco óptico; Glaucoma; Doenças do nervo óptico; Técnicas de diagnóstico oftalmológico; Internet; Medidas

ORIGINAL ARTICLES

 

Simulated cup/disc ratio: a tool for ophthalmologists

 

Razão escavação/disco simulada: uma ferramenta para oftalmologistas

 

 

Carolina Pelegrini Barbosa^I; Pedro Felipe Angelini^II; Paulo Schor^I; Augusto Paranhos Junior^I

^IPhysician , Ophthalmology Department, Escola Paulista de Medicina, Universidade Federal de São Paulo - UNIFESP - São Paulo (SP), Brazil
^IIEngineer, Escola Politécnica, Universidade de São Paulo - USP - São Paulo (SP), Brazil

Correspondence address

 

 

---

ABSTRACT

PURPOSE: To evaluate and compare the observers' ability to measure simulations of cup/disc ratios (CDR) as concentric and non-concentric circles.
METHODS: In a prospective, random, and masked setting, 43 images representing the CDR spectrum from 0.2 to 0.9 for vertical and horizontal CDR measurements were developed and presented on a computer screen to 171 participants.
RESULTS: There were satisfactory agreements according to the kappa coefficient (0.755 and 0.730 for horizontal and vertical cup disc ratios, respectively) and Lin's concordance correlation (R=0.88 and R=0.86 for horizontal and vertical measurements, respectively). However, very poor agreement was found for intermediate CDR values. The worst agreement occurred when the CDR was between 0.4 and 0.6 for both the horizontal and vertical values. The kappa coefficient was 0.37 and 0.39 for 0.4 CDR (horizontal and vertical, respectively), 0.39 and 0.38 for 0.5 CDR (horizontal and vertical, respectively) and 0.45 and 0.41 for 0.6 CDR (horizontal and vertical, respectively).
CONCLUSION: Despite a good general agreement between the gold standard and the participants' responses, the absolute agreement for intermediate CDR values was very poor for both horizontal and vertical values.

Keywords: Optic disc; Glaucoma/diagnosis; Optic nerve diseases/diagnosis; Diagnostic techniques, ophthalmological; Internet/utilization; Measures

---

RESUMO

OBJETIVO: Avaliar e comparar a habilidade de observadores em medir a razão escavação/disco (CDR) por meio de figuras esquemáticas.
MÉTODOS: Em um estudo prospectivo, randomizado e mascarado, 43 imagens representado CDR horizontais e verticais entre 0,2 e 0,9 foram desenvolvidas e apresentadas em uma tela de computador para 171 participantes.
RESULTADOS: Para todos os intervalos de CDR a concordância foi satisfatória para análise kappa (0,755 e 0,730 para CDR horizontais e verticais, respectivamente) e para concordância de Lin (R=0,88 e R=0,86 para medidas horizontais e verticais respectivamente). No entanto, a concordância foi fraca para valores intermediários de CDR. A pior concordância ocorreu para CDR horizontais e verticais entre 0,4 e 0,6.
CONCLUSÃO: Apesar da boa concordância geral entre as respostas corretas e as respostas dadas pelos participantes, a concordância absoluta para valores intermediários de CDR mostrou-se muito fraca tanto para figuras horizontais como verticais.

Descritores: Disco óptico; Glaucoma/diagnóstico; Doenças do nervo óptico/diagnóstico; Técnicas de diagnóstico oftalmológico; Internet/utilização; Medidas

---

 

 

INTRODUCTION

Glaucoma is a progressive optic neuropathy characterized by a specific pattern of optic nerve head and visual field damage^(1-2). There are several different ways to classify glaucoma optic nerve damage; one of the most commonly used in clinical and epidemiological studies is the cup/disc ratio (CDR)^(3-4). However, the CDR measurement is not the optimal method to classify glaucoma damage. Foster et al. reported problems with CDR measurement reproducibility and demonstrated that the nerve rim usually exhibits an asymmetric pattern of thinning, which often does not reflect the vertical or horizontal axis⁽²⁾. Some physicians only employ the CDR classification method to report the optic nerve condition and use it to follow the rate of progression; they do not acquire images or draw the optic nerve. Jampel and Quigley reported on the lack of agreement among glaucoma experts when they independently assessed the disc change over time in patients with established visual field loss⁽⁵⁾.

The agreement in CDR evaluation between observers may be very poor, since it is a measurement that may be difficult to teach residents and requires practice to improve its ability to evaluate the condition of the optic nerve. We developed a CDR self-evaluation Internet tool for residents and previously trained physicians. In this study, we evaluated and compared the observers' ability to use this tool to measure simulations of CDR as concentric and non-concentric circles.

 

METHODS

The study protocol was reviewed and approved by the Institutional Review Board of the Federal University of São Paulo. It was a cross-sectional, randomized, and masked study. The tests were performed on the internet. The study group was comprised of both genders. All participants were over 20 years old and had a referred visual acuity of 20/20 with or without correction of both eyes.

To gather the data over the internet, a custom made program was designed using the Web-Service paradigm. Participants could use their personal computers to participate in the study in the same way that they would access a forum website or e-mail. All participants answered 86 objective questions related to 43 figures. They could write numbers between 0.1 to 0.9 with just one decimal fraction. Each participant was presented with: concentric circumference figures (8 figures varying between 0.2 and 0.9); decentered circumferences (11 figures varying between 0.6 and 0.8 with a displacement of 0.05 for each figure); or concentric ellipses (24 figures combining each interval of CDR). The horizontal and vertical CDR of each figure was determined. The presentation of figures to participants was random, and a colored timer was included to prevent user fatigue. This timer was deep green from 0 to 20 seconds, yellow from 20 to 30 seconds, and red after 30 seconds, so the user would know how much time was remaining.

The agreement between the measurements for different CDR intervals was evaluated using the weighted kappa. Bland-Altman analysis was also used to assess the agreement between the observed and expected CDR. The mean of the differences between the two measurements (bias) and the 95% limits of agreement (95% LOA) were calculated. Lin's concordance correlation coefficient (Rc) was also used. This value is considered to be complementary to the 95% LOA and combines both precision and accuracy measurements to determine whether the observed data significantly deviates from the line of perfect concordance (i.e., line of identity)⁽⁶⁾. All statistical analysis were performed with SAS version 9.1.3 (SAS Institute, Cary, NC) and Minitab 13.3 (Minitab Inc., PA, USA).

 

RESULTS

This study included 171 participants (37% male and 63% female) who performed the test including: ophthalmologists (57%); general physicians (14%); medical students (9%); and undetermined as non-doctors healthy professional (20%). The average age was 43 ± 1.2 years varying between 20 and 60 years.

The kappa coefficient indicated a substantial overall agreement for the horizontal CDR (weighted kappa=0.755) and vertical CDR (weighted kappa=0.730). The Lin's concordance correlation coefficient also identified a satisfactory agreement for the horizontal CDR (Rc=0.8829) and vertical CDR (Rc=0.8613). However, Bland-Altman analysis (Figures 1 and 2 for horizontal and vertical measurements, respectively) showed a very poor agreement for intermediate CDR values. The worst agreement occurred when the CDR was between 0.4 and 0.6 for both horizontal and vertical values. The kappa coefficient was 0.37 and 0.39 for 0.4 CDR (horizontal and vertical, respectively), 0.39 and 0.38 for 0.5 CDR (horizontal and vertical, respectively) and 0.45 and 0.41 for 0.6 CDR (horizontal and vertical, respectively).

 

 

 

 

DISCUSSION

Cup/disc ratio measurement is commonly used to classify glaucomatous damage and monitor its progression. It is widely used in epidemiological studies^(7-11). Vitale et al. reported that the vertical CDR and Nerve Fiber Layer neural network number were the best combination to measure the neural damage resulting from glaucomatous disease because of their sensitivity and specificity⁽⁷⁾. The CDR classification is frequently employed in clinical trials. For example, the Collaborative Initial Glaucoma Treatment Study (CIGTS) used CDR classification and reported that small neuroretinal rim is a risk factor for glaucomatous optic nerve progression^(8-11). In addition, the Ocular Hypertensive Treatment Study (OHTS) also used CDR classification and showed that vertical and horizontal CDR values are important risk factors for glaucoma conversion^(9,12).

Jampel et al. proposed a statistical convention: a probability of <5% representing a significant deviation from the normal can be established. Based on this convention, the CDR above which 2.5% of the normal population lie defines the "upper limit of normal" (the other 2.5% falls below the normal distribution). Using the 97.5^th percentile enables one to avoid making the assumption that the CDR is normally distributed (it was found to be Gaussian in some studies but not in others). Jampel et al. suggested using the 97.5^th percentile value for CDR asymmetry as a second criterion for abnormality.

It would be very useful if the CDR measurement was in good agreement between observers. An important diagnostic parameter should be very reproducible; however, there is a poor agreement on subjective assessment of the optic disc even among experts⁽¹³⁾. Therefore, standardized criteria must be developed for assessing glaucomatous optic disc damage⁽¹⁴⁾. Spaeth proposed a DDLS (Disc Damage Likelihood Scale), a method that focuses on the optic disc size and the region of the optic nerve that is actually undergoing damage (neuroretinal rim) using a scale of the narrowest rim disc ratio that simultaneously accounts for the overall size of the optic nerve⁽¹⁵⁾.

The CDR classification method has significant problems, especially its poor reproducibility^(16-19). For example, some patients have small CDR ratios but significant visual field loss, whereas some have large CDR ratios with little visual field loss⁽²⁾. Finally, while the CDR is of some value in patients with concentric cupping⁽¹⁵⁾, it may be seriously misleading when the loss of the rim is limited to a single sector, as with a focal notch. In this latter situation, the CDR ratio may be recorded as small, but the disc and visual field may be badly damaged. Another problem with CDR classification is wide variability in optic disc morphology between individuals. Jonas et al. confirmed that there are healthy eyes with rather small optic discs and healthy eyes with very large optic discs, as concluded from their study on a South Indian population⁽²⁰⁾. Therefore, multiple studies have reported on different techniques to classify and measure the CDR. For example, Armaly proposed a useful classification system to describe the optic nerve, and this system exhibited a good correlation with visual field damage^(21-22). However, this classification system was limited by issues of focal rim narrowing and physiological variability based on the disc size^(16,23).

Our study showed a lack of agreement among intermediate CDR values; therefore, the CDR measurement appears more useful for extreme values. However, extreme values are not as important in clinical practice. Our results are very important as they indicate a significant lack of agreement between individuals examining virtual optic nerve samples. It is expected that there would be an even worse agreement with real optic nerve samples.

Some studies have shown that the variability on measure the CDR is reduced with experience^(24-25) and some teaching programs were developed trying to improve this tool⁽²⁶⁾. The group of this study has a certain variety because not only ophthalmologists performed the test but also general physicians and medical students. So this sample could partially justify the results of this study and a teaching tool could help our observers, but it was not the aim of this study.

The primary reason for performing this study is the wide variability between optic discs within the general population, in terms of form and size^(27-28). In addition, agreement between medical practitioners in measuring the CDR using stereoscopic images is not optimal⁽²⁵⁾, as confirmed by our study.

 

CONCLUSIONS

The results from this study show that the CDR simulation test indicated poor agreement between observers and standard CDR values, especially for intermediate values. This test demonstrates that analysis of CDR is not intuitive and that training and reference images may be necessary to improve use in clinical applications.

 

ACKNOWLEDGEMENT

Supported by Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) - Brazil.

 

REFERENCES

1. Lin SC, Singh K, Jampel HD, Hodapp EA, Smith SD, Francis BA, Dueker DK, Fechtner RD, Samples JS, Schuman JS, Minckler DS; American Academy of Ophthalmology; Ophthalmic Technology Assessment Committee Glaucoma Panel. Optic nerve head and retinal nerve fiber layer analysis: a report by the American Academy of Ophthalmology. Ophthalmology. 2007;114(10):1937-49. Erratum in: Ophthalmology. 2008;115(3):472. Comment in: Ophthalmology. 2008;115(7):1266-7; author reply 1267-8.         

2. Foster PJ, Buhrmann R, Quigley HA, Johnson GJ. The definition and classification of glaucoma in prevalence surveys. Br J Ophthalmol. 2002;86(2):238-42. Review.         

3. Sánchez Pérez A, Honrubia López FM, Larrosa Poves JM, Polo Llorens V, Melcon Sánchez-Frieras B. [The Autocad system for planimetric study of the optic disc in glaucoma: technique and reproducibility study]. Arch Soc Esp Oftalmol. 2001;76(9): 551-8. Spanish.         

4. Caprioli J. Recognizing structural damage to the optic nerve head and nerve fiber layer in glaucoma. Am J Ophthalmol. 1997;124(4):516-20. Review.         

5. Jampel HD, Friedman D, Quigley H, Vitale S, Miller R, Knezevich F, Ding Y. Agreement among glaucoma specialists in assessing progressive disc changes from photographs in open-angle glaucoma patients. Am J Ophthalmol. 2009; 147(1):39-44 e1.         

6. Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics. 1989;45(1):255-68.         

7. Vitale S, Smith TD, Quigley T, Kerrigan-Baumrind TA, Pease TE, Varma R, et al. Screening performance of functional and structural measurements of neural damage in open-angle glaucoma: a case-control study from the Baltimore Eye Survey. J Glaucoma. 2000;9(5):346-56.         

8. Comparison of glaucomatous progression between untreated patients with normal-tension glaucoma and patients with therapeutically reduced intraocular pressures. Collaborative Normal-Tension Glaucoma Study Group. Am J Ophthalmol. 1998; 126(4):487-97. Erratum in: Am J Ophthalmol 1999;127(1):120.         

9. Gordon MO, Beiser JA, Brandt JD, Heuer DK, Higginbotham EJ, Johnson CA, et al. The Ocular Hypertension Treatment Study: baseline factors that predict the onset of primary open-angle glaucoma. Arch Ophthalmol. 2002;120(6): 714-20; discussion 829-30.         

10. Jonas JB, Martus P, Budde WM, Jünemann A, Hayler J. Small neuroretinal rim and large parapapillary atrophy as predictive factors for progression of glaucomatous optic neuropathy. Ophthalmology. 2002;109(8):1561-7.         

11. Jonas JB, Martus P, Budde WM, Hayler J. Morphologic predictive factors for development of optic disc hemorrhages in glaucoma. Invest Ophthalmol Vis Sci. 2002;43(9):2956-61.         

12. Johnson CA, Cioffi GA, Liebmann JR, Sample PA, Zangwill LM, Weinreb RN. The relationship between structural and functional alterations in glaucoma: a review. Semin Ophthalmol. 2000;15(4):221-33.         

13. Lichter PR. Variability of expert observers in evaluating the optic disc. Trans Am Ophthalmol Soc. 1976;74:532-72.         

14. Abrams LS, Scott IU, Spaeth GL, Quigley HA, Varma R. Agreement among optometrists, ophthalmologists, and residents in evaluating the optic disc for glaucoma. Ophthalmology. 1994;101(10):1662-7.         

15. Spaeth GL, Henderer J, Liu C, Kesen M, Altangerel U, Bayer A, et al. The disc damage likelihood scale: reproducibility of a new method of estimating the amount of optic nerve damage caused by glaucoma. Trans Am Ophthalmol Soc. 2002;100:181-5; discussion 185-6.         

16. Douglas GR, Drance SM, Schulzer M. A correlation of fields and discs in open angle glaucoma. Can J Ophthalmol. 1974;9(4):391-8.         

17. Guthauser U, Flammer J, Niesel P. The relationship between the visual field and the optic nerve head in glaucomas. Graefes Arch Clin Exp Ophthalmol. 1987; 225(2):129-32.         

18. Miglior S, Brigatti L, Lonati C, Rossetti L, Pierrottet C, Orzalesi N. Correlation between the progression of optic disc and visual field changes in glaucoma. Curr Eye Res. 1996;15(2):145-9.         

19. Read RM, Spaeth GL. The practical clinical appraisal of the optic disc in glaucoma: the natural history of cup progression and some specific disc-field correlations. Trans Am Acad Ophthalmol Otolaryngol. 1974;78(2):OP255-74.         

20. Jonas JB, Thomas R, George R, Berenshtein E, Muliyil J. Optic disc morphology in south India: the Vellore Eye Study. Br J Ophthalmol. 2003;87(2):189-96.         

21. Armaly MF. Genetic determination of cup/disc ratio of the optic nerve. Arch Ophthalmol. 1967;78(1):35-43.         

22. Armaly MF, Sayegh RE. The cup-disc ratio. The findings of tonometry and tonography in the normal eye. Arch Ophthalmol. 1969;82(2):191-6.         

23. Hoskins HD Jr, Gelber EC. Optic disk topography and visual field defects in patients with increased intraocular pressure. Am J Ophthalmol. 1975;80(2):284-90.         

24. Garway-Heath DF. Determinants of optic disc characteristics. Ophthalmology. 2000; 107(7):1217-9. Comment on: Ophthalmology. 1999;106(8):1588-96.         

25. Varma R, Steinmann WC, Scott IU. Expert agreement in evaluating the optic disc for glaucoma. Ophthalmology. 1992;99(2):215-21.         

26. Sheen NJ, Morgan JE, Poulsen JL, North RV. Digital stereoscopic analysis of the optic disc: evaluation of a teaching program. Ophthalmology. 2004;111(10):1873-9.         

27. Iester M, Jonas JB, Mardin CY, Budde WM. Discriminant analysis models for early detection of glaucomatous optic disc changes. Br J Ophthalmol. 2000; 84(5):464-8.         

28. Jonas JB, Gusek GC, Naumann GO. Optic disc morphometry in chronic primary open-angle glaucoma. I. Morphometric intrapapillary characteristics. Graefes Arch Clin Exp Ophthalmol. 1988;226(6):522-30.         

 

 

Correspondence address:
Carolina Pelegrini Barbosa
Rua Botucatu, 820
CEP 04023-062 - São Paulo (SP)
E-mail: [email protected]

Recebido para publicação em 02.04.2010
Última versão recebida em 26.06.2010
Aprovação em 02.07.2010

 

 

Study carried out at Ophthalmology Department, Escola Paulista de Medicina, Universidade Federal de São Paulo - UNIFESP - São Paulo (SP), Brazil.
The results presented in this paper have not been previously published either in whole or part, except in abstract form.
Nota Editorial: Depois de concluída a análise do artigo sob sigilo editorial e com a anuência do Dr. Jayter Silva de Paula sobre a divulgação de seu nome como revisor, agradecemos sua participação neste processo.