Home | About us | Editorial board | Search | Ahead of print | Current issue | Archives | Submit article | Instructions| Reviewers

  Home Print this page Email this page Small font sizeDefault font sizeIncrease font size Users Online: 653    

Table of Contents   
Year : 2020  |  Volume : 10  |  Issue : 5  |  Page : 659-665
Comparison of digital and paper assessment of smile aesthetics perception

Department of Orthodontics, Faculty of Dentistry, King Abdulaziz University, Jeddah, Saudi Arabia

Date of Submission13-Jul-2020
Date of Decision27-Jul-2020
Date of Acceptance31-Jul-2020
Date of Web Publication08-Aug-2020

Correspondence Address:
Dr. Shoroog Hassan Agou
Department of Orthodontics, Faculty of Dentistry, King Abdul Aziz University, Al Ehtifalat St, Jeddah.
Saudi Arabia
Login to access the Email id

Source of Support: None, Conflict of Interest: None

DOI: 10.4103/jispcd.JISPCD_323_20

Rights and Permissions


Objectives: Despite the widespread of assessment of smile aesthetic perception in many areas, there has yet to be a direct comparison of digital and paper-based photographs for the assessment of smile aesthetics. Here we compared digital and paper-based photographs representing different smile aesthetic features using visual analog scale (VAS) scoring. Materials and Methods: One hundred students were randomly recruited from a university campus. Participants were asked to record their perception of smile aesthetics via paper and digital-based platforms. The minimum clinically important difference between platforms was set at 15 mm. The percentage of participants who rated smile attractiveness worse on digital images was recorded. The paired one-tailed Student’s t test was used to determine differences between digital and paper platforms, and Bland–Altman analysis and intraclass correlations (ICCs) were used to test for agreement between paper and digital photographs. Results: Ninety-nine subjects participated, 55 men (mean age = 22.05, standard deviation [SD] = 1.91) and 44 women (mean age 22.05, SD = 1.84). There were statistically significant differences between paper-based and digital photographs for all images except one (paired t test; P < 0.05). Digital ratings were lower than paper-based ratings for all images, and differences were clinically significant in four out of eight images. A high percentage of participants (50.5%–85.9%) rated smile attractiveness worse on digital images than on paper for all images. There was poor agreement between the two methods as assessed by ICCs and Bland–Altman analysis. Conclusion: Equivalence between paper and digital images for smile aesthetics cannot be assumed, and paper-based photographs may lead to clinically relevant overestimations of perceived attractiveness. As academic dentistry increasingly relies on digital imaging and sharing in the post-COVID-19 world, further validation of digital platforms for smile aesthetics assessment is warranted, and care should be taken when interpreting the results of studies assessing smile perception based on different platforms.

Keywords: Digital assessment, paper assessment, smile aesthetics, visual analog scale

How to cite this article:
Agou S. Comparison of digital and paper assessment of smile aesthetics perception. J Int Soc Prevent Communit Dent 2020;10:659-65

How to cite this URL:
Agou S. Comparison of digital and paper assessment of smile aesthetics perception. J Int Soc Prevent Communit Dent [serial online] 2020 [cited 2022 Dec 4];10:659-65. Available from: https://www.jispcd.org/text.asp?2020/10/5/659/291779

   Introduction Top

Dental aesthetics are important in treatment planning, and several valid measures have been developed to assess the need for treatment based on aesthetics including the Aesthetic Component of the Index of Orthodontic Treatment Need Aesthetic Component (IOTN-AC)[1] and the Dental Aesthetic Index (DAI).[2] Furthermore, smile aesthetics is one of the most important parameters to evaluate when assessing patient-reported outcomes in dentistry, despite it being a subjective construct. The perception of smile aesthetics has been investigated amongst dentists, specialist orthodontists, lay people, and students.[3],[4],[5],[6] Various rating methods have been used to assess dental aesthetics, including smile aesthetics, which can be broadly categorized into those that apply rank-order scales, where the judgments are not spaced at equal intervals (such as the 5-point ordinal ranking scale used in Mackley[7]), and those based on continuous scales, such as the visual analog scale (VAS)[8],[9] and Q-sort methods.[9],[10] VAS have been particularly extensively used in studies of smile aesthetics,[5],[6],[9] but whether the format of the accompanying images being assessed impacts the result is unknown.

Nevertheless, traditionally administered in paper format, smile aesthetics are increasingly being assessed in digital formats, where photographs of the subject are presented on a computer screen or handheld device and aesthetics are scored using VAS. VAS are psychometric instruments used to measure subjective endpoints such as characteristics, feelings, or attitudes, and they have been applied to many medical disorders (e.g., pain, tinnitus), market research, and social science investigations.[11] In a VAS, scores are usually recorded by the subject marking a 100 mm line that represents a continuum between two extremes of feeling denoted by verbal descriptions such as “least attractive” to “most attractive.”[11],[12],[13] Digital VAS offers a number of advantages including easier, automatic, more accurate scoring, improved record keeping, and universal sharing via electronic medical records. We hypothesized that paper and digital versions of VAS may not be equivalent in terms of reporting the assessment of smile aesthetics, as the fidelity of the images presented in digital format might alter viewer perception.

There has yet to be a direct comparison of digital and paper-based assessment of smile aesthetics. To fill this gap, the VAS scores derived from digital and paper-based assessments of smile aesthetics by dental and medical students were compared.

   Subjects and Methods Top


This was a cross-sectional study of 100 dental students and non-dental students from the same campus at a public university in Saudi Arabia, over a period of two months. One hundred participants were selected from year 4 and 5 student registration lists using systematic random sampling. The Research Ethics Committee at the Faculty of Dentistry of the University reviewed and approved the study protocol.

Digital and paper-based assessment of smile aesthetics

All participants were asked to complete a web-based questionnaire [Figure 1]A and a paper-based questionnaire [Figure 1]B to rate eight color smile images using VAS. Both paper and digital questionnaires were completed by all participants one after the other. The questionnaires first presented brief information about the study and asked questions on participant gender, age, faculty, and year of study. Participants were then informed that by continuing with the survey they would be considered to have given their consent to take part. The questionnaire included a brief introduction about the length of time needed, who the investigator was, and the purpose of the study. For the paper-based format, each participant was provided with a booklet that included printed photographs, on photographic paper, of the eight smile images and was asked to view each image for twenty seconds without being allowed to return to previously rated images, as suggested in a similar study by Flores-Mir et al.[14] For the web-based format, students were provided with a tablet (Apple iPad) and were asked to rate the smile presented consecutively as digital images on separate webpages with responses captured automatically. In both formats, participants were asked to rate each image on a VAS (100 mm) from 1 (least attractive) to 100 (most attractive). The webpage was developed using a combination of JavaScript and the ASP.NET framework. No incentives were given in exchange for participation in the study. Respondents were not allowed to review or change their answers.
Figure 1: (A) Digital visual analog scale (VAS)-based platform for the assessment of smile aesthetics. (B) Paper-based format

Click here to view

Smile images

The smile images were used by kind permission of Dr Hanan Omar and as published in (15). Briefly, a suitable image was first identified that, based on dental aesthetics, proportions, and gingival aspects, was deemed to be “ideal.”[16] The image was then digitally cropped to remove the chin, nose, and cheeks to show only the anterior teeth, gingivae, and lips and eliminate other factors that could affect smile perception. The image was then digitally altered using Adobe Photoshop Software to produce eight images with different smile features to test the individual smile aesthetics of reduced maxillary incisor height (image B); darker crown shade (image C); diminutive lateral incisors (image D); flattened maxillary incisor edges (image E); midline deviation (image F); diastema (image G); and increased gingival display (image H) [Figure 2]. The degree of alteration and modification were similar to a number of previous studies.[17],[18] The face validity of this type of image development has been tested in several studies.[17],[19],[20]
Figure 2: Modified dental images used in either digital or paper format. (A) Model image. (B) Reduced maxillary incisor height. (C) Darker crown shade. (D) Diminutive lateral incisors. (E) Flattened maxillary incisor edges. (F) Midline deviation. (G) Diastema (H) Increased gingival display

Click here to view

Statistical analysis

The proportion of participants scoring paper and digital images differently was expressed as percentages. VAS scores were described with means, standard deviation (SD), and 95% confidence intervals (95% CI). Both the paper-based and digital VAS scales were 100 mm long. The minimum clinically important difference was set at 15 mm (15% of total scale length), meaning that differences of ≥15% were considered clinically significant. This threshold, based on minimum clinically important differences in pain research being 9%–13% and a 15% threshold used in a study of attractiveness, has been used in a previous study of smile aesthetics.[21] Differences between groups were analyzed using the paired sample t test. To evaluate the level of agreement and test for reliability, Bland and Altman analysis[22] and intraclass correlations (ICCs) were calculated. Data were analyzed using Statistical Package for the Social Sciences (SPSS) software program, version 24.0 (IBM Statistics, Chicago, Illinois), and the significance level for all tests was set at P < 0.05.

   Results Top

Of 100 invitees, 99 subjects, 55 men (mean age 22.1 years, SD = 1.91, range 18–26) and 44 women (mean age 22.1 years, SD = 1.84, range 18–25), participated. Of these, 37 were dental students. Gender and faculty effect on the platform used were evaluated and found to be non-significant.

A comparison of the VAS scores assigned to digital and paper-based images is shown in [Table 1] and [Table 2]. The mean scores for paper-based images (range 46.6–64.9) were consistently higher than the scores assigned to digital images (range 21.5–55.7). In all cases, a few participants scored the paper and digital images the same (7.1%–16.2%), a minority assigned higher scores to digital images (7.1%–34.3%), while a majority assigned higher scores to paper-based images (50.5%–85.9%) [Table 1]. The mean paper and digital VAS scores for the eight images were significantly different for all images except image G [diastema; paired t test, all P < 0.05; [Table 2]]. The scoring differences of four out of eight images were clinically significant (≥15%). The results were similar according to gender in magnitude and direction.
Table 1: Mean visual analog scale scores assigned to the digital and paper-based images and the proportions of individuals scoring the two formats the same or differently

Click here to view
Table 2: A comparison of digital and paper-based scoring of smile aesthetics scores, including by gender

Click here to view

To further assess the reliability of the two methods for assessing smile aesthetics, the ICCs [Table 3] and Bland–Altman plots [Figure 3] were evaluated. ICCs between digital and paper ratings were all less than 0.41 (P < 0.05) for the entire cohort. Bland–Altman analysis further revealed poor inter-method agreement, with image F (midline deviation) showing the greatest difference of –35.56 ± 29.28 and limits of agreement (95% CI) ranging from –92.9 to 21.8 [Table 2].
Table 3: Intraclass correlations between digital and paper-based visual analog scale scores for each smile esthetic image

Click here to view
Figure 3: Bland–Altman plots for each image assessed by digital and paper-based visual analog scale (VAS) scoring

Click here to view

   Discussion Top

This study was the first direct comparison of digital and paper-based images using VAS scoring of smile aesthetics. The results show that there was poor agreement between digital and paper-based methods as assessed by quantitative differences between the assigned scores and two reliability measures (ICCs and Bland–Altman analysis). Half of the observed differences between methods were of sufficient magnitude to be deemed clinically significant. Overall, participants always assigned lower VAS scores to digital images than paper-based images, suggesting that the use of paper-based images might lead to overestimation of perceived attractiveness.

Previous comparisons of paper and digital VAS scoring for other subjective assessments not relying on visualizing images, such as pain, have shown good agreement between modalities. In a recent study, Delgado et al.[23] compared pain scores recorded using paper-based VAS and digitally using both a laptop and mobile phone in 100 patients presenting with pain as their main complaint and reported no clinically significant differences between paper-based VAS assessment and VAS scores obtained on laptops or mobile phones. While a difference was detected between paper-based and mobile phone-based VAS scores, this difference was small, not clinically significant, and attributed to the size of the measurement entity (the finger) relative to the platform (mobile phone), which was suggested to have reduced the specificity. In their assessment of 33 patients with upper extremity injuries, Sindhu et al.[24] found excellent and identical test–retest reliabilities (r = 0.96) of digital touchscreen and paper-based VAS, indicating that participants scored their pain almost identically. Jamison et al.[25] performed a randomized, cross-over trial of 24 individuals receiving cognitive (descriptions of pain) and sensory (lifting test weights) stimulation and found extremely high correlations (r = 0.99 and r = 0.98) for both stimuli between palmtop device and paper-based VAS formats. Finally, Kreindler et al.[26] assessed mood in 28 subjects using a VAS-based questionnaire using a handheld computer and differently sized paper-based formats and detected no loss of precision or accuracy of the mood scale data. These data have supported the use of VAS-based pain assessment in clinical practice. While the minor differences between different platforms noted in the previous studies may also apply here, the large differences seen between digital and paper-based scoring of smile aesthetics suggest that other factors contribute to poor platform agreement when the subjective assessment is visual.

Therefore, the poor agreement between the two methods observed here may have been due to several factors related to the VAS itself or the presentation of smile images in digital and paper-based formats. Regarding the VAS, similar to Delgado et al.,[23] the nature of the interaction with a paper VAS and our digital VAS format may have altered the specificity of the VAS, such as through the interaction with the slider or the duration of looking at the image. With respect to differences in perception of smile images presented as hardcopy photographs or as digital images, it has been shown that the intrinsic properties of an image, such as color homogeneity, can alter the visual perception of attractiveness.[27] Therefore, if such intrinsic properties were more or less visible in a particular format, this might have introduced bias to explain the differences between the two formats. For instance, if the hue, tint, tone, and shade differed between digital and paper images, this might have given rise to different perceptions of aesthetics of tooth color, which are known to differ according to age and gender. Younger individuals tend to value whiter shade teeth more than older individuals[28],[29] and, in our population, lighter teeth were preferred by female subjects compared to male subjects,[30] suggesting that there may also be ethnic biases in smile esthetic perception. Furthermore, it has been shown that image quality can alter the perception of smile aesthetics, with smiles rated as more attractive in higher-resolution images.[31] Other properties of the formats that might have influenced perception include the size and print quality of the paper-based images. Although it is unclear exactly which properties of the paper or digital images may have influenced smile perception in this study, the properties of the images used when assessing aesthetics digitally may make a difference and must be carefully considered, particularly if comparing the results across studies. Given that the factors influencing perception of smile aesthetics across platforms are unknown, further research is warranted in this area.

This study has some limitations. First, this was a relatively small study of medical and dental students at a single institution, so the results may not be generalizable to other populations. The differences between the two image formats (e.g., hue, saturation, intensity) that might have accounted for how participants differentially viewed the digital and paper-based images could not be objectively quantified; future studies should aim to identify the factors that affect the validity of this perceptual test. Finally, we did not adopt a crossover study design, so the participants may have experienced fatigue or changed their scores after seeing a number of images, which may have introduced systematic bias and may account for the poor inter-method agreement observed.

In conclusion, this study shows that equivalence between paper and digital images in the assessment of smile aesthetics cannot be assumed. As academic dentistry is increasingly relying on digital imaging and sharing for assessment purposes, this study has practical implications for how students, faculty, and clinicians perceive the outcomes of clinical work depending on the modality used. The findings also have implications for researchers validating digital tools in outcomes-based research. Further evaluation of digital platforms for smile aesthetics assessment is warranted, and care should be taken when interpreting the results of studies based on different platforms.


The author would like to thank Dr. Mustafa Elhusain for his assistance in data collection.

Financial support and sponsorship

Not applicable.

Conflicts of interest

There are no conflicts of interest.

Author contributions

SHA designed, conceived and conducted the study, performed data analysis, and wrote the manuscript.

Ethical policy and institutional review board statement

Not applicable.

Patient consent statement

The authors certify that they have obtained all appropriate participants’ consent forms. Data was collected anonymously. No clinical or personal information were obtained from any participants.

Data availability statement

Not applicable.

   References Top

Brook PH, Shaw WC The development of an index of orthodontic treatment priority. Eur J Orthod 1989;11:309-20.  Back to cited text no. 1
Hamamci N, Başaran G, Uysal E Dental aesthetic index scores and perception of personal dental appearance among turkish university students. Eur J Orthod 2009;31:168-73.  Back to cited text no. 2
Gul-e-Erum, Fida M Changes in smile parameters as perceived by orthodontists, dentists, artists, and laypeople. World J Orthod 2008;9:132-40.  Back to cited text no. 3
Kokich VO Jr, Kiyak HA, Shapiro PA Comparing the perception of dentists and lay people to altered dental esthetics. J Esthet Dent 1999;11:311-24.  Back to cited text no. 4
Parekh SM, Fields HW, Beck M, Rosenstiel S Attractiveness of variations in the smile arc and buccal corridor space as judged by orthodontists and laymen. Angle Orthod 2006;76:557-63.  Back to cited text no. 5
Ritter DE, Gandini LG, Pinto Ados S, Locks A Esthetic influence of negative space in the buccal corridor during smiling. Angle Orthod 2006;76:198-203.  Back to cited text no. 6
Mackley RJ An evaluation of smiles before and after orthodontic treatment. Angle Orthod 1993;63:183-9; discussion 190.  Back to cited text no. 7
Phillips C, Tulloch C, Dann C Rating of facial attractiveness. Community Dent Oral Epidemiol 1992;20:214-20.  Back to cited text no. 8
Schabel BJ, McNamara JA Jr, Franchi L, Baccetti T Q-sort assessment vs visual analog scale in the evaluation of smile esthetics. Am J Orthod Dentofacial Orthop 2009;135:S61-71.  Back to cited text no. 9
Coy K, Speltz ML, Jones K Facial appearance and attachment in infants with orofacial clefts: A replication. Cleft Palate Craniofac J 2002;39:66-72.  Back to cited text no. 10
Klimek L, Bergmann KC, Biedermann T, Bousquet J, Hellings P, Jung K, et al. Visual analogue scales (VAS): Measuring instruments for the documentation of symptoms and therapy monitoring in cases of allergic rhinitis in everyday health care: Position paper of the German Society of Allergology (AEDA) and the german society of allergy and clinical immunology (DGAKI), ENT section, in collaboration with the working group on clinical immunology, allergology and environmental medicine of the german society of otorhinolaryngology, head and neck surgery (DGHNOKHC). Allergo J Int 2017;26:16-24.  Back to cited text no. 11
Aitken RC Measurement of feelings using visual analogue scales. Proc R Soc Med 1969;62:989-93.  Back to cited text no. 12
Duncan GH, Bushnell MC, Lavigne GJ Comparison of verbal and visual analogue scales for measuring the intensity and unpleasantness of experimental pain. Pain 1989;37:295-303.  Back to cited text no. 13
Flores-Mir C, Silva E, Barriga MI, Lagravere MO, Major PW Lay person’s perception of smile aesthetics in dental and facial views. J Orthod 2004;31:204-9; discussion 201.  Back to cited text no. 14
Omar H, Tai YT Perception of smile aesthetics among dental and nondental students. J Ed Ethics Dent 2014;4:54.  Back to cited text no. 15
Levin EI Dental esthetics and the golden proportion. J Prosthet Dent 1978;40:244-52.  Back to cited text no. 16
España P, Tarazona B, Paredes V Smile esthetics from odontology students’ perspectives. Angle Orthod 2014;84:214-24.  Back to cited text no. 17
Işiksal E, Hazar S, Akyalçin S Smile esthetics: Perception and comparison of treated and untreated smiles. Am J Orthod Dentofacial Orthop 2006;129:8-16.  Back to cited text no. 18
Cracel-Nogueira F, Pinho T Assessment of the perception of smile esthetics by laypersons, dental students and dental practitioners. Int Orthod 2013;11:432-44.  Back to cited text no. 19
Witt M, Flores-Mir C Laypeople’s preferences regarding frontal dentofacial esthetics: Periodontal factors. J Am Dent Assoc 2011;142:925-37.  Back to cited text no. 20
Ioi H, Nakata S, Counts AL Influence of gingival display on smile aesthetics in japanese. Eur J Orthod 2010;32:633-7.  Back to cited text no. 21
Dogan NO Bland-Altman analysis: A paradigm to understand correlation and agreement. Turk J Emerg Med 2018;18:139-41.  Back to cited text no. 22
Delgado DA, Lambert BS, Boutris N, McCulloch PC, Robbins AB, Moreno MR, et al. Validation of digital visual analog scale pain scoring with a traditional paper-based visual analog scale in adults. J Am Acad Orthop Surg Glob Res Rev 2018;2:e088.  Back to cited text no. 23
Sindhu BS, Shechtman O, Tuckey L Validity, reliability, and responsiveness of a digital version of the visual analog scale. J Hand Ther 2011;24:356-63; quiz 364.  Back to cited text no. 24
Jamison RN, Gracely RH, Raymond SA, Levine JG, Marino B, Herrmann TJ, et al. Comparative study of electronic vs. Paper VAS ratings: A randomized, crossover trial using healthy volunteers. Pain 2002;99:341-7.  Back to cited text no. 25
Kreindler D, Levitt A, Woolridge N, Lumsden CJ Portable mood mapping: The validity and reliability of analog scale displays for mood assessment via hand-held computer. Psychiatry Res 2003;120:165-77.  Back to cited text no. 26
Fink B, Matts PJ, D’Emiliano D, Bunse L, Weege B, Röder S Colour homogeneity and visual perception of age, health and attractiveness of male facial skin. J Eur Acad Dermatol Venereol 2012;26:1486-92.  Back to cited text no. 27
Vallittu PK, Vallittu AS, Lassila VP Dental aesthetics–a survey of attitudes in different groups of patients. J Dent 1996;24:335-8.  Back to cited text no. 28
Veeraganta SK, Savadi RC, Baroudi K, Nassani MZ Differences in tooth shade value according to age, gender and skin color: A pilot study. J Indian Prosthodont Soc 2015;15:138-41.  Back to cited text no. 29
Labban N, Al-Otaibi H, Alayed A, Alshankiti K, Al-Enizy MA Assessment of the influence of gender and skin color on the preference of tooth shade in saudi population. Saudi Dent J 2017;29:102-10.  Back to cited text no. 30
Sajjadi SH, Khosravanifard B, Moazzami F, Rakhshan V, Esmaeilpour M Effects of three types of digital camera sensors on dental specialists’ perception of smile esthetics: A preliminary double-blind clinical trial. J Prosthodont 2016;25:675-81.  Back to cited text no. 31


  [Figure 1], [Figure 2], [Figure 3]

  [Table 1], [Table 2], [Table 3]


Print this article  Email this article
    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
    Article in PDF (2,160 KB)
    Citation Manager
    Access Statistics
    Reader Comments
    Email Alert *
    Add to My List *
* Registration required (free)  

   Subjects and Methods
    Article Figures
    Article Tables

 Article Access Statistics
    PDF Downloaded295    
    Comments [Add]    

Recommend this journal