Share this post on:

R strength of agreement for kappa coefficient was proposed: 0 or lower=poor, 0.010.20=slight, 0.21-0.40=fair, 0.41-0.60=moderate, 0.61-0.80=substantial, and 0.80-1.0 =almost perfect.45 Using the standards suggested by 46 Portney and Watkins , another set of statistical significance was defined: ICC<0.50 (low), ICC: 0.50 - 0.75 (moderate), ICC>0.75 (good). One concern with kappa is that it was designed for nominal random variables, therefore in cases of ordinal data the seriousness of a disagreement depends on the 47 difference between the ratings. The weighted kappa coefficient is probably the most useful method for agreement for ordinal data, but several issues of concern arise from using this 48 method in Belinostat supplier analysis. It was explained that the problem with the kappa statistic is that the kappa value depends on the prevalence in each category, which leads to difficulties in comparing the kappa values of different studies with different prevalence in the categories. Many factors can influence the magnitude of kappa, with the most common being the prevalence, bias and non-independence of ratings.49 To remedy this problem, Fleiss and Cohen44 suggested that the Intraclass correlation coefficient (ICC) is the mathematical equivalent of the weighted kappa for ordinal data and pointed that the ICC is the special case of weighted kappa when the categories are equally spaced points along one dimension. Other literature also supported that the ICC can also be used for ordinal data with equal 46 Some other distance between intervals. researchers demonstrated that to analyze the reliability of data obtained with original continuous scale, methods such as ICC, the standard error of measurement, or the bias and limits of agreement can be used.48,49 In some widely used numerical scales of psychopathology, ICC has shown to 50 produce a reliabilities interval between 0.70-0.90. Jakobsson and Westergren48 pointed out that some researchers used correlation as a measure of agreement. Correlation, like the chi-square test, is a measure of association and does not satisfactorilywww.pharmacypractice.org (ISSN: 1886-3655)Tan CL, Hassali MA, Saleem F, Shafie AA, Aljadhey H, Gan VB. Development, test-retest reliability and validity of the Pharmacy Value-Added Services Questionnaire (PVASQ). Pharmacy Practice 2015 Jul-Sep;13(3):598. doi: 10.18549/PharmPract.2015.03.measure agreement.51,52 Association can be defined as two variables that are not independent, while agreement is a special case of association where the data in the diagonal (perfect agreement) are of most interest. Therefore it should be noted that perfect association does not automatically mean perfect agreement because a perfect correlation (r=1.0) can be obtained even if the intercept is not zero and the slope is not 1.0. To illustrate this clearly, Jakobsson and Westergren showed an example when one of the observers constantly grades the scores a little higher than the other observers. This will give a high association but low agreement. Thus, correlation does not account for systematic biases. Furthermore, the correlation coefficient tends to be higher than the “true” 53 reliability. Therefore in this study, kappa and ICC were utilized to establish test-retest reliability. The results indicated evidence for the repeatability for PX-478 site construct measurements between two time points. Therefore, it is concluded that the intra-rater reliability test was established. Confirmatory Factor Analysis: Construct Va.R strength of agreement for kappa coefficient was proposed: 0 or lower=poor, 0.010.20=slight, 0.21-0.40=fair, 0.41-0.60=moderate, 0.61-0.80=substantial, and 0.80-1.0 =almost perfect.45 Using the standards suggested by 46 Portney and Watkins , another set of statistical significance was defined: ICC<0.50 (low), ICC: 0.50 - 0.75 (moderate), ICC>0.75 (good). One concern with kappa is that it was designed for nominal random variables, therefore in cases of ordinal data the seriousness of a disagreement depends on the 47 difference between the ratings. The weighted kappa coefficient is probably the most useful method for agreement for ordinal data, but several issues of concern arise from using this 48 method in analysis. It was explained that the problem with the kappa statistic is that the kappa value depends on the prevalence in each category, which leads to difficulties in comparing the kappa values of different studies with different prevalence in the categories. Many factors can influence the magnitude of kappa, with the most common being the prevalence, bias and non-independence of ratings.49 To remedy this problem, Fleiss and Cohen44 suggested that the Intraclass correlation coefficient (ICC) is the mathematical equivalent of the weighted kappa for ordinal data and pointed that the ICC is the special case of weighted kappa when the categories are equally spaced points along one dimension. Other literature also supported that the ICC can also be used for ordinal data with equal 46 Some other distance between intervals. researchers demonstrated that to analyze the reliability of data obtained with original continuous scale, methods such as ICC, the standard error of measurement, or the bias and limits of agreement can be used.48,49 In some widely used numerical scales of psychopathology, ICC has shown to 50 produce a reliabilities interval between 0.70-0.90. Jakobsson and Westergren48 pointed out that some researchers used correlation as a measure of agreement. Correlation, like the chi-square test, is a measure of association and does not satisfactorilywww.pharmacypractice.org (ISSN: 1886-3655)Tan CL, Hassali MA, Saleem F, Shafie AA, Aljadhey H, Gan VB. Development, test-retest reliability and validity of the Pharmacy Value-Added Services Questionnaire (PVASQ). Pharmacy Practice 2015 Jul-Sep;13(3):598. doi: 10.18549/PharmPract.2015.03.measure agreement.51,52 Association can be defined as two variables that are not independent, while agreement is a special case of association where the data in the diagonal (perfect agreement) are of most interest. Therefore it should be noted that perfect association does not automatically mean perfect agreement because a perfect correlation (r=1.0) can be obtained even if the intercept is not zero and the slope is not 1.0. To illustrate this clearly, Jakobsson and Westergren showed an example when one of the observers constantly grades the scores a little higher than the other observers. This will give a high association but low agreement. Thus, correlation does not account for systematic biases. Furthermore, the correlation coefficient tends to be higher than the “true” 53 reliability. Therefore in this study, kappa and ICC were utilized to establish test-retest reliability. The results indicated evidence for the repeatability for construct measurements between two time points. Therefore, it is concluded that the intra-rater reliability test was established. Confirmatory Factor Analysis: Construct Va.

Share this post on: