Page 198 - Dark Shades of Istria
P. 198
Istrian Memories in the Dark Tourism Context: The Quantitative Analysis
ining the photos, if available, which is completely in line with the options
listed by Bryman (2012, p. 290). Content analysis is a data-driven process,
which means that researchers ‘want to code the text in terms of certain
subjects and themes, which is called a categorisation of the phenomenon
or phenomena of interest’ (p. 297). It is assumed that the process of cate-
gorisation (coding) is relatively objective. As mentioned above, the inclu-
sion of one researcher only made the assessment of intra-rater reliability
possible. In this case, Mackey & Gass (2005, p. 129) and Riffe et al. (2005,
p. 145) suggest a two-step approach:
1. the researcher/coder first codes all the data of the body of the con-
tent, and
2. after a certain interval, they need to re-code the data or (only) part
of it.
According to these suggestions, the scores of the same researcher, but
at different points of time, can have the same effect as the standard inter-
rater reliability check procedure. Consequently, a two-week break was in-
troduced between the ‘original’ and re-coding. The 20 content samples
(15.9 of the body of content) for reliability testing were then randomly
selected.⁴ In comparison with the initial coding, recoding of the content
of selected reports showed merely negligible deviations – two variables
or about 0.2 of codes differed – therefore, we did not continue with
the re-coding process and accepted the high level of intra-rater reliabil-
ity.⁵
Moreover, the above-mentioned two-step approach also shows, al-
though in a simplified manner, the test-retest reliability of a construct.
The variable type (codes) prevented the mathematical calculation of in-
ternal reliability, as is customary in sociology and other social sciences in
the processing of scale variables.
The guidelines of Riffe et al. (2005, pp. 169–170) were followed for va-
lidity testing as well. According to the authors, ‘content analysis can be
a very strong research technique in terms of external validity or gen-
eralisability.’ The analysis included all available reports on public dark
⁴ On the basis of past research, Riffe et al. (2005, p. 146) suggested a sample of 5 to 25 of
the body of the content.
⁵ We assumed that at least 90 of codes must be equal, which is understood as ideal
(Mackey & Gass, 2005, p. 244); Stemler (2001, p. 2) notes 95 of agreement. However,
we (orientationally) used perceived assumption in the literature, which is characteristic
for the inter-rater reliability, since the scale for the intra-rater reliability was not observed.
198
ining the photos, if available, which is completely in line with the options
listed by Bryman (2012, p. 290). Content analysis is a data-driven process,
which means that researchers ‘want to code the text in terms of certain
subjects and themes, which is called a categorisation of the phenomenon
or phenomena of interest’ (p. 297). It is assumed that the process of cate-
gorisation (coding) is relatively objective. As mentioned above, the inclu-
sion of one researcher only made the assessment of intra-rater reliability
possible. In this case, Mackey & Gass (2005, p. 129) and Riffe et al. (2005,
p. 145) suggest a two-step approach:
1. the researcher/coder first codes all the data of the body of the con-
tent, and
2. after a certain interval, they need to re-code the data or (only) part
of it.
According to these suggestions, the scores of the same researcher, but
at different points of time, can have the same effect as the standard inter-
rater reliability check procedure. Consequently, a two-week break was in-
troduced between the ‘original’ and re-coding. The 20 content samples
(15.9 of the body of content) for reliability testing were then randomly
selected.⁴ In comparison with the initial coding, recoding of the content
of selected reports showed merely negligible deviations – two variables
or about 0.2 of codes differed – therefore, we did not continue with
the re-coding process and accepted the high level of intra-rater reliabil-
ity.⁵
Moreover, the above-mentioned two-step approach also shows, al-
though in a simplified manner, the test-retest reliability of a construct.
The variable type (codes) prevented the mathematical calculation of in-
ternal reliability, as is customary in sociology and other social sciences in
the processing of scale variables.
The guidelines of Riffe et al. (2005, pp. 169–170) were followed for va-
lidity testing as well. According to the authors, ‘content analysis can be
a very strong research technique in terms of external validity or gen-
eralisability.’ The analysis included all available reports on public dark
⁴ On the basis of past research, Riffe et al. (2005, p. 146) suggested a sample of 5 to 25 of
the body of the content.
⁵ We assumed that at least 90 of codes must be equal, which is understood as ideal
(Mackey & Gass, 2005, p. 244); Stemler (2001, p. 2) notes 95 of agreement. However,
we (orientationally) used perceived assumption in the literature, which is characteristic
for the inter-rater reliability, since the scale for the intra-rater reliability was not observed.
198
