Welcome to mReliability
mReliability is a free resource with two primary goals. First, it provides open-source functions for calculating inter-observer reliability in the MATLAB environment. Second, it provides information about different approaches to calculating inter-observer reliability. Inter-observer reliability quantifies the extent to which measurements from two or more observers are consistent and interchangeable; it constitutes an important source of validity evidence in observational measurement.
Inter-observer reliability is especially important in the fields of affective computing and computational behavioral science, where algorithms are developed to automate observational measurement tasks. In these fields, MATLAB (Matrix Laboratory) is often used as a numerical computing environment and programming language. However, indexes of inter-observer reliability are not well-known in these fields and functions for calculating them in MATLAB are not widely-available. mReliability addresses both issues.
General Information Pages
Indexes for Categorical Measurements
- Agreement or accuracy
- Bennett et al.'s S score
- Cohen's kappa coefficient
- Gwet's gamma coefficient
- Krippendorff's alpha coefficient
- Scott's pi coefficient
- Specific agreement coefficient