Establishing Reliability on the ADI-R
There are two factors in establishing your reliability: 1) showing that you have learned the standardized administration procedures, and 2) showing that you understand coding rules and have achieved at least 90% agreement with our lab. In order to achieve reliability, we must evaluate your administration and coding on three different administrations of the ADI-R.
1. Complete the ADI-R 2 ½ Day Research Training workshop.
You must complete the ADI-R Research Training before beginning the reliability process. You will want to establish reliability on the instrument before you begin to collect data for your research project.
2. Demonstrate standardized administration procedures.
You must demonstrate your understanding of standardized administration procedures on 3 separate administrations. Administrations of the ADI-R completed for the ADI-R Research Training can be included in this process. Demonstrations of administration procedures can include:
- Your pre-course review tape submitted for the ADI-R Training for Researchers
- Administrations of the ADI-R you do during the training workshop
- Your post-course taped administration submitted to us for reliability scoring
3. Establish reliability in coding the ADI-R.
You must reach 90% agreement with our lab or another designated group on 3 administrations of the ADI-R. Agreement is calculated separately for the diagnostic algorithm and for the protocol in total, and you must achieve 90% agreement on both. Administrations for achieving reliability in coding can include:
- Administrations of the ADI-R during the training workshop
- Your scoring of the administration on the post-course tape we send home with you after the training workshop
- Your post-course taped administration submitted to us for reliability scoring
If you work with someone who has established reliability on the ADI-R , you can complete the steps of establishing reliability with that person, as long as you are working at the same physical site and have frequent (i.e., daily) contact to talk about coding and administration issues. To establish reliability with someone at your site, you must achieve 90% agreement in coding with the ADI-R-reliable examiner across 3 consecutive administrations. Your ADI-R-reliable examiner also must see you administer the ADI-R during at least one of the 3 reliable administrations. We recommend that you and the reliable examiner double-score administrations (that is, code an administration independently, then check your reliability with one another) until you have established 90% agreement among yourselves on three independently scored administrations. This can include the tapes we have given you.
Training Others At Your Site:
After you establish reliability, you can then teach research group members who are in the same physical location as you are. Collaborators must be physically at the same site. Thus, though the University of Michigan and Yale are collaborating, we are not in the same location. Even though the University of Chicago and Northwestern University are in the same city, they are not the same site. We recommend that, with colleagues at your site, you double-score administrations (that is, code an administration independently, then check your reliability with one another) until you have established 90% agreement among yourselves on three independently scored administrations of the ADOS. This can include the tapes we have given you. We request that you only train people who are in your immediate research group (who are at the same site as you are and whom you see on a daily basis). If you need to be able to train others outside your immediate group, you must first get reliable with us, and then complete the process for becoming an ADI-R Trainer. [see Becoming a Trainer]
|
