There are two factors in establishing your reliability: 1) showing that you have learned the standardized administration procedures, and 2) showing that you understand coding rules and have achieved at least 80% agreement with our lab. In order to achieve reliability, we must evaluate your administration and coding on three different administrations of the Modules 1 / 2 and on three different administrations of Modules 3 / 4.
1. Complete the ADOS training workshops.
You must complete both the ADOS Training for Clinicians and the ADOS Training for Researchers before beginning the reliability process. You will want to establish reliability on the instrument before you begin to collect data for your research project.
2. Demonstrate standardized administration procedures.
You must demonstrate your understanding of standardized administration procedures on 3 separate administrations each for Modules 1 / 2 and Modules 3 / 4. Administrations of the ADOS completed for the ADOS Research Training can be included in this process. Demonstrations of administration procedures can include:
- Your pre-course review tape submitted for the ADOS Training for Researchers
- Administrations of the ADOS you do during the Research Training workshop
- Your post-course taped administration submitted to us for reliability scoring
3. Establish reliability in coding the ADOS.
You must reach 80% agreement with our lab or another designated group on 3 administrations of the ADOS Modules 1 / 2 and 3 administrations of ADOS Modules 3 / 4. Agreement is calculated separately for the diagnostic algorithm and for the protocol in total, and you must achieve 80% agreement on both. Administrations for achieving reliability in coding can include:
- Administrations of the ADOS during the Research Training workshop
- Your scoring of the administrations on the post-course tape we send home with you after the Research Training workshop
- Your post-course taped administration submitted to us for reliability scoring
If you work with someone who has established reliability on the ADOS , you can complete the steps of establishing reliability with that person, as long as you are working at the same physical site and have frequent (i.e., daily) contact to talk about coding and administration issues. To establish reliability with someone at your site, you must achieve 80% agreement in coding with the ADOS-reliable examiner across 3 consecutive administrations of Modules 1 / 2 and 3 consecutive administrations of Modules 3 / 4. Within these 6 administrations, all 4 modules must be represented. Your ADOS-reliable examiner also must see you administer all 4 modules at some point during the reliability process. We recommend that you and the reliable examiner double-score administrations (that is, code an administration independently, then check your reliability with one another) until you have established 80% agreement among yourselves on 3 independently scored administrations of the ADOS. This can include the tapes we have given you.
Training Others At Your Site:
After you establish reliability, you can then teach research group members who are in the same physical location as you are. Collaborators must be physically at the same site. Thus, though the University of Michigan and Yale are collaborating, we are not in the same location. Even though the University of Chicago and Northwestern University are in the same city, they are not the same site. We recommend that, with colleagues at your site, you double-score administrations (that is, code an administration independently, then check your reliability with one another) until you have established 80% agreement among yourselves on three independently scored administrations of the ADOS. This can include the tapes we have given you. We request that you only train people who are in your immediate research group (who are at the same site as you are and whom you see on a daily basis).If you need to be able to train others outside your immediate group, you must first get reliable with us, and then complete the process for becoming an ADOS Trainer.
[Becoming a Trainer]
