UC San Diego Linguistics Experimental Handbook

This website contains some useful information for graduate students about running experiments at UCSD Linguistics.

1. Using human subjects: IRB approval

If you are planning an experiment that uses human subjects, you first need to obtain an approval from the IRB. Do it early! Getting my first IRB approval took me almost the whole quarter! Here is what you need to do:

  1. Complete the CITI Program training. You can only run subjects once you have completed this. Note, this applies to any 199s as well.
  2. Check application due dates (for "Social and Behavioral Sciences") here.
  3. Prepare your application. All the necessary information on how to do that is found here (under "For Social and Behavioral Science Projects"). All required forms for a new application can be found here. Basically, the application consists of the following:
    • Facesheet: a short summary of the project with signatures from you, your advisor, and the chair of the deparment, it can be found here
      • To make the initial case number you need to send in the Facesheet electronically from the PDF. If you are sending it from a Mac you may get an error message saying it did not go through. It did, but it is always good to call IRB to double check.
    • Cover letter: request an expedited approval (if not, it will take even longer)
    • Application form: a blank version from the IRB can be found here
    • Consent form: a blank version from the IRB for adults can be found here; other blanks versions are also available on the IRB website
    • Language background questionnaire: example (pdf)
      • I usually make people fill out the questionnaire, and then ask them more specific questions and take notes.
  4. Get the necessary signatures on the Facesheet (from the department's chair and your advisor).
  5. File your application. All files can now be uploaded to the IRBs electronic system here. There is no need to mail in any documents. Once you have the Facesheet signed upload the final version with all signatures. Be sure to have all documents uploaded at least two weeks before the meeting date. This is not offically posted, but that is the only way you can guarantee your application will be discussed at your desired meeting date.

Several weeks after filing your application you should hear back from the IRB. Usually they'll ask you to make small revisions that you can submit online where you submitted your original application. Once they approve it, they'll send you a stamped consent form by mail: all your subjects will need to sign two copies of it (one for them and one for you).

The department has begun to compile sample applications so that reserachers can save time on new applications. To download the sample applications sign into the parody server and go to the folder parody/shared/IRB_Applications. Inside the folder you will see the file "List_of_IRB_Applications.xlsx" that includes information such as keywords about the application. Each application is zipped and password protected. To get the password email the graduate student currently in charge of Reserach and Travel Grants.

2. Undergraduate research assistants (199s)

Running an experiment is a lot of work, but you can get some help by taking undergraduate research assistants ("one-ninety-nines"). First, some things to note:

  1. Undergraduate research assistants have to sign up for a LIGN 199 class (2 or 4 units, P/NP).
  2. You are not the instructor for their 199 class. You have to find a faculty member who agrees to do that.
  3. The amount of units corresponds to the workload: 5 hours/week for 2 units, 10 hours/week for 4 units.
  4. Linguistics majors and language studies majors can get elective credit for one 4-unit 199, which is a good incentive for them (remember to mention it when you advertise your experiment!).
  5. UCSD frowns upon students taking too much P/NP credit, so while students can enroll in more than 4 credits, their advisors may warn against it. As a result it can be hard to keep students around for more than one or two quarters, since after that it is generally on a volutneer basis.

What you need to do:

  1. Find a faculty member that is willing to be the official supervisor.
  2. Advertise your experiment (preferably, the quarter before you want to run the experiment):
    • Prepare a short description of the experiment and the duties. Set some requirements (e.g., 3.5 GPA in major, specific coursework).
    • Either ask the faculty to make an announcement in their classes, or — what I found easier — send an email to Alycia at arandol at ucsd dot edu and she will email the listserv for all linguistics majors.
    • Ask students to email you back explaining why they are interested and why you should choose them. This will help you decide who you want to interview.
    • You can ask your faculty supervisor to check the GPA (or grades for particular courses) of a potential RA. The faculty have access to this information for all undergrads.
    • In the last couple years Lingua (the undergraduate Linguistics club) has been holding a 199 Fair at the beginning of every quarter where graduate students can come and pitch their projects. This has also been a useful resource for finding 199s.
  3. Get 199 forms from Alycia or here:
    • Let students fill out the first part (info about their coursework, etc.).
    • Fill out the second part (info about their duties, etc.; their final project can be a "final self-evaluation").
    • Tell students to get a signature from the faculty supervisor, and then to give the form to Alycia (or just send them directly to Alycia who can get the faculty signature later; discuss the details with the faculty supervisor).

Students have until the end of the 2nd week of the quarter to turn in their paperwork.

3. Software to run experiment

Here is a limited selection of softwares that can be used to prepare your experiment:

  1. Praat
    • Great for "listening experiments" (identification or discrimination tasks). Fairly easy to set up by following the instructions found here.
    • Cons: doesn't measure RTs.
  2. PsyScope
    • Designed to run all sorts of psychological experiments. Measures RTs.
    • Cons: only runs on Macs.
  3. DMDX
    • Designed for psychological experiments. Measures RTs to visual and auditory stimuli.
    • Cons: only runs on PCs.
  4. Ibex
    • Python-based software for self-paced reading and speeded acceptability judgment tasks. It can be run online through a web browser.
    • It seems to be both platform independent and fairly modular. The same code can be used to run an experiment both online and off-line. Crucially, it supports non-latin fonts with no hassle. It was developed by Alex Drummmond, a grad in linguistics at University of Maryland (http://www.ling.umd.edu/~alexd/).
  5. WebExp
    • To run web-based experiments.
    • You don't have to download it, it's already installed on one of our servers. First, ask Ezra to get an account on 'quiz'. Then, follow Laura Kertz's tutorial.
  6. Presentation
    1. It's commercial software, but apparently it's available on CRL computers. It's especially good for audio-visual experiments.
  7. Linger
    • Free software for self-paced reading experiments.
  8. PsychoPy
    • Python-based software, but also has a GUI for those why do not know Python.

4. Where to find subjects: SONA

Linguistics, Psychology, and Cognitive Science all share the same subject pool. The Psychology department runs the subject pool here.

Why is SONA so great?

  1. You can specify eligibility for your experiment, which means that you only run subjects that you need.
  2. You have access to a large pool of subjects so even if you are looking for subjects with, let's say, some very specific language background, there is a good chance that you'll find them.
  3. Students who don't show up get a penalty, which means that this happens less often.
  4. The system is very convenient and easy to use.

This is what you have to do:

  1. Read the guidelines. This is the most recent version to my knowledge.
  2. Before running any experiments make sure both you and anyone else running the experiment such as a 1999 has completed the CITI Program training.
  3. Make an account. To create an account as an experimeter email ucsdsonasubjectpool at gmail dot com.
  4. Once you get an account you can add your current experiment.
  5. It may take a few days for the study to be approved. Once it is you can customaize the experiment by adding participation qualificiations and experimenters.
  6. You can also add other experimters such as 199s to run the study. To have an undergraduate added as an experimenter they need a different account from their account as a subject. Have them email the email address above but cc you on the email as proof that they are working with a graduate student or faculty.

IMPORTANT: for each quarter, the last day to run the experiment is always Wednesday of week 10. You can't schedule subjects for a later date.

Compiled by Bożena Pająk

Updated by Page Piccinini

Many thanks to the contributors:
Klinton Bicknell, Rebecca Colavin, Laura Kertz, Cindy Kilpatrick

For Current Grads

The graduate program combines thorough training in theoretical linguistics and formal analysis with innovative experimental approaches to the study of language.

Inquiries about the Linguistics Graduate Program should be directed to Professor Andy Kehler, Director of Graduate Studies.