methodology in psychology

Developmental Psychology : Methodology • Good methodology does not equal good science • But good science must have good methodology • Good methodology is about making sure your question is being properly answered by your task… • And your selection of participants... Developmental Psychology : Methodology • Subjects - how many - a lot or a few ? – The larger the number the easier it is to obtain a significant result – The smaller the number the more in-depth testing • Which ? – Large numbers for surveys (only) Developmental Psychology : Methodology • Surveys (large numbers, shallow testing) – Education/paediatrics • TV preferences/ bed-times/ hours physical recreation, etc. – Psychometrics/IQ • Sex differences/age differences on standardised tests – Correlational studies • e.g. video violence and play behaviour - descriptive • Experiments (smaller numbers, more in-depth) ? – Causal analyses • e.g. video violence and play behaviour - manipulative Developmental Psychology : Methodology General observations on designing experiments with children : as much an art as a science ... Developmental Psychology : Methodology • Designing an experiment with children – What is the most important consideration ? • How well the task fits the question • How much fun the child will have √ √ √ • How the statistics are to be carried out Developmental Psychology : Methodology • Carrying out the experiment - what is the most important consideration ? • How consistent you are across subjects • How much fun the child is having √ • How much fun you are having √ • What might indicate the child should go back even if s/he hasn‟t asked to •Yawning •Inattention •Stretching/ head back •Looking away •Dropped voice Developmental Psychology : Methodology • How can you make sure the child has fun ? – embed your task in a game • An example : – Do children use counting spontaneously in order to enumerate ? – You could give them some things and ask them how many are there…. – OR you could give them a strong motive for wanting to know „how many‟… see tape…. Developmental Psychology : Methodology • Another example : – Can children work out transitive spatial relationships ? • If blue goes above red and red goes above green, what would a tower built of all three colours look like ? (after Pears and Bryant…) – You could ask…. – Or you could give a reason for a tower AND a reason for keeping the spatial relationships…. Developmental Psychology : Methodology • Make sure the task is clear – Avoid complex language and long story lines - see tape • Don‟t create a set-up which „invites‟ the child to play with it in some other way... – Make the moving parts integral with your game • Make it rewarding to win ! – Stickers can help….. Developmental Psychology : Methodology Some reminders about variables and how to build them into the design Developmental Psychology : Methodology • Designing the experiment : the variables – Independent : Task conditions (e.g. active v passive sentence instruction) – Dependent : Accuracy, reaction time, gestures, speech, eye movements, etc. – Age ? Independent or Dependent ? • Can be both Embryology of Mind : Lecture 3 Transitivity : the 5 term series task Pair AB BC CD DE “red is taller than b lue” “blue i s taller than green” “green is taller tha n w hite” “w hite i s taller tha n yellow ” “blue i s shor ter tha n red” “green is shor ter tha n blue” “w hite i s shor ter tha n green” “yellow is shor ter tha n w hite” Bryant and Trabasso „Transitive inferences and memory in young children‟. Nature, 1971 Embryology of Mind : Lecture 3 The independent variables (usually task factors) An example • Double comparative training versus single comparative training in transitivity tasks is a very important experimental variable The double version includes the inverse (in red)….. AB “red is taller than blue” “blue is shorter than red” BC “blue is taller than yellow” “yellow is shorter than blue” • CD “yellow is taller than white” “white is shorter than yellow” DE “white is longer than green” “green is shorter than white” Developmental Psychology : Methodology • Task factor : (using single v double comp training as an example) – which is better : Intra-subject or Inter-subject ? – Inter-subject gives a clean design where highly specific task factors are concerned Group A Group B Single comparative training Double comparative training Developmental Psychology : Methodology Training box with countersunk holes for rods Coloured rods trained in pairs : AB BC CD DE • A simple intersubject design Advantage: exactly the same underlying tasks can be given simultaneously under two (slightly) different conditions Group A “Red is longer than blue“ “Blue is shorter than red” Group B “Red is longer than blue“ “Blue is longer than green” “Green is shorter than blue”, “Green is shorter than blue”, etc. etc. Developmental Psychology : Methodology • But supposing you are looking at more major task differences, e.g the verbal v the concrete version of the task... – Intra or Inter-group ? Verbal Condition “Tom is taller than Jack”, Jack is taller than Bill”, etc. Non-verbal Condition AB BC Intra is better - why ? Developmental Psychology : Methodology • An intra-group design is better where individual differences could explain the group differences The smaller the group, and the bigger the difference between the conditions, therefore, the more appropriate an intra-subject design, i.e. both conditions to all subjects – But the problem is ???? • • Yes - order effects ! Developmental Psychology : Methodology • The potential problem for intra-subject designs is order effects. – The normal solution is ? – Counterbalancing ! Group A 1. Concrete task 2. Symbolic task Group B 1. Symbolic task 2. Concrete task Developmental Psychology : Methodology Bryant and Trabasso (1971) : TRAIN The design of an experiment is more than just this, however - it covers the whole structure of the study…. Phase A (successive): AB (till 9/10) BC (till 9/10) CD(till 9/10) DE (till 9/10) Phase B (simultaneous): AB BC CD DE in random order till 6/6 successively correct on each TEST (no corrective feedback) AB BC CD DE & AC AD AE BD BE CE x4 Developmental Psychology : Methodology Describing the study design in a report Developmental Psychology : Methodology • How would you write the design paragraph for this task ? • (Stimuli already described) – “The experiment was divided into a training and a test phase” – “In the training phase, Ss were presented with the four pairs ABDE, first successively and then in random order” – “In the test phase, all 4 training pairs and all 6 novel pairs deriving from the set were presented four times each in random order within 4 trial blocks of 10…” • (Further task details in the Procedure..) Developmental Psychology : Methodology • How would you write the design paragraph for the inter-subject version of this task ? – – – • (Stimuli and basic task already described) “The experiment was divided into a training and a test phase” “In the training phase, Ss were presented with the four pairs ABDE, first successively and then in random order” “In the test phase, all 4 training pairs and all 6 novel pairs deriving from the set were presented four times each in random order within 4 trial blocks of 10…” “Two training conditions were run; one using single comparative training (SC), the other using double comparative training (DC). “ “ An inter-subject design was used where half the Ss were randomly allocated to the SC condition; the other half to the DC..” • (Further task details in the Procedure..) – – Developmental Psychology : Methodology • How would you write the design paragraph for the intra-subject version of this task ? – “The experiment was divided into two main conditions : a concrete condition (CC) and a linguistic condition (LC)” – Now describe the stimuli and tasks for LC and CC – “ An intra-subject design was used in which half the subjects were randomly allocated to the CC condition followed by the LC condition, whilst for the other half it was vice-versa…” – “Both conditions were divided into a training and test phase,” etc. • i.e. describe the two different tasks first…. Developmental Psychology : Methodology Other sorts of experimental studies with children Developmental Psychology : Methodology • Are all experiments with children about this sort of experimenter-led testing ? • What other types of study might you carry out in a laboratory setting ? • A structured observational study - i.e where the behaviour is largely child-led Developmental Psychology : Methodology • What would you use this type of study for ? • Play behaviour • Spontaneous verbalisation during play/problem solving • Interactions with others Developmental Psychology : Methodology • Structured observational studies are harder to analyse - true or false ? • True ! Why ? • You (nearly always) have to work out the scoring system post-hoc Developmental Psychology : Methodology • Longer-term studies – These are almost always preferable to short „one-off‟ studies with children. • Why ? a Because there are numerous spurious reasons why children might not show their potential at first b Because learning itself can be a very important aspect of children‟s understanding Developmental Psychology : Methodology • Long-term learning-type tasks • How are the procedures different from crosssectional tasks ? 1 Corrective feedback is given on each trial 2 Learning criteria are set 3 Training is followed by test Developmental Psychology : Methodology • How are the measures different from cross-sectional tasks ? – They look at speed of acquisition (number of trials) – Learning curves – Retention during test – Although the stats are trickier, this is a very rich measure of competence Developmental Psychology : Methodology • Learning as a methodology – Richer behavioural analysis than pass/fail • depth of failure • upper limits on success • large data sets for exploring reasons for both – Where linguistic comprehension cannot be presumed • a non-verbal means of communicating task requirements Developmental Psychology : Methodology • This is an example of a learning task where the child has to learn to touch each shape once and once only Success is denoted at the end of a correct trial by a man climbing up steps QuickTime™ an d a DivX 5 decompressor are need ed to see this p icture . • Developmental Psychology : Methodology • Learning as a measure – within each individual : • criterion as a statistically significant measure of success e.g. 8/10 – across individuals, levels or task conditions : • • • • • trials to criterion/start of criterion run trial of last error no of errors savings slope of learning curve Learning within a task level expressed as number of errors learning on a simple sequence (square, circle, triangle) Number of incorrect touches 7 6 5 Normal learning curve 4 3 Without help With help 2 1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 trial number Learning as a function of task level : criterion = 4/5 Min poss trials to criterion = 5 Learning across task level 40 35 30 Game 1 sess 1 Game 1 sess 2 Game 2 sess 1 Game 2 sess 2 Game 2 sess 3 Cath1 Cath2 25 20 15 10 5 0 Number of items in sequence Developmental Psychology : Methodology New considerations regarding „special groups Developmental Psychology : Methodology • Non-normal populations e.g. autism – Make no assumptions about valid behaviour – Make sure the task engages the child‟s level of comprehension – Find out what might be preventing this…. – Be prepared to annotate the behaviour graphs Developmental Psychology : Methodology • Here the embedding of the task requirement - to touch everything once and once only - was reinforced by animating „correct‟ touches Repeat touches resulted in a low sound and a „yellow card‟ QuickTime™ an d a DivX 5 decompressor are need ed to see this p icture . • Developmental Psychology : Methodology • Make sure the task engages with the child – If it is a learning experiment, always be ready with a crashback lower level of task that you think your participant will be able to complete... – always finish a session on the note of success ! Developmental Psychology : Methodology • So how do you make testing with such children „fit‟ into a well-controlled study ? – 1. These observations are PART of the study – 2. Build them into your results by annotating the graphs or tabulating the various prompts, interruptions selfinterruptions, incomplete trials etc. – 3. Extract, if you can, „valid‟ bits of learning or performance and analyse its characteristics (e.g. perseveration, visual neglect etc) – 4. Be prepared to present all the data INDIVIDUALLY as each profile could be vastly different Developmental Psychology : Methodology • How do you select your sample ? – Supposing you are studying transitive inference in autistics (logical structure not so easily grasped) – You know that normal children pass Bryant and Trabasso‟s test at 5 years… – What autistics would you select - bearing in mind they have an IQ range from normal to severely retarded ...? Developmental Psychology : Methodology • As your „question‟ bears on fairly advanced abilities you would probably choose • high functioning children of targetted CA (e.g. 4 to 6 years old) • CA matched control children – difference at CA norm is an autistic specific problem • If it was more to do with e.g. the basic memory part of the task, then you might also choose • lower functioning children within a restricted but lower IQ range (and MA matched controls) – difference at MA norm is picking up a low IQ aspect of autism.. Developmental Psychology : Methodology • Supposing your question is not obviously specific to either (e.g. perception of emotions in faces), then you might have to choose both groups together with their controls • You may also then have to have both a linguistic and a non-linguistic way of testing – “look at this picture - what is this man feeling ?” – “look at these pictures - give me the picture of the happy man ? – Reaction times can help where verbal responses are not possible Developmental Psychology : Methodology • So for this more inclusive solution – Have IQ measures etc recorded for the sample but don‟t use them for pre-hoc selection – Avoid exclusion criteria (like language) and try to think of another way of answering your question – But use language as another level of assessment for subjects who are capable of this… Developmental Psychology : Methodology • In short, good methodology requires making sure your question is being properly answered by your task… – Use your imagination • …and your selection of participants… – Use your common sense • ...and the quality of the behaviour you are eliciting – Be enthusiastic AND systematic