Research Methodology Examination:Â Exploring Mixed Methods vs. Quantitative Research Methods
By:Â Deirdre Houghton
Astronaut, Neil Armstrong stated, “Research is creating new knowledge,” (“Neil Armstrong Quotes,” n.d.), and indeed, it does! However, what is important about research is not only the knowledge gained from the results, but also the methodology that researchers use to gather their information. Assessing the Quality of Mixed Methods Research: Toward a Comprehensive Framework, by Alicia O’Cathain and Consequences of Childhood Reading Difficulties and Behaviour problems for Educational Achievement and Employment in Early Adulthood, by Diana Smart, George J.Yousef, Ann Sanson, Margo Prior, John W. Toumbourou and Craig A. Olsson, are both interesting and detailed articles discussing different research methodologies. In following the 4-R’s (Research, Researcher, Researched and Reader), it will be demonstrated how each article illustrates very different research methodologies, thus illustrating a dichotomy that is present among researchers and their chosen methods.
Alicia O’Cathain, is a professor at the University of Sheffield (UK). Currently, she works in Health Services Research, where she teaches courses focussing on mixed method research. Furthermore, she has completed numerous studies on patient healthcare and chronic illnesses; written numerous scholarly articles on both research processes and health related interventions. Her article focussed on the methodology of mixed method research, (which employs both quantitative and qualitative methodologies), and the significance of assessment for this style of research. O’Cathain (2015) reported three routes researchers could choose from to complete mixed methods research, including “the generic research approach, the individual components approach, and the mixed methods approach.” (O’Cathain, 2015, p. 535) First, the generic research approach involved the researcher using a broad research lens while using more general criteria for their study. However, this approach may not provide enough detail in the results. Second, the individual components approach involved the researcher completing criteria specific to each of the quantitative and qualitative methods to gain valid information on their study. An issue identified with this approach is that, “concerns[regarding]…the quality of one or both components may suffer as a direct consequence of being part of mixed methods study,” (O’Cathain, 2015, p. 535) and it proposes several criteria sets to be followed, thus lengthy for the researcher. Finally, the mixed method approach involved using a determined set of benchmarks that enabled the researcher to combine qualitative and quantitative research methods to address or question the subject with a unit of criteria. Possible problems associated with this method included the researcher not having a sound understanding of the style of research (ambiguities in the process itself) or the criteria, and personal biases could weigh in. (O’Cathain, 2015, p. 536)
O’Cathain’s (2015) article provided an example where the mixed method research approach was used to evaluate the effectiveness of pamphlets made accessible to women notifying them of their “choice around…decisions [they] face in maternity care.” (O’Cathain, 2015, p. 543) The research conducted involved both Random Controlled Trials and Ethnographic research. The outcome of the research, informed the researcher that the pamphlets were “not effective in promoting informed choice…[concluding it]…was that a culture of informed compliance [that] operated, rather than one of informed choice…[thus] the culture was not conducive to leaflets promoting informed choice.” (O’Cathain, 2015, p. 543) In relation to the 4’Rs, this example illustrated that the individuals running the testing were the researcher(s); the outcome of the study of the pamphlet’s effectiveness was the researched; the research demonstrated how the study was conducted by employing the mixed method approach (both qualitative and quantitative criteria and clear assessment applied); and the reader(s) were the colleagues involved in the research study itself, and possibly other health administrators interested in the study.
O’Cathain (2015) suggested, that to produce high quality mixed methods research, a sound outline is essential to provides the researcher with “guidance…common language and… direction for further development.” (O’Cathain, 2015, p. 532) Furthermore, she recommended the framework or table developed by V. Caracelli and L. Riggins (O’Cathain, 2015, p. 537) that incorporated the methodologies of A. Tashakkori and C. Teddlie (O’Cathain, 2015, p. 537) to assist researchers in their research collection and assessment when using mixed method research, as these two format styles combined were very comprehensive.
The guideline O’Cathain (2015) referred to incorporated eight main categories or steps to assist researchers in quality research and assessment. The categories, include: Planning Quality – the researcher examines how well the mixed methods research is organized and or planned; Design Quality – the researcher examines the appropriateness of the mixed methods research; Data Quality – the researcher manages, examines, and analyzes the collected information; Interpretive Rigor – the researcher examines the strength and authentic value of the collected data; Inference Transferability – the researcher draws conclusions, that could also assist in other areas of the study; Reporting Quality – the researcher completes their research responsibilities by reporting on all aspects of the study; Synthesizability – the researcher makes sure all elements, comprising of the mixed method review, are reported on including information that comes from both qualitative measures and quantitative measures; and Utility – researcher determines and reports on the usefulness of the researched outcomes. (O’Cathain, 2015, pp. 544-552) O’Cathain (2015) emphasized that substantial criteria must be adhered to when using mixed method research and its assessment. Furthermore, she also indicated that there are still challenges associated with this research method. For example, the criteria that researchers must use for completing and assessing their research on a specific study is very extensive; and there may be conflict between the criteria used to gather data or in the assessment process itself.
The article Consequences of Childhood Reading Difficulties and Behaviour Problems for Educational Achievement and Employment in Early Adulthood, by Diana Smart, George J.Yousef, Ann Sanson, Margo Prior, John W. Toumbourou and Craig A. Olsson followed a different research methodology than O’Cathain’s, it followed quantitative research methods; thus provided extensive numerical data. The purpose of the article was to research possible connections between children’s reading difficulties (RD) and behaviour problems (BP), and the link they share with obtaining poor educational and occupational outcomes later in life. The article itself was based off data from the Australian Temperament Project (ATP). It should be noted that the ATP project is a “life course longitudinal study of psycho-social development,” (Smart, D. et al., 2017, p. 288) and provided evidence and further insight into the “significance of childhood RDs and BPs for early adult outcomes, including their direct and mediated effects.” (Smart, D. et al., 2017, p. 288) This study also indicated both direct and indirect effects that could impact those being studied. The results of this informative article, indicated that children who had BPs and RDs were “at risk for poorer educational and occupation outcomes with co-occurring problems… increasing the risk of poorer education outcomes.” (Smart, D. et al., 2017, p. 288) Furthermore, this research indicated, “the effects of childhood BPs on occupational status were mediated by secondary school non-completion, but childhood RDs were not.” (Smart, D. et al., 2017, p. 288) The conclusion of this study is significant. It stressed the importance of screening children at a young age to recognize and help decrease the development of reading difficulties and or behaviour problems; and the necessity of providing the needed support. (Smart, D. et al., p. 288) Early identification of these issues would improve an individual’s chances of being successful in school, completing school and accessing future opportunities, in further education and or occupation.
Unlike O’Cathain’s (2015) mixed method research, this study used quantitative methods to outline and emphasize its data. Information that researchers collected came primarily from a variety of comprehensive tests and reports. Research results were calculated and formatted to include percentages, ratios, and results that corresponded to specific testing criteria. Some criteria examples researchers used to examine their subjects, included “maternal age, education background, non-English speaking, number of children in the family, single parent families, and child gender,” (Smart, D. et al., 2017, p. 290) The charts and statistics included in the article were very informative.
When considering the 4-Rs in this study, the researchers included Diana Smart, George J.Yousef, Ann Sanson, Margo Prior, John W. Toumbourou, Craig A. Olsson and others whom collected and recorded data from the Australian Temperament Project. The researched are those who took part in the study and were exposed to qualitative test methods, thus demonstrating the link between reading difficulties and behaviour problems; and their progression throughout school, post-secondary school and into an occupation. The research recordings consisted of data from tests, reports and questioning over a period of many years (childhood – adulthood from the ATP). The reader(s) of the research would be those who completed the research itself. Schools, healthcare professionals, and community support agencies would likely also be interested in reading this report as it may assist in potential policymaking.
This quantitative research article focussed on the schooling, occupational struggles and successes of students with RDs and BPs. It was detailed and extensive. However, if the researcher(s) followed the mixed method research format, as demonstrated in O’Cathain’s article, it would have included more personal or qualitative criteria. Additional areas that may have been examined, if the mixed method research was used, could have included information on student/teacher/parent relationships; quality of education in the region/district; school district funding for special needs; economics of the region or country; extended family support outside of the home; home life situation (substance abuse, mental health, trauma); oral language being a traditional source of communication; and access to and availability of outside agency support.
After reading Assessing the Quality of Mixed Methods Research and Consequences of Childhood Reading Difficulties and Behaviour Problems for Educational Achievement and Employment in Early Adulthood, it is evident how involved and/or extensive different methodologies are in their organization and procedures. This thoroughness in research practice is essential to create a valid and encompassing examination of the subject or the researched, and eliminate bias. Furthermore, different research methodologies create and use specific sets of criteria to ensure tests and assessments are comprehensive and authentic. Lastly, the methodologies used by researchers to test, study, collect and report information, demonstrate that they are equally as important, as the knowledge we learn from research outcomes.
As a teaching professional, I understand how essential it is to examine and provide correct data on students abilities within a course. Although one can use quantitative data, such as multiple choice testing or looking at your class average to assess the learning situation as a whole, I feel that it does not provide me with a true snap shot of student growth, their learning process and skill level. I feel qualitative or a mix of both gives me a better picture. Therefore, I employ either mixed methods research or strictly quantitative research in my assignments and assessment practices. Providing this method of assessment provides students with the opportunity to analyze, explain “why” or “how” a situation occurred or exists, and practice making inferences all of which draw on a multitude of knowledge and skill, and gives them the opportunity to “show me what they know.” In-depth questions enable students to demonstrate to me what they know and understand. I find interviewing and self-assessments (other examples of qualitative assessments), also provide me with a better idea of how a student is progressing. Furthermore, employing the above mentioned research methods into my teaching practices also lets me take into account items such as student learning levels, Individual Education Plans, modifications and adaptations. In relating back to the articles, if the second article by D. Smart et. al., employed more qualitative research methods, the researchers may have had even more information to explain their study. Qualitative methods could have provided them a personal in-depth look as to what was happening in the lives of their research studies.
References
Neil Armstrong Quotes. (n.d.). Retrieved from        http://www.brainyquote.com/quotes/neil_armstrong_363175
O’Cathain, A. (n.d.). Assessing the Quality of Mixed Methods Research: Toward a Comprehensive Framework. SAGE Handbook of Mixed Methods in Social   & Behavioral Research, 531-556. doi:10.4135/9781506335193.n21
Smart, D., Youssef, G. J., Sanson, A., Prior, M., Toumbourou, J. W., & Olsson, C.  A. (2017). Consequences of childhood reading difficulties and behaviour       problems for educational achievement and employment in early adulthood.    British Journal of Educational Psychology, 87(2), 288-308.doi:10.1111/bjep.12150