Use of Content Based Instruction and Socratic Discussion for ESL Undergraduate Biomedical Science Students to Develop Critical Thinking Skills

Content based language instruction can assist English as a second language (ESL) students to achieve better learning and teaching outcomes, however, it is primarily used to understand content, and may not help to develop critical analysis skills. Here we describe a pilot study that used a ‘Socratic’ small-group discussion in addition to content based instruction, to enhance critical thinking skills in ESL Biomedical Science students. Students were provided a detailed protocol for the analysis of a research journal manuscript and participated in a Socratic discussion. Students wrote evaluative pieces to summarize the manuscript’s topics, both before and after the small-group discussion. Overall, a third of all students displayed an improved critical thinking score based on Bloom’s taxonomy. However, only 20% of ESL students improved their critical thinking score, while 42% of non-ESL students improved. Despite this, students agreed that the Socratic discussion improved their understanding of science and enhanced their ability to review scientific literature. Importantly, ESL students believed that the discussion made them feel positive about their ability to read scientific literature. Thus, specific tools for enhancing critical thinking in the ESL student group should further be developed, with investigation of the retention of these skills warranted.


Introduction
Australia is a culturally diverse country, with the trend repeated in Higher Education (Crawford & Candlin, 2013). Subsequently there has been an increase in students who predominantly speak a language other than English, resulting in these students requiring a significant level of support (Crawford & Candlin, 2013). It has been noted that students with English as a second language (ESL or English language learners) first need to learn basic interpersonal communication, with the more advanced cognitive academic language proficiency which enables students to analyse, evaluate and interpret concepts acquired in a formal Higher Education setting (Sanner & Wilson, 2008). To assist with this, a trend in ESL education has been the emphasis in promoting thinking as a significant component of English language pedagogy (Davidson & Dunham, 1997). However in a number of disciplines, the ability to comprehend common knowledge can be restricted by the constraints and common practices of the discipline. For example, ESL students undertaking a science related program are required to develop their basic literacy skills, at the same time as developing their scientific specific literacy. To identify key concepts in science, this involves identifying information in scientific texts, interpreting and applying this knowledge and finally formulating responses which explain these ideas (Carrier, 2005). One mechanism that can be used to enhance the scientific literacy of ESL students is to embed the development of these skills in a task-focused, content-based activity which provides students with tools that will be useful throughout the course of their studies. scientific literacy), in addition to the Socratic discussion in a small-group format (to improve critical thinking), in ESL students enrolled in a foundations unit in the Bachelor of Biomedical Sciences degree at Victoria University, Australia. It was hypothesised that after attending the seminar (with the instruction and discussion), students would show improved literacy and critical thinking skills based on the indications of higher order thinking from Bloom's taxonomy (Bloom, 1956).

Theoretical Background
The didactic method of learning involves a lecturer addressing a class of students and providing them with categorical knowledge, questions and problems as well as the solutions to them. The role of the student in this method is simply to observe, absorb and regurgitate this information in examinations, recalling by rote and learning passively. There is typically no emphasis on collecting and evaluating information and students are largely unable to apply their memorised knowledge to new problems and situations (Garside 1996;Kaddoura 2011).
Critical thinking and metacognition are considered to be essential skills for any graduate (Bailin, 2002;Gunn, Grigg & Pomahac, 2008) and necessary as basic skills for workplace integration and active participation in a democratic society (Abrami et al., 2008;Vieira, Tenreiro-Vieira & Martins, 2011). As well as discipline knowledge, science graduates are also expected to be able to demonstrate an ability to synthesise, critique and analyse information leading them to a more informed judgement, or critical thinking (Staib, 2003). These are skills highlighted in the higher order levels of Bloom's taxonomy (Bloom, 1956). However, the adoption of didactic learning has resulted in a content driven lecture focus with limited opportunities for the development of critical thinking skills. Thus, alternative modes of teaching should be utilised to enhance the development of essential scientific literacy skills such as critical and analytical thinking, problem solving and improved communication (Levy, Little & Whelan, 2011).
One method which supports this model of higher order learning is the use of discussion, and specifically small-group discussions which provide students with the opportunity to critique, evaluate and analyse key ideas and concepts, as well as enhance the overall participation of individual students via active learning practices (Abrami et al., 2008;Chowning, 2009;Pollock, Hamman & Wilson, 2009;Vieira, Tenreiro-Vieira & Martins, 2011) and resulting in improved perceptions of their learning (Pollock, Hamman & Wilson, 2009). Importantly, Pollock, Hamman and Wilson also demonstrated that small-group discussions, in general, showed equal levels of participation of students from different language backgrounds (Pollock, Hamman & Wilson, 2009).
The increase in cultural diversity in Australian universities has increased the proportion of ESL students. Importantly, education of ESL learners is complex because it involves teaching discipline specific content to students while they are developing a second language (Rosebery, Warren, & Conant, 1992). This is compounded by the dominant instructional approach which separates the teaching of English language from the teaching of academic content, because it is assumed that proficiency in English is a prerequisite for learning topic and subject specific matter (Collier, 1989;Cummins, 1981). This is problematic because it may take some time for ESL language learners to acquire a level of language proficiency comparable to native speakers (Collier, 1989;Cummins, 1981). Thus ESL students may fall behind academically if they do not learn the content of the curriculum as they acquire English (Stoddart, Pinal, Latzke and Canaday, 2002). Cooperative learning tasks that utilise small-group discussions may help students understand discipline content better and develop new perspectives (Garside, 1996). However despite a study suggesting that students from different language backgrounds perceive an improvement of their learning outcomes following small-group discussions (Pollock, Hamman & Wilson, 2009), the role of small-group discussions in promoting critical thinking and enhancing student learning in culturally diverse cohorts has not been investigated

The Study
This study was approved by the Victoria University Human Ethics Committee (HR12/187).

Participant (Subject) Characteristics
Undergraduate Biomedical students in their first year of the Bachelor of Biomedical Science degree at Victoria University participated in the study. The 59 students that participated in this study were from students enrolled in the first year subject Foundations in Biomedical Science B (RBM1502) in Semester 2, 2012 (total 91 enrolments).

Study Design
participated in the seminar which consisted of a text-based method of analysis as well as a Socratic discussion. Students were advised that participation was voluntary and they were able to withdraw from the study at any time without penalty. They were also advised that their survey responses were anonymous, and would not affect their final grade for this subject. Before reading the manuscript 'Vitamin D deficiency is the cause of common obesity' by Foss (Foss, 2009), students were asked to complete a 'Quick Write', which is similar to the 'Minute Paper' (Cross & Angelo, 1988). Instructions were to write down any "knowledge they have on the themes of vitamin D deficiency and obesity as well as any ideas about a potential link between these two topics".
Students were then introduced to the research paper by Foss (Foss, 2009) and were instructed to read it following specific guidelines. Firstly, students were required to read the entire paper (Foss, 2009) and number the paragraphs. Secondly students needed to underline key claims made and note who is making that claim. Thirdly, students were asked to highlight words or phrases they do not understand. Finally, students were provided with 5 mins to participate in discussions with their partner to compare their markings and analysis of the manuscript.
Following this specific analysis of the manuscript, students then participated in a Socratic discussion, where students stand in a circle arrangement with the facilitator included. The facilitator poses an open ended question to initiate the discussion. Students are advised to use examples from the text to support their arguments and also to draw on previous knowledge and experience. The facilitator does not lead the discussion, but merely prompts students and opens discussion topics by asking further questions. Students are then asked to complete a 'Final Analysis' in which they are asked to detail any "knowledge they have on the themes of vitamin D deficiency and obesity as well as any ideas about a potential link between these two topics" after reading the journal article and also from the Socratic discussion.
To analyse the qualitative value of the written responses, each Quick Write and Final Analysis answer was scored categorically Level 1, Level 2 or Level 3 based on Bloom's taxonomy (Table 1) which is designed to reflect higher-order critical thinking (Fang, 2005). Bloom's Taxonomy details 6 cognitive learning objectives from lowest to highest order thinking as knowledge, comprehension, application, analysis, synthesis and evaluation. For the sake of simplicity, Level 1 answers were those that demonstrated only knowledge and/or comprehension. Level 2 answers were those that demonstrated some application and/or analysis. Level 3 answers showcased synthesis and/or evaluation (Bloom, 1956) (Table 1). In this way a higher score reflects higher level critical thinking skills. Each student's results before and after the discussion were analysed for any improvement, similarity or decline of that particular student's Bloom's taxonomy ranking score. A higher score denotes a greater presence of higher order thinking. The Quick Write and Final Analysis scores were analysed using a Students Paired t-test, using p values to represent significance (p <0.05).

Survey of Students Following the Socratic Discussion
After completing the final analysis, students were asked to complete a survey intended to evaluate the Socratic discussion ( Figure 1). Students responded to ten closed questions following a Likert Rating scale from 1-5 and N/A. 5= Strongly Agree, 4 = Agree, 3 = Undecided (neither agree nor disagree), 2= Disagree, 1= Strongly Disagree and N/A = Not Applicable. The questionnaire was designed with questions that are worded both positively and negatively in order to identify the students who were not engaged in answering the evaluation. Students not engaged with the task are identifiable by a pattern of agreement to two different questions that are inherently contradictory and/or by marking all questions with the same score e.g. "3 -Undecided". Data from students who answered in this fashion were deemed invalid and were not included in the analysis. Your responses to this evaluation will be combined with those of your fellow students to provide feedback about the effectiveness of the Socratic seminar in the Bachelor of Science (Biomedicine) degree. The information gained will be used in a research publication. However your responses are completely anonymous. Completion of the survey is also voluntary and not compulsory. The median scores of each answer were analysed using a nonparametric Wilcoxon signed-rank test to analyse the medians that were significantly different from the neutral score of "3 -Undecided", using p values to represent significance (p <0.05).

Limitations
The limitations inherent to this pilot study design are that this study did not utilize a control group of students that were asked to perform a written task without reading the manuscript or performing the discussion. However, inherent in any educational study is often anecdotal evidence that supports the development of the project. In this instance, students previously complained of difficulties in reading primary literature, indicating it was often hard to comprehend. In addition, as the concept of critical thinking in this manuscript is defined based on analysis of text using a Bloom's Taxonomy ranking score, other measures may provide a different outcome to this study. Further, the applicability of this research to other instructional settings is limited to general findings.

Text Analysis of Students Work According to Bloom's Taxonomy
Student attendance in the tutorial sessions for the Foundations subject is not compulsory. In addition, all students who attended (n=59) were advised that the activities scheduled for the tutorial class (Quick Write, Final Analysis, Socratic Discussion and questionnaire) were purely voluntary. Of the 91 enrolled students, 59 attended the tutorial (63%), and all present students (n=59) participated in the Socratic discussion. Of these 59 students, 58 returned the questionnaire whereas all 59 completed both the "Quick Write" and "Final Analysis". In this cohort 33 students self-identified as non-ESL, while 25 self-identified as ESL. Brief demographic data is presented in Table 2. Textual analysis of the "Quick Write" and "Final Analysis" shows the frequency of scores and improvements (Table  3). When comparing the total frequencies of answer scores of the cohort, it was found that 'level 1' answers decreased by 11 (19%) and 'level 3' answers increased by 10 (17%) in the Final analysis when compared to the Quick Writes, suggesting that as a group, critical thinking skills were improved. At an individual level, the improvements were even greater; with 32% of students showing an improvement by one or two levels. However, this is contrasted with some students whose Final Analysis scored lower than their Quick Write (15%). Statistical analysis indicated that there was a significant difference between the scores for the Quick Write and Final Analysis (p<0.01). Specific analysis of the changes in Bloom's taxonomy ranking score segregated into ESL and non-ESL students indicated that 42% of non-ESL students had improved their score, represented as a 1 or 2 which demonstrate the increase in level of their response in the Final Analysis compared to the Quick Write. However, only 20% of ESL students showed an improvement in their ranking score (Table 4). Further, 68% of ESL students showed no change in this ranking score following the intervention, compared with 45% of non-ESL students.

Survey
Analysis of the survey questions ( Figure 1) is displayed in Table 5 and indicates that as a cohort, responses were generally positive with 8/10 questions achieving agreement (item number 4 is negatively worded therefore disagreement is in fact a positive indication). Statistical analysis of this data indicated that all responses not equal to 3 were significantly different to the median of 3 (Neutral) (all values p<0.01). Again, when the responses were segregated into ESL and non-ESL students, the only difference between the cohorts was that ESL students believed that the Socratic discussion made them feel more positive about reading papers, while the non-ESL students did not agree with this statement (Table 6). Thus, this suggests that despite a limited improvement as indicated by the increase in Bloom's taxonomy ranking score (Table 4), the ESL students believed that the detailed protocol for the analysis of the paper, as well as the discussion, provided them with confidence to engage in the task of reading primary literature.

Discussion
In Australia there has been a significant increase in the number of ESL students enrolled in higher education (Crawford & Candlin, 2013). As such, successful progression of ESL students has become a concern for universities. ESL students are often required to meet institutional standards for English language proficiency. However, science and scientific language embodies a specialised and unique view of thinking and reasoning that uses a distinct variety of language (Fang, 2005).
The subject RBM1502 at Victoria University focuses on building core scientific strengths considered as foundations of biomedical science, including statistics, communication skills and critiquing of scientific literature. In a single 2 hour tutorial session, we focused on improving students' critical thinking skills by providing them with a mechanism for reading a scientific manuscript, as well as understanding and communicating scientific concepts taken from a source of scientific literature. In this case the tutorial was focused on the scientific journal manuscript "Vitamin D deficiency is the cause of common obesity" by Foss (Foss, 2009). Our data demonstrated an improvement in the Bloom's Taxonomy ranking score, which differed between non-ESL (42%) compared to ESL (20%) students in the cohort. Further, despite both groups agreeing that the exercise was valuable, only ESL students indicated that the seminar helped to increase their "positivity" associated with reading primary literature. Certainly the question then remains, in the ESL cohort, are similar interventions that are repeated across their three year degree program required to improve the critical thinking skills of these students? experiences, so that the link between the prior knowledge and new knowledge enhances their ability to understand the task (Wang, Martin & Martin, 2002). Importantly, the scientific manuscript that was chosen for this study discussed research into areas that are often represented in the popular press. Further, as these students are enrolled in a Biomedical Science degree, there is high relevance for these topics in their studies. Confidence in an education setting is important for student retention (Favero, 2011), possibly by the development of learning strategies and social inclusion (Hryciw et al., 2013). The Socratic discussion format enhances collaborative learning, which builds on skills of cooperation and communication which may not be supported by other areas in their degree program (Topping & Winterhoff, 2001).
Critical thinking is emerging as an important concept in preparing high school students for tertiary education (Abrami et al., 2008) and for preparing graduate students for postgraduate opportunities and the workplace. In addition, students that engage in collaborative discourse and evaluation develop skills to enhance student conceptual understanding and students' skills and capabilities with scientific reasoning (Osbourne, 2010). In science based industry, professionals are expected to solve problems, work constructively with others in their field and identify errors when they are made (Gunn, Grigg & Pomahac, 2008). With new technologies, ethical issues and conceptual challenges often arise which require the application of critical thinking (Cross & Angelo, 1988). In 2011, Vieira (Vieira, 2011) suggested that critical thinking is important in the field of science as it imparts the thinker a greater understanding of a given issue, source or topic.
After completing the text-based instruction seminar with Socratic discussion, students showed a higher level of critical thinking, with an improvement in their answer scores compared to their pre-discussion score However, ESL students did not demonstrate the same level of improvement as non-ESL students. While it might seem obvious that students would know more about the topic after reading a journal manuscript than before, the analysis did not measure improvement of discipline knowledge. The methods used measure markers of critical thinking -not as what the students write but instead how they express and justify themselves. Indeed it is quite possible that students can 'understand' the journal manuscript thoroughly yet provide an answer of lower order thinking by simply recalling information and paraphrasing content. So too can a student display higher order thinking in their Quick Write without even having read the manuscript if they for instance, display critique and organise a supported argument or viewpoint.
In the peer group discussion, the same concepts were examined but using a more lay-person language and in a non-threatening environment. This suits ESL students who may have trouble grasping the concepts in the manuscript because of a high degree of scientific language. Polite (Polite & Adams, 1997) detailed the importance of real life relevance of discussions and that when students were able to personally relate the issue that the result was higher level of engagement and responses that result in a greater display of critical thinking. As described previously, the link with previous knowledge is essential to ESL students (Wang, Martin & Martin, 2002).
A small proportion of students (15%) received a Final Analysis score that was lower than their Quick Write. It is unlikely that the Socratic seminar would cause a loss of critical thinking skills, so we postulate that this may reflect a lower level of understanding of the task, the journal manuscript or simply be the result of students not engaging with the task. It is unlikely that students who did not understand the relevance and value of the task would critique, synthesise and evaluate the information even if they possess the ability to. A decreased score might therefore reflect aptitude for the task at the beginning of the session and decreased engagement and effort towards the end.
It is important also to note that this is an isolated preliminary trial of the use of the seminar incorporating the text-based instruction and the Socratic discussion to provide effective tools in the understanding of scientific literature, with comparisons between the ESL and non-ESL student cohorts. This study suggests that participation in this seminar is conducive to improving critical thinking skills, with limited success in an ESL cohort. Whether this improvement is stable over time without further intervention remains unseen. Certainly there is a strong argument for the sustained delivery of content and practices over time (Pally, 1997) to ensure the skills are scaffolded across the curriculum and their development supported. Further studies should involve an analysis of the effect of the integration of regular seminars of this kind into the first year curriculum and scaffold this across their degree program. However, it may be the case that sustained, embedded interventions of this type are need for ESL students, to ensure the improvement of their critical thinking skills. Ideally, each intervention may result in a cumulative improvement of critical thinking skills and produce students who are active thinkers and learners.

Conclusion
In conclusion, by asking the original research question "Does the use of structured text analysis with a Socratic discussion improve critical thinking skills in ESL undergraduate students?" it is possible to postulate some primary conclusions. This study shows that an improvement of Bloom's taxonomy ranking scores occurred from 'Quick Write' to 'Final Analysis' and that this is due to participation in the seminar. This improvement of scores is related to improvement in critical thinking skills as indicated by Bloom's taxonomy higher order thinking (Eber & Parker, 2007). However differences exist between the ESL and non-ESL students, so that the number of students that demonstrated an improvement was reduced in the ESL cohort. In addition to this, students generally agreed that the text-based instruction and the Socratic discussion helped them to develop their confidence in understanding of a scientific paper, improved their discipline knowledge and scientific literacy, as well as finding the experience positive. Data from this experiment serve as a promising preliminary indicator that text based instruction in addition to Socratic discussions may improve critical thinking skills when integrated into the curriculum. This presents a possibility for tertiary institutions to develop their undergraduate science courses in order to produce a higher quality and more actively-thinking graduate.