Four Tier Test Ecosystem: Diagnostic Test to Detect Misconception on Ecosystem Concepts

Article history: Received: 19 Okt 2020 Accepted: 17 Mar 2021 Published: 30 April 2021 Background: This study aims to develop and produce a diagnostic tool capable of detecting student misconceptions on the concept of ecosystems. Methods: This study used a research and development method with a modified 4D design (Define, Design, Develop and Disseminate) without disseminating or spreading. Data collection methods used in this research and development are observation, literature review, documentation, questionnaires, interviews and tests. Results: The results showed that the instruments developed were valid by validating 3 experts, namely material experts, evaluation and assessment experts, and linguists with an average percentage of each feasibility of 90% material expert judgment, 82% assessment expert assessment and evaluation. and 80% linguist assessment. Meanwhile, the instrument readability test showed a percentage of 63% in good category. The validity value of the instrument in general was 0.70 with the valid or high validity category, while the instrument reliability value was 0.82 with the very high category of reliability. The overall results of students' misconceptions on ecosystem material showed 23% of students experienced low-category miconceptions, 8% of students experienced no conceptual understanding, 17% of students experienced conceptual understanding, and 52% of students experienced partial understanding. Conclusions: Based on these results it can be said that the four-tier test instrument developed is feasible and can diagnose misconceptions in students


Introduction
Misconception is an understanding of a concept that is firmly believed to be true but the concept is not in accordance with the scientific concept of experts (Pesman, 2005;Peşman & Eryilmaz, 2010). In line with the expert's statement, Suparno (2013) explained that misconception as a concept that is incompatible with the scientific understanding accepted by experts in the related field. Suparno (2013) states that there are 5 categories that can cause misconceptions including: students, teachers, textbooks, context and teaching methods.
There are several ways that are commonly used to find out students' understanding of concepts and misconceptions, including open questions, two-tier diagnostic tests, concept maps, prediction-observation explanation, interviews, word association and drawing (Köse, 2008;Nurbaety et al., 2016). Some of these diagnostic tests have weaknesses such as diagnostic tests with interviews that can explore students' thoughts in depth, but it takes a long time to interview many students (Pakpahan et al., 2020). It is different from using multiple choices which can identify many students in a short time, but students' answers can only be just guesses (Kaltakci-Gurel et al., 2017). If using the essay test students get more time to think and write down their understanding in detail, but researchers find it difficult to detect and analyze the results of the essay test so it takes a lot of time in the process of analyzing it. Class discussions in small groups can only identify the understanding of some students because not all students dare to express their opinions directly (Kutluay, 2005;Istiyani et al., 2018). Therefore, the use of two-tier tests in detecting misconceptions is considered more relevant than interviews, multiple choice tests, essays, and class discussions. Two-tier test is a diagnostic test that is intended to measure students' misconceptions. This test component consists of the first tier which contains multiple choices, and the second tier contains reasons from the first tier. The advantages of using this instrument are: (1) Reducing the possibility of guessing; and (2) easier to manage and check than other methods, so it is very useful to use in the classroom (Kim-Chwee Daniel Tan et al., 1999;Fitriani et al., 2019).
As an initial analysis stage, the researcher tried to conduct an analysis of the Biology book used at SMAN 4 Tasikmalaya and found misconceptions in the concept of ecosystems. Based on the results of the analysis, the researcher also conducted a second pre-study to see if there were any misconceptions among students regarding the ecosystem concept by using a two-tier diagnostic test related to the concept of ecosystem and consisting of 5 questions adopted from research by Adibah, (2018) who is conducting research related to the development of a twotier test instrument on the concept of ecology. The two-tier diagnostic test was carried out in class XII MIA 5 with the pre-research results showing the percentage of student misconceptions of 39.8% with the medium category.. However, based on the literature review it was found that there are several weaknesses in which the use of the two-tier test instrument in detecting misconceptions in students is that it has results that are too overestimating for misconceptions because all wrong answers are considered misconceptions (Kutluay, 2005;Nurbaety et al., 2016). The use of two-tier also has not been able to determine the level of confidence of students in understanding a concept (Rukmana et al., 2019). In addition, the two-tier test was unable to distinguish between errors due to lack of knowledge and errors due to misconceptions, and to distinguish between correct answers based on guessing and correct answers based on authentic understanding (Türker, 2005).
The limitations on the two-tier test can be overcome significantly by including a confidence rating as an additional level for the two tier test so that a three tier test is developed to detect misconceptions in students. However, the three-tier test is only able to express one confidence rank for answers and reasons, unable to explain whether students have various levels of confidence for the two levels so a four-tier test is developed to overcome weaknesses in the three-tier test (Caleon & Subramaniam, 2010). The advantages in using the four-tier test instrument in detecting misconceptions in students are that it is able to measure differences in students' knowledge, so that they can find out the level of misconceptions experienced by students (Rukmana et al., 2019;Pakpahan et al., 2020).
Research and development related to the four tier test instrument in detecting student misconceptions has also been done a lot in physics, but only a few have been developed in biology such as the concept of plant tissue, excretion systems and photosynthesis, while the concept of ecosystem has never been developed. Therefore, the researcher tried to develop a four-tier test instrument on the ecosystem concept that had been developed previously using a two-tier test instrument. The purpose of this research and development is to determine the feasibility and accuracy of the four-tier test instrument in detecting student misconceptions on ecosystem concepts..

Methods
This research is research and development. The research and development design used was adopted from Thiagarajan, S. Semmel, D. S & Semmel, (1974) namely 4D (Define, Design, Develop and Disseminate) and the researcher did not do the disseminate stage.

Research Procedure
The first stage in this research is Define which consists of front-end analysis, student analysis, concept analysis, task analysis, and formulation of learning objectives. The second stage is the design which consists of the preparation of test standards, media selection, format selection, and initial design. The third stage is Develop which consists of expert judgement and product testing. The final product of this research is a diagnostic test for misconceptions on the concept of ecosystems.

Data Source
Data source in this study were 3 experts including expert of material, evaluation and assessment, and linguists. There were 2 trials conducted including initial field trials and final field trials. The initial field trial aims to determine the readability of the instrument, the trial is carried out in class XII MIA-4 with a total population of 32 students. The final field trial aims to determine the accuracy of the instrument in diagnosing misconceptions in students carried out in class XI MIA-1 and MIA-4 with a population of 66 students.

Data Collection and Data Analysis
Data collection techniques are in the form of literature studies, questionnaires, interviews, tests and documentation. Data analysis for the validators was given an assessment rubric related to the appropriateness of the instrument and its content. The results of the expert validator's assessment are calculated by the formula below: Information: % = Percentage of Score n = ∑ Score N = ∑ Total Score The product eligibility criteria from the results of expert judgment can be seen in the table below: Analysis of the validity and reliability of the instrument was generally carried out using ANATES version 4.2. The results of categorizing validity and reliability can be seen in Table 2 and Table 3.   Questionnaire assessment for students responses were adapted from Masriyah, (2006) with the percentage of the respondent's value using the formula below: Information: %NRS = Percentage of Student Response Score = Total Students Response Score on each question item The rubric for interpreting the results of the four-tier test is a guideline for grouping respondents' answers. Answers are grouped into 5 categories, namely Misconception, Not Understanding Concepts, Understanding Concepts, Partial Understanding and Uncodeable. The rubric for the interpretation of students' understanding can be seen in Table 4.

Result and Discussion
The development of a four-tier test instrument using a 3D development model obtained the following results:

Define
Based on problem identification and data collection, the researcher decided to develop a four tier test instrument that was able to detect students' misconceptions on ecosystems concept. This four-tier test instrument is a development of the two tier test instrument on the ecosystem concept that has been previously developed by Adibah, (2018). The define steps taken by the researcher are as follows: 1) The research subjects are at the high school level who have studied the ecosystem concept before, 2) Add 2 confidence rating that contain level of confidence for the answer to the correct and wrong answer and level confidence for answers to the reasons for choosing answers, 3) The test instrument was made of 28 questions, and 4) The content used was ecosystem concept.

Design
The design stage in making a four-tier test instrument is as follows: 1) Creating a grid of diagnostic test questions consisting of basic competencies, sub-concepts, indicators, question numbers on each indicator, 2) Determining the form of the test, namely the four-tier test, 3) Write down the four-tier test questions in the format that can be seen in Figure 1. and 4) The initial design of the draft four-tier ecosystem concept test instrument has been developed.

Figure 1. Four-Tier Test Format
Based on Figure 1. The four-tier test format can be explained that the form of a diagnostic test consists of four tiers, including: the first tier is a correct and wrong statement, the second tier is the confidence rating for student's answer in first tier, the third tier is the reason for student's answer in first tier, and the fourth tier is the confidence rating for student's answer in third tier. The confidence rating consists of 4 levels, namely: very unsure, not sure, sure and very sure. The reasons for answers there are 4 reasons that have been provided and one reason for an open answer if the four answers are not in accordance with the students' understanding.

Develop
In the development process, product testing was carried out to determine the feasibility and to measure the product's ability to detect student misconceptions. As for the tests carried out consisted of internal tests and external tests. Internal testing is carried out through expert judgment consisting of material experts, assessment and evaluation experts and language experts. The percentage of the assessment results can be seen in  The total percentage of expert judgment Figure 2. Percentage of expert judgement Based on Figure 2, it can be explained that the results of the content expert judgement show a percentage of 90% in very feasible or valid category, for assessment and evaluation expert judgment has a percentage of 83% in very feasible or valid category, and for linguist expert judgement it has a percentage of 80% in very feasible or valid category. The categorization of expert validation refers to (Arikunto, 2009). Based on the three expert judgments, it can be concluded that the four-tier test instrument can be used or is feasible, but the product must be revised according to the advice given by the experts.. The next product trial is an external trial consisting of initial field trials and final field trials. The initial field trials aimed to determine the readability of the four-tier test instrument by testing it in class XII MIA-4 with a total population of 32 students. To find out the readability of the instrument, students fill out an online questionnaire via google form. The results of the readability of the instrument can be seen in Figure 3.  Figure 3, the average readability of the instrument has a percentage of 63% in a good category, so it can be concluded that the four-tier test instrument for the ecosystem concept can be read by students. However, based on the criticisms and suggestions given by students, there are several things that must be revised by the researcher, students ask to giving addition of time for example 1 minute for 1 tier so that 112 minutes is used on 28 questions with each question having 4 tiers. Students also have difficulty solving the questions because the concepts asked in the questions have never been studied. But after conducting personal interviews with students, they explained that each concept had been studied but because it was too long so that students were unable to remember the concept. There are also some terms that students do not understand, such as scientific names that must be translated into Indonesian, and there are reasons for answers that have multiple meanings, especially in number 6, tier 3. In making questions, it must have regularity and completeness and correct spelling in a writing in order to be able to express clear ideas or thoughts so that students can understand it (Ayudia et al., 2016). Based on these suggestions, the researcher revised the four-tier test instrument before the final trial was carried out.
The final trial was conducted in class XI MIA-1 and XI MIA-4 with a total population of 66 students. The purpose of this trial is to determine whether the instrument is able and feasible to diagnose misconceptions in students. The test results have a validity of 0.70 with a high or good validity category. While the reliability was 0.80 in the high reliability category (Guilford, 1956). It is important in a study to have high validity and reliability, it is used by researchers so that the validity of the research conducted is not in doubt and it is expected to obtain valid and  (Sugiono, 2015). The results can be seen in Figure 4. Based on Figure 4. the percentage of understanding the concept of ecosystems in class XI MIA-1 and MIA-4 can be explained that the percentage of students who have misconceptions is 23%, 8% of students who do not understand the concept, 17% of students who understand the concept, 52% of students Partially understanding, and 0% students who uncodeable. The diagnostic test instrument developed in this study consists of 6 subconcepts. Based on the data obtained after conducting the final field trial, it is known that the percentage of misconceptions that occur in each sub-concept is different. For more details, it can be seen in Figure 5. Based on Figure 5, the average percentage of misconceptions in each sub-concept can be explained that misconceptions occur in each sub-concept with moderate and low categories (Suwarto, 2013). There are 16 questions in the low category and 12 questions in the medium category. The most common misconception is in the energy flow sub-concept with a percentage of 33% which is included in the Misconception category according to categorize moderate misconceptions. The sub-concepts are about the definition of the food chain and the analysis of the food web with a percentage of 65% which falls into the misconception category according to Suwarto (2013) categorized the misconception as moderate. Some of these sub-concepts are about the definition of the food chain and the analysis of the food web with a percentage of 65% which is included in the misconception category according to Suwarto (2013) categorize the misconception as high.
The researcher also conducted interviews with 3 biology teachers to determine the teacher's response to the instrument being developed. Based on the interview, it is known that the teacher has never analyzed the textbooks used by students. This can cause misconceptions in every lesson in class. In addition, the diagnostic tool used by the teacher is only a diagnostic tool used to measure learning outcomes based on what the teacher has taught, such as multiple choice questions, essays, practicums and interviews. The teacher also does not diagnose the concepts brought by students which will bring up a misconception that can cause a change in the student's mindset. If the teacher does not carry out earlier diagnostics related to misconceptions in students, there will be a potential misconception that is carried by students to the next level (Suparno, 2013;Pakpahan et al., 2020). Based on the extensive trials that have been carried out, it can be concluded that the source of the misconceptions that occur does not only come from students, but teachers and textbook sources are also able to cause student misconceptions. It is necessary to conduct an evaluation for the teacher in order to reduce the misconceptions brought by students by conducting earlier diagnostics in detecting student misconceptions. By using the four-tier test instrument, the teacher is able to see the overall understanding of students and reduce the possibility of guessing each answer (Türker, 2005). Based on interviews conducted with 3 biology teachers, they said that this instrument is very much needed in the ecosystem concept and needs to be given as a pretest periodically before entering the main ecosystem concept and also need to develop a four tier test for other biological concepts.

Conclusions
The diagnostic test instrument developed was a fourtier test instrument on the ecosystem concept using the 4D research method with no dissemination stage. The results of product validation by the three experts stated that the instrument could be used or was feasible. The instrument readability was 63% with a good category and the instrument could be used. The validity value in general was 0.7 with the high validity category while the instrument reliability value was 0.8 with the very high reliability category. The four tier test instrument developed was able to detect misconceptions, do not understand the concept, partially understand the ecosystem concept. The factors that cause misconceptions in students are students, textbooks and teachers.

Declaration statement
The authors reported no potential conflict of interest.