Criterion-related validity

Criterion- Related Validity

by Nang Aye Aye Htun

What is criterion-related validity? 

• evaluate how accurately a test measures the outcome it was designed to measure. 
• To establish criterion validity, you need to compare your test results to criterion variables. 

How to measure criterion validity

Criterion validity is assessed in two ways:

• By statistically testing  a new measurement technique against an independent criterion or standard to establish concurrent validity
• By statistically testing against a future performance to establish predictive validity

Example: Criterion validity

A researcher wants to know whether a college entrance exam is able to predict future academic performance. First-semester GPA can serve as the criterion variable, as it is an accepted measure of academic performance.The researcher can then compare the college entry exam scores of 100 students to their GPA after one semester in college. If the scores of the two tests are close, then the college entry exam has criterion validity.

Types of criterion validity

There are two types of criterion validity.

• Concurrent validity is used when the scores of a test and the criterion variables are obtained at the same time.
• Predictive validity is used when the criterion variables are measured after the scores of the test

Criterion Validity Evaluating Methods

Correlation Coefficients

Correlation coefficients, such as Pearson’s correlation coefficient (r), are used to measure the strength and direction of the relationship between the scores on a test or measurement instrument and the criterion or outcome. A high correlation indicates a strong relationship, suggesting good criterion validity.

Receiver Operating Characteristic (ROC) Analysis

ROC analysis is commonly used when the test or measurement instrument produces dichotomous outcomes (e.g., pass/fail). It evaluates the ability of the test to accurately discriminate between individuals who have the criterion or outcome and those who do not. The area under the ROC curve (AUC) is used as a measure of the test’s predictive accuracy, with higher values indicating better criterion validity.

Sensitivity and Specificity

Sensitivity refers to the proportion of individuals with the criterion or outcome who are correctly identified by the test, while specificity refers to the proportion of individuals without the criterion or outcome who are correctly identified as such. Sensitivity and specificity are typically calculated based on predetermined cutoff scores on the test and are used to evaluate the accuracy of the test in correctly classifying individuals.

Regression Analysis

Regression analysis can be used to predict the criterion or outcome variable based on the scores obtained on the test or measurement instrument. By examining the strength and significance of the regression coefficients, researchers can determine the extent to which the test predicts or correlates with the criterion or outcome.

Known Groups Method

The known groups method involves comparing the scores obtained on the test or measurement instrument between groups that are known to differ in terms of the criterion or outcome. If the test can effectively distinguish between these groups, it suggests good criterion validity.

Construct Validity

By Nutthakan O. 

What is Construct Validity?

What is Construct Validity?

Construct validity serves as a crucial quality indicator in data collection. It ensures that the measurements used effectively capture the underlying theoretical concepts or constructs being studied. By establishing construct validity, researchers ensure that their data collection methods accurately reflect the intended attributes and dimensions of the construct. This helps in preventing biases, errors, and misinterpretations, ultimately enhancing the reliability and credibility of the research outcomes. In essence, construct validity guarantees that the data collected truly measures what it is intended to measure, making it an essential benchmark for the quality of data collection in research.

Types of Construct Validity:

1. Convergent Validity: This examines the extent to which measures of related constructs align with each other. Measures of related traits are expected to correlate positively, indicating a consistent assessment of the underlying construct.

2. Discriminant Validity: This assesses the extent to which measures of unrelated constructs display weak or negative correlations. Measures of distinct concepts should not strongly correlate, confirming their independent assessment.

   
   

   Selecting Unrelated Constructs:

   In testing discriminant validity, it's important to choose theoretically distinct or opposing concepts within the same category. For instance, when assessing introversion, selecting a contrasting trait like extroversion is appropriate. This opposition allows for an expected negative correlation between the measures.

   Measuring Construct Validity:

   Pilot Study: After developing a new measure, conducting a pilot study with a small sample helps evaluate its feasibility, reliability, and validity. This ensures necessary adjustments are made to accurately test the construct.

   Statistical Analyses: Correlations are often employed to assess convergent and discriminant validity by comparing results from your test with established measures. Regression analyses can also gauge whether the measure predicts expected outcomes, reinforcing its construct validity claim.

   Incorporating construct validity measures in research methods enhances the reliability of findings and the confidence in the interpretations drawn from test scores and measurements.

   
   

   Conclusion

   
   

   Ensuring Accuracy: Construct validity safeguards the meaningfulness and credibility of research findings.
   
   

   Construct validity helps researchers ensure that their measurements are meaningful and trustworthy for making valid inferences and conclusions. It involves demonstrating that the operationalization (measurement) of a construct aligns with the theoretical understanding of that construct.

   
   

   
   

The Educational Management Guidelines by using a Community Participation Process for a Child Development Center in Phadungmart Sub- district, Janae district, Nawathiwat Province

by Kanokwann Kanyati and Recha Choosuwan

Education Administration program, Faculty of Education, Prince of Songkla UNiversity, Pattani Campus, Pattani 94000, Thailand

 Received : August 9, 2020 , Revised: Novembe 7, 2020 , Accepted : December 7, 2020

                                                                                                                                Nang Aye Aye Htun

                                                                                                                         Seminer on Dissertaion

1) Background of the research

This research studies the guidelines of educational management by using a community participation process in a Childhood Development Center in Phadungmart Sub-district, Janae District, Narathiwat Province. This study used qualitative research that was designed for this specific research. There were 40 participants who were considered key informants to the research. This group included 10 representatives of the Child Development Center Committee, 9 teachers, children’s parents, and community leaders. A purposive sampling was used to select the participants. Semi-structured interviews were used to collect as much in-depth data as possible. The data was analyzed by a content analysis technique.

2)  The research objective

To study the guidelines for educational management through a community participation process of the Child Development Center of Phadungmart Sub-district, Janae District, Narathiwat Province.

3) The research methodology

It is a qulitative research. The research tool is the interview which was held in a private place. The researcher also allowed the participants to use psedonyms with the purpose of not to affect the informants.

Research tool

Date is collected by semi-structured interview protocal. It is an open interview and the answers of the participants are not limited. In the interview, the participants are also allowed to tell stories or exeriences freely. The purpose for doing this is to build the credibility of the data. 

Tape recording and note taking are used to collect the data from the interview.

Protocol of the interview

1. study relevant reserach documents to determine the framework and scope of the appropriate tools for this research

2. study the principles of creating a semi-structured interview form

3. send the tools to the thesis advisor to check

4. request a letter form to  Department of Education Administration Faculty to verify the accuracy of the tools and methods

5. Inspect the chosen reserach tools by having three experts inspectors

6. modify interview forms based on the advice of experts and professors

7. prepare the complete interview form

Data Collection

1. Data providers were selected by considering whether the person providing the information meets the requirements to be able to discuss problems and provide information on what the researcher wants to study.

2. The researcher sent a document introducing the research and asking for cooperation in data collection.

3. The researcher worked to build good relationships with the key informants in order to establish trust so that they felt comfortable to disclose information that could assist in the objectives of the researcher. 

4. In-depth interviews were conducted, each taking approximately 11/2-2 hours, to gather information in a descriptive format.

5. Date was collected from various related documents for use in data analysis according to research objectives.

6. The interview results were sent to the key informants for confirmation.

7. A summary of the research was made.

* Key informants are the people who are related to the educational management of the Child Development Center in Phadungmat Sub-district.

Data Analysis

This research analyzed data using content analysis techniques. The data was analyzed with the following steps: 

1. The dialogue from all interviews was transcribed.

2. Data management methods were used to sort the data. The researcher prepared the data by putting each type of data in a separate data file according to the characteristics of the data.

3. The researcher re-examined the information in order to consider the accuracy and completeness of the information before using the data for analysis. 

4. The researcher used a content analysis procedure for data analysis. 

5. The researcher defined the key words and ideas related to the issues that need to be studied. The definitions helped to specify a clear direction and framework for conducting research which would be easy to remember, provide consistency, and prevent the overcomplexity of data during data analysis. 

6. The researcher analyzed and presented the data by using Microsoft Word. 

7. Drawing and verifying conclusions were done to include finding patterns, grouping, factor analysis, identification of event associations, conceptual theory links and other discoveries. The success of creating a summary depends on the completeness of the data that is summarized. 

8. The researcher confirmed the correctness of the conclusion by examining the validity and representation of the data with a multi-step method. The methods included collecting data from different data sources to gather the same information, using observation along with questioning while also studying additional information from the source documents, and asking important informants to check the conclusions obtained with the informants after the conclusion of the study.

4) the research findings

The research findings revealed that the guidelines of educational management in this particular study refer to principles and methods of educational management in accordance with the Child Development Center operating standards. There are 6 principles which are listed here. 

(1) Decentralization of educational administration by participation of communities from all regions. There are 4 areas in which these methods are used. The first is that of participation in making decisions and planning, determining measures, and creating guidelines for educational management. The second area is defining various department systems. The third area is establishing a community representative committee. And the final area is the presentation of information 

(2) Participation in the development of the potential of community personnel. This involves training methods such as career skills seminars, personnel education development and building morale in people. The second evaluation of performance is by making a policy plan and checking the implementation of project activities. 

(3) Cooperative management based on mutual trust. There are 2 methods that are used to assess development and system security. 

(4) Integrating knowledge management from communities. There are 3 methods that search for learning resources and local wisdom, planning meetings for integration of local knowledge and wisdom and procuring community folk philosophers. 

(5) Positive empowerment of the community by creating an understanding of the operations of the Child Development Center by using project presentations and inviting community members to join activities of the Child Development Center. The purpose of these activities would be to develop leaders and create engaging forums where all groups can share their ideas. 

 (6) Principles of empowerment from a network outside the Child Development Center by building an attitude of confidence in the Child Development Center. Establishing a parent network can help reach these goals as well as promote and support the development of networks in the community. 

5) Strength and weakness of the research

Although it is a qualitative research, the interview questions are well-designmed with the advice from the experts who are working in Education administration department. The data collection and analysising processes are described in details. so, overall, it is a very clear reserach paper.

6) What I have learned from the research

After reading this reserach, I have realized the importance of community participation in assisting the education sector. In addition, I come to understand that Child Development Center is not just related to the nursey or primary sector only. It is related to every students who are studying for their basic education.

The School Administration Affecting the Life Skill and Career of Students in the Opportunity Expansion School, Narathiwat Province, Thailand

The School Administration Affecting the Life Skill and Career of Students in the Opportunity Expansion School, Narathiwat Province, Thailand

Maromlee Ama and Kanokkorn Sirisuk

Faculty of Education, Songkhla Rajabhat University, 

Songkhla 90000, Thailand

                                                                                              Nang Aye Aye Htun

                                                                                              Seminer on Dissertation 143400

1) Background of the research

      This is a research to study the school administration that influenced the promotion of life skills and career of students of Educational Opportunity Expansion Schools in Narathiwat Province. 200 teachers who served as heads of four divisions in Educational Opportunity Expansion Schools in Narathiwat Province, were chosen with the purposive sampling technique for the sample. The data is collected by the five-scale questionnaires with reliability of 0.952. Statistics used for this research include percentage, mean, standard deviation, and stepwise multiple regression analysis. 

2) The researh objectives

          There are 3 research obejctives.

  1. To study the school administration of the school administrators at the Opportunity Expansion School in Narathiwat Province

2. To study the promotion of life skills and careers of students at the Opportunity Expansion School in Narathiwat Province

3. To study the school administration that affects the promotion of life skills and careers of students at the Opportunity Expansion School in Narathiwat Province

My comment: 

The objectives of this reseach is very clear. Moreover, each obejctives are related to each other and they are also in consistent with the title. 

3) Research Methodology

This research is a mix-method research which is a combination of both quantitative and qualitative research. The research tool is five-scale questionnair related to school administration and the promotion of students' life skills and career. The questionnair which included a checklist form and a 5-level rating scale with 62 items, was used to collect data.

Part 1 : General information of the respondents

Part 2: Teacher opinions test about school administrators' administration   

curricula development,

 teaching management,

 measurement and evaluation, 

development and moral enhancement,

 resource Management, 

investing in education, and coordination and networking.     

Part 3 : teachers opinion questionnaire on students' life skills and careers

 flexibility and adaptability, 

 initiative and self-guiding,

 social skills and cross-cultural learning,

 productivity and accountability, and 

 leadership and responsibility

My comment:

Since the data is collected with 5 scale questionnairs, I am concerned about the data reliability. And most importantly, all the questions are answered by the teachers' opinion only.

4) Research findings

1. The Opportunity Expansion School in Narathiwat Province, through the Educational Institution Administration of School Administrators, is at a high level overall and aspects.

2. In Narathiwat Province as a whole, the promotion of students' life skills and careers in schools is at an all-time high, boosting educational opportunities. 

3. The administration of educational institute administrators in Narathiwat Province has three variables that affect the promotion of life skills and careers of students in the Opportunity Expansion School, Narathiwat Province: Coordination and networking (Net), measurement and evaluation (Eva), and the development and morale enhancing (Enc). 

5) Strength and weakness of the research

Strength : Although the data is collected by the questinnaire, the researcher tried to cover a lot of areas which are very supportive for the obejctives. The researcher also compared the results and findings with the other research papers that are similiar to it. In addition, the recommendation which the reseracher add in the paper will be a great help for the further researches.

Weakness : Since it is just questionnairs and most importantly, the answers also also based on the teachers' opinion, I am concerned about the data reliabilty. In addition, all the participants were chosen with the purposive sampling method. 

6 ) What we have learned from the research

According to the findings, the school administration play a great role for the students life skills and future careers because they are the one who have the most right in managing and arranging the school curriculm and activities.

The Development of a Community Participation Model for Organizing Lifelong Education by Educational Institutions: A Participatory Action Research

Chittwadee Thongtua,Wirathep Pathumcharoenwatthana, Noppamonton Sibmuenpiam 

Chulalongkorn University

Journal of Education Studies Volume 42 Issue 0 Special Issue  (1-1-2014) 

Nutthakarn Onsawai

 143400 Seminar On Dissertation

Community participation in education is recognized as a valuable approach to enhance educational outcomes and foster a sense of engagement within communities. Lifelong education refers to learning opportunities that extend beyond formal schooling and encompass various stages of life. It is important to understand the conditions, factors, and models that facilitate community participation in lifelong education management to create an inclusive and effective educational environment.

The research have been motivated by the need to explore and identify the factors and conditions that influence community participation, as well as to develop a model that can guide educational institutions in promoting community involvement.

1.Research Objective

The research aims to accomplish three main objectives. 

1. To study the contexts and factors of community participation education management in educational institutions. 

2. To compare contexts and factors of community participation education in educational institutions. 

3. To develop a model of community participation in lifelong community education in educational institutions. 

◦Firstly, it intends to examine the conditions and factors associated with community participation in the management of lifelong education within local educational institutions. 

◦Secondly, it seeks to compare the factors influencing community participation in lifelong education across different educational institutions. 

◦Lastly, the research aims to develop a model of community participation for lifelong education within educational institutions using a participatory action research approach.

2. Research Questions and Type of research

Research question : "What are the conditions, factors, and model of community participation for lifelong education management in educational institutions?"

Type of research : A mixed methods research design involves combining both quantitative and qualitative data collection. 

The target population for the study was selected through purposive sampling, consisting of 1,846 individuals from reputable schools in sub-districts that were part of a project initiated by the OBEC (Office of the Basic Education Commission) in 2010.

◦Additionally, 25 individuals from one of these schools were chosen to participate in the participatory action research (PAR) process. 

◦The research employed various data collection instruments, including questionnaires, in-depth interviews, and focus group discussions. 

◦Computer programs and content analysis were used to analyze the collected data.

3. The research methodology

A mixed methods research design involves combining both quantitative and qualitative data collection. 

Phase	Step	Description
Phase 1	Theoretical framework design
	1.1 Documentation	Collect information from documents, textbooks, and related materials on community participation, lifelong education, etc.
	1.2 Research design	Formulate a research design based on the theoretical framework
	1.3 Presentation to experts	Present the research design to experts for evaluation and feedback
	1.4 Adjustment and revision of research design	Revise and modify the research design based on the suggestions and feedback provided by the experts
Phase 2	Review of the contexts and factors related to community participation in lifelong education management
	2.1 Research instrument development	Develop 1-5 rating scale questionnaires with an open-ended format.
	2.2 Revision of questionnaires	Supervisor and experts review and assess the questionnaires for content validity, completion, and inclusiveness.
	2.3 Pilot study	Conduct a pilot study with 30 pilot schools to investigate comprehensive questions and answers.
	2.4 Data collection	Administer questionnaires at a high standard school in a sub-district containing one of the project schools (182 schools). Collect information on community contexts and factors related to community participation in lifelong education management. Subjects include school administrators, subject head teachers, parental representatives, alumni representatives, scholars, community leaders, occupational group representatives, and voluntary groups from the community.
	2.5 Data analysis	Perform data analysis using computer programs to calculate frequency, percentage, mean, and standard deviation (S.D.). For closed-ended questions, qualitative data will be analyzed using content analysis, conceptualization, and induction analysis to sort data into categories and draw conclusions.
Phase 3	Analysis of results and design of the model
	3.1 Analysis of results from Phase 2	Analyze the data obtained from Phase 2, which includes information on community participation in lifelong educational management. Compare the factors of community participation among different types of educational institutions (primary schools, basic educational opportunity expansion schools, and secondary schools).
	3.2 Designing the model of community participation	Based on the analysis results, design a model of community participation in lifelong educational management. This model incorporates the identified factors and provides a framework for promoting effective community involvement in educational institutions.
Phase 4	Development and evaluation of the model using PAR
	4.1 Development of the model	Refine and further develop the model of community participation in lifelong educational management based on the findings from Phase 3.
	4.2 Evaluation of the model with Participatory Action Research (PAR)	Engage stakeholders, such as educators, community members, and relevant individuals, in the evaluation process using Participatory Action Research (PAR) methodology. Present and discuss the model with the participants, gathering their feedback and suggestions. Assess the effectiveness, feasibility, and practicality of the model in promoting community participation in lifelong educational management. Revise and improve the model based on the feedback received.

4. The strength and weakness of the research

Strengths:

1. Comprehensive approach: The research employs a comprehensive approach by considering various       aspects of community participation in lifelong education management. It examines conditions, factors, and develops a model, providing a holistic understanding of the topic.

2. Participatory Action Research (PAR): The use of PAR methodology is a strength as it actively involves stakeholders, such as educators and community members, in the research process. This participatory approach can enhance the relevance and applicability of the research findings.

3. Mixed methods approach: The research utilizes a mixed methods approach, combining quantitative and qualitative data collection and analysis methods. This approach allows for a deeper understanding of the research topic by capturing both numerical data and rich contextual information.

Weaknesses:

1. Limited Context: The research focuses on a specific context and sample, which may limit the generalize of the findings. The findings may be applicable primarily to the studied educational institutions and may not be representative of other settings.

2. Lack of external validation: It does not mention external validation or verification of the research findings. External validation through independent review of the study's results could enhance the reliability and credibility of the findings.

3. Potential bias in sampling: The use of purposive sampling may introduce bias in the selection of participants. It is essential to acknowledge and address any potential biases to ensure the research findings accurately reflect the perspectives of the broader community.

6.What we have learnt from this research?

From findings..

The developed model of community participation for lifelong education within educational institutions consisted of four components: 

a)Basic conditions for community participation, which encompassed educational organizers, target groups, knowledge provision, learning activity methodology, learning materials, educational objectives, community learning resources, and the benefits received by the community from their participation. 

b)Processes involved in community participation, including engagement in studying community-based information, brainstorming, planning, management, receiving benefits, and evaluating education management. 

c)Factors influencing the participation of educational institutions and the local community. 

d) Management of lifelong education within the educational institution.

Overall, this research highlights 

◦The significance of community participation in lifelong education management, 

◦Identifies factors that influence participation, and 

◦Provides a model to guide institutions in fostering community engagement. 

It emphasizes the value of participatory research approaches in addressing complex educational challenges and promoting inclusive decision-making processes.