Student Life Assessment Project.ppt

1、Student Life Assessment Project,Module 4: Analyzing & Reporting Assessment Results, 2013 Christie Cruise-Harper, PhD All Rights Reserved,The purpose of this module is to assist you with analyzing the data you gathered for the program/service you assessed in the 2013 2014 academic year.,Student Life

2、Assessment Project,Student Life Assessment Project,In module 1 you developed learning outcomes for your program/service using Blooms Taxonomy as a guide. Module 2 allowed you to strengthen those learning outcomes and guided you through the process of choosing learning activities. Module 3 assisted y

3、ou with choosing appropriate assessment tools/methods for the learning outcomes and learning activities you established. This module will help you with analyzing and reporting the assessment data.,Student Life Assessment Project,Student Life Assessment Project,Assessment results must be analyzed to

4、determine if student learning outcomes were met. Data is analyzed for context, understanding and to draw conclusions. Analysis of data gives the information meaning.Taken from Academic Program Assessment: Tools & Techniques for Program Improvement,Analyzing Assessment Data,Determining how to organiz

5、e, synthesize, interrelate, compare and present the assessment results are all part of analyzing the data. Assessment data can be compared to findings from previous assessments, baseline data and existing criteria.Taken from Academic Program Assessment: Tools & Techniques for Program Improvement,Ana

6、lyzing Assessment Data,Quantitative Also known as “empirical research” Refers to any research based on something that can be accurately and precisely measured. Collects numerical data in order to explain or predict a particular phenomena.Taken from University of Wisconsin Madisons Ebling Library & R

7、. Ouyangs Basic Inquiry of Quantitative Research For more information visit: http:/researchguides.ebling.library.wisc.edu/content.php?pid=325126&sid=2940225,Types of Data,Qualitative Refers to any research based on something that cannot be accurately and precisely measured. Collects narrative data t

8、o gain insights into a particular phenomena.Taken from University of Wisconsin Madisons Ebling Library & R. Ouyangs Basic Inquiry of Quantitative Research For more information visit: http:/researchguides.ebling.library.wisc.edu/content.php?pid=325126&sid=2940225,Types of Data,Analyzing Quantitative

9、Data,Its all about the numbers!,There are four types of quantitative research methods: Descriptive: collecting data for hypotheses testing* Correlational: determining whether and to what degree a relationship exists Cause-Comparative: establishing the cause-effect relationship Experimental: establis

10、hes the cause-effect relationship, but manipulates the causeSee notes section for more detailed information.Taken from R. Ouyangs Basic Inquiry of Quantitative Research,Types of Quantitative Research,The following are common methods of data collection in quantitative research: Surveys and Questionna

11、ires Structured Interviews Observation or Interaction Analysis Secondary Data or Content Analysis ExperimentsSee notes section for more detailed information.,Methods of Data Collection,In quantitative research there are two ways in which data are analyzed: Descriptive Statistics Procedures used to d

12、escribe a given collection of data. The purpose is to describe the sample at hand-the collection of cases that we have examined. Inferential Statistics Procedures that let us generalize our findings beyond the particular sample at hand to the larger population represented by that sample.Taken from D

13、iekhoff, G.M.(1996). Basic Statistics for the Social and Behavioral Sciences.,Data Analysis,Most Student Life assessment projects do not seek to generalize its findings to the entire Maryville University student body. Because our goal is to learn about the sample at hand, descriptive statistics will

14、 be the focus of the quantitative data analysis for module 4. There are three types of descriptive statistics to provide you with an overview of your data: Central Tendency Measures* Variability Measures Frequency and Percentages*,Descriptive Statistics,More on descriptive statistics later. Before b

15、eginning any data analysis, you must first identify the level of measurement associated with your quantitative data. There are four levels of measurement: Nominal Ordinal Interval Ratio (Scale),Levels of Measurement,Nominal: basic classification data; do not have meaningful numbers attached to them,

16、 but are broader categories Ordinal: have numbers attached to them and the numbers are in a certain order, but there are not equal intervals between the numbers Interval: have equal intervals between the numbers; the distance between attributes have meaning Ratio: have equal intervals between the nu

17、mbers; there is an absolute zero that is meaningfulTaken from: http:/www.uni.edu/commstudies/researchmethods/chapterfour1.html,Levels of Measurement,Once you have decided on your data collection method, decided on the level of measurement for your variables, and collected the data, you are ready to

18、begin analyzing the data. There are two software programs I recommend and they are available on campus: Qualtrics SPSS (Statistical Package for the Social Sciences),Data Analysis and Software,The following quantitative data analysis procedures are used to describe the data and can be done in Qualtri

19、cs and SPSS: Data Tabulation (Frequency Distributions, Percentiles) Descriptive Data (Central Tendency) Data Disaggregation Moderate and Advanced Analytical MethodsFor a detailed description of these analyses visit: http:/toolkit.pellinstitute.org/evaluation-guide/analyze/analyze-quantitative-data/

20、OR Chapters 1-3 in Diekhoff s Basic Statistics for the Social and Behavioral Sciences,Data Analysis and Software,Surveys and questionnaires can be developed in Qualtrics and the data can be analyzed within the software. There are tutorials to assist you with analyzing your data. Crosstabs: http:/ Un

21、derstanding Statistics: http:/ Below are several links to resources to assist you with using SPSS. http:/ http:/ http:/ http:/calcnet.mth.cmich.edu/org/spss/toc.htm,Data Analysis and Software,In addition to a narrative about your data analysis, quantitative results are presented in the following way

22、s: Charts Graphs Tables The following websites provide examples, in APA, of how quantitative results are presented: https:/owl.english.purdue.edu/owl/resource/560/19/ https:/owl.english.purdue.edu/owl/resource/560/20/,Reporting Quantitative Results,Tables and graphs can be created in Qualtrics. For

23、more information visit: http:/ http:/ Graphs can also be created in SPSS. For directions visit: http:/julius.csscr.washington.edu/pdf/spss.pdf http:/academic.udayton.edu/gregelvers/psy216/spss/graphs.htm http:/www.ats.ucla.edu/stat/spss/seminars/SPSSGraphics/spssgraph.htm,Reporting Quantitative Resu

24、lts,Analyzing Qualitative Data,I want to know how you feel!,It is important to achieve empathic understanding to comprehend the participants experience with a minimum of distortion or bias; The researcher must attempt to recognize their own personal prejudices, stereotypes, myths, assumptions and ot

25、her thoughts or feelings that may cloud the perception of other peoples experiences; Knowledge of others experience cannot be assumed regardless of familiarity with their subcultural landscape; and Do not expect participants to hold the same values as you.M. Ely, M. Anzul, T. Friedman, D. Garner, A.

26、M. Steinmetz in Doing Qualitative Research: Circles within Circles,Before Analyzing Qualitative Data,Qualitative Research Methodologies Basic Interpretive Qualitative Study* Grounded Theory Phenomenology Case Study Ethnography Postmodern Research Critical Qualitative Research Narrative AnalysisFor m

27、ore information visit: http:/www.fctl.ucf.edu/researchandscholarship/sotl/creatingsotlprojects/implementingmanaging/qualitativeresearchtypes.php,Types of Qualitative Research,Participant Observation Interviewing (formal & informal) Focus Groups Document Analysis Logs (notes/reflections from observat

28、ions and interviews) Audio and videotaping On-going data analysisM. Ely, M. Anzul, T. Friedman, D. Garner, A.M. Steinmetz in Doing Qualitative Research: Circles within Circles,Methods of Data Collection,Coding is the process of organizing data into chunks or segments before making meaning of the inf

29、ormation. Coding involves taking text data or pictures gathered during data collection, segmenting it into categories, and labeling those categories with a term, often a term used by the actual participant.Taken from: Creswell, J.W.(2009). Qualitative, Quantitative and Mixed Methods Approach, chapte

30、r 9, Qualitative Procedures.,Coding and Categorizing,Begin the coding process by first reviewing your learning outcomes as a reminder of what you are assessing. Your coding scheme will be based on your learning outcomes. For example: Undergraduate students who participate in the Multicultural Schola

31、rs Program will be able to describe their talents, strengths and social group memberships. Coding involves assigning a word, phrase, number or symbol to each coding category. For example: describe talents and strengths,Coding Qualitative Data,Codes can be pre-set or emergent. You should have both. P

32、re-set: A list of codes created in advance by the researcher based on the research question, learning outcomes, or conceptual framework. Emergent: Ideas, concepts, actions and meanings that come up from reading and analyzing the data that are not in the pre-set codes. Coding will serve as a system t

33、o help you to organize your data.For more information and an example of coding visit: http:/programeval.ucdavis.edu/documents/Tips_Tools_18_2012.pdf,Coding Qualitative Data,Once you have gone through all documents and coded them, they can now be gathered into families of codes or categories. Materia

34、ls are sorted by these categories, identifying similar phrases, patterns and relationships. As you code and categorize the data, look for the interrelationships among categories. Sorted materials are examined to isolate meaningful patterns. Identified patterns are used to create themes.,Creating Cat

35、egories,It is best to start any report of qualitative results with an overview of how data were processed and coded. The results are presented as “findings”. The findings are organized by themes with substantial evidence that links to the themes included in the findings (i.e. quotes from participant

36、s).,Reporting Qualitative Results,Using the Data,So now what?,Closing the loop - using the assessment results for program improvement.,Closing the Loop,The findings from your quantitative and/or qualitative research will yield rich information that will be included in your assessment report. These f

37、indings will provide you with insights into whats working with your program/service and areas of improvement. From these findings you will make recommendations for improvement of your program/service.,Closing the Loop,Program Improvement,You may find it helpful to create a program improvement plan t

38、o transform the recommendations made into actions for improvement.,Accreditation Planning and budgeting Maryville University requirements Student Life requirements Program promotion/marketing Recruitment/retention initiatives Publications Conference presentations Student development opportunities Pr

39、ofessional development opportunities Grant applicationsTaken from: Academic Program Assessment: Tools & Techniques for Program Improvement,Other Uses of Assessment Results,Academic Program Assessment: Tools & Techniques for Program Improvement.(2013). SUNY. Analyzing Assessment Data.(2013). http:/ww

40、w.sunyorange.edu/assessmentapa/docs/Analy zingandUtilizingAssessmentData.pdf. State University of New York (SUNY) Orange County Community College. Basic Inquiry of Quantitative Research.(n.d.). http:/ksumail.kennesaw.edu/rouyang/ED- research/details.htm. Kennesaw State University. Center for Evaluat

41、ion and Research.(2012). http:/programeval.ucdavis.edu/documents/Tips_Tools_ 18_2012.pdf. University of California at Davis.,References,Creswell, J.W.(2009). Qualitative, Quantitative and Mixed Methods Approach, chapter 9, Qualitative Procedures. Diekhoff, G.M.(1996). Basic Statistics for the Social

42、 and Behavioral Sciences. Differences Between Quantitative and Qualitative Research. (2013). http:/researchguides.ebling.library.wisc.edu/conte nt.php?pid=325126&sid=2940225. University of Wisconsin-Madison-Health Sciences Ebling Library Ely, M., Anzul, M., Friedman, T., Garner, D., Steinmetz, A.M.(

43、1991). Doing qualitative research: Circles within circles. RoutledgeFalmer: New York.,References,Faculty Center for Teaching and Learning.(2013). Types of qualitative research: Explained within a SOTL framework. http:/www.fctl.ucf.edu/researchandscholarship/sot l/creatingsotlprojects/implementingman

44、aging/quali tativeresearchtypes.php. University of Central Florida. Gathering Data and Assessing Results.(2013). http:/nnlm.gov/evaluation/guide/stage 5.pdf. National Network of Libraries of Medicine. The Pell Institute and Pathways to College Network.(2013). Evaluation Toolkit. http:/toolkit.pellinstitute.org/evaluation- guide/analyze/analyze-quantitative-data/ University of Northern Iowa.(2013). Communication Studies Research Methods Chapter Four. http:/www.uni.edu/commstudies/researchmethods/chapte rfour1.html,References,