Part 3 - Comprehensive Standards
  

3.3.1.1  The institution identifies expected outcomes, assesses the extent to
which it achieves these outcomes, and provides evidence of
improvement based on analysis of the results in educational programs, to include student learning outcomes.  (Institutional effectiveness - educational programs)

Compliance
Partial Compliance
Non-Compliance

Narrative

Background

Upon notification in July 2012 by SACSCOC that the University was moving to Monitoring Status, UNO has engaged in a complete retooling of the institutional effectiveness process.  The first Monitoring Report, submitted April 2013, focused on the key components of the revised Institutional Effectiveness system and described the implementation timeline associated with the adoption of the WEAVE software to track institutional effectiveness activities in the 2013-14 academic year. A second Monitoring Report, submitted April 2014, focused on Standard 3.3.1.1 with content related to institutional effectiveness activities for educational programs.

The second Monitoring Report was organized to respond to each area included in the directions provided in the July 2013 letter from the Commission.  The first section addressed the four areas cited: 1) identification of outcomes, including student learning outcomes, 2) assessing the extent to which it achieves those outcomes, 3) making improvements based on analysis of results for its educational programs, and 4) describing the procedures used to ensure that the sample included in the response is representative.  For clarification purposes, a description of the sampling technique was presented prior to a response for other areas of concern.  The second section of the report presented a brief summary of progress and a University action plan.  The final section of the document provided an overview of the improvements incorporated in the UNO system of institutional effectiveness since being placed on monitoring status for Standard 3.3.1.1. as well as a petition for continuation of accreditation for Good Cause.  In June 2014, the University was notified by SACSCOC that it had been removed from monitoring status.

Description of Sample

Two techniques are used to present evidence for this standard.  Student learning outcomes, department outcomes and closing the loop summative data are presented for all academic programs offered by the University of New Orleans. A sample is used to derive examples that demonstrate assessment strategies, findings and specific closing the loop actions.

The sample of academic programs was selected in the following manner.  A comprehensive list was developed of all degree programs at the undergraduate and graduate levels (See Table 1).  Approximately one third of bachelor, master’s, and doctoral programs were randomly selected from each of the five academic colleges in the University.  In cases where fewer than three programs were offered, one program was randomly selected.  Table 2 provides a summary of the number of programs from each college included in the sample.  Note that programs with concentrations (30% to 49% of major coursework is specialized) are counted as a single program for institutional effectiveness purposes, while options (50% or more of major coursework is specialized) are treated separately. 

In Fall 2013, UNO received approval to begin offering two programs via distance education:  the BA in Philosophy and the MS in Hotel, Restaurant and Tourism.  These programs assess the same student learning outcomes as the traditional programs currently offered.  Enrollment has just begun in these programs; thus, no data are included in the sample.  UNO does not offer any programs of study exclusively at off-site locations.

Table 1.  Summary of All Degree Programs by College

Level Business Admin Education and Human Development Engineering Liberal Arts Sciences Interdisciplinary Studies
Bachelor 6 5 4 15 7 1
Master's 5 6 2 14 7  
Doctoral 1 4 1 2 3  
Total 12 15 7 31 17 1
Grand Total   83

 

Table 2.  Summary of Number of Degree Programs by College Included in the Sample

Level Business Admin Education and Human Development Engineering Liberal Arts Sciences Interdisciplinary Studies
Bachelor 2 2 1 5 2 1
Master's 1 2 1 5 2  
Doctoral 1 1 1 1 1  
Total 4 5 3 11 5  
Grand Total   29

 

Identification of Expected Outcomes

In July 2012, the University was notified that a SACSCOC team reviewing the Fifth-Year Interim Report (submitted June 2011) had noted concerns regarding the “mingling” of department and student learning outcomes.  The Institutional Effectiveness Committee assessed the status of outcomes published by academic departments and concurred with the findings of the SACSCOC review team.  The Office of Institutional Effectiveness, in conjunction with the Institutional Effectiveness Committee, made a deliberate decision to separate academic program and academic department plans to address the issue of “mingled” learning outcomes.  Academic degree program plans now focus exclusively on student learning outcomes assessment.  Academic department plans focus on other program outcomes including recruitment and retention, teaching evaluations, scholarly activity, service, professional development, online course development, alumni outreach, as well as professional, industry, and community partnerships.

Student Learning Outcomes

The Institutional Effectiveness Committee assessed the quality of student learning outcomes published by academic departments and determined a need for professional development.  Beginning Fall 2012, professional development sessions were held with each academic department campus wide to discuss the purpose and development of student learning outcomes.  Following the initial professional development sessions, information on student learning outcome development, quality indicators, and examples of appropriate student learning outcomes for multiple disciplines were disseminated via the University Institutional Effectiveness website to serve as a resource for faculty and administrators.

Following professional development sessions, all academic departments revised or developed student learning outcomes for each academic degree program offered.  The student learning outcomes were reviewed by the Institutional Effectiveness Committee as part of the institutional effectiveness plan review process which occurs every spring semester.  Table 3 presents the student learning outcomes for all 83 degree programs offered at the University of New Orleans.  The student learning outcomes have been loaded into WEAVE, published in the appropriate degree sections of the 2014-15 Catalog, as well as on program websites.

Table 3.  Student Learning Outcomes for all Degree Programs

College of Business Administration
  BS Business Administration Student Learning Outcomes    
  MBA Student Learning Outcomes
Accounting BS Accounting Student Learning Outcomes
  MS Accounting Student Learning Outcomes
  MS Tax Accounting Student Learning Outcomes
Economics and Finance BS Finance Student Learning Outcomes
  MS Financial Economics Student Learning Outcomes
  PhD Financial Economics Student Learning Outcomes
Hotel, Restaurant, Tourism BS HRT Management Student Learning Outcomes
  MS HRT Management Student Learning Outcomes
Management BS Management Student Learning Outcomes
  MS Healthcare Management Student Learning Outcomes
Marketing and Logistics BS Marketing Student Learning Outcomes
College of Education and Human Development
Curriculum and Instruction BS Early Childhood Education    Student Learning Outcomes         
  BS Elementary Education Student Learning Outcomes
  BS Secondary Teaching Student Learning Outcomes
  MAT Curriculum & Instruction Student Learning Outcomes
  MEd Curriculum & Instruction Student Learning Outcomes
  PhD Curriculum & Instruction Student Learning Outcomes

Educational Leadership,

Counseling, and Foundations

BS Human Performance and

Heath Promotion

Student Learning Outcomes
  MEd Counselor Education Student Learning Outcomes
  PhD Counselor Education Student Learning Outcomes
  MEd Educational Leadership Student Learning Outcomes
  PhD Educational Leadership Student Learning Outcomes

Special Education and

Habilitative Services

BS Elementary Education -

Integrated/Merged Option

Student Learning Outcomes
  MAT Special Education Student Learning Outcomes
  MEd Special Education Student Learning Outcomes
  PhD Special Education Student Learning Outcomes
College of Engineering
  MS Engineering      Student Learning Outcomes                    
  MS Engineering Management Student Learning Outcomes
  PhD Engineering and Applied Science Student Learning Outcomes
Civil and Environmental                    BS Civil Engineering Student Learning Outcomes
Electrical BS Electrical Engineering Student Learning Outcomes
Mechanical BS Mechanical Engineering Student Learning Outcomes

Navel Architecture and  

Marine Engineering

BS Naval Architecture & Marine

Engineering

Student Learning Outcomes
College of Liberal Arts
  BA International Studies Student Learning Outcomes                             
Anthropology                                              BA Anthropology Student Learning Outcomes
Arts Administration MA Arts Administration Student Learning Outcomes
English BA English Student Learning Outcomes
  MA English Student Learning Outcomes
 

MFA Creative Writing Workshop

Student Learning Outcomes
Film and Theatre BA Film & Theatre – Theatre Arts Student Learning Outcomes
  BA Film & Theater – Writing for Production Student Learning Outcomes
  BA Film Arts Student Learning Outcomes
  MFA Film Production Student Learning Outcomes
  MFA Theatre Design Student Learning Outcomes
  MFA Theatre Performance Student Learning Outcomes
Fine Arts BA Fine Arts – Art History Student Learning Outcomes
  BA Fine Arts – Studio Art Student Learning Outcomes
Foreign Languages BA Romance Languages Student Learning Outcomes
  MA Romance Languages Student Learning Outcomes
History BA History Student Learning Outcomes
  MA History Student Learning Outcomes
Music BA Music Studies Student Learning Outcomes
  MM Music Student Learning Outcomes
Philosophy BA Philosophy Student Learning Outcomes
Planning and Urban BS Urban Studies & Planning Student Learning Outcomes
  MS Urban Studies Student Learning Outcomes
  MURP– Urban & Regional Planning Student Learning Outcomes
  Ph.D. Urban Studies Student Learning Outcomes
Political Science BA Political Science Student Learning Outcomes
  MA Political Science Student Learning Outcomes
  MPA Public Administration Student Learning Outcomes
  PhD Political Science Student Learning Outcomes
Sociology BA Sociology Student Learning Outcomes
  MA Sociology Student Learning Outcomes
College of Sciences
Biological Sciences BS Biological Sciences                             Student Learning Outcomes                    
  MS Biological Sciences Student Learning Outcomes
  PhD Integrative Biology Student Learning Outcomes
Chemistry BS Chemistry Student Learning Outcomes
  MS Chemistry Student Learning Outcomes
  PhD Chemistry Student Learning Outcomes
Computer Science BS Computer Science Student Learning Outcomes
  MS Computer Science Student Learning Outcomes
Earth and Environmental Sciences BS Earth & Environmental Science Student Learning Outcomes
  MS Earth & Environmental Science Student Learning Outcomes
Mathematics BS Mathematics Student Learning Outcomes
  MS Mathematics Student Learning Outcomes
Physics BS Physics Student Learning Outcomes
  MS Applied Physics Student Learning Outcomes
Psychology BS Psychology Student Learning Outcomes
  MS Psychology Student Learning Outcomes
  PhD Psychology Student Learning Outcomes
Interdisciplinary Studies
                                                                BIDS Interdisciplinary Studies                     Student Learning Outcomes                          

 

Other Academic Outcomes

In addition to identifying student learning outcomes for each program, each academic department identified outcomes as part of the annual institutional effectiveness process.  Departments were instructed to align their outcomes with the University strategic plan.  These outcomes typically focus on recruitment and retention, teaching evaluations, scholarly activity, service, professional development, online course development, alumni outreach, as well as professional, industry, and community partnerships.  Academic department outcomes and University Strategic Plan by Association Reports are presented by college in Table 4.

Table 4.  Academic Department Outcomes and Strategic Plan Associations

College Department Outcomes Strategic Plan Associations
College of Business Administration Department Outcomes Strategic Plan Associations
College of Education and Human Development Department Outcomes Strategic Plan Associations
College of Engineering Department Outcomes Strategic Plan Associations
College of Liberal Arts Department Outcomes Strategic Plan Associations
College of Sciences Department Outcomes Strategic Plan Associations
Interdisciplinary Studies Department Outcomes Strategic Plan Associations

 

The Institutional Effectiveness Committee developed new plan templates to include all critical components for institutional effectiveness planning and to align with the implementation of WEAVE.   The templates included the following components:  mission, student learning outcome (academic degree program) or strategic planning goals (academic department), measure, target, findings, and closing the loop/action plans.  Rubrics aligned with the templates were developed so that the Institutional Effectiveness Committee could evaluate plans for consistency and quality.  The Institutional Effectiveness Committee used the rubrics to identify areas of strength and areas where improvements were needed for each academic degree program. 

Institutional effectiveness plans submitted by faculty were reviewed by the Institutional Effectiveness Committee and in some cases revisions were required.  Table 5 summarizes the common errors associated with the development of student learning outcomes, assessment methods, and targets as well as the feedback provided by the committee.   Professional development was provided to all academic departments in three ways:  1) sessions were organized at the college and department levels, 2) ongoing support was provided via individualized consultation sessions, and 3) resources were loaded on the institutional effectiveness website to provide guidance for developing quality student learning outcomes, assessment methods and targets.  Faculty revised and resubmitted plans based on feedback received.

Assessing the Extent to which Academic Outcomes are Achieved

The Achievement Status Detail report provides information regarding the status of each program’s targets, including raw numbers and percentages of “met,” “partially met,” or “not met.”  Data related to targets with missing findings and those not reported in the 2013-14 institutional effectiveness cycle are also included.

Findings

Findings for student learning outcomes associated with individual degree programs were recorded for the fall and spring semesters in WEAVE.  Academic programs were strongly encouraged to upload supporting documentation in the form of spreadsheets or other artifacts into the Document Management repository feature of WEAVE.  This ensures that the data used to support the narrative summaries of the findings are readily available for review by faculty and administrators.  These documents may be linked to the individual findings within WEAVE, as well as within any report run from WEAVE.  Of the 29 academic programs included in the sample, 15 uploaded raw data or summaries of their data, while 14 programs did not upload supporting documentation into the WEAVE repository.

Evidence of Improvement Based on Analysis of the Results

A sample of institutional effectiveness plans prepared by academic programs is included.  As stated previously, the sample was selected in the following manner.  A comprehensive list was developed of all degree programs at the undergraduate and graduate levels.  Approximately one third of bachelor, master’s, and doctoral programs were randomly selected from each of the five academic colleges in the University.  In cases where fewer than three programs were offered, one program was randomly selected.  Note that programs with concentrations are counted as a single program for institutional effectiveness purposes, while options are treated separately.  Table 8 presents Detailed Assessment Reports for all sampled academic degree programs.

In Fall 2013, UNO received approval to begin offering two programs via distance education:  the BA in Philosophy and the MS in Hotel, Restaurant and Tourism.  These programs assess the same student learning outcomes as the traditional programs currently offered.  Enrollment has just begun in these programs; thus, no data are included in the sample.  UNO does not offer any programs of study exclusively at off-site locations.

Table 6 provides a sample of Detailed Assessment Reports from WEAVE.  Detailed Assessment Reports include outcomes, measures and targets as well as related findings and closing the loop/action plans.

Table 6.  Sample of Detailed Assessment Reports from WEAVE

College of Business Administration
Level Baccalaureate Master’s Doctoral
    MBA  
Accounting BS    
Economics and Finance     PhD

Hotel, Restaurant, Tourism

BS    
College of Engineering
Level                                        Baccalaureate Master's     Doctoral
Engineering   MS  

Engineering and Applied

Science

    PhD
Mechanical Engineering BS    
College of Education and Human Development
Level                                                         Baccalaureate          Master's             Doctoral       
Counselor Education   MEd     
Early Childhood Education BS    
Educational Administration     PhD
Secondary Teaching, Grades 6-12 BS    
Special Education   MEd  
College of Liberal Arts
Level Baccalaureate Master's Doctoral
Anthropology BA    
Arts Administration   MA  
Creative Writing Workshop   MFA  
Fine Arts – Studio Arts Option BA    
History BA MA  
Music   MM  
Philosophy BA    
Political Science     PhD
Romance Languages BA MA  
College of Sciences
Level Baccalaureate Master's Doctoral
Biological Sciences BS    
Chemistry BS   PhD
Earth and Environmental Sciences   MS  
Mathematics   MS  
Interdisciplinary Studies
Level Baccalaureate Master's Doctoral
Interdisciplinary Studies                   Bachelor    

 

Closing the Loop

Closing the loop is the most important step in the institutional effectiveness cycle – it involves analyzing data and then modifying strategies as needed to better achieve measurable objectives.  An initial problem that surfaced with this part of the institutional effectiveness process was conveying to all academic programs that even though a target was met, an action plan was still required.  Ongoing support was provided via consultation and resources disseminated on the institutional effectiveness website to provide guidance for developing quality closing the loop action plans.

Table 7 presents a summary of the types of closing the loop/action plans documented by all academic programs offered by the University.  The closing the loop efforts of academic programs based on Fall 2013 data were analyzed by the Institutional Effectiveness Committee and organized into the following eight categories:  curriculum or course content, assignment or application, pedagogy, assessment methods, personnel, target, maintenance, and other. 

Table 7.  Academic Program Closing the Loop/Action Plan Types (Fall 2013)

College Curriculum/Content Assignment/Application Pedagogy Assessment Methods Personnel Target Maintain Other
Business Administration 1 4 6 13 1 8 6 12
Education and Human Development 0 4 0 16 0 3 23 4
Engineering 3 1 0 12 0 2 0 4
Liberal Arts 9 4 12 58 0 9 63 14
Sciences 11 10 1 17 1 17 22 23
Interdisciplinary Studies 2 1 0 4 0 0 1 1
Totals 26 24 19 120 2 39 115 58

The following tables provide examples of closing the loop/action plan efforts included in sample plans.  The tables are organized by closing the loop category.  Examples of data and/or other documentation are provided to demonstrate how changes are documented.

Data collection and maintenance for individual academic programs are managed at the department level.  For academic year 2013-14, departments were not required to centralize the storage of data.  Based on the fact that not all data were housed in WEAVE, the Institutional Effectiveness Committee will now require data summaries related to findings to be uploaded in WEAVE effective the 2014-15 academic year.  Evidence of curriculum and course changes is provided via proposal approval by the University Courses and Curricula Committee.  Of the curriculum and course content examples described below, the Interdisciplinary Studies (IDS) and Chemistry curricula changes have already been approved for Fall 2014 implementation.

Closing the Loop – Category 1:  Curriculum and Course Content

Table 8.  Closing the Loop Examples –Interdisciplinary Studies, Bachelor

Issue Data Source Improvement
No defined required percentage of upper division courses in integrative learning plan

Academic program degree requirements review mandated by Academic Affairs.

2014-15 Catalog revised to reflect minimum percentage of upper division courses in integrative learning plan.

Students require more time and assignments to develop ability to integrate discipline via research.

Results from faculty-juried capstone paper in IDS 3091 course. Develop IDS 4091 course on integrative learning theories and practices to help students conduct research in the field of interdisciplinary studies. More substantial assignments including midterm, oral presentation in front of peers, capstone paper that requires integrating and citing research. Course proposal Fall 2014, course implementation Spring 2015.

 

Table 9.  Closing the Loop Examples – Chemistry, BS

Issue Data Source Improvement

Logistics surrounding ability for program to administer the ETS Major Field Test in Chemistry

ETS Major Field test measure not administered

Developed capstone assessment course and added to curriculum; approved by University Courses and Curricula Committee

Effective Spring 2014, all BS students will be required to enroll in CHEM 4000 to take the ETS Major Field Test prior to graduation

Difficulty assessing students’ overall satisfaction with program and perceived level of preparedness for advanced study

Exit survey

All BS students will be required to enroll in CHEM 4000 and take the exit survey prior to graduation

 

Table 10.  Closing the Loop Examples – College of Education and Human Development

Issue Data Source Improvement
Lack of resources (faculty and budget lines) Program review conducted by Academic Affairs in conjunction with programs and colleges

Combining Curriculum and Instruction and Special Education PhD programs

Lack of resources (faculty and budget lines) Program review conducted by Academic Affairs in conjunction with programs and colleges Dropping Early Childhood Education BS program

 

Closing the Loop – Category 2:  Assignment and Application

Table 11. Closing the Loop Examples – Fine Arts, BA, Studio Art Option

Issue Data Source Improvement

Students’ ability to demonstrate an understanding of the formal elements, processes, and critical issues in studio practice, particularly in their area of specialization.

Final research paper in Art History courses.

Revise assignment descriptions to clarify project intents and objectives as well as resources students can find on campus and online to assist in writing. Survey art history courses will focus on writing now that the ENGL 1158 prerequisite has been removed, which may be one reason students were struggling more with this assignment over the last year.

Only 60% of students were able to demonstrate their understanding of their own art practice through prepared statements presented to the faculty in their capstone course.

Students are required to write artist statements to demonstrate their own understanding of the historical context of their creative work, as well as technical issues in creating the work. Creating directed instruction at every level of curriculum related to the production of the artist statement.  This requires more attention to contemporary art resources and additional classwork in most fine art classes. This is very important and can be challenging for students who don't feel comfortable speaking publicly about their work. We have developed specific classes such as professional practice to address this topic.

 

Table 12.  Closing the Loop Examples – Mathematics, MS

Issue Data Source Improvement
Incorporating oral presentations in online courses and courses with large enrollment Oral presentation results Faculty will discuss logistics of requiring this assignment

Not all graduate level courses require written reports on research projects

Grades on written presentations All graduate students in 5000- or 6000-level classes will submit a written presentation on a research project.

 

Closing the Loop – Category 3:  Pedagogy

Table 13.  Closing the Loop Examples – Accounting, BS

Issue Data Source Improvement

Students’ ability to demonstrate a proficiency in financial accounting and governmental accounting concepts was not met in some areas.

Examinations at the end of the courses (ACCT 3121, 3122, 3124).

Find new and interesting ways to present material; handouts and reviewing problems; greater emphasis on Financial Statement presentation; greater emphasis and repetition on Current Assets; more class time for practice; implement out of class small project

Students ability to demonstrate a proficiency in individual income taxation concepts.

Target not met in Deductions, Income and Property transactions area.
Examinations at the end of course (ACCT 3152). More handouts; ask questions to determine whether the students have a good understanding of the material covered in class; review information; try other exercises, to maintain the competency in the Compliance area and have the objectives met in the other three areas (Deductions, Income, and Property transactions).

 

Table 14.  Closing the Loop Examples – History, BA

Issue Data Source Improvement

History majors’ ability to produce scholarly material by engaging with both primary and secondary sources.

Target not met, great variety of written expectations across History courses.

Numerous portfolios did not address all the elements (synthesis of historical information, primary source analysis, clear writing, and ability to design a historical argument)

Review of portfolio with rubric (History 3001)

Greater coordination and agreement for writing assignments and expectations in our upper-level history course work.

The history faculty will develop clearer expectations for student learning outcomes in upper-level history courses.

 

Table 15.  Closing the Loop Examples – Biological Science, BS

Issue Data Source Improvement
Fundamental concepts

Students complete the SALG survey instrument (www.salgsite.org) after completing BIOS 1071.                   

Teaching workshops to improve pedagogy, incorporate newer teaching techniques that emphasize scientific connections versus facts.  E.g., Flipping the classroom to allow more data analysis and thought questions to be posed.

 

Closing the Loop – Category 4:  Assessment Methods

Table 16.  Closing the Loop Example – Creative Writing Workshop, MFA

Issue Data Source Improvement

Students’ ability to learn to identify characteristics of English grammar, mechanics, and usage and apply them to their own and to the texts of others.

All graduates will take at least four workshops in their genre. In each workshop, students will write three or more original examples of the creative writing art form they are practicing and will write 300-600 word critiques of all

their peers’ work. Their work will improve both through production and analysis. As a capstone, all graduates must write a book-length thesis in their genre.

Need standardized method of assessment to facilitate data collection.
Theses judged by faculty committee, including oral defense.

Develop a rubric to improve assessment of the MFA thesis.

Although we continue to be impressed by our students' high level of achievement through a rigorous curriculum, as well as their success in publishing after they leave our program, we plan to construct a rubric, built by faculty for their specific genres, to help standardize and assist us in evaluating both the written theses and the oral thesis defenses. This should help ensure that the various committees are holding all students to the same standard.

Students are completing comprehensive exam within four year time frame, but there is no standardized method of assessing the exam.  Also, more useful data is needs to be obtained for program improvement purposes.

Required comprehensive exam to be evaluated by a faculty committee.

Develop a rubric to help improve assessment of MFA comprehensive examination.

Construct a rubric, built by faculty for their specific genres, to help standardize and assist us in evaluating the comprehensive exam.

This should help ensure that the various committees are holding all students to the same standard.

We plan to use this rubric to assess all comprehensive exams beginning in the fall semester of 2014.

Faculty will turn in this rubric to the program director upon completion.

 

Table 17.  Closing the Loop Example – Biological Sciences, BS

Issue Data Source Improvement
Students ability to achieve satisfactory adjustment to the University learning environment.

Students complete the SALG survey instrument (www.salgsite.org) after completing BIOS 1071.

Evaluate the utility of self-reported gains in attitude which may be subject to inflated self- assessments; that is, some students may perceive themselves as already being well-adjusted to the University learning environment and feel that they did not make gains in attitude during the semester. This course may be particularly subject to this perceptual bias since many of the students do not take the course until their second or third semester, by which time they may feel adequately adjusted. Whether or not they are actually well-adjusted, we need to determine if the low scores on this assessment are in part due to students prior feelings of being well- adjusted.

We will develop a pre-course attitude survey to administer during Fall 2014. If many students report low gain in positive attitude and a high pre-course self assessment, then this assessment may not be of significant value and may be dropped.

Target not met relating to students’ ability to acquire fundamental concepts in the fields of organismal biology, evolutionary biology, genetics, and ecology.

Students complete the SALG survey instrument (www.salgsite.org) after completing BIOS 1071.

Investigate the possibility of revising laboratory quizzes to include more cumulative knowledge. Weekly quizzes currently focus only on the current and previous week's laboratories, and adding questions aimed at cumulative review may help to promote a more comprehensive view of the material during the semester.

 

Closing the Loop – Category 5:  Personnel

Table 18.  Closing the Loop Example – Biological Sciences, BS

Issue Data Source Improvement
Students ability to learn core concepts of cellular and molecular biology Pre- and post-course testing in BIOS 2114 using the IMAC Cell and Molecular Biology instrument We achieved our target but want to see if this can be maintained with a different instructor for the course.

 

Closing the Loop – Category 6:  Target

Table 19.  Closing the Loop Example – Educational Administration, PhD

Issue Data Source Improvement

Four out of four students with scheduled dissertation defenses successfully completed.   While this is a strong 100% goal attainment rate, this number is relatively small compared to the nearly 70 students in the PhD program.

Dissertation: all graduates will complete a research project of publishable quality by graduation as judged by a jury of faculty members.

We would like to look for an increase in this number. While this is due somewhat to the fact that 2 of 3 program faculty from Spring 2013 left the University.  With the addition of a new faculty member in the Fall 2013, and a second tenure-track faculty member expected to begin in Fall 2014, an increased number of PhD graduates should be attainable.

 

Table 20.  Closing the Loop Example – Earth and Environmental Sciences, MS

Issue Data Source Improvement
Exceeded previous target of 80% of the students will achieve a B or better Students in 6000-level graduate courses will be required to submit a minimum of two oral PowerPoint presentations regarding a topic of relevance to the course.

Target revised to 85% of the students will achieve a B or better

Exceed target of at least 2.8 out of 5.0 for overall quality of course.

Course evaluations in 4000G and 6000 graduate level classes to demonstrate satisfaction with the program. All concentration level and graduate level courses required of the degree will achieve at least 3.0 out of 5.0 for overall quality of course.

 

Closing the Loop – Category 7:  Maintain

Table 21.  Closing the Loop Example – Anthropology, BA

Issue Data Source Improvement

100% of graduating senior anthropology majors in ANTH 4995 in the spring 2013 semester received passing assessments from the faculty jury.

Students in their senior year of study will present an in-class presentation (in ANTH 4995), to be assessed by a faculty jury.

We will continue to use a faculty jury to review, with the appropriate rubric we have developed for this purpose, student presentations. Data we collect in the spring semester will be analyzed in order to revise course work leading up to this exercise.

 

Table 22.  Closing the Loop Example – Special Education, MEd

Issue Data Source Improvement

All students (100%) earned a score of "acceptable" with a score of "2" or above in their applied project.

Scores on applied project rubric in course taken within program of study (e.g., EDSP 6060 or 6981). Continue supporting students in EDSP 6060 to complete an applied project with an acceptable score on the rubric

 

Closing the Loop – Category 8:  Other

Table 23.  Closing the Loop Example – Business Administration, MBA

Issue Data Source Improvement
Only 11% of the students measured scored at the 60th percentile or above in the ability to be able to make the appropriate decisions in Marketing Management.

Student’s relative scores in Marketing Management section of the BSG report.

Meetings are being held with the Marketing Dept. to determine the reason scores in Marketing Management appeared to be significantly lower than scores in most other measured areas.

Of the six sections of the standardized test, students met only one target – all others were partially met or not met. Student scores on the BSG report.

Analysis of the results from this first year of usage of the academic standards will be accomplished with the goal of determining if perhaps our stretch goals for the outcomes were in fact too high in most instances given our student population.

Regular MBA faculty meetings will be held to insure better coordination among the MBA teaching faculty toward the overall goals of the MBA program

 

Closing the Loop/Action Plan Analysis

As previously stated, an analysis of closing the loop/action plans was conducted by the Institutional Effectiveness Committee in Spring 2014.  Initial findings included:

  • A large number of plans were categorized as “assessment methods” which means that the plans addressed issues related to the outcome measures themselves.  This is logical in that many programs, especially those without program-specific professional accreditation, have discovered that their assessment results are either difficult to interpret or do not provide sufficient detail to inform a meaningful program improvement decision.  Implementing these action plans related to assessment methods will ensure that programs obtain more useful data in the near future.  Also, data will soon indicate whether the assessment tool or related learning is the problem.
  • A large number of plans were categorized as “maintenance” which included plans to simply maintain or continue monitoring a particular student learning outcome through continued measurement.  Feedback from faculty indicates that many instructors hesitated to make major changes until the completion of the academic year when more data are available to inform the decision.

Based on the closing the loop/action plan category tallies, it is evident that more professional development is required to ensure that academic programs create more meaningful improvement plans.  The Institutional Effectiveness Committee will review the closing the loop/action plans annually to ensure that other types of action are taken.

Progress Summary

It should be noted that the institutional effectiveness program sample is representative in that it contains plans with a range of assessment quality.  The Office of Institutional Effectiveness recognizes that the University overall is at the beginning stages of developing a culture of data-driven decision making that is appropriately documented.  The large number of closing the loop action plans related to assessment methods is evidence of this primary phase.  Our experience to date indicates that some units had only a very basic understanding of the process, so professional development activities had to start at this fundamental level.  Many faculty members had to be instructed regarding the fact that course completion and course grades were not to be used as student learning outcome assessment methods.  Other programs were further along in the evolutionary cycle, with valid measures such as portfolios, papers and projects in place, but without standardized evaluation tools such as rubrics to properly assess learning.  Still other programs, primarily those with external accreditation, had effective measures already in place and were thoroughly engaged in the program improvement process.

As a result of the emphasis on evidence-based planning and assessment, all units have made great progress toward institutionalizing the assessment process, including a transition from a manual, paper process to an electronic, web-based software system.  All academic programs identified student learning outcomes, assessment methods and targets for the 2013-14 academic year.  (These plans were then vetted by the Institutional Effectiveness Committee.)  Academic programs have followed through with conducting assessments, collecting data, and making informed decisions for program improvement purposes.  Evidence of this effort is documented in WEAVE through findings and closing the loop/action plans for the 2013-14 academic year.  The Data Entry Status Report demonstrates that all components of the institutional effectiveness process have been reported in WEAVE and are provided for each academic program. 

Data Entry Status Reports are presented by college:

College of Business Administration

College of Education and Human Development

College of Engineering

College of Liberal Arts

College of Sciences

Interdisciplinary Studies

Please note that goals, analysis questions, and annual report sections are not used for academic programs at this time.  Academic department and college plans have also been reviewed by the Institutional Effectiveness Committee and entered in WEAVE.  Findings for the expected outcomes included in these plans are reported annually at the end of the academic year.

The institutional effectiveness process at the University of New Orleans is a continuous improvement process in and of itself.  The monitoring status allowed the institution the opportunity to critically examine the current state of institutional effectiveness implementation.  In April 2014, the IE Committee discussed findings related to the Second Monitoring Report.  (Meeting presentation and minutes are presented.)  In May 2014, the committee reviewed 2014-15 institutional effectiveness plans, disseminated approvals and feedback, and requested revisions as necessary.  The Office of Institutional Effectiveness ensured that all academic programs reported Spring 2014 data in WEAVE.  The Committee was also involved in developing a process by which each academic department would align their expected outcomes with the new University 2020 Strategic Plan goals and strategies in WEAVE.  Furthermore, annual department reports now include Detailed Assessment Reports from WEAVE. 

What the University of New Orleans has learned through this focused assessment will allow the institution to improve consistently over the coming academic year.  It is apparent that the Office of Institutional Effectiveness should continue to provide specific instructions and training materials to faculty to ensure quality planning and assessment including, but not limited to:  1) detailed expectations regarding the quality of outcomes, measures, targets, findings and action plans; and 2) raw or summative data must be uploaded in Document Management repository to support narrative summaries of findings.

Supporting Documents

Document Description
Document IconASD_Report_201314Achievement Status Details Report
Document IconCL_Art_History_GradesArt History Research Paper Grades
Document IconCL_Assessment_Acct_3121Accounting Exam Assessment Results
Document IconCL_Assessment_Acct_3122Accounting Exam Assessment Results
Document IconCL_Assessment_Acct_3124Accounting Exam Assessment Results
Document IconCL_Assessment_Acct_3152Accounting Exam Assessment Results
Document IconCL_BIOS_SALGBiology SALG Survey Instrument Results
Document IconCL_CHEM_4000_ProposalChemistry Course Add Proposal
Document IconCL_CHEM_Curriculum_ChangeChemistry Curriculum Change Proposal
Document IconCL_CWW_Comp_ExamMFA Comprehensive Exam Results
Document IconCL_CWW_ThesisMFA Thesis Results
Document IconCL_IDS_3091_Capstone_Scores_RubricInterdisciplinary Studies: Capstone Rubric Scores
Document IconCL_IDS_BS_DegReq_webInterdisciplinary Studies: Academic program degree requirements review mandated by Academic Affairs
Document IconCL_IDS_Curric_ChangeInterdisciplinary Studies: Curriculum change
Document IconCL_MBA_BSG_ReportMBA BSG Report Results
Document IconCL_Special_Ed_Applied_ProjectSpecial Education Applied Project Rubric Results
Document IconCOBA_Dept_OutcomesCollege of Business Administration: Department Outcomes
Document IconCOBA_USPACollege of Business Administration: Strategic Plan Associations
Document IconCOE_Dept_OutcomesCollege of Engineering: Department Outcomes
Document IconCOE_USPACollege of Engineering: Strategic Plan Associations
Document IconCOEHD_Dept_OutcomesCollege of Education and Human Development: Department Outcomes
Document IconCOEHD_USPACollege of Education and Human Development: Strategic Plan Associations
Document IconCOLA_Dept_OutcomesCollege of Liberal Arts: Department Outcomes
Document IconCOLA_USPACollege of Liberal Arts: Strategic Plan Associations
Document IconCommon_Planning_Errors_and_Committee_FeedbackCommon Planning Errors and Committee Feedback
Document IconCOS_Dept_OutcomesCollege of Sciences: Department Outcomes
Document IconCOS_USPACollege of Sciences: Strategic Plan Associations
Document IconDAR_Accounting_BS_201314Detailed Assessment Report: Accounting, BS
Document IconDAR_Anthropology_BA_201314Detailed Assessment Report: Anthropology, BA
Document IconDAR_Arts_Admin_MA_201314Detailed Assessment Report: Arts Administration, MA
Document IconDAR_Biological_Sciences_BS_201314Detailed Assessment Report: Biological Sciences, BS
Document IconDAR_Chemistry_BS_201314Detailed Assessment Report: Chemistry, BS
Document IconDAR_Chemistry_PhD_201314Detailed Assessment Report: Chemistry, PhD
Document IconDAR_Counselor_Education_MEd_201314Detailed Assessment Report: Counselor Education, MEd
Document IconDAR_CWW_MFA_201314Detailed Assessment Report: Creative Writing Workshop, MFA
Document IconDAR_DENAS_PhD_201314Detailed Assessment Report: Engineering and Applied Science, PhD
Document IconDAR_Early_Childhood_Education_BS_201314Detailed Assessment Report: Early Childhood Education, BS
Document IconDAR_Economics_Finance_PhD_201314Detailed Assessment Report: Economics and Finance, PhD
Document IconDAR_Educational_Admin_PhD_201314Detailed Assessment Report: Educational Administration, PhD
Document IconDAR_EES_MS_201314Detailed Assessment Report: Earth and Environmental Sciences, MS
Document IconDAR_Engineering_MS_201314Detailed Assessment Report: Engineering, MS
Document IconDAR_Fine_Arts_Studio_Art_MADetailed Assessment Report: Fine Arts – Studio Arts Option, BA
Document IconDAR_History_BA_201314Detailed Assessment Report: History, BA
Document IconDAR_History_MA_201314Detailed Assessment Report: History, MA
Document IconDAR_HRT_BS_201314Detailed Assessment Report: Hotel, Restaurant, Tourism, BS
Document IconDAR_IDS_201314Detailed Assessment Report: Bachelor of Interdisciplinary Studies
Document IconDAR_Math_MS_201314Detailed Assessment Report: Mathematics, MS
Document IconDAR_MBA_201314Detailed Assessment Report: Masters in Business Administration
Document IconDAR_Mechanical_Engineering_BS_201314Detailed Assessment Report: Mechanical Engineering, BS
Document IconDAR_Music_MM_201314Detailed Assessment Report: Music, MM
Document IconDAR_Philosophy_BA_201314Detailed Assessment Report: Philosophy, BA
Document IconDAR_Political_Science_PhD_201314Detailed Assessment Report: Political Science, PhD
Document IconDAR_Romance_Languages_BA_201314Detailed Assessment Report: Romance Languages, BA
Document IconDAR_Romance_Languages_MA_201314Detailed Assessment Report: Romance Languages, MA
Document IconDAR_Special_Ed_MEd_201314Detailed Assessment Report: Special Education, MEd
Document IconDES_COBA_201314Data Entry Status Report: College of Business Administration
Document IconDES_COE_201314Data Entry Status Report: College of Engineering
Document IconDES_COEHD_201314Data Entry Status Report: College of Education and Human Development
Document IconDES_COLA_201314Data Entry Status Report: College of Liberal Arts
Document IconDES_COS_201314Data Entry Status Report: College of Sciences
Document IconDES_IDS_201314Data Entry Status Report: Interdisciplinary Studies
Document IconFilm-Theatre-Writing-for-ProductionFilm-Theatre-Writing-for-Production - Student Learning Outcomes
Document IconFinanEcon_MS_SLOM.S. Financial Economics Student Learning Outcomes
Document IconIDS_Dept_OutcomesInterdisciplinary Studies: Department Outcomes
Document IconIDS_USPAInterdisciplinary Studies: Strategic Plan Associations
Document IconIE_Committee_Meeting_Minutes_20140416IE Committee Meeting Minutes
Document IconIE_Committee_Meeting_Presentation_20140416IE Committee Meeting Presentation
Document IconIE_Plan_Evaluation_Rubric_Academic_DeptInstitutional Effectiveness Plan Evaluation Rubric: Academic and Department
Document IconIE_Plan_Template_Academic_Program_DeptInstitutional Effectiveness Plan Templates: Academic and Department
Document IconSLO-BA-AnthropologyAnthropology - Student Learning Outcomes
Document IconSLO-BA-EnglishEnglish - Student Learning Outcomes
Document IconSLO-BA-Film-ArtsFilm-Arts - Student Learning Outcomes
Document IconSLO-BA-Film-Theatre-Theatre-ArtsFilm-Theatre - Student Learning Outcomes
Document IconSLO-BA-Fine-Arts-Art-HistoryFine-Arts-Art-History - Student Learning Outcomes
Document IconSLO-BA-Fine-Arts-Studio-ArtFine-Arts-Studio-Art - Student Learning Outcomes
Document IconSLO-BA-HistoryHistory - Student Learning Outcomes
Document IconSLO-BA-International-StudiesInternational-Studies - Student Learning Outcomes
Document IconSLO-BA-Music-StudiesMusic-Studies - Student Learning Outcomes
Document IconSLO-BA-PhilosophyPhilosophy - Student Learning Outcomes
Document IconSLO-BA-Political-SciencePolitical-Science - Student Learning Outcomes
Document IconSLO-BA-Romance-LanguagesRomance-Languages - Student Learning Outcomes
Document IconSLO-BA-SociologySociology - Student Learning Outcomes
Document IconSLO-BIDS-Interdisciplinary-StudiesInterdisciplinary-Studies - Student Learning Outcomes
Document IconSLO-BS-AccountingAccounting - Student Learning Outcomes
Document IconSLO-BS-Biological-SciencesBiological-Sciences - Student Learning Outcomes
Document IconSLO-BS-Business-AdministrationBusiness-Administration - Student Learning Outcomes
Document IconSLO-BS-ChemistryChemistry - Student Learning Outcomes
Document IconSLO-BS-Civil-EngineeringCivil-Engineering - Student Learning Outcomes
Document IconSLO-BS-Computer-ScienceComputer-Science - Student Learning Outcomes
Document IconSLO-BS-Early-Childhood-EducationEarly-Childhood-Education - Student Learning Outcomes
Document IconSLO-BS-Earth-Environmental-ScienceEarth-Environmental-Science - Student Learning Outcomes
Document IconSLO-BS-Electrical-EngineeringElectrical-Engineering - Student Learning Outcomes
Document IconSLO-BS-Elementary-EducationElementary-Education - Student Learning Outcomes
Document IconSLO-BS-Elementary-Education-IntegratedMerged-OptionElementary-Education-IntegratedMerged-Option - Student Learning Outcomes
Document IconSLO-BS-FinanceFinance - Student Learning Outcomes
Document IconSLO-BS-HRT-ManagementHRT-Management - Student Learning Outcomes
Document IconSLO-BS-Human-Performance-and-Health-PromotionHuman-Performance-and-Health-Promotion - Student Learning Outcomes
Document IconSLO-BS-ManagementManagement - Student Learning Outcomes
Document IconSLO-BS-MarketingMarketing - Student Learning Outcomes
Document IconSLO-BS-MathematicsMathematics - Student Learning Outcomes
Document IconSLO-BS-Mechanical-EngineeringMechanical-Engineering - Student Learning Outcomes
Document IconSLO-BS-Naval-Architecture-Marine-EngineeringNaval-Architecture-Marine-Engineering - Student Learning Outcomes
Document IconSLO-BS-PhysicsPhysics - Student Learning Outcomes
Document IconSLO-BS-PyschologyPyschology - Student Learning Outcomes
Document IconSLO-BS-Secondary-EducationSecondary-Education - Student Learning Outcomes
Document IconSLO-BS-Urban-Studies-PlanningUrban-Studies-Planning - Student Learning Outcomes
Document IconSLO-MA-Arts-AdministrationArts-Administration - Student Learning Outcomes
Document IconSLO-MA-EnglishEnglish - Student Learning Outcomes
Document IconSLO-MA-HistoryMA-History - Student Learning Outcomes
Document IconSLO-MA-Political-ScienceMA-Political-Science - Student Learning Outcomes
Document IconSLO-MA-Romance-LanguagesMA-Romance-Languages - Student Learning Outcomes
Document IconSLO-MA-SociologyMA-Sociology - Student Learning Outcomes
Document IconSLO-MAT-Curriculum-InstructionMAT-Curriculum-Instruction - Student Learning Outcomes
Document IconSLO-MAT-Special-EducationMAT-Special-Education - Student Learning Outcomes
Document IconSLO-MBAMBA - Student Learning Outcomes
Document IconSLO-MEd-Counselor-EducationMEd-Counselor-Education - Student Learning Outcomes
Document IconSLO-MEd-Curriculum-InstructionMEd-Curriculum-Instruction - Student Learning Outcomes
Document IconSLO-MEd-Educational-LeadershipMEd-Educational-Leadership - Student Learning Outcomes
Document IconSLO-MEd-Special-EducationMEd-Special-Education - Student Learning Outcomes
Document IconSLO-MFA-Creative-Writing-WorkshopMFA-Creative-Writing-Workshop - Student Learning Outcomes
Document IconSLO-MFA-Film-ProductionMFA-Film-Production - Student Learning Outcomes
Document IconSLO-MFA-Theatre-DesignMFA-Theatre-Design - Student Learning Outcomes
Document IconSLO-MFA-Theatre-PerformanceMFA-Theatre-Performance - Student Learning Outcomes
Document IconSLO-MM-MusicMM-Music - Student Learning Outcomes
Document IconSLO-MPA-Public-AdministrationMPA-Public-Administration - Student Learning Outcomes
Document IconSLO-MS-AccountingMS-Accounting - Student Learning Outcomes
Document IconSLO-MS-Applied-PhysicsMS-Applied-Physics - Student Learning Outcomes
Document IconSLO-MS-Biological-SciencesMS-Biological-Sciences - Student Learning Outcomes
Document IconSLO-MS-ChemistryMS-Chemistry - Student Learning Outcomes
Document IconSLO-MS-Computer-ScienceMS-Computer-Science - Student Learning Outcomes
Document IconSLO-MS-Earth-Environmental-SciencesMS-Earth-Environmental-Sciences - Student Learning Outcomes
Document IconSLO-MS-EngineeringMS-Engineering - Student Learning Outcomes
Document IconSLO-MS-Engineering-ManagementMS-Engineering-Management - Student Learning Outcomes
Document IconSLO-MS-Health-Care-ManagementMS-Health-Care-Management - Student Learning Outcomes
Document IconSLO-MS-Hospitality-ManagementMS-Hospitality-Management - Student Learning Outcomes
Document IconSLO-MS-MathematicsMS-Mathematics - Student Learning Outcomes
Document IconSLO-MS-PsychologyMS-Psychology - Student Learning Outcomes
Document IconSLO-MS-Tax-AccountingMS-Tax-Accounting - Student Learning Outcomes
Document IconSLO-MS-Urban-StudiesMS-Urban-Studies - Student Learning Outcomes
Document IconSLO-MURP-Urban-Regional-PlanningMURP-Urban-Regional-Planning - Student Learning Outcomes
Document IconSLO-PhD-ChemistryPhD-Chemistry - Student Learning Outcomes
Document IconSLO-PhD-Counselor-EducationPhD-Counselor-Education - Student Learning Outcomes
Document IconSLO-PhD-Curriculum-InstructionPhD-Curriculum-Instruction - Student Learning Outcomes
Document IconSLO-PhD-Educational-AdministrationPhD-Educational-Administration - Student Learning Outcomes
Document IconSLO-PhD-Engineering-and-Applied-SciencePhD-Engineering-and-Applied-Science - Student Learning Outcomes
Document IconSLO-PhD-Financial-EconomicsPhD-Financial-Economics - Student Learning Outcomes
Document IconSLO-PhD-Integrative-BiologyPhD-Integrative-Biology - Student Learning Outcomes
Document IconSLO-PhD-Political-SciencePhD-Political-Science - Student Learning Outcomes
Document IconSLO-PhD-PsychologyPhD-Psychology - Student Learning Outcomes
Document IconSLO-PhD-Special-EducationPhD-Special-Education - Student Learning Outcomes
Document IconSLO-PhD-Urban-StudiesPhD-Urban-Studies - Student Learning Outcomes
Document IconUNO_IE_Website_Closing_the_LoopUNO Institutional Effectiveness Website: Closing the Loop
Document IconUNO_IE_Website_Collecting_Valid_EvidenceUNO Institutional Effectiveness Website: Collecting Valid Evidence Section
Document IconUNO_IE_Website_Developing_TargetsUNO Institutional Effectiveness Website: Developing Targets Section
Document IconUNO_IE_Website_OutcomesUNO Institutional Effectiveness Website: Student Learning Outcomes Section
Document IconWEAVE_BrochureWEAVE brochure