ISQ and Student Success

Questioner: Anonymous

Posed to: Pamela Chally, Intrim Provost and VP of Academic Affairs

Answered by: Pamela Chally, Intrim Provost and VP of Academic Affairs


Question 1:

The recent emphasis on ISQ bothers me as an educator. Is it my role to educate the students or make them happy? Please do not respond saying these are the same. If you want references and recent research, ISQ does not determine that. Many universities are on the path to make the ISQ result confidential and available to only the faculty while UNF posts them online by faculty and course number. Questions. 1. To what extent have you made sure that the improvement of ISQ is not coming at the cost of course quality? Many faculty are putting excessive weight on attendance, take home exams, open book exams, and grade curves so that students like them. How are colleges controlling that? Or is it a college’s policy to do only what is necessary to improve the metrics and get state money? If so, can we change the mission statement to low quality instead of high quality education?

We have been clear about our goals for UNF to increase our standing in the performance-based metrics, but never at a cost of a quality education for our students.  To draw attention, in general, to quality of instruction can only benefit students.  I do not have any data indicating that faulty are putting excessive weight on attendance, but attendance in itself is an important factor in students doing well in a class.  Alternative means of assessment are totally the decision of the faculty member.  Consequently, the choice of an open book examination is an individual faculty decision. As Academic Affairs has drawn attention to student success, we have also provided support to both faculty and students.


Question 2

It has been proven that ISQ scores are discriminating against immigrants (especially with an accent) and women. Is it in UNF’s new mission to use a statistic that has been proven to be discriminating? If the administration thinks that is not true in the case of UNF, have they done a study to make certain?

In response to a recent discussion with Faculty Association on this topic and suggested language forwarded by Terri Ellis which shows impact on gender and racial bias in evaluations, Academic Affairs requested from ITS the incorporation of suggested language in the ISQ instructions to students:

 “Student evaluations of teaching play an important role in the review of faculty. Your opinions influence the review of instructors that takes place every year. Iowa State University recognizes that student evaluations of teaching are often influenced by students’ unconscious and unintentional biases about the race and gender of the instructor. Women and instructors of color are systematically rated lower in their teaching evaluations than white men, even when there are no actual differences in the instruction or in what students have learned.  As you fill out the course evaluation please keep this in mind and make an effort to resist stereotypes about professors. Focus on your opinions about the content of the course (the assignments, the textbook, the in-class material) and not unrelated matters (the instructor’s appearance).”


Question 3

If the administration is so concerned for students, why don’t we drop the full time student requirement from 15 to 12 credits? Most students at UNF are employed, often even more than 40 hours. So forcing them to enroll for 15 credits and then blaming the faculty for their poor performance is outrageous.

Jay Coleman recently responded to a very similar question and therefore I am forwarding Jay’s response as it appropriately addresses this question:

I have a working version of a DFW model covering the entire campus (undergraduates), which includes a myriad of factors. Once I account for pre-entry characteristics, student demographics, course characteristics, the level of student engagement, instructor of the course, how much students are working off campus, etc., students attempting fewer than 15 hours actually do statistically significantly worse in their courses.  Every hour attempted below 15 is associated with about a 1.5% reduction in the odds of passing a class with an A, B, or C.

I can add that I’ve done a lot of other modelling of student success over the last 3-4 years, using a variety of outcome measures in a variety of areas, and taking less than 15 hours is frequently a negative factor. At worst, it’s a statistically insignificant one.

The question then obviously is why. Yes, there could be omitted factors involved that are skewing results, and we continue to pull in more data on more factors to help address such problems.  However, in virtually all of these models I’m accounting for the student’s academic preparation pre-entry (e.g., SAT/ACT scores and high school GPAs), which major they’re in, how much they’re working off campus, how much they’re engaged on campus (e.g., visits to the Library, Wellness Center, Game Room, Nature Trails) – i.e., things that would be expected to capture reasons why students might want to take fewer hours.

I think part of the answer is that we presume that students take the extra time that they would have been using on the fifth course and allocate it to their other four courses. Too often, they simply don’t.  Instead, that time is being spent on leisure (or other) activities instead, and depending on what those activities are, that in itself could be a bad thing.  I think another part of the answer is that hours attempted is a partial proxy for a construct that is increasingly showing up in educational research: grit.  Students with more grit – i.e., persistence, commitment, work ethic, etc. – simply do better than students with less, and high levels of grit make up for a lot.  If you’re trying fewer hours (all other things being equal), it’s just not a good sign regarding your level of grit.

Research continues, but signs are that the push for 15 has really helped the grad rate. Keep in mind that when we push taking 15, it’s for all FTICs at all points in their programs of study, not just the ones who just started.  Thus, it can have a positive and more immediate grad rate benefit for those who are late(r) in their programs.  Our 4-year grad rate has increased by nearly 50% over the last four years, from about 26% to 38.5%, and we expect next year’s number to be significantly higher.  Pushing 15 per term (or 30 per year) has a lot to do with that.


Questioner: Dr. Santavicca (Physics)

Question Posed from the floor

In past years, there’s always been an announcement at one of the last FA meetings of the year about the T&P cases for that year: how many people went out, what the outcomes were, the university committee votes. We did not have that this year. Can we get that information?

Answer from the floor by David Fenner and Others

It has been common practice that we get a P&T report usually around April. Usually, once it has gone through the provost, the provost provides us with that report; we hear about the positive, we hear about the negative, we hear about the whole picture. This year, Faculty Association has not been provided with that report. Cindy and I have both asked for it on a few occasions. We’ve been told it has to now wait till the full slate is approved by the Board of Trustees, so I go back to the point about June 18. On June 18, there will be a BOT meeting at which the positive slate is voted on. Once that happens, I believe that the provost’s office will then release to us a report about P&T cases, but, yes, this is not normal precedent, to not have a report.


President’s model

Questioner: Dr. Robert Slater

Question from the floor posed to David Fenner, FA President

your assessment of the president basing this model on a business school, is that your personal assessment? Because from what I’ve seen he’s micromanaged even down two tenure candidates he’s got now doing lit reviews and citations analysis, which is not a business school model. The business school model is the dean respects the recommendations of the committees and the people below him while he’s out fundraising. It would seem that this president is actually micromanaging the parts of the administration that he wants to and more or less ignoring parts that he doesn’t. That could explain why he’s not here.

Answer from the floor by David Fenner, FA President

What I said is my personal opinion and also that of an outsider.

Book store


Question asked from the floor

If the bookstore is a 3rd party vendor, how does a student resolve problems with bookstore?  Who would be UNF’s administrator contact over Bookstore?

Answer by Sheri Shuman, VP of  Administration & Finance

Tully Burnett, Director of Business Services, administers and oversees the bookstore contract.  If a student or faculty has an issue that they cannot resolve with the bookstore, the individual should contact Tully for him to address the concerns.

UNF FA and President’s attendance to meetings

Questioner: Anonymous

Question Posed to David Fenner, FA president

Under the UNF Constitution and as stated on the FA’s website, the UNF Faculty Association is the core of UNF’s shared governance structure. Thus, it was extraordinary to hear on from the FA president on May 9, 2019, that the UNF President will no longer be presenting a President’s Report to the Faculty Association and that in general terms neither will the UNF Vice Presidents. In both instances the UNF President and VPs will not attend meetings of the FA and only the Provost will attend and answer questions. Given this extraordinary change, can the Faculty Association President comment on his agreement with this change and describe the impact that this change will have on the Faculty Association as the core of UNF’s shared governance structure. Can the FA president also share his view on whether he believes the President hosting so-called “jam sessions” in colleges is a comparable substitute for the President’s attendance, Report, and responses to questions brought forth by the Faculty Association.

Answer from the floor by David Fenner, FA President

I want to say two things. The first thing I want to say is about the lunches:  I’ve always taken the lunches—as I think most of you have, too—as a gift. John Delaney started this six years ago. It’s been a blessing not only because we’ve had the opportunity for camaraderie and the opportunity to be fed, but also it allowed us to make certain we had a quorum to get faculty association business done. We had five years of lunches under John, and we have had a year’s worth of lunches under President Szymanski. I take those to have been a gift. What money Shari is going to continue to give us for lunches, I also take that to be a gift. The only concern I have is to make certain we have enough faculty who are participating on a regular basis to make sure that we can do the shared governance part of our work, that we can handle the curriculum and the legislative calendar and ensure that faculty voice is heard.

The second thing to say is that when I was in the graduate school, I had an opportunity to interact with Coggin in a serious way for the first time. I realized then that Coggin has a different kind of administrative structure that many of the other colleges. For a number of years under at least two deans, there was a relationship where the dean was very outward facing; the dean was interested in connecting with the community, was interested in fundraising, was really looking outside, was looking at the horizon—if we could use the captain of a ship metaphor. And the Associate Dean was responsible for running the day-to-day matters of the college.  That was certainly the case with Bobby Waldrup. I think it might’ve been the case with Jay Coleman and Jeff Steagall when they were there. That was a model that I was not used to. You know that our president comes out of a business school background and so I believe he is giving us a new model and that new model is more like a business school model where he is going to be outward facing. He’s going to be looking at the legislature, he’s going to be looking at fundraising and it’s really going to be the provost who takes on the duties of being sort of master of the ship, to extend the metaphor, who is really going to be responsible for the day-to-day operations. So an awful lot of what I’m saying is resting on the hope and expectation that Simon Rhodes will be brilliant and will connect with us in very close, intimate ways.

Increasing Student Success

Questioner : Anonymous

Question posed to Pamela Chally, Interim Provost and VP of Academic Affairs

Questions are surfacing regarding the choices made by the new administration in their effort to increase student success. Data shows that over the last few years the number of students placed in online classes by the administration has been increasing and, in some cases, have doubled. Classes that in the past had 20-25 students are now being reported to have 50, 60 and 70 students. Please explain how doubling the size of a class, and especially an online class, contributes to increasing student success?

Answer by Pamela Chally, Interim Provost and VP of Academic Affairs

The data show that the average class size for undergraduate DL really hasn’t changed much over the last five years. It was 35.47% in AY13, and in AY18 it was 36.01%, so the size hasn’t really changed. There are a few classes that have deliberately been increased; there’s one in CCEC, there may be a few in Coggin. The one in CCEC specifically has a lead instructor and has a lot of students in it, but it’s a very, very good lead instructor who has support from many distance learning coaches. One last thing I’d say is there isn’t an exact number of how many should be in a class to decide if it’s well done or not well done, but support is very important for that faculty member.

Regalia at Commencement

Questioner: Jose Franco (Mathematics & Statistics)

When we were preparing for the commencement, we were told that there is a possible interest of doing uniform regalia for the marshals for the commencement. I would like to know if you could conduct a survey among faculty, because many of us are very proud of our colors, of our regalia, and we would want that not to be taken away from us. Me, in particular, I’ll say that. So could you conduct a survey just to in that way inform the people organizing the commencement ceremonies on the sentiment of faculty.

Answered by David Fenner, FA President and Pamela Chally, Interim Provost and VP of Academic Affairs

Dr. Fenner:  So let me say two things. First thing to say is I will pass it on to the Executive Committee, and we we will pick that up as a discussion of the Executive Committee, which is going to happen in two weeks or around there.  And so YES, we’ll definitely take up the issue, and if we could do a survey the committee will do a survey.


But may I say something else, which is kind of just purely anecdotal. So when we had the inauguration not too long ago, Sharon Ashton and I had a big fight about this and the fight was about me being in one of the uniform gowns, so what I expressed was that my primary role in the inauguration is a representative of the faculty, and not a member of the Board of Trustees.  She allowed me to wear my own gown and so forth, but I did not sit with the Board of Trustees. I actually ended up sitting on the row with the deans. Now all of the Board of Trustees members were in the uniform gown, which is blue with gray bands here. And so with the vice presidents, so for somebody like Vice President Shuman being in that kind of gown is not—I mean, it’s actually a little bit nicer than what she was wearing before. She was just wearing a standard black master’s gown. I don’t know how Dr. Chally feels about this, but I felt slightly bad for Dr. Chally, that she was wearing a gown that did not have doctoral bars on it, all that stuff. I believe that the president is interested in having a kind of uniform look to some degree, but I don’t know how far it’s going to go.


Dr. Chally, responding to Dr. Fenner asking if she’d like to add anything: added: I’ll just speak for Dr. Rhodes down the road.  If I had stayed in this position, I would have ordered a gown that had stripes, but it was just in the UNF colors.  Since I’m interim, we didn’t do that.  I’m sure Dr. Rhodes will do that. I can’t really add anything more than what you said.

Alcohol related incident

Questioner: Stephynie Perkins (Communication Department)

Question from the Floor posed to Pamela Chally, Interim Provost and VP of Academic Affairs

1a. During the last week of class in the spring term, a faculty member was suspended because several students brought alcohol into the classroom unbeknownst to the faculty member. The University Police was tipped off that alcohol was in the classroom and came to the class. It was reported that the students will go before the Student Conduct Office, and that the faculty member has been suspended indefinitely until this incident can be thoroughly investigated. How and when did the University Police know about this alcohol related incident in the classroom and if they knew before the start of class, should they have informed the faculty to be on the lookout?

1b. In order for faculty not to be reprimanded or suspended, has the administration created specific step-by-step written guidelines for faculty to follow in case this kind of incident occurs again?

1c. What is the due process for faculty who are caught up in this kind of activity unbeknownst to them?


Answer from the Floor by Pamela Chally, Interim Provost and VP of Academic Affairs

1a. One of the students in the class posted a picture on Instagram that somehow made it to UPD during the class.

1b. I sent some suggestions about that to Dr. Parmelee yesterday. I don’t want to do it misjustice by reporting it without having it in front of me, so I will be very happy to answer that question and include what I had sent to Dr. Parmelee yesterday.

1c. I don’t feel that I can answer that at this point. I will say this: they certainly can appeal, they can grieve. That is certainly their right to do that.

ISQ results and Promotion of Faculty

Questioner : Anonymous

Question posed to Pamela Chally, Interim Provost and VP of Academic Affairs and Kally Malcom – Bjorklund, UFF President

This question is for the UFF union president and the UNF administration interim Provost. Can each of you weigh in on the appropriateness — contractually and scientifically — of a Promotion and  Tenure Committee or an administrator to focus narrowly on ISQ results (student evaluations) and impose a threshold score for a particular item when making judgements of teaching?

Question answered by Pamela Chally, Interim Provost and VP of Academic Affairs

Kally and I thought we would answer that together. I really appreciate that, because it comes directly form the contract.

The Collective Bargaining agreement addresses criteria for demonstrating teaching effectiveness, and the process of applying for tenure and promotion. Article 20 addresses tenure:
“Judgments of academic excellence are complex. They cannot easily be reduced to a quantitative formula, nor can the considerations that must be applied in each individual case be completely described in general terms or by numbers alone, separate from necessary qualitative assessments.”


Article 20.5 is the section titled Criteria for Tenure and Basis for Tenure Decision.

This section includes the paragraph I just mentioned, and section D states that the tenure decision shall take into account annual assignments and annual performance evaluations, among other things.


Performance Evaluations are addressed in Article 18, and on teaching effectiveness. This article offers the most specific language about how faculty can demonstrate teaching effectiveness. And it is in Article 18 where ISQs are addressed specifically.


Article 18.2(d) University Required Student Evaluations.

“(1) The University required student Instructional Satisfaction Questionnaire (or ISQ) is one tool for evaluating teaching performance, and all the required ISQs must be included in the annual evaluation portfolio. However, the evaluation of a faculty member shall not be based solely or primarily on student evaluations if the faculty member has provided other information or evidence in support of his/her teaching performance.”


Article 18.4(a)1

“There are many approaches to and dimensions of pedagogical work. Thus, the evaluation of teaching performance shall consider the range of pedagogical activities engaged in by the faculty member.”


Another section in Article 18 offers 15 examples of pedagogical activities that can be used to evaluate teaching effectiveness, and ISQs are but one of the measures listed.


The Collective Bargaining agreement has language in Articles 18 and 20, as well as several promotion articles that outline the appropriate way to evaluate teaching effectiveness. The person asking the question also seeks scientific data supporting or challenging the use of student evaluations as the singular method of assessing teaching effectiveness. In the interest of brevity, I will not use this time to point to the several studies that are out there related to the usefulness of student evaluations. The best practices for evaluating faculty teaching are already addressed clearly within the collective bargaining agreement. And that is an agreement that is negotiated not just by UFF but also by the administration, and I was so happy that Provost Chally told us this morning that they are committed to following the contract, which we all must do, so we are happy to stand together in support of Articles 18, 20, and so many more.


Dr. Chally: It’s a multifaceted decision, and it’s really important that the individual make their case as to why they should be supported

UNF hiring and retention of Faculty

Questioner: Anonymous

Question posed to Pamela Chally, Interim Provost and VP of Academic Affairs

Student success requires that UNF hire and retain the best faculty. Yet, according to studies conducted by the National Education Association, over the past decade, faculty salaries at UNF have been at, or near, the bottom of all 11 schools within the state university system. In 2018, the average salaries of faculty at all ranks were at, or near, the bottom of the state university system. In order for UNF to hire and retain the best faculty, what plans does the administration have to increase faculty salaries in 2019-20?

Question Answered by Pamela Chally, Interim Provost and VP of Academic Affairs

Obviously, David already answered that question from the Faculty Association perspective. A couple things I’d like to say: I wish all of you had higher salaries. I know you wish that also. We really have not received any new sources of money for the last 10 years, and that’s difficult. It’s really, really been difficult. But despite that, last year and the year before, small raises, 2%, were available and that was the same average of raises across the United States. I want to say one other thing about this. It’s a two- way street in terms of being able to negotiate raises, and I really look forward to working with Kally and Bess at UFF in a new, hopefully very, very collaborative relationship, so we can make increasing progress in that area. I know it’s a need.