ISQ and Student Success

Questioner: Anonymous

Posed to: Pamela Chally, Intrim Provost and VP of Academic Affairs

Answered by: Pamela Chally, Intrim Provost and VP of Academic Affairs


Question 1:

The recent emphasis on ISQ bothers me as an educator. Is it my role to educate the students or make them happy? Please do not respond saying these are the same. If you want references and recent research, ISQ does not determine that. Many universities are on the path to make the ISQ result confidential and available to only the faculty while UNF posts them online by faculty and course number. Questions. 1. To what extent have you made sure that the improvement of ISQ is not coming at the cost of course quality? Many faculty are putting excessive weight on attendance, take home exams, open book exams, and grade curves so that students like them. How are colleges controlling that? Or is it a college’s policy to do only what is necessary to improve the metrics and get state money? If so, can we change the mission statement to low quality instead of high quality education?

We have been clear about our goals for UNF to increase our standing in the performance-based metrics, but never at a cost of a quality education for our students.  To draw attention, in general, to quality of instruction can only benefit students.  I do not have any data indicating that faulty are putting excessive weight on attendance, but attendance in itself is an important factor in students doing well in a class.  Alternative means of assessment are totally the decision of the faculty member.  Consequently, the choice of an open book examination is an individual faculty decision. As Academic Affairs has drawn attention to student success, we have also provided support to both faculty and students.


Question 2

It has been proven that ISQ scores are discriminating against immigrants (especially with an accent) and women. Is it in UNF’s new mission to use a statistic that has been proven to be discriminating? If the administration thinks that is not true in the case of UNF, have they done a study to make certain?

In response to a recent discussion with Faculty Association on this topic and suggested language forwarded by Terri Ellis which shows impact on gender and racial bias in evaluations, Academic Affairs requested from ITS the incorporation of suggested language in the ISQ instructions to students:

 “Student evaluations of teaching play an important role in the review of faculty. Your opinions influence the review of instructors that takes place every year. Iowa State University recognizes that student evaluations of teaching are often influenced by students’ unconscious and unintentional biases about the race and gender of the instructor. Women and instructors of color are systematically rated lower in their teaching evaluations than white men, even when there are no actual differences in the instruction or in what students have learned.  As you fill out the course evaluation please keep this in mind and make an effort to resist stereotypes about professors. Focus on your opinions about the content of the course (the assignments, the textbook, the in-class material) and not unrelated matters (the instructor’s appearance).”


Question 3

If the administration is so concerned for students, why don’t we drop the full time student requirement from 15 to 12 credits? Most students at UNF are employed, often even more than 40 hours. So forcing them to enroll for 15 credits and then blaming the faculty for their poor performance is outrageous.

Jay Coleman recently responded to a very similar question and therefore I am forwarding Jay’s response as it appropriately addresses this question:

I have a working version of a DFW model covering the entire campus (undergraduates), which includes a myriad of factors. Once I account for pre-entry characteristics, student demographics, course characteristics, the level of student engagement, instructor of the course, how much students are working off campus, etc., students attempting fewer than 15 hours actually do statistically significantly worse in their courses.  Every hour attempted below 15 is associated with about a 1.5% reduction in the odds of passing a class with an A, B, or C.

I can add that I’ve done a lot of other modelling of student success over the last 3-4 years, using a variety of outcome measures in a variety of areas, and taking less than 15 hours is frequently a negative factor. At worst, it’s a statistically insignificant one.

The question then obviously is why. Yes, there could be omitted factors involved that are skewing results, and we continue to pull in more data on more factors to help address such problems.  However, in virtually all of these models I’m accounting for the student’s academic preparation pre-entry (e.g., SAT/ACT scores and high school GPAs), which major they’re in, how much they’re working off campus, how much they’re engaged on campus (e.g., visits to the Library, Wellness Center, Game Room, Nature Trails) – i.e., things that would be expected to capture reasons why students might want to take fewer hours.

I think part of the answer is that we presume that students take the extra time that they would have been using on the fifth course and allocate it to their other four courses. Too often, they simply don’t.  Instead, that time is being spent on leisure (or other) activities instead, and depending on what those activities are, that in itself could be a bad thing.  I think another part of the answer is that hours attempted is a partial proxy for a construct that is increasingly showing up in educational research: grit.  Students with more grit – i.e., persistence, commitment, work ethic, etc. – simply do better than students with less, and high levels of grit make up for a lot.  If you’re trying fewer hours (all other things being equal), it’s just not a good sign regarding your level of grit.

Research continues, but signs are that the push for 15 has really helped the grad rate. Keep in mind that when we push taking 15, it’s for all FTICs at all points in their programs of study, not just the ones who just started.  Thus, it can have a positive and more immediate grad rate benefit for those who are late(r) in their programs.  Our 4-year grad rate has increased by nearly 50% over the last four years, from about 26% to 38.5%, and we expect next year’s number to be significantly higher.  Pushing 15 per term (or 30 per year) has a lot to do with that.

ISQ results and Promotion of Faculty

Questioner : Anonymous

Question posed to Pamela Chally, Interim Provost and VP of Academic Affairs and Kally Malcom – Bjorklund, UFF President

This question is for the UFF union president and the UNF administration interim Provost. Can each of you weigh in on the appropriateness — contractually and scientifically — of a Promotion and  Tenure Committee or an administrator to focus narrowly on ISQ results (student evaluations) and impose a threshold score for a particular item when making judgements of teaching?

Question answered by Pamela Chally, Interim Provost and VP of Academic Affairs

Kally and I thought we would answer that together. I really appreciate that, because it comes directly form the contract.

The Collective Bargaining agreement addresses criteria for demonstrating teaching effectiveness, and the process of applying for tenure and promotion. Article 20 addresses tenure:
“Judgments of academic excellence are complex. They cannot easily be reduced to a quantitative formula, nor can the considerations that must be applied in each individual case be completely described in general terms or by numbers alone, separate from necessary qualitative assessments.”


Article 20.5 is the section titled Criteria for Tenure and Basis for Tenure Decision.

This section includes the paragraph I just mentioned, and section D states that the tenure decision shall take into account annual assignments and annual performance evaluations, among other things.


Performance Evaluations are addressed in Article 18, and on teaching effectiveness. This article offers the most specific language about how faculty can demonstrate teaching effectiveness. And it is in Article 18 where ISQs are addressed specifically.


Article 18.2(d) University Required Student Evaluations.

“(1) The University required student Instructional Satisfaction Questionnaire (or ISQ) is one tool for evaluating teaching performance, and all the required ISQs must be included in the annual evaluation portfolio. However, the evaluation of a faculty member shall not be based solely or primarily on student evaluations if the faculty member has provided other information or evidence in support of his/her teaching performance.”


Article 18.4(a)1

“There are many approaches to and dimensions of pedagogical work. Thus, the evaluation of teaching performance shall consider the range of pedagogical activities engaged in by the faculty member.”


Another section in Article 18 offers 15 examples of pedagogical activities that can be used to evaluate teaching effectiveness, and ISQs are but one of the measures listed.


The Collective Bargaining agreement has language in Articles 18 and 20, as well as several promotion articles that outline the appropriate way to evaluate teaching effectiveness. The person asking the question also seeks scientific data supporting or challenging the use of student evaluations as the singular method of assessing teaching effectiveness. In the interest of brevity, I will not use this time to point to the several studies that are out there related to the usefulness of student evaluations. The best practices for evaluating faculty teaching are already addressed clearly within the collective bargaining agreement. And that is an agreement that is negotiated not just by UFF but also by the administration, and I was so happy that Provost Chally told us this morning that they are committed to following the contract, which we all must do, so we are happy to stand together in support of Articles 18, 20, and so many more.


Dr. Chally: It’s a multifaceted decision, and it’s really important that the individual make their case as to why they should be supported

1.7 Million Dollars

Questioner: Anonymous

Posed to: Pamela Chally, Interim Provost and VP of Academic Affairs


Is it true that Dean Rainbolt “found” 1.7 million dollars that had gone unaccounted for in his college?

Where were these funds found and how is it possible they were not known of all along?

Could these funds have been put to use in hiring faculty and adjuncts rather than having departments going directly to Academic Affairs to ask for funds to put on courses?

Could the use of these funds for instructional purposes have facilitated an increase in graduation rates and perhaps retention as well?

Is it true that Dean Rainbolt discussed with the COAS chairs the possibility of using these funds for faculty and staff raises?

Would such plans not involve circumventing the contract, the union, and human resources?

Would such plans not exacerbate pay inequities across the university as a whole?

What is the responsibility of Academic Affairs for oversight over the budgetary workings of COAS?

Will Provost Chally continue to allow Dean Rainbolt to hold classes teaching members of his college about budgets? 

Does Provost Chally accept these developments as evidence of budgetary incompetence and will this evidence be used in her decision concerning the strong vote of no-confidence in Dean Rainbolt?

Response in writing by Provost Chally:

The dollars are being reviewed.   Most were known about and regularly used to pay for summer school.   Dr. Rainbolt is aware that faculty raises are not possible without Union negotiation and there was no intent, whatsoever, to circumvent UFF.  All College budgets are overseen through Dan Moon and Anne Hoover, including the COAS budget. The budget classes Dean Rainbolt is teaching do not specifically discuss how to “find” money or what to do once “found.”

Student Success

Questioner: Anonymous 

Posed to President Szymanski and Provost Chally

President Szymanski
Listening to the President talk at different time about the faculty’s role in promoting student success led me to reflect on the administrative team he has surrounded himself with to lead UNF in this effort. My reflection was not encouraging.

In the past year UNF has had one Dean fired for being caught engaging in sexual misconduct, another Dean was the subject of an investigation into allegations of promoting a hostile workplace and other complaints, and yet another Dean recently received a vote of no confidence by the majority of his faculty.

Do these individuals reflect the type of administrators the President plans to depend on to, lead UNF in promoting student success? While these administrators were hired by the past President, what plans does the current President have for hiring the best administrators going forward? Many of the circumstances that UNF now face are the product of decisions by current administrators. What is the President doing to change this and what is being done to insure that better administrative decisions are made in the future?

Student success requires effort and excellence on the part of both the faculty and the administration


President Szymanski and Provost Chally
There has been considerable discussion about the metrics and improving student success. However, most of this discussion has been about “tactics” – steps faculty can take. Can the President and Provost please talk about what they are doing in “strategic” terms at the administration level to improve student success?

For example, education research shows that online courses have notoriously high withdrawal rates. Yet UNF has ramped up online learning without apparent consideration of this fact. Is it really wise for the administration to be increasing online learning at the rate it is at this time? If not, what is being done in strategic terms to insure student success in the increasing number of online courses?

Research also shows that student retention is highly correlated with the use of tenure track faculty. Yet UNF is at a 5 year low in the percentage of tenure track faculty it employs in the classroom. For example, in the President’s own department, tenure track faculty teach less than 40% of the students – an all-time low. What is being down to improve student success through hiring more tenure track faculty?

Finally, research shows that faculty engaged in research are often more engaging in the classroom and this increases student success. Yet at UNF revenues from grants and contracts (a recognized measure of research engagement) is at a 10-year low and support for research assistants is at an all-time low. What is being done to address these deficiencies and improve research productivity and support and thereby improve student success?
Finally, in many of the above circumstances, the challenges described did not result from error or oversight. Instead they were the product of judgements and decisions on the part of administrators, many of who are still in office. What is being done to put administrators in office who recognize what is needed to reverse the challenges that UNF faces and thereby result in decisions that increase student success?


Answer from the floor by President Szymanski

So we have a writing center that can help with the issue with “finally, finally, finally” and that kind of thing. But beyond that, now there’s a couple things that I want to point out. One is an error in terms of something that was said because it is really important. There was not a dean who was fired. There was a dean who resigned from the university, and that’s an important point. We’ve had some turnover and I think the most important part, as I get to some specifics as these are pretty complex questions.


So one is hiring a provost—really, really, really, really, really critical to an institution. Pam’s worked really hard over the past year or so, but having a permanent provost position who can also partner with the President is really important to an institution. So when you think about the academic side of the house, without that it’s really a difficult sort of process. And so I’m excited, as I mentioned before, I’m excited about having a provost in place. Having someone who is going to lead our institution, is going to partner with the academic side of the house and make sure we have a strong, strong foundation for our institution, and I think that’s probably the most important part of this. You have to have the leadership, and I think we have somebody who’s coming in—we’re all going to cross our fingers and he’s going to cross his fingers, but someone who through your support is going to be successful. But that’s key to a lot of these parts of these questions. It’s important to have vision, that person who is committed, dedicated, who is someone who aspires to success. And that’s going to be key to us moving forward.


Some of the other questions that were asked. So if you do have research, just please give me cites so I can read research, to evaluate it and to make sure that I’m also learning as we go along, because some of the data that I asked Jay to quickly get for us suggest that withdrawal rate is actually lower in DL classes than it is in non-DL classes. And then the other part that I asked, I said, can you give me a little bit of a breakdown in terms of retention with respect to faculty rank? And there really is no difference among all the different faculty. It’s a real quick cut because we just didn’t have enough time to do a deep dive, but it’s actually the person who’s probably the visitor, which we talked about, who’s actually slightly better in terms of retention. There’s some data in there that I think it’s important for us to think about. We will think about a lot of these things.


There’s questions, a lot of questions about support. You have to realize that we finished at the bottom three years in a row. Support for research, support for administration, support for travel, all those kinds of things—there’d be an extra $30 million for a lot of different things. So it’s important for us to perform well with respect to the metrics because it does give you the cash flow and the resources that allow us to be discretionary and to reinvest and strategically invest in your success. So I think that’s a really important part of who we are moving forward. The reason why we talking about tactics with respect to faculty, why we didn’t sort of talk about other things, it’s because you’re the faculty. And so some of the questions are to faculty too. Jay does a wonderful presentation sort of laying this out there. David had an opportunity to hear it at a Board of Trustees meeting. It’s really not DFWs. There a huge explanatory variable above the variables explains the most in terms of  retention. Student retention is the thing we do the worst. We’re going to get zero points on the metrics with respect to student retention. You can’t compete for dollars when you get zero points. So it’s not just because of that, but it is about student success. It’s about the opportunity costs when people pay money to come to an institution. It’s about having to take classes over again, getting discouraged and dropping out. It’s about delaying graduation, the opportunity costs of what it costs when you don’t have a job for a semester and you compound that over the lifetime; it gets pretty expensive pretty quick. So when we talk about tactics, because those are the things that you can control. In your profession, you can control the quality of your teaching. You can’t always control what editors and associate editors and reviewers have to say about your research, but you can control the classroom experience. So we talk about strategy. There are many moving parts. It’s not just faculty; it’s everybody who has to play a role in moving the institution forward. The Student Association did a great job of reducing fees, then reallocating fees, to make sure that they are focusing in on student wellbeing, being very proactive.


It’s about staff and what we do and how do we make it a better environment for our students. So when we’re talking about these things, there is a strategic part to this, and in the few seconds we have, what can we move the needle on? Really, it is about classroom experience, and that’s what you control. You can always control what you do in a classroom, how motivated you make your students, how successful you want them to be, and how you can inspire them, because we’ve all been inspired; otherwise, we wouldn’t be here. So it’s not that we just focus in on talking—there is a lot of strategy going on, a lot of things around—raising funds, scholarships, reducing costs of education—that aren’t really looked upon as the faculty’s role necessarily except when it comes to the books. So the cost of books and how can we reduce that? And how can the library help with respect to open access? Those are important issues. So it’s a collective. So I just don’t want to give you the impression that it’s about the faculty—what can the faculty do? It’s what all of us need to do every day to wake up and move our institution forward. So we’ve got a lot of great things, and people appreciate them externally now. And that’s important, because the community also needs to know more about us and who we are and what we’re all about, so we get the best students and become a destination institution. We’ve got great students, and we want to continue to have good students moving forward.


So it’s kind of a long-winded answer to two pages’ worth of questioning, but I’m also happy to have this discussion when I come to your colleges, so feel free to raise your hand and ask again. We’ll have longer than a couple of minutes to respond to questions. I don’t have all the answers. It’s a collective part of this, it’s a change culture, change perspective, but we have excellent people here who are passionate, as I tell people everywhere. Passionate professors, passionate students, and passionate administrators.


Questioner: Anonymous 

Posed to: Pamela Chally, Interim Provost and VP of Academic Affairs

Q. 3a

Regarding the three new promotions in Academic Affairs,
(1) Could the provost explain how these promotions advance UNF’s mission or further our progress toward meeting the President’s goals?
(2) Could the provost please tell us the amounts of the raises that each person received with his or her promotion?
(3) Could the provost please tell us what new duties were assigned to each?
(4) Could the provost please tell us what old duties were reassigned and to whom?
(5) If no old duties were reassigned, were those in these positions prior to promotion not working 40 hour weeks? (6) Could the provost explain how promotions and raises for faculty are highly controlled by the CBA but promotions and raises for administrators can be enacted without searches, without process, without oversight, and without faculty consultation?

Q. 3b
Does the promotion of Dr. Moon to associate provost represent a reorganization of Academic Affairs?
Should a reorganization of Academic Affairs full under the UNF Constitutional mandate to include consultation with faculty and in particular with Faculty Association?
What is the new structure, if there is a new structure, for Academic Affairs?
Does Dr. Patterson now answer to Dr. Moon?
Does Christina Helbling answer to Dr. Patterson or Dr. Moon?
Does the provost continue to maintain direct reports or do her former direct reports now answer to Dr. Moon?
If the latter, has Dr. Chally abdicated duties of the provost to Dr. Moon, someone who has never been vetted by the faculty for such a role?

Q. 3c
Why was Dan Moon — a while male — promoted to associate provost instead of Karen Patterson — a black female? Dan Moon has fewer years in administration than Karen Patterson; he can cite fewer accomplishments as an administrator than Karen Patterson; he enjoys the favor of the faculty to a far lesser degree than does Karen Patterson; he is, overall, much less effective than Karen Patterson.

Q. 3d
Many faculty believe that the timing of the promotion of Dan Moon to associate provost signals that he will be appointed to the position of interim provost if the search for a permanent provost fails this year. Is this true? If it is true, how will the deans, chairs, and faculty feel about having as interim provost a person who served only two years as a department chair, less than a full year as undergraduate studies dean, and a year as interim dean of COAS where nothing — apart from cancelling the holiday party — was accomplished or even attempted?

Wouldn’t appointing Dan Moon to the role of interim provost be a strong signal to the faculty and the broader UNF community that UNF is really not serious about taking the university to the next level or meeting the ambitious goals of our new president?


Answer from the floor by Pamela Chally, Interim Provost and VP of Academic Affairs

A. Interim Provost Chally. The actions of the promotions in Academic Affairs, including Deb Miller, Cristina Helbling, and Dan Moon, are justified by our commitment to student success and our need as an institution to follow the mandates of the Board of Governors. By contributing to students’ success, the president’s goals are being strongly supported. Dr. Moon took on significant responsibilities in student affairs and increasing responsibilities in the Provost’s office, and he did an excellent job taking on all these additional responsibilities. I asked Dr. Moon to assume an interim role as associate provost, and he agreed. The position was open after Jay Coleman was named vice president. The permanent provost can determine how he or she wishes to move forward. This is a temporary assignment to an open position to get the work of the office accomplished.

Some of these questions, or at least parts of them, feel very undignified and humiliating. This time in Faculty Association meetings is an opportunity, I believe, for some to demoralize and embarrassed me and others. Aren’t we better than this? I am far from perfect. You can question and you could ask anyone who’s worked with me, and many of you have worked with me for many years, I am so far from perfect, but please come and meet with me. I don’t believe this forum is a time to humiliate others. Whether you agree with me or not, I want to share four guiding principles and this is why I can get up in the morning and why I have come to work here at UNF since 1993. I try very hard to listen to understand. Respecting others is incredibly important to me. I also believe in kindness, downright kindness, and the last one is to promote excellence at UNF. I hope you all will join me in those same principles. Thanks.

Conduct of Administrators

February 7, 2019

Questioner: Anonymous

Posed to: Pam Chally, Interim Provost & VP of Academic Affairs

My questions concern the business school and its Dean. This past summer three female office managers, with 35 years of experience among them, walked off their jobs accusing the Dean of creating a hostile workplace. A lawyer was hired to investigate. However, the inquiry was limited to if the Dean violated laws against discrimination and sexual misconduct. Not surprisingly, the lawyer found “no appearance” the Dean violated such laws. What standard of conduct are administrators at UNF held to? Is it limited to whether they break the law? What about the “highest standards of ethical behavior” and “professional practice” called for in the UNF rules and regulations? The investigator’s report contains disturbing accounts of threats, intimidation, coercion and retaliation; as well as other forms of aggressive and unwanted behavior. What is being done about this conduct? Will it be investigated under UNF’s Code of Ethical Conduct? The investigation also produced a “supplemental” report that evidenced among other issues that the Dean tried to interfere with the Presidential Search Committee to further his own candidacy and when he was unsuccessful he retaliated against a committee member. What steps are being taken in response to this second report and this evidence? In both reports, most (but not all) of the evidence came from administrators and staff who work closely with the Dean. There is also evidence that the administration has long known about many of the allegations. Is this what faculty and students can expect from the new administration? Is this who we are? Aren’t we better than this – aren’t we?

Response from the floor by Interim Provost Chally

Interim Provost Chally stated she had concerns about discussing personnel issues in a public forum. She will answer it in the future.

Instructional Resources

February 7, 2019

Questioner: Anonymous

Posed to: Pam Chally, Interim Provost and VP of Academic Affairs

Dean Rainbolt seems obsessed with money. His recent policies, his new (very complex) budget spreadsheets, the new class he is offering on budgeting, and the fact that he has TWO staff members in his office who are either exclusively or principally focused on budgeting seem to have pushed out all focus on other things. Despite this, or perhaps because of it, the resources that are distributed for instruction, especially visitor positions, have diminished under his leadership. This is felt acutely in the Department of Mathematics and Statistics. Is it true that on a number of occasions the chair of that department felt compelled to approach Academic Affairs directly for instructional resources necessary to provide students with the courses they need to progress or graduate?

Response from the floor by Interim Provost Chally

Interim Provost Chally stated Dr. Patterson is a strong advocate for more lines for the department. To the best of her knowledge, that has always occurred in the presence of Dean Rainbolt.

Not Getting Paid On Time

Questioner: Anonymous

Posted to: Pamela S. Chally, Interim Provost and Vice President for Academic Affairs

As an adjunct in COAS, it was a hardship to not get paid this fall in a timely manner. I understand several of my colleagues were victims of the same snafu. Will someone be held accountable for this error? I do not want to leave UNF, but will not continue if this is the way we are treated.

Response from the Floor by Provost Chally

Provost Chally apologized for the error and assured the faculty that the process that caused the problem has been corrected. There are new processes in place to ensure that this will not happen again. Dean’s that were involved in this situation understand that this is their responsibility.

Data Analytics Recent Announcement

Questioner: Anonymous

Posted to: David Fenner, Faculty Association President

The recent announcement of an internal search for a Vice President of Data Analytics made me look up the definition of Analytics on the web. It says that it is “the discovery, interpretation, and communication of meaningful patterns in data. It relies on the simultaneous application of statistics, computer programming and operations research to quantify performance”. In that definition, the word statistics comes first because statisticians are professionally trained to interpret and communicate data.  The previous administration had enlisted statistics professors to help precisely because the data analytics people came short in that. This year BOG Performance Metrics indicate that effort paid off with 10 points increase in the score over the year before. Why doesn’t the position announcement underscore the importance of a PhD statistician?

President Szymanski’s response copied from his report. 

The Vice President for Analytics search is ongoing. Finalists will be interviewed during the second week of October. The President addressed an anonymous question about whether or not candidates could or should have a PhD in statistics. The President stated that while that was one acceptable degree, there were finalists with other backgrounds as well.

Administrative Stipends

Questioner: Anonymous

Posted to: Pamela S. Chally, Interim Provost and Vice President for Academic Affairs

Why are administrators that return to the faculty after completion of their term continuing to be paid their administrative stipend? These administrators are no longer performing administrative services. Is it true that these “excess” payments amount to over $100,000 annually (the union says it more like $400,000)? At the completion of their professorships, faculty are not permitted to keep their stipends. The argument that individuals would not take on administrative roles unless they were allowed to keep their salary stipends is not credible (according to the union and a recent Folio Weekly article no other university in Florida has such a provision in their contract).


Response from the Floor by Provost Chally

That has been a current policy that has been in effect for a large number of years.