Assessing core skills in post-secondary students not as “EASI” as it sounds

In recent years concerns have been raised by government officials, college and university administrators, faculty, staff and students with regards to the skills and competencies attained through Ontario’s post-secondary education (PSE) sector. Why aren’t all graduates finding gainful employment? What skills do employers look for? How can we make our students more innovative and entrepreneurial? These are just a few questions that have invigorated widespread debate on the overall quality and accountability of Ontario’s PSE sector.

Amidst these concerns, the Higher Education Quality Council of Ontario (HEQCO) recently announced its plans to launch an assessment of students’ literacy, numeracy and problem-solving skills as a means to measure what they call ‘critical employability skills’. HEQCO’s Essential Adult Skills Initiative (EASI) is embarking to measure the core skills that most employers feel are foundational to success. Ten public colleges and five public universities have volunteered to participate in this new assessment initiative.

The test uses the Education and Skills Online assessment tool from the Organization for Economic Co-operation and Development (OECD), and it will be taken by first-year students and graduating students from across various disciplines. Each student will receive individualized reports of their results and how it compares internationally, while the results of each college and university will be provided solely to each institution.

“The initial goal is that colleges and universities use these results as instruments for quality enhancement,” says Harvey Weingarten, President & CEO of HEQCO.

What is a quality PSE?

The rolling out of this initiative raises several questions about what it means to deliver a quality PSE in Ontario. Does quality imply teaching students core competencies? Or is it a matter of graduates finding employment? Or might we determine it based on students’ perceived worth of the time and resources they input? Or is it perhaps all of the above?

Quality has yet to be a significant determinant of PSE funding in Ontario, though, with the anticipated funding formula revisions likely to place greater emphasis on performance indicators, it will soon become clear what sorts of indicators represent quality to the provincial government. Nevertheless, at present, it remains to be seen what specific skills or competencies are reflective of a quality PSE.

HEQCO has outlined four classes of learning outcomes: basic cognitive skills (numeracy and literacy), discipline-specific skills (specialised knowledge and skills), higher-order cognitive skills (communication and critical thinking), and transferable skills (teamwork, initiative and resilience). However, it is unclear which of these four classes is most representative of a quality PSE. And do colleges and universities have the responsibility to foster each/any/all of them?

This ambiguity may be a limiting factor on HEQCO’s upcoming initiative. For example, HEQCO claims this test will help provide insight into how effectively students are developing their literacy, numeracy and problem-solving skills to succeed throughout their lives. Using HEQCO’s definitions from above, it seems this test is more about measuring basic cognitive skills and a bit of higher-order cognitive skills. Skills such as teamwork, initiative and resilience – the transferable skills most desired by employers – are not covered in this assessment. To be fair, HEQCO is actively seeking ways to utilize reliable assessment tools capable of measuring transferable skills, which is incredibly difficult to standardize.

In short, this pilot test is more about assessing students’ basic and higher-order cognitive skills. Whether or not these competencies are the prerogative of colleges and universities rather than the K-12 sector is up for debate. While it remains to be seen what new information this assessment will provide, we can anticipate a variety of ways in which the results of EASI might impact the Ontario post-secondary education sector.

How will EASI impact Ontario colleges and universities?

Since this initiative is truly the first of its kind at the post-secondary level in Ontario, it is unclear how its results will impact the sector. It is fair to anticipate that one of the key benefits this initiative can provide is that it will give each student participant a sense of their own literacy, numeracy and problem-solving skills with international benchmarks to compare. It is also likely that each participating institution will use the results to inform their practices and see where certain areas could be improved through course curricula and/or program designs.

The final reports summarizing all of the findings will also provide the provincial government and employers with a snapshot of the cognitive skills Ontario’s first-years and graduates possess. Finally, and perhaps ideally, the EASI project will illustrate the improvements needed within the K-12 sector to ensure their students are literate and numerate, and prepared to build on those skills at the post-secondary level.

However, there is some concern that EASI could pave the way for a singular framework of learning outcomes to take root within Ontario PSE. For instance, the large emphasis on cognitive skills, while important, does not effectively address the transferable skills piece that is most representative of employer demands. It is therefore arguable to consider this test as less of an assessment of ‘critical employability skills’, and more so an assessment of ‘essential life skills’. Ensuring students have the skills needed to succeed in life is an important aspect of our education system, but it would be debateable to suggest that cognitive skills fully encapsulate that ideal.

Lastly, we know that PSE is not like K-12 education. There is no standardized curriculum and there isn’t a provincially-held learning outcomes framework that each college and university has to abide by in order to maintain their funding (at least for now). Students in different programs will exercise and develop different cognitive skills throughout their program’s duration. Some won’t even continue using skills like numeracy for the entirety of their PSE.

Moving forward: more of the same?

So if we anticipate some graduate students with equal or lower numeracy scores compared to first-years, does this mean their programs are of poor quality? Of course not. There are discipline-specific skills and the far more valued transferable skills that HEQCO has also outlined, which remain unexamined in this instance. But knowing that transferable skills are very difficult to assess in a standardized format, are we left measuring the same general cognitive skills we have always measured?

There are two main takeaways from this thought exercise: 1) It is debateable how effectively EASI will tell us much new information about our PSE students; 2) It is unclear how the results of EASI will impact the sector as a whole.

We should remain optimistic, though, as all parties seem to genuinely yearn for the highest quality PSE possible in Ontario. Like any new initiative, the effectiveness of this project will be determined by how its results inform policy; how it is used by HEQCO, how it is used by the participating colleges and universities and, most importantly, how it is used by the Ontario government to influence PSE policy.