Select Page
EFLogo high res_2012-2013

A 50-year-old company probably isn’t the first one you’d think would be focused on innovating the standardised testing model, but not all companies maintain a “startup-y kind of mindset” after that many years, either.

WYSE Travel Confederation member EF Education First, which got its start when Bertil Hult, who suffered from dyslexia, realized that language learning couldn’t simply be confined to textbooks in the classroom. With commercial flights becoming more available at the time, Hult’s approach instead involved taking students abroad to learn a language. The entrepreneurial spirit that built the company’s educational travel and cultural exchange business exists today, with perhaps no better example than the EF Standard English Test.

In designing the exam, the company aimed to avoid many of the pitfalls that have made standardized testing “high-stakes,” like the high price students pay for many exams or the difficulty of securing a spot at a testing center, while still providing a meaningful, accurate score on par with the Test of English as Foreign Language (TOEFL) or International English Language Testing System (IELTS). In doing so, it enlisted experts including psychometrician Dr Richard Luecht, former Educational Testing Service senior vice president Dr. Mari Pearlman, and Dr. Lyle Bachman, who assisted in the development of the Cambridge English exams and others.

“Put those three mega brains together and we get something like the EFSET,” says EF Executive Director Yerrie Kim.

We caught up with Kim to get a sense of how the company approached the EFSET’s design and the changes it hopes to influence in standardized testing as a whole.

EDUCATION DIVE: How widely used is the EFSET compared to, say, the TOEFL?

KIM: That’s an interesting question. The audience for the EFSET, I think, is very different from the audience for the TOEFL. The TOEFL, really, is a test that gets administered maybe to, at most, a million, maybe a million-and-a-half, a year. Potentially two million, if you stretch that number out a little bit. It complies with people who are trying to go from a country — China, Korea, France — to study in the US or the UK, for example. That’s the majority of the people who are currently taking the TOEFL or need to take the TOEFL, because those universities require that. Who the EFSET is trying to serve is not just that two million trying to go abroad, but the hundreds of millions or close to billion people who are actually learning English regardless of their current level and regardless of whether they want to or don’t want to go to university in the U.S.

As a result, in order for us to serve all the language learning population, we had to make sure that we were scalable and that we were accessible to all types of different people who were learning English for different reasons. While we have developed an EFSET that meets standardized exam development criteria, the population we’re trying to serve is much larger than who the TOEFL is catering toward right now.

How was the EFSET’s design approached to ensure it had the quality of the TOEFL or IELTS while also challenging high-stakes aspects of standardised testing?

KIM: The most important thing about standardized testing is that we’re able to provide an accurate score that is meaningful. What that means is, for all the different types of people with different ability levels, a test is meant to provide a numerical score, which allows students to be comparable to others and/or to show progress if progress has been made, according to the leveling system of the test.

This is much harder to do and requires at least two things. One is big data psychometric analysis, so lots of statistics. Another is making sure that, from the very beginning, you are thinking about the design of each of the different types of test questions that you’re going to put together. At the same time, what that means is in order to actually do that, we had to learn this from scratch.

I guess the simple answer is that we work with the industry’s best people and make sure that we have psychometric analysis as well as assessment design principles that work together to make sure that we’re able to provide standardised measurements that are precise and accurate each time a person takes a test.

With the EFSET being online and having no proctors, how do you prevent cheating?

KIM: This is another very interesting question, and also where we hope to see the most change in the way people think about the test. First of all, the EFSET was not necessarily built to satisfy university administrators from the get-go. It was really built to try to provide an accurate assessment tool in the hands of the students, so we focused on that first. We were certainly concerned about cheating, but that was very low in our priority list.

What was more on our priority list was, “How do we make an accurate test that is actually accessible to people, so that if they want to test their ability, they can do that without actually having to spend USD 300 per test or go to a test center? And is it possible to bring that experience to the home or to your library on your own device?”

When you focus on that, and you do that because you’re actually trying to serve the student, then the question of cheating really gets to be on the burden of the student. If you actually want to use this instrument to measure your English level and actually are interested in an accurate diagnosis of your skills, then you probably don’t want to be cheating because that’s just you cheating yourself. So that’s just sort of where we drew the line.

At the same time, I think it’s definitely not smart to ignore the current traditional use case of tests. As much as there are many more individuals who are learning for their own good and at their own pace, how we think about tests in the traditional sense is that a test is given by someone else to you. For example, the University of Chile — which was piloting the use of the EFSET for its university students who were preparing for the TOEFL ​— did their own individualisation. So they had rows of computers, and a professor was walking around as you would in a traditional setting to prevent or guard from cheating.

We focused our effort on providing an accurate measurement tool for the student who has genuine need to understand how they’re doing or where they are in the grand scheme of all possible fluency. And so, we wanted to make sure it was accessible to that student and not necessarily adding a burden. At the same time, for large-scale use or use in universities, we’re coming up with ways, together with the universities or school systems that are using the test, where proctoring happens in the premise where the test is used. At the same time, for the future, there are lots of discussions and developments, both inside and outside of EF, that are going to allow lots of different use cases for online tests that are much more secure than what you might expect today.

How could the EFSET’s approach best be applied to other standardised tests?

KIM: Standardised tests way too often these days are unfortunately considered the end goal. A lot of students, for many different types of reasons, are studying because they want that score. What I think we are making that grand leap toward is hopefully a change of that mindset, and a change, therefore, in how and why we assess people. In other words, what we’re trying to do here is both large-scale delivery and rapid development of test questions, and to put that together in a very smart way using an adaptive engine. That actually means that we can use less test questions to find out how good you are on a standardised scale.

If we democratise that — not just in the delivery, but also in how available these standardised exams are — then I think we can start to see it not becoming a luxury product or a thing that only a few people can afford, but something that is in the hands of every teacher and every student. The minute that happens, what you will see is that people are going to value standardised exam scores less, and therefore not be so obsessed with them, but probably more with what you can do with the skills that you learn and how much you actually know —​ and perhaps actually assess the person in a much more holistic way.

So, what we really want to do is change the way that people perceive test scores and instead provide a way that allows for language exams or any other standardised exams to be produced and delivered at mass scale, which will then fundamentally change the use case for test scores. This is not something that’s just going to happen because EF is doing it or because we want it to happen. It’s going to require sea change. But what you’re seeing now is there are lots of companies that are thinking in this direction, and I think it’s a really exciting time to actually be a part of this education revolution.