Create a secure exam environment with Linux

English Open Chat
#1

Hi,
recently in Germany a new decree was issued that computers are allowed in exams.

While this sounds good news at first (at least for me, I prefer computers to very expensive graphical calculators), the requirements for the exam-environment are very strict.

The most important points are:

  • exam environment cannot be entered or left by the student alone, only by the school (meaning: by the teacher in charge)
  • a computer in exam environment must be eye-catching
  • unallowed leaving of the environment gives an acoustical warning (I know that this is in opposition to not being able to leave it at first)
  • no network access allowed
  • no peripherals (like usb-sticks) allowed
  • the exam environment must still be active after hard poweroff / power failure

In germany, some schools use the “lernstick” (https://www.imedias.ch/themen/lernstick/index.cfm ) or other linux distributions that boot from stick. While I consider this quite safe, it doesn’t fulfil the requirement of being active after boot (you can remove the stick, turn off the computer, reboot and then insert the stick again to fake being in exam environment, and I can think of no means to prevent that).

The decree is now in official hearing, so maybe some things will be changed. But I started pondering about an exam environment and would like you to comment my thoughts.

  1. Are there any linux-based exam environments ready to use somewhere in the world?
  2. I can think of the following solution. Take a math-exam as an example, the application “GeoGebra” should be used.
  • create an x-session for geogebra alone for a specific user “exam”
  • start the exam-session with a password (more on that later). Upon starting the session, all files in exam-user are deleted.
  • disable all virtual consoles
  • put the exam user into autologin
  • to leave the exam mode, a different password must be entered, exam-user will be taken out of autologin then

One can think of programming an exam-shell. Depending on the password entered, different applications can be enabled. So the x-session would be for this shell, not for geogebra (which makes things more complicated, one would need a window manager then, I think).

The only way to get out of this that I can think of is to reboot into system repair mode, maybe this can be disabled somehow.

The password could be generated such that it is only valid on a certain day.

All of this would be much easier if network was allowed. Then entering and leaving the exam mode can be registered on a server. All clients must issue alive-signals every minute or so, the teacher sees computers that are no longer in exam mode with a big red sign on the teacher pc.The issued passoword is valid for a certain exam only.

I would very much like to see exams taken with linux-based systems. In Germany, many schools start using IPads, which - in my opinion - are for leisure rather than for serious work.

Thanks in advance for your input,

Andreas

#2

Not a technical question relating to openSUSE installation etc. Moving to ‘Looking For Something Other Than Support’ subforum…

#3

IMO
The exam requirements as you’ve described them were written by someone with without technical understanding, basically a wish list thrown together without defining first non-technical objectives and then finding someone to translate into a list of technical objectives and requirements.

So,
This is the type of project good companies (and people) run away from… Because if there is no clarity, then there will have to be substantial modifications to the requirements. And, only a technically knowledgeable person can know if the requirement list is clear, understandable and attainable.

Even your suggestion about the lernstick violates the stated requirement list.

I’d recommend…
Provide some feedback to the people who are setting the requirements.
Offer constructive criticism for what they have already published.
But, don’t spend much time on this until it’s managed competently.

IMO,
TSU
:slight_smile:

#4

As someone with a background in certification exams, I’d probably start
with the exam providers and the environments they support for their exams.

There are some that support Linux (the Linux Foundation’s exams, for
example, use a remotely proctored exam environment that’s quite good).

Others - such as the big providers (Thompson Prometric and Pearson VUE)
tend to require Windows systems for their exam environments.

But companies like that would probably be the place I’d start talking
about the use of Linux for exams, since they’ve got the background in
exam delivery and security that’s really needed to provide a defensible
exam experience.

Jim

Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

#5

Thank you for your thoughts.

I have already written a letter with questions to the government.

As far as I can see, most exam providers use the network, so this should make the government re-evaluate that point.

I’d like to ask differently now (and set aside the regulations): Do you think it is possible to make a standard SuSE-Installation exam-proof?

Like I said:

  • enter exam mode via password
  • issue “alive” signals over the network
  • leave exam mode via password

Or will there always be exits where the user can get his own files?

Andreas

#6

On Sat, 27 May 2017 08:16:01 +0000, goeba wrote:

> Thank you for your thoughts.
>
> I have already written a letter with questions to the government.
>
> As far as I can see, most exam providers use the network, so this should
> make the government re-evaluate that point.

Indeed - that seems to be to be a somewhat strange requirement. I kinda
understand the intent, though - they want people to not cheat by using
chat programs or Google (or Wolfram alpha, or whatever) - but that’s why
you have an exam proctor. I don’t think you’ll ever replace a proctor
with technology.

> I’d like to ask differently now (and set aside the regulations): Do you
> think it is possible to make a standard SuSE-Installation exam-proof?

If the user has physical control over the machine, probably not -
physical access generally overrides any software-based access controls
you put in place, regardless of the OS used. You can try locking it down
using BIOS controls, but

> Like I said:
> - enter exam mode via password - issue “alive” signals over the network
> - leave exam mode via password
>
> Or will there always be exits where the user can get his own files?

You’d have to remove access to su/sudo, and probably other tricks that
people will work out - like changing bash to run as root (or a copy of
bash). You’d have to control and validate all updates for the OS (again
regardless of the OS) to ensure no new backdoors were installed in the
system - and you’ll always be chasing that.

Jim

Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

#7

Hi Jim,

thanks for your input.

I am currently planning a project where each student has his/her own personal notebook.

On those notebooks, a bios-password is set and the students do not have root-access (and, of course, are no sudoers). The system used will be OpenSuSE (probably).

Secure boot will probably be “off”, so this setup can be hacked by replacing the system hard-disk (or by getting root-access somehow or by getting a bios-master-password).

I think that one can make this quite safe if the exam-shell uses some public-private-key procedure to ensure that it is running on the machine it was installed to.

I would very much like the “normal” installation for exams, as booting from stick introduces an extra complexity. But as you pointed out, this is very hard to be made really safe when the student has physical access to the computer.

So I will probably go this route:

  • use the normal installation with some kind of exam-mode as pointed out above for “normal” tests. Here the teacher in the room will have to walk around a bit to make sure that no cheating is going on, just like in any other test.
  • use a special exam-installation from stick for central examinations

The point “no network” must be changed to “the student must not have network-access of any kind, network for controlling the exam mode is allowed”.

It is very much more likely that students use their smartphone for cheating than taking a huge effort to hack the exam-mode of a linux-machine. Maybe one student in 10 000 has the knowledge to achieve that.

Thanks,
Andreas

#8

On Sun, 28 May 2017 16:06:01 +0000, goeba wrote:

> Hi Jim,
>
> thanks for your input.

You bet. Exam security is a challenging subject, no doubt. :slight_smile:

> I am currently planning a project where each student has his/her own
> personal notebook.
>
> On those notebooks, a bios-password is set and the students do not have
> root-access (and, of course, are no sudoers). The system used will be
> OpenSuSE (probably).

You’ll want to also make sure they can’t access the options in grub that
let them bypass the initial boot sequence, allowing them to force it into
single-user mode (or recovery mode). Disable all ability to boot from
external media, and set the internal hard drive as the first (and only)
boot device.

That also includes disabling network boot from the built-in ethernet
controller, if it supports that.

> Secure boot will probably be “off”, so this setup can be hacked by
> replacing the system hard-disk (or by getting root-access somehow or by
> getting a bios-master-password).

You can run secure boot if you like - openSUSE does support it (I don’t,
but I understand it works well).

> I think that one can make this quite safe if the exam-shell uses some
> public-private-key procedure to ensure that it is running on the machine
> it was installed to.

Off-network is going to make that difficult to validate the key -
typically, you want a signed keychain of some sort, unless you’re talking
about using something like SSH keys. For that, you’d need some sort of a
process for validating the public key provided against a locally-stored
private key, without exposing that private key.

> I would very much like the “normal” installation for exams, as booting
> from stick introduces an extra complexity. But as you pointed out, this
> is very hard to be made really safe when the student has physical access
> to the computer.

Yep.

> So I will probably go this route:
>
> - use the normal installation with some kind of exam-mode as pointed out
> above for “normal” tests. Here the teacher in the room will have to walk
> around a bit to make sure that no cheating is going on, just like in any
> other test.
> - use a special exam-installation from stick for central examinations
>
> The point “no network” must be changed to “the student must not have
> network-access of any kind, network for controlling the exam mode is
> allowed”.

That would simplify a great many things, yes. Something to keep in mind
when restricting access is that the loopback interface probably needs to
remain enabled, or some locally-installed software may not work as
expected.

> It is very much more likely that students use their smartphone for
> cheating than taking a huge effort to hack the exam-mode of a
> linux-machine. Maybe one student in 10 000 has the knowledge to achieve
> that.

If there’s one thing I’ve learned over the years, students find creative
ways to cheat on exams. That’s why a proctor is still used and required
for high-stakes exams. Nothing catches cheaters better than someone
watching. Recording the exam session from the system they’re using also
isn’t a bad idea.

Jim

Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

#9

Hi Jim,
of course a proctor is needed. The trouble is: The proctor usually sits in front of the students, so he/she cannot see what’s going on on notebook-screens. So if a student manages to get internet-access, cheating would be terribly easy.

On the other hand, I don’t see the point of buying expensive notebooks if you cannot use them in exams. At the moment, parents pay for expensive calculators, in some cases for electronic dictionaries, too. For several years now parents keep asking why those expensive gadgets are needed when everyone has a smartphone that is technically far superior to them.

Apple is giving away high-quality classrom software for “free” (meaning: People still have to buy the apple-products, and if you get used to them at school, you will probably stick to them later), I’d prefer the LInux-way!

Are you a teacher, too?

Andreas

#10

On Wed, 31 May 2017 08:36:02 +0000, goeba wrote:

> Hi Jim,
> of course a proctor is needed. The trouble is: The proctor usually sits
> in front of the students, so he/she cannot see what’s going on on
> notebook-screens. So if a student manages to get internet-access,
> cheating would be terribly easy.

Easy solution - proctor sits at the back of the classroom. :slight_smile: (I
remember having an issue exactly like this with a testing provider at a
university years ago - during the exam, the proctor started sitting in
the back of the room, and the cheating stopped).

> On the other hand, I don’t see the point of buying expensive notebooks
> if you cannot use them in exams. At the moment, parents pay for
> expensive calculators, in some cases for electronic dictionaries, too.
> For several years now parents keep asking why those expensive gadgets
> are needed when everyone has a smartphone that is technically far
> superior to them.

Yep. Part of it depends on what you’re testing for, though, too. One of
the most common complaints in IT certification exams is “why can’t I use
Google?” - the argument goes that in the real world, that’s how you find
answers.

But the purpose of testing is not to test one’s Google-fu, but to measure
one’s understand of the material actually being tested for. If you have
to look everything up on Google, you affect your own performance in doing
the job.

Something similar might apply here.

> Apple is giving away high-quality classrom software for “free” (meaning:
> People still have to buy the apple-products, and if you get used to them
> at school, you will probably stick to them later), I’d prefer the
> LInux-way!
>
> Are you a teacher, too?

Have been, currently, I manage instructors and training programs. :slight_smile:

Jim

Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

#11

Just recently there has been an article about using google in exams in German online-magazine “Spiegel Online”:

http://www.spiegel.de/lebenundlernen/schule/smartphone-an-schulen-googlen-in-der-klausur-erlauben-a-1146879.html

One could allow google for instance if

a) you want to test the students skills in googleing

b) the tasks of the exam require more than just googleing

In a normal exam, one can get some points by writing down some learned facts. Those “easy points” will go away when google is allowed, thus exams will (if taken seriously) be more difficult when google is allowed.

The big “but” is: How can you allow google and not allow online collaboration? Rich students might pay experts to do their exams for them (online and real-time).

So my conclusion is: Exams are exams, not the real world, and this will have to stay like this. If you don’t want students to learn everything by heart, you can write open-books exams or allow documents on pc’s, but one cannot allow internet-access.

I’ll keep you informed how my project proceeds!

Andreas

#12

On Thu, 01 Jun 2017 10:36:02 +0000, goeba wrote:

> Just recently there has been an article about using google in exams in
> German online-magazine “Spiegel Online”:
>
> http://tinyurl.com/ydz29768
>
> One could allow google for instance if
>
> a) you want to test the students skills in googleing

Yep, that’s valid, certainly - though grading it would be very
interesting/challenging.

> b) the tasks of the exam require more than just googleing

Maybe - but I imagine that the nature of such exam questions would lend
itself to pages that describe the answer to the questions, in which case
a Google search would naturally pick up those answers.

Unless it’s something like “use Google to learn how to do a push-up, and
then do 10”. (flailing a little for an example, but you get the idea -
the test involves a physical component that requires a physical action)

> In a normal exam, one can get some points by writing down some learned
> facts. Those “easy points” will go away when google is allowed, thus
> exams will (if taken seriously) be more difficult when google is
> allowed.

That’s the theory, yeah. Ultimately, an exam should test skills +
knowledge rather than knowledge alone - these days, knowledge isn’t often
deemed important (because “I can just google it”), but being able to do
things is.

> The big “but” is: How can you allow google and not allow online
> collaboration? Rich students might pay experts to do their exams for
> them (online and real-time).

That happens today, actually. There’s an entire industry around
protecting exam intellectual property and detecting and defeating
cheating.

> So my conclusion is: Exams are exams, not the real world, and this will
> have to stay like this. If you don’t want students to learn everything
> by heart, you can write open-books exams or allow documents on pc’s, but
> one cannot allow internet-access.

Exams are a way of measuring one’s knowledge and skills, but a passing
score basically is a measure of a minimally-qualified individual in the
thing being tested. The best example is that of a lawyer - any lawyer
you work with in the US will have passed the bar exam - but if they just
squeaked by, they’ve barely met the minimum requirements.

That’s not to say that in all cases, the exam is the end-all measure of
someone’s skills. There are rare exams (I’ve seen a few) that give
consistent results over time for the same candidate, but it comes down to
what you’re measuring (in one case, it’s cognitive ability), which is an
inherent trait rather than a learned skill.

> I’ll keep you informed how my project proceeds!

Sounds good!

Jim

Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C