Honestly, the problem is not the cheating, per se.
The problem is the lack of learning the material. You don't, IMO, directly care how they produced the answer, you care about it only as a proxy for them learning the material well enough to solve the problem.
And making people do them in person with no technology is unrealistic - not because it can't be done, but because at that point, it's not a reflection of how you'd use it outside of that classroom, and people are going to call it pointless, and IMO they'd be right. You would be correct that anyone who met that bar would have likely learned the material, but you'd also have excluded people who would have met the bar of "can use the material to the degree of familiarity needed going forward".
I think a reasonable compromise would be to let students collaborate on the exams in the classroom, without external access - while I realize some people learn better on their own in some subjects, as long as everyone contributes some portion of the work, and they go back and forth on agreeing what the right answers are, then you're going to make forward progress, even if that ruins the exam process as anything other than a class-wide metric. You could subdivide it, but then that gets riskier as there's a higher chance that the subset of people doesn't know enough to make progress. Maybe a hint system for groups, since the goal here is learning, not just grading their knowledge going in?
Not that there's not some need for metrics, but in terms of trying to check in on where students are every so often, I think you need to leverage how people often end up learning things "in the wild" - from a combination of wild searching and talking to other people, and then feedback on whether they decided you could build an airplane out of applesauce or something closer to accurate.
You don't, IMO, directly care how they produced the answer, you care about it only as a proxy for them learning the material well enough to solve the problem.
I don’t care about the answer. I care about the thought process that went into finding the answer. The answer is irrelevant.
And making people do them in person with no technology is unrealistic - not because it can't be done, but because at that point, it's not a reflection of how you'd use it outside of that classroom, and people are going to call it pointless, and IMO they'd be right.
There’s body of knowledge a person trained in a given area ought to know without use of computers or notes. There are things a person who calls themself “an engineer” or “a physicist” ought to know off the top of their head. A person going into mechanical engineering ought to have some familiarity with how to integrate without using a computer. Such is my belief.
I absolutely agree there's a minimum baseline you need, but my argument is more that I feel like, from my experiences in academia, a lot of it, particularly in intro courses, is often focused around verbatim memorization as a proxy for knowing how to use the rote memorized thing, and while I've always been quite good at rote memorization, other people often are not, and get filtered by those classes.
e.g. I saw several intro CS courses filter people who couldn't write Java or C code cold with precisely correct syntax, and baseline physics classes filter people who couldn't keep a bunch of identities memorized well, when I claim that neither of those is directly useful as a skill in most circumstances, or necessary in the problem domain.
The problem is the lack of learning the material. You don't, IMO, directly care how they produced the answer, you care about it only as a proxy for them learning the material well enough to solve the problem.
And making people do them in person with no technology is unrealistic - not because it can't be done, but because at that point, it's not a reflection of how you'd use it outside of that classroom, and people are going to call it pointless, and IMO they'd be right. You would be correct that anyone who met that bar would have likely learned the material, but you'd also have excluded people who would have met the bar of "can use the material to the degree of familiarity needed going forward".
I think a reasonable compromise would be to let students collaborate on the exams in the classroom, without external access - while I realize some people learn better on their own in some subjects, as long as everyone contributes some portion of the work, and they go back and forth on agreeing what the right answers are, then you're going to make forward progress, even if that ruins the exam process as anything other than a class-wide metric. You could subdivide it, but then that gets riskier as there's a higher chance that the subset of people doesn't know enough to make progress. Maybe a hint system for groups, since the goal here is learning, not just grading their knowledge going in?
Not that there's not some need for metrics, but in terms of trying to check in on where students are every so often, I think you need to leverage how people often end up learning things "in the wild" - from a combination of wild searching and talking to other people, and then feedback on whether they decided you could build an airplane out of applesauce or something closer to accurate.