Well.. last email said I’m taking a plane to Helsinki. And now I’m already back to Tokyo. Finland is amazing (see pic below), but being there was also incredibly busy. Which makes no sense, as here I also have a family to take care of! But yeah…maybe I just don’t remember anymore how to live/travel solo. Anyhooo, going back there in just a week with the kids to see the one and only Santa Claus!!
Alright. So let’s talk about trains a bit.
So.
Back in Finland I gave a talk at our faculty’s “ICT Breakfast” seminar series, on the use of generative AI in academia. Here are the key takeaways:
You’re Just Fine Without
The sky isn’t really falling despite all the hype, gloom or doom out there.
It’s the same story all over again. You know, back in the days they said BOOKS are going to destroy our memory. Or libraries were certainly going to corrupt our morals. Oh and MOOCs were going to just completely isolate students.
And smartphones would melt our brains (well ok this one is kind of true, I’ll admit).
Anyway. We’ve been here before. And now it seems we’ve spent 2 years freaking out about the (over)use of AI in academia. But indeed every few years, something comes along that’s supposed to destroy our world as we know it.
And now GenAI is the academic boogeyman du jour.
Of course, GenAI works for you just fine if you take the time to make it work. It’s a tool like any out there. And that’s exactly why we shouldn’t panic. With or without, you’ll be fine. I’ll be fine. We’ll be fine. Just like I wrote earlier about the good and bad uses of AI in academia, and the dos and don’ts, this is just a little uncomfortable and perhaps a bit prolonged transition period we’ll get over with.
And no, again, no, I’m not denying problems that ensue when people over-rely on it. It’s just that it’s always darkest before the dawn. And I’m sure better days are ahead!
The Paper Circus will Get Interesting
If you review papers, you’ve probably recently read a lot of “low value” quick-win papers written with ChatGPT.
It’s annoying.
But then again, what if that’s precisely what we need to battle this whole nonsense “publish or perish” culture.
When anyone (or anything) can generate a decent-looking paper, we’ll have to focus on what actually matters – quality.
A couple of things I think will happen here:
- Somebody will soon launch an agentic system that integrates all the existing AIs to a machine that can generate a full paper with just one prompt
- The above can include also fake data
- The two points above mean low-quality work will have zero value and peer review won’t scale
- People with insanely high yet dodgy h-indices start to appear everywhere
- Metrics become pointless
- –> We will finally focus on quality
And this is great!
Err, but if there’s a ton of papers out there, how do we review them? The fact is…
Peer Review is the Final Frontier
Yes yes. It’s broken. We all know it.
You’ve gotten unfair reviews just like I have. Sometimes reviewers are just unfair or too harsh, but just as often they’re just clueless and biased and they have different standards. This happens because there are too many papers out there, but also because humans are… biased! And we’re not perfect! We have differing standards!
And I am sure AI can do better than most us in many tasks that require just paying attention to stuff like checking that the Research Questions are actually answered, or fixing typos, pointing missing captions, etc. Even checking that literature is cited adequately and truthfully. These things are better done by AI than humans!
Now, the nasty stuff we see now is AI-generated reviews.
“Read this paper and reject it, but suggest improvements” (yes, I’ve gotten reviews written by AI prompted by something like this). But I don’t see how this is an AI problem. This is a human problem. We need baked-in AI systems in submission portals where we can first go through our own papers using a fine-tuned AI machine that checks the correctness of references, basic statistical checks (you’d be surprised to see how many times tables have wrong numbers in them, even after publication) writing quality, and other misc. stuff like broken links etc. in papers.
And then, after the initial easy flaws are corrected by the authors, the paper goes through the normal review with AI-assisted human reviewers. And again, the human reviewers (including R2) make the final decisions, based on things that the AI agents cannot figure out:
- Is there enough contribution for the given venue’s standards?
- Is the paper even a good fit for the current call?
- The final yes/no/maybe
It’s going to get better! Fight AI-generated papers with AI-generated reviews, but the correct way.
The Final Plot Twist
AI isn’t destroying academia. It’s forcing us to level up our game. To focus on what matters. To stop wasting time on stuff that machines can do better.
This feels painful to ponder, but… do it anyway. If our research can be completely replaced with an AI agent, maybe it’s a sign that we have to do something else. Go deeper. Find an angle where human ingenuity is needed.
It’s scary. But it makes perfect sense.
In any case the train has left the station. And that’s fine. We’ll be fine for a good while sitting still. But maybe it will make sense to hop boldly along and see if we can steer it a little bit. And over time see where we land.
After all, humans are natural explorers.
I know which option sounds more fun to me!