Grading an Essay by Noah Smith

Using ChatGPT

I wrote:

Grade on a scale from A to F, with + and -. If the essay does not seem to make any particular claim, say that you cannot grade it, and explain why. Otherwise, it earns an A if it articulates strong arguments against its claim and responds to those arguments. It earns an F if it merely insults people who disagree or claims that it understands the motives of those who disagree and that those motives are bad. It earns a B if it at least mentions that there is a possibility that the claim is wrong or indicates potential ways in which the claim could turn out to be wrong. It earns a D if it denies that the claim could possibly be wrong. It earns a C- if it neither considers the possibility that it could be wrong nor argues that the claim must certainly be right. Are those criteria clear?

ChatGPT wrote back:

Yes, the criteria are clear. Here is how I will grade the essay based on your criteria:

• A: Strong arguments against the claim are articulated and responded to.

• B: Acknowledgment of the possibility that the claim could be wrong, or indication of ways the claim might turn out to be wrong.

• C-: Neither considers the possibility of being wrong nor asserts the certainty of being right.

• D: Denies any possibility that the claim could be wrong.

• F: Insults people who disagree or ascribes bad motives to them.

If the essay does not make a clear claim or argument, I will indicate that it cannot be graded according to these criteria because a foundational aspect of a persuasive essay is the presentation of a clear claim or argument for evaluation.

Please present the essay when you're ready, and I will proceed with the grading.

I then input the latter half of a recent essay by Noah Smith.

Who has the power to stand up and push back against a flood of online Chinese and Russian government propaganda? Currently, under the U.S.’ marketplace-of-ideas doctrine, that duty falls to private citizens. It’s normal, everyday Americans who must spend hours upon hours of their day replying to viral tweets and desperately making TikTok rebuttal videos, lest their fellow Americans fall victim to the siren song of totalitarianism. We are all somehow expected to sacrifice hours of day to do thankless, unpaid work pushing back against illiberal narratives spread relentlessly by people for whom spreading those narratives is a full-time salaried job. Meanwhile, under this onslaught of Russian and Chinese messaging, the U.S. government stands serenely aloof from the fray. Administration officials will give speeches and pen the occasional op-ed in Foreign Affairs, government officials make anodyne tweets, and the Voice of America is still putting out articles that no one reads, but the kind of coordinated forceful official advocacy of liberal values that we saw under FDR is considered propaganda to be avoided at all cost. It should be obvious that this is a recipe for liberalism’s utter defeat. Imagine if a country being invaded by a conquering army declared that government shouldn’t get involved in violence, and outsourced the defense of the nation to private citizens with homemade guns? That is effectively what the U.S. government is doing by ceding the information sphere to foreign governments and private citizens. We are so afraid of being propagandists that we are refusing to counter the efforts of powers who have no such fear. The U.S. government needs to get back into the information warfare game. First, this means articulating liberal values to the American people, and explaining how U.S. government actions are intended to promote those values. The right messaging medium probably won’t be preachy films and stylized posters like it was in the 1940s (though who knows, maybe it might!). Instead it might involve 15-second videos, or podcasts by elected politicians, or whatever messaging best reaches the online populace of the 2020s. Despite being a modestly popular blogger, I’m not really the expert in what moves the masses. But someone out there is, and the government can find and employ them like it employed Frank Capra in the 1940s. Second, the government needs to act to limit the ability of the totalitarian powers to push their own messaging to our people. FDR’s Office of War Information reviewed Hollywood films to make sure they weren’t anti-American; that sort of blunt censorship was almost certainly unconstitutional, and shouldn’t be considered today. But there are analogues that do make sense. The most important step is probably to force TikTok to sell itself to an American company — a move the U.S. government has repeatedly considered and then shied away from. There are, of course, dangers to this approach. The U.S. government’s censorship efforts during the Vietnam War silenced much-needed dissent in that conflict, and the Bush Administration was famously less than forthright with the American people when persuading them to accept a war in Iraq. And there’s always the danger that the administration in power will use government resources to present events in a way that puts itself in a positive light and sours people on the opposition. These are potential abuses that we need to take seriously. Instead of censorship, our defensive efforts should focus entirely on preventing foreign governments from controlling our information ecosystem. And all U.S. government communication should be open and honest about where it comes from, so that it can be subject to the same scrutiny that any other piece of information receives. But simply sitting back and trusting in the Marketplace of Ideas to reach an optimal equilibrium is no longer a viable option for a liberal society. Totalitarian powers are on the march, and social media gives them the opportunity to reach American hearts and minds more directly than ever before. One of our government’s essential functions is to guard us against such powers. If we force it to abnegate that function, mainlining totalitarian propaganda out of fear of creating our own, we’re all a bunch of suckers.

ChatGPT replied,

The essay makes a clear claim: that the U.S. government needs to actively engage in countering foreign propaganda and disinformation, and can no longer rely solely on the "marketplace-of-ideas" doctrine where private citizens are expected to counter such efforts.

Now, according to your grading criteria:

• The essay does not articulate strong arguments against its claim and respond to those arguments; therefore, it doesn't earn an A.

• The essay does acknowledge potential issues and dangers with the government taking a more active role, suggesting that the author is aware of the possibility that the claim could have flaws or could turn out to be wrong. This would align with the criteria for a B. The mention of the dangers of government overreach, censorship, and potential for abuse reflects an understanding of the complexity and risks involved.

• There is no outright denial that the claim could possibly be wrong, so it does not earn a D.

• The essay does not solely insult people who disagree or ascribe bad motives to them, thus it does not earn an F.

• However, it doesn't simply remain neutral without considering whether the claim is right or wrong, which would have warranted a C-.

Therefore, based on your criteria, the essay would earn a B grade. It makes a case for a particular course of action, recognizes potential downsides and risks associated with this course, and avoids descending into ad hominem attacks against those who might disagree.

If I were grading it myself, I would probably give it a slightly higher grade, say B+. Noah does bring up the dangers of government getting into the propaganda business. I happen to place a higher weight than he does on those dangers, but that does not mean he should be graded down for that.

My point is that it is feasible to get ChatGPT to do this. As soon Openai allows users like me can create apps (Openai calls them GPTs), my understanding is that I will be able to create a grading app that anyone can submit essays to. I think that a project like this could be very worthwhile. I think it has the potential to improve public discourse.

Thanks to a reader for encouraging this approach. His timing could not have been better, since Openai subsequently announced the new ability to create GPTs or apps.

I welcome your advice and support.

referenced above:

@

Liberalism is losing the information war