top of page

Man vs Machine: A Case Study in Grammar Checkers

It's easy to say that grammar checkers are missing something that human editors have, but let's put this idea into practice using two of the most popular grammar checkers: ProWritingAid and Grammarly. I have chosen two well-known classics in the public domain for this: Alice in Wonderland by Lewis Carroll and Frankenstein by Mary Shelley. While I don't believe that just because something has been published it is perfect, these are two books that have been published (and therefore edited) many times. As a result, they present excellent examples as to why an experienced editor is needed when using grammar checkers.


*All text courtesy of Project Gutenberg

The Basics

Starting with Alice in Wonderland, here are some pages with the errors flagged by both Grammarly and ProWritingAid.


Image of pages from Word with markup by Grammarly and ProWritingAid

Both programs have flagged word errors (usually spelling, but sometimes incorrect usage) in red and the passive voice in purple. ProWritingAid identifies stylistic errors in orange and grammatical errors in light blue. Grammarly marks a lack of clarity with dark blue and style errors with turquoise. In approximately 26,000 words, the following number of errors were found.


Grammarly = 1114 errors

ProWritingAid (fantasy) = 731 errors

ProWritingAid (general) = 854 errors


This simple statistic already shows a discrepancy between programs.


There is minimal overlap in the errors found between programs. Below is the first page of the above text with only one program's flags shown. ProWritingAid is on the left; Grammarly is on the right.

Image of a page from Alice in Wonderland showing flags from ProWritingAid on the left and Grammarly on the right.

ProWritingAid found 20 errors; Grammarly found 29. Of these, only 5 were flagged by both programs and in only 3 of these cases are both programs in agreement about what the error.


The same thing can be seen in Frankenstein.


Text from Frankenstein showing flags from Grammarly and ProWritingAid

Even more so than with Alice in Wonderland, the results for Frankenstein (~75,000 words) are different depending on the program used.


Grammarly = 3902 errors

ProWritingAid (horror/sci-fi) = 2324 errors


It is also interesting to note that when running Frankenstein through Grammarly, it flags almost every use of the past perfect tense as incorrect. The past perfect is used to refer to events that happened in the story's past when the story is written in the past tense, and since Frankenstein uses a lot of flashbacks, the past perfect is often used. To a modern reader, this can make the text feel a little clunky, but ultimately, this is a structural and stylistic choice, not an error.


The above clearly shows that there is no such thing as completely objective when it comes to something creative such as writing, even from "objective" software.


Alice in Wonderland

Let's take a closer look at some specifics.


Paragraph from Alice in Wonderland marked up by Grammarly

Grammarly has flagged 14 errors. (Although it is worth noting that the web version only flags 4.) Following Grammarly's suggestions produces the following (changes underlined):


After finding that nothing more had happened, she decided to go into the garden at once, but, alas, for poor Alice! She found she had forgotten the little golden key when she reached the door. When she went back to the table for it, Alice found she could not possibly get it: Alice could see it quite plainly through the glass and tried her best to climb up one of the legs of the table, but it was too slippery; and when she had tired herself out with testing, the poor little thing sat down and cried.


These two versions aren't that different, however, Grammarly has changed the feeling of the text by breaking it into shorter sentences. It has also replaced most of the shes with Alices which makes it feel like Alice is repeated too many times. This was done in the name of reducing the repetition of she, but names are more intrusive to the reader than pronouns. This is why we have pronouns. They integrate into the text more smoothly. Grammarly has basically swapped one issue for another.


The revision has also introduced three changes to the meaning. By eliminating "After a while" in the first sentence, the passage of time is removed creating a disconnect in the first sentence. This gives Alice's character more impatience than she had in the original. Reach it has been changed to get it, which subtly changes the meaning. Get it is very general. It doesn't give the reader an idea of what Alice's specific struggle in getting the key is. Reach it reinforces the fact that she has shrunk and that her distance from the key is the obstacle. And then there is the change of trying to testing in the final sentence. Grammarly flags it as "appear[ing] repeatedly in this text" despite it appearing only once. This edit makes the sentence nonsensical. The words can be synonyms, but not in this case where trying is being used to mean to make an attempt, not to test. The context in which the word was used was not considered.


ProWritingAid flags nothing in this paragraph when set to check fantasy writing. This reinforces two things. 1) the choice of tool is important to get the best results and 2) there is no such thing as objective when it comes to writing.


This second point is important. Grammar checkers are often portrayed as scientific tools that are infallible, but they are created by humans and therefore cannot be fully objective. The intended function is also important. Grammarly is designed primarily for business writing, so it is looking to remove more creative and informal writing because that is what it is programmed to do. ProWritingAid is designed for fiction writing. This means it is more likely to accept non-standard grammar.


This distinction is evident in the following paragraph where both ProWritingAid and Grammarly have flagged errors. The first is ProWritingAid; the second is Grammarly.


Paragraph from Alice in Wonderland with ProWritingAid showing errors.
A paragraph from Alice in Wonderland with Grammarly showing errors.

Here are the revisions as suggested by ProWritingAid, then Grammarly:


The rabbit-hole went straight on like a tunnel for some way, and then dipped suddenly down, so suddenly that Alice had not a moment to think about stopping herself before she fell down a deep well.


The rabbit hole went straight on like a tunnel for some way and then dipped suddenly down, so suddenly that Alice did not have a moment to stop herself before falling into a very deep well.


The differences in the three versions are subtle, but in both cases, the changes have had an impact on the style of the writing and the intimacy of the text. The original has a free-flowing quality that matches the feeling of Alice falling down the well. Part of this is the length of the sentences (which both programs shortened), but it's also in the way Carroll has placed the reader inside Alice's head. The descriptions of Alice not having "a moment to think about stopping herself" and "she found herself ..." give the reader a sense of being in the moment with Alice inside her head. Both ProWritingAid and Grammarly have removed this intimacy so it feels more like the reader is watching Alice fall as opposed to falling with her. In both cases, the suggested edits are not incorrect, but they change the voice of the author.


Frankenstein

Grammarly

These issues aren't limited to Alice in Wonderland.


Here is a paragraph from Frankenstein.


A paragraph from Frankenstein checked by Grammarly

Grammarly finds six errors in this passage. Three grammatical errors involving punctuation, a sentence fragment, an unclear antecedent and a use of passive voice.


Here is the revised paragraph according to Grammarly's recommendations.


Several months passed in this manner. Beaufort grew worse; She spent all her time attending to him; her means of subsistence decreased, and in the tenth month, her father died in her arms, leaving her an orphan and a beggar. This last blow overcame her, and she knelt by Beaufort’s coffin, weeping bitterly when my father entered the chamber.


The flagging of passed is one of the best examples of why grammar checkers can't replace an editor. Grammarly calls this a sentence fragment. It isn't. It has a subject and a verb and forms a complete idea. That is the definition of a sentence, so this suggestion is objectively wrong. Removing the phrase "in this manner" satisfies the software, but this removes descriptive information from the sentence. And you would have to make this decision yourself. Following Grammarly's suggestions sends you into an unending loop as follows:


Several months she was passed in this manner. (change in meaning, change to passive voice)

Several months she passed in this manner. (change in meaning)

Several months passed in this manner. (we return to where we started)


This one sentence undermines the authority of Grammarly. The error it flags ultimately leads you back to the error it flagged when following its suggestions. In this case, it doesn't know what it's doing.


The next error "Her father," flags her as an unclear antecedent, but it isn't acting as a pronoun in this sentence. It is an adjective describing father. The software has misread the sentence and flagged an error where there is none.


The change in the last sentence, moving the comma to after coffin from after bitterly changes the meaning of the sentence. The original says that the woman was crying as "my father entered the chamber." Her tears are not the result of the narrator's father entering the room. With the edit, the father did not find her crying; she started crying because he entered. This one small edit completely changes the timing of the events and the reason for her tears.


The remaining edits are stylistic choices. The original isn't wrong and neither are the suggestions. However, in this case, I believe most of these suggestions are in opposition to the author's voice. But that is up for debate and one of the joys of working with an editor.


ProWritingAid

Here is the same passage reviewed by ProWritingAid.


A paragraph from Frankenstein assessed by ProWritingAid

When set to check horror text, ProWritingAid finds five errors in this passage. Three sentences starting with the same word in a row and two grammatical errors involving punctuation. The only thing that both programs flagged is the lack of a comma after month. This is what the paragraph looks like with ProWritingAid's suggestions:


Several months passed in this manner. Her father grew worse; her time was more entirely occupied in attending him; her means of subsistence decreased; and in the tenth month, her father died in her arms, leaving her an orphan and a beggar. This last blow overcame her, and she knelt by Beaufort’s coffin, weeping bitterly, when my father entered the chamber.


The two added commas are very minor changes. They make a slight adjustment to the rhythm, which is a valid suggestion for the author to consider. However, the suggestion that there are three sentences that start with her in a row is incorrect. Since each clause is separated by a semicolon, they are a single sentence and not three separate sentences.


The Big Takeaways

Like a word processor, grammar checkers are tools that when used effectively can help with your writing. However, like humans, they are not infallible and, unlike humans, they can't adapt on the fly.


If you are going to use a grammar checker, remember the following:

  • Do your research on the options and pick the one that works best for the type of writing you do.

  • Never take a grammar checker as the ultimate authority. Always question their suggestions and ignore the ones that don't work. If a suggestion seems incorrect, do your research to confirm it's correct before making the change. Think about what the proposed change does to the feeling and intention of the text. Does the suggestion change your intention?

  • Sometimes writing incorrectly, unusually or ungrammatically is the best choice for the narrative. Don't let grammar checkers tell you differently.


Grammar checkers are simply an aid, not a second set of eyes. The less knowledge someone has about grammar, spelling and mechanics, the less useful a grammar checker is. They are most effective in catching typos. When it comes to areas of writing that are more creative and less bound by rules—like style and syntax—they become less effective as they try to make the writing match a standard—not enhance the voice of the author. Always have another person review your work before publication. This is the best way to preserve the voice and intention in your writing.

bottom of page