A Gherkin Lexer

Learn about how to implement text indentation and a Gherkin lexer.

A lexer is an interesting topic when parsing text because it can often be confused with the parser itself. If we look at the Gherkin example in the code snippet below, we can see that there are a lot of individual characters and words, and it might be difficult to even figure out where we should start parsing this:

Press + to interact
Feature: Overdue tasks
Let users know when tasks are overdue, even when using other
features of the app
Rule: Users are notified about overdue tasks on first use of the day
Background:
Given I have overdue tasks
Example: First use of the day
Given I last used the app yesterday
When I use the app
Then I am notified about overdue tasks

Operating directly on individual characters or entire words can be complicated from a ...