Why use Hitchstory instead of Behave, Lettuce or Cucumber (Gherkin)?

HitchStory and Gherkin are both DSLs for writing user stories that can double as acceptance tests, but they have very different approaches and philosophies.

Gherkin primarily emphasizes the execution of feature documentation written in "business-facing" text.

The designers are insistent that it is primarily not an integration testing framework.

Whereas hitchstory was built primarily as an integration testing framework with:

• The use of strictly typed YAML to define specs for readability, precision, clarity, terseness and ease of maintenance.

• Specifying all behavior precisely rather than just "business facing text" but still enforcing a separation of test concerns.

• Specification documentation test triality such that documentation is autogenerated and specifications are tested.

What Gherkin got right

Cucumber and their related tools became popular for good reasons, even though they made many mistakes. They also got a lot of non-obvious stuff right:

• Emphasis on readability of the specs - something that xUnit tests are typically not good at.

• The idea that specifications should double as tests.

• The deliberate use of a simpler non-turing complete language for specifications.

• The that there should be a separation of concerns.

• Built in parameterization.

Example: Gherkin vs. HitchStory

From the training wheels come off Aslak Hellesøy described how Cucumber was used to write tests (in his view, poorly):

Scenario: Successful login
  Given a user "Aslak" with password "xyz"
  And I am on the login page
  And I fill in "User name" with "Aslak"
  And I fill in "Password" with "xyz"
  When I press "Log in"
  Then I should see "Welcome, Aslak"

And how he thinks stories should be written:

Scenario: User is greeted upon login
  Given the user "Aslak" has an account
  When he logs in
  Then he should see "Welcome, Aslak"

The "uninteresting details" are thus pushed down to the step layer and it becomes more difficult to share step code.

Since the syntax of Gherkin inhibits the expression of complex specifications, the framework kind of encourages this downward concern leakage.

With HitchStory, most stories should look something like this:

Aslak sees welcome message on login:
  steps:
  - visit: /login
  - fill form:
      username: Aslak
      password: xyz
  - click: log in button
  - should contain:
      item: welcome banner
      message: Welcome, Aslak

With documentation generated from this targeted at specific stakeholders that may strip out "uninteresting" parts of the spec.

Trying to be an English-like Language

English is vague. English is verbose. English is messy. English is imprecise. These are ideal qualities for some purposes but they are not ideal for writing specifications.

Like actual code, to be effective, specifications are exact, precise and DRY. Even non-executable specifications benefit from these qualities.

The imprecision of English is drawn out in Gherkin specifications partly due to its lack of inheritance. This encourages vague stories - they usually lack precision and critical details and end up serving as a sort of high level description of what the program it's describing does.

Parser hell

#

Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems. - Jamie Zawinksi

Because of the Englishy nature of the language, parsing is also difficult. Gherkin requires very

There are a multiplicity of examples around the web where it went wrong causing many headaches for users of Cucumber and often leading to hours of unnecessary debugging and workarounds.

One of the facets of the syntactic ambiguity is that some parsers require you to write regular expressions to parse segments of the Gherkin.

Given/When/Then are training wheels that don't come off

BDD emphasizes the notion of using Given, When and Then as a way of structuring example based test cases.

And, to be fair, almost every test case should follow this pattern. However, these keywords do not hold any meaning for the parser and are ignored.

For this reason, hitchstory advocates the Given/When/Then pattern but, apart from 'given' does not use these keywords. "Given" is a user defined mapping of arbitrary YAML that can be used to configure how set_up behaves, while the meat of the stories is in the steps.

given: something: something else steps: - step 1 # when - step 2 # then - step 3 # and - step 4 # but

Verbosity

Why does this above matter? It results in executable specifications which are naturally terser and present a higher density of information - about your story - with less text.

One of the side effects of the multiplicity of the above Given/When/Then keywords is greater verbosity.

Weakly/stringly typed

Type safety is a critical feature of programming languages. While type systems can be overly picky about code to the detriment of productivity, in general greater type safety results in fewer bugs and greater confidence in code. This is one reason why typescript is winning out over javascript, or rust over C++ for instance.

At the other end of the spectrum is "stringly typed" code - the weakest of type systems, where everything is a string. Bash and Makefiles are both stringly typed and while it can be ok for short, simple scripts, it will make debugging painful when these scripts grow very large.

Gherkin is stringly typed. Steps are parsed (often with regexes) and strings are pulled out and fed into step code.

HitchStory, on the other hand, is based upon StrictYAML and bakes in schema based parsing.

No inheritance

#

"The short answer, no. Feature files can't inherit from another feature file." - Stack Overflow

Inheritance is, and always has been about keeping code DRY. It is a thorny tool because, while it can reduce repetition it can sacrifice readability. For this reason, Gherkin left it out.

From the Stack overflow question:

"A common background is to login in a system. Login is important but it can often be hidden in the steps."

This is downward concern leakage again.

The steps that lead to login are part of the specification of the system, so why should it be hidden?

This conflict between the desire to keep steps DRY and not hide meaningful details in steps puts users of Cucumber in a difficult position.

In the article how not to repeat yourself in cucumber scenarios there are several attempts to try and how to avoid it but none work particularly well.

Executable specifications are not always suitable for the non technical

Talking about music is like dancing about architecture -- Marvin Mull

Most software packages under test don't actually interact with humans they interacts with other software.

This interaction can be via code - in which case the specification ought to be defined with snippets of code.

The interaction might instead be with a REST API - in which case, in order to be able to define the interaction, you must understand and define the semantics of REST APIs and be able to write them out if you are defining a spec.

The specification should not attempt to conceal or dumb down these details.

To be fair, this isn't a problem with the syntax of Gherkin or design of Cucumber - it is mostly a cultural affectation.

It should be useful to developers or testers even if stakeholders arent interested

Due to all of the above problems, Gherkin does not have a wide adoption among developers.

It's not like deliberately simplified languages are not popular with developers for other forms of configuration.

If programmers don't like using a tool or technology that interfaces to their code, forget about getting the business to use it.