pony_check.pony

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
"""
PonyCheck is a library for property based testing
with tight integration into PonyTest.

## Property Based Testing

In _traditional_ unit testing, the developer specifies one or more input
examples manually for the class or system under test and asserts on certain
output conditions. The difficulty here is to find enough examples to cover
all branches and cases of the class or system under test.

In property based testing, the developer defines a property, a kind of predicate
for the class or system under test that should hold for all kinds or just a
subset of possible input values. The property based testing engine then
generates a big number of random input values and checks if the property holds
for all of them. The developer only needs to specify the possible set of input
values using a Generator.

This testing technique is great for finding edge cases that would easily go
unnoticed with manually constructed test samples. In general it can lead to much
higher coverage than traditional unit testing, with much less code to write.

## How PonyCheck implements Property Based Testing

A property based test in PonyCheck consists of the following:

* A name (mostly for integration into PonyTest)
* One or more generators, depending on how your property is laid out.
  There are tons of them defined in the primitive
  [Generators](pony_check-Generators.md).
* A `property` method that asserts a certain property for each sample
  generated by the [Generator(s)](pony_check-Generator.md) with the help of
  [PropertyHelper](pony_check-PropertyHelper.md), which tries to expose a
  similar API   as [TestHelper](pony_test-TestHelper.md).
* Optionally, the method `params()` can be used to configure how PonyCheck
  executes the property by specifying a custom
  [PropertyParams](pony_check-PropertyParams.md) object.

The classical list-reverse example:

```pony
use "collections"
use "pony_check"

class ListReverseProperty is Property1[List[USize]]
  fun name(): String => "list/reverse"

  fun gen(): Generator[List[USize]] =>
    Generators.list_of[USize](Generators.usize())

  fun property(arg1: List[USize], ph: PropertyHelper) =>
    ph.array_eq[USize](arg1, arg1.reverse().reverse())
```

## Integration into PonyTest

There are two ways of integrating a [Property](pony_check-Property1.md) into
[PonyTest](pony_test--index.md):

1. In order to pass your Property to the PonyTest engine, you need to wrap it
  inside a [Property1UnitTest](pony_check-Property1UnitTest.md).

```pony
  actor Main is TestList
    new create(env: Env) => PonyTest(env, this)

    fun tag tests(test: PonyTest) =>
      test(Property1UnitTest[String](MyStringProperty))
```

2. Run as many [Properties](pony_check-Property1.md) as you wish inside one
  PonyTest [UnitTest](pony_test-UnitTest.md) using the convenience function
  [PonyCheck.for_all](pony_check-PonyCheck.md#for_all) providing a
  [Generator](pony_check-Generator), the [TestHelper](pony_test-TestHelper.md)
  and the actual property function. (Note that the property function is supplied
  in a second application of the result to `for_all`.)

```pony
  class ListReversePropertyWithinAUnitTest is UnitTest
    fun name(): String => "list/reverse/forall"

    fun apply(h: TestHelper) =>
      let gen = recover val Generators.list_of[USize](Generators.usize()) end
      PonyCheck.for_all[List[USize]](gen, h)(
        {(sample, ph) =>
          ph.array_eq[Usize](arg1, arg1.reverse().reverse())
        })
      // ... possibly more properties, using `PonyCheck.for_all`
```

Independently of how you integrate with [PonyTest](pony_test--index.md),
the PonyCheck machinery will instantiate the provided Generator, and will
execute it for a configurable number of samples.

If the property fails using an assertion method of
[PropertyHelper](pony_check-PropertyHelper.md),
the failed example will be shrunken by the generator
to obtain a smaller and more informative, still failing, sample
for reporting.

"""
use "pony_test"

primitive PonyCheck
  fun for_all[T](gen: Generator[T] val, h: TestHelper): ForAll[T] =>
    """
    Convenience method for running 1 to many properties as part of
    one PonyTest UnitTest.

    Example:

    ```pony
      class MyTestWithSomeProperties is UnitTest
        fun name(): String => "mytest/withMultipleProperties"

        fun apply(h: TestHelper) =>
          PonyCheck.for_all[U8](recover Generators.unit[U8](0) end, h)(
            {(u, h) =>
              h.assert_eq(u, 0)
              consume u
            })
    ```
    """
    ForAll[T](gen, h)

  fun for_all2[T1, T2](
    gen1: Generator[T1] val,
    gen2: Generator[T2] val,
    h: TestHelper)
    : ForAll2[T1, T2]
  =>
    ForAll2[T1, T2](gen1, gen2, h)

  fun for_all3[T1, T2, T3](
    gen1: Generator[T1] val,
    gen2: Generator[T2] val,
    gen3: Generator[T3] val,
    h: TestHelper)
    : ForAll3[T1, T2, T3]
  =>
    ForAll3[T1, T2, T3](gen1, gen2, gen3, h)

  fun for_all4[T1, T2, T3, T4](
    gen1: Generator[T1] val,
    gen2: Generator[T2] val,
    gen3: Generator[T3] val,
    gen4: Generator[T4] val,
    h: TestHelper)
    : ForAll4[T1, T2, T3, T4]
  =>
    ForAll4[T1, T2, T3, T4](gen1, gen2, gen3, gen4, h)