Generate your own types in fast-check
Contents
TLDR
If your custom type is:
class ImaCustom {constructor(public s: Set<string>, public a: string[]) {}}
… and if you're just getting started with this (like me) and thinking: "ok, now how do I randomly generate my ImaCustom
instances?".
Try generating random sets and arrays which you can then use to create your ImaCustom
s.
e.g.
import fc from "fast-check";test("Some property you want to test", () => {fc.assert(fc.property(// generate [Set<string>, string[]] tuples with some size constraintsfc.tuple(fc.set(fc.string(100), 25), fc.array(fc.string(100), 50)),([s, a]) => {const i = new ImaCustom(s, a);// assert your property}));});
That's basically what I wanted to write about 😊
A slow start to a fast-check
I guess you already know what property based testing is if you're here. Recently, I was testing something… which tbh, I didn't really know how I wanted to work. But in my mind, I could think of certain properties; certain things which had to hold on the datastructure in question when this operation happens. So - I figured I'd reach for that testing approach I know about; think is amazing; but almost never actually use 🤔
Anyway - it didn't take long to find fast-check. I had used jsverify before, but I've forgotten the API and who cares anyway - I just want to code a few properties and get on with my app… maybe actually get it to a usable state 😅
So… I'm searching online:
Ye, I know more or less what property based testing is. Great - first few examples show how to generate basic types. Cool, found the list of built-in… oh "arbitraries" they're called… hmm fancy that - and I thought I had an arbitrary name. Now if I could only get to the "how to generate your own types" in the documentation and give these stressed eyeballs a break.
Far as I can tell - that section doesn't exist in the docs. I did eventually have a 🤦♂️ moment and realised that any "custom" datastructure must be made up of more basic types.
Show me the code!
This is basically the data I'm working with here (Immutable.js Set / OrderedSet):
export interface ContentProps {include?: OrderedSet<string>;exclude?: Set<string>;}
… which is passed in to Content
's constructor to create an instance. But the constructor has some constraints:
if (include.has("") || exclude.has("")) {// ... throw Error}const invalidValues = include.intersect(exclude);if (invalidValues.size > 0) {// ... throw Error}
So basically, no empty string in the sets and they must be mutually exclusive.
Finally, a Content
can sync
given a list of file paths string[]
… and this is what I came up with for what I had in mind. NOTE: you do not need to understand or even read this code:
public sync(filePaths: string[] = []): Content {const filePathsAsSet = Set(filePaths);const setOfFilePathsNotAlreadyExcluded = filePathsAsSet.subtract(this.exclude);const oldValidSectionsInRightOrder = this.include.intersect(setOfFilePathsNotAlreadyExcluded);const newValidSections = setOfFilePathsNotAlreadyExcluded.subtract(oldValidSectionsInRightOrder);const include = oldValidSectionsInRightOrder.concat(newValidSections);const exclude = filePathsAsSet.intersect(this.exclude);return new Content({dirName: this.dirName,include,exclude});}
One property I can think of is this:
In
result = content.sync(filePaths: string[])
- any filePath infilePaths
which is also incontent.exclude
should be inresult.exclude
and not inresult.include
This is the property being demoed below, but if you want to get a feel of what sync
is about, here are a couple more properties which come to mind:
- Any existing values in
content.exclude
which are not present infilePaths
should not be inresult.exclude
- Any existing values in
content.include
which are not present infilePaths
should not be inresult.include
- Any existing values in
content.include
which are infilePaths
are kept in the same order inresult.include
Generating data
To express this property, I first want to express how the "ingredients" are generated, i.e. the arbitrary data:
const arbitrarySyncTestProps = fc.tuple(fc.set(fc.string(100), 25),fc.set(fc.string(100), 25),fc.array(fc.string(100), 50));
So - generate me a tuple of 2 sets of strings with a max size of 25 and whose strings are no more than 100 characters in length. Also, throw in an array of strings for good measure. Super simple - but now I want to get more specific… I should have no empty strings in either Set, nor in the array come to think of it - as those elements are meant to end up in a Set which doesn't want empty strings. Also, I want the two Sets to be mutually exclusive… and while we're at it - I think it won't do to just have random strings. Taking another look at the property:
any filePath in
filePaths
which is also incontent.exclude
So, maybe I should ensure I get some strings which are in both filePaths
and exclude
:
const arbitrarySyncTestProps = fc.tuple(fc.set(fc.string(100), 25),fc.set(fc.string(100), 25),fc.array(fc.string(100), 50)).map(([include, exclude, filePaths]) => {// make filePathsToAddToExclude so tests can be more meaningfullet filePathsToAddToExclude = [];if (filePaths.length > 0) {const numberOfPathsToAddToExclude = Math.floor(Math.random() * filePaths.length);for (let i = 0; i <= numberOfPathsToAddToExclude; i++) {filePathsToAddToExclude.push(filePaths[i]);}}// transform the data so it meets Content's pre-conditionsconst includeOrderedSet = OrderedSet(include.filter((x) => x !== "" && !exclude.includes(x)));const excludeSet = Set(exclude.concat(filePathsToAddToExclude).filter((x) => x !== ""));const filePathsArray = filePaths.filter((x) => x !== "");return [includeOrderedSet, excludeSet, filePathsArray];});
Expressing the property
Finally, the property can be expressed with a few simple assertions:
([include, exclude, filePaths]: [OrderedSet<string>,Set<string>,string[]]) => {const content = new Content({include,exclude,});const result = content.sync(filePaths);const pathsInBothFilePathsAndExclude: Set<string> = content.exclude.intersect(Set(filePaths));expect(pathsInBothFilePathsAndExclude.isSubset(content.exclude)).toBe(true);expect(result.include.intersect(pathsInBothFilePathsAndExclude).size).toBe(0);};
Putting it all together
import fc from "fast-check";import { Set, OrderedSet } from "immutable";import { Content } from "./Content";describe("Content", () => {describe("content.sync(filePaths: string[])", () => {test("Any filePath in filePaths which is also in content.exclude should be in result.exclude and not in result.include", () => {const arbitrarySyncTestProps = fc.tuple(fc.set(fc.string(100), 25),fc.set(fc.string(100), 25),fc.array(fc.string(100), 50)).map(([include, exclude, filePaths]) => {// make filePathsToAddToExclude so tests can be more meaningfullet filePathsToAddToExclude = [];if (filePaths.length > 0) {const numberOfPathsToAddToExclude = Math.floor(Math.random() * filePaths.length);for (let i = 0; i <= numberOfPathsToAddToExclude; i++) {filePathsToAddToExclude.push(filePaths[i]);}}// transform the data so it meets Content's pre-conditionsconst includeOrderedSet = OrderedSet(include.filter((x) => x !== "" && !exclude.includes(x)));const excludeSet = Set(exclude.concat(filePathsToAddToExclude).filter((x) => x !== ""));const filePathsArray = filePaths.filter((x) => x !== "");return [includeOrderedSet, excludeSet, filePathsArray];});fc.assert(fc.property(arbitrarySyncTestProps,([include, exclude, filePaths]: [OrderedSet<string>,Set<string>,string[]]) => {const content = new Content({include,exclude,});const result = content.sync(filePaths);const pathsInBothFilePathsAndExclude: Set<string> = content.exclude.intersect(Set(filePaths));expect(pathsInBothFilePathsAndExclude.isSubset(content.exclude)).toBe(true);expect(result.include.intersect(pathsInBothFilePathsAndExclude).size).toBe(0);}));});});});
In conclusion
It was fun to revisit property based testing. One issue I know will come up is the matter of performance. This is taking circa 5 seconds to run so I know its going to be problem. Maybe I'll only run the property based tests when an env var is set, or configure fast-check
to "be more fast!" … somehow (e.g. generate less tests). In any case, it's still useful and I have already found a couple of issues with my implementation.
Any feedback on this is more than welcome 👇
🍻