maj/JTest

Files

Tom Hicks c4898f20fe Adds Project.md: team reference doc on the JTest C++ Jasmine-style testing framework

2026-09-24 05:22:00 -07:00

16 KiB

Raw Blame History

JTest — A C++ Test Framework That Feels Like Jasmine

This document is the single point of reference for anyone new to the JTest project. It explains what we are building, why, how it maps to the JavaScript testing framework Jasmine, and how the pieces fit together today. If you only read one file, read this one first.

1. What is JTest, and who is it for?

One-sentence summary: JTest is a C++ unit-testing framework whose API is deliberately modelled on JavaScript's Jasmine, so that anyone who already knows Jasmine can write C++ tests without relearning the vocabulary.

The job-to-be-done: a developer testing C++ code wants the same comfort, readability, and rhythm they get from Jasmine — describe/it/expect/toBe — but in a native, header-and-source C++ form, compiled locally with clang++.

Who the users are:

Test authors — C++ developers who write test files using describe, it, expect(...).toBe(...) etc.
Framework maintainers — the small team (you) that extends JTest itself.

What it is NOT (yet): it is not a drop-in for GoogleTest, not a CI runner, and not complete. It is an evolving design. See §8 Current State & Roadmap.

2. Why the name "Jasmine"? (the vocabulary mapping)

Jasmine is one of the most widely recognised behaviour-driven testing (BDD) frameworks in JavaScript. JTest reuses its three pillars one-for-one, plus the Jasmine "pending" and "failure" idioms. The mapping below is the core mental model of the whole project.

Concept in JTest	Jasmine equivalent	What it does
`describe(label, makeTests)`	`describe`	Names a group of related tests; can nest.
`xdescribe(...)`	`xdescribe`	Same as `describe`, but the whole group is disabled.
`it(label, fn)`	`it` / `spec`	Declares a single test case that runs `fn`.
`xit(label, fn)`	`xit`	Declares a test that is disabled (skipped).
`expect(x)`	`expect`	Returns an `Expectable<T>` you assert on.
`.toBe(matcher)`	`toBe`	Assert equality / truth via a custom matcher function.
`.toBeTrue/False/Null/Equal(...)`	`toBe` family	Type-specific boolean/null/equality matchers.
`.toThrow(...)`	`toThrow`	Assert a callable throws (optionally a matching exception).
`.nevermore()`	`.not`	Inverts the next assertion (negation). See §4.4.
`expect(x).nevermore().toEqual(y)`	`expect(x).not.toBe(y)`	Negative assertion.
`beforeAll/afterAll/beforeEach/afterEach`	same names	Lifecycle hooks, supplied via `DescribeOptions`.
`fail(reason)`	`fail`	Force a test to fail with a message.
`pending(reason)`	`pending`/`xit`	Mark a test as pending (skipped) with a reason.

Naming note for maintainers: JTest can't use Jasmine's .not() because not is awkward in C++. The current chosen name is nevermore(). A shortlist of alternatives was tracked in include/JTest/Expectable.h — revisit before anything depends on it.

3. Project layout

JTest/
├── Project.md            <-- you are here: the team reference doc
├── README.md             <-- high-level, plus a TODO list (see §8)
├── LICENSE               <-- MIT
├── Makefile              <-- builds the framework + the example test binary
├── include/JTest/        <-- public headers (the API surface)
├── src/JTest/            <-- framework implementation (.cpp)
├── examples/             <-- example test sources that exercise the API
└── build/                <-- generated objects/binaries (gitignored target dir)

3.1 Headers (`include/JTest/`) — the public API

Header	Responsibility
`JTest.h`	Umbrella header. Declares the top-level functions: `execute`, `describe`, `xdescribe`, `it`, `xit`, `fail`, `pending`, and the `expect<T>(T)` template. Start here.
`Expectable.h`	The `Expectable<T>` matcher class — all the `toBe/toBeEqual/toBeTrue/.../toThrow` matchers plus negation. Header-only, templated.
`TestBundle.h`	A node in the test tree: holds `tests_` (leaf tests), `children_` (nested bundles), the four lifecycle hooks, and `disabled_`.
`Test.h`	A single leaf test: `label_`, `test_method_` (a `test_fn`), `disabled_`.
`TestResults.h`	Accumulated counters: `total / passed / failed / skipped`, with `operator+`, `+=`, and `operator<<` for reporting.
`TestOptions.h`	Placeholder for per-test options — currently an empty class.
`DescribeOptions.h`	Per-group options: `beforeAll/afterAll/beforeEach/afterEach` setters (fluent, return `*this`).
`ConfigureFunction.h`	`configure_fn = std::function<void()>` and a `combine(...)` helper that chains two optional hooks.
`TestFunction.h`	`test_fn = std::function<void()>` — the signature every `it(...)` body is wrapped in.
`TestStatus.h`	`enum TestStatus { Unknown, Passed, Failed, Skipped }`.
`TestException.h`	Base exception (`std::runtime_error`) carrying label/path/function/file/line; the foundation of test failures.
`TestFailedException.h`	`TestException` subclass = a reported test failure (`reason()`).
`TestPendingException.h`	`TestException` subclass = a skipped/pending test.
`FailedExpectation.h`	`TestFailedException` subclass thrown by a matcher when an assertion fails.

3.2 Sources (`src/JTest/`)

.cpp implementations for: ConfigureFunction, DescribeOptions, FailedExpectation, JTest (the runtime: describe/it/execute), Test, TestBundle, TestException, TestFailedException, TestOptions, TestPendingException, TestResults, TestStatus.

3.3 Examples (`examples/`)

example.cpp — the example test driver. Contains several describe blocks and a main() that runs them, collects a TestResults, and prints it. This is the file you read to learn the API by example.
ClassToTest.h / ClassToTest.cpp — currently empty stubs (a MyNS namespace placeholder). Intended to be the first "class under test".

4. How the API works (concepts)

4.1 Describing and nesting

A test file builds a tree of describe blocks. Each describe takes a label and a factory function (make_testbundle_fn) that returns the inner TestBundle. Nesting is done by returning TestBundles (or other describe results) from those factories.

describe("ClassToTest", [](){
    return TestBundle(
        {
            it("should do the thing", [](){
                // body
            }),
            it("should do the other thing", [](){
                pending("we haven't made the other thing yet");
            }),
        },
        DescribeOptions()
            .beforeEach([](){ /* setup */ })
            .afterEach([](){ /* teardown */ })
            .beforeAll([](){ /* once before all */ })
            .afterAll([](){ /* once after all */ }));
});

xdescribe(...) / xit(...) are the "disabled" variants.

4.2 Expect and matchers

expect(actual) returns an Expectable<T>. You chain a matcher onto it.

expect(MyAddFunction(2, 2)).toEqual(4);      // true → pass; false → throw
expect(x).toBeTrue();                        // boolean-ish
expect(ptr).toBeNull();                      // pointer
expect(fn).toThrow(std::runtime_error("...")); // callable throws

Matchers implemented today:

toBe(matcher) — matcher is std::optional<std::string>(const T&); returns a value on failure → throws FailedExpectation.
toBeFalse(), toBeTrue(), toBeNull(), toEqual(value).
toThrow() / toThrow(exception) / toThrow(string) / toThrow(matcher) — several overloads for "assert this callable throws".

4.3 Negation (the `not` problem)

Expectable::nevermore() flips the internal is_inverted_ flag so the next matcher asserts the negative — JTest's stand-in for Jasmine's .not().

expect(2 + 2).nevermore().toEqual(5);   // passes: they are not equal

4.4 Running tests

The top-level entry point is execute(...). There are two overloads:

execute(TestBundle bundle, path="") — runs a group; recursively handles beforeAll → children → tests (each wrapped in beforeEach/afterEach) → afterAll.
execute(Test test, bundle_label) — runs one test; increments the right counter and prints a line.

A typical main() (see examples/example.cpp) accumulates results and prints them:

TestResults results;
results += run_group_a(args);
// ...
std::cout << results << std::endl;

4.5 Failure model (exceptions carry the signal)

JTest uses exceptions to flow test outcomes, then catches them at the execute(Test...) level to turn them into counters + output:

A matcher fails → throws FailedExpectation (a TestFailedException).
fail(reason) → throws TestFailedException.
pending(reason) → throws TestPendingException.
Any uncaught std::exception / ... → counted as a failure ("Unhandled exception running test").

Result line formats (seen in real output — see §7):

🚧 Pending Test: <path>::<label>
❌ Failed Test: <path>::<label> reason: <reason>
🔥 Unhandled exception running test: <path>::<label> what: <what>
Final summary: Total: N, Passed: N, Failed: N, Skipped: N

4.6 Lifecycle hooks and inheritance

beforeAll/afterAll/beforeEach/afterEach are stored on a TestBundle as maybe_configure_fn (an optional<configure_fn>). combine(first, second) chains two hooks so parent hooks run relative to child hooks. describe merges DescribeOptions hooks into the returned bundle via combine.

5. Building and running

Toolchain: clang++, C++17, macOS / clang++ is assumed. The Makefile targets clang++ with -std=c++17 and -arch arm64.

make            # build framework objects + examples/example  (all: target)
make run        # build then execute the example test binary
make clean      # wipe + recreate the build/ directories

make builds build/examples/example by linking the compiled framework objects (build/JTest/*.o) with the example object.
make run is make's run target: build the example and run it.
No external test runner is wired up yet — the "runner" is main() in examples/example.cpp, which hand-calls execute(...) for each group. This is intentional for now; an automated/self-testing harness is on the roadmap.

5.1 Real run output (verified)

Running make run against examples/example.cpp (as it stands) prints:

🚧 Pending Test: ClassToTest::<anonymous>::should do the other thing
❌ Failed Test: ClassToTest::<anonymous>::should not do the bad thing reason: it did the bad thing
🔥 Unhandled exception running test: ClassToTest::<anonymous>::should throw an exception if we do the other bad thing what: Bad thing happen.
Total: 4, Passed: 1, Failed: 2, Skipped: 1

This is the current expected output: one passing, two failing, one pending.

6. Design notes & conventions to follow

These are the "gotchas" that save the next person hours:

Headers include their own forward-declarations and then re-open the namespace after including dependencies. This pattern (declare class Foo; → include → reopen namespace) appears throughout include/JTest/. Match it.
it/xit return TestBundle (not Test) today. This is a known rough edge; the plan is to return Test and change describe to accept a variant. Do not "fix" this unilaterally.
The matcher toBeTruthy/Equal/Null/False and toThrow families use std::enable_if/std::is_convertible/std::is_invocable to compile only when the relevant operation makes sense for T. When adding a matcher, mirror this SFINAE style.
Negation is nevermore(), not not() — not is not usable as C++ method syntax. Keep that decision until it's revisited.
Outcome = exception. Don't invent a second "return a status" path; route failures/pending through the existing exception types and the execute catch-alls.
Fully-qualified names use :: between path and label (ClassToTest::<anonymous>::should do the other thing). Anonymous bundles print as <anonymous>.
TestOptions is a placeholder — per-test options aren't wired yet.
Expectable::make_failure_message only uses operator<< for T — a known limitation; failure text assumes a streamable type.

7. How to add a test (recipe for test authors)

Put your tests in examples/ (or wherever the driver runs them).
Each "suite file" exposes one or more TestResults <name>(const vector<string>&) functions, each calling execute(describe(...), args) and returning the TestResults.
Register the suite in main() by adding results += <name>(args); and uncommenting it (see examples/example.cpp lines around main).
Build & run: make run. Read the printed line per test, then the totals line.

8. Current State & Roadmap

This project is early-stage. The following come directly from README.md (the authoritative TODO list) and the TODO comments in the code. Treat these as the open work; do not assume they're done.

Framework behaviour

Make Expectable::toThrow work with arguments, member functions, static member functions, and possibly constructors/destructors.
Report all errors at the end instead of printing inline (prerequisite for parallel execution).
Make JTest self-testing (possibly namespace magic to run a special instance).
Mocks and Spies — design, then add.

API cleanups / design decisions

Expectable<T>:
- Clean up failure messages.
- Only use ostream in failure messages when operator<< is defined for T.
- Pick the final name for negation (nevermore() is the placeholder).
describe/xdescribe / it/xit:
- Make it/xit return a Test instead of a TestBundle.
- Make describe/xdescribe accept a variant of Test or TestBundle.
- Make TestOptions actually useful.
- Set the label/path at the it/describe call site, not at execute time.
- A disabled bundle should report all child tests as skipped/disabled.
- Execute tests in parallel.
- Verify before*/after* inheritance: parent before* before child, parent after* after child.
- When reporting an unhandled exception in a test, include the fully qualified test name and label.

Reporting

DescribeOptions: add operator<< and a print(ostream&).
TestResults: add (currently commented-out) skipped/failed/errored vectors to hold detailed per-test info; consider making the counters private.

Error reporting plumbing

FailedExpectation: figure out how to capture the label/path when creating an expectation from an expect<T>(T) call (thread it out of the executing execute method), then add it when the FailedExpectation is caught.

9. Quick glossary (for new team members)

TestBundle — a node in the test tree (a group). Holds tests, child groups, and the four lifecycle hooks.
Test — a leaf: label + test_fn + disabled flag.
test_fn — std::function<void()>; the body of an it.
configure_fn / maybe_configure_fn — std::function<void()> / its optional; the body of a lifecycle hook.
Expectable — the fluent matcher object returned by expect(x).
TestResults — the counters (total/passed/failed/skipped) accumulated as tests run and printed at the end.
combine(a, b) — chains two optional hooks into one.

10. How to keep this document fresh

This file is living documentation. When you change the public API in include/JTest/, update §2 (vocabulary), §3 (layout), or §6 (conventions). When you close a README TODO, remove it from §8. When you add a new header or a new matcher, add it to §3.1 / §4.2.

16 KiB Raw Blame History