Spinel

How often does Spinel give a wrong result?

Spinel can give different results than CRuby for the same Ruby program. Spinel targets parity with CRuby, so we call these wrong results.

The numbers on this page come from 402 small Ruby programs. Each one found a bug in Spinel at some point. I collect them from my reviews of Spinel's new commits, and most of them use rare Ruby features. Every day, a Ruby script runs each program with CRuby 4.0.7 and with Spinel built from the latest master, and compares the outputs. You can read every program in the list at the end of this page. If you don't know Spinel yet, start with my one-page intro to Spinel.

What I check

An example of a wrong result

If we run this program with CRuby, it prints true. If we compile it with Spinel and run it, it prints false.

p 2**64 == Rational(2**64, 1)

CRuby

true

Spinel

false

Each program falls into one of four groups

Same result as CRubyThe program prints the same output and stops the same way.
RefusedSpinel doesn't compile the program, and says why.
Wrong resultThe program finishes and prints a different output.
Error or crashThe program raises an error that CRuby doesn't raise, or it crashes, or Spinel's C code doesn't compile.

Every day

The results of each day

Each bar is one run of the set on that day's Spinel.

Programs by result, per day

Spinel's changes

Pull requests merged each day

Pull requests merged into Spinel, per day

The programs

Every program in the set

Open a program to see its code and today's result. For a wrong result, you also see the first output line that differs between CRuby and Spinel. The date is the day the program joined the set.