y-cruncher - A Multi-Threaded Pi-Program

From a high-school project that went a little too far...

By Alexander J. Yee

(Last updated: July 9, 2024)

 

Shortcuts:

 

The first scalable multi-threaded Pi-benchmark for multi-core systems...

 

How fast can your computer compute Pi?

 

y-cruncher is a program that can compute Pi and other constants to trillions of digits.

It is the first of its kind that is multi-threaded and scalable to multi-core systems. Ever since its launch in 2009, it has become a common benchmarking and stress-testing application for overclockers and hardware enthusiasts.

 

y-cruncher has been used to set several world records for the most digits of Pi ever computed.

 

Current Release:

Windows: Version 0.8.5 Build 9541 (Released: July 9, 2024)

Linux      : Version 0.8.5 Build 9541 (Released: July 9, 2024)

 

Official Mersenneforum Subforum.

Official HWBOT forum thread.

 

News:

 

Version v0.8.5 Released: (July 1, 2024) - permalink

 

y-cruncher v0.8.5 has been released without Zen5 optimizations for now. Once I get final Zen5 hardware, I intend to follow up with an update to add it.

 

Originally, I intended to wait for the Zen5 binary to be ready before releasing v0.8.5. But it is now July and multiple media outlets are asking me about it since all signs are pointing at an end-of-July launch for Zen5.

 

y-cruncher v0.8.5 is a rather big update from v0.8.4 with new optimizations and a new benchmark category (the BBP). Thus hardware reviewers will need to rerun all benchmarks on older hardware. And this takes time.

 

So to allow reviewers to get started on this early, I decided to branch-cut v0.8.5 early and push out a release without the Zen5 binary. When the final hardware arrives and the final optimizations/tuning is done, there will be an update to this release just for that. All existing binaries will not be affected and will be speed identical to today's release.

 

Related: Version 0.8.5 Preview

 

 

 

Note that the CPU Utilization of 3.33% is wrong. There is a yet-to-be-resolved issue that is causing long runs on many cores to bug out this measurement.

 

Pi Record Smashed at 202 Trillion Digits: (June 28, 2024) - permalink

 

StorageReview wasn't kidding when they said they were running a very large computation at the end of this video.

 

The previous record of 105 trillion digits set earlier this year was frought with problems. And with those lessions learned, this time it was done right.

 

Compared to last time, these changes were made:

  1. Switched to an Intel machine which lacks the superalignment hazard that plagues AMD processors. While the Intel machine was less powerful than the AMD one from last time, compute power is not the bottleneck. Thus getting rid of the hazard is more important.

  2. Better storage configuration with most issues from last time resolved. So no more silly bottlenecks holding everything back.

  3. Picked up fixes for 2 of the y-cruncher bugs discovered during the 105 trillion digit run. (the N63 bug + error handling crash)

See StorageReview's blog for more details!

 

 

Computation Statistics:

Decimal Digits: 202,112,290,000,000
Total Time:

104 days

(February 6, 2024 - May 20, 2024)

CPU: 2 x Intel Xeon Platinum 8592+ (128 cores, SMT off)
Memory: 1.0 TB DDR5
Storage:

Swap: 22 x Solidigm P5336

Digit Output: 6 x Solidigm P5336

OS: Windows 10 Build 20348
Software: y-cruncher v0.8.3.9532 (with developer fixes)
Validation File: Validation File
Verification: Screenshot

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

This computation was interrupted just once. It was started before the previous 105 trillion digit run even finished. And thus it began using the same (bugged) version of y-cruncher which broke that computation. Thus the interruption was necessary to switch to a patched binary with the bare minimum fixes to finish it off.

 

Barring that program swap, this easily could have been a single contiguous run.

 

Overall, this was a really fast computation relative to its size. And there were no errors or other complications. So y-cruncher's fault tolerance wasn't even tested this time.

 

Given that this computation went so well and there's nothing else to talk about, we can ask the question, how far can the Pi record be pushed on today's hardware?

 

 

What's the Limit?

 

When trying to estimate what is possible, we need to look at the typical bottlenecks that prevent going higher. And given the current setup of running on a single system, the limiters are:

Historically speaking, cost has not been a major issue. While the hardware involved is very expensive, it is not "consumed". So the cost comes from power consumption, opportunity cost of the hardware, and maintainence. Recent records have also been trending towards advertisement compaigns for products and services where there is more willingness to spend resources.

 

 

Time Limit:

 

While there is technically no time limit to a computation, there are some implicit ones imposed by external forces.

 

The main time limit is human patience. 100 - 200 days seems to be the limit of what people are willing to plan for. While some of the Pi records have run longer than that, they typically were the result of complications that led to overruns. To my knowledge, nobody has willingly attempted a computation knowing that it would take longer than a year.

 

Let's say you were willing to wait for more than a year. How far is the limit now? At some point, the exponential growth of hardware and software* improvements will catch up to you. This means that you can set a record faster by waiting for better hardware/software to arrive, then running it later. As of today, I estimate this limit to be around 3 years. Meaning that if you wanted to run a computation that would take 3 years, you're probably better off waiting 2 years for hardware/software to arrive that can do it in less than 1 year.

 

*I hesitate to include software as part of "exponential growth". But in reality, exponential improvement in hardware is only possible if software is updated to utilize it. (think parallelism, vectorization, etc...) Though in another (comedic) view, software in general does in fact "grow exponentially" - in the wrong direction (i.e. modern bloatware).

 

 

Storage in one System:

 

This is perhaps the most obvious of the limits. You can't run anything larger than you can store. As of today, the densest form of storage is actually enterprise SSD. (Enterprise SSD is denser than even hard drives.) Thus with enterprise SSDs, you can cram roughly 1 - 3 petabytes of it in one system. This puts a limit of around 500 trillion to 1 quadrillion (1015) digits.

 

It is possible to further expand storage by moving off system, but then you start testing the bandwidth capability the network or whatever interconnect is used. This opens up a different can of worms around hierarchical storage - which is something that y-cruncher working towards, but is not there yet.

 

 

Limit of Reliability:

 

This is somewhat related to the time limit. The longer a computation runs, the probability of something failing catastrophicly increases exponentially.

 

y-cruncher has already gone to extreme lengths to extend this limit:

In all, these will allow large computations to be done quite unreliable hardware.

 

In the history of y-cruncher Pi records, there has been one soft error and multiple hard drive failures - all occuring during the early records by Shigeru Kondo. Everything since then has been rather clean - partly because of these experiences leading to better education and thus better preparation in later computations.

 

And because of this, it's hard to estimate what the limit is on in modern times since reliability hasn't really been tested. If we assume that most modern hardware tends to start falling apart after "a few years", this would probably be the upper-bound of long computations.

 

 

Putting it all together, I don't see y-cruncher being used to go beyond 500 trillion digits in the next year or so - the limiter being time of computation. Though I wouldn't bet against it if enterprise SSDs keep rapidly increasing in performance and density.

 

 

 

Older News

 

Records Set by y-cruncher:

y-cruncher has been used to set a number of world record sized computations.

 

Blue: Current World Record

Green: Former World Record

Red: Unverified computation. Does not qualify as a world record until verified using an alternate formula.

Date Announced Date Completed: Source: Who: Constant: Decimal Digits: Time: Computer:
June 28, 2024 May 20, 2024 Source

Jordan Ranous

Kevin O’Brien

Brian Beeler

(StorageReview)

Pi 202,112,290,000,000

Compute:  104 days

Verify:  4 days

Validation File

2 x Intel Xeon Platinum 8592+
1.0 TB
1.6 PB storage

May 12, 2024 May 11, 2024   Dmitriy Grigoryev Zeta(5) 260,000,000,000 Compute:  5.11 days
Verify:  6.93 days
Intel Xeon W7-3465X
1 TB
March 14, 2024 February 27, 2024 Source

Jordan Ranous

Kevin O’Brien

Brian Beeler

(StorageReview)

Pi 105,000,000,000,000

Compute:  75 days

Verify:  4 days

Validation File

2 x AMD Epyc 9754
1.5 TB
960 TB storage

February 13, 2024 February 12, 2024   Jordan Ranous Log(2) 3,000,000,000,000

Compute:  42.7 hours
Verify:  58.3 hours

2 x Intel Xeon Platinum 8460H
512 GB
January 17, 2024 December 12, 2023   Jordan Ranous Gamma(1/4) 1,000,000,000,000

Compute:  22.6 hours

Verify:  22.8 hours

2 x Intel Xeon Platinum 8450H
512 GB
December 26, 2023 December 24, 2023   Jordan Ranous e 35,000,000,000,000

Compute:  94.5 hours

Verify:  92.5 hours

2 x Intel Xeon Platinum 8460H
512 GB

December 26, 2023 December 25, 2023   Jordan Ranous Square Root of 2 20,000,000,000,000

Compute:  29.2 hours

Verify:  21.6 hours

Intel Xeon Platinum 8450H
512 GB
Intel Xeon Platinum 8460H
512 GB
December 26, 2023 December 22, 2023  

Andrew Sun

Zeta(3) - Apery's Constant 2,020,569,031,595 Compute:  5.61 days

Verify:  5.93 days

Intel Xeon Platinum 8347C
505 GB
Intel Xeon Platinum 8347C
507 GB
December 18, 2023 December 15, 2023   Jordan Ranous Gamma(1/3) 1,000,000,000,000

Compute:  17.5 hours

Verify:  23.3 hours

2 x Intel Xeon Platinum 8450H

512 GB

December 2, 2023 November 27, 2023   Jordan Ranous Golden Ratio 20,000,000,000,000

Compute:  76.1 hours

Verify:  30.0 hours

AMD Epyc 9654 - 1.5 TB

Intel Xeon Platinum 8450H

September 9, 2023 September 7, 2023  

Andrew Sun

Euler-Mascheroni Constant 1,337,000,000,000

Compute:  28.5 days

Verify:  41.3 days

Intel Xeon Platinum 83470C

400 GB

July 17, 2022 July 15, 2022   Seungmin Kim Lemniscate 1,200,000,000,100

Compute:  32.2 days

Verify:  46.5 days

2 x Intel Xeon Gold 6140
377 GB

June 8, 2022 March 21, 2022   Emma Haruka Iwao Pi 100,000,000,000,000

Compute:  158 days

Verify:  12.6 hours

Validation File

128 vCPU Intel Ice Lake (GCP)
864 GB
663 TB storage

March 14, 2022 March 9, 2022   Seungmin Kim Catalan's Constant 1,200,000,000,100 Compute:  48.6 days

Verify:  47.3 days

2 x Intel Xeon Gold 6140
2 x Intel Xeon E5-2680 v3

August 17, 2021 August 14, 2021 Source UAS Grisons Pi 62,831,853,071,796 Compute:  108 days
Verify:  34.4 hours
AMD Epyc 7542
1 TB
34 + 4 Hard Drives
September 13, 2020 September 6, 2020   Seungmin Kim Log(10) 1,200,000,000,100

Compute:  14.5 days

Verify:  22.5 days

2 x Intel Xeon E5-2699 v3
756 GB
2 x Intel Xeon Gold 5220
754 GB
January 29, 2020 January 29, 2020 Blog Timothy Mullican Pi 50,000,000,000,000

Compute:  303 days

Verify:  17.2 hours

Validation File

4 x Intel Xeon E7-4880 v2

315 GB

48 Hard Drives

March 14, 2019 January 21, 2019

Blogs

1 + 2

Emma Haruka Iwao Pi 31,415,926,535,897 Compute:  121 days

Verify:  20.0 hours

Validation File

2 x Undisclosed Intel Xeon
> 1.40 TB DDR4
> 240 TB SSD
November 15, 2016 November 11, 2016 Blog
Sponsor
Peter Trueb Pi 22,459,157,718,361 Compute:  105 days

Verify:  28 hours

Validation File

4 x Xeon E7-8890 v3
1.25 TB DDR4
20 x 6 TB 7200 RPM Seagate
October 8, 2014 October 7, 2014  

Sandon Van Ness

(houkouonchi)

Pi 13,300,000,000,000

Compute:  208 days

Verify:  182 hours

Validation File

2 x Xeon E5-4650L
192 GB DDR3 @ 1333 MHz
24 x 4 TB + 30 x 3 TB
December 28, 2013 December 28, 2013 Source Shigeru Kondo Pi 12,100,000,000,050

Compute: 94 days

Verify: 46 hours

2 x Xeon E5-2690
128 GB DDR3 @ 1600 MHz
24 x 3 TB

See the complete list including other notably large computations. If you want to set a record yourself, the rules are in that link.

 

 

Features:

 

The main computational features of y-cruncher are:

 

Download:

Sample Screenshot: 1 trillion digits of Pi

Core i7 5960X @ 4.0 GHz - 64 DDR4 @ 2400 MHz - 16 HDs

 

Latest Releases: (July 9, 2024)

Downloading any of these files constitutes as acceptance of the license agreement.

OS Download Link Size

Windows

y-cruncher v0.8.5.9541.zip

39.5 MB

Linux (Static)

y-cruncher v0.8.5.9541-static.tar.xz

30.6 MB

Linux (Dynamic)

y-cruncher v0.8.5.9541-dynamic.tar.xz

24.3 MB

 

 

 

 

 

 

 

 

Downloads can also be found on GitHub. Use this if you prefer HTTPS.

 

The Linux version comes in both statically and dynamically linked versions. The static version should work on most Linux distributions, but lacks TBB and NUMA binding. The dynamic version supports all features, but is less portable due to the DLL dependency hell.

 

HWBOT submission is back with this release. So I expect the leaderboards to be rewritten soon.

 

System Requirements:

Windows:

Linux:

All Systems:

Very old systems that don't meet these requirements may be able to run older versions of y-cruncher. Support goes all the way back to even before Windows XP.

 

Version History:

 

Other Downloads (for C++ programmers):

 

Advanced Documentation:

 

 

Benchmarks:

Comparison Chart: (Last updated: July 8, 2024)

 

Computations of Pi to various sizes. All times in seconds. All computations done entirely in ram.

The timings include the time needed to convert the digits to decimal representation, but not the time needed to write out the digits to disk.

 

Blue: Benchmarks are up-to-date with the latest version of y-cruncher.

Green: Benchmarks were done with an old version of y-cruncher that is comparable in performance with the current release.

Red: Benchmarks are significantly out-of-date due to being run with an old version of y-cruncher that is no longer comparable with the current release.

Purple: Benchmarks are from unreleased internal builds that are not speed comparable with the current release.

 

 

Laptops + Low-Power:

Processor(s): Core i7 11800H Ryzen 9 7940HS
Generation: Intel Tiger Lake Intel Zen4
Cores/Threads: 8/16 8/16
Processor Speed: ~2.5 - 2.8 GHz (60W PL) ~4.1 - 4.8 GHz (50W PL)
Memory: 64 GB @ 3200 MT/s 96 GB - 5600 MT/s
Program Version: v0.8.5 (18-CNL) v0.8.5 (22-ZN4)
Instruction Set: x64 AVX512-VBMI x64 AVX512-GFNI
25,000,000 0.504 0.410
50,000,000 1.122 0.910
100,000,000 2.576 2.041
250,000,000 6.891 5.662
500,000,000 15.166 12.486
1,000,000,000 33.950 27.654
2,500,000,000 96.181 79.921
5,000,000,000 211.777 199.153
10,000,000,000 476.878 501.327
Credit:    
Processor(s): Core i7 6820HK Core i7 11800H Core i7 11800H
Generation: Intel Skylake Intel Tiger Lake Intel Tiger Lake
Cores/Threads: 4/8 8/16 8/16
Processor Speed: 3.2 GHz (stock) ~2.5 GHz (45W PL) ~3.0 GHz (60W PL)
Memory: 64 GB @ 2133 MT/s 64 GB @ 3200 MT/s 64 GB @ 3200 MT/s
Version: v0.8.1 (14-BDW) v0.8.1 (18-CNL) v0.8.1 (18-CNL)
Instruction Set: x64 AVX2 + ADX x64 AVX512-VBMI x64 AVX512-VBMI
25,000,000 1.500 0.655 0.530
50,000,000 3.307 1.406 1.125
100,000,000 7.238 3.005 2.447
250,000,000 20.596 8.576 6.855
500,000,000 45.967 19.747 15.356
1,000,000,000 102.885 42.727 34.308
2,500,000,000 290.824 123.523 96.918
5,000,000,000 640.506 247.705 218.782
10,000,000,000 1,391.204 526.212 480.197
Credit:      
Processor(s): Core i3 8121U Core i7 11800H
Generation: Intel Cannon Lake Intel Tiger Lake
Cores/Threads: 2/4 8/16
Processor Speed: ~2.5 - 3.2 GHz (stock) ~2.5 - 2.8 GHz (45W PL)
Memory: 8 GB @ 2400 MT/s 64 GB @ 3200 MT/s
Version: v0.8.1 (14-BDW) v0.8.1 (17-SKX) v0.8.1 (18-CNL) v0.8.1 (14-BDW) v0.8.1 (17-SKX) v0.8.1 (18-CNL)
Instruction Set: x64 AVX2 + ADX x64 AVX512-DQ x64 AVX512-VBMI x64 AVX2 + ADX x64 AVX512-DQ x64 AVX512-VBMI
25,000,000 2.857 2.467 1.988 0.907 0.853 0.655
50,000,000 6.446 5.501 4.392 2.075 1.862 1.406
100,000,000 14.335 12.257 9.490 4.176 3.749 3.005
250,000,000 42.566 36.204 27.137 12.014 10.705 8.576
500,000,000 99.040 85.443 64.359 28.805 24.123 19.747
1,000,000,000 228.863 198.405 151.605 63.898 55.264 42.727
2,500,000,000       187.882 148.423 123.523
5,000,000,000       375.130 327.776 247.705
10,000,000,000       794.573 709.606 526.212
Credit:            

 

 

 

Mainstream Desktops:

Processor(s): Ryzen 7 1800X Ryzen 9 3950X Ryzen 9 7950X
Generation: AMD Zen 1 AMD Zen 2 AMD Zen 4
Cores/Threads: 8/16 16/32 16/32
Processor Speed: stock stock stock
Memory: 64 GB - 2866 MT/s 32 GB - 3200 MT/s 28 GB - 5200 MT/s
Program Version: v0.8.5 (17-ZN1) v0.8.5 (17-ZN2) v0.8.5 (22-ZN4)
Instruction Set: x64 AVX2 x64 AVX2 x64 AVX512-GFNI
25,000,000 1.092 0.527 0.287
50,000,000 2.393 1.112 0.615
100,000,000 5.337 2.387 1.365
250,000,000 15.340 6.302 3.813
500,000,000 34.074 13.519 7.985
1,000,000,000 76.415 29.470 16.841
2,500,000,000 218.467 84.116 45.703
5,000,000,000 495.367 187.272 100.214
10,000,000,000 1112.598 412.102 218.732
25,000,000,000     615.070
Credit:      
Processor(s): Ryzen 5 7600 Core i9 11700K Ryzen 9 3950X Ryzen 9 5950X Core i9 13900KS Ryzen 9 7950X
Generation: AMD Zen 4 Intel Rocket Lake AMD Zen 2 AMD Zen 3 Intel Raptor Lake AMD Zen 4
Cores/Threads: 6/12 8/16 16/32 16/32 24/32 16/32
Processor Speed:   stock stock stock 5.7/4.5 GHz stock
Memory: 32 GB 32 GB - 3200 MT/s 128 GB - 2666 MT/s 64 GB - 3200 MT/s 96 GB - 8000 MT/s 128 GB - 4400 MT/s 128 GB - 5200 MT/s
Program Version: v0.8.1 (22-ZN4) v0.8.1 (18-CNL) v0.8.1 (19-ZN2) v0.8.1 (19-ZN2) v0.8.1 (14-BDW) v0.8.1 (22-ZN4)
Instruction Set: x64 AVX512-GFNI x64 AVX512-VBMI x64 AVX2 + ADX x64 AVX2 + ADX x64 AVX2 + ADX x64 AVX512-GFNI
25,000,000 0.439 0.501 0.588 0.490 0.241 0.312 0.307
50,000,000   1.114 1.257 1.090 0.525 0.679 0.654
100,000,000   2.223 2.685 2.345 1.132 1.517 1.410
250,000,000   6.220 7.251 6.371 3.185 4.157 3.820
500,000,000 13.378 13.573 15.556 13.395 7.065 8.883 8.062
1,000,000,000 29.497 30.415 33.925 29.301 15.901 18.542 17.039
2,500,000,000 83.421 86.119 96.695 82.204 44.888 50.743 46.467
5,000,000,000 181.647 193.718 215.333 181.355 99.566 110.379 101.345
10,000,000,000     473.958 399.012   241.162 220.522
25,000,000,000     1,361.732     680.344 623.493
Credit: Joel Rufin Oliver Kruse

 

Oliver Kruse 曾 铮    
Processor(s): Core i7 920 FX-8350 Core i7 4770K Ryzen 7 1800X Ryzen 7 3800X
Generation: Intel Nehalem AMD Piledriver Intel Haswell AMD Zen 1 AMD Zen 2
Cores/Threads: 4/8 8/8 4/8 8/16 8/16
Processor Speed: 3.5 GHz stock 4.0 GHz stock stock
Memory: 12 GB - 1333 MT/s 32 GB - 1600 MT/s 32 GB - 2133 MT/s 64 GB - 2866 MT/s 32 GB - 3600 MT/s
Program Version: v0.8.1 (08-NHM) v0.8.1 (11-BD1) v0.8.1 (13-HSW) v0.8.1 (17-ZN1) v0.8.1 (19-ZN2)
Instruction Set: x64 SSE4.1 x64 FMA4 x64 AVX2 x64 AVX2 + ADX x64 AVX2 + ADX
25,000,000 7.032 3.677 1.546 1.150 0.654
50,000,000 17.174 7.703 3.259 2.527 1.415
100,000,000 36.164 16.576 6.987 5.555 3.028
250,000,000 105.789 46.597 19.588 15.760 8.404
500,000,000 236.096 103.165 43.197 34.659 18.440
1,000,000,000 531.676 230.780 96.845 78.690 41.097
2,500,000,000   669.594 274.336 220.278 117.788
5,000,000,000   1,460.714 606.605 493.388 266.719
10,000,000,000       1,078.187  
25,000,000,000          
Credit:         Oliver Kruse

 

 

 

High-End Desktops:

Processor(s): Core i7 5960X Core i9 7900X Core i9 10980XE
Generation: Intel Haswell Intel Skylake X Intel Cascake Lake
Cores/Threads: 8/16 10/20 18/36
Processor Speed: 4.0 GHz ~3.6 GHz (200W PL) 3.3 GHz (AVX512)
Memory: 64 GB - 2400 MT/s 128 GB - 3000 MT/s 128 GB - 3600 MT/s
Program Version: v0.8.5 (13-HSW) v0.8.5 (17-SKX) v0.8.5 (17-SKX)
Instruction Set: x64 AVX2 x64 AVX512-DQ x64 AVX512-DQ
25,000,000 0.727 0.409 0.286
50,000,000 1.626 0.885 0.567
100,000,000 3.524 1.916 1.245
250,000,000 10.089 5.488 3.541
500,000,000 22.546 12.419 7.976
1,000,000,000 50.538 27.822 17.822
2,500,000,000 146.103 78.850 50.888
5,000,000,000 314.891 174.063 113.511
10,000,000,000 681.296 380.010 245.876
25,000,000,000   1,064.718 676.923
Credit:      
Processor(s): Core i7 5960X Threadripper 1950X Core i9 7900X Core i9 7940X Threadripper 3990X Xeon W7-2495X Xeon W9-3475X
Generation: Intel Haswell AMD Zen 1 Intel Skylake X Intel Skylake X AMD Zen 2 Intel Sapphire Rapids Intel Sapphire Rapids
Cores/Threads: 8/16 16/32 10/20 14/28 64/128 24/48 36/72
Processor Speed: 4.0 GHz stock ~3.6 GHz (200W PL) 3.6 GHz (AVX512) 2.9 GHz 4.1-4.9 GHz 4.2-4.9 GHz
Memory: 64 GB - 2400 MT/s 64 GB - 2800 MT/s 128 GB - 3000 MT/s 128 GB - 3466 MT/s ~141 GB - 2666 MT/s 64 GB - 6400 MT/s 128 GB - 6400 MT/s
Program Version: v0.8.1 (13-HSW) v0.8.1 (17-ZN1) v0.8.1 (17-SKX) v0.8.1 (17-SKX) v0.8.1 (19-ZN2) v0.8.1 (18-CNL) v0.8.3 (18-CNL)
Instruction Set: x64 AVX2 x64 AVX2 + ADX x64 AVX512-DQ x64 AVX512-DQ x64 AVX2 + ADX x64 AVX512-VBMI x64 AVX512-VBMI
25,000,000 0.807 0.756 0.522 0.404 0.584 0.170 0.201
50,000,000 1.743 1.579 1.028 0.721 1.181 0.340 0.321
100,000,000 3.647 3.273 2.048 1.451 2.409 0.726 0.586
250,000,000 10.088 8.990 5.752 4.056 5.724 2.068 1.413
500,000,000 22.075 19.604 12.830 9.017 10.881 4.588 2.627
1,000,000,000 49.232 43.014 28.906 20.518 21.496 10.190 5.924
2,500,000,000 139.404 121.645 82.764 60.636 58.009 28.881 16.345
5,000,000,000 311.388 271.983 186.233 137.906 126.513 64.158 36.139
10,000,000,000 669.736 613.450 401.820 302.121 274.050 124.826 78.816
25,000,000,000     1,125.775 843.498 768.212   225.482
Credit:   Oliver Kruse     Paul Underwood 曾 铮

 

 

Multi-Processor Workstation/Servers:

 

Due to high core count and the effect of NUMA (Non-Uniform Memory Access), performance on multi-processor systems are extremely sensitive to various settings. Therefore, these benchmarks may not be entirely representative of what the hardware is capable of.

Processor(s):

Xeon Platinum 8375C

(AWS x2iedn.32xlarge)

Xeon Platinum 8488C

(AWS m7i.48xlarge)

Epyc 9R14

(AWS m7a.48xlarge)

Epyc 9R14

(AWS hpc7a.96xlarge)

Epyc 9754
Generation: Intel Sapphire Rapids Intel Sapphire Rapids AMD Genoa AMD Bergamo
Cores/Threads: 64/128 96/192 192/192 128/256 128/128
Processor Speed: 2.9 GHz 2.4 GHz 2.6 GHz 2.25 - 3.1 GHz
Memory: 4 TB 744 GB 740 GB 768 GB - 4800 MT/s
Program Version: v0.8.1 (18-CNL) v0.8.1 (18-CNL) v0.8.1 (22-ZN4) v0.8.1 (22-ZN4)
Instruction Set: x64 AVX512-VBMI x64 AVX512-VBMI x64 AVX512-GFNI x64 AVX512-GFNI
25,000,000 0.250 0.163 0.216 0.213 0.245 0.229
50,000,000 0.454 0.289 0.285 0.279 0.350 0.433
100,000,000 0.844 0.531 0.642 0.635 0.853 0.876
250,000,000 1.976 1.288 1.776 1.716 2.224 2.133
500,000,000 3.794 2.499 3.728 3.621 4.186 3.850
1,000,000,000 7.650 5.149 6.547 6.265 7.063 6.495
2,500,000,000 20.425 13.633 13.554 12.500 15.338 14.477
5,000,000,000 45.675 29.655 25.334 22.377 29.072 28.133
10,000,000,000 101.468 64.026 51.134 44.059 58.797 59.007
25,000,000,000 297.622 182.920 140.286 120.282 156.797 164.281
50,000,000,000 678.016 410.842 321.970 275.297 350.391 368.548
100,000,000,000 1,549.991 943.182 771.266 672.558 829.957 853.717
250,000,000,000 4,488.317          
500,000,000,000 9,685.971          
Credit: Greg Hogan Tim Wesley

 

Processor(s): Xeon Platinum 8124M Xeon Gold 6148 Xeon Platinum 8175M Xeon Platinum 8275CL Epyc 7742 Epyc 7B12 Epyc 7742
Generation: Intel Skylake Purley Intel Skylake Purley Intel Skylake Purley Intel Cascade Lake AMD Rome AMD Rome AMD Rome
Sockets/Cores/Threads: 2/36/72 2/40/40 2/48/96 2/48/96 2/128/256 2/112/224 2/128/256
Processor Speed: 3.0 GHz 2.4 GHz 2.5 GHz 3.0 GHz   2.25 GHz 2.25 GHz
Memory: 137 GB - ?? 188 GB - ?? ~756 GB - ?? 192 GB ~504 GB ~882 GB 2 TB
Program Version: v0.7.5 (17-SKX) v0.7.6 (17-SKX) v0.7.6 (17-SKX) v0.7.8 (17-SKX) v0.7.7 (17-ZN1) v0.7.8 (19-ZN2) v0.7.8 (19-ZN2)
Instruction Set: x64 AVX512-DQ x64 AVX512-DQ x64 AVX512-DQ x64 AVX512-DQ x64 AVX2 + ADX x64 AVX2 + ADX x64 AVX2 + ADX
25,000,000 0.540 0.329 0.294 0.283 0.534 0.439 0.513
50,000,000 0.981 0.683 0.617 0.544 1.027 0.838 0.920
100,000,000 1.905 1.456 1.305 1.169 2.298 1.796 1.887
250,000,000 5.085 3.737 3.591 3.125 5.854 4.509 4.650
500,000,000 10.372 7.750 7.293 6.309 10.502 8.196 8.066
1,000,000,000 21.217 16.550 15.041 13.042 17.836 14.252 13.246
2,500,000,000 55.701 45.693 39.329 34.028 35.485 30.592 27.011
5,000,000,000 118.151 99.078 83.601 71.777 62.432 58.405 49.940
10,000,000,000 247.928 212.984 176.695 153.169 115.543 116.900 98.156
25,000,000,000   599.653 491.988 425.442 307.995 314.907 258.081
50,000,000,000     1,081.181   690.662 741.633 598.716
100,000,000,000           1715.123 1,370.714
250,000,000,000             3,872.397
Credit: Jacob Coleman Oliver Kruse newalex Xinyu Miao Carsten Spille Greg Hogan Song Pengei
Processor(s): Xeon E5-2683 v3 Xeon E7-8880 v3 Xeon E5-2687W v4 Xeon E5-2686 v4 Xeon E5-2696 v4 Epyc 7601 Xeon Gold 6130F
Generation: Intel Haswell Intel Haswell Intel Broadwell Intel Broadwell Intel Broadwell AMD Naples Intel Skylake Purley
Sockets/Cores/Threads: 2/28/56 4/64/128 2/24/48 2/36/72 2/44/88 2/64/128 2/32/64
Processor Speed: 2.03 GHz 2.3 GHz 3.0 GHz 2.3 GHz 2.2 GHz 2.2 GHz 2.1 GHz
Memory: 128 GB - ??? 2 TB - ??? 64 GB 504 GB - ??? 768 GB - ??? 256 GB - ?? 256 GB - ??
Program Version: v0.6.9 (13-HSW) v0.7.1 (13-HSW) v0.7.6 (14-BDW) v0.7.7 (14-BDW) v0.7.1 (14-BDW) v0.7.3 (17-ZN1) v0.7.3 (17-SKX)
Instruction Set: x64 AVX2 x64 AVX2 x64 AVX2 + ADX x64 AVX2 + ADX x64 AVX2 + ADX x64 AVX2 + ADX x64 AVX512-DQ
25,000,000 0.907 1.176 0.490 0.494 0.715 2.459 1.150
50,000,000 1.745 2.321 1.072 0.982 1.344 4.347 1.883
100,000,000 3.317 4.217 2.303 2.193 2.673 6.996 3.341
250,000,000 8.339 8.781 6.196 6.044 6.853 14.258 7.731
500,000,000 17.708 15.879 13.046 12.582 14.538 24.930 15.346
1,000,000,000 37.311 32.078 27.763 26.852 31.260 47.837 31.301
2,500,000,000 102.131 78.251 76.202 73.596 84.271 111.139 82.871
5,000,000,000 218.917 164.157 165.046 160.094 192.889 228.252 179.488
10,000,000,000 471.802 346.307 356.487 346.305 417.322 482.777 387.530
25,000,000,000 1,511.852 957.966 1,006.131 980.784 1,186.881 1,184.144 1,063.850
50,000,000,000   2,096.169 2,202.558 2,156.854 2,601.476    
100,000,000,000   4,442.742     6,037.704    
250,000,000,000   17,428.450          
Credit: Shigeru Kondo Jacob Coleman Cameron Giesbrecht newalex "yoyo" Dave Graham

 

 

Fastest Times:

The full chart of rankings for each size can be found here:

These fastest times may include unreleased betas.


Got a faster time? Let me know: a-yee@u.northwestern.edu

Note that I usually do not respond to these emails. I simply put them into the charts which I update periodically (typically within 2 weeks).

 

 

Performance Tips:

 

Decimal Digits of Pi - Times in Seconds

Core i9 7940X @ 3.7 GHz AVX512

Memory Frequency: 2666 MT/s 3466 MT/s
25,000,000 0.839 0.758
50,000,000 1.424 1.338
100,000,000 2.701 2.425
250,000,000 6.489 5.877
500,000,000 13.307 11.917
1,000,000,000 27.913 24.915
2,500,000,000 76.837 68.322
5,000,000,000 168.058 148.737
10,000,000,000 365.047 322.115
25,000,000,000 1,037.527 916.039

High core count Skylake X processors are known to be heavily bottlenecked by memory bandwidth.

Memory Bandwidth:

 

Because of the memory-intensive nature of computing Pi and other constants, y-cruncher needs a lot of memory bandwidth to perform well. In fact, the program has been noticably memory bound on nearly all high-end desktops since 2012 as well as the majority of multi-socket systems since at least 2006.

 

Recommendations:

Don't be surprised if y-cruncher exposes instabilities that other applications and stress-tests do not. y-cruncher is unusual in that it simultaneously places a heavy load on both the CPU and the entire memory subsystem.

 

 

 

Parallel Performance:

 

y-cruncher has a lot of settings for tuning parallel performance. By default, it makes a best effort to analyze the hardware and pick the best settings. But because of the virtually unlimited combinations of processor topologies, it's difficult for y-cruncher to optimally pick the best settings for everything. So sometimes the best performance can only be achieved with manual settings.

*These are advanced settings that cannot be changed if you're using the benchmark option in the console UI. To change them, you will need to either run benchmark mode from the command line or use the custom compute menu.

 

Load imbalance is a faily common problem in y-cruncher. The usual causes are:

  1. The number of logical cores is not a power-of-two.
  2. The cores are not homogenous. Common reasons include:
    • The cores are clocked at different speeds.
    • The cores have access to different amounts of memory bandwidth due an imbalanced NUMA topology.
    • The cores are different generation cores hidden behind a virtual machine.
  3. CPU-intensive background processes are interfering with y-cruncher's ability to use all the hardware. This applies to all forms of system jitter.

 

 

Large Pages:

 

Large pages used to not matter in the past, but they do now in the post-Spectre/Meltdown world. Mitigations for the Meltdown vulnerability can have a noticeable performance drop for y-cruncher (up to 5% has been observed). It turns out that turning on large pages can mitigate the penalty for this mitigation. (pun intended)

 

Refer to the memory allocation guide on how to turn on large pages.

 

 

Swap Mode:

 

This is probably one of the most complicated features in y-cruncher.

 

 

Known Issues:

 

Everything in this section is in the process of being re-verified and moved to: https://github.com/Mysticial/y-cruncher/issues

 

 

Performance Issues:


Algorithms and Developments:

 

FAQ:

 

Pi and other Constants:

 

Program Usage:

 

Hardware and Overclocking:

 

Academia:

 

Programming:

 

Other:

 

Links:

Here's some interesting sites dedicated to the computation of Pi and other constants:

 

Questions or Comments

Contact me via e-mail. I'm pretty good with responding unless it gets caught in my school's junk mail filter.

You can also find me on Twitter as @Mysticial.