Configuration of the Successor Supercomputer (2025)

The basic configuration of the new supercomputer will follow that of the current one, with significant enhancements: the network connection to SINET6 will be upgraded to 100 Gbps, and the computational performance per CPU core will approximately double.

For GPU-equipped nodes, the following cutting-edge hardware will be introduced: NVIDIA DGX B200 for advanced AI analysis, NVIDIA L40S and PEZY-SC3 to provide cost-effective solutions for genome analysis.

For more details about the Successor Supercomputer (2025)’s configuration, see below.

Overview

Classification			Specifications
Compute Nodes	HPC CPU optimized nodes Total number of CPU cores: 13,184 CPU Total computing performance: 241.644 TFLOPS Total memory capacity: 91.136 TB	Type 1 AMD EPYC 9654 CPU	50 nodes 9,600 CPU cores 184.3 TFLOPS 76.8 TB total memory（8GB memory/CPU core）
		Type 2 AMD EPYC 7702 CPU（Former Thin compute node Type 1b）	28 nodes 3,584 CPU cores 57.344 TFLOPS 14.336 TB total memory（4GB memory/CPU core）
	Memory optimisation node Total number of CPU cores: 960 Total computing performance: 18.0 TFLOPS Total memory capacity: 21 TB	Type 1 6TB memory installed	2 nodes 384 CPU cores 7.2 TFLOPS 12 TB total memory（32GB memory/CPU core）
		Type 2 3TB memory installed	3 nodes 576 CPU cores 10.8 TFLOPS 9 TB total memory（16GB memory/CPU core）
	Accelerator-optimised node	Type 1 NVIDIA B200 on GPU For AI computing	4 nodes
		Type 2 GPU nodes for genome analysis	Being procured
		Type 3 Equipped with PEZY ZettaScaler3 as accelerator For genome analysis	2 nodes
Storage	Analysis storage For user home directories in the general analysis division and personal genome analysis division.		Lustre file system 13.3PB
	Database storage For DDBJ database including DRA		Lustre file system 40.5PB
	Tape equipment For backing up DDBJ databases		15PB
Inter-node interconnect network			InfiniBand HDR100 100Gbps fat tree For storage, full bi-division For compute nodes, 1/2 of full bi-section, but 400 Gbps full bi-section between accelerator-optimised node Type 1

HPC CPU optimized nodes

HPC CPU optimized nodes Type 1 (50 computers)

Compute nodes with AMD EPYC 9654 processors.

GIGABYTE R283-Z94

component	model number	number of computation	performance per node, etc
CPU	AMD EPYC 9654 (96 cores) Base 2.4GHz, Max 3.7GHz, 1.8TFlops	2	Total 192 cores, 3.6TFlops
Memory	64GB DDR5-4800	24	Total 1536GB (8GB per CPU core)
Storage (OS)	1TB NVMe SSD	1
Storage (Data)	30TB NVMe SSD	1
Network	InfiniBand HDR100	1	100Gbps

HPC CPU optimized node Type 2 (28 computers)

Compute nodes with AMD EPYC 7702 processors(This is the same as the former Thin compute node Type 1b)

DELL PowerEdge R6525

component	model number	number of computation	performance per node, etc
CPU	AMD EPYC 7702 (64 cores) Base 2.0GHz, Max 3.35GHz, 1.024TFlops	2	Total 128 ccores, 2.048TFlops
Memory	32GB DDR4-2666	16	Totak 512GB (4GB per CPU core)
Storage (OS)	1.6TB NVMe SSD	1
Storage (Data)	900GB SAS HDD	1
Network	InfiniBand 4xEDR	1	100Gbps

Memory optimisation node

Memory optimisation node (2 computers)

This is a compute node with 6 TB of memory per computer.

GIGABYTE R283-ZK0-AAL1

component	model number	number of computation	performance per node, etc
CPU	AMD EPYC 9654 (96 cores) Base 2.4GHz, Max 3.7GHz, 1.8TFlops	2	Total 192 cores, 3.6TFlops
Memory	128GB DDR5-4800	48	Total 6.144 TB (32GB per CPU core)
Storage	30TB SATA HDD	1	30TB
Network	InfiniBand HDR100	1	100Gbps

Memory optimisation node Type 2 (3 computers)

It is a compute node with 3TB of memory in one computer.

GIGABYTE R203-Z94-AAD2

component	model number	number of computation	performance per node, etc
CPU	AMD EPYC 9654 (96 cores) Base 2.4GHz, Max 3.7GHz, 1.8TFlops	2	Total 192 cores, 3.6TFlopw
Memory	128GB DDR5-4800	24	Total 3.072TB (16GB per CPU core)
Storage	30TB SATA HDD	1	30TB
Network	InfiniBand HDR100	1	100Gbps

Accelerator-optimised node

Accelerator-optimised node Type 1 (4 computers)

It is a compute node equipped with eight NVIDIA B200 GPUs per node. It is a GPU-equipped compute node suitable for AI computations.

NVIDIA DGX B200

component	model number	number of computation	performance per node, etc
CPU	Intel Xeon Platinum 8570 (56 cores) Base 2.1GHz, Max 4.0GHz, 1.97TFlops	2	Total 112 cores, 3.94TFlops
Memory	合計2TB		Total 2TB (17.9GB per CPU core)
GPU	NVIDIA Blackwell B200	8
Storage (OS)	1.9TB NVMe SSD	2	Total 3.8TB
Storage (Data)	3.84TB NVMe SSD	8	Total 30.7TB
Network	InfiniBand NDR	1	400Gbps

Accelerator-optimised node Type 2 (Being procured)

It is a GPU-equipped compute node suitable for genome analysis computations.

component	performance per node, etc
CPU	Total 64 cores
Memory	Total 768GB (12GB per CPU core)
Memory	Four NVIDIA L40S.
Storage (OS)	Total 489GB
Storage (Data)	Total 15TB

Accelerator-optimised node Type 3 (2台)

This node is equipped with the PEZY-SC3 accelerator. It is a compute node suitable for genome analysis computations.

PEZY ZettaScaler3.0

component	model number	number of computation	performance per node, etc
CPU	AMD EPYC 7713P (64 cores) Base 2.0GHz, Max 3.67GHz, 1.02TFlops	1	Total 64 cores, 1.02TFlops
Memory	128GB DDR4 3200MHz	16	Total 2TB (32GB per CPU core)
Accelerator	PEZY-SC3	4
Storage (OS)	0.256TB NVMe SSD	2	Total 0.512TB
Storage (Data)	4TB NVMe SSD	4	Total 16TB
Network	InfiniBand HDR100	1	100Gbps

(Appendix) GPU and Accelerator Performance Comparison

Accerelater	NVIDIA V100	NVIDIA A100	NVIDIA L40S	NVIDIA H100	NVIDIA B200	PEZY-SC3
Benchmark Run Time of GATK Compatible Software [hh:mm:ss]	3:07:29	2:06:14	1:45:05	1:56:24	N/A	1:02:55
Architecture	Volta	Ampere	Ada Lovelace	Hopper	Blackwell	PEZY
Memory size [GB]	16 or 32	40 or 80	40	80 or 94	N/D	32
Memory bandwidth [GB/s]	900	2039	864	3352	N/D	1200
FP64 [TFlops]	7.8	9.7	N/A	33.5	N/D	19.66
FP32 [TFlops]	15.7	19.5	91.6	66.9	N/D	39.32
FP16,BF16 [TFlops]	112	624	733	1979	N/D	78.64
FP8 [TFlops]	N/A	N/A	1466	3958	N/D	N/A
TF32 tensor core [TFlops]	125	312	366	989	N/D	N/A
FP16 tensor core [TFlops]	N/A	624	733	1979	N/D	N/A
FP8 tensor core [TFlops]	N/A	N/A	1466	3958	N/D	N/A
INT8 tensor core [TOPS]	N/A	1248	1466	3958	N/D	N/A

N/A : Not Available, N/D : Not Disclosed
Benchmark run times for GATK compatible software is the computing time when using the NA18945 sample from the 1KGP 30x data set. For more details on the benchmark, check the Benchmark(NVIDIA Parabriks) page on our website.

Storage

Analysis storage

Usage	Capacity	Peak Performance	Configuration
home directories in the general analysis division	10PB	read 60GB/sec, write 70GB/sec	DDN ES400NVX2 + DDN SS9024
home directories in the personal genome analysis division	8PB	read 40GB/sec, write 40GB/sec	DDN ES400NVX2 + DDN SS9024

Database storage

Usage	Capacity	Peak Performance	Configuration
DDBJ database storage	40PB	read 150GB/sec, write 150GB/sec	DDN ES400NVX2 + DDN SS9024
DDBJ database storage	10PB	read 60GB/sec, write 70GB/sec	DDN ES400NVX2 + DDN SS9024

Overview​

HPC CPU optimized nodes​

HPC CPU optimized nodes Type 1 (50 computers)​

HPC CPU optimized node Type 2 (28 computers)​

Memory optimisation node​

Memory optimisation node (2 computers)​

Memory optimisation node Type 2 (3 computers)​

Accelerator-optimised node​

Accelerator-optimised node Type 1 (4 computers)​

Accelerator-optimised node Type 2 (Being procured)​

Accelerator-optimised node Type 3 (2台)​

(Appendix) GPU and Accelerator Performance Comparison​

Storage​

Analysis storage​

Database storage​