P1 NGC HPC
The P1 NGC HPC is hosted at the National Genome Centre, for GDPR-compliant work on sensitive data and large datasets. VMware vSphere isolates projects from one another, and storage is petabyte-scale. NGC's supercomputer page describes the infrastructure.
Getting Access
Who's eligible?
Requirements: PhD or higher (exceptions may apply), valid Danish university email, and registered P1 affiliation.
Each project must bring a signed Data Processing Agreement as its record. It must name NGC as a data processor and allow the data to be stored there. A project that poses a high risk to the people whose personal data it processes also needs a Data Protection Impact Assessment (DPIA).
Do I need to talk to someone first?
No. The optional workload survey tells the P1 compute coordinator about your workload. Fill it in early for NGC, since GDPR projects need their data-processing paperwork lined up.
How do I sign up?
- Request P1 affiliation in your P1 profile (Update Profile in the header). P1 staff approve it.
- Get the user application form from NGC. Only NGC issues it; their how to apply for access page explains how.
- Complete and sign it, then email it to the compute coordinator.
- Fill in the access request below. The coordinator then knows the paperwork is coming and raises the request with NGC.
You join the NGC Slack channel once you have access.
What if I don't qualify?
PhD P1 affiliates and above can skip this section. If you are a P1 member currently enrolled in an MSc, a research assistant (RA) role, or similar, you are not eligible for access without a written approval of exception from a responsible person (a P1 co-lead or faculty member) who can vouch for your request. Fill in the form and have the responsible person send a confirmation to compute-governance-p1@aicentre.dk so they can audit the request. The exception applies across all P1 clusters.
Connecting
The P1 NGC HPC is air-gapped. Access needs multi-factor authentication and the Omnissa Remote
Desktop client, and you get instructions once registered. From the remote desktop, ssh -X <your-username>@login reaches the login node.
Useful hosts:
https://console.cld076.vmc/status # Internal status page
cld076-0004.cld076.vmc # Internal SFTP
sftp.spc.ngc.dk # External Ingress/Egress (SFTP)Using the Cluster
The cluster runs PBS, and iqsub opens a quick interactive session. Jobs name your project's
group and account, which you get when access is approved. Scheduler flags are in NGC's internal
documentation, linked from the status page.
The cluster has no internet access, so git clone from a public forge, pip install and conda do not work. Code, data, packages and results pass through the SFTP staging area at /data/upload under admin supervision, and anything leaving needs your NGC admin's approval. Ask your NGC admin
for access to /data/upload as your project's data gateway.
The gallery shows how to move code, data and dependencies through the staging area. See every example that runs on P1 NGC HPC.
Policy & Support
What are the limits?
Compute
- Max wall time
- Not published; ask your NGC admin or the compute coordinator
- GPUs per job
- Not published; ask your NGC admin or the compute coordinator
- GPUs at once, per person
- Not published; ask your NGC admin or the compute coordinator
Storage
- Home directory
- Set per project at onboarding
- Project storage
- Set per project at onboarding
- Shared pool
- About 2 PB (245 TB SSD, 368 TB HDD, 1.44 PB cold)
- Only use this GDPR-compliant cluster for projects that need it
- Each project needs a signed Data Processing Agreement naming NGC as data processor
- High-risk projects also need a DPIA
How much storage do I get?
Your quota is set when your project is onboarded. The staging area at /data/upload is a gateway
only. Clear your files from it once they have landed, so the next transfer does not compete with
them.
Your Data Processing Agreement governs retention, and personal data may not stay once the project ends. Agree deletion or archiving with your NGC admin and the compute coordinator well before then.
What if I need more GPUs?
Default per-user limits keep the cluster fair when demand is high. Use this form when a run needs more than the default and needs to be protected: a multi-day training, a deadline, or a workload that cannot checkpoint. Tell us the size and the window, and the compute coordinator arranges it with the cluster operator.
If your workload checkpoints cleanly and you only want GPUs that are sitting idle, check whether your cluster lets you submit past the default limit directly. That path is faster and needs no request, at the cost of the job being cancellable when demand picks up.
Who do I contact?
- Technical Support
- Broken or missing packages, scheduler issues
- Policy Support
- Priority access, queueing, access exceptions
- General Questions
- Open chat with the P1 compute community
- Compute Coordinator
- Onboarding, workload guidance, general questions. Always happy to help
- Helped with your work? Acknowledge ORCID 0000-0002-2740-1651
Hardware Specification
- Nodes
- 8 compute (Lenovo VX665) + 3 storage (VX650)
- CPU per compute node
- 2× AMD EPYC 9354 (32 cores each)
- RAM per compute node
- 768 GB
- GPUs
- 14× NVIDIA H100 PCIe (80 GB)
- Storage
- About 2 PB total (245 TB SSD, 368 TB HDD, 1.44 PB cold)
- Topology
- Air-gapped, per-project isolation through VMware vSphere
- Compliance
- GDPR-compliant infrastructure
- Operating system
- Rocky Linux
- Scheduler
- TORQUE (PBS)