# P1 DTU HPC

> **Provider documentation.** This page is an overview to help you onboard and get started. The technical and advanced documentation is maintained by DTU DCC, who host and operate the system.
>
> DTU DCC documentation: https://www.hpc.dtu.dk/?page_id=5028

The P1 DTU HPC is P1's partition at DTU. It suits medium to large-scale machine learning research
by P1 members (PhD and above).

## Getting Access

### Who's eligible?

**Requirements:** PhD or higher (exceptions may apply), valid Danish university email, and
registered P1 affiliation.

### Do I need to talk to someone first?

No. The optional [workload survey](/survey/) tells the P1 compute coordinator about your
workload, so we can point you to the right resource.

### How do I sign up?

1. Request P1 affiliation in [your P1 profile](https://www.aicentre.dk/p1adb) (**Update Profile** in the header). P1
   staff approve it.
2. Fill in the DTU account form below. It only accepts official university email addresses.
   Henning Christiansen (head of DTU's compute centre) processes it, and you get the account
   details by email.

**Form:** [DTU Account Signup Form](https://forms.cloud.microsoft/e/DG5qCfs6Wm) Use your official university email address.

### What if I don't qualify?

**Form:** [Access exception request](https://forms.gle/jSp12oBU2BcM6GJR7) For MSc students, research assistants, and similar roles who need cluster access with written approval from a responsible P1 co-lead or faculty member.

## Connecting

The login node is `login9.hpc.dtu.dk`. Off DTU's network, connect to the DTU VPN first;
[DTU IT's VPN guide](https://www.hpc.dtu.dk/?page_id=2501) covers setup and MFA.

```bash
aicentre check -p dtu --connect
```

`aicentre check` shows the Host block for `~/.ssh/config` and whether your key is set up, then
tests the login.

## Using the Cluster

The cluster runs IBM Spectrum LSF with two queues:

- `p1`: batch jobs, seven nodes
- `p1i`: interactive sessions, one node, for installing packages, test runs and short debugging

`p1i` is the only P1 queue for interactive GPU work. Everything else runs as a batch job. Home
directories hold 30 GB, so project data goes under `/dtu/p1/`. See DTU's
[LSF guide](https://www.hpc.dtu.dk/?page_id=2698) for scheduler flags and its
[transfer nodes](https://www.hpc.dtu.dk/?page_id=2671) for very large data moves.

From the gallery:

- [an interactive session with a GPU](/gallery/int-gpu/) on `p1i`
- [VS Code on the login node](/gallery/rem-lgn/)
- [moving code and data](/gallery/xfr-dat/) on and off the cluster

See [every example that runs on P1 DTU HPC](/gallery/?provider=dtu).

## Policy & Support

### What are the limits?

**Compute**

| Limit | Value |
| --- | --- |
| Max wall time | 72 hours |
| GPUs per job | 2 (one node) |
| GPUs at once, per person | About half the cluster, so roughly 8 of the 16 H100s |

**Storage**

| Limit | Value |
| --- | --- |
| Home directory | 30 GB |
| Project storage | 500 GB default, under /dtu/p1/ |
| Shared pool | 60 TiB |

For more storage, contact the governance group at
[compute-governance-p1@aicentre.dk](mailto:compute-governance-p1@aicentre.dk).

### How much storage do I get?

Your 500 GB project allowance comes out of a 60 TiB shared pool. Use it as working space, not an
archive. Keep code in git and a second copy of anything you cannot regenerate on institutional
storage or your own machine. Before a project ends, ask the compute coordinator what to move off.

> **GDPR Compliance**
>
> Only non-GDPR data belongs here, such as public datasets and open benchmarks. Your home directory
> is private, but the cluster is not approved for personal data, whatever you do at file level. If
> your project involves personal data, talk to the compute coordinator before you start.

### What if I need more GPUs?

**Form:** [Planned GPU allocation](https://forms.cloud.microsoft/e/yN8Vxi7KTc) Request a raised GPU limit for a run that also needs protecting from cancellation: a multi-day training, a deadline, or a workload that cannot checkpoint.

**Citing this cluster:** if the P1 DTU HPC contributed to your work, please cite the DTU Computing Center resources DOI, [10.48714/DTU.HPC.0001](https://doi.org/10.48714/DTU.HPC.0001). That covers help from the DTU DCC team or the P1 compute coordinator just as much as time spent running jobs.

### Who do I contact?

- DTU HPC Support Team: support@hpc.dtu.dk
- P1 compute coordinator: bstja@dtu.dk
- Book a consultation with the coordinator: https://book.starostka.io/benjamin/compute-needs
- Helped with your work? Acknowledge the coordinator's ORCID: https://orcid.org/0000-0002-2740-1651

## Hardware Specification

| | |
| --- | --- |
| Nodes | 8× Lenovo ThinkSystem SR665 V3 (7 batch, 1 interactive) |
| CPU per node | 2× AMD EPYC 9354 (32 cores each) |
| RAM per node | 768 GB |
| GPUs | 16× NVIDIA H100 PCIe (80 GB), 2 per node |
| Shared storage | 60 TiB at /dtu/p1, 500 GB default project quota |
| Home directory | 30 GB |
| Storage fabric | NFSv4.2 over 100 Gbit EDR InfiniBand with RDMA, 11× Micron 7450 NVMe (RAID 5) |
| Operating system | AlmaLinux |
| Scheduler | IBM Spectrum LSF |
| Queues | p1 (batch, 7 nodes) / p1i (interactive, 1 node) |