Intro to GPUs on Explorer

Intro to GPUs on Explorer

What a GPU is for, which ones Explorer has, and how to ask for one in a job, so the right work lands on the right hardware. Video 4 of 13 in the series Research Computing on Explorer and AICR, built from Northeastern Research Computing's training material. Liam, in for Bear, narrates.

Chapters 0:00 Opening 0:12 The idea 0:34 Terms in this lecture 0:55 I · The GPU Partitions 2:07 II · What A GPU Is 2:45 III · Three Ways In 3:09 IV · In The Browser 3:42 V · Interactive And Batch 4:26 VI · Watching The GPU 4:51 VII · Using It Well 5:36 VIII · Habits 5:57 Recap 6:26 Your turn

Your turn. Paste this into Claude: Here is a PyTorch loop I run on a GPU: [paste it]. First, tell me whether it processes samples one at a time or in batches, and rewrite it to batch if it doesn't. Second, write the DataLoader line with workers and pinned memory. Third, write me the Slurm srun command for a one-hour test on one GPU and the sbatch script for the real run. Then check two things yourself: did it keep the arithmetic identical when it batched? did it ask for only one GPU?

Sources

• gpu_simple.py, example1/, example2/, "Introduction to GPUs on Explorer", Northeastern Research Computing, presenter Arsalan Akhter; github.com/northeastern-rc-training/Intro-to-GPUs-2026. Read 2026-10-06. Questions about the clusters: rchelp@northeastern.edu.

Made with the brutalist.art toolkit: Manim, Remotion and Kokoro, free and local. Narration is synthetic (Liam, in for Bear). This is an independent educational film, not an official Northeastern publication.

▶ The full series playlist — Brutalist https://www.youtube.com/playlist?list=PLG9H-C6rp5RU

What this video covers

• 0:00 — Opening
• 0:12 — The idea
• 0:34 — Terms in this lecture
• 0:55 — I · The GPU Partitions
• 2:07 — II · What A GPU Is
• 2:45 — III · Three Ways In
• 3:09 — IV · In The Browser
• 3:42 — V · Interactive And Batch
• 4:26 — VI · Watching The GPU
• 4:51 — VII · Using It Well
• 5:36 — VIII · Habits
• 5:57 — Recap
• 6:26 — Your turn

Transcript

A lightly formatted transcript of the video, with timestamps so you can jump to any section on YouTube.

Opening

[0:00] Research computing on Explorer and AICR. Intro to GPUs on Explorer. What a GPU is for, which ones Explorer has, and how to ask for one in a job so the right work lands on the right hardware.

The idea

[0:12] Ciao, this is Liam in for Bear. First, a correction. A GPU is not a faster CPU. It's thousands of small, simple cores working at once, which is why it wins at some jobs and loses at others. This lecture is the beginner's introduction to the GPUs on Northeastern's Explorer cluster, which partitions and card types there are, what a GPU is, the three ways to get one, how to watch it work, and the two code habits that keep it busy.

Terms in this lecture

[0:34] Six terms: a core, one processing unit that runs instructions on its own. A GPU, a chip with thousands of small cores for parallel work. A partition, a named group of cluster nodes with its own limits. GRES, generic resources, Slurm's name for the GPUs you ask for. Srun, the command that gives you a live session on a node. And Sbatch, the command that submits a script to run without you. Start with where the GPUs

I · The GPU Partitions

[0:56] Are. Explorer groups its GPU nodes in the partitions, and six of the ones you'll meet. GPU, the general one up to 8 hours. GPU interactive for open on demand sessions, 2 hours. GPU short, 2 hours too, but prioritized so quick tests schedule fast. Multigpu for jobs that need several cards, 24 hours. Courses GPU for classes, 24 hours. And sharing GPUs that Northeastern PIs lend to everyone, 1 hour at a time. Which cards live in a partition? Ask Sinfo for the GRES column. On the GPU partition, the answer is a mix. V100s in two flavors, T4s, A100s in two node sizes, and H200s, eight to a node. The number after

The card is how many sit in each node. Before you submit, it pays to know what's free. Sinfo with the node list, the GRES and the GRES in use shows that per node. And to check one node, name it. One trap, a node in the mixed state isn't telling you about its GPUs. Mixed describes its CPUs. Only the GRES used column says whether a card is free. There's also a GPU monitor app on Open On Demand. It colors every GPU node as idle, mixed, or allocated and lists how many GPU jobs are waiting in the queue by card type, so you can pick a card with a shorter line.

II · What A GPU Is

[2:07] So, what is a GPU? Start with a core, one unit that runs instructions on its own. A CPU has a few large, complex cores. A GPU has thousands of small, simple ones. For a problem you can split into thousands of identical little pieces, the GPU is the right tool. Two pictures from the training, delivering thousands of pizzas across a city. A few Ferraris or thousands of bicycles? The bicycles. Or a room of professors at whiteboards against a stadium of students with calculators? The professors win at a hard proof, but to multiply a very large matrix, a multiplying and an add for every cell, the stadium wins. Machine learning, scientific simulation, data analysis, those are stadium jobs. Three ways to

III · Three Ways In

[2:46] Use them. Open On Demand, a website where a GPU session is a form. An interactive session from the terminal with S run. And a batch job, a script you submit with S batch and walk away from. One setup for all three. Log in, go to your scratch directory, and clone the training repository. Everything the demos use is in there. A Python environment script, a tiny GPU check, and the example code.

IV · In The Browser

[3:10] First way, the browser. On the Open On Demand dashboard, open standard apps and pick VS Code Server. The form asks for a simple password for the editor, a partition, sharing here, and the number of GPUs, one. Leave the rest as it is and launch. When the session connects, open the cloned folder in VS Code, then a terminal from the view menu. Make the environment script executable, run it, and check that which Python points into the new environment. Then the smallest possible test, the script prints the CUDA_VISIBLE_DEVICES variable. Zero means the scheduler gave you a GPU.

V · Interactive And Batch

[3:43] Second way, a live session from the terminal. S run on the sharing partition with one GPU drops you onto a GPU node. Activate the same environment and run the fuller check, which asks PyTorch for the card. Here's that check. It prints the PyTorch version, whether CUDA is available, the GPU's name, and then puts a tiny tensor on the card and prints where it lives. Open it in VI, add an NVIDIA SMI line, and run it again to see the card itself. Third way, batch the same work as a script with Sbatch lines. The GPU partition, one GPU, four CPUs, 16 GB, 1 hour, and log files. It activates the environment and runs the check. Submit it with Sbatch. It runs when a card is free, writes its output to the logs folder, and you read it

VI · Watching The GPU

[4:26] Later. Is the card actually working? Put NVIDIA SMI in your script to print its state while the job runs. And after a job finishes, GPU logs with the job ID gives a minute-by-minute record. This real one from the training for 5 minutes, nothing. Then memory climbs to 23%, then 78 with utilization still zero. The model is loading. Only at the end does utilization reach 56%. Most of this job's time the GPU was waiting. Two

VII · Using It Well

[4:52] Habits keep a GPU busy. First, batching. The naive version sends 50,000 single rows to the card, one request at a time, and pays the overhead 50,000 times. The batched version sends all 50,000 rows at once and runs the model once. Same arithmetic, far less overhead, and the card's thousands of cores all have work. Picture it, 50,000 tiny parcels through the door one at a time, the card idle between each, or one crate and every core busy at once. Second, data loading. With zero workers, the CPU prepares one batch at a time, and the GPU waits for each. With workers, pinned memory, and a non-blocking copy, the next batch is ready before the card finishes the current one. Batch your data, give the loader workers, keep the card working.

VIII · Habits

[5:36] Four habits to leave with. Choose the right partition, GPU short for a quick job, and Sbatch for anything long. Request only what you need, and check a finished job with seff to see how many CPUs it really used. Use the GPU tune libraries, CUDA and cuDNN rather than plain code, and profile before you optimize. Stuck? Email rc.help@northeastern.edu or come to office hours. Let's recap

Recap

[5:58] With Claude. Six GPU partitions, GPU short schedules fast, and sharing is lent by PIs. Sinfo with the GRES column shows the cards and the gres used column not mixed says what's free. A GPU is thousands of simple cores, the stadium not the professors. Three ways in VS Code on open on demand, srun with one GPU, or an sbatch script. GPU log shows the cards memory and utilization minute by minute. And batch your data and give the data loader workers so the card stays busy.

Your turn

[6:27] Your turn. Paste this into Claude. Here's a PyTorch loop I run on a GPU. Paste it. First, tell me whether it processes samples one at a time or in batches and rewrite it to batch if it doesn't. Second, write the data loader line with workers and pinned memory. Third, write me the Slurm srun command for a 1-hour test on one GPU and the sbatch script for the real run. Then check two things yourself. Did it keep the arithmetic identical when it batched? And did it ask for only one GPU? Intro to GPUs on Explorer at Nick Bear Brown.