Discussions

• 7 You must be logged in to vote

  🤖

  Imbalanced GPU memory with DDP, single machine multiple GPUs

  Alexfinkelshtein asked Mar 17, 2021 in DDP / multi-GPU / multi-node · Unanswered

  

  

  

  

  5
• 1 You must be logged in to vote

  🤖

  No response when training the model using SLURM

  morestart asked Nov 26, 2024 in DDP / multi-GPU / multi-node · Unanswered

  

  

  0
• 1 You must be logged in to vote

  ⚡

  Dynamic Dataloader

  leonardcaquot94 asked Nov 22, 2024 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Unanswered

  

  

  0
• 1 You must be logged in to vote

  ⚡

  Low GPU Utilization

  accelerator: cuda Compute Unified Device Architecture GPU performance

  Sakurakdx asked Aug 15, 2022 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Unanswered

  

  

  

  

  4
• 1 You must be logged in to vote

  ⚡

  [Bug] ValueError: Expected 'baseclass' to be a subclass type or a tuple of subclass types: () when running 'python run.py fit --config src/configs/retail.yaml'

  enermyfree asked Nov 22, 2024 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Unanswered

  

  

  1
• 1 You must be logged in to vote

  💬

  Best practices for automated parameter sweeps

  HenrikAsmuth started Nov 18, 2024 in General

  

  

  2
• 1 You must be logged in to vote

  💬

  Is there anyway to suppress print message like "INFO: LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]"

  XuShenLZ started Nov 20, 2024 in General

  

  

  0
• 1 You must be logged in to vote

  💬

  Pipeline Parallelism Support

  mitchadam started Nov 14, 2024 in General

  

  

  0
• 1 You must be logged in to vote

  🤖

  DDP share information before processes are properly initialized.

  AndrasSalamon asked Nov 12, 2024 in DDP / multi-GPU / multi-node · Closed · Unanswered

  

  

  0
• 1 You must be logged in to vote

  ⚡

  Use a pretrained model for training another model

  haertelkar asked Nov 9, 2024 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Closed · Unanswered

  

  

  1
• 8 You must be logged in to vote

  ⚡

  Gist on how to use Schedule-Free in Lightning

  CompRhys asked Apr 10, 2024 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Unanswered

  

  

  

  

  10
• 2 You must be logged in to vote

  ⚡

  Log the same metric when using multiple dataloaders in validation_step()

  thangld201 asked Sep 13, 2023 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Unanswered

  

  

  1
• 2 You must be logged in to vote

  ⚡

  self.global_step understanding

  ajinkyaambatwar asked Jun 16, 2021 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Unanswered

  

  

  

  

  3
• 6 You must be logged in to vote

  ⚡

  Timed out initializing process group in store based....

  EvanZ asked Nov 9, 2022 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Unanswered

  

  

  

  

  6
• 1 You must be logged in to vote

  🤖

  ddp_weakref is not set during the backward pass

  lokesh-vr-17773 asked Nov 4, 2024 in DDP / multi-GPU / multi-node · Unanswered

  

  

  0
• 7 You must be logged in to vote

  ⚡

  How to overwrite checkpoints with ModelCheckpoint

  nsabir2011 asked Nov 24, 2022 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Unanswered

  

  

  1
• 1 You must be logged in to vote

  ⚡

  What is trainer/global_step in wandb logging?

  edmcman asked Oct 30, 2024 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Unanswered

  

  

  1
• 3 You must be logged in to vote

  ⚡

  Gradient checkpointing with DDP in a loop

  shivammehta25 asked Nov 11, 2021 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Unanswered

  

  

  

  

  8
• 2 You must be logged in to vote

  🤖

  Optuna HPO & Lightning Multi-GPU Training using DDP on SLURM - ValueError: World Size does not Match

  eTuDpy asked May 30, 2024 in DDP / multi-GPU / multi-node · Unanswered

  

  

  

  

  4
• 1 You must be logged in to vote

  ⚡

  Exact behaviour of .prepare_data() and .setup() in LightningDataModule when passing DM or dataloaders to Trainer in respect to internal hooks.

  tiefenthaler asked Oct 29, 2024 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Unanswered

  

  

  0
• 1 You must be logged in to vote

  💬

  How to change auto-requeue hpc.ckpt path

  arijit-hub started Oct 22, 2024 in General · Closed

  

  

  1
• 1 You must be logged in to vote

  ⚡

  Active Learning Trainer

  BerAnton asked Oct 25, 2024 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🤖

  Collect Threads after Trainer.fit

  42elenz asked Oct 23, 2024 in DDP / multi-GPU / multi-node · Unanswered

  

  

  0
• 1 You must be logged in to vote

  ⚡

  Logging in Multi - GPU and new on_validation_step function

  42elenz asked Oct 23, 2024 in Lightning Trainer API: Trainer, LightningModule, LightningDataModule · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🤖

  Both GPUs are getting same data from IterableDataset with DDP

  akshat-suwalka asked Jul 22, 2024 in DDP / multi-GPU / multi-node · Unanswered

  

  

  1