A4 & A4X TRTLLM GKE single-host inference benchmarking recipes#146

Open

hmhv1222 wants to merge 32 commits intomainfrom

mhvictorhau-20260303-a4-a4x-trtllm-benchmarking

Commits on Mar 13, 2026

A4 & A4X TRTLLM GKE Single-Host Inference Recipes, ReadMe and Config Files
hmhv1222
committed
Add benchmarking configs warning paragraphs in TRTLLM inference ReadMe
hmhv1222
committed
Remove memory resources constraint in A4X serving-launcher.yaml and change default number of GPUs in A4 TRTLLM inference recipe
hmhv1222
committed

Commits on Mar 30, 2026

Adding WAN Recipe for A4 (#144)

depksingh
authored and
hmhv1222
committed
Adding the A4x wan receipe (#143)

Priya-Quad
authored and
hmhv1222
committed
a4x cs llama 405b

notabee
authored and
hmhv1222
committed
Adding a recipe for Llama3.1-70B with gbs 256 scaled Recipe from 16 nodes. (#148)

incredere
authored and
hmhv1222
committed
Update submit.slurm

notabee
authored and
hmhv1222
committed
Add Qwen3 235B A22B recipe on 16-node B200 #recipebot

weikuo0506
authored and
hmhv1222
committed
Update README to match Qwen3 235B 16-node recipe template

weikuo0506
authored and
hmhv1222
committed
Update README to match the exact template structure

weikuo0506
authored and
hmhv1222
committed
temp

Alina-PANG
authored and
hmhv1222
committed
Add Qwen3 235B A22B FP8MX GBS8192 recipe on 32-node B200 #recipebot

weikuo0506
authored and
hmhv1222
committed
Update README to use exact strict formatting of template

weikuo0506
authored and
hmhv1222
committed
Move 16node-BF16-GBS4096 files into recipe directory #recipebot

weikuo0506
authored and
hmhv1222
committed
Add GPT-OSS 120B 8-node BF16 recipe #recipebot

weikuo0506
authored and
hmhv1222
committed
Add DeepSeek V3 32-node FP8MX SEQ4096 GBS4096 recipe #recipebot

weikuo0506
authored and
hmhv1222
committed
Restructure qwen3 recipes by nemo version

weikuo0506
authored and
hmhv1222
committed
Add QWEN3 235B 32-node BF16 SEQ4096 GBS4096 recipe #recipebot

weikuo0506
authored and
hmhv1222
committed
Add Llama 3.1 405B FP8CS 16-node recipe on a4 (nemo2602)

Alina-PANG
authored and
hmhv1222
committed
Add recipe for Llama3.1 405B FP8CS B200 256GPUs

Alina-PANG
authored and
hmhv1222
committed
Add Qwen3 30B Nemo Pretraining recipe on A4

Alina-PANG
authored and
hmhv1222
committed
Update DeepSeek V3 32-node BF16 NEMO26.02 recipe with optimized launcher flags #recipebot

weikuo0506
authored and
hmhv1222
committed
Move 32node recipe into nemo2602 subdirectory and update README

weikuo0506
authored and
hmhv1222
committed
Move 32node NEMO25.11 recipe into nemo2511 subdirectory

weikuo0506
authored and
hmhv1222
committed
#recipebot Add NeMo pretraining A4 Llama3.1 70b recipe

Alina-PANG
authored and
hmhv1222
committed
chore: Migrate gsutil usage to gcloud storage

gurusai-voleti
authored and
hmhv1222
committed
chore: update

gurusai-voleti
authored and
hmhv1222
committed
Add Qwen3 235B A22B Megatron-Bridge pretraining recipe on A4X Slurm

notabee
authored and
hmhv1222
committed
Add Qwen3 235B A22B pretraining recipe on A4X Slurm Cluster

notabee
authored and
hmhv1222
committed
Qwen3 235B A22B, Qwen 2.5 VL 7B & Llama 3.1 405B config files and launcher modification
hmhv1222
committed
Support running VL models with trtllm-launcher.sh
hmhv1222
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

A4 & A4X TRTLLM GKE single-host inference benchmarking recipes#146

A4 & A4X TRTLLM GKE single-host inference benchmarking recipes#146
hmhv1222 wants to merge 32 commits intomainfrom
mhvictorhau-20260303-a4-a4x-trtllm-benchmarking

Commits on Mar 13, 2026

A4 & A4X TRTLLM GKE Single-Host Inference Recipes, ReadMe and Config Files

Add benchmarking configs warning paragraphs in TRTLLM inference ReadMe

Remove memory resources constraint in A4X serving-launcher.yaml and change default number of GPUs in A4 TRTLLM inference recipe

Commits on Mar 30, 2026

Adding WAN Recipe for A4 (#144)

Adding the A4x wan receipe (#143)

a4x cs llama 405b

Adding a recipe for Llama3.1-70B with gbs 256 scaled Recipe from 16 nodes. (#148)

Update submit.slurm

Add Qwen3 235B A22B recipe on 16-node B200 #recipebot

Update README to match Qwen3 235B 16-node recipe template

Update README to match the exact template structure

temp

Add Qwen3 235B A22B FP8MX GBS8192 recipe on 32-node B200 #recipebot

Update README to use exact strict formatting of template

Move 16node-BF16-GBS4096 files into recipe directory #recipebot

Add GPT-OSS 120B 8-node BF16 recipe #recipebot

Add DeepSeek V3 32-node FP8MX SEQ4096 GBS4096 recipe #recipebot

Restructure qwen3 recipes by nemo version

Add QWEN3 235B 32-node BF16 SEQ4096 GBS4096 recipe #recipebot

Add Llama 3.1 405B FP8CS 16-node recipe on a4 (nemo2602)

Add recipe for Llama3.1 405B FP8CS B200 256GPUs

Add Qwen3 30B Nemo Pretraining recipe on A4

Update DeepSeek V3 32-node BF16 NEMO26.02 recipe with optimized launcher flags #recipebot

Move 32node recipe into nemo2602 subdirectory and update README

Move 32node NEMO25.11 recipe into nemo2511 subdirectory

#recipebot Add NeMo pretraining A4 Llama3.1 70b recipe

chore: Migrate gsutil usage to gcloud storage

chore: update

Add Qwen3 235B A22B Megatron-Bridge pretraining recipe on A4X Slurm

Add Qwen3 235B A22B pretraining recipe on A4X Slurm Cluster

Qwen3 235B A22B, Qwen 2.5 VL 7B & Llama 3.1 405B config files and launcher modification

Support running VL models with trtllm-launcher.sh