Fix VLM no image inference issue and add tests#634

meatybobby wants to merge 10 commits intomainNVIDIA-NeMo/Export-Deploy:mainfrom

bobchen/vlm_fixNVIDIA-NeMo/Export-Deploy:bobchen/vlm_fixCopy head branch name to clipboard