Drivers that don't support PRIME will not have initialized the PRIME
specific private component of struct drm_file. If called for such
drivers, the drm_gem_remove_prime_handles() function will crash. Fix
it by checking for PRIME support prior to removing the PRIME handles.

Signed-off-by: Thierry Reding <treding@nvidia.com>
Signed-off-by: Dave Airlie <airlied@gmail.com>