Supplier Free Shipping For Some Products

I want some products from a Supplier to have free shipping. But not all their products. I have 2 shipping methods available for Supplier A. One is UPS Real Time and other is Free Shipping.

I checked the Free Shipping box for a specific product.

But when I go to purchase that product it has both shipping methods available. Any idea why?

If I disable the Free Shipping option for the Supplier than I get no shipping options for this product. This makes sense since the product is marked Free Shipping but that method is not available to that Supplier.

Version 4.6.2 using Supplier Add-On