To get cold water, will I have to let run to remove all of the warm water that is being recirculated from the hot line?

You will have some tempered water in the cold water line where the valve is yes this is the tradeoff due to your home not having a dedicated return if you had a dedicated return line you could use a dedicated recirculating pump however this pump is not designed for that operation.