Device-free localization (DFL) systems locate a person in an environment by measuring the changes in received signal on links in a wireless network. A fingerprint-based DFL method collects a training database of measurement fingerprints and uses a machine learning classifier to determine a person's location from a new fingerprint. However, as the environment changes over time due to furniture or other objects being moved, the fingerprints diverge from those in the database. This paper addresses, for DFL methods that use received signal strength as measurements, the degradation caused as a result of environmental changes. We perform experiments to quantify how changes in an environment affect accuracy, through a repetitive process of randomly moving an item in a residential home and then conducting a localization experiment, and then repeating. We quantify the degradation as well as consider ways to be more robust to environmental change. We find that the localization error rate doubles, on average, for every six random changes in the environment. We find that the random forests classifier has the lowest error rate among four tested. We present a correlation method for selecting channels which decreases the localization error rate from 4.8% to 1.6%.