Lightweight physical layer security schemes that have recently attracted a lot of attention include physical unclonable functions (PUFs), RF fingerprinting / proximity based authentication and secret key generation (SKG) from wireless fading coefficients. In this paper, we propose a fast, privacy-preserving, zero-round-trip-time (0-RTT), multi-factor authentication protocol, that for the first time brings all these elements together, i.e., PUFs, proximity estimation and SKG. We use Kalman filters to extract proximity estimates from real measurements of received signal strength (RSS) in an indoor environment to provide soft fingerprints for node authentication. By leveraging node mobility, a multitude of such fingerprints are extracted to provide resistance to impersonation type of attacks e.g., a false base station. Upon removal of the proximity fingerprints, the residual measurements are then used as an entropy source for the distillation of symmetric keys and subsequently used as resumption secrets in a 0-RTT fast authentication protocol. Both schemes are incorporated in a challenge-response PUF-based mutual authentication protocol, shown to be secure through formal proofs using Burrows, Abadi, and Needham (BAN) and Mao and Boyd (MB) logic, as well as the Tamarin-prover. Our protocol showcases that in future networks purely physical layer security solutions are tangible and can provide an alternative to public key infrastructure in specific scenarios.