Graphic-pattern-based implicit authentication has been successfully exploited to elevate the security of smartphones. On-screen pressure is one of the key features in such approach since it can reveal users' touch pattern. However, state-of-the-art approaches rely on a system API to obtain on-screen pressure, which is not adequately accurate and cannot meet the demands of robust implicit authentication. To bridge this gap, we propose PresSafe, a novel implicit authentication system that utilizes the smartphone's built-in barometer sensor to measure pressure during the unlocking process, and to utilize the pressure data in authentication. A key technical challenge in utilizing barometer sensing, however, is to understand the user activity through measured pressure. To overcome this challenge, PresSafe leverages barometer data along with data from other conventional but heterogeneous ambient sensors to produce accurate and robust user activity descriptions. PresSafe utilizes a transfer learning based hybrid workflow to integrate user activity representation learning with a lightweight classical authentication algorithm to obtain a unified model. This approach offloads computational cost from the terminal and addresses privacy concerns. To ensure applicability of our approach despite data heterogeneity and insufficient training data, we utilize a channel-adaptive data processing mechanism. Extensive experiments utilizing more than 70,000 records from 23 volunteers in 6 different locations show that PresSafe achieves an FAR of 0.45 %, an FRR of 0.49 %, and an EER of 0.47 %, which clearly demonstrate its superiority over several existing solutions.