Blockchain Technology is an emerging technology with the potential to disrupt a number of application domains. Though blockchain platforms like Bitcoin and Ethereum have seen immense success and acceptability, their nature of being public and anonymous make them unsuitable for many enterprise level use-cases. To address this issue, Linux Foundation has started an open source umbrella initiative, known as the Hyperledger Platforms. Under this initiative, a number of private blockchain platforms have been developed which can be used for different enterprise level applications. However, the scalability and performance of these private blockchains must be examined to understand their suitability for different use-cases. Recent researches and projects on performance benchmarking for private blockchain systems are very specific to use-cases and are generally tied to a blockchain platform. In this article, we present BlockMeter, an application agnostic performance benchmarking framework for private blockchain platforms. This framework can be utilised to measure the key performance matrices of any application deployed on top of an external private blockchain application in real-time. In this article, we present the architecture of the framework and discuss its different implementation aspects. Then, to showcase the applicability of the framework, we use BlockMeter to evaluate the two most widely used Hyperledger platforms, Hyperledger Fabric and Hyperledger Sawtooth, against a number of use-cases.