Bone is an organic-inorganic hierarchical biocomposite. Its basic building block is mineralized collagen fibers with both intrafibrillar and extrafibrillar mineralization, which is believed to be regulated by non-collagenous proteins (NCPs) with polyanionic domains. In this study, collagen fibrils with both intrafibrillar and extrafibrillar mineralization were successfully prepared and the mechanism of biomineralization was proposed. Building on this success, a unique biomimetic lamellar scaffold composed of collagen fibrils with both intrafibrillar and extrafibrillar mineralization was fabricated using a combination of self-compression and unidirectional freeze drying approach. To achieve intrafibrillar mineralization, polyacrylic acid (PAA) was used to sequester calcium and phosphate ions to form fluidic PAA-amorphous calcium phosphate (PAA-ACP) nanoprecursors. At the presence of sodium tripolyphosphate (TPP), PAA-ACP nanoprecursors were modulated to orderly deposit within the gap zone of collagen fibrils. The effect of PAA concentration on the intrafibrillar and extrafibrillar mineralization of reconstituted collagen fibrils was investigated. It was found that with the decrease of PAA concentration, the inhibitory effect of PAA on mineralization and the stability of ACP nanoprecursors decreased. As a result, more minerals were deposited both within and on the surface of the collagen fibrils. Moreover, with the ability to reproduce biomineralization of collagen fibrils, it allowed us to fabricate a biomimetic hierarchical collagen/hydroxyapatite scaffolds composed of both intrafibrillar and extrafibrillar minerals using a bottom-up approach. This technique renders a promising biomimetic scaffold, which will be suitable for bone repair and regeneration.