Extracting building information from Very-High-Resolution (VHR) satellite images is critical for urban mapping and monitoring. Traditional manual annotation methods are labor-intensive and costly, making automated solutions highly desirable. Segment Anything Model (SAM), a foundation model trained mostly on natural images, has recently shown high performance on diverse segmentation tasks. However, due to differences in perspective and the average size of objects in the images, SAM exhibits lower performance when extracting buildings from satellite imagery. These limitations, derived from differences in image domains, can be addressed by fine-tuning the model with satellite images and preprocessing the input images. However, various hyperparameters, such as learning rate, batch size, and optimizer type, deeply impact the performance of the fine-tuned model, and thus, in-depth investigations on these hyperparameters are critical for model adaptation. To identify the optimal hyperparameter configuration, we conducted extensive experiments with combinations of hyperparameter settings using Korea Multi-Purpose Satellite (KOMPSAT) images. Additionally, various upscaling methods and objectby-object preprocessing techniques were compared and evaluated, leading to the proposal of an effective preprocessing approach. With the optimal combination, an F1 Score of 0.862, an Intersection over Union (IoU) of 0.761, and a mean IoU (mIoU) of 0.705 were achieved using AdamW optimizer, object-by-object cropping, and 100-pixel buffering. The proposed hyperparameter optimization method in our research underscores the effectiveness of fine-tuning SAM for accurate building extraction in VHR satellite imagery, thereby enabling more reliable data interpretation and decision-making processes in automated remote sensing applications.