The endeavour of predicting stock prices using different mathematical and technological methods and tools is not new. But the recent advancements and curiosity regarding big data and machine learning have added a new dimension to it. In this research study, we investigated the feasibility and performance of the multiple regression method in the prediction of stock prices. Here, multiple regression was used on the basis of the incremental machine learning setting. The study conducted an experiment to predict the closing price of stocks of six different organizations enlisted in the Dhaka Stock Exchange (DSE). Three years of historical stock market data (2017-2019) of these organizations have been used. Here, the Multiple Regression, Squared Loss Function, and Stochastic Gradient Descent (SGD) algorithms are used as a predictor, loss function, and optimizer respectively. The model incrementally learned from the data of several stock-related attributes and predicted the closing price of the next day. The performance of prediction was then analysed and assessed on the basis of the rolling Mean Absolute Error (MAE) metric. The rolling MAE scores found in the experiment are quite promising.