Modelling the mass distributions of strong gravitational lenses is often necessary to use them as astrophysical and cosmological probes. With the high number of lens systems ( 10 5 ) expected from upcoming surveys, it is timely to explore efficient modeling approaches beyond traditional Markov-Chain Monte-Carlo techniques that are time consuming. We train a Convolutional Neural Network (CNN) on images of galaxy-scale lens systems to predict the five parameters of the Singular Isothermal Ellipsoid (SIE) mass model (lens center x and y, complex ellipticity e x and e y , and Einstein radius θ E ). To train the network, we simulate images based on real observations from the Hyper Suprime-Cam Survey for the lens galaxies and from the Hubble Ultra Deep Field as lensed galaxies. We tested different network architectures and the effect of different data sets such as using only double or quad systems defined based on the source center, and using different input distributions of θ E . We find that the CNN performs well and obtain with the network trained on both doubles and quads with a uniform distribution of θ E > 0.5 the following median values with 1σ scatter: ∆x = (0.00 +0.30 −0.30 ) , ∆y = (0.00 +0.30 −0.29 ) , ∆θ E = (0.07 +0.29 −0.12 ) , ∆e x = −0.01 +0.08 −0.09 and ∆e y = 0.00 +0.08 −0.09 . The bias in θ E is driven by systems with small θ E . Therefore, when we further predict the multiple lensed image positions and time delays based on the network output, we apply the network to the sample limited to θ E > 0.8 . In this case, the offset between the predicted and input lensed image positions is (0.00 +0.29 −0.29 ) and (0.00 +0.32 −0.31 ) for the x and y coordinates, respectively. For the fractional difference between the predicted and true time delay, we obtain 0.04 +0.27 −0.05 . Our CNN model is able to predict the SIE parameter values in fractions of a second on a single CPU and with the output we can predict the image positions and time delays in an automated way, such that we are able to process efficiently the huge amount of expected galaxy-scale lens detections in the near future.