Abstract-Creating accurate models of stochastically selfassembling systems is a key step in developing control strategies, centralized or distributed, for the self-assembly process. This paper comparatively studies several aspects of developing probabilistic models for programmable self-assembling systems of stochastically interacting modules. In particular, we systematically investigate Markov models as well as hidden Markov models to predict the self-assembly process dynamics. We consider a case study leveraging our fluidic self-assembly robotic system. The ground truth is obtained through a high-fidelity simulation, calibrated using real experimental data. We first consider Markov models and employ the formalism of chemical reaction networks. In order to compute the model parameters, i.e. the reaction rates in the network, three different methods are studied. We then investigate the validity of the underlying well-mixed assumption, and thus the Markov property, for our system through estimation of the diffusion coefficient, through two different approaches. The system is shown to be borderline well-mixed, motivating extension of the initial Markov models to more complex models in order to achieve improved model accuracy. We formulate an automatic method for creating a hidden Markov model starting from a Markov model, based on a previously existing systematic method. Sample trajectories of the models are realized using the Gillespie's method. The resulting hidden Markov model is shown to achieve an improved accuracy over the standard Markov model.