The application of state-dependent speciation and extinction (SSE) models to phylogenetic trees has revealed an important role for traits in diversification. However, this role remains comparatively unexplored on islands, which can include multiple independent clades resulting from different colonization events. Here, we perform a robustness study to identify how trait-dependence in rates of island colonization, extinction and speciation (CES rates) affects the estimation accuracy of a phylogenetic model that assumes no rate variation between trait states. We extend the DAISIE (Dynamic Assembly of Islands through Speciation, Immigration and Extinction) simulation model to include state-dependent rates, and evaluate the robustness of the DAISIE inference model using simulated data. Our results show that when the CES rate differences between trait states are moderate, DAISIE shows negligible error for a variety of island diversity metrics. However, for large differences in speciation rates, we find large errors when reconstructing clade size variation and non-endemic species diversity through time. We conclude that for many biologically realistic scenarios with trait-dependent speciation and colonization, island diversity dynamics can be accurately estimated without the need to explicitly model trait dynamics. Nonetheless, our new simulation model may provide a useful tool for studying patterns of trait variation.