The goal of this paper is to obtain online abstractions for coupled multi-agent systems in a decentralized manner. A discrete model which captures the motion capabilities of each agent is derived over a bounded time-horizon, by discretizing a corresponding overapproximation of the agent's reachable states. The individual abstractions' composition provides a correct representation of the coupled continuous system over the horizon and renders the approach appropriate for control synthesis under high-level specifications which are assigned to the agents over this time window. Sufficient conditions are also provided for the space and time discretization to guarantee the derivation of deterministic abstractions with tunable transition capabilities.