For high-end industrial image processing applications with real-time requirements, FPGAs are often used as custom accelerators. High level synthesis tools, such as CatapultC, provide a compelling means of speeding up the algorithmic hardware design. However, increasing image resolutions make it ever more difficult to obtain sufficient throughput from external SDRAM frame buffers while providing simple, low-latency memory resources for the data path. To address these issues, this paper proposes a platform-based design with a custom memory system of buffers, caches and an optimized commercial memory controller that improves available SDRAM bandwidth by up to 4×. This greatly facilitates the high level synthesis flow, which is demonstrated by implementing two memory-intensive algorithms using 47.0 Gbit/s and 5.7 Gbit/s of on-chip and off-chip memory bandwidth respectively