Sensor nodes are being used in numerous domains for data collection and analysis. The ability to perform on device data processing increases the functionality and lifetime of a network as it avoids network transmission. Previous work has developed algorithms for sorting on sensor nodes with flash memory. These algorithms favour reads over writes due to the asymmetric costs. However, previous algorithms have not exploited the ability to perform random reads at the same cost as sequential reads. In this paper, we propose a new algorithm called Flash MinSort that uses random reads to rapidly sort in flash memory using a small amount of memory. The algorithm works especially well for sensor data which is often temporally clustered. Experimental results on random and real sensor data show that Flash MinSort is two to ten times faster than previous approaches for small memory sizes where external merge sort is not executable.

Many embedded system applications involve storing and querying large datasets. Existing research in this area has focused on adapting and applying conventional database algorithms to embedded devices. Algorithms designed for processing queries on embedded devices must be able to execute given the small amount of available memory and energy constraints. Most embedded devices use flash memory to store large amounts of data. Flash memory has unique performance characteristics that can be exploited to improve algorithm performance. In this paper, we describe the Flash MinSort external sorting algorithm that uses an index, generated at runtime, to take advantage of fast random reads in flash memory. This algorithm adapts to the amount of memory available and performs best in applications where sort keys are clustered. Experimental results show that Flash MinSort is two to ten times faster than previous approaches for small memory sizes where external merge sort is not executable.

No abstract