BackgroundDifferentiation of telomere length is of vital importance because telomere length is closely related with several deadly diseases such as cancer. Additionally, G-quadruplex and i-motif formation in telomeric DNA have been shown to act as a negative regulator of telomere elongation by telomerase in vivo and are considered as an attractive drug target for cancer chemotherapy.ResultsIn this assay, Ag nanoclusters templated by hyperbranched polyethyleneimine (PEI–Ag NCs) are designed as a new novel resonance Rayleigh scattering (RRS) probe for sensitive differentiation of telomere length and monitoring special motifs (G-quadruplex and i-motif). In this assay, free PEI–Ag NC probe or DNA sequence alone emits low intensities of RRS, while the formation of PEI–Ag NCs/DNA complexes yields greatly enhanced RRS signals; however, when PEI–Ag NCs react with G-quadruplex or i-motif, the intensities of RRS exhibit slight changes. At the same concentration, the enhancement of RRS signal is directly proportional to the length of telomere, and the sensitivity of 64 bases is the highest with the linear range of 0.3–50 nM (limit of detection 0.12 nM). On the other hand, due to the conversion of telomere DNA molecules among multiple surrounding conditions, a DNA logic gate is developed on the basis of two chemical input signals (K+ and H+) and a change in RRS intensity as the output signal.ConclusionOur results indicate that PEI–Ag NCs can serve as a novel RRS probe to identify DNA length and monitor G-quadruplex/i-motif through the different increasing degrees of RRS intensity. Meanwhile, the novel attributes of the nanoprobe stand superior to those involving dyes or labeled DNA because of no chemical modification, low cost, green, and high efficiency.Electronic supplementary materialThe online version of this article (10.1186/s12951-018-0407-5) contains supplementary material, which is available to authorized users.