An appealing challenge in Neuroscience is to identify network architecture from neural activity. A key requirement is the knowledge of statistical input-output relation of single neurons in vivo. Using a recent exact solution of spike-timing for leaky integrate-and-fire neurons under noisy inputs balanced near threshold, we construct a unified framework that links synaptic inputs, spiking nonlinearity, and network architecture, with statistics of population activity. The framework predicts structured higher-order interactions of neurons receiving common inputs under different architectures: It unveils two network motifs behind sparse activity reported in visual neurons. Comparing model's prediction with monkey's V1 neurons, we found excitatory inputs to pairs explain the sparse activity characterized by negative triple-wise interactions, ruling out shared inhibition. While the model predicts variation in the structured activity according to local circuitries, we show strong negative interactions are in general a signature of excitatory inputs to neuron pairs, whenever background activity is sparse.1 Keywords network architecture, sparse activity, common inputs, higher-order interactions, leaky integrateand-fire neuron model, threshold regime, input-output relation, triple-wise interactions, spontaneous activity