We introduce the alternating Schur algebra AS F (n, d) as the commutant of the action of the alternating group A d on the d-fold tensor power of an n-dimensional F -vector space. When F has characteristic different from 2, we give a basis of AS F (n, d) in terms of bipartite graphs, and a graphical interpretation of the structure constants. We introduce the abstract Koszul duality functor on modules for the even part of any Z/2Z-graded algebra. The algebra AS F (n, d) is Z/2Z-graded, having the classical Schur algebra S F (n, d) as its even part. This leads to an approach to Koszul duality for S F (n, d)-modules that is amenable to combinatorial methods. We characterize the category of AS F (n, d)-modules in terms of S F (n, d)-modules and their Koszul duals. We use the graphical basis of AS F (n, d) to study the dependence of the behavior of derived Koszul duality on n and d.