Given an oriented surface bounding a handlebody, we study the subgroup of its mapping class group defined as the intersection of the handlebody group and the second term of the Johnson filtration: A ∩ J 2 . We introduce two trace-like operators, inspired by Morita's trace, and show that their kernels coincide with the images by the second Johnson homomorphism τ 2 of J 2 and A ∩ J 2 , respectively. In particular, we answer by the negative to a question asked by Levine about an algebraic description of τ 2 (A ∩ J 2 ). By the same techniques, and for a Heegaard surface in S 3 , we also compute the image by τ 2 of the intersection of the Goeritz group G with J 2 .