We consider a truss as a graph consisting of nodes and edges, and combine graph embedding (GE) and reinforcement learning (RL) to develop an agent for generating a stable assembly path for a truss with arbitrary configuration. GE is a method of embedding the features of a graph into a vector space. By using GE, the agent can obtain numerical information on neighboring members and nodes considering their connectivity. Since the stability of a structure is strongly affected by the relative positions of members and nodes, feature extraction by GE should be effective in considering the stability of a truss. The proposed method not only can train agents using trusses with arbitrary connectivity but also can apply trained agents to trusses with arbitrary connectivity, ensuring the versatility of the trained agents' applicability. In the numerical examples, the trained agents are verified to find rational assembly sequences for various trusses more than 1000 times faster than metaheuristic approaches. The trained agent is further implemented as a user-friendly component compatible with 3D modeling software.