Context. Abundances of galaxies at redshifts z > 4 are difficult to obtain from damped Ly α (DLA) systems in the sightlines of quasars (QSOs) due to the Ly α forest blanketing and the low number of high-redshift quasars detected. Gamma-ray bursts (GRBs) with their higher luminosity are well suited to study galaxies out to the formation of the first stars at z > 10. Its large wavelength coverage makes the X-shooter spectrograph an excellent tool to study the interstellar medium (ISM) of high redshift galaxies, in particular if the redshift is not known beforehand. Aims. We want to determine the properties of a GRB host at z = 4.66723 from absorption lines. This is one of the highest redshifts where a detailed analysis with medium-resolution data is possible. Furthermore, we determine the dust extinction using the spectral energy distribution from X-rays to near infrared. Finally, we put the results in context to other GRBs at z > 4 and properties of (high redshift) QSO absorbers. Methods. The velocity components of the resonant and fine-structure absorption lines are fitted with Voigt-profiles and the metallicity determined from S, Si, Fe and O. Si II* together with the energy released in the UV restframe as determined from GROND photometric data gives us the distance of the absorbing material from the GRB. The extinction is determined from the spectral slope using X-ray spectral information and the flux calibrated X-shooter spectrum, cross calibrated with photometric data from GROND. We also collect information on all GRB hosts with z > 4. Results. We measure a relatively high metallicity of [M/H] = −1.0 ± 0.1 from S, the distance of the material showing fine-structure lines is 0.3 to 1.0 kpc. The extinction is moderate with A V = 0.24 ± 0.06 mag. Low-and high ionization as well as fine-structure lines show a complicated kinematic structure probably pointing to a merger in progress. We detect one intervening system at z = 2.18. Conclusions. GRB-DLAs have a shallower evolution of metallicity with redshift than QSO absorbers and no evolution in their HI column density or ionization fraction. GRB hosts at high redshift seem to continue the trend of the metallicity-luminosity relation towards lower metallicities but the sample is still too small to draw a definite conclusion. While the detection of GRBs at z > 4 with current satellites is still difficult, they are very important for our understanding of the early epochs of star-and galaxy-formation in the Universe.