The task of turning agricultural waste into practical construction and building materials has been placed before civil engineers. Coffee husk is produced in vast amounts due to the global commerce of coffee beans, which are incinerated into ash when used as fuel, producing coffee husk ash (CHA). Even though many researchers have worked on the utilization of CHA in concrete, they have been used as partial cement replacement but not as a replacement of aggregates. The experimental study of the performance of concrete on fine aggregate replaced partially with CHA is represented in this paper. The fine aggregate is replaced by 0%, 2%, 4%, 6%, and 8% by weight of CHA. The performance of the partially replaced fine aggregate with CHA is reviewed by considering the compressive strength and workability of fresh concrete and the splitting tensile strength, flexural strength, durability under acid and alkaline media, thermal conductivity, and rapid chloride permeability test of hardened concrete. The results indicate that the partial replacement of fine aggregate with 4% of CHA (CHA04) in concrete provides a positive impact to all the selected performance parameters. The compressive strength, flexural strength, and splitting tensile of the CHA04 mix were 43.4 MPa, 3.7 MPa, and 2.44 MPa, respectively, which were 28.4%, 19.35%, and 1.66%, respectively, greater than normal concrete mix (CHA00). Even the study of acid and alkaline attack on the CHA04 mix showed lesser strength reduction as compared to other mixes. The RCPT showed less chloride permeability, and the thermal conductivity is higher for CHA04, indicating lesser voids compared to other mixes. With the help of this investigation, it can be said that fine aggregate replacement with 4% CHA has the best strength and durability properties compared to regular concrete.