Aggregation of amyloid-beta (Aβ) peptide is the major event underlying neuronal damage in Alzheimer's disease (AD). Specific lipids and their homeostasis play important roles in this and other neurodegenerative disorders. The complex interplay between the lipids and the generation, clearance or deposition of Aβ has been intensively investigated and is reviewed in this chapter. Membrane lipids can have an important influence on the biogenesis of Aβ from its precursor protein. In particular, increased cholesterol in the plasma membrane augments Aβ generation and shows a strong positive correlation with AD progression. Furthermore, apolipoprotein E, which transports cholesterol in the cerebrospinal fluid and is known to interact with Aβ or compete with it for the lipoprotein receptor binding, significantly influences Aβ clearance in an isoform-specific manner and is the major genetic risk factor for AD. Aβ is an amphiphilic peptide that interacts with various lipids, proteins and their assemblies, which can lead to variation in Aβ aggregation in vitro and in vivo. Upon interaction with the lipid raft components, such as cholesterol, gangliosides and phospholipids, Aβ can aggregate on the cell membrane and thereby disrupt it, perhaps by forming channel-like pores. This leads to perturbed cellular calcium homeostasis, suggesting that Aβ-lipid interactions at the cell membrane probably trigger the neurotoxic cascade in AD. Here, we overview the roles of specific lipids, lipid assemblies and apolipoprotein E in Aβ processing, clearance and aggregation, and discuss the contribution of these factors to the neurotoxicity in AD.