Something suffers diminishing returns when, after a certain point, having more of it becomes pointless or detrimental.
Optimizing everything to perfection is almost impossible. After picking the “low hanging fruit”, further optimization can cost more than the returns you’ll gain.
Optimize until reaching the point of Diminishing Returns, then focus on something else.
See also: [Perfectionism]