Depends on what you want to take pictures of - for portraits and for street-photography, filters are not absolutely necessary, for landscape and architecture they are very important.
I personally use a yellow-green about 90 % of the time - it darkens the sky & gives better clouf separation like the more often recommended 'normal' yellow, without bleaching lips and other red objects as much, and it also differentiates greens (like foliage) better.
The only othe filter I occasionally use is the orange - when I deliberately want to have a dramatic look, e.g. when shooting (modern) architecture, or when shooting far away things in hazy light.
The red filter is to melodramatic for my tast (but tastes differ), and I never found a use for green or blue.
Roman