is there a way to use not equal in facet filter?

Yes, e.g

age:!= 24

Numerical not equals only support in 0.25 but 0.24 supports string not equals.