Hello Christina,
That is so obvious because when you select All Values it is trying to display all the city names in one cell only resulting in #MULTIVALUE to avoid that you can replace the formula in the cell by as shown below.
=If IsError([Cityfilter]) Then "All Values" Else [Cityfilter]
Where cityfilter is the thing you are showing in that cell.
Regards
Niraj