Second, you are complicating your collections a bit, in my opinion. As stated in the other comments, you cannot disj a key that is not in there; your map function is returning the values associated with the keys :card1 etc, so trying to disj the key on the results will do nothing.
Now, the fact you are turning this into a set only matters if you expect the values in your original map to possibly be duplicated in that map. Is it possible to have more than one Wisp card with the same damage, etc? If it is possible that :card5 and :card8, for example, could be identical values, then turning the map into a set will remove those duplicates. If it is not possible that cards would be identical, then the map already has unique keys that cannot be duplicated and so I'm not sure what you are gaining by transforming it into a set.