Answer: The Shulchan Aruch (OC 147:1) writes that one mustn’t touch a sefer Torah without a cloth. The Rema writes that while some extend this prohibition to other kisvei kodesh, holy writings, that isn’t our minhag. Nonetheless, he cautions that one should ideally wash one’s hands before handling them (See Beis Yosef OC 147:1).
The Magen Avraham (147:1) writes that Rema was referring to printed sefarim. The Rema would agree, however, that one must wash one’s hands before touching a kosher megilla.
The Elya Rabba (147:1) and Panim Meiros (1:76) however, disagree, writing that when the Rema writes that it is ideal to wash one’s hands, he was referring to writings such as the megilla which don’t contain the same kedusha as a sefer Torah. One only needs to wash one’s hands if they are dirty (See Aruch Hashulchan OC 147:5; Kaf Hachaim OC 147:5; 7).
The Shaarei Teshuva (691:4) quotes R’ Yaakov Emden (Mor Uketzia OC 691; Sheilas Yaavetz 2:103) who forbids touching a megilla, though notes that people are not particular to wash their hands first. He concludes, however, that it is befitting to do so. Similarly, the Chasam Sofer (Hagahos OC 691) records that his teacher, R’ Nosson Adler was particular to wash his hands before holding the megilla. Nonetheless, he refers to Radvaz (2:771) who notes that people are not particular about this.
In conclusion, it is ideal for one to wash one’s hands before touching the megilla.