Yes, the Polish letter ą is different that a "regular" a. It is pronounced with a nasal sound, very similar to the French word "un." To an English speaker, Wąpielsk may sound more like Wompielsk (since the ą is followed by a p, it gets more of a nasal M sound).
It looks like there has only been once place in Poland (historically and presently) named Wąpielsk. It is the one you mention, which is some distance from the Podlasie province (the topic of this board). In the late 1800s, the Roman Catholic parish that served Wąpielsk was in the village of Radziki Wielkie. Looks like the village may have been renamed Radziki Duże (or was absorbed into Radziki Duże), and it still exists today. Here's a link to a webpage about the church:http://www.diecezjaplocka.pl/index.php?akcja=spis_wyswietlPa...
They have sacramental records from 1910 forward, so if you're interested in writing to Poland for records for your ancestors, you'd have to send to the Płock diocese for them:http://www.diecezjaplocka.pl/
Some dioceses will accept request via email and they will inform you of their costs and how to pay them for research.
The LDS church has microfilmed records from the parish of Radziki Duże:http://www.familysearch.org/eng/library/fhlcatalog/supermain...
It would be much less expensive to order the films from your local Family History Center and look at them yourself. Although the records are in Polish, Russian, and Latin, there are lots of translation aids available both online and in publications that can help you decipher the records yourself.