Ahead of the December 5 Rampur Sadar assembly bypoll, Deputy Chief Minister Brijesh Pathak on Saturday said while the Narendra Modi government is working to empower Pasmanda Muslims through its schemes, other political parties have used the community just for votes.