A controversial proposal to transform a former Dudley farm into a Muslim cemetery is about to move from small committee hearings to a townwide vote.

Dudley selectmen decided this week to bring the matter to Town Meeting on May 23, when residents will be asked whether the town should be allowed to buy the land and preempt the plans of an Islamic group.

The 56-acre parcel has space for 16,000 graves off a narrow, winding road.

“Whenever we have a major issue or a controversial issue, we always let the people voice their opinion,’’ Selectman Paul Joseph said Thursday.

Town Meeting approval would not direct the town to purchase the land, but instead would give Dudley the authority to buy it. Until this point, the proposal has been discussed by the zoning and health boards.

“Honest to gosh, I don’t know if we will or not’’ buy the property, Joseph said. “It just gives us an option.’’

The cemetery plans by the Islamic Society of Greater Worcester have generated a firestorm. Although most residents said they are concerned about traffic and possible contamination of ground water, some supporters of the project have attributed part of the backlash to anti-Muslim bias.

The Islamic Society — which currently uses the Muslim graveyard in Enfield, Conn., 60 miles from Worcester — already has signed an agreement to buy the property for about $285,000. But Joseph said the town has the right of first refusal because the property falls under a state program that provides property-tax subsidies for agricultural land.

Jay Talerman, the attorney for the Islamic Society, disputed that argument. The group’s standing as a nonprofit charitable organization precludes the town’s right of first refusal, Talerman said.

Apart from the legalese, Talerman questioned why the town would increase its financial burden to block a cemetery.

“They’re not seeking to tax people for better schools, or more police and firefighters, or better roads,’’ he said. “They’re seeking to tax people for the sole purpose of stopping a Muslim cemetery.’’

Joseph lashed out at Talerman in an interview Thursday, accusing the lawyer of demonizing the town of 11,000 south of Worcester.

“He went on this one-man campaign to paint the town inappropriately and deceitfully as a racist and bigoted town. He went too far,’’ Joseph said.

The selectman said the issue has generated more discussion than any other during his 12 years on the board.

“I’ve never had as many people talk to me directly about an issue,’’ Joseph said. “But in only one case do I believe that the person was opposed to it because it’s a Muslim cemetery.’’

However, he said that Islamic Society officials were not transparent about their true intentions for the property, where they have predicted they would bury only about 15 bodies a year.

If that were the case, Joseph said, “they probably only need between three and five acres. They wanted to turn it into a regional cemetery and never said they wanted to turn it into a regional cemetery.’’

Brian MacQuarrie can be reached at brian.macquarrie @globe.com.