Imagine trying to file a tax return only to be rejected and told it has already been filed.

Maybe you had a lapse of memory but most likely you have become the latest victim of tax fraud.

The Internal Revenue Service and the Justice Department have been cracking down on suspected identity theft perpetrators to curb the growing issue of refund fraud and identity theft.

IRS Spokesman Mark Green said the issue is a growing problem throughout the country, but the Gainesville area is a particular problem.

"Identity theft, unfortunately, always has been around and unfortunately (the IRS is) another component of identity theft," he said. "Scam artists have ventured into the area of stealing people's Social Security numbers in order to prepare returns for their own personal gain."

The nationwide sweep targeted 105 people in 23 states leading to several arrests and indictments.

In all, there were 939 criminal charges filed and 69 indictments.

"IRS auditors and investigators conducted extensive compliance visits to money service businesses in nine locations across the country, including Georgia," said Green.

The effort was to ensure check-cashing businesses were not facilitating refund fraud and to deter others from considering the scheme.

"This unprecedented effort against identity theft sends a strong, unmistakable message to anyone considering participating in a refund fraud scheme this tax season," said IRS Commissioner Doug Shulman in a news release. "We are aggressively pursuing cases across the nation with the Justice Department and people will be going to jail."

Social Security numbers can be stolen through a variety of means including schools, banks, hospitals or some other affiliation that requires the use of personal information.

But sometimes Social Security numbers can be bought off the street.

Gainesville attorney David McLeod said many times the issue involves illegal immigrants using a fake Social Security number.

"They will oftentimes have multiple Social Security numbers, which also can result in tax fraud as well," McLeod said.

Gainesville attorney Arturo Corso, who represents much of the area's Hispanic population in legal issues, said many illegal aliens will purchase a Social Security number so they can obtain work.

But in some cases the number they purchased belongs to a living person.

"It causes that person, often in another state, to have a tax liability," Corso said. "That does occur, but the person that's using the number here in Hall County is using a number to work and they're having their taxes withheld but maybe they claimed exemptions from withholding so not all of the taxes that should be withheld are."

Illegal immigrants often don't have a choice but to obtain a Social Security number illegally or they may not find work.

"If they're not here legally they can't work except by illegal means," McLeod said. "I don't mean to say that I condone it because it's a violation of the law but it's a necessity to do it."

That could lead to the IRS believing the person who owns the Social Security number should owe more than they filed.

The issue doesn't stop at filing illegal tax returns, though. A large issue also involves illegal immigrants receiving undocumented pay, McLeod said.

"I think for a lot of that population there's a whole underground economy where lots of things are done with cash, so to some extent that's tax fraud if you're not reporting it," he said.

McLeod said recent legislation targeting illegal immigrants has caused government officials to address the issue more closely.

"It's a confusing situation politically," he said.

But the issue doesn't only involve illegal immigrants. Other groups are also participating in large-scale scams.

Last week, more than 60 in Elberton fell victim to a scam involving an email offering $1,000 in federal stimulus money. The victims provided their personal information and when they filed their 2011 tax returns they never received the money from the IRS because the Social Security numbers had already been used.

"There are always people that are scamming, without question," McLeod said. "I just think primarily I have seen it with illegal immigrants."