Block Spam Based on Keyword(s)


BBhumeBB

 

Posted

I can't take it anymore. Every day I log in, I keep finding emails for [censored] and its driving me insane. I hit the spam button 9 million times and still it comes back - every day, more and more.

Please, somebody, take the time to update the in-game email system so it scans/spams email on the basis of keywords. This would help *so much* its not even funny.

Thanks for listening.


 

Posted

That particular website's spammers have been mispelling thier site's name (by one letter) for a little while now, probably because they are expecting a measure like this one.
But the mispelling is close enough, as you have been spammed by it and still recognize it for the real name instead.
A filter based on keywords could be wiggled around by spaces inside the words, splitting them, or as they've been doing, by mispelling.

I feel for ya, buddy. But I feel that the oft-repeated suggestion of a filter on the recieving side for Emails would be a better course of action. (Filtering out Emails based on thier relationship to the reciever: Stranger, Trials, SG&Friends, etc.)

And/or increasing the required level to send Email from L10 to L15, so that disposable FREE trial accounts (cap at L14!) couldn't get PL'ed in minutes to send RMTmail, then dumped, then repeated ad infinitum.


 

Posted

This doesn't help. They just resort to breaking it up, and avoiding the keywords altogether while still spamming you.

All it really accomplishes is wasting developer time, which frankly irritates me a bit, because I'm sort of annoyed with it myself. :/ There's really not a lot that can be done about it that's not going to have a negative impact on the players as well (see Trial Account nerfing).


 

Posted

Someone should make a boilerplate response for this.

I think eliminating the ability of trial accounts to send mail is probably reasonable. It would have far less of an impact on legitimate ones than eliminating their ability to send tells did.


 

Posted

Agreed. I was also thinking about just auto-deleting any emails with the common spammers.

Do trial accounts really need to be able to send mail? Hell, in all the time I've been playing the game, I think I've sent one in game email, and received maybe three that had to do with sg stuff. Every single one past that has been spam.

I'm sure some people use it more than most of us, but this spam has been slowly moving from annoying to making me just no longer want to log in and spam all of the 10-20 emails I get after one-two days of being logged out.

On another note, why don't the devs just purchase some inf, track the ip's and accounts backwards through the trades and delete all of their mules, etc. Hard to sell stuff you no longer have.

--Umbra.


 

Posted

[ QUOTE ]

Do trial accounts really need to be able to send mail?

[/ QUOTE ]
For the most part, no.
In the case of giving your pal getting a trial account and telling him "Email my CharacterX your info so we can hook up", yes.

But in that case, I have a solution.
Allow all characters at L1 to send Emails, but have a filter where recievers do not recieve Emails from Trial Accounts unless they turn on a setting (default is off) and have the option to recieve (or not) from strangers (not friends or SG) on paid account by choice of setting.
Guy with friend can turn on "Recieve from trials", get the buddy's character name and they can play together, /friend the buddy, then turn filter back on. Everyone else has spam mail never hitting thier screen.

We used to only have /hide and /unhide. They managed to granulate that and the results are appreciated.


 

Posted

[ QUOTE ]
But in that case, I have a solution.

[/ QUOTE ]

I too have a solution for communicating with friends!

AIM, Yahoo, MSN Messenger, ICQ, Xfire, Gmail Chat...


 

Posted

[ QUOTE ]
Someone should make a boilerplate response for this..

[/ QUOTE ]

Someone did. *cough*


[ QUOTE ]

Common topic : Email changes in response to RMT Spam

There are quite a few ideas floated typically for dealing with RMT spam. Trial account limitations were one thing tried by the devs, and worked reasonably well until lately (early 2009.) This is a common topic in the suggestion forum.

Since it is, a tip on searching.
1. Click on "Search" up at the top of the forum.
2. On the left, under "Forum(s) to search," select "Suggestions and ideas."
3. On the right, Keyword Search Terms. This is probably where your problem was if you did search. Try the following, exactly as typed:

+mail -"re: "

This will search for anything with "mail" in the title. Why not "email?" Because not everybody spells it the same. Some will call it email, others "e-mail" or "e mail," and still others may just call it the "mail system" or "Filters on Mail." This gives you a better chance of catching all of those. The -re: portion of it removes replies, so you'll see the root of every thread that comes up, letting you see just how many threads there are on this. (The last helps for other subjects, as well.)

4. Click the "In subject and body" radio button. Normally I say just in the subject, but this comes up in a few different ways, so for this search - both.

5. Leave Username Search blank.

6. Date range, Newer Than, change the 1 to a 3, and the time to Months.

7. Result format doesn't matter. Click on "Submit."

What do we see with this result? There are 34 returned threads as I type this (3/23/09.) My first page? Here's part of it.

Give me an option to block emails, Im sick of spam
Change email to mail system
We need another ignore list
Lock out e-mail until 15
Be able to disable or restrict e-mail
Maybe the worst idea ever, or maybe not.
Tired of RMT Emails?
Shared storage slots
Email link not turning red
In-game E-mail spamming

Going to just one month would have cut it down to just 13 results, in this case. Three months, though, is a decent search range. You can see already the reason for putting in "Mail" instead of "Email." And those without it in the title have it in the body somewhere.

Personal Opinion

I am, as always, strongly in favor of client-side filtering. Something like:
Accept email from:
o Friends
o Global Friends
0 Supergroup
o Teammates
0 Coalition
0 Any

with "Any," honestly, being the default - I use the system and help people I run across in PVP zones, for instance.

Why not keyword filtering (blacklisting?) Well, say I'm spamming from a (as far as I know) made up site called infspammer .com. I can tell you right now, just from *regular* email experience, that the second you block the entire name, it's going to get changed. www. infspammer. com is not the same. Block infspammer? 1nfspammer. i n f s pa a m m e r. in fs pa mm er. 1nfsp@mmer. 1nfsp@/|/| /|/|er. And if you finally manage to blow all that time and money setting up and testing the blocks for all the permutations of that - I spend five bucks and get buymyinf.com, and you get to start all over again. A blacklist is just more work for less gain. After all, years after the huge push from overseas, I still get the occasional creative spelling of "viagra" showing up in my own (whitelisted) junk mail filter.

That said, I'm also strongly leaning towards totally disabling email on trial accounts by this point. Yes, it's another trial limitation... but the RMTers are getting around it by PL'ing up characters already. As much as I *dislike* the idea, I can't help but think that it'd be needed at this point.



[/ QUOTE ]


 

Posted

[ QUOTE ]
A filter based on keywords could be wiggled around by spaces inside the words, splitting them, or as they've been doing, by mispelling.

[/ QUOTE ]

Not if that filter is Bayesian. A Bayesian filter looks at words, words in proximity to other words, words in relation to other words, etc. It looks at the "language" of spam, not just the individual letters that compose it, and it uses that "language", the composition of advertisements designed to catch the eye and convince the reader to buy, to catch and remove the spam. It's highly effective when it's properly implemented, generally at or above 99% effectiveness, and with false positive counts below 1%.


 

Posted

[ QUOTE ]
[ QUOTE ]
A filter based on keywords could be wiggled around by spaces inside the words, splitting them, or as they've been doing, by mispelling.

[/ QUOTE ]

Not if that filter is Bayesian. A Bayesian filter looks at words, words in proximity to other words, words in relation to other words, etc. It looks at the "language" of spam, not just the individual letters that compose it, and it uses that "language", the composition of advertisements designed to catch the eye and convince the reader to buy, to catch and remove the spam. It's highly effective when it's properly implemented, generally at or above 99% effectiveness, and with false positive counts below 1%.

[/ QUOTE ]
That sounds rather clever.


 

Posted

[ QUOTE ]
[ QUOTE ]
[ QUOTE ]
A filter based on keywords could be wiggled around by spaces inside the words, splitting them, or as they've been doing, by mispelling.

[/ QUOTE ]

Not if that filter is Bayesian. A Bayesian filter looks at words, words in proximity to other words, words in relation to other words, etc. It looks at the "language" of spam, not just the individual letters that compose it, and it uses that "language", the composition of advertisements designed to catch the eye and convince the reader to buy, to catch and remove the spam. It's highly effective when it's properly implemented, generally at or above 99% effectiveness, and with false positive counts below 1%.

[/ QUOTE ]
That sounds rather clever.

[/ QUOTE ]

The question is, do we want to pay the cost (processing wise, not to mention trying to fit that into the email system) of that while simultaneously doing all an MMO does?

I'm still more in favor of clientside filters in a situation like this.


 

Posted

Email doesn't need to be efficient in CoH dump email processing for all servers into one system and have it process email filtering separate from the game. If it take 15 to 30 minutes for an email to pass hands so be it.


 

Posted

[ QUOTE ]
[ QUOTE ]
A filter based on keywords could be wiggled around by spaces inside the words, splitting them, or as they've been doing, by mispelling.

[/ QUOTE ]

Not if that filter is Bayesian. A Bayesian filter looks at words, words in proximity to other words, words in relation to other words, etc. It looks at the "language" of spam, not just the individual letters that compose it, and it uses that "language", the composition of advertisements designed to catch the eye and convince the reader to buy, to catch and remove the spam. It's highly effective when it's properly implemented, generally at or above 99% effectiveness, and with false positive counts below 1%.

[/ QUOTE ]

Welcome to about two iterations of the spammer/antispam software wars ago.


My characters - all on Virtue.
Gabe's Internet [censored] Theory
RMT spammers WILL steal your credit card.

 

Posted

[ QUOTE ]
[ QUOTE ]
[ QUOTE ]
A filter based on keywords could be wiggled around by spaces inside the words, splitting them, or as they've been doing, by mispelling.

[/ QUOTE ]

Not if that filter is Bayesian. A Bayesian filter looks at words, words in proximity to other words, words in relation to other words, etc. It looks at the "language" of spam, not just the individual letters that compose it, and it uses that "language", the composition of advertisements designed to catch the eye and convince the reader to buy, to catch and remove the spam. It's highly effective when it's properly implemented, generally at or above 99% effectiveness, and with false positive counts below 1%.

[/ QUOTE ]

Welcome to about two iterations of the spammer/antispam software wars ago.

[/ QUOTE ]

I know. I've been following Bayesian filtering for years, since this article was first posted. As soon as the spammers figure out it's in place, they start adding random garbage phrases and words to their e-mails in an attempt to poison the list, in hopes of either generating enough false positives to force the company to remove the filter, or to skew the weighting on the negatives enough to get more through.

But it's still better than no filter at all, which is what we (appear to) have now, and Bayesian is still a very, very good algorithm when it's implemented well (not naive).


 

Posted

[ QUOTE ]
[ QUOTE ]
[ QUOTE ]
[ QUOTE ]
A filter based on keywords could be wiggled around by spaces inside the words, splitting them, or as they've been doing, by mispelling.

[/ QUOTE ]

Not if that filter is Bayesian. A Bayesian filter looks at words, words in proximity to other words, words in relation to other words, etc. It looks at the "language" of spam, not just the individual letters that compose it, and it uses that "language", the composition of advertisements designed to catch the eye and convince the reader to buy, to catch and remove the spam. It's highly effective when it's properly implemented, generally at or above 99% effectiveness, and with false positive counts below 1%.

[/ QUOTE ]
That sounds rather clever.

[/ QUOTE ]

The question is, do we want to pay the cost (processing wise, not to mention trying to fit that into the email system) of that while simultaneously doing all an MMO does?

[/ QUOTE ]

The filtering should be done on a separate system. It wouldn't impact the game at all. Person composes e-mail, clicks Send. e-mail is routed to a separate server where it's run through the filter, then flagged as safe or spam. Safe gets routed back to the game servers, spam gets held for manual review (at which point false positives could be forwarded to the game servers).

It's really just a database, a list of words and relative weightings, and a simple algorithm that sifts through e-mail text and checks those words against what's in the database. Not terribly complex, not even something which would require a state of the art server. There wouldn't be any overhead generated on the game servers. At least, if it were implemented correctly.

[ QUOTE ]
I'm still more in favor of clientside filters in a situation like this.

[/ QUOTE ]

Which, on the other hand, would create additional load, and be vulnerable to direct manipulation via malware/viruses/worms/etc.


 

Posted

[ QUOTE ]
Email doesn't need to be efficient in CoH dump email processing for all servers into one system and have it process email filtering separate from the game. If it take 15 to 30 minutes for an email to pass hands so be it.

[/ QUOTE ]

I can't imagine it taking that long unless the server were a 286 with no math coprocessor. With anything reasonably modern, say Athlon XP/P3 onward, it shouldn't take more than a few seconds per e-mail, and that's including the routing time and expectation that it would be running several hundred e-mails through the filter at once.


 

Posted

I feel like this is neglecting one major facet of institutional implementation, which is that it has to be someone's job to design and install the filter, and that they have to be given that job by someone with the authority to do so. Barring unforseen internal emphasis, if this ball got rolling now, the system might be in place in, oh, three to four Issues. And that's being optimistic about estimations involving hardware requisition and setup. This is not going to make it to the top of anyone's priority list.

Speaking of which:
[ QUOTE ]
spam gets held for manual review

[/ QUOTE ]

Not likely to happen.


 

Posted

How many people use the system? I know we always have a few very vocal protestors any time someone suggests that it be removed, but how many people actually would be really inconvenienced if it was?

I would rather have an insignificant minority inconvenienced than the rest of us...


 

Posted

Depends on what you consider "use," I suppose.

I actually do use it with some regularity - when I need more length and permanence than a tell. And I see it in supergroups as well - where the members may not *be* emailing it back and forth, but the leaders will send an email with things like the SG's website, rules, any prerequisites (IE, specialized builds) and the like, or if they have to kick someone for inactivity or whatnot. That, to me, is fairly common.


 

Posted

The nonsense I get advertising v1@gr@ and so on indicates how that goes.

I admit to being baffled by spam advertising that sticks subjects like "Your account will be closed" or "Emergency at home" on a text loaded with nonsense like leet-spelled drug names or semi-coherent descriptions of pr0n0grafik dlites awaiting me, or whatever it might be. Surely the advertisers realize this will infuriate and repulse 1000 people for every 1 it might attract. I guess the infinitesimal costs of internet advertising means a tiny success rate is still protiftable - that's leaving aside the ones intended to get you to click on a trapped website or open a loaded attachment.

It is as if spammage of this variety has taken on a life of its own, independent of any desire to actually SELL anything.


My scrapper doesn't need an AoE. She IS an AoE.

 

Posted

There's only one way to stop this.

Require the entry of real human ID(social security or otherwise) to create accounts.

That way, a malicious registration can ban the human being behind the keyboard from further business with NCsoft instead of an email or some gobbledygook person who does not exist.


 

Posted

There is the potential issue that RMT spammers use stolen identity data to drive their manufacture of accounts. Also whether NCSoft wants to put personal info data (beyond what is already there, including credit card data) on their systems where it is itself subject to identity theft. These days, a service does not necessarily want any more data than they absolutely need from their customers.

Some customers would also refuse to use their SSN this way - I'd think twice myself (as I do whenever any entity but the IRS requests my social).


My scrapper doesn't need an AoE. She IS an AoE.

 

Posted

I say the best way to keep spam out is to use what was earlier said. Give us the choice as far as who we receive email from. That way no stranger could email us. Besides why would a stranger need to email us anyway. Why couldn't they just talk to us when we are in the game with tells if they can send tells.


Ebony Fists: Level 50 DM/Regen Scrapper, Gloom Piston Robotics/Dark mastermind level 34, QueenFireMare: Level 34 Fire blaster (pure fire),

 

Posted

[ QUOTE ]
I say the best way to keep spam out is to use what was earlier said. Give us the choice as far as who we receive email from. That way no stranger could email us. Besides why would a stranger need to email us anyway. Why couldn't they just talk to us when we are in the game with tells if they can send tells.

[/ QUOTE ]
I haven't read the whole thread but yes, this is a common suggestion. Just make it so that by default you can only receive email from friends, global friends, and SG/coalition members.

The problem is this: the spammers will just switch to sending /tells, because they can be sent to someone who is offline. And since almost all of us want to be able to receive tells for team invites, I don't see an easy solution to that problem.


Freedom: Blazing Larb, Fiery Fulcrum, Sardan Reborn, Arctic-Frenzy, Wasabi Sam, Mr Smashtastic.