07-31-2011, 10:46 AM
In short, Spamalyser tries to detect spam posts (not spam registrations) through a number of means and can perform some actions on detected spam.
Note that this plugin isn't exactly "complete", but should be "good enough" to use.
How is this different to other anti-spam plugins?
There are a number of other MyBB plugins which try to prevent spam. However, I find that most of these can get rather pedantic and can easily block legitimate users with little, if any, means of them working around the block.
The aim of this plugin was to be somewhat more permissive than restrictive, as well as not have to rely on external services (although this plugin is capable of utilising information pulled from them). This tries to deny as little as possible to legit users - for example, a solution such as blocking link posting for new users means that all users need to have a number of posts before they are able to post links, whereas Spamalyser allows this activity as long as it doesn't look spammy.
Stop Forum Spam and Fassim plugins, for example, seem to only perform blocks on user registration (before they even register, in fact). As Spamalyser is more permissive, it only performs checks on posting (this also has the slight benefit of being able to work if guest posting is enabled, although Spamalyser isn't as effective with guests than it is with registered users). Also, performing spam analysis on posts means that Spamalyser is able to use a greater number of inputs to judge whether a post is spam or not.
Of course, the downside is that Spamalyser will not try to do anything about spam registrations.
Can this be used with other anti-spam plugins?
Most of them yes. I'm not sure whether this works with the Akismet plugin or not (although it's somewhat pointless to do so, because Spamalyser supports Akismet lookups).
If you use it with the Stop Forum Spam plugin, you may wish to disable SFS lookups in Spamalyser.
How does this plugin work?
When a user tries to post a new thread or reply, or edit a post, the plugin will check whether the user passes a number of thresholds (such as post count) and if so, deems the user "safe" and stops there.
If the user fails to pass the threshold test, the main Spamalyser engine kicks in and analyses the post to determine a "spam weighting" (likeliness of it being spam). It then compares this weighting against some configurable action thresholds to decide on whether it should do anything to the post. If the weighting meets the thresholds, Spamalyser can currently (depending on what you enable):
How is the "spam weighting" calculated?
Quite a number of means, but most of the code is link analysis. Spammers ultimately want to post links, so it seems like a good place to start. Every link posted will add to the weighting, and links with similar keywords or to the same domain get penalised more heavily.
Spamalyser can also make some judgements based on the poster's online time and other factors, and has a number of features to attempt to reduce the number of false positives (for example, by examining the user's previous posts).
External lookups to services such as Stop Forum Spam and Akismet are also supported, and you can specify the amount of weighting to give to these services.
You can try looking in the Spamalyser settings which show all the methods used by this plugin to detect spam.
Configuring Spamalyser
Spamalyser has quite a number of options, designed to give the administrator a fair amount of control over the spam analysis process.
The defaults are relatively permissive. From my testing, you can probably reduce the Unapprove Threshold from 10 to 5 and capture a lot more spam.
Note that, by default, external lookups are disabled for localhost installs, but enabled for non-localhost installs.
I know there are a lot of options so it may be difficult to get your head around what they all do. You can try inspecting the log (clicking on the weighting will show how it's calculated) to see how some posts' weights are determined and perhaps get a rough idea of what some of the options do.
Disable Link Analysis
If you wish to, you can completely disable the internal link analysis routine. To do so, set the following three settings to 0:
Akismet Note
To enable Akismet, you must enter in an API key - and you must ensure that this key is valid because Spamalyser won't check it. Also, your server must be able to either make requests via cURL or fsockopen()
Upgrading
I'm not going to bother maintaining upgrade paths for this plugin. This means that if you wish to upgrade to a newer version, you'll have to uninstall the old version, upload the files for the new version, then install it.
Obviously this means that you'll lose any setting changes you've made, as well as log entries, so you may wish to note down any custom settings you've set before uninstalling. Do note that the meaning of some settings may change in the future, so refer to the changelog to ensure your custom settings have the same meaning.
Performance Issues
As this plugin performs external lookups by default, for all analysed posts, posting may be slower for posts which are examined. Although the services queried (SFS, Akismet and Google) should have reasonably fast servers, you can choose to disable these lookups if you wish. [lookups are performed through MyBB's fetch_remote_file() function]
As for internal link analysis, the algorithm isn't exactly fast (still should be significantly faster than MyBB's post parser), but should be acceptable, especially since it's only done during posting.
Limitations
Note that this plugin isn't exactly "complete", but should be "good enough" to use.
How is this different to other anti-spam plugins?
There are a number of other MyBB plugins which try to prevent spam. However, I find that most of these can get rather pedantic and can easily block legitimate users with little, if any, means of them working around the block.
The aim of this plugin was to be somewhat more permissive than restrictive, as well as not have to rely on external services (although this plugin is capable of utilising information pulled from them). This tries to deny as little as possible to legit users - for example, a solution such as blocking link posting for new users means that all users need to have a number of posts before they are able to post links, whereas Spamalyser allows this activity as long as it doesn't look spammy.
Stop Forum Spam and Fassim plugins, for example, seem to only perform blocks on user registration (before they even register, in fact). As Spamalyser is more permissive, it only performs checks on posting (this also has the slight benefit of being able to work if guest posting is enabled, although Spamalyser isn't as effective with guests than it is with registered users). Also, performing spam analysis on posts means that Spamalyser is able to use a greater number of inputs to judge whether a post is spam or not.
Of course, the downside is that Spamalyser will not try to do anything about spam registrations.
Can this be used with other anti-spam plugins?
Most of them yes. I'm not sure whether this works with the Akismet plugin or not (although it's somewhat pointless to do so, because Spamalyser supports Akismet lookups).
If you use it with the Stop Forum Spam plugin, you may wish to disable SFS lookups in Spamalyser.
How does this plugin work?
When a user tries to post a new thread or reply, or edit a post, the plugin will check whether the user passes a number of thresholds (such as post count) and if so, deems the user "safe" and stops there.
If the user fails to pass the threshold test, the main Spamalyser engine kicks in and analyses the post to determine a "spam weighting" (likeliness of it being spam). It then compares this weighting against some configurable action thresholds to decide on whether it should do anything to the post. If the weighting meets the thresholds, Spamalyser can currently (depending on what you enable):
- Report the post,
- Unapprove the post, or
- Block the post (displays an error when user tries to submit the post)
How is the "spam weighting" calculated?
Quite a number of means, but most of the code is link analysis. Spammers ultimately want to post links, so it seems like a good place to start. Every link posted will add to the weighting, and links with similar keywords or to the same domain get penalised more heavily.
Spamalyser can also make some judgements based on the poster's online time and other factors, and has a number of features to attempt to reduce the number of false positives (for example, by examining the user's previous posts).
External lookups to services such as Stop Forum Spam and Akismet are also supported, and you can specify the amount of weighting to give to these services.
You can try looking in the Spamalyser settings which show all the methods used by this plugin to detect spam.
Configuring Spamalyser
Spamalyser has quite a number of options, designed to give the administrator a fair amount of control over the spam analysis process.
The defaults are relatively permissive. From my testing, you can probably reduce the Unapprove Threshold from 10 to 5 and capture a lot more spam.
Note that, by default, external lookups are disabled for localhost installs, but enabled for non-localhost installs.
I know there are a lot of options so it may be difficult to get your head around what they all do. You can try inspecting the log (clicking on the weighting will show how it's calculated) to see how some posts' weights are determined and perhaps get a rough idea of what some of the options do.
Disable Link Analysis
If you wish to, you can completely disable the internal link analysis routine. To do so, set the following three settings to 0:
- Weight Per Simple Link
- Weight Per Complex Link
- Duplicate Keyword Bias
Akismet Note
To enable Akismet, you must enter in an API key - and you must ensure that this key is valid because Spamalyser won't check it. Also, your server must be able to either make requests via cURL or fsockopen()
Upgrading
I'm not going to bother maintaining upgrade paths for this plugin. This means that if you wish to upgrade to a newer version, you'll have to uninstall the old version, upload the files for the new version, then install it.
Obviously this means that you'll lose any setting changes you've made, as well as log entries, so you may wish to note down any custom settings you've set before uninstalling. Do note that the meaning of some settings may change in the future, so refer to the changelog to ensure your custom settings have the same meaning.
Performance Issues
As this plugin performs external lookups by default, for all analysed posts, posting may be slower for posts which are examined. Although the services queried (SFS, Akismet and Google) should have reasonably fast servers, you can choose to disable these lookups if you wish. [lookups are performed through MyBB's fetch_remote_file() function]
As for internal link analysis, the algorithm isn't exactly fast (still should be significantly faster than MyBB's post parser), but should be acceptable, especially since it's only done during posting.
Limitations
- The plugin assumes that MyCode is always enabled (so only detects [url] type links), which is probably in-line with the assumptions made by most spammers
- Google searching behaviour is somewhat erratic - it's a bit difficult to generate a representative search query from a post. Can be tweaked, though I'm not too worried because spammers can theoretically bypass this by slightly varying their posts across forums
- Reports currently only go to the database regardless of what reporting medium you have selected
