Spamalyser

ZiNgA BuRgA · (This post was last modified: 08-18-2011 08:28 PM by ZiNgA BuRgA.)

In short, Spamalyser tries to detect spam posts (not spam registrations) through a number of means and can perform some actions on detected spam.
Note that this plugin isn't exactly "complete", but should be "good enough" to use.

How is this different to other anti-spam plugins?
There are a number of other MyBB plugins which try to prevent spam.  However, I find that most of these can get rather pedantic and can easily block legitimate users with little, if any, means of them working around the block.
The aim of this plugin was to be somewhat more permissive than restrictive, as well as not have to rely on external services (although this plugin is capable of utilising information pulled from them).  This tries to deny as little as possible to legit users - for example, a solution such as blocking link posting for new users means that all users need to have a number of posts before they are able to post links, whereas Spamalyser allows this activity as long as it doesn't look spammy.

Stop Forum Spam and Fassim plugins, for example, seem to only perform blocks on user registration (before they even register, in fact).  As Spamalyser is more permissive, it only performs checks on posting (this also has the slight benefit of being able to work if guest posting is enabled, although Spamalyser isn't as effective with guests than it is with registered users).  Also, performing spam analysis on posts means that Spamalyser is able to use a greater number of inputs to judge whether a post is spam or not.
Of course, the downside is that Spamalyser will not try to do anything about spam registrations.

Can this be used with other anti-spam plugins?
Most of them yes.  I'm not sure whether this works with the Akismet plugin or not (although it's somewhat pointless to do so, because Spamalyser supports Akismet lookups).
If you use it with the Stop Forum Spam plugin, you may wish to disable SFS lookups in Spamalyser.

How does this plugin work?
When a user tries to post a new thread or reply, or edit a post, the plugin will check whether the user passes a number of thresholds (such as post count) and if so, deems the user "safe" and stops there.
If the user fails to pass the threshold test, the main Spamalyser engine kicks in and analyses the post to determine a "spam weighting" (likeliness of it being spam).  It then compares this weighting against some configurable action thresholds to decide on whether it should do anything to the post.  If the weighting meets the thresholds, Spamalyser can currently (depending on what you enable):

Report the post,
Unapprove the post, or
Block the post (displays an error when user tries to submit the post)

All weighting calculations are logged and can be viewed via ACP -> Tools -> Spamalyser Log

How is the "spam weighting" calculated?
Quite a number of means, but most of the code is link analysis.  Spammers ultimately want to post links, so it seems like a good place to start.  Every link posted will add to the weighting, and links with similar keywords or to the same domain get penalised more heavily.
Spamalyser can also make some judgements based on the poster's online time and other factors, and has a number of features to attempt to reduce the number of false positives (for example, by examining the user's previous posts).
External lookups to services such as Stop Forum Spam and Akismet are also supported, and you can specify the amount of weighting to give to these services.

You can try looking in the Spamalyser settings which show all the methods used by this plugin to detect spam.

Configuring Spamalyser
Spamalyser has quite a number of options, designed to give the administrator a fair amount of control over the spam analysis process.
The defaults are relatively permissive.  From my testing, you can probably reduce the Unapprove Threshold from 10 to 5 and capture a lot more spam.
Note that, by default, external lookups are disabled for localhost installs, but enabled for non-localhost installs.

I know there are a lot of options so it may be difficult to get your head around what they all do.  You can try inspecting the log (clicking on the weighting will show how it's calculated) to see how some posts' weights are determined and perhaps get a rough idea of what some of the options do.

Disable Link Analysis
If you wish to, you can completely disable the internal link analysis routine.  To do so, set the following three settings to 0:

Weight Per Simple Link
Weight Per Complex Link
Duplicate Keyword Bias

Akismet Note
To enable Akismet, you must enter in an API key - and you must ensure that this key is valid because Spamalyser won't check it.  Also, your server must be able to either make requests via cURL or fsockopen()

Upgrading
I'm not going to bother maintaining upgrade paths for this plugin.  This means that if you wish to upgrade to a newer version, you'll have to uninstall the old version, upload the files for the new version, then install it.
Obviously this means that you'll lose any setting changes you've made, as well as log entries, so you may wish to note down any custom settings you've set before uninstalling.  Do note that the meaning of some settings may change in the future, so refer to the changelog to ensure your custom settings have the same meaning.

Performance Issues
As this plugin performs external lookups by default, for all analysed posts, posting may be slower for posts which are examined.  Although the services queried (SFS, Akismet and Google) should have reasonably fast servers, you can choose to disable these lookups if you wish. [lookups are performed through MyBB's fetch_remote_file() function]
As for internal link analysis, the algorithm isn't exactly fast (still should be significantly faster than MyBB's post parser), but should be acceptable, especially since it's only done during posting.

Limitations

The plugin assumes that MyCode is always enabled (so only detects [url] type links), which is probably in-line with the assumptions made by most spammers
Google searching behaviour is somewhat erratic - it's a bit difficult to generate a representative search query from a post. Can be tweaked, though I'm not too worried because spammers can theoretically bypass this by slightly varying their posts across forums
Reports currently only go to the database regardless of what reporting medium you have selected

Hamster24 · 01-08-2015 05:46 PM

where can i find the compatibility line in the plugin to make it 18.
as i am getting error not compat with 1.8.3

sorry i find it its in sp_admin.php
but i have a question i was changing compat lines in other plugins and changed to 18.
but in spamalyser
existing compat lines were like 14*,16* so i changed it to 18*.
so what difference would it make b/w 18 and 18*?

***ZiNgA BuRgA*** · 01-09-2015 03:55 PM

* means any arbitrary text. So "18" would match nothing.

Be aware of potential issues when trying to use this on 1.8, as it isn't really compatible yet (see previous posts for more info).

Hamster24 · 01-09-2015 07:09 PM

oh ok i disabled it.
btw
is there any plan to upgrade it?
it is(was) awesome plugin

Hamster24 · 01-10-2015 03:50 AM

since i disabled the plugin i got 40+ SPAM threads created that too in weird unknown languages. please yumi update it to 1.8.
earlier when spamalyser was active it seemed people have stopped spamming in this world but i was wrong.

Hamster24 · 01-11-2015 06:29 PM

(09-23-2014 04:57 PM)Sama34 Wrote: The `spamalyser_weight_markreport` setting needs to be revised in 1.8, since the reportedposts table has been changed:
PHP Code:
1 2 3 4 5 6
		$numreports = $db->fetch_field($db->query('
			SELECT COUNT(DISTINCT p.pid) AS numreports
			FROM '.TABLE_PREFIX.'posts p
			INNER JOIN '.TABLE_PREFIX.'reportedposts r ON r.pid=p.pid
			WHERE p.uid='.$user['uid'].' AND p.visible=1 AND r.reportstatus!=0'.$qx
		), 'numreports');
Should probably be:
PHP Code:
1 2 3 4 5 6
		$numreports = $db->fetch_field($db->query('
			SELECT COUNT(DISTINCT p.pid) AS numreports
			FROM '.TABLE_PREFIX.'posts p
			INNER JOIN '.TABLE_PREFIX.'reportedcontent r ON r.id=p.pid
			WHERE p.uid='.$user['uid'].' AND p.visible=1 AND r.reportstatus!=0 AND (r.type="post" OR type="")'.$qx
		), 'numreports');
Cheers.

Yumi
is this code correct and if yes where to change it.

Hamster24 · 01-12-2015 07:38 PM

today users reported they weren't able to reply or post new thread ang get some sort of mysql error.

Hamster24 · 03-05-2015 04:20 AM

Please update this to 1.8 version. it was the most useful (after xthreads) plugin of mybb. totally stops spam

Sama34 · 03-05-2015 03:14 PM

Just read some posts above.

Hamster24 · 03-05-2015 05:35 PM

ah at last found it its the sp_main file

Hamster24 · 03-05-2015 06:58 PM

i changed the code you gave in the posts above in the sp_main file and spamalyser started to work fine for guest replies i tested it and it showed me error in a nice popup on the right and side which fade away after little while.

but just few minutes ago i tried that with guest threads. and i get this error.
Image attached: http://i.imgur.com/ah0eT7P.png

afterwards when i see the forum the thread is created but the thread is not reported

how can i fix it when it works for replies.

also in spamalyser logs what is /admin/spamalyser_img/action_.gif i what does it symbolize as i am getting a broken image (404) so don't know what this image means.
other images such as /admin/spamalyser_img/action_blocked.gif is showing fine.

Spamalyser
Author	Message