Codeable info

The WordPress Plugin Compatibility Procedure Needs To Be Reviewed

Posted on by in Blog

WordPress developer Dalton Rooney raised some interesting points about the WordPress plugin repository. Those of you who frequent the repository a lot will be aware of the compatibility box to the right hand side of all plugins. The box allows you to vote whether a specific version of a plugin is working with a specific version of WordPress (it defaults to the latest version of each). It’s an incredibly handy resource that is unfortunately underused as it’s incredibly useful to know whether a plugin works with the version of WordPress you are using.

Dalton today spoke about a minor annoyance about this setup. He notes that his Portfolio Slideshow plugin had previously had 7 votes stating that the plugin worked fine.

Plugin Compatibility Box

Unfortunately, for the latest version of WordPress, some people have reported that it doesn’t work. Since the compatibility box defaults to the latest version of WordPress, this suggests to potential downloaders that the plugin isn’t working correctly.

Plugin Compatibility Box

He quite rightly points out that there is little incentive for people to vote when a plugin is working fine but a great incentive when something isn’t working correctly. I must admit that I have been guilty of this many times in the past i.e. only taking the time to vote when the plugin isn’t working correctly. Due to this, the compatibility box results can be skewed negatively.

Dalton notes that:

99% of the problems I’ve seen with our plugin fall into the following three categories: 1) Didn’t read the instructions. 2) Poorly coded themes missing the header or footer functions or hard-coding scripts, or 3) Conflicts with lousy plugins that hard-code scripts instead of enqueing them.

I’d like to create a little survey that runs whenever someone tries to click that “Broken” button and ask if they read the instructions and the FAQ before they are allowed to vote. And, if a problem is reported to the forum & then marked as resolved, their vote is automatically changed to “Works!”

I can understand Daltons frustration with this. There are thousands of plugins and themes in the repository that are poorly coded. Every plugin has the potential to conflict with something else because of this issue. It’s unfair that anyone can simply check that a plugin is not working without testing it in the proper environment.

As many of you know, I have tested hundreds of plugins in order to review them on WP Mods. I appreciate that Dalton is a plugin developer who spends a lot of time adding good instructions and making sure the code is of a high standard. From my experience, this is not always the case. For every three or four plugins I test out for WP Mods, there is always one that simply doesn’t work.

When I test a new plugin I make sure that no other plugins are activated and always use the Twenty Ten (or recently Twenty Eleven) default WordPress theme. Unfortunately, some just do not work. I have sent many emails to developers advising that I am attempting to review their plugin. Many have resolved the issue though some have not even replied. It’s something that frustrates me as you can spend a lot of time trying to get a plugin to work.

Likewise, not all developers add good instructions for downloaders. Today I reviewed a questions and answers plugin for an article that publishes in a week or so. I could not get the plugin to work but I eventually found a response in the support forums that said that in order to make the plugin work you had to re-save your permalink structure. Unfortunately, there was no note about this in the installation instructions.

Compatibility Procedure Needs To Be Reviewed

Bearing in mind what I noted above, I believe that Dalton is right; the compatibility procedure needs to be reviewed. The compatibility box is an incredibly useful feature and can save others a lot of time by advising them that a plugin is not working, however the procedure in which a vote is cast should be reviewed.

Perhaps instead of a broken link vote the user is taken to a feedback form or support thread where they note the problem they are having with the plugin. This can then be reviewed by the plugin developer. They can then either explain to the user what they are doing wrong or upgrade their plugin accordingly. If they state that they cannot fix it and/or a set period of time passes without a solution, the negative vote would be cast.

This would make the compatibility box more reliable as far as negative votes go though it’s prudent to assume that if the procedure to cast a negative vote is longer, there is less chance of negative votes being cast. This is a good thing though, as with the method, if the vote is not cast as negative, it means that the developer answered the users query and updated the installation instructions or fixed the issue and updated their plugin.

Dalton suggested that if a developer resolves and issue the vote should be changed to confirm that the plugin works. I would change this slightly by adding a negative vote to the previous version but automatically adding a positive vote to the new version. This ensures that users know that they should download the latest version.

Do you agree that the current system of casting votes on whether a plugin works needs to be reviewed? I’d love to hear your opinion on this issue :)


Codeable info

Comments (18)

Comment by Kevin Muldoon says:

Thank you but there’s no need. Drop by the forums if you have time. I’m sure members would benefit from chatting with a plugin developer :)

Comment by seocosenza says:

thank you for these further informations.
everytime there’s a new wp release i just wait a couple of days and i update my wp core too. i think it’s really importat to keep a fresh and up-to-date cms.
i’ll try to submit to wp directory my plugin giving the most in fact of support, information and above all precise information on wp versioning :)

thank you so much, i feel i have to give something to you for this consulting!
let me know what can i do for your precious support!
Andrea :)

Comment by Kevin Muldoon says:

Updates to the WordPress core is the main reason many plugins stop working. I would personally continue to code for the latest release and stress in your plugin information page that you have to be using the latest version of WordPress.

Your friends should update too :)

Comment by seocosenza says:

hi Kevin,
thank you very much for your reply! :)
i just developed a foursquare based wp plugin and i run it on wordpress 3.2.1 but some friends of mine had problems with wp 3.1.2. is it somehing so usual?

in addition, thank you for your point of view about compatibility.
the most frustating thing is that i developed the plugin following the codex.wordpress guide!

thank you again!
Andrea :)

Comment by Kevin Muldoon says:

Side affects? I’m not sure what you mean.

For me, the most reliable way to check a plugin is by testing it myself.

The directory compatibility isn’t fullproof – I consider it a guide i.e. a few people saying a plugin isn’t compatible doesn’t mean the plugin won’t work, it may just mean a few people had problems installing or using it.


Comment by seocosenza says:

hi Kevin,
i read just now your post, because i have developer a simple wp plugin based on foursquare.
I tested it on 4 different blogs with different fingerprint, plugins and themes, with no issues.
Some friend of mine told me of unexpected error and it is really really frustating to me.

Is there an official way to check a plugin compatibility or not?
I did not sumbit to the offical plugins directory yet, but i’m really scared to do for these side effects.
can you suggest me something to do?

thank you in advance,
andrea (from italy) :)

Comment by tzdk says:

Don’t like the word closed and recommended you already have on frontpage where Automattic or Automattic associates show off their stuff ;)

I think “regulated” is a better word. Point being that there is hardly any as of now and that cause problems. Does “community” benefit from no regulation?

If you look at Mozillas add-on sites you can’t say they are free of “crap” but they do have review teams and various ways of sorting/searching site. Mostly meaningful. In a WP context I would think that focus should be on update/dev status. Perfectly fine if a dev starts to develope for Drupal but then make it clear to people that X project is most likely dead. There are way too much unnecessary stuff to wade through as it is. In worst cases you get a 404 after having dug out the sometimes hidden website of dev. That is a problem from any point of view right?

Those devs who actually work their butt off, deserve donations and fame (micro payment will rule one day!) – perhaps have ideas of future business or whatever will benefit from some sort of “selection” other than “most downloaded” and that type of hype. One that cannot be argued since then new issue will be censorship and what not.

In fact it will probably be a good start if .org site began to inform Devs about good practices with regards to how to promote, support your stuff. Much more than guidelines for readme.txt is needed. Send new signals. Delevering enough information to make it possible to evaluate and select would be fantastic. I think this is where they go hmmm because .org is simply seen as a mighty storage site. No regulation is the idea! – opposite “featured” section but no rules without exceptions. Also themes are something totally different, make so much sense… Ok, but then kiss user friendlyness good bye. I wonder if “community” has any interest in that? Almost feels like plugin spam sometimes ;)

Comment by Kevin Muldoon says:

Yeah there definitely needs to be a change of some kind. I do still find value in there be some sort of voting system – or at least a system that advises you when something is clearly not working. Though the current method is not very practical.

Comment by Kevin Muldoon says:

There have been rumours about certain plugins getting a ‘recommended’ or ‘approved’ status. Not sure if it will ever see the light of day though.

I don’t know if having a closed shop, like the apple store, is the way to go. WordPress flourished because of its open source status. I do agree that there is a lot of crap in the plugin directory though perhaps this is a small price to pay to have such a wide variety or plugins and such a large community.

Perhaps not though. The android market is open source however there are thousands of really poor products released there. There’s a lot of programs with viruses too – it’s a bit like the wild west. An ‘recommended’ plugin list could be a happy medium.

Comment by Dalton says:

Hi Kevin,
Thanks for picking this up! I’m glad to see I’m not the only one who sees the flaw in this system.

Just one clarification. I was suggesting that if someone reports the plugin as broken and then posts in the forum about it, if it turns out that the problem was actually caused by another plugin or theme then there might be some automated way to change the vote.

In reality, there are so many possibilities for problems with WP plugins and themes that it seems unlikely that a perfect solution could ever exist. But I like to imagine that there is a way to fine tune the system so the results are more accurate and less misleading.

Comment by tzdk says:

There would not be any questions asked to plugin site if there were “daily care”. Setup is nice, changlog tab and all but also pretty obvious that it just rolls from there. Upload whatever, done. Feel like donating? Here is PayPal link which replaced link to DEV site. Most stupid upgrade ever. Why encourage people to check out what they install? or potentially even donate to? Make sense to who?

All this opposite themes which must be approved by review team. Not so difficult to see what is needed? Plugins can be checked in other ways than public opinion. Also the world don’t need close to 15000 plugins. Cleanup team needed. Sounds harsh perhaps but I am totally in favor of a system where DEV need to prove their existence on a regulary basis. If plugin is pretty much dead just remove it, bye bye. Set up a new site for “Most likely dead stuff but no code is EVER deleted so go crazy” if necessary. Get rid of the fluff. Make it easier for people to select. Might even improve validity of votes.

Comment by tzdk says:

I don’t like any voting :)

You need a generic WP with 2010 theme and quite a bit of know-how to evaluate such questions. You want to do that for the countless plugins coming in daily? Nobody does. Focus should be on DEVS. Indirectly or directly they need to be informed that .org site might be a “storage” but don’t mess about too much. Best advertisement for paying up for plugins is .org site! – time for an overhaul and you don’t get that by improving voting system which has questionable value in any case.

Check W3 Total Cache btw. It is NOT broken with WP version 3.1.3 but pluign itself is partly broken and then people get annoyed and suddenly find voting interesting. With a minimal knowledge of how virtual/internet behavior works you can tell this is what will happen.

Comment by Kevin Muldoon says:

It does offer value on occasion. If I’m looking for a plugin and I see several votes stating that it doesn’t work then I don’t download it as I know it will be a waste of time.

I do agree that in most cases the information is not that reilable. I also agree that Devs should take responsibility for ‘it doesn’t work’ type questions (as noted in my article), which is why we need a different method of voting.

Comment by Kevin Muldoon says:

Well the voting defaults to the latest version of WordPress. The user would have to change it to say that the version they were using wasn’t working, though I’m sure some people don’t check that and say it just doesn’t work (and select the newest version of WordPress instead).

Comment by Nathan B says:

Right, but I mean non-plugin-developers who don’t want to update from WordPress 3.1 to 3.13 (or soon 3.2), for example. So they stay on an older WP version, which may have security holes (that the new version fixes). The plugin voting, I thought, was supposed to be a mechanism to assure them their sites and plugins would still work if they upgraded. I do like the Hotfix plugin though — it may have been this blog that I learned about it.

Comment by tzdk says:

I have no idea how you can see any value in this voting system regarding compatibility. How many votes you think are really valid? As Dalton points out many users are not in a position to declare yes/no to this.

Devs are not told to do much else than make a decent readme.txt by .org site. Rest is wild west in all possible ways, plugins are not themes which are so much more important! Well, may be it should be added that the uploading dev have full responsibility for keeping “compatible with” or what it is called updated? I can think of other non-code requirements. Like when you upload an image to a sharing site it can get removed if not viewed in 30 days. Get the idea? If a dev. has left the shop but can’t be bothered to announce that people should know – and certainly not start voting away to figure out if it still works a little. Totally meaningless and not part of any research, testing or something valid. Devs should take responsibility for such “does it work” questions.

Comment by Nathan B says:

Excellent questions I’ve wondered about too; good ideas. Thanks for raising them. One of the reasons this is so important is that many site owners or developers delay updating to the current version of WordPress because they’re afraid their plugins will break; a good reporting system to assure them otherwise is important.

Comment by Kevin Muldoon says:

I can understand why they hold back. It is time consuming to update plugins on a regular basis, particularly if you have released several or more to the community.

Codeable info