How to enhance YOURLS for spam control/throttling, unshortening and weekly stats

So I've been using YOURLS link shortener PHP code to power mrte.ch. YOURLS is a fantastic system created by Ozh Richard – @Ozh on Twitter and Lester Chan – @GamerZ on Twitter.

The latest updates are incredible and provide tons of features and control but there were a couple of spots that I've enhanced:

pre-validating URL/link,
site blacklisting,
domain spam throttling,
unshortening already shortened links and
top weekly stats.

Below is the code that I've added to both the shortener page and the API pages, just before the "yourls_add_new_link" calls. There's nothing fancy about my approach (yet) but I figured this would help anyone who needs these features. I'm more of function vs. form kind of person when it comes to some things, so please ignore the crass and simplistic approach, for me it just works.

Pre-validating Submitted URL

Since most of these enhancement are before the "yourls_add_new_link" has had a chance to clean up the url, make sure to check the submitted URL with "yourls_sanitize_url" function and use a URL validator like this:

Code:

function valid_url($str) {
return ( ! preg_match('/^(http|https|ftp):\/\/([A-Z0-9][A-Z0-9_-]*(?:\.[A-Z0-9][A-Z0-9_-]*)+):?(\d+)?\/?/i', $str))
? FALSE : TRUE;
}

$url = yourls_sanitize_url($_REQUEST['url']);

if (!valid_url($url)) {
die( 'Invalid url, go back and try again' );
}

Site Blacklisting
All I did here is create a crude array of top level domains that have spammed me before. I added the array to the config.php file so that it's available from the general include. It's populated manually for now as I await ozh's enhancements to add blacklisting, etc.

config.php code:

Code:

$yourls_banned_URL = array(
'mrte.ch', 'yoursitesucksforspamming.org', 'diespammers.org', 'idontthinkso.org', 'nothanksloser.com'
);

Note: please note how the first site is mrte.ch, yes I found folks submitting mrte.ch shortened links to the mrte.ch shortener, not anymore....

Main index.php page and also your API page: include just before "yourls_add_new_link" call in the submit section:

Code:

$url = $_REQUEST['url'];

foreach ($yourls_banned_URL as $banned_url) {
if (stristr($url, $banned_url)) {
die( 'Banned site' );
}
}

Note: please let me know if there's a more optimal way of doing this, thanks.

Domain Spam Throttling
This just check for today's submitted sites and if more than 5 have been submitted from the same domain for that day, it just dies. I may add a check to automatically blacklist if within a specific time period and a greater throttle number. I'll also remove all the other existing links, etc. if they are spamming, I'll update this posting if/when i do.

Main index.php page and also your API page: include just before "yourls_add_new_link" call in the submit section (I split $spamCheckQuery for easier reading only):

Code:

$spamCheckURLArray = parse_url(stripslashes($url));

$spamCheckQuery = "SELECT count(*) FROM `yourls_url_table` WHERE `url` LIKE '%";
$spamCheckQuery .= $spamCheckURLArray['host'] . "%' AND timestamp >= CURDATE()";
$spamLinkCount = $ydb->get_var($spamCheckQuery);

if ($spamLinkCount > 5) {
die( 'Nice try on spamming...' );
}

Unshortening Already Shortened Links
I found Slinky a PHP URL Unshortener or "lengthener" as they put it and decided to put this into place because I was finding folks submitting links that were already shortened by bit.ly, tr.im, etc. I also wanted to make sure that I kept things nice and clean. Here's how I use Slinky, note there's a syntax bug in their code (missing semi-colon or something, can't remember) once you fix that you are on your way. Just download the code from Slinky and drop there include on your site.

Code:

require_once( dirname(__FILE__).'/includes/slinky.php' );

$url = $_REQUEST['url'];

$slinky = new Slinky( $url );
$slinky->set_service_from_url();
// you should be able to remove the 'login' and 'apiKey' lines if you don't want bit.ly unshortening, etc.
$slinky->service->set( 'login', 'YOUR_BIT.LY_API_LOGIN_IF_YOU_HAVE_ONE' );
$slinky->service->set( 'apiKey', 'YOUR_BIT.LY_API_KEY_IF_YOU_HAVE_ONE' );
$url = $slinky->long();

Top Weekly Stats
I wanted to show top weekly stats on my main shortener page so I whipped up this little ditty.

Code:

$urlt2 = '';

// Connect To Database
$sqlcode = "SELECT url, keyword, clicks FROM `yourls_url_table` WHERE timestamp >= SUBDATE(CURDATE(), 7) ";
$sqlcode .= "order by clicks desc limit 8";

$query = $ydb->get_results($sqlcode);

if ($query) {
foreach( $query as $query_result ) {
$thisURLArray = parse_url(stripslashes($query_result->url));
$urlt2 .= '<a href="http://mrte.ch/' . $query_result->keyword .'" target="blank">';
$urlt2 .= str_replace('www.', '', $thisURLArray['host']) . '</a> (' . $query_result->clicks . ') ';
}
}

echo '<b>Week\'s Top links:</b> ' . $urlt2 . "<br><br>\n\r";

To test or validate your result you can pull up your data like this, you can also change the limit from 8 to whatever you want to show:

Quote

SELECT url, keyword, clicks, timestamp, SUBDATE(CURDATE(), 7)
FROM `yourls_url_table` WHERE timestamp >= SUBDATE(CURDATE(), 7)
order by clicks desc limit 8

Note: SUBDATE should probably be 6 instead of 7 but I wanted to be a little more inclusive.

In Conclusion
Again I'd like to extend my honest gratitude to Ozh and YOURLS for a great system and hope this helps someone. Please post a message if you were able to use any of this, if you find any bugs or have any performance enhancements, etc.

-- mel reyes: self-professed geek

« Last Edit: April 11, 2010, 03:44:42 PM by mrtech »

• mel reyes • Blog • Plaxo • LinkedIn • Twitter •
• Support mrtech.com get our toolbar •

December 08, 2009, 11:45:11 AM

About The Author

mrtech
Administrator
Posts: I am a geek!!

MR Tech Forums

Accesibility Menu:

Main Menu:

Main Content:

How to enhance YOURLS for spam control/throttling, unshortening and weekly stats

About The Author

Comments

Add a Comment