Archive for the ‘Tech’ Category

SaltThePass mobile app now available on iTunes, Google Play and Amazon

July 31st, 2013

A few months ago I released, which is a password generator that will help you generate unique, secure passwords for all of the websites you visit based on a single Master Password that you remember.

I’ve been working on a mobile / offline iOS and Android app that gives you all of the features of the website.  The apps are now in the Apple iTunes App Store, Google Play App Store and the Amazon Appstore.

Let me know if you use them!
iPad, iPhone and Android apps available

How to deal with a WordPress wp-comments-post.php SPAM attack

May 9th, 2013

This morning I woke up to several website monitoring alarms going off.  My websites were becoming intermittently unavailable due to extremely high server load (>190).  It appears had been under a WordPress comment-SPAM attack from thousands of IP addresses overnight.  After a few hours of investigation, configuration changes and cleanup, I think I’ve resolved the issue.  I’m still under attack, but the changes I’ve made have removed all of the comment SPAM and have reduced the server load back to normal.

Below is a chronicle of how I investigated the problem, how I cleaned up the SPAM, and how I’m preventing it from happening again.


The first thing I do when website monitoring alarms are going off (I use Pingdom and Cacti) is to log into the server and check its load.  Load is an indicator of how busy your server is.  Anything greater than the number of CPUs on your server is cause for alarm.  My load is usually around 2.0 — when I logged in, it was 196:

[nicjansma@server3 ~]$ uptime
06:09:48 up 104 days, 11:25,  1 user,  load average: 196.32, 167.75, 156.40

Next, I checked top and found that mysqld was likely the cause of the high load because it was using 200-1000% of the CPU:

top - 06:16:45 up 104 days, 11:32, 2 users, load average: 97.69, 162.31, 161.74
Tasks: 597 total, 1 running, 596 sleeping, 0 stopped, 0 zombie
Cpu(s): 3.8%us, 19.1%sy, 0.0%ni, 10.7%id, 66.2%wa, 0.0%hi, 0.1%si, 0.0%st
Mem: 12186928k total, 12069408k used, 117520k free, 5868k buffers
Swap: 4194296k total, 2691868k used, 1502428k free, 3894808k cached

24846 mysql 20 0 26.6g 6.0g 2.6g S 260.6 51.8 18285:17 mysqld

Using SHOW PROCESSLIST in MySQL (via phpMyAdmin), I saw about 100 processes working on the wp_comments table in the WordPress database.

I was already starting to guess that I was under some sort of WordPress comment SPAM attack, so I checked out my Apache access_log and found nearly 800,000 POSTS to wp-comments-post.php since yesterday.  They all look a bit like this:

[nicjansma@server3 ~]$ grep POST access_log - - [09/May/2013:06:07:29 -0700] "POST /wp-comments-post.php HTTP/1.1" 302 20 "" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1;)"

What’s worse, the SPAMs were coming from over 3,000 unique IP addresses.  Essentially, it was a distributed denial of service (DDoS) attack:

[nicjansma@server3 ~]$ grep POST access_log | awk '{print $1}' | sort | uniq -c | wc -l
3105 was getting hundreds of thousands of POSTS to wp-comments-post.php, which was causing Apache and MySQL to do a whole lot of work checking them against Akismet for SPAM and saving in the WordPress database.  I logged into the WordPress Admin interface, which verified the problem as well:

There are 809,345 comments in your spam queue right now.


Stopping the Attack

First things first, if you’re under an attack like this, the quickest thing you can do to stop the attack is by disabling comments on your WordPress site.  There are a few ways of doing this.

One way is to go into Settings > Discussion > and un-check Allow people to post comments on new articles.

The second way is to rename wp-comments-post.php, which is what spammers use directly to add comments to your blog.  I renamed my file wp-comments-post.php.bak temporarily, so I could change it back later.  In addition, I created a 0-byte placeholder file called wp-comments-post.php so the POSTS will look to the spammers like they succeeded, but the 0-byte file takes up less server resources than a 404 page:

[nicjansma@server3 ~]$ mv wp-comments-post.php wp-comments-post.php.bak && touch wp-comments-post.php

Either of these methods should stop the SPAM attack immediately.  5 minutes after I did this, my server load was back down to ~2.0.

Now that the spammers are essentially POSTing data to your blank wp-comments-post.php file, new comments shouldn’t be appearing in your blog.  While this will reduce the overhead of the SPAM attack, they are still consuming your bandwidth and web server connections with their POSTs.  To stop the spammers from even sending a single packet to your webserver, you can create a small script that automatically drops packets from IPs that are posting several times to wp-comments-post.php.  This is easily done via a simple script like my Autoban Website Spammers via the Apache Access log post.  Change THRESHOLD to something small like 10, and SEARCHTERM to wp-comments-post.php and you will be automatically dropping packets from IPs that try to post more than 10 comments a day.

Cleaning up the Mess

At this point, I still had 800,000+ SPAMs in my WordPress moderation queue.  I feel bad for Akismet, they actually classified them all!

I tried removing the SPAM comments by going to Comments > Spam > Empty Spam, but I think it was too much for Apache to handle and it crashed.  Time to remove them from MySQL instead!

Via phpMyAdmin, I found that not only were there 800,000+ SPAMs in the database, the wp_comments table was over 3.6 GB and the wp_commentmeta was at 8.1 GB!

Here’s how to clean out the wp_comments table from any comments marked as SPAM:

DELETE FROM wp_comments WHERE comment_approved = 'spam';

OPTIMIZE TABLE wp_comments

In addition to the wp_comments table, the wp_commentmeta table has metadata about all of the comments. You can safely remove any comment metadata for comments that are no longer there:

DELETE FROM wp_commentmeta WHERE comment_id NOT IN (SELECT comment_id FROM wp_comments)

OPTIMIZE TABLE wp_commentmeta

For me, this removed 800,000+ rows of wp_comments (bringing it down from 3.6 GB to just 207 KB) and 2,395,512 rows of wp_commentmeta (bringing it down from 8.1 GB to just 136 KB).

Preventing Future Attacks

There are a few preventative measures you can take to stop SPAM attacks like these.

NOTE: Remember to rename your wp-comments-post.php.bak (or turn Comments back on) after you’re happy with the prevention techniques you’re using.

  1. Disable Comments on your blog entirely (Settings > Discussion > Allow people to post comments on new articles.) (probably not desirable for most people)
  2. Turn off Comments for older posts (spammers seem to target older posts that rank higher in search results). Here’s a way to disable comments automatically after 30 days.
  3. Rename wp-comments-post.php to something else, such as my-comments-post.php. Comment spammers often just assume your code is at the wp-comments-post.php URL and won’t check your site’s HTML to verify this is the case. If you rename wp-comments-post.php and change all occurrences of that URL in your theme, your site should continue to work while the spammers hit a bogus URL. You can follow this renaming guide for more details.
  4. Enable a Captcha for your comments so automated bots are less likely to be able to SPAM your blog. I’ve had great success with Are You A Human.
  5. The Autoban Website Spammers via the Apache Access log post describes my method for automatically dropping packets from bad citizen IP addresses.

After all of these changes, my server load is back to normal and I’m not getting any new SPAM comments.  The DDoS is still hitting my server, but their IP addresses are slowly getting packets dropped via my script every 10 minutes.

Hopefully these steps can help others out there.  Good luck! Fighting spammers is a never-ending battle!

2012 Minifigures Available

April 17th, 2013

Thanks to Christoph‘s hard work taking photos of all 529 minifigures released in 2012, the 2012 minifigs are now available for purchase in the Unofficial Minifigure Catalog app.

To purchase the update, first update the database to the latest version (Settings > Database) and then go to Settings > Collections and look for the purchase button.


April 15th, 2013

UserTiming is one of the W3C specs that I helped design while working at Microsoft through the W3C WebPerf working group.  It helps developers measure the performance of their web applications by giving them access to high precision timestamps. It also provides a standardized API that analytics scripts and developer tools can use to display performance metrics.

UserTiming is natively supported in IE 10 and prefixed in Chrome 25+.  I wanted to use the interface for a few of my projects so I created a small polyfill to help patch other browsers that don’t support it natively. Luckily, a JavaScript version of UserTiming can be implemented and be 100% API functional — you just lose some precision and performance vs. native browser support.

So here it is: UserTiming.js


UserTiming.js is a polyfill that adds UserTiming support to browsers that do not natively support it.

UserTiming is accessed via the PerformanceTimeline, and requires support, so UserTiming.js adds a limited version of these interfaces if the browser does not support them (which is likely the case if the browser does not natively support UserTiming).

As of 2013-04-15, UserTiming is natively supported by the following browsers:

  • IE 10+
  • Chrome 25+ (prefixed)

UserTiming.js has been verified to add UserTiming support to the following browsers:

  • IE 6-9
  • Firefox 3.6+ (previous versions not tested)
  • Safari 4.0.5+ (previous versions not tested)
  • Opera 10.50+ (previous versions not tested)

UserTiming.js will detect native implementations of UserTiming, and the PerformanceTimeline and will not make any changes if those interfaces already exist.  When a prefixed version is found, it is copied over to the unprefixed name.

UserTiming.js can be found on GitHub and as the npm usertiming module.


April 14th, 2013

It’s not you, it’s me.

A few months ago I released a small JavaScript micro-framework: breakup.js

Serially enumerating over a collection (such as using async.forEachSeries()in Node.js or jQuery.each() in the browser) can lead to performance and responsiveness issues if processing or looping through the collection takes too long. In some browsers, enumerating over a large number of elements (or doing a lot of work on each element) may cause the browser to become unresponsive, and possibly prompt the user to stop running the script.

breakup.js helps solve this problem by breaking up the enumeration into time-based chunks, and yielding to the environment if a threshold of time has passed before continuing.  This will help avoid a Long Running Script dialog in browsers as they are given a chance to update their UI.  It is meant to be a simple, drop-in replacement for async.forEachSeries().  It also provides breakup.each() as a replacement for jQuery.each() (though the developer may have to modify code-flow to deal with the asynchronous nature of breakup.js).

breakup.js does this by keeping track of how much time the enumeration has taken after processing each item.  If the enumeration time has passed a threshold (the default is 50ms, but this can be customized), the enumeration will yield before resuming.  Yielding can be done immediately in environments that support it (such as process.nextTick() in Node.js and setImmediate() in modern browsers), and will fallback to a setTimeout(..., 4) in older browsers.  This yield will allow the environment to do any UI and other processing work it wants to do.  In browsers, this will help reduce the chance of a Long Running Script dialog.

breakup.js is primarily meant to be used in a browser environment, as Node.js code is already asynchronously driven. You won’t see a Long Running Script dialog in Node.js. However, you’re welcome to use the breakup Node.js module if you want have more control over how much  time your enumerations take.  For example, if you have thousands of items to enumerate and you want to process them lazily, you could set the threshold to 100ms with a 10000ms wait time and specify the forceYield parameter, so other work is prioritized.

Check it out on Github or via npm.

March 26th, 2013

As many geeks do, I have a collection of about 30-odd domain names that I’ve purchased over the past few years for awesome-at-the-time ideas that I just never found the time to work on.

Last month, I resolved stop collecting these domains and instead make some visible progress on them, one at a time.

SaltThePass is my first project.  Do you have an account on LinkedIn, Evernote, or Yahoo?  All of these sites had password breaches in the last year that compromised their user’s logins and passwords.  One big problem people face today is managing all of the passwords they use for all of the sites that they visit.  People often re-use the same password on many sites because it would be impossible to remember hundreds of different passwords.  Unfortunately, this means that if a single site is hacked and your password is revealed, the attacker may have access to your account on all of the other sites you visit.

To help solve this problem, I created  Salt The Pass is a password generator that will help you generate unique, secure passwords for all of the websites you visit based on a single Master Password that you remember.  You don’t need to install any additional software, and you can access your passwords from anywhere you have internet access.

Check it out at and let me know what you think!

Using Modern Browser APIs to Improve the Performance of Your Web Applications

February 26th, 2013

Last night I gave a short presentation on Using Modern Browser APIs to Improve the Performance of Your Web Applications at GrWebDev.

It’s available on SlideShare:

Usinng Modern Browser APIs SlideShare deck

Two other presentations I gave late last year are available here as well:

Switch your HTPC back to Media Center after logging out of Remote Desktop

May 23rd, 2012

I have a Windows 7 Media Center PC hooked up to the TV in our living room.  It’s paired to a 4-stream Ceton CableCard adapter and is great for watching both TV and movies.

Sometimes I need to Remote Desktop (RDP) into the machine to install updates or make other changes.  During this, and after logging out, the Media Center PC is left at the login-screen on the TV.  So the next time I sit down to watch TV, I have to find the wireless keyboard and enter my password to log back in.

Since this can get annoying, I’ve created a small script on the desktop that automatically switches the console session (what’s shown on the TV) back to the primary user and re-starts Media Center.  This way, the next person that uses the TV doesn’t have to log back in.  When I’m done in the RDP session, I simply start the batch script and it logs me out of RDP and logs the TV back in.

Here’s the simple script:

call %windir%\system32\tscon.exe 1 /dest:console
start "Media Center" /max %windir%\ehome\ehshell.exe
exit /b 1

PngOutBatch: Optimize your PNGs by running PngOut multiple times

May 15th, 2012

PngOut is a command-line tool that can losslessly reduce the file size of your PNGs. In many cases, it can reduce the size of a PNG by 10-15%. I’ve even seen some cases where it was able to reduce the file size by over 50%.

There are several other PNG compression utilties out there, such as pngcrush and AdvanceCOMP, but I’ve found PngOut to be the best optimizer most of the time.

There’s an excellent tutorial on PngOut for first-timers.  Running PngOut is pretty easy, simply run it once agaist your PNG:

PngOut.exe [image.png]

However, to get the best optimization of your images, you can run PngOut multiple times with different block sizes (eg, /b1024) and randomized initial tables (/r).

There’s a commercial program, PngOutWin that can run through all of the block sizes using multiple CPU cores, but I wanted something free that I could run from the command line.

To aid in this, I created a simple DOS batch script that runs PngOut through 9 different block sizes (from 0 to 8192), with each block size run multiple times with random initial tables.

While the first iteration of PngOut does all of the heavy lifting, I’ve sometimes found that using the different block sizes can eek out a few extra bytes (sometimes 100-bytes or more than the initial pass).  You may not care about optimizing your PNG to the absolute last byte possible, but I try to run any new PNGs ready for production in my websites and mobile apps through this batch script before they’re committed to the wild.

Running PngOutBatch is as easy as running PngOut:

PngOutBatch.cmd [image.png] [number of iterations per block size - defaults to 5]

PngOutBatch will show progress as it reduces the file size.  Here’s a sample compressing the PNG logo from

Blocksize: 0
Iteration #1: Saved 2529 bytes
Iteration #2: No savings
Iteration #3: No savings
Iteration #4: No savings
Iteration #5: No savings
Blocksize: 128
Iteration #1: Saved 606 bytes
Iteration #2: Saved 10 bytes
Iteration #3: No savings
Iteration #4: Saved 2 bytes
Iteration #5: No savings
Blocksize: 192
Iteration #1: No savings
Iteration #2: No savings
Iteration #3: No savings
Iteration #4: No savings
Iteration #5: No savings
Blocksize: 256
Iteration #1: Saved 1 bytes
Iteration #2: No savings
Iteration #3: Saved 5 bytes
Iteration #4: Saved 11 bytes
Iteration #5: No savings
Blocksize: 512
Iteration #1: No savings
Iteration #2: No savings
Iteration #3: No savings
Iteration #4: No savings
Iteration #5: No savings
Blocksize: 1024
Iteration #1: No savings
Iteration #2: No savings
Iteration #3: No savings
Iteration #4: No savings
Iteration #5: No savings
Blocksize: 2048
Iteration #1: No savings
Iteration #2: No savings
Iteration #3: No savings
Iteration #4: No savings
Iteration #5: No savings
Blocksize: 4096
Iteration #1: No savings
Iteration #2: No savings
Iteration #3: No savings
Iteration #4: No savings
Iteration #5: No savings
Blocksize: 8192
Iteration #1: No savings
Iteration #2: No savings
Iteration #3: No savings
Iteration #4: No savings
Iteration #5: No savings
D:\temp\test.png: SUCCESS: 17260 bytes originally, 14096 bytes final: 3164 bytes saved

The first block size (0) reduced the file by 2529 bytes, then the 128-byte block size further reduced it by 606, 10 then 2 bytes. The 192-byte block size didn’t help, but a 256-byte block size reduced the file size by 1, 5 then 11 more bytes.  Larger block sizes didn’t help, but at the end of the day we reduced the PNG by 3164 bytes (18%), and 635 bytes (25% more) than if we had only run it once.

The PngOutBatch.cmd script is hosted at Gist.Github if you want to use it or contribute changes.

DIY Cloud Backup using Amazon EC2 and EBS

February 20th, 2012

I’ve created a small set of scripts that allows you to use Amazon Web Services to backup files to your own personal “cloud”. It’s available at GitHub for you to download or fork.


  • Uses rsync over ssh to securely backup your Windows machines to Amazon’s EC2 (Elastic Compute Cloud) cloud, with persistent storage provided by Amazon EBS (Elastic Block Store)
  • Rsync efficiently mirrors your data to the cloud by only transmitting changed deltas, not entire files
  • An Amazon EC2 instance is used as a temporary server inside Amazon’s data center to backup your files, and it is only running while you are actively performing the rsync
  • An Amazon EBS volume holds your backup and is only attached during the rsync, though you could attach it to any other EC2 instance later for data retrieval, or snapshot it to S3 for point-in-time backup


There are several online backup services available, from Mozy to Carbonite to Dropbox. They all provide various levels of backup services for little or no cost. They usually require you to run one of their apps on your machine, which backs up your files periodically to their “cloud” of storage.

While these services may suffice for the majority of people, you may wish to take a little more control of your backup process. For example, you are trusting their client app to do the right thing, and for your files to be stored securely in their data centers. They may also put limits on the rate they upload your backups, change their cost, or even go out of business.

On the other hand, one of the simplest tools to backup files is a program called rsync, which has been around for a long time. It efficiently transfers files over a network, and can be used to only transfer the parts of a file that have changed since the last sync. Rsync can be run on Linux or Windows machines through Cygwin. It can be run over SSH, so backups are performed with encryption. The problem is you need a Linux rsync server somewhere as the remote backup destination.

Instead of relying on one of the commercial backup services, I wanted to create a DIY backup “cloud” that I had complete control of. This script uses Amazon Web Services, a service from Amazon that offers on-demand compute instances (EC2) and storage volumes (EBS). It uses the amazingly simple, reliable and efficient rsync protocol to back up your documents quickly to Amazon’s data centers, only using an EC2 instance for the duration of the rsync. Your backups are stored on EBS volumes in Amazon’s data center, and you have complete control over them. By using this DIY method of backup, you get complete control of your backup experience. No upload rate-limiting, no client program constantly running on your computer. You can even do things like encrypt the volume you’re backing up to. NOTE: As of 2014-05-21, EBS volumes can be encrypted automatically.

The only service you’re paying for is Amazon EC2 and EBS, which is pretty cheap, and not likely to disappear any time soon. For example, my monthly EC2 costs for perfoming a weekly backup are less than a dollar, and EBS costs at this time are as cheap as $0.10/GB/mo.

These scripts are provided to give you a simple way to backup your files via rsync to Amazon’s infrastructure, and can be easily adapted to your needs.

How It Works

This script is a simple DOS batch script that can be run to launch an EC2 instance, perform the rsync, stop the instance, and check on the status of your instances.

After you’ve created your personal backup “cloud” (see Amazon Cloud Setup), and have the Required Tools, you simply run the amazon-cloud-backup.cmd -start to startup a new EC2 instance. Internally, this uses the Amazon API Developer Tools to start the instance via ec2-run-instances. There’s a custom bootscript for the instance, that works well with the Amazon Linux AMIs to enable root access to the machine over SSH (they initially only offer the user ec2-user SSH access). We need root access to perform the mount of the volume.

After the instance is started, the script attaches your personal EBS volume to the device. Its remote address is queried viaec2-describe-instances and SSH is used to mount the EBS volume to a backup point (eg, /backup). Once this is completed, your remote EC2 instance and EBS volume are ready for you to rsync.

To start the rsync, you simply need to run amazon-cloud-backup.cmd -rsync [options]. Rsync is started over SSH, and your files are backed up to the remote volume.

Once the backup is complete, you can stop the EC2 instance at any time by running amazon-cloud-backup.cmd -stop, or get the status of the instance by running amazon-cloud-backup.cmd -status. You can also check on the free space on the volume by running amazon-cloud-backup.cmd -volumestatus.

There are a couple things you will need to configure to set this all up. First you need to sign up for Amazon Web Services and generate the appropriate keys and certificates. Then you need a few helper programs on your machine, for example rsync.exe and ssh.exe. Finally, you need to set a few settings in amazon-cloud-backup.cmd so the backup is tailored to your keys and requirements.

Amazon “Cloud” Setup

To use this script, you need to have an Amazon Web Services account. You can sign up for one at Once you have an Amazon Web Services account, you will also need to sign up for Amazon EC2.

Once you have access to EC2, you will need to do the following.

  1. Create a X.509 Certificate so we can enable API access to the Amazon Web Service API. You can get this in your Security Credentials page. Click on the X.509 Certificates tab, then Create a new Certificate. Download both the X.509 Private Key and Certificate files (pk-xyz.pem and cert-xyz.pem).
  2. Determine which Amazon Region you want to work out of. See their Reference page for details. For example, I’m in the Pacific Northwest so I chose us-west-2 (Oregon) as the Region.
  3. Create an EC2 Key Pair so you can log into your EC2 instance via SSH. You can do this in the AWS Management Console. Click on Create a Key Pair, name it (for example, “amazon-cloud-backup-rsync”) and download the .pem file.
  4. Create an EBS Volume in the AWS Management Console. Click on Volumes and then Create Volume. You can create whatever size volume you want, though you should note that you will pay monthly charges for the volume size, not the size of your backed up files.
  5. Determine which EC2 AMI (Amazon Machine Image) you want to use. I’m using the Amazon Linux AMI: EBS Backed 32-bit image. This is a Linux image provided and maintained by Amazon. You’ll need to pick the appropriate AMI ID for your region. If you do not use one of the Amazon-provided AMIs, you may need to modify for the backup to work.
  6. Create a new EC2 Security Group that allows SSH access. In the AWS Management Console, under EC2, open the Security Groups pane. Select Create Security Group and name it “ssh” or something similar. Once added, edit its Inbound rules to allow port 22 from all sources “″. If you know what your remote IP address is ahead of time, you could limit the source to that IP.
  7. Launch an EC2 instance with the “ssh” Security Group. After you launch the instance, you can use the Attach Volume button in theVolumes pane to attach your new volume as /dev/sdb.
  8. Log-in to your EC2 instance using ssh (see Required Toolsbelow) and fdisk the volume and create a filesystem. For example:
    ssh -i my-rsync-key.pem
    [ec2-user] sudo fdisk /dev/sdb
    [ec2-user] sudo mkfs.ext4 /dev/sdb1
  9. Your Amazon personal “Cloud” is now setup.

Many of the choices you’ve made in this section will need to be set as configuration options in the amazon-cloud-backup.cmd script.

Required Tools

You will need a couple tools on your Windows machine to perform the rsync backup and query the Amazon Web Services API.

  1. First, you’ll need a few binaries (rsync.exe, ssh.exe) on your system to facilitate the ssh/rsync transfer. Cygwin can be used to accomplish this. You can easily install Cygwin from After installing, pluck a couple files from the bin/folder and put them into this directory. The binaries you need are:

    You may also need a couple libraries to ensure those binaries run:

  2. You will need the Amazon API Developer Tools, downloaded from Place them in a sub-directory called amazon-tools\

Script Configuration

Now you simply have to configure amazon-cloud-backup.cmd.

Most of the settings can be left at their defaults, but you will likely need to change the locations and name of your X.509 Certificate and EC2 Key Pair.


Once you’ve done the steps in Amazon “Cloud” Setup, Required Tools and Script Configuration, you just need to run the amazon-cloud-backup.cmd script.

These simple steps will launch your EC2 instance, perform the rsync, and then stop the instance.

amazon-cloud-backup.cmd -launch
amazon-cloud-backup.cmd -rsync
amazon-cloud-backup.cmd -stop

After -stop, your EC2 instance will stop and the EBS volume will be un-attached.


The source code is available at GitHub. Feel free to send pull requests for improvements!