Web | Z-Car

Installing LetsEncrypt Certbot on Amazon Linux Server

By Jimmy June 21, 2017 Blogging, Linux, Programming, Software, Web, Wordpress No Comments

The LetsEncrypt Certbot is an awesome tool for getting and maintaining SSL certificates for your sites. In most cases, it is extremely easy to setup and maintain. However, installing it on Amazon’s proprietary Linux server in EC2, it often fails. These are the steps I took to get it to install properly in my environment.

wget https://dl.eff.org/certbot-auto
chmod a+x ./certbot-auto
sudo yum install libffi-devel
sudo yum install openssl-devel
sudo chown -R ec2-user:ec2-user .well-known/
./certbot-auto –no-self-upgrade –no-bootstrap

After performing these commands, you should now be able to use Certbot as you would normally.

Certbot will place your completed SSL certs in the following paths.

Certificate: /etc/letsencrypt/live/YOUR_WEBSITE_HERE/cert.pem
Full Chain: /etc/letsencrypt/live/YOUR_WEBSITE_HERE/fullchain.pem
Private Key: /etc/letsencrypt/live/YOUR_WEBSITE_HERE/privkey.pem

To setup auto renew, you will need to add the following line to your crontab file. Make sure you are running as root when you do so using sudo su.

Adding this line will run the renewal process twice per day.

0 2,14 * * * /home/ec2-user/certbot-auto renew

If you have any issues, leave a message in the comments so we can help.

Some useful custom SpamAssassin rules and settings – modify your own

By Jimmy November 12, 2016 Linux, Programming, Security, Software, Web No Comments

I have a Linux server that I use for managing web sites, development work, etc. In addition, I use it to host a mail server for two domains that I have had since 1993. Because of their age, and the number of email addresses that were used on them over the years, they receive a LOT of Spam. In order to manage this huge volume of SPAM, I use SpamAssassin, and a ton of custom rules in Postfix to minimize the amount of spam that ultimately reached my inbox.

Here is a collection of score changes, and custom filters that I use. These change on a fairly regular basis, as the Spammers are always making changes. I will try and keep this updated fairly often.

If you have any questions on how to write a rule, or have a suggestion for a good one, leave a comment or email.

score RAZOR2_CHECK 5
score BAYES_999 1.0
score BAYES_00 -4
score T_FREEMAIL_DOC_PDF 2
score DIGEST_MULTIPLE 5
score MPART_ALT_DIFF 5
score RCVD_IN_MSPIKE_L5 4
score URIBL_BLACK 5
score URIBL_DBL_SPAM 5
score DCC_CHECK 5
score PYZOR_CHECK 5

mimeheader ZIP_ATTACHED Content-Type =~ /zip|xls|docm|doc/i
describe ZIP_ATTACHED email contains a zip file attachment
score ZIP_ATTACHED 4.5
header CUSTOM_PHP_ID_SPAM X-PHP-Originating-Script =~ /class.php/
score CUSTOM_PHP_ID_SPAM 5

header CUSTOM_UNQ_ID_SPAM X-MC-Unique =~ /randcase/
score CUSTOM_UNQ_ID_SPAM 5

header CUSTOM_UA_ID_SPAM User-Agent =~ /Mutt/
score CUSTOM_UA_ID_SPAM 5

rawbody CUSTOM_GMAIL_SPAM /style\=\”color\:\#245dc1\;text\-decoration\:none\;/
score CUSTOM_GMAIL_SPAM 4

rawbody CUSTOM_WHATSAP_SPAM /background\:\#d9d9d9\;font\-family\:arial\;font\-weight\:normal\;font\-size\:11px\;color\:\#808080\;/
score CUSTOM_WHATSAP_SPAM 4

rawbody CUSTOM_FEDEX_SPAM /style\=\”text\-decoration\:none\;color\:\#4d148c\;\” alt\=\”Privacy policy\” title\=\”Privacy policy/
score CUSTOM_FEDEX_SPAM 4

body CUSTOM_FARGO_SPAM /FARGO\, ND 58103/
score CUSTOM_FARGO_SPAM 4

rawbody CUSTOM_INLINE_IMAGE /src=”cid:/
score CUSTOM_INLINE_IMAGE 5.5

rawbody CUSTOM_TRACKING_CODE /img src=”(.*)\.us(.*)\?email/
score CUSTOM_TRACKING_CODE 5.5

rawbody CUSTOM_ENDS_IN_GUID /[A-Za-z0-9]{8}[A-Za-z0-9]{4}[A-Za-z0-9]{4}[A-Za-z0-9]{4}[A-Za-z0-9]{12}$/
score CUSTOM_ENDS_IN_GUID 7.5

rawbody CUSTOM_ALT_IN_GUID /alt=”[A-Za-z0-9]{8}[A-Za-z0-9]{4}[A-Za-z0-9]{4}[A-Za-z0-9]{4}[A-Za-z0-9]{12}”/
score CUSTOM_ALT_IN_GUID 0.5

Building a custom version of Apache 2.4 on CentOS – Support for socket.io proxying

By Jimmy May 12, 2016 Linux, PHP, Web No Comments

Last year I needed a way to create a specific Apache version that would support proper handling of proxying socket.io connections to a Node client. I had to hunt around, however here are some notes I created to get the version I needed built.

There are a lot of threads on the Internet that point to running Apache 2.4 on RHEL 6 as being a difficult setup. It’s actually quite easy, thanks to Apache’s wonderful packaging. Since Apache builds their source packages so they can easily be compiled into RPMs. (All of these steps were performed on a fresh installation of CentOS 6.6.)

First we need to install all of the tools for building RPMs and create the directory structure –

yum -y install rpm-build
mkdir -p ~/rpmbuild/{SOURCES,SPECS,BUILD,RPMS,SRPMS}

Let’s start by downloading the Apache Httpd sources and trying to compile –

cd ~/rpmbuild/SOURCES
wget http://www.gtlib.gatech.edu/pub/apache/httpd/httpd-2.4.4.tar.bz2

Now we can identify the missing dependencies and figure out how to continue –

# rpmbuild -tb httpd-2.4.4.tar.bz2 
error: Failed build dependencies:
    autoconf is needed by httpd-2.4.4-1.x86_64
    apr-devel >= 1.4.0 is needed by httpd-2.4.4-1.x86_64
    apr-util-devel >= 1.4.0 is needed by httpd-2.4.4-1.x86_64
    pcre-devel >= 5.0 is needed by httpd-2.4.4-1.x86_64
    openldap-devel is needed by httpd-2.4.4-1.x86_64
    lua-devel is needed by httpd-2.4.4-1.x86_64
    libxml2-devel is needed by httpd-2.4.4-1.x86_64
    distcache-devel is needed by httpd-2.4.4-1.x86_64

We have packages available for autoconf, pcre-devel, openldap-devel, lua-devel, and libxml2-devel. APR is included in RHEL and CentOS, but it’s unfortunately an old version, so we’ll have to recompile that too. distcache is often the problem people are reporting when installing Apache 2.4, but continue reading for a nice trick to make this easier.

Next, we’ll download the sources of all of the custom packages we need to compile for Apache (your versions may change) –

cd ~/rpmbuild/SOURCES
wget http://www.gtlib.gatech.edu/pub/apache/apr/apr-1.4.6.tar.bz2
wget http://www.gtlib.gatech.edu/pub/apache/apr/apr-util-1.5.2.tar.bz2

Each of these can now be easily used to create RPMs for installation. Let’s start with APR –

cd ~/rpmbuild/SOURCES
# Install apr dependencies
yum -y install autoconf libtool doxygen
rpmbuild -tb apr-1.4.6.tar.bz2
# Install our freshly build apr RPMs
rpm -ivh ~/rpmbuild/RPMS/x86_64/apr-1.4.6-1.x86_64.rpm ~/rpmbuild/RPMS/x86_64/apr-devel-1.4.6-1.x86_64.rpm
# Install apr-util dependencies
yum -y install expat-devel libuuid-devel db4-devel postgresql-devel mysql-devel freetds-devel unixODBC-devel openldap-devel nss-devel
# For some reason this has failed for me once or twice, but completed successfully the next time.
rpmbuild -tb apr-util-1.5.2.tar.bz2
rpm -ivh ~/rpmbuild/RPMS/x86_64/apr-util-1.5.2-1.x86_64.rpm ~/rpmbuild/RPMS/x86_64/apr-util-devel-1.5.2-1.x86_64.rpm

Installing distcache on RHEL can be a pain, but we can take advantage of Fedora’s SRPM to get us started –

cd ~/rpmbuild/SRPMS
wget http://www.gtlib.gatech.edu/pub/fedora.redhat/linux/releases/18/Fedora/source/SRPMS/d/distcache-1.4.5-23.src.rpm
rpmbuild --rebuild distcache-1.4.5-23.src.rpm
rpm -ivh ~/rpmbuild/RPMS/x86_64/distcache-1.4.5-23.x86_64.rpm ~/rpmbuild/RPMS/x86_64/distcache-devel-1.4.5-23.x86_64.rpm

Now that we have apr and distcache taken care of, the Apache compilation and install is quite easy –

cd ~/rpmbuild/SOURCES/
# Install remaining httpd dependencies
yum -y install pcre-devel lua-devel libxml2-devel
rpmbuild -tb httpd-2.4.4.tar.bz2

Now you’re ready to install httpd, but you’ll get one last error –

$ rpm -ivh ~/rpmbuild/RPMS/x86_64/httpd-2.4.4-1.x86_64.rpm 
    error: Failed dependencies:
        /etc/mime.types is needed by httpd-2.4.4-1.x86_64

Let’s find out which package provides that file –

$ yum whatprovides "/etc/mime.types"
Loaded plugins: fastestmirror, security
Loading mirror speeds from cached hostfile
 * base: ftp.osuosl.org
 * epel: ftp.osuosl.org
 * extras: ftp.osuosl.org
 * updates: ftp.usf.edu
mailcap-2.1.31-2.el6.noarch : Helper application and MIME type associations for file types
Repo        : base
Matched from:
Filename    : /etc/mime.types

We’re finally ready to install httpd –

yum -y install mailcap
rpm -ivh ~/rpmbuild/RPMS/x86_64/httpd-2.4.4-1.x86_64.rpm

Now you’re all ready to go with Apache 2.4! And since you’ve built proper RPMs, you’re also ready to deploy the packages out to all of your servers.

ProxyPass        /    ws://localhost:8080/
ProxyPassReverse /    ws://localhost:8080/
ProxyPass        /    wss://localhost:8080/
ProxyPassReverse /    wss://localhost:8080/

ProxyPass / http://localhost:8080/
ProxyPassReverse / http://localhost:8080/

This works

RewriteEngine on

    RewriteCond %{QUERY_STRING} transport=polling
    RewriteRule /(.*)$ http://localhost:$port/$1 [P]

    ProxyRequests off
    ProxyPass /socket.io-client/ ws://localhost:$port/socket.io-client/
    ProxyPassReverse /socket.io-client/ ws://localhost:$port/socket.io-client/

    ProxyPass / http://localhost:$port/
    ProxyPassReverse / http://localhost:$port/

Node.js – fs.open / fs.writeFile, mode, file permissions, umask, and how they interact

By Jimmy April 9, 2014 Node.js, Programming, Software, Web No Comments

While working with Node.js recently, I ran into a situation with the mode parameter used in several functions; fs.open, fs.appendFile, fs.writeFile, etc. When trying to create a new file and specifying a specific mode parameter to allow user and group writes, I found that the file would get created, however would have the wrong permissions. After much experimenting, I realized that the Linux server’s umask was getting applied, and preventing the creation of a file with the permissions I wanted.

The work around turned out to be fairly simple, although it does create a slight security hole, and should only be used in cases where you can control access to the application.

I simply wrapped the file create calls with the following code :

oldmask = process.umask(newmask);
fs.open(path, flags, [mode], callback)
process.umask(oldmask);

This only needs be performed when the open (or other method) will create a new file. Let me know if you have experienced a similar issue, and if you have found other solutions to resolve.

How to escape Mongo keys using Node.js in a Flash

By Jimmy April 3, 2014 Mongo DB, Node.js, Programming, Software, Web No Comments

Dale is Flash Gordon’s constant companion in his adventures, as well as his one true love.

Don’t be scared, dealing with Mongo is not that difficult, especially with some help from Flash. While some people know of Mongo as a fictional planet where the comic and movie serials of Flash Gordon takes place, ruled by a tyrant named Ming the Merciless, who governs with an iron hand, others are familiar with Mongo as a cross-platform document-oriented database system.

Many first-time users of Mongo quickly find out that it does not allow insertion of keys with a dot (.) or dollar sign ($). While understandable, this would seem to defeat the purpose of having a free-form document that is not rigidly defined. Luckily, there is an easy way to address this issue in Node.js. Below is a simple code snippet that I wrote which will escape the dot (.) or dollar sign ($) in any of your keys.

Just pass the document into the function, and your object will be escaped. The dot (.) will be replaced with _dot_ and the ampersand (&) will be replaced with _amp_. If required, you can change these to whatever you need. If you need to unescape, you can easily swap the key.replace params. I will leave this as an exercise to be completed by the reader…

function escapeKeys(obj) {
    if (!(Boolean(obj) && typeof obj == 'object'
      && Object.keys(obj).length > 0)) {
        return false;
    }
    Object.keys(obj).forEach(function(key) {
        if (typeof(obj[key]) == 'object') {
            escapeKeys(obj[key]);
        } else {
            if (key.indexOf('.') !== -1) {
                var newkey = key.replace(/\./g, '_dot_');
                obj[newkey] = obj[key];
                delete obj[key];
            }
            if (key.indexOf('$') !== -1) {
                var newkey = key.replace(/\$/g, '_amp_');
                obj[newkey] = obj[key];
                delete obj[key];
            }

        }
    });
    return true;
}

Syk is ruled by the witch-queen Azura, who is feared by everyone on Mongo.

MongoDB is an open-source document database, and the leading NoSQL database.

Google Chrome’s new False Start “Feature”

By Jimmy December 10, 2010 Chrome, Software, Web 15 Comments

In the past few weeks, millions of Google Chrome users have started to experience issues accessing isolated web sites. These users are typically accessing sites using HTTPS, and have often successfully accessed these sites in the past, but no longer can. Accessing these sites using Internet Explorer or FireFox are successful. When attempting to access the site, Chrome will just spin forever attempting to load the page, or will display an Error 101 (net::ERR_CONNECTION_RESET): Unknown error.

So, what is happening?

Google Chrome now has a “feature” called False Start that is designed to speed up secure communications over SSL and TLS. False Start eliminates one of the round-trip messages needed to set up a secure channel between a Web browser and Web server. While this is a nice little improvement to speed up sites that use HTTPS, many web sites do not yet support the ability to handle False Start, and this is why the pages will not load. Chrome has a built-in set of sites that it knows does not support False Start, and Chrome disables False Start when communicating with them. However, it is clear that Google appears to underestimate the number of sites this issue is affecting. In particular, we are getting reports from lots of people who use private internal web applications that can no longer work with Chrome. Even when a web server (Apache, IIS, Tomcat) is updated, it seems as if many load balancers are still running very old versions of software. If they have not been updated recently, and SSL/TLS negotiation is handled at the load balancer level, you will have issues. A10 Networks only recently updated their code, and it is unclear what version of F5 BigIP supports False Start.

What can you do?

if you control the servers, make sure your server and load balancer version are up to date. Complain to vendors who are not yet supporting False Start. If you are using Chrome to access a web site that you do not control, you can use Chrome command line options to allow Chrome to access offending websites.

First Option is –use-system-ssl. This forces Chrome to use your systems SSL library (Windows-SCHANNEL) rather than Chrome’s built-in NSS.

You must specify –use-system-ssl in the command line, without extra spaces inside. Your shortcut should look like : “C:\…blahblah…\chrome.exe” –use-system-ssl

Second Option is –disable-ssl-false-start. This forces Chrome to not use False Start, but still use internal NSS stack.

You must specify –disable-ssl-false-start in the command line, without extra spaces inside. Your shortcut should look like : “C:\…blahblah…\chrome.exe” –disable-ssl-false-start

Leave me a message if you use Chrome and have been bitten by this “feature”