Web | Z-Car

Unable to renew using LetsEncrypt Certbot-Auto with CRON – Fail

By Jimmy August 1, 2017 Programming, Security, Web No Comments

Recently I had a fail when trying to automate the renewal of my LetsEncrypt SSL certificates using a CRON job, I experienced the following error when running “certbot-auto renew” :

Error: couldn’t get currently installed version for //.local/share/letsencrypt/bin/letsencrypt:
//.local/share/letsencrypt/lib64/python2.6/site-packages/cryptography/__init__.py:26: DeprecationWarning: Python 2.6 is no longer supported by the Python core team, please upgrade your Python. A future version of cryptography will drop support for Python 2.6
DeprecationWarning
An unexpected error occurred:
ContextualVersionConflict: (setuptools 0.9.8 (/.local/share/letsencrypt/lib/python2.6/site-packages), Requirement.parse(‘setuptools>=1.0′), set([‘certbot’]))

At first glance, this failure would appear to have something to do with Python 2.6 and the cryptography site-package. However, this actually just turns out to be a warning. The unexpected error below is what causes the issue. If you look closely at the error (couldn’t get currently installed version for //.local/share/letsencrypt/bin/letsencrypt) line and the line starting with ContextualVersionConflict, you will see that it is trying to access (/.local/share/letsencrypt/lib/python2.6/site-packages). In both cases, the issue is with //.local/, when running as a cronjob, this needs to point to /root/.local. There are a couple ways to fix this.

One, you can edit your crontab and edit HOME=/ to HOME=/root/
Two, you can create a wrapper script that changes your HOME directory before calling certbot-auto renew
Lastly, you can choose to do what I did and directly edit the certbot-auto script. I hard-coded the values for HOME and XDG_DATA_HOME. This has the disadvantage of getting wiped out if you ever update that script, however it also guarantees that no matter how that script is executed, it will always be pointing to the correct location.

To edit the script, add the following lines :
HOME=/root/
XDG_DATA_HOME=/root/.local/share

before the line that has :
VENV_NAME=”letsencrypt”

Leave a comment if this solved your problem, or if you are experiencing any other strange issues running certbot-auto from a cron job.

Installing LetsEncrypt Certbot on Amazon Linux Server

By Jimmy June 21, 2017 Blogging, Linux, Programming, Software, Web, Wordpress No Comments

The LetsEncrypt Certbot is an awesome tool for getting and maintaining SSL certificates for your sites. In most cases, it is extremely easy to setup and maintain. However, installing it on Amazon’s proprietary Linux server in EC2, it often fails. These are the steps I took to get it to install properly in my environment.

wget https://dl.eff.org/certbot-auto
chmod a+x ./certbot-auto
sudo yum install libffi-devel
sudo yum install openssl-devel
sudo chown -R ec2-user:ec2-user .well-known/
./certbot-auto –no-self-upgrade –no-bootstrap

After performing these commands, you should now be able to use Certbot as you would normally.

Certbot will place your completed SSL certs in the following paths.

Certificate: /etc/letsencrypt/live/YOUR_WEBSITE_HERE/cert.pem
Full Chain: /etc/letsencrypt/live/YOUR_WEBSITE_HERE/fullchain.pem
Private Key: /etc/letsencrypt/live/YOUR_WEBSITE_HERE/privkey.pem

To setup auto renew, you will need to add the following line to your crontab file. Make sure you are running as root when you do so using sudo su.

Adding this line will run the renewal process twice per day.

0 2,14 * * * /home/ec2-user/certbot-auto renew

If you have any issues, leave a message in the comments so we can help.

Some useful custom SpamAssassin rules and settings – modify your own

By Jimmy November 12, 2016 Linux, Programming, Security, Software, Web No Comments

I have a Linux server that I use for managing web sites, development work, etc. In addition, I use it to host a mail server for two domains that I have had since 1993. Because of their age, and the number of email addresses that were used on them over the years, they receive a LOT of Spam. In order to manage this huge volume of SPAM, I use SpamAssassin, and a ton of custom rules in Postfix to minimize the amount of spam that ultimately reached my inbox.

Here is a collection of score changes, and custom filters that I use. These change on a fairly regular basis, as the Spammers are always making changes. I will try and keep this updated fairly often.

If you have any questions on how to write a rule, or have a suggestion for a good one, leave a comment or email.

score RAZOR2_CHECK 5
score BAYES_999 1.0
score BAYES_00 -4
score T_FREEMAIL_DOC_PDF 2
score DIGEST_MULTIPLE 5
score MPART_ALT_DIFF 5
score RCVD_IN_MSPIKE_L5 4
score URIBL_BLACK 5
score URIBL_DBL_SPAM 5
score DCC_CHECK 5
score PYZOR_CHECK 5

mimeheader ZIP_ATTACHED Content-Type =~ /zip|xls|docm|doc/i
describe ZIP_ATTACHED email contains a zip file attachment
score ZIP_ATTACHED 4.5
header CUSTOM_PHP_ID_SPAM X-PHP-Originating-Script =~ /class.php/
score CUSTOM_PHP_ID_SPAM 5

header CUSTOM_UNQ_ID_SPAM X-MC-Unique =~ /randcase/
score CUSTOM_UNQ_ID_SPAM 5

header CUSTOM_UA_ID_SPAM User-Agent =~ /Mutt/
score CUSTOM_UA_ID_SPAM 5

rawbody CUSTOM_GMAIL_SPAM /style\=\”color\:\#245dc1\;text\-decoration\:none\;/
score CUSTOM_GMAIL_SPAM 4

rawbody CUSTOM_WHATSAP_SPAM /background\:\#d9d9d9\;font\-family\:arial\;font\-weight\:normal\;font\-size\:11px\;color\:\#808080\;/
score CUSTOM_WHATSAP_SPAM 4

rawbody CUSTOM_FEDEX_SPAM /style\=\”text\-decoration\:none\;color\:\#4d148c\;\” alt\=\”Privacy policy\” title\=\”Privacy policy/
score CUSTOM_FEDEX_SPAM 4

body CUSTOM_FARGO_SPAM /FARGO\, ND 58103/
score CUSTOM_FARGO_SPAM 4

rawbody CUSTOM_INLINE_IMAGE /src=”cid:/
score CUSTOM_INLINE_IMAGE 5.5

rawbody CUSTOM_TRACKING_CODE /img src=”(.*)\.us(.*)\?email/
score CUSTOM_TRACKING_CODE 5.5

rawbody CUSTOM_ENDS_IN_GUID /[A-Za-z0-9]{8}[A-Za-z0-9]{4}[A-Za-z0-9]{4}[A-Za-z0-9]{4}[A-Za-z0-9]{12}$/
score CUSTOM_ENDS_IN_GUID 7.5

rawbody CUSTOM_ALT_IN_GUID /alt=”[A-Za-z0-9]{8}[A-Za-z0-9]{4}[A-Za-z0-9]{4}[A-Za-z0-9]{4}[A-Za-z0-9]{12}”/
score CUSTOM_ALT_IN_GUID 0.5

Building a custom version of Apache 2.4 on CentOS – Support for socket.io proxying

By Jimmy May 12, 2016 Linux, PHP, Web No Comments

Last year I needed a way to create a specific Apache version that would support proper handling of proxying socket.io connections to a Node client. I had to hunt around, however here are some notes I created to get the version I needed built.

There are a lot of threads on the Internet that point to running Apache 2.4 on RHEL 6 as being a difficult setup. It’s actually quite easy, thanks to Apache’s wonderful packaging. Since Apache builds their source packages so they can easily be compiled into RPMs. (All of these steps were performed on a fresh installation of CentOS 6.6.)

First we need to install all of the tools for building RPMs and create the directory structure –

yum -y install rpm-build
mkdir -p ~/rpmbuild/{SOURCES,SPECS,BUILD,RPMS,SRPMS}

Let’s start by downloading the Apache Httpd sources and trying to compile –

cd ~/rpmbuild/SOURCES
wget http://www.gtlib.gatech.edu/pub/apache/httpd/httpd-2.4.4.tar.bz2

Now we can identify the missing dependencies and figure out how to continue –

# rpmbuild -tb httpd-2.4.4.tar.bz2 
error: Failed build dependencies:
    autoconf is needed by httpd-2.4.4-1.x86_64
    apr-devel >= 1.4.0 is needed by httpd-2.4.4-1.x86_64
    apr-util-devel >= 1.4.0 is needed by httpd-2.4.4-1.x86_64
    pcre-devel >= 5.0 is needed by httpd-2.4.4-1.x86_64
    openldap-devel is needed by httpd-2.4.4-1.x86_64
    lua-devel is needed by httpd-2.4.4-1.x86_64
    libxml2-devel is needed by httpd-2.4.4-1.x86_64
    distcache-devel is needed by httpd-2.4.4-1.x86_64

We have packages available for autoconf, pcre-devel, openldap-devel, lua-devel, and libxml2-devel. APR is included in RHEL and CentOS, but it’s unfortunately an old version, so we’ll have to recompile that too. distcache is often the problem people are reporting when installing Apache 2.4, but continue reading for a nice trick to make this easier.

Next, we’ll download the sources of all of the custom packages we need to compile for Apache (your versions may change) –

cd ~/rpmbuild/SOURCES
wget http://www.gtlib.gatech.edu/pub/apache/apr/apr-1.4.6.tar.bz2
wget http://www.gtlib.gatech.edu/pub/apache/apr/apr-util-1.5.2.tar.bz2

Each of these can now be easily used to create RPMs for installation. Let’s start with APR –

cd ~/rpmbuild/SOURCES
# Install apr dependencies
yum -y install autoconf libtool doxygen
rpmbuild -tb apr-1.4.6.tar.bz2
# Install our freshly build apr RPMs
rpm -ivh ~/rpmbuild/RPMS/x86_64/apr-1.4.6-1.x86_64.rpm ~/rpmbuild/RPMS/x86_64/apr-devel-1.4.6-1.x86_64.rpm
# Install apr-util dependencies
yum -y install expat-devel libuuid-devel db4-devel postgresql-devel mysql-devel freetds-devel unixODBC-devel openldap-devel nss-devel
# For some reason this has failed for me once or twice, but completed successfully the next time.
rpmbuild -tb apr-util-1.5.2.tar.bz2
rpm -ivh ~/rpmbuild/RPMS/x86_64/apr-util-1.5.2-1.x86_64.rpm ~/rpmbuild/RPMS/x86_64/apr-util-devel-1.5.2-1.x86_64.rpm

Installing distcache on RHEL can be a pain, but we can take advantage of Fedora’s SRPM to get us started –

cd ~/rpmbuild/SRPMS
wget http://www.gtlib.gatech.edu/pub/fedora.redhat/linux/releases/18/Fedora/source/SRPMS/d/distcache-1.4.5-23.src.rpm
rpmbuild --rebuild distcache-1.4.5-23.src.rpm
rpm -ivh ~/rpmbuild/RPMS/x86_64/distcache-1.4.5-23.x86_64.rpm ~/rpmbuild/RPMS/x86_64/distcache-devel-1.4.5-23.x86_64.rpm

Now that we have apr and distcache taken care of, the Apache compilation and install is quite easy –

cd ~/rpmbuild/SOURCES/
# Install remaining httpd dependencies
yum -y install pcre-devel lua-devel libxml2-devel
rpmbuild -tb httpd-2.4.4.tar.bz2

Now you’re ready to install httpd, but you’ll get one last error –

$ rpm -ivh ~/rpmbuild/RPMS/x86_64/httpd-2.4.4-1.x86_64.rpm 
    error: Failed dependencies:
        /etc/mime.types is needed by httpd-2.4.4-1.x86_64

Let’s find out which package provides that file –

$ yum whatprovides "/etc/mime.types"
Loaded plugins: fastestmirror, security
Loading mirror speeds from cached hostfile
 * base: ftp.osuosl.org
 * epel: ftp.osuosl.org
 * extras: ftp.osuosl.org
 * updates: ftp.usf.edu
mailcap-2.1.31-2.el6.noarch : Helper application and MIME type associations for file types
Repo        : base
Matched from:
Filename    : /etc/mime.types

We’re finally ready to install httpd –

yum -y install mailcap
rpm -ivh ~/rpmbuild/RPMS/x86_64/httpd-2.4.4-1.x86_64.rpm

Now you’re all ready to go with Apache 2.4! And since you’ve built proper RPMs, you’re also ready to deploy the packages out to all of your servers.

ProxyPass        /    ws://localhost:8080/
ProxyPassReverse /    ws://localhost:8080/
ProxyPass        /    wss://localhost:8080/
ProxyPassReverse /    wss://localhost:8080/

ProxyPass / http://localhost:8080/
ProxyPassReverse / http://localhost:8080/

This works

RewriteEngine on

    RewriteCond %{QUERY_STRING} transport=polling
    RewriteRule /(.*)$ http://localhost:$port/$1 [P]

    ProxyRequests off
    ProxyPass /socket.io-client/ ws://localhost:$port/socket.io-client/
    ProxyPassReverse /socket.io-client/ ws://localhost:$port/socket.io-client/

    ProxyPass / http://localhost:$port/
    ProxyPassReverse / http://localhost:$port/

Node.js – fs.open / fs.writeFile, mode, file permissions, umask, and how they interact

By Jimmy April 9, 2014 Node.js, Programming, Software, Web No Comments

While working with Node.js recently, I ran into a situation with the mode parameter used in several functions; fs.open, fs.appendFile, fs.writeFile, etc. When trying to create a new file and specifying a specific mode parameter to allow user and group writes, I found that the file would get created, however would have the wrong permissions. After much experimenting, I realized that the Linux server’s umask was getting applied, and preventing the creation of a file with the permissions I wanted.

The work around turned out to be fairly simple, although it does create a slight security hole, and should only be used in cases where you can control access to the application.

I simply wrapped the file create calls with the following code :

oldmask = process.umask(newmask);
fs.open(path, flags, [mode], callback)
process.umask(oldmask);

This only needs be performed when the open (or other method) will create a new file. Let me know if you have experienced a similar issue, and if you have found other solutions to resolve.

How to escape Mongo keys using Node.js in a Flash

By Jimmy April 3, 2014 Mongo DB, Node.js, Programming, Software, Web No Comments

Dale is Flash Gordon’s constant companion in his adventures, as well as his one true love.

Don’t be scared, dealing with Mongo is not that difficult, especially with some help from Flash. While some people know of Mongo as a fictional planet where the comic and movie serials of Flash Gordon takes place, ruled by a tyrant named Ming the Merciless, who governs with an iron hand, others are familiar with Mongo as a cross-platform document-oriented database system.

Many first-time users of Mongo quickly find out that it does not allow insertion of keys with a dot (.) or dollar sign ($). While understandable, this would seem to defeat the purpose of having a free-form document that is not rigidly defined. Luckily, there is an easy way to address this issue in Node.js. Below is a simple code snippet that I wrote which will escape the dot (.) or dollar sign ($) in any of your keys.

Just pass the document into the function, and your object will be escaped. The dot (.) will be replaced with _dot_ and the ampersand (&) will be replaced with _amp_. If required, you can change these to whatever you need. If you need to unescape, you can easily swap the key.replace params. I will leave this as an exercise to be completed by the reader…

function escapeKeys(obj) {
    if (!(Boolean(obj) && typeof obj == 'object'
      && Object.keys(obj).length > 0)) {
        return false;
    }
    Object.keys(obj).forEach(function(key) {
        if (typeof(obj[key]) == 'object') {
            escapeKeys(obj[key]);
        } else {
            if (key.indexOf('.') !== -1) {
                var newkey = key.replace(/\./g, '_dot_');
                obj[newkey] = obj[key];
                delete obj[key];
            }
            if (key.indexOf('$') !== -1) {
                var newkey = key.replace(/\$/g, '_amp_');
                obj[newkey] = obj[key];
                delete obj[key];
            }

        }
    });
    return true;
}

Syk is ruled by the witch-queen Azura, who is feared by everyone on Mongo.

MongoDB is an open-source document database, and the leading NoSQL database.

« Previous