CentOS Apache MySQL PHP

For a while now, I've been learning the hard way how a lot of configurations published on the Internet are much less effective than their authors purport. First we need to back up to a set of core principles in order to then find the configuration that will work for production servers.

Production Server Principles

  • As fast as possible,
  • As scalable as needed,
  • As secure as expected,
  • While being rock-solid stable
    • No process failure, memory leaks, or unexpected/unexplained rebooting

This generally does away with a lot of the recommendations, if one looks closely as the configurations being recommended. A little bit of testing shows up problems fairly quickly. Good log and performance monitoring goes a long way of informing on bad decisions.

Application Version Selection

  • CentOS latest stable version shipping (7.2), or any other mainstream RHEL derivative
  • Apache, latest stable version shipping for CentOS (2.4)
  • PHP, latest stable version, from the Remi repository (5.6)
    • The main reason for this choice has to do with built-in Opcache and some speed improvements, otherwise the latest vanilla CentOS/EPEL distribution would be the choice
    • PHP 7 is not yet supported by many WordPress Plugins, and so doesn't have required functionality
  • MySQL, latest stable version from Oracle
    • MariaDB is nice but they don't have a binary compatible version for 5.6, and their 10.x product is not nearly as popular. Also, the built-in Opcache in 5.6 is an advantage

Apache and PHP Configuration

With WordPress, a lot of the focus is on the Apache and PHP configuration. That is, how the web server (using SSL) establishes connections, reads requests, hands off appropriate elements to PHP, and returns requested content/functionality.

First, we really aren't going to get any additional speed out of MPM:Worker or MPM:Event. Therefore MPM:Prefork is what we want to rely on, and can best manage, given modest scalability/load requirements.

PHP Session Handling

WordPress does not use PHP Sessions, and plugins need not, therefore:

  • Eradicate plugins which use @session_start(); which includes (as per latest scan):

  • wp-affiliate-platform,

  • wp-spamshield,
  • woocommerce-amazon-s3-storage, and
  • php-compatibility-checker (which is only needed for testing, in any case)
cd /var/www/html
grep -r 'session_start'

Caching Configuration in WordPress

W3 Total Cache

General Settings

  • Page Cache, Disk: Enhanced
  • Minify (disabled)
  • Database Cache, Disk
  • Object Cache (disabled)
  • Browser Cache (disabled, we do this manually in httpd.conf)
  • CDN (disabled)
  • Use single network configuration file
  • Purge Policy: Posts page, Post page

Page Cache

  • Cache posts, SSL, Don't cache logged in
  • Prime page cache, 900, 10
  • Preload post cache upon publish

  • Rejected Cookies:

wptouch_switch_toggle
ap_id
cart_in_use
eMember_in_use
  • Never Cache the Following Pages
wp-.*\.php
index\.php
[a-z0-9_\-]*sitemap[a-z0-9_\-]*\.(xml|xsl|html?)(\.gz)?
favorites\.php
cart
checkout
shop
/shop*
  • Note: must include any changes to permalinks and the pages above

Database Cache

  • Don't cache for logged in
  • Ignore Query Stems
gdsr_
wp_rg_
_wp_session_
_wc_session_

Autoptimize

  • Optimize HTML, Keep HTML Comments
  • Optimize Javascript, aggregate inline JS
  • Optimize CSS, Remove Google Fonts
  • Save aggregated as static files = uncheck

SSL Configuration and Issues

There are a lot of SSL issues, in many cases the key is to ensure that the settings for Apache in general, mod_ssl in particular, default locations (directory paths inherited) and both http and https virtual hosts are not in conflict. Since later configuration directives override earlier ones, the fewer directives downstream, the better, especially as configuration files proliferate.

With LetsEncrypt, it is important that the /etc/httpd/sites-available/domain.com.conf file(s) have the same domains as are in the /etc/httpd/conf.d/ssl.conf file. While it may be possible to merge this info into a single or two files, the configuration may be best managed (and understood) as having three parts:

  • /etc/httpd/conf/httpd.conf - main configuration for Apache. Should include everything that can be inherited by all vhosts and ssl vhosts. Can include what is kept in .htaccess files
  • /etc/httpd/sites-available/domain.com.conf - this is one vhosts file. There may be others. For a single wordpress multi-site installation, only one file is actually needed (since all sites have the same root directory, and can share the same SSL certificate).
  • /etc/httpd/conf.d/ssl.conf - this is the one ssl vhosts file, and also is where other SSL directives are (after SSLEngine On)

SNI Certificates and Directives

Using SNI, it is important also to have a fallback to allow non-SNI clients to access a site, with the directive SSLStrictSNIVHostCheck.

Directory, Virtual Host Configuration

From the perspective of a single website which may use a single or multi-site WordPress installation, there need be only a single directory location, and a single host configuration. The main complications are that LetsEncrypt wants to use the /etc/https/sites-enabled directory to look for virtual host files (at least one), and also that SSL directives should be in *:443 listens and not *:80 listens, so those should be separate configurations.

This may mean (still to be determined) that only two configuration files are needed, the /etc/httpd/conf/httpd.conf and the /etc/httpd/sites-available/domain.com.conf file. A single certificate can be generated, and all additional domains can be set as domain aliases.

Note that for the SSL Cert, only the root server names, and the actual webserver fqdn are needed, rather than including various www. and other redirected domains, which can be handled with mod_rewrite without problems.

Security and Performance

Exotic Performance Tuning

Testing Tools

Code Cleanup

A good part of speed issues is the actual site code (php/js/css/html) and when it comes to WordPress, especially WordPress plugins, there are a lot of potential conflicts. Blocking JS and CSS is a big part of the problem, as well as removing all the default crap that is not needed (such as various webfonts).

Concatenated Installation and Configuration

Create user, grant permissions, generate SSH key

Add new user, password, grand sudo rights

useradd xyz && passwd xyz
SoMEthing!29characterzlongrandom12oujaodlf0asy
usermod -aG wheel xyz

Sign into that user, create .ssh directory

su - xyz
sudo ls -l
SoMEp4s5w0rd29characterzlongrandomSH1t
sudo mkdir /home/xyz/.ssh
sudo chown xyz:xyz /home/xyz/.ssh

Generate a set of keys on client computer (not server)

sudo ssh-keygen -t rsa -f ~/.ssh/xyz
sudo scp ~/.ssh/xyz.pub xyz@xyz.server.com:/home/xyz/.ssh/authorized_keys

Back at the server, set security on the key

sudo chmod 700 -R /home/xyz/.ssh && chmod 600 /home/xyz/.ssh/authorized_keys

Configure SSHD

Edit sshd_config file

sudo nano /etc/ssh/sshd_config

Remove password and root login in SSH

PasswordAuthentication no
PermitRootLogin no
~~

Restart sshd

~~
sudo systemctl restart sshd

Go ahead and test login with the new user

Configure EPEL and REMI repositories

sudo yum -y update
sudo yum -y install wget
sudo wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
sudo rpm -Uvh epel-release-latest-7*.rpm
wget https://rpms.remirepo.net/enterprise/remi-release-7.rpm
sudo rpm -Uvh remi-release-7.rpm
sudo yum -y install yum-utils
sudo yum-config-manager --enable remi-php56

Check on the repolist with yum repolist

Install and configure firewalld

firewalld is a firewall for CentOS that uses iptables commands, but not the iptables service. One advantage of FirewallD is that it can be altered at runtime, without needing to destry existing connections. See a comparison between firewalld and iptables.

sudo yum -y install firewalld firewall-config
sudo systemctl start firewalld
sudo firewall-cmd --permanent --add-service=ssh
sudo firewall-cmd --permanent --add-service=http
sudo firewall-cmd --permanent --add-service=https
sudo firewall-cmd --permanent --zone=public --add-port=10000/tcp
sudo firewall-cmd --reload
sudo systemctl enable firewalld

See Using firewalld on centos 7 and Introduction to firewalld on centos 7.

Install and configure Mosh

sudo yum -y install mosh

Create a services file to run mosh as a service

sudo nano /etc/firewalld/services/mosh.xml

Add the following to the mosh.xml file

<?xml version="1.0" encoding="utf-8"?>
<service>
  <short>MOSH</short>
<description>Mosh extends ssh for intermittent connect</description>
  <port protocol="udp" port="60001"/>
  <port protocol="udp" port="60002"/>
  <port protocol="udp" port="60003"/>
  <port protocol="udp" port="60004"/>
  <port protocol="udp" port="60005"/>
  <port protocol="udp" port="60006"/>
  <port protocol="udp" port="60007"/>
  <port protocol="udp" port="60008"/>
  <port protocol="udp" port="60009"/>
</service>

Reload, add, add permanent, and restart firewalld

sudo firewall-cmd --reload
sudo firewall-cmd --add-service=mosh
sudo firewall-cmd --add-service=mosh --permanent
sudo systemctl restart firewalld

Syntax for Mosh (on the client): mosh --ssh="ssh -v -i xyz" xyz@abc.classics.io

Remove postfix

sudo yum remove -y postfix

Disable avahi services

sudo systemctl disable avahi-daemon.socket avahi-daemon.service
sudo systemctl mask avahi-daemon.socket avahi-daemon.service
sudo systemctl stop avahi-daemon.socket avahi-daemon.service

Set Time, Hostname, DNS

sudo timedatectl set-timezone Asia/Tokyo
sudo hostnamectl set-hostname xyz.domain.com --static
sudo echo "nameserver 8.8.8.8" > /etc/resolv.conf
sudo echo "nameserver 8.8.4.4" > /etc/resolv.conf

Linode Longview - for Linode installations

Find out Distributor ID and Version

cat /etc/redhat-release

Edit the longview.repo config

sudo nano /etc/yum.repos.d/longview.repo

Append the DID and Ver in the baseurl

[longview]
name=Longview Repo
baseurl=https://yum-longview.linode.com/CentOS/7.3.1611/noarch/
enabled=1
gpgcheck=1

Get the Linode key and import that

sudo wget https://yum-longview.linode.com/linode.key
sudo rpm --import linode.key
sudo mkdir -p /etc/linode/

Install via that longview.key command for the specific monitor at Linode Longview

Should see the key in this file

sudo nano /etc/linode/longview.key

Fix the kdump error - for Linode installations

nano /etc/default/grub

Change crashkernel=auto to crashkernel=512M

grub2-mkconfig -o /boot/grub2/grub.cfg

Install and Configure Apache, SSL, PHP

sudo yum -y install httpd
sudo yum -y install php56 mod_fcgid fcgi
sudo yum -y install php-pecl-zendopcache

SSL Certs with LetsEncrypt

sudo yum install mod_ssl openssl

Edit the ssl.conf file

sudo nano /etc/httpd/conf.d/ssl.conf

/etc/httpd/conf.d/ssl.conf configuration

Listen *:443 https

SSLStrictSNIVHostCheck  off
SSLPassPhraseDialog     exec:/usr/libexec/httpd-ssl-pass-dialog
SSLSessionCache         shmcb:/run/httpd/sslcache(512000)
SSLSessionCacheTimeout  300

SSLRandomSeed startup file:/dev/urandom 256
SSLRandomSeed connect builtin

SSLCryptoDevice builtin

SSLUseStapling                   on
SSLStaplingResponderTimeout  5
SSLStaplingReturnResponderErrors off
SSLStaplingCache                 shmcb:/run/ocsp(128000)

SSLVerifyClient none
BrowserMatch "MSIE [2-5]" \
         nokeepalive ssl-unclean-shutdown \
         downgrade-1.0 force-response-1.0


  DocumentRoot "/var/www/html"
  ServerName host.domain.com
  ServerAlias host.domain.com

  SSLCertificateFile      /etc/letsencrypt/live/host.domain.com/cert.pem
  SSLCertificateKeyFile   /etc/letsencrypt/live/host.domain.com/privkey.pem
  SSLCertificateChainFile /etc/letsencrypt/live/host.domain.com/chain.pem

  SSLOptions +StdEnvVars

  ErrorLog logs/ssl_error_log
  TransferLog logs/ssl_access_log
  LogLevel warn
  CustomLog logs/ssl_request_log \
          "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"

  SSLEngine on
  Header always set Strict-Transport-Security "max-age=63072000; includeSubdomains; preload"

  SSLProtocol         all -SSLv3
  SSLCipherSuite      ALL:+HIGH:+TLSv1:!ADH:!EXP:!SSLv2:!MEDIUM:!LOW:!NULL:!aNULL
  SSLHonorCipherOrder on
  SSLCompression      off


Create Debian-Style Symlinked Virtual Hosts

sudo mkdir /etc/httpd/sites-available
sudo mkdir /etc/httpd/sites-enabled

Make a base named virtual host config for the server

sudo nano /etc/httpd/sites-available/domain.com.conf

Add a basic vhost entry


ServerName host.domain.com ServerAlias domain.com

Note that there can be fewer or more aspects of these vhost entries.

Symlink into /sites-enabled/

cd /etc/httpd/sites-enabled
ln -s /etc/httpd/sites-available/domain.com.conf

Edit httpd.conf

sudo nano /etc/httpd/conf/httpd.conf

Include this at the end of the .conf

IncludeOptional sites-enabled/*.conf

Install LetsEncrypt

Install git first

sudo yum install git

Clone a handy letsencrypt client from github

sudo git clone https://github.com/letsencrypt/letsencrypt /opt/letsencrypt

Generate an SSL Cert

cd /opt/letsencrypt
sudo ./letsencrypt-auto --apache -d host.domain.com

If there are errors, make sure the /etc/httpd/sites-available/domain.com.conf and the /etc/httpd/conf.d/ssl.conf files both have the same domain names and aliases. Otherwise can also try without using the -auto option.

Set path of SSL Certs for the main server

sudo nano /etc/httpd/conf.d/ssl.conf

Change the three entries for the certificate locations

SSLCertificateFile /etc/letsencrypt/live/host.domain.com/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/host.domain.com/privkey.pem
SSLCertificateChainFile /etc/letsencrypt/live/host.domain.com/chain.pem

Restart Apache

sudo systemctl stop httpd; systemctl start httpd

Inexhaustive List of Configuration Files

CentOS

  • sudo nano /etc/passwd - epass
  • sudo nano /etc/ssh/sshd_config - esshd

Apache, SSL

  • sudo nano /etc/httpd/conf/httpd.conf - eap
  • sudo nano /etc/httpd/sites-available/domain.com.conf - evhost
  • sudo nano /etc/httpd/conf.d/ssl.conf - essl
  • sudo nano /etc/httpd/conf.modules.d/00-mpm.conf - empm
  • sudo nano /etc/httpd/conf.modules.d/00-base.conf - ebase
  • stop httpd; start httpd - rap

PHP, Opcache

  • sudo nano /etc/php.ini -
  • sudo nano /etc/httpd/conf.d/php.conf -
  • sudo nano /etc/php.d/opcache.ini -
  • sudo nano /etc/httpd/conf.modules.d/10-fcgid.conf -
  • sudo nano /etc/httpd/conf.d/fcgid.conf -
  • sudo nano /etc/httpd/conf.modules.d/00-ssl.conf -

MySQL

  • sudo nano /etc/httpd/conf.d/phpMyAdmin.conf -

WordPress

  • sudo nano /var/www/html/wp-config.php -
  • sudo nano /var/www/html/.htaccess -