alanthing

Linux and other things

Never Touch Your Local /etc/hosts File in OS X Again

This post originally featured on the Echo & Co. blog.

In each of my posts on setting up a local development environment on OS X, it’s mentioned that you need to add your website’s domain, even though it’s local, in your /etc/hosts file. My preferred way to edit the hosts file on OS X is using Gas Mask. If you wanted to create the local virtual host projectx.dev, you would add the line 127.0.0.1 projectx.dev in /etc/hosts or with Gas Mask, and then use that same value in either ServerName in Apache or server_name in Nginx. This can be tedious for adding new sites. Luckily there’s a way to set this up once and then never have to edit your hosts file again for adding new local virtual hosts.

You’ll need a copy of dnsmasq, and I find this is most easily installed via Homebrew. If you haven’t already, grab either Xcode or Xcode Command Line Tools and install Homebrew.

The steps below will install dnsmasq from Homebrew, configure dnsmasq to return the IP address ‘127.0.0.1’ for all requests to the fake top-level-domain “.dev,” start dnsmasq on boot (don’t worry, it’s an extremely light-weight process), and configure OS X to use dnsmasq for queries ending in “.dev.”

1
2
3
4
5
6
7
brew install dnsmasq
mkdir -pv $(brew --prefix)/etc/
echo 'address=/.dev/127.0.0.1' > $(brew --prefix)/etc/dnsmasq.conf
sudo cp -v $(brew --prefix dnsmasq)/homebrew.mxcl.dnsmasq.plist /Library/LaunchDaemons
sudo launchctl load -w /Library/LaunchDaemons/homebrew.mxcl.dnsmasq.plist
sudo mkdir -v /etc/resolver
sudo bash -c 'echo "nameserver 127.0.0.1" > /etc/resolver/dev'

That’s it! You can run scutil --dns to show all of your current resolvers, and you should see that all requests for a domain ending in .dev will go to the DNS server at 127.0.0.1:

1
2
3
resolver #9
  domain   : dev
  nameserver[0] : 127.0.0.1

If you ping any domain that ends in .dev, you’ll get the IP address 127.0.0.1 back as a result:

1
2
3
$ ping -c 1 thereisnowaythisisarealdomain.dev
PING thereisnowaythisisarealdomain.dev (127.0.0.1): 56 data bytes
64 bytes from 127.0.0.1: icmp_seq=0 ttl=64 time=0.057 ms

Note that if you’re using Mountain Lion you may need to reboot before the /etc/resolver settings take effect globally. I was able to get pings to work right away but Chrome would not resolve properly until I restarted.

Now you can set up a new virtual host with anydomain.dev, and it’ll be available as soon as you reload Apache or Nginx! You could extend this by enabling Mass Virtual Hosting for Apache or similar with Nginx, though both require consistent layout of directories. Have fun configuring less stuff on your system!

Redirect All Port 80 Requests to Port 8080

This post originally featured on the Echo & Co. blog.

In my previous post, I walked through how to set up a local environment using Nginx running on port 8080 so as to avoid running anything as root or with sudo. Something that I’ve found incredibly annoying is when I forget to specify the port I get an error in my browser, or Chrome might even suggest something based on a search term. It’s fairly easy though to configure Apache to route everything to another port.

Create the file /etc/apache2/other/port8080-redirect.conf as root. It’s probably easiest to hop into Terminal and use nano:

1
sudo nano -w /etc/apache2/other/port8080-redirect.conf

Enter the following lines and save and exit the editor.

1
2
3
4
5
6
<VirtualHost _default_:80>
  DocumentRoot /Library/WebServer/Documents
  RewriteEngine On
  # Redirect all requests to the local Apache server to port 8080
  RewriteRule ^.*$ http://%{HTTP_HOST}:8080%{REQUEST_URI}
</VirtualHost>

Even though the DocumentRoot doesn’t appear to be necessary, I wasn’t able to get this working without it.

Go into System Preferences, open the Sharing preference pane, and check the box for Web Sharing. If you have trouble getting the check to “stick,” you can run the following in Terminal: sudo apachectl restart

Now if you go in your browser and hit a domain you have defined in /etc/hosts, like http://drupal.local/**, you’ll automatically be redirected to http://drupal.local:8080/**

OS X 10.7 Lion Development: Nginx, PHP, MariaDB With Homebrew

This post originally featured on the Echo & Co. blog.

Nginx is quickly becoming a popular, low resource alternative to Apache for many websites. This doesn’t come without challenges, such as using PHP as CGI due to not having mod_php available. Nginx also does not use any Apache configuration rules, nor does it use .htaccess or anything like it, so it requires additional configuration regardless of the web application being deployed. A big help in getting Nginx started with Drupal is António P. P. Almeida’s drupal-with-nginx configuration, which makes it fairly simple to deploy in Linux. But what about local development on OS X? Read on to learn get all of the required components set up for your system, as well as the modifications necessary to get drupal-with-nginx set up on OS X.

Prerequisites

We’ll be building all of our packages with Homebrew, which is, in my opinion, one of the best ways to easily add lot’s of great open-source software on OS X. Homebrew requires that you have a compiler, so you can either install the huge Xcode package, or I would recommend Apple’s Xcode Command Line Tools which is a much smaller download and officially supported by Homebrew.

Once you have either Xcode or Xcode Command Line Tools installed, install Homebrew.

Note that for all commands below that are starting with a $, the dollar sign is showing a command-line prompt in Terminal, and you should not actually type it as part of the commands. I also make heavy use of $(brew --prefix) to make these instructions persist passed current Homebrew formula versions, and hopefully also for an installation with Homebrew in a path other than /usr/local, though I have not tested it.

Also note that many times in this post you will see /n; make sure you type those or include them with your copy and paste, they are not CMS errors :)

Database: MariaDB

I’ve covered before why I like MariaDB, but you could easily swap this out with MySQL if you would rather. We’ll start by installing MariaDB with Homebrew.

1
2
3
4
5
6
7
8
9
brew install mariadb pidof (note: OS X may ask you if you want to install 'javac')
unset TMPDIR
mysql_install_db --verbose --user=`whoami` --basedir="$(brew --prefix mariadb)" --datadir=$(brew --prefix)/var/mysql --tmpdir=/tmp
cp $(brew --prefix mariadb)/share/mysql/my-small.cnf $(brew --prefix mariadb)/my.cnf
sed -i "" 's/max_allowed_packet = 1.*M/max_allowed_packet = 2G/g' $(brew --prefix mariadb)/my.cnf
[ ! -d ~/Library/LaunchAgents ] && mkdir ~/Library/LaunchAgents
[ -f $(brew --prefix mariadb)/homebrew.mxcl.mariadb.plist ] && cp -v $(brew --prefix mariadb)/homebrew.mxcl.mariadb.plist ~/Library/LaunchAgents/
[ -f ~/Library/LaunchAgents/homebrew.mxcl.mariadb.plist ] && launchctl load -w ~/Library/LaunchAgents/homebrew.mxcl.mariadb.plist
$(brew --prefix mariadb)/bin/mysql_secure_installation

Note: you could alternatively run: $(brew --prefix mariadb)/bin/mysqladmin -u root password 'new-password' instead of mysql_secure_installation

PHP

OS X comes with PHP installed, but it doesn’t come with PHP-FPM. While it’s likely possible to run PHP as FastCGI with the built-in OS X, I prefer to install PHP with Homebrew since we’re using Homebrew for everything else, and it keeps everything self-contained in Homebrew’s root (defaults to /usr/local). Note that the brew tap command requires Homebrew 0.9 or greater.

1
2
3
4
5
6
7
8
brew tap josegonzalez/php
brew install php --with-mariadb --with-suhosin --with-fpm
mkdir -v $(brew --prefix php)/var/log
cp -v $(brew --prefix)/etc/php-fpm.conf.default $(brew --prefix)/etc/php-fpm.conf
sed -i '' 's|;\(daemonize[[:space:]]*=[[:space:]]*\)yes|\1no|g' $(brew --prefix)/etc/php-fpm.conf
[ ! -d ~/Library/LaunchAgents ] && mkdir ~/Library/LaunchAgents
[ -f $(brew --prefix php)/org.php-fpm.plist ] && cp -v $(brew --prefix php)/org.php-fpm.plist ~/Library/LaunchAgents/
[ -f ~/Library/LaunchAgents/org.php-fpm.plist ] && launchctl load -w ~/Library/LaunchAgents/org.php-fpm.plist

I would recommend the following settings for full compatibility with drupal-with-nginx, and to set the time zone to silence a lot of PHP warnings:

1
2
3
4
sed -i '' 's|;\(pm.status_path[[:space:]]*=[[:space:]]*/\)\(status\)|\1fpm-\2|g' $(brew --prefix)/etc/php-fpm.conf
sed -i '' 's|;\(ping.path[[:space:]]*=[[:space:]]*/ping\)|\1|g' $(brew --prefix)/etc/php-fpm.conf
sed -i '' 's|;\(ping.response[[:space:]]*=[[:space:]]*pong\)|\1|g' $(brew --prefix)/etc/php-fpm.conf
sed -i '' "s|;\(date\.timezone[[:space:]]*=\).*|\1 $(php -d 'error_reporting=' -r 'echo date("e", time());')|g" $(brew --prefix)/etc/php.ini

By default, PHP-FPM runs on a socket, which means that connections to PHP-FPM will require using TCP. You also have the option to use Unix sockets, which means slightly less overhead in PHP-FPM connections. Note that the drupal-with-nginx repository is set up for TCP by default, though if you choose to run the following command I will tell you how to use Unix sockets with Nginx.

1
sed -i '' 's|\(listen[[:space:]]*=[[:space:]]*\)127.0.0.1:9000|\1var/www.sock|g' $(brew --prefix)/etc/php-fpm.conf

Optional: PHP Extensions

The third-party Homebrew keg that we “tapped” into also provides easy formulas for PHP extensions. None of these are required to run Nginx and Drupal locally. You may also note that uploadprogress does not work with anything but mod_php, but by installing it now you could theoretically use the same PHP installation with Apache if you wanted and already have it ready to go. Feel free to omit it, or any of these below, though I would at least recommend APC for performance reasons.

1
2
3
4
5
6
7
8
9
10
11
brew install uploadprogress-php
echo -e "\n[uploadprogress]\nextension=\"$(brew --prefix uploadprogress-php)/uploadprogress.so\"" >> $(brew --prefix)/etc/php.ini
brew install apc-php
echo -e "\n[apc]\nextension=\"$(brew --prefix apc-php)/apc.so\"\napc.enabled=1 \napc.shm_segments=1 \napc.shm_size=64M \napc.ttl=7200 \napc.user_ttl=7200 \napc.num_files_hint=1024 \napc.mmap_file_mask=/tmp/apc.XXXXXX \napc.enable_cli=1" >> $(brew --prefix)/php.ini
brew install memcache-php
echo -e "\n[memcache]\nextension=\"$(brew --prefix memcache-php)/memcache.so\"" >> $(brew --prefix)/etc/php.ini
echo -e "memcache.hash_strategy=\"consistent\"" >> $(brew --prefix)/etc/php.ini
brew install xdebug-php
echo -e "\n[xdebug]\nzend_extension=\"$(brew --prefix xdebug-php)/xdebug.so\"" >> $(brew --prefix)/etc/php.ini
brew install xhprof-php
echo -e "\n[xhprof]\nextension=\"$(brew --prefix xhprof-php)/xhprof.so\"" >> $(brew --prefix)/etc/php.ini

Once you’ve finished configuring PHP, you can reload the settings for PHP-FPM (or, you could find the pid of the first php-fpm process and send it the SIGUSR2 signal; this is easier):

1
[ -f ~/Library/LaunchAgents/org.php-fpm.plist ] && launchctl unload -w ~/Library/LaunchAgents/org.php-fpm.plist && launchctl load -w ~/Library/LaunchAgents/org.php-fpm.plist

Nginx

After compiling MariaDB and PHP, you’re probably not too excited about compiling another application. Luckily, Nginx is a faily quick build, at least compared to MariaDB and PHP. We’ll include some build options not on by default since they add features references in drupal-with-nginx, and we’ll also add some 3rd party extensions as well. We’ll start by grabbing those extensions:

1
2
curl -s -L -o /tmp/nginx-upload-progress.tar.gz https://github.com/masterzen/nginx-upload-progress-module/tarball/v0.9.0 && mkdir /tmp/nginx-upload-progress && tar zxpf /tmp/nginx-upload-progress.tar.gz --strip-components 1 -C /tmp/nginx-upload-progress && rm /tmp/nginx-upload-progress.tar.gz
curl -s -L -o /tmp/nginx-fair.tar.gz http://github.com/gnosek/nginx-upstream-fair/tarball/master && mkdir /tmp/nginx-fair && tar zxpf /tmp/nginx-fair.tar.gz --strip-components 1 -C /tmp/nginx-fair && rm /tmp/nginx-fair.tar.gz

The next section is one giant line of sed regex that will edit the Homebrew formula for nginx to add the additional compile options that we need. Make sure it all gets entered as one line (yes, you should use copy and paste here)!

1
sed -i '-default' 's/\([[:space:]]*\['\''--\)\(with-webdav\)\('\'',[[:space:]]*"\)\(Compile with support for WebDAV module\)\("\]\)/\1\2\3\4\5,%\1with-realip\3Compile with support for RealIP module\5,%\1with-gzip_static\3Compile with support for Gzip Static module\5,%\1with-uploadprogress\3Compile with support for Upload Progress module\5,%\1with-fair\3Compile with support for Fair module\5,%\1with-mp4\3Compile with support for MP4 module\5,%\1with-flv\3Compile with support for FLV module\5,%\1with-stub_status\3Compile with support for Stub Status module\5/; s/\([[:space:]]* args << "--\)\(with-http_dav_module\)\(" if ARGV.include? '\''--with-\)\(webdav\)\('\''.*\)/\1\2\3\4\5%\1with-http_realip_module\3realip\5%\1with-http_gzip_static_module\3gzip_static\5%\1add-module=\/tmp\/nginx-upload-progress\3uploadprogress\5%\1add-module=\/tmp\/nginx-fair\3fair\5%\1with-http_mp4_module\3mp4\5%\1with-http_flv_module\3flv\5%\1with-http_stub_status_module\3stub_status\5/; y/%/\n/' $(brew --prefix)/Library/Formula/nginx.rb

Now we’ll install Nginx with our new build options and extensions and start it.

1
2
3
4
5
6
brew install nginx --with-realip --with-gzip_static --with-mp4 --with-flv --with-stub_status --with-uploadprogress --with-fair
[ $? -eq 0 ] && rm -rf /tmp/nginx-upload-progress /tmp/nginx-fair
mkdir -vp $(brew --prefix nginx)/var/{microcache,log,run}
[ ! -d ~/Library/LaunchAgents ] && mkdir ~/Library/LaunchAgents
[ -f $(brew --prefix nginx)/homebrew.mxcl.nginx.plist ] && cp -v $(brew --prefix nginx)/homebrew.mxcl.nginx.plist ~/Library/LaunchAgents/
[ -f ~/Library/LaunchAgents/homebrew.mxcl.nginx.plist ] && launchctl load -w ~/Library/LaunchAgents/homebrew.mxcl.nginx.plist

Nginx-With-Drupal

We’re now ready to set up Nginx to work with Drupal. I’ve created a fork of the original repository to make some necessary changes for OS X. I also make several changes for the stable 1.0.x branch of Nginx, which we’ve just installed, rather than the unstable 1.1.x branch that is in the original configuration.

If you want to stick with the original project, check out the changes I made for OS X and 1.0.x and you can apply them yourself and skip the git steps below.

1
2
3
4
5
6
7
8
[ -d $(brew --prefix)/etc/nginx ] && mv -v $(brew --prefix)/etc/nginx $(brew --prefix)/etc/nginx-default
git clone https://github.com/alanthing/drupal-with-nginx.git $(brew --prefix)/etc/nginx
cd $(brew --prefix)/etc/nginx
git checkout osx-1.0.x
mkdir sites-enabled
cd sites-enabled
ln -s ../sites-available/000-default
cp -a ../sites-available/example.com.conf yournewsite.conf

That’s about as much as I can automate for you with copy+paste-able commands! You’ll want to do the following to yournewsite.conf, which you can rename to be anything, to configure Nginx for your website:

  • Change server_name, access_log, and error_log to use your local domain name for your virtual host
  • Change root to the path of your Drupal installation. On my system, this may be /Users/alan/Sites/drupal-7.14. Note that you cannot use ~ in place of /Users/name
  • Unless you have a valid SSL certificate, you’ll probably want to completely delete the second half of the file. So find the line containing } HTTP server and remove all following lines
  • If you’re using Boost with Drupal 7, or Drupal 6 with/without Boost, note that you’ll want to comment out the include sites-available/drupal.conf line and uncomment the other relevant one for your site
  • If you want to get additional status messages from PHP-FPM, uncomment include php_fpm_status_vhost.conf in this file, and also include php_fpm_status_allowed_hosts.conf in $(brew --prefix)/etc/nginx/nginx.conf
  • If you configured PHP-FPM earlier to use Unix sockets instead of TCP, open nginx.conf and comment out include upstream_phpcgi_tcp.conf and uncomment include upstream_phpcgi_unix.conf
  • Depending on the location of your files directory, you may need to edit the sites-available/drupal.conf (or other drupal*.conf) file and change the relevant sites/default/files paths appropriately

Once you’re finished editing your virtual host conf file (and possibly nginx.conf), you can reload nginx easily:

1
$(brew --prefix)/sbin/nginx -s reload

Bonus: Drush

You’ll need Drush to install a new Drupal site, as install.php is blocked for security reasons by default. Also, you’ll find cron.php is inaccessible as well. There’s a weird little hack required to get Drush installed with Homebrew without requiring sudo, so below is an example of how to both get around the sudo requirement and set up a blank Drupal 7 website (note that using the root MySQL user is bad form, but this is meant more as a quick demo of using drush with this setup than a recommended setup).

1
2
3
4
5
6
7
8
9
10
touch $(brew --prefix php)/lib/php/.lock
chmod 0644 $(brew --prefix php)/lib/php/.lock
$(brew --prefix)/bin/pear channel-discover pear.drush.org
$(brew --prefix)/bin/pear install drush/drush
brew unlink php && brew link php
cd ~/Sites 2>/dev/null || mkdir ~/Sites && cd ~/Sites
drush dl
mysql -uroot -p'yourpassword' -e'create database drupal;'
cd drupal-7.14
drush si --db-url=mysql://root:yourpassword@localhost/drupal

Can I Get Your Number? DNS Made Simple

This post originally featured on the Echo & Co. blog.

At EchoDitto, we get a lot of questions about how hosting works, and in particular about changing DNS records for new websites or websites who are moving to new hosting providers. Here’s an old but true analogy to make understanding DNS easier: the IP address of your website is like your phone number (it changes if you move houses – at least, your landline does), and DNS is like the phonebook. If someone wants to call you at your new house they might look you up by name in the phonebook, but if their phonebook hasn’t been updated since you moved they’re going to call your old house.

Here’s how it works in more detail:

When visitors want to access your website their computers take the domain name that they type in (echoditto.com, for example) and look up the IP address for that domain by way of Domain Name Service, or DNS.

To get your DNS records for users to get to your website, first you need a domain name. Go Daddy and Network Solutions are examples of registrars where you can buy domain names.

Continuing the analogy, the first thing a visitor’s computer needs to know is which “phonebook” your website’s IP address is listed in. The registrar will set something called the “name server” for your domain, which is akin to telling the user which “phonebook” (name server) to look you up in. The name server contains the details about how your domain name, example.com, will get to the computer that actually contains (or “hosts”) your website.

In a phonebook, you look up a name and get a phone number. It’s similar with DNS – the user’s computer will look up your domain in the name server and get an IP address, which is a numerical address of a computer on the internet. The name server gives the user’s computer the exact IP address of the computer that hosts your website. Once it has that information the user’s computer can “call” the number, which takes them directly to your website.

DNS contains another field along with the IP address, called a TTL, or time-to-live. This time, in seconds, is how long your computer should remember the number in the phonebook before it looks it up again. Computers check in with DNS name servers before visiting a site again to make sure the IP address of a website hasn’t changed, and the TTL tells them how often to do that. TTL times can vary, but often you may find it initially set for 1 to 4 hours on average websites. The higher the setting is, the less often the DNS name server will be asked about the IP address and other DNS records.

When launching a website on a new server, or moving web hosts, you want to temporarily change the TTL to be really low. That makes website visitors check their “phonebook” more frequently, which means they will stop going to the old IP address as soon as possible after a change is made. It can take a while for a new TTL to take effect for all users if using the typical defaults of 1 to 4 hours, so when we’re launching a new site, or changing hosting, we do this ahead of time. Once visitors worldwide have the new lower TTL, we can change the DNS entry for the website to point to the new IP address, and then visitors will see your new website, or your current website on the new host, very quickly.

Have other questions about DNS? Ask your computer to look up http://www.youtube.com/watch?v=oN7ripK5uGM to watch a video that explains all of this in another way.

Node.js and NPM on CentOS

This post originally featured on the Echo & Co. blog.

Update: This no longer appears to be necessary as of nodejs 0.8.0. It may have been fixed earlier but I noticed neither of these changes are necessary anymore. Something new though, I had problems with node-gyp, and the solution was to install python26 with yum and then re-run the npm command with PYTHON="/usr/bin/python26" npm install -deps or similar.

The preferred way to install node and NPM seems to be installing from source, but I’m a perennial fan of using packages to keep things tidy, especially if I need to uninstall something. I started by going to the Node.js download page, and through to Installing with a package manager. I installed the yum release RPM for the tchol.org repository as directed and installed nodejs and npm with yum. From there, I ran into two problems but thankfully they were fairly easy to resolve.

The first thing I wanted to do was install forever globally so I could use it to keep applications running persistently. But running npm install forever -g kept stalling. The npm RPM installed from the tchol.org repository creates the symlink /usr/lib/node_modules to /usr/lib/nodejs. That’s fine, but /usr/lib/nodejs is owned by root:root. Running a npm install command with sudo attempted to set the ownership of the NPM modules as nobody:user, and NPM wasn’t exiting due to permissions for some reason.

I have ACLs enabled on my file system, so I fixed this by allowing nobody write access to /usr/lib/nodejs:

sudo setfacl -m u:nobody:rwx /usr/lib/nodejs

If you don’t have ACLs enabled on your filesystem, you could allow nobody to be the folder owner:

sudo chown nobody:nobody /usr/lib/nodejs

The second problem I ran into was, after installing forever, it wouldn’t run. The nodejs package installed the binary as /usr/bin/nodejs but /usr/bin/forever begins with #!/usr/bin/env node, which will not return a valid interpreter. I wanted to install a symlink into /usr/local/bin, but some users don’t always have that in their path, so I created a link via:

sudo ln -s /usr/bin/nodejs /usr/bin/node

Now I’m able to install anything I need to with NPM and run my Node.js applications with Forever. I’m still very new to Node, so if you think I should be doing things a better way, let me know in the comments!

Alan’s 2011 Favorites

This post originally featured on the Echo & Co. blog.

Favorite Links of 2011

Google Reader – between the Android app, Reeder for iPad, and the website, keeps me up to date on the news.

Stack Overflow and other Stack Exchange sites – one of the best ways to get help and help others.

9GAG – thanks to (our summer intern) Vicky for showing this to me, I waste a lot of time looking at internet memes.

Best Entertainment of 2011

Friday Night Lights – the final season was aired on NBC in the first half of the year. It gets my vote for the best drama of all time.

Community – one of the smartest and most brilliant comedies in recent memory. It’s a shame NBC doesn’t get it and has decided to “bench” it for the immediate future. It doesn’t get good Nielson rating because a majority of their audience watches it on Netflix/Hulu/DVR/etc.

Thor – it won’t win any awards for best screenplay, but 1) it was a beautiful movie with great special effects, and 2) the closer we get to an Avengers movie, the more this geek gets excited!

Favorite Restaurant

Rachel’s Creperie – at the end of the block of my new co-working space, offers a new way to try many kinds of foods you’d normally just slap in between two pieces of boring bread.

Favorite Game

Farkle! – the link explains all of the rules. All it requires is 6 dice and 1 or more friends. It channels Yahtzee but has some fun twists.

Favorite Mobile App

Words With Friends – I know it’s more like the game of 2010, but this year they added the Android version so I can play Facebook and iPhone friends, and it gives my brain a quick puzzle when I have some downtime.

Favorite Technology Innovation

Music in the Cloud, either Google Music, Amazon Cloud Player, or iCloud – they make listening to the same music on my laptop and my phone a snap. Sync-less bliss…

Favorite Cause

It wasn’t new to 2011, but I think 2011 is when Kickstarter really became huge. It’s amazing to see peoples’ dreams come to life.

OS X 10.7 Lion Development: MacPorts

This post originally featured on the Echo & Co. blog.

OS X Lion comes with most of the tools you would need to do “MAMP” (Mac OS X, Apache, MySQL/MariaDB, PHP) development, as outlined in my previous posts once you add a database. So then why would you want to use MacPorts? Setting your development environment up in MacPorts isolates the binaries, libraries, and configuration files, completely separate from the existing OS X install (with the exception of startup scripts). You can also tweak the configuration files on your own, apply your own patches, and apply updates that MacPorts may get before Apple pushes them. It will take more time because you’ll be compiling everything, but you have all of the control. Read on for how to get things set up.

Prerequisites

We’ll going to be compiling a lot of source code so you’ll need to have Xcode 4 installed. I initially tried with the OS X GCC Installer but eventually ran into a problem where a portfile was expecting an Xcode binary to check to a prerequisite. I’m sure some hacking could’ve resolved it to avoid having to install the very large Xcode package, but at the end of the day it’s better to just know that it’ll work as expected.

Once you have Xcode 4 installed, you’ll need to install MacPorts. It’s a simple click-through installer. Come back to this guide once MacPorts is ready.

Note that for all commands before that are starting with a $, the dollar sign is showing a command-line prompt in Terminal, and you should not actually type it as part of the commands.

Install packages

Update the ports tree to get the latest from the MacPorts server in case the installer had an older tree:

1
sudo /opt/local/bin/port selfupdate

All of the packages that we need can be installed with a single command:

1
sudo /opt/local/bin/port install apache2 php5 php5-mysql mysql5-server php5-gd php5-mbstring php5-apc lynx phpmyadmin

Compiling all of these tools will take a while, so grab a book and come back when the command is completed.

Configure PHP

The following will set up PHP with some development-friendly settings.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
$ sudo cp -av /opt/local/etc/php5/php.ini-development /opt/local/etc/php5/php.ini
$ sudo sed -i "" 's/\(default_socket[ ]\{0,1\}=\)/\1 \/opt\/local\/var\/run\/mysql5\/mysqld.sock/g' /opt/local/etc/php5/php.ini
$ sudo sh -c "cat >> /opt/local/etc/php5/php.ini <<'EOF'


;;
;; User customizations below
;;

; Original - memory_limit = 128M
memory_limit = 196M
; Original - post_max_size = 8M
post_max_size = 200M
; Original - upload_max_filesize = 2M
upload_max_filesize = 100M
; Original - default_socket_timeout = 60
default_socket_timeout = 600
; Original - max_execution_time = 30
max_execution_time = 300
; Original - max_input_time = 60
max_input_time = 600
; Original - ;date.timezone =
date.timezone = 'America/New_York'
EOF"

Now configure the APC extension (installed above with php5-apc) which will increase PHP performance dramatically.

1
2
3
4
5
6
7
8
9
10
$ sudo sh -c "cat > /opt/local/var/db/php5/apc-config.ini <<'EOF'
[apc]
apc.enabled = 1
apc.shm_segments = 1
apc.shm_size = 96M
apc.cache_by_default = 1
apc.stat = 1
apc.rfc1867 = 1
apc.stat = 7200
EOF"

Configure Apache

Start by setting up PHP to work with Apache properly:

1
2
3
4
5
6
7
8
9
10
11
$ sudo cp -av /opt/local/apache2/conf/httpd.conf /opt/local/apache2/conf/httpd.conf-default
$ sudo /opt/local/apache2/bin/apxs -a -e -n "php5" /opt/local/apache2/modules/libphp5.so
$ sudo bash -c "cat >> /opt/local/apache2/conf/httpd.conf <<'EOF'

## Add mod_php information
Include conf/extra/mod_php.conf
# Add index.php to the list of files that will be served as directory indexes.
<IfModule dir_module>
  DirectoryIndex index.php index.html
</IfModule>
EOF"

Now we’ll move httpd-vhosts.conf to ~/Sites for easy editing of new virtual hosts, as well as create a ~/Sites/logs folder:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
$ sudo mv -v /opt/local/apache2/conf/extra/httpd-vhosts.conf /opt/local/apache2/conf/extra/httpd-vhosts.conf-default </br>
$ [ ! -d ~/Sites ] && mkdir -v ~/Sites
$ cp -av /opt/local/apache2/conf/extra/httpd-vhosts.conf-default ~/Sites/httpd-vhosts.conf
$ sudo ln -s ~/Sites/httpd-vhosts.conf /opt/local/apache2/conf/extra/httpd-vhosts.conf
$ sudo sed -i "" 's/\#\(.*httpd-vhosts\.conf\)/\1/' /opt/local/apache2/conf/httpd.conf
$ [ ! -d ~/Sites/logs ] && mkdir ~/Sites/logs
$ sudo perl -pi -e 'BEGIN{undef $/;} s/\<VirtualHost .*\n//sg;' ${USERHOME}/Sites/httpd-vhosts.conf </br>
$ sudo sh -c "cat >> ~/Sites/httpd-vhosts.conf <<'EOF' </br>
 </br>
## Change /Users/name to the path of your home folder </br>
#<VirtualHost *:80> </br>
#  ServerName project.local </br>
#  CustomLog "/Users/name/Sites/logs/project.local-access_log" combined </br>
#  ErrorLog "/Users/name/Sites/logs/project.local-error_log" </br>
#  DocumentRoot "/Users/name/Sites/project.local" </br>
#</VirtualHost> </br>
EOF"

Allow userdirs so the MacPorts Apache “feels” more like the built-in Apache:

1
sudo sed -i "" 's/\#\(.*httpd-userdir\.conf\)/\1/' /opt/local/apache2/conf/httpd.conf

Start Apache and set it to load on boot:

1
sudo /opt/local/bin/port load apache2

Configure MySQL

Set up the configuration file:

1
2
sudo cp -av /opt/local/share/mysql5/mysql/my-small.cnf /opt/local/etc/mysql5/my.cnf
sudo sed -i "" 's/max_allowed_packet = 1.*M/max_allowed_packet = 1G/g' /opt/local/etc/mysql5/my.cnf

Initialize MySQL and run the secure installation script:

1
2
3
sudo -u _mysql /opt/local/bin/mysql_install_db5
sudo /opt/local/bin/port load mysql5-server
sudo /opt/local/lib/mysql5/bin/mysql_secure_installation

Configure phpMyAdmin

Add a config file for Apache and reload Apache:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
$ sudo sh -c "cat >> /opt/local/apache2/conf/httpd.conf <<'EOF'

# phpMyAdmin
Include conf/extra/phpmyadmin.conf
EOF"
$ sudo sh -c "cat >> /opt/local/apache2/conf/extra/phpmyadmin.conf <<'EOF'
AliasMatch ^/phpmyadmin(?:/)?(/.*)?$ /opt/local/www/phpmyadmin$1
AliasMatch ^/phpMyAdmin(?:/)?(/.*)?$ /opt/local/www/phpmyadmin$1

<Directory "/opt/local/www/phpmyadmin">
 Options -Indexes
 AllowOverride None
 Order allow,deny
 Allow from all

 LanguagePriority en de es fr ja ko pt-br ru
 ForceLanguagePriority Prefer Fallback
</Directory>
EOF"
$ sudo /opt/local/bin/port unload apache2
$ sudo /opt/local/bin/port load apache2

Basic set up of phpMyAdmin:

1
2
3
sudo sed -i "" "s/blowfish_secret\'\] = \'/blowfish_secret\'\] = \'`cat /dev/urandom | strings | grep -o '[[:alnum:]]' | head -n 50 | tr -d '\n'`/" /opt/local/www/phpmyadmin/config.inc.php
sudo /opt/local/bin/mysql5 -uroot -proot < /opt/local/www/phpmyadmin/scripts/create_tables.sql
sudo /opt/local/bin/mysql5 -uroot -proot -e"CREATE USER 'pma'@'localhost' IDENTIFIED BY 'pmapass'; GRANT SELECT, INSERT, DELETE, UPDATE ON phpmyadmin.* TO pma@localhost;"

Hooray!

Now you can edit ~/Sites/httpd-vhosts.conf and add a new virtual host. You’d need to reload Apache after doing so by running sudo port unload apache2 && sudo port load apache2. phpMyAdmin or the mysql5 binary should provide you a way to create new databases and database users and you can then set up a local site.

If you find MacPorts too heavy a separation from OSX, or too slow to compile, you should check out my previous blog posts on setting up a MAMP environment with as many built-in tools as possible only by adding a MySQL installer or MySQL/MariaDB via Homebrew.

OS X 10.7 Lion Development: Native Apache & PHP With Homebrew MySQL or MariaDB

This post originally featured on the Echo & Co. blog.

OS X Lion ships with Apache and PHP, which both require a little bit of tweaking to get fully-functional for “MAMP” local development. The one thing Lion does not ship with is a database. This will be very similar to my previous post on local development but this time we’ll be using Homebrew to install either MySQL or MariaDB for the database. Since we’ll be using a compiler for Homebrew, I’ll also cover how to add APC and other PECL modules that you can add to OS X.

Note that for all commands before that are starting with a $, the dollar sign is showing a command-line prompt in Terminal, and you should not actually type it as part of the commands.

Apache

We’ll set things up so we won’t need sudo often in the future, and so we can manage multiple VirtualHosts. We’ll keep the Apache information and our website roots in ~/Sites, and Apache logs in ~/Sites/logs

1
2
3
4
5
[ ! -d ~/Sites ] && mkdir ~/Sites
touch ~/Sites/httpd-vhosts.conf
sudo ln -s ~/Sites/httpd-vhosts.conf /etc/apache2/other
mkdir ~/Sites/logs
chmod 0777 ~/Sites/logs

Edit the new ~/Sites/httpd-vhosts.conf file and add the following. Note that you’ll need to change all instances of “/Users/name” to your actual home folder path.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
# 
# Use name-based virtual hosting. 
# 
NameVirtualHost *:80

# 
# Set up permissions for VirtualHosts in ~/Sites 
# 
<Directory "/Users/name/Sites">
    Options Indexes FollowSymLinks MultiViews
    AllowOverride All
    Order allow,deny
    Allow from all
</Directory>

# For http://localhost in the OS X default location 
<VirtualHost _default_:80>
    ServerName localhost
    DocumentRoot /Library/WebServer/Documents
</VirtualHost>

# 
# VirtualHosts below 
# 

## Template 
#<VirtualHost *:80> 
#    ServerName domain.local 
#    CustomLog "/Users/name/Sites/logs/domain.local-access_log" combined 
#    ErrorLog "/Users/name/Sites/logs/domain.local-error_log" 
#    DocumentRoot "/Users/name/Sites/domain.local" 
#</VirtualHost>

Launch System Preferences and go to Sharing and toggle Web Sharing off and on so it’s started and reloaded with the new settings. Then click on the blue underlined link under “Your computer’s website is available at this address:” to ensure Apache is working correctly, and you should see text saying “It works!”

To add a site, duplicate the <VirtualHost> section under the Template, remove the comments, and edit appropriately. Edit /etc/hosts (or use Gas Mask) and add an entry for 127.0.0.1 for your project.local, and make the appropriate change in the conf file, along with the correct DocumentRoot and Log file locations. See Apache’s documentation for more information.

PHP

OS X has shipped with PHP for quite some time, but not had it enabled by default. We’ll enable mod_php for Apache, and also set up an /etc/php.ini file based on the default one shipped with Lion with some development-friendly changes. Change your timezone as needed.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
$ sudo sh -c "grep php /etc/apache2/httpd.conf|grep LoadModule|cut -d'#' -f2 > /etc/apache2/other/php5-loadmodule.conf"
$ sudo cp -a /etc/php.ini.default /etc/php.ini
$ sudo sh -c "cat >> /etc/php.ini <<'EOF' 


;; 
;; User customizations below 
;; 

; Original - memory_limit = 128M 
memory_limit = 196M 
; Original - post_max_size = 8M 
post_max_size = 200M 
; Original - upload_max_filesize = 2M 
upload_max_filesize = 100M 
; Original - default_socket_timeout = 60 
default_socket_timeout = 600 
; Original - max_execution_time = 30 
max_execution_time = 300 
; Original - max_input_time = 60 
max_input_time = 600 
; Original - display_errors = Off 
display_errors = on 
; Original - display_startup_errors = Off 
display_startup_errors = on 
; Original - ;date.timezone = 
date.timezone = 'America/New_York' 
EOF"

Optional: Lion ships with PEAR but not installed, whereas Snow Leopard it was installed. This will install PEAR from the phar archive, upgrade it, and add ‘pear’ and ‘pecl’ aliases to your shell.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
$ sudo /usr/bin/php /usr/lib/php/install-pear-nozlib.phar
$ cat >> ~/.bashrc <<'EOF' 

alias pear="php /usr/lib/php/pear/pearcmd.php" 
alias pecl="php /usr/lib/php/pear/peclcmd.php" 
EOF
$ . ~/.bashrc
$ sudo pear channel-update pear.php.net
$ sudo pecl channel-update pecl.php.net
$ sudo pear upgrade --force pear
$ sudo pear upgrade
$ sudo pecl upgrade
$ sudo sh -c "cat >> /etc/php.ini <<'EOF' 
; Original - ;include_path = ".:/php/includes" 
include_path = ".:/usr/lib/php/pear" 
EOF"

We’ll cover APC and other PECL modules after we have Homebrew up and running.

Toggle Web Sharing in System Preferences > Sharing for the new PHP options to take effect.

Homebrew

Hop onto the Mac App Store and download Xcode 4. It’s free! When it’s finished downloading, you’ll have an Application called Install Xcode in the Applications folder, so run that and click through and you’ll be all set. If it seems like overkill since we’re not covering iOS or Mac app development, Xcode ships with the compiler we need for Homebrew and installing PECL modules.

To install Homebrew, go to https://github.com/mxcl/homebrew/wiki/installation and run the ruby script in Terminal. If you get the error message: Error: Cannot write to /usr/local, run:

1
2
sudo chmod g+w /usr/local
sudo chgrp staff /usr/local

MariaDB / MySQL

MariaDB is a fork of MySQL that has additional features and is a drop-in replacement for MySQL. MariaDB is supported by Drupal 7, and works with Drupal 6 as well. Make the choice to go with either MariaDB or MySQL and continue; I’ll cover how to setup and use both.

MySQL

To install MySQL, simply run:

1
brew install mysql

cmake is a dependency of MySQL, and cmake needs java, so if you get a pop-up asking you to install a Java runtime, be sure to click Install and proceed. Grab a book because compiling MySQL and its dependencies will take several minutes. Now, to configure MySQL (includes raising packet limits for easier use in a non-production scenario) and start:

1
2
3
4
5
unset TMPDIR
mysql_install_db --verbose --user=`whoami` --basedir="$(brew --prefix mysql)" --datadir=/usr/local/var/mysql --tmpdir=/tmp
cp $(brew --prefix mysql)/support-files/my-small.cnf /usr/local/var/mysql/my.cnf
sed -i "" 's/max_allowed_packet = 1.*M/max_allowed_packet = 2G/g' /usr/local/var/mysql/my.cnf
[ ! -d ~/Library/LaunchAgents ] && mkdir ~/Library/LaunchAgents

MariaDB

To install MariaDB, run:

1
brew install mariadb pidof

pidof is referenced in some of the MariaDB scripts but is not listed as a prerequisite in MariaDB’s Homebrew formula, so we need to install it. To configure and start:

1
2
3
4
5
unset TMPDIR
mysql_install_db --verbose --user=`whoami` --basedir="$(brew --prefix mariadb)" --datadir=/usr/local/var/mysql --tmpdir=/tmp
cp $(brew --prefix mariadb)/share/mysql/my-small.cnf /usr/local/var/mysql/my.cnf
sed -i "" 's/max_allowed_packet = 1.*M/max_allowed_packet = 2G/g' /usr/local/var/mysql/my.cnf
[ ! -d ~/Library/LaunchAgents ] && mkdir ~/Library/LaunchAgents

APC and other PECL modules

PECL modules require compiling, which is why I excluded it from my previous blog post because I did not require Xcode to be installed. An easy one is uploadprogress which is often recommended for Drupal development:

1
2
3
4
5
6
$ sudo pecl install uploadprogress
$ sudo sh -c "cat >> /etc/php.ini <<'EOF' 

; Enable PECL extension uploadprogress 
extension=uploadprogress.so 
EOF"

APC is easy too, but requires you to install pcre libraries. With Homebrew, this is easy, and we’ll add some basic APC parameters:

1
2
3
4
5
6
7
8
9
10
11
12
13
$ brew install pcre
$ sudo pecl install apc
$ sudo sh -c "cat >> /etc/php.ini <<'EOF' 

; Enable PECL extension APC 
extension=apc.so 
apc.enabled = 1 
apc.shm_segments = 1 
apc.shm_size = 96M 
apc.cache_by_default = 1 
apc.stat = 1 
apc.rfc1867 = 1 
EOF"

Non-PECL extensions, like mcrypt, require some additional work. Check out this guide for building mcrypt.

Make PHP and MariaDB / MySQL Play Nice

If you were to run php -i|egrep 'mysql.*default_socket' you would see that PHP was compiled to expect the MySQL socket file in /var/mysql, but MariaDB / MySQL will place it in /tmp. This easiest fix is to tell PHP to look in /tmp:

1
sudo sed -i "" 's|/var/mysql/mysql\.sock|/tmp/mysql.sock|g' /etc/php.ini

Toggle Web Sharing in System Preferences > Sharing for the new PHP options to take effect.

Hooray!

You should now be all set to keep adding more VirtualHosts in httpd-vhosts.conf and begin development on your local machine. Stay tuned for instructions on how to use MacPorts!

Update: If you’re interested in leveraging more from Homebrew, like replacing Apache and PHP all from Homebrew, check out the homebrew-alt repository on GitHub. Newer versions of Homebrew allow you to install formulae from URLs or alternate folders, and homebrew-alt provides formulae that replace built-in OS X applications (something official formulae will not do). For example, you could install PHP with PHP-FPM, something the OS X-provided PHP does not offer. Have fun exploring!

OS X 10.7 Lion Development: Native MAMP With MySQL Installer

This post originally featured on the Echo & Co. blog.

With the release of Lion, there are some subtle differences to setting up a local MAMP (Mac OS X, Apache, MySQL, PHP) environment compared to Snow Leopard. In an effort to keep this from being overly wordy and just get to the good stuff, we’ll dive right in, so read on to get started.

Note that for all commands before that are starting with a $, the dollar sign is showing a command-line prompt in Terminal, and you should not actually type it as part of the commands.

Apache

We’ll set things up so we won’t need sudo often in the future, and so we can manage multiple VirtualHosts. We’ll keep the Apache information and our website roots in ~/Sites, and Apache logs in ~/Sites/logs

1
2
3
4
5
$ [ ! -d ~/Sites ] && mkdir ~/Sites
$ touch ~/Sites/httpd-vhosts.conf
$ sudo ln -s ~/Sites/httpd-vhosts.conf /etc/apache2/other
$ mkdir ~/Sites/logs
$ chmod 0777 ~/Sites/logs

Edit the new ~/Sites/httpd-vhosts.conf file and add the following. Note that you’ll need to change all instances of “/Users/name” to your actual home folder path.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
# 
# Use name-based virtual hosting. 
# 
NameVirtualHost *:80

# 
# Set up permissions for VirtualHosts in ~/Sites 
# 
<Directory "/Users/name/Sites">
    Options Indexes FollowSymLinks MultiViews
    AllowOverride All
    Order allow,deny
    Allow from all
</Directory>

# For http://localhost in the OS X default location 
<VirtualHost _default_:80>
    ServerName localhost
    DocumentRoot /Library/WebServer/Documents
</VirtualHost>

# 
# VirtualHosts below 
# 

## Template 
#<VirtualHost *:80> 
#    ServerName domain.local 
#    CustomLog "/Users/name/Sites/logs/domain.local-access_log" combined 
#    ErrorLog "/Users/name/Sites/logs/domain.local-error_log" 
#    DocumentRoot "/Users/name/Sites/domain.local" 
#</VirtualHost>

Launch System Preferences and go to Sharing and toggle Web Sharing off and on so it’s started and reloaded with the new settings. Then click on the blue underlined link under “Your computer’s website is available at this address:” to ensure Apache is working correctly, and you should see text saying “It works!”

To add a site, duplicate the <VirtualHost> section under the Template, remove the comments, and edit appropriately. Edit /etc/hosts (or use Gas Mask) and add an entry for 127.0.0.1 for your project.local, and make the appropriate change in the conf file, along with the correct DocumentRoot and Log file locations. See Apache’s documentation for more information.

PHP

OS X has shipped with PHP for quite some time, but not had it enabled by default. We’ll enable mod_php for Apache, and also set up an /etc/php.ini file based on the default one shipped with Lion with some development-friendly changes. Change your timezone as needed.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
$ sudo bash -c "grep php /etc/apache2/httpd.conf|grep LoadModule|cut -d'#' -f2 > /etc/apache2/other/php5-loadmodule.conf"
$ sudo cp -a /etc/php.ini.default /etc/php.ini
$ sudo bash -c "cat >> /etc/php.ini <<'EOF' 


;; 
;; User customizations below 
;; 

; Original - memory_limit = 128M 
memory_limit = 196M 
; Original - post_max_size = 8M 
post_max_size = 200M 
; Original - upload_max_filesize = 2M 
upload_max_filesize = 100M 
; Original - default_socket_timeout = 60 
default_socket_timeout = 600 
; Original - max_execution_time = 30 
max_execution_time = 300 
; Original - max_input_time = 60 
max_input_time = 600 
; Original - display_errors = Off 
display_errors = on 
; Original - display_startup_errors = Off 
display_startup_errors = on 
; Original - ;date.timezone = 
date.timezone = 'America/New_York' 
EOF"

Optional: Lion ships with PEAR but not installed, whereas Snow Leopard it was installed. This will install PEAR from the phar archive, upgrade it, and add pear and pecl aliases to your shell.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
$ sudo /usr/bin/php /usr/lib/php/install-pear-nozlib.phar
$ cat >> ~/.bashrc <<'EOF' 

alias pear="php /usr/lib/php/pear/pearcmd.php" 
alias pecl="php /usr/lib/php/pear/peclcmd.php" 
EOF
$ . ~/.bashrc
$ sudo pear channel-update pear.php.net
$ sudo pecl channel-update pecl.php.net
$ sudo pear upgrade --force pear
$ sudo pear upgrade
$ sudo pecl upgrade
$ sudo sh -c "cat >> /etc/php.ini <<'EOF' 
; Original - ;include_path = ".:/php/includes" 
include_path = ".:/usr/lib/php/pear" 
EOF"

Note that installing most pecl modules will require having a compiler installed via Xcode. I’ll be addressing installing uploadprogress and APC in a later post, since this walkthrough requires no compiling.

Toggle Web Sharing in System Preferences > Sharing for the new PHP options to take effect.

MySQL

MySQL is the only thing not shipped with OS X that we need for our development environment. Go to the download site for OS X pre-built binaries at http://dev.mysql.com/downloads/mysql/index.html#macosx-dmg and choose the DMG Archive most appropriate for your system. As of this writing, the most recent version says OS X 10.6, but it will work on 10.7 as well.

Open the downloaded disk image and begin by installing the mysql-5.x.x-osx10.6-x86_64.pkg file. Once the Installer script is completed, install the Preference Pane by double-clicking on MySQL.prefPane. If you are not sure where to install the preference pane, choose “Install for this user only.”

Optional: If you wish to have MySQL start on boot, you do need to additionally install the MySQLStartupItem.pkg file. The preference pane has a toggle for starting on boot, but it will not work until you install the pkg and you run the following in Terminal:

1
sudo chown -R root:wheel /Library/StartupItems/MySQLCOM

Open System Preferences and MySQL and start the database by clicking the Start MySQL Server button.

Optional: The MySQL installer installed its files to /usr/local/mysql, and the binaries are in /usr/local/mysql/bin which is not in the $PATH of any user by default. Rather than edit the $PATH shell variable, we add symbolic links in /usr/local/bin (a location already in $PATH) to a few MySQL binaries. Add more or omit more binaries as desired:

1
2
3
4
5
6
[ ! -d /usr/local/bin ] && sudo mkdir -p /usr/local && sudo chmod 0777 /usr/local && mkdir /usr/local/bin && sudo chmod 0755 /usr/local
cd /usr/local/bin
ln -s /usr/local/mysql/bin/mysql
ln -s /usr/local/mysql/bin/mysqladmin
ln -s /usr/local/mysql/bin/mysqldump
ln -s /usr/local/mysql/support-files/mysql.server

This script ships with MySQL and only needs to be run once, and it should be run with sudo. As you run it, you can accept all other defaults after you set the root user’s password:

1
sudo /usr/local/mysql/bin/mysql_secure_installation

After installing MySQL, several sample my.cnf files are created but none is placed where MySQL can find it. Start with a “small” configuration file and make a few changes to increase the max_allowed_packet variable:

1
2
sudo cp /usr/local/mysql/support-files/my-small.cnf /usr/local/mysql/data/my.cnf
sudo sed -i "" 's/max_allowed_packet = 1.*M/max_allowed_packet = 2G/g' /usr/local/mysql/data/my.cnf

To load in these changes, go back to the MySQL System Preferences pane, and restart the server by pressing “Stop MySQL Server” followed by “Start MySQL Server.”

Make PHP and MySQL Play Nice

If you were to run php -i|egrep 'mysql.*default_socket' you would see that PHP was compiled to expect the MySQL socket file in /var/mysql, but MySQL will place it in /tmp. This easiest fix is to tell PHP to look in /tmp:

1
sudo sed -i "" 's/\/var\/mysql\/mysql\.sock/\/tmp\/mysql\.sock/g' /etc/php.ini

Toggle Web Sharing in System Preferences > Sharing for the new PHP options to take effect.

Hooray!

You should now be all set to keep adding more VirtualHosts in httpd-vhosts.conf and begin development on your local machine. Stay tuned for instructions on how to use MariaDB or MySQL with Homebrew instead of the Oracle MySQL installer, and MacPorts!

Use Syslog Instead of Watchdog

This post originally featured on the Echo & Co. blog.

In this example, we choose one of our servers to receive the Syslog traffic and the others will send traffic to it. In a large environment, you should use a non-public-facing server to act as a syslog server for your sites, ideally using a database to store data, but we’ll just output everything to a file to keep things simple for this guide. If you decide to use a database like MySQL to store your data, this can still provide you with a starting point, but just know that using your same production database server defeats my intended purpose since I was ultimately trying to alleviate the amount of writes on the database.

Start by using yum to install rsyslog. If you’re using another distro, rsyslog is likely included in the base repositories. Also, note that CentOS 6 uses rsyslog by default so you may skip this step if it’s already installed on your system. You’ll need this on all servers running Drupal or your rsyslog server:

1
yum install rsyslog

For CentOS 5, you’ll need to unload the default syslog and switch to rsyslog. The following will ensure it starts on boot as well. Until otherwise noted, run the following commands on all servers:

1
2
3
/sbin/service syslog stop && /sbin/service rsyslog start
/sbin/chkconfig rsyslog on
/sbin/chkconfig syslog off

Rather than hacking the config file to bits, I find it easier to create an include folder and keep things organized by files. I also like to back up default files created by RPMs so if an update leaves behind rsyslog.rpmnew, you can compare it to the default to see the changes:

1
2
mkdir -v /etc/rsyslog.d
cp -av /etc/rsyslog.conf{,-default}

Add the following to /etc/rsyslog.conf below the last line containing “$ModLoad”:

1
2
# Include all files in /etc/rsyslog.d
$IncludeConfig /etc/rsyslog.d/*.conf

Reload the current rsyslog configuration:

1
/sbin/service rsyslog reload

The next several steps will only need to be run on the syslog server. Back up and edit /etc/sysconfig/rsyslog to accept UDP connections from other servers. If you’re using iptables, open up port 514 on UDP.

1
cp -a /etc/sysconfig/rsyslog /etc/sysconfig/rsyslog-default

Edit /etc/sysconfig.rsyslog and change SYSLOGD_OPTIONS=“-m 0” to SYSLOGD_OPTIONS=“-m 0 -r514”

Create a root folder for website logs to go:

1
mkdir /var/log/websites

Create the following file to work with example.com (this format will allow you to create other files for anotherexample.org, etc.): /etc/rsyslog.d/example.conf. Add the following:

1
2
3
4
# example.com
$template DailyPerHostLogs-example,"/var/log/websites/%SYSLOGTAG:F,58:1%/%SYSLOGTAG:F,58:1%.%$YEAR%-%$MONTH%-%$DAY%.log"
:syslogtag, contains, "example"                    -?DailyPerHostLogs-example
& ~

This will create /var/log/websites/example/example.2011-08-17.log and new files for each day. To add other websites, duplicate the file and replace example with another string. We’ll later use “example” in the Drupal configuration for Syslog later that will correlate to the correct file.

For those curious about this configuration: a template is needed for dynamic file paths with year-month-day. The syslogtag variable will use the string sent by Drupal in the Syslog module settings. Drupal will send it as example:, so :F,58:1 will use the colon as the field separator and grab the first field. Even though it would appear as though this single template could be used on multiple sites, I found I still had to have a unique template for each website or all websites would dump into the same file (perhaps this is remedied in newer versions of rsyslog). The last line means that the syslog input will not be sent to any other syslog files.

Reload the current rsyslog configuration, and the server is set up:

1
/sbin/service rsyslog reload

On the clients, we’ll create simple .conf files for each website. Create /etc/rsyslog.d/example.conf on each additional web server containing:

1
2
# example.com
:syslogtag, contains, "example"     @192.168.1.20

Note that the IP address can also be a domain name, though I would not recommend it to keep another process from being needed for the lookup.

Reload the current rsyslog configuration, and each web client is set up:

1
/sbin/service rsyslog reload

Enable the syslog module and go to http://example.com/admin/settings/logging/syslog**. Set the Syslog Identity to match the value used in /etc/rsyslog.d/example.conf; in this case, example**. Using the rsyslog filter we’re using, it does not matter which facility we send to. Save the configuration.

rsyslog-screenshot

Go to your rsyslog server and watch the /var/log/websites/example/example.2011-08-16.log file. After verifying that the server is receiving all messages, disable the Watchdog module in **http://example.org/admin/build/modules/list**.

There are improvements that can be had here, like using TCP instead of UDP, using LogAnalyzer to filter through the files, and more. This was meant to be an introduction into getting combined text files of your watchdog output while reducing the strain on your database by not writing to it as often.