Cchost/Developers/Victor's Dev Setup
I recently had the opportunity to install a new development machine using the latest tools. This is how my ccMixter/ccHost development machine was set up.
- Ubuntu: 8.04 - the Hardy Heron - released in April 2008.
- Apache: Apache/2.2.8 (Ubuntu)
- PHP: PHP/5.2.4-2ubuntu5.3 with Suhosin-Patch
- mySQL: 5.0.51a (Ver 14.12 Distrib 5.0.51a, for debian-linux-gnu (i486) using readline 5.2)
- Subversion: Client 1.4.6
- RapidSVN 0.9.4
- Meld: 18.104.22.168
- ccHost 5.0.1 (Committed revision 11342 Nov. 29, 2008)
I used the Wubi installer to get Ubuntu on my Windows laptop.
I end up as user 'victor' in group 'victor'.
After installation, there were 195 'updates' and I got all of those.
I then went out and got and verified the pieces of LAMP:
sudo apt-get install apache2 sudo apt-get install mysql sudo apt-get install php5 libapache2-mod-php5 sudo apt-get install php5-cli
Because I'm using the PHP module (not CGI) Apache will perform operations in 'nix as a user in a group that will need write permissions to several directories. In order to ease this problem I change the default group that apache will run in to the same as me ('victor'):
cd /etc/apache2 sudo vi envvars
Then edit the line:
and change www-data to victor
I next want to enable mod_rewrite:
cd /etc/apache2/mods-enabled ln -s ../mods-available/rewrite.load rewrite.load
And you need to enable .htaccess overwrites:
cd /etc/apache2/sites-available sudo vi default
Under <Directory /var/www/> change the line:
I prefer to keep my localhost website for junky experiments so I create a virtual Apache host for ccMixter. Create the new file:
sudo vi /etc/apache2/sites-available/cchost
Enter the following:
<VirtualHost *> DocumentRoot /var/www/cchost ServerName cchost </VirtualHost>
Then enable it:
ln -s /etc/apache2/sites-available/cchost /etc/apache2/sites-enabled/cchost
Then map the name for my local machine:
sudo vi /etc/hosts
Add the line:
Setting up the ccHost Database
Even though I'm going to mirror the ccMixter database here, it doesn't matter what the names are here locally:
mysql -p -u root mysql> CREATE DATABASE ccmixter; mysql> GRANT ALL ON ccmixter.* TO 'ccm_user'@'localhost' IDENTIFIED BY 'ccm_password';
I like curl
sudo apt-get install curl
I want to make the mirroring with ccMixter smooth so I create a directory that matches where we have getID3 on the ccMixter server:
sudo mkdir /var/getid3
Then I grab the zip from the net, unzip and do some clean up:
curl http://internap.dl.sourceforge.net/sourceforge/getid3/getid3-1.7.8b3.zip > getid3.zip unzip getid3.zip rm getid3.zip rm -f -r demos
The Creative Commons code server uses Subversion. There's a very usable and worthwhile GUI on Ubuntu and a fine diff program that I get:
sudo apt-get install subversion sudo aptitude install rapidsvn sudo aptitude install meld
Writable Repository Access
Since I need writable access to the CC code repository, I had to register my public key with the system.
If you only plan to do read-only updates from SVN you can skip this section entirely.
If you already have keys registered in your system you can skip this section entirely.
If you don't have ssh keys then you can use ssh-keygen and let the CC folks know about it to get writable access. And you can skip the rest of this section entirely.
My key was originally generated on Windows using the Putty tools. The 'nix system prefers OpenSSH version of the key files so I had to translate the PPK (Putty) version to that. I probably could have gone back to Windows to translate these but instead I installed the 'nix version the Putty tools:
sudo apt-get install putty
From my home directory I ran my key file through that to generate the OpenSSH key files:
puttygen -L victors_key.ppk > .ssh/id_dsa.pub puttygen victors_key.ppk -O private-openssh -o .ssh/id_dsa
The .ssh/id_dsa naming convention is magic and required, the files must be private read-only:
chmod 600 .ssh/id_dsa chmod 600 .ssh/id_dsa.pub
The ssh-agent daemon was already running in my install so all I had to do was register the keys (the passphrase was prompted for):
From this point on, all SVN access to the CC server would be done passing these keys back and forth and I never had to think about it again.
Getting the Code
Moving to /var/www I got the code:
svn co svn+ssh://firstname.lastname@example.org/svnroot/cchost/trunk cchost
If you're only doing read-only updates then its:
svn co http://code.creativecommons.org/svnroot/cchost/trunk cchost
IS THAT RIGHT????
We need 'cchost' directory writable by the web server. The group is already correct ('victor') because I created it so all we have to do is enable write by the group:
chmod 775 cchost
For cchost log files we write them to a directory that is not accessible to the web. Just to make the mirroring easier I created a log directory in the same location as ccMixter's on my local machine. (The actual location is different than the one here but the steps are the same):
sudo mkdir /var/log/cchost sudo chgrp victor /var/log/cchost sudo chmod 775 /var/log/cchost
Verifying ccHost Install
Finally ready to see if our setup is alive. We start by restarting Apache to get our config changes to take hold:
sudo /etc/init.d/apache2 restart
Kill all instances of the browser, then browse to http://localhost/cchost
NOTE: You may have to log out of Ubuntu or possibly reboot and get all the config stuff to take.
That brings up the 'cchost has not been properly install...' with a link to ccamdin' subdirectory. Follow that link to start the install.
After the install the Setting up your PHP environment screen tells me that /etc/php5/apache2/php.ini should be edited with different defaults and I go do that.
Click on the 'One more thing' link. Then rename the /var/www/cchost/ccadmin to something bogus.
Click through all the links and ccHost is up and running with defaults.
Import ccMixter Config
I have a bash script that will get the latest config from ccmixter.org and import it into the dev machine. The actual script using tons of piping in just three commands:
sqlite3 ~/.mozilla/firefox/vkuuxfit.default/cookies.sqlite "SELECT ':1',lastAccessed,':2',value FROM moz_cookies where name = 'lepsog3' and host = 'ccmixter.org'" | sed 's/:1|/ccmixter.org FALSE \/ FALSE /; s/|:2|/ lepsog3 /' > ccm_cookie curl -b ccm_cookie http://ccmixter.org/export | sed 's/ccmixter.org/cchost.org/; s/Download, Sample, Cut-up, Share./ccHost DEV SITE/' > cch_import cd cchost/bin php -f cc-host-config-import.php ../cch_import
Below I break it up to annotate it.
The first thing I want to do is call http://ccmixter.org/export while logged in as admin. I can fake this by grabbing the ccMixter cookie out of Mozilla's Firefox. The cookie is usually in a directory like:
I run a SQL query on it with place holders to a file called 'x':
sqlite3 cookies.sqlite \ "SELECT ':1',lastAccessed,':2',value FROM moz_cookies where name='lepsog3' and host='ccmixter.org'" \ > /var/www/x
I then substitute the place holders so that it forms a proper browser cookie:
cd /var/www cat x | sed 's/:1|/ccmixter.orgTABFALSETAB\/TABFALSETAB/; s/|:2|/TABlepsog3TAB/' > ccm_cookie
NOTE: In order to make things clear in this wiki, I've bolded the tab characters - you should replace those with actual tab characters (ascii 7).
I now have a cookie and can use curl to call over to ccMixter and get the config as if I was logged in as admin. I save it to a file called 'xx'
curl -b ccm_cookie http://ccmixter.org/export > xx
The file 'xx' refers to ccmixter.org all over the place so I replace it with my dev domain. I also replace the site description in the banner so I don't forget which site I'm on (!):
cat xx | sed 's/ccmixter.org/cchost.org/; s/Download, Sample, Cut-up, Share./ccHost DEV SITE/' > cch_import
At this point I could browse to my dev site and using the import command:
but that's overkill considering we're right on the machine and can call a script that does the import:
cd cchost/bin php -f cc-host-config-import.php ../cch_import
Import ccMixter Data
To mirror the data I use mysqldump on the CC server, sftp the results down to my dev laptop, then import the sql file using mysql command line. I leave that part as an exercise for the reader. (Just remember to NOT transfer the cc_tbl_config table.)