Code Elements: January 2007

Tuesday, January 30, 2007

Example Development Environment

This is an outline of a development architecture I put together for a client.

A example software development environment utilizing, Linux, Apache, PHP, and Subversion.

No testing environment is shown here but it is easily added by cloning the dev env.

Outline of the Sandbox Environment

The Development Environment

The Production Environment

Friday, January 26, 2007

PHP Data Objects

DataObject is a design pattern that incapsualtes the database with a simple to use interface. This alows the Controller to interact with the database layer in an SQL'less way. To acomplish this you build a base DataObject abstract class that implements the base functionality and sets up the interface for a Data Object. Then for each of your 'Business Objects' (such as customer), you extend the DataObject class and implement the various methods provided by DataObject. (see attached code for details).

You'll in the posted PHP files below that the code gets more 'complex' as we move down the stack (Controller->BusinessObject->DataObject). This is the beauty of OOP since we have implemented the DataObject only once in our application (it may be used by many Business Objects). As long as we maintain its interface, we are free to refactore its implementation details and add functionallity. And there is only one file to change.

Retrival of object from the database

In the simple case of the controller needing to retrieve a customer. Traditionally this might involve building a query of something like...

This process involes the Controller having far too much knowelge of the Database layer. This tight coupling should be avoided.

The same process using a database object would go something like this...

Search the database for an Object

This is quite elegant with DO's. Just simply create a new DO, set what you do know about what you are seaching for then execute the object find method.

Saving and Updating Objects

Again this is very simple also. For a populated customer DO object, just issue its insert method and to update, use the update method.

Here is a Proof of Concept Controller PHP script

/** * This is just a Proof Of Concept, this is not Production code. * * This is the controller script, that in a very simple * way allows you to Insert/update and delete customers * using the Data Object Class. */ require("../includes/bootstrap.php"); require_once('CustomerDO.php'); $feedback = ""; // collect the id of the image to edit $customer_id = isset($_GET['customer_id']) ? hd($_GET['customer_id']) : null; /** * Create the CustomerDO object. The DB connection * is an argument of the constructor (Dependency Injection) */ $customer = new CustomerDO($db); /** * If there is a customer id provided, retrieve it from the DB */ if ($customer_id) { $customer->get($customer_id); } if (isset($_GET['submit'])) { $customer->first_name = hd($_GET['first_name']); $customer->last_name = hd($_GET['last_name']); if($customer->customer_id){ $customer->update(); $feedback = "Update Complete"; } else { $customer->insert(); $feedback = "Creation Complete"; } } else if(isset($_GET['delete']) && $customer->customer_id) { $customer->delete(); $feedback = "Delete Complete"; } // Set values for the template $first_name = $customer->first_name; $last_name = $customer->last_name; $customer_id = $customer->customer_id; // Just using simple include rather than template engine(i.e. Smarty) include("customer/index.tpl");

Here is the Customer Data Object that extends DataObject

require_once('DataObject.php'); /** * This is the Business Object class for Customer. * It extends the DataObject class and implemets * the needed interface methods of DataObject * */ class CustomerDO extends DataObject { /** * Specific C.R.U.D. queries for Customer * */ const QUERY_GET = " SELECT * FROM customer WHERE customer_id = ?"; const QUERY_INSERT = " INSERT INTO customer SET `customer_id` = ?, `first_name` = ?, `last_name` = ?, `create_date` = NOW(), `modified_date` = NOW()"; const QUERY_UPDATE = " UPDATE customer SET `first_name` = ?, `last_name` = ?, `modified_date` = NOW() WHERE customer_id = ?"; const QUERY_DELETE = " DELETE FROM customer WHERE customer_id = ?"; const QUERY_FIND = " SELECT * FROM customer"; /** * Allow for an optional Database handle param. This * Inversion Of Control allows for ease of unit testing. i.e. you * can supplay a mock database object * * @param Object $db DatabaseHandle */ public function __construct($db=null) { $this->init(); $this->db = $db; } /** * Set up the initial values for this objects * attributes. * */ private function init() { $this->attributes = array( 'customer_id' => '', 'first_name' => '', 'last_name' => '' ); } /** * Retrieve a specific object by its primary key * * @param mixed $data_object_id */ public function get($data_object_id) { parent::get($data_object_id, self::QUERY_GET); } /** * Persist a new object to the DB * */ public function insert() { $this->customer_id = $this->db->nextId(parent::DB_SEQUENCE_TABLE); $params = array( $this->customer_id, $this->first_name, $this->last_name ); $result = $this->db->query(self::QUERY_INSERT ,$params); db_check_error($result); } /** * Update an existing Object in the DB * */ public function update() { $params = array( $this->first_name, $this->last_name, $this->customer_id ); $result = $this->db->query(self::QUERY_UPDATE ,$params); db_check_error($result); } /** * Permanently remove an object from persistence * */ public function delete() { $params = array($this->customer_id); $row = $this->db->query(self::QUERY_DELETE ,$params); db_check_error($row); } /** * Query for all records matching the state of the current * Object. * * @param boolean $as_detached_recordset * @param string $order_by_field "{fieldname} {ASC | DESC}" * @param string $group_by_fields "{fieldname1, fieldname2...}" * @return ReadOnlyResultSet or array if $as_detached_recordset = true */ public function find($as_detached_recordset=false, $order_by_field=false, $group_by_fields=false, $return_structure=false) { return parent::find(self::QUERY_FIND, $as_detached_recordset, $order_by_field, $group_by_fields,$return_structure); } /** * Utility function to retrieve the number of objects in persistance * matching a given 'find' * * @return unknown */ public function count() { return count($this->find(true)); } }

And here is the Base Data Object Class

/** * This is the base DataObject Class. * * @package default * @author Sam **/ abstract class DataObject { protected $attributes; protected $db; // for debugging use public $sql; protected $attrib_to_dbfield_mapping = array(); /** * Default get implementation * * @param unknown_type $newsletter_id */ protected function get($do_id, $get_query) { $params = array($do_id); $row = $this->db->getRow($get_query ,$params,DB_FETCHMODE_ASSOC); db_check_error($row); if ($row) { foreach ($this->attributes as $class_attribute => $value) { if (key_exists($class_attribute,$row)) { $this->$class_attribute = $row[$class_attribute]; } } } } protected abstract function insert(); protected abstract function update(); protected abstract function delete(); protected abstract function count(); const DB_SEQUENCE_TABLE = 'sequence_ids'; /** * @todo implement OR, right now only AND is supported... */ const CONJUNCTION_AND = 'AND'; const CONJUNCTION_OR = 'OR'; public function __get ($attribute) { if (! key_exists($attribute,$this->attributes)) { throw new Exception("Attribute: [$attribute], not defined for ".__CLASS__); } return $this->attributes[$attribute]; } public function __set ($attribute, $value) { if (! key_exists($attribute,$this->attributes)) { throw new Exception("Attribute: [$attribute], not defined for ".__CLASS__); } $this->attributes[$attribute] = $value; } public function getAttributeList() { return array_keys($this->attributes); } /** * Used to build clauses for sql queries. * Assumptions: * - All date values are supplied as UNIX timestamps * * @param string $field_name * @param string $clause The expected format of the $clause param is * - '{ <= | <|>= |> } {value}' * - i.e. ">= 17645329" * - i.e. "!= Y" * - i.e. "Y" // = (equals) is inferred by default * a space is required to seperate the boolean clause and * the value. If no boolean clause is given, '=' is * assumed. */ protected function buildWhereClause($field_name, $clause) { $clause_boolean = '='; //default is equals '=' $clause_righthand_value = $clause; $hack_booleans = array('<=','>=','<>','=','!=','<','>'); foreach ($hack_booleans as $boolean) { if (strpos($clause,$boolean)===0) { $clause_boolean = substr($clause,0,strlen($boolean)); $clause_righthand_value = trim(substr($clause,strlen($boolean)+1)); break; } } } /** * check date special case (righthand value should be a timestamp) * - be sure to use the attribute name if it exists (rather than the db field name). That way we can * rely on the '_date' naming convention. */ $date_attribute_name = array_search($field_name,$this->attrib_to_dbfield_mapping) ? array_search($field_name,$this->attrib_to_dbfield_mapping) : $field_name; if (strstr($date_attribute_name,'_date')) { if (! is_numeric($clause_righthand_value)) { throw new Exception("The righthand value for [{$field_name}]: not in correct format. Should be {boolean} {unix timestamp}. (" ."i.e. usage: 'expire_date' => '>= '.time())"); } $where = "`{$field_name}` {$clause_boolean} FROM_UNIXTIME('" . mysql_escape_string($clause_righthand_value) . "')"; // Not date, so put clause together normally } else { $where = "`{$field_name}` {$clause_boolean} '" . mysql_escape_string($clause_righthand_value) . "'"; } return $where; } /** * Maps DO attribute names to DB field names. If a mapping for * the given attribute does not exist, the attribute name is simply * returned * * @param string $attribute_name * @return string */ protected function mappedAttributeName($attribute_name) { return isset($this->attrib_to_dbfield_mapping[$attribute_name]) ? $this->attrib_to_dbfield_mapping[$attribute_name] : $attribute_name; } /** * Query for all records matching the state of the current * Object. * * @param strinh $find_query The sql query (w/out the where clause) * @param boolean $as_detached_recordset * @param string $order_by_field "{fieldname} {ASC | DESC}" * @param string $group_by_fields "{fieldname1, fieldname2...}" * @return ReadOnlyResultSet or array if $as_detached_recordset = true */ public function find($find_query, $as_detached_recordset=false, $order_by_field=false, $group_by_fields=false, $return_structure=false, $limit_terms=false) { if (! $as_detached_recordset && $return_structure) { throw new Exception('Usage: If you requests a $return_structure, you must specify $as_detached_recordset=true'); } $where_clauses = array(); // Loop through the properties build individual where clauses foreach ($this->attributes as $attrib_key=>$attrib_values) { /* * attribute values can have multi value so force to array & * deal only with the array case. * This allows you to define something like * WHERE * username != 'Anonymous' AND * username != '' */ $attrib_values = is_array($attrib_values) ? $attrib_values : array($attrib_values); foreach ($attrib_values as $attrib_value) { if (trim($attrib_value)!='') { $where_clauses[] = $this->buildWhereClause($this->mappedAttributeName($attrib_key), $attrib_value); } } } // If we have a where clause, build it $this->sql = $find_query; if (count($where_clauses)> 0){ $this->sql .= " WHERE " . implode(' '.DataObject::CONJUNCTION_AND.' ', $where_clauses); } if ($group_by_fields) { $group_by_fields = explode(',',$group_by_fields); $group_by_fields = array_map('trim',$group_by_fields); $this->sql .= " GROUP BY `".implode("`, `",$group_by_fields)."`"; } if ($order_by_field) { $order_by_field = explode(',',$order_by_field); foreach ($order_by_field as &$order_by) { $direction = stristr('DESC',$order_by) ? "DESC" : "ASC"; $order_by = str_ireplace(array("DESC","ASC"),'',$order_by); $order_by = "`".trim($order_by)."`"." {$direction} "; } $this->sql .= " ORDER BY ".implode("`, `",$order_by_field); } if ($limit_terms) { $this->sql .= " LIMIT ".$limit_terms; } $results = null; if ($as_detached_recordset) { $results = $this->db->getAll($this->sql,array(),DB_FETCHMODE_ASSOC); db_check_error($results); if($return_structure) { $results = $this->mapReturnStructure($results,$return_structure); } return $results; } else { $result_set = $this->db->query($this->sql); db_check_error($result_set); require_once('ReadOnlyResultSet.php'); return new ReadOnlyResultSet($result_set); } } /** * Takes the indexed array result of a query and re-structures it to * the supplied assocc array. * - ie * <code> * if $result_data is * [0]=>array('name'=>'foo','age'=>10, 'height'=>6.5, 'weight'=>100), * [1]=>array('name'=>'bar','age'=>13, 'height'=>6.5, 'weight'=>120) * and $return_structure is * 'name'=>array('age','weight') * then the final returned array would be... * 'foo'=>array('age'=>10,'weight'=>100), * 'bar'=>array('age'=>13,'weight'=>120) * * if there is only 1 array value in the return structure.. * i.e. 'name'=>'age' (or 'name'=>array('age') * then the final returned array would be...(no keys on value) * 'foo'=>10, * 'bar'=>13 * </code> * * @param array $result_data * @param array $return_structure */ private function mapReturnStructure($result_data, $return_structure) { $result = array(); foreach ($result_data as $datum) { $key = key($return_structure); $struct_values = is_array($return_structure[$key]) ? $return_structure[$key] : array($return_structure[$key]); foreach ($datum as $datum_key => $datum_value) { if (in_array($datum_key,$struct_values)) { if (count($struct_values)>1) { $result[$datum[$key]][$datum_key] = $datum_value; } else { $result[$datum[$key]] = $datum_value; } } } } return $result; } }

Very Simple Captcha

Very Simple Captcha or "Poor Man's Captcha"

This is a very simple way to implement a Captcha concept. It is less effective than say a generated on the fly image (using something like GD), but it is far better than using nothing at all.

Summary of Usage

In the PHP scripts that controls a page that requires a Captcha, simpley call the static function Captcha::generateCaptchaCode().
Then in the template of the page use Captcha::getHTMLCaptcha() to retrieve the HTML markups for the images representing the code.
Then on submision of the form with the captch, check the submitted value against the code in the session ($_SESSION['captcha_code'] by default)

Example Captcha Image

Simple Usage Example

Code for the Static Captcha class

/** * Very simple captcha solution. I had a client who's * server was not capable of generating images on the * fly (GD), so I just generated my own images (digits * 0-9) and use them. Not the most secure Captcha, but * better than nothing. * */ class Captcha { const CAPTCHA_IMAGE_DIR = "/images/captcha/"; // Range of characters available for captcha private static $available_captcha_chars = array( '1', '2', '3', '4', '5', '6', '7', '8', '9', '0' ); private static $captcha_image_map = array( 1 => '_0n3', 2 => '_tw0', 3 => '_thr33', 4 => '_f0ur', 5 => '_f1v3', 6 => '_51x', 7 => '_53v3n', 8 => '_31ght', 9 => '_n1n3', 0 => '_z3r0', ); /** * This is the key used to store the catcha in the $_SESSION. * * @var string */ private static $session_catcha_name = "captcha_code"; /** * Generate the random code and place it in the session * * @param int $number_of_captcha_chars * @return void */ public static function generateCaptchaCode($number_of_captcha_chars = 4) { // reset code $captcha_code = ''; $available_captcha_chars = self::$available_captcha_chars; $rand_max = count($available_captcha_chars) - 1; // loop through and generate the code letter by letter for ($i = 0; $i <$number_of_captcha_chars; $i++) { // get random index $char_index = rand(0, $rand_max); // append char to captcha string $captcha_code .= $available_captcha_chars[$char_index]; } // Set the current catcha code into the session $_SESSION[self::$session_catcha_name] = $captcha_code; return $captcha_code; } /** * For a given captcha code, returns its html * image representation * * @param string $captcha_code * @return string */ public static function getHTMLCaptcha ($captcha_code) { $HTML_output = "<div>\n"; for ($i=0;$i<strlen($captcha_code);$i++) src="'">\n"; } $HTML_output .= "</div>\n"; return $HTML_output; } /** * Verifiy that the given captcha matches that * in the session * * @param string $supplied_captcha * @return boolean */ public static function validateCaptcha($supplied_captcha) { return strtoupper($supplied_captcha)==$_SESSION[self::$session_catcha_name]; } }

Tuesday, January 23, 2007

Drupal Caching

Drupal 5.0 With caching off

ab -c100 -n10 http://example.com/index.php
This is ApacheBench, Version 2.0.41-dev <$Revision: 1.141 > apache-2.0
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/

Benchmarking example.com (be patient).....done

Server Software:        Apache/2.0.59
Server Hostname:        example.com
Server Port:            80

Document Path:          /index.php
Document Length:        10541 bytes

Concurrency Level:      100
Time taken for tests:   3.417284 seconds
Complete requests:      10
Failed requests:        6
  (Connect: 0, Length: 6, Exceptions: 0)
Write errors:           0
Total transferred:      110154 bytes
HTML transferred:       104774 bytes
Requests per second:    2.93 [#/sec] (mean)
Time per request:       34172.840 [ms] (mean)
Time per request:       341.728 [ms] (mean, across all concurrent requests)
Transfer rate:          31.31 [Kbytes/sec] received

Connection Times (ms)
             min  mean[+/-sd] median   max
Connect:       92   93   0.6     93      94
Processing:  1351 2433 652.3   2757    3322
Waiting:     1158 2222 668.5   2566    3129
Total:       1444 2526 652.6   2850    3416

Percentage of the requests served within a certain time (ms)
 50%   2850
 66%   3059
 75%   3087
 80%   3095
 90%   3416
 95%   3416
 98%   3416
 99%   3416
100%   3416 (longest request)

Drupal 5.0 With caching on (Normal)

[sam@vps ~]$ ab -c100 -n10 http://example.com/index.php
This is ApacheBench, Version 2.0.41-dev <$Revision: 1.141 > apache-2.0
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/

Benchmarking example.com (be patient).....done

Server Software:        Apache/2.0.59
Server Hostname:        example.com
Server Port:            80

Document Path:          /index.php
Document Length:        10435 bytes

Concurrency Level:      100
Time taken for tests:   0.506276 seconds
Complete requests:      10
Failed requests:        0
Write errors:           0
Total transferred:      148446 bytes
HTML transferred:       139122 bytes
Requests per second:    19.75 [#/sec] (mean)
Time per request:       5062.760 [ms] (mean)
Time per request:       50.628 [ms] (mean, across all concurrent requests)
Transfer rate:          284.43 [Kbytes/sec] received

Connection Times (ms)
             min  mean[+/-sd] median   max
Connect:       93   93   0.7     93      94
Processing:   264  326  53.6    323     412
Waiting:      127  196  41.4    203     282
Total:        358  420  53.5    417     505

Percentage of the requests served within a certain time (ms)
 50%    417
 66%    430
 75%    473
 80%    492
 90%    505
 95%    505
 98%    505
 99%    505
100%    505 (longest request)

Drupal 5.0 With caching on (aggressive)

[sam@vps ~]$ ab -c100 -n10 http://example.com/index.php
This is ApacheBench, Version 2.0.41-dev <$Revision: 1.141 INSERT:CONTENT:ENDgt; apache-2.0
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/Benchmarking example.com (be patient).....done

Server Software:        Apache/2.0.59
Server Hostname:        example.com
Server Port:            80

Document Path:          /index.php
Document Length:        10541 bytes

Concurrency Level:      100
Time taken for tests:   0.469280 seconds
Complete requests:      10
Failed requests:        0
Write errors:           0
Total transferred:      144128 bytes
HTML transferred:       134804 bytes
Requests per second:    21.31 [#/sec] (mean)
Time per request:       4692.800 [ms] (mean)
Time per request:       46.928 [ms] (mean, across all concurrent requests)
Transfer rate:          298.33 [Kbytes/sec] received

Connection Times (ms)
             min  mean[+/-sd] median   max
Connect:       92   92   0.9     93      93
Processing:   245  314  44.4    336     377
Waiting:      133  162  19.0    169     188
Total:        338  407  44.3    429     469
WARNING: The median and mean for the initial connection time are not within a normal deviation
       These results are probably not that reliable.

Percentage of the requests served within a certain time (ms)
 50%    429
 66%    430
 75%    440
 80%    464
 90%    469
 95%    469
 98%    469
 99%    469
100%    469 (longest request)

Ponder Results

wow, certainly not a thorough test by any strech, but from ~3 req/sec (caching off) to ~20 req/sec with it on!
That is a quite impressive gain, 85%.

Saturday, January 20, 2007

Character Encoding 101

Although the discussion of character encoding for web developers is vast, This article by Tommy Olsson on Sitepoint is a great starting point. It does a supurb job of explaining the basics and defining the needed terms.
Another great article on encoding is over at Joel On Software

Get Lazy

After much deliberation and discussions with friends (thanks Mark), I have decided to embrace PHP5's built in __autoload function. I have chosen a method briefly documented on phpKitchen. For those of you who do not know what PHP5 __autoload is, no worry go here for a review

The short of it is for a class in the path {base class path}/persist/database/DbAdapter.php the class in the file dbAdapter.php would be named persist_database_DbAdapter. You then define an __autoload function similar to this...

So now what does this get us?? Instead of something like this...

can simply do...

$dbAdapter = persist_database_DbAdapter::getInstance();

Then when PHP tries to load the unknown class, persist_database_DbAdapter, the __autoload function will be called which will build the appropriate require_once statement to locate the class.

Now some may see this as casting the slight complexity of all those require_once statements only to add complexity of having strange class names like persist_database_DbAdapter. I may agree with this point of view but I would argue that there are much greater benefits to using __autoload() than just getting rid of those require_once statements and I will briefly summarize two of them below.

The first advantage (lesser of the two), is one of code readability. Whereas above I described the class name of persist_database_DbAdapter as ”strange”, I would suggest that the long class name actually helps to describe the true purpose of the class, making the code more readable by just the nature of the class name. Thus reducing the need to clutter the code with /* comments */.

The second advantage is much more substantial one and it involves what is often referred to as Lazy Initialization. What I often see happening in PHP is that required files for a Class are often added at the top of the file (outside of any methods). With this setup, whenever the Class is referenced all these requires are parsed and included regardless of if all the files they are including will actually be used on the particular Request cycle. Depending on the size of some of the files being parsed, this can be quite an expensive operation. For example, with the code below...

If getBar() is never called for this particular use of the class Bar, the process time of require_once(CLASS_PATH.'/foo/Bar.php') has been waisted. On the other hand if we use __autoload() the above code becomes simply...

So, in this case the require_once(CLASS_PATH.'/foo/Bar.php'); only occurs the instant before it is actually required (and never if it is not required). This is essentially the performance enhancement Pattern know as Lazy Initialization and when you are dealing with large and/or numerous class files, the processor time savings can be quite substantial.

Sunday, January 14, 2007

Back to school

For those of you who are into education for the knowlege (rather than the diploma), be sure to check out MIT Opencourseware. The couse selection is imense and the course materials are thourough and well organized. You will not get any ‘official’ credit for taking these courses but you cannot beat the price.

Code Elements