August 28, 2008

Does fruit and veg drag on?

Do we eat enough fruit and vegetables? I think I'm - personally - borderline with the recommended "five per day"; I take it as a flag to remind me that I should watch my dietin this area rather than something I keep careful track of - and in any case why should an apple and an orange count as "2" whereas 2 apples (or 2 oranges) would just count as 1.

The flag has encourgaed Lisa and I to have a delivery, every 2 weeks, of a vegetable box contgaining whatever's in plentiful supply at the current time of year, augmented by a number of other tempting, wierd wonderful and unusual things that we probably wouldn't pickk off the shelves for ourselves. This thing on the left (do you know what it is?) has appeared in this week's box - together with instructions - and I was left with a choice of "should I eat it or protograph it? You see which option won out ... but I guess I can now go off and eat it if Lisa hasn't already done so.

[[Oh - it's a Dragon Fruit which is why there's that dreadful pun in the title of this item. G and S fruit, who make up and deliver the box, may be found here]]

Posted by gje at 07:50 AM | Comments (0)

More about Graham Ellis of Well House Consultants

Easterholic

I've always been a little intrigued by awkward algorithms and conversions - such as Orndance Survery grid references to Latitude and Longitude (see here) and when Easter will fall (see here) for some earlier comments - but little did I think when I posted about Ken Palm and his Domain Name Sampling scheme where he's trying to sell someting he borrow for free for hundreds of pounds (see here) the other day that I would get an email from someone with a similar - or (let's be fair) far deeper interest in the date of Easter. Yet I shouldn't be surprised - for when I posted up about the Domain Name Registry of America and wwmdirectory in the past, I've received a number of responses from a number of people who strike me as honest and hard working who have either been taken in or - in the case of the Gentleman of Easter - have also smelled a rat.

Oh - the "Date of Easter" page is to be found at www.dateofeaster.net/ and thanks to the alert provided by Ken Palm, also at www.dateofeaster.com. David Harper (David's home page) writes "Ironically, I ought to thank Ken for alerting me to the fact that the domain was finally available. Maybe I should send a postcard to his Colorado mailing address, which appears to be the local UPS store ...";

Posted by gje at 07:48 AM | Comments (0)

August 27, 2008

What is my real and my effective ID? [Linux]

When you're logged in to a Linux or Unix system, you should normally be running through a user account - and not as the system administrator (root). The root account should be reserved for systems admin tasks where the extra authority is needed.

The who, users, whoami and id commands all tell you who you are running as ...


[trainee@crocus ~]$ who
trainee pts/0 Aug 25 15:39 (192.168.200.90)
[trainee@crocus ~]$ who am i
trainee pts/0 Aug 25 15:39 (192.168.200.90)
[trainee@crocus ~]$ users
trainee
[trainee@crocus ~]$ id -a
uid=500(trainee) gid=48(apache) groups=48(apache)
context=user_u:system_r:unconfined_t
[trainee@crocus ~]$ whoami
trainee
[trainee@crocus ~]$

If you need root access, you should gain it using the su command with the - option - and that changes your effective user id to root, but leaves your real id as your original user.

The who and the users command will report your real id.
The id -a and the whoami command will report your effective id.

[trainee@crocus ~]$ su -
Password: ********
[root@crocus ~]# who
trainee pts/0 Aug 25 15:39 (192.168.200.90)
[root@crocus ~]# who am i
trainee pts/0 Aug 25 15:39 (192.168.200.90)
[root@crocus ~]# users
trainee
[root@crocus ~]# id -a
uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),
4(adm),6(disk),10(wheel) context=root:system_r:unconfined_t
[root@crocus ~]# whoami
root
[root@crocus ~]#

[b]Remember[/b] to exit as soon as you have done your root-y stuff from the root account!

Posted by gje at 06:01 PM | Comments (0)
Useful link: Linux training

August 26, 2008

The Longest Possible Day

I post here every day - or I try to at least. What is the longest possible gap between posts for me to leave, and still achieve that? Well - it's 48 hours and I'm very close to that today.

Not that it's been quiet - in fact I had my head down yesterday, taking a whole lot of pages that we transferred from the old server like the Open source Quiz pages and updating them to something more recent (quiz index, a quiz replacement page and some of the old questions), labelling images on pages for images searches at www.wwuu.co.uk, and loading Truetype fonts into our PHP installation, and enabling the necessary functions. This has allowed me - amongst other things to add dynamic wording on our First Great Western Train Running Diagram so that users of The Coffeeshop can see what's up at a glance, even on a day when there's a lot up. The we found that there were some sluggish responses from our server, which co-incidentally (?) always seem to come from at times when certain visitors are around. So they had to be looked into and dealt with on top of everything else, and as a matter of some priority.

Today has been "just" a Linux course - well, I say just a course but it's been fabulous fun as usual, and a great group of five, and it carries on tomorrow. Really rewarding course to give. Plus this, plus that, plus the other. But that's all lead to a gap that's going to be around 44 hours by the time I post his.

I was musing on the hotel side as to our longest gap between checkins. We have a lot of hotel guests from the USA, and they often arrive at Heathrow on morning flights before coming to Melksham, checking in to the hotel as early as 10 or 11 in the morning. If we know ahead of time, we're quite happy with this - especially at the start of the week, for a week's stay. And then at the end of the day, there are the people who spend a full day at home or work before travelling to their Melksham Hotel, perhaps not realising just how far it is for them - or guests who come in to Bristol Airport n the last flight from Dublin ... which then gets delayed. Again - we're quite open to this (we wouldn't be in the business if we weren't) but it can make for a mighty long day. Longest gap between checkins? On a Sunday - around 12 hours!

Posted by gje at 07:48 PM | Comments (0)

August 25, 2008

Ken Palm, iTime, and Domain Name Tasting

Have you come across "domain name tasting" - try before you buy from ICANN. It is the practice of registering a domain name (and placing pay-per-click ads on it for a few days) to gauge whether the ads will make more than the annual cost of the domain. Read more about it here (link opens in separate window). Apparently, "domain name tasting" has developed from a five day grace period on domain registration which allows people to turn in domains that they have registered by mistake.

How did I come across Domain Name Tasting? From researching a Ken Palm of i-Time Marketing, Falcon, Colorado ... or of flkdomaindiscovery.com or of thepreferredurl.com or of Davenport, Iowa.

Ken wrote to me on Monday, offering to sell me phpquiz.com for $557.

He wrote to me on Tuesday, with something he had forgotten to mention.

On Thursday, he sent me a reminder and a nice link I could buy through.

Saturday, and he tells me about a limited period discount he's offering.

And on Sunday he writes "Make me an offer" ...

I don't actually want to register phpquiz.com - we already have www.phpquiz.co.uk, but something in Ken's emails - his extravagant claims for the domain - made me have a look into him and I discovered what appears to be a "scam", where he tastes a domain for a few days, during which time he tries to sell it to interested parties in exactly the way he has tried to sell to us. I understand that if you are REALLY interested in a domain that Ken Palm is looking to sell, are willing to take a very slight risk and wait a few days for him to return it, you'll be able to register it for $10 or so. I learned that from here and here

Posted by gje at 12:47 AM | Comments (0)

August 24, 2008

More HowTo diagrams - MySQL, Tomcat and Java

Here are some more diagrams "off the wall" from last week - showing "howto" and "whatis" information for Tomcat, Java and MySQL

Some Tomcat and Java Fundamentals

On the right of the board ... the main java programs ...
• the Java compiler is called javac
• the Java Virtual machins is in a program called java
• the jar utility is used to bundle and unbundle lots of java files ("classes") and directories ("packages") and associated files too.
• javap may be used to look inside a class file if you want to find what externally visible members it contains.

The following environment variables are ones you should know about ...
• CLASSPATH which defines where you find Java classes
• PATH for the path on which the operating system searches for programs
• JAVA_HOME to tell Tomcat where it should Java

On the left of the board ... the crucial configuration files ...
• server.xml to define the main server configuration
• web.xml to define the base parameters for each web applications. Settings may be overridden by individual web.xml files within each applications.
• tomcat-users.xml to define the access account to the Tomcat Manager and potentially to other applications too if your programmers have used the facility.

A diagram of Tomcat ...
A SERVER (outer box) runs a SERVICE (next box) on an ENGINE (next box) supporting a HOST (inner box). A number of connectors (red splodges on the left) give access in to the server / service, supporting different protocols with different metrics. Very often, a production server will have just two services running - a SHUTDOWN service which may or may not be on the default 8005 port, and either an http or an ajp service - common port number are 8080, 8082 and 8009.

Some other words that crop up ...
• ant is a Java build tool - defining what has to be done to put together and install an application through a series of rules and dependencies. For those readers who have come across make, you may consider ant to be the Java equivalent.
• jasper The class that runs in your Tomcat Engine and lets you run JSPs (Java Server Pages)
• jikes The Just in Time Java Compiler ... rather than precompile everything, Just in Time technology lets you compile Java on the fly.

Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here

The Connector Story

The majority of users of the Tomcat server also run an Apache httpd web server to "front" it - for load balancing, logging, serving out static pages, providing PHP support on the same domain as Java, for security, and for other reasons. The way it typically works is that it's httpd which is providing the post that's visible to web clients (browsers) and then httpd acts as a client to the Tomcat server - it's rather like you calling up your telephone company and talking to a single individual on the phone, but that individual going off and talking to others and / or looking things up on systems to help get you the answer whether it's direct from the person you're talking to or not.

There are at last 8 ways of combining httpd and Tomcat, with new options developing over time, older ones being deprecated and even (in the case of jk) a method being deprecated and then recalled. This board tells the story - but the full story is told (and the practicals done) on the course.

If I summarise the story ... these days, you'll probably choose mod_proxy or mod_jk, each of which has its own metrics. In some specific circumstances, mod_rewrite comes into its own too (and mod_rewrite is a fabulous module for other things!

Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here

MySQL - release levels

MySQL was an excellent database produce even back at release 3.23 - that's when it really took off - but it has come a long way sense, right through to the MySQL company being bought by Sun Microsystems earlier this (2008) year for - we are told - 1 billion dollars.

This board summarises some of the metrics that have changed between the various releases, and some of the things that you (the person deploying them or programming for them) really should be aware of.

In olden days, data was held in a table format called isam which was replaced by myisam ... or rather myisam was added, as the MySQL team is very careful to maintain data integrity so the old ones will still work. MyISAM tables do not support transactions, which were added around release 4.0, together this InnoDb tables to support them. Stored procedures were added at release 5.

One of the most significant changes, though, was actually made between releases 4.0 and 4.1, when the drivers and in particular the security of passwords were beefed up. That's no bad thing in that the connection from the clients to the MySQL server is now better protected that it was, but it has the unfortunate side effect thatit means that you can upgrade your MySQL from a 4.0 or earlier release to a 4.1 or later release and find that you applications no longer work ... that you need to rebuild PHP for example! There ARE ways that you can instruct a recent MySQL to accept logins to specific accounts with the older and less secure passwords, which is an excellent short term fix, but that's going beyond the scope of this particular board.

Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here

Table Types in MySQL

The MySQL database supports a number of different table types (formats) which are listed here.

• Isam Which are the oldest type, and now largely superseded by MyIsam which have better data integrity in the even of an unexpected system halt, and are portable in their binary format between bigendian and littleendian processor systems.
• MyIsam The default table type - ideal for many straightforward applications, with the data held (but you should NOT access it directly!) in directories with the same name as the database, and in there in files with the same name as the table names.
• Heap A memory based table, ideal for very quick access but limited as to the column types you may use, and lost on server restart.
• bdb MySQL supports the Berkeley Database Format which allows you to use MySQL based applications to directly edit thinsg like NIS tables, mail aliases and the X Windows System colour tables.
• InnoDb InnoDb tables are typically used for the more "meaty" applications. They're stored in large files which are preallocated to help with efficiency at the Operating System level, and provide the ability to checkpoint and rollback - so that's transaction processing. They have not taken over from MyISAM tables as they're more involved to set up / use / administer, and some of the locking issues can leave them as being less rather than more practical in some circumstances.
• Cluster A highly specialised, memory based table type which (I hope) you'll not come across in your typical, day to day, use of MySQL. I understand it's designed for telephoney switching applications ...

Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here

A balancing scheme for servers. Also MySQL account strategys

The upper diagram on this page shows a scheme for load balancing (rather than clustering!) between a series of near-identically configured servers. All users of the web site contact httpd on the leftmost box, which farms out the heavy work to Tomcat instances on the following boxes. The Tomcat instances do MySQL selects (i.e. read from the database) on their own box, but save any changes back to the rightmost box. Finally, the MySQL master is on the rightmost box, with slave servers handing off it on all the other boxes.

For a read-mostly application (and this scheme was drawn up for such!) this is a good solution. For low traffic installations it can all work on one box, but that can be widened out as need be. In terms of system failure and recovery, the MySQL master / client scheme allows for easy failower, and in terms of all the http traffic coming in at the front, you could use a heartbeat script on the second (and other) machines to keep track, or an extra balancer.

The lower diagram shows the authorisation levels for a MySQL account:
• Accounts have global permissions over all databases granted in the user table in the mysql database, plus
• Permissions of a per database basis granted in the db table plus
• per-table permissions plus
• per-column permissions.

Since some permission types simply don't apply once you get down to the more limited scopes, there are a large number of possible permissions than can be granted globally, reducing to just a few on a per column basis. Remember that the permissions granted in each of the categories are ADDATIVE so that if some one is granted select_priv in the user table they can read anything being served, even if select_priv is turned off to them for specific databases.

Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here

One distribution for three separate computers

The MySQL distribution includes the Mysql daemon (mysqld) which you'll want to install on your database server computer, the MySQL client programs which you may want to install on your own desktop, and the libraries that you may need to have on your web server prior to building PHP. You may be a little suprised that all three come in the same download / distribution and wonder why they're not separated my view is that actually the MySQL folks have made the right decision on this one - it just needs a wee bit of explanation.

The board goes on to show you how the various elements work together, and how the daemon provides a "gatehouse" through which all database accesses must be made. Within the data compound, you'll see a division into a series of databases, with a number of tables within each of them.

Did you notice the file names /etc/my.cnf and ~/.my.cnf also on the board? MySQL is a database product and keeps the majority of its configuration information internally within its tables (if it didn't could well make a comment about having a dog and barking yourself). However, a handful of parameters are needed before the daemon and its client programs can actually reach the data - and those parameters go into a file called /etc/my.cnf (system wide) or .my.cnf in each user's home directory for each parameter that's set on a peruser basis.

Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here

Are these diagrams useful? They're the second (and final) instalment of a set - and youcan find the first instalment here

Posted by gje at 10:08 AM | Comments (0)
Useful link: Java training

3 hours from Milton Keynes

I finished training at the end of Friday in Milton Keynes ... with a 2 to 3 hour drive home through the traffic travelling to start the Bank Holiday. Why rush it? I kept off the motorways, and stopped on the way to take a picture or two. Here are some of the images from the beside road from Milton Keynes to Melksham ...


Walton Hall, the Open University, Milton Keynes


Wind Turbines in the Oxfordshire Countryside - taken from the A420 between Faringdon and Shrivenham.


Where the A420 (Oxford to Swindon road crosses the main London to Swindon and South Wales Railway, you'll find this twin arch bridge. It didn't always look like this - originally, the arch you can see on the left was for the road ... and the Wilts and Berks canal passed through the arch on the right.


Silbury Hill, near Avebury on the Marlborough Downs


Bailing the hay, on the Marlborough Downs - near Windmill Hill.


And so to Melksham - the old Spa houses on the Devizes Road, pictured in the low evening sun.

Posted by gje at 12:20 AM | Comments (0)

August 23, 2008

July child ponders on August children

It's the "result season" when each year we hear about the successes of our 18 year olds at A levels, and 16 year olds at GCSEs, with an inevitable (it seems) rise in pass rates and a cry of "they're getting too easy". I'm not going to jump into that particular controversy, but I will congratulate anyone who's just got their results on what they've achieved - if you put in hard work to get those results, then congratulations are in order.

But there's one subject that has come up - at least in the media I have been following - this year, in relation to how "August Children" are said to be disadvantaged. It works like this ... children in the UK start school, at the age of 5, in September - so that there are children who are nearly 6 with September and October birthdays, mixed in with children who have only just stopped saying "I am 4" with August and July birthdays. And that's a substantial gap that young in life.

Unless the system is radically changed (for example, to have three sets of classes each year throughout the school system, with September, January and May starts) there's nothing prescriptive that could be done. But I'm not sure that doing anything prescriptive is the right answer anyway - the system should be flexible enough to be tailorable for each individual child, with the professionals such as the teachers able to work with the parents [[as appropriate]] to get the best of each and every young person.

I heard parents talking of their 29th August child being immature for school, and of how being the little one in the class effected them all the way through their school career and beyond, but then feedback (via email and otherwise) to the programs from young people, some of whom are still pupils quite early in the system, saying how much they have personally gained from being the younger ones - how it has helped to make them more robust and more mature and being in a group that's slightly older has set them higher targets to go for and achieve. Exactly my point that the solution should not be a prescriptive one.

A declaration of interest here. I'm a July child, and still have a vague memory of utter confusion the first day I went to Crofton School. Whether I was just unprepared, or too young, or if it happens that first day for everyone like that I have no way of knowing. And I most certainly have school day memories very strongly coloured by being bullied. And yet on the other hand, my parents did make the most enormous sacrifices (Dad may well read this - THANK YOU) to ensure that I wasn't just another pea in the pod; being at the younger end, I could have been easily lead off the rails and I know that I could get easily bored which wouldn't have been exactly a recipe for highly concentrated success from that initial environment.

So I put my position as the "youngster of the class" - in the following environments at Dulwich and Sevenoaks - as a distinct advantage and challenge. For sure, there was break time bullying especially at Sevenoaks, but that in turn lead me towards other activities which were in themselves educational while others were kicking footballs around and indulging in something akin to tribal warfare. And it lead me to a hardened and perhaps fierce independence where I didn't rely on others.

I wasn't top of the class at either of the schools - but there must of been something that the teachers recognised - perhaps they saw me as "coasting along". I might describe my approach, if truth be know, as only and lazily doing what was needed on subjects that didn't interest me. So they put me through an accelerated stream where I took 3 years in 2 and ended up in following years in classes with a 2 year age spread, and sitting my 'A' levels while still just 16. I managed to get through all four, although not at the 'A' and 'A*' grades that are almost the norm these days [[hey - we are looking back 35 years!]] ... and that early school finish gave me the opportunity - and what a golden opportunity - to take a "gap" year before starting University, getting some real work experience under my belt. And that was such a good year too which has helped make me some things I'm proud to be today.

Do I have a conclusion? If this entry is anything more than a rambling, it's a plea to the bean counters and school system managers to treat each and every child based on their individual needs and circumstances. My own example shows how everyone IS different and it shouldn't be forgotten. And it also says a big "thank you" to those who looked at me as an individual and helped (sometimes having to push me quite hard!) me come out with what - I feel - is an experience that set me up well for life.

Posted by gje at 09:23 AM | Comments (0)

What is built in to this httpd and PHP?

If you build your own tailored Apache httpd and PHP, you should take a full backup when you're done, and also fully document which options you selected and modules you included in the build.

You SHOULD do so, of course ... but it's something that you can overlook all too easily. If you find yourself looking at your own system (or perhaps one you have taken over) and wondering "and what is built into THIS?" there are ways to find out.

The file config.nice is saved by the ./configure that you'll run prior to building Apache httpd, and it contains a log of the options you selected:

#! /bin/sh
#
# Created by configure
 
"./configure" \
"--enable-so" \
"--enable-proxy" \
"--enable-proxy-http" \
"--enable-proxy-balancer" \
"--enable-rewrite" \
"--enable-proxy-ajp" \
"--prefix=/usr/local/apache2.2" \
"$@"

If you have an already installed web server (without access to those development directories) the httpd daemon itself can be called up with the -l option, to give you a list of compiled-in modules. This is the same example as the build above, and has been tuned to allow me to install PHP (the so module) and to allow for a variety of rewriting or URLs and proxying on to application servers - further httpds, Tomcats running Java, etc ...

[trainee@crocus ~]$ cd /usr/local/apache2.2
[trainee@crocus apache2.2]$ ./bin/httpd -l
Compiled in modules:
core.c
mod_authn_file.c
mod_authn_default.c
mod_authz_host.c
mod_authz_groupfile.c
mod_authz_user.c
mod_authz_default.c
mod_auth_basic.c
mod_include.c
mod_filter.c
mod_log_config.c
mod_env.c
mod_setenvif.c
mod_proxy.c
mod_proxy_connect.c
mod_proxy_ftp.c
mod_proxy_http.c
mod_proxy_ajp.c
mod_proxy_balancer.c
prefork.c
http_core.c
mod_mime.c
mod_status.c
mod_autoindex.c
mod_asis.c
mod_cgi.c
mod_negotiation.c
mod_dir.c
mod_actions.c
mod_userdir.c
mod_alias.c
mod_rewrite.c
mod_so.c
[trainee@crocus apache2.2]$

So you now know what you've installed in your httpd web server, and you can learn more by looking at the httpd.conf file, and any other files it includes. But what about your PHP?

You should find that PHP is configured in via lines in the httpd.conf file:
LoadModule php5_module modules/libphp5.so
AddType application/x-httpd-php .php .phtml
and that the library really is in the modules subdirectory.

For details of the build, use PHP to tell you via a test page that reads as follows:
<?php phpinfo() ?>
(Yes - just that one line) which when you browse to it will give you a long web page report.

You can also run "phpinfo" from PHP on the command line:

bash-3.2$ php -r 'phpinfo();'
PHP Version => 5.2.x
 
Build Date => Jul 17 2008 07:00:56
Configure Command => './configure' '--with-apxs2=/usr/local/apache2/bin/apxs'
'--with-mysql=/usr/local/mysql' '--with-gd' '--with-png-dir=/usr'
'--with-jpeg-dir=/usr'
Server API => Command Line Interface
Virtual Directory Support => disabled
Configuration File (php.ini) Path => /usr/local/lib
And so on ...

(Note - this is a much trimmed example!)

Posted by gje at 07:57 AM | Comments (0)
Useful link: PHP training

August 22, 2008

mod_proxy and mod_proxy_ajp - httpd

Under Apache httpd version 2.2, mod_proxy_ajp provides a neat way for you to forward requests to an appropriately configured Apache Tomcat server using the more terse ajp protocol rather than the verbose http. Here are some sample configuration lines from httpd.conf, forwarding requests under the /coffee directory via http and requests under /whiskey via ajp.

ProxyPass /coffee http://192.168.200.153:9082/latmjdemo
ProxyPassReverse /coffee http://192.168.200.153:9082/latmjdemo

ProxyPass /whiskey ajp://192.168.200.153:9009/latmjdemo
ProxyPassReverse /whiskey ajp://192.168.200.153:9009/latmjdemo

This is purely a demo - you probably wouldn't mix them in this way in real life.

mod_proxy_ajp provides a quick and easy alternative to mod_jk if you're running Apache httpd 2.2 (or later in due course), but the example above lacks the flexibility of worker pools and the like that mod_jk provide. To some extend mod_proxy_balancer can help mod_proxy_ajp, but I would expect both mod_jk and mod_proxy_ajp to both have their own proponents and optimum uses

Posted by gje at 07:17 AM | Comments (0)

Diagrams to show you how - Tomcat, Java, PHP

I like to work with a flipchart occasionally, and I have been doing so quite a bit this week, which is a week that I'm giving a wide ranging web server deployment course under Linux, covering both LAMP / PHP technologies, and Tomcat / Java too. Why do I like using a flipchart? Because it encourages me to come up with a number of simple but effective diagrams which I can look back to and add to my notes / re-use. Here are some from the last couple of days ...

The ancestry of Java and C# from C and C++

The left hand diagram on this page shows how the C language was extended into C++, with source code compatibility maintained - resulting in a complex language. Java (originally called Oak, but I have crossed that name out!) took the best of C++ but removed the need for compatibility, simplified, added network awareness and the virtual machine concept. Never the less, it is C / C++ based.

J++ from Microsoft took the Java standard and added things to it - which was great if you wanted to run J++ but caused issues if you developed code under J+ and expected it to run on a standard Java Virtual Machine; J++ was decidedly NOT the way it had been intended for Java to be taken forward, and it had a short life.

Microsoft's C# follows the same philosophy that Java followed - best of C++, network aware, virtual machines, simplified ... so it's little wonder that it bears more than a passing resemblance to Java. It came a long a bit later, though, and has learned from Java's early experience. As a "differentiator", Java is the more portable across platforms, but C# runs in a .net virtual machine meaning that developers in the environment have an extra choice of other languages, such as Visual Basic.

The diagram to the right shows how Java source code (in a .java file) is compiled into a class file via the javac program, and is then run within a Java Virtual Machine or JVM. The JVM is rather like the engine of a car, and just line buying a car ... you need more than just the engine - in this case you need the whole of a JRE or Java Runtime Environment, which provides your application with all the extra standard classes it needs in order to run.

There's a choice when you download Java between a JRE (lower half of the diagram) and a full JSDK (Java Software Development Kit) which is the JRE plus the compiler an tools. If you see the terms JDK, SDK and JSDK mixed and matched, they're all the same thing - it's just Sun's habit of renaming things from time to time.

Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here

Releases of Java, and which download to choose

Sun have actually done a very good job with Java in terms of an initial specification that hasn't had to be broken with any sort of incompatible source code - which means that it goes up very nicely from Java 1.0 to Java release 1.6.

But would you buy into a product that was 10 years old, and still at release one-point-summat? The marketing folks thought you wouldn't so Java was re-christened Java 2 - we have releases like Java 2 1.3 and Java 2 1.4 - and then with a leap to show it was really well established, we went on to Java 5 and now Java 6. All very well in terms of showing change and progress in the product, but it's really still 1.6.

My diagram also shows Apache Tomcat releases - Tomcat 6 works well with Java 6 (good, and nice to see some synchronisation!). Java 5 goes with - err - Tomcat 5.5 and Java 2 1.4 with Tomcat 5.0. It is also possible to run Tomcat 5.5 with Java 6, and Tomcat 5.0 with Java 5, but you'll need some extra patches if you want to run Tomcat 5.5 with Java 2 1.4 (and, no, I am NOT going back any further)

On the right of my diagram, I was helping my users select which of all the plethora of options they'll need to download to run Tomcat on their Linux servers, supporting Servlets and JSPs. It worked out as follows:
• Sun's Java, rather than anyone else or the oft-supplied gcj / gij.
• The standard edition, rather than micro or enterprise. True, one of the Enterprise jars provides Tomcat support, but that's also included with the Tomcat distribution.
• The Development kit rather than just the Runtime Environment. "Why do I need a compiler on the live server" you may ask. Because you'll be running JSPs which include Java source code you'll need to be able to handle!
• The appropriate download for Linux (Java class files may be portable, but the JVM most certainly is NOT!)
• The appropriate download for the hardware (processor) you're running on - for example i586. (Same portability note!)
• and of course a version that's a release that's compatible with the Tomcat you wish to run.

Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here

Three guises for the Java Virtual Machine

You can run your Java programs as stand alone programs on your computer, on your web server (in the form of Servlets or JSPs) and in your browser (as applets).

In a stand alone program, the programmer provides a method called main which runs, once, when the program is called up.

For a servlet, running on a web server with an engine such as Tomcat's Catalina, the programmer takes a standard pre-written class and overwrites (extends) one of more of its methods - such as init, destroy, doGet and doPost. Unlike a stand alone program, these don't run just once when Tomcat invokes them - either the doGet or the doPost method will run many times, retained in the server's memory - like turning the handle of a machine each time a visitor browses to the URL which triggers them.

Applets run within a browser plugin. When a web page that includes and applet is loaded, a method called init is run once, followed by a method called start when the window becomes visible. The paint method id called repeatedly to keep updating / refreshing the output - giving dynamic graphics - until the window ceases to be visible when the stop method is called. When the user moves on to the next page or kills the browser window, the destroy method is called.

Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here

A flowchart for each stage of a web application

A well designed web based application should work in a manner to something like I am showing in this flowchart. Whether you're looking at a doGet or doPost method in a Java Servlet, or a phase of a PHP application, the pattern is a common one ...

• read any cookie or hidden field to see if this is a continuing session or if we know who the user is
• if we do know the user, read in his details - "shopping cart" - to date, which will be from a file of a database such as MySQL
• process the data that's been entered, which I characterise as finishing up from the previous page. You'll see a multiway branch such as a switch used here, as it's a very good idea (later slide) to have all the stages of an application controlled from a single process.
• prepare for the next page. That will be a branch again and USUALLY for the next page in the sequence, but if the user made an error in entering his data it will refresh the previous page.
• Save the (updated) details back to the shopping cart
• Read it the HTML template for the response page
• Complete and send out the next page / form

Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here

The 4 layer model

You shouldn't write all the code for a web application in a single file. The file will get too big and too hard to maintain, and you'll be preventing yourself from reusing parts of the code in other pages / applications. You'll also end up with a file that's got a mixture or program code and HTML in it, meaning that it can only be maintained by a person who is skilled in BOTH.

What is better? Try the 4 layer model.

1. Your Top level / controlling code
2. The application logic (also known as the business logic) which contains all the database and calculation code.
3. The Web helpers - your standard routines for (example) ensuring that you are not open to injection attacks, that input boxes are "sticky", and that forms have a consistent way of handling errors
4. Your HTML template that gives you the look and feel of each page.

Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here

Handling any characters in your input

There's a joke about the mother who gave the son the middle names "DROP DATABASE". When she signed him up, on line, for his primary school, the school's computer mysteriously lost all its records ...

In a well programmed system, problems like this should not occur and even database keywords should be acceptable as inputs. But you have to check this in your programming, and as well as keywords be aware of how quotes, less than signs, ampersands, and even spaces are handled.

This diagram shows how user inputs from the web need to be cleaned up to provide the "real" values to be used in calculations, and then need further processing / protection if they're to be stored in a database, or echoed back on a web page.

The same principles apply in Perl (vis CGI), Java, and PHP ... and other languages. The function names in orange on the diagram are from PHP.

Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here

Things to consider in PHP ...

If you're looking to write an easy to use, secure PHP page, here are some of the things you should consider ...
• How to prevent injection attacks (see previous board)
• How to provide sticky fields, so that user who fails to complete a form correctly is NOT penalised by being given a new BLANK form to try again (I think we've all see when sites that drop fields, haven't we?)
• Preventing users bookmarking a page in the middle of a series so that they can just land there, unexpectedly, at a later date - the "Hogwarts effect"
• How to maintain users's sessions
• The best and consistent way to handle errors.
There are considerations for other languages too - it just so happens that this example is a PHP slide!

In PHP, some variables are automatically provided to you when you start your web page processing, and on this board they're listed down the right hand side of the screen. They are:
$_GET - parameters supplied via the URL / GET method
$_POST - parameters supplied via a POSTed form
$_COOKIE - cookies returned with the request
$_ENV - environment variables from the server
$_SERVER - a wide range of variables from the web server / details of the requesy.

$_GET, $_POST, $_COOKIE and $_ENV are also combined into another single array called $_REQUEST which is very useful to you if you want to check for the presence of an input field, whether it was supplied by any of the methods.

Finally, this diagram mentions $_SESSION which is a little different - it's populated by the session_start function rather than by the starting server process, and it contains the information saved for this user's previous page in his / her current session.

Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here

Versions of PHP and code portability

PHP is a vibrant language - so much so that there have been a number of issues / changes over the years, and configuration options added, that mean that PHP code written on one server may not run straight off on another. This board goes through recent(ish) versions, and highlights some of the issues.

Up to and including PHP 4.1, variables were populated directly from form fields which was regarded as a security issue when coding is in the hands of newcomers who don't appreciate the need to initialise their storage. From 4.1 onwards (4.1 supports both methods), you should use $_GET, $_POST or $_REQUEST. But you'll see a delta sign on my diagram - if you have old code, you can support the old mode too ..

As of PHP 5.0, the Object model was changed and if you assign an object, you're copying a pointer in PHP 5, but cloning the whole object in PHP 4. It means, incredibly, that an assignment statement may have a different effect depending on the version you're running.

As PHP 5 was introduced, the MySQL drivers (that had been supplied with PHP4) were withdrawn, due to the tightened license with MySQL. If you want have MySQL support in PHP 5, you download and unpack MySQL first (no need to actually install it - just have the libraries available) and then build PHP.

Also at PHP 5, a second API to the (now loaded from elsewhere) MySQL drivers was provided. The ones provided prior to that date - with names starting with mysql_ - were judged imperfect in Computer Scientist's terms and the mysqli_ ones which were added are considered better. To use the new ones means you have to change your code, so you'll probably want to consider building your PHP with both sets!

Also noted at the top right of this slide - other things to remember with regards to code portability across different versions and settings of PHP
• Register Globals
• Short Tags
• Magic Quotes

Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here

OOO Arrrr!

There are lots of file types with extensions ending in "ar" which usually stands for "archive"!

.tar files have been around for many, many years - "tape archives" produced and also unpacked by the tar utility. The format is very basic - they're uncompressed, and they lack a single index at the beginning which mean they're very inefficient if you were to even THINK of using them for random access.

.jar files are designated as "Java Archives" and written (and read) by the jar utility which is supplied as a part of the Java distribution - but in fact the can contain any file and directory structure, and they're actually ".zip" file compatible. Which means that they are compressed, and they include a list of contents which allow them to easily be used for randomly accessing the files / elements they contain.

.war files ... "web archives" are .jar files! what tells them apart from .jar files is the specific layout of the files they contain, and a certain subdirectory that must be present, which means that Tomcat will assume that they can be directly deployed (instructions for the URLs to be used contained within!) to allow a "distribute and play" approach to Java Application Upgrades.

Similarly, .ear and .sar files are generated to a specific layout by jar - enterprise archives and service archives. .rar files - resource archives - are something else!

Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here

Regular Expressions - an easy introduction

If you're looking to see if two values are equal in programming terms, that's usually easy enough - but what is you want to see if (example) a user's data entry "looks like" a postcode - that's an altogether tougher prospect, and we commonly use a regular expression to make the test.

A regular expression is the description of a pattern and can contain a number of elements.

Anchors - things like "does it start with ..."

Literal characters - "does it contain exactly this character"

Character groups - "does it contain a character from this list"

Counts "does it contain one or more of ..."

All these elements were (are) present in grep - the "Regular Expression processor" and they're shown in blue on the board. egrep provides / provided extensions - shown in brown, which included the ability to group together sections of the regular expression, and added an "or" operator described in post terms as "alternation".

Larry Wall's Perl added yet further options - a tiny sampling of those are added in red - and languages like Tcl (John Ousterhout) and PHP added "POSIX" standard alternatives - in Green. Larry decided to support the green POSIX stuff too, and the PHP team supported Larry's style ... and so the whole this is very flexible as well as being a complex story.

For this week's web server deployment course, Regular Expressions actually came up in the form of configuration settings for the Apache httpd web server - especially for mod_rewrite which lets you tell the web server to divert a request which is apparently for a fixed web page to a script that's going - for example - to access a database, passing in the name of the page that was called up as a parameter to the page. It's very clever stuff, allowing us to create whole directories of virtual documents.

Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here

It's incredible just how long it has taken me to document a few diagrams - but it also goes a long way to proving that a picture paints a thousand words, and to demonstrating just how much useful information I can pack into a short course such as the one I've been giving on site in Milton Keynes (link - what would such a course cost?) this week.

Update - more similar diagrams If you found these diagrams useful, I have added a second series of them here covering the conclusion of the course - with more on Java and Tomcat, and a whole series on MySQL too!

Posted by gje at 01:15 AM | Comments (0)
Useful links: PHP training, Java training

August 21, 2008

Dialects of English and Unix

English is spoken as a first language by over 300 million people, and by about five times that number if you include people who can speak it as a second or third language. But that doesn't mean that word and wording is going to be the same within different variants, let alone the subtleties and drifts, where somethings that's just a mild comment in English can be offensive to an American, and vice versa.

In a way, it's a bit the same with flavours of Unix and Linux - something that works on Solaris may not work on OS X, and a Linux command that you've come to rely on on your shared hosting service turns out to have not been loaded on the basic build on your new dedicated host - ah - these things are sent to try us!

Take the ps command - for process status,

In the old SunOS 4.1.3 days, it used to be second nature to type ps -aux but that was a Berkeley Software Distribution (bsd) based Unix and when Sun switched to System V.4 (AT&T based) at Solaris 2 / SunOS 5, the recommended command became ps elf which presented broadly the same information is a somewhat different format.

You'll notice that the - (minus sign) has gone from the "option" list, and I recall being told that happened because they're not really options, but rather they are a description of how the thing is to work. And if they're not options, you don't need a minus sign! ((The same thing applies to tar - where it's tar cvf rather than tar -cvf))

There days, under Linux, I find some delegates using ps ef and others using ps aux. But there are other useful options ... and I surprised one of my delegates by suggesting he use

ps auxwww

Odd? Yes, it is a bit. The "w" descriptor means "wide" - or in this case "wide, wide, wide" ... so that lines reported are not truncated to the screen width. ps auxwww outputs the complete command line used to start all processes, and in this instance was vital to allow my delegate to see all the various options used in starting up Tomcat, including the option that specified which particular JVM release it was to use, and where that was to be loaded from.

Posted by gje at 05:56 PM | Comments (0)

Well House Consultants Ltd. Copyright 2008