Archive for the ‘System Administration’ Category

Staging servers

Monday, August 22nd, 2011

At GORGES we prefer to set up a staging server or website for our bigger projects.  I would even call it a requirement since it can be used by the customer and our quality assurance staff to review both progress and the release candidate before publishing a website to a production site.

Usually developers work directly on their workstations or laptops, and a development environment has both web and database services running similar to a production server. Ideally a development environment should have the same versions of services as the production server, for example PHP 5.2 or Ruby 1.9.  On more than one occasion I have been bitten by writing MySQL 4.3 or 5 code, but the production server has an older version of MySQL without sub-query support so the queries had to be rewritten.

Usually these development services are customized for development, for example the database does not have to be hardware-optimized and the web service can have debugging tools enabled.  I prefer to turn on all warning messages, so any identified problems that are not severe enough to halt page serving will be shown either in a log file or onscreen.

Just as important with having a staging server is to have clean and well-structured publishing scripts.  If you are relying on a manual process to upload files using FTP, then it is possible to skip an updated file by mistake.  Publishing scripts come in all sorts of flavors and approaches – we have both “pull” methods that use server-side scripts, and “push” scripts that execute on the developer’s computer and automatically upload files to the server.  The push scripts are built on Capistrano, which although it is based on ruby we sometimes use at Gorges for non-ruby websites.

We routinely protect our staging servers from prying eyes and search engines by using HTAuth username and passwords.  Once a website is scanned by a Google, Bing, or other web crawler then it is out of your control.

Having a staging server doesn’t mean much unless it is actually used to test new features and look for bugs.  Quality assurance is important, and I have yet to meet a developer that produces bug-free code or see a unit testing system that covers 100% of a web application feature set.

Not covered in this post is the issue of database migrations between developer environment and a staging server, and also from a staging server to a production machine.  These can be tricky, but as with all challenges the proper planning will guide you to the best solution.

After the application or update goes live to the production server most staging servers are continued as “sand boxes” to support additional updates and testing.

worked in academia, corporate research labs and several technology startup companies prior to GORGES. His expertise is software architecture, database development, and system administration. Matt brings GORGES over 25 years experience developing fast and robust software on a multitude of platforms and languages.

Change tracking with phpMyAdmin

Monday, February 14th, 2011

A couple of years back, I posted a feature request in the phpMyAdmin tracker. I was planning to implement this feature myself and I wrote a fairly complete specification. I submitted that with my feature request.

Much to my excitement, I recently discovered that this feature is now available in phpMyAdmin, and pretty much to my exact specifications!

The feature is change tracking – essentially a simple automatic schema version control system.

You can think of this idea as auto-generated, forward-only migrations. In other words, it doesn’t enable you to easily revert to a previous schema version, except by re-initializing the schema to a previous version, destroying existing data – but it also doesn’t involve writing migrations by hand.

You just open your schema, turn on tracking, and then proceed to work with phpMyAdmin the way you normally would – making changes to the schema (and/or data) using the GUI interface; your choice of SQL statements are tracked and recorded, and at the end, a snapshot of the complete schema is taken, enabling you to upgrade from one schema version to another, or initialize the schema to a specific version.

If you’re not already using database migrations, this could be a big timesaver, and should lighten the work of keeping your local/test/production databases in sync – as well as maintaining a history of what was changed and when. Nice!

Thanks, phpMyAdmin Team!

Rasmus Schultz has worked for web development companies, advertising agencies and a music software company during his extensive development career. His main strengths are software development and database design. Rasmus has more than a decade of experience with many development platforms, languages and standards.

GORGES Greenery

Monday, February 14th, 2011

There are good reasons to work toward reduced impacts. We know most of the environment-saving actions we read about will also save us money. Clients express appreciation when suppliers do their bit towards a sustainable future.

Our pro bono effort to help with Gulf oil cleanup (www.gulfsaversolutions.com) was a type of singular environmental action. That’s worthy, but what counts largest are the routine actions.

Here are some routine efforts that we make at GORGES.  If you would like to know about our experience with any of these, please contact me or Don.

Communicate electronically:  We send invoices electronically unless we are explicitly asked to send by postal mail.  This significantly reduces paper use, and we have found as a bonus that clients often pay more quickly with this method.

We send newsletters by email. It’s rare that anyone we are talking with declines to receive the newsletter. We use a service to send it and monitor the sending, and almost one-half of our recipients open the newsletter.  If given a choice, we ask businesses that send print newsletters to us by postal mail, or other recurring correspondence to switch us to email. If they are not setup to do that, we offer to help with the advances.

Reduce paper usage:  Over the holidays some of my family visited and we toured the GORGES offices.  My sister marveled at how little printed paper there was in everyone’s work area.  It’s true – we have set up collaborative file repositories and work almost exclusively electronically.  There are days when our group of seventeen prints nothing.

Turn off appliances:  This is obvious, but it does work.  Even devices on standby mode consume some power.  We supply power strips to each workstation area and encourage staff to turn off the entire strip at the end of the day.  Turning monitors off is common enough; turning the whole computer off helps even further.

When we switched offices last summer I changed our phone extensions from using many individual power adapters to a single power-over-ethernet switch device in our server closet.  The energy savings may be marginal, but the added benefit of reduced wires and freeing up a workstation power outlet was nice.

The biggest savings come from deactivating devices entirely.  By upgrading some servers at our co-location facility, we have decommissioned five servers in favor of two new ones.  An added benefit is reducing number of our uninterruptible power supply units, which require expensive lead-acid batteries that only last 2-3 years.

Reused & recycled items:  We built out our office with re-purposed furniture, saving thousands of dollars.  There may be a scratch or three on the new desks, but it sure beats the fold-up tables we had during our early startup years.  And old items and equipment are not always sent directly to the trash – we are regular contributors to the Finger Lakes ReUse Center.  Of course we also do the standard office paper, plastic, and metal recycling.

Plants:  We have many indoor plants. They are great to look at and help to clean the air, particularly in rooms with minimal air movement or no windows. Employing electrical devices to do the same is costly, annoying, and boring.

We have an active thermostat control program. We are after all a bunch of techies, so we can handle programming the devices!  We can all handle an extra degree without being uncomfortable, and this reduces our power consumption.

Location:  We love working downtown, and have remained here despite seeing other tech companies move to the office parks and ‘burbs.  Several staff (including myself) walk to work, and others bicycle or take public transportation.  We have one client who provides bus fares to all employees for going to and from the office (and even to travel to meetings).

In summary, I’m sure you can think of other ways to reduce your power or materials consumption in your own office as well; it all counts.  Let us help if our experience or technical skills can contribute to reducing consumption.

worked in academia, corporate research labs and several technology startup companies prior to GORGES. His expertise is software architecture, database development, and system administration. Matt brings GORGES over 25 years experience developing fast and robust software on a multitude of platforms and languages.

Virtualization made Expensive

Friday, February 4th, 2011

I attended a seminar in which a network installation company was extolling the virtues of virtualization. One of their slides showed virtualization contributing much to the cost of overall IT operations in the near future. I just did not get it – did not fit my day-to-day experience.

The answer to my first inquiry did not clear it up. The presenter answered something about management policies and approvals for virtualization. Thinking it was just one of those security paranoia things, I let it rest.

But coming back to it I still wondered, why would something so quick, simple, and useful be expensive and management intensive? Then the little light came on, it’s because they were speaking in the world of Microsoft servers and each virtualized server needs a license.

In our shop the virtualization decision has been pushed down to project management level and on down to the developer level. A new application is often virtualized after completion, adjusted to be a production version, and then pushed to the cloud.

Developers treat virtualization as a handy “envelope”, a quick and useful place to put an untrustworthy current version of a new client’s application, or anything else that needs compartmentalization. Developers virtualize as needed. They do not need permission. They use Linux or Centos so there are no cost issues.

Once again. Open Source brings the productivity tool we all need at a price our clients can afford.

Adding a Fax Machine to Asterisk

Monday, January 10th, 2011

Recently we needed to formally integrate a fax machine into our Trixbox / Asterisk office phone system.  In the past we just had a simple phone splitter and hoped the line wasn’t in use.  We have grown to 15 people in our office, so yelling “Is everyone off the phone lines?” is a bit disruptive to our office staff.

The first step was to research a solution, and a Cisco Linksys PAP2T device seemed like the best device.  Usually it is bundled with a service contract by VOIP Internet phone companies, and there are many unlocked units available ($32 incl. shipping – I love eBay).

Once the device arrived, I first created a SIP (Session Initiation Protocol) extension in our Trixbox configuration panel (we used extension 40).

Next we need to know the IP number used by the PAP2T so we can use a web browser to program it to interface with our Trixbox phone server.  First plug in the power connector, an ethernet cable to a DHCP-enabled network, and an analog phone into line 1.  The analog phone should have a dial tone, so from this phone type four asterisks (“****”) to activate the PAP2T voice-based setup menu.  When prompted, type “110#” to get the IP number of the PAP2T.  Another way of determining the IP# is to

From a web browser, enter the IP# of the PAP2T device (our device used http://192.168.1.131).  You should next see the PAP2T configuration screen.  Click the Admin Login link at the top-right, then click the Line 1 tab.  These fields had to be entered:

  • Proxy: 192.168.1.2 (use the IP# of your Trixbox server)
  • Display name: fax
  • User ID: 40 (this is the extension)
  • Password: **** (no, I’m not going to reveal our password here!)

Different trixbox setups may require different configuration settings, but these four items were all that we needed.  Click the submit button any you’re ready for faxing.

Note that this is not the only way to integrate a PAP2T with Trixbox.  Other solutions used TFTP to program the PAP2T during a device restart, but this seemed unnecessary compared with the simplicity of the above instructions.  And there are plenty of software or service solutions (e.g. HylaFax, asterfax), but our fax machine is also an office scanner and copier so we wanted a hardware solution.

Someday we may add an automatic fax signal detector and route a fax automatically to our fax machine (it seems to be a simple Trixbox setting or extension), but that’s a future project.

http://en.wikipedia.org/wiki/Session_Initiation_Protocol
worked in academia, corporate research labs and several technology startup companies prior to GORGES. His expertise is software architecture, database development, and system administration. Matt brings GORGES over 25 years experience developing fast and robust software on a multitude of platforms and languages.

Saving brain cells with WinSCP

Wednesday, September 22nd, 2010

One of my daily challenges is getting access to the hundreds of different web sites that we host in order to make edits, tweak file permissions, or fix bugs. It’s easy to remember a handful of different URLs, login IDs, and passwords, but my photographic memory only has a 5-1/4″ floppy for storage, so as the number grows I need help.

One tool that’s helped me achieve more ninja-like response times is WinSCP, a free GUI-based FTP/SFTP client. Like most FTP clients, it lets me maintain a list of stored sessions that I can open in a flash, but what sets it apart is the slick integration with the popular PuTTY SSH client.

I came to WinSCP by a round-about path – for a long time I used FileZilla for SFTP and PuTTY for SSH, but the effort of maintaining a separate list of site logins for each tool ate up valuable brain cycles and finger motions that could have been used profitably for other tasks. When I discovered WinSCP, it was like a floor wax and a dessert topping in one – the tool lets you launch both PuTTY and SFTP sessions to a given server in a single stroke. Sometimes you need a little command shell, sometimes you need a lot – now you don’t have to choose.

Setting up the PuTTY integration takes a few minutes, but it’s fairly simple. In WinSCP, open the Preferences dialog and select the Integration / Applications heading on the left. Enter the path to your PuTTY installation (you need to install PuTTY separately), and set the other options the way you like them – the most useful one lets you pass the session password to PuTTY so you don’t need to enter it again. You can set it to launch a PuTTY session every time you connect to a server via SFTP, or you can just use the convenient button on the main toolbar to launch PuTTY when you need it. Either way, the seconds you save by not having to re-enter the login ID and password each time can add up in the long run.

One minor annoyance that I recently solved was setting the color preferences for PuTTY when launched from WinSCP – I knew how to do this with stand-alone PuTTY, but the integrated version always came up with a blue-on-black scheme that was a challenge for my aging eyes. To fix this, just use the normal PuTTY settings dialog (in PuTTY, right-click on the title bar and select Change Settings), but be sure to select “WinSCP temporary session” in the list of saved sessions, and save your changes to that.

WinSCP has a host of other features I haven’t even explored yet, including remote directory-synchronization functions and extensive scripting and command-line support. You can also generate Windows shortcuts to saved sessions, which could be stored with
your other project files to make your life still easier.

If you want to spend more time with your family and less time searching for passwords, I’d recommend giving WinSCP a try.

Ted Caldwell is an Application Developer and Senior Software Architect for GORGES. Ted's work encompasses all aspects of web application development, including requirements analysis, database and software design, programming, project management, and system administration. Ted focuses on delivering practical, cost-effective technology solutions for GORGES clients, using a variety of custom and off-the-shelf software tools and frameworks.
©2013 GORGES - All rights reserved
where programming meets design and lives happily ever after